2024-05-08T21:35:46.391 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T21:35:46.392 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698072 --name pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi --owner scheduled_pdonnell@teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/blogbench}} -- /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698072/orig.config.yaml 2024-05-08T21:35:46.454 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T21:35:46.550 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698072 branch: wip-pdonnell-testing-20240508.150423-reef description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/blogbench}} email: pdonnell@redhat.com first_in_suite: false job_id: '7698072' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-pdonnell-testing-20240508.150423-reef ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240508.150423-reef sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 owner: scheduled_pdonnell@teuthology priority: 65 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2738 sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 sleep_before_teardown: 0 subset: 17/512 suite: fs suite_branch: wip-pdonnell-testing-20240508.150423-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi131.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK8bR10z5wNoFiddZ7dZ//vz4Q6mQgFpCWy71PjKH9nonkxjgZRw+OFdf36hISCPUG9t4N6IeZD+x2xeqzawisI= smithi132.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPBvsaukJqA6PBwG8nB3b/8/xPJpur1JqfebbMvzCoLc82K2tXUb4jz6Vd/5wm3i2hjQH7/QTaqKraWuatM1Ghs= smithi157.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKEc38WN7l4z03FAtek/wdrSSzEmVG+PrBQkkk3lumpwKAdTDhe9Glj04Br8dEpsDxHxCZCOBr23QFhnXVqqHLs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_19:22:21 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T21:35:46.551 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa; will attempt to use it 2024-05-08T21:35:46.551 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa/tasks 2024-05-08T21:35:46.551 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T21:35:46.553 INFO:teuthology.task.internal:Checking packages... 2024-05-08T21:35:46.579 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'd21e4f961318b56f0d915caa2490c3711bf13c34' 2024-05-08T21:35:46.579 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T21:35:46.579 INFO:teuthology.packaging:ref: None 2024-05-08T21:35:46.579 INFO:teuthology.packaging:tag: None 2024-05-08T21:35:46.580 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240508.150423-reef 2024-05-08T21:35:46.580 INFO:teuthology.packaging:sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:35:46.580 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-pdonnell-testing-20240508.150423-reef 2024-05-08T21:35:46.809 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1322.gd21e4f96 2024-05-08T21:35:46.811 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T21:35:46.817 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T21:35:46.817 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T21:35:46.833 INFO:teuthology.task.internal:Saving configuration 2024-05-08T21:35:46.854 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T21:35:46.860 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T21:35:46.888 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698072', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 21:30:04.651931', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK8bR10z5wNoFiddZ7dZ//vz4Q6mQgFpCWy71PjKH9nonkxjgZRw+OFdf36hISCPUG9t4N6IeZD+x2xeqzawisI='} 2024-05-08T21:35:46.910 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698072', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 21:30:04.649880', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPBvsaukJqA6PBwG8nB3b/8/xPJpur1JqfebbMvzCoLc82K2tXUb4jz6Vd/5wm3i2hjQH7/QTaqKraWuatM1Ghs='} 2024-05-08T21:35:46.930 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698072', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 21:30:04.653434', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKEc38WN7l4z03FAtek/wdrSSzEmVG+PrBQkkk3lumpwKAdTDhe9Glj04Br8dEpsDxHxCZCOBr23QFhnXVqqHLs='} 2024-05-08T21:35:46.930 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T21:35:46.937 INFO:teuthology.task.internal:roles: ubuntu@smithi131.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-08T21:35:46.937 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-08T21:35:46.937 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-08T21:35:46.937 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T21:35:47.037 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1107d05160>, signals=[15]) 2024-05-08T21:35:47.038 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T21:35:47.044 INFO:teuthology.task.internal:Opening connections... 2024-05-08T21:35:47.044 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi131.front.sepia.ceph.com 2024-05-08T21:35:47.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T21:35:47.119 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi132.front.sepia.ceph.com 2024-05-08T21:35:47.120 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T21:35:47.192 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2024-05-08T21:35:47.193 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T21:35:47.267 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T21:35:47.275 DEBUG:teuthology.orchestra.run.smithi131:> uname -m 2024-05-08T21:35:47.296 INFO:teuthology.orchestra.run.smithi131.stdout:x86_64 2024-05-08T21:35:47.297 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release 2024-05-08T21:35:47.356 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="Red Hat Enterprise Linux" 2024-05-08T21:35:47.356 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="8.6 (Ootpa)" 2024-05-08T21:35:47.356 INFO:teuthology.orchestra.run.smithi131.stdout:ID="rhel" 2024-05-08T21:35:47.357 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="fedora" 2024-05-08T21:35:47.357 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="8.6" 2024-05-08T21:35:47.357 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el8" 2024-05-08T21:35:47.357 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-08T21:35:47.357 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31" 2024-05-08T21:35:47.357 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-08T21:35:47.357 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://www.redhat.com/" 2024-05-08T21:35:47.357 INFO:teuthology.orchestra.run.smithi131.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-08T21:35:47.357 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T21:35:47.357 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:35:47.357 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-08T21:35:47.357 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-08T21:35:47.357 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-08T21:35:47.357 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-08T21:35:47.358 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2024-05-08T21:35:47.383 DEBUG:teuthology.orchestra.run.smithi132:> uname -m 2024-05-08T21:35:47.403 INFO:teuthology.orchestra.run.smithi132.stdout:x86_64 2024-05-08T21:35:47.404 DEBUG:teuthology.orchestra.run.smithi132:> cat /etc/os-release 2024-05-08T21:35:47.462 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="Red Hat Enterprise Linux" 2024-05-08T21:35:47.462 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="8.6 (Ootpa)" 2024-05-08T21:35:47.463 INFO:teuthology.orchestra.run.smithi132.stdout:ID="rhel" 2024-05-08T21:35:47.463 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE="fedora" 2024-05-08T21:35:47.463 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="8.6" 2024-05-08T21:35:47.463 INFO:teuthology.orchestra.run.smithi132.stdout:PLATFORM_ID="platform:el8" 2024-05-08T21:35:47.463 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-08T21:35:47.463 INFO:teuthology.orchestra.run.smithi132.stdout:ANSI_COLOR="0;31" 2024-05-08T21:35:47.463 INFO:teuthology.orchestra.run.smithi132.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-08T21:35:47.463 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://www.redhat.com/" 2024-05-08T21:35:47.463 INFO:teuthology.orchestra.run.smithi132.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-08T21:35:47.463 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T21:35:47.463 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-05-08T21:35:47.463 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-08T21:35:47.463 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-08T21:35:47.463 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-08T21:35:47.464 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-08T21:35:47.464 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2024-05-08T21:35:47.491 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2024-05-08T21:35:47.511 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2024-05-08T21:35:47.511 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2024-05-08T21:35:47.570 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="Red Hat Enterprise Linux" 2024-05-08T21:35:47.570 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="8.6 (Ootpa)" 2024-05-08T21:35:47.570 INFO:teuthology.orchestra.run.smithi157.stdout:ID="rhel" 2024-05-08T21:35:47.570 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="fedora" 2024-05-08T21:35:47.570 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="8.6" 2024-05-08T21:35:47.570 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el8" 2024-05-08T21:35:47.570 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-08T21:35:47.570 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2024-05-08T21:35:47.570 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-08T21:35:47.570 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://www.redhat.com/" 2024-05-08T21:35:47.571 INFO:teuthology.orchestra.run.smithi157.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-08T21:35:47.571 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T21:35:47.571 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-08T21:35:47.571 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-08T21:35:47.571 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-08T21:35:47.571 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-08T21:35:47.571 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-08T21:35:47.571 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2024-05-08T21:35:47.591 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T21:35:47.601 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T21:35:47.608 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T21:35:47.608 DEBUG:teuthology.orchestra.run.smithi131:> test '!' -e /home/ubuntu/cephtest 2024-05-08T21:35:47.610 DEBUG:teuthology.orchestra.run.smithi132:> test '!' -e /home/ubuntu/cephtest 2024-05-08T21:35:47.613 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2024-05-08T21:35:47.630 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T21:35:47.636 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T21:35:47.636 DEBUG:teuthology.orchestra.run.smithi131:> test -z $(ls -A /var/lib/ceph) 2024-05-08T21:35:47.670 DEBUG:teuthology.orchestra.run.smithi132:> test -z $(ls -A /var/lib/ceph) 2024-05-08T21:35:47.672 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2024-05-08T21:35:47.699 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T21:35:47.765 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T21:35:47.780 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-08T21:35:47.781 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-05-08T21:35:47.781 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-05-08T21:35:47.781 DEBUG:teuthology.orchestra.run.smithi131:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T21:35:47.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T21:35:47.800 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2024-05-08T21:35:48.052 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-05-08T21:35:48.052 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-05-08T21:35:48.052 DEBUG:teuthology.orchestra.run.smithi131:> uname -r 2024-05-08T21:35:48.072 INFO:teuthology.orchestra.run.smithi131.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-05-08T21:35:48.072 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-513.9.1.el8_9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-05-08T21:35:48.072 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-08T21:35:48.073 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum upgrade -y linux-firmware 2024-05-08T21:35:49.588 INFO:teuthology.orchestra.run.smithi131.stdout:Updating Subscription Management repositories. 2024-05-08T21:35:49.588 INFO:teuthology.orchestra.run.smithi131.stdout:Unable to read consumer identity 2024-05-08T21:35:49.856 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:35:49.856 INFO:teuthology.orchestra.run.smithi131.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T21:35:49.856 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:35:51.763 INFO:teuthology.orchestra.run.smithi131.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-05-08T21:35:52.414 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 26 MB/s | 16 MB 00:00 2024-05-08T21:35:56.047 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 605 kB/s | 24 kB 00:00 2024-05-08T21:35:56.896 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-05-08T21:35:56.897 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-05-08T21:35:56.897 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-05-08T21:35:57.048 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-05-08T21:35:57.048 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-05-08T21:35:57.270 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/8/flavors/default/x86_64/ 2024-05-08T21:35:57.270 DEBUG:teuthology.orchestra.run.smithi131:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/8/flavors/default/x86_64/ --input-file=- 2024-05-08T21:35:57.273 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi131.front.sepia.ceph.com to complete... 2024-05-08T21:36:11.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08 21:36:11 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/8/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [425099368/425099368] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-08T21:36:11.524 INFO:teuthology.orchestra.run.smithi131.stderr:FINISHED --2024-05-08 21:36:11-- 2024-05-08T21:36:11.524 INFO:teuthology.orchestra.run.smithi131.stderr:Total wall clock time: 14s 2024-05-08T21:36:11.524 INFO:teuthology.orchestra.run.smithi131.stderr:Downloaded: 1 files, 405M in 14s (28.9 MB/s) 2024-05-08T21:36:11.526 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-05-08T21:36:11.526 DEBUG:teuthology.orchestra.run.smithi131:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-08T21:36:13.455 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying... ######################################## 2024-05-08T21:36:13.536 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing... ######################################## 2024-05-08T21:36:13.687 INFO:teuthology.orchestra.run.smithi131.stdout:Updating / installing... 2024-05-08T21:36:56.651 INFO:teuthology.orchestra.run.smithi131.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-05-08T21:36:58.212 INFO:teuthology.orchestra.run.smithi131.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-05-08T21:37:22.138 INFO:teuthology.orchestra.run.smithi131.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-05-08T21:39:40.839 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi131.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-08T21:39:40.840 DEBUG:teuthology.orchestra.run.smithi131:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-08T21:39:40.897 INFO:teuthology.orchestra.run.smithi131.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-05-08T21:39:40.897 INFO:teuthology.orchestra.run.smithi131.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-05-08T21:39:40.897 INFO:teuthology.orchestra.run.smithi131.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-05-08T21:39:40.897 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-05-08T21:39:40.897 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-05-08T21:39:40.897 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-05-08T21:39:40.897 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-05-08T21:39:40.897 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-05-08T21:39:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-05-08T21:39:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-08T21:39:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-08T21:39:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-08T21:39:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-08T21:39:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-08T21:39:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-08T21:39:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-08T21:39:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-08T21:39:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-08T21:39:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-08T21:39:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-08T21:39:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-08T21:39:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-08T21:39:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-08T21:39:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-08T21:39:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-08T21:39:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-08T21:39:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-08T21:39:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-08T21:39:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-08T21:39:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-05-08T21:39:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-05-08T21:39:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-05-08T21:39:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-08T21:39:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-05-08T21:39:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-08T21:39:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-08T21:39:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-05-08T21:39:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-05-08T21:39:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-05-08T21:39:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-05-08T21:39:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-05-08T21:39:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-05-08T21:39:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-05-08T21:39:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-05-08T21:39:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-08T21:39:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-05-08T21:39:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-05-08T21:39:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-05-08T21:39:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-05-08T21:39:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-05-08T21:39:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-05-08T21:39:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-05-08T21:39:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-05-08T21:39:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-05-08T21:39:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-05-08T21:39:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-05-08T21:39:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-05-08T21:39:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-05-08T21:39:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-05-08T21:39:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-05-08T21:39:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-05-08T21:39:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-05-08T21:39:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-05-08T21:39:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-05-08T21:39:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-05-08T21:39:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-05-08T21:39:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-05-08T21:39:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-05-08T21:39:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-05-08T21:39:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-05-08T21:39:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-05-08T21:39:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-05-08T21:39:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-05-08T21:39:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-05-08T21:39:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-05-08T21:39:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-05-08T21:39:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-05-08T21:39:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-05-08T21:39:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-05-08T21:39:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-05-08T21:39:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-05-08T21:39:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-05-08T21:39:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-05-08T21:39:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-05-08T21:39:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-05-08T21:39:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-05-08T21:39:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-05-08T21:39:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-05-08T21:39:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-05-08T21:39:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-08T21:39:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-05-08T21:39:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-05-08T21:39:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-05-08T21:39:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-05-08T21:39:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-05-08T21:39:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-05-08T21:39:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-05-08T21:39:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-05-08T21:39:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-05-08T21:39:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-05-08T21:39:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-05-08T21:39:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-05-08T21:39:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-05-08T21:39:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-05-08T21:39:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-05-08T21:39:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-05-08T21:39:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-05-08T21:39:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-05-08T21:39:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-05-08T21:39:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-05-08T21:39:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-05-08T21:39:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-05-08T21:39:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-05-08T21:39:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-08T21:39:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-08T21:39:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-05-08T21:39:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-05-08T21:39:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-05-08T21:39:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-05-08T21:39:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-05-08T21:39:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-05-08T21:39:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-05-08T21:39:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-05-08T21:39:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-05-08T21:39:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-05-08T21:39:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-05-08T21:39:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-05-08T21:39:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-05-08T21:39:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-08T21:39:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-08T21:39:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-05-08T21:39:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-05-08T21:39:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-05-08T21:39:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-05-08T21:39:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-05-08T21:39:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-05-08T21:39:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-05-08T21:39:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-05-08T21:39:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-05-08T21:39:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-05-08T21:39:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-05-08T21:39:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-05-08T21:39:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-05-08T21:39:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-05-08T21:39:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-05-08T21:39:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-05-08T21:39:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-05-08T21:39:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-05-08T21:39:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-05-08T21:39:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-05-08T21:39:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-05-08T21:39:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-05-08T21:39:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-05-08T21:39:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-05-08T21:39:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-05-08T21:39:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-05-08T21:39:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-05-08T21:39:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-08T21:39:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-08T21:39:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-05-08T21:39:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-05-08T21:39:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-05-08T21:39:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-05-08T21:39:40.910 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-05-08T21:39:40.910 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-05-08T21:39:40.910 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-05-08T21:39:40.910 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-05-08T21:39:40.910 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-05-08T21:39:40.910 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-05-08T21:39:40.910 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-05-08T21:39:40.910 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-08T21:39:40.910 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-05-08T21:39:40.910 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-05-08T21:39:40.910 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-05-08T21:39:40.910 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-05-08T21:39:40.910 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-05-08T21:39:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-05-08T21:39:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-05-08T21:39:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-08T21:39:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-05-08T21:39:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-05-08T21:39:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-05-08T21:39:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-05-08T21:39:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-05-08T21:39:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-05-08T21:39:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-08T21:39:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-08T21:39:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-08T21:39:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-05-08T21:39:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-08T21:39:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-05-08T21:39:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-08T21:39:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-08T21:39:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-08T21:39:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-08T21:39:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-08T21:39:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-08T21:39:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-05-08T21:39:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-05-08T21:39:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-05-08T21:39:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-05-08T21:39:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-05-08T21:39:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-05-08T21:39:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-05-08T21:39:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-08T21:39:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-08T21:39:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-08T21:39:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-08T21:39:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-08T21:39:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-05-08T21:39:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-05-08T21:39:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-05-08T21:39:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-08T21:39:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-08T21:39:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-08T21:39:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-08T21:39:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-08T21:39:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-08T21:39:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-05-08T21:39:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-05-08T21:39:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-08T21:39:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-05-08T21:39:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-05-08T21:39:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-05-08T21:39:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-05-08T21:39:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-08T21:39:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-08T21:39:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-08T21:39:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-05-08T21:39:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-05-08T21:39:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-05-08T21:39:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-08T21:39:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-05-08T21:39:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-05-08T21:39:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-05-08T21:39:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-08T21:39:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-05-08T21:39:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-05-08T21:39:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-08T21:39:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-05-08T21:39:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-08T21:39:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-05-08T21:39:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-05-08T21:39:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-05-08T21:39:40.916 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-05-08T21:39:40.916 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-05-08T21:39:40.916 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-05-08T21:39:40.916 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-05-08T21:39:40.916 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-05-08T21:39:40.916 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-05-08T21:39:40.916 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-05-08T21:39:40.916 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-05-08T21:39:40.916 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-05-08T21:39:40.916 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-05-08T21:39:40.916 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-05-08T21:39:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-05-08T21:39:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-05-08T21:39:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-08T21:39:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-05-08T21:39:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-05-08T21:39:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-08T21:39:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-05-08T21:39:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-05-08T21:39:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-05-08T21:39:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-05-08T21:39:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-05-08T21:39:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-05-08T21:39:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-05-08T21:39:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-05-08T21:39:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-05-08T21:39:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-05-08T21:39:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-05-08T21:39:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-08T21:39:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-08T21:39:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-05-08T21:39:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-05-08T21:39:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-05-08T21:39:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-05-08T21:39:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-05-08T21:39:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-05-08T21:39:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-05-08T21:39:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-05-08T21:39:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-08T21:39:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-08T21:39:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-05-08T21:39:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-05-08T21:39:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-05-08T21:39:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-05-08T21:39:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-05-08T21:39:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-05-08T21:39:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-05-08T21:39:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-05-08T21:39:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-05-08T21:39:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-08T21:39:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-05-08T21:39:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-08T21:39:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-08T21:39:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-08T21:39:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-08T21:39:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-08T21:39:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-08T21:39:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-08T21:39:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-08T21:39:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-08T21:39:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-05-08T21:39:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-05-08T21:39:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-05-08T21:39:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-08T21:39:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-05-08T21:39:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-05-08T21:39:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-05-08T21:39:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-05-08T21:39:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-05-08T21:39:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-08T21:39:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-05-08T21:39:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-05-08T21:39:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-05-08T21:39:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-05-08T21:39:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-05-08T21:39:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-05-08T21:39:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-05-08T21:39:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-05-08T21:39:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-05-08T21:39:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-05-08T21:39:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-05-08T21:39:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-05-08T21:39:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-05-08T21:39:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-05-08T21:39:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-05-08T21:39:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-05-08T21:39:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-05-08T21:39:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-05-08T21:39:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-05-08T21:39:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-08T21:39:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-05-08T21:39:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-05-08T21:39:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-05-08T21:39:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-05-08T21:39:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-05-08T21:39:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-05-08T21:39:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-05-08T21:39:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-05-08T21:39:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-05-08T21:39:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-05-08T21:39:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-05-08T21:39:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-05-08T21:39:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-05-08T21:39:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-05-08T21:39:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-05-08T21:39:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-05-08T21:39:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-05-08T21:39:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-05-08T21:39:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-05-08T21:39:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-08T21:39:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-05-08T21:39:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-05-08T21:39:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-08T21:39:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-05-08T21:39:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-05-08T21:39:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-05-08T21:39:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-05-08T21:39:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-05-08T21:39:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-05-08T21:39:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-05-08T21:39:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-05-08T21:39:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-05-08T21:39:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-05-08T21:39:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-05-08T21:39:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-05-08T21:39:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-05-08T21:39:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-05-08T21:39:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-08T21:39:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-05-08T21:39:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-05-08T21:39:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-05-08T21:39:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-05-08T21:39:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-05-08T21:39:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-05-08T21:39:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-05-08T21:39:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-05-08T21:39:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-05-08T21:39:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-05-08T21:39:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-05-08T21:39:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-05-08T21:39:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-05-08T21:39:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-05-08T21:39:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-05-08T21:39:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-05-08T21:39:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-05-08T21:39:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-05-08T21:39:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-05-08T21:39:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-05-08T21:39:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-05-08T21:39:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-05-08T21:39:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-05-08T21:39:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-05-08T21:39:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-05-08T21:39:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-05-08T21:39:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-05-08T21:39:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-05-08T21:39:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-05-08T21:39:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-05-08T21:39:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-05-08T21:39:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-05-08T21:39:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-05-08T21:39:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-05-08T21:39:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-05-08T21:39:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-05-08T21:39:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-05-08T21:39:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-05-08T21:39:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-05-08T21:39:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-05-08T21:39:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-05-08T21:39:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-05-08T21:39:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-05-08T21:39:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-05-08T21:39:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-05-08T21:39:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-05-08T21:39:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-05-08T21:39:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-05-08T21:39:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-08T21:39:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-08T21:39:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-08T21:39:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-08T21:39:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-08T21:39:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-08T21:39:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-08T21:39:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-08T21:39:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-08T21:39:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-08T21:39:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-08T21:39:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-08T21:39:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-05-08T21:39:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-05-08T21:39:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-05-08T21:39:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-08T21:39:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-05-08T21:39:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-05-08T21:39:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-05-08T21:39:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-05-08T21:39:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-05-08T21:39:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-05-08T21:39:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-05-08T21:39:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-05-08T21:39:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-05-08T21:39:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-05-08T21:39:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-05-08T21:39:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-05-08T21:39:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-05-08T21:39:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-05-08T21:39:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-05-08T21:39:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-05-08T21:39:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-05-08T21:39:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-05-08T21:39:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-05-08T21:39:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-05-08T21:39:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-05-08T21:39:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-05-08T21:39:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-05-08T21:39:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-05-08T21:39:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-05-08T21:39:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-05-08T21:39:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-05-08T21:39:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-08T21:39:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-08T21:39:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-05-08T21:39:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-08T21:39:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-08T21:39:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-08T21:39:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-08T21:39:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-08T21:39:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-08T21:39:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-08T21:39:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-08T21:39:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-08T21:39:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-08T21:39:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-08T21:39:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-08T21:39:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-08T21:39:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-08T21:39:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-08T21:39:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-08T21:39:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-08T21:39:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-08T21:39:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-08T21:39:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-08T21:39:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-08T21:39:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-08T21:39:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-08T21:39:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-05-08T21:39:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-05-08T21:39:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-08T21:39:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-05-08T21:39:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-05-08T21:39:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-05-08T21:39:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-05-08T21:39:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-08T21:39:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-05-08T21:39:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-08T21:39:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-08T21:39:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-08T21:39:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-08T21:39:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-08T21:39:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-05-08T21:39:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-05-08T21:39:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-08T21:39:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-05-08T21:39:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-08T21:39:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-05-08T21:39:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-08T21:39:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-05-08T21:39:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-08T21:39:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-05-08T21:39:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-05-08T21:39:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-08T21:39:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-05-08T21:39:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-08T21:39:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-05-08T21:39:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-08T21:39:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-05-08T21:39:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-08T21:39:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-05-08T21:39:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-08T21:39:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-05-08T21:39:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-05-08T21:39:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-08T21:39:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-08T21:39:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-05-08T21:39:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-05-08T21:39:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-05-08T21:39:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-08T21:39:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-05-08T21:39:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-05-08T21:39:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-05-08T21:39:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-05-08T21:39:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-08T21:39:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-05-08T21:39:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-08T21:39:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-08T21:39:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-08T21:39:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-05-08T21:39:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-08T21:39:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-05-08T21:39:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-05-08T21:39:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-05-08T21:39:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-08T21:39:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-05-08T21:39:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-05-08T21:39:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-05-08T21:39:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-05-08T21:39:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-05-08T21:39:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-05-08T21:39:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-05-08T21:39:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-05-08T21:39:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-08T21:39:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-08T21:39:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-08T21:39:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-08T21:39:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-08T21:39:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-08T21:39:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-08T21:39:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-08T21:39:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-08T21:39:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-05-08T21:39:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-05-08T21:39:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-05-08T21:39:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-05-08T21:39:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-05-08T21:39:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-05-08T21:39:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-05-08T21:39:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-05-08T21:39:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-05-08T21:39:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-05-08T21:39:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-05-08T21:39:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-05-08T21:39:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-05-08T21:39:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-05-08T21:39:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-05-08T21:39:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-05-08T21:39:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-05-08T21:39:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-05-08T21:39:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-05-08T21:39:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-08T21:39:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-05-08T21:39:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-05-08T21:39:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-05-08T21:39:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-05-08T21:39:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-05-08T21:39:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-05-08T21:39:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-05-08T21:39:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-05-08T21:39:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-05-08T21:39:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-05-08T21:39:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-05-08T21:39:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-05-08T21:39:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-05-08T21:39:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-05-08T21:39:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-05-08T21:39:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-05-08T21:39:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-05-08T21:39:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-05-08T21:39:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-05-08T21:39:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-05-08T21:39:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-05-08T21:39:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-05-08T21:39:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-05-08T21:39:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-08T21:39:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-05-08T21:39:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-05-08T21:39:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-05-08T21:39:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-05-08T21:39:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-05-08T21:39:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-05-08T21:39:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-05-08T21:39:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-08T21:39:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-05-08T21:39:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-05-08T21:39:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-05-08T21:39:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-05-08T21:39:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-05-08T21:39:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-05-08T21:39:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-08T21:39:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-08T21:39:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-05-08T21:39:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-05-08T21:39:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-05-08T21:39:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-05-08T21:39:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-05-08T21:39:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-08T21:39:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-05-08T21:39:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-08T21:39:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-08T21:39:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-08T21:39:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-05-08T21:39:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-05-08T21:39:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-05-08T21:39:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-05-08T21:39:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-05-08T21:39:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-08T21:39:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-05-08T21:39:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-05-08T21:39:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-05-08T21:39:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-05-08T21:39:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-05-08T21:39:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-08T21:39:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-08T21:39:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-08T21:39:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-05-08T21:39:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-08T21:39:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-05-08T21:39:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-05-08T21:39:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-05-08T21:39:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-05-08T21:39:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-08T21:39:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-08T21:39:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-08T21:39:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-08T21:39:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-05-08T21:39:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-08T21:39:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-08T21:39:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-08T21:39:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-08T21:39:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-05-08T21:39:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-08T21:39:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-05-08T21:39:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-08T21:39:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-05-08T21:39:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-08T21:39:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-05-08T21:39:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-08T21:39:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-05-08T21:39:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-08T21:39:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-08T21:39:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-08T21:39:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-05-08T21:39:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-05-08T21:39:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-08T21:39:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-08T21:39:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-08T21:39:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-08T21:39:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-08T21:39:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-05-08T21:39:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-08T21:39:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-08T21:39:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-05-08T21:39:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-08T21:39:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-08T21:39:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-08T21:39:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-08T21:39:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-08T21:39:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-05-08T21:39:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-08T21:39:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-08T21:39:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-05-08T21:39:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-05-08T21:39:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-08T21:39:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-05-08T21:39:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-08T21:39:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-08T21:39:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-08T21:39:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-08T21:39:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-08T21:39:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-08T21:39:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-08T21:39:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-08T21:39:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-08T21:39:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-08T21:39:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-05-08T21:39:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-05-08T21:39:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-08T21:39:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-05-08T21:39:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-05-08T21:39:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-08T21:39:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-08T21:39:40.950 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-08T21:39:40.950 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-08T21:39:40.950 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-08T21:39:40.950 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-08T21:39:40.950 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-05-08T21:39:40.950 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-08T21:39:40.950 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-08T21:39:40.950 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-05-08T21:39:40.950 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-05-08T21:39:40.950 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-05-08T21:39:40.950 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-08T21:39:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-05-08T21:39:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-05-08T21:39:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-05-08T21:39:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-08T21:39:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-05-08T21:39:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-08T21:39:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-05-08T21:39:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-08T21:39:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-05-08T21:39:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-08T21:39:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-05-08T21:39:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-08T21:39:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-05-08T21:39:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-08T21:39:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-05-08T21:39:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-08T21:39:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-05-08T21:39:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-08T21:39:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-08T21:39:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-08T21:39:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-08T21:39:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-08T21:39:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-05-08T21:39:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-05-08T21:39:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-08T21:39:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-08T21:39:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-08T21:39:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-05-08T21:39:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-08T21:39:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-08T21:39:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-08T21:39:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-08T21:39:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-08T21:39:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-08T21:39:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-05-08T21:39:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-08T21:39:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-05-08T21:39:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-08T21:39:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-08T21:39:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-05-08T21:39:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-08T21:39:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-08T21:39:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-05-08T21:39:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-08T21:39:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-05-08T21:39:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-08T21:39:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-05-08T21:39:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-08T21:39:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-08T21:39:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-05-08T21:39:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-08T21:39:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-08T21:39:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-08T21:39:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-08T21:39:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-08T21:39:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-05-08T21:39:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-08T21:39:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-05-08T21:39:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-08T21:39:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-08T21:39:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-08T21:39:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-08T21:39:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-05-08T21:39:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-08T21:39:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-05-08T21:39:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-08T21:39:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-05-08T21:39:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-08T21:39:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-05-08T21:39:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-08T21:39:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-08T21:39:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-05-08T21:39:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-08T21:39:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-08T21:39:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-08T21:39:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-08T21:39:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-05-08T21:39:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-08T21:39:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-08T21:39:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-08T21:39:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-05-08T21:39:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-08T21:39:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-08T21:39:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-08T21:39:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-08T21:39:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-05-08T21:39:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-08T21:39:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-05-08T21:39:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-08T21:39:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-05-08T21:39:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-08T21:39:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-08T21:39:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-05-08T21:39:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-08T21:39:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-08T21:39:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-05-08T21:39:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-08T21:39:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-05-08T21:39:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-05-08T21:39:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-08T21:39:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-05-08T21:39:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-08T21:39:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-05-08T21:39:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-05-08T21:39:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-08T21:39:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-05-08T21:39:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-05-08T21:39:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-05-08T21:39:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-05-08T21:39:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-08T21:39:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-08T21:39:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-08T21:39:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-05-08T21:39:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-05-08T21:39:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-05-08T21:39:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-05-08T21:39:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-05-08T21:39:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-08T21:39:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-08T21:39:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-05-08T21:39:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-05-08T21:39:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-08T21:39:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-08T21:39:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-08T21:39:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-05-08T21:39:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-05-08T21:39:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-05-08T21:39:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-05-08T21:39:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-05-08T21:39:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-05-08T21:39:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-05-08T21:39:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-05-08T21:39:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-05-08T21:39:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-05-08T21:39:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-05-08T21:39:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-05-08T21:39:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-05-08T21:39:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-05-08T21:39:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-05-08T21:39:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-05-08T21:39:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-05-08T21:39:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-05-08T21:39:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-08T21:39:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-05-08T21:39:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-08T21:39:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-08T21:39:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-08T21:39:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-08T21:39:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-05-08T21:39:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-05-08T21:39:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-05-08T21:39:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-05-08T21:39:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-05-08T21:39:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-05-08T21:39:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-05-08T21:39:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-05-08T21:39:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-05-08T21:39:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-05-08T21:39:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-08T21:39:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-08T21:39:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-08T21:39:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-05-08T21:39:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-08T21:39:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-05-08T21:39:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-05-08T21:39:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-05-08T21:39:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-05-08T21:39:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-08T21:39:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-05-08T21:39:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-05-08T21:39:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-05-08T21:39:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-08T21:39:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-05-08T21:39:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-05-08T21:39:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-05-08T21:39:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-05-08T21:39:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-05-08T21:39:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-05-08T21:39:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-05-08T21:39:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-05-08T21:39:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-05-08T21:39:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-05-08T21:39:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-05-08T21:39:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-05-08T21:39:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-05-08T21:39:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-05-08T21:39:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-08T21:39:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-05-08T21:39:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-08T21:39:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-05-08T21:39:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-05-08T21:39:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-05-08T21:39:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-05-08T21:39:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-08T21:39:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-05-08T21:39:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-05-08T21:39:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-08T21:39:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-05-08T21:39:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-05-08T21:39:40.966 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-08T21:39:40.966 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-05-08T21:39:40.966 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-08T21:39:40.966 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-08T21:39:40.966 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-08T21:39:40.966 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-08T21:39:40.966 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-08T21:39:40.966 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-08T21:39:40.966 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-08T21:39:40.966 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-08T21:39:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-08T21:39:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-05-08T21:39:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-05-08T21:39:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-08T21:39:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-05-08T21:39:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-08T21:39:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-08T21:39:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-08T21:39:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-08T21:39:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-08T21:39:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-08T21:39:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-08T21:39:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-08T21:39:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-08T21:39:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-05-08T21:39:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-05-08T21:39:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-08T21:39:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-08T21:39:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-05-08T21:39:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-05-08T21:39:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-08T21:39:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-08T21:39:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-05-08T21:39:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-05-08T21:39:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-05-08T21:39:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-05-08T21:39:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-05-08T21:39:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-05-08T21:39:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-05-08T21:39:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-05-08T21:39:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-08T21:39:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-05-08T21:39:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-05-08T21:39:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-08T21:39:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-08T21:39:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-08T21:39:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-08T21:39:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-08T21:39:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-08T21:39:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-08T21:39:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-08T21:39:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-08T21:39:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-08T21:39:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-05-08T21:39:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-08T21:39:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-08T21:39:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-08T21:39:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-08T21:39:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-08T21:39:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-08T21:39:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-05-08T21:39:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-08T21:39:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-08T21:39:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-08T21:39:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-08T21:39:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-08T21:39:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-08T21:39:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-08T21:39:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-08T21:39:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-08T21:39:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-08T21:39:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-08T21:39:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-08T21:39:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-05-08T21:39:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-05-08T21:39:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-05-08T21:39:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-05-08T21:39:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-05-08T21:39:40.972 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-05-08T21:39:40.972 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-05-08T21:39:40.972 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-05-08T21:39:40.972 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-08T21:39:40.972 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-05-08T21:39:40.972 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-05-08T21:39:40.972 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-08T21:39:40.972 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-08T21:39:40.972 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-05-08T21:39:40.972 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-05-08T21:39:40.972 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-08T21:39:40.972 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-05-08T21:39:40.972 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-05-08T21:39:40.973 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-08T21:39:40.973 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-05-08T21:39:40.973 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-05-08T21:39:40.973 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-05-08T21:39:40.973 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-08T21:39:40.973 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-05-08T21:39:40.973 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-08T21:39:40.973 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-05-08T21:39:40.973 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-05-08T21:39:40.973 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-08T21:39:40.973 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-05-08T21:39:40.973 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-08T21:39:40.973 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-08T21:39:40.973 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-08T21:39:40.973 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-08T21:39:40.974 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-08T21:39:40.974 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-05-08T21:39:40.974 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-05-08T21:39:40.974 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-08T21:39:40.974 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-08T21:39:40.974 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-08T21:39:40.974 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-08T21:39:40.974 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-05-08T21:39:40.974 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-05-08T21:39:40.974 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-08T21:39:40.974 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-05-08T21:39:40.974 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-05-08T21:39:40.974 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-08T21:39:40.974 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-08T21:39:40.975 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-05-08T21:39:40.975 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-05-08T21:39:40.975 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-05-08T21:39:40.975 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-05-08T21:39:40.975 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-05-08T21:39:40.975 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-05-08T21:39:40.975 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-05-08T21:39:40.975 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-05-08T21:39:40.975 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-05-08T21:39:40.975 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-05-08T21:39:40.975 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-08T21:39:40.975 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-08T21:39:40.975 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-05-08T21:39:40.975 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-05-08T21:39:40.976 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-05-08T21:39:40.976 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-08T21:39:40.976 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-05-08T21:39:40.976 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-05-08T21:39:40.976 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-05-08T21:39:40.976 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-08T21:39:40.976 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-05-08T21:39:40.976 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-05-08T21:39:40.976 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-08T21:39:40.976 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-08T21:39:40.976 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-08T21:39:40.976 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-05-08T21:39:40.976 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-08T21:39:40.977 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-05-08T21:39:40.977 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-08T21:39:40.977 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-05-08T21:39:40.977 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-05-08T21:39:40.977 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-08T21:39:40.977 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-05-08T21:39:40.977 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-05-08T21:39:40.977 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-05-08T21:39:40.977 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-08T21:39:40.977 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-08T21:39:40.977 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-05-08T21:39:40.977 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-08T21:39:40.978 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-05-08T21:39:40.978 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-05-08T21:39:40.978 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-08T21:39:40.978 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-08T21:39:40.978 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-08T21:39:40.978 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-08T21:39:40.978 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-08T21:39:40.978 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-05-08T21:39:40.978 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-05-08T21:39:40.978 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-05-08T21:39:40.978 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-05-08T21:39:40.978 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-08T21:39:40.978 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-05-08T21:39:40.978 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-08T21:39:40.978 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-05-08T21:39:40.979 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-05-08T21:39:40.979 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-05-08T21:39:40.979 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-05-08T21:39:40.979 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-05-08T21:39:40.979 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-08T21:39:40.979 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-05-08T21:39:40.979 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-08T21:39:40.979 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-05-08T21:39:40.979 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-05-08T21:39:40.979 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-05-08T21:39:40.979 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-05-08T21:39:40.979 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-05-08T21:39:40.979 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-05-08T21:39:40.979 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-08T21:39:40.980 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-05-08T21:39:40.980 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-05-08T21:39:40.980 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-05-08T21:39:40.980 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-08T21:39:40.980 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-08T21:39:40.980 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-08T21:39:40.980 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-08T21:39:40.980 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-08T21:39:40.980 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-08T21:39:40.980 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-08T21:39:40.980 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-08T21:39:40.980 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-08T21:39:40.980 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-08T21:39:40.980 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-08T21:39:40.981 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-08T21:39:40.981 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-08T21:39:40.981 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-08T21:39:40.981 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-08T21:39:40.981 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-08T21:39:40.981 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-08T21:39:40.981 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-08T21:39:40.981 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-05-08T21:39:40.981 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-05-08T21:39:40.981 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-08T21:39:40.981 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-05-08T21:39:40.981 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-05-08T21:39:40.981 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-05-08T21:39:40.981 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-05-08T21:39:40.982 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-05-08T21:39:40.982 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-08T21:39:40.982 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-05-08T21:39:40.982 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-05-08T21:39:40.982 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-08T21:39:40.982 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-05-08T21:39:40.982 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-05-08T21:39:40.982 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-08T21:39:40.982 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-05-08T21:39:40.982 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-08T21:39:40.982 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-05-08T21:39:40.982 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-05-08T21:39:40.982 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-05-08T21:39:40.982 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-05-08T21:39:40.983 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-05-08T21:39:40.983 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-05-08T21:39:40.983 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-05-08T21:39:40.983 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-05-08T21:39:40.983 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-05-08T21:39:40.983 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-08T21:39:40.983 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-05-08T21:39:40.983 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-05-08T21:39:40.983 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-05-08T21:39:40.983 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-08T21:39:40.983 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-05-08T21:39:40.983 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-05-08T21:39:40.983 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-05-08T21:39:40.983 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-05-08T21:39:40.983 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-05-08T21:39:40.984 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-05-08T21:39:40.984 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-05-08T21:39:40.984 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-05-08T21:39:40.984 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-05-08T21:39:40.984 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-05-08T21:39:40.984 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-05-08T21:39:40.984 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-08T21:39:40.984 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-05-08T21:39:40.984 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-05-08T21:39:40.984 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-05-08T21:39:40.984 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-08T21:39:40.984 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-08T21:39:40.984 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-05-08T21:39:40.984 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-05-08T21:39:40.984 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-08T21:39:40.985 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-05-08T21:39:40.985 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-05-08T21:39:40.985 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-05-08T21:39:40.985 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-05-08T21:39:40.985 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-05-08T21:39:40.985 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-05-08T21:39:40.985 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-05-08T21:39:40.985 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-08T21:39:40.985 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-05-08T21:39:40.985 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-05-08T21:39:40.985 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-05-08T21:39:40.985 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-08T21:39:40.985 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-08T21:39:40.985 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-05-08T21:39:40.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-08T21:39:40.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-08T21:39:40.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-08T21:39:40.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-05-08T21:39:40.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-05-08T21:39:40.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-05-08T21:39:40.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-05-08T21:39:40.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-08T21:39:40.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-05-08T21:39:40.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-08T21:39:40.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-08T21:39:40.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-08T21:39:40.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-05-08T21:39:40.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-05-08T21:39:40.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-05-08T21:39:40.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-05-08T21:39:40.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-05-08T21:39:40.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-05-08T21:39:40.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-05-08T21:39:40.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-05-08T21:39:40.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-05-08T21:39:40.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-05-08T21:39:40.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-05-08T21:39:40.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-05-08T21:39:40.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-05-08T21:39:40.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-05-08T21:39:40.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-08T21:39:40.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-05-08T21:39:40.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-05-08T21:39:40.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-08T21:39:40.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-08T21:39:40.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-05-08T21:39:40.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-08T21:39:40.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-05-08T21:39:40.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-05-08T21:39:40.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-08T21:39:40.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-05-08T21:39:40.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-05-08T21:39:40.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-08T21:39:40.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-08T21:39:40.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-08T21:39:40.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-08T21:39:40.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-08T21:39:40.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-08T21:39:40.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-08T21:39:40.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-05-08T21:39:40.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-08T21:39:40.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-08T21:39:40.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-08T21:39:40.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-08T21:39:40.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-08T21:39:40.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-05-08T21:39:40.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-05-08T21:39:40.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-08T21:39:40.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-08T21:39:40.990 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-08T21:39:40.990 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-05-08T21:39:40.990 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-05-08T21:39:40.990 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-08T21:39:40.990 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-08T21:39:40.990 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-05-08T21:39:40.990 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-08T21:39:40.990 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-05-08T21:39:40.990 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-05-08T21:39:40.990 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-08T21:39:40.990 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-05-08T21:39:40.990 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-05-08T21:39:40.990 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-05-08T21:39:40.990 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-05-08T21:39:40.991 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-05-08T21:39:40.991 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-05-08T21:39:40.991 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-08T21:39:40.991 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-05-08T21:39:40.991 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-05-08T21:39:40.991 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-05-08T21:39:40.991 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-05-08T21:39:40.991 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-05-08T21:39:40.991 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-08T21:39:40.991 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-08T21:39:40.991 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-05-08T21:39:40.991 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-05-08T21:39:40.991 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-05-08T21:39:40.991 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-05-08T21:39:40.991 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-05-08T21:39:40.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-05-08T21:39:40.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-08T21:39:40.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-08T21:39:40.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-08T21:39:40.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-08T21:39:40.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-08T21:39:40.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-08T21:39:40.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-08T21:39:40.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-08T21:39:40.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-08T21:39:40.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-08T21:39:40.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-08T21:39:40.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-05-08T21:39:40.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-05-08T21:39:40.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-08T21:39:40.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-08T21:39:40.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-08T21:39:40.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-05-08T21:39:40.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-05-08T21:39:40.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-08T21:39:40.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-08T21:39:40.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-05-08T21:39:40.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-05-08T21:39:40.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-08T21:39:40.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-08T21:39:40.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-05-08T21:39:40.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-05-08T21:39:40.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-05-08T21:39:40.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-08T21:39:40.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-08T21:39:40.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-08T21:39:40.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-08T21:39:40.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-05-08T21:39:40.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-08T21:39:40.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-05-08T21:39:40.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-08T21:39:40.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-08T21:39:40.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-08T21:39:40.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-05-08T21:39:40.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-05-08T21:39:40.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-05-08T21:39:40.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-08T21:39:40.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-05-08T21:39:40.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-05-08T21:39:40.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-05-08T21:39:40.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-05-08T21:39:40.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-08T21:39:40.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-05-08T21:39:40.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-05-08T21:39:40.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-08T21:39:40.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-05-08T21:39:40.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-05-08T21:39:40.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-05-08T21:39:40.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-05-08T21:39:40.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-05-08T21:39:40.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-05-08T21:39:40.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-05-08T21:39:40.996 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-05-08T21:39:40.996 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-05-08T21:39:40.996 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-05-08T21:39:40.996 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-05-08T21:39:40.996 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-05-08T21:39:40.996 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-05-08T21:39:40.996 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-05-08T21:39:40.996 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-05-08T21:39:40.996 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-05-08T21:39:40.996 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-05-08T21:39:40.996 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-05-08T21:39:40.996 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-05-08T21:39:40.996 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-05-08T21:39:40.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-05-08T21:39:40.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-08T21:39:40.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-05-08T21:39:40.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-05-08T21:39:40.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-05-08T21:39:40.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-05-08T21:39:40.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-05-08T21:39:40.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-05-08T21:39:40.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-05-08T21:39:40.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-05-08T21:39:40.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-05-08T21:39:40.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-05-08T21:39:40.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-05-08T21:39:40.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-05-08T21:39:40.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-05-08T21:39:40.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-05-08T21:39:40.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-05-08T21:39:40.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-05-08T21:39:40.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-05-08T21:39:40.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-05-08T21:39:40.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-05-08T21:39:40.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-05-08T21:39:40.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-05-08T21:39:40.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-05-08T21:39:40.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-05-08T21:39:40.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-05-08T21:39:40.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-05-08T21:39:40.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-05-08T21:39:40.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-05-08T21:39:40.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-08T21:39:40.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-05-08T21:39:40.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-08T21:39:40.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-05-08T21:39:40.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-08T21:39:40.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-05-08T21:39:40.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-05-08T21:39:40.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-05-08T21:39:40.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-05-08T21:39:40.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-05-08T21:39:41.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-05-08T21:39:41.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-05-08T21:39:41.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-05-08T21:39:41.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-05-08T21:39:41.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-05-08T21:39:41.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-05-08T21:39:41.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-05-08T21:39:41.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-05-08T21:39:41.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-05-08T21:39:41.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-05-08T21:39:41.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-05-08T21:39:41.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-05-08T21:39:41.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-05-08T21:39:41.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-05-08T21:39:41.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-05-08T21:39:41.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-05-08T21:39:41.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-05-08T21:39:41.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-05-08T21:39:41.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-05-08T21:39:41.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-05-08T21:39:41.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-05-08T21:39:41.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-05-08T21:39:41.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-05-08T21:39:41.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-05-08T21:39:41.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-05-08T21:39:41.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-05-08T21:39:41.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-05-08T21:39:41.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-05-08T21:39:41.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-05-08T21:39:41.002 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-05-08T21:39:41.002 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-05-08T21:39:41.002 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-05-08T21:39:41.002 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-05-08T21:39:41.002 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-05-08T21:39:41.002 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-05-08T21:39:41.002 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-05-08T21:39:41.002 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-05-08T21:39:41.002 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-05-08T21:39:41.002 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-05-08T21:39:41.002 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-05-08T21:39:41.002 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-05-08T21:39:41.002 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-05-08T21:39:41.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-05-08T21:39:41.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-05-08T21:39:41.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-05-08T21:39:41.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-05-08T21:39:41.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-05-08T21:39:41.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-05-08T21:39:41.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-05-08T21:39:41.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-05-08T21:39:41.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-05-08T21:39:41.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-05-08T21:39:41.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-05-08T21:39:41.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-05-08T21:39:41.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-08T21:39:41.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-05-08T21:39:41.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-05-08T21:39:41.004 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-05-08T21:39:41.004 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-08T21:39:41.004 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-05-08T21:39:41.004 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-08T21:39:41.004 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-05-08T21:39:41.004 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-08T21:39:41.004 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-05-08T21:39:41.004 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-08T21:39:41.004 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-08T21:39:41.004 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-08T21:39:41.004 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-05-08T21:39:41.004 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-05-08T21:39:41.004 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-05-08T21:39:41.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-05-08T21:39:41.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-05-08T21:39:41.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-05-08T21:39:41.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-05-08T21:39:41.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-05-08T21:39:41.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-05-08T21:39:41.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-05-08T21:39:41.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-05-08T21:39:41.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-05-08T21:39:41.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-05-08T21:39:41.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-05-08T21:39:41.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-05-08T21:39:41.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-05-08T21:39:41.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-05-08T21:39:41.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-05-08T21:39:41.006 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-05-08T21:39:41.006 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-05-08T21:39:41.006 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-05-08T21:39:41.006 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-05-08T21:39:41.006 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-05-08T21:39:41.006 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-05-08T21:39:41.006 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-05-08T21:39:41.006 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-05-08T21:39:41.006 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-05-08T21:39:41.006 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-05-08T21:39:41.006 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-05-08T21:39:41.006 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-08T21:39:41.007 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-05-08T21:39:41.007 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-05-08T21:39:41.007 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-05-08T21:39:41.007 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-05-08T21:39:41.007 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-05-08T21:39:41.007 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-05-08T21:39:41.007 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-05-08T21:39:41.007 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-05-08T21:39:41.007 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-05-08T21:39:41.007 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-05-08T21:39:41.007 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-05-08T21:39:41.007 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-05-08T21:39:41.007 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-05-08T21:39:41.007 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-05-08T21:39:41.008 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-05-08T21:39:41.008 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-05-08T21:39:41.008 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-05-08T21:39:41.008 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-08T21:39:41.008 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-05-08T21:39:41.008 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-05-08T21:39:41.008 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-05-08T21:39:41.008 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-05-08T21:39:41.008 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-05-08T21:39:41.008 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-05-08T21:39:41.008 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-05-08T21:39:41.008 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-05-08T21:39:41.008 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-05-08T21:39:41.009 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-05-08T21:39:41.009 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-05-08T21:39:41.009 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-05-08T21:39:41.009 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-05-08T21:39:41.009 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-08T21:39:41.009 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-05-08T21:39:41.009 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-05-08T21:39:41.009 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-05-08T21:39:41.009 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-05-08T21:39:41.009 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-05-08T21:39:41.009 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-05-08T21:39:41.009 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-05-08T21:39:41.009 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-05-08T21:39:41.009 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-05-08T21:39:41.010 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-05-08T21:39:41.010 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-05-08T21:39:41.010 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-05-08T21:39:41.010 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-05-08T21:39:41.010 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-05-08T21:39:41.010 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-05-08T21:39:41.010 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-05-08T21:39:41.010 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-05-08T21:39:41.010 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-05-08T21:39:41.010 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-05-08T21:39:41.010 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-05-08T21:39:41.010 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-05-08T21:39:41.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-05-08T21:39:41.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-08T21:39:41.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-08T21:39:41.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-08T21:39:41.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-08T21:39:41.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-08T21:39:41.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-08T21:39:41.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-08T21:39:41.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-08T21:39:41.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-08T21:39:41.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-08T21:39:41.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-08T21:39:41.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-08T21:39:41.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-08T21:39:41.012 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-08T21:39:41.012 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-08T21:39:41.012 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-08T21:39:41.012 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-08T21:39:41.012 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-08T21:39:41.012 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-08T21:39:41.012 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-08T21:39:41.012 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-05-08T21:39:41.012 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-08T21:39:41.012 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-08T21:39:41.012 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-08T21:39:41.012 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-05-08T21:39:41.012 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-05-08T21:39:41.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-05-08T21:39:41.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-05-08T21:39:41.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-05-08T21:39:41.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-05-08T21:39:41.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-05-08T21:39:41.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-05-08T21:39:41.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-05-08T21:39:41.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-05-08T21:39:41.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-05-08T21:39:41.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-05-08T21:39:41.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-05-08T21:39:41.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-05-08T21:39:41.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-05-08T21:39:41.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-05-08T21:39:41.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-05-08T21:39:41.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-05-08T21:39:41.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-05-08T21:39:41.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-05-08T21:39:41.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-08T21:39:41.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-05-08T21:39:41.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-08T21:39:41.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-05-08T21:39:41.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-05-08T21:39:41.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-05-08T21:39:41.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-05-08T21:39:41.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-05-08T21:39:41.015 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-05-08T21:39:41.015 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-05-08T21:39:41.015 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-05-08T21:39:41.015 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-05-08T21:39:41.015 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-05-08T21:39:41.015 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-05-08T21:39:41.015 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-05-08T21:39:41.015 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-08T21:39:41.015 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-08T21:39:41.015 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-08T21:39:41.015 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-08T21:39:41.015 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-08T21:39:41.015 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-08T21:39:41.016 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-08T21:39:41.016 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-08T21:39:41.016 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-08T21:39:41.016 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-08T21:39:41.016 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-08T21:39:41.016 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-08T21:39:41.016 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-08T21:39:41.016 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-08T21:39:41.016 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-08T21:39:41.016 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-08T21:39:41.016 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-08T21:39:41.017 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-08T21:39:41.017 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-08T21:39:41.017 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-08T21:39:41.017 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-08T21:39:41.017 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-05-08T21:39:41.017 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-05-08T21:39:41.017 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-05-08T21:39:41.017 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-05-08T21:39:41.017 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-05-08T21:39:41.017 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-05-08T21:39:41.017 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-05-08T21:39:41.017 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-05-08T21:39:41.017 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-05-08T21:39:41.017 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-05-08T21:39:41.018 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-05-08T21:39:41.018 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-05-08T21:39:41.018 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-05-08T21:39:41.018 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-05-08T21:39:41.018 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-05-08T21:39:41.018 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-05-08T21:39:41.018 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-08T21:39:41.018 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-05-08T21:39:41.018 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-05-08T21:39:41.018 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-05-08T21:39:41.018 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-05-08T21:39:41.018 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-05-08T21:39:41.018 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-08T21:39:41.019 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-05-08T21:39:41.019 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-05-08T21:39:41.019 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-05-08T21:39:41.019 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-08T21:39:41.019 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-08T21:39:41.019 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-08T21:39:41.019 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-08T21:39:41.019 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-08T21:39:41.019 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-08T21:39:41.019 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-08T21:39:41.019 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-08T21:39:41.019 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-08T21:39:41.019 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-08T21:39:41.019 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-08T21:39:41.020 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-08T21:39:41.020 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-08T21:39:41.020 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-08T21:39:41.020 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-08T21:39:41.020 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-08T21:39:41.020 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-08T21:39:41.020 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-08T21:39:41.020 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-08T21:39:41.020 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-08T21:39:41.020 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-08T21:39:41.020 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-08T21:39:41.020 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-08T21:39:41.020 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-08T21:39:41.021 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-05-08T21:39:41.021 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-05-08T21:39:41.021 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-08T21:39:41.021 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-05-08T21:39:41.021 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-05-08T21:39:41.021 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-05-08T21:39:41.021 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-05-08T21:39:41.021 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-08T21:39:41.021 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-05-08T21:39:41.021 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-05-08T21:39:41.021 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-05-08T21:39:41.021 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-05-08T21:39:41.021 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-05-08T21:39:41.021 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-05-08T21:39:41.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-05-08T21:39:41.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-05-08T21:39:41.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-05-08T21:39:41.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-05-08T21:39:41.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-05-08T21:39:41.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-05-08T21:39:41.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-05-08T21:39:41.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-05-08T21:39:41.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-08T21:39:41.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-08T21:39:41.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-08T21:39:41.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-08T21:39:41.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-08T21:39:41.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-08T21:39:41.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-08T21:39:41.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-08T21:39:41.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-08T21:39:41.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-08T21:39:41.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-08T21:39:41.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-08T21:39:41.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-08T21:39:41.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-08T21:39:41.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-08T21:39:41.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-05-08T21:39:41.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-08T21:39:41.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-08T21:39:41.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-08T21:39:41.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-08T21:39:41.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-08T21:39:41.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-08T21:39:41.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-08T21:39:41.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-08T21:39:41.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-08T21:39:41.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-08T21:39:41.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-08T21:39:41.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-08T21:39:41.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-08T21:39:41.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-08T21:39:41.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-05-08T21:39:41.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-05-08T21:39:41.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-08T21:39:41.025 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-08T21:39:41.025 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-08T21:39:41.025 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-08T21:39:41.025 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-08T21:39:41.025 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-08T21:39:41.025 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-08T21:39:41.025 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-08T21:39:41.025 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-08T21:39:41.025 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-08T21:39:41.025 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-08T21:39:41.025 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-08T21:39:41.025 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-08T21:39:41.025 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-05-08T21:39:41.026 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-08T21:39:41.026 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-05-08T21:39:41.026 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-05-08T21:39:41.026 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-08T21:39:41.026 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-08T21:39:41.026 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-05-08T21:39:41.026 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-05-08T21:39:41.026 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-08T21:39:41.026 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-08T21:39:41.026 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-05-08T21:39:41.026 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-05-08T21:39:41.026 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-08T21:39:41.026 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-05-08T21:39:41.026 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-08T21:39:41.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-08T21:39:41.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-05-08T21:39:41.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-05-08T21:39:41.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-05-08T21:39:41.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-08T21:39:41.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-05-08T21:39:41.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-08T21:39:41.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-05-08T21:39:41.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-05-08T21:39:41.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-05-08T21:39:41.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-05-08T21:39:41.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-05-08T21:39:41.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-05-08T21:39:41.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-05-08T21:39:41.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-05-08T21:39:41.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-05-08T21:39:41.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-05-08T21:39:41.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-05-08T21:39:41.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-05-08T21:39:41.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-05-08T21:39:41.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-05-08T21:39:41.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-08T21:39:41.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-08T21:39:41.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-08T21:39:41.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-05-08T21:39:41.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-05-08T21:39:41.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-05-08T21:39:41.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-05-08T21:39:41.029 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-08T21:39:41.029 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-05-08T21:39:41.029 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-05-08T21:39:41.029 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-08T21:39:41.029 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-05-08T21:39:41.029 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-05-08T21:39:41.029 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-08T21:39:41.029 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-05-08T21:39:41.029 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-08T21:39:41.029 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-05-08T21:39:41.029 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-08T21:39:41.029 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-08T21:39:41.029 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-05-08T21:39:41.030 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-05-08T21:39:41.030 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-05-08T21:39:41.030 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-05-08T21:39:41.030 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-05-08T21:39:41.030 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-05-08T21:39:41.030 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-05-08T21:39:41.030 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-05-08T21:39:41.030 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-05-08T21:39:41.030 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-05-08T21:39:41.030 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-05-08T21:39:41.030 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-05-08T21:39:41.030 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-05-08T21:39:41.030 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-05-08T21:39:41.031 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-05-08T21:39:41.031 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-05-08T21:39:41.031 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-05-08T21:39:41.031 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-05-08T21:39:41.031 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-05-08T21:39:41.031 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-05-08T21:39:41.031 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-05-08T21:39:41.031 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-05-08T21:39:41.031 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-05-08T21:39:41.031 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-05-08T21:39:41.031 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-05-08T21:39:41.031 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-05-08T21:39:41.031 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-05-08T21:39:41.032 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-05-08T21:39:41.032 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-05-08T21:39:41.032 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-05-08T21:39:41.032 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-05-08T21:39:41.032 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-05-08T21:39:41.032 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-05-08T21:39:41.032 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-05-08T21:39:41.032 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-05-08T21:39:41.032 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-05-08T21:39:41.032 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-05-08T21:39:41.032 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-05-08T21:39:41.032 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-05-08T21:39:41.033 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-05-08T21:39:41.033 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-05-08T21:39:41.033 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-05-08T21:39:41.033 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-05-08T21:39:41.033 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-05-08T21:39:41.033 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-05-08T21:39:41.033 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-05-08T21:39:41.033 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-05-08T21:39:41.033 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-05-08T21:39:41.033 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-05-08T21:39:41.033 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-05-08T21:39:41.033 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-05-08T21:39:41.033 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-05-08T21:39:41.033 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-05-08T21:39:41.034 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-05-08T21:39:41.034 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-05-08T21:39:41.034 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-05-08T21:39:41.034 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-05-08T21:39:41.034 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-05-08T21:39:41.034 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-05-08T21:39:41.034 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-05-08T21:39:41.034 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-05-08T21:39:41.034 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-05-08T21:39:41.034 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-05-08T21:39:41.034 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-05-08T21:39:41.034 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-05-08T21:39:41.034 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-05-08T21:39:41.035 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-05-08T21:39:41.035 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-05-08T21:39:41.035 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-05-08T21:39:41.035 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-05-08T21:39:41.035 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-05-08T21:39:41.035 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-05-08T21:39:41.035 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-05-08T21:39:41.035 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-05-08T21:39:41.035 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-05-08T21:39:41.035 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-05-08T21:39:41.035 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-05-08T21:39:41.035 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-05-08T21:39:41.035 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-05-08T21:39:41.035 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-05-08T21:39:41.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-05-08T21:39:41.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-05-08T21:39:41.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-05-08T21:39:41.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-05-08T21:39:41.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-05-08T21:39:41.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-05-08T21:39:41.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-05-08T21:39:41.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-05-08T21:39:41.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-05-08T21:39:41.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-05-08T21:39:41.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-05-08T21:39:41.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-05-08T21:39:41.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-05-08T21:39:41.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-05-08T21:39:41.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-05-08T21:39:41.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-05-08T21:39:41.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-05-08T21:39:41.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-05-08T21:39:41.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-05-08T21:39:41.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-05-08T21:39:41.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-05-08T21:39:41.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-05-08T21:39:41.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-05-08T21:39:41.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-05-08T21:39:41.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-05-08T21:39:41.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-05-08T21:39:41.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-05-08T21:39:41.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-05-08T21:39:41.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-05-08T21:39:41.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-05-08T21:39:41.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-05-08T21:39:41.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-05-08T21:39:41.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-05-08T21:39:41.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-05-08T21:39:41.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-05-08T21:39:41.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-08T21:39:41.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-08T21:39:41.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-05-08T21:39:41.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-05-08T21:39:41.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-08T21:39:41.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-05-08T21:39:41.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-05-08T21:39:41.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-05-08T21:39:41.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-05-08T21:39:41.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-05-08T21:39:41.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-05-08T21:39:41.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-08T21:39:41.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-08T21:39:41.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-05-08T21:39:41.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-08T21:39:41.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-08T21:39:41.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-05-08T21:39:41.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-08T21:39:41.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-05-08T21:39:41.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-05-08T21:39:41.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-05-08T21:39:41.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-08T21:39:41.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-05-08T21:39:41.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-05-08T21:39:41.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-05-08T21:39:41.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-05-08T21:39:41.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-05-08T21:39:41.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-05-08T21:39:41.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-05-08T21:39:41.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-05-08T21:39:41.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-05-08T21:39:41.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-05-08T21:39:41.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-05-08T21:39:41.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-05-08T21:39:41.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-05-08T21:39:41.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-05-08T21:39:41.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-05-08T21:39:41.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-08T21:39:41.045 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-05-08T21:39:41.045 DEBUG:teuthology.orchestra.run.smithi131:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-08T21:39:41.068 INFO:teuthology.orchestra.run.smithi131.stdout:postinstall scriptlet (using /bin/sh): 2024-05-08T21:39:41.069 INFO:teuthology.orchestra.run.smithi131.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-08T21:39:41.069 INFO:teuthology.orchestra.run.smithi131.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-08T21:39:41.069 INFO:teuthology.orchestra.run.smithi131.stdout:fi 2024-05-08T21:39:41.069 INFO:teuthology.orchestra.run.smithi131.stdout:for file in vmlinuz System.map config; do 2024-05-08T21:39:41.069 INFO:teuthology.orchestra.run.smithi131.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-05-08T21:39:41.069 INFO:teuthology.orchestra.run.smithi131.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-05-08T21:39:41.069 INFO:teuthology.orchestra.run.smithi131.stdout: fi 2024-05-08T21:39:41.069 INFO:teuthology.orchestra.run.smithi131.stdout:done 2024-05-08T21:39:41.069 INFO:teuthology.orchestra.run.smithi131.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-05-08T21:39:41.069 INFO:teuthology.orchestra.run.smithi131.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-05-08T21:39:41.069 INFO:teuthology.orchestra.run.smithi131.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-08T21:39:41.070 INFO:teuthology.orchestra.run.smithi131.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-08T21:39:41.070 INFO:teuthology.orchestra.run.smithi131.stdout:fi 2024-05-08T21:39:41.070 INFO:teuthology.orchestra.run.smithi131.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-05-08T21:39:41.070 INFO:teuthology.orchestra.run.smithi131.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-05-08T21:39:41.070 INFO:teuthology.orchestra.run.smithi131.stdout:fi 2024-05-08T21:39:41.070 INFO:teuthology.orchestra.run.smithi131.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-08T21:39:41.070 INFO:teuthology.orchestra.run.smithi131.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-08T21:39:41.070 INFO:teuthology.orchestra.run.smithi131.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-05-08T21:39:41.070 INFO:teuthology.orchestra.run.smithi131.stdout:fi 2024-05-08T21:39:41.071 DEBUG:teuthology.orchestra.run.smithi131:> sudo rpm -qi grub2-tools 2024-05-08T21:39:41.145 INFO:teuthology.orchestra.run.smithi131.stdout:Name : grub2-tools 2024-05-08T21:39:41.145 INFO:teuthology.orchestra.run.smithi131.stdout:Epoch : 1 2024-05-08T21:39:41.145 INFO:teuthology.orchestra.run.smithi131.stdout:Version : 2.02 2024-05-08T21:39:41.145 INFO:teuthology.orchestra.run.smithi131.stdout:Release : 123.el8 2024-05-08T21:39:41.145 INFO:teuthology.orchestra.run.smithi131.stdout:Architecture: x86_64 2024-05-08T21:39:41.146 INFO:teuthology.orchestra.run.smithi131.stdout:Install Date: Thu 14 Dec 2023 12:39:42 AM UTC 2024-05-08T21:39:41.146 INFO:teuthology.orchestra.run.smithi131.stdout:Group : System Environment/Base 2024-05-08T21:39:41.146 INFO:teuthology.orchestra.run.smithi131.stdout:Size : 9265450 2024-05-08T21:39:41.146 INFO:teuthology.orchestra.run.smithi131.stdout:License : GPLv3+ 2024-05-08T21:39:41.146 INFO:teuthology.orchestra.run.smithi131.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2024-05-08T21:39:41.146 INFO:teuthology.orchestra.run.smithi131.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2024-05-08T21:39:41.146 INFO:teuthology.orchestra.run.smithi131.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2024-05-08T21:39:41.146 INFO:teuthology.orchestra.run.smithi131.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2024-05-08T21:39:41.146 INFO:teuthology.orchestra.run.smithi131.stdout:Relocations : (not relocatable) 2024-05-08T21:39:41.146 INFO:teuthology.orchestra.run.smithi131.stdout:Packager : Red Hat, Inc. 2024-05-08T21:39:41.146 INFO:teuthology.orchestra.run.smithi131.stdout:Vendor : Red Hat, Inc. 2024-05-08T21:39:41.146 INFO:teuthology.orchestra.run.smithi131.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T21:39:41.146 INFO:teuthology.orchestra.run.smithi131.stdout:Summary : Support tools for GRUB. 2024-05-08T21:39:41.146 INFO:teuthology.orchestra.run.smithi131.stdout:Description : 2024-05-08T21:39:41.147 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:39:41.147 INFO:teuthology.orchestra.run.smithi131.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T21:39:41.147 INFO:teuthology.orchestra.run.smithi131.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T21:39:41.147 INFO:teuthology.orchestra.run.smithi131.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T21:39:41.147 INFO:teuthology.orchestra.run.smithi131.stdout:hardware devices. 2024-05-08T21:39:41.147 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:39:41.147 INFO:teuthology.orchestra.run.smithi131.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T21:39:41.148 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T21:39:41.148 INFO:teuthology.task.kernel:Updating grub on smithi131 to boot 6.9.0-rc3-g806ec36b2112 2024-05-08T21:39:41.148 DEBUG:teuthology.orchestra.run.smithi131:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T21:39:42.340 INFO:teuthology.orchestra.run.smithi131.stderr:Generating grub configuration file ... 2024-05-08T21:39:44.183 INFO:teuthology.orchestra.run.smithi131.stderr:done 2024-05-08T21:39:44.186 DEBUG:teuthology.orchestra.run.smithi131:> mktemp 2024-05-08T21:39:44.202 INFO:teuthology.orchestra.run.smithi131.stdout:/tmp/tmp.c0wrlyY5rB 2024-05-08T21:39:44.202 DEBUG:teuthology.orchestra.run.smithi131:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.c0wrlyY5rB 2024-05-08T21:39:44.276 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0666 /tmp/tmp.c0wrlyY5rB 2024-05-08T21:39:44.438 DEBUG:teuthology.orchestra.remote:smithi131:/tmp/tmp.c0wrlyY5rB is 6KB 2024-05-08T21:39:44.486 DEBUG:teuthology.orchestra.run.smithi131:> rm -fr /tmp/tmp.c0wrlyY5rB 2024-05-08T21:39:44.502 DEBUG:teuthology.orchestra.run.smithi131:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T21:39:44.567 INFO:teuthology.orchestra.run.smithi131.stdout:2db68f91c8f90721a6596efe657a5324-0-rescue.conf 2024-05-08T21:39:44.567 INFO:teuthology.orchestra.run.smithi131.stdout:2db68f91c8f90721a6596efe657a5324-6.9.0-rc3-g806ec36b2112.conf 2024-05-08T21:39:44.567 INFO:teuthology.orchestra.run.smithi131.stdout:963f8b6c99184de696879b3f7da9ecce-0-rescue.conf 2024-05-08T21:39:44.567 INFO:teuthology.orchestra.run.smithi131.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-372.9.1.el8.x86_64.conf 2024-05-08T21:39:44.567 INFO:teuthology.orchestra.run.smithi131.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-513.9.1.el8_9.x86_64.conf 2024-05-08T21:39:44.568 DEBUG:teuthology.orchestra.run.smithi131:> sudo grub2-set-default 2db68f91c8f90721a6596efe657a5324-6.9.0-rc3-g806ec36b2112 2024-05-08T21:39:44.725 DEBUG:teuthology.orchestra.run.smithi131:> sudo shutdown -r now 2024-05-08T21:40:14.729 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-08T21:40:14.729 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-05-08T21:40:14.730 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T21:40:33.182 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.131 2024-05-08T21:40:42.184 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-05-08T21:40:42.185 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T21:41:42.187 DEBUG:teuthology.orchestra.remote:timed out 2024-05-08T21:41:54.188 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-05-08T21:41:54.189 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T21:41:54.698 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-05-08T21:41:55.155 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-05-08T21:41:55.155 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-05-08T21:41:55.155 DEBUG:teuthology.orchestra.run.smithi131:> uname -r 2024-05-08T21:41:55.196 INFO:teuthology.orchestra.run.smithi131.stdout:6.9.0-rc3-g806ec36b2112 2024-05-08T21:41:55.196 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-05-08T21:41:55.196 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T21:41:55.196 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2024-05-08T21:41:56.197 DEBUG:teuthology.parallel:result is None 2024-05-08T21:41:56.197 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T21:41:56.204 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T21:41:56.204 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T21:41:56.209 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T21:41:56.212 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T21:41:56.251 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T21:41:56.257 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T21:41:56.267 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T21:41:56.267 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T21:41:56.295 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T21:41:56.297 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T21:41:56.349 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T21:41:56.356 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T21:41:56.356 DEBUG:teuthology.orchestra.run.smithi132:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T21:41:56.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T21:41:56.382 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T21:41:56.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T21:41:56.401 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T21:41:56.405 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T21:41:56.424 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T21:41:56.478 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T21:41:56.496 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T21:41:56.515 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T21:41:56.519 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T21:41:56.546 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T21:41:56.582 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T21:41:56.586 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T21:41:56.594 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T21:41:56.594 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T21:41:56.598 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T21:41:56.600 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T21:41:56.660 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T21:41:56.670 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T21:41:56.670 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T21:41:56.675 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T21:41:56.677 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T21:41:56.711 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T21:41:56.822 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T21:41:56.932 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T21:41:57.001 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T21:41:57.080 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-08T21:41:57.080 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T21:41:57.145 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T21:41:57.194 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T21:41:57.250 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T21:41:57.330 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T21:41:57.377 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-05-08T21:41:57.378 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T21:41:57.445 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T21:41:57.493 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T21:41:57.564 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T21:41:57.608 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T21:41:57.655 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-08T21:41:57.655 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T21:41:57.723 DEBUG:teuthology.orchestra.run.smithi131:> sudo service rsyslog restart 2024-05-08T21:41:57.727 DEBUG:teuthology.orchestra.run.smithi132:> sudo service rsyslog restart 2024-05-08T21:41:57.730 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2024-05-08T21:41:57.795 INFO:teuthology.orchestra.run.smithi132.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T21:41:57.832 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T21:41:57.855 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T21:41:58.287 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T21:41:58.295 INFO:teuthology.task.internal:Starting timer... 2024-05-08T21:41:58.295 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T21:41:58.305 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T21:41:58.314 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-05-08T21:41:58.315 DEBUG:teuthology.orchestra.run.smithi131:> sudo service auditd rotate 2024-05-08T21:41:58.469 INFO:teuthology.orchestra.run.smithi131.stdout:Rotating logs: 2024-05-08T21:41:58.472 DEBUG:teuthology.orchestra.run.smithi132:> sudo service auditd rotate 2024-05-08T21:41:58.536 INFO:teuthology.orchestra.run.smithi132.stdout:Rotating logs: 2024-05-08T21:41:58.538 DEBUG:teuthology.orchestra.run.smithi157:> sudo service auditd rotate 2024-05-08T21:41:58.607 INFO:teuthology.orchestra.run.smithi157.stdout:Rotating logs: 2024-05-08T21:41:58.609 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T21:41:58.609 DEBUG:teuthology.orchestra.run.smithi131:> /usr/sbin/getenforce 2024-05-08T21:41:58.666 INFO:teuthology.orchestra.run.smithi131.stdout:Permissive 2024-05-08T21:41:58.667 DEBUG:teuthology.orchestra.run.smithi132:> /usr/sbin/getenforce 2024-05-08T21:41:58.687 INFO:teuthology.orchestra.run.smithi132.stdout:Permissive 2024-05-08T21:41:58.687 DEBUG:teuthology.orchestra.run.smithi157:> /usr/sbin/getenforce 2024-05-08T21:41:58.715 INFO:teuthology.orchestra.run.smithi157.stdout:Permissive 2024-05-08T21:41:58.715 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi131.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi132.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi157.front.sepia.ceph.com': 'permissive'} 2024-05-08T21:41:58.715 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T21:41:58.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T21:41:58.784 DEBUG:teuthology.orchestra.run.smithi132:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T21:41:58.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T21:41:58.812 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T21:41:58.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T21:41:58.839 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T21:41:58.839 DEBUG:teuthology.orchestra.run.smithi131:> sudo /usr/sbin/setenforce permissive 2024-05-08T21:41:58.906 DEBUG:teuthology.orchestra.run.smithi132:> sudo /usr/sbin/setenforce permissive 2024-05-08T21:41:58.934 DEBUG:teuthology.orchestra.run.smithi157:> sudo /usr/sbin/setenforce permissive 2024-05-08T21:41:58.962 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T21:41:58.971 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T21:41:58.990 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T21:41:58.991 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi131.front.sepia.ceph.com,smithi132.front.sepia.ceph.com,smithi157.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T21:53:48.615 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi131.front.sepia.ceph.com'), Remote(name='ubuntu@smithi132.front.sepia.ceph.com'), Remote(name='ubuntu@smithi157.front.sepia.ceph.com')] 2024-05-08T21:53:48.616 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-05-08T21:53:48.617 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T21:53:48.710 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-05-08T21:53:48.827 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-05-08T21:53:48.827 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-05-08T21:53:48.828 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T21:53:48.901 DEBUG:teuthology.orchestra.run.smithi132:> true 2024-05-08T21:53:48.968 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-05-08T21:53:48.968 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-05-08T21:53:48.969 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T21:53:49.043 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-05-08T21:53:49.117 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-05-08T21:53:49.118 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T21:53:49.131 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T21:53:49.131 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T21:53:49.132 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T21:53:49.137 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T21:53:49.137 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T21:53:49.140 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T21:53:49.140 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T21:53:49.172 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T21:53:49.187 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T21:53:49.192 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T21:53:49.203 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T21:53:49.208 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T21:53:49.236 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T21:53:49.267 INFO:teuthology.orchestra.run.smithi132.stderr:sudo: ntpd: command not found 2024-05-08T21:53:49.279 INFO:teuthology.orchestra.run.smithi132.stdout:506 Cannot talk to daemon 2024-05-08T21:53:49.286 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: ntpd: command not found 2024-05-08T21:53:49.294 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T21:53:49.297 INFO:teuthology.orchestra.run.smithi157.stdout:506 Cannot talk to daemon 2024-05-08T21:53:49.308 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T21:53:49.312 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T21:53:49.327 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T21:53:49.338 INFO:teuthology.orchestra.run.smithi131.stderr:sudo: ntpd: command not found 2024-05-08T21:53:49.366 INFO:teuthology.orchestra.run.smithi131.stdout:506 Cannot talk to daemon 2024-05-08T21:53:49.388 INFO:teuthology.orchestra.run.smithi132.stderr:bash: ntpq: command not found 2024-05-08T21:53:49.392 INFO:teuthology.orchestra.run.smithi132.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T21:53:49.392 INFO:teuthology.orchestra.run.smithi132.stdout:=============================================================================== 2024-05-08T21:53:49.392 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:53:49.393 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:53:49.393 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:53:49.393 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:53:49.398 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T21:53:49.410 INFO:teuthology.orchestra.run.smithi157.stderr:bash: ntpq: command not found 2024-05-08T21:53:49.414 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T21:53:49.414 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-05-08T21:53:49.414 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:53:49.414 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:53:49.414 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:53:49.414 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:53:49.429 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T21:53:49.523 INFO:teuthology.orchestra.run.smithi131.stderr:bash: ntpq: command not found 2024-05-08T21:53:49.530 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T21:53:49.530 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2024-05-08T21:53:49.530 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:53:49.530 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:53:49.530 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:53:49.530 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:53:49.531 INFO:teuthology.run_tasks:Running task pexec... 2024-05-08T21:53:49.541 INFO:teuthology.task.pexec:Executing custom commands... 2024-05-08T21:53:49.542 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi131.front.sepia.ceph.com 2024-05-08T21:53:49.542 DEBUG:teuthology.orchestra.run.smithi131:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-08T21:53:49.542 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi132.front.sepia.ceph.com 2024-05-08T21:53:49.542 DEBUG:teuthology.orchestra.run.smithi132:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-08T21:53:49.543 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi157.front.sepia.ceph.com 2024-05-08T21:53:49.543 DEBUG:teuthology.orchestra.run.smithi157:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-08T21:53:49.918 INFO:teuthology.orchestra.run.smithi132.stdout:Updating Subscription Management repositories. 2024-05-08T21:53:49.920 INFO:teuthology.orchestra.run.smithi157.stdout:Updating Subscription Management repositories. 2024-05-08T21:53:50.027 INFO:teuthology.orchestra.run.smithi131.stdout:Updating Subscription Management repositories. 2024-05-08T21:53:51.709 INFO:teuthology.orchestra.run.smithi157.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 34 kB/s | 4.5 kB 00:00 2024-05-08T21:53:51.845 INFO:teuthology.orchestra.run.smithi132.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 22 kB/s | 4.5 kB 00:00 2024-05-08T21:53:51.990 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 49 kB/s | 4.5 kB 00:00 2024-05-08T21:53:52.457 INFO:teuthology.orchestra.run.smithi157.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 39 kB/s | 4.1 kB 00:00 2024-05-08T21:53:52.540 INFO:teuthology.orchestra.run.smithi132.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 19 kB/s | 4.1 kB 00:00 2024-05-08T21:53:52.712 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 31 kB/s | 4.1 kB 00:00 2024-05-08T21:53:55.423 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-05-08T21:53:55.424 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-05-08T21:53:55.424 INFO:teuthology.orchestra.run.smithi132.stdout: Package Architecture Version Repository Size 2024-05-08T21:53:55.424 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-05-08T21:53:55.424 INFO:teuthology.orchestra.run.smithi132.stdout:Resetting modules: 2024-05-08T21:53:55.424 INFO:teuthology.orchestra.run.smithi132.stdout: container-tools 2024-05-08T21:53:55.425 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-05-08T21:53:55.425 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-05-08T21:53:55.425 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-05-08T21:53:55.425 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-05-08T21:53:55.535 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-05-08T21:53:55.536 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-05-08T21:53:55.537 INFO:teuthology.orchestra.run.smithi157.stdout: Package Architecture Version Repository Size 2024-05-08T21:53:55.537 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-05-08T21:53:55.537 INFO:teuthology.orchestra.run.smithi157.stdout:Resetting modules: 2024-05-08T21:53:55.537 INFO:teuthology.orchestra.run.smithi157.stdout: container-tools 2024-05-08T21:53:55.537 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-08T21:53:55.537 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-05-08T21:53:55.537 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-05-08T21:53:55.537 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-08T21:53:55.739 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-05-08T21:53:55.759 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-05-08T21:53:55.759 INFO:teuthology.orchestra.run.smithi131.stdout: Package Architecture Version Repository Size 2024-05-08T21:53:55.759 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-05-08T21:53:55.759 INFO:teuthology.orchestra.run.smithi131.stdout:Resetting modules: 2024-05-08T21:53:55.759 INFO:teuthology.orchestra.run.smithi131.stdout: container-tools 2024-05-08T21:53:55.759 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:53:55.759 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-05-08T21:53:55.759 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-05-08T21:53:55.759 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:54:04.652 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-05-08T21:54:05.144 INFO:teuthology.orchestra.run.smithi132.stdout:Updating Subscription Management repositories. 2024-05-08T21:54:06.893 INFO:teuthology.orchestra.run.smithi132.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 kB/s | 4.5 kB 00:00 2024-05-08T21:54:07.522 INFO:teuthology.orchestra.run.smithi132.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 27 kB/s | 4.1 kB 00:00 2024-05-08T21:54:08.185 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-05-08T21:54:08.677 INFO:teuthology.orchestra.run.smithi157.stdout:Updating Subscription Management repositories. 2024-05-08T21:54:10.239 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-05-08T21:54:10.475 INFO:teuthology.orchestra.run.smithi157.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 35 kB/s | 4.5 kB 00:00 2024-05-08T21:54:10.557 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-05-08T21:54:10.720 INFO:teuthology.orchestra.run.smithi132.stdout:========================================================================================================== 2024-05-08T21:54:10.721 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-05-08T21:54:10.721 INFO:teuthology.orchestra.run.smithi132.stdout:========================================================================================================== 2024-05-08T21:54:10.721 INFO:teuthology.orchestra.run.smithi132.stdout:Installing group/module packages: 2024-05-08T21:54:10.721 INFO:teuthology.orchestra.run.smithi132.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-08T21:54:10.721 INFO:teuthology.orchestra.run.smithi132.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-08T21:54:10.721 INFO:teuthology.orchestra.run.smithi132.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-08T21:54:10.721 INFO:teuthology.orchestra.run.smithi132.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-08T21:54:10.721 INFO:teuthology.orchestra.run.smithi132.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T21:54:10.721 INFO:teuthology.orchestra.run.smithi132.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-08T21:54:10.721 INFO:teuthology.orchestra.run.smithi132.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-08T21:54:10.721 INFO:teuthology.orchestra.run.smithi132.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-08T21:54:10.721 INFO:teuthology.orchestra.run.smithi132.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-08T21:54:10.721 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-05-08T21:54:10.721 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-08T21:54:10.722 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-08T21:54:10.722 INFO:teuthology.orchestra.run.smithi132.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-08T21:54:10.722 INFO:teuthology.orchestra.run.smithi132.stdout:Installing weak dependencies: 2024-05-08T21:54:10.722 INFO:teuthology.orchestra.run.smithi132.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T21:54:10.722 INFO:teuthology.orchestra.run.smithi132.stdout:Installing module profiles: 2024-05-08T21:54:10.722 INFO:teuthology.orchestra.run.smithi132.stdout: container-tools/common 2024-05-08T21:54:10.722 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-05-08T21:54:10.722 INFO:teuthology.orchestra.run.smithi132.stdout:Enabling module streams: 2024-05-08T21:54:10.722 INFO:teuthology.orchestra.run.smithi132.stdout: container-tools rhel8 2024-05-08T21:54:10.722 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-05-08T21:54:10.722 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-05-08T21:54:10.722 INFO:teuthology.orchestra.run.smithi132.stdout:========================================================================================================== 2024-05-08T21:54:10.722 INFO:teuthology.orchestra.run.smithi132.stdout:Install 13 Packages 2024-05-08T21:54:10.722 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-05-08T21:54:10.723 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 26 M 2024-05-08T21:54:10.723 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 87 M 2024-05-08T21:54:10.723 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-05-08T21:54:10.915 INFO:teuthology.orchestra.run.smithi131.stdout:Updating Subscription Management repositories. 2024-05-08T21:54:10.935 INFO:teuthology.orchestra.run.smithi132.stdout:(1/13): python3-pyxdg-0.25-16.el8.noarch.rpm 443 kB/s | 94 kB 00:00 2024-05-08T21:54:10.968 INFO:teuthology.orchestra.run.smithi132.stdout:(2/13): python3-pytoml-0.1.14-5.git7dea353.el8. 101 kB/s | 25 kB 00:00 2024-05-08T21:54:10.993 INFO:teuthology.orchestra.run.smithi132.stdout:(3/13): criu-libs-3.18-4.module+el8.9.0+20326+3 146 kB/s | 39 kB 00:00 2024-05-08T21:54:11.052 INFO:teuthology.orchestra.run.smithi132.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 350 kB/s | 41 kB 00:00 2024-05-08T21:54:11.136 INFO:teuthology.orchestra.run.smithi132.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 6.0 MB/s | 1.0 MB 00:00 2024-05-08T21:54:11.148 INFO:teuthology.orchestra.run.smithi157.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 34 kB/s | 4.1 kB 00:00 2024-05-08T21:54:11.169 INFO:teuthology.orchestra.run.smithi132.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2169 6.2 MB/s | 739 kB 00:00 2024-05-08T21:54:11.245 INFO:teuthology.orchestra.run.smithi132.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 2.2 MB/s | 239 kB 00:00 2024-05-08T21:54:11.395 INFO:teuthology.orchestra.run.smithi132.stdout:(8/13): buildah-1.31.5-1.module+el8.9.0+21697+6 22 MB/s | 8.8 MB 00:00 2024-05-08T21:54:11.420 INFO:teuthology.orchestra.run.smithi132.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 876 kB/s | 154 kB 00:00 2024-05-08T21:54:11.511 INFO:teuthology.orchestra.run.smithi132.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21697+ 11 MB/s | 3.7 MB 00:00 2024-05-08T21:54:11.612 INFO:teuthology.orchestra.run.smithi132.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 13 MB/s | 2.5 MB 00:00 2024-05-08T21:54:11.646 INFO:teuthology.orchestra.run.smithi132.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 367 kB/s | 49 kB 00:00 2024-05-08T21:54:11.754 INFO:teuthology.orchestra.run.smithi132.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 23 MB/s | 8.1 MB 00:00 2024-05-08T21:54:11.755 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-05-08T21:54:11.755 INFO:teuthology.orchestra.run.smithi132.stdout:Total 25 MB/s | 26 MB 00:01 2024-05-08T21:54:12.034 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-05-08T21:54:12.055 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-05-08T21:54:12.056 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-05-08T21:54:12.283 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-05-08T21:54:12.292 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-05-08T21:54:12.701 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 28 kB/s | 4.5 kB 00:00 2024-05-08T21:54:13.090 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-05-08T21:54:13.342 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 59 kB/s | 4.1 kB 00:00 2024-05-08T21:54:13.633 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-08T21:54:13.871 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-08T21:54:13.978 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-05-08T21:54:14.032 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-08T21:54:14.142 INFO:teuthology.orchestra.run.smithi157.stdout:========================================================================================================== 2024-05-08T21:54:14.142 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-05-08T21:54:14.142 INFO:teuthology.orchestra.run.smithi157.stdout:========================================================================================================== 2024-05-08T21:54:14.142 INFO:teuthology.orchestra.run.smithi157.stdout:Installing group/module packages: 2024-05-08T21:54:14.142 INFO:teuthology.orchestra.run.smithi157.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-08T21:54:14.142 INFO:teuthology.orchestra.run.smithi157.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-08T21:54:14.142 INFO:teuthology.orchestra.run.smithi157.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-08T21:54:14.142 INFO:teuthology.orchestra.run.smithi157.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-08T21:54:14.142 INFO:teuthology.orchestra.run.smithi157.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T21:54:14.142 INFO:teuthology.orchestra.run.smithi157.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-08T21:54:14.142 INFO:teuthology.orchestra.run.smithi157.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-08T21:54:14.143 INFO:teuthology.orchestra.run.smithi157.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-08T21:54:14.143 INFO:teuthology.orchestra.run.smithi157.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-08T21:54:14.143 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-05-08T21:54:14.143 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-08T21:54:14.143 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-08T21:54:14.143 INFO:teuthology.orchestra.run.smithi157.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-08T21:54:14.143 INFO:teuthology.orchestra.run.smithi157.stdout:Installing weak dependencies: 2024-05-08T21:54:14.143 INFO:teuthology.orchestra.run.smithi157.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T21:54:14.143 INFO:teuthology.orchestra.run.smithi157.stdout:Installing module profiles: 2024-05-08T21:54:14.143 INFO:teuthology.orchestra.run.smithi157.stdout: container-tools/common 2024-05-08T21:54:14.143 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-08T21:54:14.143 INFO:teuthology.orchestra.run.smithi157.stdout:Enabling module streams: 2024-05-08T21:54:14.143 INFO:teuthology.orchestra.run.smithi157.stdout: container-tools rhel8 2024-05-08T21:54:14.143 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-08T21:54:14.143 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-05-08T21:54:14.144 INFO:teuthology.orchestra.run.smithi157.stdout:========================================================================================================== 2024-05-08T21:54:14.144 INFO:teuthology.orchestra.run.smithi157.stdout:Install 13 Packages 2024-05-08T21:54:14.144 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-08T21:54:14.144 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 26 M 2024-05-08T21:54:14.144 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 87 M 2024-05-08T21:54:14.144 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-05-08T21:54:14.225 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-08T21:54:14.286 INFO:teuthology.orchestra.run.smithi157.stdout:(1/13): python3-pyxdg-0.25-16.el8.noarch.rpm 667 kB/s | 94 kB 00:00 2024-05-08T21:54:14.311 INFO:teuthology.orchestra.run.smithi157.stdout:(2/13): criu-libs-3.18-4.module+el8.9.0+20326+3 237 kB/s | 39 kB 00:00 2024-05-08T21:54:14.336 INFO:teuthology.orchestra.run.smithi157.stdout:(3/13): python3-pytoml-0.1.14-5.git7dea353.el8. 129 kB/s | 25 kB 00:00 2024-05-08T21:54:14.395 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-08T21:54:14.396 INFO:teuthology.orchestra.run.smithi157.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 376 kB/s | 41 kB 00:00 2024-05-08T21:54:14.535 INFO:teuthology.orchestra.run.smithi157.stdout:(5/13): cockpit-podman-75-1.module+el8.9.0+2169 5.2 MB/s | 739 kB 00:00 2024-05-08T21:54:14.579 INFO:teuthology.orchestra.run.smithi157.stdout:(6/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 3.8 MB/s | 1.0 MB 00:00 2024-05-08T21:54:14.621 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-08T21:54:14.704 INFO:teuthology.orchestra.run.smithi157.stdout:(7/13): buildah-1.31.5-1.module+el8.9.0+21697+6 24 MB/s | 8.8 MB 00:00 2024-05-08T21:54:14.729 INFO:teuthology.orchestra.run.smithi157.stdout:(8/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 1.2 MB/s | 239 kB 00:00 2024-05-08T21:54:14.821 INFO:teuthology.orchestra.run.smithi157.stdout:(9/13): netavark-1.7.0-2.module+el8.9.0+21697+6 15 MB/s | 3.7 MB 00:00 2024-05-08T21:54:14.847 INFO:teuthology.orchestra.run.smithi157.stdout:(10/13): python3-podman-4.6.0-2.module+el8.9.0+ 1.1 MB/s | 154 kB 00:00 2024-05-08T21:54:14.882 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-08T21:54:14.988 INFO:teuthology.orchestra.run.smithi157.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 15 MB/s | 2.5 MB 00:00 2024-05-08T21:54:15.005 INFO:teuthology.orchestra.run.smithi157.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 312 kB/s | 49 kB 00:00 2024-05-08T21:54:15.131 INFO:teuthology.orchestra.run.smithi157.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 20 MB/s | 8.1 MB 00:00 2024-05-08T21:54:15.131 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-05-08T21:54:15.132 INFO:teuthology.orchestra.run.smithi157.stdout:Total 26 MB/s | 26 MB 00:00 2024-05-08T21:54:15.415 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-05-08T21:54:15.436 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-05-08T21:54:15.436 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-05-08T21:54:15.665 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-05-08T21:54:15.673 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-05-08T21:54:15.777 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T21:54:16.051 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T21:54:16.412 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-05-08T21:54:16.435 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-05-08T21:54:16.449 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-08T21:54:16.609 INFO:teuthology.orchestra.run.smithi131.stdout:========================================================================================================== 2024-05-08T21:54:16.609 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-05-08T21:54:16.609 INFO:teuthology.orchestra.run.smithi131.stdout:========================================================================================================== 2024-05-08T21:54:16.609 INFO:teuthology.orchestra.run.smithi131.stdout:Installing group/module packages: 2024-05-08T21:54:16.609 INFO:teuthology.orchestra.run.smithi131.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-08T21:54:16.610 INFO:teuthology.orchestra.run.smithi131.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-08T21:54:16.610 INFO:teuthology.orchestra.run.smithi131.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-08T21:54:16.610 INFO:teuthology.orchestra.run.smithi131.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-08T21:54:16.610 INFO:teuthology.orchestra.run.smithi131.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T21:54:16.610 INFO:teuthology.orchestra.run.smithi131.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-08T21:54:16.610 INFO:teuthology.orchestra.run.smithi131.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-08T21:54:16.610 INFO:teuthology.orchestra.run.smithi131.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-08T21:54:16.610 INFO:teuthology.orchestra.run.smithi131.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-08T21:54:16.610 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-05-08T21:54:16.610 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-08T21:54:16.610 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-08T21:54:16.610 INFO:teuthology.orchestra.run.smithi131.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-08T21:54:16.610 INFO:teuthology.orchestra.run.smithi131.stdout:Installing weak dependencies: 2024-05-08T21:54:16.611 INFO:teuthology.orchestra.run.smithi131.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T21:54:16.611 INFO:teuthology.orchestra.run.smithi131.stdout:Installing module profiles: 2024-05-08T21:54:16.611 INFO:teuthology.orchestra.run.smithi131.stdout: container-tools/common 2024-05-08T21:54:16.611 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:54:16.611 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling module streams: 2024-05-08T21:54:16.611 INFO:teuthology.orchestra.run.smithi131.stdout: container-tools rhel8 2024-05-08T21:54:16.611 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:54:16.611 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-05-08T21:54:16.611 INFO:teuthology.orchestra.run.smithi131.stdout:========================================================================================================== 2024-05-08T21:54:16.611 INFO:teuthology.orchestra.run.smithi131.stdout:Install 13 Packages 2024-05-08T21:54:16.611 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:54:16.612 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 26 M 2024-05-08T21:54:16.612 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 87 M 2024-05-08T21:54:16.612 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-05-08T21:54:16.756 INFO:teuthology.orchestra.run.smithi131.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 173 kB/s | 25 kB 00:00 2024-05-08T21:54:16.773 INFO:teuthology.orchestra.run.smithi131.stdout:(2/13): criu-libs-3.18-4.module+el8.9.0+20326+3 246 kB/s | 39 kB 00:00 2024-05-08T21:54:16.798 INFO:teuthology.orchestra.run.smithi131.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 506 kB/s | 94 kB 00:00 2024-05-08T21:54:16.874 INFO:teuthology.orchestra.run.smithi131.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 349 kB/s | 41 kB 00:00 2024-05-08T21:54:16.890 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-08T21:54:16.915 INFO:teuthology.orchestra.run.smithi131.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 7.1 MB/s | 1.0 MB 00:00 2024-05-08T21:54:17.007 INFO:teuthology.orchestra.run.smithi131.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2169 5.4 MB/s | 739 kB 00:00 2024-05-08T21:54:17.041 INFO:teuthology.orchestra.run.smithi131.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 1.9 MB/s | 239 kB 00:00 2024-05-08T21:54:17.091 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-08T21:54:17.171 INFO:teuthology.orchestra.run.smithi131.stdout:(8/13): python3-podman-4.6.0-2.module+el8.9.0+2 1.2 MB/s | 154 kB 00:00 2024-05-08T21:54:17.255 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-08T21:54:17.284 INFO:teuthology.orchestra.run.smithi131.stdout:(9/13): netavark-1.7.0-2.module+el8.9.0+21697+6 13 MB/s | 3.7 MB 00:00 2024-05-08T21:54:17.356 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-08T21:54:17.426 INFO:teuthology.orchestra.run.smithi131.stdout:(10/13): buildah-1.31.5-1.module+el8.9.0+21697+ 14 MB/s | 8.8 MB 00:00 2024-05-08T21:54:17.457 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-08T21:54:17.535 INFO:teuthology.orchestra.run.smithi131.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 10 MB/s | 2.5 MB 00:00 2024-05-08T21:54:17.586 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-08T21:54:17.586 INFO:teuthology.orchestra.run.smithi131.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 371 kB/s | 49 kB 00:00 2024-05-08T21:54:17.636 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-08T21:54:17.685 INFO:teuthology.orchestra.run.smithi131.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 16 MB/s | 8.1 MB 00:00 2024-05-08T21:54:17.685 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-05-08T21:54:17.685 INFO:teuthology.orchestra.run.smithi131.stdout:Total 24 MB/s | 26 MB 00:01 2024-05-08T21:54:17.786 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T21:54:17.887 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-08T21:54:18.164 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-08T21:54:18.166 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-05-08T21:54:18.191 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-05-08T21:54:18.192 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-05-08T21:54:18.200 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T21:54:18.200 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-08T21:54:18.200 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-08T21:54:18.201 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-08T21:54:18.201 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-08T21:54:18.201 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-08T21:54:18.201 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-08T21:54:18.201 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-08T21:54:18.201 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T21:54:18.201 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T21:54:18.201 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-08T21:54:18.201 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-08T21:54:18.201 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-08T21:54:18.603 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-05-08T21:54:18.613 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-05-08T21:54:19.032 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T21:54:19.263 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T21:54:19.497 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-05-08T21:54:19.648 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-08T21:54:20.065 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-08T21:54:20.272 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-08T21:54:20.476 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-08T21:54:20.555 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-08T21:54:20.642 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-08T21:54:20.751 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-08T21:54:20.834 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-08T21:54:20.959 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T21:54:21.112 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-08T21:54:21.346 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-08T21:54:21.378 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T21:54:21.378 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-08T21:54:21.378 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-08T21:54:21.378 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-08T21:54:21.378 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-08T21:54:21.378 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-08T21:54:21.378 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-08T21:54:21.378 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-08T21:54:21.378 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T21:54:21.378 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T21:54:21.378 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-08T21:54:21.379 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-08T21:54:21.379 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-08T21:54:22.293 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T21:54:22.542 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T21:54:22.968 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-08T21:54:23.964 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-08T21:54:24.215 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-08T21:54:24.382 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T21:54:25.122 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T21:54:25.122 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-08T21:54:25.122 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-08T21:54:25.122 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-08T21:54:25.122 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-08T21:54:25.122 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-08T21:54:25.122 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-08T21:54:25.122 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-08T21:54:25.122 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T21:54:25.122 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T21:54:25.122 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-08T21:54:25.123 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-08T21:54:25.123 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-08T21:54:34.043 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-08T21:54:34.044 INFO:teuthology.orchestra.run.smithi132.stdout:Installed products updated. 2024-05-08T21:54:34.622 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-05-08T21:54:34.622 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-05-08T21:54:34.622 INFO:teuthology.orchestra.run.smithi132.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:54:34.622 INFO:teuthology.orchestra.run.smithi132.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:54:34.622 INFO:teuthology.orchestra.run.smithi132.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:54:34.622 INFO:teuthology.orchestra.run.smithi132.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-08T21:54:34.622 INFO:teuthology.orchestra.run.smithi132.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:54:34.623 INFO:teuthology.orchestra.run.smithi132.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:54:34.623 INFO:teuthology.orchestra.run.smithi132.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:54:34.623 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-08T21:54:34.623 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-08T21:54:34.623 INFO:teuthology.orchestra.run.smithi132.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:54:34.623 INFO:teuthology.orchestra.run.smithi132.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:54:34.623 INFO:teuthology.orchestra.run.smithi132.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:54:34.623 INFO:teuthology.orchestra.run.smithi132.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-08T21:54:34.623 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-05-08T21:54:34.623 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-05-08T21:54:34.782 DEBUG:teuthology.parallel:result is None 2024-05-08T21:54:36.431 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-08T21:54:36.431 INFO:teuthology.orchestra.run.smithi157.stdout:Installed products updated. 2024-05-08T21:54:36.995 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-08T21:54:36.995 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-05-08T21:54:36.995 INFO:teuthology.orchestra.run.smithi157.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:54:36.995 INFO:teuthology.orchestra.run.smithi157.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:54:36.995 INFO:teuthology.orchestra.run.smithi157.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:54:36.995 INFO:teuthology.orchestra.run.smithi157.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-08T21:54:36.996 INFO:teuthology.orchestra.run.smithi157.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:54:36.996 INFO:teuthology.orchestra.run.smithi157.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:54:36.996 INFO:teuthology.orchestra.run.smithi157.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:54:36.996 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-08T21:54:36.996 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-08T21:54:36.996 INFO:teuthology.orchestra.run.smithi157.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:54:36.996 INFO:teuthology.orchestra.run.smithi157.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:54:36.996 INFO:teuthology.orchestra.run.smithi157.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:54:36.996 INFO:teuthology.orchestra.run.smithi157.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-08T21:54:36.996 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-08T21:54:36.996 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-05-08T21:54:37.202 DEBUG:teuthology.parallel:result is None 2024-05-08T21:54:42.368 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-08T21:54:42.369 INFO:teuthology.orchestra.run.smithi131.stdout:Installed products updated. 2024-05-08T21:54:43.040 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:54:43.040 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-05-08T21:54:43.040 INFO:teuthology.orchestra.run.smithi131.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:54:43.040 INFO:teuthology.orchestra.run.smithi131.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:54:43.040 INFO:teuthology.orchestra.run.smithi131.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:54:43.040 INFO:teuthology.orchestra.run.smithi131.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-08T21:54:43.040 INFO:teuthology.orchestra.run.smithi131.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:54:43.041 INFO:teuthology.orchestra.run.smithi131.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:54:43.041 INFO:teuthology.orchestra.run.smithi131.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:54:43.041 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-08T21:54:43.041 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-08T21:54:43.041 INFO:teuthology.orchestra.run.smithi131.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:54:43.041 INFO:teuthology.orchestra.run.smithi131.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:54:43.041 INFO:teuthology.orchestra.run.smithi131.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:54:43.041 INFO:teuthology.orchestra.run.smithi131.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-08T21:54:43.041 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:54:43.042 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-05-08T21:54:43.351 DEBUG:teuthology.parallel:result is None 2024-05-08T21:54:43.351 INFO:teuthology.run_tasks:Running task install... 2024-05-08T21:54:43.360 DEBUG:teuthology.task.install:project ceph 2024-05-08T21:54:43.360 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}} 2024-05-08T21:54:43.361 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'} 2024-05-08T21:54:43.361 INFO:teuthology.task.install:Using flavor: default 2024-05-08T21:54:43.367 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T21:54:43.367 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-05-08T21:54:43.367 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'tag': None, 'wait_for_package': False} 2024-05-08T21:54:43.368 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:54:43.370 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'tag': None, 'wait_for_package': False} 2024-05-08T21:54:43.370 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:54:43.371 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'tag': None, 'wait_for_package': False} 2024-05-08T21:54:43.371 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:54:43.522 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/ 2024-05-08T21:54:43.523 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1322.gd21e4f96 2024-05-08T21:54:43.602 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/ 2024-05-08T21:54:43.602 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1322.gd21e4f96 2024-05-08T21:54:43.616 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/ 2024-05-08T21:54:43.616 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1322.gd21e4f96 2024-05-08T21:54:43.763 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T21:54:43.764 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-08T21:54:43.764 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T21:54:43.792 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T21:54:43.792 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-05-08T21:54:43.792 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T21:54:43.798 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T21:54:43.798 DEBUG:teuthology.orchestra.run.smithi157:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d21e4f961318b56f0d915caa2490c3711bf13c34/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T21:54:43.826 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T21:54:43.827 DEBUG:teuthology.orchestra.run.smithi132:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d21e4f961318b56f0d915caa2490c3711bf13c34/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T21:54:43.849 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T21:54:43.849 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-08T21:54:43.849 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T21:54:43.871 DEBUG:teuthology.orchestra.run.smithi157:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T21:54:43.899 DEBUG:teuthology.orchestra.run.smithi132:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T21:54:43.915 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T21:54:43.916 DEBUG:teuthology.orchestra.run.smithi131:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d21e4f961318b56f0d915caa2490c3711bf13c34/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T21:54:43.954 DEBUG:teuthology.orchestra.run.smithi157:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T21:54:43.981 DEBUG:teuthology.orchestra.run.smithi132:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T21:54:44.023 DEBUG:teuthology.orchestra.run.smithi131:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T21:54:44.025 INFO:teuthology.orchestra.run.smithi157.stdout:check_obsoletes = 1 2024-05-08T21:54:44.025 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum clean all 2024-05-08T21:54:44.049 INFO:teuthology.orchestra.run.smithi132.stdout:check_obsoletes = 1 2024-05-08T21:54:44.050 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum clean all 2024-05-08T21:54:44.115 DEBUG:teuthology.orchestra.run.smithi131:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T21:54:44.180 INFO:teuthology.orchestra.run.smithi131.stdout:check_obsoletes = 1 2024-05-08T21:54:44.183 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum clean all 2024-05-08T21:54:44.406 INFO:teuthology.orchestra.run.smithi157.stdout:Updating Subscription Management repositories. 2024-05-08T21:54:44.432 INFO:teuthology.orchestra.run.smithi132.stdout:Updating Subscription Management repositories. 2024-05-08T21:54:44.630 INFO:teuthology.orchestra.run.smithi131.stdout:Updating Subscription Management repositories. 2024-05-08T21:54:45.674 INFO:teuthology.orchestra.run.smithi157.stdout:63 files removed 2024-05-08T21:54:45.732 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T21:54:46.054 INFO:teuthology.orchestra.run.smithi132.stdout:63 files removed 2024-05-08T21:54:46.069 INFO:teuthology.orchestra.run.smithi157.stdout:Updating Subscription Management repositories. 2024-05-08T21:54:46.159 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T21:54:46.401 INFO:teuthology.orchestra.run.smithi131.stdout:63 files removed 2024-05-08T21:54:46.460 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T21:54:46.496 INFO:teuthology.orchestra.run.smithi132.stdout:Updating Subscription Management repositories. 2024-05-08T21:54:46.904 INFO:teuthology.orchestra.run.smithi131.stdout:Updating Subscription Management repositories. 2024-05-08T21:54:47.934 INFO:teuthology.orchestra.run.smithi157.stdout:ceph packages for x86_64 163 kB/s | 80 kB 00:00 2024-05-08T21:54:48.514 INFO:teuthology.orchestra.run.smithi157.stdout:ceph noarch packages 21 kB/s | 11 kB 00:00 2024-05-08T21:54:48.526 INFO:teuthology.orchestra.run.smithi132.stdout:ceph packages for x86_64 164 kB/s | 80 kB 00:00 2024-05-08T21:54:48.852 INFO:teuthology.orchestra.run.smithi132.stdout:ceph noarch packages 39 kB/s | 11 kB 00:00 2024-05-08T21:54:48.874 INFO:teuthology.orchestra.run.smithi131.stdout:ceph packages for x86_64 207 kB/s | 80 kB 00:00 2024-05-08T21:54:48.935 INFO:teuthology.orchestra.run.smithi157.stdout:ceph source packages 3.8 kB/s | 1.5 kB 00:00 2024-05-08T21:54:49.129 INFO:teuthology.orchestra.run.smithi157.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2024-05-08T21:54:49.217 INFO:teuthology.orchestra.run.smithi131.stdout:ceph noarch packages 37 kB/s | 11 kB 00:00 2024-05-08T21:54:49.228 INFO:teuthology.orchestra.run.smithi132.stdout:ceph source packages 4.3 kB/s | 1.5 kB 00:00 2024-05-08T21:54:49.370 INFO:teuthology.orchestra.run.smithi132.stdout:Copr repo for python3-asyncssh owned by ceph 29 kB/s | 3.5 kB 00:00 2024-05-08T21:54:49.517 INFO:teuthology.orchestra.run.smithi131.stdout:ceph source packages 5.5 kB/s | 1.5 kB 00:00 2024-05-08T21:54:49.702 INFO:teuthology.orchestra.run.smithi131.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-05-08T21:54:50.215 INFO:teuthology.orchestra.run.smithi132.stdout:Extra Packages for Enterprise Linux 20 MB/s | 16 MB 00:00 2024-05-08T21:54:50.921 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 9.2 MB/s | 16 MB 00:01 2024-05-08T21:54:52.886 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 5.2 MB/s | 16 MB 00:03 2024-05-08T21:54:53.814 INFO:teuthology.orchestra.run.smithi132.stdout:lab-extras 460 kB/s | 24 kB 00:00 2024-05-08T21:54:54.515 INFO:teuthology.orchestra.run.smithi132.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 14 MB/s | 8.8 MB 00:00 2024-05-08T21:54:54.605 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 417 kB/s | 24 kB 00:00 2024-05-08T21:54:55.956 INFO:teuthology.orchestra.run.smithi157.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 48 MB/s | 61 MB 00:01 2024-05-08T21:54:57.103 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 286 kB/s | 24 kB 00:00 2024-05-08T21:54:57.417 INFO:teuthology.orchestra.run.smithi132.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 61 MB 00:01 2024-05-08T21:54:58.747 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 39 MB/s | 61 MB 00:01 2024-05-08T21:55:06.689 INFO:teuthology.orchestra.run.smithi157.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 16 MB/s | 8.8 MB 00:00 2024-05-08T21:55:08.892 INFO:teuthology.orchestra.run.smithi132.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 MB/s | 67 MB 00:01 2024-05-08T21:55:09.557 INFO:teuthology.orchestra.run.smithi157.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 54 MB/s | 67 MB 00:01 2024-05-08T21:55:11.414 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 42 MB/s | 67 MB 00:01 2024-05-08T21:55:22.472 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 12 MB/s | 8.8 MB 00:00 2024-05-08T21:55:23.940 INFO:teuthology.orchestra.run.smithi132.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T21:55:23.941 INFO:teuthology.orchestra.run.smithi132.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T21:55:23.960 INFO:teuthology.orchestra.run.smithi132.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-08T21:55:23.961 INFO:teuthology.orchestra.run.smithi132.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-08T21:55:23.963 INFO:teuthology.orchestra.run.smithi132.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-08T21:55:23.964 INFO:teuthology.orchestra.run.smithi132.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-08T21:55:23.967 INFO:teuthology.orchestra.run.smithi132.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-08T21:55:23.971 INFO:teuthology.orchestra.run.smithi132.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-08T21:55:23.973 INFO:teuthology.orchestra.run.smithi132.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-08T21:55:24.192 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-05-08T21:55:24.206 INFO:teuthology.orchestra.run.smithi132.stdout:=================================================================================================================================== 2024-05-08T21:55:24.206 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-05-08T21:55:24.206 INFO:teuthology.orchestra.run.smithi132.stdout:=================================================================================================================================== 2024-05-08T21:55:24.206 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-05-08T21:55:24.206 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-08T21:55:24.206 INFO:teuthology.orchestra.run.smithi132.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-08T21:55:24.206 INFO:teuthology.orchestra.run.smithi132.stdout: ceph x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.4 k 2024-05-08T21:55:24.206 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 5.5 M 2024-05-08T21:55:24.206 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 860 k 2024-05-08T21:55:24.206 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 166 k 2024-05-08T21:55:24.207 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 1.6 M 2024-05-08T21:55:24.207 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 138 k 2024-05-08T21:55:24.207 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 3.5 M 2024-05-08T21:55:24.207 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 7.4 M 2024-05-08T21:55:24.207 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-rook noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 49 k 2024-05-08T21:55:24.207 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 13 M 2024-05-08T21:55:24.207 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 47 M 2024-05-08T21:55:24.207 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-volume noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 263 k 2024-05-08T21:55:24.207 INFO:teuthology.orchestra.run.smithi132.stdout: cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 170 k 2024-05-08T21:55:24.207 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 261 k 2024-05-08T21:55:24.207 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-top noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 18 k 2024-05-08T21:55:24.207 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-08T21:55:24.207 INFO:teuthology.orchestra.run.smithi132.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-08T21:55:24.208 INFO:teuthology.orchestra.run.smithi132.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-08T21:55:24.208 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 31 k 2024-05-08T21:55:24.208 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 782 k 2024-05-08T21:55:24.208 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 139 k 2024-05-08T21:55:24.208 INFO:teuthology.orchestra.run.smithi132.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-08T21:55:24.208 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-08T21:55:24.208 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-08T21:55:24.208 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-08T21:55:24.208 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-08T21:55:24.208 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-08T21:55:24.208 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 197 k 2024-05-08T21:55:24.208 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 370 k 2024-05-08T21:55:24.208 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 365 k 2024-05-08T21:55:24.208 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 107 k 2024-05-08T21:55:24.209 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 95 k 2024-05-08T21:55:24.209 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.5 M 2024-05-08T21:55:24.209 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 181 k 2024-05-08T21:55:24.209 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-08T21:55:24.209 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrading: 2024-05-08T21:55:24.209 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-08T21:55:24.209 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-08T21:55:24.209 INFO:teuthology.orchestra.run.smithi132.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-08T21:55:24.209 INFO:teuthology.orchestra.run.smithi132.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-08T21:55:24.209 INFO:teuthology.orchestra.run.smithi132.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-08T21:55:24.209 INFO:teuthology.orchestra.run.smithi132.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-08T21:55:24.209 INFO:teuthology.orchestra.run.smithi132.stdout: librados2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.7 M 2024-05-08T21:55:24.209 INFO:teuthology.orchestra.run.smithi132.stdout: librbd1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.6 M 2024-05-08T21:55:24.210 INFO:teuthology.orchestra.run.smithi132.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-08T21:55:24.210 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-08T21:55:24.210 INFO:teuthology.orchestra.run.smithi132.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-08T21:55:24.210 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-08T21:55:24.210 INFO:teuthology.orchestra.run.smithi132.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-08T21:55:24.210 INFO:teuthology.orchestra.run.smithi132.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-08T21:55:24.210 INFO:teuthology.orchestra.run.smithi132.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-08T21:55:24.210 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-08T21:55:24.210 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-05-08T21:55:24.210 INFO:teuthology.orchestra.run.smithi132.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-08T21:55:24.210 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 22 M 2024-05-08T21:55:24.210 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 24 k 2024-05-08T21:55:24.210 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 2.3 M 2024-05-08T21:55:24.211 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 245 k 2024-05-08T21:55:24.211 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 4.8 M 2024-05-08T21:55:24.211 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 18 M 2024-05-08T21:55:24.211 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 16 k 2024-05-08T21:55:24.211 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-selinux x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 24 k 2024-05-08T21:55:24.211 INFO:teuthology.orchestra.run.smithi132.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-08T21:55:24.211 INFO:teuthology.orchestra.run.smithi132.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-08T21:55:24.211 INFO:teuthology.orchestra.run.smithi132.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-08T21:55:24.211 INFO:teuthology.orchestra.run.smithi132.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-08T21:55:24.211 INFO:teuthology.orchestra.run.smithi132.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-08T21:55:24.211 INFO:teuthology.orchestra.run.smithi132.stdout: libcephsqlite x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 142 k 2024-05-08T21:55:24.211 INFO:teuthology.orchestra.run.smithi132.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-08T21:55:24.212 INFO:teuthology.orchestra.run.smithi132.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-08T21:55:24.212 INFO:teuthology.orchestra.run.smithi132.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-08T21:55:24.212 INFO:teuthology.orchestra.run.smithi132.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-08T21:55:24.212 INFO:teuthology.orchestra.run.smithi132.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-08T21:55:24.212 INFO:teuthology.orchestra.run.smithi132.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-08T21:55:24.212 INFO:teuthology.orchestra.run.smithi132.stdout: libradosstriper1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 474 k 2024-05-08T21:55:24.212 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-08T21:55:24.212 INFO:teuthology.orchestra.run.smithi132.stdout: librgw2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.5 M 2024-05-08T21:55:24.212 INFO:teuthology.orchestra.run.smithi132.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-08T21:55:24.212 INFO:teuthology.orchestra.run.smithi132.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-08T21:55:24.212 INFO:teuthology.orchestra.run.smithi132.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-08T21:55:24.212 INFO:teuthology.orchestra.run.smithi132.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-08T21:55:24.212 INFO:teuthology.orchestra.run.smithi132.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-08T21:55:24.212 INFO:teuthology.orchestra.run.smithi132.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-08T21:55:24.213 INFO:teuthology.orchestra.run.smithi132.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-08T21:55:24.213 INFO:teuthology.orchestra.run.smithi132.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-08T21:55:24.213 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-08T21:55:24.213 INFO:teuthology.orchestra.run.smithi132.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-08T21:55:24.213 INFO:teuthology.orchestra.run.smithi132.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-08T21:55:24.213 INFO:teuthology.orchestra.run.smithi132.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-08T21:55:24.213 INFO:teuthology.orchestra.run.smithi132.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-08T21:55:24.213 INFO:teuthology.orchestra.run.smithi132.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-08T21:55:24.213 INFO:teuthology.orchestra.run.smithi132.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-08T21:55:24.213 INFO:teuthology.orchestra.run.smithi132.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-08T21:55:24.213 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-08T21:55:24.213 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 45 k 2024-05-08T21:55:24.213 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 129 k 2024-05-08T21:55:24.213 INFO:teuthology.orchestra.run.smithi132.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-08T21:55:24.214 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-08T21:55:24.214 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-08T21:55:24.214 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-08T21:55:24.214 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-08T21:55:24.214 INFO:teuthology.orchestra.run.smithi132.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-08T21:55:24.214 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-08T21:55:24.214 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-08T21:55:24.214 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-08T21:55:24.214 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-08T21:55:24.214 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-08T21:55:24.214 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T21:55:24.214 INFO:teuthology.orchestra.run.smithi132.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-08T21:55:24.214 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-08T21:55:24.215 INFO:teuthology.orchestra.run.smithi132.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-08T21:55:24.215 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T21:55:24.215 INFO:teuthology.orchestra.run.smithi132.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-08T21:55:24.215 INFO:teuthology.orchestra.run.smithi132.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-08T21:55:24.215 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T21:55:24.215 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-08T21:55:24.215 INFO:teuthology.orchestra.run.smithi132.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-08T21:55:24.215 INFO:teuthology.orchestra.run.smithi132.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-08T21:55:24.215 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-08T21:55:24.215 INFO:teuthology.orchestra.run.smithi132.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-08T21:55:24.215 INFO:teuthology.orchestra.run.smithi132.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-08T21:55:24.215 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-08T21:55:24.215 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-08T21:55:24.216 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-08T21:55:24.216 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-08T21:55:24.216 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-08T21:55:24.216 INFO:teuthology.orchestra.run.smithi132.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-08T21:55:24.216 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T21:55:24.216 INFO:teuthology.orchestra.run.smithi132.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-08T21:55:24.216 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-08T21:55:24.216 INFO:teuthology.orchestra.run.smithi132.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-08T21:55:24.216 INFO:teuthology.orchestra.run.smithi132.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-08T21:55:24.216 INFO:teuthology.orchestra.run.smithi132.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-08T21:55:24.216 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-08T21:55:24.216 INFO:teuthology.orchestra.run.smithi132.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-08T21:55:24.216 INFO:teuthology.orchestra.run.smithi132.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-08T21:55:24.216 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-08T21:55:24.217 INFO:teuthology.orchestra.run.smithi132.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-08T21:55:24.217 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-08T21:55:24.217 INFO:teuthology.orchestra.run.smithi132.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-08T21:55:24.217 INFO:teuthology.orchestra.run.smithi132.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-08T21:55:24.217 INFO:teuthology.orchestra.run.smithi132.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-08T21:55:24.217 INFO:teuthology.orchestra.run.smithi132.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-08T21:55:24.217 INFO:teuthology.orchestra.run.smithi132.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-08T21:55:24.217 INFO:teuthology.orchestra.run.smithi132.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-08T21:55:24.217 INFO:teuthology.orchestra.run.smithi132.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-08T21:55:24.217 INFO:teuthology.orchestra.run.smithi132.stdout:Enabling module streams: 2024-05-08T21:55:24.217 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql 10 2024-05-08T21:55:24.217 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-05-08T21:55:24.217 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-05-08T21:55:24.217 INFO:teuthology.orchestra.run.smithi132.stdout:=================================================================================================================================== 2024-05-08T21:55:24.218 INFO:teuthology.orchestra.run.smithi132.stdout:Install 130 Packages 2024-05-08T21:55:24.218 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrade 16 Packages 2024-05-08T21:55:24.218 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-05-08T21:55:24.218 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 215 M 2024-05-08T21:55:24.218 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-05-08T21:55:24.475 INFO:teuthology.orchestra.run.smithi132.stdout:(1/146): ceph-18.2.2-1322.gd21e4f96.el8.x86_64. 55 kB/s | 6.4 kB 00:00 2024-05-08T21:55:24.659 INFO:teuthology.orchestra.run.smithi132.stdout:(2/146): ceph-fuse-18.2.2-1322.gd21e4f96.el8.x8 4.6 MB/s | 860 kB 00:00 2024-05-08T21:55:24.717 INFO:teuthology.orchestra.run.smithi132.stdout:(3/146): ceph-immutable-object-cache-18.2.2-132 2.8 MB/s | 166 kB 00:00 2024-05-08T21:55:24.754 INFO:teuthology.orchestra.run.smithi157.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T21:55:24.755 INFO:teuthology.orchestra.run.smithi157.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T21:55:24.773 INFO:teuthology.orchestra.run.smithi157.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-08T21:55:24.774 INFO:teuthology.orchestra.run.smithi157.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-08T21:55:24.775 INFO:teuthology.orchestra.run.smithi157.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-08T21:55:24.777 INFO:teuthology.orchestra.run.smithi157.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-08T21:55:24.779 INFO:teuthology.orchestra.run.smithi157.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-08T21:55:24.783 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-08T21:55:24.784 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-08T21:55:25.001 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-05-08T21:55:25.023 INFO:teuthology.orchestra.run.smithi132.stdout:(4/146): ceph-mds-18.2.2-1322.gd21e4f96.el8.x86 7.6 MB/s | 2.3 MB 00:00 2024-05-08T21:55:25.024 INFO:teuthology.orchestra.run.smithi157.stdout:=================================================================================================================================== 2024-05-08T21:55:25.024 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-05-08T21:55:25.024 INFO:teuthology.orchestra.run.smithi157.stdout:=================================================================================================================================== 2024-05-08T21:55:25.024 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-05-08T21:55:25.024 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-08T21:55:25.024 INFO:teuthology.orchestra.run.smithi157.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-08T21:55:25.024 INFO:teuthology.orchestra.run.smithi157.stdout: ceph x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.4 k 2024-05-08T21:55:25.024 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 5.5 M 2024-05-08T21:55:25.024 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 860 k 2024-05-08T21:55:25.024 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 166 k 2024-05-08T21:55:25.024 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 1.6 M 2024-05-08T21:55:25.025 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 138 k 2024-05-08T21:55:25.025 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 3.5 M 2024-05-08T21:55:25.025 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 7.4 M 2024-05-08T21:55:25.025 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 49 k 2024-05-08T21:55:25.025 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 13 M 2024-05-08T21:55:25.025 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 47 M 2024-05-08T21:55:25.025 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 263 k 2024-05-08T21:55:25.025 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 170 k 2024-05-08T21:55:25.025 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 261 k 2024-05-08T21:55:25.025 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 18 k 2024-05-08T21:55:25.025 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-08T21:55:25.025 INFO:teuthology.orchestra.run.smithi157.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-08T21:55:25.025 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-08T21:55:25.025 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 31 k 2024-05-08T21:55:25.026 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 782 k 2024-05-08T21:55:25.026 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 139 k 2024-05-08T21:55:25.026 INFO:teuthology.orchestra.run.smithi157.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-08T21:55:25.026 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-08T21:55:25.026 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-08T21:55:25.026 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-08T21:55:25.026 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-08T21:55:25.026 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-08T21:55:25.026 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 197 k 2024-05-08T21:55:25.026 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 370 k 2024-05-08T21:55:25.026 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 365 k 2024-05-08T21:55:25.026 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 107 k 2024-05-08T21:55:25.027 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 95 k 2024-05-08T21:55:25.027 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.5 M 2024-05-08T21:55:25.027 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 181 k 2024-05-08T21:55:25.027 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-08T21:55:25.027 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrading: 2024-05-08T21:55:25.027 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-08T21:55:25.027 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-08T21:55:25.027 INFO:teuthology.orchestra.run.smithi157.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-08T21:55:25.027 INFO:teuthology.orchestra.run.smithi157.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-08T21:55:25.027 INFO:teuthology.orchestra.run.smithi157.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-08T21:55:25.027 INFO:teuthology.orchestra.run.smithi157.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-08T21:55:25.027 INFO:teuthology.orchestra.run.smithi157.stdout: librados2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.7 M 2024-05-08T21:55:25.027 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.6 M 2024-05-08T21:55:25.027 INFO:teuthology.orchestra.run.smithi157.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-08T21:55:25.028 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-08T21:55:25.028 INFO:teuthology.orchestra.run.smithi157.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-08T21:55:25.028 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-08T21:55:25.028 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-08T21:55:25.028 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-08T21:55:25.028 INFO:teuthology.orchestra.run.smithi157.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-08T21:55:25.028 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-08T21:55:25.028 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-05-08T21:55:25.028 INFO:teuthology.orchestra.run.smithi157.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-08T21:55:25.028 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 22 M 2024-05-08T21:55:25.028 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 24 k 2024-05-08T21:55:25.028 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 2.3 M 2024-05-08T21:55:25.028 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 245 k 2024-05-08T21:55:25.029 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 4.8 M 2024-05-08T21:55:25.029 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 18 M 2024-05-08T21:55:25.029 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 16 k 2024-05-08T21:55:25.029 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 24 k 2024-05-08T21:55:25.029 INFO:teuthology.orchestra.run.smithi157.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-08T21:55:25.029 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-08T21:55:25.029 INFO:teuthology.orchestra.run.smithi157.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-08T21:55:25.029 INFO:teuthology.orchestra.run.smithi157.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-08T21:55:25.029 INFO:teuthology.orchestra.run.smithi157.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-08T21:55:25.029 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 142 k 2024-05-08T21:55:25.029 INFO:teuthology.orchestra.run.smithi157.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-08T21:55:25.029 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-08T21:55:25.029 INFO:teuthology.orchestra.run.smithi157.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-08T21:55:25.029 INFO:teuthology.orchestra.run.smithi157.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-08T21:55:25.030 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-08T21:55:25.030 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-08T21:55:25.030 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 474 k 2024-05-08T21:55:25.030 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-08T21:55:25.030 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.5 M 2024-05-08T21:55:25.030 INFO:teuthology.orchestra.run.smithi157.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-08T21:55:25.030 INFO:teuthology.orchestra.run.smithi157.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-08T21:55:25.030 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-08T21:55:25.030 INFO:teuthology.orchestra.run.smithi157.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-08T21:55:25.030 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-08T21:55:25.030 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-08T21:55:25.030 INFO:teuthology.orchestra.run.smithi157.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-08T21:55:25.030 INFO:teuthology.orchestra.run.smithi157.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-08T21:55:25.031 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-08T21:55:25.031 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-08T21:55:25.031 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-08T21:55:25.031 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-08T21:55:25.031 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-08T21:55:25.031 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-08T21:55:25.031 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-08T21:55:25.031 INFO:teuthology.orchestra.run.smithi157.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-08T21:55:25.031 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-08T21:55:25.031 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 45 k 2024-05-08T21:55:25.031 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 129 k 2024-05-08T21:55:25.031 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-08T21:55:25.031 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-08T21:55:25.031 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-08T21:55:25.032 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-08T21:55:25.032 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-08T21:55:25.032 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-08T21:55:25.032 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-08T21:55:25.032 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-08T21:55:25.032 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-08T21:55:25.032 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-08T21:55:25.032 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-08T21:55:25.032 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T21:55:25.032 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-08T21:55:25.032 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-08T21:55:25.032 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-08T21:55:25.032 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T21:55:25.032 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-08T21:55:25.033 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-08T21:55:25.033 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T21:55:25.033 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-08T21:55:25.033 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-08T21:55:25.033 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-08T21:55:25.033 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-08T21:55:25.033 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-08T21:55:25.033 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-08T21:55:25.033 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-08T21:55:25.033 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-08T21:55:25.033 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-08T21:55:25.033 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-08T21:55:25.033 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-08T21:55:25.034 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-08T21:55:25.034 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T21:55:25.034 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-08T21:55:25.034 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-08T21:55:25.034 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-08T21:55:25.034 INFO:teuthology.orchestra.run.smithi157.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-08T21:55:25.034 INFO:teuthology.orchestra.run.smithi157.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-08T21:55:25.034 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-08T21:55:25.034 INFO:teuthology.orchestra.run.smithi157.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-08T21:55:25.034 INFO:teuthology.orchestra.run.smithi157.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-08T21:55:25.034 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-08T21:55:25.034 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-08T21:55:25.034 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-08T21:55:25.035 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-08T21:55:25.035 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-08T21:55:25.035 INFO:teuthology.orchestra.run.smithi157.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-08T21:55:25.035 INFO:teuthology.orchestra.run.smithi157.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-08T21:55:25.035 INFO:teuthology.orchestra.run.smithi157.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-08T21:55:25.035 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-08T21:55:25.035 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-08T21:55:25.035 INFO:teuthology.orchestra.run.smithi157.stdout:Enabling module streams: 2024-05-08T21:55:25.035 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql 10 2024-05-08T21:55:25.035 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-08T21:55:25.035 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-05-08T21:55:25.035 INFO:teuthology.orchestra.run.smithi157.stdout:=================================================================================================================================== 2024-05-08T21:55:25.035 INFO:teuthology.orchestra.run.smithi157.stdout:Install 130 Packages 2024-05-08T21:55:25.035 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrade 16 Packages 2024-05-08T21:55:25.036 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-08T21:55:25.036 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 215 M 2024-05-08T21:55:25.036 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-05-08T21:55:25.151 INFO:teuthology.orchestra.run.smithi132.stdout:(5/146): ceph-base-18.2.2-1322.gd21e4f96.el8.x8 7.0 MB/s | 5.5 MB 00:00 2024-05-08T21:55:25.210 INFO:teuthology.orchestra.run.smithi132.stdout:(6/146): ceph-mgr-18.2.2-1322.gd21e4f96.el8.x86 8.1 MB/s | 1.6 MB 00:00 2024-05-08T21:55:25.593 INFO:teuthology.orchestra.run.smithi132.stdout:(7/146): ceph-mon-18.2.2-1322.gd21e4f96.el8.x86 11 MB/s | 4.8 MB 00:00 2024-05-08T21:55:25.633 INFO:teuthology.orchestra.run.smithi157.stdout:(1/146): ceph-18.2.2-1322.gd21e4f96.el8.x86_64. 14 kB/s | 6.4 kB 00:00 2024-05-08T21:55:26.807 INFO:teuthology.orchestra.run.smithi157.stdout:(2/146): ceph-fuse-18.2.2-1322.gd21e4f96.el8.x8 733 kB/s | 860 kB 00:01 2024-05-08T21:55:26.932 INFO:teuthology.orchestra.run.smithi157.stdout:(3/146): ceph-immutable-object-cache-18.2.2-132 1.3 MB/s | 166 kB 00:00 2024-05-08T21:55:26.960 INFO:teuthology.orchestra.run.smithi132.stdout:(8/146): ceph-radosgw-18.2.2-1322.gd21e4f96.el8 9.6 MB/s | 13 MB 00:01 2024-05-08T21:55:27.052 INFO:teuthology.orchestra.run.smithi132.stdout:(9/146): ceph-selinux-18.2.2-1322.gd21e4f96.el8 265 kB/s | 24 kB 00:00 2024-05-08T21:55:27.141 INFO:teuthology.orchestra.run.smithi157.stdout:(4/146): ceph-base-18.2.2-1322.gd21e4f96.el8.x8 2.8 MB/s | 5.5 MB 00:01 2024-05-08T21:55:27.766 INFO:teuthology.orchestra.run.smithi157.stdout:(5/146): ceph-mgr-18.2.2-1322.gd21e4f96.el8.x86 2.5 MB/s | 1.6 MB 00:00 2024-05-08T21:55:27.919 INFO:teuthology.orchestra.run.smithi132.stdout:(10/146): ceph-osd-18.2.2-1322.gd21e4f96.el8.x8 6.8 MB/s | 18 MB 00:02 2024-05-08T21:55:28.036 INFO:teuthology.orchestra.run.smithi132.stdout:(11/146): cephfs-mirror-18.2.2-1322.gd21e4f96.e 2.2 MB/s | 261 kB 00:00 2024-05-08T21:55:28.083 INFO:teuthology.orchestra.run.smithi157.stdout:(6/146): ceph-mds-18.2.2-1322.gd21e4f96.el8.x86 2.0 MB/s | 2.3 MB 00:01 2024-05-08T21:55:28.103 INFO:teuthology.orchestra.run.smithi132.stdout:(12/146): libcephfs-devel-18.2.2-1322.gd21e4f96 466 kB/s | 31 kB 00:00 2024-05-08T21:55:28.503 INFO:teuthology.orchestra.run.smithi132.stdout:(13/146): libcephfs2-18.2.2-1322.gd21e4f96.el8. 1.9 MB/s | 782 kB 00:00 2024-05-08T21:55:28.728 INFO:teuthology.orchestra.run.smithi132.stdout:(14/146): ceph-common-18.2.2-1322.gd21e4f96.el8 5.0 MB/s | 22 MB 00:04 2024-05-08T21:55:28.753 INFO:teuthology.orchestra.run.smithi132.stdout:(15/146): libcephsqlite-18.2.2-1322.gd21e4f96.e 567 kB/s | 142 kB 00:00 2024-05-08T21:55:28.829 INFO:teuthology.orchestra.run.smithi132.stdout:(16/146): librados-devel-18.2.2-1322.gd21e4f96. 1.4 MB/s | 139 kB 00:00 2024-05-08T21:55:28.937 INFO:teuthology.orchestra.run.smithi132.stdout:(17/146): libradosstriper1-18.2.2-1322.gd21e4f9 2.5 MB/s | 474 kB 00:00 2024-05-08T21:55:29.046 INFO:teuthology.orchestra.run.smithi132.stdout:(18/146): python3-ceph-argparse-18.2.2-1322.gd2 412 kB/s | 45 kB 00:00 2024-05-08T21:55:29.188 INFO:teuthology.orchestra.run.smithi132.stdout:(19/146): python3-ceph-common-18.2.2-1322.gd21e 910 kB/s | 129 kB 00:00 2024-05-08T21:55:29.272 INFO:teuthology.orchestra.run.smithi132.stdout:(20/146): python3-cephfs-18.2.2-1322.gd21e4f96. 2.3 MB/s | 197 kB 00:00 2024-05-08T21:55:29.372 INFO:teuthology.orchestra.run.smithi132.stdout:(21/146): python3-rados-18.2.2-1322.gd21e4f96.e 3.6 MB/s | 370 kB 00:00 2024-05-08T21:55:29.467 INFO:teuthology.orchestra.run.smithi157.stdout:(7/146): ceph-common-18.2.2-1322.gd21e4f96.el8. 5.1 MB/s | 22 MB 00:04 2024-05-08T21:55:29.480 INFO:teuthology.orchestra.run.smithi132.stdout:(22/146): python3-rbd-18.2.2-1322.gd21e4f96.el8 3.3 MB/s | 365 kB 00:00 2024-05-08T21:55:29.533 INFO:teuthology.orchestra.run.smithi157.stdout:(8/146): ceph-mon-18.2.2-1322.gd21e4f96.el8.x86 2.7 MB/s | 4.8 MB 00:01 2024-05-08T21:55:29.566 INFO:teuthology.orchestra.run.smithi132.stdout:(23/146): python3-rgw-18.2.2-1322.gd21e4f96.el8 1.3 MB/s | 107 kB 00:00 2024-05-08T21:55:29.610 INFO:teuthology.orchestra.run.smithi157.stdout:(9/146): ceph-selinux-18.2.2-1322.gd21e4f96.el8 324 kB/s | 24 kB 00:00 2024-05-08T21:55:29.657 INFO:teuthology.orchestra.run.smithi132.stdout:(24/146): rbd-fuse-18.2.2-1322.gd21e4f96.el8.x8 1.0 MB/s | 95 kB 00:00 2024-05-08T21:55:29.816 INFO:teuthology.orchestra.run.smithi131.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T21:55:29.817 INFO:teuthology.orchestra.run.smithi131.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T21:55:29.837 INFO:teuthology.orchestra.run.smithi131.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-08T21:55:29.838 INFO:teuthology.orchestra.run.smithi131.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-08T21:55:29.840 INFO:teuthology.orchestra.run.smithi131.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-08T21:55:29.842 INFO:teuthology.orchestra.run.smithi131.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-08T21:55:29.844 INFO:teuthology.orchestra.run.smithi131.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-08T21:55:29.867 INFO:teuthology.orchestra.run.smithi131.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-08T21:55:29.867 INFO:teuthology.orchestra.run.smithi131.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-08T21:55:30.080 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-05-08T21:55:30.098 INFO:teuthology.orchestra.run.smithi131.stdout:=================================================================================================================================== 2024-05-08T21:55:30.098 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-05-08T21:55:30.098 INFO:teuthology.orchestra.run.smithi131.stdout:=================================================================================================================================== 2024-05-08T21:55:30.099 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-05-08T21:55:30.099 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-08T21:55:30.099 INFO:teuthology.orchestra.run.smithi131.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-08T21:55:30.099 INFO:teuthology.orchestra.run.smithi131.stdout: ceph x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.4 k 2024-05-08T21:55:30.099 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 5.5 M 2024-05-08T21:55:30.099 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 860 k 2024-05-08T21:55:30.099 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 166 k 2024-05-08T21:55:30.099 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 1.6 M 2024-05-08T21:55:30.099 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 138 k 2024-05-08T21:55:30.099 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 3.5 M 2024-05-08T21:55:30.099 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 7.4 M 2024-05-08T21:55:30.100 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 49 k 2024-05-08T21:55:30.100 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 13 M 2024-05-08T21:55:30.100 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 47 M 2024-05-08T21:55:30.100 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 263 k 2024-05-08T21:55:30.100 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 170 k 2024-05-08T21:55:30.100 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 261 k 2024-05-08T21:55:30.100 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-top noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 18 k 2024-05-08T21:55:30.100 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-08T21:55:30.100 INFO:teuthology.orchestra.run.smithi131.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-08T21:55:30.100 INFO:teuthology.orchestra.run.smithi131.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-08T21:55:30.100 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 31 k 2024-05-08T21:55:30.100 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 782 k 2024-05-08T21:55:30.101 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 139 k 2024-05-08T21:55:30.101 INFO:teuthology.orchestra.run.smithi131.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-08T21:55:30.101 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-08T21:55:30.101 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-08T21:55:30.101 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-08T21:55:30.101 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-08T21:55:30.101 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-08T21:55:30.101 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 197 k 2024-05-08T21:55:30.101 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 370 k 2024-05-08T21:55:30.101 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 365 k 2024-05-08T21:55:30.101 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 107 k 2024-05-08T21:55:30.101 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 95 k 2024-05-08T21:55:30.102 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.5 M 2024-05-08T21:55:30.102 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 181 k 2024-05-08T21:55:30.102 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-08T21:55:30.102 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrading: 2024-05-08T21:55:30.102 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-08T21:55:30.102 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-08T21:55:30.102 INFO:teuthology.orchestra.run.smithi131.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-08T21:55:30.102 INFO:teuthology.orchestra.run.smithi131.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-08T21:55:30.102 INFO:teuthology.orchestra.run.smithi131.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-08T21:55:30.102 INFO:teuthology.orchestra.run.smithi131.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-08T21:55:30.102 INFO:teuthology.orchestra.run.smithi131.stdout: librados2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.7 M 2024-05-08T21:55:30.102 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.6 M 2024-05-08T21:55:30.102 INFO:teuthology.orchestra.run.smithi131.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-08T21:55:30.102 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-08T21:55:30.103 INFO:teuthology.orchestra.run.smithi131.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-08T21:55:30.103 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-08T21:55:30.103 INFO:teuthology.orchestra.run.smithi131.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-08T21:55:30.103 INFO:teuthology.orchestra.run.smithi131.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-08T21:55:30.103 INFO:teuthology.orchestra.run.smithi131.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-08T21:55:30.103 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-08T21:55:30.104 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-05-08T21:55:30.104 INFO:teuthology.orchestra.run.smithi131.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-08T21:55:30.104 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 22 M 2024-05-08T21:55:30.104 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 24 k 2024-05-08T21:55:30.104 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 2.3 M 2024-05-08T21:55:30.104 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 245 k 2024-05-08T21:55:30.104 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 4.8 M 2024-05-08T21:55:30.104 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 18 M 2024-05-08T21:55:30.104 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 16 k 2024-05-08T21:55:30.104 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 24 k 2024-05-08T21:55:30.104 INFO:teuthology.orchestra.run.smithi131.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-08T21:55:30.104 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-08T21:55:30.104 INFO:teuthology.orchestra.run.smithi131.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-08T21:55:30.105 INFO:teuthology.orchestra.run.smithi131.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-08T21:55:30.105 INFO:teuthology.orchestra.run.smithi131.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-08T21:55:30.105 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 142 k 2024-05-08T21:55:30.105 INFO:teuthology.orchestra.run.smithi131.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-08T21:55:30.105 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-08T21:55:30.105 INFO:teuthology.orchestra.run.smithi131.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-08T21:55:30.105 INFO:teuthology.orchestra.run.smithi131.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-08T21:55:30.105 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-08T21:55:30.105 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-08T21:55:30.105 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 474 k 2024-05-08T21:55:30.105 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-08T21:55:30.105 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.5 M 2024-05-08T21:55:30.105 INFO:teuthology.orchestra.run.smithi131.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-08T21:55:30.105 INFO:teuthology.orchestra.run.smithi131.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-08T21:55:30.106 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-08T21:55:30.106 INFO:teuthology.orchestra.run.smithi131.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-08T21:55:30.106 INFO:teuthology.orchestra.run.smithi131.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-08T21:55:30.106 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-08T21:55:30.106 INFO:teuthology.orchestra.run.smithi131.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-08T21:55:30.106 INFO:teuthology.orchestra.run.smithi131.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-08T21:55:30.106 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-08T21:55:30.106 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-08T21:55:30.106 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-08T21:55:30.106 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-08T21:55:30.106 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-08T21:55:30.108 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-08T21:55:30.108 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-08T21:55:30.108 INFO:teuthology.orchestra.run.smithi131.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-08T21:55:30.108 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-08T21:55:30.108 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 45 k 2024-05-08T21:55:30.108 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 129 k 2024-05-08T21:55:30.108 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-08T21:55:30.109 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-08T21:55:30.109 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-08T21:55:30.109 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-08T21:55:30.109 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-08T21:55:30.109 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-08T21:55:30.109 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-08T21:55:30.109 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-08T21:55:30.109 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-08T21:55:30.109 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-08T21:55:30.109 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-08T21:55:30.109 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T21:55:30.109 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-08T21:55:30.109 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-08T21:55:30.109 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-08T21:55:30.110 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T21:55:30.110 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-08T21:55:30.110 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-08T21:55:30.110 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T21:55:30.110 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-08T21:55:30.110 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-08T21:55:30.110 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-08T21:55:30.110 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-08T21:55:30.110 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-08T21:55:30.110 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-08T21:55:30.110 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-08T21:55:30.110 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-08T21:55:30.111 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-08T21:55:30.111 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-08T21:55:30.111 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-08T21:55:30.111 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-08T21:55:30.111 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T21:55:30.111 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-08T21:55:30.111 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-08T21:55:30.111 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-08T21:55:30.111 INFO:teuthology.orchestra.run.smithi131.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-08T21:55:30.111 INFO:teuthology.orchestra.run.smithi131.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-08T21:55:30.111 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-08T21:55:30.111 INFO:teuthology.orchestra.run.smithi131.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-08T21:55:30.111 INFO:teuthology.orchestra.run.smithi131.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-08T21:55:30.112 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-08T21:55:30.112 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-08T21:55:30.112 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-08T21:55:30.112 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-08T21:55:30.112 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-08T21:55:30.112 INFO:teuthology.orchestra.run.smithi131.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-08T21:55:30.112 INFO:teuthology.orchestra.run.smithi131.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-08T21:55:30.112 INFO:teuthology.orchestra.run.smithi131.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-08T21:55:30.112 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-08T21:55:30.112 INFO:teuthology.orchestra.run.smithi131.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-08T21:55:30.112 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling module streams: 2024-05-08T21:55:30.112 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql 10 2024-05-08T21:55:30.112 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:55:30.113 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-05-08T21:55:30.113 INFO:teuthology.orchestra.run.smithi131.stdout:=================================================================================================================================== 2024-05-08T21:55:30.113 INFO:teuthology.orchestra.run.smithi131.stdout:Install 130 Packages 2024-05-08T21:55:30.113 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrade 16 Packages 2024-05-08T21:55:30.113 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:55:30.114 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 215 M 2024-05-08T21:55:30.114 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-05-08T21:55:30.590 INFO:teuthology.orchestra.run.smithi131.stdout:(1/146): ceph-18.2.2-1322.gd21e4f96.el8.x86_64. 16 kB/s | 6.4 kB 00:00 2024-05-08T21:55:31.032 INFO:teuthology.orchestra.run.smithi132.stdout:(25/146): rbd-mirror-18.2.2-1322.gd21e4f96.el8. 2.5 MB/s | 3.5 MB 00:01 2024-05-08T21:55:31.117 INFO:teuthology.orchestra.run.smithi157.stdout:(10/146): ceph-radosgw-18.2.2-1322.gd21e4f96.el 7.9 MB/s | 13 MB 00:01 2024-05-08T21:55:31.132 INFO:teuthology.orchestra.run.smithi132.stdout:(26/146): rbd-nbd-18.2.2-1322.gd21e4f96.el8.x86 1.8 MB/s | 181 kB 00:00 2024-05-08T21:55:31.148 INFO:teuthology.orchestra.run.smithi131.stdout:(2/146): ceph-fuse-18.2.2-1322.gd21e4f96.el8.x8 1.5 MB/s | 860 kB 00:00 2024-05-08T21:55:31.217 INFO:teuthology.orchestra.run.smithi157.stdout:(11/146): cephfs-mirror-18.2.2-1322.gd21e4f96.e 2.6 MB/s | 261 kB 00:00 2024-05-08T21:55:31.224 INFO:teuthology.orchestra.run.smithi132.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1322.g 265 kB/s | 24 kB 00:00 2024-05-08T21:55:31.257 INFO:teuthology.orchestra.run.smithi131.stdout:(3/146): ceph-immutable-object-cache-18.2.2-132 1.5 MB/s | 166 kB 00:00 2024-05-08T21:55:31.292 INFO:teuthology.orchestra.run.smithi157.stdout:(12/146): libcephfs-devel-18.2.2-1322.gd21e4f96 413 kB/s | 31 kB 00:00 2024-05-08T21:55:31.316 INFO:teuthology.orchestra.run.smithi132.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1322.gd21e4f9 1.5 MB/s | 138 kB 00:00 2024-05-08T21:55:31.601 INFO:teuthology.orchestra.run.smithi157.stdout:(13/146): libcephfs2-18.2.2-1322.gd21e4f96.el8. 2.5 MB/s | 782 kB 00:00 2024-05-08T21:55:31.632 INFO:teuthology.orchestra.run.smithi131.stdout:(4/146): ceph-mds-18.2.2-1322.gd21e4f96.el8.x86 6.1 MB/s | 2.3 MB 00:00 2024-05-08T21:55:31.676 INFO:teuthology.orchestra.run.smithi157.stdout:(14/146): libcephsqlite-18.2.2-1322.gd21e4f96.e 1.8 MB/s | 142 kB 00:00 2024-05-08T21:55:31.883 INFO:teuthology.orchestra.run.smithi131.stdout:(5/146): ceph-mgr-18.2.2-1322.gd21e4f96.el8.x86 6.2 MB/s | 1.6 MB 00:00 2024-05-08T21:55:31.885 INFO:teuthology.orchestra.run.smithi157.stdout:(15/146): ceph-osd-18.2.2-1322.gd21e4f96.el8.x8 4.8 MB/s | 18 MB 00:03 2024-05-08T21:55:31.910 INFO:teuthology.orchestra.run.smithi157.stdout:(16/146): librados-devel-18.2.2-1322.gd21e4f96. 595 kB/s | 139 kB 00:00 2024-05-08T21:55:31.941 INFO:teuthology.orchestra.run.smithi132.stdout:(29/146): librgw2-18.2.2-1322.gd21e4f96.el8.x86 2.1 MB/s | 6.5 MB 00:03 2024-05-08T21:55:31.966 INFO:teuthology.orchestra.run.smithi131.stdout:(6/146): ceph-base-18.2.2-1322.gd21e4f96.el8.x8 3.1 MB/s | 5.5 MB 00:01 2024-05-08T21:55:32.044 INFO:teuthology.orchestra.run.smithi157.stdout:(17/146): libradosstriper1-18.2.2-1322.gd21e4f9 2.9 MB/s | 474 kB 00:00 2024-05-08T21:55:32.152 INFO:teuthology.orchestra.run.smithi157.stdout:(18/146): python3-ceph-argparse-18.2.2-1322.gd2 413 kB/s | 45 kB 00:00 2024-05-08T21:55:32.261 INFO:teuthology.orchestra.run.smithi157.stdout:(19/146): python3-ceph-common-18.2.2-1322.gd21e 1.2 MB/s | 129 kB 00:00 2024-05-08T21:55:32.370 INFO:teuthology.orchestra.run.smithi157.stdout:(20/146): python3-cephfs-18.2.2-1322.gd21e4f96. 1.8 MB/s | 197 kB 00:00 2024-05-08T21:55:32.575 INFO:teuthology.orchestra.run.smithi131.stdout:(7/146): ceph-mon-18.2.2-1322.gd21e4f96.el8.x86 7.0 MB/s | 4.8 MB 00:00 2024-05-08T21:55:32.578 INFO:teuthology.orchestra.run.smithi157.stdout:(21/146): python3-rados-18.2.2-1322.gd21e4f96.e 1.7 MB/s | 370 kB 00:00 2024-05-08T21:55:32.704 INFO:teuthology.orchestra.run.smithi157.stdout:(22/146): python3-rbd-18.2.2-1322.gd21e4f96.el8 2.8 MB/s | 365 kB 00:00 2024-05-08T21:55:32.812 INFO:teuthology.orchestra.run.smithi157.stdout:(23/146): python3-rgw-18.2.2-1322.gd21e4f96.el8 986 kB/s | 107 kB 00:00 2024-05-08T21:55:32.946 INFO:teuthology.orchestra.run.smithi157.stdout:(24/146): rbd-fuse-18.2.2-1322.gd21e4f96.el8.x8 713 kB/s | 95 kB 00:00 2024-05-08T21:55:33.366 INFO:teuthology.orchestra.run.smithi132.stdout:(30/146): ceph-mgr-dashboard-18.2.2-1322.gd21e4 1.7 MB/s | 3.5 MB 00:02 2024-05-08T21:55:33.541 INFO:teuthology.orchestra.run.smithi132.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1322.gd2 1.4 MB/s | 245 kB 00:00 2024-05-08T21:55:33.583 INFO:teuthology.orchestra.run.smithi132.stdout:(32/146): ceph-mgr-rook-18.2.2-1322.gd21e4f96.e 1.2 MB/s | 49 kB 00:00 2024-05-08T21:55:33.650 INFO:teuthology.orchestra.run.smithi132.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1322.gd 245 kB/s | 16 kB 00:00 2024-05-08T21:55:33.822 INFO:teuthology.orchestra.run.smithi132.stdout:(34/146): ceph-volume-18.2.2-1322.gd21e4f96.el8 1.5 MB/s | 263 kB 00:00 2024-05-08T21:55:33.909 INFO:teuthology.orchestra.run.smithi132.stdout:(35/146): cephadm-18.2.2-1322.gd21e4f96.el8.noa 1.9 MB/s | 170 kB 00:00 2024-05-08T21:55:33.962 INFO:teuthology.orchestra.run.smithi157.stdout:(25/146): rbd-mirror-18.2.2-1322.gd21e4f96.el8. 3.4 MB/s | 3.5 MB 00:01 2024-05-08T21:55:34.035 INFO:teuthology.orchestra.run.smithi132.stdout:(36/146): cephfs-top-18.2.2-1322.gd21e4f96.el8. 145 kB/s | 18 kB 00:00 2024-05-08T21:55:34.046 INFO:teuthology.orchestra.run.smithi157.stdout:(26/146): rbd-nbd-18.2.2-1322.gd21e4f96.el8.x86 2.1 MB/s | 181 kB 00:00 2024-05-08T21:55:34.130 INFO:teuthology.orchestra.run.smithi157.stdout:(27/146): librgw2-18.2.2-1322.gd21e4f96.el8.x86 2.9 MB/s | 6.5 MB 00:02 2024-05-08T21:55:34.155 INFO:teuthology.orchestra.run.smithi157.stdout:(28/146): ceph-grafana-dashboards-18.2.2-1322.g 224 kB/s | 24 kB 00:00 2024-05-08T21:55:34.205 INFO:teuthology.orchestra.run.smithi157.stdout:(29/146): ceph-mgr-cephadm-18.2.2-1322.gd21e4f9 1.8 MB/s | 138 kB 00:00 2024-05-08T21:55:34.425 INFO:teuthology.orchestra.run.smithi131.stdout:(8/146): ceph-radosgw-18.2.2-1322.gd21e4f96.el8 7.1 MB/s | 13 MB 00:01 2024-05-08T21:55:34.426 INFO:teuthology.orchestra.run.smithi132.stdout:(37/146): gperftools-libs-2.7-9.el8.x86_64.rpm 782 kB/s | 306 kB 00:00 2024-05-08T21:55:34.492 INFO:teuthology.orchestra.run.smithi131.stdout:(9/146): ceph-selinux-18.2.2-1322.gd21e4f96.el8 366 kB/s | 24 kB 00:00 2024-05-08T21:55:34.493 INFO:teuthology.orchestra.run.smithi132.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 885 kB/s | 59 kB 00:00 2024-05-08T21:55:34.569 INFO:teuthology.orchestra.run.smithi132.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 1.0 MB/s | 75 kB 00:00 2024-05-08T21:55:35.061 INFO:teuthology.orchestra.run.smithi132.stdout:(40/146): ceph-test-18.2.2-1322.gd21e4f96.el8.x 5.9 MB/s | 47 MB 00:08 2024-05-08T21:55:35.086 INFO:teuthology.orchestra.run.smithi132.stdout:(41/146): python3-asyncssh-2.7.0-2.el8.noarch.r 921 kB/s | 476 kB 00:00 2024-05-08T21:55:35.089 INFO:teuthology.orchestra.run.smithi157.stdout:(30/146): ceph-mgr-dashboard-18.2.2-1322.gd21e4 3.8 MB/s | 3.5 MB 00:00 2024-05-08T21:55:35.153 INFO:teuthology.orchestra.run.smithi132.stdout:(42/146): python3-beautifulsoup4-4.6.3-2.el8.1. 2.7 MB/s | 185 kB 00:00 2024-05-08T21:55:35.212 INFO:teuthology.orchestra.run.smithi132.stdout:(43/146): python3-cachetools-3.1.1-4.el8.noarch 566 kB/s | 33 kB 00:00 2024-05-08T21:55:35.231 INFO:teuthology.orchestra.run.smithi157.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1322.gd2 1.7 MB/s | 245 kB 00:00 2024-05-08T21:55:35.237 INFO:teuthology.orchestra.run.smithi132.stdout:(44/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 248 kB/s | 44 kB 00:00 2024-05-08T21:55:35.270 INFO:teuthology.orchestra.run.smithi132.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 279 kB/s | 16 kB 00:00 2024-05-08T21:55:35.298 INFO:teuthology.orchestra.run.smithi157.stdout:(32/146): ceph-mgr-rook-18.2.2-1322.gd21e4f96.e 738 kB/s | 49 kB 00:00 2024-05-08T21:55:35.371 INFO:teuthology.orchestra.run.smithi132.stdout:(46/146): python3-cherrypy-18.4.0-1.el8.noarch. 3.8 MB/s | 384 kB 00:00 2024-05-08T21:55:35.373 INFO:teuthology.orchestra.run.smithi157.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1322.gd 218 kB/s | 16 kB 00:00 2024-05-08T21:55:35.396 INFO:teuthology.orchestra.run.smithi132.stdout:(47/146): python3-cheroot-8.5.2-1.el8.noarch.rp 1.1 MB/s | 173 kB 00:00 2024-05-08T21:55:35.422 INFO:teuthology.orchestra.run.smithi132.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 1.6 MB/s | 81 kB 00:00 2024-05-08T21:55:35.455 INFO:teuthology.orchestra.run.smithi132.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 190 kB/s | 11 kB 00:00 2024-05-08T21:55:35.480 INFO:teuthology.orchestra.run.smithi132.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 300 kB/s | 18 kB 00:00 2024-05-08T21:55:35.507 INFO:teuthology.orchestra.run.smithi157.stdout:(34/146): ceph-volume-18.2.2-1322.gd21e4f96.el8 1.9 MB/s | 263 kB 00:00 2024-05-08T21:55:35.539 INFO:teuthology.orchestra.run.smithi132.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 837 kB/s | 49 kB 00:00 2024-05-08T21:55:35.582 INFO:teuthology.orchestra.run.smithi157.stdout:(35/146): cephadm-18.2.2-1322.gd21e4f96.el8.noa 2.2 MB/s | 170 kB 00:00 2024-05-08T21:55:35.631 INFO:teuthology.orchestra.run.smithi132.stdout:(52/146): python3-more-itertools-7.2.0-3.el8.no 646 kB/s | 59 kB 00:00 2024-05-08T21:55:35.649 INFO:teuthology.orchestra.run.smithi157.stdout:(36/146): cephfs-top-18.2.2-1322.gd21e4f96.el8. 271 kB/s | 18 kB 00:00 2024-05-08T21:55:35.673 INFO:teuthology.orchestra.run.smithi132.stdout:(53/146): python3-kubernetes-11.0.0-6.el8.noarc 5.4 MB/s | 1.2 MB 00:00 2024-05-08T21:55:35.698 INFO:teuthology.orchestra.run.smithi132.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 891 kB/s | 60 kB 00:00 2024-05-08T21:55:35.740 INFO:teuthology.orchestra.run.smithi132.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 4.1 MB/s | 283 kB 00:00 2024-05-08T21:55:35.766 INFO:teuthology.orchestra.run.smithi132.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 233 kB/s | 16 kB 00:00 2024-05-08T21:55:35.799 INFO:teuthology.orchestra.run.smithi132.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 577 kB/s | 34 kB 00:00 2024-05-08T21:55:35.824 INFO:teuthology.orchestra.run.smithi132.stdout:(58/146): python3-routes-2.4.1-12.el8.noarch.rp 3.3 MB/s | 196 kB 00:00 2024-05-08T21:55:35.857 INFO:teuthology.orchestra.run.smithi157.stdout:(37/146): gperftools-libs-2.7-9.el8.x86_64.rpm 1.4 MB/s | 306 kB 00:00 2024-05-08T21:55:35.858 INFO:teuthology.orchestra.run.smithi132.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 1.1 MB/s | 64 kB 00:00 2024-05-08T21:55:35.883 INFO:teuthology.orchestra.run.smithi132.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 320 kB/s | 19 kB 00:00 2024-05-08T21:55:35.925 INFO:teuthology.orchestra.run.smithi157.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 882 kB/s | 59 kB 00:00 2024-05-08T21:55:35.950 INFO:teuthology.orchestra.run.smithi132.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 261 kB/s | 24 kB 00:00 2024-05-08T21:55:35.966 INFO:teuthology.orchestra.run.smithi157.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 1.8 MB/s | 75 kB 00:00 2024-05-08T21:55:35.976 INFO:teuthology.orchestra.run.smithi132.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 318 kB/s | 29 kB 00:00 2024-05-08T21:55:36.010 INFO:teuthology.orchestra.run.smithi132.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 470 kB/s | 27 kB 00:00 2024-05-08T21:55:36.043 INFO:teuthology.orchestra.run.smithi132.stdout:(64/146): python3-waitress-1.4.3-1.el8.noarch.r 3.7 MB/s | 255 kB 00:00 2024-05-08T21:55:36.077 INFO:teuthology.orchestra.run.smithi132.stdout:(65/146): python3-webob-1.8.5-1.el8.1.noarch.rp 3.7 MB/s | 251 kB 00:00 2024-05-08T21:55:36.102 INFO:teuthology.orchestra.run.smithi132.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2024-05-08T21:55:36.125 INFO:teuthology.orchestra.run.smithi157.stdout:(40/146): python3-asyncssh-2.7.0-2.el8.noarch.r 2.9 MB/s | 476 kB 00:00 2024-05-08T21:55:36.136 INFO:teuthology.orchestra.run.smithi132.stdout:(67/146): python3-webtest-2.0.33-1.el8.noarch.r 1.4 MB/s | 85 kB 00:00 2024-05-08T21:55:36.161 INFO:teuthology.orchestra.run.smithi132.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 392 kB/s | 23 kB 00:00 2024-05-08T21:55:36.220 INFO:teuthology.orchestra.run.smithi132.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 70 kB 00:00 2024-05-08T21:55:36.311 INFO:teuthology.orchestra.run.smithi132.stdout:(70/146): thrift-0.13.0-2.el8.x86_64.rpm 9.9 MB/s | 1.7 MB 00:00 2024-05-08T21:55:36.370 INFO:teuthology.orchestra.run.smithi132.stdout:(71/146): python3-packaging-16.8-9.el8.noarch.r 321 kB/s | 48 kB 00:00 2024-05-08T21:55:36.381 INFO:teuthology.orchestra.run.smithi157.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 171 kB/s | 44 kB 00:00 2024-05-08T21:55:36.412 INFO:teuthology.orchestra.run.smithi132.stdout:(72/146): python3-jsonpointer-1.10-11.el8.noarc 199 kB/s | 20 kB 00:00 2024-05-08T21:55:36.509 INFO:teuthology.orchestra.run.smithi157.stdout:(42/146): python3-beautifulsoup4-4.6.3-2.el8.1. 1.4 MB/s | 185 kB 00:00 2024-05-08T21:55:36.537 INFO:teuthology.orchestra.run.smithi132.stdout:(73/146): python3-prettytable-0.7.2-14.el8.noar 263 kB/s | 44 kB 00:00 2024-05-08T21:55:36.551 INFO:teuthology.orchestra.run.smithi157.stdout:(43/146): python3-cachetools-3.1.1-4.el8.noarch 795 kB/s | 33 kB 00:00 2024-05-08T21:55:36.621 INFO:teuthology.orchestra.run.smithi132.stdout:(74/146): ceph-mgr-diskprediction-local-18.2.2- 1.6 MB/s | 7.4 MB 00:04 2024-05-08T21:55:36.646 INFO:teuthology.orchestra.run.smithi132.stdout:(75/146): python3-jsonpatch-1.21-2.el8.noarch.r 244 kB/s | 27 kB 00:00 2024-05-08T21:55:36.660 INFO:teuthology.orchestra.run.smithi157.stdout:(44/146): ceph-mgr-diskprediction-local-18.2.2- 3.0 MB/s | 7.4 MB 00:02 2024-05-08T21:55:36.676 INFO:teuthology.orchestra.run.smithi157.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 130 kB/s | 16 kB 00:00 2024-05-08T21:55:36.680 INFO:teuthology.orchestra.run.smithi132.stdout:(76/146): python3-pyasn1-modules-0.3.7-6.el8.no 412 kB/s | 110 kB 00:00 2024-05-08T21:55:36.727 INFO:teuthology.orchestra.run.smithi157.stdout:(46/146): python3-cherrypy-18.4.0-1.el8.noarch. 7.5 MB/s | 384 kB 00:00 2024-05-08T21:55:36.734 INFO:teuthology.orchestra.run.smithi131.stdout:(10/146): ceph-osd-18.2.2-1322.gd21e4f96.el8.x8 3.9 MB/s | 18 MB 00:04 2024-05-08T21:55:36.755 INFO:teuthology.orchestra.run.smithi132.stdout:(77/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 946 kB/s | 126 kB 00:00 2024-05-08T21:55:36.794 INFO:teuthology.orchestra.run.smithi157.stdout:(47/146): python3-google-auth-1.1.1-10.el8.noar 1.2 MB/s | 81 kB 00:00 2024-05-08T21:55:36.822 INFO:teuthology.orchestra.run.smithi132.stdout:(78/146): python3-werkzeug-0.12.2-4.el8.noarch. 2.6 MB/s | 457 kB 00:00 2024-05-08T21:55:36.836 INFO:teuthology.orchestra.run.smithi157.stdout:(48/146): python3-cheroot-8.5.2-1.el8.noarch.rp 983 kB/s | 173 kB 00:00 2024-05-08T21:55:36.853 INFO:teuthology.orchestra.run.smithi157.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 189 kB/s | 11 kB 00:00 2024-05-08T21:55:36.855 INFO:teuthology.orchestra.run.smithi132.stdout:(79/146): bison-3.0.4-10.el8.x86_64.rpm 3.8 MB/s | 688 kB 00:00 2024-05-08T21:55:36.870 INFO:teuthology.orchestra.run.smithi157.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 521 kB/s | 18 kB 00:00 2024-05-08T21:55:36.875 INFO:teuthology.orchestra.run.smithi131.stdout:(11/146): cephfs-mirror-18.2.2-1322.gd21e4f96.e 1.8 MB/s | 261 kB 00:00 2024-05-08T21:55:36.906 INFO:teuthology.orchestra.run.smithi132.stdout:(80/146): libtool-2.4.6-25.el8.x86_64.rpm 8.3 MB/s | 709 kB 00:00 2024-05-08T21:55:36.931 INFO:teuthology.orchestra.run.smithi157.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 805 kB/s | 49 kB 00:00 2024-05-08T21:55:36.932 INFO:teuthology.orchestra.run.smithi132.stdout:(81/146): flex-2.6.1-9.el8.x86_64.rpm 1.8 MB/s | 320 kB 00:00 2024-05-08T21:55:36.951 INFO:teuthology.orchestra.run.smithi131.stdout:(12/146): libcephfs-devel-18.2.2-1322.gd21e4f96 414 kB/s | 31 kB 00:00 2024-05-08T21:55:36.957 INFO:teuthology.orchestra.run.smithi132.stdout:(82/146): python3-markupsafe-0.23-19.el8.x86_64 385 kB/s | 39 kB 00:00 2024-05-08T21:55:36.970 INFO:teuthology.orchestra.run.smithi157.stdout:(52/146): python3-kubernetes-11.0.0-6.el8.noarc 9.9 MB/s | 1.2 MB 00:00 2024-05-08T21:55:36.987 INFO:teuthology.orchestra.run.smithi157.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2024-05-08T21:55:37.013 INFO:teuthology.orchestra.run.smithi157.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 1.4 MB/s | 60 kB 00:00 2024-05-08T21:55:37.049 INFO:teuthology.orchestra.run.smithi132.stdout:(83/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.8 MB/s | 259 kB 00:00 2024-05-08T21:55:37.055 INFO:teuthology.orchestra.run.smithi157.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 4.1 MB/s | 283 kB 00:00 2024-05-08T21:55:37.071 INFO:teuthology.orchestra.run.smithi157.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 266 kB/s | 16 kB 00:00 2024-05-08T21:55:37.108 INFO:teuthology.orchestra.run.smithi132.stdout:(84/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 689 kB/s | 103 kB 00:00 2024-05-08T21:55:37.122 INFO:teuthology.orchestra.run.smithi157.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 504 kB/s | 34 kB 00:00 2024-05-08T21:55:37.124 INFO:teuthology.orchestra.run.smithi132.stdout:(85/146): uuid-1.6.2-43.el8.x86_64.rpm 332 kB/s | 64 kB 00:00 2024-05-08T21:55:37.147 INFO:teuthology.orchestra.run.smithi157.stdout:(58/146): python3-routes-2.4.1-12.el8.noarch.rp 2.5 MB/s | 196 kB 00:00 2024-05-08T21:55:37.164 INFO:teuthology.orchestra.run.smithi157.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 1.5 MB/s | 64 kB 00:00 2024-05-08T21:55:37.176 INFO:teuthology.orchestra.run.smithi131.stdout:(13/146): libcephfs2-18.2.2-1322.gd21e4f96.el8. 3.4 MB/s | 782 kB 00:00 2024-05-08T21:55:37.198 INFO:teuthology.orchestra.run.smithi157.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 375 kB/s | 19 kB 00:00 2024-05-08T21:55:37.223 INFO:teuthology.orchestra.run.smithi157.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 409 kB/s | 24 kB 00:00 2024-05-08T21:55:37.248 INFO:teuthology.orchestra.run.smithi157.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 584 kB/s | 29 kB 00:00 2024-05-08T21:55:37.260 INFO:teuthology.orchestra.run.smithi131.stdout:(14/146): libcephsqlite-18.2.2-1322.gd21e4f96.e 1.7 MB/s | 142 kB 00:00 2024-05-08T21:55:37.275 INFO:teuthology.orchestra.run.smithi132.stdout:(86/146): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2024-05-08T21:55:37.293 INFO:teuthology.orchestra.run.smithi157.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 412 kB/s | 27 kB 00:00 2024-05-08T21:55:37.307 INFO:teuthology.orchestra.run.smithi157.stdout:(64/146): python3-waitress-1.4.3-1.el8.noarch.r 4.3 MB/s | 255 kB 00:00 2024-05-08T21:55:37.308 INFO:teuthology.orchestra.run.smithi132.stdout:(87/146): python3-jinja2-2.10.1-3.el8.noarch.rp 2.6 MB/s | 538 kB 00:00 2024-05-08T21:55:37.341 INFO:teuthology.orchestra.run.smithi157.stdout:(65/146): python3-webob-1.8.5-1.el8.1.noarch.rp 4.9 MB/s | 251 kB 00:00 2024-05-08T21:55:37.352 INFO:teuthology.orchestra.run.smithi131.stdout:(15/146): librados-devel-18.2.2-1322.gd21e4f96. 1.5 MB/s | 139 kB 00:00 2024-05-08T21:55:37.366 INFO:teuthology.orchestra.run.smithi157.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2024-05-08T21:55:37.391 INFO:teuthology.orchestra.run.smithi157.stdout:(67/146): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2024-05-08T21:55:37.417 INFO:teuthology.orchestra.run.smithi157.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 456 kB/s | 23 kB 00:00 2024-05-08T21:55:37.433 INFO:teuthology.orchestra.run.smithi132.stdout:(88/146): python3-numpy-1.14.3-10.el8.x86_64.rp 12 MB/s | 3.7 MB 00:00 2024-05-08T21:55:37.459 INFO:teuthology.orchestra.run.smithi132.stdout:(89/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.5 MB/s | 225 kB 00:00 2024-05-08T21:55:37.475 INFO:teuthology.orchestra.run.smithi157.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 70 kB 00:00 2024-05-08T21:55:37.484 INFO:teuthology.orchestra.run.smithi132.stdout:(90/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.5 MB/s | 323 kB 00:00 2024-05-08T21:55:37.526 INFO:teuthology.orchestra.run.smithi157.stdout:(70/146): thrift-0.13.0-2.el8.x86_64.rpm 13 MB/s | 1.7 MB 00:00 2024-05-08T21:55:37.527 INFO:teuthology.orchestra.run.smithi131.stdout:(16/146): libradosstriper1-18.2.2-1322.gd21e4f9 2.6 MB/s | 474 kB 00:00 2024-05-08T21:55:37.576 INFO:teuthology.orchestra.run.smithi132.stdout:(91/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.4 MB/s | 354 kB 00:00 2024-05-08T21:55:37.659 INFO:teuthology.orchestra.run.smithi157.stdout:(71/146): python3-jsonpointer-1.10-11.el8.noarc 109 kB/s | 20 kB 00:00 2024-05-08T21:55:37.684 INFO:teuthology.orchestra.run.smithi157.stdout:(72/146): python3-prettytable-0.7.2-14.el8.noar 288 kB/s | 44 kB 00:00 2024-05-08T21:55:37.693 INFO:teuthology.orchestra.run.smithi132.stdout:(92/146): python3-mako-1.0.6-14.el8.noarch.rpm 752 kB/s | 157 kB 00:00 2024-05-08T21:55:37.760 INFO:teuthology.orchestra.run.smithi157.stdout:(73/146): python3-jsonpatch-1.21-2.el8.noarch.r 354 kB/s | 27 kB 00:00 2024-05-08T21:55:37.810 INFO:teuthology.orchestra.run.smithi157.stdout:(74/146): python3-pyasn1-modules-0.3.7-6.el8.no 731 kB/s | 110 kB 00:00 2024-05-08T21:55:37.826 INFO:teuthology.orchestra.run.smithi132.stdout:(93/146): openblas-0.3.15-6.el8.x86_64.rpm 20 MB/s | 5.0 MB 00:00 2024-05-08T21:55:37.902 INFO:teuthology.orchestra.run.smithi132.stdout:(94/146): openblas-threads-0.3.15-6.el8.x86_64. 12 MB/s | 5.2 MB 00:00 2024-05-08T21:55:37.935 INFO:teuthology.orchestra.run.smithi157.stdout:(75/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 721 kB/s | 126 kB 00:00 2024-05-08T21:55:37.969 INFO:teuthology.orchestra.run.smithi157.stdout:(76/146): python3-werkzeug-0.12.2-4.el8.noarch. 2.8 MB/s | 457 kB 00:00 2024-05-08T21:55:37.977 INFO:teuthology.orchestra.run.smithi132.stdout:(95/146): automake-1.16.1-8.el8.noarch.rpm 2.5 MB/s | 714 kB 00:00 2024-05-08T21:55:38.036 INFO:teuthology.orchestra.run.smithi132.stdout:(96/146): libpq-13.11-1.el8.x86_64.rpm 948 kB/s | 198 kB 00:00 2024-05-08T21:55:38.086 INFO:teuthology.orchestra.run.smithi132.stdout:(97/146): jq-1.6-7.el8.x86_64.rpm 1.1 MB/s | 202 kB 00:00 2024-05-08T21:55:38.161 INFO:teuthology.orchestra.run.smithi157.stdout:(77/146): bison-3.0.4-10.el8.x86_64.rpm 3.0 MB/s | 688 kB 00:00 2024-05-08T21:55:38.161 INFO:teuthology.orchestra.run.smithi132.stdout:(98/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-05-08T21:55:38.194 INFO:teuthology.orchestra.run.smithi157.stdout:(78/146): flex-2.6.1-9.el8.x86_64.rpm 1.4 MB/s | 320 kB 00:00 2024-05-08T21:55:38.261 INFO:teuthology.orchestra.run.smithi131.stdout:(17/146): ceph-common-18.2.2-1322.gd21e4f96.el8 2.7 MB/s | 22 MB 00:08 2024-05-08T21:55:38.261 INFO:teuthology.orchestra.run.smithi132.stdout:(99/146): python3-pytz-2017.2-11.el8.noarch.rpm 191 kB/s | 54 kB 00:00 2024-05-08T21:55:38.319 INFO:teuthology.orchestra.run.smithi131.stdout:(18/146): python3-ceph-argparse-18.2.2-1322.gd2 770 kB/s | 45 kB 00:00 2024-05-08T21:55:38.337 INFO:teuthology.orchestra.run.smithi157.stdout:(79/146): libtool-2.4.6-25.el8.x86_64.rpm 3.9 MB/s | 709 kB 00:00 2024-05-08T21:55:38.387 INFO:teuthology.orchestra.run.smithi157.stdout:(80/146): python3-markupsafe-0.23-19.el8.x86_64 202 kB/s | 39 kB 00:00 2024-05-08T21:55:38.487 INFO:teuthology.orchestra.run.smithi132.stdout:(100/146): python3-scipy-1.0.0-21.module+el8.9. 30 MB/s | 14 MB 00:00 2024-05-08T21:55:38.554 INFO:teuthology.orchestra.run.smithi132.stdout:(101/146): postgresql-contrib-10.23-4.module+el 2.7 MB/s | 812 kB 00:00 2024-05-08T21:55:38.554 INFO:teuthology.orchestra.run.smithi157.stdout:(81/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.2 MB/s | 259 kB 00:00 2024-05-08T21:55:38.579 INFO:teuthology.orchestra.run.smithi157.stdout:(82/146): uuid-1.6.2-43.el8.x86_64.rpm 333 kB/s | 64 kB 00:00 2024-05-08T21:55:38.596 INFO:teuthology.orchestra.run.smithi132.stdout:(102/146): postgresql-10.23-4.module+el8.9.0+21 3.5 MB/s | 1.5 MB 00:00 2024-05-08T21:55:38.713 INFO:teuthology.orchestra.run.smithi132.stdout:(103/146): postgresql-server-10.23-4.module+el8 22 MB/s | 5.1 MB 00:00 2024-05-08T21:55:38.738 INFO:teuthology.orchestra.run.smithi132.stdout:(104/146): libverto-devel-0.3.0-5.el8.x86_64.rp 97 kB/s | 18 kB 00:00 2024-05-08T21:55:38.763 INFO:teuthology.orchestra.run.smithi157.stdout:(83/146): python3-babel-2.5.1-7.el8.noarch.rpm 26 MB/s | 4.8 MB 00:00 2024-05-08T21:55:38.763 INFO:teuthology.orchestra.run.smithi132.stdout:(105/146): python3-jwt-1.6.1-2.el8.noarch.rpm 255 kB/s | 43 kB 00:00 2024-05-08T21:55:38.780 INFO:teuthology.orchestra.run.smithi157.stdout:(84/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 459 kB/s | 103 kB 00:00 2024-05-08T21:55:38.803 INFO:teuthology.orchestra.run.smithi131.stdout:(19/146): ceph-test-18.2.2-1322.gd21e4f96.el8.x 11 MB/s | 47 MB 00:04 2024-05-08T21:55:38.828 INFO:teuthology.orchestra.run.smithi131.stdout:(20/146): python3-ceph-common-18.2.2-1322.gd21e 253 kB/s | 129 kB 00:00 2024-05-08T21:55:38.855 INFO:teuthology.orchestra.run.smithi132.stdout:(106/146): python3-oauthlib-2.1.0-1.el8.noarch. 1.1 MB/s | 155 kB 00:00 2024-05-08T21:55:38.911 INFO:teuthology.orchestra.run.smithi131.stdout:(21/146): python3-cephfs-18.2.2-1322.gd21e4f96. 1.8 MB/s | 197 kB 00:00 2024-05-08T21:55:38.913 INFO:teuthology.orchestra.run.smithi132.stdout:(107/146): python3-requests-oauthlib-1.0.0-1.el 283 kB/s | 43 kB 00:00 2024-05-08T21:55:38.930 INFO:teuthology.orchestra.run.smithi157.stdout:(85/146): python3-jinja2-2.10.1-3.el8.noarch.rp 3.2 MB/s | 538 kB 00:00 2024-05-08T21:55:38.981 INFO:teuthology.orchestra.run.smithi132.stdout:(108/146): libaio-devel-0.3.112-1.el8.x86_64.rp 149 kB/s | 19 kB 00:00 2024-05-08T21:55:39.012 INFO:teuthology.orchestra.run.smithi131.stdout:(22/146): python3-rados-18.2.2-1322.gd21e4f96.e 2.0 MB/s | 370 kB 00:00 2024-05-08T21:55:39.022 INFO:teuthology.orchestra.run.smithi132.stdout:(109/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 404 kB/s | 44 kB 00:00 2024-05-08T21:55:39.046 INFO:teuthology.orchestra.run.smithi131.stdout:(23/146): python3-rbd-18.2.2-1322.gd21e4f96.el8 2.7 MB/s | 365 kB 00:00 2024-05-08T21:55:39.048 INFO:teuthology.orchestra.run.smithi132.stdout:(110/146): python3-pycparser-2.14-14.el8.noarch 352 kB/s | 109 kB 00:00 2024-05-08T21:55:39.097 INFO:teuthology.orchestra.run.smithi157.stdout:(86/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.9 MB/s | 323 kB 00:00 2024-05-08T21:55:39.114 INFO:teuthology.orchestra.run.smithi131.stdout:(24/146): python3-rgw-18.2.2-1322.gd21e4f96.el8 1.0 MB/s | 107 kB 00:00 2024-05-08T21:55:39.146 INFO:teuthology.orchestra.run.smithi131.stdout:(25/146): rbd-fuse-18.2.2-1322.gd21e4f96.el8.x8 951 kB/s | 95 kB 00:00 2024-05-08T21:55:39.213 INFO:teuthology.orchestra.run.smithi131.stdout:(26/146): rbd-nbd-18.2.2-1322.gd21e4f96.el8.x86 2.7 MB/s | 181 kB 00:00 2024-05-08T21:55:39.222 INFO:teuthology.orchestra.run.smithi157.stdout:(87/146): python3-numpy-1.14.3-10.el8.x86_64.rp 8.4 MB/s | 3.7 MB 00:00 2024-05-08T21:55:39.247 INFO:teuthology.orchestra.run.smithi157.stdout:(88/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.5 MB/s | 225 kB 00:00 2024-05-08T21:55:39.339 INFO:teuthology.orchestra.run.smithi131.stdout:(27/146): librgw2-18.2.2-1322.gd21e4f96.el8.x86 3.6 MB/s | 6.5 MB 00:01 2024-05-08T21:55:39.348 INFO:teuthology.orchestra.run.smithi157.stdout:(89/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.8 MB/s | 354 kB 00:00 2024-05-08T21:55:39.364 INFO:teuthology.orchestra.run.smithi131.stdout:(28/146): ceph-grafana-dashboards-18.2.2-1322.g 161 kB/s | 24 kB 00:00 2024-05-08T21:55:39.456 INFO:teuthology.orchestra.run.smithi131.stdout:(29/146): ceph-mgr-cephadm-18.2.2-1322.gd21e4f9 1.2 MB/s | 138 kB 00:00 2024-05-08T21:55:39.464 INFO:teuthology.orchestra.run.smithi157.stdout:(90/146): openblas-threads-0.3.15-6.el8.x86_64. 24 MB/s | 5.2 MB 00:00 2024-05-08T21:55:39.507 INFO:teuthology.orchestra.run.smithi132.stdout:(111/146): pcre2-devel-10.32-2.el8.x86_64.rpm 1.1 MB/s | 605 kB 00:00 2024-05-08T21:55:39.524 INFO:teuthology.orchestra.run.smithi157.stdout:(91/146): python3-mako-1.0.6-14.el8.noarch.rpm 891 kB/s | 157 kB 00:00 2024-05-08T21:55:39.590 INFO:teuthology.orchestra.run.smithi132.stdout:(112/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 406 kB/s | 220 kB 00:00 2024-05-08T21:55:39.632 INFO:teuthology.orchestra.run.smithi132.stdout:(113/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 375 kB/s | 229 kB 00:00 2024-05-08T21:55:39.658 INFO:teuthology.orchestra.run.smithi132.stdout:(114/146): libselinux-devel-2.9-5.el8.x86_64.rp 1.3 MB/s | 200 kB 00:00 2024-05-08T21:55:39.665 INFO:teuthology.orchestra.run.smithi157.stdout:(92/146): openblas-0.3.15-6.el8.x86_64.rpm 25 MB/s | 5.0 MB 00:00 2024-05-08T21:55:39.715 INFO:teuthology.orchestra.run.smithi157.stdout:(93/146): automake-1.16.1-8.el8.noarch.rpm 3.7 MB/s | 714 kB 00:00 2024-05-08T21:55:39.750 INFO:teuthology.orchestra.run.smithi131.stdout:(30/146): ceph-mgr-dashboard-18.2.2-1322.gd21e4 9.1 MB/s | 3.5 MB 00:00 2024-05-08T21:55:39.757 INFO:teuthology.orchestra.run.smithi157.stdout:(94/146): libpq-13.11-1.el8.x86_64.rpm 2.1 MB/s | 198 kB 00:00 2024-05-08T21:55:39.783 INFO:teuthology.orchestra.run.smithi132.stdout:(115/146): keyutils-libs-devel-1.5.10-9.el8.x86 250 kB/s | 48 kB 00:00 2024-05-08T21:55:39.831 INFO:teuthology.orchestra.run.smithi131.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1322.gd2 3.0 MB/s | 245 kB 00:00 2024-05-08T21:55:39.841 INFO:teuthology.orchestra.run.smithi132.stdout:(116/146): libsepol-devel-2.9-3.el8.x86_64.rpm 417 kB/s | 87 kB 00:00 2024-05-08T21:55:39.878 INFO:teuthology.orchestra.run.smithi157.stdout:(95/146): jq-1.6-7.el8.x86_64.rpm 1.2 MB/s | 202 kB 00:00 2024-05-08T21:55:39.898 INFO:teuthology.orchestra.run.smithi131.stdout:(32/146): ceph-mgr-rook-18.2.2-1322.gd21e4f96.e 742 kB/s | 49 kB 00:00 2024-05-08T21:55:39.900 INFO:teuthology.orchestra.run.smithi132.stdout:(117/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 238 kB/s | 58 kB 00:00 2024-05-08T21:55:39.908 INFO:teuthology.orchestra.run.smithi157.stdout:(96/146): python3-pytz-2017.2-11.el8.noarch.rpm 361 kB/s | 54 kB 00:00 2024-05-08T21:55:39.925 INFO:teuthology.orchestra.run.smithi132.stdout:(118/146): libcom_err-devel-1.45.6-4.el8.x86_64 271 kB/s | 39 kB 00:00 2024-05-08T21:55:39.957 INFO:teuthology.orchestra.run.smithi131.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1322.gd 279 kB/s | 16 kB 00:00 2024-05-08T21:55:40.043 INFO:teuthology.orchestra.run.smithi131.stdout:(34/146): ceph-volume-18.2.2-1322.gd21e4f96.el8 3.0 MB/s | 263 kB 00:00 2024-05-08T21:55:40.067 INFO:teuthology.orchestra.run.smithi132.stdout:(119/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.2 MB/s | 172 kB 00:00 2024-05-08T21:55:40.092 INFO:teuthology.orchestra.run.smithi157.stdout:(97/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-05-08T21:55:40.092 INFO:teuthology.orchestra.run.smithi132.stdout:(120/146): fuse-2.9.7-17.el8.x86_64.rpm 331 kB/s | 83 kB 00:00 2024-05-08T21:55:40.118 INFO:teuthology.orchestra.run.smithi132.stdout:(121/146): libgfortran-8.5.0-20.el8.x86_64.rpm 2.9 MB/s | 645 kB 00:00 2024-05-08T21:55:40.126 INFO:teuthology.orchestra.run.smithi131.stdout:(35/146): rbd-mirror-18.2.2-1322.gd21e4f96.el8. 3.4 MB/s | 3.5 MB 00:01 2024-05-08T21:55:40.151 INFO:teuthology.orchestra.run.smithi131.stdout:(36/146): cephadm-18.2.2-1322.gd21e4f96.el8.noa 1.5 MB/s | 170 kB 00:00 2024-05-08T21:55:40.177 INFO:teuthology.orchestra.run.smithi132.stdout:(122/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.1 MB/s | 238 kB 00:00 2024-05-08T21:55:40.193 INFO:teuthology.orchestra.run.smithi131.stdout:(37/146): cephfs-top-18.2.2-1322.gd21e4f96.el8. 270 kB/s | 18 kB 00:00 2024-05-08T21:55:40.227 INFO:teuthology.orchestra.run.smithi132.stdout:(123/146): elfutils-libelf-devel-0.189-3.el8.x8 568 kB/s | 62 kB 00:00 2024-05-08T21:55:40.252 INFO:teuthology.orchestra.run.smithi132.stdout:(124/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 456 kB/s | 73 kB 00:00 2024-05-08T21:55:40.292 INFO:teuthology.orchestra.run.smithi157.stdout:(98/146): python3-scipy-1.0.0-21.module+el8.9.0 33 MB/s | 14 MB 00:00 2024-05-08T21:55:40.342 INFO:teuthology.orchestra.run.smithi157.stdout:(99/146): postgresql-10.23-4.module+el8.9.0+213 6.0 MB/s | 1.5 MB 00:00 2024-05-08T21:55:40.361 INFO:teuthology.orchestra.run.smithi132.stdout:(125/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 3.0 MB/s | 562 kB 00:00 2024-05-08T21:55:40.386 INFO:teuthology.orchestra.run.smithi132.stdout:(126/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 352 kB/s | 47 kB 00:00 2024-05-08T21:55:40.444 INFO:teuthology.orchestra.run.smithi132.stdout:(127/146): openssl-devel-1.1.1k-12.el8_9.x86_64 11 MB/s | 2.3 MB 00:00 2024-05-08T21:55:40.459 INFO:teuthology.orchestra.run.smithi157.stdout:(100/146): postgresql-contrib-10.23-4.module+el 4.8 MB/s | 812 kB 00:00 2024-05-08T21:55:40.520 INFO:teuthology.orchestra.run.smithi132.stdout:(128/146): python3-cryptography-3.2.1-7.el8_9.x 3.4 MB/s | 559 kB 00:00 2024-05-08T21:55:40.551 INFO:teuthology.orchestra.run.smithi157.stdout:(101/146): postgresql-server-10.23-4.module+el8 24 MB/s | 5.1 MB 00:00 2024-05-08T21:55:40.562 INFO:teuthology.orchestra.run.smithi132.stdout:(129/146): NetworkManager-initscripts-updown-1. 822 kB/s | 144 kB 00:00 2024-05-08T21:55:40.637 INFO:teuthology.orchestra.run.smithi132.stdout:(130/146): libuuid-devel-2.32.1-44.el8_9.1.x86_ 519 kB/s | 100 kB 00:00 2024-05-08T21:55:40.643 INFO:teuthology.orchestra.run.smithi157.stdout:(102/146): python3-packaging-16.8-9.el8.noarch. 263 kB/s | 48 kB 00:00 2024-05-08T21:55:40.652 INFO:teuthology.orchestra.run.smithi131.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 128 kB/s | 59 kB 00:00 2024-05-08T21:55:40.796 INFO:teuthology.orchestra.run.smithi132.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.4 MB/s | 232 kB 00:00 2024-05-08T21:55:40.996 INFO:teuthology.orchestra.run.smithi132.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.5 MB/s | 303 kB 00:00 2024-05-08T21:55:41.103 INFO:teuthology.orchestra.run.smithi131.stdout:(39/146): gperftools-libs-2.7-9.el8.x86_64.rpm 322 kB/s | 306 kB 00:00 2024-05-08T21:55:41.118 INFO:teuthology.orchestra.run.smithi157.stdout:(103/146): ceph-test-18.2.2-1322.gd21e4f96.el8. 4.1 MB/s | 47 MB 00:11 2024-05-08T21:55:41.128 INFO:teuthology.orchestra.run.smithi131.stdout:(40/146): libunwind-1.3.1-3.el8.x86_64.rpm 158 kB/s | 75 kB 00:00 2024-05-08T21:55:41.143 INFO:teuthology.orchestra.run.smithi157.stdout:(104/146): libverto-devel-0.3.0-5.el8.x86_64.rp 30 kB/s | 18 kB 00:00 2024-05-08T21:55:41.169 INFO:teuthology.orchestra.run.smithi157.stdout:(105/146): python3-jwt-1.6.1-2.el8.noarch.rpm 81 kB/s | 43 kB 00:00 2024-05-08T21:55:41.204 INFO:teuthology.orchestra.run.smithi132.stdout:(133/146): iproute-6.2.0-5.el8_9.x86_64.rpm 4.0 MB/s | 854 kB 00:00 2024-05-08T21:55:41.227 INFO:teuthology.orchestra.run.smithi157.stdout:(106/146): python3-pycparser-2.14-14.el8.noarch 1.4 MB/s | 109 kB 00:00 2024-05-08T21:55:41.228 INFO:teuthology.orchestra.run.smithi131.stdout:(41/146): ceph-mgr-diskprediction-local-18.2.2- 4.2 MB/s | 7.4 MB 00:01 2024-05-08T21:55:41.253 INFO:teuthology.orchestra.run.smithi131.stdout:(42/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 348 kB/s | 44 kB 00:00 2024-05-08T21:55:41.269 INFO:teuthology.orchestra.run.smithi157.stdout:(107/146): python3-oauthlib-2.1.0-1.el8.noarch. 1.0 MB/s | 155 kB 00:00 2024-05-08T21:55:41.303 INFO:teuthology.orchestra.run.smithi157.stdout:(108/146): python3-requests-oauthlib-1.0.0-1.el 318 kB/s | 43 kB 00:00 2024-05-08T21:55:41.320 INFO:teuthology.orchestra.run.smithi131.stdout:(43/146): python3-asyncssh-2.7.0-2.el8.noarch.r 2.1 MB/s | 476 kB 00:00 2024-05-08T21:55:41.346 INFO:teuthology.orchestra.run.smithi132.stdout:(134/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.9 MB/s | 711 kB 00:00 2024-05-08T21:55:41.353 INFO:teuthology.orchestra.run.smithi157.stdout:(109/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 524 kB/s | 44 kB 00:00 2024-05-08T21:55:41.370 INFO:teuthology.orchestra.run.smithi157.stdout:(110/146): libaio-devel-0.3.112-1.el8.x86_64.rp 131 kB/s | 19 kB 00:00 2024-05-08T21:55:41.370 INFO:teuthology.orchestra.run.smithi131.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 282 kB/s | 33 kB 00:00 2024-05-08T21:55:41.404 INFO:teuthology.orchestra.run.smithi157.stdout:(111/146): pcre2-devel-10.32-2.el8.x86_64.rpm 5.9 MB/s | 605 kB 00:00 2024-05-08T21:55:41.405 INFO:teuthology.orchestra.run.smithi132.stdout:(135/146): librados2-18.2.2-1322.gd21e4f96.el8. 4.2 MB/s | 3.7 MB 00:00 2024-05-08T21:55:41.437 INFO:teuthology.orchestra.run.smithi131.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 145 kB/s | 16 kB 00:00 2024-05-08T21:55:41.464 INFO:teuthology.orchestra.run.smithi132.stdout:(136/146): librbd1-18.2.2-1322.gd21e4f96.el8.x8 4.0 MB/s | 3.6 MB 00:00 2024-05-08T21:55:41.479 INFO:teuthology.orchestra.run.smithi157.stdout:(112/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.0 MB/s | 220 kB 00:00 2024-05-08T21:55:41.496 INFO:teuthology.orchestra.run.smithi157.stdout:(113/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.6 MB/s | 229 kB 00:00 2024-05-08T21:55:41.530 INFO:teuthology.orchestra.run.smithi157.stdout:(114/146): libselinux-devel-2.9-5.el8.x86_64.rp 1.6 MB/s | 200 kB 00:00 2024-05-08T21:55:41.547 INFO:teuthology.orchestra.run.smithi132.stdout:(137/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 7.3 MB/s | 1.5 MB 00:00 2024-05-08T21:55:41.589 INFO:teuthology.orchestra.run.smithi131.stdout:(46/146): python3-cherrypy-18.4.0-1.el8.noarch. 2.5 MB/s | 384 kB 00:00 2024-05-08T21:55:41.590 INFO:teuthology.orchestra.run.smithi132.stdout:(138/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 1.8 MB/s | 333 kB 00:00 2024-05-08T21:55:41.613 INFO:teuthology.orchestra.run.smithi131.stdout:(47/146): python3-cheroot-8.5.2-1.el8.noarch.rp 713 kB/s | 173 kB 00:00 2024-05-08T21:55:41.646 INFO:teuthology.orchestra.run.smithi157.stdout:(115/146): libsepol-devel-2.9-3.el8.x86_64.rpm 579 kB/s | 87 kB 00:00 2024-05-08T21:55:41.673 INFO:teuthology.orchestra.run.smithi132.stdout:(139/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 5.1 MB/s | 1.1 MB 00:00 2024-05-08T21:55:41.696 INFO:teuthology.orchestra.run.smithi131.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 750 kB/s | 81 kB 00:00 2024-05-08T21:55:41.698 INFO:teuthology.orchestra.run.smithi132.stdout:(140/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.4 MB/s | 221 kB 00:00 2024-05-08T21:55:41.732 INFO:teuthology.orchestra.run.smithi132.stdout:(141/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 1.7 MB/s | 254 kB 00:00 2024-05-08T21:55:41.747 INFO:teuthology.orchestra.run.smithi131.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 83 kB/s | 11 kB 00:00 2024-05-08T21:55:41.801 INFO:teuthology.orchestra.run.smithi157.stdout:(116/146): keyutils-libs-devel-1.5.10-9.el8.x86 150 kB/s | 48 kB 00:00 2024-05-08T21:55:41.805 INFO:teuthology.orchestra.run.smithi131.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2024-05-08T21:55:41.830 INFO:teuthology.orchestra.run.smithi157.stdout:(117/146): libcom_err-devel-1.45.6-4.el8.x86_64 210 kB/s | 39 kB 00:00 2024-05-08T21:55:41.831 INFO:teuthology.orchestra.run.smithi131.stdout:(51/146): python3-beautifulsoup4-4.6.3-2.el8.1. 308 kB/s | 185 kB 00:00 2024-05-08T21:55:41.849 INFO:teuthology.orchestra.run.smithi132.stdout:(142/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.3 MB/s | 237 kB 00:00 2024-05-08T21:55:41.855 INFO:teuthology.orchestra.run.smithi157.stdout:(118/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 177 kB/s | 58 kB 00:00 2024-05-08T21:55:41.874 INFO:teuthology.orchestra.run.smithi132.stdout:(143/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 699 kB/s | 99 kB 00:00 2024-05-08T21:55:41.900 INFO:teuthology.orchestra.run.smithi132.stdout:(144/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 894 kB/s | 180 kB 00:00 2024-05-08T21:55:41.914 INFO:teuthology.orchestra.run.smithi131.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 449 kB/s | 49 kB 00:00 2024-05-08T21:55:41.931 INFO:teuthology.orchestra.run.smithi157.stdout:(119/146): fuse-2.9.7-17.el8.x86_64.rpm 639 kB/s | 83 kB 00:00 2024-05-08T21:55:41.956 INFO:teuthology.orchestra.run.smithi131.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 469 kB/s | 59 kB 00:00 2024-05-08T21:55:41.973 INFO:teuthology.orchestra.run.smithi157.stdout:(120/146): libgfortran-8.5.0-20.el8.x86_64.rpm 4.4 MB/s | 645 kB 00:00 2024-05-08T21:55:41.998 INFO:teuthology.orchestra.run.smithi157.stdout:(121/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.2 MB/s | 172 kB 00:00 2024-05-08T21:55:42.033 INFO:teuthology.orchestra.run.smithi132.stdout:(145/146): util-linux-user-2.32.1-44.el8_9.1.x8 648 kB/s | 103 kB 00:00 2024-05-08T21:55:42.059 INFO:teuthology.orchestra.run.smithi131.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 416 kB/s | 60 kB 00:00 2024-05-08T21:55:42.107 INFO:teuthology.orchestra.run.smithi131.stdout:(55/146): python3-kubernetes-11.0.0-6.el8.noarc 3.2 MB/s | 1.2 MB 00:00 2024-05-08T21:55:42.116 INFO:teuthology.orchestra.run.smithi132.stdout:(146/146): util-linux-2.32.1-44.el8_9.1.x86_64. 9.3 MB/s | 2.5 MB 00:00 2024-05-08T21:55:42.119 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-05-08T21:55:42.119 INFO:teuthology.orchestra.run.smithi132.stdout:Total 12 MB/s | 215 MB 00:17 2024-05-08T21:55:42.166 INFO:teuthology.orchestra.run.smithi131.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 148 kB/s | 16 kB 00:00 2024-05-08T21:55:42.173 INFO:teuthology.orchestra.run.smithi157.stdout:(122/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 983 kB/s | 238 kB 00:00 2024-05-08T21:55:42.199 INFO:teuthology.orchestra.run.smithi157.stdout:(123/146): elfutils-libelf-devel-0.189-3.el8.x8 308 kB/s | 62 kB 00:00 2024-05-08T21:55:42.199 INFO:teuthology.orchestra.run.smithi131.stdout:(57/146): python3-pecan-1.3.2-9.el8.noarch.rpm 1.1 MB/s | 283 kB 00:00 2024-05-08T21:55:42.224 INFO:teuthology.orchestra.run.smithi131.stdout:(58/146): python3-repoze-lru-0.7-6.el8.noarch.r 287 kB/s | 34 kB 00:00 2024-05-08T21:55:42.241 INFO:teuthology.orchestra.run.smithi157.stdout:(124/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 272 kB/s | 73 kB 00:00 2024-05-08T21:55:42.291 INFO:teuthology.orchestra.run.smithi157.stdout:(125/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 4.7 MB/s | 562 kB 00:00 2024-05-08T21:55:42.291 INFO:teuthology.orchestra.run.smithi131.stdout:(59/146): python3-routes-2.4.1-12.el8.noarch.rp 1.5 MB/s | 196 kB 00:00 2024-05-08T21:55:42.325 INFO:teuthology.orchestra.run.smithi131.stdout:(60/146): python3-rsa-4.9-2.el8.noarch.rpm 512 kB/s | 64 kB 00:00 2024-05-08T21:55:42.382 INFO:teuthology.orchestra.run.smithi131.stdout:(61/146): python3-simplegeneric-0.8.1-17.el8.no 119 kB/s | 19 kB 00:00 2024-05-08T21:55:42.417 INFO:teuthology.orchestra.run.smithi131.stdout:(62/146): python3-singledispatch-3.4.0.3-18.el8 191 kB/s | 24 kB 00:00 2024-05-08T21:55:42.443 INFO:teuthology.orchestra.run.smithi131.stdout:(63/146): python3-tempora-1.14.1-5.el8.noarch.r 249 kB/s | 29 kB 00:00 2024-05-08T21:55:42.449 INFO:teuthology.orchestra.run.smithi157.stdout:(126/146): python3-cryptography-3.2.1-7.el8_9.x 3.5 MB/s | 559 kB 00:00 2024-05-08T21:55:42.493 INFO:teuthology.orchestra.run.smithi131.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 248 kB/s | 27 kB 00:00 2024-05-08T21:55:42.508 INFO:teuthology.orchestra.run.smithi157.stdout:(127/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 176 kB/s | 47 kB 00:00 2024-05-08T21:55:42.543 INFO:teuthology.orchestra.run.smithi131.stdout:(65/146): python3-waitress-1.4.3-1.el8.noarch.r 2.0 MB/s | 255 kB 00:00 2024-05-08T21:55:42.558 INFO:teuthology.orchestra.run.smithi157.stdout:(128/146): openssl-devel-1.1.1k-12.el8_9.x86_64 6.5 MB/s | 2.3 MB 00:00 2024-05-08T21:55:42.585 INFO:teuthology.orchestra.run.smithi131.stdout:(66/146): python3-webob-1.8.5-1.el8.1.noarch.rp 1.7 MB/s | 251 kB 00:00 2024-05-08T21:55:42.592 INFO:teuthology.orchestra.run.smithi157.stdout:(129/146): libuuid-devel-2.32.1-44.el8_9.1.x86_ 1.2 MB/s | 100 kB 00:00 2024-05-08T21:55:42.610 INFO:teuthology.orchestra.run.smithi131.stdout:(67/146): python3-websocket-client-0.56.0-5.el8 522 kB/s | 61 kB 00:00 2024-05-08T21:55:42.617 INFO:teuthology.orchestra.run.smithi157.stdout:(130/146): NetworkManager-initscripts-updown-1. 862 kB/s | 144 kB 00:00 2024-05-08T21:55:42.652 INFO:teuthology.orchestra.run.smithi131.stdout:(68/146): python3-webtest-2.0.33-1.el8.noarch.r 785 kB/s | 85 kB 00:00 2024-05-08T21:55:42.719 INFO:teuthology.orchestra.run.smithi131.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.0 MB/s | 70 kB 00:00 2024-05-08T21:55:42.745 INFO:teuthology.orchestra.run.smithi131.stdout:(70/146): python3-zc-lockfile-2.0-2.el8.noarch. 144 kB/s | 23 kB 00:00 2024-05-08T21:55:42.759 INFO:teuthology.orchestra.run.smithi157.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.6 MB/s | 232 kB 00:00 2024-05-08T21:55:42.853 INFO:teuthology.orchestra.run.smithi131.stdout:(71/146): thrift-0.13.0-2.el8.x86_64.rpm 7.2 MB/s | 1.7 MB 00:00 2024-05-08T21:55:42.879 INFO:teuthology.orchestra.run.smithi131.stdout:(72/146): python3-prettytable-0.7.2-14.el8.noar 330 kB/s | 44 kB 00:00 2024-05-08T21:55:42.904 INFO:teuthology.orchestra.run.smithi131.stdout:(73/146): python3-jsonpointer-1.10-11.el8.noarc 109 kB/s | 20 kB 00:00 2024-05-08T21:55:42.926 INFO:teuthology.orchestra.run.smithi157.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.8 MB/s | 303 kB 00:00 2024-05-08T21:55:43.029 INFO:teuthology.orchestra.run.smithi131.stdout:(74/146): python3-jsonpatch-1.21-2.el8.noarch.r 177 kB/s | 27 kB 00:00 2024-05-08T21:55:43.051 INFO:teuthology.orchestra.run.smithi157.stdout:(133/146): librbd1-18.2.2-1322.gd21e4f96.el8.x8 7.8 MB/s | 3.6 MB 00:00 2024-05-08T21:55:43.054 INFO:teuthology.orchestra.run.smithi131.stdout:(75/146): python3-pyasn1-modules-0.3.7-6.el8.no 548 kB/s | 110 kB 00:00 2024-05-08T21:55:43.080 INFO:teuthology.orchestra.run.smithi131.stdout:(76/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 744 kB/s | 126 kB 00:00 2024-05-08T21:55:43.110 INFO:teuthology.orchestra.run.smithi157.stdout:(134/146): iproute-6.2.0-5.el8_9.x86_64.rpm 4.5 MB/s | 854 kB 00:00 2024-05-08T21:55:43.143 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-05-08T21:55:43.180 INFO:teuthology.orchestra.run.smithi131.stdout:(77/146): python3-werkzeug-0.12.2-4.el8.noarch. 3.0 MB/s | 457 kB 00:00 2024-05-08T21:55:43.210 INFO:teuthology.orchestra.run.smithi157.stdout:(135/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.4 MB/s | 711 kB 00:00 2024-05-08T21:55:43.230 INFO:teuthology.orchestra.run.smithi131.stdout:(78/146): flex-2.6.1-9.el8.x86_64.rpm 2.1 MB/s | 320 kB 00:00 2024-05-08T21:55:43.261 INFO:teuthology.orchestra.run.smithi157.stdout:(136/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 9.8 MB/s | 1.5 MB 00:00 2024-05-08T21:55:43.272 INFO:teuthology.orchestra.run.smithi131.stdout:(79/146): bison-3.0.4-10.el8.x86_64.rpm 3.1 MB/s | 688 kB 00:00 2024-05-08T21:55:43.327 INFO:teuthology.orchestra.run.smithi157.stdout:(137/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.8 MB/s | 333 kB 00:00 2024-05-08T21:55:43.339 INFO:teuthology.orchestra.run.smithi131.stdout:(80/146): python3-markupsafe-0.23-19.el8.x86_64 358 kB/s | 39 kB 00:00 2024-05-08T21:55:43.389 INFO:teuthology.orchestra.run.smithi131.stdout:(81/146): libtool-2.4.6-25.el8.x86_64.rpm 3.3 MB/s | 709 kB 00:00 2024-05-08T21:55:43.394 INFO:teuthology.orchestra.run.smithi157.stdout:(138/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 8.0 MB/s | 1.1 MB 00:00 2024-05-08T21:55:43.423 INFO:teuthology.orchestra.run.smithi131.stdout:(82/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.7 MB/s | 259 kB 00:00 2024-05-08T21:55:43.465 INFO:teuthology.orchestra.run.smithi131.stdout:(83/146): uuid-1.6.2-43.el8.x86_64.rpm 509 kB/s | 64 kB 00:00 2024-05-08T21:55:43.495 INFO:teuthology.orchestra.run.smithi157.stdout:(139/146): librados2-18.2.2-1322.gd21e4f96.el8. 4.0 MB/s | 3.7 MB 00:00 2024-05-08T21:55:43.531 INFO:teuthology.orchestra.run.smithi131.stdout:(84/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 729 kB/s | 103 kB 00:00 2024-05-08T21:55:43.539 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-05-08T21:55:43.539 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-05-08T21:55:43.553 INFO:teuthology.orchestra.run.smithi157.stdout:(140/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 978 kB/s | 221 kB 00:00 2024-05-08T21:55:43.579 INFO:teuthology.orchestra.run.smithi157.stdout:(141/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 1.3 MB/s | 254 kB 00:00 2024-05-08T21:55:43.644 INFO:teuthology.orchestra.run.smithi131.stdout:(85/146): python3-jinja2-2.10.1-3.el8.noarch.rp 2.9 MB/s | 538 kB 00:00 2024-05-08T21:55:43.670 INFO:teuthology.orchestra.run.smithi157.stdout:(142/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.3 MB/s | 237 kB 00:00 2024-05-08T21:55:43.696 INFO:teuthology.orchestra.run.smithi157.stdout:(143/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 848 kB/s | 99 kB 00:00 2024-05-08T21:55:43.721 INFO:teuthology.orchestra.run.smithi157.stdout:(144/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.0 MB/s | 180 kB 00:00 2024-05-08T21:55:43.791 INFO:teuthology.orchestra.run.smithi131.stdout:(86/146): python3-babel-2.5.1-7.el8.noarch.rpm 13 MB/s | 4.8 MB 00:00 2024-05-08T21:55:43.866 INFO:teuthology.orchestra.run.smithi131.stdout:(87/146): python3-numpy-1.14.3-10.el8.x86_64.rp 11 MB/s | 3.7 MB 00:00 2024-05-08T21:55:43.891 INFO:teuthology.orchestra.run.smithi131.stdout:(88/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.3 MB/s | 323 kB 00:00 2024-05-08T21:55:43.904 INFO:teuthology.orchestra.run.smithi157.stdout:(145/146): util-linux-2.32.1-44.el8_9.1.x86_64. 11 MB/s | 2.5 MB 00:00 2024-05-08T21:55:43.930 INFO:teuthology.orchestra.run.smithi157.stdout:(146/146): util-linux-user-2.32.1-44.el8_9.1.x8 440 kB/s | 103 kB 00:00 2024-05-08T21:55:43.934 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-05-08T21:55:43.935 INFO:teuthology.orchestra.run.smithi157.stdout:Total 11 MB/s | 215 MB 00:18 2024-05-08T21:55:43.991 INFO:teuthology.orchestra.run.smithi131.stdout:(89/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.8 MB/s | 354 kB 00:00 2024-05-08T21:55:44.016 INFO:teuthology.orchestra.run.smithi131.stdout:(90/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 998 kB/s | 225 kB 00:00 2024-05-08T21:55:44.100 INFO:teuthology.orchestra.run.smithi131.stdout:(91/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.4 MB/s | 157 kB 00:00 2024-05-08T21:55:44.167 INFO:teuthology.orchestra.run.smithi131.stdout:(92/146): openblas-threads-0.3.15-6.el8.x86_64. 19 MB/s | 5.2 MB 00:00 2024-05-08T21:55:44.301 INFO:teuthology.orchestra.run.smithi131.stdout:(93/146): automake-1.16.1-8.el8.noarch.rpm 3.5 MB/s | 714 kB 00:00 2024-05-08T21:55:44.376 INFO:teuthology.orchestra.run.smithi131.stdout:(94/146): openblas-0.3.15-6.el8.x86_64.rpm 14 MB/s | 5.0 MB 00:00 2024-05-08T21:55:44.401 INFO:teuthology.orchestra.run.smithi131.stdout:(95/146): libpq-13.11-1.el8.x86_64.rpm 846 kB/s | 198 kB 00:00 2024-05-08T21:55:44.426 INFO:teuthology.orchestra.run.smithi131.stdout:(96/146): jq-1.6-7.el8.x86_64.rpm 1.6 MB/s | 202 kB 00:00 2024-05-08T21:55:44.494 INFO:teuthology.orchestra.run.smithi131.stdout:(97/146): python3-pytz-2017.2-11.el8.noarch.rpm 463 kB/s | 54 kB 00:00 2024-05-08T21:55:44.535 INFO:teuthology.orchestra.run.smithi131.stdout:(98/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-05-08T21:55:44.694 INFO:teuthology.orchestra.run.smithi131.stdout:(99/146): postgresql-10.23-4.module+el8.9.0+213 7.5 MB/s | 1.5 MB 00:00 2024-05-08T21:55:44.744 INFO:teuthology.orchestra.run.smithi131.stdout:(100/146): postgresql-contrib-10.23-4.module+el 3.8 MB/s | 812 kB 00:00 2024-05-08T21:55:44.771 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-05-08T21:55:44.782 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-05-08T21:55:44.898 INFO:teuthology.orchestra.run.smithi131.stdout:(101/146): libverto-devel-0.3.0-5.el8.x86_64.rp 117 kB/s | 18 kB 00:00 2024-05-08T21:55:44.964 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-05-08T21:55:45.062 INFO:teuthology.orchestra.run.smithi131.stdout:(102/146): python3-scipy-1.0.0-21.module+el8.9. 21 MB/s | 14 MB 00:00 2024-05-08T21:55:45.087 INFO:teuthology.orchestra.run.smithi131.stdout:(103/146): python3-jwt-1.6.1-2.el8.noarch.rpm 228 kB/s | 43 kB 00:00 2024-05-08T21:55:45.219 INFO:teuthology.orchestra.run.smithi131.stdout:(104/146): postgresql-server-10.23-4.module+el8 9.6 MB/s | 5.1 MB 00:00 2024-05-08T21:55:45.254 INFO:teuthology.orchestra.run.smithi131.stdout:(105/146): python3-pycparser-2.14-14.el8.noarch 653 kB/s | 109 kB 00:00 2024-05-08T21:55:45.279 INFO:teuthology.orchestra.run.smithi131.stdout:(106/146): python3-oauthlib-2.1.0-1.el8.noarch. 713 kB/s | 155 kB 00:00 2024-05-08T21:55:45.330 INFO:teuthology.orchestra.run.smithi131.stdout:(107/146): python3-requests-oauthlib-1.0.0-1.el 388 kB/s | 43 kB 00:00 2024-05-08T21:55:45.361 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-05-08T21:55:45.361 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-05-08T21:55:45.397 INFO:teuthology.orchestra.run.smithi131.stdout:(108/146): libaio-devel-0.3.112-1.el8.x86_64.rp 131 kB/s | 19 kB 00:00 2024-05-08T21:55:45.447 INFO:teuthology.orchestra.run.smithi131.stdout:(109/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 262 kB/s | 44 kB 00:00 2024-05-08T21:55:45.480 INFO:teuthology.orchestra.run.smithi131.stdout:(110/146): pcre2-devel-10.32-2.el8.x86_64.rpm 3.9 MB/s | 605 kB 00:00 2024-05-08T21:55:45.564 INFO:teuthology.orchestra.run.smithi131.stdout:(111/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.3 MB/s | 229 kB 00:00 2024-05-08T21:55:45.597 INFO:teuthology.orchestra.run.smithi131.stdout:(112/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2024-05-08T21:55:45.623 INFO:teuthology.orchestra.run.smithi131.stdout:(113/146): libselinux-devel-2.9-5.el8.x86_64.rp 1.4 MB/s | 200 kB 00:00 2024-05-08T21:55:45.690 INFO:teuthology.orchestra.run.smithi131.stdout:(114/146): keyutils-libs-devel-1.5.10-9.el8.x86 384 kB/s | 48 kB 00:00 2024-05-08T21:55:45.757 INFO:teuthology.orchestra.run.smithi131.stdout:(115/146): libsepol-devel-2.9-3.el8.x86_64.rpm 548 kB/s | 87 kB 00:00 2024-05-08T21:55:45.790 INFO:teuthology.orchestra.run.smithi131.stdout:(116/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 346 kB/s | 58 kB 00:00 2024-05-08T21:55:45.816 INFO:teuthology.orchestra.run.smithi131.stdout:(117/146): libcom_err-devel-1.45.6-4.el8.x86_64 307 kB/s | 39 kB 00:00 2024-05-08T21:55:45.941 INFO:teuthology.orchestra.run.smithi131.stdout:(118/146): fuse-2.9.7-17.el8.x86_64.rpm 451 kB/s | 83 kB 00:00 2024-05-08T21:55:46.008 INFO:teuthology.orchestra.run.smithi131.stdout:(119/146): libgfortran-8.5.0-20.el8.x86_64.rpm 2.9 MB/s | 645 kB 00:00 2024-05-08T21:55:46.041 INFO:teuthology.orchestra.run.smithi131.stdout:(120/146): libquadmath-8.5.0-20.el8.x86_64.rpm 765 kB/s | 172 kB 00:00 2024-05-08T21:55:46.067 INFO:teuthology.orchestra.run.smithi131.stdout:(121/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 1.2 MB/s | 73 kB 00:00 2024-05-08T21:55:46.092 INFO:teuthology.orchestra.run.smithi131.stdout:(122/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.5 MB/s | 238 kB 00:00 2024-05-08T21:55:46.200 INFO:teuthology.orchestra.run.smithi131.stdout:(123/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 4.1 MB/s | 562 kB 00:00 2024-05-08T21:55:46.234 INFO:teuthology.orchestra.run.smithi131.stdout:(124/146): elfutils-libelf-devel-0.189-3.el8.x8 320 kB/s | 62 kB 00:00 2024-05-08T21:55:46.310 INFO:teuthology.orchestra.run.smithi131.stdout:(125/146): openssl-devel-1.1.1k-12.el8_9.x86_64 11 MB/s | 2.3 MB 00:00 2024-05-08T21:55:46.335 INFO:teuthology.orchestra.run.smithi131.stdout:(126/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 352 kB/s | 47 kB 00:00 2024-05-08T21:55:46.377 INFO:teuthology.orchestra.run.smithi131.stdout:(127/146): python3-cryptography-3.2.1-7.el8_9.x 3.9 MB/s | 559 kB 00:00 2024-05-08T21:55:46.427 INFO:teuthology.orchestra.run.smithi131.stdout:(128/146): libuuid-devel-2.32.1-44.el8_9.1.x86_ 1.1 MB/s | 100 kB 00:00 2024-05-08T21:55:46.452 INFO:teuthology.orchestra.run.smithi131.stdout:(129/146): NetworkManager-initscripts-updown-1. 1.0 MB/s | 144 kB 00:00 2024-05-08T21:55:46.528 INFO:teuthology.orchestra.run.smithi131.stdout:(130/146): python3-packaging-16.8-9.el8.noarch. 320 kB/s | 48 kB 00:00 2024-05-08T21:55:46.576 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-05-08T21:55:46.586 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-05-08T21:55:46.653 INFO:teuthology.orchestra.run.smithi131.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.8 MB/s | 232 kB 00:00 2024-05-08T21:55:46.837 INFO:teuthology.orchestra.run.smithi131.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.6 MB/s | 303 kB 00:00 2024-05-08T21:55:46.903 INFO:teuthology.orchestra.run.smithi131.stdout:(133/146): librados2-18.2.2-1322.gd21e4f96.el8. 7.8 MB/s | 3.7 MB 00:00 2024-05-08T21:55:47.004 INFO:teuthology.orchestra.run.smithi131.stdout:(134/146): iproute-6.2.0-5.el8_9.x86_64.rpm 5.0 MB/s | 854 kB 00:00 2024-05-08T21:55:47.046 INFO:teuthology.orchestra.run.smithi131.stdout:(135/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.9 MB/s | 711 kB 00:00 2024-05-08T21:55:47.171 INFO:teuthology.orchestra.run.smithi131.stdout:(136/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 8.8 MB/s | 1.5 MB 00:00 2024-05-08T21:55:47.213 INFO:teuthology.orchestra.run.smithi131.stdout:(137/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.0 MB/s | 333 kB 00:00 2024-05-08T21:55:47.305 INFO:teuthology.orchestra.run.smithi131.stdout:(138/146): librbd1-18.2.2-1322.gd21e4f96.el8.x8 4.2 MB/s | 3.6 MB 00:00 2024-05-08T21:55:47.355 INFO:teuthology.orchestra.run.smithi131.stdout:(139/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 5.8 MB/s | 1.1 MB 00:00 2024-05-08T21:55:47.381 INFO:teuthology.orchestra.run.smithi131.stdout:(140/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.3 MB/s | 221 kB 00:00 2024-05-08T21:55:47.456 INFO:teuthology.orchestra.run.smithi131.stdout:(141/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 1.6 MB/s | 254 kB 00:00 2024-05-08T21:55:47.481 INFO:teuthology.orchestra.run.smithi131.stdout:(142/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.8 MB/s | 237 kB 00:00 2024-05-08T21:55:47.515 INFO:teuthology.orchestra.run.smithi131.stdout:(143/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.3 MB/s | 180 kB 00:00 2024-05-08T21:55:47.632 INFO:teuthology.orchestra.run.smithi131.stdout:(144/146): util-linux-2.32.1-44.el8_9.1.x86_64. 16 MB/s | 2.5 MB 00:00 2024-05-08T21:55:47.657 INFO:teuthology.orchestra.run.smithi131.stdout:(145/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 494 kB/s | 99 kB 00:00 2024-05-08T21:55:47.747 INFO:teuthology.orchestra.run.smithi131.stdout:(146/146): util-linux-user-2.32.1-44.el8_9.1.x8 443 kB/s | 103 kB 00:00 2024-05-08T21:55:47.750 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-05-08T21:55:47.750 INFO:teuthology.orchestra.run.smithi131.stdout:Total 12 MB/s | 215 MB 00:17 2024-05-08T21:55:49.937 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-05-08T21:55:50.381 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-05-08T21:55:50.381 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-05-08T21:55:50.656 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-05-08T21:55:50.790 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-08T21:55:51.115 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T21:55:51.332 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T21:55:51.534 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-08T21:55:51.730 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T21:55:51.801 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T21:55:52.013 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T21:55:52.047 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-05-08T21:55:52.060 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-05-08T21:55:52.087 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T21:55:52.345 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-08T21:55:52.355 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-05-08T21:55:52.490 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-08T21:55:52.530 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T21:55:52.638 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T21:55:52.735 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T21:55:52.833 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-05-08T21:55:52.952 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T21:55:53.047 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T21:55:53.115 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-08T21:55:53.269 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T21:55:53.372 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T21:55:53.440 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T21:55:53.544 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T21:55:53.600 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T21:55:53.630 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:55:53.859 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-08T21:55:54.011 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:55:54.069 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T21:55:54.134 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T21:55:54.188 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:55:54.311 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-05-08T21:55:54.313 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:55:54.487 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:55:54.527 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T21:55:54.591 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:55:54.916 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T21:55:54.932 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T21:55:55.161 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:55:55.229 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T21:55:55.316 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T21:55:55.554 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:55:55.561 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-08T21:55:55.744 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:55:55.841 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-08T21:55:55.900 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:55:55.999 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-08T21:55:56.091 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:55:56.212 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:55:56.393 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-08T21:55:56.544 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T21:55:56.658 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-08T21:55:56.796 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:55:56.827 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T21:55:56.894 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T21:55:56.901 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:55:57.092 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-08T21:55:57.096 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T21:55:57.173 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T21:55:57.371 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T21:55:57.381 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-08T21:55:57.508 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T21:55:57.670 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T21:55:57.743 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-08T21:55:57.976 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-05-08T21:55:58.197 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-08T21:55:58.248 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-08T21:55:58.304 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T21:55:58.462 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T21:55:58.489 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-08T21:55:58.512 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T21:55:58.669 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:55:58.745 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T21:55:58.772 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:55:58.981 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-08T21:55:58.985 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T21:55:59.052 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T21:55:59.108 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T21:55:59.210 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T21:55:59.267 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T21:55:59.312 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T21:55:59.368 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T21:55:59.372 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T21:55:59.527 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T21:55:59.575 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T21:55:59.666 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T21:55:59.919 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-08T21:55:59.988 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T21:56:00.126 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T21:56:00.186 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T21:56:00.206 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T21:56:00.277 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-05-08T21:56:00.376 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T21:56:00.426 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-05-08T21:56:00.427 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-05-08T21:56:00.617 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-05-08T21:56:00.633 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T21:56:00.927 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-05-08T21:56:00.985 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T21:56:01.129 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T21:56:01.160 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T21:56:01.239 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-05-08T21:56:01.383 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:56:01.509 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-08T21:56:01.710 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 31/162 2024-05-08T21:56:01.837 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:56:01.850 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T21:56:01.865 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 32/162 2024-05-08T21:56:02.058 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:56:02.116 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-05-08T21:56:02.145 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 33/162 2024-05-08T21:56:02.211 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:56:02.273 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-05-08T21:56:02.445 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:56:02.471 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-05-08T21:56:02.561 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:56:02.727 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-05-08T21:56:02.928 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T21:56:02.990 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-05-08T21:56:03.119 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-05-08T21:56:03.223 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T21:56:03.269 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-08T21:56:03.349 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T21:56:03.442 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-05-08T21:56:03.457 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 31/162 2024-05-08T21:56:03.595 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 32/162 2024-05-08T21:56:03.629 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-08T21:56:03.712 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-05-08T21:56:03.870 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 33/162 2024-05-08T21:56:03.907 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-08T21:56:03.918 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T21:56:04.097 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-08T21:56:04.568 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-08T21:56:04.816 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-05-08T21:56:04.821 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-08T21:56:05.008 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:56:05.132 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-05-08T21:56:05.140 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:56:05.324 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T21:56:05.423 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T21:56:05.448 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-05-08T21:56:05.607 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T21:56:05.650 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T21:56:05.662 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T21:56:05.778 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T21:56:05.986 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T21:56:06.013 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-05-08T21:56:06.272 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-08T21:56:06.505 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-08T21:56:06.669 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T21:56:06.716 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 41/162 2024-05-08T21:56:06.860 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 42/162 2024-05-08T21:56:06.882 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T21:56:07.086 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T21:56:07.167 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T21:56:07.398 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T21:56:07.403 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T21:56:07.478 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T21:56:07.556 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T21:56:07.734 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-08T21:56:07.750 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-05-08T21:56:07.791 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T21:56:07.980 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-08T21:56:08.003 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-08T21:56:08.194 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T21:56:08.219 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-08T21:56:08.431 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 41/162 2024-05-08T21:56:08.588 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T21:56:08.599 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 42/162 2024-05-08T21:56:08.646 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T21:56:08.834 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T21:56:08.874 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-05-08T21:56:08.895 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T21:56:08.986 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T21:56:09.002 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T21:56:09.066 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-05-08T21:56:09.133 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T21:56:09.173 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-05-08T21:56:09.206 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T21:56:09.239 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-05-08T21:56:09.423 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-05-08T21:56:09.463 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-08T21:56:09.533 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-05-08T21:56:09.640 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-05-08T21:56:09.686 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-08T21:56:09.805 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-05-08T21:56:09.866 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-05-08T21:56:09.908 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T21:56:10.023 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-05-08T21:56:10.179 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-08T21:56:10.323 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-05-08T21:56:10.343 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T21:56:10.389 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 31/162 2024-05-08T21:56:10.545 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 32/162 2024-05-08T21:56:10.633 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-05-08T21:56:10.634 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T21:56:10.650 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T21:56:10.831 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 33/162 2024-05-08T21:56:10.862 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-05-08T21:56:10.882 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-05-08T21:56:11.070 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-05-08T21:56:11.081 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-05-08T21:56:11.278 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-05-08T21:56:11.287 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-05-08T21:56:11.460 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-05-08T21:56:11.642 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-05-08T21:56:11.686 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-05-08T21:56:11.967 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-05-08T21:56:11.980 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-05-08T21:56:11.998 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-05-08T21:56:12.276 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-05-08T21:56:12.286 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-05-08T21:56:12.375 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-05-08T21:56:12.518 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-05-08T21:56:12.546 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-05-08T21:56:12.687 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-05-08T21:56:12.698 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-05-08T21:56:12.703 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-05-08T21:56:12.922 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T21:56:12.947 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-05-08T21:56:12.968 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-05-08T21:56:13.222 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-05-08T21:56:13.289 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-05-08T21:56:13.486 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-05-08T21:56:13.609 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-05-08T21:56:13.688 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-05-08T21:56:13.863 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-05-08T21:56:13.882 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-05-08T21:56:14.115 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-05-08T21:56:14.154 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-05-08T21:56:14.326 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-05-08T21:56:14.360 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-05-08T21:56:14.445 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-05-08T21:56:14.544 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-05-08T21:56:14.737 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : jq-1.6-7.el8.x86_64 71/162 2024-05-08T21:56:14.820 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-05-08T21:56:14.960 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : automake-1.16.1-8.el8.noarch 72/162 2024-05-08T21:56:14.965 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T21:56:15.066 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-05-08T21:56:15.152 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : uuid-1.6.2-43.el8.x86_64 73/162 2024-05-08T21:56:15.205 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 73/162 2024-05-08T21:56:15.251 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-05-08T21:56:15.338 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-05-08T21:56:15.400 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-05-08T21:56:15.462 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-05-08T21:56:15.568 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-08T21:56:15.869 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-05-08T21:56:15.869 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-08T21:56:15.869 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 75/162 2024-05-08T21:56:15.869 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-05-08T21:56:15.910 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-packaging-16.8-9.el8.noarch 70/162 2024-05-08T21:56:15.998 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 41/162 2024-05-08T21:56:16.087 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-05-08T21:56:16.107 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-05-08T21:56:16.190 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-05-08T21:56:16.202 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 42/162 2024-05-08T21:56:16.285 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-08T21:56:16.414 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T21:56:16.485 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : jq-1.6-7.el8.x86_64 72/162 2024-05-08T21:56:16.490 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-08T21:56:16.509 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T21:56:16.716 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : automake-1.16.1-8.el8.noarch 73/162 2024-05-08T21:56:16.765 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T21:56:16.807 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-08T21:56:16.877 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T21:56:16.915 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : uuid-1.6.2-43.el8.x86_64 74/162 2024-05-08T21:56:16.975 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 74/162 2024-05-08T21:56:17.058 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-08T21:56:17.116 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-08T21:56:17.172 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 75/162 2024-05-08T21:56:17.264 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-08T21:56:17.330 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 76/162 2024-05-08T21:56:17.351 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-08T21:56:17.463 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-08T21:56:17.548 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T21:56:17.550 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 77/162 2024-05-08T21:56:17.660 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-08T21:56:17.690 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-08T21:56:17.872 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-08T21:56:17.898 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-08T21:56:18.135 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-08T21:56:18.190 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T21:56:18.235 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-08T21:56:18.329 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-08T21:56:18.455 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-08T21:56:18.490 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-08T21:56:18.540 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T21:56:18.586 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T21:56:18.695 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-08T21:56:18.746 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-08T21:56:18.793 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-packaging-16.8-9.el8.noarch 49/162 2024-05-08T21:56:18.893 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-08T21:56:18.996 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-08T21:56:19.011 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : fuse-2.9.7-17.el8.x86_64 50/162 2024-05-08T21:56:19.083 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-08T21:56:19.191 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-08T21:56:19.217 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 51/162 2024-05-08T21:56:19.303 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-08T21:56:19.447 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 52/162 2024-05-08T21:56:19.462 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-08T21:56:19.547 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-08T21:56:19.678 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 53/162 2024-05-08T21:56:19.700 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-08T21:56:19.727 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-08T21:56:19.853 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 54/162 2024-05-08T21:56:19.908 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-08T21:56:19.913 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-08T21:56:20.119 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-08T21:56:20.175 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-08T21:56:20.200 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 55/162 2024-05-08T21:56:20.336 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-08T21:56:20.496 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-08T21:56:20.498 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/162 2024-05-08T21:56:20.770 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/162 2024-05-08T21:56:20.784 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-08T21:56:20.983 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 58/162 2024-05-08T21:56:21.040 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-08T21:56:21.214 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-08T21:56:21.257 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 59/162 2024-05-08T21:56:21.289 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-08T21:56:21.490 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-08T21:56:21.725 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 60/162 2024-05-08T21:56:21.813 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-08T21:56:22.016 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-08T21:56:22.020 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 61/162 2024-05-08T21:56:22.341 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/162 2024-05-08T21:56:22.342 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-08T21:56:22.607 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 63/162 2024-05-08T21:56:22.617 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-08T21:56:22.621 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-08T21:56:22.813 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T21:56:22.841 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-08T21:56:22.926 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-08T21:56:23.066 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 65/162 2024-05-08T21:56:23.155 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-08T21:56:23.382 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 66/162 2024-05-08T21:56:23.440 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-08T21:56:23.623 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-08T21:56:23.633 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 67/162 2024-05-08T21:56:23.766 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-08T21:56:23.800 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T21:56:23.873 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 68/162 2024-05-08T21:56:23.897 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T21:56:24.006 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-08T21:56:24.067 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 104/162 2024-05-08T21:56:24.070 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 69/162 2024-05-08T21:56:24.360 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 70/162 2024-05-08T21:56:24.580 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-05-08T21:56:24.686 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-05-08T21:56:24.813 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-08T21:56:24.989 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T21:56:25.007 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : jq-1.6-7.el8.x86_64 72/162 2024-05-08T21:56:25.063 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T21:56:25.265 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 104/162 2024-05-08T21:56:25.299 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : automake-1.16.1-8.el8.noarch 73/162 2024-05-08T21:56:25.496 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : uuid-1.6.2-43.el8.x86_64 74/162 2024-05-08T21:56:25.587 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 74/162 2024-05-08T21:56:25.795 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 75/162 2024-05-08T21:56:25.997 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 76/162 2024-05-08T21:56:26.216 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 77/162 2024-05-08T21:56:26.424 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-08T21:56:26.621 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-08T21:56:27.302 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-08T21:56:27.555 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-08T21:56:27.719 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-08T21:56:27.830 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:56:27.927 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-08T21:56:28.116 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:56:28.186 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-08T21:56:28.455 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-08T21:56:28.695 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-08T21:56:28.709 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:56:28.882 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-08T21:56:28.990 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:56:29.044 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T21:56:29.070 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-08T21:56:29.109 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T21:56:29.274 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T21:56:29.277 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-08T21:56:29.536 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-08T21:56:29.736 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-08T21:56:29.992 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-08T21:56:30.222 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:56:30.223 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-08T21:56:30.439 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-08T21:56:30.720 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-08T21:56:30.817 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:56:31.121 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T21:56:31.195 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T21:56:31.355 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T21:56:31.982 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-08T21:56:32.526 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-08T21:56:32.793 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-08T21:56:33.144 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-08T21:56:33.448 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-08T21:56:33.716 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-08T21:56:34.554 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-08T21:56:34.778 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T21:56:34.873 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T21:56:35.072 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 104/162 2024-05-08T21:56:41.797 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:56:42.086 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:56:42.797 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:56:43.414 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T21:56:43.585 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T21:56:43.735 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T21:56:48.824 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T21:56:48.824 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /sys 2024-05-08T21:56:48.824 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /proc 2024-05-08T21:56:48.825 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /dev 2024-05-08T21:56:48.825 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /run 2024-05-08T21:56:48.825 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /mnt 2024-05-08T21:56:48.825 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /var/tmp 2024-05-08T21:56:48.825 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /home 2024-05-08T21:56:48.825 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /tmp 2024-05-08T21:56:48.825 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /dev 2024-05-08T21:56:48.825 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-05-08T21:56:49.197 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T21:56:50.705 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T21:56:50.705 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /sys 2024-05-08T21:56:50.705 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /proc 2024-05-08T21:56:50.705 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /dev 2024-05-08T21:56:50.705 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /run 2024-05-08T21:56:50.705 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /mnt 2024-05-08T21:56:50.705 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /var/tmp 2024-05-08T21:56:50.705 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /home 2024-05-08T21:56:50.705 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /tmp 2024-05-08T21:56:50.706 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /dev 2024-05-08T21:56:50.706 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-08T21:56:50.839 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T21:56:51.014 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T21:56:51.094 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T21:56:51.232 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T21:56:51.438 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T21:56:51.913 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T21:56:52.104 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T21:56:52.122 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T21:56:52.730 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T21:56:52.900 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T21:56:53.119 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T21:56:53.268 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T21:56:53.741 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T21:56:53.883 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T21:56:53.943 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T21:56:53.961 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T21:56:54.083 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T21:56:54.202 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 113/162 2024-05-08T21:56:54.447 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 114/162 2024-05-08T21:56:54.702 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T21:56:55.729 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T21:56:55.731 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T21:56:55.902 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T21:56:56.005 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T21:56:56.016 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 113/162 2024-05-08T21:56:56.270 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 114/162 2024-05-08T21:56:56.497 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T21:56:56.973 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T21:56:57.162 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T21:56:57.255 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T21:56:57.528 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T21:56:57.609 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 118/162 2024-05-08T21:56:57.774 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T21:56:57.822 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T21:56:57.860 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T21:56:58.028 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T21:56:58.040 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T21:56:58.737 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T21:56:58.900 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T21:56:58.993 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T21:56:59.332 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 121/162 2024-05-08T21:56:59.352 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 118/162 2024-05-08T21:56:59.508 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T21:56:59.513 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T21:56:59.547 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T21:56:59.548 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T21:56:59.725 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T21:56:59.737 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T21:56:59.745 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T21:56:59.788 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T21:56:59.995 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T21:57:00.342 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T21:57:00.553 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T21:57:01.004 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 121/162 2024-05-08T21:57:01.207 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T21:57:01.242 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T21:57:01.484 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T21:57:01.585 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T21:57:01.692 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T21:57:02.039 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T21:57:02.233 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T21:57:04.587 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T21:57:04.587 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /sys 2024-05-08T21:57:04.587 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /proc 2024-05-08T21:57:04.587 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /dev 2024-05-08T21:57:04.587 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /run 2024-05-08T21:57:04.587 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /mnt 2024-05-08T21:57:04.588 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /var/tmp 2024-05-08T21:57:04.588 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /home 2024-05-08T21:57:04.588 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /tmp 2024-05-08T21:57:04.588 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /dev 2024-05-08T21:57:04.588 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:57:04.940 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T21:57:05.018 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T21:57:05.486 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 126/162 2024-05-08T21:57:05.846 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-05-08T21:57:06.114 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T21:57:06.676 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T21:57:06.721 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T21:57:06.795 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T21:57:06.927 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T21:57:07.091 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-05-08T21:57:07.132 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 126/162 2024-05-08T21:57:07.193 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T21:57:07.360 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-05-08T21:57:07.382 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T21:57:07.470 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-05-08T21:57:07.609 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T21:57:07.717 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T21:57:07.846 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-08T21:57:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T21:57:08.015 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-08T21:57:08.090 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T21:57:08.133 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T21:57:08.219 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 134/162 2024-05-08T21:57:08.394 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T21:57:08.653 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-05-08T21:57:08.956 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-05-08T21:57:09.163 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T21:57:09.426 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-08T21:57:09.611 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-08T21:57:09.824 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 134/162 2024-05-08T21:57:10.528 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:57:10.877 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:57:10.915 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:57:10.925 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T21:57:11.121 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T21:57:11.247 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 113/162 2024-05-08T21:57:11.407 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-08T21:57:11.490 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 114/162 2024-05-08T21:57:11.758 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-08T21:57:11.794 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T21:57:11.988 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-08T21:57:12.201 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 139/162 2024-05-08T21:57:12.209 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:57:12.392 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 140/162 2024-05-08T21:57:12.549 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:57:12.590 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:57:12.709 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 141/162 2024-05-08T21:57:12.986 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-08T21:57:12.987 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T21:57:13.043 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-08T21:57:13.209 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T21:57:13.219 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T21:57:13.258 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T21:57:13.422 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-08T21:57:13.462 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-05-08T21:57:13.660 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-08T21:57:13.709 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T21:57:13.789 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T21:57:13.840 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 139/162 2024-05-08T21:57:14.009 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T21:57:14.023 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T21:57:14.072 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 140/162 2024-05-08T21:57:14.199 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-08T21:57:14.332 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 141/162 2024-05-08T21:57:14.353 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T21:57:14.411 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-08T21:57:14.572 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T21:57:14.613 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-08T21:57:14.613 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T21:57:14.658 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-08T21:57:14.705 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T21:57:14.814 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T21:57:14.824 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T21:57:14.896 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T21:57:14.896 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T21:57:15.053 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-05-08T21:57:15.124 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T21:57:15.124 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T21:57:15.125 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 118/162 2024-05-08T21:57:15.256 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T21:57:15.334 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T21:57:15.337 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T21:57:15.340 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T21:57:15.404 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T21:57:15.547 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T21:57:15.561 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T21:57:15.577 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-08T21:57:15.595 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T21:57:15.625 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T21:57:15.751 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-08T21:57:15.769 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-08T21:57:15.916 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-08T21:57:16.019 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-08T21:57:16.019 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:57:16.126 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-08T21:57:16.127 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T21:57:16.236 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:57:16.237 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T21:57:16.409 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T21:57:16.409 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T21:57:16.495 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T21:57:16.496 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:57:16.618 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T21:57:16.618 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T21:57:16.728 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:57:16.728 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T21:57:16.896 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T21:57:16.974 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 121/162 2024-05-08T21:57:17.003 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T21:57:17.003 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:57:17.182 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-08T21:57:17.200 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T21:57:17.261 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:57:17.263 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T21:57:17.358 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-08T21:57:17.403 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-08T21:57:17.403 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T21:57:17.472 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T21:57:17.545 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T21:57:17.607 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-08T21:57:17.607 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:57:17.722 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T21:57:17.883 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:57:17.883 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T21:57:18.116 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T21:57:18.116 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:57:18.126 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T21:57:18.314 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T21:57:18.361 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T21:57:18.361 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 1/162 2024-05-08T21:57:18.361 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2/162 2024-05-08T21:57:18.362 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 3/162 2024-05-08T21:57:18.362 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 4/162 2024-05-08T21:57:18.362 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 5/162 2024-05-08T21:57:18.362 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 6/162 2024-05-08T21:57:18.362 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 7/162 2024-05-08T21:57:18.362 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 8/162 2024-05-08T21:57:18.362 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:57:18.362 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:57:18.362 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:57:18.363 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T21:57:18.363 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 13/162 2024-05-08T21:57:18.363 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 14/162 2024-05-08T21:57:18.363 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 15/162 2024-05-08T21:57:18.363 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 16/162 2024-05-08T21:57:18.363 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 17/162 2024-05-08T21:57:18.363 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 18/162 2024-05-08T21:57:18.363 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:57:18.363 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 20/162 2024-05-08T21:57:18.363 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 21/162 2024-05-08T21:57:18.363 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 22/162 2024-05-08T21:57:18.363 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 23/162 2024-05-08T21:57:18.363 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 24/162 2024-05-08T21:57:18.364 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 25/162 2024-05-08T21:57:18.364 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 26/162 2024-05-08T21:57:18.364 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 27/162 2024-05-08T21:57:18.364 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 28/162 2024-05-08T21:57:18.364 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 29/162 2024-05-08T21:57:18.364 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 30/162 2024-05-08T21:57:18.364 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 31/162 2024-05-08T21:57:18.364 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 32/162 2024-05-08T21:57:18.364 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 33/162 2024-05-08T21:57:18.365 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 34/162 2024-05-08T21:57:18.365 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 35/162 2024-05-08T21:57:18.365 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 36/162 2024-05-08T21:57:18.365 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 37/162 2024-05-08T21:57:18.365 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 38/162 2024-05-08T21:57:18.365 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-08T21:57:18.365 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-08T21:57:18.365 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-08T21:57:18.365 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-08T21:57:18.365 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-08T21:57:18.365 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-08T21:57:18.365 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-08T21:57:18.366 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-08T21:57:18.366 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-08T21:57:18.366 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-08T21:57:18.366 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-08T21:57:18.366 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-08T21:57:18.366 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-08T21:57:18.366 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-08T21:57:18.366 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-08T21:57:18.366 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-08T21:57:18.367 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-08T21:57:18.367 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-08T21:57:18.367 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-08T21:57:18.367 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-08T21:57:18.367 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-08T21:57:18.367 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-08T21:57:18.367 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-08T21:57:18.367 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-08T21:57:18.367 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-08T21:57:18.367 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T21:57:18.368 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-08T21:57:18.368 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-08T21:57:18.368 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-08T21:57:18.368 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-08T21:57:18.368 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-08T21:57:18.368 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-08T21:57:18.368 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-08T21:57:18.368 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-05-08T21:57:18.368 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 73/162 2024-05-08T21:57:18.368 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-05-08T21:57:18.369 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 75/162 2024-05-08T21:57:18.369 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-05-08T21:57:18.369 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 77/162 2024-05-08T21:57:18.369 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 78/162 2024-05-08T21:57:18.369 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : bison-3.0.4-10.el8.x86_64 79/162 2024-05-08T21:57:18.369 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flex-2.6.1-9.el8.x86_64 80/162 2024-05-08T21:57:18.369 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 81/162 2024-05-08T21:57:18.369 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 82/162 2024-05-08T21:57:18.369 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 83/162 2024-05-08T21:57:18.369 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 84/162 2024-05-08T21:57:18.369 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 85/162 2024-05-08T21:57:18.369 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 86/162 2024-05-08T21:57:18.369 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 87/162 2024-05-08T21:57:18.370 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 88/162 2024-05-08T21:57:18.370 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 89/162 2024-05-08T21:57:18.370 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 90/162 2024-05-08T21:57:18.370 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 91/162 2024-05-08T21:57:18.370 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 92/162 2024-05-08T21:57:18.370 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 93/162 2024-05-08T21:57:18.370 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 94/162 2024-05-08T21:57:18.370 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : automake-1.16.1-8.el8.noarch 95/162 2024-05-08T21:57:18.370 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libpq-13.11-1.el8.x86_64 96/162 2024-05-08T21:57:18.370 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : jq-1.6-7.el8.x86_64 97/162 2024-05-08T21:57:18.370 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 98/162 2024-05-08T21:57:18.370 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 99/162 2024-05-08T21:57:18.371 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 100/162 2024-05-08T21:57:18.371 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 101/162 2024-05-08T21:57:18.371 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 102/162 2024-05-08T21:57:18.371 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 103/162 2024-05-08T21:57:18.371 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-05-08T21:57:18.371 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-05-08T21:57:18.371 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-05-08T21:57:18.371 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-05-08T21:57:18.371 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-05-08T21:57:18.371 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-05-08T21:57:18.371 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-05-08T21:57:18.371 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-05-08T21:57:18.371 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-05-08T21:57:18.372 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-05-08T21:57:18.372 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-05-08T21:57:18.372 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-05-08T21:57:18.372 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-05-08T21:57:18.372 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-05-08T21:57:18.372 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-05-08T21:57:18.372 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-05-08T21:57:18.372 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-05-08T21:57:18.372 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-05-08T21:57:18.372 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 122/162 2024-05-08T21:57:18.372 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 123/162 2024-05-08T21:57:18.372 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 124/162 2024-05-08T21:57:18.372 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 125/162 2024-05-08T21:57:18.372 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 126/162 2024-05-08T21:57:18.373 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 127/162 2024-05-08T21:57:18.373 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 128/162 2024-05-08T21:57:18.373 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 129/162 2024-05-08T21:57:18.373 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 130/162 2024-05-08T21:57:18.373 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T21:57:18.373 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-08T21:57:18.373 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 133/162 2024-05-08T21:57:18.373 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-08T21:57:18.373 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-08T21:57:18.373 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-08T21:57:18.373 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-08T21:57:18.373 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-08T21:57:18.373 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-08T21:57:18.374 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-08T21:57:18.374 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-08T21:57:18.374 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-08T21:57:18.374 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-08T21:57:18.374 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-08T21:57:18.374 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-08T21:57:18.374 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-08T21:57:18.374 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-08T21:57:18.374 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-08T21:57:18.374 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-08T21:57:18.374 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-08T21:57:18.374 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-08T21:57:18.375 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-08T21:57:18.375 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-08T21:57:18.375 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-08T21:57:18.375 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-08T21:57:18.375 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:57:18.375 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-08T21:57:18.375 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:57:18.375 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-08T21:57:18.375 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:57:18.375 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-08T21:57:18.376 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:57:18.376 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T21:57:18.616 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T21:57:18.616 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:57:18.816 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:57:19.000 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-08T21:57:19.000 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T21:57:19.903 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T21:57:19.903 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 1/162 2024-05-08T21:57:19.903 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2/162 2024-05-08T21:57:19.903 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 3/162 2024-05-08T21:57:19.904 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 4/162 2024-05-08T21:57:19.904 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 5/162 2024-05-08T21:57:19.904 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 6/162 2024-05-08T21:57:19.904 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 7/162 2024-05-08T21:57:19.904 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 8/162 2024-05-08T21:57:19.904 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:57:19.904 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:57:19.904 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:57:19.904 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T21:57:19.904 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 13/162 2024-05-08T21:57:19.904 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 14/162 2024-05-08T21:57:19.905 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 15/162 2024-05-08T21:57:19.905 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 16/162 2024-05-08T21:57:19.905 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 17/162 2024-05-08T21:57:19.905 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 18/162 2024-05-08T21:57:19.905 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:57:19.905 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 20/162 2024-05-08T21:57:19.905 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 21/162 2024-05-08T21:57:19.905 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 22/162 2024-05-08T21:57:19.905 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 23/162 2024-05-08T21:57:19.905 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 24/162 2024-05-08T21:57:19.905 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 25/162 2024-05-08T21:57:19.905 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 26/162 2024-05-08T21:57:19.905 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 27/162 2024-05-08T21:57:19.905 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 28/162 2024-05-08T21:57:19.906 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 29/162 2024-05-08T21:57:19.906 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 30/162 2024-05-08T21:57:19.906 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 31/162 2024-05-08T21:57:19.906 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 32/162 2024-05-08T21:57:19.906 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 33/162 2024-05-08T21:57:19.906 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 34/162 2024-05-08T21:57:19.906 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 35/162 2024-05-08T21:57:19.906 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 36/162 2024-05-08T21:57:19.906 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 37/162 2024-05-08T21:57:19.906 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 38/162 2024-05-08T21:57:19.906 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-08T21:57:19.906 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-08T21:57:19.906 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-08T21:57:19.906 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-08T21:57:19.907 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-08T21:57:19.907 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-08T21:57:19.907 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-08T21:57:19.907 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-08T21:57:19.907 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-08T21:57:19.907 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-08T21:57:19.907 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-08T21:57:19.907 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-08T21:57:19.907 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-08T21:57:19.907 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-08T21:57:19.908 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-08T21:57:19.908 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-08T21:57:19.908 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-08T21:57:19.908 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-08T21:57:19.908 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-08T21:57:19.908 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-08T21:57:19.908 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-08T21:57:19.908 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-08T21:57:19.908 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-08T21:57:19.908 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-08T21:57:19.908 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-08T21:57:19.908 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T21:57:19.908 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-08T21:57:19.909 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-08T21:57:19.909 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-08T21:57:19.909 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-08T21:57:19.909 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-08T21:57:19.909 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-08T21:57:19.909 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-08T21:57:19.909 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 72/162 2024-05-08T21:57:19.909 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/162 2024-05-08T21:57:19.909 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/162 2024-05-08T21:57:19.909 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 75/162 2024-05-08T21:57:19.909 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 76/162 2024-05-08T21:57:19.909 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/162 2024-05-08T21:57:19.909 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/162 2024-05-08T21:57:19.909 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flex-2.6.1-9.el8.x86_64 79/162 2024-05-08T21:57:19.909 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 80/162 2024-05-08T21:57:19.910 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 81/162 2024-05-08T21:57:19.910 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 82/162 2024-05-08T21:57:19.910 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 83/162 2024-05-08T21:57:19.910 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 84/162 2024-05-08T21:57:19.910 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 85/162 2024-05-08T21:57:19.910 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 86/162 2024-05-08T21:57:19.910 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/162 2024-05-08T21:57:19.910 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/162 2024-05-08T21:57:19.910 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/162 2024-05-08T21:57:19.910 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 90/162 2024-05-08T21:57:19.910 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 91/162 2024-05-08T21:57:19.910 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 92/162 2024-05-08T21:57:19.911 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 93/162 2024-05-08T21:57:19.911 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : automake-1.16.1-8.el8.noarch 94/162 2024-05-08T21:57:19.911 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libpq-13.11-1.el8.x86_64 95/162 2024-05-08T21:57:19.911 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : jq-1.6-7.el8.x86_64 96/162 2024-05-08T21:57:19.911 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 97/162 2024-05-08T21:57:19.911 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 98/162 2024-05-08T21:57:19.911 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 99/162 2024-05-08T21:57:19.911 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 100/162 2024-05-08T21:57:19.912 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 101/162 2024-05-08T21:57:19.912 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 102/162 2024-05-08T21:57:19.912 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 103/162 2024-05-08T21:57:19.912 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-05-08T21:57:19.912 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-05-08T21:57:19.913 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-05-08T21:57:19.913 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-05-08T21:57:19.913 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-05-08T21:57:19.913 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-05-08T21:57:19.913 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-05-08T21:57:19.913 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-05-08T21:57:19.914 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-05-08T21:57:19.914 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-05-08T21:57:19.914 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-05-08T21:57:19.914 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-05-08T21:57:19.914 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-05-08T21:57:19.914 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-05-08T21:57:19.915 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-05-08T21:57:19.915 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-05-08T21:57:19.915 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-05-08T21:57:19.915 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-05-08T21:57:19.915 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 122/162 2024-05-08T21:57:19.915 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 123/162 2024-05-08T21:57:19.916 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 124/162 2024-05-08T21:57:19.916 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 125/162 2024-05-08T21:57:19.916 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 126/162 2024-05-08T21:57:19.916 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 127/162 2024-05-08T21:57:19.916 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 128/162 2024-05-08T21:57:19.917 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 129/162 2024-05-08T21:57:19.917 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 130/162 2024-05-08T21:57:19.917 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T21:57:19.917 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-08T21:57:19.917 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 133/162 2024-05-08T21:57:19.917 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-08T21:57:19.917 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-08T21:57:19.918 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-08T21:57:19.918 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-08T21:57:19.918 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-08T21:57:19.918 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-08T21:57:19.918 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-08T21:57:19.918 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-08T21:57:19.918 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-08T21:57:19.918 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-08T21:57:19.918 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-08T21:57:19.918 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-08T21:57:19.919 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-08T21:57:19.919 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-08T21:57:19.919 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-08T21:57:19.919 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-08T21:57:19.919 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-08T21:57:19.919 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-08T21:57:19.919 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-08T21:57:19.919 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-08T21:57:19.919 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-08T21:57:19.919 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-08T21:57:19.919 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:57:19.919 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-08T21:57:19.919 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:57:19.919 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-08T21:57:19.920 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:57:19.920 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-08T21:57:23.176 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T21:57:23.720 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 126/162 2024-05-08T21:57:24.098 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-05-08T21:57:24.315 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T21:57:25.350 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T21:57:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-05-08T21:57:25.892 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-05-08T21:57:26.135 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T21:57:26.392 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-08T21:57:26.567 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-08T21:57:26.796 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 134/162 2024-05-08T21:57:30.375 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:57:30.796 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:57:30.860 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:57:31.273 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-08T21:57:31.286 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-08T21:57:31.286 INFO:teuthology.orchestra.run.smithi157.stdout:Installed products updated. 2024-05-08T21:57:31.645 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-08T21:57:31.905 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-08T21:57:31.905 INFO:teuthology.orchestra.run.smithi157.stdout:Upgraded: 2024-05-08T21:57:31.905 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-08T21:57:31.905 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-08T21:57:31.905 INFO:teuthology.orchestra.run.smithi157.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-08T21:57:31.905 INFO:teuthology.orchestra.run.smithi157.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:57:31.905 INFO:teuthology.orchestra.run.smithi157.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:57:31.905 INFO:teuthology.orchestra.run.smithi157.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:57:31.905 INFO:teuthology.orchestra.run.smithi157.stdout: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.905 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.905 INFO:teuthology.orchestra.run.smithi157.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:57:31.905 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:57:31.906 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:57:31.906 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:57:31.906 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:57:31.906 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:57:31.906 INFO:teuthology.orchestra.run.smithi157.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-08T21:57:31.906 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-08T21:57:31.906 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-05-08T21:57:31.906 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-08T21:57:31.906 INFO:teuthology.orchestra.run.smithi157.stdout: automake-1.16.1-8.el8.noarch 2024-05-08T21:57:31.906 INFO:teuthology.orchestra.run.smithi157.stdout: bison-3.0.4-10.el8.x86_64 2024-05-08T21:57:31.906 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.906 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.906 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.907 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.907 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:31.907 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.907 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.907 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.907 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:31.907 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:31.907 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:31.907 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:31.907 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:31.907 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.907 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.907 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:31.908 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.908 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.908 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.908 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:31.908 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:31.908 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.908 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:31.908 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-08T21:57:31.908 INFO:teuthology.orchestra.run.smithi157.stdout: flex-2.6.1-9.el8.x86_64 2024-05-08T21:57:31.908 INFO:teuthology.orchestra.run.smithi157.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-08T21:57:31.908 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-08T21:57:31.908 INFO:teuthology.orchestra.run.smithi157.stdout: jq-1.6-7.el8.x86_64 2024-05-08T21:57:31.908 INFO:teuthology.orchestra.run.smithi157.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-08T21:57:31.908 INFO:teuthology.orchestra.run.smithi157.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-08T21:57:31.909 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-08T21:57:31.909 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.909 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.909 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.909 INFO:teuthology.orchestra.run.smithi157.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-08T21:57:31.909 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-08T21:57:31.909 INFO:teuthology.orchestra.run.smithi157.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-08T21:57:31.909 INFO:teuthology.orchestra.run.smithi157.stdout: libpq-13.11-1.el8.x86_64 2024-05-08T21:57:31.909 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-08T21:57:31.909 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-08T21:57:31.909 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.909 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.909 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-08T21:57:31.909 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.910 INFO:teuthology.orchestra.run.smithi157.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-08T21:57:31.910 INFO:teuthology.orchestra.run.smithi157.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-08T21:57:31.910 INFO:teuthology.orchestra.run.smithi157.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-08T21:57:31.910 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-08T21:57:31.910 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:57:31.910 INFO:teuthology.orchestra.run.smithi157.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-08T21:57:31.910 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-08T21:57:31.910 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-08T21:57:31.910 INFO:teuthology.orchestra.run.smithi157.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-08T21:57:31.910 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-08T21:57:31.910 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-08T21:57:31.910 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:57:31.910 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-08T21:57:31.911 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-08T21:57:31.911 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-08T21:57:31.911 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:57:31.911 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:57:31.911 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:57:31.911 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-08T21:57:31.911 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-08T21:57:31.911 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-08T21:57:31.911 INFO:teuthology.orchestra.run.smithi157.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-08T21:57:31.911 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-08T21:57:31.911 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.911 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.911 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.911 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-08T21:57:31.912 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-08T21:57:31.912 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-08T21:57:31.912 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-08T21:57:31.912 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-08T21:57:31.912 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-08T21:57:31.912 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-08T21:57:31.912 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-08T21:57:31.912 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-08T21:57:31.912 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-08T21:57:31.912 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-08T21:57:31.912 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-08T21:57:31.912 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-08T21:57:31.912 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-08T21:57:31.913 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-08T21:57:31.913 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-08T21:57:31.913 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-08T21:57:31.913 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-08T21:57:31.913 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-08T21:57:31.913 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-08T21:57:31.913 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-08T21:57:31.913 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-08T21:57:31.914 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-08T21:57:31.914 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend-2.6-1.el8.noarch 2024-05-08T21:57:31.914 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-08T21:57:31.914 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-08T21:57:31.914 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-08T21:57:31.914 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-08T21:57:31.914 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-08T21:57:31.914 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-08T21:57:31.914 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.914 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.914 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-08T21:57:31.914 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-08T21:57:31.914 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.915 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-08T21:57:31.915 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-08T21:57:31.915 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-08T21:57:31.915 INFO:teuthology.orchestra.run.smithi157.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-08T21:57:31.915 INFO:teuthology.orchestra.run.smithi157.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-08T21:57:31.915 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-08T21:57:31.915 INFO:teuthology.orchestra.run.smithi157.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-08T21:57:31.915 INFO:teuthology.orchestra.run.smithi157.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-08T21:57:31.915 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-08T21:57:31.915 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-08T21:57:31.915 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-08T21:57:31.915 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-08T21:57:31.915 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-08T21:57:31.915 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.915 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.916 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:31.916 INFO:teuthology.orchestra.run.smithi157.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-08T21:57:31.916 INFO:teuthology.orchestra.run.smithi157.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-08T21:57:31.916 INFO:teuthology.orchestra.run.smithi157.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-08T21:57:31.916 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-08T21:57:31.916 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-08T21:57:31.917 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-08T21:57:31.917 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-08T21:57:31.917 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-08T21:57:31.917 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-05-08T21:57:32.137 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 139/162 2024-05-08T21:57:32.172 DEBUG:teuthology.parallel:result is None 2024-05-08T21:57:32.723 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 140/162 2024-05-08T21:57:33.079 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 141/162 2024-05-08T21:57:33.388 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-08T21:57:33.586 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T21:57:33.615 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T21:57:33.852 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-05-08T21:57:34.096 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T21:57:34.220 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T21:57:34.484 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T21:57:34.516 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T21:57:34.693 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-08T21:57:34.930 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-08T21:57:35.140 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-08T21:57:35.140 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T21:57:35.256 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-08T21:57:35.256 INFO:teuthology.orchestra.run.smithi132.stdout:Installed products updated. 2024-05-08T21:57:35.440 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T21:57:35.440 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T21:57:35.714 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T21:57:35.715 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T21:57:35.880 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-05-08T21:57:35.880 INFO:teuthology.orchestra.run.smithi132.stdout:Upgraded: 2024-05-08T21:57:35.880 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-08T21:57:35.880 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-08T21:57:35.880 INFO:teuthology.orchestra.run.smithi132.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-08T21:57:35.881 INFO:teuthology.orchestra.run.smithi132.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:57:35.881 INFO:teuthology.orchestra.run.smithi132.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:57:35.881 INFO:teuthology.orchestra.run.smithi132.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:57:35.881 INFO:teuthology.orchestra.run.smithi132.stdout: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.881 INFO:teuthology.orchestra.run.smithi132.stdout: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.881 INFO:teuthology.orchestra.run.smithi132.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:57:35.881 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:57:35.881 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:57:35.881 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:57:35.881 INFO:teuthology.orchestra.run.smithi132.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:57:35.881 INFO:teuthology.orchestra.run.smithi132.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:57:35.881 INFO:teuthology.orchestra.run.smithi132.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-08T21:57:35.881 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-08T21:57:35.882 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-05-08T21:57:35.882 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-08T21:57:35.882 INFO:teuthology.orchestra.run.smithi132.stdout: automake-1.16.1-8.el8.noarch 2024-05-08T21:57:35.882 INFO:teuthology.orchestra.run.smithi132.stdout: bison-3.0.4-10.el8.x86_64 2024-05-08T21:57:35.882 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.882 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.882 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.882 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.882 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:35.882 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.882 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.882 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.882 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:35.882 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:35.883 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:35.883 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:35.883 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:35.883 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.883 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.883 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:35.883 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.883 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.883 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.883 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:35.883 INFO:teuthology.orchestra.run.smithi132.stdout: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:35.883 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.883 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:35.883 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-08T21:57:35.884 INFO:teuthology.orchestra.run.smithi132.stdout: flex-2.6.1-9.el8.x86_64 2024-05-08T21:57:35.884 INFO:teuthology.orchestra.run.smithi132.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-08T21:57:35.884 INFO:teuthology.orchestra.run.smithi132.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-08T21:57:35.884 INFO:teuthology.orchestra.run.smithi132.stdout: jq-1.6-7.el8.x86_64 2024-05-08T21:57:35.884 INFO:teuthology.orchestra.run.smithi132.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-08T21:57:35.884 INFO:teuthology.orchestra.run.smithi132.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-08T21:57:35.884 INFO:teuthology.orchestra.run.smithi132.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-08T21:57:35.884 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.884 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.884 INFO:teuthology.orchestra.run.smithi132.stdout: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.884 INFO:teuthology.orchestra.run.smithi132.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-08T21:57:35.884 INFO:teuthology.orchestra.run.smithi132.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-08T21:57:35.884 INFO:teuthology.orchestra.run.smithi132.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-08T21:57:35.884 INFO:teuthology.orchestra.run.smithi132.stdout: libpq-13.11-1.el8.x86_64 2024-05-08T21:57:35.885 INFO:teuthology.orchestra.run.smithi132.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-08T21:57:35.885 INFO:teuthology.orchestra.run.smithi132.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-08T21:57:35.885 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.885 INFO:teuthology.orchestra.run.smithi132.stdout: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.885 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-08T21:57:35.885 INFO:teuthology.orchestra.run.smithi132.stdout: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.885 INFO:teuthology.orchestra.run.smithi132.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-08T21:57:35.885 INFO:teuthology.orchestra.run.smithi132.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-08T21:57:35.885 INFO:teuthology.orchestra.run.smithi132.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-08T21:57:35.885 INFO:teuthology.orchestra.run.smithi132.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-08T21:57:35.885 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:57:35.885 INFO:teuthology.orchestra.run.smithi132.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-08T21:57:35.885 INFO:teuthology.orchestra.run.smithi132.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-08T21:57:35.885 INFO:teuthology.orchestra.run.smithi132.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-08T21:57:35.886 INFO:teuthology.orchestra.run.smithi132.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-08T21:57:35.886 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-08T21:57:35.886 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-08T21:57:35.886 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:57:35.886 INFO:teuthology.orchestra.run.smithi132.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-08T21:57:35.886 INFO:teuthology.orchestra.run.smithi132.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-08T21:57:35.886 INFO:teuthology.orchestra.run.smithi132.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-08T21:57:35.886 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:57:35.886 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:57:35.887 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:57:35.887 INFO:teuthology.orchestra.run.smithi132.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-08T21:57:35.887 INFO:teuthology.orchestra.run.smithi132.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-08T21:57:35.887 INFO:teuthology.orchestra.run.smithi132.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-08T21:57:35.887 INFO:teuthology.orchestra.run.smithi132.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-08T21:57:35.887 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-08T21:57:35.887 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.887 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.887 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.887 INFO:teuthology.orchestra.run.smithi132.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-08T21:57:35.887 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-08T21:57:35.887 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-08T21:57:35.887 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-08T21:57:35.887 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-08T21:57:35.888 INFO:teuthology.orchestra.run.smithi132.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-08T21:57:35.888 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-08T21:57:35.888 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-08T21:57:35.888 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-08T21:57:35.888 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-08T21:57:35.888 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-08T21:57:35.888 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-08T21:57:35.888 INFO:teuthology.orchestra.run.smithi132.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-08T21:57:35.888 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-08T21:57:35.888 INFO:teuthology.orchestra.run.smithi132.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-08T21:57:35.888 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-08T21:57:35.888 INFO:teuthology.orchestra.run.smithi132.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-08T21:57:35.888 INFO:teuthology.orchestra.run.smithi132.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-08T21:57:35.888 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-08T21:57:35.889 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-08T21:57:35.889 INFO:teuthology.orchestra.run.smithi132.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-08T21:57:35.889 INFO:teuthology.orchestra.run.smithi132.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-08T21:57:35.889 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-08T21:57:35.889 INFO:teuthology.orchestra.run.smithi132.stdout: python3-portend-2.6-1.el8.noarch 2024-05-08T21:57:35.889 INFO:teuthology.orchestra.run.smithi132.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-08T21:57:35.889 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-08T21:57:35.889 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-08T21:57:35.890 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-08T21:57:35.890 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-08T21:57:35.890 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-08T21:57:35.890 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.890 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.890 INFO:teuthology.orchestra.run.smithi132.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-08T21:57:35.890 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-08T21:57:35.890 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.890 INFO:teuthology.orchestra.run.smithi132.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-08T21:57:35.890 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-08T21:57:35.890 INFO:teuthology.orchestra.run.smithi132.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-08T21:57:35.890 INFO:teuthology.orchestra.run.smithi132.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-08T21:57:35.890 INFO:teuthology.orchestra.run.smithi132.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-08T21:57:35.891 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-08T21:57:35.891 INFO:teuthology.orchestra.run.smithi132.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-08T21:57:35.891 INFO:teuthology.orchestra.run.smithi132.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-08T21:57:35.891 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-08T21:57:35.891 INFO:teuthology.orchestra.run.smithi132.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-08T21:57:35.891 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-08T21:57:35.891 INFO:teuthology.orchestra.run.smithi132.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-08T21:57:35.891 INFO:teuthology.orchestra.run.smithi132.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-08T21:57:35.891 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.891 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.891 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:35.892 INFO:teuthology.orchestra.run.smithi132.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-08T21:57:35.892 INFO:teuthology.orchestra.run.smithi132.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-08T21:57:35.892 INFO:teuthology.orchestra.run.smithi132.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-08T21:57:35.892 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-08T21:57:35.892 INFO:teuthology.orchestra.run.smithi132.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-08T21:57:35.892 INFO:teuthology.orchestra.run.smithi132.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-08T21:57:35.892 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-05-08T21:57:35.892 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-05-08T21:57:35.995 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T21:57:36.139 DEBUG:teuthology.parallel:result is None 2024-05-08T21:57:36.254 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-08T21:57:36.467 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-08T21:57:36.746 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-08T21:57:36.746 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:57:37.039 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:57:37.039 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T21:57:37.322 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T21:57:37.322 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:57:37.572 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:57:37.572 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T21:57:37.838 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T21:57:37.839 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:57:39.139 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:57:39.288 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-08T21:57:39.288 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T21:57:39.919 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T21:57:39.919 INFO:teuthology.orchestra.run.smithi131.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-08T21:57:39.919 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:57:40.659 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 1/162 2024-05-08T21:57:40.659 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2/162 2024-05-08T21:57:40.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 3/162 2024-05-08T21:57:40.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 4/162 2024-05-08T21:57:40.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 5/162 2024-05-08T21:57:40.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 6/162 2024-05-08T21:57:40.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 7/162 2024-05-08T21:57:40.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 8/162 2024-05-08T21:57:40.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:57:40.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:57:40.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:57:40.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T21:57:40.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 13/162 2024-05-08T21:57:40.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 14/162 2024-05-08T21:57:40.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 15/162 2024-05-08T21:57:40.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 16/162 2024-05-08T21:57:40.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 17/162 2024-05-08T21:57:40.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 18/162 2024-05-08T21:57:40.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:57:40.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 20/162 2024-05-08T21:57:40.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 21/162 2024-05-08T21:57:40.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 22/162 2024-05-08T21:57:40.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 23/162 2024-05-08T21:57:40.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 24/162 2024-05-08T21:57:40.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 25/162 2024-05-08T21:57:40.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 26/162 2024-05-08T21:57:40.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 27/162 2024-05-08T21:57:40.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 28/162 2024-05-08T21:57:40.662 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 29/162 2024-05-08T21:57:40.662 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 30/162 2024-05-08T21:57:40.662 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 31/162 2024-05-08T21:57:40.662 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 32/162 2024-05-08T21:57:40.662 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 33/162 2024-05-08T21:57:40.675 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 34/162 2024-05-08T21:57:40.675 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 35/162 2024-05-08T21:57:40.675 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 36/162 2024-05-08T21:57:40.675 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 37/162 2024-05-08T21:57:40.675 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 38/162 2024-05-08T21:57:40.675 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-08T21:57:40.675 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-08T21:57:40.675 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-08T21:57:40.675 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-08T21:57:40.676 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-08T21:57:40.676 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-08T21:57:40.676 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-08T21:57:40.676 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-08T21:57:40.676 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-08T21:57:40.676 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-08T21:57:40.676 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-08T21:57:40.676 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-08T21:57:40.676 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-08T21:57:40.676 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-08T21:57:40.676 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-08T21:57:40.676 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-08T21:57:40.676 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-08T21:57:40.676 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-08T21:57:40.677 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-08T21:57:40.677 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-08T21:57:40.677 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-08T21:57:40.677 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-08T21:57:40.677 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-08T21:57:40.677 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-08T21:57:40.677 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-08T21:57:40.677 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T21:57:40.677 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-08T21:57:40.677 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-08T21:57:40.677 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-08T21:57:40.677 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-08T21:57:40.677 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-08T21:57:40.678 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-08T21:57:40.678 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-08T21:57:40.678 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 72/162 2024-05-08T21:57:40.678 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/162 2024-05-08T21:57:40.678 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/162 2024-05-08T21:57:40.678 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 75/162 2024-05-08T21:57:40.678 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 76/162 2024-05-08T21:57:40.678 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/162 2024-05-08T21:57:40.678 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/162 2024-05-08T21:57:40.678 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flex-2.6.1-9.el8.x86_64 79/162 2024-05-08T21:57:40.678 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 80/162 2024-05-08T21:57:40.678 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 81/162 2024-05-08T21:57:40.678 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 82/162 2024-05-08T21:57:40.679 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 83/162 2024-05-08T21:57:40.679 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 84/162 2024-05-08T21:57:40.679 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 85/162 2024-05-08T21:57:40.679 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 86/162 2024-05-08T21:57:40.679 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/162 2024-05-08T21:57:40.679 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/162 2024-05-08T21:57:40.679 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/162 2024-05-08T21:57:40.679 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 90/162 2024-05-08T21:57:40.679 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 91/162 2024-05-08T21:57:40.679 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 92/162 2024-05-08T21:57:40.679 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 93/162 2024-05-08T21:57:40.679 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : automake-1.16.1-8.el8.noarch 94/162 2024-05-08T21:57:40.680 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libpq-13.11-1.el8.x86_64 95/162 2024-05-08T21:57:40.680 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : jq-1.6-7.el8.x86_64 96/162 2024-05-08T21:57:40.680 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 97/162 2024-05-08T21:57:40.680 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 98/162 2024-05-08T21:57:40.680 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 99/162 2024-05-08T21:57:40.680 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 100/162 2024-05-08T21:57:40.680 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 101/162 2024-05-08T21:57:40.680 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 102/162 2024-05-08T21:57:40.680 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 103/162 2024-05-08T21:57:40.680 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 104/162 2024-05-08T21:57:40.680 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 105/162 2024-05-08T21:57:40.680 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 106/162 2024-05-08T21:57:40.680 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 107/162 2024-05-08T21:57:40.681 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 108/162 2024-05-08T21:57:40.681 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 109/162 2024-05-08T21:57:40.681 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 110/162 2024-05-08T21:57:40.681 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/162 2024-05-08T21:57:40.681 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 112/162 2024-05-08T21:57:40.681 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 113/162 2024-05-08T21:57:40.681 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 114/162 2024-05-08T21:57:40.681 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 115/162 2024-05-08T21:57:40.681 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 116/162 2024-05-08T21:57:40.681 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 117/162 2024-05-08T21:57:40.681 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 118/162 2024-05-08T21:57:40.681 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 119/162 2024-05-08T21:57:40.681 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 120/162 2024-05-08T21:57:40.681 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 121/162 2024-05-08T21:57:40.682 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 122/162 2024-05-08T21:57:40.682 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 123/162 2024-05-08T21:57:40.682 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 124/162 2024-05-08T21:57:40.682 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 125/162 2024-05-08T21:57:40.682 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 126/162 2024-05-08T21:57:40.682 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 127/162 2024-05-08T21:57:40.682 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 128/162 2024-05-08T21:57:40.682 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 129/162 2024-05-08T21:57:40.682 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 130/162 2024-05-08T21:57:40.682 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T21:57:40.682 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-08T21:57:40.682 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 133/162 2024-05-08T21:57:40.682 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-08T21:57:40.683 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-08T21:57:40.683 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-08T21:57:40.683 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-08T21:57:40.683 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-08T21:57:40.683 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-08T21:57:40.683 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-08T21:57:40.683 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-08T21:57:40.683 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-08T21:57:40.683 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-08T21:57:40.683 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-08T21:57:40.683 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-08T21:57:40.683 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-08T21:57:40.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-08T21:57:40.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-08T21:57:40.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-08T21:57:40.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-08T21:57:40.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-08T21:57:40.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-08T21:57:40.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-08T21:57:40.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-08T21:57:40.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-08T21:57:40.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:57:40.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-08T21:57:40.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:57:40.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-08T21:57:40.684 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:57:40.685 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-08T21:57:53.363 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-08T21:57:53.364 INFO:teuthology.orchestra.run.smithi131.stdout:Installed products updated. 2024-05-08T21:57:54.036 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:57:54.036 INFO:teuthology.orchestra.run.smithi131.stdout:Upgraded: 2024-05-08T21:57:54.036 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-08T21:57:54.036 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-08T21:57:54.036 INFO:teuthology.orchestra.run.smithi131.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-08T21:57:54.036 INFO:teuthology.orchestra.run.smithi131.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:57:54.036 INFO:teuthology.orchestra.run.smithi131.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:57:54.036 INFO:teuthology.orchestra.run.smithi131.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:57:54.036 INFO:teuthology.orchestra.run.smithi131.stdout: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.037 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.037 INFO:teuthology.orchestra.run.smithi131.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:57:54.037 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:57:54.037 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:57:54.037 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:57:54.037 INFO:teuthology.orchestra.run.smithi131.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:57:54.037 INFO:teuthology.orchestra.run.smithi131.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:57:54.037 INFO:teuthology.orchestra.run.smithi131.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-08T21:57:54.037 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-08T21:57:54.037 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-05-08T21:57:54.037 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-08T21:57:54.037 INFO:teuthology.orchestra.run.smithi131.stdout: automake-1.16.1-8.el8.noarch 2024-05-08T21:57:54.038 INFO:teuthology.orchestra.run.smithi131.stdout: bison-3.0.4-10.el8.x86_64 2024-05-08T21:57:54.038 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.038 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.038 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.038 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.038 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:54.038 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.038 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.038 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.038 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:54.038 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:54.038 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:54.039 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:54.039 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:54.039 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.039 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.039 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:54.039 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.039 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.039 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.039 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:54.039 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:54.039 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.039 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:57:54.039 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-08T21:57:54.040 INFO:teuthology.orchestra.run.smithi131.stdout: flex-2.6.1-9.el8.x86_64 2024-05-08T21:57:54.040 INFO:teuthology.orchestra.run.smithi131.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-08T21:57:54.040 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-08T21:57:54.040 INFO:teuthology.orchestra.run.smithi131.stdout: jq-1.6-7.el8.x86_64 2024-05-08T21:57:54.040 INFO:teuthology.orchestra.run.smithi131.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-08T21:57:54.040 INFO:teuthology.orchestra.run.smithi131.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-08T21:57:54.040 INFO:teuthology.orchestra.run.smithi131.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-08T21:57:54.040 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.040 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.040 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.040 INFO:teuthology.orchestra.run.smithi131.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-08T21:57:54.040 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-08T21:57:54.040 INFO:teuthology.orchestra.run.smithi131.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-08T21:57:54.040 INFO:teuthology.orchestra.run.smithi131.stdout: libpq-13.11-1.el8.x86_64 2024-05-08T21:57:54.040 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-08T21:57:54.041 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-08T21:57:54.041 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.041 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.041 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-08T21:57:54.041 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.041 INFO:teuthology.orchestra.run.smithi131.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-08T21:57:54.041 INFO:teuthology.orchestra.run.smithi131.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-08T21:57:54.041 INFO:teuthology.orchestra.run.smithi131.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-08T21:57:54.041 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-08T21:57:54.041 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:57:54.041 INFO:teuthology.orchestra.run.smithi131.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-08T21:57:54.041 INFO:teuthology.orchestra.run.smithi131.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-08T21:57:54.042 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-08T21:57:54.042 INFO:teuthology.orchestra.run.smithi131.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-08T21:57:54.042 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-08T21:57:54.042 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-08T21:57:54.042 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:57:54.042 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-08T21:57:54.042 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-08T21:57:54.042 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-08T21:57:54.042 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:57:54.043 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:57:54.043 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:57:54.043 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-08T21:57:54.043 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-08T21:57:54.043 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-08T21:57:54.043 INFO:teuthology.orchestra.run.smithi131.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-08T21:57:54.043 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-08T21:57:54.043 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.043 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.043 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.043 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-08T21:57:54.043 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-08T21:57:54.043 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-08T21:57:54.043 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-08T21:57:54.044 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-08T21:57:54.044 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-08T21:57:54.044 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-08T21:57:54.044 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-08T21:57:54.044 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-08T21:57:54.044 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-08T21:57:54.044 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-08T21:57:54.044 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-08T21:57:54.044 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-08T21:57:54.044 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-08T21:57:54.044 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-08T21:57:54.044 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-08T21:57:54.044 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-08T21:57:54.045 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-08T21:57:54.045 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-08T21:57:54.045 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-08T21:57:54.045 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-08T21:57:54.045 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-08T21:57:54.045 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-08T21:57:54.045 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend-2.6-1.el8.noarch 2024-05-08T21:57:54.045 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-08T21:57:54.045 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-08T21:57:54.045 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-08T21:57:54.045 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-08T21:57:54.046 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-08T21:57:54.046 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-08T21:57:54.046 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.046 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.046 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-08T21:57:54.046 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-08T21:57:54.046 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.046 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-08T21:57:54.046 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-08T21:57:54.046 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-08T21:57:54.046 INFO:teuthology.orchestra.run.smithi131.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-08T21:57:54.046 INFO:teuthology.orchestra.run.smithi131.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-08T21:57:54.047 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-08T21:57:54.047 INFO:teuthology.orchestra.run.smithi131.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-08T21:57:54.047 INFO:teuthology.orchestra.run.smithi131.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-08T21:57:54.047 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-08T21:57:54.047 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-08T21:57:54.047 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-08T21:57:54.047 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-08T21:57:54.047 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-08T21:57:54.047 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.047 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.047 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:57:54.047 INFO:teuthology.orchestra.run.smithi131.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-08T21:57:54.047 INFO:teuthology.orchestra.run.smithi131.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-08T21:57:54.047 INFO:teuthology.orchestra.run.smithi131.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-08T21:57:54.048 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-08T21:57:54.048 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-08T21:57:54.048 INFO:teuthology.orchestra.run.smithi131.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-08T21:57:54.048 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:57:54.048 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-05-08T21:57:54.422 DEBUG:teuthology.parallel:result is None 2024-05-08T21:57:54.422 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:57:54.573 DEBUG:teuthology.orchestra.run.smithi131:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T21:57:54.626 INFO:teuthology.orchestra.run.smithi131.stdout:18.2.2-1322.gd21e4f96.el8 2024-05-08T21:57:54.627 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322.gd21e4f96.el8 2024-05-08T21:57:54.627 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322.gd21e4f96 is installed. 2024-05-08T21:57:54.629 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:57:54.772 DEBUG:teuthology.orchestra.run.smithi132:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T21:57:54.801 INFO:teuthology.orchestra.run.smithi132.stdout:18.2.2-1322.gd21e4f96.el8 2024-05-08T21:57:54.801 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322.gd21e4f96.el8 2024-05-08T21:57:54.802 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322.gd21e4f96 is installed. 2024-05-08T21:57:54.804 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:57:55.022 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T21:57:55.052 INFO:teuthology.orchestra.run.smithi157.stdout:18.2.2-1322.gd21e4f96.el8 2024-05-08T21:57:55.052 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322.gd21e4f96.el8 2024-05-08T21:57:55.052 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322.gd21e4f96 is installed. 2024-05-08T21:57:55.054 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T21:57:55.054 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-08T21:57:55.054 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T21:57:55.146 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-05-08T21:57:55.146 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T21:57:55.196 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-08T21:57:55.196 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T21:57:55.245 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T21:57:55.245 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-08T21:57:55.245 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T21:57:55.312 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T21:57:55.418 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-05-08T21:57:55.418 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T21:57:55.450 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T21:57:55.522 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-08T21:57:55.522 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T21:57:55.555 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T21:57:55.626 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T21:57:55.626 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-08T21:57:55.626 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T21:57:55.693 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T21:57:55.798 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-05-08T21:57:55.798 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T21:57:55.829 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T21:57:55.901 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-08T21:57:55.901 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T21:57:55.933 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T21:57:56.004 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T21:57:56.005 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-08T21:57:56.005 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T21:57:56.071 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T21:57:56.175 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-05-08T21:57:56.176 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T21:57:56.208 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T21:57:56.279 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-08T21:57:56.279 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T21:57:56.311 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T21:57:56.381 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-08T21:57:56.504 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'subvols': {'create': 2}} 2024-05-08T21:57:56.505 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:57:56.505 INFO:tasks.cephadm:Cluster fsid is 06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T21:57:56.505 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-08T21:57:56.505 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.131', 'mon.b': '172.21.15.132', 'mon.c': '172.21.15.157'} 2024-05-08T21:57:56.505 INFO:tasks.cephadm:First mon is mon.a on smithi131 2024-05-08T21:57:56.505 INFO:tasks.cephadm:First mgr is x 2024-05-08T21:57:56.506 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-08T21:57:56.506 DEBUG:teuthology.orchestra.run.smithi131:> sudo hostname $(hostname -s) 2024-05-08T21:57:56.572 DEBUG:teuthology.orchestra.run.smithi132:> sudo hostname $(hostname -s) 2024-05-08T21:57:56.607 DEBUG:teuthology.orchestra.run.smithi157:> sudo hostname $(hostname -s) 2024-05-08T21:57:56.640 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-08T21:57:56.640 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:57:56.783 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/79222/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-1322-gd21e4f96', 'node_name': '172.21.2.4+braggi04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-1322.gd21e4f96'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-05-08 19:11:17.581213', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.150423-reef', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'archs': ['arm64', 'source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/79220/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-1322-gd21e4f96', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-1322.gd21e4f96'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-05-08 18:26:58.979379', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.150423-reef', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-08T21:57:57.016 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-pdonnell-testing-20240508.150423-reef, sha1 d21e4f961318b56f0d915caa2490c3711bf13c34 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:57:57.018 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm 2024-05-08T21:57:57.018 DEBUG:teuthology.orchestra.run.smithi131:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T21:57:57.411 INFO:teuthology.orchestra.run.smithi131.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 8 21:57 /home/ubuntu/cephtest/cephadm 2024-05-08T21:57:57.412 DEBUG:teuthology.orchestra.run.smithi132:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T21:57:57.768 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 8 21:57 /home/ubuntu/cephtest/cephadm 2024-05-08T21:57:57.768 DEBUG:teuthology.orchestra.run.smithi157:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T21:57:58.281 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 8 21:57 /home/ubuntu/cephtest/cephadm 2024-05-08T21:57:58.282 DEBUG:teuthology.orchestra.run.smithi131:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T21:57:58.331 DEBUG:teuthology.orchestra.run.smithi132:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T21:57:58.353 DEBUG:teuthology.orchestra.run.smithi157:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T21:57:58.415 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 on all hosts... 2024-05-08T21:57:58.415 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 pull 2024-05-08T21:57:58.419 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 pull 2024-05-08T21:57:58.421 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 pull 2024-05-08T21:57:59.399 INFO:teuthology.orchestra.run.smithi157.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T21:57:59.411 INFO:teuthology.orchestra.run.smithi132.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T21:58:00.304 INFO:teuthology.orchestra.run.smithi131.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T21:58:26.395 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-05-08T21:58:26.396 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T21:58:26.396 INFO:teuthology.orchestra.run.smithi132.stdout: "image_id": "af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c", 2024-05-08T21:58:26.396 INFO:teuthology.orchestra.run.smithi132.stdout: "repo_digests": [ 2024-05-08T21:58:26.396 INFO:teuthology.orchestra.run.smithi132.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e8ec72da4fcdc307fe9b94614a1c07d7ebae386ad20c4998c8d80e9b80a48985" 2024-05-08T21:58:26.396 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-05-08T21:58:26.396 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-05-08T21:58:28.976 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T21:58:28.977 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T21:58:28.977 INFO:teuthology.orchestra.run.smithi131.stdout: "image_id": "af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c", 2024-05-08T21:58:28.977 INFO:teuthology.orchestra.run.smithi131.stdout: "repo_digests": [ 2024-05-08T21:58:28.977 INFO:teuthology.orchestra.run.smithi131.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e8ec72da4fcdc307fe9b94614a1c07d7ebae386ad20c4998c8d80e9b80a48985" 2024-05-08T21:58:28.977 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-08T21:58:28.977 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T21:58:33.619 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-05-08T21:58:33.620 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T21:58:33.620 INFO:teuthology.orchestra.run.smithi157.stdout: "image_id": "af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c", 2024-05-08T21:58:33.620 INFO:teuthology.orchestra.run.smithi157.stdout: "repo_digests": [ 2024-05-08T21:58:33.620 INFO:teuthology.orchestra.run.smithi157.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e8ec72da4fcdc307fe9b94614a1c07d7ebae386ad20c4998c8d80e9b80a48985" 2024-05-08T21:58:33.620 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2024-05-08T21:58:33.620 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-05-08T21:58:33.640 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /etc/ceph 2024-05-08T21:58:33.717 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph 2024-05-08T21:58:33.754 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph 2024-05-08T21:58:33.785 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 777 /etc/ceph 2024-05-08T21:58:33.853 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 777 /etc/ceph 2024-05-08T21:58:33.884 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /etc/ceph 2024-05-08T21:58:33.912 INFO:tasks.cephadm:Writing seed config... 2024-05-08T21:58:33.914 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-08T21:58:33.914 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-08T21:58:33.914 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-08T21:58:33.914 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-08T21:58:33.914 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-08T21:58:33.914 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-08T21:58:33.914 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-08T21:58:33.914 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-08T21:58:33.914 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-08T21:58:33.914 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-08T21:58:33.914 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-08T21:58:33.915 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-08T21:58:33.915 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-08T21:58:33.915 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-08T21:58:33.915 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-08T21:58:33.915 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-08T21:58:33.915 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-08T21:58:33.915 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-05-08T21:58:33.915 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-08T21:58:33.915 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-08T21:58:33.915 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-08T21:58:33.915 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-08T21:58:33.915 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-08T21:58:33.915 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-08T21:58:33.916 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-08T21:58:33.916 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-08T21:58:33.916 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-08T21:58:33.916 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-08T21:58:33.916 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-08T21:58:33.916 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-08T21:58:33.916 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-08T21:58:33.917 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-08T21:58:33.917 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-08T21:58:33.917 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-08T21:58:33.917 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-08T21:58:33.917 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-08T21:58:33.917 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-08T21:58:33.918 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-08T21:58:33.918 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-08T21:58:33.918 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-08T21:58:33.918 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-08T21:58:33.918 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-08T21:58:33.919 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-08T21:58:33.919 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-08T21:58:33.919 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-08T21:58:33.920 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-08T21:58:33.920 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-08T21:58:33.962 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 06f1a696-0d86-11ef-bc97-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-08T21:58:33.962 DEBUG:teuthology.orchestra.run.smithi131:mon.a> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mon.a.service 2024-05-08T21:58:34.006 DEBUG:teuthology.orchestra.run.smithi131:mgr.x> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mgr.x.service 2024-05-08T21:58:34.051 INFO:tasks.cephadm:Bootstrapping... 2024-05-08T21:58:34.051 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 -v bootstrap --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.131 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-08T21:58:34.070 INFO:journalctl@ceph.mon.a.smithi131.stdout:-- Logs begin at Wed 2024-05-08 21:41:05 UTC. -- 2024-05-08T21:58:34.115 INFO:journalctl@ceph.mgr.x.smithi131.stdout:-- Logs begin at Wed 2024-05-08 21:41:05 UTC. -- 2024-05-08T21:58:34.267 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-05-08T21:58:34.267 INFO:teuthology.orchestra.run.smithi131.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34', '-v', 'bootstrap', '--fsid', '06f1a696-0d86-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.131', '--skip-admin-label'] 2024-05-08T21:58:34.313 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stdout 4.6.1 2024-05-08T21:58:34.313 INFO:teuthology.orchestra.run.smithi131.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-08T21:58:34.314 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying podman|docker is present... 2024-05-08T21:58:34.358 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stdout 4.6.1 2024-05-08T21:58:34.358 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying lvm2 is present... 2024-05-08T21:58:34.358 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying time synchronization is in place... 2024-05-08T21:58:34.372 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-08T21:58:34.372 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-08T21:58:34.384 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-08T21:58:34.384 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-05-08T21:58:34.397 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout enabled 2024-05-08T21:58:34.409 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout active 2024-05-08T21:58:34.409 INFO:teuthology.orchestra.run.smithi131.stdout:Unit chronyd.service is enabled and running 2024-05-08T21:58:34.409 INFO:teuthology.orchestra.run.smithi131.stdout:Repeating the final host check... 2024-05-08T21:58:34.453 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stdout 4.6.1 2024-05-08T21:58:34.453 INFO:teuthology.orchestra.run.smithi131.stdout:podman (/bin/podman) version 4.6.1 is present 2024-05-08T21:58:34.453 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl is present 2024-05-08T21:58:34.453 INFO:teuthology.orchestra.run.smithi131.stdout:lvcreate is present 2024-05-08T21:58:34.465 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-08T21:58:34.465 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-08T21:58:34.477 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-08T21:58:34.477 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-05-08T21:58:34.490 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout enabled 2024-05-08T21:58:34.502 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout active 2024-05-08T21:58:34.502 INFO:teuthology.orchestra.run.smithi131.stdout:Unit chronyd.service is enabled and running 2024-05-08T21:58:34.502 INFO:teuthology.orchestra.run.smithi131.stdout:Host looks OK 2024-05-08T21:58:34.502 INFO:teuthology.orchestra.run.smithi131.stdout:Cluster fsid: 06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T21:58:34.502 INFO:teuthology.orchestra.run.smithi131.stdout:Acquiring lock 140623941609960 on /run/cephadm/06f1a696-0d86-11ef-bc97-c7b262605968.lock 2024-05-08T21:58:34.503 INFO:teuthology.orchestra.run.smithi131.stdout:Lock 140623941609960 acquired on /run/cephadm/06f1a696-0d86-11ef-bc97-c7b262605968.lock 2024-05-08T21:58:34.503 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying IP 172.21.15.131 port 3300 ... 2024-05-08T21:58:34.503 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying IP 172.21.15.131 port 6789 ... 2024-05-08T21:58:34.504 INFO:teuthology.orchestra.run.smithi131.stdout:Base mon IP(s) is [172.21.15.131:3300, 172.21.15.131:6789], mon addrv is [v2:172.21.15.131:3300,v1:172.21.15.131:6789] 2024-05-08T21:58:34.511 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2024-05-08T21:58:34.511 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.131 metric 100 2024-05-08T21:58:34.518 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-08T21:58:34.518 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-05-08T21:58:34.518 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1799sec hoplimit 64 pref medium 2024-05-08T21:58:34.525 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-08T21:58:34.525 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-08T21:58:34.525 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-08T21:58:34.525 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-08T21:58:34.525 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-05-08T21:58:34.525 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-08T21:58:34.526 INFO:teuthology.orchestra.run.smithi131.stdout:Mon IP `172.21.15.131` is in CIDR network `172.21.0.0/20` 2024-05-08T21:58:34.526 INFO:teuthology.orchestra.run.smithi131.stdout:Mon IP `172.21.15.131` is in CIDR network `172.21.0.0/20` 2024-05-08T21:58:34.526 INFO:teuthology.orchestra.run.smithi131.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-08T21:58:34.527 INFO:teuthology.orchestra.run.smithi131.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-08T21:58:34.528 INFO:teuthology.orchestra.run.smithi131.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T21:58:34.975 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stdout af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c 2024-05-08T21:58:34.975 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T21:58:34.975 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Getting image source signatures 2024-05-08T21:58:34.975 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Copying blob sha256:7850648fa4707607dddc5eec7674b8c9296086cf87f20a8b1acf1b014b952e9f 2024-05-08T21:58:34.975 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-08T21:58:34.976 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Copying config sha256:af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c 2024-05-08T21:58:34.976 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-08T21:58:35.724 INFO:teuthology.orchestra.run.smithi131.stdout:ceph: stdout ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable) 2024-05-08T21:58:35.725 INFO:teuthology.orchestra.run.smithi131.stdout:Ceph version: ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable) 2024-05-08T21:58:35.725 INFO:teuthology.orchestra.run.smithi131.stdout:Extracting ceph user uid/gid from container image... 2024-05-08T21:58:36.941 INFO:teuthology.orchestra.run.smithi131.stdout:stat: stdout 167 167 2024-05-08T21:58:36.941 INFO:teuthology.orchestra.run.smithi131.stdout:Creating initial keys... 2024-05-08T21:58:37.581 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph-authtool: stdout AQCN9TtmUo5yDhAAZdpQAa1H67RiEFUFB7h0qA== 2024-05-08T21:58:38.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph-authtool: stdout AQCN9TtmI7ovNRAAMZ5kkypkGdosEyfwfjB1jA== 2024-05-08T21:58:38.862 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph-authtool: stdout AQCO9TtmTMfOIBAAmcW/nW8yeFP3WyYEEoQJ5Q== 2024-05-08T21:58:38.863 INFO:teuthology.orchestra.run.smithi131.stdout:Creating initial monmap... 2024-05-08T21:58:39.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-08T21:58:39.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-05-08T21:58:39.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T21:58:39.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-08T21:58:39.516 INFO:teuthology.orchestra.run.smithi131.stdout:monmaptool for a [v2:172.21.15.131:3300,v1:172.21.15.131:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-08T21:58:39.516 INFO:teuthology.orchestra.run.smithi131.stdout:setting min_mon_release = pacific 2024-05-08T21:58:39.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: set fsid to 06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T21:58:39.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-08T21:58:39.516 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:58:39.516 INFO:teuthology.orchestra.run.smithi131.stdout:Creating mon... 2024-05-08T21:58:40.227 INFO:teuthology.orchestra.run.smithi131.stdout:create mon.a on 2024-05-08T21:58:40.510 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-05-08T21:58:40.706 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-08T21:58:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-06f1a696-0d86-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-06f1a696-0d86-11ef-bc97-c7b262605968.target. 2024-05-08T21:58:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-06f1a696-0d86-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-06f1a696-0d86-11ef-bc97-c7b262605968.target. 2024-05-08T21:58:41.257 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mon.a 2024-05-08T21:58:41.257 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Failed to reset failed state of unit ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-08T21:58:41.454 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-06f1a696-0d86-11ef-bc97-c7b262605968.target.wants/ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-06f1a696-0d86-11ef-bc97-c7b262605968@.service. 2024-05-08T21:58:41.558 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:41 smithi131 systemd[1]: Starting Ceph mon.a for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T21:58:43.069 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T21:58:43.069 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout disabled 2024-05-08T21:58:43.086 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:43 smithi131 bash[36728]: 4928a25930b10890cac0367c3152d79bf1e571674c2052cfe16bc8707d8eb094 2024-05-08T21:58:43.086 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:43 smithi131 systemd[1]: Started Ceph mon.a for 06f1a696-0d86-11ef-bc97-c7b262605968. 2024-05-08T21:58:43.086 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:43 smithi131 ceph-mon[36765]: mkfs 06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T21:58:43.087 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T21:58:43.087 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-05-08T21:58:43.087 INFO:teuthology.orchestra.run.smithi131.stdout:firewalld.service is not enabled 2024-05-08T21:58:43.087 INFO:teuthology.orchestra.run.smithi131.stdout:Not possible to enable service . firewalld.service is not available 2024-05-08T21:58:43.087 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mon to start... 2024-05-08T21:58:43.087 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mon... 2024-05-08T21:58:43.309 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:43 smithi131 ceph-mon[36765]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T21:58:44.089 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout cluster: 2024-05-08T21:58:44.095 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout id: 06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T21:58:44.096 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-08T21:58:44.096 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-08T21:58:44.096 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout services: 2024-05-08T21:58:44.096 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.626347s) 2024-05-08T21:58:44.096 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-08T21:58:44.096 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-08T21:58:44.096 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-08T21:58:44.096 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout data: 2024-05-08T21:58:44.096 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-08T21:58:44.096 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-08T21:58:44.096 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-08T21:58:44.096 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout pgs: 2024-05-08T21:58:44.096 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-08T21:58:44.096 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.619+0000 7feb2a400700 1 Processor -- start 2024-05-08T21:58:44.097 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.620+0000 7feb2a400700 1 -- start start 2024-05-08T21:58:44.097 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.620+0000 7feb2a400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb24106780 0x7feb24106ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:44.097 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.620+0000 7feb2a400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb24107170 con 0x7feb24106780 2024-05-08T21:58:44.097 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.621+0000 7feb23400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb24106780 0x7feb24106ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:44.097 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.621+0000 7feb23400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb24106780 0x7feb24106ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42580/0 (socket says 172.21.15.131:42580) 2024-05-08T21:58:44.097 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.621+0000 7feb23400700 1 -- 172.21.15.131:0/3521904885 learned_addr learned my addr 172.21.15.131:0/3521904885 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:58:44.097 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.621+0000 7feb23400700 1 -- 172.21.15.131:0/3521904885 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb24107930 con 0x7feb24106780 2024-05-08T21:58:44.097 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.622+0000 7feb23400700 1 --2- 172.21.15.131:0/3521904885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb24106780 0x7feb24106ba0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7feb140098a0 tx=0x7feb14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85969924ade8469f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:44.097 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.622+0000 7feb22000700 1 -- 172.21.15.131:0/3521904885 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb14004160 con 0x7feb24106780 2024-05-08T21:58:44.097 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.622+0000 7feb22000700 1 -- 172.21.15.131:0/3521904885 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb140042c0 con 0x7feb24106780 2024-05-08T21:58:44.097 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.622+0000 7feb22000700 1 -- 172.21.15.131:0/3521904885 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb140045a0 con 0x7feb24106780 2024-05-08T21:58:44.097 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.622+0000 7feb2a400700 1 -- 172.21.15.131:0/3521904885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb24106780 msgr2=0x7feb24106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:44.097 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.622+0000 7feb2a400700 1 --2- 172.21.15.131:0/3521904885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb24106780 0x7feb24106ba0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7feb140098a0 tx=0x7feb14009bb0 comp rx=0 tx=0).stop 2024-05-08T21:58:44.097 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.623+0000 7feb2a400700 1 -- 172.21.15.131:0/3521904885 shutdown_connections 2024-05-08T21:58:44.098 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.623+0000 7feb2a400700 1 --2- 172.21.15.131:0/3521904885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb24106780 0x7feb24106ba0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:44.098 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.623+0000 7feb2a400700 1 -- 172.21.15.131:0/3521904885 >> 172.21.15.131:0/3521904885 conn(0x7feb24101d20 msgr2=0x7feb24104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:44.098 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.623+0000 7feb2a400700 1 -- 172.21.15.131:0/3521904885 shutdown_connections 2024-05-08T21:58:44.098 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.623+0000 7feb2a400700 1 -- 172.21.15.131:0/3521904885 wait complete. 2024-05-08T21:58:44.098 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.624+0000 7feb2a400700 1 Processor -- start 2024-05-08T21:58:44.098 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.624+0000 7feb2a400700 1 -- start start 2024-05-08T21:58:44.098 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.624+0000 7feb2a400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb24106780 0x7feb2419bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:44.098 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.624+0000 7feb2a400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2419c330 con 0x7feb24106780 2024-05-08T21:58:44.098 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.624+0000 7feb23400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb24106780 0x7feb2419bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:44.098 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.624+0000 7feb23400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb24106780 0x7feb2419bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42588/0 (socket says 172.21.15.131:42588) 2024-05-08T21:58:44.098 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.624+0000 7feb23400700 1 -- 172.21.15.131:0/1324281632 learned_addr learned my addr 172.21.15.131:0/1324281632 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:58:44.098 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.625+0000 7feb23400700 1 -- 172.21.15.131:0/1324281632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb14009550 con 0x7feb24106780 2024-05-08T21:58:44.098 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.625+0000 7feb23400700 1 --2- 172.21.15.131:0/1324281632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb24106780 0x7feb2419bdf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7feb14004000 tx=0x7feb14004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:44.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.625+0000 7feb1be00700 1 -- 172.21.15.131:0/1324281632 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb14011630 con 0x7feb24106780 2024-05-08T21:58:44.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.625+0000 7feb1be00700 1 -- 172.21.15.131:0/1324281632 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb14011790 con 0x7feb24106780 2024-05-08T21:58:44.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.625+0000 7feb2a400700 1 -- 172.21.15.131:0/1324281632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2419c530 con 0x7feb24106780 2024-05-08T21:58:44.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.625+0000 7feb1be00700 1 -- 172.21.15.131:0/1324281632 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb14011a30 con 0x7feb24106780 2024-05-08T21:58:44.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.625+0000 7feb2a400700 1 -- 172.21.15.131:0/1324281632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2419c9d0 con 0x7feb24106780 2024-05-08T21:58:44.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.625+0000 7feb1be00700 1 -- 172.21.15.131:0/1324281632 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7feb14022070 con 0x7feb24106780 2024-05-08T21:58:44.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.626+0000 7feb1be00700 1 -- 172.21.15.131:0/1324281632 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7feb1401c450 con 0x7feb24106780 2024-05-08T21:58:44.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.626+0000 7feb2a400700 1 -- 172.21.15.131:0/1324281632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb24195d50 con 0x7feb24106780 2024-05-08T21:58:44.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.629+0000 7feb1be00700 1 -- 172.21.15.131:0/1324281632 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7feb1401b450 con 0x7feb24106780 2024-05-08T21:58:44.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.702+0000 7feb2a400700 1 -- 172.21.15.131:0/1324281632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7feb24107f20 con 0x7feb24106780 2024-05-08T21:58:44.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.703+0000 7feb1be00700 1 -- 172.21.15.131:0/1324281632 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7feb1401b5f0 con 0x7feb24106780 2024-05-08T21:58:44.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.704+0000 7feb2a400700 1 -- 172.21.15.131:0/1324281632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb24106780 msgr2=0x7feb2419bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:44.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.704+0000 7feb2a400700 1 --2- 172.21.15.131:0/1324281632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb24106780 0x7feb2419bdf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7feb14004000 tx=0x7feb14004750 comp rx=0 tx=0).stop 2024-05-08T21:58:44.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.704+0000 7feb2a400700 1 -- 172.21.15.131:0/1324281632 shutdown_connections 2024-05-08T21:58:44.100 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.704+0000 7feb2a400700 1 --2- 172.21.15.131:0/1324281632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb24106780 0x7feb2419bdf0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:44.100 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.704+0000 7feb2a400700 1 -- 172.21.15.131:0/1324281632 >> 172.21.15.131:0/1324281632 conn(0x7feb24101d20 msgr2=0x7feb24192e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:44.100 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.704+0000 7feb2a400700 1 -- 172.21.15.131:0/1324281632 shutdown_connections 2024-05-08T21:58:44.100 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:43.704+0000 7feb2a400700 1 -- 172.21.15.131:0/1324281632 wait complete. 2024-05-08T21:58:44.100 INFO:teuthology.orchestra.run.smithi131.stdout:mon is available 2024-05-08T21:58:44.100 INFO:teuthology.orchestra.run.smithi131.stdout:Assimilating anything we can from ceph.conf... 2024-05-08T21:58:44.344 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:44 smithi131 ceph-mon[36765]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T21:58:44.344 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:44 smithi131 ceph-mon[36765]: monmap e1: 1 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T21:58:44.344 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:44 smithi131 ceph-mon[36765]: fsmap 2024-05-08T21:58:44.344 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:44 smithi131 ceph-mon[36765]: osdmap e1: 0 total, 0 up, 0 in 2024-05-08T21:58:44.344 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:44 smithi131 ceph-mon[36765]: mgrmap e1: no daemons active 2024-05-08T21:58:44.344 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:44 smithi131 ceph-mon[36765]: from='client.? 172.21.15.131:0/1324281632' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-08T21:58:44.975 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-08T21:58:44.975 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [global] 2024-05-08T21:58:44.975 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout fsid = 06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T21:58:44.975 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.131:3300,v1:172.21.15.131:6789] 2024-05-08T21:58:44.975 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-08T21:58:44.975 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-08T21:58:44.975 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-08T21:58:44.975 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-08T21:58:44.975 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-08T21:58:44.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-08T21:58:44.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-08T21:58:44.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-08T21:58:44.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [osd] 2024-05-08T21:58:44.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-08T21:58:44.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-08T21:58:44.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-08T21:58:44.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-08T21:58:44.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-08T21:58:44.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.535+0000 7f4d8a400700 1 Processor -- start 2024-05-08T21:58:44.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.535+0000 7f4d8a400700 1 -- start start 2024-05-08T21:58:44.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.535+0000 7f4d8a400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d84106780 0x7f4d84106ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:44.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.535+0000 7f4d8a400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d84107170 con 0x7f4d84106780 2024-05-08T21:58:44.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.536+0000 7f4d83400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d84106780 0x7f4d84106ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:44.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.536+0000 7f4d83400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d84106780 0x7f4d84106ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60072/0 (socket says 172.21.15.131:60072) 2024-05-08T21:58:44.977 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.536+0000 7f4d83400700 1 -- 172.21.15.131:0/2974709816 learned_addr learned my addr 172.21.15.131:0/2974709816 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:58:44.977 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.537+0000 7f4d83400700 1 -- 172.21.15.131:0/2974709816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d84107930 con 0x7f4d84106780 2024-05-08T21:58:44.977 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.537+0000 7f4d83400700 1 --2- 172.21.15.131:0/2974709816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d84106780 0x7f4d84106ba0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4d740098a0 tx=0x7f4d74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937f392b3d0a2bea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:44.977 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.537+0000 7f4d82000700 1 -- 172.21.15.131:0/2974709816 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d74004160 con 0x7f4d84106780 2024-05-08T21:58:44.977 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.537+0000 7f4d82000700 1 -- 172.21.15.131:0/2974709816 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d740042c0 con 0x7f4d84106780 2024-05-08T21:58:44.977 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.537+0000 7f4d82000700 1 -- 172.21.15.131:0/2974709816 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d740045a0 con 0x7f4d84106780 2024-05-08T21:58:44.977 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.538+0000 7f4d8a400700 1 -- 172.21.15.131:0/2974709816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d84106780 msgr2=0x7f4d84106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:44.977 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.538+0000 7f4d8a400700 1 --2- 172.21.15.131:0/2974709816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d84106780 0x7f4d84106ba0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4d740098a0 tx=0x7f4d74009bb0 comp rx=0 tx=0).stop 2024-05-08T21:58:44.977 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.538+0000 7f4d8a400700 1 -- 172.21.15.131:0/2974709816 shutdown_connections 2024-05-08T21:58:44.977 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.538+0000 7f4d8a400700 1 --2- 172.21.15.131:0/2974709816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d84106780 0x7f4d84106ba0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:44.977 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.538+0000 7f4d8a400700 1 -- 172.21.15.131:0/2974709816 >> 172.21.15.131:0/2974709816 conn(0x7f4d84101d20 msgr2=0x7f4d84104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:44.977 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.538+0000 7f4d8a400700 1 -- 172.21.15.131:0/2974709816 shutdown_connections 2024-05-08T21:58:44.977 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.538+0000 7f4d8a400700 1 -- 172.21.15.131:0/2974709816 wait complete. 2024-05-08T21:58:44.977 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.539+0000 7f4d8a400700 1 Processor -- start 2024-05-08T21:58:44.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.539+0000 7f4d8a400700 1 -- start start 2024-05-08T21:58:44.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.539+0000 7f4d8a400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d84106780 0x7f4d8419be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:44.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.539+0000 7f4d8a400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8419c3d0 con 0x7f4d84106780 2024-05-08T21:58:44.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.540+0000 7f4d83400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d84106780 0x7f4d8419be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:44.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.540+0000 7f4d83400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d84106780 0x7f4d8419be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60088/0 (socket says 172.21.15.131:60088) 2024-05-08T21:58:44.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.540+0000 7f4d83400700 1 -- 172.21.15.131:0/4054758681 learned_addr learned my addr 172.21.15.131:0/4054758681 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:58:44.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.540+0000 7f4d83400700 1 -- 172.21.15.131:0/4054758681 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d74009550 con 0x7f4d84106780 2024-05-08T21:58:44.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.540+0000 7f4d83400700 1 --2- 172.21.15.131:0/4054758681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d84106780 0x7f4d8419be90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4d74004000 tx=0x7f4d74004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:44.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.541+0000 7f4d7be00700 1 -- 172.21.15.131:0/4054758681 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d74011630 con 0x7f4d84106780 2024-05-08T21:58:44.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.541+0000 7f4d7be00700 1 -- 172.21.15.131:0/4054758681 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d74011790 con 0x7f4d84106780 2024-05-08T21:58:44.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.541+0000 7f4d8a400700 1 -- 172.21.15.131:0/4054758681 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d8419c5d0 con 0x7f4d84106780 2024-05-08T21:58:44.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.541+0000 7f4d7be00700 1 -- 172.21.15.131:0/4054758681 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d74011a30 con 0x7f4d84106780 2024-05-08T21:58:44.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.541+0000 7f4d8a400700 1 -- 172.21.15.131:0/4054758681 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d8419ca70 con 0x7f4d84106780 2024-05-08T21:58:44.979 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.541+0000 7f4d7be00700 1 -- 172.21.15.131:0/4054758681 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f4d74022070 con 0x7f4d84106780 2024-05-08T21:58:44.979 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.541+0000 7f4d7be00700 1 -- 172.21.15.131:0/4054758681 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4d7401c790 con 0x7f4d84106780 2024-05-08T21:58:44.979 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.542+0000 7f4d8a400700 1 -- 172.21.15.131:0/4054758681 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d84107ca0 con 0x7f4d84106780 2024-05-08T21:58:44.979 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.545+0000 7f4d7be00700 1 -- 172.21.15.131:0/4054758681 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f4d7401bcf0 con 0x7f4d84106780 2024-05-08T21:58:44.979 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.614+0000 7f4d8a400700 1 -- 172.21.15.131:0/4054758681 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f4d84107e30 con 0x7f4d84106780 2024-05-08T21:58:44.979 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.618+0000 7f4d7be00700 1 -- 172.21.15.131:0/4054758681 <== mon.0 v2:172.21.15.131:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4d7401be90 con 0x7f4d84106780 2024-05-08T21:58:44.979 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.619+0000 7f4d7be00700 1 -- 172.21.15.131:0/4054758681 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f4d74037610 con 0x7f4d84106780 2024-05-08T21:58:44.979 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.620+0000 7f4d8a400700 1 -- 172.21.15.131:0/4054758681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d84106780 msgr2=0x7f4d8419be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:44.979 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.620+0000 7f4d8a400700 1 --2- 172.21.15.131:0/4054758681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d84106780 0x7f4d8419be90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4d74004000 tx=0x7f4d74004750 comp rx=0 tx=0).stop 2024-05-08T21:58:44.979 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.620+0000 7f4d8a400700 1 -- 172.21.15.131:0/4054758681 shutdown_connections 2024-05-08T21:58:44.979 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.620+0000 7f4d8a400700 1 --2- 172.21.15.131:0/4054758681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d84106780 0x7f4d8419be90 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:44.979 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.620+0000 7f4d8a400700 1 -- 172.21.15.131:0/4054758681 >> 172.21.15.131:0/4054758681 conn(0x7f4d84101d20 msgr2=0x7f4d8407b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:44.980 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.620+0000 7f4d8a400700 1 -- 172.21.15.131:0/4054758681 shutdown_connections 2024-05-08T21:58:44.980 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:44.620+0000 7f4d8a400700 1 -- 172.21.15.131:0/4054758681 wait complete. 2024-05-08T21:58:44.980 INFO:teuthology.orchestra.run.smithi131.stdout:Generating new minimal ceph.conf... 2024-05-08T21:58:45.228 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:45 smithi131 ceph-mon[36765]: from='client.? 172.21.15.131:0/4054758681' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-08T21:58:45.228 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:45 smithi131 ceph-mon[36765]: from='client.? 172.21.15.131:0/4054758681' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-08T21:58:45.892 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.435+0000 7f7a73800700 1 Processor -- start 2024-05-08T21:58:45.892 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.435+0000 7f7a73800700 1 -- start start 2024-05-08T21:58:45.892 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.435+0000 7f7a73800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c07be30 0x7f7a6c07a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:45.892 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.435+0000 7f7a73800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c07a820 con 0x7f7a6c07be30 2024-05-08T21:58:45.892 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.436+0000 7f7a70e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c07be30 0x7f7a6c07a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:45.892 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.436+0000 7f7a70e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c07be30 0x7f7a6c07a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60104/0 (socket says 172.21.15.131:60104) 2024-05-08T21:58:45.892 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.436+0000 7f7a70e00700 1 -- 172.21.15.131:0/3770816552 learned_addr learned my addr 172.21.15.131:0/3770816552 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:58:45.892 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.436+0000 7f7a70e00700 1 -- 172.21.15.131:0/3770816552 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6c07a960 con 0x7f7a6c07be30 2024-05-08T21:58:45.892 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.436+0000 7f7a70e00700 1 --2- 172.21.15.131:0/3770816552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c07be30 0x7f7a6c07a2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7a600098a0 tx=0x7f7a60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd3336f68625fbb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:45.892 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.437+0000 7f7a6b400700 1 -- 172.21.15.131:0/3770816552 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a60004160 con 0x7f7a6c07be30 2024-05-08T21:58:45.892 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.437+0000 7f7a6b400700 1 -- 172.21.15.131:0/3770816552 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7a600042c0 con 0x7f7a6c07be30 2024-05-08T21:58:45.893 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.437+0000 7f7a6b400700 1 -- 172.21.15.131:0/3770816552 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a60011a00 con 0x7f7a6c07be30 2024-05-08T21:58:45.893 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.437+0000 7f7a73800700 1 -- 172.21.15.131:0/3770816552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c07be30 msgr2=0x7f7a6c07a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:45.893 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.437+0000 7f7a73800700 1 --2- 172.21.15.131:0/3770816552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c07be30 0x7f7a6c07a2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7a600098a0 tx=0x7f7a60009bb0 comp rx=0 tx=0).stop 2024-05-08T21:58:45.893 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.437+0000 7f7a73800700 1 -- 172.21.15.131:0/3770816552 shutdown_connections 2024-05-08T21:58:45.893 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.437+0000 7f7a73800700 1 --2- 172.21.15.131:0/3770816552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c07be30 0x7f7a6c07a2e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:45.893 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.437+0000 7f7a73800700 1 -- 172.21.15.131:0/3770816552 >> 172.21.15.131:0/3770816552 conn(0x7f7a6c101d20 msgr2=0x7f7a6c104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:45.893 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.437+0000 7f7a73800700 1 -- 172.21.15.131:0/3770816552 shutdown_connections 2024-05-08T21:58:45.893 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.438+0000 7f7a73800700 1 -- 172.21.15.131:0/3770816552 wait complete. 2024-05-08T21:58:45.893 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.438+0000 7f7a73800700 1 Processor -- start 2024-05-08T21:58:45.893 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.439+0000 7f7a73800700 1 -- start start 2024-05-08T21:58:45.893 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.439+0000 7f7a73800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c07be30 0x7f7a6c19c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:45.893 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.439+0000 7f7a73800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c19cb90 con 0x7f7a6c07be30 2024-05-08T21:58:45.893 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.439+0000 7f7a70e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c07be30 0x7f7a6c19c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:45.894 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.439+0000 7f7a70e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c07be30 0x7f7a6c19c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60114/0 (socket says 172.21.15.131:60114) 2024-05-08T21:58:45.894 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.439+0000 7f7a70e00700 1 -- 172.21.15.131:0/1337107318 learned_addr learned my addr 172.21.15.131:0/1337107318 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:58:45.894 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.439+0000 7f7a70e00700 1 -- 172.21.15.131:0/1337107318 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a60009550 con 0x7f7a6c07be30 2024-05-08T21:58:45.894 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.440+0000 7f7a70e00700 1 --2- 172.21.15.131:0/1337107318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c07be30 0x7f7a6c19c650 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009130 tx=0x7f7a60011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:45.894 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.440+0000 7f7a69600700 1 -- 172.21.15.131:0/1337107318 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a60003b70 con 0x7f7a6c07be30 2024-05-08T21:58:45.894 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.440+0000 7f7a73800700 1 -- 172.21.15.131:0/1337107318 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a6c19cd90 con 0x7f7a6c07be30 2024-05-08T21:58:45.894 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.440+0000 7f7a69600700 1 -- 172.21.15.131:0/1337107318 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7a60003cd0 con 0x7f7a6c07be30 2024-05-08T21:58:45.894 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.440+0000 7f7a69600700 1 -- 172.21.15.131:0/1337107318 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a600197d0 con 0x7f7a6c07be30 2024-05-08T21:58:45.894 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.440+0000 7f7a73800700 1 -- 172.21.15.131:0/1337107318 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a6c19d230 con 0x7f7a6c07be30 2024-05-08T21:58:45.894 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.440+0000 7f7a69600700 1 -- 172.21.15.131:0/1337107318 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f7a60019a10 con 0x7f7a6c07be30 2024-05-08T21:58:45.894 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.440+0000 7f7a69600700 1 -- 172.21.15.131:0/1337107318 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7a6000b720 con 0x7f7a6c07be30 2024-05-08T21:58:45.894 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.441+0000 7f7a73800700 1 -- 172.21.15.131:0/1337107318 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a6c10cab0 con 0x7f7a6c07be30 2024-05-08T21:58:45.894 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.444+0000 7f7a69600700 1 -- 172.21.15.131:0/1337107318 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f7a6002a060 con 0x7f7a6c07be30 2024-05-08T21:58:45.895 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.513+0000 7f7a73800700 1 -- 172.21.15.131:0/1337107318 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f7a6c10cc70 con 0x7f7a6c07be30 2024-05-08T21:58:45.895 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.514+0000 7f7a69600700 1 -- 172.21.15.131:0/1337107318 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7f7a60019e50 con 0x7f7a6c07be30 2024-05-08T21:58:45.895 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.515+0000 7f7a73800700 1 -- 172.21.15.131:0/1337107318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c07be30 msgr2=0x7f7a6c19c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:45.895 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.515+0000 7f7a73800700 1 --2- 172.21.15.131:0/1337107318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c07be30 0x7f7a6c19c650 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009130 tx=0x7f7a60011ad0 comp rx=0 tx=0).stop 2024-05-08T21:58:45.895 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.515+0000 7f7a73800700 1 -- 172.21.15.131:0/1337107318 shutdown_connections 2024-05-08T21:58:45.895 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.515+0000 7f7a73800700 1 --2- 172.21.15.131:0/1337107318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c07be30 0x7f7a6c19c650 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:45.895 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.515+0000 7f7a73800700 1 -- 172.21.15.131:0/1337107318 >> 172.21.15.131:0/1337107318 conn(0x7f7a6c101d20 msgr2=0x7f7a6c1028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:45.895 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.515+0000 7f7a73800700 1 -- 172.21.15.131:0/1337107318 shutdown_connections 2024-05-08T21:58:45.895 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:45.515+0000 7f7a73800700 1 -- 172.21.15.131:0/1337107318 wait complete. 2024-05-08T21:58:45.895 INFO:teuthology.orchestra.run.smithi131.stdout:Restarting the monitor... 2024-05-08T21:58:46.201 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:45 smithi131 systemd[1]: Stopping Ceph mon.a for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T21:58:46.201 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:46 smithi131 ceph-mon[36765]: from='client.? 172.21.15.131:0/1337107318' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:58:46.483 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:46 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mon-a[36742]: 2024-05-08T21:58:46.199+0000 7f3bda200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:58:46.483 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:46 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mon-a[36742]: 2024-05-08T21:58:46.199+0000 7f3bda200700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-08T21:58:46.483 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:46 smithi131 bash[37096]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mon-a 2024-05-08T21:58:47.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:46 smithi131 systemd[1]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mon.a.service: Succeeded. 2024-05-08T21:58:47.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:46 smithi131 systemd[1]: Stopped Ceph mon.a for 06f1a696-0d86-11ef-bc97-c7b262605968. 2024-05-08T21:58:47.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:46 smithi131 systemd[1]: Starting Ceph mon.a for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T21:58:47.224 INFO:teuthology.orchestra.run.smithi131.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-08T21:58:47.484 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 bash[37222]: e4dc3ef057c3ffa92ba40b33e0ef3b3547737309b2137282f04650d01f324506 2024-05-08T21:58:47.484 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 systemd[1]: Started Ceph mon.a for 06f1a696-0d86-11ef-bc97-c7b262605968. 2024-05-08T21:58:47.484 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: set uid:gid to 167:167 (ceph:ceph) 2024-05-08T21:58:47.484 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable), process ceph-mon, pid 7 2024-05-08T21:58:47.484 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: pidfile_write: ignore empty --pid-file 2024-05-08T21:58:47.484 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: load: jerasure load: lrc 2024-05-08T21:58:47.485 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: RocksDB version: 7.9.2 2024-05-08T21:58:47.485 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Git sha 0 2024-05-08T21:58:47.485 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Compile date 2024-05-08 17:55:50 2024-05-08T21:58:47.485 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: DB SUMMARY 2024-05-08T21:58:47.485 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: DB Session ID: 1U0XPBJ0IQE4NE1C7766 2024-05-08T21:58:47.485 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: CURRENT file: CURRENT 2024-05-08T21:58:47.485 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: IDENTITY file: IDENTITY 2024-05-08T21:58:47.485 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-08T21:58:47.485 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-08T21:58:47.485 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99453 ; 2024-05-08T21:58:47.485 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.error_if_exists: 0 2024-05-08T21:58:47.485 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.create_if_missing: 0 2024-05-08T21:58:47.485 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.paranoid_checks: 1 2024-05-08T21:58:47.486 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-08T21:58:47.486 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-08T21:58:47.486 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-08T21:58:47.486 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.env: 0x5568d09f1820 2024-05-08T21:58:47.486 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.fs: PosixFileSystem 2024-05-08T21:58:47.486 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.info_log: 0x5568f5d1f560 2024-05-08T21:58:47.486 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_file_opening_threads: 16 2024-05-08T21:58:47.486 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.statistics: (nil) 2024-05-08T21:58:47.486 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.use_fsync: 0 2024-05-08T21:58:47.486 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_log_file_size: 0 2024-05-08T21:58:47.486 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-08T21:58:47.486 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-08T21:58:47.486 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.keep_log_file_num: 1000 2024-05-08T21:58:47.486 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.recycle_log_file_num: 0 2024-05-08T21:58:47.487 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.allow_fallocate: 1 2024-05-08T21:58:47.487 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.allow_mmap_reads: 0 2024-05-08T21:58:47.487 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.allow_mmap_writes: 0 2024-05-08T21:58:47.487 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.use_direct_reads: 0 2024-05-08T21:58:47.487 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-08T21:58:47.487 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.create_missing_column_families: 0 2024-05-08T21:58:47.487 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.db_log_dir: 2024-05-08T21:58:47.487 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.wal_dir: 2024-05-08T21:58:47.487 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-08T21:58:47.487 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-08T21:58:47.487 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-08T21:58:47.487 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-08T21:58:47.487 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-08T21:58:47.487 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-08T21:58:47.487 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.advise_random_on_open: 1 2024-05-08T21:58:47.488 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.db_write_buffer_size: 0 2024-05-08T21:58:47.488 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.write_buffer_manager: 0x5568f4fb4320 2024-05-08T21:58:47.488 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-08T21:58:47.488 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-08T21:58:47.488 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-08T21:58:47.488 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.rate_limiter: (nil) 2024-05-08T21:58:47.488 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-08T21:58:47.488 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.wal_recovery_mode: 2 2024-05-08T21:58:47.488 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.enable_thread_tracking: 0 2024-05-08T21:58:47.488 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.enable_pipelined_write: 0 2024-05-08T21:58:47.488 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.unordered_write: 0 2024-05-08T21:58:47.488 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-08T21:58:47.488 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-08T21:58:47.488 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-08T21:58:47.488 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-08T21:58:47.489 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.row_cache: None 2024-05-08T21:58:47.489 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.wal_filter: None 2024-05-08T21:58:47.489 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-08T21:58:47.489 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.allow_ingest_behind: 0 2024-05-08T21:58:47.489 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.two_write_queues: 0 2024-05-08T21:58:47.489 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.manual_wal_flush: 0 2024-05-08T21:58:47.489 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.wal_compression: 0 2024-05-08T21:58:47.489 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.atomic_flush: 0 2024-05-08T21:58:47.489 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-08T21:58:47.489 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-08T21:58:47.489 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-08T21:58:47.489 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.log_readahead_size: 0 2024-05-08T21:58:47.489 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-08T21:58:47.489 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.best_efforts_recovery: 0 2024-05-08T21:58:47.489 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-08T21:58:47.489 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-08T21:58:47.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.allow_data_in_errors: 0 2024-05-08T21:58:47.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.db_host_id: __hostname__ 2024-05-08T21:58:47.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-08T21:58:47.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_background_jobs: 2 2024-05-08T21:58:47.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_background_compactions: -1 2024-05-08T21:58:47.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_subcompactions: 1 2024-05-08T21:58:47.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-08T21:58:47.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-08T21:58:47.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-08T21:58:47.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_total_wal_size: 0 2024-05-08T21:58:47.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-08T21:58:47.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-08T21:58:47.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-08T21:58:47.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-08T21:58:47.491 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_open_files: -1 2024-05-08T21:58:47.491 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.bytes_per_sync: 0 2024-05-08T21:58:47.491 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-08T21:58:47.491 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-08T21:58:47.491 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.compaction_readahead_size: 0 2024-05-08T21:58:47.491 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_background_flushes: -1 2024-05-08T21:58:47.491 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Compression algorithms supported: 2024-05-08T21:58:47.491 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-08T21:58:47.491 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: kZSTD supported: 0 2024-05-08T21:58:47.491 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: kXpressCompression supported: 0 2024-05-08T21:58:47.491 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: kLZ4HCCompression supported: 1 2024-05-08T21:58:47.491 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: kZlibCompression supported: 1 2024-05-08T21:58:47.491 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: kSnappyCompression supported: 1 2024-05-08T21:58:47.491 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: kLZ4Compression supported: 1 2024-05-08T21:58:47.492 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: kBZip2Compression supported: 0 2024-05-08T21:58:47.492 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-08T21:58:47.492 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-08T21:58:47.492 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-08T21:58:47.492 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-08T21:58:47.492 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-08T21:58:47.492 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.merge_operator: 2024-05-08T21:58:47.492 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.compaction_filter: None 2024-05-08T21:58:47.492 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.compaction_filter_factory: None 2024-05-08T21:58:47.492 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.sst_partitioner_factory: None 2024-05-08T21:58:47.492 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-08T21:58:47.492 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-08T21:58:47.492 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5568f5d1f660) 2024-05-08T21:58:47.492 INFO:journalctl@ceph.mon.a.smithi131.stdout: cache_index_and_filter_blocks: 1 2024-05-08T21:58:47.492 INFO:journalctl@ceph.mon.a.smithi131.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-08T21:58:47.493 INFO:journalctl@ceph.mon.a.smithi131.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-08T21:58:47.493 INFO:journalctl@ceph.mon.a.smithi131.stdout: pin_top_level_index_and_filter: 1 2024-05-08T21:58:47.493 INFO:journalctl@ceph.mon.a.smithi131.stdout: index_type: 0 2024-05-08T21:58:47.493 INFO:journalctl@ceph.mon.a.smithi131.stdout: data_block_index_type: 0 2024-05-08T21:58:47.493 INFO:journalctl@ceph.mon.a.smithi131.stdout: index_shortening: 1 2024-05-08T21:58:47.493 INFO:journalctl@ceph.mon.a.smithi131.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-08T21:58:47.493 INFO:journalctl@ceph.mon.a.smithi131.stdout: checksum: 4 2024-05-08T21:58:47.493 INFO:journalctl@ceph.mon.a.smithi131.stdout: no_block_cache: 0 2024-05-08T21:58:47.493 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_cache: 0x5568f50371f0 2024-05-08T21:58:47.493 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_cache_name: BinnedLRUCache 2024-05-08T21:58:47.493 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_cache_options: 2024-05-08T21:58:47.493 INFO:journalctl@ceph.mon.a.smithi131.stdout: capacity : 536870912 2024-05-08T21:58:47.493 INFO:journalctl@ceph.mon.a.smithi131.stdout: num_shard_bits : 4 2024-05-08T21:58:47.493 INFO:journalctl@ceph.mon.a.smithi131.stdout: strict_capacity_limit : 0 2024-05-08T21:58:47.493 INFO:journalctl@ceph.mon.a.smithi131.stdout: high_pri_pool_ratio: 0.000 2024-05-08T21:58:47.493 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_cache_compressed: (nil) 2024-05-08T21:58:47.494 INFO:journalctl@ceph.mon.a.smithi131.stdout: persistent_cache: (nil) 2024-05-08T21:58:47.494 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_size: 4096 2024-05-08T21:58:47.494 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_size_deviation: 10 2024-05-08T21:58:47.494 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_restart_interval: 16 2024-05-08T21:58:47.494 INFO:journalctl@ceph.mon.a.smithi131.stdout: index_block_restart_interval: 1 2024-05-08T21:58:47.494 INFO:journalctl@ceph.mon.a.smithi131.stdout: metadata_block_size: 4096 2024-05-08T21:58:47.494 INFO:journalctl@ceph.mon.a.smithi131.stdout: partition_filters: 0 2024-05-08T21:58:47.494 INFO:journalctl@ceph.mon.a.smithi131.stdout: use_delta_encoding: 1 2024-05-08T21:58:47.494 INFO:journalctl@ceph.mon.a.smithi131.stdout: filter_policy: bloomfilter 2024-05-08T21:58:47.494 INFO:journalctl@ceph.mon.a.smithi131.stdout: whole_key_filtering: 1 2024-05-08T21:58:47.494 INFO:journalctl@ceph.mon.a.smithi131.stdout: verify_compression: 0 2024-05-08T21:58:47.494 INFO:journalctl@ceph.mon.a.smithi131.stdout: read_amp_bytes_per_bit: 0 2024-05-08T21:58:47.494 INFO:journalctl@ceph.mon.a.smithi131.stdout: format_version: 5 2024-05-08T21:58:47.494 INFO:journalctl@ceph.mon.a.smithi131.stdout: enable_index_compression: 1 2024-05-08T21:58:47.495 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_align: 0 2024-05-08T21:58:47.495 INFO:journalctl@ceph.mon.a.smithi131.stdout: max_auto_readahead_size: 262144 2024-05-08T21:58:47.495 INFO:journalctl@ceph.mon.a.smithi131.stdout: prepopulate_block_cache: 0 2024-05-08T21:58:47.495 INFO:journalctl@ceph.mon.a.smithi131.stdout: initial_auto_readahead_size: 8192 2024-05-08T21:58:47.495 INFO:journalctl@ceph.mon.a.smithi131.stdout: num_file_reads_for_auto_readahead: 2 2024-05-08T21:58:47.495 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.write_buffer_size: 33554432 2024-05-08T21:58:47.495 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_write_buffer_number: 2 2024-05-08T21:58:47.495 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.compression: NoCompression 2024-05-08T21:58:47.495 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.bottommost_compression: Disabled 2024-05-08T21:58:47.495 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.prefix_extractor: nullptr 2024-05-08T21:58:47.495 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-08T21:58:47.495 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.num_levels: 7 2024-05-08T21:58:47.495 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-08T21:58:47.495 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-08T21:58:47.495 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-08T21:58:47.496 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-08T21:58:47.496 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-08T21:58:47.496 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-08T21:58:47.496 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-08T21:58:47.496 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-08T21:58:47.496 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-08T21:58:47.496 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-08T21:58:47.496 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-08T21:58:47.496 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-08T21:58:47.496 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-08T21:58:47.496 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.compression_opts.level: 32767 2024-05-08T21:58:47.496 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.compression_opts.strategy: 0 2024-05-08T21:58:47.496 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-08T21:58:47.496 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-08T21:58:47.496 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-08T21:58:47.496 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-08T21:58:47.497 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.compression_opts.enabled: false 2024-05-08T21:58:47.497 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-08T21:58:47.497 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-08T21:58:47.497 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-08T21:58:47.497 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-08T21:58:47.497 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.target_file_size_base: 67108864 2024-05-08T21:58:47.497 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-08T21:58:47.497 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-08T21:58:47.497 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-08T21:58:47.497 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-08T21:58:47.497 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-08T21:58:47.497 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-08T21:58:47.497 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-08T21:58:47.497 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-08T21:58:47.497 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-08T21:58:47.497 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-08T21:58:47.498 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-08T21:58:47.498 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-08T21:58:47.498 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-08T21:58:47.498 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-08T21:58:47.498 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.arena_block_size: 1048576 2024-05-08T21:58:47.498 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-08T21:58:47.498 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-08T21:58:47.498 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.disable_auto_compactions: 0 2024-05-08T21:58:47.498 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-08T21:58:47.498 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-08T21:58:47.498 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-08T21:58:47.498 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-08T21:58:47.498 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-08T21:58:47.498 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-08T21:58:47.499 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-08T21:58:47.499 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-08T21:58:47.499 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-08T21:58:47.499 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-08T21:58:47.499 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.table_properties_collectors: 2024-05-08T21:58:47.499 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.inplace_update_support: 0 2024-05-08T21:58:47.499 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-08T21:58:47.499 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-08T21:58:47.499 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-08T21:58:47.499 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-08T21:58:47.499 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.bloom_locality: 0 2024-05-08T21:58:47.499 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.max_successive_merges: 0 2024-05-08T21:58:47.499 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-08T21:58:47.499 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.paranoid_file_checks: 0 2024-05-08T21:58:47.499 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.force_consistency_checks: 1 2024-05-08T21:58:47.500 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.report_bg_io_stats: 0 2024-05-08T21:58:47.500 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.ttl: 2592000 2024-05-08T21:58:47.500 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-08T21:58:47.500 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-08T21:58:47.500 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-08T21:58:47.500 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.enable_blob_files: false 2024-05-08T21:58:47.500 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.min_blob_size: 0 2024-05-08T21:58:47.500 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.blob_file_size: 268435456 2024-05-08T21:58:47.500 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-08T21:58:47.500 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-08T21:58:47.500 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-08T21:58:47.500 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-08T21:58:47.500 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-08T21:58:47.500 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.blob_file_starting_level: 0 2024-05-08T21:58:47.500 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-08T21:58:47.501 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-08T21:58:47.501 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-08T21:58:47.501 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a8f43da1-a8ef-4233-b7de-df2cb0915fb4 2024-05-08T21:58:47.501 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715205527241599, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-08T21:58:47.501 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-08T21:58:47.501 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715205527244520, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94655, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 316, "table_properties": {"data_size": 92660, "index_size": 277, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13781, "raw_average_key_size": 50, "raw_value_size": 85133, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 275, "num_filter_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715205527, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a8f43da1-a8ef-4233-b7de-df2cb0915fb4", "db_session_id": "1U0XPBJ0IQE4NE1C7766", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-08T21:58:47.501 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715205527244711, "job": 1, "event": "recovery_finished"} 2024-05-08T21:58:47.501 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-08T21:58:47.501 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T21:58:47.501 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5568f50d4000 2024-05-08T21:58:47.501 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: DB pointer 0x5568f50c2000 2024-05-08T21:58:47.501 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T21:58:47.501 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T21:58:47.501 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** DB Stats ** 2024-05-08T21:58:47.502 INFO:journalctl@ceph.mon.a.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T21:58:47.502 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T21:58:47.502 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T21:58:47.502 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T21:58:47.502 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T21:58:47.502 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T21:58:47.502 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T21:58:47.502 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-05-08T21:58:47.502 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** Compaction Stats [default] ** 2024-05-08T21:58:47.502 INFO:journalctl@ceph.mon.a.smithi131.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T21:58:47.502 INFO:journalctl@ceph.mon.a.smithi131.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T21:58:47.502 INFO:journalctl@ceph.mon.a.smithi131.stdout: L0 2/0 94.27 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 49.8 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-08T21:58:47.503 INFO:journalctl@ceph.mon.a.smithi131.stdout: Sum 2/0 94.27 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 49.8 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-08T21:58:47.503 INFO:journalctl@ceph.mon.a.smithi131.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 49.8 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-08T21:58:47.503 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-05-08T21:58:47.503 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** Compaction Stats [default] ** 2024-05-08T21:58:47.503 INFO:journalctl@ceph.mon.a.smithi131.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T21:58:47.503 INFO:journalctl@ceph.mon.a.smithi131.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T21:58:47.503 INFO:journalctl@ceph.mon.a.smithi131.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 49.8 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-08T21:58:47.503 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-05-08T21:58:47.503 INFO:journalctl@ceph.mon.a.smithi131.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T21:58:47.503 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-05-08T21:58:47.503 INFO:journalctl@ceph.mon.a.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T21:58:47.503 INFO:journalctl@ceph.mon.a.smithi131.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T21:58:47.503 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T21:58:47.503 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T21:58:47.503 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T21:58:47.504 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T21:58:47.504 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative compaction: 0.00 GB write, 9.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T21:58:47.504 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval compaction: 0.00 GB write, 9.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T21:58:47.504 INFO:journalctl@ceph.mon.a.smithi131.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T21:58:47.504 INFO:journalctl@ceph.mon.a.smithi131.stdout: Block cache BinnedLRUCache@0x5568f50371f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-05-08T21:58:47.504 INFO:journalctl@ceph.mon.a.smithi131.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-05-08T21:58:47.504 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-05-08T21:58:47.504 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T21:58:47.504 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: starting mon.a rank 0 at public addrs [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] at bind addrs [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T21:58:47.504 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: mon.a@-1(???) e1 preinit fsid 06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T21:58:47.504 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: mon.a@-1(???).mds e1 new map 2024-05-08T21:58:47.504 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: mon.a@-1(???).mds e1 print_map 2024-05-08T21:58:47.504 INFO:journalctl@ceph.mon.a.smithi131.stdout: e1 2024-05-08T21:58:47.504 INFO:journalctl@ceph.mon.a.smithi131.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T21:58:47.505 INFO:journalctl@ceph.mon.a.smithi131.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T21:58:47.505 INFO:journalctl@ceph.mon.a.smithi131.stdout: legacy client fscid: -1 2024-05-08T21:58:47.505 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-05-08T21:58:47.505 INFO:journalctl@ceph.mon.a.smithi131.stdout: No filesystems configured 2024-05-08T21:58:47.505 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T21:58:47.505 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T21:58:47.505 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T21:58:47.505 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T21:58:47.505 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-08T21:58:47.505 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T21:58:47.505 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: monmap e1: 1 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T21:58:47.505 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: fsmap 2024-05-08T21:58:47.505 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: osdmap e1: 0 total, 0 up, 0 in 2024-05-08T21:58:47.505 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:47 smithi131 ceph-mon[37260]: mgrmap e1: no daemons active 2024-05-08T21:58:48.147 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.709+0000 7f774a600700 1 Processor -- start 2024-05-08T21:58:48.147 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.709+0000 7f774a600700 1 -- start start 2024-05-08T21:58:48.147 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.709+0000 7f774a600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f774407be30 0x7f774407a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:48.147 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.709+0000 7f774a600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774407a820 con 0x7f774407be30 2024-05-08T21:58:48.147 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.710+0000 7f7749200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f774407be30 0x7f774407a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:48.147 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.710+0000 7f7749200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f774407be30 0x7f774407a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60116/0 (socket says 172.21.15.131:60116) 2024-05-08T21:58:48.147 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.710+0000 7f7749200700 1 -- 172.21.15.131:0/299732298 learned_addr learned my addr 172.21.15.131:0/299732298 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:58:48.147 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.710+0000 7f7749200700 1 -- 172.21.15.131:0/299732298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f774407a960 con 0x7f774407be30 2024-05-08T21:58:48.147 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.711+0000 7f7749200700 1 --2- 172.21.15.131:0/299732298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f774407be30 0x7f774407a2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f77380098a0 tx=0x7f7738009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781001a2e578a584 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:48.147 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.711+0000 7f7743400700 1 -- 172.21.15.131:0/299732298 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7738004160 con 0x7f774407be30 2024-05-08T21:58:48.147 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.711+0000 7f7743400700 1 -- 172.21.15.131:0/299732298 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f77380042c0 con 0x7f774407be30 2024-05-08T21:58:48.148 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.711+0000 7f7743400700 1 -- 172.21.15.131:0/299732298 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7738011a00 con 0x7f774407be30 2024-05-08T21:58:48.148 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.712+0000 7f774a600700 1 -- 172.21.15.131:0/299732298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f774407be30 msgr2=0x7f774407a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:48.148 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.712+0000 7f774a600700 1 --2- 172.21.15.131:0/299732298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f774407be30 0x7f774407a2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f77380098a0 tx=0x7f7738009bb0 comp rx=0 tx=0).stop 2024-05-08T21:58:48.148 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.712+0000 7f774a600700 1 -- 172.21.15.131:0/299732298 shutdown_connections 2024-05-08T21:58:48.148 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.712+0000 7f774a600700 1 --2- 172.21.15.131:0/299732298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f774407be30 0x7f774407a2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:48.148 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.712+0000 7f774a600700 1 -- 172.21.15.131:0/299732298 >> 172.21.15.131:0/299732298 conn(0x7f7744101bc0 msgr2=0x7f7744103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:48.148 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.712+0000 7f774a600700 1 -- 172.21.15.131:0/299732298 shutdown_connections 2024-05-08T21:58:48.148 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.712+0000 7f774a600700 1 -- 172.21.15.131:0/299732298 wait complete. 2024-05-08T21:58:48.148 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.713+0000 7f774a600700 1 Processor -- start 2024-05-08T21:58:48.148 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.713+0000 7f774a600700 1 -- start start 2024-05-08T21:58:48.149 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.713+0000 7f774a600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f774407be30 0x7f774419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:48.149 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.713+0000 7f774a600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774419ca30 con 0x7f774407be30 2024-05-08T21:58:48.149 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.713+0000 7f7749200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f774407be30 0x7f774419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:48.149 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.713+0000 7f7749200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f774407be30 0x7f774419c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60118/0 (socket says 172.21.15.131:60118) 2024-05-08T21:58:48.149 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.713+0000 7f7749200700 1 -- 172.21.15.131:0/99846169 learned_addr learned my addr 172.21.15.131:0/99846169 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:58:48.149 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.714+0000 7f7749200700 1 -- 172.21.15.131:0/99846169 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7738009550 con 0x7f774407be30 2024-05-08T21:58:48.149 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.714+0000 7f7749200700 1 --2- 172.21.15.131:0/99846169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f774407be30 0x7f774419c4f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7738011a20 tx=0x7f7738011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:48.149 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.714+0000 7f7741600700 1 -- 172.21.15.131:0/99846169 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7738003af0 con 0x7f774407be30 2024-05-08T21:58:48.149 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.714+0000 7f7741600700 1 -- 172.21.15.131:0/99846169 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7738003c50 con 0x7f774407be30 2024-05-08T21:58:48.149 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.714+0000 7f7741600700 1 -- 172.21.15.131:0/99846169 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7738019760 con 0x7f774407be30 2024-05-08T21:58:48.149 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.714+0000 7f774a600700 1 -- 172.21.15.131:0/99846169 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f774419cc30 con 0x7f774407be30 2024-05-08T21:58:48.150 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.714+0000 7f774a600700 1 -- 172.21.15.131:0/99846169 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f774419d0d0 con 0x7f774407be30 2024-05-08T21:58:48.150 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.715+0000 7f7741600700 1 -- 172.21.15.131:0/99846169 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f773801a430 con 0x7f774407be30 2024-05-08T21:58:48.150 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.715+0000 7f7741600700 1 -- 172.21.15.131:0/99846169 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7738025b90 con 0x7f774407be30 2024-05-08T21:58:48.150 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.716+0000 7f774a600700 1 -- 172.21.15.131:0/99846169 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f774410c930 con 0x7f774407be30 2024-05-08T21:58:48.150 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.719+0000 7f7741600700 1 -- 172.21.15.131:0/99846169 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f773802a060 con 0x7f774407be30 2024-05-08T21:58:48.150 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.788+0000 7f774a600700 1 -- 172.21.15.131:0/99846169 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f774419f960 con 0x7f774407be30 2024-05-08T21:58:48.150 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.791+0000 7f7741600700 1 -- 172.21.15.131:0/99846169 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f773801a8d0 con 0x7f774407be30 2024-05-08T21:58:48.150 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.792+0000 7f774a600700 1 -- 172.21.15.131:0/99846169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f774407be30 msgr2=0x7f774419c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:48.150 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.792+0000 7f774a600700 1 --2- 172.21.15.131:0/99846169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f774407be30 0x7f774419c4f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7738011a20 tx=0x7f7738011b00 comp rx=0 tx=0).stop 2024-05-08T21:58:48.150 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.792+0000 7f774a600700 1 -- 172.21.15.131:0/99846169 shutdown_connections 2024-05-08T21:58:48.150 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.792+0000 7f774a600700 1 --2- 172.21.15.131:0/99846169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f774407be30 0x7f774419c4f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:48.150 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.792+0000 7f774a600700 1 -- 172.21.15.131:0/99846169 >> 172.21.15.131:0/99846169 conn(0x7f7744101bc0 msgr2=0x7f7744102750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:48.150 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.792+0000 7f774a600700 1 -- 172.21.15.131:0/99846169 shutdown_connections 2024-05-08T21:58:48.150 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:47.792+0000 7f774a600700 1 -- 172.21.15.131:0/99846169 wait complete. 2024-05-08T21:58:48.151 INFO:teuthology.orchestra.run.smithi131.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-08T21:58:48.176 INFO:teuthology.orchestra.run.smithi131.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-08T21:58:48.176 INFO:teuthology.orchestra.run.smithi131.stdout:Creating mgr... 2024-05-08T21:58:48.176 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-08T21:58:48.177 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-08T21:58:48.569 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mgr.x 2024-05-08T21:58:48.569 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Failed to reset failed state of unit ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-08T21:58:48.766 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-06f1a696-0d86-11ef-bc97-c7b262605968.target.wants/ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-06f1a696-0d86-11ef-bc97-c7b262605968@.service. 2024-05-08T21:58:48.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:48 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/99846169' entity='client.admin' 2024-05-08T21:58:48.892 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:48 smithi131 systemd[1]: Starting Ceph mgr.x for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T21:58:49.335 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T21:58:49.336 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout disabled 2024-05-08T21:58:49.351 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T21:58:49.351 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-05-08T21:58:49.351 INFO:teuthology.orchestra.run.smithi131.stdout:firewalld.service is not enabled 2024-05-08T21:58:49.351 INFO:teuthology.orchestra.run.smithi131.stdout:Not possible to enable service . firewalld.service is not available 2024-05-08T21:58:49.368 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T21:58:49.368 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout disabled 2024-05-08T21:58:49.383 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T21:58:49.383 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-05-08T21:58:49.384 INFO:teuthology.orchestra.run.smithi131.stdout:firewalld.service is not enabled 2024-05-08T21:58:49.384 INFO:teuthology.orchestra.run.smithi131.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-08T21:58:49.384 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mgr to start... 2024-05-08T21:58:49.384 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mgr... 2024-05-08T21:58:49.576 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:49 smithi131 bash[37497]: 756987c4fd85c4d4c560098dac16267d0279bf33fc2f0bf92552da45dd6ad2f4 2024-05-08T21:58:49.576 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:49 smithi131 systemd[1]: Started Ceph mgr.x for 06f1a696-0d86-11ef-bc97-c7b262605968. 2024-05-08T21:58:50.105 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:49 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2134176346' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T21:58:50.106 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:49 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:58:49.848+0000 7ff5d33e1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T21:58:50.106 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:49 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:58:49.983+0000 7ff5d33e1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T21:58:50.106 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:50 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:58:50.104+0000 7ff5d33e1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T21:58:50.300 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-08T21:58:50.300 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-05-08T21:58:50.300 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsid": "06f1a696-0d86-11ef-bc97-c7b262605968", 2024-05-08T21:58:50.300 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T21:58:50.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T21:58:50.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T21:58:50.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T21:58:50.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:58:50.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T21:58:50.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T21:58:50.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 0 2024-05-08T21:58:50.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:58:50.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T21:58:50.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T21:58:50.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:58:50.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-08T21:58:50.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T21:58:50.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:58:50.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-08T21:58:50.302 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T21:58:50.302 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:58:50.302 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T21:58:50.302 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:58:50.302 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T21:58:50.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T21:58:50.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T21:58:50.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T21:58:50.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T21:58:50.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T21:58:50.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:58:50.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T21:58:50.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T21:58:50.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T21:58:50.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T21:58:50.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T21:58:50.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T21:58:50.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T21:58:50.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T21:58:50.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T21:58:50.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:58:50.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T21:58:50.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:58:50.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T21:58:50.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T21:58:50.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:58:50.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T21:58:50.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T21:58:50.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T21:58:50.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T21:58:50.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T21:58:50.313 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T21:58:50.313 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T21:58:50.313 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:58:50.313 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T21:58:50.313 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:58:50.313 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T21:58:50.313 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:58:50.313 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T21:58:43.079588+0000", 2024-05-08T21:58:50.313 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T21:58:50.313 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:58:50.313 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T21:58:50.313 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-05-08T21:58:50.313 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.840+0000 7ffb8b200700 1 Processor -- start 2024-05-08T21:58:50.314 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.840+0000 7ffb8b200700 1 -- start start 2024-05-08T21:58:50.314 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.841+0000 7ffb8b200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb84106780 0x7ffb84106ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:50.314 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.841+0000 7ffb8b200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb84107170 con 0x7ffb84106780 2024-05-08T21:58:50.314 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.841+0000 7ffb83e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb84106780 0x7ffb84106ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:50.314 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.841+0000 7ffb83e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb84106780 0x7ffb84106ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60146/0 (socket says 172.21.15.131:60146) 2024-05-08T21:58:50.314 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.841+0000 7ffb83e00700 1 -- 172.21.15.131:0/3947823433 learned_addr learned my addr 172.21.15.131:0/3947823433 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:58:50.314 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.842+0000 7ffb83e00700 1 -- 172.21.15.131:0/3947823433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb84107930 con 0x7ffb84106780 2024-05-08T21:58:50.314 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.842+0000 7ffb83e00700 1 --2- 172.21.15.131:0/3947823433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb84106780 0x7ffb84106ba0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ffb740098a0 tx=0x7ffb74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fed135b6b858a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:50.314 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.842+0000 7ffb82a00700 1 -- 172.21.15.131:0/3947823433 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb74004160 con 0x7ffb84106780 2024-05-08T21:58:50.314 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.842+0000 7ffb82a00700 1 -- 172.21.15.131:0/3947823433 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffb740042c0 con 0x7ffb84106780 2024-05-08T21:58:50.314 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.842+0000 7ffb82a00700 1 -- 172.21.15.131:0/3947823433 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb74011a00 con 0x7ffb84106780 2024-05-08T21:58:50.314 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.843+0000 7ffb8b200700 1 -- 172.21.15.131:0/3947823433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb84106780 msgr2=0x7ffb84106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:50.315 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.843+0000 7ffb8b200700 1 --2- 172.21.15.131:0/3947823433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb84106780 0x7ffb84106ba0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ffb740098a0 tx=0x7ffb74009bb0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.315 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.843+0000 7ffb8b200700 1 -- 172.21.15.131:0/3947823433 shutdown_connections 2024-05-08T21:58:50.315 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.843+0000 7ffb8b200700 1 --2- 172.21.15.131:0/3947823433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb84106780 0x7ffb84106ba0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.315 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.843+0000 7ffb8b200700 1 -- 172.21.15.131:0/3947823433 >> 172.21.15.131:0/3947823433 conn(0x7ffb84101d20 msgr2=0x7ffb84104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:50.315 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.843+0000 7ffb8b200700 1 -- 172.21.15.131:0/3947823433 shutdown_connections 2024-05-08T21:58:50.315 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.843+0000 7ffb8b200700 1 -- 172.21.15.131:0/3947823433 wait complete. 2024-05-08T21:58:50.315 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.844+0000 7ffb8b200700 1 Processor -- start 2024-05-08T21:58:50.316 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.844+0000 7ffb8b200700 1 -- start start 2024-05-08T21:58:50.316 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.845+0000 7ffb8b200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb84106780 0x7ffb8419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:58:50.316 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.845+0000 7ffb8b200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8419cbd0 con 0x7ffb84106780 2024-05-08T21:58:50.316 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.845+0000 7ffb83e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb84106780 0x7ffb8419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:58:50.316 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.845+0000 7ffb83e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb84106780 0x7ffb8419c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60154/0 (socket says 172.21.15.131:60154) 2024-05-08T21:58:50.316 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.845+0000 7ffb83e00700 1 -- 172.21.15.131:0/2134176346 learned_addr learned my addr 172.21.15.131:0/2134176346 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:58:50.316 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.845+0000 7ffb83e00700 1 -- 172.21.15.131:0/2134176346 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb74009550 con 0x7ffb84106780 2024-05-08T21:58:50.316 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.845+0000 7ffb83e00700 1 --2- 172.21.15.131:0/2134176346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb84106780 0x7ffb8419c690 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009130 tx=0x7ffb74011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:58:50.316 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.846+0000 7ffb80c00700 1 -- 172.21.15.131:0/2134176346 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb74003ab0 con 0x7ffb84106780 2024-05-08T21:58:50.316 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.846+0000 7ffb80c00700 1 -- 172.21.15.131:0/2134176346 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffb74003c10 con 0x7ffb84106780 2024-05-08T21:58:50.316 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.846+0000 7ffb80c00700 1 -- 172.21.15.131:0/2134176346 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb740247d0 con 0x7ffb84106780 2024-05-08T21:58:50.317 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.846+0000 7ffb8b200700 1 -- 172.21.15.131:0/2134176346 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb8419cdd0 con 0x7ffb84106780 2024-05-08T21:58:50.317 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.846+0000 7ffb8b200700 1 -- 172.21.15.131:0/2134176346 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8419d270 con 0x7ffb84106780 2024-05-08T21:58:50.317 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.846+0000 7ffb80c00700 1 -- 172.21.15.131:0/2134176346 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7ffb74024d00 con 0x7ffb84106780 2024-05-08T21:58:50.317 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.846+0000 7ffb80c00700 1 -- 172.21.15.131:0/2134176346 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ffb740193c0 con 0x7ffb84106780 2024-05-08T21:58:50.317 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.847+0000 7ffb8b200700 1 -- 172.21.15.131:0/2134176346 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb84068010 con 0x7ffb84106780 2024-05-08T21:58:50.317 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.849+0000 7ffb80c00700 1 -- 172.21.15.131:0/2134176346 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7ffb7401abb0 con 0x7ffb84106780 2024-05-08T21:58:50.317 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.926+0000 7ffb8b200700 1 -- 172.21.15.131:0/2134176346 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ffb8410ab00 con 0x7ffb84106780 2024-05-08T21:58:50.317 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.926+0000 7ffb80c00700 1 -- 172.21.15.131:0/2134176346 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7ffb7401ad50 con 0x7ffb84106780 2024-05-08T21:58:50.318 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.927+0000 7ffb8b200700 1 -- 172.21.15.131:0/2134176346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb84106780 msgr2=0x7ffb8419c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:58:50.318 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.927+0000 7ffb8b200700 1 --2- 172.21.15.131:0/2134176346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb84106780 0x7ffb8419c690 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009130 tx=0x7ffb74011ad0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.318 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.927+0000 7ffb8b200700 1 -- 172.21.15.131:0/2134176346 shutdown_connections 2024-05-08T21:58:50.318 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.927+0000 7ffb8b200700 1 --2- 172.21.15.131:0/2134176346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb84106780 0x7ffb8419c690 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:58:50.318 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.927+0000 7ffb8b200700 1 -- 172.21.15.131:0/2134176346 >> 172.21.15.131:0/2134176346 conn(0x7ffb84101d20 msgr2=0x7ffb84102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:58:50.318 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.928+0000 7ffb8b200700 1 -- 172.21.15.131:0/2134176346 shutdown_connections 2024-05-08T21:58:50.318 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:49.928+0000 7ffb8b200700 1 -- 172.21.15.131:0/2134176346 wait complete. 2024-05-08T21:58:50.318 INFO:teuthology.orchestra.run.smithi131.stdout:mgr not available, waiting (1/15)... 2024-05-08T21:58:50.390 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:50 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:58:50.218+0000 7ff5d33e1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T21:58:51.390 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:51 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:58:51.089+0000 7ff5d33e1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:58:51.660 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:51 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:58:51.405+0000 7ff5d33e1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T21:58:51.660 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:51 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:58:51.518+0000 7ff5d33e1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T21:58:51.660 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:51 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:58:51.659+0000 7ff5d33e1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T21:58:52.025 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:51 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:58:51.855+0000 7ff5d33e1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T21:58:52.390 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:52 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:58:52.024+0000 7ff5d33e1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T21:58:52.391 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:52 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:58:52.134+0000 7ff5d33e1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T21:58:52.867 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:52 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:58:52.629+0000 7ff5d33e1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T21:58:53.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:52 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1499688076' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T21:58:53.529 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:53 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:58:53.153+0000 7ff5d33e1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T21:58:53.529 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:53 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:58:53.406+0000 7ff5d33e1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T21:58:53.890 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:53 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:58:53.528+0000 7ff5d33e1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T21:58:54.853 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:54 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:58:54.525+0000 7ff5d33e1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T21:58:54.853 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:54 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:58:54.652+0000 7ff5d33e1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T21:58:54.853 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:54 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:58:54.851+0000 7ff5d33e1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T21:58:56.140 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:55 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:58:55.680+0000 7ff5d33e1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T21:58:56.140 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:55 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:58:55.925+0000 7ff5d33e1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T21:58:56.640 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:56 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:58:56.183+0000 7ff5d33e1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T21:58:57.640 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:57 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:58:57.336+0000 7ff5d33e1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T21:58:58.074 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:57 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:58:57.708+0000 7ff5d33e1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:58:58.074 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:57 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:58:57.826+0000 7ff5d33e1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T21:58:58.369 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:58 smithi131 ceph-mon[37260]: Activating manager daemon x 2024-05-08T21:58:58.369 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:58 smithi131 ceph-mon[37260]: mgrmap e2: x(active, starting, since 0.00427745s) 2024-05-08T21:58:58.369 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:58 smithi131 ceph-mon[37260]: from='mgr.14100 172.21.15.131:0/1754718745' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T21:58:58.369 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:58 smithi131 ceph-mon[37260]: from='mgr.14100 172.21.15.131:0/1754718745' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T21:58:58.369 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:58 smithi131 ceph-mon[37260]: from='mgr.14100 172.21.15.131:0/1754718745' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T21:58:58.369 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:58 smithi131 ceph-mon[37260]: from='mgr.14100 172.21.15.131:0/1754718745' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:58:58.369 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:58 smithi131 ceph-mon[37260]: from='mgr.14100 172.21.15.131:0/1754718745' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T21:58:58.370 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:58 smithi131 ceph-mon[37260]: Manager daemon x is now available 2024-05-08T21:58:58.370 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:58:58 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:58:58.080+0000 7ff5d33e1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T21:58:59.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:59 smithi131 ceph-mon[37260]: from='mgr.14100 172.21.15.131:0/1754718745' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T21:58:59.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:59 smithi131 ceph-mon[37260]: from='mgr.14100 172.21.15.131:0/1754718745' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T21:58:59.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:59 smithi131 ceph-mon[37260]: from='mgr.14100 172.21.15.131:0/1754718745' entity='mgr.x' 2024-05-08T21:58:59.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:59 smithi131 ceph-mon[37260]: from='mgr.14100 172.21.15.131:0/1754718745' entity='mgr.x' 2024-05-08T21:58:59.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:59 smithi131 ceph-mon[37260]: from='mgr.14100 172.21.15.131:0/1754718745' entity='mgr.x' 2024-05-08T21:58:59.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:58:59 smithi131 ceph-mon[37260]: mgrmap e3: x(active, since 1.01019s) 2024-05-08T21:59:01.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:01 smithi131 ceph-mon[37260]: mgrmap e4: x(active, since 2s) 2024-05-08T21:59:02.011 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-08T21:59:02.013 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-05-08T21:59:02.013 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsid": "06f1a696-0d86-11ef-bc97-c7b262605968", 2024-05-08T21:59:02.013 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T21:59:02.014 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T21:59:02.014 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T21:59:02.014 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T21:59:02.014 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:59:02.014 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T21:59:02.014 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T21:59:02.014 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 0 2024-05-08T21:59:02.014 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:59:02.014 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T21:59:02.014 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T21:59:02.014 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:59:02.014 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-08T21:59:02.014 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T21:59:02.014 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:59:02.014 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-08T21:59:02.015 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T21:59:02.015 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:59:02.015 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T21:59:02.015 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:59:02.015 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T21:59:02.015 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T21:59:02.015 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T21:59:02.023 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T21:59:02.024 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T21:59:02.024 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T21:59:02.024 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:59:02.024 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T21:59:02.025 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T21:59:02.025 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T21:59:02.025 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T21:59:02.025 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T21:59:02.025 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T21:59:02.025 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T21:59:02.025 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T21:59:02.025 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T21:59:02.025 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:59:02.025 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T21:59:02.025 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:59:02.025 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T21:59:02.025 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T21:59:02.025 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:59:02.025 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T21:59:02.026 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T21:59:02.026 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T21:59:02.026 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T21:59:02.026 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T21:59:02.026 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T21:59:02.026 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T21:59:02.026 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:59:02.026 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T21:59:02.026 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:59:02.026 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T21:59:02.026 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:59:02.026 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T21:58:43.079588+0000", 2024-05-08T21:59:02.026 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T21:59:02.027 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:59:02.027 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T21:59:02.027 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-05-08T21:59:02.027 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.757+0000 7f097a000700 1 Processor -- start 2024-05-08T21:59:02.027 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.757+0000 7f097a000700 1 -- start start 2024-05-08T21:59:02.027 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.757+0000 7f097a000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0974106780 0x7f0974106ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:02.027 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.757+0000 7f097a000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0974107170 con 0x7f0974106780 2024-05-08T21:59:02.027 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.758+0000 7f0973400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0974106780 0x7f0974106ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:02.027 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.758+0000 7f0973400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0974106780 0x7f0974106ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60168/0 (socket says 172.21.15.131:60168) 2024-05-08T21:59:02.027 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.758+0000 7f0973400700 1 -- 172.21.15.131:0/424985027 learned_addr learned my addr 172.21.15.131:0/424985027 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:02.027 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.758+0000 7f0973400700 1 -- 172.21.15.131:0/424985027 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0974107930 con 0x7f0974106780 2024-05-08T21:59:02.027 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.758+0000 7f0973400700 1 --2- 172.21.15.131:0/424985027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0974106780 0x7f0974106ba0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f09640098a0 tx=0x7f0964009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470701a41ddeb4dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:02.027 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.758+0000 7f0972000700 1 -- 172.21.15.131:0/424985027 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0964004160 con 0x7f0974106780 2024-05-08T21:59:02.027 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.759+0000 7f0972000700 1 -- 172.21.15.131:0/424985027 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f09640042c0 con 0x7f0974106780 2024-05-08T21:59:02.028 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.759+0000 7f0972000700 1 -- 172.21.15.131:0/424985027 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0964011a00 con 0x7f0974106780 2024-05-08T21:59:02.028 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.759+0000 7f097a000700 1 -- 172.21.15.131:0/424985027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0974106780 msgr2=0x7f0974106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:02.028 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.759+0000 7f097a000700 1 --2- 172.21.15.131:0/424985027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0974106780 0x7f0974106ba0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f09640098a0 tx=0x7f0964009bb0 comp rx=0 tx=0).stop 2024-05-08T21:59:02.028 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.759+0000 7f097a000700 1 -- 172.21.15.131:0/424985027 shutdown_connections 2024-05-08T21:59:02.028 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.759+0000 7f097a000700 1 --2- 172.21.15.131:0/424985027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0974106780 0x7f0974106ba0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:02.028 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.759+0000 7f097a000700 1 -- 172.21.15.131:0/424985027 >> 172.21.15.131:0/424985027 conn(0x7f0974101d20 msgr2=0x7f0974104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:02.028 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.759+0000 7f097a000700 1 -- 172.21.15.131:0/424985027 shutdown_connections 2024-05-08T21:59:02.028 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.759+0000 7f097a000700 1 -- 172.21.15.131:0/424985027 wait complete. 2024-05-08T21:59:02.028 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.760+0000 7f097a000700 1 Processor -- start 2024-05-08T21:59:02.028 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.760+0000 7f097a000700 1 -- start start 2024-05-08T21:59:02.028 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.761+0000 7f097a000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0974106780 0x7f097407bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:02.028 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.761+0000 7f097a000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097407c060 con 0x7f0974106780 2024-05-08T21:59:02.028 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.761+0000 7f0973400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0974106780 0x7f097407bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:02.028 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.761+0000 7f0973400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0974106780 0x7f097407bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60178/0 (socket says 172.21.15.131:60178) 2024-05-08T21:59:02.029 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.761+0000 7f0973400700 1 -- 172.21.15.131:0/1499688076 learned_addr learned my addr 172.21.15.131:0/1499688076 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:02.029 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.761+0000 7f0973400700 1 -- 172.21.15.131:0/1499688076 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0964009550 con 0x7f0974106780 2024-05-08T21:59:02.029 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.762+0000 7f0973400700 1 --2- 172.21.15.131:0/1499688076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0974106780 0x7f097407bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0964009130 tx=0x7f0964011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:02.029 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.762+0000 7f096be00700 1 -- 172.21.15.131:0/1499688076 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0964003ab0 con 0x7f0974106780 2024-05-08T21:59:02.029 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.762+0000 7f096be00700 1 -- 172.21.15.131:0/1499688076 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0964003c10 con 0x7f0974106780 2024-05-08T21:59:02.029 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.762+0000 7f097a000700 1 -- 172.21.15.131:0/1499688076 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f097407a2a0 con 0x7f0974106780 2024-05-08T21:59:02.029 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.762+0000 7f096be00700 1 -- 172.21.15.131:0/1499688076 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09640247d0 con 0x7f0974106780 2024-05-08T21:59:02.029 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.762+0000 7f097a000700 1 -- 172.21.15.131:0/1499688076 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f097407a740 con 0x7f0974106780 2024-05-08T21:59:02.029 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.762+0000 7f096be00700 1 -- 172.21.15.131:0/1499688076 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f0964024d50 con 0x7f0974106780 2024-05-08T21:59:02.029 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.763+0000 7f096be00700 1 -- 172.21.15.131:0/1499688076 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f096401a430 con 0x7f0974106780 2024-05-08T21:59:02.029 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.763+0000 7f097a000700 1 -- 172.21.15.131:0/1499688076 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0974068010 con 0x7f0974106780 2024-05-08T21:59:02.029 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.766+0000 7f096be00700 1 -- 172.21.15.131:0/1499688076 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f096401abb0 con 0x7f0974106780 2024-05-08T21:59:02.029 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.844+0000 7f097a000700 1 -- 172.21.15.131:0/1499688076 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f097410ab60 con 0x7f0974106780 2024-05-08T21:59:02.029 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.844+0000 7f096be00700 1 -- 172.21.15.131:0/1499688076 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f0964032070 con 0x7f0974106780 2024-05-08T21:59:02.029 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.845+0000 7f097a000700 1 -- 172.21.15.131:0/1499688076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0974106780 msgr2=0x7f097407bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:02.030 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.845+0000 7f097a000700 1 --2- 172.21.15.131:0/1499688076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0974106780 0x7f097407bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0964009130 tx=0x7f0964011ad0 comp rx=0 tx=0).stop 2024-05-08T21:59:02.030 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.845+0000 7f097a000700 1 -- 172.21.15.131:0/1499688076 shutdown_connections 2024-05-08T21:59:02.030 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.845+0000 7f097a000700 1 --2- 172.21.15.131:0/1499688076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0974106780 0x7f097407bb20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:02.030 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.845+0000 7f097a000700 1 -- 172.21.15.131:0/1499688076 >> 172.21.15.131:0/1499688076 conn(0x7f0974101d20 msgr2=0x7f0974102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:02.030 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.846+0000 7f097a000700 1 -- 172.21.15.131:0/1499688076 shutdown_connections 2024-05-08T21:59:02.030 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:58:52.846+0000 7f097a000700 1 -- 172.21.15.131:0/1499688076 wait complete. 2024-05-08T21:59:02.030 INFO:teuthology.orchestra.run.smithi131.stdout:mgr not available, waiting (2/15)... 2024-05-08T21:59:05.065 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-08T21:59:05.065 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:04 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1585394360' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T21:59:05.069 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-05-08T21:59:05.069 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsid": "06f1a696-0d86-11ef-bc97-c7b262605968", 2024-05-08T21:59:05.069 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T21:59:05.069 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T21:59:05.069 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T21:59:05.069 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T21:59:05.069 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:59:05.069 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T21:59:05.069 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T21:59:05.069 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 0 2024-05-08T21:59:05.069 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:59:05.070 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T21:59:05.070 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T21:59:05.070 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:59:05.070 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_age": 17, 2024-05-08T21:59:05.070 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T21:59:05.070 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:59:05.070 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-08T21:59:05.070 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T21:59:05.070 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:59:05.070 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T21:59:05.070 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:59:05.070 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T21:59:05.070 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T21:59:05.071 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T21:59:05.071 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T21:59:05.071 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T21:59:05.071 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T21:59:05.071 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:59:05.071 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T21:59:05.071 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T21:59:05.071 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T21:59:05.080 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T21:59:05.080 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T21:59:05.080 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T21:59:05.080 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T21:59:05.080 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T21:59:05.080 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T21:59:05.080 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:59:05.080 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T21:59:05.080 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:59:05.080 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T21:59:05.080 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T21:59:05.081 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:59:05.081 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T21:59:05.081 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T21:59:05.081 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T21:59:05.081 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T21:59:05.081 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T21:59:05.081 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T21:59:05.081 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T21:59:05.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:59:05.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T21:59:05.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:59:05.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T21:59:05.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:59:05.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T21:58:43.079588+0000", 2024-05-08T21:59:05.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T21:59:05.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:59:05.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T21:59:05.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-05-08T21:59:05.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.440+0000 7f8e60a00700 1 Processor -- start 2024-05-08T21:59:05.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.440+0000 7f8e60a00700 1 -- start start 2024-05-08T21:59:05.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.440+0000 7f8e60a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e5c07be70 0x7f8e5c07a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:05.083 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.440+0000 7f8e60a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c07a8f0 con 0x7f8e5c07be70 2024-05-08T21:59:05.083 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.441+0000 7f8e5a800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e5c07be70 0x7f8e5c07a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:05.083 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.441+0000 7f8e5a800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e5c07be70 0x7f8e5c07a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46184/0 (socket says 172.21.15.131:46184) 2024-05-08T21:59:05.083 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.441+0000 7f8e5a800700 1 -- 172.21.15.131:0/2474267137 learned_addr learned my addr 172.21.15.131:0/2474267137 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:05.083 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.441+0000 7f8e5a800700 1 -- 172.21.15.131:0/2474267137 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5c07aa30 con 0x7f8e5c07be70 2024-05-08T21:59:05.083 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.441+0000 7f8e5a800700 1 --2- 172.21.15.131:0/2474267137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e5c07be70 0x7f8e5c07a320 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0098a0 tx=0x7f8e4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54220237b8f8a916 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:05.083 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.442+0000 7f8e59400700 1 -- 172.21.15.131:0/2474267137 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e4c004160 con 0x7f8e5c07be70 2024-05-08T21:59:05.083 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.442+0000 7f8e59400700 1 -- 172.21.15.131:0/2474267137 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8e4c0042c0 con 0x7f8e5c07be70 2024-05-08T21:59:05.083 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.442+0000 7f8e59400700 1 -- 172.21.15.131:0/2474267137 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e4c011a00 con 0x7f8e5c07be70 2024-05-08T21:59:05.083 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.442+0000 7f8e60a00700 1 -- 172.21.15.131:0/2474267137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e5c07be70 msgr2=0x7f8e5c07a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:05.083 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.442+0000 7f8e60a00700 1 --2- 172.21.15.131:0/2474267137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e5c07be70 0x7f8e5c07a320 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0098a0 tx=0x7f8e4c009bb0 comp rx=0 tx=0).stop 2024-05-08T21:59:05.083 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.442+0000 7f8e60a00700 1 -- 172.21.15.131:0/2474267137 shutdown_connections 2024-05-08T21:59:05.084 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.442+0000 7f8e60a00700 1 --2- 172.21.15.131:0/2474267137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e5c07be70 0x7f8e5c07a320 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:05.084 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.442+0000 7f8e60a00700 1 -- 172.21.15.131:0/2474267137 >> 172.21.15.131:0/2474267137 conn(0x7f8e5c101bc0 msgr2=0x7f8e5c103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:05.084 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.442+0000 7f8e60a00700 1 -- 172.21.15.131:0/2474267137 shutdown_connections 2024-05-08T21:59:05.084 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.442+0000 7f8e60a00700 1 -- 172.21.15.131:0/2474267137 wait complete. 2024-05-08T21:59:05.084 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.443+0000 7f8e60a00700 1 Processor -- start 2024-05-08T21:59:05.084 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.444+0000 7f8e60a00700 1 -- start start 2024-05-08T21:59:05.084 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.444+0000 7f8e60a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e5c07be70 0x7f8e5c19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:05.084 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.444+0000 7f8e60a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c19ca30 con 0x7f8e5c07be70 2024-05-08T21:59:05.084 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.444+0000 7f8e5a800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e5c07be70 0x7f8e5c19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:05.084 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.444+0000 7f8e5a800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e5c07be70 0x7f8e5c19c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46186/0 (socket says 172.21.15.131:46186) 2024-05-08T21:59:05.084 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.444+0000 7f8e5a800700 1 -- 172.21.15.131:0/1585394360 learned_addr learned my addr 172.21.15.131:0/1585394360 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:05.084 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.444+0000 7f8e5a800700 1 -- 172.21.15.131:0/1585394360 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c009550 con 0x7f8e5c07be70 2024-05-08T21:59:05.084 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.445+0000 7f8e5a800700 1 --2- 172.21.15.131:0/1585394360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e5c07be70 0x7f8e5c19c4f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c011a20 tx=0x7f8e4c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:05.085 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.445+0000 7f8e53400700 1 -- 172.21.15.131:0/1585394360 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e4c003af0 con 0x7f8e5c07be70 2024-05-08T21:59:05.085 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.445+0000 7f8e53400700 1 -- 172.21.15.131:0/1585394360 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8e4c003c50 con 0x7f8e5c07be70 2024-05-08T21:59:05.085 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.445+0000 7f8e60a00700 1 -- 172.21.15.131:0/1585394360 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5c19cc30 con 0x7f8e5c07be70 2024-05-08T21:59:05.085 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.445+0000 7f8e53400700 1 -- 172.21.15.131:0/1585394360 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e4c019760 con 0x7f8e5c07be70 2024-05-08T21:59:05.085 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.445+0000 7f8e60a00700 1 -- 172.21.15.131:0/1585394360 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5c19d0d0 con 0x7f8e5c07be70 2024-05-08T21:59:05.085 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.446+0000 7f8e60a00700 1 -- 172.21.15.131:0/1585394360 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e5c10c950 con 0x7f8e5c07be70 2024-05-08T21:59:05.085 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.446+0000 7f8e53400700 1 -- 172.21.15.131:0/1585394360 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7f8e4c01a430 con 0x7f8e5c07be70 2024-05-08T21:59:05.085 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.447+0000 7f8e53400700 1 --2- 172.21.15.131:0/1585394360 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f8e40038700 0x7f8e4003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:05.085 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.447+0000 7f8e53400700 1 -- 172.21.15.131:0/1585394360 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8e4c04c820 con 0x7f8e5c07be70 2024-05-08T21:59:05.085 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.447+0000 7f8e59e00700 1 --2- 172.21.15.131:0/1585394360 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f8e40038700 0x7f8e4003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:05.085 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.448+0000 7f8e59e00700 1 --2- 172.21.15.131:0/1585394360 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f8e40038700 0x7f8e4003abc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8e48006fd0 tx=0x7f8e48006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:05.085 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.453+0000 7f8e53400700 1 -- 172.21.15.131:0/1585394360 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8e4c02d9a0 con 0x7f8e5c07be70 2024-05-08T21:59:05.086 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.740+0000 7f8e60a00700 1 -- 172.21.15.131:0/1585394360 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f8e5c19f960 con 0x7f8e5c07be70 2024-05-08T21:59:05.086 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.740+0000 7f8e53400700 1 -- 172.21.15.131:0/1585394360 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f8e4c02d7c0 con 0x7f8e5c07be70 2024-05-08T21:59:05.086 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.742+0000 7f8e60a00700 1 -- 172.21.15.131:0/1585394360 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f8e40038700 msgr2=0x7f8e4003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:05.086 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.742+0000 7f8e60a00700 1 --2- 172.21.15.131:0/1585394360 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f8e40038700 0x7f8e4003abc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8e48006fd0 tx=0x7f8e48006e40 comp rx=0 tx=0).stop 2024-05-08T21:59:05.086 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.742+0000 7f8e60a00700 1 -- 172.21.15.131:0/1585394360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e5c07be70 msgr2=0x7f8e5c19c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:05.086 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.742+0000 7f8e60a00700 1 --2- 172.21.15.131:0/1585394360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e5c07be70 0x7f8e5c19c4f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c011a20 tx=0x7f8e4c011b00 comp rx=0 tx=0).stop 2024-05-08T21:59:05.086 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.742+0000 7f8e60a00700 1 -- 172.21.15.131:0/1585394360 shutdown_connections 2024-05-08T21:59:05.086 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.742+0000 7f8e60a00700 1 --2- 172.21.15.131:0/1585394360 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f8e40038700 0x7f8e4003abc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:05.086 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.742+0000 7f8e60a00700 1 --2- 172.21.15.131:0/1585394360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e5c07be70 0x7f8e5c19c4f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:05.086 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.743+0000 7f8e60a00700 1 -- 172.21.15.131:0/1585394360 >> 172.21.15.131:0/1585394360 conn(0x7f8e5c101bc0 msgr2=0x7f8e5c102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:05.087 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.743+0000 7f8e60a00700 1 -- 172.21.15.131:0/1585394360 shutdown_connections 2024-05-08T21:59:05.087 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:04.743+0000 7f8e60a00700 1 -- 172.21.15.131:0/1585394360 wait complete. 2024-05-08T21:59:05.087 INFO:teuthology.orchestra.run.smithi131.stdout:mgr is available 2024-05-08T21:59:06.081 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-08T21:59:06.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [global] 2024-05-08T21:59:06.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout fsid = 06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T21:59:06.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-08T21:59:06.082 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:05 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/894913415' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-08T21:59:06.089 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-08T21:59:06.089 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-08T21:59:06.089 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-08T21:59:06.089 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-08T21:59:06.090 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-08T21:59:06.090 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-08T21:59:06.090 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-08T21:59:06.090 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [osd] 2024-05-08T21:59:06.090 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-08T21:59:06.090 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-08T21:59:06.090 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-08T21:59:06.090 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-08T21:59:06.090 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-08T21:59:06.090 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.506+0000 7f9e6b000700 1 Processor -- start 2024-05-08T21:59:06.090 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.506+0000 7f9e6b000700 1 -- start start 2024-05-08T21:59:06.090 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.506+0000 7f9e6b000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e64106600 0x7f9e64106a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:06.090 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.506+0000 7f9e6b000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e64106ff0 con 0x7f9e64106600 2024-05-08T21:59:06.091 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.506+0000 7f9e69c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e64106600 0x7f9e64106a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:06.091 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.506+0000 7f9e69c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e64106600 0x7f9e64106a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46190/0 (socket says 172.21.15.131:46190) 2024-05-08T21:59:06.091 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.506+0000 7f9e69c00700 1 -- 172.21.15.131:0/1744445929 learned_addr learned my addr 172.21.15.131:0/1744445929 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:06.091 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.507+0000 7f9e69c00700 1 -- 172.21.15.131:0/1744445929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e641077b0 con 0x7f9e64106600 2024-05-08T21:59:06.091 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.507+0000 7f9e69c00700 1 --2- 172.21.15.131:0/1744445929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e64106600 0x7f9e64106a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9e580098a0 tx=0x7f9e58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b59a23a05ed52f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:06.091 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.507+0000 7f9e63e00700 1 -- 172.21.15.131:0/1744445929 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e58004160 con 0x7f9e64106600 2024-05-08T21:59:06.091 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.507+0000 7f9e63e00700 1 -- 172.21.15.131:0/1744445929 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9e580042c0 con 0x7f9e64106600 2024-05-08T21:59:06.091 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.507+0000 7f9e63e00700 1 -- 172.21.15.131:0/1744445929 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e58011a00 con 0x7f9e64106600 2024-05-08T21:59:06.091 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.508+0000 7f9e6b000700 1 -- 172.21.15.131:0/1744445929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e64106600 msgr2=0x7f9e64106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:06.091 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.508+0000 7f9e6b000700 1 --2- 172.21.15.131:0/1744445929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e64106600 0x7f9e64106a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9e580098a0 tx=0x7f9e58009bb0 comp rx=0 tx=0).stop 2024-05-08T21:59:06.091 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.508+0000 7f9e6b000700 1 -- 172.21.15.131:0/1744445929 shutdown_connections 2024-05-08T21:59:06.091 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.508+0000 7f9e6b000700 1 --2- 172.21.15.131:0/1744445929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e64106600 0x7f9e64106a20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:06.091 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.508+0000 7f9e6b000700 1 -- 172.21.15.131:0/1744445929 >> 172.21.15.131:0/1744445929 conn(0x7f9e64101bc0 msgr2=0x7f9e64103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:06.091 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.508+0000 7f9e6b000700 1 -- 172.21.15.131:0/1744445929 shutdown_connections 2024-05-08T21:59:06.092 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.508+0000 7f9e6b000700 1 -- 172.21.15.131:0/1744445929 wait complete. 2024-05-08T21:59:06.092 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.509+0000 7f9e6b000700 1 Processor -- start 2024-05-08T21:59:06.092 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.509+0000 7f9e6b000700 1 -- start start 2024-05-08T21:59:06.092 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.510+0000 7f9e6b000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e64106600 0x7f9e6419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:06.092 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.510+0000 7f9e6b000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6419ca90 con 0x7f9e64106600 2024-05-08T21:59:06.092 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.510+0000 7f9e69c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e64106600 0x7f9e6419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:06.092 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.510+0000 7f9e69c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e64106600 0x7f9e6419c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46204/0 (socket says 172.21.15.131:46204) 2024-05-08T21:59:06.092 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.510+0000 7f9e69c00700 1 -- 172.21.15.131:0/894913415 learned_addr learned my addr 172.21.15.131:0/894913415 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:06.092 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.510+0000 7f9e69c00700 1 -- 172.21.15.131:0/894913415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e58009550 con 0x7f9e64106600 2024-05-08T21:59:06.092 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.510+0000 7f9e69c00700 1 --2- 172.21.15.131:0/894913415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e64106600 0x7f9e6419c550 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9e58011710 tx=0x7f9e58011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:06.092 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.511+0000 7f9e62000700 1 -- 172.21.15.131:0/894913415 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e58003aa0 con 0x7f9e64106600 2024-05-08T21:59:06.093 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.511+0000 7f9e62000700 1 -- 172.21.15.131:0/894913415 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9e58003c00 con 0x7f9e64106600 2024-05-08T21:59:06.093 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.511+0000 7f9e6b000700 1 -- 172.21.15.131:0/894913415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e6419cc90 con 0x7f9e64106600 2024-05-08T21:59:06.093 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.511+0000 7f9e62000700 1 -- 172.21.15.131:0/894913415 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e58024800 con 0x7f9e64106600 2024-05-08T21:59:06.093 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.511+0000 7f9e6b000700 1 -- 172.21.15.131:0/894913415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6419d130 con 0x7f9e64106600 2024-05-08T21:59:06.093 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.512+0000 7f9e62000700 1 -- 172.21.15.131:0/894913415 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7f9e5801abd0 con 0x7f9e64106600 2024-05-08T21:59:06.093 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.512+0000 7f9e6b000700 1 -- 172.21.15.131:0/894913415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e6410a9e0 con 0x7f9e64106600 2024-05-08T21:59:06.093 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.512+0000 7f9e62000700 1 --2- 172.21.15.131:0/894913415 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f9e4c038700 0x7f9e4c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:06.093 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.512+0000 7f9e62000700 1 -- 172.21.15.131:0/894913415 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9e5804c200 con 0x7f9e64106600 2024-05-08T21:59:06.093 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.513+0000 7f9e69200700 1 --2- 172.21.15.131:0/894913415 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f9e4c038700 0x7f9e4c03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:06.093 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.513+0000 7f9e69200700 1 --2- 172.21.15.131:0/894913415 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f9e4c038700 0x7f9e4c03abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9e50006fd0 tx=0x7f9e50006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:06.093 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.517+0000 7f9e62000700 1 -- 172.21.15.131:0/894913415 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9e58024960 con 0x7f9e64106600 2024-05-08T21:59:06.094 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.717+0000 7f9e6b000700 1 -- 172.21.15.131:0/894913415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f9e6419f9c0 con 0x7f9e64106600 2024-05-08T21:59:06.094 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.718+0000 7f9e62000700 1 -- 172.21.15.131:0/894913415 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f9e5802c350 con 0x7f9e64106600 2024-05-08T21:59:06.094 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.721+0000 7f9e6b000700 1 -- 172.21.15.131:0/894913415 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f9e4c038700 msgr2=0x7f9e4c03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:06.094 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.721+0000 7f9e6b000700 1 --2- 172.21.15.131:0/894913415 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f9e4c038700 0x7f9e4c03abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9e50006fd0 tx=0x7f9e50006e40 comp rx=0 tx=0).stop 2024-05-08T21:59:06.094 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.721+0000 7f9e6b000700 1 -- 172.21.15.131:0/894913415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e64106600 msgr2=0x7f9e6419c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:06.094 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.721+0000 7f9e6b000700 1 --2- 172.21.15.131:0/894913415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e64106600 0x7f9e6419c550 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9e58011710 tx=0x7f9e58011ad0 comp rx=0 tx=0).stop 2024-05-08T21:59:06.094 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.721+0000 7f9e6b000700 1 -- 172.21.15.131:0/894913415 shutdown_connections 2024-05-08T21:59:06.094 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.721+0000 7f9e6b000700 1 --2- 172.21.15.131:0/894913415 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f9e4c038700 0x7f9e4c03abc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:06.094 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.721+0000 7f9e6b000700 1 --2- 172.21.15.131:0/894913415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e64106600 0x7f9e6419c550 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:06.094 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.721+0000 7f9e6b000700 1 -- 172.21.15.131:0/894913415 >> 172.21.15.131:0/894913415 conn(0x7f9e64101bc0 msgr2=0x7f9e641026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:06.094 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.721+0000 7f9e6b000700 1 -- 172.21.15.131:0/894913415 shutdown_connections 2024-05-08T21:59:06.094 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:05.721+0000 7f9e6b000700 1 -- 172.21.15.131:0/894913415 wait complete. 2024-05-08T21:59:06.095 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling cephadm module... 2024-05-08T21:59:07.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:06 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/4141610217' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-08T21:59:07.141 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:06 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: ignoring --setuser ceph since I am not root 2024-05-08T21:59:07.141 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:06 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: ignoring --setgroup ceph since I am not root 2024-05-08T21:59:07.197 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.492+0000 7f5dccc00700 1 Processor -- start 2024-05-08T21:59:07.197 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.492+0000 7f5dccc00700 1 -- start start 2024-05-08T21:59:07.197 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.492+0000 7f5dccc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5dc807be70 0x7f5dc807a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:07.197 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.492+0000 7f5dccc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc807a8f0 con 0x7f5dc807be70 2024-05-08T21:59:07.197 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.493+0000 7f5dc7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5dc807be70 0x7f5dc807a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:07.197 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.493+0000 7f5dc7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5dc807be70 0x7f5dc807a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46220/0 (socket says 172.21.15.131:46220) 2024-05-08T21:59:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.493+0000 7f5dc7400700 1 -- 172.21.15.131:0/1407807577 learned_addr learned my addr 172.21.15.131:0/1407807577 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.493+0000 7f5dc7400700 1 -- 172.21.15.131:0/1407807577 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc807aa30 con 0x7f5dc807be70 2024-05-08T21:59:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.494+0000 7f5dc7400700 1 --2- 172.21.15.131:0/1407807577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5dc807be70 0x7f5dc807a320 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5db80098a0 tx=0x7f5db8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4efc29abde6fcb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.494+0000 7f5dc6000700 1 -- 172.21.15.131:0/1407807577 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5db8004160 con 0x7f5dc807be70 2024-05-08T21:59:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.494+0000 7f5dc6000700 1 -- 172.21.15.131:0/1407807577 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5db80042c0 con 0x7f5dc807be70 2024-05-08T21:59:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.494+0000 7f5dc6000700 1 -- 172.21.15.131:0/1407807577 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5db8011a00 con 0x7f5dc807be70 2024-05-08T21:59:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.494+0000 7f5dccc00700 1 -- 172.21.15.131:0/1407807577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5dc807be70 msgr2=0x7f5dc807a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.494+0000 7f5dccc00700 1 --2- 172.21.15.131:0/1407807577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5dc807be70 0x7f5dc807a320 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5db80098a0 tx=0x7f5db8009bb0 comp rx=0 tx=0).stop 2024-05-08T21:59:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.495+0000 7f5dccc00700 1 -- 172.21.15.131:0/1407807577 shutdown_connections 2024-05-08T21:59:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.495+0000 7f5dccc00700 1 --2- 172.21.15.131:0/1407807577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5dc807be70 0x7f5dc807a320 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.495+0000 7f5dccc00700 1 -- 172.21.15.131:0/1407807577 >> 172.21.15.131:0/1407807577 conn(0x7f5dc8101bc0 msgr2=0x7f5dc8103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.495+0000 7f5dccc00700 1 -- 172.21.15.131:0/1407807577 shutdown_connections 2024-05-08T21:59:07.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.495+0000 7f5dccc00700 1 -- 172.21.15.131:0/1407807577 wait complete. 2024-05-08T21:59:07.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.496+0000 7f5dccc00700 1 Processor -- start 2024-05-08T21:59:07.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.496+0000 7f5dccc00700 1 -- start start 2024-05-08T21:59:07.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.496+0000 7f5dccc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5dc807be70 0x7f5dc819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:07.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.496+0000 7f5dccc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc819ca30 con 0x7f5dc807be70 2024-05-08T21:59:07.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.496+0000 7f5dc7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5dc807be70 0x7f5dc819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:07.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.496+0000 7f5dc7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5dc807be70 0x7f5dc819c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46236/0 (socket says 172.21.15.131:46236) 2024-05-08T21:59:07.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.496+0000 7f5dc7400700 1 -- 172.21.15.131:0/4141610217 learned_addr learned my addr 172.21.15.131:0/4141610217 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:07.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.497+0000 7f5dc7400700 1 -- 172.21.15.131:0/4141610217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db8009550 con 0x7f5dc807be70 2024-05-08T21:59:07.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.497+0000 7f5dc7400700 1 --2- 172.21.15.131:0/4141610217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5dc807be70 0x7f5dc819c4f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5db8011a20 tx=0x7f5db8011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:07.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.497+0000 7f5dbfe00700 1 -- 172.21.15.131:0/4141610217 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5db8003af0 con 0x7f5dc807be70 2024-05-08T21:59:07.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.497+0000 7f5dbfe00700 1 -- 172.21.15.131:0/4141610217 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5db8003c50 con 0x7f5dc807be70 2024-05-08T21:59:07.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.497+0000 7f5dccc00700 1 -- 172.21.15.131:0/4141610217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc819cc30 con 0x7f5dc807be70 2024-05-08T21:59:07.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.497+0000 7f5dbfe00700 1 -- 172.21.15.131:0/4141610217 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5db8019760 con 0x7f5dc807be70 2024-05-08T21:59:07.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.497+0000 7f5dccc00700 1 -- 172.21.15.131:0/4141610217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc819d0d0 con 0x7f5dc807be70 2024-05-08T21:59:07.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.498+0000 7f5dbfe00700 1 -- 172.21.15.131:0/4141610217 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7f5db801a430 con 0x7f5dc807be70 2024-05-08T21:59:07.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.498+0000 7f5dccc00700 1 -- 172.21.15.131:0/4141610217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc810c950 con 0x7f5dc807be70 2024-05-08T21:59:07.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.499+0000 7f5dbfe00700 1 --2- 172.21.15.131:0/4141610217 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f5dac038700 0x7f5dac03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:07.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.499+0000 7f5dbfe00700 1 -- 172.21.15.131:0/4141610217 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5db804c820 con 0x7f5dc807be70 2024-05-08T21:59:07.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.499+0000 7f5dc6a00700 1 --2- 172.21.15.131:0/4141610217 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f5dac038700 0x7f5dac03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:07.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.499+0000 7f5dc6a00700 1 --2- 172.21.15.131:0/4141610217 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f5dac038700 0x7f5dac03abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5db4006fd0 tx=0x7f5db4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:07.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.504+0000 7f5dbfe00700 1 -- 172.21.15.131:0/4141610217 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5db802d9a0 con 0x7f5dc807be70 2024-05-08T21:59:07.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.762+0000 7f5dccc00700 1 -- 172.21.15.131:0/4141610217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f5dc819f960 con 0x7f5dc807be70 2024-05-08T21:59:07.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.797+0000 7f5dbfe00700 1 -- 172.21.15.131:0/4141610217 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f5db802bd00 con 0x7f5dc807be70 2024-05-08T21:59:07.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.799+0000 7f5dbfe00700 1 -- 172.21.15.131:0/4141610217 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f5db802dd90 con 0x7f5dc807be70 2024-05-08T21:59:07.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.802+0000 7f5dccc00700 1 -- 172.21.15.131:0/4141610217 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f5dac038700 msgr2=0x7f5dac03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:07.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.802+0000 7f5dccc00700 1 --2- 172.21.15.131:0/4141610217 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f5dac038700 0x7f5dac03abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5db4006fd0 tx=0x7f5db4006e40 comp rx=0 tx=0).stop 2024-05-08T21:59:07.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.802+0000 7f5dccc00700 1 -- 172.21.15.131:0/4141610217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5dc807be70 msgr2=0x7f5dc819c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:07.201 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.802+0000 7f5dccc00700 1 --2- 172.21.15.131:0/4141610217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5dc807be70 0x7f5dc819c4f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5db8011a20 tx=0x7f5db8011b00 comp rx=0 tx=0).stop 2024-05-08T21:59:07.201 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.802+0000 7f5dccc00700 1 -- 172.21.15.131:0/4141610217 shutdown_connections 2024-05-08T21:59:07.201 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.802+0000 7f5dccc00700 1 --2- 172.21.15.131:0/4141610217 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f5dac038700 0x7f5dac03abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:07.201 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.802+0000 7f5dccc00700 1 --2- 172.21.15.131:0/4141610217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5dc807be70 0x7f5dc819c4f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:07.201 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.802+0000 7f5dccc00700 1 -- 172.21.15.131:0/4141610217 >> 172.21.15.131:0/4141610217 conn(0x7f5dc8101bc0 msgr2=0x7f5dc8102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:07.201 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.802+0000 7f5dccc00700 1 -- 172.21.15.131:0/4141610217 shutdown_connections 2024-05-08T21:59:07.201 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:06.802+0000 7f5dccc00700 1 -- 172.21.15.131:0/4141610217 wait complete. 2024-05-08T21:59:07.545 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:07 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:07.418+0000 7fe4c2a69200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T21:59:07.798 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:07 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:07.544+0000 7fe4c2a69200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T21:59:07.798 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:07 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:07.663+0000 7fe4c2a69200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T21:59:07.798 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:07 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:07.776+0000 7fe4c2a69200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T21:59:08.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:07 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/4141610217' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-08T21:59:08.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:07 smithi131 ceph-mon[37260]: mgrmap e5: x(active, since 8s) 2024-05-08T21:59:08.297 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-05-08T21:59:08.297 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-08T21:59:08.297 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T21:59:08.298 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-08T21:59:08.298 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-08T21:59:08.298 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-05-08T21:59:08.298 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.620+0000 7f0be9200700 1 Processor -- start 2024-05-08T21:59:08.298 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.621+0000 7f0be9200700 1 -- start start 2024-05-08T21:59:08.298 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.621+0000 7f0be9200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0be4106780 0x7f0be4106ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:08.298 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.621+0000 7f0be9200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be4107170 con 0x7f0be4106780 2024-05-08T21:59:08.298 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.621+0000 7f0be1e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0be4106780 0x7f0be4106ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:08.298 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.621+0000 7f0be1e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0be4106780 0x7f0be4106ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46262/0 (socket says 172.21.15.131:46262) 2024-05-08T21:59:08.298 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.621+0000 7f0be1e00700 1 -- 172.21.15.131:0/966533124 learned_addr learned my addr 172.21.15.131:0/966533124 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:08.298 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.622+0000 7f0be1e00700 1 -- 172.21.15.131:0/966533124 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be4107930 con 0x7f0be4106780 2024-05-08T21:59:08.298 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.622+0000 7f0be1e00700 1 --2- 172.21.15.131:0/966533124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0be4106780 0x7f0be4106ba0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0bd40098a0 tx=0x7f0bd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50177ae02aab7d80 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:08.299 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.622+0000 7f0be0a00700 1 -- 172.21.15.131:0/966533124 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bd4004160 con 0x7f0be4106780 2024-05-08T21:59:08.299 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.622+0000 7f0be0a00700 1 -- 172.21.15.131:0/966533124 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0bd40042c0 con 0x7f0be4106780 2024-05-08T21:59:08.299 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.622+0000 7f0be0a00700 1 -- 172.21.15.131:0/966533124 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bd4011a00 con 0x7f0be4106780 2024-05-08T21:59:08.299 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.623+0000 7f0be9200700 1 -- 172.21.15.131:0/966533124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0be4106780 msgr2=0x7f0be4106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:08.299 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.623+0000 7f0be9200700 1 --2- 172.21.15.131:0/966533124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0be4106780 0x7f0be4106ba0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0bd40098a0 tx=0x7f0bd4009bb0 comp rx=0 tx=0).stop 2024-05-08T21:59:08.299 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.623+0000 7f0be9200700 1 -- 172.21.15.131:0/966533124 shutdown_connections 2024-05-08T21:59:08.299 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.623+0000 7f0be9200700 1 --2- 172.21.15.131:0/966533124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0be4106780 0x7f0be4106ba0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:08.299 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.623+0000 7f0be9200700 1 -- 172.21.15.131:0/966533124 >> 172.21.15.131:0/966533124 conn(0x7f0be4101d20 msgr2=0x7f0be4104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:08.299 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.623+0000 7f0be9200700 1 -- 172.21.15.131:0/966533124 shutdown_connections 2024-05-08T21:59:08.299 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.623+0000 7f0be9200700 1 -- 172.21.15.131:0/966533124 wait complete. 2024-05-08T21:59:08.299 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.624+0000 7f0be9200700 1 Processor -- start 2024-05-08T21:59:08.299 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.624+0000 7f0be9200700 1 -- start start 2024-05-08T21:59:08.299 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.625+0000 7f0be9200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0be4106780 0x7f0be419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:08.299 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.625+0000 7f0be9200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be419cbd0 con 0x7f0be4106780 2024-05-08T21:59:08.300 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.625+0000 7f0be1e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0be4106780 0x7f0be419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:08.300 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.625+0000 7f0be1e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0be4106780 0x7f0be419c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46276/0 (socket says 172.21.15.131:46276) 2024-05-08T21:59:08.300 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.625+0000 7f0be1e00700 1 -- 172.21.15.131:0/3804791490 learned_addr learned my addr 172.21.15.131:0/3804791490 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:08.300 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.625+0000 7f0be1e00700 1 -- 172.21.15.131:0/3804791490 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd4009550 con 0x7f0be4106780 2024-05-08T21:59:08.300 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.625+0000 7f0be1e00700 1 --2- 172.21.15.131:0/3804791490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0be4106780 0x7f0be419c690 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009130 tx=0x7f0bd4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:08.300 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.626+0000 7f0bdaa00700 1 -- 172.21.15.131:0/3804791490 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bd4003ab0 con 0x7f0be4106780 2024-05-08T21:59:08.300 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.626+0000 7f0bdaa00700 1 -- 172.21.15.131:0/3804791490 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0bd4003c10 con 0x7f0be4106780 2024-05-08T21:59:08.300 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.626+0000 7f0be9200700 1 -- 172.21.15.131:0/3804791490 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be419cdd0 con 0x7f0be4106780 2024-05-08T21:59:08.300 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.626+0000 7f0bdaa00700 1 -- 172.21.15.131:0/3804791490 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bd40247d0 con 0x7f0be4106780 2024-05-08T21:59:08.300 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.626+0000 7f0be9200700 1 -- 172.21.15.131:0/3804791490 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be419d270 con 0x7f0be4106780 2024-05-08T21:59:08.300 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.627+0000 7f0bdaa00700 1 -- 172.21.15.131:0/3804791490 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f0bd402d3f0 con 0x7f0be4106780 2024-05-08T21:59:08.300 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.627+0000 7f0be9200700 1 -- 172.21.15.131:0/3804791490 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be410ab60 con 0x7f0be4106780 2024-05-08T21:59:08.300 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.628+0000 7f0bdaa00700 1 --2- 172.21.15.131:0/3804791490 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f0bc8038700 0x7f0bc803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:08.300 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.628+0000 7f0bdaa00700 1 -- 172.21.15.131:0/3804791490 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0bd404c520 con 0x7f0be4106780 2024-05-08T21:59:08.300 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.628+0000 7f0be1400700 1 -- 172.21.15.131:0/3804791490 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f0bc8038700 msgr2=0x7f0bc803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/1564023164 2024-05-08T21:59:08.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.628+0000 7f0be1400700 1 --2- 172.21.15.131:0/3804791490 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f0bc8038700 0x7f0bc803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T21:59:08.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.633+0000 7f0bdaa00700 1 -- 172.21.15.131:0/3804791490 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0bd4010740 con 0x7f0be4106780 2024-05-08T21:59:08.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.829+0000 7f0be1400700 1 -- 172.21.15.131:0/3804791490 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f0bc8038700 msgr2=0x7f0bc803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/1564023164 2024-05-08T21:59:08.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.829+0000 7f0be1400700 1 --2- 172.21.15.131:0/3804791490 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f0bc8038700 0x7f0bc803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T21:59:08.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.898+0000 7f0be9200700 1 -- 172.21.15.131:0/3804791490 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f0be419fbf0 con 0x7f0be4106780 2024-05-08T21:59:08.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.898+0000 7f0bdaa00700 1 -- 172.21.15.131:0/3804791490 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f0bd4010380 con 0x7f0be4106780 2024-05-08T21:59:08.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.901+0000 7f0be9200700 1 -- 172.21.15.131:0/3804791490 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f0bc8038700 msgr2=0x7f0bc803abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T21:59:08.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.901+0000 7f0be9200700 1 --2- 172.21.15.131:0/3804791490 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f0bc8038700 0x7f0bc803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:08.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.901+0000 7f0be9200700 1 -- 172.21.15.131:0/3804791490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0be4106780 msgr2=0x7f0be419c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:08.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.901+0000 7f0be9200700 1 --2- 172.21.15.131:0/3804791490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0be4106780 0x7f0be419c690 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009130 tx=0x7f0bd4011ad0 comp rx=0 tx=0).stop 2024-05-08T21:59:08.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.901+0000 7f0be9200700 1 -- 172.21.15.131:0/3804791490 shutdown_connections 2024-05-08T21:59:08.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.901+0000 7f0be9200700 1 --2- 172.21.15.131:0/3804791490 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f0bc8038700 0x7f0bc803abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:08.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.901+0000 7f0be9200700 1 --2- 172.21.15.131:0/3804791490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0be4106780 0x7f0be419c690 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:08.301 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.901+0000 7f0be9200700 1 -- 172.21.15.131:0/3804791490 >> 172.21.15.131:0/3804791490 conn(0x7f0be4101d20 msgr2=0x7f0be4102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:08.302 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.901+0000 7f0be9200700 1 -- 172.21.15.131:0/3804791490 shutdown_connections 2024-05-08T21:59:08.302 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:07.901+0000 7f0be9200700 1 -- 172.21.15.131:0/3804791490 wait complete. 2024-05-08T21:59:08.302 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for the mgr to restart... 2024-05-08T21:59:08.302 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mgr epoch 5... 2024-05-08T21:59:08.798 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:08 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:08.653+0000 7fe4c2a69200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:59:09.097 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:08 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3804791490' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-08T21:59:09.098 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:08 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:08.981+0000 7fe4c2a69200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T21:59:09.390 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:09 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:09.096+0000 7fe4c2a69200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T21:59:09.390 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:09 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:09.237+0000 7fe4c2a69200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T21:59:09.715 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:09 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:09.434+0000 7fe4c2a69200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T21:59:09.716 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:09 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:09.603+0000 7fe4c2a69200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T21:59:10.140 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:09 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:09.714+0000 7fe4c2a69200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T21:59:10.640 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:10 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:10.188+0000 7fe4c2a69200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T21:59:10.932 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:10 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:10.680+0000 7fe4c2a69200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T21:59:11.390 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:10 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:10.931+0000 7fe4c2a69200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T21:59:11.391 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:11 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:11.055+0000 7fe4c2a69200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T21:59:12.340 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:12 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:12.018+0000 7fe4c2a69200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T21:59:12.340 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:12 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:12.145+0000 7fe4c2a69200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T21:59:12.640 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:12 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:12.339+0000 7fe4c2a69200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T21:59:13.640 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:13 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:13.158+0000 7fe4c2a69200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T21:59:13.640 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:13 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:13.400+0000 7fe4c2a69200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T21:59:14.140 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:13 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:13.657+0000 7fe4c2a69200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T21:59:15.141 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:14 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:14.799+0000 7fe4c2a69200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T21:59:15.546 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:15 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:15.172+0000 7fe4c2a69200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:59:15.546 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:15 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:15.290+0000 7fe4c2a69200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T21:59:15.890 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:15 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:15.545+0000 7fe4c2a69200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T21:59:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:15 smithi131 ceph-mon[37260]: Active manager daemon x restarted 2024-05-08T21:59:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:15 smithi131 ceph-mon[37260]: Activating manager daemon x 2024-05-08T21:59:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:15 smithi131 ceph-mon[37260]: osdmap e2: 0 total, 0 up, 0 in 2024-05-08T21:59:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:15 smithi131 ceph-mon[37260]: mgrmap e6: x(active, starting, since 0.00420112s) 2024-05-08T21:59:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:15 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:59:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:15 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T21:59:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:15 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T21:59:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:15 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T21:59:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:15 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T21:59:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:15 smithi131 ceph-mon[37260]: Manager daemon x is now available 2024-05-08T21:59:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:15 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' 2024-05-08T21:59:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:15 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' 2024-05-08T21:59:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:15 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:59:15.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:15 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:59:16.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:16 smithi131 ceph-mon[37260]: Found migration_current of "None". Setting to last migration. 2024-05-08T21:59:16.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:16 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T21:59:16.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:16 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T21:59:16.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:16 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' 2024-05-08T21:59:16.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:16 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' 2024-05-08T21:59:16.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:16 smithi131 ceph-mon[37260]: mgrmap e7: x(active, since 1.00706s) 2024-05-08T21:59:16.932 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-05-08T21:59:16.940 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-08T21:59:16.940 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-08T21:59:16.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-05-08T21:59:16.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.730+0000 7f29e0e00700 1 Processor -- start 2024-05-08T21:59:16.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.731+0000 7f29e0e00700 1 -- start start 2024-05-08T21:59:16.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.731+0000 7f29e0e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29dc07be90 0x7f29dc07a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:16.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.731+0000 7f29e0e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc07a910 con 0x7f29dc07be90 2024-05-08T21:59:16.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.731+0000 7f29d9e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29dc07be90 0x7f29dc07a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:16.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.731+0000 7f29d9e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29dc07be90 0x7f29dc07a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46290/0 (socket says 172.21.15.131:46290) 2024-05-08T21:59:16.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.731+0000 7f29d9e00700 1 -- 172.21.15.131:0/3732551026 learned_addr learned my addr 172.21.15.131:0/3732551026 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:16.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.732+0000 7f29d9e00700 1 -- 172.21.15.131:0/3732551026 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29dc07aa50 con 0x7f29dc07be90 2024-05-08T21:59:16.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.732+0000 7f29d9e00700 1 --2- 172.21.15.131:0/3732551026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29dc07be90 0x7f29dc07a340 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0098a0 tx=0x7f29cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305d627fc466ade9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:16.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.732+0000 7f29d8a00700 1 -- 172.21.15.131:0/3732551026 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29cc004160 con 0x7f29dc07be90 2024-05-08T21:59:16.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.732+0000 7f29d8a00700 1 -- 172.21.15.131:0/3732551026 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f29cc0042c0 con 0x7f29dc07be90 2024-05-08T21:59:16.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.732+0000 7f29d8a00700 1 -- 172.21.15.131:0/3732551026 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29cc011a00 con 0x7f29dc07be90 2024-05-08T21:59:16.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.733+0000 7f29e0e00700 1 -- 172.21.15.131:0/3732551026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29dc07be90 msgr2=0x7f29dc07a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:16.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.733+0000 7f29e0e00700 1 --2- 172.21.15.131:0/3732551026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29dc07be90 0x7f29dc07a340 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0098a0 tx=0x7f29cc009bb0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.733+0000 7f29e0e00700 1 -- 172.21.15.131:0/3732551026 shutdown_connections 2024-05-08T21:59:16.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.733+0000 7f29e0e00700 1 --2- 172.21.15.131:0/3732551026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29dc07be90 0x7f29dc07a340 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.733+0000 7f29e0e00700 1 -- 172.21.15.131:0/3732551026 >> 172.21.15.131:0/3732551026 conn(0x7f29dc101d20 msgr2=0x7f29dc104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:16.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.733+0000 7f29e0e00700 1 -- 172.21.15.131:0/3732551026 shutdown_connections 2024-05-08T21:59:16.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.733+0000 7f29e0e00700 1 -- 172.21.15.131:0/3732551026 wait complete. 2024-05-08T21:59:16.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.734+0000 7f29e0e00700 1 Processor -- start 2024-05-08T21:59:16.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.735+0000 7f29e0e00700 1 -- start start 2024-05-08T21:59:16.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.735+0000 7f29e0e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29dc07be90 0x7f29dc1a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:16.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.735+0000 7f29e0e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc1a0fb0 con 0x7f29dc07be90 2024-05-08T21:59:16.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.735+0000 7f29d9e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29dc07be90 0x7f29dc1a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:16.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.735+0000 7f29d9e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29dc07be90 0x7f29dc1a0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46294/0 (socket says 172.21.15.131:46294) 2024-05-08T21:59:16.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.735+0000 7f29d9e00700 1 -- 172.21.15.131:0/689115888 learned_addr learned my addr 172.21.15.131:0/689115888 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:16.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.736+0000 7f29d9e00700 1 -- 172.21.15.131:0/689115888 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc009550 con 0x7f29dc07be90 2024-05-08T21:59:16.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.736+0000 7f29d9e00700 1 --2- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29dc07be90 0x7f29dc1a0a70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f29cc011710 tx=0x7f29cc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:16.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.736+0000 7f29d2a00700 1 -- 172.21.15.131:0/689115888 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29cc0039e0 con 0x7f29dc07be90 2024-05-08T21:59:16.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.736+0000 7f29d2a00700 1 -- 172.21.15.131:0/689115888 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f29cc003b40 con 0x7f29dc07be90 2024-05-08T21:59:16.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.736+0000 7f29e0e00700 1 -- 172.21.15.131:0/689115888 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29dc1a11b0 con 0x7f29dc07be90 2024-05-08T21:59:16.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.736+0000 7f29d2a00700 1 -- 172.21.15.131:0/689115888 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29cc019690 con 0x7f29dc07be90 2024-05-08T21:59:16.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.736+0000 7f29e0e00700 1 -- 172.21.15.131:0/689115888 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29dc1a1650 con 0x7f29dc07be90 2024-05-08T21:59:16.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.737+0000 7f29d2a00700 1 -- 172.21.15.131:0/689115888 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f29cc019950 con 0x7f29dc07be90 2024-05-08T21:59:16.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.738+0000 7f29d2a00700 1 --2- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f29c0038700 0x7f29c003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:16.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.738+0000 7f29d2a00700 1 -- 172.21.15.131:0/689115888 --> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f29c003b2d0 con 0x7f29c0038700 2024-05-08T21:59:16.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.738+0000 7f29d2a00700 1 -- 172.21.15.131:0/689115888 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f29cc04c800 con 0x7f29dc07be90 2024-05-08T21:59:16.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.738+0000 7f29d9400700 1 -- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f29c0038700 msgr2=0x7f29c003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/1564023164 2024-05-08T21:59:16.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.738+0000 7f29d9400700 1 --2- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f29c0038700 0x7f29c003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T21:59:16.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.939+0000 7f29d9400700 1 -- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f29c0038700 msgr2=0x7f29c003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/1564023164 2024-05-08T21:59:16.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:08.939+0000 7f29d9400700 1 --2- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f29c0038700 0x7f29c003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T21:59:16.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:09.340+0000 7f29d9400700 1 -- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f29c0038700 msgr2=0x7f29c003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/1564023164 2024-05-08T21:59:16.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:09.340+0000 7f29d9400700 1 --2- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f29c0038700 0x7f29c003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T21:59:16.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:10.141+0000 7f29d9400700 1 -- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f29c0038700 msgr2=0x7f29c003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/1564023164 2024-05-08T21:59:16.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:10.141+0000 7f29d9400700 1 --2- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f29c0038700 0x7f29c003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T21:59:16.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:11.742+0000 7f29d9400700 1 -- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f29c0038700 msgr2=0x7f29c003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/1564023164 2024-05-08T21:59:16.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:11.743+0000 7f29d9400700 1 --2- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f29c0038700 0x7f29c003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-08T21:59:16.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:14.944+0000 7f29d9400700 1 -- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f29c0038700 msgr2=0x7f29c003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/1564023164 2024-05-08T21:59:16.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:14.944+0000 7f29d9400700 1 --2- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f29c0038700 0x7f29c003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-05-08T21:59:16.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:15.550+0000 7f29d2a00700 1 -- 172.21.15.131:0/689115888 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mgrmap(e 6) v1 ==== 45308+0+0 (secure 0 0 0) 0x7f29cc019c50 con 0x7f29dc07be90 2024-05-08T21:59:16.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:15.550+0000 7f29d2a00700 1 -- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f29c0038700 msgr2=0x7f29c003abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T21:59:16.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:15.550+0000 7f29d2a00700 1 --2- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f29c0038700 0x7f29c003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:16.552+0000 7f29d2a00700 1 -- 172.21.15.131:0/689115888 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7f29cc00b3e0 con 0x7f29dc07be90 2024-05-08T21:59:16.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:16.552+0000 7f29d2a00700 1 --2- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f29c003bfe0 0x7f29c003e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:16.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:16.552+0000 7f29d2a00700 1 -- 172.21.15.131:0/689115888 --> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f29c003b2d0 con 0x7f29c003bfe0 2024-05-08T21:59:16.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:16.552+0000 7f29d9400700 1 --2- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f29c003bfe0 0x7f29c003e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:16.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:16.553+0000 7f29d9400700 1 --2- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f29c003bfe0 0x7f29c003e3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f29c8003a10 tx=0x7f29c8009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:16.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:16.554+0000 7f29d2a00700 1 -- 172.21.15.131:0/689115888 <== mgr.14118 v2:172.21.15.131:6800/1418340798 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f29c003b2d0 con 0x7f29c003bfe0 2024-05-08T21:59:16.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:16.561+0000 7f29e0e00700 1 -- 172.21.15.131:0/689115888 --> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f29dc1a17e0 con 0x7f29c003bfe0 2024-05-08T21:59:16.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:16.561+0000 7f29d2a00700 1 -- 172.21.15.131:0/689115888 <== mgr.14118 v2:172.21.15.131:6800/1418340798 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f29c8008850 con 0x7f29c003bfe0 2024-05-08T21:59:16.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:16.562+0000 7f29e0e00700 1 -- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f29c003bfe0 msgr2=0x7f29c003e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:16.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:16.562+0000 7f29e0e00700 1 --2- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f29c003bfe0 0x7f29c003e3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f29c8003a10 tx=0x7f29c8009290 comp rx=0 tx=0).stop 2024-05-08T21:59:16.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:16.562+0000 7f29e0e00700 1 -- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29dc07be90 msgr2=0x7f29dc1a0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:16.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:16.562+0000 7f29e0e00700 1 --2- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29dc07be90 0x7f29dc1a0a70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f29cc011710 tx=0x7f29cc011ad0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:16.562+0000 7f29e0e00700 1 -- 172.21.15.131:0/689115888 shutdown_connections 2024-05-08T21:59:16.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:16.562+0000 7f29e0e00700 1 --2- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29dc07be90 0x7f29dc1a0a70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:16.562+0000 7f29e0e00700 1 --2- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f29c003bfe0 0x7f29c003e3d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.946 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:16.562+0000 7f29e0e00700 1 --2- 172.21.15.131:0/689115888 >> [v2:172.21.15.131:6800/1564023164,v1:172.21.15.131:6801/1564023164] conn(0x7f29c0038700 0x7f29c003abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:16.946 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:16.562+0000 7f29e0e00700 1 -- 172.21.15.131:0/689115888 >> 172.21.15.131:0/689115888 conn(0x7f29dc101d20 msgr2=0x7f29dc104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:16.946 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:16.562+0000 7f29e0e00700 1 -- 172.21.15.131:0/689115888 shutdown_connections 2024-05-08T21:59:16.946 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:16.562+0000 7f29e0e00700 1 -- 172.21.15.131:0/689115888 wait complete. 2024-05-08T21:59:16.946 INFO:teuthology.orchestra.run.smithi131.stdout:mgr epoch 5 is available 2024-05-08T21:59:16.946 INFO:teuthology.orchestra.run.smithi131.stdout:Setting orchestrator backend to cephadm... 2024-05-08T21:59:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:17 smithi131 ceph-mon[37260]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-08T21:59:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:17 smithi131 ceph-mon[37260]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-08T21:59:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:17 smithi131 ceph-mon[37260]: [08/May/2024:21:59:16] ENGINE Bus STARTING 2024-05-08T21:59:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:17 smithi131 ceph-mon[37260]: [08/May/2024:21:59:16] ENGINE Serving on https://172.21.15.131:7150 2024-05-08T21:59:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:17 smithi131 ceph-mon[37260]: [08/May/2024:21:59:16] ENGINE Serving on http://172.21.15.131:8765 2024-05-08T21:59:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:17 smithi131 ceph-mon[37260]: [08/May/2024:21:59:16] ENGINE Bus STARTED 2024-05-08T21:59:17.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:17 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:59:17.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:17 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' 2024-05-08T21:59:17.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:17 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:59:17.949 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.332+0000 7f4f78000700 1 Processor -- start 2024-05-08T21:59:17.949 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.332+0000 7f4f78000700 1 -- start start 2024-05-08T21:59:17.949 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.333+0000 7f4f78000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f70106790 0x7f4f70106bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:17.949 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.333+0000 7f4f78000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f70107180 con 0x7f4f70106790 2024-05-08T21:59:17.950 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.333+0000 7f4f75600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f70106790 0x7f4f70106bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:17.950 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.333+0000 7f4f75600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f70106790 0x7f4f70106bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33090/0 (socket says 172.21.15.131:33090) 2024-05-08T21:59:17.950 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.333+0000 7f4f75600700 1 -- 172.21.15.131:0/4159592780 learned_addr learned my addr 172.21.15.131:0/4159592780 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:17.950 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.334+0000 7f4f75600700 1 -- 172.21.15.131:0/4159592780 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f70107940 con 0x7f4f70106790 2024-05-08T21:59:17.950 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.334+0000 7f4f75600700 1 --2- 172.21.15.131:0/4159592780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f70106790 0x7f4f70106bb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4f640098a0 tx=0x7f4f64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433f6abee3f5a224 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:17.950 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.334+0000 7f4f6fe00700 1 -- 172.21.15.131:0/4159592780 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f64004160 con 0x7f4f70106790 2024-05-08T21:59:17.951 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.334+0000 7f4f6fe00700 1 -- 172.21.15.131:0/4159592780 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4f640042c0 con 0x7f4f70106790 2024-05-08T21:59:17.951 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.334+0000 7f4f6fe00700 1 -- 172.21.15.131:0/4159592780 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f64011a00 con 0x7f4f70106790 2024-05-08T21:59:17.951 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.335+0000 7f4f78000700 1 -- 172.21.15.131:0/4159592780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f70106790 msgr2=0x7f4f70106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:17.951 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.335+0000 7f4f78000700 1 --2- 172.21.15.131:0/4159592780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f70106790 0x7f4f70106bb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4f640098a0 tx=0x7f4f64009bb0 comp rx=0 tx=0).stop 2024-05-08T21:59:17.951 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.335+0000 7f4f78000700 1 -- 172.21.15.131:0/4159592780 shutdown_connections 2024-05-08T21:59:17.952 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.335+0000 7f4f78000700 1 --2- 172.21.15.131:0/4159592780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f70106790 0x7f4f70106bb0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:17.952 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.335+0000 7f4f78000700 1 -- 172.21.15.131:0/4159592780 >> 172.21.15.131:0/4159592780 conn(0x7f4f70101d50 msgr2=0x7f4f70104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:17.952 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.335+0000 7f4f78000700 1 -- 172.21.15.131:0/4159592780 shutdown_connections 2024-05-08T21:59:17.952 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.335+0000 7f4f78000700 1 -- 172.21.15.131:0/4159592780 wait complete. 2024-05-08T21:59:17.952 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.336+0000 7f4f78000700 1 Processor -- start 2024-05-08T21:59:17.952 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.336+0000 7f4f78000700 1 -- start start 2024-05-08T21:59:17.953 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.337+0000 7f4f78000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f70106790 0x7f4f7019c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:17.953 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.337+0000 7f4f78000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7019cc80 con 0x7f4f70106790 2024-05-08T21:59:17.953 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.337+0000 7f4f75600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f70106790 0x7f4f7019c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:17.953 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.337+0000 7f4f75600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f70106790 0x7f4f7019c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33092/0 (socket says 172.21.15.131:33092) 2024-05-08T21:59:17.953 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.337+0000 7f4f75600700 1 -- 172.21.15.131:0/3072693302 learned_addr learned my addr 172.21.15.131:0/3072693302 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:17.954 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.337+0000 7f4f75600700 1 -- 172.21.15.131:0/3072693302 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f64009550 con 0x7f4f70106790 2024-05-08T21:59:17.954 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.337+0000 7f4f75600700 1 --2- 172.21.15.131:0/3072693302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f70106790 0x7f4f7019c740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4f64011710 tx=0x7f4f64011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:17.954 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.338+0000 7f4f6e000700 1 -- 172.21.15.131:0/3072693302 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f64003aa0 con 0x7f4f70106790 2024-05-08T21:59:17.954 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.338+0000 7f4f6e000700 1 -- 172.21.15.131:0/3072693302 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4f64003c00 con 0x7f4f70106790 2024-05-08T21:59:17.954 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.338+0000 7f4f78000700 1 -- 172.21.15.131:0/3072693302 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f7019ce80 con 0x7f4f70106790 2024-05-08T21:59:17.954 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.338+0000 7f4f6e000700 1 -- 172.21.15.131:0/3072693302 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f64024800 con 0x7f4f70106790 2024-05-08T21:59:17.955 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.338+0000 7f4f78000700 1 -- 172.21.15.131:0/3072693302 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f7019d320 con 0x7f4f70106790 2024-05-08T21:59:17.955 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.339+0000 7f4f6e000700 1 -- 172.21.15.131:0/3072693302 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7f4f6401a430 con 0x7f4f70106790 2024-05-08T21:59:17.955 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.339+0000 7f4f78000700 1 -- 172.21.15.131:0/3072693302 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f7010ab70 con 0x7f4f70106790 2024-05-08T21:59:17.955 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.339+0000 7f4f6e000700 1 --2- 172.21.15.131:0/3072693302 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f4f58038620 0x7f4f5803aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:17.955 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.339+0000 7f4f6e000700 1 -- 172.21.15.131:0/3072693302 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4f6404c410 con 0x7f4f70106790 2024-05-08T21:59:17.955 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.340+0000 7f4f74c00700 1 --2- 172.21.15.131:0/3072693302 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f4f58038620 0x7f4f5803aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:17.956 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.340+0000 7f4f74c00700 1 --2- 172.21.15.131:0/3072693302 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f4f58038620 0x7f4f5803aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c006fd0 tx=0x7f4f5c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:17.956 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.345+0000 7f4f6e000700 1 -- 172.21.15.131:0/3072693302 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4f6402c9a0 con 0x7f4f70106790 2024-05-08T21:59:17.956 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.565+0000 7f4f78000700 1 -- 172.21.15.131:0/3072693302 --> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f70103070 con 0x7f4f58038620 2024-05-08T21:59:17.956 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.576+0000 7f4f6e000700 1 -- 172.21.15.131:0/3072693302 <== mgr.14118 v2:172.21.15.131:6800/1418340798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4f70103070 con 0x7f4f58038620 2024-05-08T21:59:17.956 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.578+0000 7f4f78000700 1 -- 172.21.15.131:0/3072693302 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f4f58038620 msgr2=0x7f4f5803aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:17.957 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.578+0000 7f4f78000700 1 --2- 172.21.15.131:0/3072693302 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f4f58038620 0x7f4f5803aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c006fd0 tx=0x7f4f5c006e40 comp rx=0 tx=0).stop 2024-05-08T21:59:17.957 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.578+0000 7f4f78000700 1 -- 172.21.15.131:0/3072693302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f70106790 msgr2=0x7f4f7019c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:17.957 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.578+0000 7f4f78000700 1 --2- 172.21.15.131:0/3072693302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f70106790 0x7f4f7019c740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4f64011710 tx=0x7f4f64011ad0 comp rx=0 tx=0).stop 2024-05-08T21:59:17.957 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.578+0000 7f4f78000700 1 -- 172.21.15.131:0/3072693302 shutdown_connections 2024-05-08T21:59:17.957 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.578+0000 7f4f78000700 1 --2- 172.21.15.131:0/3072693302 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f4f58038620 0x7f4f5803aae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:17.957 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.578+0000 7f4f78000700 1 --2- 172.21.15.131:0/3072693302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f70106790 0x7f4f7019c740 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:17.958 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.578+0000 7f4f78000700 1 -- 172.21.15.131:0/3072693302 >> 172.21.15.131:0/3072693302 conn(0x7f4f70101d50 msgr2=0x7f4f70102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:17.958 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.579+0000 7f4f78000700 1 -- 172.21.15.131:0/3072693302 shutdown_connections 2024-05-08T21:59:17.958 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:17.579+0000 7f4f78000700 1 -- 172.21.15.131:0/3072693302 wait complete. 2024-05-08T21:59:18.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:18 smithi131 ceph-mon[37260]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:59:18.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:18 smithi131 ceph-mon[37260]: mgrmap e8: x(active, since 2s) 2024-05-08T21:59:18.959 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-08T21:59:18.960 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.381+0000 7f1600000700 1 Processor -- start 2024-05-08T21:59:18.960 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.381+0000 7f1600000700 1 -- start start 2024-05-08T21:59:18.960 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.381+0000 7f1600000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15f8106780 0x7f15f8106ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:18.960 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.381+0000 7f1600000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f8107170 con 0x7f15f8106780 2024-05-08T21:59:18.960 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.382+0000 7f15fd600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15f8106780 0x7f15f8106ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:18.960 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.382+0000 7f15fd600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15f8106780 0x7f15f8106ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33102/0 (socket says 172.21.15.131:33102) 2024-05-08T21:59:18.960 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.382+0000 7f15fd600700 1 -- 172.21.15.131:0/723347151 learned_addr learned my addr 172.21.15.131:0/723347151 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:18.960 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.382+0000 7f15fd600700 1 -- 172.21.15.131:0/723347151 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f8107930 con 0x7f15f8106780 2024-05-08T21:59:18.960 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.383+0000 7f15fd600700 1 --2- 172.21.15.131:0/723347151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15f8106780 0x7f15f8106ba0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0098a0 tx=0x7f15ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e02d5a9602948ff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:18.960 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.383+0000 7f15f7e00700 1 -- 172.21.15.131:0/723347151 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15ec004160 con 0x7f15f8106780 2024-05-08T21:59:18.960 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.383+0000 7f15f7e00700 1 -- 172.21.15.131:0/723347151 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f15ec0042c0 con 0x7f15f8106780 2024-05-08T21:59:18.960 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.383+0000 7f15f7e00700 1 -- 172.21.15.131:0/723347151 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15ec011a00 con 0x7f15f8106780 2024-05-08T21:59:18.960 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.383+0000 7f1600000700 1 -- 172.21.15.131:0/723347151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15f8106780 msgr2=0x7f15f8106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:18.960 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.383+0000 7f1600000700 1 --2- 172.21.15.131:0/723347151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15f8106780 0x7f15f8106ba0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0098a0 tx=0x7f15ec009bb0 comp rx=0 tx=0).stop 2024-05-08T21:59:18.961 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.383+0000 7f1600000700 1 -- 172.21.15.131:0/723347151 shutdown_connections 2024-05-08T21:59:18.961 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.383+0000 7f1600000700 1 --2- 172.21.15.131:0/723347151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15f8106780 0x7f15f8106ba0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:18.961 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.383+0000 7f1600000700 1 -- 172.21.15.131:0/723347151 >> 172.21.15.131:0/723347151 conn(0x7f15f8101d20 msgr2=0x7f15f8104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:18.961 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.384+0000 7f1600000700 1 -- 172.21.15.131:0/723347151 shutdown_connections 2024-05-08T21:59:18.961 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.384+0000 7f1600000700 1 -- 172.21.15.131:0/723347151 wait complete. 2024-05-08T21:59:18.961 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.385+0000 7f1600000700 1 Processor -- start 2024-05-08T21:59:18.961 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.385+0000 7f1600000700 1 -- start start 2024-05-08T21:59:18.961 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.385+0000 7f1600000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15f8106780 0x7f15f819c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:18.961 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.385+0000 7f1600000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f819cb90 con 0x7f15f8106780 2024-05-08T21:59:18.961 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.385+0000 7f15fd600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15f8106780 0x7f15f819c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:18.961 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.385+0000 7f15fd600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15f8106780 0x7f15f819c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33114/0 (socket says 172.21.15.131:33114) 2024-05-08T21:59:18.961 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.385+0000 7f15fd600700 1 -- 172.21.15.131:0/3771270175 learned_addr learned my addr 172.21.15.131:0/3771270175 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:18.961 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.386+0000 7f15fd600700 1 -- 172.21.15.131:0/3771270175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ec009550 con 0x7f15f8106780 2024-05-08T21:59:18.961 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.386+0000 7f15fd600700 1 --2- 172.21.15.131:0/3771270175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15f8106780 0x7f15f819c650 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009130 tx=0x7f15ec011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:18.962 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.386+0000 7f15f6000700 1 -- 172.21.15.131:0/3771270175 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15ec003ab0 con 0x7f15f8106780 2024-05-08T21:59:18.962 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.386+0000 7f15f6000700 1 -- 172.21.15.131:0/3771270175 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f15ec003c10 con 0x7f15f8106780 2024-05-08T21:59:18.962 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.386+0000 7f1600000700 1 -- 172.21.15.131:0/3771270175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f819cd90 con 0x7f15f8106780 2024-05-08T21:59:18.962 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.386+0000 7f15f6000700 1 -- 172.21.15.131:0/3771270175 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15ec0247d0 con 0x7f15f8106780 2024-05-08T21:59:18.962 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.386+0000 7f1600000700 1 -- 172.21.15.131:0/3771270175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f819d230 con 0x7f15f8106780 2024-05-08T21:59:18.962 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.387+0000 7f15f6000700 1 -- 172.21.15.131:0/3771270175 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f15ec01a430 con 0x7f15f8106780 2024-05-08T21:59:18.962 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.387+0000 7f1600000700 1 -- 172.21.15.131:0/3771270175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f810ab60 con 0x7f15f8106780 2024-05-08T21:59:18.962 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.388+0000 7f15f6000700 1 --2- 172.21.15.131:0/3771270175 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f15e0038750 0x7f15e003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:18.962 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.388+0000 7f15f6000700 1 -- 172.21.15.131:0/3771270175 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f15ec04c890 con 0x7f15f8106780 2024-05-08T21:59:18.962 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.388+0000 7f15fcc00700 1 --2- 172.21.15.131:0/3771270175 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f15e0038750 0x7f15e003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:18.962 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.389+0000 7f15fcc00700 1 --2- 172.21.15.131:0/3771270175 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f15e0038750 0x7f15e003ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f15e4006fd0 tx=0x7f15e4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:18.962 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.393+0000 7f15f6000700 1 -- 172.21.15.131:0/3771270175 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f15ec018b40 con 0x7f15f8106780 2024-05-08T21:59:18.962 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.597+0000 7f1600000700 1 -- 172.21.15.131:0/3771270175 --> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f15f819fb20 con 0x7f15e0038750 2024-05-08T21:59:18.963 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.598+0000 7f15f6000700 1 -- 172.21.15.131:0/3771270175 <== mgr.14118 v2:172.21.15.131:6800/1418340798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f15f819fb20 con 0x7f15e0038750 2024-05-08T21:59:18.963 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.600+0000 7f1600000700 1 -- 172.21.15.131:0/3771270175 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f15e0038750 msgr2=0x7f15e003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:18.963 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.600+0000 7f1600000700 1 --2- 172.21.15.131:0/3771270175 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f15e0038750 0x7f15e003ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f15e4006fd0 tx=0x7f15e4006e40 comp rx=0 tx=0).stop 2024-05-08T21:59:18.963 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.601+0000 7f1600000700 1 -- 172.21.15.131:0/3771270175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15f8106780 msgr2=0x7f15f819c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:18.963 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.601+0000 7f1600000700 1 --2- 172.21.15.131:0/3771270175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15f8106780 0x7f15f819c650 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009130 tx=0x7f15ec011ad0 comp rx=0 tx=0).stop 2024-05-08T21:59:18.963 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.601+0000 7f1600000700 1 -- 172.21.15.131:0/3771270175 shutdown_connections 2024-05-08T21:59:18.963 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.601+0000 7f1600000700 1 --2- 172.21.15.131:0/3771270175 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f15e0038750 0x7f15e003ac10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:18.963 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.601+0000 7f1600000700 1 --2- 172.21.15.131:0/3771270175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15f8106780 0x7f15f819c650 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:18.963 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.601+0000 7f1600000700 1 -- 172.21.15.131:0/3771270175 >> 172.21.15.131:0/3771270175 conn(0x7f15f8101d20 msgr2=0x7f15f8102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:18.963 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.601+0000 7f1600000700 1 -- 172.21.15.131:0/3771270175 shutdown_connections 2024-05-08T21:59:18.963 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:18.601+0000 7f1600000700 1 -- 172.21.15.131:0/3771270175 wait complete. 2024-05-08T21:59:18.963 INFO:teuthology.orchestra.run.smithi131.stdout:Generating ssh key... 2024-05-08T21:59:19.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:19 smithi131 ceph-mon[37260]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:59:20.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.402+0000 7f99b5a00700 1 Processor -- start 2024-05-08T21:59:20.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.403+0000 7f99b5a00700 1 -- start start 2024-05-08T21:59:20.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.403+0000 7f99b5a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b007be90 0x7f99b007a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:20.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.403+0000 7f99b5a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b007a910 con 0x7f99b007be90 2024-05-08T21:59:20.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.403+0000 7f99aea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b007be90 0x7f99b007a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:20.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.403+0000 7f99aea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b007be90 0x7f99b007a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33118/0 (socket says 172.21.15.131:33118) 2024-05-08T21:59:20.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.403+0000 7f99aea00700 1 -- 172.21.15.131:0/1445348559 learned_addr learned my addr 172.21.15.131:0/1445348559 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:20.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.404+0000 7f99aea00700 1 -- 172.21.15.131:0/1445348559 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b007aa50 con 0x7f99b007be90 2024-05-08T21:59:20.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.404+0000 7f99aea00700 1 --2- 172.21.15.131:0/1445348559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b007be90 0x7f99b007a340 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f99a00098a0 tx=0x7f99a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759f35f178a1ed34 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:20.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.404+0000 7f99ad600700 1 -- 172.21.15.131:0/1445348559 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99a0004160 con 0x7f99b007be90 2024-05-08T21:59:20.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.404+0000 7f99ad600700 1 -- 172.21.15.131:0/1445348559 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f99a00042c0 con 0x7f99b007be90 2024-05-08T21:59:20.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.404+0000 7f99ad600700 1 -- 172.21.15.131:0/1445348559 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99a0011a00 con 0x7f99b007be90 2024-05-08T21:59:20.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.405+0000 7f99b5a00700 1 -- 172.21.15.131:0/1445348559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b007be90 msgr2=0x7f99b007a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:20.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.405+0000 7f99b5a00700 1 --2- 172.21.15.131:0/1445348559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b007be90 0x7f99b007a340 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f99a00098a0 tx=0x7f99a0009bb0 comp rx=0 tx=0).stop 2024-05-08T21:59:20.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.405+0000 7f99b5a00700 1 -- 172.21.15.131:0/1445348559 shutdown_connections 2024-05-08T21:59:20.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.405+0000 7f99b5a00700 1 --2- 172.21.15.131:0/1445348559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b007be90 0x7f99b007a340 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:20.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.405+0000 7f99b5a00700 1 -- 172.21.15.131:0/1445348559 >> 172.21.15.131:0/1445348559 conn(0x7f99b0101d20 msgr2=0x7f99b0104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:20.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.405+0000 7f99b5a00700 1 -- 172.21.15.131:0/1445348559 shutdown_connections 2024-05-08T21:59:20.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.405+0000 7f99b5a00700 1 -- 172.21.15.131:0/1445348559 wait complete. 2024-05-08T21:59:20.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.406+0000 7f99b5a00700 1 Processor -- start 2024-05-08T21:59:20.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.407+0000 7f99b5a00700 1 -- start start 2024-05-08T21:59:20.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.407+0000 7f99b5a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b007be90 0x7f99b01a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:20.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.407+0000 7f99b5a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b01a0fb0 con 0x7f99b007be90 2024-05-08T21:59:20.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.407+0000 7f99aea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b007be90 0x7f99b01a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:20.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.407+0000 7f99aea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b007be90 0x7f99b01a0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33124/0 (socket says 172.21.15.131:33124) 2024-05-08T21:59:20.324 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.407+0000 7f99aea00700 1 -- 172.21.15.131:0/206397548 learned_addr learned my addr 172.21.15.131:0/206397548 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:20.324 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.407+0000 7f99aea00700 1 -- 172.21.15.131:0/206397548 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a0009550 con 0x7f99b007be90 2024-05-08T21:59:20.324 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.408+0000 7f99aea00700 1 --2- 172.21.15.131:0/206397548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b007be90 0x7f99b01a0a70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f99a0011710 tx=0x7f99a0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:20.324 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.408+0000 7f99a7400700 1 -- 172.21.15.131:0/206397548 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99a00039e0 con 0x7f99b007be90 2024-05-08T21:59:20.324 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.408+0000 7f99a7400700 1 -- 172.21.15.131:0/206397548 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f99a0003b40 con 0x7f99b007be90 2024-05-08T21:59:20.324 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.408+0000 7f99a7400700 1 -- 172.21.15.131:0/206397548 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99a0019690 con 0x7f99b007be90 2024-05-08T21:59:20.324 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.408+0000 7f99b5a00700 1 -- 172.21.15.131:0/206397548 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b01a11b0 con 0x7f99b007be90 2024-05-08T21:59:20.324 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.408+0000 7f99b5a00700 1 -- 172.21.15.131:0/206397548 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b01a1650 con 0x7f99b007be90 2024-05-08T21:59:20.324 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.409+0000 7f99a7400700 1 -- 172.21.15.131:0/206397548 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f99a001aa60 con 0x7f99b007be90 2024-05-08T21:59:20.324 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.409+0000 7f99b5a00700 1 -- 172.21.15.131:0/206397548 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99b019a220 con 0x7f99b007be90 2024-05-08T21:59:20.324 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.410+0000 7f99a7400700 1 --2- 172.21.15.131:0/206397548 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f9994038700 0x7f999403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:20.324 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.410+0000 7f99a7400700 1 -- 172.21.15.131:0/206397548 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f99a004c910 con 0x7f99b007be90 2024-05-08T21:59:20.325 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.410+0000 7f99ae000700 1 --2- 172.21.15.131:0/206397548 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f9994038700 0x7f999403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:20.325 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.410+0000 7f99ae000700 1 --2- 172.21.15.131:0/206397548 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f9994038700 0x7f999403abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f999c006fd0 tx=0x7f999c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:20.325 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.415+0000 7f99a7400700 1 -- 172.21.15.131:0/206397548 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f99a000b3e0 con 0x7f99b007be90 2024-05-08T21:59:20.325 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.621+0000 7f99b5a00700 1 -- 172.21.15.131:0/206397548 --> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f99b002cfb0 con 0x7f9994038700 2024-05-08T21:59:20.325 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.951+0000 7f99a7400700 1 -- 172.21.15.131:0/206397548 <== mgr.14118 v2:172.21.15.131:6800/1418340798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f99b002cfb0 con 0x7f9994038700 2024-05-08T21:59:20.325 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.954+0000 7f99b5a00700 1 -- 172.21.15.131:0/206397548 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f9994038700 msgr2=0x7f999403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:20.325 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.954+0000 7f99b5a00700 1 --2- 172.21.15.131:0/206397548 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f9994038700 0x7f999403abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f999c006fd0 tx=0x7f999c006e40 comp rx=0 tx=0).stop 2024-05-08T21:59:20.325 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.954+0000 7f99b5a00700 1 -- 172.21.15.131:0/206397548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b007be90 msgr2=0x7f99b01a0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:20.325 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.954+0000 7f99b5a00700 1 --2- 172.21.15.131:0/206397548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b007be90 0x7f99b01a0a70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f99a0011710 tx=0x7f99a0011ad0 comp rx=0 tx=0).stop 2024-05-08T21:59:20.325 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.954+0000 7f99b5a00700 1 -- 172.21.15.131:0/206397548 shutdown_connections 2024-05-08T21:59:20.325 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.954+0000 7f99b5a00700 1 --2- 172.21.15.131:0/206397548 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f9994038700 0x7f999403abc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:20.325 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.954+0000 7f99b5a00700 1 --2- 172.21.15.131:0/206397548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b007be90 0x7f99b01a0a70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:20.325 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.954+0000 7f99b5a00700 1 -- 172.21.15.131:0/206397548 >> 172.21.15.131:0/206397548 conn(0x7f99b0101d20 msgr2=0x7f99b0104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:20.326 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.954+0000 7f99b5a00700 1 -- 172.21.15.131:0/206397548 shutdown_connections 2024-05-08T21:59:20.326 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:19.954+0000 7f99b5a00700 1 -- 172.21.15.131:0/206397548 wait complete. 2024-05-08T21:59:20.326 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: Generating public/private rsa key pair. 2024-05-08T21:59:20.326 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: Your identification has been saved in /tmp/tmplv8aozm5/key. 2024-05-08T21:59:20.326 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: Your public key has been saved in /tmp/tmplv8aozm5/key.pub. 2024-05-08T21:59:20.326 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: The key fingerprint is: 2024-05-08T21:59:20.326 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: SHA256:yZaXKgQM2lU2KkuYTrviTgYQU4mg8UTe0kzUCWAOw/4 ceph-06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T21:59:20.326 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: The key's randomart image is: 2024-05-08T21:59:20.326 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: +---[RSA 3072]----+ 2024-05-08T21:59:20.327 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: |X+B+=++. | 2024-05-08T21:59:20.327 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: |o^.O oo. | 2024-05-08T21:59:20.327 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: |B.O B | 2024-05-08T21:59:20.327 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: |+o.+ . . o . | 2024-05-08T21:59:20.327 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: |.oo . S o | 2024-05-08T21:59:20.327 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: |. .E . . o | 2024-05-08T21:59:20.327 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: |.+ . . | 2024-05-08T21:59:20.327 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: |= . | 2024-05-08T21:59:20.327 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: |.o | 2024-05-08T21:59:20.327 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: +----[SHA256]-----+ 2024-05-08T21:59:20.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:20 smithi131 ceph-mon[37260]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:59:20.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:20 smithi131 ceph-mon[37260]: Generating ssh key... 2024-05-08T21:59:20.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:20 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' 2024-05-08T21:59:20.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:20 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' 2024-05-08T21:59:21.286 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/DYJrtSvN+EyYLCXPyBB89BReFW9jYLCighbA8FEY5dXop6TdteqCQE0iILXoM+1dp7KDLbtHA7srGbWI/BoCp6ztA5tsQ5VVwCCZSyADR6FPojJEt6rVSQdIaM8hiKHnXJcB/JyUcQYEL+fLd79DIpiUJc9JEbReSKZv19wX12e8+npfAESdWriQDufWnAOvGhfmaIuepxaQjdEPDlD5NzQ7LVFvOqxzbmQZxyf8M8aAfy8KBVQIgylSdUyS4N/uUUKcb7rP5g7Tu5aohKGcchPRAhZj4TdSiTzPWKD/Jx7EFhq0+jjTcvNxaE4cdLxqMUb4yrna6xjozj68cGBwwSANce69N0tVfXiEXpH+yxcuFav9K4J55B+Z9QuA7zysdaqnKqNt+FFkxVgeUl33dR3qkZD1YCHKr0UbQiGIoU+YzfsWEheQ4uUGjl7XyV0Xeuh1SYL3jDp0r/nN+MD21HvUPGSKwhzaVBT9mYlAbDYNQ07tfZMclvB69eIirwU= ceph-06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T21:59:21.286 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.697+0000 7fbd0d200700 1 Processor -- start 2024-05-08T21:59:21.286 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.697+0000 7fbd0d200700 1 -- start start 2024-05-08T21:59:21.286 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.697+0000 7fbd0d200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd0807be90 0x7fbd0807a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:21.286 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.697+0000 7fbd0d200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0807a910 con 0x7fbd0807be90 2024-05-08T21:59:21.286 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.697+0000 7fbd05e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd0807be90 0x7fbd0807a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:21.286 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.697+0000 7fbd05e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd0807be90 0x7fbd0807a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33128/0 (socket says 172.21.15.131:33128) 2024-05-08T21:59:21.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.697+0000 7fbd05e00700 1 -- 172.21.15.131:0/1845931379 learned_addr learned my addr 172.21.15.131:0/1845931379 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:21.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.698+0000 7fbd05e00700 1 -- 172.21.15.131:0/1845931379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0807aa50 con 0x7fbd0807be90 2024-05-08T21:59:21.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.698+0000 7fbd05e00700 1 --2- 172.21.15.131:0/1845931379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd0807be90 0x7fbd0807a340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80098a0 tx=0x7fbcf8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97b503c57b6c14 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:21.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.698+0000 7fbd04a00700 1 -- 172.21.15.131:0/1845931379 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcf8004160 con 0x7fbd0807be90 2024-05-08T21:59:21.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.698+0000 7fbd04a00700 1 -- 172.21.15.131:0/1845931379 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbcf80042c0 con 0x7fbd0807be90 2024-05-08T21:59:21.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.698+0000 7fbd04a00700 1 -- 172.21.15.131:0/1845931379 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcf8011a00 con 0x7fbd0807be90 2024-05-08T21:59:21.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.699+0000 7fbd0d200700 1 -- 172.21.15.131:0/1845931379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd0807be90 msgr2=0x7fbd0807a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:21.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.699+0000 7fbd0d200700 1 --2- 172.21.15.131:0/1845931379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd0807be90 0x7fbd0807a340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80098a0 tx=0x7fbcf8009bb0 comp rx=0 tx=0).stop 2024-05-08T21:59:21.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.699+0000 7fbd0d200700 1 -- 172.21.15.131:0/1845931379 shutdown_connections 2024-05-08T21:59:21.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.699+0000 7fbd0d200700 1 --2- 172.21.15.131:0/1845931379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd0807be90 0x7fbd0807a340 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:21.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.699+0000 7fbd0d200700 1 -- 172.21.15.131:0/1845931379 >> 172.21.15.131:0/1845931379 conn(0x7fbd08101d20 msgr2=0x7fbd08104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:21.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.699+0000 7fbd0d200700 1 -- 172.21.15.131:0/1845931379 shutdown_connections 2024-05-08T21:59:21.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.699+0000 7fbd0d200700 1 -- 172.21.15.131:0/1845931379 wait complete. 2024-05-08T21:59:21.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.700+0000 7fbd0d200700 1 Processor -- start 2024-05-08T21:59:21.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.700+0000 7fbd0d200700 1 -- start start 2024-05-08T21:59:21.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.700+0000 7fbd0d200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd0807be90 0x7fbd081a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:21.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.700+0000 7fbd0d200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd081a0fb0 con 0x7fbd0807be90 2024-05-08T21:59:21.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.701+0000 7fbd05e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd0807be90 0x7fbd081a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:21.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.701+0000 7fbd05e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd0807be90 0x7fbd081a0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33130/0 (socket says 172.21.15.131:33130) 2024-05-08T21:59:21.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.701+0000 7fbd05e00700 1 -- 172.21.15.131:0/2292564319 learned_addr learned my addr 172.21.15.131:0/2292564319 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:21.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.701+0000 7fbd05e00700 1 -- 172.21.15.131:0/2292564319 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf8009550 con 0x7fbd0807be90 2024-05-08T21:59:21.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.701+0000 7fbd05e00700 1 --2- 172.21.15.131:0/2292564319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd0807be90 0x7fbd081a0a70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8011710 tx=0x7fbcf8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:21.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.701+0000 7fbcfea00700 1 -- 172.21.15.131:0/2292564319 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcf80039e0 con 0x7fbd0807be90 2024-05-08T21:59:21.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.701+0000 7fbcfea00700 1 -- 172.21.15.131:0/2292564319 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbcf8003b40 con 0x7fbd0807be90 2024-05-08T21:59:21.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.701+0000 7fbd0d200700 1 -- 172.21.15.131:0/2292564319 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd081a11b0 con 0x7fbd0807be90 2024-05-08T21:59:21.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.701+0000 7fbcfea00700 1 -- 172.21.15.131:0/2292564319 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcf8019690 con 0x7fbd0807be90 2024-05-08T21:59:21.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.702+0000 7fbd0d200700 1 -- 172.21.15.131:0/2292564319 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd081a1650 con 0x7fbd0807be90 2024-05-08T21:59:21.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.703+0000 7fbcfea00700 1 -- 172.21.15.131:0/2292564319 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fbcf801aa60 con 0x7fbd0807be90 2024-05-08T21:59:21.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.703+0000 7fbd0d200700 1 -- 172.21.15.131:0/2292564319 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd0819a220 con 0x7fbd0807be90 2024-05-08T21:59:21.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.703+0000 7fbcfea00700 1 --2- 172.21.15.131:0/2292564319 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7fbcec038750 0x7fbcec03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:21.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.703+0000 7fbcfea00700 1 -- 172.21.15.131:0/2292564319 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbcf804c890 con 0x7fbd0807be90 2024-05-08T21:59:21.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.703+0000 7fbd05400700 1 --2- 172.21.15.131:0/2292564319 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7fbcec038750 0x7fbcec03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:21.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.704+0000 7fbd05400700 1 --2- 172.21.15.131:0/2292564319 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7fbcec038750 0x7fbcec03ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4006fd0 tx=0x7fbcf4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:21.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.708+0000 7fbcfea00700 1 -- 172.21.15.131:0/2292564319 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbcf8019900 con 0x7fbd0807be90 2024-05-08T21:59:21.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.912+0000 7fbd0d200700 1 -- 172.21.15.131:0/2292564319 --> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd0802cfb0 con 0x7fbcec038750 2024-05-08T21:59:21.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.913+0000 7fbcfea00700 1 -- 172.21.15.131:0/2292564319 <== mgr.14118 v2:172.21.15.131:6800/1418340798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fbd0802cfb0 con 0x7fbcec038750 2024-05-08T21:59:21.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.915+0000 7fbd0d200700 1 -- 172.21.15.131:0/2292564319 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7fbcec038750 msgr2=0x7fbcec03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:21.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.915+0000 7fbd0d200700 1 --2- 172.21.15.131:0/2292564319 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7fbcec038750 0x7fbcec03ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4006fd0 tx=0x7fbcf4006e40 comp rx=0 tx=0).stop 2024-05-08T21:59:21.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.915+0000 7fbd0d200700 1 -- 172.21.15.131:0/2292564319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd0807be90 msgr2=0x7fbd081a0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:21.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.915+0000 7fbd0d200700 1 --2- 172.21.15.131:0/2292564319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd0807be90 0x7fbd081a0a70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8011710 tx=0x7fbcf8011ad0 comp rx=0 tx=0).stop 2024-05-08T21:59:21.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.915+0000 7fbd0d200700 1 -- 172.21.15.131:0/2292564319 shutdown_connections 2024-05-08T21:59:21.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.916+0000 7fbd0d200700 1 --2- 172.21.15.131:0/2292564319 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7fbcec038750 0x7fbcec03ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:21.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.916+0000 7fbd0d200700 1 --2- 172.21.15.131:0/2292564319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd0807be90 0x7fbd081a0a70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:21.290 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.916+0000 7fbd0d200700 1 -- 172.21.15.131:0/2292564319 >> 172.21.15.131:0/2292564319 conn(0x7fbd08101d20 msgr2=0x7fbd08104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:21.290 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.916+0000 7fbd0d200700 1 -- 172.21.15.131:0/2292564319 shutdown_connections 2024-05-08T21:59:21.290 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:20.916+0000 7fbd0d200700 1 -- 172.21.15.131:0/2292564319 wait complete. 2024-05-08T21:59:21.290 INFO:teuthology.orchestra.run.smithi131.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-08T21:59:21.290 INFO:teuthology.orchestra.run.smithi131.stdout:Adding key to root@localhost authorized_keys... 2024-05-08T21:59:21.290 INFO:teuthology.orchestra.run.smithi131.stdout:Adding host smithi131... 2024-05-08T21:59:22.349 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:21 smithi131 ceph-mon[37260]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:59:23.226 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:22 smithi131 ceph-mon[37260]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi131", "addr": "172.21.15.131", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:59:23.226 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:22 smithi131 ceph-mon[37260]: Deploying cephadm binary to smithi131 2024-05-08T21:59:25.361 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout Added host 'smithi131' with addr '172.21.15.131' 2024-05-08T21:59:25.361 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.728+0000 7fc6da200700 1 Processor -- start 2024-05-08T21:59:25.361 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.729+0000 7fc6da200700 1 -- start start 2024-05-08T21:59:25.362 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.729+0000 7fc6da200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6d4106790 0x7fc6d4106bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:25.362 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.729+0000 7fc6da200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d4107180 con 0x7fc6d4106790 2024-05-08T21:59:25.362 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.729+0000 7fc6d3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6d4106790 0x7fc6d4106bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:25.362 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.729+0000 7fc6d3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6d4106790 0x7fc6d4106bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33140/0 (socket says 172.21.15.131:33140) 2024-05-08T21:59:25.362 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.729+0000 7fc6d3400700 1 -- 172.21.15.131:0/2897403150 learned_addr learned my addr 172.21.15.131:0/2897403150 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:25.362 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.730+0000 7fc6d3400700 1 -- 172.21.15.131:0/2897403150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d4107940 con 0x7fc6d4106790 2024-05-08T21:59:25.362 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.730+0000 7fc6d3400700 1 --2- 172.21.15.131:0/2897403150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6d4106790 0x7fc6d4106bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc6c40098a0 tx=0x7fc6c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e2680c42b3e70c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:25.362 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.730+0000 7fc6d2000700 1 -- 172.21.15.131:0/2897403150 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6c4004160 con 0x7fc6d4106790 2024-05-08T21:59:25.363 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.730+0000 7fc6d2000700 1 -- 172.21.15.131:0/2897403150 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc6c40042c0 con 0x7fc6d4106790 2024-05-08T21:59:25.363 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.730+0000 7fc6d2000700 1 -- 172.21.15.131:0/2897403150 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6c4011a00 con 0x7fc6d4106790 2024-05-08T21:59:25.363 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.731+0000 7fc6da200700 1 -- 172.21.15.131:0/2897403150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6d4106790 msgr2=0x7fc6d4106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:25.363 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.731+0000 7fc6da200700 1 --2- 172.21.15.131:0/2897403150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6d4106790 0x7fc6d4106bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc6c40098a0 tx=0x7fc6c4009bb0 comp rx=0 tx=0).stop 2024-05-08T21:59:25.363 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.731+0000 7fc6da200700 1 -- 172.21.15.131:0/2897403150 shutdown_connections 2024-05-08T21:59:25.363 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.731+0000 7fc6da200700 1 --2- 172.21.15.131:0/2897403150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6d4106790 0x7fc6d4106bb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:25.363 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.731+0000 7fc6da200700 1 -- 172.21.15.131:0/2897403150 >> 172.21.15.131:0/2897403150 conn(0x7fc6d4101d50 msgr2=0x7fc6d4104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:25.363 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.731+0000 7fc6da200700 1 -- 172.21.15.131:0/2897403150 shutdown_connections 2024-05-08T21:59:25.364 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.731+0000 7fc6da200700 1 -- 172.21.15.131:0/2897403150 wait complete. 2024-05-08T21:59:25.364 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.732+0000 7fc6da200700 1 Processor -- start 2024-05-08T21:59:25.364 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.732+0000 7fc6da200700 1 -- start start 2024-05-08T21:59:25.364 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.732+0000 7fc6da200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6d4106790 0x7fc6d407bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:25.364 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.732+0000 7fc6da200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d407c060 con 0x7fc6d4106790 2024-05-08T21:59:25.364 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.733+0000 7fc6d3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6d4106790 0x7fc6d407bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:25.364 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.733+0000 7fc6d3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6d4106790 0x7fc6d407bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33148/0 (socket says 172.21.15.131:33148) 2024-05-08T21:59:25.364 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.733+0000 7fc6d3400700 1 -- 172.21.15.131:0/2519950821 learned_addr learned my addr 172.21.15.131:0/2519950821 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:25.364 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.733+0000 7fc6d3400700 1 -- 172.21.15.131:0/2519950821 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c4009550 con 0x7fc6d4106790 2024-05-08T21:59:25.364 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.733+0000 7fc6d3400700 1 --2- 172.21.15.131:0/2519950821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6d4106790 0x7fc6d407bb20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4011710 tx=0x7fc6c4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:25.364 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.733+0000 7fc6cbe00700 1 -- 172.21.15.131:0/2519950821 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6c4003aa0 con 0x7fc6d4106790 2024-05-08T21:59:25.364 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.734+0000 7fc6cbe00700 1 -- 172.21.15.131:0/2519950821 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc6c4003c00 con 0x7fc6d4106790 2024-05-08T21:59:25.365 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.734+0000 7fc6da200700 1 -- 172.21.15.131:0/2519950821 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d407a2a0 con 0x7fc6d4106790 2024-05-08T21:59:25.365 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.734+0000 7fc6cbe00700 1 -- 172.21.15.131:0/2519950821 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6c4024800 con 0x7fc6d4106790 2024-05-08T21:59:25.365 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.734+0000 7fc6da200700 1 -- 172.21.15.131:0/2519950821 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d407a740 con 0x7fc6d4106790 2024-05-08T21:59:25.365 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.735+0000 7fc6da200700 1 -- 172.21.15.131:0/2519950821 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d40625d0 con 0x7fc6d4106790 2024-05-08T21:59:25.365 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.735+0000 7fc6cbe00700 1 -- 172.21.15.131:0/2519950821 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fc6c401abd0 con 0x7fc6d4106790 2024-05-08T21:59:25.365 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.735+0000 7fc6cbe00700 1 --2- 172.21.15.131:0/2519950821 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7fc6b8038750 0x7fc6b803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:25.365 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.735+0000 7fc6cbe00700 1 -- 172.21.15.131:0/2519950821 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc6c404c6f0 con 0x7fc6d4106790 2024-05-08T21:59:25.365 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.736+0000 7fc6d2a00700 1 --2- 172.21.15.131:0/2519950821 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7fc6b8038750 0x7fc6b803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:25.365 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.736+0000 7fc6d2a00700 1 --2- 172.21.15.131:0/2519950821 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7fc6b8038750 0x7fc6b803ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0006fd0 tx=0x7fc6c0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:25.365 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.740+0000 7fc6cbe00700 1 -- 172.21.15.131:0/2519950821 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc6c4018760 con 0x7fc6d4106790 2024-05-08T21:59:25.365 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:21.947+0000 7fc6da200700 1 -- 172.21.15.131:0/2519950821 --> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi131", "addr": "172.21.15.131", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6d407ada0 con 0x7fc6b8038750 2024-05-08T21:59:25.365 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:24.966+0000 7fc6cbe00700 1 -- 172.21.15.131:0/2519950821 <== mgr.14118 v2:172.21.15.131:6800/1418340798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fc6d407ada0 con 0x7fc6b8038750 2024-05-08T21:59:25.365 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:24.969+0000 7fc6da200700 1 -- 172.21.15.131:0/2519950821 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7fc6b8038750 msgr2=0x7fc6b803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:25.365 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:24.969+0000 7fc6da200700 1 --2- 172.21.15.131:0/2519950821 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7fc6b8038750 0x7fc6b803ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0006fd0 tx=0x7fc6c0006e40 comp rx=0 tx=0).stop 2024-05-08T21:59:25.366 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:24.969+0000 7fc6da200700 1 -- 172.21.15.131:0/2519950821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6d4106790 msgr2=0x7fc6d407bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:25.366 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:24.969+0000 7fc6da200700 1 --2- 172.21.15.131:0/2519950821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6d4106790 0x7fc6d407bb20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4011710 tx=0x7fc6c4011ad0 comp rx=0 tx=0).stop 2024-05-08T21:59:25.366 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:24.969+0000 7fc6da200700 1 -- 172.21.15.131:0/2519950821 shutdown_connections 2024-05-08T21:59:25.366 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:24.969+0000 7fc6da200700 1 --2- 172.21.15.131:0/2519950821 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7fc6b8038750 0x7fc6b803ac10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:25.366 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:24.969+0000 7fc6da200700 1 --2- 172.21.15.131:0/2519950821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6d4106790 0x7fc6d407bb20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:25.366 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:24.969+0000 7fc6da200700 1 -- 172.21.15.131:0/2519950821 >> 172.21.15.131:0/2519950821 conn(0x7fc6d4101d50 msgr2=0x7fc6d4102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:25.366 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:24.969+0000 7fc6da200700 1 -- 172.21.15.131:0/2519950821 shutdown_connections 2024-05-08T21:59:25.366 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:24.969+0000 7fc6da200700 1 -- 172.21.15.131:0/2519950821 wait complete. 2024-05-08T21:59:25.366 INFO:teuthology.orchestra.run.smithi131.stdout:Deploying unmanaged mon service... 2024-05-08T21:59:26.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:25 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' 2024-05-08T21:59:26.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:25 smithi131 ceph-mon[37260]: Added host smithi131 2024-05-08T21:59:26.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:25 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:59:26.723 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-08T21:59:26.724 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.856+0000 7f586a200700 1 Processor -- start 2024-05-08T21:59:26.724 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.856+0000 7f586a200700 1 -- start start 2024-05-08T21:59:26.724 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.856+0000 7f586a200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f586407be10 0x7f586407a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:26.724 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.856+0000 7f586a200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586407a800 con 0x7f586407be10 2024-05-08T21:59:26.724 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.857+0000 7f5868e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f586407be10 0x7f586407a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:26.724 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.857+0000 7f5868e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f586407be10 0x7f586407a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48990/0 (socket says 172.21.15.131:48990) 2024-05-08T21:59:26.724 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.857+0000 7f5868e00700 1 -- 172.21.15.131:0/3268427220 learned_addr learned my addr 172.21.15.131:0/3268427220 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:26.724 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.857+0000 7f5868e00700 1 -- 172.21.15.131:0/3268427220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f586407a940 con 0x7f586407be10 2024-05-08T21:59:26.724 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.857+0000 7f5868e00700 1 --2- 172.21.15.131:0/3268427220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f586407be10 0x7f586407a2c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f58580098a0 tx=0x7f5858009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236829fb09983b2c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:26.724 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.858+0000 7f5863400700 1 -- 172.21.15.131:0/3268427220 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5858004160 con 0x7f586407be10 2024-05-08T21:59:26.724 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.858+0000 7f5863400700 1 -- 172.21.15.131:0/3268427220 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f58580042c0 con 0x7f586407be10 2024-05-08T21:59:26.724 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.858+0000 7f5863400700 1 -- 172.21.15.131:0/3268427220 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5858011a00 con 0x7f586407be10 2024-05-08T21:59:26.725 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.858+0000 7f586a200700 1 -- 172.21.15.131:0/3268427220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f586407be10 msgr2=0x7f586407a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:26.725 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.858+0000 7f586a200700 1 --2- 172.21.15.131:0/3268427220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f586407be10 0x7f586407a2c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f58580098a0 tx=0x7f5858009bb0 comp rx=0 tx=0).stop 2024-05-08T21:59:26.725 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.859+0000 7f586a200700 1 -- 172.21.15.131:0/3268427220 shutdown_connections 2024-05-08T21:59:26.725 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.859+0000 7f586a200700 1 --2- 172.21.15.131:0/3268427220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f586407be10 0x7f586407a2c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:26.725 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.859+0000 7f586a200700 1 -- 172.21.15.131:0/3268427220 >> 172.21.15.131:0/3268427220 conn(0x7f5864101bc0 msgr2=0x7f5864103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:26.725 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.859+0000 7f586a200700 1 -- 172.21.15.131:0/3268427220 shutdown_connections 2024-05-08T21:59:26.725 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.859+0000 7f586a200700 1 -- 172.21.15.131:0/3268427220 wait complete. 2024-05-08T21:59:26.725 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.860+0000 7f586a200700 1 Processor -- start 2024-05-08T21:59:26.725 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.860+0000 7f586a200700 1 -- start start 2024-05-08T21:59:26.725 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.861+0000 7f586a200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f586407be10 0x7f586419c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:26.725 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.861+0000 7f586a200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586419ca20 con 0x7f586407be10 2024-05-08T21:59:26.725 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.861+0000 7f5868e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f586407be10 0x7f586419c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:26.725 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.861+0000 7f5868e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f586407be10 0x7f586419c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49006/0 (socket says 172.21.15.131:49006) 2024-05-08T21:59:26.726 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.861+0000 7f5868e00700 1 -- 172.21.15.131:0/274676268 learned_addr learned my addr 172.21.15.131:0/274676268 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:26.726 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.861+0000 7f5868e00700 1 -- 172.21.15.131:0/274676268 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5858009550 con 0x7f586407be10 2024-05-08T21:59:26.726 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.862+0000 7f5868e00700 1 --2- 172.21.15.131:0/274676268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f586407be10 0x7f586419c4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5858011a20 tx=0x7f5858011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:26.726 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.862+0000 7f5861600700 1 -- 172.21.15.131:0/274676268 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5858003af0 con 0x7f586407be10 2024-05-08T21:59:26.726 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.862+0000 7f5861600700 1 -- 172.21.15.131:0/274676268 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5858003c50 con 0x7f586407be10 2024-05-08T21:59:26.726 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.862+0000 7f5861600700 1 -- 172.21.15.131:0/274676268 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5858019760 con 0x7f586407be10 2024-05-08T21:59:26.726 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.862+0000 7f586a200700 1 -- 172.21.15.131:0/274676268 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f586419cc20 con 0x7f586407be10 2024-05-08T21:59:26.726 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.862+0000 7f586a200700 1 -- 172.21.15.131:0/274676268 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f586419d0c0 con 0x7f586407be10 2024-05-08T21:59:26.726 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.863+0000 7f5861600700 1 -- 172.21.15.131:0/274676268 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f585801a430 con 0x7f586407be10 2024-05-08T21:59:26.726 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.863+0000 7f586a200700 1 -- 172.21.15.131:0/274676268 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f586410c950 con 0x7f586407be10 2024-05-08T21:59:26.726 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.864+0000 7f5861600700 1 --2- 172.21.15.131:0/274676268 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f584c038700 0x7f584c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:26.726 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.864+0000 7f5861600700 1 -- 172.21.15.131:0/274676268 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f585804c9d0 con 0x7f586407be10 2024-05-08T21:59:26.727 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.864+0000 7f5863e00700 1 --2- 172.21.15.131:0/274676268 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f584c038700 0x7f584c03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:26.727 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.864+0000 7f5863e00700 1 --2- 172.21.15.131:0/274676268 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f584c038700 0x7f584c03abc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5850006fd0 tx=0x7f5850006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:26.727 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:25.873+0000 7f5861600700 1 -- 172.21.15.131:0/274676268 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f585802d9a0 con 0x7f586407be10 2024-05-08T21:59:26.727 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:26.103+0000 7f586a200700 1 -- 172.21.15.131:0/274676268 --> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f5864102ea0 con 0x7f584c038700 2024-05-08T21:59:26.727 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:26.109+0000 7f5861600700 1 -- 172.21.15.131:0/274676268 <== mgr.14118 v2:172.21.15.131:6800/1418340798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f5864102ea0 con 0x7f584c038700 2024-05-08T21:59:26.727 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:26.111+0000 7f586a200700 1 -- 172.21.15.131:0/274676268 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f584c038700 msgr2=0x7f584c03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:26.727 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:26.112+0000 7f586a200700 1 --2- 172.21.15.131:0/274676268 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f584c038700 0x7f584c03abc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5850006fd0 tx=0x7f5850006e40 comp rx=0 tx=0).stop 2024-05-08T21:59:26.727 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:26.112+0000 7f586a200700 1 -- 172.21.15.131:0/274676268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f586407be10 msgr2=0x7f586419c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:26.727 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:26.112+0000 7f586a200700 1 --2- 172.21.15.131:0/274676268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f586407be10 0x7f586419c4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5858011a20 tx=0x7f5858011b00 comp rx=0 tx=0).stop 2024-05-08T21:59:26.727 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:26.112+0000 7f586a200700 1 -- 172.21.15.131:0/274676268 shutdown_connections 2024-05-08T21:59:26.727 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:26.112+0000 7f586a200700 1 --2- 172.21.15.131:0/274676268 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f584c038700 0x7f584c03abc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:26.727 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:26.112+0000 7f586a200700 1 --2- 172.21.15.131:0/274676268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f586407be10 0x7f586419c4e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:26.727 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:26.112+0000 7f586a200700 1 -- 172.21.15.131:0/274676268 >> 172.21.15.131:0/274676268 conn(0x7f5864101bc0 msgr2=0x7f5864102790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:26.728 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:26.112+0000 7f586a200700 1 -- 172.21.15.131:0/274676268 shutdown_connections 2024-05-08T21:59:26.728 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:26.112+0000 7f586a200700 1 -- 172.21.15.131:0/274676268 wait complete. 2024-05-08T21:59:26.728 INFO:teuthology.orchestra.run.smithi131.stdout:Deploying unmanaged mgr service... 2024-05-08T21:59:27.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:27 smithi131 ceph-mon[37260]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:59:27.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:27 smithi131 ceph-mon[37260]: Saving service mon spec with placement count:5 2024-05-08T21:59:27.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:27 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' 2024-05-08T21:59:27.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:27 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' 2024-05-08T21:59:27.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:27 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' 2024-05-08T21:59:27.802 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-08T21:59:27.802 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.165+0000 7f5d72200700 1 Processor -- start 2024-05-08T21:59:27.802 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.166+0000 7f5d72200700 1 -- start start 2024-05-08T21:59:27.802 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.166+0000 7f5d72200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d6c106790 0x7f5d6c106bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:27.802 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.166+0000 7f5d72200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c107180 con 0x7f5d6c106790 2024-05-08T21:59:27.802 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.167+0000 7f5d6b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d6c106790 0x7f5d6c106bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:27.802 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.167+0000 7f5d6b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d6c106790 0x7f5d6c106bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49012/0 (socket says 172.21.15.131:49012) 2024-05-08T21:59:27.802 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.167+0000 7f5d6b400700 1 -- 172.21.15.131:0/2349548422 learned_addr learned my addr 172.21.15.131:0/2349548422 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:27.802 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.167+0000 7f5d6b400700 1 -- 172.21.15.131:0/2349548422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d6c107940 con 0x7f5d6c106790 2024-05-08T21:59:27.802 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.167+0000 7f5d6b400700 1 --2- 172.21.15.131:0/2349548422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d6c106790 0x7f5d6c106bb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0098a0 tx=0x7f5d5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe700990e163241 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:27.802 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.167+0000 7f5d6a000700 1 -- 172.21.15.131:0/2349548422 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d5c004160 con 0x7f5d6c106790 2024-05-08T21:59:27.802 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.168+0000 7f5d6a000700 1 -- 172.21.15.131:0/2349548422 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d5c0042c0 con 0x7f5d6c106790 2024-05-08T21:59:27.803 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.168+0000 7f5d6a000700 1 -- 172.21.15.131:0/2349548422 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d5c0119b0 con 0x7f5d6c106790 2024-05-08T21:59:27.803 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.168+0000 7f5d72200700 1 -- 172.21.15.131:0/2349548422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d6c106790 msgr2=0x7f5d6c106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:27.803 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.168+0000 7f5d72200700 1 --2- 172.21.15.131:0/2349548422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d6c106790 0x7f5d6c106bb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0098a0 tx=0x7f5d5c009bb0 comp rx=0 tx=0).stop 2024-05-08T21:59:27.803 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.169+0000 7f5d72200700 1 -- 172.21.15.131:0/2349548422 shutdown_connections 2024-05-08T21:59:27.803 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.169+0000 7f5d72200700 1 --2- 172.21.15.131:0/2349548422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d6c106790 0x7f5d6c106bb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:27.803 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.169+0000 7f5d72200700 1 -- 172.21.15.131:0/2349548422 >> 172.21.15.131:0/2349548422 conn(0x7f5d6c101d50 msgr2=0x7f5d6c104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:27.803 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.169+0000 7f5d72200700 1 -- 172.21.15.131:0/2349548422 shutdown_connections 2024-05-08T21:59:27.803 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.169+0000 7f5d72200700 1 -- 172.21.15.131:0/2349548422 wait complete. 2024-05-08T21:59:27.803 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.170+0000 7f5d72200700 1 Processor -- start 2024-05-08T21:59:27.803 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.170+0000 7f5d72200700 1 -- start start 2024-05-08T21:59:27.803 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.170+0000 7f5d72200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d6c106790 0x7f5d6c19c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:27.803 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.170+0000 7f5d72200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c19cd20 con 0x7f5d6c106790 2024-05-08T21:59:27.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.170+0000 7f5d6b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d6c106790 0x7f5d6c19c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:27.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.170+0000 7f5d6b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d6c106790 0x7f5d6c19c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49018/0 (socket says 172.21.15.131:49018) 2024-05-08T21:59:27.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.170+0000 7f5d6b400700 1 -- 172.21.15.131:0/180521917 learned_addr learned my addr 172.21.15.131:0/180521917 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:27.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.171+0000 7f5d6b400700 1 -- 172.21.15.131:0/180521917 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5c009550 con 0x7f5d6c106790 2024-05-08T21:59:27.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.171+0000 7f5d6b400700 1 --2- 172.21.15.131:0/180521917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d6c106790 0x7f5d6c19c7e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c011670 tx=0x7f5d5c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:27.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.171+0000 7f5d63e00700 1 -- 172.21.15.131:0/180521917 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d5c003af0 con 0x7f5d6c106790 2024-05-08T21:59:27.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.171+0000 7f5d63e00700 1 -- 172.21.15.131:0/180521917 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d5c003c50 con 0x7f5d6c106790 2024-05-08T21:59:27.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.171+0000 7f5d72200700 1 -- 172.21.15.131:0/180521917 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d6c19cf20 con 0x7f5d6c106790 2024-05-08T21:59:27.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.171+0000 7f5d63e00700 1 -- 172.21.15.131:0/180521917 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d5c024820 con 0x7f5d6c106790 2024-05-08T21:59:27.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.171+0000 7f5d72200700 1 -- 172.21.15.131:0/180521917 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d6c19d3c0 con 0x7f5d6c106790 2024-05-08T21:59:27.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.172+0000 7f5d63e00700 1 -- 172.21.15.131:0/180521917 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f5d5c024aa0 con 0x7f5d6c106790 2024-05-08T21:59:27.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.173+0000 7f5d63e00700 1 --2- 172.21.15.131:0/180521917 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f5d50038700 0x7f5d5003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:27.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.173+0000 7f5d63e00700 1 -- 172.21.15.131:0/180521917 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5d5c04c810 con 0x7f5d6c106790 2024-05-08T21:59:27.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.173+0000 7f5d72200700 1 -- 172.21.15.131:0/180521917 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d6c10ab70 con 0x7f5d6c106790 2024-05-08T21:59:27.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.173+0000 7f5d6aa00700 1 --2- 172.21.15.131:0/180521917 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f5d50038700 0x7f5d5003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:27.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.174+0000 7f5d6aa00700 1 --2- 172.21.15.131:0/180521917 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f5d50038700 0x7f5d5003abc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5d58006fd0 tx=0x7f5d58006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:27.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.178+0000 7f5d63e00700 1 -- 172.21.15.131:0/180521917 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5d5c010380 con 0x7f5d6c106790 2024-05-08T21:59:27.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.438+0000 7f5d72200700 1 -- 172.21.15.131:0/180521917 --> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f5d6c1030b0 con 0x7f5d50038700 2024-05-08T21:59:27.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.444+0000 7f5d63e00700 1 -- 172.21.15.131:0/180521917 <== mgr.14118 v2:172.21.15.131:6800/1418340798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f5d6c1030b0 con 0x7f5d50038700 2024-05-08T21:59:27.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.447+0000 7f5d72200700 1 -- 172.21.15.131:0/180521917 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f5d50038700 msgr2=0x7f5d5003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:27.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.447+0000 7f5d72200700 1 --2- 172.21.15.131:0/180521917 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f5d50038700 0x7f5d5003abc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5d58006fd0 tx=0x7f5d58006e40 comp rx=0 tx=0).stop 2024-05-08T21:59:27.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.447+0000 7f5d72200700 1 -- 172.21.15.131:0/180521917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d6c106790 msgr2=0x7f5d6c19c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:27.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.447+0000 7f5d72200700 1 --2- 172.21.15.131:0/180521917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d6c106790 0x7f5d6c19c7e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c011670 tx=0x7f5d5c011a30 comp rx=0 tx=0).stop 2024-05-08T21:59:27.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.447+0000 7f5d72200700 1 -- 172.21.15.131:0/180521917 shutdown_connections 2024-05-08T21:59:27.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.447+0000 7f5d72200700 1 --2- 172.21.15.131:0/180521917 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f5d50038700 0x7f5d5003abc0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:27.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.447+0000 7f5d72200700 1 --2- 172.21.15.131:0/180521917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d6c106790 0x7f5d6c19c7e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:27.806 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.447+0000 7f5d72200700 1 -- 172.21.15.131:0/180521917 >> 172.21.15.131:0/180521917 conn(0x7f5d6c101d50 msgr2=0x7f5d6c1029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:27.806 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.447+0000 7f5d72200700 1 -- 172.21.15.131:0/180521917 shutdown_connections 2024-05-08T21:59:27.806 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:27.447+0000 7f5d72200700 1 -- 172.21.15.131:0/180521917 wait complete. 2024-05-08T21:59:28.701 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:28 smithi131 ceph-mon[37260]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:59:28.701 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:28 smithi131 ceph-mon[37260]: Saving service mgr spec with placement count:2 2024-05-08T21:59:28.701 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:28 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' 2024-05-08T21:59:29.085 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.434+0000 7fd996000700 1 Processor -- start 2024-05-08T21:59:29.085 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.434+0000 7fd996000700 1 -- start start 2024-05-08T21:59:29.085 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.434+0000 7fd996000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd990107250 0x7fd990109640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:29.085 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.434+0000 7fd996000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd990079e30 con 0x7fd990107250 2024-05-08T21:59:29.085 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.434+0000 7fd98f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd990107250 0x7fd990109640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:29.085 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.435+0000 7fd98f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd990107250 0x7fd990109640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49032/0 (socket says 172.21.15.131:49032) 2024-05-08T21:59:29.085 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.435+0000 7fd98f400700 1 -- 172.21.15.131:0/1293222430 learned_addr learned my addr 172.21.15.131:0/1293222430 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:29.085 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.435+0000 7fd98f400700 1 -- 172.21.15.131:0/1293222430 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd990109b80 con 0x7fd990107250 2024-05-08T21:59:29.086 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.435+0000 7fd98f400700 1 --2- 172.21.15.131:0/1293222430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd990107250 0x7fd990109640 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd9800098a0 tx=0x7fd980009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1de9559b12967 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:29.086 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.435+0000 7fd98ea00700 1 -- 172.21.15.131:0/1293222430 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd980004160 con 0x7fd990107250 2024-05-08T21:59:29.086 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.435+0000 7fd98ea00700 1 -- 172.21.15.131:0/1293222430 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9800042c0 con 0x7fd990107250 2024-05-08T21:59:29.086 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.435+0000 7fd98ea00700 1 -- 172.21.15.131:0/1293222430 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9800119b0 con 0x7fd990107250 2024-05-08T21:59:29.086 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.436+0000 7fd996000700 1 -- 172.21.15.131:0/1293222430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd990107250 msgr2=0x7fd990109640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:29.086 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.436+0000 7fd996000700 1 --2- 172.21.15.131:0/1293222430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd990107250 0x7fd990109640 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd9800098a0 tx=0x7fd980009bb0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.086 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.436+0000 7fd996000700 1 -- 172.21.15.131:0/1293222430 shutdown_connections 2024-05-08T21:59:29.086 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.436+0000 7fd996000700 1 --2- 172.21.15.131:0/1293222430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd990107250 0x7fd990109640 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.086 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.436+0000 7fd996000700 1 -- 172.21.15.131:0/1293222430 >> 172.21.15.131:0/1293222430 conn(0x7fd990100c20 msgr2=0x7fd990103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:29.087 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.436+0000 7fd996000700 1 -- 172.21.15.131:0/1293222430 shutdown_connections 2024-05-08T21:59:29.087 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.436+0000 7fd996000700 1 -- 172.21.15.131:0/1293222430 wait complete. 2024-05-08T21:59:29.087 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.437+0000 7fd996000700 1 Processor -- start 2024-05-08T21:59:29.087 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.437+0000 7fd996000700 1 -- start start 2024-05-08T21:59:29.087 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.438+0000 7fd996000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd990107250 0x7fd99019c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:29.087 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.438+0000 7fd996000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99019cbd0 con 0x7fd990107250 2024-05-08T21:59:29.087 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.438+0000 7fd98f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd990107250 0x7fd99019c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:29.087 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.438+0000 7fd98f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd990107250 0x7fd99019c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49036/0 (socket says 172.21.15.131:49036) 2024-05-08T21:59:29.087 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.438+0000 7fd98f400700 1 -- 172.21.15.131:0/765281304 learned_addr learned my addr 172.21.15.131:0/765281304 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:29.087 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.438+0000 7fd98f400700 1 -- 172.21.15.131:0/765281304 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd980009550 con 0x7fd990107250 2024-05-08T21:59:29.087 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.438+0000 7fd98f400700 1 --2- 172.21.15.131:0/765281304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd990107250 0x7fd99019c690 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd980011670 tx=0x7fd980011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:29.087 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.439+0000 7fd98cc00700 1 -- 172.21.15.131:0/765281304 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd980003b20 con 0x7fd990107250 2024-05-08T21:59:29.087 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.439+0000 7fd98cc00700 1 -- 172.21.15.131:0/765281304 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd980003c80 con 0x7fd990107250 2024-05-08T21:59:29.087 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.439+0000 7fd996000700 1 -- 172.21.15.131:0/765281304 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd99019cdd0 con 0x7fd990107250 2024-05-08T21:59:29.088 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.439+0000 7fd98cc00700 1 -- 172.21.15.131:0/765281304 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd980019770 con 0x7fd990107250 2024-05-08T21:59:29.088 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.439+0000 7fd996000700 1 -- 172.21.15.131:0/765281304 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd99019d270 con 0x7fd990107250 2024-05-08T21:59:29.088 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.440+0000 7fd996000700 1 -- 172.21.15.131:0/765281304 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd99010a910 con 0x7fd990107250 2024-05-08T21:59:29.088 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.440+0000 7fd98cc00700 1 -- 172.21.15.131:0/765281304 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fd98001a430 con 0x7fd990107250 2024-05-08T21:59:29.088 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.440+0000 7fd98cc00700 1 --2- 172.21.15.131:0/765281304 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7fd974038750 0x7fd97403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:29.088 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.440+0000 7fd98cc00700 1 -- 172.21.15.131:0/765281304 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd98004c9e0 con 0x7fd990107250 2024-05-08T21:59:29.088 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.441+0000 7fd987e00700 1 --2- 172.21.15.131:0/765281304 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7fd974038750 0x7fd97403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:29.088 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.442+0000 7fd987e00700 1 --2- 172.21.15.131:0/765281304 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7fd974038750 0x7fd97403ac10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd978006fd0 tx=0x7fd978006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:29.088 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.446+0000 7fd98cc00700 1 -- 172.21.15.131:0/765281304 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd98002d9a0 con 0x7fd990107250 2024-05-08T21:59:29.088 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.669+0000 7fd996000700 1 -- 172.21.15.131:0/765281304 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fd99019fb00 con 0x7fd990107250 2024-05-08T21:59:29.088 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.673+0000 7fd98cc00700 1 -- 172.21.15.131:0/765281304 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fd98002d350 con 0x7fd990107250 2024-05-08T21:59:29.088 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.676+0000 7fd996000700 1 -- 172.21.15.131:0/765281304 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7fd974038750 msgr2=0x7fd97403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:29.088 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.676+0000 7fd996000700 1 --2- 172.21.15.131:0/765281304 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7fd974038750 0x7fd97403ac10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd978006fd0 tx=0x7fd978006e40 comp rx=0 tx=0).stop 2024-05-08T21:59:29.088 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.676+0000 7fd996000700 1 -- 172.21.15.131:0/765281304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd990107250 msgr2=0x7fd99019c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:29.089 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.676+0000 7fd996000700 1 --2- 172.21.15.131:0/765281304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd990107250 0x7fd99019c690 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd980011670 tx=0x7fd980011a30 comp rx=0 tx=0).stop 2024-05-08T21:59:29.089 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.676+0000 7fd996000700 1 -- 172.21.15.131:0/765281304 shutdown_connections 2024-05-08T21:59:29.089 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.676+0000 7fd996000700 1 --2- 172.21.15.131:0/765281304 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7fd974038750 0x7fd97403ac10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.089 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.676+0000 7fd996000700 1 --2- 172.21.15.131:0/765281304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd990107250 0x7fd99019c690 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:29.089 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.676+0000 7fd996000700 1 -- 172.21.15.131:0/765281304 >> 172.21.15.131:0/765281304 conn(0x7fd990100c20 msgr2=0x7fd990101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:29.089 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.676+0000 7fd996000700 1 -- 172.21.15.131:0/765281304 shutdown_connections 2024-05-08T21:59:29.089 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:28.676+0000 7fd996000700 1 -- 172.21.15.131:0/765281304 wait complete. 2024-05-08T21:59:30.052 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:29 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' 2024-05-08T21:59:30.052 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:29 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/765281304' entity='client.admin' 2024-05-08T21:59:30.232 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.602+0000 7ffaa5200700 1 Processor -- start 2024-05-08T21:59:30.232 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.602+0000 7ffaa5200700 1 -- start start 2024-05-08T21:59:30.232 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.602+0000 7ffaa5200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa0106790 0x7ffaa0106bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:30.232 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.602+0000 7ffaa5200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa0107180 con 0x7ffaa0106790 2024-05-08T21:59:30.232 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.603+0000 7ffa9de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa0106790 0x7ffaa0106bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:30.232 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.603+0000 7ffa9de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa0106790 0x7ffaa0106bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49042/0 (socket says 172.21.15.131:49042) 2024-05-08T21:59:30.232 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.603+0000 7ffa9de00700 1 -- 172.21.15.131:0/261434000 learned_addr learned my addr 172.21.15.131:0/261434000 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:30.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.603+0000 7ffa9de00700 1 -- 172.21.15.131:0/261434000 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa0107940 con 0x7ffaa0106790 2024-05-08T21:59:30.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.603+0000 7ffa9de00700 1 --2- 172.21.15.131:0/261434000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa0106790 0x7ffaa0106bb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffa900098a0 tx=0x7ffa90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd21cc48f7749ad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:30.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.604+0000 7ffa9ca00700 1 -- 172.21.15.131:0/261434000 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa90004160 con 0x7ffaa0106790 2024-05-08T21:59:30.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.604+0000 7ffa9ca00700 1 -- 172.21.15.131:0/261434000 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa900042c0 con 0x7ffaa0106790 2024-05-08T21:59:30.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.604+0000 7ffa9ca00700 1 -- 172.21.15.131:0/261434000 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa900119b0 con 0x7ffaa0106790 2024-05-08T21:59:30.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.604+0000 7ffaa5200700 1 -- 172.21.15.131:0/261434000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa0106790 msgr2=0x7ffaa0106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:30.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.604+0000 7ffaa5200700 1 --2- 172.21.15.131:0/261434000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa0106790 0x7ffaa0106bb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffa900098a0 tx=0x7ffa90009bb0 comp rx=0 tx=0).stop 2024-05-08T21:59:30.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.604+0000 7ffaa5200700 1 -- 172.21.15.131:0/261434000 shutdown_connections 2024-05-08T21:59:30.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.604+0000 7ffaa5200700 1 --2- 172.21.15.131:0/261434000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa0106790 0x7ffaa0106bb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:30.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.604+0000 7ffaa5200700 1 -- 172.21.15.131:0/261434000 >> 172.21.15.131:0/261434000 conn(0x7ffaa0101d50 msgr2=0x7ffaa0104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:30.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.605+0000 7ffaa5200700 1 -- 172.21.15.131:0/261434000 shutdown_connections 2024-05-08T21:59:30.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.605+0000 7ffaa5200700 1 -- 172.21.15.131:0/261434000 wait complete. 2024-05-08T21:59:30.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.606+0000 7ffaa5200700 1 Processor -- start 2024-05-08T21:59:30.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.606+0000 7ffaa5200700 1 -- start start 2024-05-08T21:59:30.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.606+0000 7ffaa5200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa0106790 0x7ffaa019c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:30.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.607+0000 7ffaa5200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa019cd20 con 0x7ffaa0106790 2024-05-08T21:59:30.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.607+0000 7ffa9de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa0106790 0x7ffaa019c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:30.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.607+0000 7ffa9de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa0106790 0x7ffaa019c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49058/0 (socket says 172.21.15.131:49058) 2024-05-08T21:59:30.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.607+0000 7ffa9de00700 1 -- 172.21.15.131:0/2725380913 learned_addr learned my addr 172.21.15.131:0/2725380913 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:30.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.607+0000 7ffa9de00700 1 -- 172.21.15.131:0/2725380913 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa90009550 con 0x7ffaa0106790 2024-05-08T21:59:30.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.607+0000 7ffa9de00700 1 --2- 172.21.15.131:0/2725380913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa0106790 0x7ffaa019c7e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffa90003e60 tx=0x7ffa90011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:30.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.608+0000 7ffa96a00700 1 -- 172.21.15.131:0/2725380913 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa900038b0 con 0x7ffaa0106790 2024-05-08T21:59:30.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.608+0000 7ffa96a00700 1 -- 172.21.15.131:0/2725380913 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa90003a10 con 0x7ffaa0106790 2024-05-08T21:59:30.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.608+0000 7ffaa5200700 1 -- 172.21.15.131:0/2725380913 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa019cf20 con 0x7ffaa0106790 2024-05-08T21:59:30.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.608+0000 7ffa96a00700 1 -- 172.21.15.131:0/2725380913 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa90019530 con 0x7ffaa0106790 2024-05-08T21:59:30.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.608+0000 7ffaa5200700 1 -- 172.21.15.131:0/2725380913 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa019d3c0 con 0x7ffaa0106790 2024-05-08T21:59:30.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.609+0000 7ffa96a00700 1 -- 172.21.15.131:0/2725380913 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7ffa900197a0 con 0x7ffaa0106790 2024-05-08T21:59:30.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.609+0000 7ffaa5200700 1 -- 172.21.15.131:0/2725380913 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa010ab70 con 0x7ffaa0106790 2024-05-08T21:59:30.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.609+0000 7ffa96a00700 1 --2- 172.21.15.131:0/2725380913 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7ffa84038700 0x7ffa8403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:30.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.609+0000 7ffa96a00700 1 -- 172.21.15.131:0/2725380913 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ffa9004c9d0 con 0x7ffaa0106790 2024-05-08T21:59:30.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.609+0000 7ffa9d400700 1 --2- 172.21.15.131:0/2725380913 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7ffa84038700 0x7ffa8403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:30.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.610+0000 7ffa9d400700 1 --2- 172.21.15.131:0/2725380913 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7ffa84038700 0x7ffa8403abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c006fd0 tx=0x7ffa8c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:30.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.615+0000 7ffa96a00700 1 -- 172.21.15.131:0/2725380913 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ffa9002b340 con 0x7ffaa0106790 2024-05-08T21:59:30.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.846+0000 7ffaa5200700 1 -- 172.21.15.131:0/2725380913 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7ffaa010ade0 con 0x7ffaa0106790 2024-05-08T21:59:30.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.850+0000 7ffa96a00700 1 -- 172.21.15.131:0/2725380913 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7ffa9000b3e0 con 0x7ffaa0106790 2024-05-08T21:59:30.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.853+0000 7ffaa5200700 1 -- 172.21.15.131:0/2725380913 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7ffa84038700 msgr2=0x7ffa8403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:30.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.853+0000 7ffaa5200700 1 --2- 172.21.15.131:0/2725380913 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7ffa84038700 0x7ffa8403abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c006fd0 tx=0x7ffa8c006e40 comp rx=0 tx=0).stop 2024-05-08T21:59:30.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.853+0000 7ffaa5200700 1 -- 172.21.15.131:0/2725380913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa0106790 msgr2=0x7ffaa019c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:30.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.853+0000 7ffaa5200700 1 --2- 172.21.15.131:0/2725380913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa0106790 0x7ffaa019c7e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffa90003e60 tx=0x7ffa90011a30 comp rx=0 tx=0).stop 2024-05-08T21:59:30.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.853+0000 7ffaa5200700 1 -- 172.21.15.131:0/2725380913 shutdown_connections 2024-05-08T21:59:30.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.853+0000 7ffaa5200700 1 --2- 172.21.15.131:0/2725380913 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7ffa84038700 0x7ffa8403abc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:30.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.853+0000 7ffaa5200700 1 --2- 172.21.15.131:0/2725380913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa0106790 0x7ffaa019c7e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:30.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.853+0000 7ffaa5200700 1 -- 172.21.15.131:0/2725380913 >> 172.21.15.131:0/2725380913 conn(0x7ffaa0101d50 msgr2=0x7ffaa01029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:30.236 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.853+0000 7ffaa5200700 1 -- 172.21.15.131:0/2725380913 shutdown_connections 2024-05-08T21:59:30.236 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:29.853+0000 7ffaa5200700 1 -- 172.21.15.131:0/2725380913 wait complete. 2024-05-08T21:59:30.236 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling the dashboard module... 2024-05-08T21:59:31.135 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:30 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2725380913' entity='client.admin' 2024-05-08T21:59:31.135 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:30 smithi131 ceph-mon[37260]: from='mgr.14118 172.21.15.131:0/4263778175' entity='mgr.x' 2024-05-08T21:59:32.105 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:31 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/421353424' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-08T21:59:32.105 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:31 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: ignoring --setuser ceph since I am not root 2024-05-08T21:59:32.105 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:31 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: ignoring --setgroup ceph since I am not root 2024-05-08T21:59:32.242 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.663+0000 7f2051800700 1 Processor -- start 2024-05-08T21:59:32.242 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.664+0000 7f2051800700 1 -- start start 2024-05-08T21:59:32.242 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.664+0000 7f2051800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f204c106780 0x7f204c106ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:32.242 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.664+0000 7f2051800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c107170 con 0x7f204c106780 2024-05-08T21:59:32.242 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.664+0000 7f204aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f204c106780 0x7f204c106ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:32.242 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.665+0000 7f204aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f204c106780 0x7f204c106ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49060/0 (socket says 172.21.15.131:49060) 2024-05-08T21:59:32.242 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.665+0000 7f204aa00700 1 -- 172.21.15.131:0/2002070678 learned_addr learned my addr 172.21.15.131:0/2002070678 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:32.242 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.665+0000 7f204aa00700 1 -- 172.21.15.131:0/2002070678 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c107930 con 0x7f204c106780 2024-05-08T21:59:32.243 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.665+0000 7f204aa00700 1 --2- 172.21.15.131:0/2002070678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f204c106780 0x7f204c106ba0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f203c0098a0 tx=0x7f203c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d63ec8880ed8ef0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:32.243 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.665+0000 7f2049600700 1 -- 172.21.15.131:0/2002070678 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f203c004160 con 0x7f204c106780 2024-05-08T21:59:32.243 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.665+0000 7f2049600700 1 -- 172.21.15.131:0/2002070678 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f203c0042c0 con 0x7f204c106780 2024-05-08T21:59:32.243 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.665+0000 7f2049600700 1 -- 172.21.15.131:0/2002070678 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f203c0119b0 con 0x7f204c106780 2024-05-08T21:59:32.243 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.666+0000 7f2051800700 1 -- 172.21.15.131:0/2002070678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f204c106780 msgr2=0x7f204c106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:32.243 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.666+0000 7f2051800700 1 --2- 172.21.15.131:0/2002070678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f204c106780 0x7f204c106ba0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f203c0098a0 tx=0x7f203c009bb0 comp rx=0 tx=0).stop 2024-05-08T21:59:32.243 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.666+0000 7f2051800700 1 -- 172.21.15.131:0/2002070678 shutdown_connections 2024-05-08T21:59:32.243 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.666+0000 7f2051800700 1 --2- 172.21.15.131:0/2002070678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f204c106780 0x7f204c106ba0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:32.243 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.666+0000 7f2051800700 1 -- 172.21.15.131:0/2002070678 >> 172.21.15.131:0/2002070678 conn(0x7f204c101d80 msgr2=0x7f204c1041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:32.243 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.666+0000 7f2051800700 1 -- 172.21.15.131:0/2002070678 shutdown_connections 2024-05-08T21:59:32.243 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.666+0000 7f2051800700 1 -- 172.21.15.131:0/2002070678 wait complete. 2024-05-08T21:59:32.243 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.668+0000 7f2051800700 1 Processor -- start 2024-05-08T21:59:32.243 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.668+0000 7f2051800700 1 -- start start 2024-05-08T21:59:32.243 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.668+0000 7f2051800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f204c106780 0x7f204c1a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:32.244 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.668+0000 7f2051800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c1a1070 con 0x7f204c106780 2024-05-08T21:59:32.244 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.668+0000 7f204aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f204c106780 0x7f204c1a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:32.244 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.669+0000 7f204aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f204c106780 0x7f204c1a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49062/0 (socket says 172.21.15.131:49062) 2024-05-08T21:59:32.244 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.669+0000 7f204aa00700 1 -- 172.21.15.131:0/421353424 learned_addr learned my addr 172.21.15.131:0/421353424 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:32.244 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.669+0000 7f204aa00700 1 -- 172.21.15.131:0/421353424 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c009550 con 0x7f204c106780 2024-05-08T21:59:32.244 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.669+0000 7f204aa00700 1 --2- 172.21.15.131:0/421353424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f204c106780 0x7f204c1a0b30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f203c003e60 tx=0x7f203c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:32.244 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.669+0000 7f2043400700 1 -- 172.21.15.131:0/421353424 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f203c0038b0 con 0x7f204c106780 2024-05-08T21:59:32.244 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.669+0000 7f2051800700 1 -- 172.21.15.131:0/421353424 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204c1a1270 con 0x7f204c106780 2024-05-08T21:59:32.244 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.669+0000 7f2043400700 1 -- 172.21.15.131:0/421353424 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f203c003a10 con 0x7f204c106780 2024-05-08T21:59:32.244 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.669+0000 7f2043400700 1 -- 172.21.15.131:0/421353424 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f203c019530 con 0x7f204c106780 2024-05-08T21:59:32.244 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.670+0000 7f2051800700 1 -- 172.21.15.131:0/421353424 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204c1a1710 con 0x7f204c106780 2024-05-08T21:59:32.244 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.671+0000 7f2043400700 1 -- 172.21.15.131:0/421353424 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f203c0197a0 con 0x7f204c106780 2024-05-08T21:59:32.244 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.671+0000 7f2051800700 1 -- 172.21.15.131:0/421353424 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f204c10ab60 con 0x7f204c106780 2024-05-08T21:59:32.244 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.671+0000 7f2043400700 1 --2- 172.21.15.131:0/421353424 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f2030038700 0x7f203003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:32.245 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.671+0000 7f2043400700 1 -- 172.21.15.131:0/421353424 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f203c04c7c0 con 0x7f204c106780 2024-05-08T21:59:32.245 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.671+0000 7f204a000700 1 --2- 172.21.15.131:0/421353424 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f2030038700 0x7f203003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:32.245 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.672+0000 7f204a000700 1 --2- 172.21.15.131:0/421353424 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f2030038700 0x7f203003abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2038006fd0 tx=0x7f2038006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:32.245 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.676+0000 7f2043400700 1 -- 172.21.15.131:0/421353424 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f203c02b340 con 0x7f204c106780 2024-05-08T21:59:32.245 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:30.959+0000 7f2051800700 1 -- 172.21.15.131:0/421353424 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f204c1a3fa0 con 0x7f204c106780 2024-05-08T21:59:32.245 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:31.856+0000 7f2043400700 1 -- 172.21.15.131:0/421353424 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f203c019d30 con 0x7f204c106780 2024-05-08T21:59:32.245 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:31.858+0000 7f2043400700 1 -- 172.21.15.131:0/421353424 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f203c003b80 con 0x7f204c106780 2024-05-08T21:59:32.245 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:31.861+0000 7f2051800700 1 -- 172.21.15.131:0/421353424 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f2030038700 msgr2=0x7f203003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:32.245 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:31.861+0000 7f2051800700 1 --2- 172.21.15.131:0/421353424 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f2030038700 0x7f203003abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2038006fd0 tx=0x7f2038006e40 comp rx=0 tx=0).stop 2024-05-08T21:59:32.245 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:31.861+0000 7f2051800700 1 -- 172.21.15.131:0/421353424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f204c106780 msgr2=0x7f204c1a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:32.245 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:31.861+0000 7f2051800700 1 --2- 172.21.15.131:0/421353424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f204c106780 0x7f204c1a0b30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f203c003e60 tx=0x7f203c011a30 comp rx=0 tx=0).stop 2024-05-08T21:59:32.245 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:31.861+0000 7f2051800700 1 -- 172.21.15.131:0/421353424 shutdown_connections 2024-05-08T21:59:32.245 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:31.861+0000 7f2051800700 1 --2- 172.21.15.131:0/421353424 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f2030038700 0x7f203003abc0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:32.245 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:31.861+0000 7f2051800700 1 --2- 172.21.15.131:0/421353424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f204c106780 0x7f204c1a0b30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:32.246 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:31.861+0000 7f2051800700 1 -- 172.21.15.131:0/421353424 >> 172.21.15.131:0/421353424 conn(0x7f204c101d80 msgr2=0x7f204c102980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:32.246 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:31.861+0000 7f2051800700 1 -- 172.21.15.131:0/421353424 shutdown_connections 2024-05-08T21:59:32.246 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:31.861+0000 7f2051800700 1 -- 172.21.15.131:0/421353424 wait complete. 2024-05-08T21:59:32.763 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:32 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:32.497+0000 7f9e0bf90200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T21:59:32.763 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:32 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:32.627+0000 7f9e0bf90200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T21:59:33.039 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:32 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/421353424' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-08T21:59:33.039 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:32 smithi131 ceph-mon[37260]: mgrmap e9: x(active, since 16s) 2024-05-08T21:59:33.040 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:32 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:32.762+0000 7f9e0bf90200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T21:59:33.040 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:32 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:32.874+0000 7f9e0bf90200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T21:59:33.397 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-05-08T21:59:33.397 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-08T21:59:33.397 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T21:59:33.397 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-08T21:59:33.398 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-08T21:59:33.398 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-05-08T21:59:33.398 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.716+0000 7fb01cc00700 1 Processor -- start 2024-05-08T21:59:33.398 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.717+0000 7fb01cc00700 1 -- start start 2024-05-08T21:59:33.398 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.717+0000 7fb01cc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb018105040 0x7fb018105460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:33.398 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.717+0000 7fb01cc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb018079e30 con 0x7fb018105040 2024-05-08T21:59:33.398 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.717+0000 7fb015e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb018105040 0x7fb018105460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:33.398 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.717+0000 7fb015e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb018105040 0x7fb018105460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49088/0 (socket says 172.21.15.131:49088) 2024-05-08T21:59:33.398 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.717+0000 7fb015e00700 1 -- 172.21.15.131:0/2465192185 learned_addr learned my addr 172.21.15.131:0/2465192185 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:33.398 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.718+0000 7fb015e00700 1 -- 172.21.15.131:0/2465192185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0181059a0 con 0x7fb018105040 2024-05-08T21:59:33.398 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.718+0000 7fb015e00700 1 --2- 172.21.15.131:0/2465192185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb018105040 0x7fb018105460 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb0080098a0 tx=0x7fb008009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84585f1c7a396709 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:33.399 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.718+0000 7fb014a00700 1 -- 172.21.15.131:0/2465192185 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb008004160 con 0x7fb018105040 2024-05-08T21:59:33.399 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.718+0000 7fb014a00700 1 -- 172.21.15.131:0/2465192185 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0080042c0 con 0x7fb018105040 2024-05-08T21:59:33.399 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.718+0000 7fb014a00700 1 -- 172.21.15.131:0/2465192185 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0080119b0 con 0x7fb018105040 2024-05-08T21:59:33.399 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.719+0000 7fb01cc00700 1 -- 172.21.15.131:0/2465192185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb018105040 msgr2=0x7fb018105460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:33.399 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.719+0000 7fb01cc00700 1 --2- 172.21.15.131:0/2465192185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb018105040 0x7fb018105460 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb0080098a0 tx=0x7fb008009bb0 comp rx=0 tx=0).stop 2024-05-08T21:59:33.399 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.719+0000 7fb01cc00700 1 -- 172.21.15.131:0/2465192185 shutdown_connections 2024-05-08T21:59:33.399 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.719+0000 7fb01cc00700 1 --2- 172.21.15.131:0/2465192185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb018105040 0x7fb018105460 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:33.399 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.719+0000 7fb01cc00700 1 -- 172.21.15.131:0/2465192185 >> 172.21.15.131:0/2465192185 conn(0x7fb018100c20 msgr2=0x7fb018103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:33.399 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.719+0000 7fb01cc00700 1 -- 172.21.15.131:0/2465192185 shutdown_connections 2024-05-08T21:59:33.399 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.719+0000 7fb01cc00700 1 -- 172.21.15.131:0/2465192185 wait complete. 2024-05-08T21:59:33.399 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.721+0000 7fb01cc00700 1 Processor -- start 2024-05-08T21:59:33.399 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.721+0000 7fb01cc00700 1 -- start start 2024-05-08T21:59:33.400 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.722+0000 7fb01cc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb018105040 0x7fb01819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:33.400 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.722+0000 7fb01cc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01819cba0 con 0x7fb018105040 2024-05-08T21:59:33.400 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.722+0000 7fb015e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb018105040 0x7fb01819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:33.400 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.722+0000 7fb015e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb018105040 0x7fb01819c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49090/0 (socket says 172.21.15.131:49090) 2024-05-08T21:59:33.400 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.722+0000 7fb015e00700 1 -- 172.21.15.131:0/3572445856 learned_addr learned my addr 172.21.15.131:0/3572445856 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:33.400 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.722+0000 7fb015e00700 1 -- 172.21.15.131:0/3572445856 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb008009550 con 0x7fb018105040 2024-05-08T21:59:33.400 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.722+0000 7fb015e00700 1 --2- 172.21.15.131:0/3572445856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb018105040 0x7fb01819c660 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb008011670 tx=0x7fb008011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:33.400 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.723+0000 7fb00ea00700 1 -- 172.21.15.131:0/3572445856 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb008003b20 con 0x7fb018105040 2024-05-08T21:59:33.400 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.723+0000 7fb00ea00700 1 -- 172.21.15.131:0/3572445856 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb008003c80 con 0x7fb018105040 2024-05-08T21:59:33.400 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.723+0000 7fb00ea00700 1 -- 172.21.15.131:0/3572445856 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb008019770 con 0x7fb018105040 2024-05-08T21:59:33.400 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.723+0000 7fb01cc00700 1 -- 172.21.15.131:0/3572445856 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01819cda0 con 0x7fb018105040 2024-05-08T21:59:33.401 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.723+0000 7fb01cc00700 1 -- 172.21.15.131:0/3572445856 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01819d240 con 0x7fb018105040 2024-05-08T21:59:33.401 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.724+0000 7fb00ea00700 1 -- 172.21.15.131:0/3572445856 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fb00801a430 con 0x7fb018105040 2024-05-08T21:59:33.401 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.724+0000 7fb01cc00700 1 -- 172.21.15.131:0/3572445856 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb018196370 con 0x7fb018105040 2024-05-08T21:59:33.401 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.724+0000 7fb00ea00700 1 --2- 172.21.15.131:0/3572445856 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7faffc0387a0 0x7faffc03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:33.401 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.724+0000 7fb00ea00700 1 -- 172.21.15.131:0/3572445856 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb00804c980 con 0x7fb018105040 2024-05-08T21:59:33.401 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.725+0000 7fb015400700 1 -- 172.21.15.131:0/3572445856 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7faffc0387a0 msgr2=0x7faffc03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/1418340798 2024-05-08T21:59:33.401 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.725+0000 7fb015400700 1 --2- 172.21.15.131:0/3572445856 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7faffc0387a0 0x7faffc03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T21:59:33.401 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.734+0000 7fb00ea00700 1 -- 172.21.15.131:0/3572445856 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb00802d9a0 con 0x7fb018105040 2024-05-08T21:59:33.402 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.926+0000 7fb015400700 1 -- 172.21.15.131:0/3572445856 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7faffc0387a0 msgr2=0x7faffc03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/1418340798 2024-05-08T21:59:33.402 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:32.926+0000 7fb015400700 1 --2- 172.21.15.131:0/3572445856 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7faffc0387a0 0x7faffc03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T21:59:33.402 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.013+0000 7fb01cc00700 1 -- 172.21.15.131:0/3572445856 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fb018106a20 con 0x7fb018105040 2024-05-08T21:59:33.402 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.014+0000 7fb00ea00700 1 -- 172.21.15.131:0/3572445856 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fb00802d350 con 0x7fb018105040 2024-05-08T21:59:33.402 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.016+0000 7fb01cc00700 1 -- 172.21.15.131:0/3572445856 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7faffc0387a0 msgr2=0x7faffc03ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T21:59:33.402 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.016+0000 7fb01cc00700 1 --2- 172.21.15.131:0/3572445856 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7faffc0387a0 0x7faffc03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:33.402 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.016+0000 7fb01cc00700 1 -- 172.21.15.131:0/3572445856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb018105040 msgr2=0x7fb01819c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:33.402 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.016+0000 7fb01cc00700 1 --2- 172.21.15.131:0/3572445856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb018105040 0x7fb01819c660 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb008011670 tx=0x7fb008011a30 comp rx=0 tx=0).stop 2024-05-08T21:59:33.402 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.016+0000 7fb01cc00700 1 -- 172.21.15.131:0/3572445856 shutdown_connections 2024-05-08T21:59:33.402 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.016+0000 7fb01cc00700 1 --2- 172.21.15.131:0/3572445856 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7faffc0387a0 0x7faffc03ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:33.402 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.016+0000 7fb01cc00700 1 --2- 172.21.15.131:0/3572445856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb018105040 0x7fb01819c660 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:33.402 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.016+0000 7fb01cc00700 1 -- 172.21.15.131:0/3572445856 >> 172.21.15.131:0/3572445856 conn(0x7fb018100c20 msgr2=0x7fb0181934b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:33.402 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.016+0000 7fb01cc00700 1 -- 172.21.15.131:0/3572445856 shutdown_connections 2024-05-08T21:59:33.403 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.017+0000 7fb01cc00700 1 -- 172.21.15.131:0/3572445856 wait complete. 2024-05-08T21:59:33.403 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for the mgr to restart... 2024-05-08T21:59:33.403 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mgr epoch 9... 2024-05-08T21:59:34.072 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:33 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3572445856' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-08T21:59:34.072 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:33 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:33.748+0000 7f9e0bf90200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:59:34.327 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:34 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:34.070+0000 7f9e0bf90200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T21:59:34.328 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:34 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:34.184+0000 7f9e0bf90200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T21:59:34.640 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:34 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:34.326+0000 7f9e0bf90200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T21:59:34.641 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:34 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:34.524+0000 7f9e0bf90200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T21:59:35.140 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:34 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:34.692+0000 7f9e0bf90200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T21:59:35.141 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:34 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:34.801+0000 7f9e0bf90200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T21:59:35.640 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:35 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:35.280+0000 7f9e0bf90200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T21:59:36.028 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:35 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:35.773+0000 7f9e0bf90200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T21:59:36.390 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:36 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:36.027+0000 7f9e0bf90200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T21:59:36.390 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:36 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:36.149+0000 7f9e0bf90200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T21:59:37.390 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:37 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:37.110+0000 7f9e0bf90200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T21:59:37.391 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:37 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:37.236+0000 7f9e0bf90200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T21:59:37.890 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:37 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:37.430+0000 7f9e0bf90200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T21:59:38.640 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:38 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:38.245+0000 7f9e0bf90200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T21:59:38.641 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:38 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:38.485+0000 7f9e0bf90200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T21:59:39.140 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:38 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:38.740+0000 7f9e0bf90200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T21:59:40.140 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:39 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:39.887+0000 7f9e0bf90200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T21:59:40.640 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:40 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:40.263+0000 7f9e0bf90200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:59:40.640 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:40 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:40.383+0000 7f9e0bf90200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T21:59:40.890 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 21:59:40 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T21:59:40.638+0000 7f9e0bf90200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T21:59:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:40 smithi131 ceph-mon[37260]: Active manager daemon x restarted 2024-05-08T21:59:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:40 smithi131 ceph-mon[37260]: Activating manager daemon x 2024-05-08T21:59:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:40 smithi131 ceph-mon[37260]: osdmap e3: 0 total, 0 up, 0 in 2024-05-08T21:59:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:40 smithi131 ceph-mon[37260]: mgrmap e10: x(active, starting, since 0.00408346s) 2024-05-08T21:59:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:59:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T21:59:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T21:59:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T21:59:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T21:59:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:40 smithi131 ceph-mon[37260]: Manager daemon x is now available 2024-05-08T21:59:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:59:41.959 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T21:59:41.959 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T21:59:41.959 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:41 smithi131 ceph-mon[37260]: mgrmap e11: x(active, since 1.00878s) 2024-05-08T21:59:42.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-05-08T21:59:42.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-08T21:59:42.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-08T21:59:42.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-05-08T21:59:42.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.851+0000 7f46f5a00700 1 Processor -- start 2024-05-08T21:59:42.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.851+0000 7f46f5a00700 1 -- start start 2024-05-08T21:59:42.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.851+0000 7f46f5a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46f01067b0 0x7f46f0106bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:42.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.851+0000 7f46f5a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f01071a0 con 0x7f46f01067b0 2024-05-08T21:59:42.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.851+0000 7f46eea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46f01067b0 0x7f46f0106bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:42.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.851+0000 7f46eea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46f01067b0 0x7f46f0106bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53382/0 (socket says 172.21.15.131:53382) 2024-05-08T21:59:42.102 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.852+0000 7f46eea00700 1 -- 172.21.15.131:0/4216164321 learned_addr learned my addr 172.21.15.131:0/4216164321 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:42.102 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.852+0000 7f46eea00700 1 -- 172.21.15.131:0/4216164321 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f0107960 con 0x7f46f01067b0 2024-05-08T21:59:42.102 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.852+0000 7f46eea00700 1 --2- 172.21.15.131:0/4216164321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46f01067b0 0x7f46f0106bd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f46e00098a0 tx=0x7f46e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f119ed7ff61da8d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:42.102 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.852+0000 7f46ee000700 1 -- 172.21.15.131:0/4216164321 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46e0004160 con 0x7f46f01067b0 2024-05-08T21:59:42.102 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.852+0000 7f46ee000700 1 -- 172.21.15.131:0/4216164321 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46e00042c0 con 0x7f46f01067b0 2024-05-08T21:59:42.102 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.852+0000 7f46ee000700 1 -- 172.21.15.131:0/4216164321 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46e00119b0 con 0x7f46f01067b0 2024-05-08T21:59:42.102 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.853+0000 7f46f5a00700 1 -- 172.21.15.131:0/4216164321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46f01067b0 msgr2=0x7f46f0106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:42.102 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.853+0000 7f46f5a00700 1 --2- 172.21.15.131:0/4216164321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46f01067b0 0x7f46f0106bd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f46e00098a0 tx=0x7f46e0009bb0 comp rx=0 tx=0).stop 2024-05-08T21:59:42.102 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.853+0000 7f46f5a00700 1 -- 172.21.15.131:0/4216164321 shutdown_connections 2024-05-08T21:59:42.102 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.853+0000 7f46f5a00700 1 --2- 172.21.15.131:0/4216164321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46f01067b0 0x7f46f0106bd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:42.102 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.853+0000 7f46f5a00700 1 -- 172.21.15.131:0/4216164321 >> 172.21.15.131:0/4216164321 conn(0x7f46f0101db0 msgr2=0x7f46f01041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:42.102 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.853+0000 7f46f5a00700 1 -- 172.21.15.131:0/4216164321 shutdown_connections 2024-05-08T21:59:42.102 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.853+0000 7f46f5a00700 1 -- 172.21.15.131:0/4216164321 wait complete. 2024-05-08T21:59:42.103 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.854+0000 7f46f5a00700 1 Processor -- start 2024-05-08T21:59:42.103 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.855+0000 7f46f5a00700 1 -- start start 2024-05-08T21:59:42.103 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.855+0000 7f46f5a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46f01067b0 0x7f46f019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:42.103 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.855+0000 7f46f5a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f019cc20 con 0x7f46f01067b0 2024-05-08T21:59:42.103 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.855+0000 7f46eea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46f01067b0 0x7f46f019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:42.103 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.855+0000 7f46eea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46f01067b0 0x7f46f019c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53388/0 (socket says 172.21.15.131:53388) 2024-05-08T21:59:42.103 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.855+0000 7f46eea00700 1 -- 172.21.15.131:0/4143768406 learned_addr learned my addr 172.21.15.131:0/4143768406 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:42.103 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.856+0000 7f46eea00700 1 -- 172.21.15.131:0/4143768406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e0009550 con 0x7f46f01067b0 2024-05-08T21:59:42.103 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.856+0000 7f46eea00700 1 --2- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46f01067b0 0x7f46f019c6e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f46e0003e60 tx=0x7f46e0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:42.103 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.860+0000 7f46e7400700 1 -- 172.21.15.131:0/4143768406 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46e0003a60 con 0x7f46f01067b0 2024-05-08T21:59:42.103 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.860+0000 7f46e7400700 1 -- 172.21.15.131:0/4143768406 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46e0003bc0 con 0x7f46f01067b0 2024-05-08T21:59:42.103 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.860+0000 7f46e7400700 1 -- 172.21.15.131:0/4143768406 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46e00196c0 con 0x7f46f01067b0 2024-05-08T21:59:42.103 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.860+0000 7f46f5a00700 1 -- 172.21.15.131:0/4143768406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f019ce20 con 0x7f46f01067b0 2024-05-08T21:59:42.103 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.860+0000 7f46f5a00700 1 -- 172.21.15.131:0/4143768406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f019d2c0 con 0x7f46f01067b0 2024-05-08T21:59:42.103 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.861+0000 7f46e7400700 1 -- 172.21.15.131:0/4143768406 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f46e0019930 con 0x7f46f01067b0 2024-05-08T21:59:42.104 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.862+0000 7f46e7400700 1 --2- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f46d4038780 0x7f46d403ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:42.104 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.862+0000 7f46f5a00700 1 -- 172.21.15.131:0/4143768406 --> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f46f010ab90 con 0x7f46d4038780 2024-05-08T21:59:42.104 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.862+0000 7f46e7400700 1 -- 172.21.15.131:0/4143768406 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f46e004cb20 con 0x7f46f01067b0 2024-05-08T21:59:42.104 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.862+0000 7f46e7e00700 1 -- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f46d4038780 msgr2=0x7f46d403ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/1418340798 2024-05-08T21:59:42.104 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:33.862+0000 7f46e7e00700 1 --2- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f46d4038780 0x7f46d403ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T21:59:42.104 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:34.063+0000 7f46e7e00700 1 -- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f46d4038780 msgr2=0x7f46d403ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/1418340798 2024-05-08T21:59:42.104 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:34.063+0000 7f46e7e00700 1 --2- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f46d4038780 0x7f46d403ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T21:59:42.104 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:34.464+0000 7f46e7e00700 1 -- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f46d4038780 msgr2=0x7f46d403ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/1418340798 2024-05-08T21:59:42.104 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:34.464+0000 7f46e7e00700 1 --2- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f46d4038780 0x7f46d403ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T21:59:42.104 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:35.265+0000 7f46e7e00700 1 -- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f46d4038780 msgr2=0x7f46d403ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/1418340798 2024-05-08T21:59:42.104 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:35.265+0000 7f46e7e00700 1 --2- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f46d4038780 0x7f46d403ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T21:59:42.104 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:36.866+0000 7f46e7e00700 1 -- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f46d4038780 msgr2=0x7f46d403ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/1418340798 2024-05-08T21:59:42.104 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:36.866+0000 7f46e7e00700 1 --2- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f46d4038780 0x7f46d403ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-08T21:59:42.104 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:40.069+0000 7f46e7e00700 1 -- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f46d4038780 msgr2=0x7f46d403ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/1418340798 2024-05-08T21:59:42.104 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:40.069+0000 7f46e7e00700 1 --2- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f46d4038780 0x7f46d403ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-05-08T21:59:42.105 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:40.643+0000 7f46e7400700 1 -- 172.21.15.131:0/4143768406 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mgrmap(e 10) v1 ==== 45321+0+0 (secure 0 0 0) 0x7f46e000fd50 con 0x7f46f01067b0 2024-05-08T21:59:42.105 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:40.643+0000 7f46e7400700 1 -- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f46d4038780 msgr2=0x7f46d403ac40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T21:59:42.105 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:40.643+0000 7f46e7400700 1 --2- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f46d4038780 0x7f46d403ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:42.105 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:41.646+0000 7f46e7400700 1 -- 172.21.15.131:0/4143768406 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 11) v1 ==== 45448+0+0 (secure 0 0 0) 0x7f46e004e660 con 0x7f46f01067b0 2024-05-08T21:59:42.105 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:41.646+0000 7f46e7400700 1 --2- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f46d403bff0 0x7f46d403e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:42.105 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:41.647+0000 7f46e7400700 1 -- 172.21.15.131:0/4143768406 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f46f010ab90 con 0x7f46d403bff0 2024-05-08T21:59:42.105 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:41.647+0000 7f46e7e00700 1 --2- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f46d403bff0 0x7f46d403e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:42.105 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:41.648+0000 7f46e7e00700 1 --2- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f46d403bff0 0x7f46d403e3e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f46d8003a10 tx=0x7f46d8009270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:42.105 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:41.649+0000 7f46e7400700 1 -- 172.21.15.131:0/4143768406 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f46f010ab90 con 0x7f46d403bff0 2024-05-08T21:59:42.105 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:41.668+0000 7f46f5a00700 1 -- 172.21.15.131:0/4143768406 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f46f010ad00 con 0x7f46d403bff0 2024-05-08T21:59:42.105 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:41.669+0000 7f46e7400700 1 -- 172.21.15.131:0/4143768406 <== mgr.14150 v2:172.21.15.131:6800/2485665042 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f46f010ad00 con 0x7f46d403bff0 2024-05-08T21:59:42.105 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:41.669+0000 7f46f5a00700 1 -- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f46d403bff0 msgr2=0x7f46d403e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:42.105 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:41.669+0000 7f46f5a00700 1 --2- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f46d403bff0 0x7f46d403e3e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f46d8003a10 tx=0x7f46d8009270 comp rx=0 tx=0).stop 2024-05-08T21:59:42.105 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:41.669+0000 7f46f5a00700 1 -- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46f01067b0 msgr2=0x7f46f019c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:42.105 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:41.669+0000 7f46f5a00700 1 --2- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46f01067b0 0x7f46f019c6e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f46e0003e60 tx=0x7f46e0011a30 comp rx=0 tx=0).stop 2024-05-08T21:59:42.106 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:41.669+0000 7f46f5a00700 1 -- 172.21.15.131:0/4143768406 shutdown_connections 2024-05-08T21:59:42.106 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:41.669+0000 7f46f5a00700 1 --2- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f46d403bff0 0x7f46d403e3e0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:42.106 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:41.669+0000 7f46f5a00700 1 --2- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46f01067b0 0x7f46f019c6e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:42.106 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:41.669+0000 7f46f5a00700 1 --2- 172.21.15.131:0/4143768406 >> [v2:172.21.15.131:6800/1418340798,v1:172.21.15.131:6801/1418340798] conn(0x7f46d4038780 0x7f46d403ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:42.106 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:41.669+0000 7f46f5a00700 1 -- 172.21.15.131:0/4143768406 >> 172.21.15.131:0/4143768406 conn(0x7f46f0101db0 msgr2=0x7f46f0102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:42.106 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:41.670+0000 7f46f5a00700 1 -- 172.21.15.131:0/4143768406 shutdown_connections 2024-05-08T21:59:42.106 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:41.670+0000 7f46f5a00700 1 -- 172.21.15.131:0/4143768406 wait complete. 2024-05-08T21:59:42.106 INFO:teuthology.orchestra.run.smithi131.stdout:mgr epoch 9 is available 2024-05-08T21:59:42.106 INFO:teuthology.orchestra.run.smithi131.stdout:Generating a dashboard self-signed certificate... 2024-05-08T21:59:43.038 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:42 smithi131 ceph-mon[37260]: [08/May/2024:21:59:41] ENGINE Bus STARTING 2024-05-08T21:59:43.038 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:42 smithi131 ceph-mon[37260]: [08/May/2024:21:59:41] ENGINE Serving on http://172.21.15.131:8765 2024-05-08T21:59:43.039 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:42 smithi131 ceph-mon[37260]: [08/May/2024:21:59:41] ENGINE Serving on https://172.21.15.131:7150 2024-05-08T21:59:43.039 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:42 smithi131 ceph-mon[37260]: [08/May/2024:21:59:41] ENGINE Bus STARTED 2024-05-08T21:59:43.039 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:42 smithi131 ceph-mon[37260]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-08T21:59:43.039 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:42 smithi131 ceph-mon[37260]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-08T21:59:43.039 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:42 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T21:59:43.407 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-08T21:59:43.407 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.526+0000 7f4976e00700 1 Processor -- start 2024-05-08T21:59:43.407 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.527+0000 7f4976e00700 1 -- start start 2024-05-08T21:59:43.407 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.527+0000 7f4976e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49701071d0 0x7f49701095c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:43.407 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.527+0000 7f4976e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4970079eb0 con 0x7f49701071d0 2024-05-08T21:59:43.407 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.528+0000 7f496fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49701071d0 0x7f49701095c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:43.407 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.528+0000 7f496fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49701071d0 0x7f49701095c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53452/0 (socket says 172.21.15.131:53452) 2024-05-08T21:59:43.407 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.528+0000 7f496fe00700 1 -- 172.21.15.131:0/2355407503 learned_addr learned my addr 172.21.15.131:0/2355407503 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:43.407 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.528+0000 7f496fe00700 1 -- 172.21.15.131:0/2355407503 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4970109b00 con 0x7f49701071d0 2024-05-08T21:59:43.407 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.528+0000 7f496fe00700 1 --2- 172.21.15.131:0/2355407503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49701071d0 0x7f49701095c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f49600098a0 tx=0x7f4960009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a81a65401b3e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:43.407 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.529+0000 7f496ea00700 1 -- 172.21.15.131:0/2355407503 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4960004160 con 0x7f49701071d0 2024-05-08T21:59:43.408 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.529+0000 7f496ea00700 1 -- 172.21.15.131:0/2355407503 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49600042c0 con 0x7f49701071d0 2024-05-08T21:59:43.408 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.529+0000 7f496ea00700 1 -- 172.21.15.131:0/2355407503 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49600119b0 con 0x7f49701071d0 2024-05-08T21:59:43.408 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.529+0000 7f4976e00700 1 -- 172.21.15.131:0/2355407503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49701071d0 msgr2=0x7f49701095c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:43.408 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.529+0000 7f4976e00700 1 --2- 172.21.15.131:0/2355407503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49701071d0 0x7f49701095c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f49600098a0 tx=0x7f4960009bb0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.408 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.529+0000 7f4976e00700 1 -- 172.21.15.131:0/2355407503 shutdown_connections 2024-05-08T21:59:43.408 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.529+0000 7f4976e00700 1 --2- 172.21.15.131:0/2355407503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49701071d0 0x7f49701095c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.408 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.529+0000 7f4976e00700 1 -- 172.21.15.131:0/2355407503 >> 172.21.15.131:0/2355407503 conn(0x7f4970100c20 msgr2=0x7f4970103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:43.408 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.529+0000 7f4976e00700 1 -- 172.21.15.131:0/2355407503 shutdown_connections 2024-05-08T21:59:43.408 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.530+0000 7f4976e00700 1 -- 172.21.15.131:0/2355407503 wait complete. 2024-05-08T21:59:43.408 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.530+0000 7f4976e00700 1 Processor -- start 2024-05-08T21:59:43.408 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.531+0000 7f4976e00700 1 -- start start 2024-05-08T21:59:43.408 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.531+0000 7f4976e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49701071d0 0x7f497019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:43.408 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.531+0000 7f4976e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497019cc20 con 0x7f49701071d0 2024-05-08T21:59:43.408 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.531+0000 7f496fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49701071d0 0x7f497019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:43.409 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.531+0000 7f496fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49701071d0 0x7f497019c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53464/0 (socket says 172.21.15.131:53464) 2024-05-08T21:59:43.409 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.531+0000 7f496fe00700 1 -- 172.21.15.131:0/295600756 learned_addr learned my addr 172.21.15.131:0/295600756 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:43.409 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.531+0000 7f496fe00700 1 -- 172.21.15.131:0/295600756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4960009550 con 0x7f49701071d0 2024-05-08T21:59:43.409 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.532+0000 7f496fe00700 1 --2- 172.21.15.131:0/295600756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49701071d0 0x7f497019c6e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4960011670 tx=0x7f4960011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:43.409 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.532+0000 7f496cc00700 1 -- 172.21.15.131:0/295600756 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4960003b20 con 0x7f49701071d0 2024-05-08T21:59:43.409 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.532+0000 7f496cc00700 1 -- 172.21.15.131:0/295600756 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4960003c80 con 0x7f49701071d0 2024-05-08T21:59:43.409 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.532+0000 7f4976e00700 1 -- 172.21.15.131:0/295600756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f497019ce20 con 0x7f49701071d0 2024-05-08T21:59:43.409 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.532+0000 7f496cc00700 1 -- 172.21.15.131:0/295600756 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4960019770 con 0x7f49701071d0 2024-05-08T21:59:43.409 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.532+0000 7f4976e00700 1 -- 172.21.15.131:0/295600756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f497019d2c0 con 0x7f49701071d0 2024-05-08T21:59:43.409 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.533+0000 7f496cc00700 1 -- 172.21.15.131:0/295600756 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 11) v1 ==== 45448+0+0 (secure 0 0 0) 0x7f496001ab60 con 0x7f49701071d0 2024-05-08T21:59:43.409 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.533+0000 7f4976e00700 1 -- 172.21.15.131:0/295600756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f497010a940 con 0x7f49701071d0 2024-05-08T21:59:43.409 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.534+0000 7f496cc00700 1 --2- 172.21.15.131:0/295600756 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f4954038620 0x7f495403aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:43.409 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.534+0000 7f496cc00700 1 -- 172.21.15.131:0/295600756 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f496004c9b0 con 0x7f49701071d0 2024-05-08T21:59:43.409 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.534+0000 7f496f400700 1 --2- 172.21.15.131:0/295600756 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f4954038620 0x7f495403aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:43.410 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.535+0000 7f496f400700 1 --2- 172.21.15.131:0/295600756 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f4954038620 0x7f495403aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f495c006fd0 tx=0x7f495c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:43.410 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.539+0000 7f496cc00700 1 -- 172.21.15.131:0/295600756 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f496001a430 con 0x7f49701071d0 2024-05-08T21:59:43.410 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:42.761+0000 7f4976e00700 1 -- 172.21.15.131:0/295600756 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f497019fb00 con 0x7f4954038620 2024-05-08T21:59:43.410 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.012+0000 7f496cc00700 1 -- 172.21.15.131:0/295600756 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f497019fb00 con 0x7f4954038620 2024-05-08T21:59:43.410 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.015+0000 7f4976e00700 1 -- 172.21.15.131:0/295600756 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f4954038620 msgr2=0x7f495403aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:43.410 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.015+0000 7f4976e00700 1 --2- 172.21.15.131:0/295600756 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f4954038620 0x7f495403aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f495c006fd0 tx=0x7f495c006e40 comp rx=0 tx=0).stop 2024-05-08T21:59:43.410 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.015+0000 7f4976e00700 1 -- 172.21.15.131:0/295600756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49701071d0 msgr2=0x7f497019c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:43.410 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.015+0000 7f4976e00700 1 --2- 172.21.15.131:0/295600756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49701071d0 0x7f497019c6e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4960011670 tx=0x7f4960011a30 comp rx=0 tx=0).stop 2024-05-08T21:59:43.410 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.015+0000 7f4976e00700 1 -- 172.21.15.131:0/295600756 shutdown_connections 2024-05-08T21:59:43.410 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.015+0000 7f4976e00700 1 --2- 172.21.15.131:0/295600756 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f4954038620 0x7f495403aae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.410 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.015+0000 7f4976e00700 1 --2- 172.21.15.131:0/295600756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49701071d0 0x7f497019c6e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:43.410 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.015+0000 7f4976e00700 1 -- 172.21.15.131:0/295600756 >> 172.21.15.131:0/295600756 conn(0x7f4970100c20 msgr2=0x7f49701016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:43.410 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.015+0000 7f4976e00700 1 -- 172.21.15.131:0/295600756 shutdown_connections 2024-05-08T21:59:43.410 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.015+0000 7f4976e00700 1 -- 172.21.15.131:0/295600756 wait complete. 2024-05-08T21:59:43.411 INFO:teuthology.orchestra.run.smithi131.stdout:Creating initial admin user... 2024-05-08T21:59:44.011 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:43 smithi131 ceph-mon[37260]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:59:44.011 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:43 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T21:59:44.011 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:43 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T21:59:44.011 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:43 smithi131 ceph-mon[37260]: mgrmap e12: x(active, since 2s) 2024-05-08T21:59:44.011 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:43 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T21:59:44.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$s45tsWlLq8W0kDUDYCZXBuzSyB8CpFKDSSc4Jfa3agmmy.vsT8B5.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715205584, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-08T21:59:44.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.868+0000 7f6e5da00700 1 Processor -- start 2024-05-08T21:59:44.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.868+0000 7f6e5da00700 1 -- start start 2024-05-08T21:59:44.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.868+0000 7f6e5da00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e58106780 0x7f6e58106ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:44.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.868+0000 7f6e5da00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e58107170 con 0x7f6e58106780 2024-05-08T21:59:44.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.869+0000 7f6e56a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e58106780 0x7f6e58106ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:44.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.869+0000 7f6e56a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e58106780 0x7f6e58106ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47100/0 (socket says 172.21.15.131:47100) 2024-05-08T21:59:44.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.869+0000 7f6e56a00700 1 -- 172.21.15.131:0/1402918407 learned_addr learned my addr 172.21.15.131:0/1402918407 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:44.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.869+0000 7f6e56a00700 1 -- 172.21.15.131:0/1402918407 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e58107930 con 0x7f6e58106780 2024-05-08T21:59:44.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.869+0000 7f6e56a00700 1 --2- 172.21.15.131:0/1402918407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e58106780 0x7f6e58106ba0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6e480098a0 tx=0x7f6e48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c72665223a1e6d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:44.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.870+0000 7f6e55600700 1 -- 172.21.15.131:0/1402918407 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e48004160 con 0x7f6e58106780 2024-05-08T21:59:44.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.870+0000 7f6e55600700 1 -- 172.21.15.131:0/1402918407 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e480042c0 con 0x7f6e58106780 2024-05-08T21:59:44.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.870+0000 7f6e55600700 1 -- 172.21.15.131:0/1402918407 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e480119b0 con 0x7f6e58106780 2024-05-08T21:59:44.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.871+0000 7f6e5da00700 1 -- 172.21.15.131:0/1402918407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e58106780 msgr2=0x7f6e58106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:44.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.871+0000 7f6e5da00700 1 --2- 172.21.15.131:0/1402918407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e58106780 0x7f6e58106ba0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6e480098a0 tx=0x7f6e48009bb0 comp rx=0 tx=0).stop 2024-05-08T21:59:44.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.871+0000 7f6e5da00700 1 -- 172.21.15.131:0/1402918407 shutdown_connections 2024-05-08T21:59:44.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.871+0000 7f6e5da00700 1 --2- 172.21.15.131:0/1402918407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e58106780 0x7f6e58106ba0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:44.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.871+0000 7f6e5da00700 1 -- 172.21.15.131:0/1402918407 >> 172.21.15.131:0/1402918407 conn(0x7f6e58101d20 msgr2=0x7f6e58104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:44.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.871+0000 7f6e5da00700 1 -- 172.21.15.131:0/1402918407 shutdown_connections 2024-05-08T21:59:44.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.871+0000 7f6e5da00700 1 -- 172.21.15.131:0/1402918407 wait complete. 2024-05-08T21:59:44.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.872+0000 7f6e5da00700 1 Processor -- start 2024-05-08T21:59:44.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.873+0000 7f6e5da00700 1 -- start start 2024-05-08T21:59:44.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.873+0000 7f6e5da00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e58106780 0x7f6e5819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:44.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.873+0000 7f6e5da00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5819cbc0 con 0x7f6e58106780 2024-05-08T21:59:44.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.873+0000 7f6e56a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e58106780 0x7f6e5819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:44.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.873+0000 7f6e56a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e58106780 0x7f6e5819c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47114/0 (socket says 172.21.15.131:47114) 2024-05-08T21:59:44.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.873+0000 7f6e56a00700 1 -- 172.21.15.131:0/2032672239 learned_addr learned my addr 172.21.15.131:0/2032672239 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:44.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.874+0000 7f6e56a00700 1 -- 172.21.15.131:0/2032672239 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e48009550 con 0x7f6e58106780 2024-05-08T21:59:44.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.874+0000 7f6e56a00700 1 --2- 172.21.15.131:0/2032672239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e58106780 0x7f6e5819c680 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6e48003e60 tx=0x7f6e48011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:44.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.874+0000 7f6e4f400700 1 -- 172.21.15.131:0/2032672239 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e48003a60 con 0x7f6e58106780 2024-05-08T21:59:44.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.874+0000 7f6e4f400700 1 -- 172.21.15.131:0/2032672239 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e48003bc0 con 0x7f6e58106780 2024-05-08T21:59:44.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.874+0000 7f6e4f400700 1 -- 172.21.15.131:0/2032672239 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e480196c0 con 0x7f6e58106780 2024-05-08T21:59:44.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.874+0000 7f6e5da00700 1 -- 172.21.15.131:0/2032672239 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e5819cdc0 con 0x7f6e58106780 2024-05-08T21:59:44.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.875+0000 7f6e5da00700 1 -- 172.21.15.131:0/2032672239 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e5819d260 con 0x7f6e58106780 2024-05-08T21:59:44.826 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.876+0000 7f6e4f400700 1 -- 172.21.15.131:0/2032672239 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f6e48019880 con 0x7f6e58106780 2024-05-08T21:59:44.826 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.876+0000 7f6e4f400700 1 --2- 172.21.15.131:0/2032672239 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f6e3c0387e0 0x7f6e3c03aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:44.826 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.876+0000 7f6e4f400700 1 -- 172.21.15.131:0/2032672239 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6e48051660 con 0x7f6e58106780 2024-05-08T21:59:44.826 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.876+0000 7f6e5da00700 1 -- 172.21.15.131:0/2032672239 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e5810ab60 con 0x7f6e58106780 2024-05-08T21:59:44.826 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.876+0000 7f6e56000700 1 --2- 172.21.15.131:0/2032672239 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f6e3c0387e0 0x7f6e3c03aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:44.826 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.877+0000 7f6e56000700 1 --2- 172.21.15.131:0/2032672239 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f6e3c0387e0 0x7f6e3c03aca0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6e44006fd0 tx=0x7f6e44006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:44.826 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:43.881+0000 7f6e4f400700 1 -- 172.21.15.131:0/2032672239 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6e48052060 con 0x7f6e58106780 2024-05-08T21:59:44.826 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:44.118+0000 7f6e5da00700 1 -- 172.21.15.131:0/2032672239 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f6e580623f0 con 0x7f6e3c0387e0 2024-05-08T21:59:44.826 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:44.345+0000 7f6e4f400700 1 -- 172.21.15.131:0/2032672239 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f6e580623f0 con 0x7f6e3c0387e0 2024-05-08T21:59:44.826 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:44.348+0000 7f6e5da00700 1 -- 172.21.15.131:0/2032672239 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f6e3c0387e0 msgr2=0x7f6e3c03aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:44.826 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:44.348+0000 7f6e5da00700 1 --2- 172.21.15.131:0/2032672239 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f6e3c0387e0 0x7f6e3c03aca0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6e44006fd0 tx=0x7f6e44006e40 comp rx=0 tx=0).stop 2024-05-08T21:59:44.826 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:44.348+0000 7f6e5da00700 1 -- 172.21.15.131:0/2032672239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e58106780 msgr2=0x7f6e5819c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:44.826 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:44.348+0000 7f6e5da00700 1 --2- 172.21.15.131:0/2032672239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e58106780 0x7f6e5819c680 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6e48003e60 tx=0x7f6e48011a30 comp rx=0 tx=0).stop 2024-05-08T21:59:44.827 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:44.348+0000 7f6e5da00700 1 -- 172.21.15.131:0/2032672239 shutdown_connections 2024-05-08T21:59:44.827 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:44.348+0000 7f6e5da00700 1 --2- 172.21.15.131:0/2032672239 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f6e3c0387e0 0x7f6e3c03aca0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:44.827 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:44.348+0000 7f6e5da00700 1 --2- 172.21.15.131:0/2032672239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e58106780 0x7f6e5819c680 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:44.827 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:44.348+0000 7f6e5da00700 1 -- 172.21.15.131:0/2032672239 >> 172.21.15.131:0/2032672239 conn(0x7f6e58101d20 msgr2=0x7f6e581027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:44.827 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:44.348+0000 7f6e5da00700 1 -- 172.21.15.131:0/2032672239 shutdown_connections 2024-05-08T21:59:44.827 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:44.348+0000 7f6e5da00700 1 -- 172.21.15.131:0/2032672239 wait complete. 2024-05-08T21:59:44.827 INFO:teuthology.orchestra.run.smithi131.stdout:Fetching dashboard port number... 2024-05-08T21:59:45.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:45 smithi131 ceph-mon[37260]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:59:45.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:45 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T21:59:45.830 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 8443 2024-05-08T21:59:45.830 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.253+0000 7fac29400700 1 Processor -- start 2024-05-08T21:59:45.830 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.253+0000 7fac29400700 1 -- start start 2024-05-08T21:59:45.831 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.254+0000 7fac29400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac24106790 0x7fac24106bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:45.831 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.254+0000 7fac29400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac24107180 con 0x7fac24106790 2024-05-08T21:59:45.831 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.254+0000 7fac22800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac24106790 0x7fac24106bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:45.831 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.254+0000 7fac22800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac24106790 0x7fac24106bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47118/0 (socket says 172.21.15.131:47118) 2024-05-08T21:59:45.831 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.254+0000 7fac22800700 1 -- 172.21.15.131:0/2023678143 learned_addr learned my addr 172.21.15.131:0/2023678143 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:45.831 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.254+0000 7fac22800700 1 -- 172.21.15.131:0/2023678143 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac24107940 con 0x7fac24106790 2024-05-08T21:59:45.831 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.255+0000 7fac22800700 1 --2- 172.21.15.131:0/2023678143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac24106790 0x7fac24106bb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac140098a0 tx=0x7fac14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1684c3de72d2b36 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:45.831 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.255+0000 7fac21e00700 1 -- 172.21.15.131:0/2023678143 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac14004160 con 0x7fac24106790 2024-05-08T21:59:45.831 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.255+0000 7fac21e00700 1 -- 172.21.15.131:0/2023678143 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac140042c0 con 0x7fac24106790 2024-05-08T21:59:45.832 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.255+0000 7fac21e00700 1 -- 172.21.15.131:0/2023678143 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac140119b0 con 0x7fac24106790 2024-05-08T21:59:45.832 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.255+0000 7fac29400700 1 -- 172.21.15.131:0/2023678143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac24106790 msgr2=0x7fac24106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:45.832 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.255+0000 7fac29400700 1 --2- 172.21.15.131:0/2023678143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac24106790 0x7fac24106bb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac140098a0 tx=0x7fac14009bb0 comp rx=0 tx=0).stop 2024-05-08T21:59:45.832 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.255+0000 7fac29400700 1 -- 172.21.15.131:0/2023678143 shutdown_connections 2024-05-08T21:59:45.832 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.255+0000 7fac29400700 1 --2- 172.21.15.131:0/2023678143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac24106790 0x7fac24106bb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:45.832 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.256+0000 7fac29400700 1 -- 172.21.15.131:0/2023678143 >> 172.21.15.131:0/2023678143 conn(0x7fac24101d50 msgr2=0x7fac24104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:45.832 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.256+0000 7fac29400700 1 -- 172.21.15.131:0/2023678143 shutdown_connections 2024-05-08T21:59:45.832 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.256+0000 7fac29400700 1 -- 172.21.15.131:0/2023678143 wait complete. 2024-05-08T21:59:45.832 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.256+0000 7fac29400700 1 Processor -- start 2024-05-08T21:59:45.832 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.257+0000 7fac29400700 1 -- start start 2024-05-08T21:59:45.832 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.257+0000 7fac29400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac24106790 0x7fac2419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:45.832 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.257+0000 7fac29400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2419aa80 con 0x7fac24106790 2024-05-08T21:59:45.832 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.257+0000 7fac22800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac24106790 0x7fac2419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:45.832 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.257+0000 7fac22800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac24106790 0x7fac2419a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47126/0 (socket says 172.21.15.131:47126) 2024-05-08T21:59:45.832 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.257+0000 7fac22800700 1 -- 172.21.15.131:0/2274608285 learned_addr learned my addr 172.21.15.131:0/2274608285 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:45.833 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.258+0000 7fac22800700 1 -- 172.21.15.131:0/2274608285 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac14009550 con 0x7fac24106790 2024-05-08T21:59:45.833 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.258+0000 7fac22800700 1 --2- 172.21.15.131:0/2274608285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac24106790 0x7fac2419a540 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fac14011670 tx=0x7fac14011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:45.833 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.258+0000 7fac1b400700 1 -- 172.21.15.131:0/2274608285 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac14003af0 con 0x7fac24106790 2024-05-08T21:59:45.833 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.258+0000 7fac1b400700 1 -- 172.21.15.131:0/2274608285 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac14003c50 con 0x7fac24106790 2024-05-08T21:59:45.833 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.258+0000 7fac29400700 1 -- 172.21.15.131:0/2274608285 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac2419ac80 con 0x7fac24106790 2024-05-08T21:59:45.833 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.258+0000 7fac1b400700 1 -- 172.21.15.131:0/2274608285 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac14024820 con 0x7fac24106790 2024-05-08T21:59:45.833 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.258+0000 7fac29400700 1 -- 172.21.15.131:0/2274608285 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2419b120 con 0x7fac24106790 2024-05-08T21:59:45.833 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.259+0000 7fac29400700 1 -- 172.21.15.131:0/2274608285 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac2410ab70 con 0x7fac24106790 2024-05-08T21:59:45.833 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.259+0000 7fac1b400700 1 -- 172.21.15.131:0/2274608285 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fac1401ab60 con 0x7fac24106790 2024-05-08T21:59:45.833 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.260+0000 7fac1b400700 1 --2- 172.21.15.131:0/2274608285 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fac080387a0 0x7fac0803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:45.833 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.260+0000 7fac1b400700 1 -- 172.21.15.131:0/2274608285 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fac1404c820 con 0x7fac24106790 2024-05-08T21:59:45.833 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.260+0000 7fac1be00700 1 --2- 172.21.15.131:0/2274608285 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fac080387a0 0x7fac0803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:45.833 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.260+0000 7fac1be00700 1 --2- 172.21.15.131:0/2274608285 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fac080387a0 0x7fac0803ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fac0c006fd0 tx=0x7fac0c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:45.833 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.265+0000 7fac1b400700 1 -- 172.21.15.131:0/2274608285 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fac14018760 con 0x7fac24106790 2024-05-08T21:59:45.834 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.467+0000 7fac29400700 1 -- 172.21.15.131:0/2274608285 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fac2419d9b0 con 0x7fac24106790 2024-05-08T21:59:45.834 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.467+0000 7fac1b400700 1 -- 172.21.15.131:0/2274608285 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fac1400b3e0 con 0x7fac24106790 2024-05-08T21:59:45.834 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.469+0000 7fac29400700 1 -- 172.21.15.131:0/2274608285 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fac080387a0 msgr2=0x7fac0803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:45.834 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.469+0000 7fac29400700 1 --2- 172.21.15.131:0/2274608285 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fac080387a0 0x7fac0803ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fac0c006fd0 tx=0x7fac0c006e40 comp rx=0 tx=0).stop 2024-05-08T21:59:45.834 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.469+0000 7fac29400700 1 -- 172.21.15.131:0/2274608285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac24106790 msgr2=0x7fac2419a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:45.834 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.469+0000 7fac29400700 1 --2- 172.21.15.131:0/2274608285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac24106790 0x7fac2419a540 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fac14011670 tx=0x7fac14011a30 comp rx=0 tx=0).stop 2024-05-08T21:59:45.834 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.470+0000 7fac29400700 1 -- 172.21.15.131:0/2274608285 shutdown_connections 2024-05-08T21:59:45.834 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.470+0000 7fac29400700 1 --2- 172.21.15.131:0/2274608285 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fac080387a0 0x7fac0803ac60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:45.834 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.470+0000 7fac29400700 1 --2- 172.21.15.131:0/2274608285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac24106790 0x7fac2419a540 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:45.834 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.470+0000 7fac29400700 1 -- 172.21.15.131:0/2274608285 >> 172.21.15.131:0/2274608285 conn(0x7fac24101d50 msgr2=0x7fac24102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:45.834 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.470+0000 7fac29400700 1 -- 172.21.15.131:0/2274608285 shutdown_connections 2024-05-08T21:59:45.834 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:45.470+0000 7fac29400700 1 -- 172.21.15.131:0/2274608285 wait complete. 2024-05-08T21:59:45.846 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T21:59:45.846 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout disabled 2024-05-08T21:59:45.863 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T21:59:45.863 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-05-08T21:59:45.863 INFO:teuthology.orchestra.run.smithi131.stdout:firewalld.service is not enabled 2024-05-08T21:59:45.863 INFO:teuthology.orchestra.run.smithi131.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-08T21:59:45.868 INFO:teuthology.orchestra.run.smithi131.stdout:Ceph Dashboard is now available at: 2024-05-08T21:59:45.868 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:59:45.868 INFO:teuthology.orchestra.run.smithi131.stdout: URL: https://smithi131.front.sepia.ceph.com:8443/ 2024-05-08T21:59:45.868 INFO:teuthology.orchestra.run.smithi131.stdout: User: admin 2024-05-08T21:59:45.868 INFO:teuthology.orchestra.run.smithi131.stdout: Password: ffj1mu92kx 2024-05-08T21:59:45.868 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:59:45.869 INFO:teuthology.orchestra.run.smithi131.stdout:Saving cluster configuration to /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config directory 2024-05-08T21:59:45.870 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling autotune for osd_memory_target 2024-05-08T21:59:46.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:46 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2274608285' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-08T21:59:46.874 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.296+0000 7f7c9d200700 1 Processor -- start 2024-05-08T21:59:46.874 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.297+0000 7f7c9d200700 1 -- start start 2024-05-08T21:59:46.875 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.297+0000 7f7c9d200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c98107200 0x7f7c981095f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:46.875 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.297+0000 7f7c9d200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c98079e50 con 0x7f7c98107200 2024-05-08T21:59:46.875 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.297+0000 7f7c95e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c98107200 0x7f7c981095f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:46.875 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.297+0000 7f7c95e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c98107200 0x7f7c981095f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47140/0 (socket says 172.21.15.131:47140) 2024-05-08T21:59:46.875 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.297+0000 7f7c95e00700 1 -- 172.21.15.131:0/3276370782 learned_addr learned my addr 172.21.15.131:0/3276370782 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:46.875 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.298+0000 7f7c95e00700 1 -- 172.21.15.131:0/3276370782 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c98109b30 con 0x7f7c98107200 2024-05-08T21:59:46.875 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.298+0000 7f7c95e00700 1 --2- 172.21.15.131:0/3276370782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c98107200 0x7f7c981095f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7c880098a0 tx=0x7f7c88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230ee51ac18dbf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:46.875 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.298+0000 7f7c94a00700 1 -- 172.21.15.131:0/3276370782 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c88004160 con 0x7f7c98107200 2024-05-08T21:59:46.875 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.298+0000 7f7c94a00700 1 -- 172.21.15.131:0/3276370782 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c880042c0 con 0x7f7c98107200 2024-05-08T21:59:46.875 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.298+0000 7f7c94a00700 1 -- 172.21.15.131:0/3276370782 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c880119b0 con 0x7f7c98107200 2024-05-08T21:59:46.875 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.299+0000 7f7c9d200700 1 -- 172.21.15.131:0/3276370782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c98107200 msgr2=0x7f7c981095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:46.875 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.299+0000 7f7c9d200700 1 --2- 172.21.15.131:0/3276370782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c98107200 0x7f7c981095f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7c880098a0 tx=0x7f7c88009bb0 comp rx=0 tx=0).stop 2024-05-08T21:59:46.875 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.299+0000 7f7c9d200700 1 -- 172.21.15.131:0/3276370782 shutdown_connections 2024-05-08T21:59:46.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.299+0000 7f7c9d200700 1 --2- 172.21.15.131:0/3276370782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c98107200 0x7f7c981095f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:46.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.299+0000 7f7c9d200700 1 -- 172.21.15.131:0/3276370782 >> 172.21.15.131:0/3276370782 conn(0x7f7c98100c50 msgr2=0x7f7c98103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:46.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.299+0000 7f7c9d200700 1 -- 172.21.15.131:0/3276370782 shutdown_connections 2024-05-08T21:59:46.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.299+0000 7f7c9d200700 1 -- 172.21.15.131:0/3276370782 wait complete. 2024-05-08T21:59:46.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.300+0000 7f7c9d200700 1 Processor -- start 2024-05-08T21:59:46.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.300+0000 7f7c9d200700 1 -- start start 2024-05-08T21:59:46.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.301+0000 7f7c9d200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c98107200 0x7f7c9819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:46.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.301+0000 7f7c9d200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9819cba0 con 0x7f7c98107200 2024-05-08T21:59:46.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.301+0000 7f7c95e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c98107200 0x7f7c9819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:46.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.301+0000 7f7c95e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c98107200 0x7f7c9819c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47144/0 (socket says 172.21.15.131:47144) 2024-05-08T21:59:46.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.301+0000 7f7c95e00700 1 -- 172.21.15.131:0/2258818713 learned_addr learned my addr 172.21.15.131:0/2258818713 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:46.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.301+0000 7f7c95e00700 1 -- 172.21.15.131:0/2258818713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c88009550 con 0x7f7c98107200 2024-05-08T21:59:46.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.301+0000 7f7c95e00700 1 --2- 172.21.15.131:0/2258818713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c98107200 0x7f7c9819c660 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7c88011670 tx=0x7f7c88011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:46.876 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.302+0000 7f7c8ea00700 1 -- 172.21.15.131:0/2258818713 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c88003b20 con 0x7f7c98107200 2024-05-08T21:59:46.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.302+0000 7f7c8ea00700 1 -- 172.21.15.131:0/2258818713 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c88003c80 con 0x7f7c98107200 2024-05-08T21:59:46.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.302+0000 7f7c9d200700 1 -- 172.21.15.131:0/2258818713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c9819cda0 con 0x7f7c98107200 2024-05-08T21:59:46.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.302+0000 7f7c8ea00700 1 -- 172.21.15.131:0/2258818713 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c88019770 con 0x7f7c98107200 2024-05-08T21:59:46.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.302+0000 7f7c9d200700 1 -- 172.21.15.131:0/2258818713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c9819d240 con 0x7f7c98107200 2024-05-08T21:59:46.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.303+0000 7f7c8ea00700 1 -- 172.21.15.131:0/2258818713 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f7c8801a430 con 0x7f7c98107200 2024-05-08T21:59:46.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.303+0000 7f7c9d200700 1 -- 172.21.15.131:0/2258818713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c9810a910 con 0x7f7c98107200 2024-05-08T21:59:46.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.303+0000 7f7c8ea00700 1 --2- 172.21.15.131:0/2258818713 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7c7c0387a0 0x7f7c7c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:46.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.303+0000 7f7c8ea00700 1 -- 172.21.15.131:0/2258818713 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7c8804cb00 con 0x7f7c98107200 2024-05-08T21:59:46.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.304+0000 7f7c95400700 1 --2- 172.21.15.131:0/2258818713 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7c7c0387a0 0x7f7c7c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:46.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.304+0000 7f7c95400700 1 --2- 172.21.15.131:0/2258818713 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7c7c0387a0 0x7f7c7c03ac60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7c84006fd0 tx=0x7f7c84006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:46.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.308+0000 7f7c8ea00700 1 -- 172.21.15.131:0/2258818713 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7c8802d9a0 con 0x7f7c98107200 2024-05-08T21:59:46.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.515+0000 7f7c9d200700 1 -- 172.21.15.131:0/2258818713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f7c9810ab50 con 0x7f7c98107200 2024-05-08T21:59:46.877 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.515+0000 7f7c8ea00700 1 -- 172.21.15.131:0/2258818713 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f7c8802d350 con 0x7f7c98107200 2024-05-08T21:59:46.878 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.518+0000 7f7c9d200700 1 -- 172.21.15.131:0/2258818713 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7c7c0387a0 msgr2=0x7f7c7c03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:46.878 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.518+0000 7f7c9d200700 1 --2- 172.21.15.131:0/2258818713 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7c7c0387a0 0x7f7c7c03ac60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7c84006fd0 tx=0x7f7c84006e40 comp rx=0 tx=0).stop 2024-05-08T21:59:46.878 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.518+0000 7f7c9d200700 1 -- 172.21.15.131:0/2258818713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c98107200 msgr2=0x7f7c9819c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:46.878 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.518+0000 7f7c9d200700 1 --2- 172.21.15.131:0/2258818713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c98107200 0x7f7c9819c660 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7c88011670 tx=0x7f7c88011a30 comp rx=0 tx=0).stop 2024-05-08T21:59:46.878 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.518+0000 7f7c9d200700 1 -- 172.21.15.131:0/2258818713 shutdown_connections 2024-05-08T21:59:46.878 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.518+0000 7f7c9d200700 1 --2- 172.21.15.131:0/2258818713 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7c7c0387a0 0x7f7c7c03ac60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:46.878 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.518+0000 7f7c9d200700 1 --2- 172.21.15.131:0/2258818713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c98107200 0x7f7c9819c660 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:46.878 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.518+0000 7f7c9d200700 1 -- 172.21.15.131:0/2258818713 >> 172.21.15.131:0/2258818713 conn(0x7f7c98100c50 msgr2=0x7f7c981017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:46.878 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.518+0000 7f7c9d200700 1 -- 172.21.15.131:0/2258818713 shutdown_connections 2024-05-08T21:59:46.878 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:46.518+0000 7f7c9d200700 1 -- 172.21.15.131:0/2258818713 wait complete. 2024-05-08T21:59:47.990 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.304+0000 7fa6c0c00700 1 Processor -- start 2024-05-08T21:59:47.996 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.304+0000 7fa6c0c00700 1 -- start start 2024-05-08T21:59:47.997 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.305+0000 7fa6c0c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6bc07be30 0x7fa6bc07a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:47.997 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.305+0000 7fa6c0c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc07a820 con 0x7fa6bc07be30 2024-05-08T21:59:47.997 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.305+0000 7fa6b9e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6bc07be30 0x7fa6bc07a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:47.997 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.305+0000 7fa6b9e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6bc07be30 0x7fa6bc07a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47160/0 (socket says 172.21.15.131:47160) 2024-05-08T21:59:47.997 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.305+0000 7fa6b9e00700 1 -- 172.21.15.131:0/881623314 learned_addr learned my addr 172.21.15.131:0/881623314 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:47.997 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.305+0000 7fa6b9e00700 1 -- 172.21.15.131:0/881623314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc07a960 con 0x7fa6bc07be30 2024-05-08T21:59:47.997 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.306+0000 7fa6b9e00700 1 --2- 172.21.15.131:0/881623314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6bc07be30 0x7fa6bc07a2e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac0098a0 tx=0x7fa6ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d5bcab7d58153f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:47.997 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.306+0000 7fa6b8a00700 1 -- 172.21.15.131:0/881623314 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6ac004160 con 0x7fa6bc07be30 2024-05-08T21:59:47.997 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.306+0000 7fa6b8a00700 1 -- 172.21.15.131:0/881623314 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6ac0042c0 con 0x7fa6bc07be30 2024-05-08T21:59:47.997 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.306+0000 7fa6b8a00700 1 -- 172.21.15.131:0/881623314 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6ac0119b0 con 0x7fa6bc07be30 2024-05-08T21:59:47.997 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.307+0000 7fa6c0c00700 1 -- 172.21.15.131:0/881623314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6bc07be30 msgr2=0x7fa6bc07a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:47.998 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.307+0000 7fa6c0c00700 1 --2- 172.21.15.131:0/881623314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6bc07be30 0x7fa6bc07a2e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac0098a0 tx=0x7fa6ac009bb0 comp rx=0 tx=0).stop 2024-05-08T21:59:47.998 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.307+0000 7fa6c0c00700 1 -- 172.21.15.131:0/881623314 shutdown_connections 2024-05-08T21:59:47.998 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.307+0000 7fa6c0c00700 1 --2- 172.21.15.131:0/881623314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6bc07be30 0x7fa6bc07a2e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:47.998 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.307+0000 7fa6c0c00700 1 -- 172.21.15.131:0/881623314 >> 172.21.15.131:0/881623314 conn(0x7fa6bc101d50 msgr2=0x7fa6bc104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:47.998 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.307+0000 7fa6c0c00700 1 -- 172.21.15.131:0/881623314 shutdown_connections 2024-05-08T21:59:47.998 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.307+0000 7fa6c0c00700 1 -- 172.21.15.131:0/881623314 wait complete. 2024-05-08T21:59:47.998 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.308+0000 7fa6c0c00700 1 Processor -- start 2024-05-08T21:59:47.998 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.308+0000 7fa6c0c00700 1 -- start start 2024-05-08T21:59:47.998 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.308+0000 7fa6c0c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6bc07be30 0x7fa6bc19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:47.998 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.308+0000 7fa6c0c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc19cc40 con 0x7fa6bc07be30 2024-05-08T21:59:47.998 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.308+0000 7fa6b9e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6bc07be30 0x7fa6bc19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:47.998 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.308+0000 7fa6b9e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6bc07be30 0x7fa6bc19c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47176/0 (socket says 172.21.15.131:47176) 2024-05-08T21:59:47.998 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.308+0000 7fa6b9e00700 1 -- 172.21.15.131:0/4215095911 learned_addr learned my addr 172.21.15.131:0/4215095911 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:47.998 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.309+0000 7fa6b9e00700 1 -- 172.21.15.131:0/4215095911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac009550 con 0x7fa6bc07be30 2024-05-08T21:59:47.999 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.309+0000 7fa6b9e00700 1 --2- 172.21.15.131:0/4215095911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6bc07be30 0x7fa6bc19c700 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac011980 tx=0x7fa6ac011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:47.999 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.309+0000 7fa6b2a00700 1 -- 172.21.15.131:0/4215095911 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6ac003b70 con 0x7fa6bc07be30 2024-05-08T21:59:47.999 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.309+0000 7fa6b2a00700 1 -- 172.21.15.131:0/4215095911 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6ac01a8f0 con 0x7fa6bc07be30 2024-05-08T21:59:47.999 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.309+0000 7fa6b2a00700 1 -- 172.21.15.131:0/4215095911 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6ac0197d0 con 0x7fa6bc07be30 2024-05-08T21:59:47.999 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.309+0000 7fa6c0c00700 1 -- 172.21.15.131:0/4215095911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6bc19ce40 con 0x7fa6bc07be30 2024-05-08T21:59:47.999 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.309+0000 7fa6c0c00700 1 -- 172.21.15.131:0/4215095911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6bc19d2e0 con 0x7fa6bc07be30 2024-05-08T21:59:47.999 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.310+0000 7fa6b2a00700 1 -- 172.21.15.131:0/4215095911 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fa6ac01a430 con 0x7fa6bc07be30 2024-05-08T21:59:47.999 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.310+0000 7fa6c0c00700 1 -- 172.21.15.131:0/4215095911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6bc10cb10 con 0x7fa6bc07be30 2024-05-08T21:59:47.999 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.311+0000 7fa6b2a00700 1 --2- 172.21.15.131:0/4215095911 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa6a0038750 0x7fa6a003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:47.999 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.311+0000 7fa6b2a00700 1 -- 172.21.15.131:0/4215095911 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa6ac04cbf0 con 0x7fa6bc07be30 2024-05-08T21:59:47.999 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.311+0000 7fa6b9400700 1 --2- 172.21.15.131:0/4215095911 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa6a0038750 0x7fa6a003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:47.999 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.312+0000 7fa6b9400700 1 --2- 172.21.15.131:0/4215095911 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa6a0038750 0x7fa6a003ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8006fd0 tx=0x7fa6a8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:47.999 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.316+0000 7fa6b2a00700 1 -- 172.21.15.131:0/4215095911 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa6ac00b3e0 con 0x7fa6bc07be30 2024-05-08T21:59:48.000 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.348+0000 7fa6b2a00700 1 -- 172.21.15.131:0/4215095911 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7fa6ac02b260 con 0x7fa6bc07be30 2024-05-08T21:59:48.000 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.624+0000 7fa6c0c00700 1 -- 172.21.15.131:0/4215095911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fa6bc10cd50 con 0x7fa6bc07be30 2024-05-08T21:59:48.000 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.627+0000 7fa6b2a00700 1 -- 172.21.15.131:0/4215095911 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7fa6ac018530 con 0x7fa6bc07be30 2024-05-08T21:59:48.000 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.630+0000 7fa6c0c00700 1 -- 172.21.15.131:0/4215095911 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa6a0038750 msgr2=0x7fa6a003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:48.000 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.630+0000 7fa6c0c00700 1 --2- 172.21.15.131:0/4215095911 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa6a0038750 0x7fa6a003ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8006fd0 tx=0x7fa6a8006e40 comp rx=0 tx=0).stop 2024-05-08T21:59:48.000 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.630+0000 7fa6c0c00700 1 -- 172.21.15.131:0/4215095911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6bc07be30 msgr2=0x7fa6bc19c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:48.000 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.630+0000 7fa6c0c00700 1 --2- 172.21.15.131:0/4215095911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6bc07be30 0x7fa6bc19c700 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac011980 tx=0x7fa6ac011a60 comp rx=0 tx=0).stop 2024-05-08T21:59:48.000 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.630+0000 7fa6c0c00700 1 -- 172.21.15.131:0/4215095911 shutdown_connections 2024-05-08T21:59:48.000 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.630+0000 7fa6c0c00700 1 --2- 172.21.15.131:0/4215095911 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa6a0038750 0x7fa6a003ac10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:48.000 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.630+0000 7fa6c0c00700 1 --2- 172.21.15.131:0/4215095911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6bc07be30 0x7fa6bc19c700 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:48.000 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.630+0000 7fa6c0c00700 1 -- 172.21.15.131:0/4215095911 >> 172.21.15.131:0/4215095911 conn(0x7fa6bc101d50 msgr2=0x7fa6bc1029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:48.000 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.630+0000 7fa6c0c00700 1 -- 172.21.15.131:0/4215095911 shutdown_connections 2024-05-08T21:59:48.000 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-08T21:59:47.630+0000 7fa6c0c00700 1 -- 172.21.15.131:0/4215095911 wait complete. 2024-05-08T21:59:48.000 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-08T21:59:48.000 INFO:teuthology.orchestra.run.smithi131.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-08T21:59:48.001 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:59:48.001 INFO:teuthology.orchestra.run.smithi131.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-08T21:59:48.001 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:59:48.001 INFO:teuthology.orchestra.run.smithi131.stdout:Or, if you are only running a single cluster on this host: 2024-05-08T21:59:48.001 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:59:48.001 INFO:teuthology.orchestra.run.smithi131.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-08T21:59:48.001 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:59:48.001 INFO:teuthology.orchestra.run.smithi131.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-08T21:59:48.001 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:59:48.001 INFO:teuthology.orchestra.run.smithi131.stdout: ceph telemetry on 2024-05-08T21:59:48.001 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:59:48.001 INFO:teuthology.orchestra.run.smithi131.stdout:For more information see: 2024-05-08T21:59:48.001 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:59:48.001 INFO:teuthology.orchestra.run.smithi131.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-08T21:59:48.002 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T21:59:48.002 INFO:teuthology.orchestra.run.smithi131.stdout:Bootstrap complete. 2024-05-08T21:59:48.053 INFO:tasks.cephadm:Fetching config... 2024-05-08T21:59:48.054 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-08T21:59:48.054 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-08T21:59:48.096 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-08T21:59:48.096 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-08T21:59:48.096 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-08T21:59:48.180 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-08T21:59:48.180 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-08T21:59:48.180 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-08T21:59:48.302 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-08T21:59:48.302 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-08T21:59:48.302 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-08T21:59:48.386 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-08T21:59:48.387 DEBUG:teuthology.orchestra.run.smithi131:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/DYJrtSvN+EyYLCXPyBB89BReFW9jYLCighbA8FEY5dXop6TdteqCQE0iILXoM+1dp7KDLbtHA7srGbWI/BoCp6ztA5tsQ5VVwCCZSyADR6FPojJEt6rVSQdIaM8hiKHnXJcB/JyUcQYEL+fLd79DIpiUJc9JEbReSKZv19wX12e8+npfAESdWriQDufWnAOvGhfmaIuepxaQjdEPDlD5NzQ7LVFvOqxzbmQZxyf8M8aAfy8KBVQIgylSdUyS4N/uUUKcb7rP5g7Tu5aohKGcchPRAhZj4TdSiTzPWKD/Jx7EFhq0+jjTcvNxaE4cdLxqMUb4yrna6xjozj68cGBwwSANce69N0tVfXiEXpH+yxcuFav9K4J55B+Z9QuA7zysdaqnKqNt+FFkxVgeUl33dR3qkZD1YCHKr0UbQiGIoU+YzfsWEheQ4uUGjl7XyV0Xeuh1SYL3jDp0r/nN+MD21HvUPGSKwhzaVBT9mYlAbDYNQ07tfZMclvB69eIirwU= ceph-06f1a696-0d86-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T21:59:48.518 INFO:teuthology.orchestra.run.smithi131.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/DYJrtSvN+EyYLCXPyBB89BReFW9jYLCighbA8FEY5dXop6TdteqCQE0iILXoM+1dp7KDLbtHA7srGbWI/BoCp6ztA5tsQ5VVwCCZSyADR6FPojJEt6rVSQdIaM8hiKHnXJcB/JyUcQYEL+fLd79DIpiUJc9JEbReSKZv19wX12e8+npfAESdWriQDufWnAOvGhfmaIuepxaQjdEPDlD5NzQ7LVFvOqxzbmQZxyf8M8aAfy8KBVQIgylSdUyS4N/uUUKcb7rP5g7Tu5aohKGcchPRAhZj4TdSiTzPWKD/Jx7EFhq0+jjTcvNxaE4cdLxqMUb4yrna6xjozj68cGBwwSANce69N0tVfXiEXpH+yxcuFav9K4J55B+Z9QuA7zysdaqnKqNt+FFkxVgeUl33dR3qkZD1YCHKr0UbQiGIoU+YzfsWEheQ4uUGjl7XyV0Xeuh1SYL3jDp0r/nN+MD21HvUPGSKwhzaVBT9mYlAbDYNQ07tfZMclvB69eIirwU= ceph-06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T21:59:48.537 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:48 smithi131 ceph-mon[37260]: mgrmap e13: x(active, since 6s) 2024-05-08T21:59:48.537 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:48 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/4215095911' entity='client.admin' 2024-05-08T21:59:48.546 DEBUG:teuthology.orchestra.run.smithi132:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/DYJrtSvN+EyYLCXPyBB89BReFW9jYLCighbA8FEY5dXop6TdteqCQE0iILXoM+1dp7KDLbtHA7srGbWI/BoCp6ztA5tsQ5VVwCCZSyADR6FPojJEt6rVSQdIaM8hiKHnXJcB/JyUcQYEL+fLd79DIpiUJc9JEbReSKZv19wX12e8+npfAESdWriQDufWnAOvGhfmaIuepxaQjdEPDlD5NzQ7LVFvOqxzbmQZxyf8M8aAfy8KBVQIgylSdUyS4N/uUUKcb7rP5g7Tu5aohKGcchPRAhZj4TdSiTzPWKD/Jx7EFhq0+jjTcvNxaE4cdLxqMUb4yrna6xjozj68cGBwwSANce69N0tVfXiEXpH+yxcuFav9K4J55B+Z9QuA7zysdaqnKqNt+FFkxVgeUl33dR3qkZD1YCHKr0UbQiGIoU+YzfsWEheQ4uUGjl7XyV0Xeuh1SYL3jDp0r/nN+MD21HvUPGSKwhzaVBT9mYlAbDYNQ07tfZMclvB69eIirwU= ceph-06f1a696-0d86-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T21:59:48.592 INFO:teuthology.orchestra.run.smithi132.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/DYJrtSvN+EyYLCXPyBB89BReFW9jYLCighbA8FEY5dXop6TdteqCQE0iILXoM+1dp7KDLbtHA7srGbWI/BoCp6ztA5tsQ5VVwCCZSyADR6FPojJEt6rVSQdIaM8hiKHnXJcB/JyUcQYEL+fLd79DIpiUJc9JEbReSKZv19wX12e8+npfAESdWriQDufWnAOvGhfmaIuepxaQjdEPDlD5NzQ7LVFvOqxzbmQZxyf8M8aAfy8KBVQIgylSdUyS4N/uUUKcb7rP5g7Tu5aohKGcchPRAhZj4TdSiTzPWKD/Jx7EFhq0+jjTcvNxaE4cdLxqMUb4yrna6xjozj68cGBwwSANce69N0tVfXiEXpH+yxcuFav9K4J55B+Z9QuA7zysdaqnKqNt+FFkxVgeUl33dR3qkZD1YCHKr0UbQiGIoU+YzfsWEheQ4uUGjl7XyV0Xeuh1SYL3jDp0r/nN+MD21HvUPGSKwhzaVBT9mYlAbDYNQ07tfZMclvB69eIirwU= ceph-06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T21:59:48.605 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/DYJrtSvN+EyYLCXPyBB89BReFW9jYLCighbA8FEY5dXop6TdteqCQE0iILXoM+1dp7KDLbtHA7srGbWI/BoCp6ztA5tsQ5VVwCCZSyADR6FPojJEt6rVSQdIaM8hiKHnXJcB/JyUcQYEL+fLd79DIpiUJc9JEbReSKZv19wX12e8+npfAESdWriQDufWnAOvGhfmaIuepxaQjdEPDlD5NzQ7LVFvOqxzbmQZxyf8M8aAfy8KBVQIgylSdUyS4N/uUUKcb7rP5g7Tu5aohKGcchPRAhZj4TdSiTzPWKD/Jx7EFhq0+jjTcvNxaE4cdLxqMUb4yrna6xjozj68cGBwwSANce69N0tVfXiEXpH+yxcuFav9K4J55B+Z9QuA7zysdaqnKqNt+FFkxVgeUl33dR3qkZD1YCHKr0UbQiGIoU+YzfsWEheQ4uUGjl7XyV0Xeuh1SYL3jDp0r/nN+MD21HvUPGSKwhzaVBT9mYlAbDYNQ07tfZMclvB69eIirwU= ceph-06f1a696-0d86-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T21:59:48.652 INFO:teuthology.orchestra.run.smithi157.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/DYJrtSvN+EyYLCXPyBB89BReFW9jYLCighbA8FEY5dXop6TdteqCQE0iILXoM+1dp7KDLbtHA7srGbWI/BoCp6ztA5tsQ5VVwCCZSyADR6FPojJEt6rVSQdIaM8hiKHnXJcB/JyUcQYEL+fLd79DIpiUJc9JEbReSKZv19wX12e8+npfAESdWriQDufWnAOvGhfmaIuepxaQjdEPDlD5NzQ7LVFvOqxzbmQZxyf8M8aAfy8KBVQIgylSdUyS4N/uUUKcb7rP5g7Tu5aohKGcchPRAhZj4TdSiTzPWKD/Jx7EFhq0+jjTcvNxaE4cdLxqMUb4yrna6xjozj68cGBwwSANce69N0tVfXiEXpH+yxcuFav9K4J55B+Z9QuA7zysdaqnKqNt+FFkxVgeUl33dR3qkZD1YCHKr0UbQiGIoU+YzfsWEheQ4uUGjl7XyV0Xeuh1SYL3jDp0r/nN+MD21HvUPGSKwhzaVBT9mYlAbDYNQ07tfZMclvB69eIirwU= ceph-06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T21:59:48.666 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-08T21:59:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:59:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.606+0000 7f8bcd800700 1 -- 172.21.15.131:0/3676305749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc8102b80 msgr2=0x7f8bc8102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.606+0000 7f8bcd800700 1 --2- 172.21.15.131:0/3676305749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc8102b80 0x7f8bc8102fa0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009910 tx=0x7f8bb8009c20 comp rx=0 tx=0).stop 2024-05-08T21:59:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.606+0000 7f8bcd800700 1 -- 172.21.15.131:0/3676305749 shutdown_connections 2024-05-08T21:59:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.606+0000 7f8bcd800700 1 --2- 172.21.15.131:0/3676305749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc8102b80 0x7f8bc8102fa0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.607+0000 7f8bcd800700 1 -- 172.21.15.131:0/3676305749 >> 172.21.15.131:0/3676305749 conn(0x7f8bc80fe120 msgr2=0x7f8bc8100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.607+0000 7f8bcd800700 1 -- 172.21.15.131:0/3676305749 shutdown_connections 2024-05-08T21:59:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.607+0000 7f8bcd800700 1 -- 172.21.15.131:0/3676305749 wait complete. 2024-05-08T21:59:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.608+0000 7f8bcd800700 1 Processor -- start 2024-05-08T21:59:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.608+0000 7f8bcd800700 1 -- start start 2024-05-08T21:59:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.608+0000 7f8bcd800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc8102b80 0x7f8bc8197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.608+0000 7f8bcd800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc8198500 con 0x7f8bc8102b80 2024-05-08T21:59:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.608+0000 7f8bc6a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc8102b80 0x7f8bc8197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.608+0000 7f8bc6a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc8102b80 0x7f8bc8197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47196/0 (socket says 172.21.15.131:47196) 2024-05-08T21:59:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.608+0000 7f8bc6a00700 1 -- 172.21.15.131:0/1345437417 learned_addr learned my addr 172.21.15.131:0/1345437417 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:50.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.609+0000 7f8bc6a00700 1 -- 172.21.15.131:0/1345437417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb80095f0 con 0x7f8bc8102b80 2024-05-08T21:59:50.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.609+0000 7f8bc6a00700 1 --2- 172.21.15.131:0/1345437417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc8102b80 0x7f8bc8197fc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8005950 tx=0x7f8bb80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:50.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.609+0000 7f8bbf400700 1 -- 172.21.15.131:0/1345437417 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8bb801d070 con 0x7f8bc8102b80 2024-05-08T21:59:50.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.609+0000 7f8bbf400700 1 -- 172.21.15.131:0/1345437417 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bb800bc10 con 0x7f8bc8102b80 2024-05-08T21:59:50.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.609+0000 7f8bbf400700 1 -- 172.21.15.131:0/1345437417 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8bb8003a10 con 0x7f8bc8102b80 2024-05-08T21:59:50.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.609+0000 7f8bcd800700 1 -- 172.21.15.131:0/1345437417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc8198700 con 0x7f8bc8102b80 2024-05-08T21:59:50.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.609+0000 7f8bcd800700 1 -- 172.21.15.131:0/1345437417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc8198ba0 con 0x7f8bc8102b80 2024-05-08T21:59:50.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.610+0000 7f8bbf400700 1 -- 172.21.15.131:0/1345437417 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f8bb8003c80 con 0x7f8bc8102b80 2024-05-08T21:59:50.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.610+0000 7f8bcd800700 1 -- 172.21.15.131:0/1345437417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc81918a0 con 0x7f8bc8102b80 2024-05-08T21:59:50.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.611+0000 7f8bbf400700 1 --2- 172.21.15.131:0/1345437417 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8bac0387f0 0x7f8bac03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:50.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.611+0000 7f8bbf400700 1 -- 172.21.15.131:0/1345437417 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8bb804c520 con 0x7f8bc8102b80 2024-05-08T21:59:50.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.611+0000 7f8bc6000700 1 --2- 172.21.15.131:0/1345437417 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8bac0387f0 0x7f8bac03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:50.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.612+0000 7f8bc6000700 1 --2- 172.21.15.131:0/1345437417 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8bac0387f0 0x7f8bac03acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4006fd0 tx=0x7f8bb4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:50.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.616+0000 7f8bbf400700 1 -- 172.21.15.131:0/1345437417 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8bb800ba70 con 0x7f8bc8102b80 2024-05-08T21:59:50.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.855+0000 7f8bcd800700 1 -- 172.21.15.131:0/1345437417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f8bc80625d0 con 0x7f8bc8102b80 2024-05-08T21:59:50.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.859+0000 7f8bbf400700 1 -- 172.21.15.131:0/1345437417 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f8bc80625d0 con 0x7f8bc8102b80 2024-05-08T21:59:50.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.862+0000 7f8bcd800700 1 -- 172.21.15.131:0/1345437417 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8bac0387f0 msgr2=0x7f8bac03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:50.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.862+0000 7f8bcd800700 1 --2- 172.21.15.131:0/1345437417 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8bac0387f0 0x7f8bac03acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4006fd0 tx=0x7f8bb4006e40 comp rx=0 tx=0).stop 2024-05-08T21:59:50.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.862+0000 7f8bcd800700 1 -- 172.21.15.131:0/1345437417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc8102b80 msgr2=0x7f8bc8197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:50.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.862+0000 7f8bcd800700 1 --2- 172.21.15.131:0/1345437417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc8102b80 0x7f8bc8197fc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8005950 tx=0x7f8bb80057c0 comp rx=0 tx=0).stop 2024-05-08T21:59:50.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.862+0000 7f8bcd800700 1 -- 172.21.15.131:0/1345437417 shutdown_connections 2024-05-08T21:59:50.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.862+0000 7f8bcd800700 1 --2- 172.21.15.131:0/1345437417 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8bac0387f0 0x7f8bac03acb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:50.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.862+0000 7f8bcd800700 1 --2- 172.21.15.131:0/1345437417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc8102b80 0x7f8bc8197fc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:50.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.862+0000 7f8bcd800700 1 -- 172.21.15.131:0/1345437417 >> 172.21.15.131:0/1345437417 conn(0x7f8bc80fe120 msgr2=0x7f8bc80fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:50.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.862+0000 7f8bcd800700 1 -- 172.21.15.131:0/1345437417 shutdown_connections 2024-05-08T21:59:50.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:50.862+0000 7f8bcd800700 1 -- 172.21.15.131:0/1345437417 wait complete. 2024-05-08T21:59:51.263 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-08T21:59:51.263 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-08T21:59:51.623 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:59:52.103 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:51 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1345437417' entity='client.admin' 2024-05-08T21:59:53.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.334+0000 7f8e40c00700 1 -- 172.21.15.131:0/105577214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e3c100750 msgr2=0x7f8e3c100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:53.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.334+0000 7f8e40c00700 1 --2- 172.21.15.131:0/105577214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e3c100750 0x7f8e3c100b70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009910 tx=0x7f8e2c009c20 comp rx=0 tx=0).stop 2024-05-08T21:59:53.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.334+0000 7f8e40c00700 1 -- 172.21.15.131:0/105577214 shutdown_connections 2024-05-08T21:59:53.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.334+0000 7f8e40c00700 1 --2- 172.21.15.131:0/105577214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e3c100750 0x7f8e3c100b70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:53.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.334+0000 7f8e40c00700 1 -- 172.21.15.131:0/105577214 >> 172.21.15.131:0/105577214 conn(0x7f8e3c0fbd10 msgr2=0x7f8e3c0fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:53.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.334+0000 7f8e40c00700 1 -- 172.21.15.131:0/105577214 shutdown_connections 2024-05-08T21:59:53.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.334+0000 7f8e40c00700 1 -- 172.21.15.131:0/105577214 wait complete. 2024-05-08T21:59:53.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.335+0000 7f8e40c00700 1 Processor -- start 2024-05-08T21:59:53.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.335+0000 7f8e40c00700 1 -- start start 2024-05-08T21:59:53.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.336+0000 7f8e40c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e3c100750 0x7f8e3c197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:53.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.336+0000 7f8e40c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c198370 con 0x7f8e3c100750 2024-05-08T21:59:53.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.336+0000 7f8e3b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e3c100750 0x7f8e3c197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:53.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.336+0000 7f8e3b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e3c100750 0x7f8e3c197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47228/0 (socket says 172.21.15.131:47228) 2024-05-08T21:59:53.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.336+0000 7f8e3b400700 1 -- 172.21.15.131:0/3198173161 learned_addr learned my addr 172.21.15.131:0/3198173161 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:53.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.336+0000 7f8e3b400700 1 -- 172.21.15.131:0/3198173161 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2c0095f0 con 0x7f8e3c100750 2024-05-08T21:59:53.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.336+0000 7f8e3b400700 1 --2- 172.21.15.131:0/3198173161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e3c100750 0x7f8e3c197e30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c0092e0 tx=0x7f8e2c009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:53.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.337+0000 7f8e33e00700 1 -- 172.21.15.131:0/3198173161 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e2c01d070 con 0x7f8e3c100750 2024-05-08T21:59:53.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.337+0000 7f8e33e00700 1 -- 172.21.15.131:0/3198173161 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e2c00be20 con 0x7f8e3c100750 2024-05-08T21:59:53.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.337+0000 7f8e33e00700 1 -- 172.21.15.131:0/3198173161 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e2c022d60 con 0x7f8e3c100750 2024-05-08T21:59:53.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.337+0000 7f8e40c00700 1 -- 172.21.15.131:0/3198173161 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e3c198570 con 0x7f8e3c100750 2024-05-08T21:59:53.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.337+0000 7f8e40c00700 1 -- 172.21.15.131:0/3198173161 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3c198a10 con 0x7f8e3c100750 2024-05-08T21:59:53.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.338+0000 7f8e33e00700 1 -- 172.21.15.131:0/3198173161 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f8e2c00b960 con 0x7f8e3c100750 2024-05-08T21:59:53.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.338+0000 7f8e40c00700 1 -- 172.21.15.131:0/3198173161 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e3c191870 con 0x7f8e3c100750 2024-05-08T21:59:53.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.338+0000 7f8e33e00700 1 --2- 172.21.15.131:0/3198173161 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8e20038840 0x7f8e2003ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:53.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.338+0000 7f8e33e00700 1 -- 172.21.15.131:0/3198173161 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8e2c04cdf0 con 0x7f8e3c100750 2024-05-08T21:59:53.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.338+0000 7f8e3aa00700 1 --2- 172.21.15.131:0/3198173161 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8e20038840 0x7f8e2003ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:53.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.339+0000 7f8e3aa00700 1 --2- 172.21.15.131:0/3198173161 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8e20038840 0x7f8e2003ad00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8e28006fd0 tx=0x7f8e28006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:53.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.344+0000 7f8e33e00700 1 -- 172.21.15.131:0/3198173161 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8e2c02bba0 con 0x7f8e3c100750 2024-05-08T21:59:53.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.548+0000 7f8e40c00700 1 -- 172.21.15.131:0/3198173161 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e3c0613e0 con 0x7f8e20038840 2024-05-08T21:59:53.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.552+0000 7f8e33e00700 1 -- 172.21.15.131:0/3198173161 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8e3c0613e0 con 0x7f8e20038840 2024-05-08T21:59:53.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.554+0000 7f8e40c00700 1 -- 172.21.15.131:0/3198173161 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8e20038840 msgr2=0x7f8e2003ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:53.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.554+0000 7f8e40c00700 1 --2- 172.21.15.131:0/3198173161 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8e20038840 0x7f8e2003ad00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8e28006fd0 tx=0x7f8e28006e40 comp rx=0 tx=0).stop 2024-05-08T21:59:53.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.554+0000 7f8e40c00700 1 -- 172.21.15.131:0/3198173161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e3c100750 msgr2=0x7f8e3c197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:53.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.554+0000 7f8e40c00700 1 --2- 172.21.15.131:0/3198173161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e3c100750 0x7f8e3c197e30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c0092e0 tx=0x7f8e2c009f40 comp rx=0 tx=0).stop 2024-05-08T21:59:53.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.554+0000 7f8e40c00700 1 -- 172.21.15.131:0/3198173161 shutdown_connections 2024-05-08T21:59:53.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.554+0000 7f8e40c00700 1 --2- 172.21.15.131:0/3198173161 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8e20038840 0x7f8e2003ad00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:53.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.554+0000 7f8e40c00700 1 --2- 172.21.15.131:0/3198173161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e3c100750 0x7f8e3c197e30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:53.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.554+0000 7f8e40c00700 1 -- 172.21.15.131:0/3198173161 >> 172.21.15.131:0/3198173161 conn(0x7f8e3c0fbd10 msgr2=0x7f8e3c0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:53.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.554+0000 7f8e40c00700 1 -- 172.21.15.131:0/3198173161 shutdown_connections 2024-05-08T21:59:53.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:53.554+0000 7f8e40c00700 1 -- 172.21.15.131:0/3198173161 wait complete. 2024-05-08T21:59:53.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:53 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T21:59:53.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:53 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T21:59:53.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:53 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi131", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:59:53.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:53 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T21:59:53.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:53 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:59:53.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:53 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T21:59:53.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:53 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T21:59:53.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:53 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:59:53.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:53 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:59:53.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:53 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:59:53.975 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi132 2024-05-08T21:59:53.975 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-05-08T21:59:53.975 DEBUG:teuthology.orchestra.run.smithi132:> dd of=/etc/ceph/ceph.conf 2024-05-08T21:59:53.996 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-05-08T21:59:53.996 DEBUG:teuthology.orchestra.run.smithi132:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T21:59:54.056 INFO:tasks.cephadm:Adding host smithi132 to orchestrator... 2024-05-08T21:59:54.056 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph orch host add smithi132 2024-05-08T21:59:54.423 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:59:54.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:54 smithi131 ceph-mon[37260]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:59:54.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:54 smithi131 ceph-mon[37260]: Updating smithi131:/etc/ceph/ceph.conf 2024-05-08T21:59:55.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.664+0000 7fe3d3c00700 1 -- 172.21.15.131:0/2293407232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3cc1068b0 msgr2=0x7fe3cc106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:55.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.664+0000 7fe3d3c00700 1 --2- 172.21.15.131:0/2293407232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3cc1068b0 0x7fe3cc106c90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009910 tx=0x7fe3c0009c20 comp rx=0 tx=0).stop 2024-05-08T21:59:55.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.664+0000 7fe3d3c00700 1 -- 172.21.15.131:0/2293407232 shutdown_connections 2024-05-08T21:59:55.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.664+0000 7fe3d3c00700 1 --2- 172.21.15.131:0/2293407232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3cc1068b0 0x7fe3cc106c90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:55.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.664+0000 7fe3d3c00700 1 -- 172.21.15.131:0/2293407232 >> 172.21.15.131:0/2293407232 conn(0x7fe3cc0fc3c0 msgr2=0x7fe3cc0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:55.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.664+0000 7fe3d3c00700 1 -- 172.21.15.131:0/2293407232 shutdown_connections 2024-05-08T21:59:55.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.664+0000 7fe3d3c00700 1 -- 172.21.15.131:0/2293407232 wait complete. 2024-05-08T21:59:55.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.665+0000 7fe3d3c00700 1 Processor -- start 2024-05-08T21:59:55.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.666+0000 7fe3d3c00700 1 -- start start 2024-05-08T21:59:55.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.666+0000 7fe3d3c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3cc1068b0 0x7fe3cc196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:55.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.666+0000 7fe3d3c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc196670 con 0x7fe3cc1068b0 2024-05-08T21:59:55.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.666+0000 7fe3d1200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3cc1068b0 0x7fe3cc196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:55.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.666+0000 7fe3d1200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3cc1068b0 0x7fe3cc196130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56390/0 (socket says 172.21.15.131:56390) 2024-05-08T21:59:55.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.666+0000 7fe3d1200700 1 -- 172.21.15.131:0/2513719948 learned_addr learned my addr 172.21.15.131:0/2513719948 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T21:59:55.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.667+0000 7fe3d1200700 1 -- 172.21.15.131:0/2513719948 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c00095f0 con 0x7fe3cc1068b0 2024-05-08T21:59:55.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.667+0000 7fe3d1200700 1 --2- 172.21.15.131:0/2513719948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3cc1068b0 0x7fe3cc196130 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0006010 tx=0x7fe3c0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:55.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.667+0000 7fe3c9600700 1 -- 172.21.15.131:0/2513719948 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3c001d070 con 0x7fe3cc1068b0 2024-05-08T21:59:55.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.667+0000 7fe3d3c00700 1 -- 172.21.15.131:0/2513719948 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3cc1968d0 con 0x7fe3cc1068b0 2024-05-08T21:59:55.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.667+0000 7fe3c9600700 1 -- 172.21.15.131:0/2513719948 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3c000bbb0 con 0x7fe3cc1068b0 2024-05-08T21:59:55.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.667+0000 7fe3c9600700 1 -- 172.21.15.131:0/2513719948 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3c0003810 con 0x7fe3cc1068b0 2024-05-08T21:59:55.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.668+0000 7fe3d3c00700 1 -- 172.21.15.131:0/2513719948 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3cc196d50 con 0x7fe3cc1068b0 2024-05-08T21:59:55.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.668+0000 7fe3c9600700 1 -- 172.21.15.131:0/2513719948 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7fe3c000b6f0 con 0x7fe3cc1068b0 2024-05-08T21:59:55.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.669+0000 7fe3d3c00700 1 -- 172.21.15.131:0/2513719948 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3cc101ff0 con 0x7fe3cc1068b0 2024-05-08T21:59:55.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.669+0000 7fe3c9600700 1 --2- 172.21.15.131:0/2513719948 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe3b40387a0 0x7fe3b403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:59:55.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.669+0000 7fe3c9600700 1 -- 172.21.15.131:0/2513719948 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe3c004c3e0 con 0x7fe3cc1068b0 2024-05-08T21:59:55.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.669+0000 7fe3cbe00700 1 --2- 172.21.15.131:0/2513719948 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe3b40387a0 0x7fe3b403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:59:55.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.670+0000 7fe3cbe00700 1 --2- 172.21.15.131:0/2513719948 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe3b40387a0 0x7fe3b403ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8006fd0 tx=0x7fe3b8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:59:55.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.674+0000 7fe3c9600700 1 -- 172.21.15.131:0/2513719948 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe3c0029540 con 0x7fe3cc1068b0 2024-05-08T21:59:55.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:55 smithi131 ceph-mon[37260]: Updating smithi131:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T21:59:55.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:55 smithi131 ceph-mon[37260]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-05-08T21:59:55.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:55.892+0000 7fe3d3c00700 1 -- 172.21.15.131:0/2513719948 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi132", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3cc10abb0 con 0x7fe3b40387a0 2024-05-08T21:59:57.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:56 smithi131 ceph-mon[37260]: Updating smithi131:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T21:59:57.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:56 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T21:59:57.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:56 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T21:59:57.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:56 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T21:59:57.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:56 smithi131 ceph-mon[37260]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi132", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:59:58.107 INFO:teuthology.orchestra.run.smithi131.stdout:Added host 'smithi132' with addr '172.21.15.132' 2024-05-08T21:59:58.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:58.106+0000 7fe3c9600700 1 -- 172.21.15.131:0/2513719948 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fe3cc10abb0 con 0x7fe3b40387a0 2024-05-08T21:59:58.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:58.108+0000 7fe3d3c00700 1 -- 172.21.15.131:0/2513719948 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe3b40387a0 msgr2=0x7fe3b403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:58.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:58.108+0000 7fe3d3c00700 1 --2- 172.21.15.131:0/2513719948 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe3b40387a0 0x7fe3b403ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8006fd0 tx=0x7fe3b8006e40 comp rx=0 tx=0).stop 2024-05-08T21:59:58.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:58.108+0000 7fe3d3c00700 1 -- 172.21.15.131:0/2513719948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3cc1068b0 msgr2=0x7fe3cc196130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:59:58.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:58.108+0000 7fe3d3c00700 1 --2- 172.21.15.131:0/2513719948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3cc1068b0 0x7fe3cc196130 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0006010 tx=0x7fe3c0004ff0 comp rx=0 tx=0).stop 2024-05-08T21:59:58.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:58.109+0000 7fe3d3c00700 1 -- 172.21.15.131:0/2513719948 shutdown_connections 2024-05-08T21:59:58.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:58.109+0000 7fe3d3c00700 1 --2- 172.21.15.131:0/2513719948 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe3b40387a0 0x7fe3b403ac60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:58.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:58.109+0000 7fe3d3c00700 1 --2- 172.21.15.131:0/2513719948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3cc1068b0 0x7fe3cc196130 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:59:58.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:58.109+0000 7fe3d3c00700 1 -- 172.21.15.131:0/2513719948 >> 172.21.15.131:0/2513719948 conn(0x7fe3cc0fc3c0 msgr2=0x7fe3cc108560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:59:58.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:58.109+0000 7fe3d3c00700 1 -- 172.21.15.131:0/2513719948 shutdown_connections 2024-05-08T21:59:58.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T21:59:58.109+0000 7fe3d3c00700 1 -- 172.21.15.131:0/2513719948 wait complete. 2024-05-08T21:59:58.131 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:57 smithi131 ceph-mon[37260]: Deploying cephadm binary to smithi132 2024-05-08T21:59:58.500 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-08T21:59:58.850 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:59:59.374 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T21:59:59.374 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:59 smithi131 ceph-mon[37260]: Added host smithi132 2024-05-08T21:59:59.374 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:59:59.374 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T21:59:59.374 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 21:59:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:00.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.028+0000 7fef55800700 1 -- 172.21.15.131:0/194380097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef501068b0 msgr2=0x7fef50106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:00.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.028+0000 7fef55800700 1 --2- 172.21.15.131:0/194380097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef501068b0 0x7fef50106c90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fef40009910 tx=0x7fef40009c20 comp rx=0 tx=0).stop 2024-05-08T22:00:00.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.028+0000 7fef55800700 1 -- 172.21.15.131:0/194380097 shutdown_connections 2024-05-08T22:00:00.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.028+0000 7fef55800700 1 --2- 172.21.15.131:0/194380097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef501068b0 0x7fef50106c90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:00.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.028+0000 7fef55800700 1 -- 172.21.15.131:0/194380097 >> 172.21.15.131:0/194380097 conn(0x7fef500fc3c0 msgr2=0x7fef500fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:00.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.028+0000 7fef55800700 1 -- 172.21.15.131:0/194380097 shutdown_connections 2024-05-08T22:00:00.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.028+0000 7fef55800700 1 -- 172.21.15.131:0/194380097 wait complete. 2024-05-08T22:00:00.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.029+0000 7fef55800700 1 Processor -- start 2024-05-08T22:00:00.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.029+0000 7fef55800700 1 -- start start 2024-05-08T22:00:00.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.029+0000 7fef55800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef501068b0 0x7fef50198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:00.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.029+0000 7fef55800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef50198880 con 0x7fef501068b0 2024-05-08T22:00:00.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.030+0000 7fef4ea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef501068b0 0x7fef50198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:00.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.030+0000 7fef4ea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef501068b0 0x7fef50198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56416/0 (socket says 172.21.15.131:56416) 2024-05-08T22:00:00.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.030+0000 7fef4ea00700 1 -- 172.21.15.131:0/522938685 learned_addr learned my addr 172.21.15.131:0/522938685 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:00:00.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.030+0000 7fef4ea00700 1 -- 172.21.15.131:0/522938685 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef400095f0 con 0x7fef501068b0 2024-05-08T22:00:00.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.030+0000 7fef4ea00700 1 --2- 172.21.15.131:0/522938685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef501068b0 0x7fef50198340 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fef40006010 tx=0x7fef40004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:00.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.030+0000 7fef47400700 1 -- 172.21.15.131:0/522938685 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef4001d070 con 0x7fef501068b0 2024-05-08T22:00:00.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.030+0000 7fef47400700 1 -- 172.21.15.131:0/522938685 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef4000bbb0 con 0x7fef501068b0 2024-05-08T22:00:00.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.030+0000 7fef47400700 1 -- 172.21.15.131:0/522938685 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef40003810 con 0x7fef501068b0 2024-05-08T22:00:00.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.030+0000 7fef55800700 1 -- 172.21.15.131:0/522938685 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef50198ae0 con 0x7fef501068b0 2024-05-08T22:00:00.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.031+0000 7fef55800700 1 -- 172.21.15.131:0/522938685 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef50198f60 con 0x7fef501068b0 2024-05-08T22:00:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.032+0000 7fef47400700 1 -- 172.21.15.131:0/522938685 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7fef4000bd20 con 0x7fef501068b0 2024-05-08T22:00:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.032+0000 7fef55800700 1 -- 172.21.15.131:0/522938685 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef50104790 con 0x7fef501068b0 2024-05-08T22:00:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.032+0000 7fef47400700 1 --2- 172.21.15.131:0/522938685 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fef340387f0 0x7fef3403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.032+0000 7fef47400700 1 -- 172.21.15.131:0/522938685 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fef4004c410 con 0x7fef501068b0 2024-05-08T22:00:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.033+0000 7fef4e000700 1 --2- 172.21.15.131:0/522938685 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fef340387f0 0x7fef3403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:00.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.033+0000 7fef4e000700 1 --2- 172.21.15.131:0/522938685 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fef340387f0 0x7fef3403acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fef3c006fd0 tx=0x7fef3c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:00.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.038+0000 7fef47400700 1 -- 172.21.15.131:0/522938685 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fef40029360 con 0x7fef501068b0 2024-05-08T22:00:00.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.247+0000 7fef55800700 1 -- 172.21.15.131:0/522938685 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fef5010abb0 con 0x7fef340387f0 2024-05-08T22:00:00.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.248+0000 7fef47400700 1 -- 172.21.15.131:0/522938685 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7fef5010abb0 con 0x7fef340387f0 2024-05-08T22:00:00.249 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:00:00.249 INFO:teuthology.orchestra.run.smithi131.stdout:[{"addr": "172.21.15.131", "hostname": "smithi131", "labels": [], "status": ""}, {"addr": "172.21.15.132", "hostname": "smithi132", "labels": [], "status": ""}] 2024-05-08T22:00:00.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.251+0000 7fef55800700 1 -- 172.21.15.131:0/522938685 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fef340387f0 msgr2=0x7fef3403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:00.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.251+0000 7fef55800700 1 --2- 172.21.15.131:0/522938685 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fef340387f0 0x7fef3403acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fef3c006fd0 tx=0x7fef3c006e40 comp rx=0 tx=0).stop 2024-05-08T22:00:00.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.251+0000 7fef55800700 1 -- 172.21.15.131:0/522938685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef501068b0 msgr2=0x7fef50198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:00.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.251+0000 7fef55800700 1 --2- 172.21.15.131:0/522938685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef501068b0 0x7fef50198340 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fef40006010 tx=0x7fef40004ff0 comp rx=0 tx=0).stop 2024-05-08T22:00:00.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.251+0000 7fef55800700 1 -- 172.21.15.131:0/522938685 shutdown_connections 2024-05-08T22:00:00.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.251+0000 7fef55800700 1 --2- 172.21.15.131:0/522938685 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fef340387f0 0x7fef3403acb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:00.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.251+0000 7fef55800700 1 --2- 172.21.15.131:0/522938685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef501068b0 0x7fef50198340 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:00.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.251+0000 7fef55800700 1 -- 172.21.15.131:0/522938685 >> 172.21.15.131:0/522938685 conn(0x7fef500fc3c0 msgr2=0x7fef50108560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:00.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.251+0000 7fef55800700 1 -- 172.21.15.131:0/522938685 shutdown_connections 2024-05-08T22:00:00.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:00.251+0000 7fef55800700 1 -- 172.21.15.131:0/522938685 wait complete. 2024-05-08T22:00:00.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:00 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:00.689 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi157 2024-05-08T22:00:00.689 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-08T22:00:00.689 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.conf 2024-05-08T22:00:00.711 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-08T22:00:00.711 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:00:00.771 INFO:tasks.cephadm:Adding host smithi157 to orchestrator... 2024-05-08T22:00:00.771 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph orch host add smithi157 2024-05-08T22:00:01.122 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:00:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:01 smithi131 ceph-mon[37260]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:00:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:01 smithi131 ceph-mon[37260]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:02.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.430+0000 7fa287200700 1 -- 172.21.15.131:0/3949796806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa280101a80 msgr2=0x7fa280101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:02.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.430+0000 7fa287200700 1 --2- 172.21.15.131:0/3949796806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa280101a80 0x7fa280101e60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa270009910 tx=0x7fa270009c20 comp rx=0 tx=0).stop 2024-05-08T22:00:02.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.430+0000 7fa287200700 1 -- 172.21.15.131:0/3949796806 shutdown_connections 2024-05-08T22:00:02.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.430+0000 7fa287200700 1 --2- 172.21.15.131:0/3949796806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa280101a80 0x7fa280101e60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:02.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.430+0000 7fa287200700 1 -- 172.21.15.131:0/3949796806 >> 172.21.15.131:0/3949796806 conn(0x7fa280076450 msgr2=0x7fa2800fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:02.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.430+0000 7fa287200700 1 -- 172.21.15.131:0/3949796806 shutdown_connections 2024-05-08T22:00:02.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.430+0000 7fa287200700 1 -- 172.21.15.131:0/3949796806 wait complete. 2024-05-08T22:00:02.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.431+0000 7fa287200700 1 Processor -- start 2024-05-08T22:00:02.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.431+0000 7fa287200700 1 -- start start 2024-05-08T22:00:02.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.432+0000 7fa287200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa280101a80 0x7fa280196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:02.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.432+0000 7fa287200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2801967a0 con 0x7fa280101a80 2024-05-08T22:00:02.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.432+0000 7fa27fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa280101a80 0x7fa280196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:02.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.432+0000 7fa27fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa280101a80 0x7fa280196260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56430/0 (socket says 172.21.15.131:56430) 2024-05-08T22:00:02.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.432+0000 7fa27fe00700 1 -- 172.21.15.131:0/4156942486 learned_addr learned my addr 172.21.15.131:0/4156942486 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:00:02.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.433+0000 7fa27fe00700 1 -- 172.21.15.131:0/4156942486 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2700095f0 con 0x7fa280101a80 2024-05-08T22:00:02.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.433+0000 7fa27fe00700 1 --2- 172.21.15.131:0/4156942486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa280101a80 0x7fa280196260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa270006010 tx=0x7fa270004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:02.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.433+0000 7fa27cc00700 1 -- 172.21.15.131:0/4156942486 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa27001d070 con 0x7fa280101a80 2024-05-08T22:00:02.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.433+0000 7fa27cc00700 1 -- 172.21.15.131:0/4156942486 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa27000bc10 con 0x7fa280101a80 2024-05-08T22:00:02.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.433+0000 7fa27cc00700 1 -- 172.21.15.131:0/4156942486 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2700038f0 con 0x7fa280101a80 2024-05-08T22:00:02.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.433+0000 7fa287200700 1 -- 172.21.15.131:0/4156942486 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2801969a0 con 0x7fa280101a80 2024-05-08T22:00:02.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.433+0000 7fa287200700 1 -- 172.21.15.131:0/4156942486 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa280196dc0 con 0x7fa280101a80 2024-05-08T22:00:02.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.434+0000 7fa27cc00700 1 -- 172.21.15.131:0/4156942486 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7fa27000b6f0 con 0x7fa280101a80 2024-05-08T22:00:02.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.434+0000 7fa287200700 1 -- 172.21.15.131:0/4156942486 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa280068980 con 0x7fa280101a80 2024-05-08T22:00:02.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.435+0000 7fa27cc00700 1 --2- 172.21.15.131:0/4156942486 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa2640387f0 0x7fa26403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:02.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.435+0000 7fa27cc00700 1 -- 172.21.15.131:0/4156942486 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa27004c4f0 con 0x7fa280101a80 2024-05-08T22:00:02.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.435+0000 7fa27f400700 1 --2- 172.21.15.131:0/4156942486 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa2640387f0 0x7fa26403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:02.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.435+0000 7fa27f400700 1 --2- 172.21.15.131:0/4156942486 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa2640387f0 0x7fa26403acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa26c006fd0 tx=0x7fa26c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:02.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.440+0000 7fa27cc00700 1 -- 172.21.15.131:0/4156942486 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa270029540 con 0x7fa280101a80 2024-05-08T22:00:02.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:02.653+0000 7fa287200700 1 -- 172.21.15.131:0/4156942486 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}) v1 -- 0x7fa28010ac90 con 0x7fa2640387f0 2024-05-08T22:00:03.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:02 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:03.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:02 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:03.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:02 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:03.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:02 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:03.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:02 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi132", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:00:03.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:02 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:03.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:02 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:00:03.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:02 smithi131 ceph-mon[37260]: Updating smithi132:/etc/ceph/ceph.conf 2024-05-08T22:00:03.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:02 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:03.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:02 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:03.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:02 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:04.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:03 smithi131 ceph-mon[37260]: Updating smithi132:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:00:04.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:03 smithi131 ceph-mon[37260]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:00:04.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:03 smithi131 ceph-mon[37260]: Updating smithi132:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T22:00:04.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:03 smithi131 ceph-mon[37260]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:04.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:03 smithi131 ceph-mon[37260]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:00:04.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:04.871+0000 7fa27cc00700 1 -- 172.21.15.131:0/4156942486 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fa28010ac90 con 0x7fa2640387f0 2024-05-08T22:00:04.872 INFO:teuthology.orchestra.run.smithi131.stdout:Added host 'smithi157' with addr '172.21.15.157' 2024-05-08T22:00:04.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:04.874+0000 7fa287200700 1 -- 172.21.15.131:0/4156942486 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa2640387f0 msgr2=0x7fa26403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:04.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:04.874+0000 7fa287200700 1 --2- 172.21.15.131:0/4156942486 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa2640387f0 0x7fa26403acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa26c006fd0 tx=0x7fa26c006e40 comp rx=0 tx=0).stop 2024-05-08T22:00:04.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:04.874+0000 7fa287200700 1 -- 172.21.15.131:0/4156942486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa280101a80 msgr2=0x7fa280196260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:04.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:04.874+0000 7fa287200700 1 --2- 172.21.15.131:0/4156942486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa280101a80 0x7fa280196260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa270006010 tx=0x7fa270004ff0 comp rx=0 tx=0).stop 2024-05-08T22:00:04.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:04.874+0000 7fa287200700 1 -- 172.21.15.131:0/4156942486 shutdown_connections 2024-05-08T22:00:04.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:04.874+0000 7fa287200700 1 --2- 172.21.15.131:0/4156942486 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa2640387f0 0x7fa26403acb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:04.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:04.874+0000 7fa287200700 1 --2- 172.21.15.131:0/4156942486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa280101a80 0x7fa280196260 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:04.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:04.874+0000 7fa287200700 1 -- 172.21.15.131:0/4156942486 >> 172.21.15.131:0/4156942486 conn(0x7fa280076450 msgr2=0x7fa280104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:04.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:04.874+0000 7fa287200700 1 -- 172.21.15.131:0/4156942486 shutdown_connections 2024-05-08T22:00:04.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:04.874+0000 7fa287200700 1 -- 172.21.15.131:0/4156942486 wait complete. 2024-05-08T22:00:05.208 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:04 smithi131 ceph-mon[37260]: Deploying cephadm binary to smithi157 2024-05-08T22:00:05.208 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:04 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:05.208 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:04 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:00:05.237 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-08T22:00:05.593 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:00:06.138 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:05 smithi131 ceph-mon[37260]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:06.138 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:05 smithi131 ceph-mon[37260]: Added host smithi157 2024-05-08T22:00:06.138 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:05 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:06.138 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:05 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:06.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.752+0000 7f0a51800700 1 -- 172.21.15.131:0/2585640644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a4c1068b0 msgr2=0x7f0a4c106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:06.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.752+0000 7f0a51800700 1 --2- 172.21.15.131:0/2585640644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a4c1068b0 0x7f0a4c106c90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c009910 tx=0x7f0a3c009c20 comp rx=0 tx=0).stop 2024-05-08T22:00:06.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.752+0000 7f0a51800700 1 -- 172.21.15.131:0/2585640644 shutdown_connections 2024-05-08T22:00:06.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.752+0000 7f0a51800700 1 --2- 172.21.15.131:0/2585640644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a4c1068b0 0x7f0a4c106c90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:06.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.752+0000 7f0a51800700 1 -- 172.21.15.131:0/2585640644 >> 172.21.15.131:0/2585640644 conn(0x7f0a4c0fc3c0 msgr2=0x7f0a4c0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:06.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.752+0000 7f0a51800700 1 -- 172.21.15.131:0/2585640644 shutdown_connections 2024-05-08T22:00:06.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.752+0000 7f0a51800700 1 -- 172.21.15.131:0/2585640644 wait complete. 2024-05-08T22:00:06.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.753+0000 7f0a51800700 1 Processor -- start 2024-05-08T22:00:06.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.753+0000 7f0a51800700 1 -- start start 2024-05-08T22:00:06.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.753+0000 7f0a51800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a4c1068b0 0x7f0a4c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:06.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.753+0000 7f0a51800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c198880 con 0x7f0a4c1068b0 2024-05-08T22:00:06.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.754+0000 7f0a4aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a4c1068b0 0x7f0a4c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:06.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.754+0000 7f0a4aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a4c1068b0 0x7f0a4c198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47148/0 (socket says 172.21.15.131:47148) 2024-05-08T22:00:06.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.754+0000 7f0a4aa00700 1 -- 172.21.15.131:0/4011009654 learned_addr learned my addr 172.21.15.131:0/4011009654 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:00:06.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.754+0000 7f0a4aa00700 1 -- 172.21.15.131:0/4011009654 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c0095f0 con 0x7f0a4c1068b0 2024-05-08T22:00:06.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.754+0000 7f0a4aa00700 1 --2- 172.21.15.131:0/4011009654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a4c1068b0 0x7f0a4c198340 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c006010 tx=0x7f0a3c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:06.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.754+0000 7f0a43400700 1 -- 172.21.15.131:0/4011009654 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a3c01d070 con 0x7f0a4c1068b0 2024-05-08T22:00:06.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.755+0000 7f0a43400700 1 -- 172.21.15.131:0/4011009654 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a3c00bbb0 con 0x7f0a4c1068b0 2024-05-08T22:00:06.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.755+0000 7f0a51800700 1 -- 172.21.15.131:0/4011009654 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a4c198ae0 con 0x7f0a4c1068b0 2024-05-08T22:00:06.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.755+0000 7f0a43400700 1 -- 172.21.15.131:0/4011009654 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a3c003810 con 0x7f0a4c1068b0 2024-05-08T22:00:06.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.755+0000 7f0a51800700 1 -- 172.21.15.131:0/4011009654 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a4c198f60 con 0x7f0a4c1068b0 2024-05-08T22:00:06.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.756+0000 7f0a43400700 1 -- 172.21.15.131:0/4011009654 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f0a3c00b6f0 con 0x7f0a4c1068b0 2024-05-08T22:00:06.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.756+0000 7f0a51800700 1 -- 172.21.15.131:0/4011009654 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a4c101ff0 con 0x7f0a4c1068b0 2024-05-08T22:00:06.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.756+0000 7f0a43400700 1 --2- 172.21.15.131:0/4011009654 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f0a300387a0 0x7f0a3003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:06.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.756+0000 7f0a43400700 1 -- 172.21.15.131:0/4011009654 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0a3c04c4a0 con 0x7f0a4c1068b0 2024-05-08T22:00:06.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.756+0000 7f0a4a000700 1 --2- 172.21.15.131:0/4011009654 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f0a300387a0 0x7f0a3003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:06.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.757+0000 7f0a4a000700 1 --2- 172.21.15.131:0/4011009654 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f0a300387a0 0x7f0a3003ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0a38006fd0 tx=0x7f0a38006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:06.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.761+0000 7f0a43400700 1 -- 172.21.15.131:0/4011009654 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0a3c029540 con 0x7f0a4c1068b0 2024-05-08T22:00:06.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.966+0000 7f0a51800700 1 -- 172.21.15.131:0/4011009654 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0a4c10abb0 con 0x7f0a300387a0 2024-05-08T22:00:06.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.967+0000 7f0a43400700 1 -- 172.21.15.131:0/4011009654 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+241 (secure 0 0 0) 0x7f0a4c10abb0 con 0x7f0a300387a0 2024-05-08T22:00:06.968 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:00:06.968 INFO:teuthology.orchestra.run.smithi131.stdout:[{"addr": "172.21.15.131", "hostname": "smithi131", "labels": [], "status": ""}, {"addr": "172.21.15.132", "hostname": "smithi132", "labels": [], "status": ""}, {"addr": "172.21.15.157", "hostname": "smithi157", "labels": [], "status": ""}] 2024-05-08T22:00:06.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.970+0000 7f0a51800700 1 -- 172.21.15.131:0/4011009654 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f0a300387a0 msgr2=0x7f0a3003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:06.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.970+0000 7f0a51800700 1 --2- 172.21.15.131:0/4011009654 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f0a300387a0 0x7f0a3003ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0a38006fd0 tx=0x7f0a38006e40 comp rx=0 tx=0).stop 2024-05-08T22:00:06.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.970+0000 7f0a51800700 1 -- 172.21.15.131:0/4011009654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a4c1068b0 msgr2=0x7f0a4c198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:06.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.970+0000 7f0a51800700 1 --2- 172.21.15.131:0/4011009654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a4c1068b0 0x7f0a4c198340 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c006010 tx=0x7f0a3c004ff0 comp rx=0 tx=0).stop 2024-05-08T22:00:06.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.970+0000 7f0a51800700 1 -- 172.21.15.131:0/4011009654 shutdown_connections 2024-05-08T22:00:06.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.970+0000 7f0a51800700 1 --2- 172.21.15.131:0/4011009654 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f0a300387a0 0x7f0a3003ac60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:06.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.970+0000 7f0a51800700 1 --2- 172.21.15.131:0/4011009654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a4c1068b0 0x7f0a4c198340 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:06.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.970+0000 7f0a51800700 1 -- 172.21.15.131:0/4011009654 >> 172.21.15.131:0/4011009654 conn(0x7f0a4c0fc3c0 msgr2=0x7f0a4c108560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:06.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.970+0000 7f0a51800700 1 -- 172.21.15.131:0/4011009654 shutdown_connections 2024-05-08T22:00:06.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:06.970+0000 7f0a51800700 1 -- 172.21.15.131:0/4011009654 wait complete. 2024-05-08T22:00:07.352 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-08T22:00:07.352 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-08T22:00:07.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:07 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:07.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:07 smithi131 ceph-mon[37260]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:07.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:07 smithi131 ceph-mon[37260]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:00:07.705 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:00:08.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.915+0000 7fa541600700 1 -- 172.21.15.131:0/2534180558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa53c1066f0 msgr2=0x7fa53c106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:08.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.915+0000 7fa541600700 1 --2- 172.21.15.131:0/2534180558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa53c1066f0 0x7fa53c106ad0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009910 tx=0x7fa52c009c20 comp rx=0 tx=0).stop 2024-05-08T22:00:08.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.915+0000 7fa541600700 1 -- 172.21.15.131:0/2534180558 shutdown_connections 2024-05-08T22:00:08.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.915+0000 7fa541600700 1 --2- 172.21.15.131:0/2534180558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa53c1066f0 0x7fa53c106ad0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:08.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.915+0000 7fa541600700 1 -- 172.21.15.131:0/2534180558 >> 172.21.15.131:0/2534180558 conn(0x7fa53c0fc1e0 msgr2=0x7fa53c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:08.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.915+0000 7fa541600700 1 -- 172.21.15.131:0/2534180558 shutdown_connections 2024-05-08T22:00:08.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.915+0000 7fa541600700 1 -- 172.21.15.131:0/2534180558 wait complete. 2024-05-08T22:00:08.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.916+0000 7fa541600700 1 Processor -- start 2024-05-08T22:00:08.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.916+0000 7fa541600700 1 -- start start 2024-05-08T22:00:08.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.917+0000 7fa541600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa53c1066f0 0x7fa53c198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:08.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.917+0000 7fa541600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c198660 con 0x7fa53c1066f0 2024-05-08T22:00:08.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.917+0000 7fa53be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa53c1066f0 0x7fa53c198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:08.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.917+0000 7fa53be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa53c1066f0 0x7fa53c198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47174/0 (socket says 172.21.15.131:47174) 2024-05-08T22:00:08.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.917+0000 7fa53be00700 1 -- 172.21.15.131:0/1586436009 learned_addr learned my addr 172.21.15.131:0/1586436009 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:00:08.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.917+0000 7fa53be00700 1 -- 172.21.15.131:0/1586436009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c0095f0 con 0x7fa53c1066f0 2024-05-08T22:00:08.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.918+0000 7fa53be00700 1 --2- 172.21.15.131:0/1586436009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa53c1066f0 0x7fa53c198120 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa52c006010 tx=0x7fa52c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:08.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.918+0000 7fa538c00700 1 -- 172.21.15.131:0/1586436009 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa52c01d070 con 0x7fa53c1066f0 2024-05-08T22:00:08.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.918+0000 7fa538c00700 1 -- 172.21.15.131:0/1586436009 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa52c00bbb0 con 0x7fa53c1066f0 2024-05-08T22:00:08.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.918+0000 7fa541600700 1 -- 172.21.15.131:0/1586436009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa53c1988c0 con 0x7fa53c1066f0 2024-05-08T22:00:08.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.918+0000 7fa538c00700 1 -- 172.21.15.131:0/1586436009 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa52c003810 con 0x7fa53c1066f0 2024-05-08T22:00:08.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.918+0000 7fa541600700 1 -- 172.21.15.131:0/1586436009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa53c198d40 con 0x7fa53c1066f0 2024-05-08T22:00:08.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.919+0000 7fa538c00700 1 -- 172.21.15.131:0/1586436009 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7fa52c0039c0 con 0x7fa53c1066f0 2024-05-08T22:00:08.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.919+0000 7fa541600700 1 -- 172.21.15.131:0/1586436009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa53c101e30 con 0x7fa53c1066f0 2024-05-08T22:00:08.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.920+0000 7fa538c00700 1 --2- 172.21.15.131:0/1586436009 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa5200387f0 0x7fa52003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:08.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.920+0000 7fa538c00700 1 -- 172.21.15.131:0/1586436009 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa52c04c560 con 0x7fa53c1066f0 2024-05-08T22:00:08.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.920+0000 7fa53b400700 1 --2- 172.21.15.131:0/1586436009 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa5200387f0 0x7fa52003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:08.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.921+0000 7fa53b400700 1 --2- 172.21.15.131:0/1586436009 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa5200387f0 0x7fa52003acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa528006fd0 tx=0x7fa528006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:08.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:08.926+0000 7fa538c00700 1 -- 172.21.15.131:0/1586436009 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa52c02abe0 con 0x7fa53c1066f0 2024-05-08T22:00:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:09.138+0000 7fa541600700 1 -- 172.21.15.131:0/1586436009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fa53c10a9f0 con 0x7fa53c1066f0 2024-05-08T22:00:09.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:09.547+0000 7fa538c00700 1 -- 172.21.15.131:0/1586436009 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fa52c026070 con 0x7fa53c1066f0 2024-05-08T22:00:09.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:09.550+0000 7fa541600700 1 -- 172.21.15.131:0/1586436009 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa5200387f0 msgr2=0x7fa52003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:09.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:09.550+0000 7fa541600700 1 --2- 172.21.15.131:0/1586436009 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa5200387f0 0x7fa52003acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa528006fd0 tx=0x7fa528006e40 comp rx=0 tx=0).stop 2024-05-08T22:00:09.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:09.550+0000 7fa541600700 1 -- 172.21.15.131:0/1586436009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa53c1066f0 msgr2=0x7fa53c198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:09.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:09.550+0000 7fa541600700 1 --2- 172.21.15.131:0/1586436009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa53c1066f0 0x7fa53c198120 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa52c006010 tx=0x7fa52c004ff0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:09.550+0000 7fa541600700 1 -- 172.21.15.131:0/1586436009 shutdown_connections 2024-05-08T22:00:09.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:09.550+0000 7fa541600700 1 --2- 172.21.15.131:0/1586436009 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa5200387f0 0x7fa52003acb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:09.550+0000 7fa541600700 1 --2- 172.21.15.131:0/1586436009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa53c1066f0 0x7fa53c198120 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:09.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:09.550+0000 7fa541600700 1 -- 172.21.15.131:0/1586436009 >> 172.21.15.131:0/1586436009 conn(0x7fa53c0fc1e0 msgr2=0x7fa53c108200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:09.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:09.550+0000 7fa541600700 1 -- 172.21.15.131:0/1586436009 shutdown_connections 2024-05-08T22:00:09.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:09.550+0000 7fa541600700 1 -- 172.21.15.131:0/1586436009 wait complete. 2024-05-08T22:00:09.552 INFO:teuthology.orchestra.run.smithi131.stderr:adjusted tunables profile to default 2024-05-08T22:00:09.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:09 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:09.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:09 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:09.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:09 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:09.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:09 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:09.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:09 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:00:09.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:09 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:09.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:09 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:00:09.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:09 smithi131 ceph-mon[37260]: Updating smithi157:/etc/ceph/ceph.conf 2024-05-08T22:00:09.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:09 smithi131 ceph-mon[37260]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:09.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:09 smithi131 ceph-mon[37260]: Updating smithi157:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:00:09.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:09 smithi131 ceph-mon[37260]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:00:09.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:09 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1586436009' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-08T22:00:09.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:09 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:09.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:09 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:09.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:09 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:09.933 INFO:tasks.cephadm:Adding mon.a on smithi131 2024-05-08T22:00:09.933 INFO:tasks.cephadm:Adding mon.b on smithi132 2024-05-08T22:00:09.933 INFO:tasks.cephadm:Adding mon.c on smithi157 2024-05-08T22:00:09.934 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi131:172.21.15.131=a;smithi132:172.21.15.132=b;smithi157:172.21.15.157=c' 2024-05-08T22:00:10.180 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:00:10.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:10 smithi131 ceph-mon[37260]: Updating smithi157:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T22:00:10.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:10 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1586436009' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-08T22:00:10.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:10 smithi131 ceph-mon[37260]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:00:11.130 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.127+0000 7fad0eeb7700 1 -- 172.21.15.157:0/1572349923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad08105800 msgr2=0x7fad08105be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:11.130 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.127+0000 7fad0eeb7700 1 --2- 172.21.15.157:0/1572349923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad08105800 0x7fad08105be0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7facfc009910 tx=0x7facfc009c20 comp rx=0 tx=0).stop 2024-05-08T22:00:11.130 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.127+0000 7fad0eeb7700 1 -- 172.21.15.157:0/1572349923 shutdown_connections 2024-05-08T22:00:11.130 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.127+0000 7fad0eeb7700 1 --2- 172.21.15.157:0/1572349923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad08105800 0x7fad08105be0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:11.130 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.127+0000 7fad0eeb7700 1 -- 172.21.15.157:0/1572349923 >> 172.21.15.157:0/1572349923 conn(0x7fad080fb330 msgr2=0x7fad080fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:11.131 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.127+0000 7fad0eeb7700 1 -- 172.21.15.157:0/1572349923 shutdown_connections 2024-05-08T22:00:11.131 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.127+0000 7fad0eeb7700 1 -- 172.21.15.157:0/1572349923 wait complete. 2024-05-08T22:00:11.131 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.128+0000 7fad0eeb7700 1 Processor -- start 2024-05-08T22:00:11.131 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.128+0000 7fad0eeb7700 1 -- start start 2024-05-08T22:00:11.131 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.128+0000 7fad0eeb7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad08105800 0x7fad08196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:11.131 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.128+0000 7fad0eeb7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad08196690 con 0x7fad08105800 2024-05-08T22:00:11.131 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.128+0000 7fad0cc53700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad08105800 0x7fad08196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:11.131 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.128+0000 7fad0cc53700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad08105800 0x7fad08196150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.157:38742/0 (socket says 172.21.15.157:38742) 2024-05-08T22:00:11.131 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.128+0000 7fad0cc53700 1 -- 172.21.15.157:0/1274050886 learned_addr learned my addr 172.21.15.157:0/1274050886 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-08T22:00:11.131 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.129+0000 7fad0cc53700 1 -- 172.21.15.157:0/1274050886 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facfc0095f0 con 0x7fad08105800 2024-05-08T22:00:11.131 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.129+0000 7fad0cc53700 1 --2- 172.21.15.157:0/1274050886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad08105800 0x7fad08196150 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7facfc004f40 tx=0x7facfc005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:11.131 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.129+0000 7fad05ffb700 1 -- 172.21.15.157:0/1274050886 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facfc01d070 con 0x7fad08105800 2024-05-08T22:00:11.131 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.129+0000 7fad0eeb7700 1 -- 172.21.15.157:0/1274050886 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad081968f0 con 0x7fad08105800 2024-05-08T22:00:11.132 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.129+0000 7fad05ffb700 1 -- 172.21.15.157:0/1274050886 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facfc00bbb0 con 0x7fad08105800 2024-05-08T22:00:11.132 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.129+0000 7fad05ffb700 1 -- 172.21.15.157:0/1274050886 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facfc003810 con 0x7fad08105800 2024-05-08T22:00:11.132 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.129+0000 7fad0eeb7700 1 -- 172.21.15.157:0/1274050886 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad08196d70 con 0x7fad08105800 2024-05-08T22:00:11.133 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.130+0000 7fad0eeb7700 1 -- 172.21.15.157:0/1274050886 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad08100f40 con 0x7fad08105800 2024-05-08T22:00:11.133 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.130+0000 7fad05ffb700 1 -- 172.21.15.157:0/1274050886 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7facfc00bd20 con 0x7fad08105800 2024-05-08T22:00:11.133 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.130+0000 7fad05ffb700 1 --2- 172.21.15.157:0/1274050886 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7facf80387a0 0x7facf803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:11.133 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.131+0000 7fad05ffb700 1 -- 172.21.15.157:0/1274050886 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7facfc04c3c0 con 0x7fad08105800 2024-05-08T22:00:11.133 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.131+0000 7fad07fff700 1 --2- 172.21.15.157:0/1274050886 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7facf80387a0 0x7facf803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:11.133 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.131+0000 7fad07fff700 1 --2- 172.21.15.157:0/1274050886 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7facf80387a0 0x7facf803ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7facf4006fd0 tx=0x7facf4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:11.137 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.136+0000 7fad05ffb700 1 -- 172.21.15.157:0/1274050886 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7facfc02ab60 con 0x7fad08105800 2024-05-08T22:00:11.342 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.341+0000 7fad0eeb7700 1 -- 172.21.15.157:0/1274050886 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi131:172.21.15.131=a;smithi132:172.21.15.132=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fad08109bc0 con 0x7facf80387a0 2024-05-08T22:00:11.350 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.349+0000 7fad05ffb700 1 -- 172.21.15.157:0/1274050886 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fad08109bc0 con 0x7facf80387a0 2024-05-08T22:00:11.350 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mon update... 2024-05-08T22:00:11.352 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.351+0000 7fad0eeb7700 1 -- 172.21.15.157:0/1274050886 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7facf80387a0 msgr2=0x7facf803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:11.352 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.351+0000 7fad0eeb7700 1 --2- 172.21.15.157:0/1274050886 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7facf80387a0 0x7facf803ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7facf4006fd0 tx=0x7facf4006e40 comp rx=0 tx=0).stop 2024-05-08T22:00:11.352 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.351+0000 7fad0eeb7700 1 -- 172.21.15.157:0/1274050886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad08105800 msgr2=0x7fad08196150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:11.352 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.351+0000 7fad0eeb7700 1 --2- 172.21.15.157:0/1274050886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad08105800 0x7fad08196150 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7facfc004f40 tx=0x7facfc005020 comp rx=0 tx=0).stop 2024-05-08T22:00:11.352 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.351+0000 7fad0eeb7700 1 -- 172.21.15.157:0/1274050886 shutdown_connections 2024-05-08T22:00:11.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.351+0000 7fad0eeb7700 1 --2- 172.21.15.157:0/1274050886 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7facf80387a0 0x7facf803ac60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:11.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.351+0000 7fad0eeb7700 1 --2- 172.21.15.157:0/1274050886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad08105800 0x7fad08196150 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:11.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.351+0000 7fad0eeb7700 1 -- 172.21.15.157:0/1274050886 >> 172.21.15.157:0/1274050886 conn(0x7fad080fb330 msgr2=0x7fad08068c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:11.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.351+0000 7fad0eeb7700 1 -- 172.21.15.157:0/1274050886 shutdown_connections 2024-05-08T22:00:11.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:11.351+0000 7fad0eeb7700 1 -- 172.21.15.157:0/1274050886 wait complete. 2024-05-08T22:00:11.741 DEBUG:teuthology.orchestra.run.smithi132:mon.b> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mon.b.service 2024-05-08T22:00:11.743 DEBUG:teuthology.orchestra.run.smithi157:mon.c> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mon.c.service 2024-05-08T22:00:11.745 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T22:00:11.745 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T22:00:11.774 INFO:journalctl@ceph.mon.c.smithi157.stdout:-- Logs begin at Wed 2024-05-08 21:35:00 UTC. -- 2024-05-08T22:00:11.780 INFO:journalctl@ceph.mon.b.smithi132.stdout:-- Logs begin at Wed 2024-05-08 21:34:20 UTC. -- 2024-05-08T22:00:11.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:11 smithi131 ceph-mon[37260]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:11.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:11 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:11.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:11 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:00:11.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:11 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:11.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:11 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:00:11.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:11 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:11.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:11 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:00:11.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:11 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:12.199 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:00:12.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:12 smithi131 ceph-mon[37260]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi131:172.21.15.131=a;smithi132:172.21.15.132=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:00:12.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:12 smithi131 ceph-mon[37260]: Saving service mon spec with placement smithi131:172.21.15.131=a;smithi132:172.21.15.132=b;smithi157:172.21.15.157=c;count:3 2024-05-08T22:00:12.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:12 smithi131 ceph-mon[37260]: Deploying daemon mon.c on smithi157 2024-05-08T22:00:13.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.691+0000 7f0a2c134700 1 -- 172.21.15.157:0/583841161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a24101a80 msgr2=0x7f0a24101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:13.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.691+0000 7f0a2c134700 1 --2- 172.21.15.157:0/583841161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a24101a80 0x7f0a24101e60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0a18009910 tx=0x7f0a18009c20 comp rx=0 tx=0).stop 2024-05-08T22:00:13.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.691+0000 7f0a2c134700 1 -- 172.21.15.157:0/583841161 shutdown_connections 2024-05-08T22:00:13.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.691+0000 7f0a2c134700 1 --2- 172.21.15.157:0/583841161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a24101a80 0x7f0a24101e60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:13.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.691+0000 7f0a2c134700 1 -- 172.21.15.157:0/583841161 >> 172.21.15.157:0/583841161 conn(0x7f0a240fb2b0 msgr2=0x7f0a240fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:13.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.691+0000 7f0a2c134700 1 -- 172.21.15.157:0/583841161 shutdown_connections 2024-05-08T22:00:13.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.691+0000 7f0a2c134700 1 -- 172.21.15.157:0/583841161 wait complete. 2024-05-08T22:00:13.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.691+0000 7f0a2c134700 1 Processor -- start 2024-05-08T22:00:13.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.691+0000 7f0a2c134700 1 -- start start 2024-05-08T22:00:13.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.692+0000 7f0a2c134700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a24101a80 0x7f0a241961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:13.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.692+0000 7f0a2c134700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a24196720 con 0x7f0a24101a80 2024-05-08T22:00:13.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.692+0000 7f0a29ed0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a24101a80 0x7f0a241961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:13.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.692+0000 7f0a29ed0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a24101a80 0x7f0a241961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.157:38756/0 (socket says 172.21.15.157:38756) 2024-05-08T22:00:13.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.692+0000 7f0a29ed0700 1 -- 172.21.15.157:0/2650096700 learned_addr learned my addr 172.21.15.157:0/2650096700 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-08T22:00:13.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.693+0000 7f0a29ed0700 1 -- 172.21.15.157:0/2650096700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a180095f0 con 0x7f0a24101a80 2024-05-08T22:00:13.695 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.693+0000 7f0a29ed0700 1 --2- 172.21.15.157:0/2650096700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a24101a80 0x7f0a241961e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0a18004f40 tx=0x7f0a18005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:13.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.693+0000 7f0a16ffd700 1 -- 172.21.15.157:0/2650096700 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a1801d070 con 0x7f0a24101a80 2024-05-08T22:00:13.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.693+0000 7f0a2c134700 1 -- 172.21.15.157:0/2650096700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a24196920 con 0x7f0a24101a80 2024-05-08T22:00:13.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.693+0000 7f0a16ffd700 1 -- 172.21.15.157:0/2650096700 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a18004370 con 0x7f0a24101a80 2024-05-08T22:00:13.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.693+0000 7f0a16ffd700 1 -- 172.21.15.157:0/2650096700 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a18003970 con 0x7f0a24101a80 2024-05-08T22:00:13.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.693+0000 7f0a2c134700 1 -- 172.21.15.157:0/2650096700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a24196d40 con 0x7f0a24101a80 2024-05-08T22:00:13.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.694+0000 7f0a2c134700 1 -- 172.21.15.157:0/2650096700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a24069210 con 0x7f0a24101a80 2024-05-08T22:00:13.698 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.694+0000 7f0a16ffd700 1 -- 172.21.15.157:0/2650096700 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f0a18003c30 con 0x7f0a24101a80 2024-05-08T22:00:13.698 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.694+0000 7f0a16ffd700 1 --2- 172.21.15.157:0/2650096700 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f0a100387a0 0x7f0a1003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:13.698 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.694+0000 7f0a16ffd700 1 -- 172.21.15.157:0/2650096700 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0a1804c440 con 0x7f0a24101a80 2024-05-08T22:00:13.698 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.694+0000 7f0a296cf700 1 --2- 172.21.15.157:0/2650096700 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f0a100387a0 0x7f0a1003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:13.698 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.695+0000 7f0a296cf700 1 --2- 172.21.15.157:0/2650096700 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f0a100387a0 0x7f0a1003ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0a20006fd0 tx=0x7f0a20006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:13.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:13.700+0000 7f0a16ffd700 1 -- 172.21.15.157:0/2650096700 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0a1802a3b0 con 0x7f0a24101a80 2024-05-08T22:00:13.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:13 smithi131 ceph-mon[37260]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:14.027 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:14.025+0000 7f0a2c134700 1 -- 172.21.15.157:0/2650096700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0a241975d0 con 0x7f0a24101a80 2024-05-08T22:00:14.027 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:14.026+0000 7f0a16ffd700 1 -- 172.21.15.157:0/2650096700 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f0a18026030 con 0x7f0a24101a80 2024-05-08T22:00:14.027 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-08T22:00:14.027 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","modified":"2024-05-08T21:58:39.180260Z","created":"2024-05-08T21:58:39.180260Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:3300","nonce":0},{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-08T22:00:14.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:14.029+0000 7f0a2c134700 1 -- 172.21.15.157:0/2650096700 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f0a100387a0 msgr2=0x7f0a1003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:14.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:14.029+0000 7f0a2c134700 1 --2- 172.21.15.157:0/2650096700 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f0a100387a0 0x7f0a1003ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0a20006fd0 tx=0x7f0a20006e40 comp rx=0 tx=0).stop 2024-05-08T22:00:14.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:14.029+0000 7f0a2c134700 1 -- 172.21.15.157:0/2650096700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a24101a80 msgr2=0x7f0a241961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:14.030 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:14.029+0000 7f0a2c134700 1 --2- 172.21.15.157:0/2650096700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a24101a80 0x7f0a241961e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0a18004f40 tx=0x7f0a18005020 comp rx=0 tx=0).stop 2024-05-08T22:00:14.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:14.029+0000 7f0a2c134700 1 -- 172.21.15.157:0/2650096700 shutdown_connections 2024-05-08T22:00:14.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:14.029+0000 7f0a2c134700 1 --2- 172.21.15.157:0/2650096700 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f0a100387a0 0x7f0a1003ac60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:14.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:14.029+0000 7f0a2c134700 1 --2- 172.21.15.157:0/2650096700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a24101a80 0x7f0a241961e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:14.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:14.029+0000 7f0a2c134700 1 -- 172.21.15.157:0/2650096700 >> 172.21.15.157:0/2650096700 conn(0x7f0a240fb2b0 msgr2=0x7f0a24104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:14.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:14.029+0000 7f0a2c134700 1 -- 172.21.15.157:0/2650096700 shutdown_connections 2024-05-08T22:00:14.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:14.029+0000 7f0a2c134700 1 -- 172.21.15.157:0/2650096700 wait complete. 2024-05-08T22:00:14.031 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-05-08T22:00:14.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:14 smithi131 ceph-mon[37260]: from='client.? 172.21.15.157:0/2650096700' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T22:00:15.028 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 systemd[1]: Started Ceph mon.c for 06f1a696-0d86-11ef-bc97-c7b262605968. 2024-05-08T22:00:15.028 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: RocksDB version: 7.9.2 2024-05-08T22:00:15.028 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Git sha 0 2024-05-08T22:00:15.028 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Compile date 2024-05-08 17:55:50 2024-05-08T22:00:15.028 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: DB SUMMARY 2024-05-08T22:00:15.028 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: DB Session ID: CH9A2BOEHG6DPN8TS48Z 2024-05-08T22:00:15.028 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: CURRENT file: CURRENT 2024-05-08T22:00:15.028 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: IDENTITY file: IDENTITY 2024-05-08T22:00:15.028 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-08T22:00:15.028 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-08T22:00:15.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-08T22:00:15.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.error_if_exists: 0 2024-05-08T22:00:15.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.create_if_missing: 0 2024-05-08T22:00:15.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.paranoid_checks: 1 2024-05-08T22:00:15.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-08T22:00:15.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-08T22:00:15.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-08T22:00:15.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.env: 0x558d7e1cc820 2024-05-08T22:00:15.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.fs: PosixFileSystem 2024-05-08T22:00:15.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.info_log: 0x558d80ae9520 2024-05-08T22:00:15.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_file_opening_threads: 16 2024-05-08T22:00:15.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.statistics: (nil) 2024-05-08T22:00:15.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.use_fsync: 0 2024-05-08T22:00:15.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_log_file_size: 0 2024-05-08T22:00:15.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-08T22:00:15.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-08T22:00:15.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.keep_log_file_num: 1000 2024-05-08T22:00:15.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.recycle_log_file_num: 0 2024-05-08T22:00:15.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.allow_fallocate: 1 2024-05-08T22:00:15.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.allow_mmap_reads: 0 2024-05-08T22:00:15.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.allow_mmap_writes: 0 2024-05-08T22:00:15.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.use_direct_reads: 0 2024-05-08T22:00:15.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-08T22:00:15.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.create_missing_column_families: 0 2024-05-08T22:00:15.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.db_log_dir: 2024-05-08T22:00:15.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.wal_dir: 2024-05-08T22:00:15.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-08T22:00:15.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-08T22:00:15.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-08T22:00:15.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-08T22:00:15.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-08T22:00:15.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-08T22:00:15.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.advise_random_on_open: 1 2024-05-08T22:00:15.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.db_write_buffer_size: 0 2024-05-08T22:00:15.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.write_buffer_manager: 0x558d7fd7e320 2024-05-08T22:00:15.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-08T22:00:15.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-08T22:00:15.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-08T22:00:15.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.rate_limiter: (nil) 2024-05-08T22:00:15.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-08T22:00:15.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.wal_recovery_mode: 2 2024-05-08T22:00:15.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.enable_thread_tracking: 0 2024-05-08T22:00:15.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.enable_pipelined_write: 0 2024-05-08T22:00:15.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.unordered_write: 0 2024-05-08T22:00:15.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-08T22:00:15.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-08T22:00:15.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-08T22:00:15.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-08T22:00:15.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.row_cache: None 2024-05-08T22:00:15.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.wal_filter: None 2024-05-08T22:00:15.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-08T22:00:15.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.allow_ingest_behind: 0 2024-05-08T22:00:15.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.two_write_queues: 0 2024-05-08T22:00:15.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.manual_wal_flush: 0 2024-05-08T22:00:15.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.wal_compression: 0 2024-05-08T22:00:15.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.atomic_flush: 0 2024-05-08T22:00:15.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-08T22:00:15.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-08T22:00:15.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-08T22:00:15.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.log_readahead_size: 0 2024-05-08T22:00:15.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-08T22:00:15.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.best_efforts_recovery: 0 2024-05-08T22:00:15.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-08T22:00:15.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-08T22:00:15.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.allow_data_in_errors: 0 2024-05-08T22:00:15.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.db_host_id: __hostname__ 2024-05-08T22:00:15.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-08T22:00:15.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_background_jobs: 2 2024-05-08T22:00:15.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_background_compactions: -1 2024-05-08T22:00:15.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_subcompactions: 1 2024-05-08T22:00:15.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-08T22:00:15.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-08T22:00:15.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-08T22:00:15.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_total_wal_size: 0 2024-05-08T22:00:15.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-08T22:00:15.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-08T22:00:15.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-08T22:00:15.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-08T22:00:15.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_open_files: -1 2024-05-08T22:00:15.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.bytes_per_sync: 0 2024-05-08T22:00:15.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-08T22:00:15.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-08T22:00:15.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.compaction_readahead_size: 0 2024-05-08T22:00:15.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_background_flushes: -1 2024-05-08T22:00:15.035 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Compression algorithms supported: 2024-05-08T22:00:15.035 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-08T22:00:15.035 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: kZSTD supported: 0 2024-05-08T22:00:15.035 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: kXpressCompression supported: 0 2024-05-08T22:00:15.035 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: kLZ4HCCompression supported: 1 2024-05-08T22:00:15.035 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: kZlibCompression supported: 1 2024-05-08T22:00:15.035 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: kSnappyCompression supported: 1 2024-05-08T22:00:15.035 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: kLZ4Compression supported: 1 2024-05-08T22:00:15.035 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: kBZip2Compression supported: 0 2024-05-08T22:00:15.035 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-08T22:00:15.035 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-08T22:00:15.036 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-08T22:00:15.036 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-08T22:00:15.036 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-08T22:00:15.036 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.merge_operator: 2024-05-08T22:00:15.036 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.compaction_filter: None 2024-05-08T22:00:15.036 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.compaction_filter_factory: None 2024-05-08T22:00:15.036 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.sst_partitioner_factory: None 2024-05-08T22:00:15.036 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-08T22:00:15.036 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-08T22:00:15.036 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558d80ae9620) 2024-05-08T22:00:15.036 INFO:journalctl@ceph.mon.c.smithi157.stdout: cache_index_and_filter_blocks: 1 2024-05-08T22:00:15.036 INFO:journalctl@ceph.mon.c.smithi157.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-08T22:00:15.036 INFO:journalctl@ceph.mon.c.smithi157.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-08T22:00:15.037 INFO:journalctl@ceph.mon.c.smithi157.stdout: pin_top_level_index_and_filter: 1 2024-05-08T22:00:15.037 INFO:journalctl@ceph.mon.c.smithi157.stdout: index_type: 0 2024-05-08T22:00:15.037 INFO:journalctl@ceph.mon.c.smithi157.stdout: data_block_index_type: 0 2024-05-08T22:00:15.037 INFO:journalctl@ceph.mon.c.smithi157.stdout: index_shortening: 1 2024-05-08T22:00:15.037 INFO:journalctl@ceph.mon.c.smithi157.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-08T22:00:15.037 INFO:journalctl@ceph.mon.c.smithi157.stdout: checksum: 4 2024-05-08T22:00:15.037 INFO:journalctl@ceph.mon.c.smithi157.stdout: no_block_cache: 0 2024-05-08T22:00:15.037 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_cache: 0x558d7fe011f0 2024-05-08T22:00:15.037 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_cache_name: BinnedLRUCache 2024-05-08T22:00:15.037 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_cache_options: 2024-05-08T22:00:15.037 INFO:journalctl@ceph.mon.c.smithi157.stdout: capacity : 536870912 2024-05-08T22:00:15.037 INFO:journalctl@ceph.mon.c.smithi157.stdout: num_shard_bits : 4 2024-05-08T22:00:15.037 INFO:journalctl@ceph.mon.c.smithi157.stdout: strict_capacity_limit : 0 2024-05-08T22:00:15.037 INFO:journalctl@ceph.mon.c.smithi157.stdout: high_pri_pool_ratio: 0.000 2024-05-08T22:00:15.037 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_cache_compressed: (nil) 2024-05-08T22:00:15.038 INFO:journalctl@ceph.mon.c.smithi157.stdout: persistent_cache: (nil) 2024-05-08T22:00:15.038 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_size: 4096 2024-05-08T22:00:15.038 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_size_deviation: 10 2024-05-08T22:00:15.038 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_restart_interval: 16 2024-05-08T22:00:15.038 INFO:journalctl@ceph.mon.c.smithi157.stdout: index_block_restart_interval: 1 2024-05-08T22:00:15.038 INFO:journalctl@ceph.mon.c.smithi157.stdout: metadata_block_size: 4096 2024-05-08T22:00:15.038 INFO:journalctl@ceph.mon.c.smithi157.stdout: partition_filters: 0 2024-05-08T22:00:15.038 INFO:journalctl@ceph.mon.c.smithi157.stdout: use_delta_encoding: 1 2024-05-08T22:00:15.038 INFO:journalctl@ceph.mon.c.smithi157.stdout: filter_policy: bloomfilter 2024-05-08T22:00:15.038 INFO:journalctl@ceph.mon.c.smithi157.stdout: whole_key_filtering: 1 2024-05-08T22:00:15.038 INFO:journalctl@ceph.mon.c.smithi157.stdout: verify_compression: 0 2024-05-08T22:00:15.038 INFO:journalctl@ceph.mon.c.smithi157.stdout: read_amp_bytes_per_bit: 0 2024-05-08T22:00:15.038 INFO:journalctl@ceph.mon.c.smithi157.stdout: format_version: 5 2024-05-08T22:00:15.039 INFO:journalctl@ceph.mon.c.smithi157.stdout: enable_index_compression: 1 2024-05-08T22:00:15.039 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_align: 0 2024-05-08T22:00:15.039 INFO:journalctl@ceph.mon.c.smithi157.stdout: max_auto_readahead_size: 262144 2024-05-08T22:00:15.039 INFO:journalctl@ceph.mon.c.smithi157.stdout: prepopulate_block_cache: 0 2024-05-08T22:00:15.039 INFO:journalctl@ceph.mon.c.smithi157.stdout: initial_auto_readahead_size: 8192 2024-05-08T22:00:15.039 INFO:journalctl@ceph.mon.c.smithi157.stdout: num_file_reads_for_auto_readahead: 2 2024-05-08T22:00:15.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.write_buffer_size: 33554432 2024-05-08T22:00:15.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_write_buffer_number: 2 2024-05-08T22:00:15.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.compression: NoCompression 2024-05-08T22:00:15.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.bottommost_compression: Disabled 2024-05-08T22:00:15.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.prefix_extractor: nullptr 2024-05-08T22:00:15.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-08T22:00:15.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.num_levels: 7 2024-05-08T22:00:15.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-08T22:00:15.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-08T22:00:15.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-08T22:00:15.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-08T22:00:15.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-08T22:00:15.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-08T22:00:15.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-08T22:00:15.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-08T22:00:15.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-08T22:00:15.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-08T22:00:15.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-08T22:00:15.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-08T22:00:15.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-08T22:00:15.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.compression_opts.level: 32767 2024-05-08T22:00:15.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.compression_opts.strategy: 0 2024-05-08T22:00:15.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-08T22:00:15.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-08T22:00:15.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-08T22:00:15.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-08T22:00:15.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.compression_opts.enabled: false 2024-05-08T22:00:15.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-08T22:00:15.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-08T22:00:15.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-08T22:00:15.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-08T22:00:15.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.target_file_size_base: 67108864 2024-05-08T22:00:15.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-08T22:00:15.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-08T22:00:15.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-08T22:00:15.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-08T22:00:15.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-08T22:00:15.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-08T22:00:15.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-08T22:00:15.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-08T22:00:15.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-08T22:00:15.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-08T22:00:15.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-08T22:00:15.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-08T22:00:15.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-08T22:00:15.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-08T22:00:15.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.arena_block_size: 1048576 2024-05-08T22:00:15.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-08T22:00:15.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-08T22:00:15.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.disable_auto_compactions: 0 2024-05-08T22:00:15.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-08T22:00:15.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-08T22:00:15.043 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-08T22:00:15.043 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-08T22:00:15.043 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-08T22:00:15.043 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-08T22:00:15.043 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-08T22:00:15.043 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-08T22:00:15.043 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-08T22:00:15.043 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-08T22:00:15.043 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.table_properties_collectors: 2024-05-08T22:00:15.043 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.inplace_update_support: 0 2024-05-08T22:00:15.043 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-08T22:00:15.043 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-08T22:00:15.044 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-08T22:00:15.044 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-08T22:00:15.044 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.bloom_locality: 0 2024-05-08T22:00:15.044 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.max_successive_merges: 0 2024-05-08T22:00:15.044 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-08T22:00:15.044 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.paranoid_file_checks: 0 2024-05-08T22:00:15.044 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.force_consistency_checks: 1 2024-05-08T22:00:15.044 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.report_bg_io_stats: 0 2024-05-08T22:00:15.044 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.ttl: 2592000 2024-05-08T22:00:15.044 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-08T22:00:15.044 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-08T22:00:15.044 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-08T22:00:15.045 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.enable_blob_files: false 2024-05-08T22:00:15.045 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.min_blob_size: 0 2024-05-08T22:00:15.045 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.blob_file_size: 268435456 2024-05-08T22:00:15.045 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-08T22:00:15.045 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-08T22:00:15.045 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-08T22:00:15.045 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-08T22:00:15.045 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-08T22:00:15.045 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.blob_file_starting_level: 0 2024-05-08T22:00:15.045 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-08T22:00:15.045 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-08T22:00:15.045 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-08T22:00:15.045 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: fc21975e-2f05-4f98-b063-2dc3fa725956 2024-05-08T22:00:15.045 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715205614652042, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-08T22:00:15.046 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-08T22:00:15.046 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715205614652636, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715205614, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "fc21975e-2f05-4f98-b063-2dc3fa725956", "db_session_id": "CH9A2BOEHG6DPN8TS48Z", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-08T22:00:15.046 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715205614652724, "job": 1, "event": "recovery_finished"} 2024-05-08T22:00:15.046 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-08T22:00:15.046 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T22:00:15.046 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558d7fe9e000 2024-05-08T22:00:15.046 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: DB pointer 0x558d7fe8c000 2024-05-08T22:00:15.046 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T22:00:15.046 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T22:00:15.046 INFO:journalctl@ceph.mon.c.smithi157.stdout: ** DB Stats ** 2024-05-08T22:00:15.046 INFO:journalctl@ceph.mon.c.smithi157.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T22:00:15.046 INFO:journalctl@ceph.mon.c.smithi157.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T22:00:15.046 INFO:journalctl@ceph.mon.c.smithi157.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T22:00:15.047 INFO:journalctl@ceph.mon.c.smithi157.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T22:00:15.047 INFO:journalctl@ceph.mon.c.smithi157.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T22:00:15.047 INFO:journalctl@ceph.mon.c.smithi157.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T22:00:15.047 INFO:journalctl@ceph.mon.c.smithi157.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T22:00:15.047 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-05-08T22:00:15.047 INFO:journalctl@ceph.mon.c.smithi157.stdout: ** Compaction Stats [default] ** 2024-05-08T22:00:15.047 INFO:journalctl@ceph.mon.c.smithi157.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T22:00:15.047 INFO:journalctl@ceph.mon.c.smithi157.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T22:00:15.047 INFO:journalctl@ceph.mon.c.smithi157.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T22:00:15.047 INFO:journalctl@ceph.mon.c.smithi157.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T22:00:15.047 INFO:journalctl@ceph.mon.c.smithi157.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T22:00:15.047 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-05-08T22:00:15.047 INFO:journalctl@ceph.mon.c.smithi157.stdout: ** Compaction Stats [default] ** 2024-05-08T22:00:15.048 INFO:journalctl@ceph.mon.c.smithi157.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T22:00:15.048 INFO:journalctl@ceph.mon.c.smithi157.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T22:00:15.048 INFO:journalctl@ceph.mon.c.smithi157.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T22:00:15.048 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-05-08T22:00:15.048 INFO:journalctl@ceph.mon.c.smithi157.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T22:00:15.048 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-05-08T22:00:15.048 INFO:journalctl@ceph.mon.c.smithi157.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T22:00:15.048 INFO:journalctl@ceph.mon.c.smithi157.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T22:00:15.048 INFO:journalctl@ceph.mon.c.smithi157.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T22:00:15.048 INFO:journalctl@ceph.mon.c.smithi157.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T22:00:15.048 INFO:journalctl@ceph.mon.c.smithi157.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T22:00:15.048 INFO:journalctl@ceph.mon.c.smithi157.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T22:00:15.048 INFO:journalctl@ceph.mon.c.smithi157.stdout: Cumulative compaction: 0.00 GB write, 0.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T22:00:15.048 INFO:journalctl@ceph.mon.c.smithi157.stdout: Interval compaction: 0.00 GB write, 0.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T22:00:15.048 INFO:journalctl@ceph.mon.c.smithi157.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T22:00:15.049 INFO:journalctl@ceph.mon.c.smithi157.stdout: Block cache BinnedLRUCache@0x558d7fe011f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-08T22:00:15.049 INFO:journalctl@ceph.mon.c.smithi157.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-08T22:00:15.049 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-05-08T22:00:15.049 INFO:journalctl@ceph.mon.c.smithi157.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T22:00:15.049 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-08T22:00:15.049 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: using public_addr v2:172.21.15.157:0/0 -> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] 2024-05-08T22:00:15.049 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: starting mon.c rank -1 at public addrs [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] at bind addrs [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T22:00:15.049 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: mon.c@-1(???) e0 preinit fsid 06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T22:00:15.049 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: mon.c@-1(synchronizing).mds e1 new map 2024-05-08T22:00:15.049 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-08T22:00:15.049 INFO:journalctl@ceph.mon.c.smithi157.stdout: e1 2024-05-08T22:00:15.049 INFO:journalctl@ceph.mon.c.smithi157.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T22:00:15.049 INFO:journalctl@ceph.mon.c.smithi157.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T22:00:15.049 INFO:journalctl@ceph.mon.c.smithi157.stdout: legacy client fscid: -1 2024-05-08T22:00:15.049 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-05-08T22:00:15.050 INFO:journalctl@ceph.mon.c.smithi157.stdout: No filesystems configured 2024-05-08T22:00:15.050 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-08T22:00:15.050 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-08T22:00:15.050 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-08T22:00:15.050 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-08T22:00:15.050 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-08T22:00:15.050 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-08T22:00:15.050 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T22:00:15.050 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T22:00:15.050 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T22:00:15.050 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T22:00:15.050 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: Deploying cephadm binary to smithi157 2024-05-08T22:00:15.050 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:15.050 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:00:15.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:15.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: Added host smithi157 2024-05-08T22:00:15.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:15.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:15.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:15.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:15.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:00:15.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:15.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:15.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:15.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:15.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:00:15.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:15.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:00:15.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: Updating smithi157:/etc/ceph/ceph.conf 2024-05-08T22:00:15.051 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:15.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: Updating smithi157:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:00:15.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:00:15.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1586436009' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-08T22:00:15.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:15.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:15.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:15.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: Updating smithi157:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T22:00:15.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1586436009' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-08T22:00:15.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:00:15.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:15.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:15.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:00:15.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:15.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:00:15.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:15.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:00:15.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:15.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi131:172.21.15.131=a;smithi132:172.21.15.132=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:00:15.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: Saving service mon spec with placement smithi131:172.21.15.131=a;smithi132:172.21.15.132=b;smithi157:172.21.15.157=c;count:3 2024-05-08T22:00:15.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: Deploying daemon mon.c on smithi157 2024-05-08T22:00:15.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:15.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: from='client.? 172.21.15.157:0/2650096700' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T22:00:15.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:14 smithi157 ceph-mon[77746]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-08T22:00:15.427 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T22:00:15.427 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T22:00:15.652 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:00:19.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.885+0000 7f85f7af4700 1 -- 172.21.15.157:0/3934516008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85dc0035c0 msgr2=0x7f85dc005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:19.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.885+0000 7f85f7af4700 1 --2- 172.21.15.157:0/3934516008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85dc0035c0 0x7f85dc005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f85f00ff800 tx=0x7f85ec013e90 comp rx=0 tx=0).stop 2024-05-08T22:00:19.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.885+0000 7f85f7af4700 1 -- 172.21.15.157:0/3934516008 shutdown_connections 2024-05-08T22:00:19.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.885+0000 7f85f7af4700 1 --2- 172.21.15.157:0/3934516008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85dc0035c0 0x7f85dc005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:19.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.885+0000 7f85f7af4700 1 -- 172.21.15.157:0/3934516008 >> 172.21.15.157:0/3934516008 conn(0x7f85f00fb400 msgr2=0x7f85f00fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:19.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.885+0000 7f85f7af4700 1 -- 172.21.15.157:0/3934516008 shutdown_connections 2024-05-08T22:00:19.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.885+0000 7f85f7af4700 1 -- 172.21.15.157:0/3934516008 wait complete. 2024-05-08T22:00:19.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.886+0000 7f85f7af4700 1 Processor -- start 2024-05-08T22:00:19.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.886+0000 7f85f7af4700 1 -- start start 2024-05-08T22:00:19.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.886+0000 7f85f7af4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85dc0035c0 0x7f85f0196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:19.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.886+0000 7f85f7af4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f01967d0 0x7f85f019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:19.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.886+0000 7f85f7af4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f0196d60 con 0x7f85dc0035c0 2024-05-08T22:00:19.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.886+0000 7f85f7af4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f0196ed0 con 0x7f85f01967d0 2024-05-08T22:00:19.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.886+0000 7f85f508f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f01967d0 0x7f85f019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:19.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.887+0000 7f85f508f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f01967d0 0x7f85f019ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54630/0 (socket says 172.21.15.157:54630) 2024-05-08T22:00:19.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.887+0000 7f85f508f700 1 -- 172.21.15.157:0/3020963244 learned_addr learned my addr 172.21.15.157:0/3020963244 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-08T22:00:19.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.887+0000 7f85f5890700 1 --2- 172.21.15.157:0/3020963244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85dc0035c0 0x7f85f0196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:19.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.887+0000 7f85f508f700 1 -- 172.21.15.157:0/3020963244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f01967d0 msgr2=0x7f85f019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-08T22:00:19.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.887+0000 7f85f508f700 1 -- 172.21.15.157:0/3020963244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f01967d0 msgr2=0x7f85f019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T22:00:19.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.887+0000 7f85f508f700 1 --2- 172.21.15.157:0/3020963244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f01967d0 0x7f85f019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:00:19.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.887+0000 7f85f508f700 1 --2- 172.21.15.157:0/3020963244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f01967d0 0x7f85f019ab80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:00:19.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.887+0000 7f85f5890700 1 -- 172.21.15.157:0/3020963244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f01967d0 msgr2=0x7f85f019ab80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:00:19.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.887+0000 7f85f5890700 1 --2- 172.21.15.157:0/3020963244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f01967d0 0x7f85f019ab80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:19.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.887+0000 7f85f5890700 1 -- 172.21.15.157:0/3020963244 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ec0138e0 con 0x7f85dc0035c0 2024-05-08T22:00:19.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.887+0000 7f85f5890700 1 --2- 172.21.15.157:0/3020963244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85dc0035c0 0x7f85f0196290 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f85ec01c000 tx=0x7f85ec005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:19.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.887+0000 7f85e6ffd700 1 -- 172.21.15.157:0/3020963244 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f85ec018070 con 0x7f85dc0035c0 2024-05-08T22:00:19.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.887+0000 7f85e6ffd700 1 -- 172.21.15.157:0/3020963244 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85ec00fcd0 con 0x7f85dc0035c0 2024-05-08T22:00:19.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.888+0000 7f85f7af4700 1 -- 172.21.15.157:0/3020963244 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f019b120 con 0x7f85dc0035c0 2024-05-08T22:00:19.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.888+0000 7f85e6ffd700 1 -- 172.21.15.157:0/3020963244 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f85ec003ae0 con 0x7f85dc0035c0 2024-05-08T22:00:19.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.888+0000 7f85f7af4700 1 -- 172.21.15.157:0/3020963244 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f019b670 con 0x7f85dc0035c0 2024-05-08T22:00:19.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.888+0000 7f85e6ffd700 1 -- 172.21.15.157:0/3020963244 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f85ec003c60 con 0x7f85dc0035c0 2024-05-08T22:00:19.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.889+0000 7f85e6ffd700 1 --2- 172.21.15.157:0/3020963244 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f85d40385e0 0x7f85d403aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:19.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.889+0000 7f85e6ffd700 1 -- 172.21.15.157:0/3020963244 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f85ec04e070 con 0x7f85dc0035c0 2024-05-08T22:00:19.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.889+0000 7f85f7af4700 1 -- 172.21.15.157:0/3020963244 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85f018ffd0 con 0x7f85dc0035c0 2024-05-08T22:00:19.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.889+0000 7f85f508f700 1 --2- 172.21.15.157:0/3020963244 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f85d40385e0 0x7f85d403aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:19.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.890+0000 7f85f508f700 1 --2- 172.21.15.157:0/3020963244 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f85d40385e0 0x7f85d403aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009180 tx=0x7f85e0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:19.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:19.895+0000 7f85e6ffd700 1 -- 172.21.15.157:0/3020963244 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f85ec01c410 con 0x7f85dc0035c0 2024-05-08T22:00:20.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:19 smithi157 ceph-mon[77746]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:20.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:00:20.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:00:20.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:19 smithi157 ceph-mon[77746]: mon.a calling monitor election 2024-05-08T22:00:20.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:00:20.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:19 smithi157 ceph-mon[77746]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:20.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:00:20.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:19 smithi157 ceph-mon[77746]: mon.c calling monitor election 2024-05-08T22:00:20.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:00:20.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:19 smithi157 ceph-mon[77746]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:20.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:00:20.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:00:20.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:19 smithi157 ceph-mon[77746]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T22:00:20.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:19 smithi157 ceph-mon[77746]: monmap e2: 2 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T22:00:20.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:19 smithi157 ceph-mon[77746]: fsmap 2024-05-08T22:00:20.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:19 smithi157 ceph-mon[77746]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:00:20.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:19 smithi157 ceph-mon[77746]: mgrmap e13: x(active, since 39s) 2024-05-08T22:00:20.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:19 smithi157 ceph-mon[77746]: overall HEALTH_OK 2024-05-08T22:00:20.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:20.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:20.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:20.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:00:20.028 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:19 smithi131 ceph-mon[37260]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:00:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:00:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:19 smithi131 ceph-mon[37260]: mon.a calling monitor election 2024-05-08T22:00:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:00:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:19 smithi131 ceph-mon[37260]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:00:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:19 smithi131 ceph-mon[37260]: mon.c calling monitor election 2024-05-08T22:00:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:00:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:19 smithi131 ceph-mon[37260]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:00:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:00:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:19 smithi131 ceph-mon[37260]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T22:00:20.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:19 smithi131 ceph-mon[37260]: monmap e2: 2 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T22:00:20.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:19 smithi131 ceph-mon[37260]: fsmap 2024-05-08T22:00:20.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:19 smithi131 ceph-mon[37260]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:00:20.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:19 smithi131 ceph-mon[37260]: mgrmap e13: x(active, since 39s) 2024-05-08T22:00:20.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:19 smithi131 ceph-mon[37260]: overall HEALTH_OK 2024-05-08T22:00:20.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:20.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:20.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:20.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:00:20.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:20.170 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:20.169+0000 7f85f7af4700 1 -- 172.21.15.157:0/3020963244 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f85f019b940 con 0x7f85dc0035c0 2024-05-08T22:00:20.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:20.170+0000 7f85e6ffd700 1 -- 172.21.15.157:0/3020963244 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1010 (secure 0 0 0) 0x7f85ec02d3b0 con 0x7f85dc0035c0 2024-05-08T22:00:20.171 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-08T22:00:20.171 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":2,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","modified":"2024-05-08T22:00:14.678809Z","created":"2024-05-08T21:58:39.180260Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:3300","nonce":0},{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-08T22:00:20.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:20.172+0000 7f85f7af4700 1 -- 172.21.15.157:0/3020963244 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f85d40385e0 msgr2=0x7f85d403aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:20.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:20.172+0000 7f85f7af4700 1 --2- 172.21.15.157:0/3020963244 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f85d40385e0 0x7f85d403aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009180 tx=0x7f85e0008040 comp rx=0 tx=0).stop 2024-05-08T22:00:20.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:20.172+0000 7f85f7af4700 1 -- 172.21.15.157:0/3020963244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85dc0035c0 msgr2=0x7f85f0196290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:20.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:20.172+0000 7f85f7af4700 1 --2- 172.21.15.157:0/3020963244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85dc0035c0 0x7f85f0196290 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f85ec01c000 tx=0x7f85ec005050 comp rx=0 tx=0).stop 2024-05-08T22:00:20.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:20.172+0000 7f85f7af4700 1 -- 172.21.15.157:0/3020963244 shutdown_connections 2024-05-08T22:00:20.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:20.172+0000 7f85f7af4700 1 --2- 172.21.15.157:0/3020963244 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f85d40385e0 0x7f85d403aaa0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:20.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:20.172+0000 7f85f7af4700 1 --2- 172.21.15.157:0/3020963244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85dc0035c0 0x7f85f0196290 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:20.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:20.172+0000 7f85f7af4700 1 --2- 172.21.15.157:0/3020963244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f01967d0 0x7f85f019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:20.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:20.172+0000 7f85f7af4700 1 -- 172.21.15.157:0/3020963244 >> 172.21.15.157:0/3020963244 conn(0x7f85f00fb400 msgr2=0x7f85f0069730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:20.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:20.172+0000 7f85f7af4700 1 -- 172.21.15.157:0/3020963244 shutdown_connections 2024-05-08T22:00:20.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:20.172+0000 7f85f7af4700 1 -- 172.21.15.157:0/3020963244 wait complete. 2024-05-08T22:00:20.174 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 2 2024-05-08T22:00:21.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:20 smithi157 ceph-mon[77746]: Deploying daemon mon.b on smithi132 2024-05-08T22:00:21.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:20 smithi157 ceph-mon[77746]: from='client.? 172.21.15.157:0/3020963244' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T22:00:21.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:20 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:00:21.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:20 smithi131 ceph-mon[37260]: Deploying daemon mon.b on smithi132 2024-05-08T22:00:21.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:20 smithi131 ceph-mon[37260]: from='client.? 172.21.15.157:0/3020963244' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T22:00:21.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:20 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:00:21.594 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T22:00:21.595 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T22:00:21.826 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:00:21.913 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:21 smithi157 ceph-mon[77746]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:22.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:21 smithi131 ceph-mon[37260]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:22.141 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:00:21 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:00:21.680+0000 7f9dc8000700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-08T22:00:22.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:22 smithi132 bash[78142]: 3a5e4aecd9be241b3f095b5cb26920acb174e16e5314b522cd731536d50276ab 2024-05-08T22:00:22.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:22 smithi132 systemd[1]: Started Ceph mon.b for 06f1a696-0d86-11ef-bc97-c7b262605968. 2024-05-08T22:00:27.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:27 smithi157 ceph-mon[77746]: mon.c calling monitor election 2024-05-08T22:00:27.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:27 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:00:27.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:27 smithi157 ceph-mon[77746]: mon.a calling monitor election 2024-05-08T22:00:27.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:27 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:00:27.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:27 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:00:27.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:27 smithi157 ceph-mon[77746]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:27.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:27 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:00:27.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:27 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:00:27.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:27 smithi157 ceph-mon[77746]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:27.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:27 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:00:27.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:27 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:00:27.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:27 smithi157 ceph-mon[77746]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:27.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:27 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:00:27.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:27 smithi157 ceph-mon[77746]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T22:00:27.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:27 smithi157 ceph-mon[77746]: monmap e3: 3 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],b=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T22:00:27.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:27 smithi157 ceph-mon[77746]: fsmap 2024-05-08T22:00:27.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:27 smithi157 ceph-mon[77746]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:00:27.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:27 smithi157 ceph-mon[77746]: mgrmap e13: x(active, since 46s) 2024-05-08T22:00:27.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:27 smithi157 ceph-mon[77746]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-05-08T22:00:27.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:27 smithi157 ceph-mon[77746]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-05-08T22:00:27.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:27 smithi157 ceph-mon[77746]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-05-08T22:00:27.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:27 smithi157 ceph-mon[77746]: mon.b (rank 2) addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] is down (out of quorum) 2024-05-08T22:00:27.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:27 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:27.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:27 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:27.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:27 smithi131 ceph-mon[37260]: mon.c calling monitor election 2024-05-08T22:00:27.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:27 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:00:27.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:27 smithi131 ceph-mon[37260]: mon.a calling monitor election 2024-05-08T22:00:27.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:27 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:00:27.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:27 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:00:27.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:27 smithi131 ceph-mon[37260]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:27.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:27 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:00:27.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:27 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:00:27.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:27 smithi131 ceph-mon[37260]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:27.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:27 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:00:27.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:27 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:00:27.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:27 smithi131 ceph-mon[37260]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:27.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:27 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:00:27.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:27 smithi131 ceph-mon[37260]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T22:00:27.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:27 smithi131 ceph-mon[37260]: monmap e3: 3 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],b=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T22:00:27.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:27 smithi131 ceph-mon[37260]: fsmap 2024-05-08T22:00:27.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:27 smithi131 ceph-mon[37260]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:00:27.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:27 smithi131 ceph-mon[37260]: mgrmap e13: x(active, since 46s) 2024-05-08T22:00:27.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:27 smithi131 ceph-mon[37260]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-05-08T22:00:27.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:27 smithi131 ceph-mon[37260]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-05-08T22:00:27.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:27 smithi131 ceph-mon[37260]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-05-08T22:00:27.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:27 smithi131 ceph-mon[37260]: mon.b (rank 2) addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] is down (out of quorum) 2024-05-08T22:00:27.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:27 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:27.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:27 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:28.717 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.716+0000 7f0e054e3700 1 -- 172.21.15.157:0/592302656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dec0062a0 msgr2=0x7f0dec008710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:28.717 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.716+0000 7f0e054e3700 1 --2- 172.21.15.157:0/592302656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dec0062a0 0x7f0dec008710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0e00101ab0 tx=0x7f0de800fe90 comp rx=0 tx=0).stop 2024-05-08T22:00:28.717 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.716+0000 7f0e054e3700 1 -- 172.21.15.157:0/592302656 shutdown_connections 2024-05-08T22:00:28.717 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.716+0000 7f0e054e3700 1 --2- 172.21.15.157:0/592302656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dec0062a0 0x7f0dec008710 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:28.718 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.716+0000 7f0e054e3700 1 -- 172.21.15.157:0/592302656 >> 172.21.15.157:0/592302656 conn(0x7f0e000fc500 msgr2=0x7f0e000fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:28.718 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.716+0000 7f0e054e3700 1 -- 172.21.15.157:0/592302656 shutdown_connections 2024-05-08T22:00:28.718 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.716+0000 7f0e054e3700 1 -- 172.21.15.157:0/592302656 wait complete. 2024-05-08T22:00:28.720 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.717+0000 7f0e054e3700 1 Processor -- start 2024-05-08T22:00:28.720 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.717+0000 7f0e054e3700 1 -- start start 2024-05-08T22:00:28.720 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.717+0000 7f0e054e3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dec0062a0 0x7f0e00196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:28.720 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.717+0000 7f0e054e3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e00196ac0 0x7f0e0019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:28.720 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.717+0000 7f0e054e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e0019b360 0x7f0e0019b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:28.720 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.717+0000 7f0e054e3700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e001971b0 con 0x7f0e00196ac0 2024-05-08T22:00:28.720 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.717+0000 7f0e054e3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e001a30d0 con 0x7f0e0019b360 2024-05-08T22:00:28.720 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.717+0000 7f0e054e3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e001a3210 con 0x7f0dec0062a0 2024-05-08T22:00:28.721 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.717+0000 7f0dfeffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dec0062a0 0x7f0e00196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:28.721 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.717+0000 7f0dfeffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dec0062a0 0x7f0e00196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54640/0 (socket says 172.21.15.157:54640) 2024-05-08T22:00:28.721 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.717+0000 7f0dfeffd700 1 -- 172.21.15.157:0/861108191 learned_addr learned my addr 172.21.15.157:0/861108191 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-08T22:00:28.721 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.718+0000 7f0dfe7fc700 1 --2- 172.21.15.157:0/861108191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e00196ac0 0x7f0e0019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:28.721 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.718+0000 7f0dfeffd700 1 -- 172.21.15.157:0/861108191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dec0062a0 msgr2=0x7f0e00196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-08T22:00:28.721 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.718+0000 7f0dfeffd700 1 -- 172.21.15.157:0/861108191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dec0062a0 msgr2=0x7f0e00196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T22:00:28.721 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.718+0000 7f0dfeffd700 1 --2- 172.21.15.157:0/861108191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dec0062a0 0x7f0e00196580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:00:28.721 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.718+0000 7f0dff7fe700 1 --2- 172.21.15.157:0/861108191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e0019b360 0x7f0e0019b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:28.721 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.718+0000 7f0dfeffd700 1 --2- 172.21.15.157:0/861108191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dec0062a0 0x7f0e00196580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:00:28.721 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.718+0000 7f0dff7fe700 1 -- 172.21.15.157:0/861108191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e0019b360 msgr2=0x7f0e0019b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-08T22:00:28.721 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.718+0000 7f0dff7fe700 1 -- 172.21.15.157:0/861108191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e0019b360 msgr2=0x7f0e0019b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T22:00:28.721 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.718+0000 7f0dff7fe700 1 --2- 172.21.15.157:0/861108191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e0019b360 0x7f0e0019b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:00:28.721 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.718+0000 7f0dff7fe700 1 --2- 172.21.15.157:0/861108191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e0019b360 0x7f0e0019b810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:00:28.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.718+0000 7f0dfe7fc700 1 -- 172.21.15.157:0/861108191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dec0062a0 msgr2=0x7f0e00196580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:00:28.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.718+0000 7f0dfe7fc700 1 --2- 172.21.15.157:0/861108191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dec0062a0 0x7f0e00196580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:28.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.718+0000 7f0dfe7fc700 1 -- 172.21.15.157:0/861108191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e0019b360 msgr2=0x7f0e0019b810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:00:28.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.718+0000 7f0dfe7fc700 1 --2- 172.21.15.157:0/861108191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e0019b360 0x7f0e0019b810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:28.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.718+0000 7f0dfe7fc700 1 -- 172.21.15.157:0/861108191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de800f8e0 con 0x7f0e00196ac0 2024-05-08T22:00:28.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.718+0000 7f0dfe7fc700 1 --2- 172.21.15.157:0/861108191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e00196ac0 0x7f0e0019ae20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0df000b550 tx=0x7f0df000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:28.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.718+0000 7f0df7fff700 1 -- 172.21.15.157:0/861108191 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df0004230 con 0x7f0e00196ac0 2024-05-08T22:00:28.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.718+0000 7f0df7fff700 1 -- 172.21.15.157:0/861108191 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0df0004390 con 0x7f0e00196ac0 2024-05-08T22:00:28.723 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.718+0000 7f0e054e3700 1 -- 172.21.15.157:0/861108191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e001a34f0 con 0x7f0e00196ac0 2024-05-08T22:00:28.723 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.718+0000 7f0df7fff700 1 -- 172.21.15.157:0/861108191 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df0011af0 con 0x7f0e00196ac0 2024-05-08T22:00:28.724 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.718+0000 7f0e054e3700 1 -- 172.21.15.157:0/861108191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e001a3a40 con 0x7f0e00196ac0 2024-05-08T22:00:28.724 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.719+0000 7f0df7fff700 1 -- 172.21.15.157:0/861108191 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f0df0011c50 con 0x7f0e00196ac0 2024-05-08T22:00:28.724 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.720+0000 7f0e054e3700 1 -- 172.21.15.157:0/861108191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e00102130 con 0x7f0e00196ac0 2024-05-08T22:00:28.724 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.720+0000 7f0df7fff700 1 --2- 172.21.15.157:0/861108191 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f0de0038c20 0x7f0de003b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:28.724 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.720+0000 7f0df7fff700 1 -- 172.21.15.157:0/861108191 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0df004ac90 con 0x7f0e00196ac0 2024-05-08T22:00:28.724 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.720+0000 7f0dfeffd700 1 --2- 172.21.15.157:0/861108191 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f0de0038c20 0x7f0de003b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:28.724 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.720+0000 7f0dfeffd700 1 --2- 172.21.15.157:0/861108191 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f0de0038c20 0x7f0de003b0e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0de8005b40 tx=0x7f0de8005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:28.726 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:28.725+0000 7f0df7fff700 1 -- 172.21.15.157:0/861108191 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0df001a400 con 0x7f0e00196ac0 2024-05-08T22:00:28.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:28 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:28.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:28 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:28.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:28 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:00:28.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:28 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:28.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:28 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:28.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:28 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:00:28.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:28 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:28.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:28 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:28.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:28 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:00:28.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:28 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:28.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:28 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:28.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:28 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:00:29.014 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:29.013+0000 7f0e054e3700 1 -- 172.21.15.157:0/861108191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0e00108700 con 0x7f0e00196ac0 2024-05-08T22:00:29.014 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:29.013+0000 7f0df7fff700 1 -- 172.21.15.157:0/861108191 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1278 (secure 0 0 0) 0x7f0df000f2e0 con 0x7f0e00196ac0 2024-05-08T22:00:29.015 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-08T22:00:29.015 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":3,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","modified":"2024-05-08T22:00:22.365975Z","created":"2024-05-08T21:58:39.180260Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:3300","nonce":0},{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:3300","nonce":0},{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-08T22:00:29.017 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:29.016+0000 7f0e054e3700 1 -- 172.21.15.157:0/861108191 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f0de0038c20 msgr2=0x7f0de003b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:29.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:29.016+0000 7f0e054e3700 1 --2- 172.21.15.157:0/861108191 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f0de0038c20 0x7f0de003b0e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0de8005b40 tx=0x7f0de8005a90 comp rx=0 tx=0).stop 2024-05-08T22:00:29.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:29.016+0000 7f0e054e3700 1 -- 172.21.15.157:0/861108191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e00196ac0 msgr2=0x7f0e0019ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:29.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:29.016+0000 7f0e054e3700 1 --2- 172.21.15.157:0/861108191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e00196ac0 0x7f0e0019ae20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0df000b550 tx=0x7f0df000b910 comp rx=0 tx=0).stop 2024-05-08T22:00:29.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:29.016+0000 7f0e054e3700 1 -- 172.21.15.157:0/861108191 shutdown_connections 2024-05-08T22:00:29.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:29.016+0000 7f0e054e3700 1 --2- 172.21.15.157:0/861108191 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f0de0038c20 0x7f0de003b0e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:29.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:29.016+0000 7f0e054e3700 1 --2- 172.21.15.157:0/861108191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e0019b360 0x7f0e0019b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:29.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:29.016+0000 7f0e054e3700 1 --2- 172.21.15.157:0/861108191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dec0062a0 0x7f0e00196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:29.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:29.016+0000 7f0e054e3700 1 --2- 172.21.15.157:0/861108191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e00196ac0 0x7f0e0019ae20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:29.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:29.016+0000 7f0e054e3700 1 -- 172.21.15.157:0/861108191 >> 172.21.15.157:0/861108191 conn(0x7f0e000fc500 msgr2=0x7f0e001024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:29.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:29.016+0000 7f0e054e3700 1 -- 172.21.15.157:0/861108191 shutdown_connections 2024-05-08T22:00:29.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:29.016+0000 7f0e054e3700 1 -- 172.21.15.157:0/861108191 wait complete. 2024-05-08T22:00:29.019 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 3 2024-05-08T22:00:29.375 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-08T22:00:29.376 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-08T22:00:29.745 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:00:29.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: mon.c calling monitor election 2024-05-08T22:00:29.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:00:29.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: mon.a calling monitor election 2024-05-08T22:00:29.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:00:29.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:00:29.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:29.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:00:29.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:00:29.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:29.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:00:29.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:00:29.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:29.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:00:29.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T22:00:29.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: monmap e3: 3 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],b=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T22:00:29.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: fsmap 2024-05-08T22:00:29.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:00:29.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: mgrmap e13: x(active, since 46s) 2024-05-08T22:00:29.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-05-08T22:00:29.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-05-08T22:00:29.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-05-08T22:00:29.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: mon.b (rank 2) addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] is down (out of quorum) 2024-05-08T22:00:29.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:29.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:29.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:29.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:29.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:00:29.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:29.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:29.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:29 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:00:30.284 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:30 smithi131 ceph-mon[37260]: mon.b calling monitor election 2024-05-08T22:00:30.284 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:30 smithi131 ceph-mon[37260]: Updating smithi131:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:00:30.284 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:30 smithi131 ceph-mon[37260]: mon.b calling monitor election 2024-05-08T22:00:30.284 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:30 smithi131 ceph-mon[37260]: mon.c calling monitor election 2024-05-08T22:00:30.284 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:30 smithi131 ceph-mon[37260]: mon.a calling monitor election 2024-05-08T22:00:30.284 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:30 smithi131 ceph-mon[37260]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T22:00:30.284 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:30 smithi131 ceph-mon[37260]: monmap e3: 3 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],b=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T22:00:30.284 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:30 smithi131 ceph-mon[37260]: fsmap 2024-05-08T22:00:30.284 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:30 smithi131 ceph-mon[37260]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:00:30.285 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:30 smithi131 ceph-mon[37260]: mgrmap e13: x(active, since 48s) 2024-05-08T22:00:30.285 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:30 smithi131 ceph-mon[37260]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-05-08T22:00:30.285 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:30 smithi131 ceph-mon[37260]: Cluster is now healthy 2024-05-08T22:00:30.285 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:30 smithi131 ceph-mon[37260]: overall HEALTH_OK 2024-05-08T22:00:30.285 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:30 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:30.285 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:30 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:30.285 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:30 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:30.285 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:30 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:30.285 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:30 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:30.285 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:30 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:30.285 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:30 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:30.285 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:30 smithi131 ceph-mon[37260]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T22:00:30.285 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:30 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:00:30.285 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:30 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:00:30.286 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:30 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:30.286 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:30 smithi131 ceph-mon[37260]: Reconfiguring daemon mon.a on smithi131 2024-05-08T22:00:30.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:30 smithi132 ceph-mon[78180]: mon.b calling monitor election 2024-05-08T22:00:30.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:30 smithi132 ceph-mon[78180]: Updating smithi131:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:00:30.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:30 smithi132 ceph-mon[78180]: mon.b calling monitor election 2024-05-08T22:00:30.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:30 smithi132 ceph-mon[78180]: mon.c calling monitor election 2024-05-08T22:00:30.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:30 smithi132 ceph-mon[78180]: mon.a calling monitor election 2024-05-08T22:00:30.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:30 smithi132 ceph-mon[78180]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T22:00:30.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:30 smithi132 ceph-mon[78180]: monmap e3: 3 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],b=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T22:00:30.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:30 smithi132 ceph-mon[78180]: fsmap 2024-05-08T22:00:30.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:30 smithi132 ceph-mon[78180]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:00:30.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:30 smithi132 ceph-mon[78180]: mgrmap e13: x(active, since 48s) 2024-05-08T22:00:30.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:30 smithi132 ceph-mon[78180]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-05-08T22:00:30.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:30 smithi132 ceph-mon[78180]: Cluster is now healthy 2024-05-08T22:00:30.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:30 smithi132 ceph-mon[78180]: overall HEALTH_OK 2024-05-08T22:00:30.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:30 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:30.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:30 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:30.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:30 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:30.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:30 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:30.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:30 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:30.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:30 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:30.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:30 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:30.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:30 smithi132 ceph-mon[78180]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T22:00:30.342 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:30 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:00:30.342 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:30 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:00:30.342 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:30 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:30.342 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:30 smithi132 ceph-mon[78180]: Reconfiguring daemon mon.a on smithi131 2024-05-08T22:00:30.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:30 smithi157 ceph-mon[77746]: mon.b calling monitor election 2024-05-08T22:00:30.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:30 smithi157 ceph-mon[77746]: Updating smithi131:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:00:30.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:30 smithi157 ceph-mon[77746]: mon.b calling monitor election 2024-05-08T22:00:30.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:30 smithi157 ceph-mon[77746]: mon.c calling monitor election 2024-05-08T22:00:30.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:30 smithi157 ceph-mon[77746]: mon.a calling monitor election 2024-05-08T22:00:30.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:30 smithi157 ceph-mon[77746]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T22:00:30.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:30 smithi157 ceph-mon[77746]: monmap e3: 3 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],b=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T22:00:30.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:30 smithi157 ceph-mon[77746]: fsmap 2024-05-08T22:00:30.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:30 smithi157 ceph-mon[77746]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:00:30.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:30 smithi157 ceph-mon[77746]: mgrmap e13: x(active, since 48s) 2024-05-08T22:00:30.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:30 smithi157 ceph-mon[77746]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-05-08T22:00:30.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:30 smithi157 ceph-mon[77746]: Cluster is now healthy 2024-05-08T22:00:30.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:30 smithi157 ceph-mon[77746]: overall HEALTH_OK 2024-05-08T22:00:30.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:30 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:30.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:30 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:30.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:30 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:30.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:30 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:30.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:30 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:30.528 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:30 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:30.528 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:30 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:30.528 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:30 smithi157 ceph-mon[77746]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T22:00:30.528 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:30 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:00:30.528 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:30 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:00:30.528 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:30 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:30.528 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:30 smithi157 ceph-mon[77746]: Reconfiguring daemon mon.a on smithi131 2024-05-08T22:00:31.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.293+0000 7f9f0a800700 1 -- 172.21.15.131:0/1448258454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f040733d0 msgr2=0x7f9f040737b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:31.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.293+0000 7f9f0a800700 1 --2- 172.21.15.131:0/1448258454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f040733d0 0x7f9f040737b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8009910 tx=0x7f9ef8009c20 comp rx=0 tx=0).stop 2024-05-08T22:00:31.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.293+0000 7f9f0a800700 1 -- 172.21.15.131:0/1448258454 shutdown_connections 2024-05-08T22:00:31.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.293+0000 7f9f0a800700 1 --2- 172.21.15.131:0/1448258454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f040733d0 0x7f9f040737b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:31.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.293+0000 7f9f0a800700 1 -- 172.21.15.131:0/1448258454 >> 172.21.15.131:0/1448258454 conn(0x7f9f040fc1e0 msgr2=0x7f9f040fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:31.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.293+0000 7f9f0a800700 1 -- 172.21.15.131:0/1448258454 shutdown_connections 2024-05-08T22:00:31.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.293+0000 7f9f0a800700 1 -- 172.21.15.131:0/1448258454 wait complete. 2024-05-08T22:00:31.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.294+0000 7f9f0a800700 1 Processor -- start 2024-05-08T22:00:31.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.294+0000 7f9f0a800700 1 -- start start 2024-05-08T22:00:31.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.294+0000 7f9f0a800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f040733d0 0x7f9f04198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:31.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.294+0000 7f9f0a800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f041988c0 0x7f9f0419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:31.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.294+0000 7f9f0a800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f0419d1b0 0x7f9f0419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:31.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.294+0000 7f9f0a800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f04198fb0 con 0x7f9f041988c0 2024-05-08T22:00:31.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.294+0000 7f9f0a800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f041a4f20 con 0x7f9f040733d0 2024-05-08T22:00:31.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.294+0000 7f9f0a800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f041a5060 con 0x7f9f0419d1b0 2024-05-08T22:00:31.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.295+0000 7f9f08a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f041988c0 0x7f9f0419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:31.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.295+0000 7f9f09400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f040733d0 0x7f9f04198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:31.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.295+0000 7f9f08a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f041988c0 0x7f9f0419cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50822/0 (socket says 172.21.15.131:50822) 2024-05-08T22:00:31.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.295+0000 7f9f08a00700 1 -- 172.21.15.131:0/4009413175 learned_addr learned my addr 172.21.15.131:0/4009413175 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:00:31.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.295+0000 7f9f09e00700 1 --2- 172.21.15.131:0/4009413175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f0419d1b0 0x7f9f0419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:31.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.295+0000 7f9f08a00700 1 -- 172.21.15.131:0/4009413175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f0419d1b0 msgr2=0x7f9f0419d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:31.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.295+0000 7f9f08a00700 1 --2- 172.21.15.131:0/4009413175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f0419d1b0 0x7f9f0419d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:31.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.295+0000 7f9f08a00700 1 -- 172.21.15.131:0/4009413175 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f040733d0 msgr2=0x7f9f04198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:31.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.295+0000 7f9f08a00700 1 --2- 172.21.15.131:0/4009413175 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f040733d0 0x7f9f04198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:31.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.295+0000 7f9f08a00700 1 -- 172.21.15.131:0/4009413175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef80095f0 con 0x7f9f041988c0 2024-05-08T22:00:31.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.295+0000 7f9f08a00700 1 --2- 172.21.15.131:0/4009413175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f041988c0 0x7f9f0419cc70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000b780 tx=0x7f9ef000ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:31.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.295+0000 7f9f02000700 1 -- 172.21.15.131:0/4009413175 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0004230 con 0x7f9f041988c0 2024-05-08T22:00:31.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.296+0000 7f9f02000700 1 -- 172.21.15.131:0/4009413175 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ef0004390 con 0x7f9f041988c0 2024-05-08T22:00:31.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.296+0000 7f9f0a800700 1 -- 172.21.15.131:0/4009413175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f041a5340 con 0x7f9f041988c0 2024-05-08T22:00:31.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.296+0000 7f9f02000700 1 -- 172.21.15.131:0/4009413175 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0011b90 con 0x7f9f041988c0 2024-05-08T22:00:31.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.296+0000 7f9f0a800700 1 -- 172.21.15.131:0/4009413175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f041a5840 con 0x7f9f041988c0 2024-05-08T22:00:31.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.297+0000 7f9f02000700 1 -- 172.21.15.131:0/4009413175 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f9ef0011cf0 con 0x7f9f041988c0 2024-05-08T22:00:31.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.297+0000 7f9f0a800700 1 -- 172.21.15.131:0/4009413175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f04074a80 con 0x7f9f041988c0 2024-05-08T22:00:31.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.297+0000 7f9f02000700 1 --2- 172.21.15.131:0/4009413175 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f9eec038c70 0x7f9eec03b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:31.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.297+0000 7f9f02000700 1 -- 172.21.15.131:0/4009413175 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9ef004a120 con 0x7f9f041988c0 2024-05-08T22:00:31.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.297+0000 7f9f09400700 1 --2- 172.21.15.131:0/4009413175 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f9eec038c70 0x7f9eec03b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:31.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.298+0000 7f9f09400700 1 --2- 172.21.15.131:0/4009413175 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f9eec038c70 0x7f9eec03b130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8004690 tx=0x7f9ef8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:31.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.302+0000 7f9f02000700 1 -- 172.21.15.131:0/4009413175 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9ef00182c0 con 0x7f9f041988c0 2024-05-08T22:00:31.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:31 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:00:31.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:31 smithi132 ceph-mon[78180]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:31.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:31 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:31.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:31 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:00:31.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:31 smithi131 ceph-mon[37260]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:31.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:31 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:31.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.504+0000 7f9f0a800700 1 -- 172.21.15.131:0/4009413175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f9f0410cc60 con 0x7f9f041988c0 2024-05-08T22:00:31.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.505+0000 7f9f02000700 1 -- 172.21.15.131:0/4009413175 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7f9ef0016030 con 0x7f9f041988c0 2024-05-08T22:00:31.506 INFO:teuthology.orchestra.run.smithi131.stdout:# minimal ceph.conf for 06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T22:00:31.506 INFO:teuthology.orchestra.run.smithi131.stdout:[global] 2024-05-08T22:00:31.506 INFO:teuthology.orchestra.run.smithi131.stdout: fsid = 06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T22:00:31.506 INFO:teuthology.orchestra.run.smithi131.stdout: mon_host = [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] 2024-05-08T22:00:31.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.507+0000 7f9f0a800700 1 -- 172.21.15.131:0/4009413175 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f9eec038c70 msgr2=0x7f9eec03b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:31.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.507+0000 7f9f0a800700 1 --2- 172.21.15.131:0/4009413175 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f9eec038c70 0x7f9eec03b130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8004690 tx=0x7f9ef8005fb0 comp rx=0 tx=0).stop 2024-05-08T22:00:31.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.507+0000 7f9f0a800700 1 -- 172.21.15.131:0/4009413175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f041988c0 msgr2=0x7f9f0419cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:31.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.507+0000 7f9f0a800700 1 --2- 172.21.15.131:0/4009413175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f041988c0 0x7f9f0419cc70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000b780 tx=0x7f9ef000ba90 comp rx=0 tx=0).stop 2024-05-08T22:00:31.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.507+0000 7f9f0a800700 1 -- 172.21.15.131:0/4009413175 shutdown_connections 2024-05-08T22:00:31.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.507+0000 7f9f0a800700 1 --2- 172.21.15.131:0/4009413175 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f9eec038c70 0x7f9eec03b130 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:31.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.507+0000 7f9f0a800700 1 --2- 172.21.15.131:0/4009413175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f0419d1b0 0x7f9f0419d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:31.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.507+0000 7f9f0a800700 1 --2- 172.21.15.131:0/4009413175 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f040733d0 0x7f9f04198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:31.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.507+0000 7f9f0a800700 1 --2- 172.21.15.131:0/4009413175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f041988c0 0x7f9f0419cc70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:31.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.507+0000 7f9f0a800700 1 -- 172.21.15.131:0/4009413175 >> 172.21.15.131:0/4009413175 conn(0x7f9f040fc1e0 msgr2=0x7f9f04104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:31.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.507+0000 7f9f0a800700 1 -- 172.21.15.131:0/4009413175 shutdown_connections 2024-05-08T22:00:31.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:31.507+0000 7f9f0a800700 1 -- 172.21.15.131:0/4009413175 wait complete. 2024-05-08T22:00:31.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:31 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:00:31.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:31 smithi157 ceph-mon[77746]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:31.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:31 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:31.888 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-08T22:00:31.888 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-08T22:00:31.888 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T22:00:31.956 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-08T22:00:31.957 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:00:32.063 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-05-08T22:00:32.063 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T22:00:32.094 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-05-08T22:00:32.094 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:00:32.161 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-08T22:00:32.162 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T22:00:32.194 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-08T22:00:32.195 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:00:32.262 INFO:tasks.cephadm:Adding mgr.x on smithi131 2024-05-08T22:00:32.262 INFO:tasks.cephadm:Adding mgr.y on smithi132 2024-05-08T22:00:32.263 INFO:tasks.cephadm:Adding mgr.z on smithi157 2024-05-08T22:00:32.263 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi131=x;smithi132=y;smithi157=z' 2024-05-08T22:00:32.470 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:32 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:32.471 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:32 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:32.471 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:32 smithi157 ceph-mon[77746]: Reconfiguring mon.b (monmap changed)... 2024-05-08T22:00:32.471 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:32 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:00:32.471 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:32 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:00:32.471 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:32 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:32.471 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:32 smithi157 ceph-mon[77746]: Reconfiguring daemon mon.b on smithi132 2024-05-08T22:00:32.471 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:32 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/4009413175' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:32.471 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:32 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:32.471 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:32 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:32.471 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:32 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:00:32.471 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:32 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:00:32.471 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:32 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:32.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:32 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:32.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:32 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:32.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:32 smithi132 ceph-mon[78180]: Reconfiguring mon.b (monmap changed)... 2024-05-08T22:00:32.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:32 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:00:32.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:32 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:00:32.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:32 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:32.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:32 smithi132 ceph-mon[78180]: Reconfiguring daemon mon.b on smithi132 2024-05-08T22:00:32.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:32 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/4009413175' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:32.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:32 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:32.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:32 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:32.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:32 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:00:32.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:32 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:00:32.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:32 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:32.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:32 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:32.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:32 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:32.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:32 smithi131 ceph-mon[37260]: Reconfiguring mon.b (monmap changed)... 2024-05-08T22:00:32.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:32 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:00:32.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:32 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:00:32.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:32 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:32.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:32 smithi131 ceph-mon[37260]: Reconfiguring daemon mon.b on smithi132 2024-05-08T22:00:32.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:32 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/4009413175' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:32.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:32 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:32.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:32 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:32.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:32 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:00:32.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:32 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:00:32.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:32 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:32.846 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:00:33.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:33 smithi157 ceph-mon[77746]: Reconfiguring mon.c (monmap changed)... 2024-05-08T22:00:33.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:33 smithi157 ceph-mon[77746]: Reconfiguring daemon mon.c on smithi157 2024-05-08T22:00:33.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:33 smithi157 ceph-mon[77746]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:33.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:33 smithi132 ceph-mon[78180]: Reconfiguring mon.c (monmap changed)... 2024-05-08T22:00:33.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:33 smithi132 ceph-mon[78180]: Reconfiguring daemon mon.c on smithi157 2024-05-08T22:00:33.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:33 smithi132 ceph-mon[78180]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:33.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:33 smithi131 ceph-mon[37260]: Reconfiguring mon.c (monmap changed)... 2024-05-08T22:00:33.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:33 smithi131 ceph-mon[37260]: Reconfiguring daemon mon.c on smithi157 2024-05-08T22:00:33.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:33 smithi131 ceph-mon[37260]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:34.125 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.123+0000 7f8f45210700 1 -- 172.21.15.157:0/1169895532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f400ff2e0 msgr2=0x7f8f400ff700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:34.125 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.123+0000 7f8f45210700 1 --2- 172.21.15.157:0/1169895532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f400ff2e0 0x7f8f400ff700 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009990 tx=0x7f8f30009ca0 comp rx=0 tx=0).stop 2024-05-08T22:00:34.125 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.124+0000 7f8f45210700 1 -- 172.21.15.157:0/1169895532 shutdown_connections 2024-05-08T22:00:34.125 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.124+0000 7f8f45210700 1 --2- 172.21.15.157:0/1169895532 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f40102c60 0x7f8f40109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:34.125 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.124+0000 7f8f45210700 1 --2- 172.21.15.157:0/1169895532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f400ff2e0 0x7f8f400ff700 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:34.126 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.124+0000 7f8f45210700 1 --2- 172.21.15.157:0/1169895532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f401022c0 0x7f8f40102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:34.126 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.124+0000 7f8f45210700 1 -- 172.21.15.157:0/1169895532 >> 172.21.15.157:0/1169895532 conn(0x7f8f400fac40 msgr2=0x7f8f400fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:34.126 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.124+0000 7f8f45210700 1 -- 172.21.15.157:0/1169895532 shutdown_connections 2024-05-08T22:00:34.126 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.124+0000 7f8f45210700 1 -- 172.21.15.157:0/1169895532 wait complete. 2024-05-08T22:00:34.126 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.124+0000 7f8f45210700 1 Processor -- start 2024-05-08T22:00:34.128 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.124+0000 7f8f45210700 1 -- start start 2024-05-08T22:00:34.128 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.125+0000 7f8f45210700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f400ff2e0 0x7f8f40197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:34.128 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.125+0000 7f8f45210700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f401022c0 0x7f8f40197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:34.129 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.125+0000 7f8f45210700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f40102c60 0x7f8f4019e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:34.129 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.125+0000 7f8f45210700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4019eb80 con 0x7f8f400ff2e0 2024-05-08T22:00:34.129 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.125+0000 7f8f45210700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4019ecc0 con 0x7f8f40102c60 2024-05-08T22:00:34.129 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.125+0000 7f8f45210700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4019ee50 con 0x7f8f401022c0 2024-05-08T22:00:34.129 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.125+0000 7f8f3e59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f401022c0 0x7f8f40197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:34.129 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.125+0000 7f8f3e59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f401022c0 0x7f8f40197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52612/0 (socket says 172.21.15.157:52612) 2024-05-08T22:00:34.129 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.125+0000 7f8f3e59c700 1 -- 172.21.15.157:0/961807658 learned_addr learned my addr 172.21.15.157:0/961807658 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-08T22:00:34.129 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.125+0000 7f8f3ed9d700 1 --2- 172.21.15.157:0/961807658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f400ff2e0 0x7f8f40197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:34.129 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.125+0000 7f8f3f59e700 1 --2- 172.21.15.157:0/961807658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f40102c60 0x7f8f4019e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:34.129 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.125+0000 7f8f3e59c700 1 -- 172.21.15.157:0/961807658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f40102c60 msgr2=0x7f8f4019e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:34.129 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.125+0000 7f8f3e59c700 1 --2- 172.21.15.157:0/961807658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f40102c60 0x7f8f4019e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:34.130 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.125+0000 7f8f3e59c700 1 -- 172.21.15.157:0/961807658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f400ff2e0 msgr2=0x7f8f40197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:34.130 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.125+0000 7f8f3e59c700 1 --2- 172.21.15.157:0/961807658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f400ff2e0 0x7f8f40197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:34.130 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.125+0000 7f8f3e59c700 1 -- 172.21.15.157:0/961807658 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f300095f0 con 0x7f8f401022c0 2024-05-08T22:00:34.130 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.125+0000 7f8f3f59e700 1 --2- 172.21.15.157:0/961807658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f40102c60 0x7f8f4019e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:00:34.130 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.125+0000 7f8f3e59c700 1 --2- 172.21.15.157:0/961807658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f401022c0 0x7f8f40197580 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400b560 tx=0x7f8f3400b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:34.130 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.125+0000 7f8f27fff700 1 -- 172.21.15.157:0/961807658 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34011830 con 0x7f8f401022c0 2024-05-08T22:00:34.130 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.125+0000 7f8f45210700 1 -- 172.21.15.157:0/961807658 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4019f130 con 0x7f8f401022c0 2024-05-08T22:00:34.130 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.125+0000 7f8f27fff700 1 -- 172.21.15.157:0/961807658 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f34011e50 con 0x7f8f401022c0 2024-05-08T22:00:34.130 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.125+0000 7f8f27fff700 1 -- 172.21.15.157:0/961807658 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400f2e0 con 0x7f8f401022c0 2024-05-08T22:00:34.130 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.126+0000 7f8f45210700 1 -- 172.21.15.157:0/961807658 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4019f630 con 0x7f8f401022c0 2024-05-08T22:00:34.130 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.126+0000 7f8f27fff700 1 -- 172.21.15.157:0/961807658 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f8f3400f440 con 0x7f8f401022c0 2024-05-08T22:00:34.130 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.126+0000 7f8f27fff700 1 --2- 172.21.15.157:0/961807658 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8f280389d0 0x7f8f2803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:34.131 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.127+0000 7f8f27fff700 1 -- 172.21.15.157:0/961807658 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8f3404aed0 con 0x7f8f401022c0 2024-05-08T22:00:34.131 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.126+0000 7f8f45210700 1 -- 172.21.15.157:0/961807658 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f401085a0 con 0x7f8f401022c0 2024-05-08T22:00:34.131 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.127+0000 7f8f3ed9d700 1 --2- 172.21.15.157:0/961807658 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8f280389d0 0x7f8f2803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:34.131 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.127+0000 7f8f3ed9d700 1 --2- 172.21.15.157:0/961807658 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8f280389d0 0x7f8f2803ae90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8f30006010 tx=0x7f8f300037b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:34.132 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.132+0000 7f8f27fff700 1 -- 172.21.15.157:0/961807658 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8f340178b0 con 0x7f8f401022c0 2024-05-08T22:00:34.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:34 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:34.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:34 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:34.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:34 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:00:34.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:34 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:34.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:34 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:00:34.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:34 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:34.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.337+0000 7f8f45210700 1 -- 172.21.15.157:0/961807658 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi131=x;smithi132=y;smithi157=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f400613e0 con 0x7f8f280389d0 2024-05-08T22:00:34.346 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.345+0000 7f8f27fff700 1 -- 172.21.15.157:0/961807658 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f8f400613e0 con 0x7f8f280389d0 2024-05-08T22:00:34.346 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mgr update... 2024-05-08T22:00:34.349 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.348+0000 7f8f45210700 1 -- 172.21.15.157:0/961807658 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8f280389d0 msgr2=0x7f8f2803ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:34.349 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.348+0000 7f8f45210700 1 --2- 172.21.15.157:0/961807658 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8f280389d0 0x7f8f2803ae90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8f30006010 tx=0x7f8f300037b0 comp rx=0 tx=0).stop 2024-05-08T22:00:34.349 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.348+0000 7f8f45210700 1 -- 172.21.15.157:0/961807658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f401022c0 msgr2=0x7f8f40197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:34.349 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.348+0000 7f8f45210700 1 --2- 172.21.15.157:0/961807658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f401022c0 0x7f8f40197580 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400b560 tx=0x7f8f3400b920 comp rx=0 tx=0).stop 2024-05-08T22:00:34.349 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.348+0000 7f8f45210700 1 -- 172.21.15.157:0/961807658 shutdown_connections 2024-05-08T22:00:34.349 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.348+0000 7f8f45210700 1 --2- 172.21.15.157:0/961807658 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8f280389d0 0x7f8f2803ae90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:34.349 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.348+0000 7f8f45210700 1 --2- 172.21.15.157:0/961807658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f40102c60 0x7f8f4019e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:34.349 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.348+0000 7f8f45210700 1 --2- 172.21.15.157:0/961807658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f400ff2e0 0x7f8f40197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:34.349 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.348+0000 7f8f45210700 1 --2- 172.21.15.157:0/961807658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f401022c0 0x7f8f40197580 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:34.349 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.348+0000 7f8f45210700 1 -- 172.21.15.157:0/961807658 >> 172.21.15.157:0/961807658 conn(0x7f8f400fac40 msgr2=0x7f8f401056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:34.350 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.348+0000 7f8f45210700 1 -- 172.21.15.157:0/961807658 shutdown_connections 2024-05-08T22:00:34.350 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:00:34.348+0000 7f8f45210700 1 -- 172.21.15.157:0/961807658 wait complete. 2024-05-08T22:00:34.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:34 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:34.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:34 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:34.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:34 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:00:34.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:34 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:34.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:34 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:00:34.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:34 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:34.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:34 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:34.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:34 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:34.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:34 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:00:34.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:34 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:34.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:34 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:00:34.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:34 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:34.767 DEBUG:teuthology.orchestra.run.smithi132:mgr.y> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mgr.y.service 2024-05-08T22:00:34.770 DEBUG:teuthology.orchestra.run.smithi157:mgr.z> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mgr.z.service 2024-05-08T22:00:34.772 INFO:tasks.cephadm:Deploying OSDs... 2024-05-08T22:00:34.772 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-08T22:00:34.772 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T22:00:34.802 INFO:journalctl@ceph.mgr.y.smithi132.stdout:-- Logs begin at Wed 2024-05-08 21:34:20 UTC. -- 2024-05-08T22:00:34.804 INFO:journalctl@ceph.mgr.z.smithi157.stdout:-- Logs begin at Wed 2024-05-08 21:35:00 UTC. -- 2024-05-08T22:00:34.819 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T22:00:34.819 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_1 2024-05-08T22:00:34.905 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T22:00:34.905 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:00:34.905 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 448 Links: 1 2024-05-08T22:00:34.905 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:00:34.905 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:00:34.906 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-05-08 21:59:51.626163753 +0000 2024-05-08T22:00:34.906 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-05-08 21:52:58.199148228 +0000 2024-05-08T22:00:34.906 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-05-08 21:52:58.199148228 +0000 2024-05-08T22:00:34.906 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-05-08 21:52:58.174148350 +0000 2024-05-08T22:00:34.906 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T22:00:35.009 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-05-08T22:00:35.009 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-05-08T22:00:35.010 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000320396 s, 1.6 MB/s 2024-05-08T22:00:35.012 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T22:00:35.055 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_2 2024-05-08T22:00:35.138 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T22:00:35.138 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:00:35.138 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 454 Links: 1 2024-05-08T22:00:35.138 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:00:35.138 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:00:35.138 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-05-08 21:59:51.694163392 +0000 2024-05-08T22:00:35.138 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-05-08 21:52:58.688145845 +0000 2024-05-08T22:00:35.138 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-05-08 21:52:58.688145845 +0000 2024-05-08T22:00:35.138 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-05-08 21:52:58.665145957 +0000 2024-05-08T22:00:35.139 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T22:00:35.241 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-05-08T22:00:35.241 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-05-08T22:00:35.241 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000294887 s, 1.7 MB/s 2024-05-08T22:00:35.244 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T22:00:35.287 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_3 2024-05-08T22:00:35.371 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T22:00:35.371 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:00:35.371 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 460 Links: 1 2024-05-08T22:00:35.371 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:00:35.371 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:00:35.371 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-05-08 21:59:51.755163068 +0000 2024-05-08T22:00:35.371 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-05-08 21:52:59.229143208 +0000 2024-05-08T22:00:35.371 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-05-08 21:52:59.229143208 +0000 2024-05-08T22:00:35.371 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-05-08 21:52:59.207143315 +0000 2024-05-08T22:00:35.371 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T22:00:35.472 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-05-08T22:00:35.473 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-05-08T22:00:35.473 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000271988 s, 1.9 MB/s 2024-05-08T22:00:35.475 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T22:00:35.519 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_4 2024-05-08T22:00:35.602 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T22:00:35.602 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:00:35.602 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 466 Links: 1 2024-05-08T22:00:35.603 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:00:35.603 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:00:35.603 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-05-08 21:59:51.819162728 +0000 2024-05-08T22:00:35.603 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-05-08 21:52:59.753140654 +0000 2024-05-08T22:00:35.603 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-05-08 21:52:59.753140654 +0000 2024-05-08T22:00:35.603 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-05-08 21:52:59.730140766 +0000 2024-05-08T22:00:35.603 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T22:00:35.604 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:35 smithi157 ceph-mon[77746]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi131=x;smithi132=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:00:35.604 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:35 smithi157 ceph-mon[77746]: Saving service mgr spec with placement smithi131=x;smithi132=y;smithi157=z;count:3 2024-05-08T22:00:35.604 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:35.604 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:00:35.604 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:35.604 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:00:35.604 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:35.604 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:00:35.604 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T22:00:35.605 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:00:35.605 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:35.605 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:35 smithi157 ceph-mon[77746]: Deploying daemon mgr.z on smithi157 2024-05-08T22:00:35.605 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:35 smithi157 ceph-mon[77746]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:35.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:35 smithi131 ceph-mon[37260]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi131=x;smithi132=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:00:35.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:35 smithi131 ceph-mon[37260]: Saving service mgr spec with placement smithi131=x;smithi132=y;smithi157=z;count:3 2024-05-08T22:00:35.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:35.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:00:35.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:35.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:00:35.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:35.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:00:35.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T22:00:35.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:00:35.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:35.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:35 smithi131 ceph-mon[37260]: Deploying daemon mgr.z on smithi157 2024-05-08T22:00:35.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:35 smithi131 ceph-mon[37260]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:35.703 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-05-08T22:00:35.703 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-05-08T22:00:35.703 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000313804 s, 1.6 MB/s 2024-05-08T22:00:35.706 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T22:00:35.750 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-05-08T22:00:35.750 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T22:00:35.771 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T22:00:35.771 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_1 2024-05-08T22:00:35.832 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T22:00:35.832 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:00:35.832 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 6h/6d Inode: 212590 Links: 1 2024-05-08T22:00:35.832 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:00:35.832 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:00:35.832 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-05-08 22:00:21.947125093 +0000 2024-05-08T22:00:35.832 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-05-08 21:51:22.666500927 +0000 2024-05-08T22:00:35.832 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-05-08 21:51:22.666500927 +0000 2024-05-08T22:00:35.832 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: - 2024-05-08T22:00:35.832 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T22:00:35.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:35 smithi132 ceph-mon[78180]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi131=x;smithi132=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:00:35.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:35 smithi132 ceph-mon[78180]: Saving service mgr spec with placement smithi131=x;smithi132=y;smithi157=z;count:3 2024-05-08T22:00:35.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:35.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:00:35.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:35.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:00:35.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:35.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:00:35.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T22:00:35.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:00:35.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:35.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:35 smithi132 ceph-mon[78180]: Deploying daemon mgr.z on smithi157 2024-05-08T22:00:35.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:35 smithi132 ceph-mon[78180]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:35.870 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-05-08T22:00:35.870 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-05-08T22:00:35.870 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000277628 s, 1.8 MB/s 2024-05-08T22:00:35.871 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T22:00:35.930 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_2 2024-05-08T22:00:35.989 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T22:00:35.989 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:00:35.989 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 6h/6d Inode: 212674 Links: 1 2024-05-08T22:00:35.989 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:00:35.989 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:00:35.989 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-05-08 22:00:21.947125093 +0000 2024-05-08T22:00:35.989 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-05-08 21:51:23.057490437 +0000 2024-05-08T22:00:35.989 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-05-08 21:51:23.057490437 +0000 2024-05-08T22:00:35.989 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: - 2024-05-08T22:00:35.990 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T22:00:36.056 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-05-08T22:00:36.057 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-05-08T22:00:36.057 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000215168 s, 2.4 MB/s 2024-05-08T22:00:36.057 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T22:00:36.116 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_3 2024-05-08T22:00:36.173 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T22:00:36.174 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:00:36.174 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 6h/6d Inode: 212773 Links: 1 2024-05-08T22:00:36.174 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:00:36.174 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:00:36.174 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-05-08 22:00:21.948125066 +0000 2024-05-08T22:00:36.174 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-05-08 21:51:23.412480914 +0000 2024-05-08T22:00:36.174 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-05-08 21:51:23.412480914 +0000 2024-05-08T22:00:36.174 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: - 2024-05-08T22:00:36.174 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T22:00:36.213 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:35 smithi157 systemd[1]: Starting Ceph mgr.z for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:00:36.240 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-05-08T22:00:36.240 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-05-08T22:00:36.241 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000210193 s, 2.4 MB/s 2024-05-08T22:00:36.242 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T22:00:36.300 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_4 2024-05-08T22:00:36.359 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T22:00:36.359 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:00:36.359 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 6h/6d Inode: 209426 Links: 1 2024-05-08T22:00:36.359 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:00:36.359 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:00:36.359 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-05-08 22:00:21.948125066 +0000 2024-05-08T22:00:36.359 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-05-08 21:51:23.749471874 +0000 2024-05-08T22:00:36.359 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-05-08 21:51:23.749471874 +0000 2024-05-08T22:00:36.359 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: - 2024-05-08T22:00:36.360 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T22:00:36.425 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-05-08T22:00:36.425 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-05-08T22:00:36.425 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000241831 s, 2.1 MB/s 2024-05-08T22:00:36.426 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T22:00:36.484 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-08T22:00:36.484 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T22:00:36.502 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T22:00:36.502 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_1 2024-05-08T22:00:36.526 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:36 smithi157 ceph-mgr[79623]: -- 172.21.15.157:0/3718162071 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55804b8701a0 con 0x55804b847000 2024-05-08T22:00:36.526 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:36 smithi157 bash[79586]: 107228a62c872fb462908fb2ce336beb486deb7ab3d8549173c7c323fb5a07c7 2024-05-08T22:00:36.526 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:36 smithi157 systemd[1]: Started Ceph mgr.z for 06f1a696-0d86-11ef-bc97-c7b262605968. 2024-05-08T22:00:36.543 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T22:00:36.543 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:00:36.544 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 209762 Links: 1 2024-05-08T22:00:36.544 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:00:36.544 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:00:36.544 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-05-08 22:00:35.928352752 +0000 2024-05-08T22:00:36.544 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-05-08 21:51:31.373240630 +0000 2024-05-08T22:00:36.544 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-05-08 21:51:31.373240630 +0000 2024-05-08T22:00:36.544 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2024-05-08T22:00:36.544 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T22:00:36.613 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-05-08T22:00:36.613 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-05-08T22:00:36.613 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 9.1885e-05 s, 5.6 MB/s 2024-05-08T22:00:36.614 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T22:00:36.674 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_2 2024-05-08T22:00:36.735 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T22:00:36.735 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:00:36.735 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 214176 Links: 1 2024-05-08T22:00:36.735 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:00:36.735 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:00:36.735 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-05-08 22:00:35.928352752 +0000 2024-05-08T22:00:36.735 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-05-08 21:51:31.715234596 +0000 2024-05-08T22:00:36.735 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-05-08 21:51:31.715234596 +0000 2024-05-08T22:00:36.735 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2024-05-08T22:00:36.735 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T22:00:36.800 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:36 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:00:36.792+0000 7facda7b4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T22:00:36.803 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-05-08T22:00:36.803 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-05-08T22:00:36.803 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 8.966e-05 s, 5.7 MB/s 2024-05-08T22:00:36.804 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T22:00:36.864 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_3 2024-05-08T22:00:36.922 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T22:00:36.922 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:00:36.922 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 212503 Links: 1 2024-05-08T22:00:36.922 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:00:36.922 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:00:36.922 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-05-08 22:00:35.928352752 +0000 2024-05-08T22:00:36.922 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-05-08 21:51:32.059228526 +0000 2024-05-08T22:00:36.922 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-05-08 21:51:32.059228526 +0000 2024-05-08T22:00:36.923 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2024-05-08T22:00:36.923 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T22:00:36.989 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-05-08T22:00:36.989 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-05-08T22:00:36.989 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 9.0367e-05 s, 5.7 MB/s 2024-05-08T22:00:36.990 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T22:00:37.048 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_4 2024-05-08T22:00:37.074 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:36 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:00:36.892+0000 7facda7b4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T22:00:37.074 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:36 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:00:36.984+0000 7facda7b4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T22:00:37.091 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T22:00:37.091 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:00:37.091 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 214286 Links: 1 2024-05-08T22:00:37.091 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:00:37.091 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:00:37.091 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-05-08 22:00:35.928352752 +0000 2024-05-08T22:00:37.091 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-05-08 21:51:32.420222156 +0000 2024-05-08T22:00:37.091 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-05-08 21:51:32.420222156 +0000 2024-05-08T22:00:37.092 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2024-05-08T22:00:37.092 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T22:00:37.156 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-05-08T22:00:37.156 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-05-08T22:00:37.157 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.00010749 s, 4.8 MB/s 2024-05-08T22:00:37.157 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T22:00:37.215 INFO:tasks.cephadm:Deploying osd.0 on smithi131 with /dev/vg_nvme/lv_4... 2024-05-08T22:00:37.215 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T22:00:37.441 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:37 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:00:37.072+0000 7facda7b4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T22:00:37.539 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:37 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:37.539 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:37 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:37.539 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:37 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:37.539 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:37 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:00:37.539 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:37 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T22:00:37.539 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:37 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:00:37.539 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:37 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:37.539 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:37 smithi132 ceph-mon[78180]: Deploying daemon mgr.y on smithi132 2024-05-08T22:00:37.540 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:37 smithi132 ceph-mon[78180]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:37.573 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:00:37.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:37 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:37.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:37 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:37.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:37 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:37.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:37 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:00:37.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:37 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T22:00:37.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:37 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:00:37.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:37 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:37.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:37 smithi131 ceph-mon[37260]: Deploying daemon mgr.y on smithi132 2024-05-08T22:00:37.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:37 smithi131 ceph-mon[37260]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:37.725 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:37 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:37.725 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:37 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:37.726 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:37 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:37.726 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:37 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:00:37.726 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:37 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T22:00:37.726 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:37 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:00:37.726 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:37 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:37.726 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:37 smithi157 ceph-mon[77746]: Deploying daemon mgr.y on smithi132 2024-05-08T22:00:37.726 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:37 smithi157 ceph-mon[77746]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:37.840 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:37 smithi132 systemd[1]: Starting Ceph mgr.y for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:00:37.977 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:37 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:00:37.724+0000 7facda7b4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:00:38.276 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:37 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:00:37.976+0000 7facda7b4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T22:00:38.276 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:38 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:00:38.062+0000 7facda7b4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T22:00:38.276 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:38 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:00:38.171+0000 7facda7b4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T22:00:38.590 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:38 smithi132 ceph-mgr[79689]: -- 172.21.15.132:0/2317066344 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5557240e01a0 con 0x5557240b6c00 2024-05-08T22:00:38.590 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:38 smithi132 bash[79652]: d6c7d8e08decacc2091b77b01aaa13e3cab6da6fb3516198de13c0f3c3d57ccb 2024-05-08T22:00:38.590 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:38 smithi132 systemd[1]: Started Ceph mgr.y for 06f1a696-0d86-11ef-bc97-c7b262605968. 2024-05-08T22:00:38.714 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:38 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:00:38.615+0000 7f83a596e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T22:00:38.776 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:38 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:00:38.331+0000 7facda7b4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T22:00:38.776 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:38 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:00:38.458+0000 7facda7b4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T22:00:38.776 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:38 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:00:38.548+0000 7facda7b4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T22:00:38.915 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:38 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:00:38.712+0000 7f83a596e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T22:00:38.915 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:38 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:00:38.811+0000 7f83a596e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T22:00:38.915 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:38 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:00:38.913+0000 7f83a596e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T22:00:39.276 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:38 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:00:38.966+0000 7facda7b4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T22:00:39.470 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T22:00:39.470 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T22:00:39.470 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-05-08T22:00:39.470 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-05-08T22:00:39.470 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0539535 s, 194 MB/s 2024-05-08T22:00:39.470 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-05-08T22:00:39.577 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:39 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:39.577 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:39 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:39.577 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:39 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:39.577 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:39 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:39.577 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:39 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:00:39.577 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:39 smithi132 ceph-mon[78180]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:39.590 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:39 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:00:39.391+0000 7facda7b4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T22:00:39.590 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:39 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:39.590 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:39 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:39.591 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:39 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:39.591 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:39 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:39.591 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:39 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:00:39.591 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:39 smithi157 ceph-mon[77746]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:39.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:39 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:39.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:39 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:39.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:39 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:39.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:39 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:39.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:39 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:00:39.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:39 smithi131 ceph-mon[37260]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:39.829 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:39 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:00:39.575+0000 7f83a596e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:00:39.901 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_4 2024-05-08T22:00:40.026 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:39 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:00:39.589+0000 7facda7b4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T22:00:40.026 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:39 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:00:39.684+0000 7facda7b4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T22:00:40.090 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:39 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:00:39.828+0000 7f83a596e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T22:00:40.090 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:39 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:00:39.913+0000 7f83a596e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T22:00:40.090 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:40 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:00:40.022+0000 7f83a596e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T22:00:40.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:40.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:40.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:40.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:40.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:40.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:00:40.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:40.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:40 smithi131 ceph-mon[37260]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T22:00:40.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:00:40.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:00:40.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:40.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:40 smithi131 ceph-mon[37260]: Reconfiguring daemon mgr.x on smithi131 2024-05-08T22:00:40.590 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:40 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:00:40.180+0000 7f83a596e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T22:00:40.590 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:40 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:00:40.306+0000 7f83a596e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T22:00:40.590 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:40 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:00:40.391+0000 7f83a596e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T22:00:40.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:40.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:40.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:40.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:40.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:40.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:00:40.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:40.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:40 smithi132 ceph-mon[78180]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T22:00:40.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:00:40.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:00:40.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:40.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:40 smithi132 ceph-mon[78180]: Reconfiguring daemon mgr.x on smithi131 2024-05-08T22:00:40.702 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:40 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:00:40.453+0000 7facda7b4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T22:00:40.702 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:40 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:00:40.550+0000 7facda7b4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T22:00:40.702 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:40 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:00:40.699+0000 7facda7b4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T22:00:40.702 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:40.702 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:40.702 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:40.702 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:40.702 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:40.702 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:00:40.703 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:40.703 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:40 smithi157 ceph-mon[77746]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T22:00:40.703 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:00:40.703 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:00:40.703 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:40.703 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:40 smithi157 ceph-mon[77746]: Reconfiguring daemon mgr.x on smithi131 2024-05-08T22:00:40.772 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:00:41.090 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:40 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:00:40.762+0000 7f83a596e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T22:00:41.443 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:41 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:00:41.157+0000 7f83a596e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T22:00:41.444 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:41 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:00:41.350+0000 7f83a596e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T22:00:41.701 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:41 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:00:41.443+0000 7f83a596e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T22:00:41.702 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:41 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:00:41.356+0000 7facda7b4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T22:00:41.702 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:41 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:00:41.537+0000 7facda7b4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T22:00:42.026 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:41 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:00:41.733+0000 7facda7b4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T22:00:42.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:41 smithi157 ceph-mon[77746]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:42.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:42.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:42.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:42.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:00:42.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:42.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:00:42.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:42.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.050+0000 7f4b9f400700 1 -- 172.21.15.131:0/1001889347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b98102ba0 msgr2=0x7f4b98109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:42.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.050+0000 7f4b9f400700 1 --2- 172.21.15.131:0/1001889347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b98102ba0 0x7f4b98109450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4b9000b0d0 tx=0x7f4b9000b3e0 comp rx=0 tx=0).stop 2024-05-08T22:00:42.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.050+0000 7f4b9f400700 1 -- 172.21.15.131:0/1001889347 shutdown_connections 2024-05-08T22:00:42.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.050+0000 7f4b9f400700 1 --2- 172.21.15.131:0/1001889347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b98102ba0 0x7f4b98109450 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:42.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.050+0000 7f4b9f400700 1 --2- 172.21.15.131:0/1001889347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b98101000 0x7f4b98101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:42.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.050+0000 7f4b9f400700 1 --2- 172.21.15.131:0/1001889347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b98102200 0x7f4b98102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:42.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.050+0000 7f4b9f400700 1 -- 172.21.15.131:0/1001889347 >> 172.21.15.131:0/1001889347 conn(0x7f4b980fc5a0 msgr2=0x7f4b980fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:00:42.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.050+0000 7f4b9f400700 1 -- 172.21.15.131:0/1001889347 shutdown_connections 2024-05-08T22:00:42.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.050+0000 7f4b9f400700 1 -- 172.21.15.131:0/1001889347 wait complete. 2024-05-08T22:00:42.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.051+0000 7f4b9f400700 1 Processor -- start 2024-05-08T22:00:42.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.051+0000 7f4b9f400700 1 -- start start 2024-05-08T22:00:42.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.052+0000 7f4b9f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b98101000 0x7f4b98199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:42.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.052+0000 7f4b9f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b98102200 0x7f4b98199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:42.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.052+0000 7f4b9f400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b98102ba0 0x7f4b981a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:42.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.052+0000 7f4b9f400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b981a0d90 con 0x7f4b98102200 2024-05-08T22:00:42.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.052+0000 7f4b9f400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b981a0ed0 con 0x7f4b98102ba0 2024-05-08T22:00:42.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.052+0000 7f4b9f400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b981a1060 con 0x7f4b98101000 2024-05-08T22:00:42.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.052+0000 7f4b9ca00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b98101000 0x7f4b98199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:42.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.052+0000 7f4b97e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b98102200 0x7f4b98199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:42.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.052+0000 7f4b9ca00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b98101000 0x7f4b98199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60784/0 (socket says 172.21.15.131:60784) 2024-05-08T22:00:42.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.052+0000 7f4b9ca00700 1 -- 172.21.15.131:0/569581255 learned_addr learned my addr 172.21.15.131:0/569581255 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:00:42.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.052+0000 7f4b9d400700 1 --2- 172.21.15.131:0/569581255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b98102ba0 0x7f4b981a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:42.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.052+0000 7f4b9ca00700 1 -- 172.21.15.131:0/569581255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b98102ba0 msgr2=0x7f4b981a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:42.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.052+0000 7f4b9ca00700 1 --2- 172.21.15.131:0/569581255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b98102ba0 0x7f4b981a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:42.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.052+0000 7f4b9ca00700 1 -- 172.21.15.131:0/569581255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b98102200 msgr2=0x7f4b98199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:00:42.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.052+0000 7f4b9ca00700 1 --2- 172.21.15.131:0/569581255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b98102200 0x7f4b98199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:00:42.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.052+0000 7f4b9ca00700 1 -- 172.21.15.131:0/569581255 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b90009d00 con 0x7f4b98101000 2024-05-08T22:00:42.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.052+0000 7f4b9d400700 1 --2- 172.21.15.131:0/569581255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b98102ba0 0x7f4b981a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:00:42.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.052+0000 7f4b97e00700 1 --2- 172.21.15.131:0/569581255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b98102200 0x7f4b98199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:00:42.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.053+0000 7f4b9ca00700 1 --2- 172.21.15.131:0/569581255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b98101000 0x7f4b98199250 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00e9a0 tx=0x7f4b8c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:42.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.053+0000 7f4b95600700 1 -- 172.21.15.131:0/569581255 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c009790 con 0x7f4b98101000 2024-05-08T22:00:42.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.053+0000 7f4b95600700 1 -- 172.21.15.131:0/569581255 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b8c00c810 con 0x7f4b98101000 2024-05-08T22:00:42.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.053+0000 7f4b9f400700 1 -- 172.21.15.131:0/569581255 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b981a1340 con 0x7f4b98101000 2024-05-08T22:00:42.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.053+0000 7f4b95600700 1 -- 172.21.15.131:0/569581255 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c003c70 con 0x7f4b98101000 2024-05-08T22:00:42.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.053+0000 7f4b9f400700 1 -- 172.21.15.131:0/569581255 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b981a1890 con 0x7f4b98101000 2024-05-08T22:00:42.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.054+0000 7f4b95600700 1 -- 172.21.15.131:0/569581255 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f4b8c004220 con 0x7f4b98101000 2024-05-08T22:00:42.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.054+0000 7f4b9f400700 1 -- 172.21.15.131:0/569581255 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b9804ec60 con 0x7f4b98101000 2024-05-08T22:00:42.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.054+0000 7f4b95600700 1 --2- 172.21.15.131:0/569581255 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f4b800389d0 0x7f4b8003ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:00:42.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.054+0000 7f4b95600700 1 -- 172.21.15.131:0/569581255 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4b8c014070 con 0x7f4b98101000 2024-05-08T22:00:42.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.055+0000 7f4b97e00700 1 --2- 172.21.15.131:0/569581255 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f4b800389d0 0x7f4b8003ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:00:42.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.055+0000 7f4b97e00700 1 --2- 172.21.15.131:0/569581255 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f4b800389d0 0x7f4b8003ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4b84011490 tx=0x7f4b84009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:00:42.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.059+0000 7f4b95600700 1 -- 172.21.15.131:0/569581255 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4b8c01f760 con 0x7f4b98101000 2024-05-08T22:00:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:41 smithi132 ceph-mon[78180]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:00:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:00:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:41 smithi131 ceph-mon[37260]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:00:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:00:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:42.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:42.271+0000 7f4b9f400700 1 -- 172.21.15.131:0/569581255 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b980748f0 con 0x7f4b800389d0 2024-05-08T22:00:42.590 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:42 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:00:42.203+0000 7f83a596e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T22:00:42.590 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:42 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:00:42.299+0000 7f83a596e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T22:00:42.590 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:42 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:00:42.447+0000 7f83a596e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T22:00:42.953 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:42 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:00:42.953 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:42 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:00:42.953 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:42 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:42.953 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:42 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:00:42.666+0000 7facda7b4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T22:00:43.051 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:42 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:00:43.051 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:42 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:00:43.051 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:42 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:43.072 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:42 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:00:43.072 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:42 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:00:43.073 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:42 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:43.232 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:42 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:00:42.952+0000 7facda7b4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:00:43.232 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:43 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:00:43.040+0000 7facda7b4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T22:00:43.232 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:00:43 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:00:43.231+0000 7facda7b4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T22:00:43.340 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:43 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:00:43.072+0000 7f83a596e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T22:00:43.340 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:43 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:00:43.253+0000 7f83a596e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T22:00:43.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:43.713+0000 7f4b95600700 1 -- 172.21.15.131:0/569581255 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mgrmap(e 14) v1 ==== 90708+0+0 (secure 0 0 0) 0x7f4b8c00cc80 con 0x7f4b98101000 2024-05-08T22:00:43.719 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:43 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:00:43.446+0000 7f83a596e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T22:00:44.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:43 smithi157 ceph-mon[77746]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:00:44.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:43 smithi157 ceph-mon[77746]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:44.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:43 smithi157 ceph-mon[77746]: Standby manager daemon z started 2024-05-08T22:00:44.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:43 smithi157 ceph-mon[77746]: from='mgr.? 172.21.15.157:0/2833923516' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T22:00:44.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:43 smithi157 ceph-mon[77746]: from='mgr.? 172.21.15.157:0/2833923516' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T22:00:44.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:43 smithi157 ceph-mon[77746]: from='mgr.? 172.21.15.157:0/2833923516' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T22:00:44.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:43 smithi157 ceph-mon[77746]: from='mgr.? 172.21.15.157:0/2833923516' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T22:00:44.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:43 smithi132 ceph-mon[78180]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:00:44.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:43 smithi132 ceph-mon[78180]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:44.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:43 smithi132 ceph-mon[78180]: Standby manager daemon z started 2024-05-08T22:00:44.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:43 smithi132 ceph-mon[78180]: from='mgr.? 172.21.15.157:0/2833923516' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T22:00:44.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:43 smithi132 ceph-mon[78180]: from='mgr.? 172.21.15.157:0/2833923516' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T22:00:44.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:43 smithi132 ceph-mon[78180]: from='mgr.? 172.21.15.157:0/2833923516' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T22:00:44.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:43 smithi132 ceph-mon[78180]: from='mgr.? 172.21.15.157:0/2833923516' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T22:00:44.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:43 smithi131 ceph-mon[37260]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:00:44.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:43 smithi131 ceph-mon[37260]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:44.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:43 smithi131 ceph-mon[37260]: Standby manager daemon z started 2024-05-08T22:00:44.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:43 smithi131 ceph-mon[37260]: from='mgr.? 172.21.15.157:0/2833923516' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T22:00:44.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:43 smithi131 ceph-mon[37260]: from='mgr.? 172.21.15.157:0/2833923516' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T22:00:44.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:43 smithi131 ceph-mon[37260]: from='mgr.? 172.21.15.157:0/2833923516' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T22:00:44.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:43 smithi131 ceph-mon[37260]: from='mgr.? 172.21.15.157:0/2833923516' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T22:00:44.660 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:44 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:00:44.375+0000 7f83a596e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T22:00:44.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:44 smithi132 ceph-mon[78180]: mgrmap e14: x(active, since 63s), standbys: z 2024-05-08T22:00:44.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:44 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T22:00:44.938 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:44 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:00:44.660+0000 7f83a596e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:00:44.939 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:44 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:00:44.747+0000 7f83a596e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T22:00:45.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:44 smithi157 ceph-mon[77746]: mgrmap e14: x(active, since 63s), standbys: z 2024-05-08T22:00:45.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:44 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T22:00:45.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:44 smithi131 ceph-mon[37260]: mgrmap e14: x(active, since 63s), standbys: z 2024-05-08T22:00:45.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:44 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T22:00:45.340 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:00:44 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:00:44.937+0000 7f83a596e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T22:00:45.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:00:45.783+0000 7f4b95600700 1 -- 172.21.15.131:0/569581255 <== mon.1 v2:172.21.15.157:3300/0 8 ==== mgrmap(e 15) v1 ==== 135818+0+0 (secure 0 0 0) 0x7f4b8c0217c0 con 0x7f4b98101000 2024-05-08T22:00:46.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:45 smithi157 ceph-mon[77746]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:46.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:45 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1990771532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fba15ea-c8d0-49f9-a7cc-80efdeff2934"}]: dispatch 2024-05-08T22:00:46.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:45 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1990771532' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fba15ea-c8d0-49f9-a7cc-80efdeff2934"}]': finished 2024-05-08T22:00:46.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:45 smithi157 ceph-mon[77746]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T22:00:46.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:45 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:00:46.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:45 smithi157 ceph-mon[77746]: Standby manager daemon y started 2024-05-08T22:00:46.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:45 smithi157 ceph-mon[77746]: from='mgr.? 172.21.15.132:0/2348765468' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T22:00:46.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:45 smithi157 ceph-mon[77746]: from='mgr.? 172.21.15.132:0/2348765468' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T22:00:46.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:45 smithi157 ceph-mon[77746]: from='mgr.? 172.21.15.132:0/2348765468' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T22:00:46.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:45 smithi157 ceph-mon[77746]: from='mgr.? 172.21.15.132:0/2348765468' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T22:00:46.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:45 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2440337007' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:00:46.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:45 smithi132 ceph-mon[78180]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:46.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:45 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1990771532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fba15ea-c8d0-49f9-a7cc-80efdeff2934"}]: dispatch 2024-05-08T22:00:46.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:45 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1990771532' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fba15ea-c8d0-49f9-a7cc-80efdeff2934"}]': finished 2024-05-08T22:00:46.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:45 smithi132 ceph-mon[78180]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T22:00:46.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:45 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:00:46.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:45 smithi132 ceph-mon[78180]: Standby manager daemon y started 2024-05-08T22:00:46.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:45 smithi132 ceph-mon[78180]: from='mgr.? 172.21.15.132:0/2348765468' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T22:00:46.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:45 smithi132 ceph-mon[78180]: from='mgr.? 172.21.15.132:0/2348765468' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T22:00:46.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:45 smithi132 ceph-mon[78180]: from='mgr.? 172.21.15.132:0/2348765468' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T22:00:46.092 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:45 smithi132 ceph-mon[78180]: from='mgr.? 172.21.15.132:0/2348765468' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T22:00:46.092 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:45 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2440337007' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:00:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:45 smithi131 ceph-mon[37260]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:45 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1990771532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fba15ea-c8d0-49f9-a7cc-80efdeff2934"}]: dispatch 2024-05-08T22:00:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:45 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1990771532' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fba15ea-c8d0-49f9-a7cc-80efdeff2934"}]': finished 2024-05-08T22:00:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:45 smithi131 ceph-mon[37260]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T22:00:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:45 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:00:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:45 smithi131 ceph-mon[37260]: Standby manager daemon y started 2024-05-08T22:00:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:45 smithi131 ceph-mon[37260]: from='mgr.? 172.21.15.132:0/2348765468' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T22:00:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:45 smithi131 ceph-mon[37260]: from='mgr.? 172.21.15.132:0/2348765468' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T22:00:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:45 smithi131 ceph-mon[37260]: from='mgr.? 172.21.15.132:0/2348765468' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T22:00:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:45 smithi131 ceph-mon[37260]: from='mgr.? 172.21.15.132:0/2348765468' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T22:00:46.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:45 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2440337007' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:00:47.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:46 smithi132 ceph-mon[78180]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-05-08T22:00:47.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:46 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T22:00:47.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:46 smithi131 ceph-mon[37260]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-05-08T22:00:47.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:46 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T22:00:47.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:46 smithi157 ceph-mon[77746]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-05-08T22:00:47.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:46 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T22:00:48.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:47 smithi132 ceph-mon[78180]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:48.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:47 smithi131 ceph-mon[37260]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:48.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:47 smithi157 ceph-mon[77746]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:50.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:49 smithi132 ceph-mon[78180]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:50.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:49 smithi131 ceph-mon[37260]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:50.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:49 smithi157 ceph-mon[77746]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:52.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:51 smithi132 ceph-mon[78180]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:52.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:51 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T22:00:52.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:51 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:52.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:51 smithi131 ceph-mon[37260]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:52.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:51 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T22:00:52.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:51 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:52.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:51 smithi157 ceph-mon[77746]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:52.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:51 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T22:00:52.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:51 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:00:53.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:52 smithi132 ceph-mon[78180]: Deploying daemon osd.0 on smithi131 2024-05-08T22:00:53.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:52 smithi131 ceph-mon[37260]: Deploying daemon osd.0 on smithi131 2024-05-08T22:00:53.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:52 smithi157 ceph-mon[77746]: Deploying daemon osd.0 on smithi131 2024-05-08T22:00:54.014 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:53 smithi131 ceph-mon[37260]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:54.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:53 smithi132 ceph-mon[78180]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:54.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:53 smithi157 ceph-mon[77746]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:56.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:55 smithi132 ceph-mon[78180]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:56.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:55 smithi131 ceph-mon[37260]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:56.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:55 smithi157 ceph-mon[77746]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:58.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:57 smithi131 ceph-mon[37260]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:58.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:57 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:58.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:57 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:00:58.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:57 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:58.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:57 smithi157 ceph-mon[77746]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:58.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:57 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:58.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:57 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:00:58.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:57 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:58.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:57 smithi132 ceph-mon[78180]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:00:58.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:57 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:00:58.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:57 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:00:58.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:57 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:00.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:59 smithi131 ceph-mon[37260]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:01:00.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:00.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:00.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:00.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:01:00.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:00:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:00.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:59 smithi132 ceph-mon[78180]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:01:00.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:00.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:00.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:00.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:01:00.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:00:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:00.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:00.130+0000 7f4b95600700 1 -- 172.21.15.131:0/569581255 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4b980748f0 con 0x7f4b800389d0 2024-05-08T22:01:00.131 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 0 on host 'smithi131' 2024-05-08T22:01:00.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:00.132+0000 7f4b9f400700 1 -- 172.21.15.131:0/569581255 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f4b800389d0 msgr2=0x7f4b8003ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:00.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:00.132+0000 7f4b9f400700 1 --2- 172.21.15.131:0/569581255 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f4b800389d0 0x7f4b8003ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4b84011490 tx=0x7f4b84009260 comp rx=0 tx=0).stop 2024-05-08T22:01:00.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:00.132+0000 7f4b9f400700 1 -- 172.21.15.131:0/569581255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b98101000 msgr2=0x7f4b98199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:00.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:00.132+0000 7f4b9f400700 1 --2- 172.21.15.131:0/569581255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b98101000 0x7f4b98199250 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00e9a0 tx=0x7f4b8c00ecb0 comp rx=0 tx=0).stop 2024-05-08T22:01:00.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:00.132+0000 7f4b9f400700 1 -- 172.21.15.131:0/569581255 shutdown_connections 2024-05-08T22:01:00.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:00.132+0000 7f4b9f400700 1 --2- 172.21.15.131:0/569581255 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f4b800389d0 0x7f4b8003ae90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:00.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:00.132+0000 7f4b9f400700 1 --2- 172.21.15.131:0/569581255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b98102ba0 0x7f4b981a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:00.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:00.132+0000 7f4b9f400700 1 --2- 172.21.15.131:0/569581255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b98101000 0x7f4b98199250 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:00.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:00.132+0000 7f4b9f400700 1 --2- 172.21.15.131:0/569581255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b98102200 0x7f4b98199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:00.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:00.132+0000 7f4b9f400700 1 -- 172.21.15.131:0/569581255 >> 172.21.15.131:0/569581255 conn(0x7f4b980fc5a0 msgr2=0x7f4b980731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:00.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:00.133+0000 7f4b9f400700 1 -- 172.21.15.131:0/569581255 shutdown_connections 2024-05-08T22:01:00.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:00.133+0000 7f4b9f400700 1 -- 172.21.15.131:0/569581255 wait complete. 2024-05-08T22:01:00.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:59 smithi157 ceph-mon[77746]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:01:00.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:00.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:00.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:00.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:01:00.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:00:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:00.636 DEBUG:teuthology.orchestra.run.smithi131:osd.0> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.0.service 2024-05-08T22:01:00.640 INFO:tasks.cephadm:Deploying osd.1 on smithi132 with /dev/vg_nvme/lv_4... 2024-05-08T22:01:00.640 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T22:01:00.721 INFO:journalctl@ceph.osd.0.smithi131.stdout:-- Logs begin at Wed 2024-05-08 21:41:05 UTC. -- 2024-05-08T22:01:00.854 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:01:01.307 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:01 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:01.307 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:01 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:01.307 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:01 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:01:01.307 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:01 smithi132 ceph-mon[78180]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:01:01.307 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:01 smithi132 ceph-mon[78180]: from='osd.0 [v2:172.21.15.131:6802/206657594,v1:172.21.15.131:6803/206657594]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T22:01:01.307 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:01 smithi132 ceph-mon[78180]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T22:01:01.357 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 08 22:01:01 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-0[55470]: 2024-05-08T22:01:00.997+0000 7f0eb92fb7c0 -1 osd.0 0 log_to_monitors true 2024-05-08T22:01:01.358 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:01 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:01.358 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:01 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:01.358 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:01 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:01:01.358 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:01 smithi131 ceph-mon[37260]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:01:01.358 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:01 smithi131 ceph-mon[37260]: from='osd.0 [v2:172.21.15.131:6802/206657594,v1:172.21.15.131:6803/206657594]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T22:01:01.358 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:01 smithi131 ceph-mon[37260]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T22:01:01.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:01 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:01.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:01 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:01.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:01 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:01:01.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:01 smithi157 ceph-mon[77746]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:01:01.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:01 smithi157 ceph-mon[77746]: from='osd.0 [v2:172.21.15.131:6802/206657594,v1:172.21.15.131:6803/206657594]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T22:01:01.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:01 smithi157 ceph-mon[77746]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T22:01:02.225 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T22:01:02.225 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T22:01:02.225 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2024-05-08T22:01:02.225 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2024-05-08T22:01:02.225 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0151119 s, 694 MB/s 2024-05-08T22:01:02.225 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2024-05-08T22:01:02.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:02 smithi131 ceph-mon[37260]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T22:01:02.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:02 smithi131 ceph-mon[37260]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T22:01:02.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:02 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:01:02.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:02 smithi131 ceph-mon[37260]: from='osd.0 [v2:172.21.15.131:6802/206657594,v1:172.21.15.131:6803/206657594]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-08T22:01:02.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:02 smithi131 ceph-mon[37260]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-08T22:01:02.415 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:02 smithi132 ceph-mon[78180]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T22:01:02.415 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:02 smithi132 ceph-mon[78180]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T22:01:02.415 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:02 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:01:02.415 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:02 smithi132 ceph-mon[78180]: from='osd.0 [v2:172.21.15.131:6802/206657594,v1:172.21.15.131:6803/206657594]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-08T22:01:02.415 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:02 smithi132 ceph-mon[78180]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-08T22:01:02.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:02 smithi157 ceph-mon[77746]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T22:01:02.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:02 smithi157 ceph-mon[77746]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T22:01:02.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:02 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:01:02.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:02 smithi157 ceph-mon[77746]: from='osd.0 [v2:172.21.15.131:6802/206657594,v1:172.21.15.131:6803/206657594]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-08T22:01:02.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:02 smithi157 ceph-mon[77746]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-08T22:01:02.620 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi132:vg_nvme/lv_4 2024-05-08T22:01:02.836 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:01:03.139 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 08 22:01:02 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-0[55470]: 2024-05-08T22:01:02.677+0000 7f0eae200700 -1 osd.0 0 waiting for initial osdmap 2024-05-08T22:01:03.139 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 08 22:01:02 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-0[55470]: 2024-05-08T22:01:02.690+0000 7f0ea3600700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:01:03.306 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:03 smithi132 ceph-mon[78180]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-08T22:01:03.306 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:03 smithi132 ceph-mon[78180]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T22:01:03.306 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:03 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:01:03.306 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:03 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:01:03.306 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:03 smithi132 ceph-mon[78180]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:01:03.306 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:03 smithi132 ceph-mon[78180]: from='osd.0 ' entity='osd.0' 2024-05-08T22:01:03.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:03 smithi131 ceph-mon[37260]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-08T22:01:03.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:03 smithi131 ceph-mon[37260]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T22:01:03.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:03 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:01:03.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:03 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:01:03.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:03 smithi131 ceph-mon[37260]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:01:03.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:03 smithi131 ceph-mon[37260]: from='osd.0 ' entity='osd.0' 2024-05-08T22:01:03.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:03 smithi157 ceph-mon[77746]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-08T22:01:03.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:03 smithi157 ceph-mon[77746]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T22:01:03.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:03 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:01:03.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:03 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:01:03.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:03 smithi157 ceph-mon[77746]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:01:03.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:03 smithi157 ceph-mon[77746]: from='osd.0 ' entity='osd.0' 2024-05-08T22:01:03.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.893+0000 7f1fd27b8700 1 -- 172.21.15.132:0/2387094099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fcc104440 msgr2=0x7f1fcc106830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:03.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.893+0000 7f1fd27b8700 1 --2- 172.21.15.132:0/2387094099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fcc104440 0x7f1fcc106830 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009990 tx=0x7f1fbc009ca0 comp rx=0 tx=0).stop 2024-05-08T22:01:03.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.893+0000 7f1fd27b8700 1 -- 172.21.15.132:0/2387094099 shutdown_connections 2024-05-08T22:01:03.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.893+0000 7f1fd27b8700 1 --2- 172.21.15.132:0/2387094099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fcc106d70 0x7f1fcc109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:03.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.893+0000 7f1fd27b8700 1 --2- 172.21.15.132:0/2387094099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fcc101b10 0x7f1fcc103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:03.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.893+0000 7f1fd27b8700 1 --2- 172.21.15.132:0/2387094099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fcc104440 0x7f1fcc106830 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:03.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.893+0000 7f1fd27b8700 1 -- 172.21.15.132:0/2387094099 >> 172.21.15.132:0/2387094099 conn(0x7f1fcc0fb4c0 msgr2=0x7f1fcc0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:03.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.893+0000 7f1fd27b8700 1 -- 172.21.15.132:0/2387094099 shutdown_connections 2024-05-08T22:01:03.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.893+0000 7f1fd27b8700 1 -- 172.21.15.132:0/2387094099 wait complete. 2024-05-08T22:01:03.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.894+0000 7f1fd27b8700 1 Processor -- start 2024-05-08T22:01:03.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.894+0000 7f1fd27b8700 1 -- start start 2024-05-08T22:01:03.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.894+0000 7f1fd27b8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fcc101b10 0x7f1fcc196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:03.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.894+0000 7f1fd27b8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fcc104440 0x7f1fcc197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:03.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.894+0000 7f1fd27b8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fcc106d70 0x7f1fcc19e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:03.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.894+0000 7f1fd27b8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc19eb20 con 0x7f1fcc101b10 2024-05-08T22:01:03.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.894+0000 7f1fd27b8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc19ec60 con 0x7f1fcc104440 2024-05-08T22:01:03.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.894+0000 7f1fd27b8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc19edf0 con 0x7f1fcc106d70 2024-05-08T22:01:03.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.894+0000 7f1fcb7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fcc104440 0x7f1fcc197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:03.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.894+0000 7f1fcb7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fcc104440 0x7f1fcc197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:42816/0 (socket says 172.21.15.132:42816) 2024-05-08T22:01:03.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.894+0000 7f1fcb7fe700 1 -- 172.21.15.132:0/3259754717 learned_addr learned my addr 172.21.15.132:0/3259754717 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-05-08T22:01:03.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.894+0000 7f1fcbfff700 1 --2- 172.21.15.132:0/3259754717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fcc101b10 0x7f1fcc196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:03.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.894+0000 7f1fd0d55700 1 --2- 172.21.15.132:0/3259754717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fcc106d70 0x7f1fcc19e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:03.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.894+0000 7f1fcb7fe700 1 -- 172.21.15.132:0/3259754717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fcc106d70 msgr2=0x7f1fcc19e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:03.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.894+0000 7f1fcb7fe700 1 --2- 172.21.15.132:0/3259754717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fcc106d70 0x7f1fcc19e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:03.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.894+0000 7f1fcb7fe700 1 -- 172.21.15.132:0/3259754717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fcc101b10 msgr2=0x7f1fcc196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:03.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.894+0000 7f1fcb7fe700 1 --2- 172.21.15.132:0/3259754717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fcc101b10 0x7f1fcc196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:03.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.894+0000 7f1fcb7fe700 1 -- 172.21.15.132:0/3259754717 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fbc0095f0 con 0x7f1fcc104440 2024-05-08T22:01:03.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.894+0000 7f1fd0d55700 1 --2- 172.21.15.132:0/3259754717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fcc106d70 0x7f1fcc19e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:01:03.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.895+0000 7f1fcb7fe700 1 --2- 172.21.15.132:0/3259754717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fcc104440 0x7f1fcc197520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc0095c0 tx=0x7f1fbc00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:03.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.895+0000 7f1fc97fa700 1 -- 172.21.15.132:0/3259754717 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc01d070 con 0x7f1fcc104440 2024-05-08T22:01:03.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.895+0000 7f1fc97fa700 1 -- 172.21.15.132:0/3259754717 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fbc022470 con 0x7f1fcc104440 2024-05-08T22:01:03.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.895+0000 7f1fd27b8700 1 -- 172.21.15.132:0/3259754717 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fcc19f070 con 0x7f1fcc104440 2024-05-08T22:01:03.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.895+0000 7f1fc97fa700 1 -- 172.21.15.132:0/3259754717 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc017650 con 0x7f1fcc104440 2024-05-08T22:01:03.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.895+0000 7f1fd27b8700 1 -- 172.21.15.132:0/3259754717 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fcc19f480 con 0x7f1fcc104440 2024-05-08T22:01:03.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.896+0000 7f1fd27b8700 1 -- 172.21.15.132:0/3259754717 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fcc0fd040 con 0x7f1fcc104440 2024-05-08T22:01:03.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.897+0000 7f1fc97fa700 1 -- 172.21.15.132:0/3259754717 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 15) v1 ==== 135818+0+0 (secure 0 0 0) 0x7f1fbc022890 con 0x7f1fcc104440 2024-05-08T22:01:03.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.897+0000 7f1fc97fa700 1 --2- 172.21.15.132:0/3259754717 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1fb40a1090 0x7f1fb40a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:03.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.897+0000 7f1fc97fa700 1 -- 172.21.15.132:0/3259754717 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f1fbc0cc450 con 0x7f1fcc104440 2024-05-08T22:01:03.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.897+0000 7f1fcbfff700 1 --2- 172.21.15.132:0/3259754717 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1fb40a1090 0x7f1fb40a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:03.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.898+0000 7f1fcbfff700 1 --2- 172.21.15.132:0/3259754717 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1fb40a1090 0x7f1fb40a3550 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0005950 tx=0x7f1fc000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:03.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:03.902+0000 7f1fc97fa700 1 -- 172.21.15.132:0/3259754717 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1fbc097800 con 0x7f1fcc104440 2024-05-08T22:01:04.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:04.105+0000 7f1fd27b8700 1 -- 172.21.15.132:0/3259754717 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1fcc02ce60 con 0x7f1fb40a1090 2024-05-08T22:01:04.495 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:04 smithi132 ceph-mon[78180]: purged_snaps scrub starts 2024-05-08T22:01:04.495 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:04 smithi132 ceph-mon[78180]: purged_snaps scrub ok 2024-05-08T22:01:04.495 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:04 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:01:04.495 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:04 smithi132 ceph-mon[78180]: Detected new or changed devices on smithi131 2024-05-08T22:01:04.495 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:04 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:04.495 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:04 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:04.495 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:04 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:01:04.496 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:04 smithi132 ceph-mon[78180]: Adjusting osd_memory_target on smithi131 to 17269M 2024-05-08T22:01:04.496 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:04 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:04.496 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:04 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:04.496 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:04 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:01:04.496 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:04 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:04.496 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:04 smithi132 ceph-mon[78180]: osd.0 [v2:172.21.15.131:6802/206657594,v1:172.21.15.131:6803/206657594] boot 2024-05-08T22:01:04.496 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:04 smithi132 ceph-mon[78180]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T22:01:04.496 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:04 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:01:04.496 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:04 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:01:04.496 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:04 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:01:04.496 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:04 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:04.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:04 smithi157 ceph-mon[77746]: purged_snaps scrub starts 2024-05-08T22:01:04.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:04 smithi157 ceph-mon[77746]: purged_snaps scrub ok 2024-05-08T22:01:04.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:04 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:01:04.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:04 smithi157 ceph-mon[77746]: Detected new or changed devices on smithi131 2024-05-08T22:01:04.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:04 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:04.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:04 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:04.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:04 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:01:04.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:04 smithi157 ceph-mon[77746]: Adjusting osd_memory_target on smithi131 to 17269M 2024-05-08T22:01:04.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:04 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:04.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:04 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:04.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:04 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:01:04.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:04 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:04.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:04 smithi157 ceph-mon[77746]: osd.0 [v2:172.21.15.131:6802/206657594,v1:172.21.15.131:6803/206657594] boot 2024-05-08T22:01:04.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:04 smithi157 ceph-mon[77746]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T22:01:04.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:04 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:01:04.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:04 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:01:04.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:04 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:01:04.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:04 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:04.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:04 smithi131 ceph-mon[37260]: purged_snaps scrub starts 2024-05-08T22:01:04.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:04 smithi131 ceph-mon[37260]: purged_snaps scrub ok 2024-05-08T22:01:04.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:04 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:01:04.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:04 smithi131 ceph-mon[37260]: Detected new or changed devices on smithi131 2024-05-08T22:01:04.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:04 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:04.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:04 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:04.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:04 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:01:04.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:04 smithi131 ceph-mon[37260]: Adjusting osd_memory_target on smithi131 to 17269M 2024-05-08T22:01:04.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:04 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:04.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:04 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:04.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:04 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:01:04.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:04 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:04.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:04 smithi131 ceph-mon[37260]: osd.0 [v2:172.21.15.131:6802/206657594,v1:172.21.15.131:6803/206657594] boot 2024-05-08T22:01:04.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:04 smithi131 ceph-mon[37260]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T22:01:04.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:04 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:01:04.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:04 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:01:04.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:04 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:01:04.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:04 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:06.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:05 smithi157 ceph-mon[77746]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:01:06.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:05 smithi157 ceph-mon[77746]: pgmap v40: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:06.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:05 smithi157 ceph-mon[77746]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T22:01:06.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:05 smithi132 ceph-mon[78180]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:01:06.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:05 smithi132 ceph-mon[78180]: pgmap v40: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:06.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:05 smithi132 ceph-mon[78180]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T22:01:06.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:05 smithi131 ceph-mon[37260]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:01:06.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:05 smithi131 ceph-mon[37260]: pgmap v40: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:06.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:05 smithi131 ceph-mon[37260]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T22:01:07.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:06 smithi157 ceph-mon[77746]: from='client.? 172.21.15.132:0/3069193413' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6d2536e-d9a6-4073-8297-987d9e885657"}]: dispatch 2024-05-08T22:01:07.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:06 smithi157 ceph-mon[77746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6d2536e-d9a6-4073-8297-987d9e885657"}]: dispatch 2024-05-08T22:01:07.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:06 smithi157 ceph-mon[77746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6d2536e-d9a6-4073-8297-987d9e885657"}]': finished 2024-05-08T22:01:07.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:06 smithi157 ceph-mon[77746]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T22:01:07.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:06 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:01:07.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:06 smithi157 ceph-mon[77746]: from='client.? 172.21.15.132:0/1078777382' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:01:07.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:06 smithi132 ceph-mon[78180]: from='client.? 172.21.15.132:0/3069193413' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6d2536e-d9a6-4073-8297-987d9e885657"}]: dispatch 2024-05-08T22:01:07.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:06 smithi132 ceph-mon[78180]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6d2536e-d9a6-4073-8297-987d9e885657"}]: dispatch 2024-05-08T22:01:07.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:06 smithi132 ceph-mon[78180]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6d2536e-d9a6-4073-8297-987d9e885657"}]': finished 2024-05-08T22:01:07.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:06 smithi132 ceph-mon[78180]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T22:01:07.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:06 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:01:07.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:06 smithi132 ceph-mon[78180]: from='client.? 172.21.15.132:0/1078777382' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:01:07.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:06 smithi131 ceph-mon[37260]: from='client.? 172.21.15.132:0/3069193413' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6d2536e-d9a6-4073-8297-987d9e885657"}]: dispatch 2024-05-08T22:01:07.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:06 smithi131 ceph-mon[37260]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6d2536e-d9a6-4073-8297-987d9e885657"}]: dispatch 2024-05-08T22:01:07.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:06 smithi131 ceph-mon[37260]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6d2536e-d9a6-4073-8297-987d9e885657"}]': finished 2024-05-08T22:01:07.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:06 smithi131 ceph-mon[37260]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T22:01:07.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:06 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:01:07.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:06 smithi131 ceph-mon[37260]: from='client.? 172.21.15.132:0/1078777382' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:01:08.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:07 smithi157 ceph-mon[77746]: pgmap v43: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:08.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:07 smithi132 ceph-mon[78180]: pgmap v43: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:08.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:07 smithi131 ceph-mon[37260]: pgmap v43: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:10.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:09 smithi157 ceph-mon[77746]: pgmap v44: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:10.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:09 smithi132 ceph-mon[78180]: pgmap v44: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:10.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:09 smithi131 ceph-mon[37260]: pgmap v44: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:12.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:11 smithi157 ceph-mon[77746]: pgmap v45: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:12.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:11 smithi132 ceph-mon[78180]: pgmap v45: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:12.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:11 smithi131 ceph-mon[37260]: pgmap v45: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:13.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:12 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T22:01:13.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:12 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:13.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:12 smithi132 ceph-mon[78180]: Deploying daemon osd.1 on smithi132 2024-05-08T22:01:13.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:12 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T22:01:13.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:12 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:13.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:12 smithi157 ceph-mon[77746]: Deploying daemon osd.1 on smithi132 2024-05-08T22:01:13.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:12 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T22:01:13.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:12 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:13.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:12 smithi131 ceph-mon[37260]: Deploying daemon osd.1 on smithi132 2024-05-08T22:01:14.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:13 smithi132 ceph-mon[78180]: pgmap v46: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:14.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:13 smithi157 ceph-mon[77746]: pgmap v46: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:14.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:13 smithi131 ceph-mon[37260]: pgmap v46: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:16.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:15 smithi132 ceph-mon[78180]: pgmap v47: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:16.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:15 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:16.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:15 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:16.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:15 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:01:16.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:15 smithi157 ceph-mon[77746]: pgmap v47: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:16.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:15 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:16.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:15 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:16.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:15 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:01:16.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:15 smithi131 ceph-mon[37260]: pgmap v47: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:16.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:15 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:16.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:15 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:16.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:15 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:01:18.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:17 smithi157 ceph-mon[77746]: pgmap v48: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:18.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:17 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:18.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:17 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:18.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:17 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:18.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:17 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:01:18.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:17 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:18.299 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:17 smithi132 ceph-mon[78180]: pgmap v48: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:18.299 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:17 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:18.300 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:17 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:18.300 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:17 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:18.300 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:17 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:01:18.300 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:17 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:18.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:17 smithi131 ceph-mon[37260]: pgmap v48: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:18.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:17 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:18.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:17 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:18.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:17 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:18.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:17 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:01:18.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:17 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:18.419 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 1 on host 'smithi132' 2024-05-08T22:01:18.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:18.418+0000 7f1fc97fa700 1 -- 172.21.15.132:0/3259754717 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1fcc02ce60 con 0x7f1fb40a1090 2024-05-08T22:01:18.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:18.420+0000 7f1fd27b8700 1 -- 172.21.15.132:0/3259754717 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1fb40a1090 msgr2=0x7f1fb40a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:18.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:18.420+0000 7f1fd27b8700 1 --2- 172.21.15.132:0/3259754717 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1fb40a1090 0x7f1fb40a3550 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0005950 tx=0x7f1fc000b410 comp rx=0 tx=0).stop 2024-05-08T22:01:18.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:18.420+0000 7f1fd27b8700 1 -- 172.21.15.132:0/3259754717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fcc104440 msgr2=0x7f1fcc197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:18.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:18.420+0000 7f1fd27b8700 1 --2- 172.21.15.132:0/3259754717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fcc104440 0x7f1fcc197520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc0095c0 tx=0x7f1fbc00f690 comp rx=0 tx=0).stop 2024-05-08T22:01:18.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:18.420+0000 7f1fd27b8700 1 -- 172.21.15.132:0/3259754717 shutdown_connections 2024-05-08T22:01:18.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:18.420+0000 7f1fd27b8700 1 --2- 172.21.15.132:0/3259754717 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1fb40a1090 0x7f1fb40a3550 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:18.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:18.420+0000 7f1fd27b8700 1 --2- 172.21.15.132:0/3259754717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fcc106d70 0x7f1fcc19e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:18.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:18.420+0000 7f1fd27b8700 1 --2- 172.21.15.132:0/3259754717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fcc101b10 0x7f1fcc196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:18.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:18.420+0000 7f1fd27b8700 1 --2- 172.21.15.132:0/3259754717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fcc104440 0x7f1fcc197520 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:18.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:18.420+0000 7f1fd27b8700 1 -- 172.21.15.132:0/3259754717 >> 172.21.15.132:0/3259754717 conn(0x7f1fcc0fb4c0 msgr2=0x7f1fcc102510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:18.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:18.420+0000 7f1fd27b8700 1 -- 172.21.15.132:0/3259754717 shutdown_connections 2024-05-08T22:01:18.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:01:18.420+0000 7f1fd27b8700 1 -- 172.21.15.132:0/3259754717 wait complete. 2024-05-08T22:01:18.799 DEBUG:teuthology.orchestra.run.smithi132:osd.1> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.1.service 2024-05-08T22:01:18.802 INFO:tasks.cephadm:Deploying osd.2 on smithi157 with /dev/vg_nvme/lv_4... 2024-05-08T22:01:18.802 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T22:01:18.832 INFO:journalctl@ceph.osd.1.smithi132.stdout:-- Logs begin at Wed 2024-05-08 21:34:20 UTC. -- 2024-05-08T22:01:19.031 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:01:19.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:19.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:19.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:01:19.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:19 smithi157 ceph-mon[77746]: pgmap v49: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:19.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:19.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:19.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:01:19.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:19 smithi131 ceph-mon[37260]: pgmap v49: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:19.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:19 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:19.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:19 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:19.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:19 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:01:19.697 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:19 smithi132 ceph-mon[78180]: pgmap v49: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:20.090 INFO:journalctl@ceph.osd.1.smithi132.stdout:May 08 22:01:19 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-1[83287]: 2024-05-08T22:01:19.810+0000 7f299c51f7c0 -1 osd.1 0 log_to_monitors true 2024-05-08T22:01:20.399 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T22:01:20.399 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T22:01:20.399 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-05-08T22:01:20.399 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-05-08T22:01:20.399 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0132316 s, 792 MB/s 2024-05-08T22:01:20.400 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-05-08T22:01:20.686 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:20 smithi132 ceph-mon[78180]: from='osd.1 [v2:172.21.15.132:6800/2358880332,v1:172.21.15.132:6801/2358880332]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T22:01:20.686 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:20 smithi132 ceph-mon[78180]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T22:01:20.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:20 smithi157 ceph-mon[77746]: from='osd.1 [v2:172.21.15.132:6800/2358880332,v1:172.21.15.132:6801/2358880332]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T22:01:20.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:20 smithi157 ceph-mon[77746]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T22:01:20.792 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_4 2024-05-08T22:01:20.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:20 smithi131 ceph-mon[37260]: from='osd.1 [v2:172.21.15.132:6800/2358880332,v1:172.21.15.132:6801/2358880332]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T22:01:20.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:20 smithi131 ceph-mon[37260]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T22:01:21.008 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:01:21.496 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:21 smithi157 ceph-mon[77746]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T22:01:21.496 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:21 smithi157 ceph-mon[77746]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T22:01:21.497 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:21 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:01:21.497 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:21 smithi157 ceph-mon[77746]: from='osd.1 [v2:172.21.15.132:6800/2358880332,v1:172.21.15.132:6801/2358880332]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-05-08T22:01:21.497 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:21 smithi157 ceph-mon[77746]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-05-08T22:01:21.497 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:21 smithi157 ceph-mon[77746]: pgmap v51: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:21.497 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:21 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:21.497 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:21 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:21.498 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:21 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:01:21.498 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:21 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:21.498 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:21 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:21.498 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:21 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:01:21.498 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:21 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:21.834 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:21 smithi132 ceph-mon[78180]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T22:01:21.834 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:21 smithi132 ceph-mon[78180]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T22:01:21.834 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:21 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:01:21.834 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:21 smithi132 ceph-mon[78180]: from='osd.1 [v2:172.21.15.132:6800/2358880332,v1:172.21.15.132:6801/2358880332]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-05-08T22:01:21.834 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:21 smithi132 ceph-mon[78180]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-05-08T22:01:21.834 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:21 smithi132 ceph-mon[78180]: pgmap v51: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:21.834 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:21 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:21.835 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:21 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:21.835 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:21 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:01:21.835 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:21 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:21.835 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:21 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:21.835 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:21 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:01:21.835 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:21 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:21 smithi131 ceph-mon[37260]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T22:01:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:21 smithi131 ceph-mon[37260]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T22:01:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:21 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:01:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:21 smithi131 ceph-mon[37260]: from='osd.1 [v2:172.21.15.132:6800/2358880332,v1:172.21.15.132:6801/2358880332]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-05-08T22:01:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:21 smithi131 ceph-mon[37260]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-05-08T22:01:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:21 smithi131 ceph-mon[37260]: pgmap v51: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:21 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:21 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:21 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:01:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:21 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:21.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:21 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:21.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:21 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:01:21.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:21 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:22.059 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.057+0000 7fc5c97d9700 1 -- 172.21.15.157:0/2680033141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5c4102b80 msgr2=0x7fc5c4109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:22.059 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.057+0000 7fc5c97d9700 1 --2- 172.21.15.157:0/2680033141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5c4102b80 0x7fc5c4109430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00b0a0 tx=0x7fc5bc00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:01:22.059 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.057+0000 7fc5c97d9700 1 -- 172.21.15.157:0/2680033141 shutdown_connections 2024-05-08T22:01:22.059 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.057+0000 7fc5c97d9700 1 --2- 172.21.15.157:0/2680033141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5c4102b80 0x7fc5c4109430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:22.059 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.057+0000 7fc5c97d9700 1 --2- 172.21.15.157:0/2680033141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5c4100fe0 0x7fc5c4101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:22.059 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.057+0000 7fc5c97d9700 1 --2- 172.21.15.157:0/2680033141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c41021e0 0x7fc5c4102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:22.059 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.057+0000 7fc5c97d9700 1 -- 172.21.15.157:0/2680033141 >> 172.21.15.157:0/2680033141 conn(0x7fc5c40fc580 msgr2=0x7fc5c40fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:22.060 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.057+0000 7fc5c97d9700 1 -- 172.21.15.157:0/2680033141 shutdown_connections 2024-05-08T22:01:22.060 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.057+0000 7fc5c97d9700 1 -- 172.21.15.157:0/2680033141 wait complete. 2024-05-08T22:01:22.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.058+0000 7fc5c97d9700 1 Processor -- start 2024-05-08T22:01:22.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.058+0000 7fc5c97d9700 1 -- start start 2024-05-08T22:01:22.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.059+0000 7fc5c97d9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5c4100fe0 0x7fc5c4197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:22.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.059+0000 7fc5c97d9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5c41021e0 0x7fc5c41975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:22.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.059+0000 7fc5c97d9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c4102b80 0x7fc5c419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:22.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.059+0000 7fc5c97d9700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c419ebd0 con 0x7fc5c4102b80 2024-05-08T22:01:22.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.059+0000 7fc5c97d9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c419ed10 con 0x7fc5c4100fe0 2024-05-08T22:01:22.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.059+0000 7fc5c97d9700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c419eea0 con 0x7fc5c41021e0 2024-05-08T22:01:22.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.059+0000 7fc5c27fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5c41021e0 0x7fc5c41975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:22.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.059+0000 7fc5c27fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5c41021e0 0x7fc5c41975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43970/0 (socket says 172.21.15.157:43970) 2024-05-08T22:01:22.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.059+0000 7fc5c27fc700 1 -- 172.21.15.157:0/2000183559 learned_addr learned my addr 172.21.15.157:0/2000183559 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-08T22:01:22.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.059+0000 7fc5c2ffd700 1 --2- 172.21.15.157:0/2000183559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5c4100fe0 0x7fc5c4197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:22.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.059+0000 7fc5c37fe700 1 --2- 172.21.15.157:0/2000183559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c4102b80 0x7fc5c419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:22.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.059+0000 7fc5c27fc700 1 -- 172.21.15.157:0/2000183559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5c4100fe0 msgr2=0x7fc5c4197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:22.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.059+0000 7fc5c27fc700 1 --2- 172.21.15.157:0/2000183559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5c4100fe0 0x7fc5c4197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:22.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.059+0000 7fc5c27fc700 1 -- 172.21.15.157:0/2000183559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c4102b80 msgr2=0x7fc5c419e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:22.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.059+0000 7fc5c27fc700 1 --2- 172.21.15.157:0/2000183559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c4102b80 0x7fc5c419e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:22.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.059+0000 7fc5c27fc700 1 -- 172.21.15.157:0/2000183559 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc009d00 con 0x7fc5c41021e0 2024-05-08T22:01:22.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.059+0000 7fc5c27fc700 1 --2- 172.21.15.157:0/2000183559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5c41021e0 0x7fc5c41975d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400b560 tx=0x7fc5b400b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:22.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.059+0000 7fc5bbfff700 1 -- 172.21.15.157:0/2000183559 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4011830 con 0x7fc5c41021e0 2024-05-08T22:01:22.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.059+0000 7fc5c97d9700 1 -- 172.21.15.157:0/2000183559 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5c419f180 con 0x7fc5c41021e0 2024-05-08T22:01:22.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.059+0000 7fc5bbfff700 1 -- 172.21.15.157:0/2000183559 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5b4011e50 con 0x7fc5c41021e0 2024-05-08T22:01:22.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.059+0000 7fc5bbfff700 1 -- 172.21.15.157:0/2000183559 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b400f2e0 con 0x7fc5c41021e0 2024-05-08T22:01:22.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.059+0000 7fc5c97d9700 1 -- 172.21.15.157:0/2000183559 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5c419f680 con 0x7fc5c41021e0 2024-05-08T22:01:22.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.061+0000 7fc5c97d9700 1 -- 172.21.15.157:0/2000183559 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5c41083c0 con 0x7fc5c41021e0 2024-05-08T22:01:22.065 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.062+0000 7fc5bbfff700 1 -- 172.21.15.157:0/2000183559 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 15) v1 ==== 135818+0+0 (secure 0 0 0) 0x7fc5b4011990 con 0x7fc5c41021e0 2024-05-08T22:01:22.065 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.062+0000 7fc5bbfff700 1 --2- 172.21.15.157:0/2000183559 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc5b00a1090 0x7fc5b00a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:22.065 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.062+0000 7fc5bbfff700 1 -- 172.21.15.157:0/2000183559 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2029+0+0 (secure 0 0 0) 0x7fc5b40cafc0 con 0x7fc5c41021e0 2024-05-08T22:01:22.065 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.063+0000 7fc5c2ffd700 1 --2- 172.21.15.157:0/2000183559 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc5b00a1090 0x7fc5b00a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:22.065 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.063+0000 7fc5c2ffd700 1 --2- 172.21.15.157:0/2000183559 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc5b00a1090 0x7fc5b00a3550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4102040 tx=0x7fc5ac00b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:22.067 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.066+0000 7fc5bbfff700 1 -- 172.21.15.157:0/2000183559 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc5b40960c0 con 0x7fc5c41021e0 2024-05-08T22:01:22.090 INFO:journalctl@ceph.osd.1.smithi132.stdout:May 08 22:01:21 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-1[83287]: 2024-05-08T22:01:21.833+0000 7f2991b83700 -1 osd.1 0 waiting for initial osdmap 2024-05-08T22:01:22.090 INFO:journalctl@ceph.osd.1.smithi132.stdout:May 08 22:01:21 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-1[83287]: 2024-05-08T22:01:21.836+0000 7f2989970700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:01:22.269 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:22.268+0000 7fc5c97d9700 1 -- 172.21.15.157:0/2000183559 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5c402ce10 con 0x7fc5b00a1090 2024-05-08T22:01:22.670 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:22 smithi157 ceph-mon[77746]: Detected new or changed devices on smithi132 2024-05-08T22:01:22.670 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:22 smithi157 ceph-mon[77746]: Adjusting osd_memory_target on smithi132 to 17114M 2024-05-08T22:01:22.670 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:22 smithi157 ceph-mon[77746]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-05-08T22:01:22.670 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:22 smithi157 ceph-mon[77746]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T22:01:22.670 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:22 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:01:22.670 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:22 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:01:22.670 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:22 smithi157 ceph-mon[77746]: from='osd.1 ' entity='osd.1' 2024-05-08T22:01:22.670 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:22 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:01:22.670 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:22 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:01:22.670 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:22 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:22.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:22 smithi132 ceph-mon[78180]: Detected new or changed devices on smithi132 2024-05-08T22:01:22.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:22 smithi132 ceph-mon[78180]: Adjusting osd_memory_target on smithi132 to 17114M 2024-05-08T22:01:22.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:22 smithi132 ceph-mon[78180]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-05-08T22:01:22.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:22 smithi132 ceph-mon[78180]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T22:01:22.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:22 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:01:22.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:22 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:01:22.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:22 smithi132 ceph-mon[78180]: from='osd.1 ' entity='osd.1' 2024-05-08T22:01:22.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:22 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:01:22.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:22 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:01:22.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:22 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:22.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:22 smithi131 ceph-mon[37260]: Detected new or changed devices on smithi132 2024-05-08T22:01:22.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:22 smithi131 ceph-mon[37260]: Adjusting osd_memory_target on smithi132 to 17114M 2024-05-08T22:01:22.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:22 smithi131 ceph-mon[37260]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-05-08T22:01:22.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:22 smithi131 ceph-mon[37260]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T22:01:22.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:22 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:01:22.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:22 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:01:22.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:22 smithi131 ceph-mon[37260]: from='osd.1 ' entity='osd.1' 2024-05-08T22:01:22.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:22 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:01:22.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:22 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:01:22.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:22 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:23.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:23 smithi157 ceph-mon[77746]: purged_snaps scrub starts 2024-05-08T22:01:23.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:23 smithi157 ceph-mon[77746]: purged_snaps scrub ok 2024-05-08T22:01:23.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:23 smithi157 ceph-mon[77746]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:01:23.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:23 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:01:23.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:23 smithi157 ceph-mon[77746]: pgmap v53: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:23.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:23 smithi157 ceph-mon[77746]: osd.1 [v2:172.21.15.132:6800/2358880332,v1:172.21.15.132:6801/2358880332] boot 2024-05-08T22:01:23.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:23 smithi157 ceph-mon[77746]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T22:01:23.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:23 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:01:23.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:23 smithi132 ceph-mon[78180]: purged_snaps scrub starts 2024-05-08T22:01:23.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:23 smithi132 ceph-mon[78180]: purged_snaps scrub ok 2024-05-08T22:01:23.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:23 smithi132 ceph-mon[78180]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:01:23.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:23 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:01:23.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:23 smithi132 ceph-mon[78180]: pgmap v53: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:23.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:23 smithi132 ceph-mon[78180]: osd.1 [v2:172.21.15.132:6800/2358880332,v1:172.21.15.132:6801/2358880332] boot 2024-05-08T22:01:23.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:23 smithi132 ceph-mon[78180]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T22:01:23.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:23 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:01:23.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:23 smithi131 ceph-mon[37260]: purged_snaps scrub starts 2024-05-08T22:01:23.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:23 smithi131 ceph-mon[37260]: purged_snaps scrub ok 2024-05-08T22:01:23.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:23 smithi131 ceph-mon[37260]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:01:23.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:23 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:01:23.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:23 smithi131 ceph-mon[37260]: pgmap v53: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:01:23.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:23 smithi131 ceph-mon[37260]: osd.1 [v2:172.21.15.132:6800/2358880332,v1:172.21.15.132:6801/2358880332] boot 2024-05-08T22:01:23.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:23 smithi131 ceph-mon[37260]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T22:01:23.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:23 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:01:25.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:24 smithi131 ceph-mon[37260]: osdmap e14: 2 total, 2 up, 2 in 2024-05-08T22:01:25.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:24 smithi131 ceph-mon[37260]: from='client.? 172.21.15.157:0/4135390449' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "992291bf-2d07-42f7-8722-efdbe3dafb17"}]: dispatch 2024-05-08T22:01:25.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:24 smithi131 ceph-mon[37260]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "992291bf-2d07-42f7-8722-efdbe3dafb17"}]: dispatch 2024-05-08T22:01:25.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:24 smithi131 ceph-mon[37260]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "992291bf-2d07-42f7-8722-efdbe3dafb17"}]': finished 2024-05-08T22:01:25.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:24 smithi131 ceph-mon[37260]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T22:01:25.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:24 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:01:25.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:24 smithi131 ceph-mon[37260]: from='client.? 172.21.15.157:0/366830228' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:01:25.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:24 smithi157 ceph-mon[77746]: osdmap e14: 2 total, 2 up, 2 in 2024-05-08T22:01:25.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:24 smithi157 ceph-mon[77746]: from='client.? 172.21.15.157:0/4135390449' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "992291bf-2d07-42f7-8722-efdbe3dafb17"}]: dispatch 2024-05-08T22:01:25.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:24 smithi157 ceph-mon[77746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "992291bf-2d07-42f7-8722-efdbe3dafb17"}]: dispatch 2024-05-08T22:01:25.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:24 smithi157 ceph-mon[77746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "992291bf-2d07-42f7-8722-efdbe3dafb17"}]': finished 2024-05-08T22:01:25.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:24 smithi157 ceph-mon[77746]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T22:01:25.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:24 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:01:25.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:24 smithi157 ceph-mon[77746]: from='client.? 172.21.15.157:0/366830228' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:01:25.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:24 smithi132 ceph-mon[78180]: osdmap e14: 2 total, 2 up, 2 in 2024-05-08T22:01:25.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:24 smithi132 ceph-mon[78180]: from='client.? 172.21.15.157:0/4135390449' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "992291bf-2d07-42f7-8722-efdbe3dafb17"}]: dispatch 2024-05-08T22:01:25.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:24 smithi132 ceph-mon[78180]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "992291bf-2d07-42f7-8722-efdbe3dafb17"}]: dispatch 2024-05-08T22:01:25.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:24 smithi132 ceph-mon[78180]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "992291bf-2d07-42f7-8722-efdbe3dafb17"}]': finished 2024-05-08T22:01:25.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:24 smithi132 ceph-mon[78180]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T22:01:25.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:24 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:01:25.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:24 smithi132 ceph-mon[78180]: from='client.? 172.21.15.157:0/366830228' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:01:26.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:25 smithi131 ceph-mon[37260]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:26.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:25 smithi157 ceph-mon[77746]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:26.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:25 smithi132 ceph-mon[78180]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:28.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:27 smithi157 ceph-mon[77746]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:28.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:27 smithi131 ceph-mon[37260]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:28.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:27 smithi132 ceph-mon[78180]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:30.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:29 smithi131 ceph-mon[37260]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:30.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:29 smithi157 ceph-mon[77746]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:30.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:29 smithi132 ceph-mon[78180]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:31.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:30 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T22:01:31.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:30 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:31.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:30 smithi131 ceph-mon[37260]: Deploying daemon osd.2 on smithi157 2024-05-08T22:01:31.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:30 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T22:01:31.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:30 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:31.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:30 smithi157 ceph-mon[77746]: Deploying daemon osd.2 on smithi157 2024-05-08T22:01:31.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:30 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T22:01:31.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:30 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:31.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:30 smithi132 ceph-mon[78180]: Deploying daemon osd.2 on smithi157 2024-05-08T22:01:31.997 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:31 smithi157 ceph-mon[77746]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:32.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:31 smithi131 ceph-mon[37260]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:32.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:31 smithi132 ceph-mon[78180]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:34.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:33 smithi131 ceph-mon[37260]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:34.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:33 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:01:34.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:33 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:34.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:33 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:34.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:33 smithi157 ceph-mon[77746]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:34.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:33 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:01:34.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:33 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:34.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:33 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:34.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:33 smithi132 ceph-mon[78180]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:34.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:33 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:01:34.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:33 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:34.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:33 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:35.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:35.674+0000 7fc5bbfff700 1 -- 172.21.15.157:0/2000183559 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc5c402ce10 con 0x7fc5b00a1090 2024-05-08T22:01:35.675 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 2 on host 'smithi157' 2024-05-08T22:01:35.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:35.676+0000 7fc5c97d9700 1 -- 172.21.15.157:0/2000183559 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc5b00a1090 msgr2=0x7fc5b00a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:35.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:35.676+0000 7fc5c97d9700 1 --2- 172.21.15.157:0/2000183559 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc5b00a1090 0x7fc5b00a3550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4102040 tx=0x7fc5ac00b5b0 comp rx=0 tx=0).stop 2024-05-08T22:01:35.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:35.676+0000 7fc5c97d9700 1 -- 172.21.15.157:0/2000183559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5c41021e0 msgr2=0x7fc5c41975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:35.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:35.676+0000 7fc5c97d9700 1 --2- 172.21.15.157:0/2000183559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5c41021e0 0x7fc5c41975d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400b560 tx=0x7fc5b400b920 comp rx=0 tx=0).stop 2024-05-08T22:01:35.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:35.676+0000 7fc5c97d9700 1 -- 172.21.15.157:0/2000183559 shutdown_connections 2024-05-08T22:01:35.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:35.676+0000 7fc5c97d9700 1 --2- 172.21.15.157:0/2000183559 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc5b00a1090 0x7fc5b00a3550 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:35.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:35.676+0000 7fc5c97d9700 1 --2- 172.21.15.157:0/2000183559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c4102b80 0x7fc5c419e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:35.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:35.676+0000 7fc5c97d9700 1 --2- 172.21.15.157:0/2000183559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5c4100fe0 0x7fc5c4197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:35.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:35.676+0000 7fc5c97d9700 1 --2- 172.21.15.157:0/2000183559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5c41021e0 0x7fc5c41975d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:35.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:35.676+0000 7fc5c97d9700 1 -- 172.21.15.157:0/2000183559 >> 172.21.15.157:0/2000183559 conn(0x7fc5c40fc580 msgr2=0x7fc5c4105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:35.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:35.676+0000 7fc5c97d9700 1 -- 172.21.15.157:0/2000183559 shutdown_connections 2024-05-08T22:01:35.678 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:01:35.677+0000 7fc5c97d9700 1 -- 172.21.15.157:0/2000183559 wait complete. 2024-05-08T22:01:36.088 DEBUG:teuthology.orchestra.run.smithi157:osd.2> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.2.service 2024-05-08T22:01:36.090 INFO:tasks.cephadm:Deploying osd.3 on smithi131 with /dev/vg_nvme/lv_3... 2024-05-08T22:01:36.090 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T22:01:36.122 INFO:journalctl@ceph.osd.2.smithi157.stdout:-- Logs begin at Wed 2024-05-08 21:35:00 UTC. -- 2024-05-08T22:01:36.268 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:35 smithi157 ceph-mon[77746]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:36.268 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:36.268 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:36.268 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:36.268 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:01:36.268 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:36.268 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:01:36.269 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:36.269 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:36.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:35 smithi132 ceph-mon[78180]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:36.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:36.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:36.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:36.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:01:36.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:36.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:01:36.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:36.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:36.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:35 smithi131 ceph-mon[37260]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:36.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:36.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:36.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:36.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:01:36.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:36.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:01:36.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:36.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:36.458 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:01:37.775 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 08 22:01:37 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-2[83563]: 2024-05-08T22:01:37.502+0000 7fd70f3847c0 -1 osd.2 0 log_to_monitors true 2024-05-08T22:01:38.080 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:37 smithi157 ceph-mon[77746]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:38.080 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:37 smithi157 ceph-mon[77746]: from='osd.2 [v2:172.21.15.157:6800/3561264016,v1:172.21.15.157:6801/3561264016]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T22:01:38.080 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:37 smithi157 ceph-mon[77746]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T22:01:38.202 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T22:01:38.202 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T22:01:38.202 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-05-08T22:01:38.202 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-05-08T22:01:38.202 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0485073 s, 216 MB/s 2024-05-08T22:01:38.202 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-05-08T22:01:38.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:37 smithi132 ceph-mon[78180]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:38.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:37 smithi132 ceph-mon[78180]: from='osd.2 [v2:172.21.15.157:6800/3561264016,v1:172.21.15.157:6801/3561264016]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T22:01:38.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:37 smithi132 ceph-mon[78180]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T22:01:38.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:37 smithi131 ceph-mon[37260]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:38.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:37 smithi131 ceph-mon[37260]: from='osd.2 [v2:172.21.15.157:6800/3561264016,v1:172.21.15.157:6801/3561264016]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T22:01:38.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:37 smithi131 ceph-mon[37260]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T22:01:38.633 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_3 2024-05-08T22:01:38.987 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:01:39.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:39 smithi157 ceph-mon[77746]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T22:01:39.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:39 smithi157 ceph-mon[77746]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T22:01:39.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:39 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:01:39.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:39 smithi157 ceph-mon[77746]: from='osd.2 [v2:172.21.15.157:6800/3561264016,v1:172.21.15.157:6801/3561264016]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-08T22:01:39.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:39 smithi157 ceph-mon[77746]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-08T22:01:39.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:39 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:39.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:39 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:39.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:39 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:01:39.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:39 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:39.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:39 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:39.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:39 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:01:39.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:39 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:39.287 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:39 smithi131 ceph-mon[37260]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T22:01:39.288 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:39 smithi131 ceph-mon[37260]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T22:01:39.288 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:39 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:01:39.288 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:39 smithi131 ceph-mon[37260]: from='osd.2 [v2:172.21.15.157:6800/3561264016,v1:172.21.15.157:6801/3561264016]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-08T22:01:39.288 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:39 smithi131 ceph-mon[37260]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-08T22:01:39.288 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:39 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:39.288 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:39 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:39.288 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:39 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:01:39.288 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:39 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:39.288 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:39 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:39.288 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:39 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:01:39.288 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:39 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:39.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:39 smithi132 ceph-mon[78180]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T22:01:39.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:39 smithi132 ceph-mon[78180]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T22:01:39.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:39 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:01:39.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:39 smithi132 ceph-mon[78180]: from='osd.2 [v2:172.21.15.157:6800/3561264016,v1:172.21.15.157:6801/3561264016]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-08T22:01:39.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:39 smithi132 ceph-mon[78180]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-08T22:01:39.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:39 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:39.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:39 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:39.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:39 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:01:39.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:39 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:39.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:39 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:39.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:39 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:01:39.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:39 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:39.776 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 08 22:01:39 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-2[83563]: 2024-05-08T22:01:39.479+0000 7fd7049e8700 -1 osd.2 0 waiting for initial osdmap 2024-05-08T22:01:39.776 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 08 22:01:39 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-2[83563]: 2024-05-08T22:01:39.484+0000 7fd6fc7d5700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:01:40.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.214+0000 7fa7e5400700 1 -- 172.21.15.131:0/3554552652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7e0074fd0 msgr2=0x7fa7e0073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:40.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.214+0000 7fa7e5400700 1 --2- 172.21.15.131:0/3554552652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7e0074fd0 0x7fa7e0073430 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0009990 tx=0x7fa7d0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:01:40.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.214+0000 7fa7e5400700 1 -- 172.21.15.131:0/3554552652 shutdown_connections 2024-05-08T22:01:40.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.214+0000 7fa7e5400700 1 --2- 172.21.15.131:0/3554552652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7e010d5e0 0x7fa7e010f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:40.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.214+0000 7fa7e5400700 1 --2- 172.21.15.131:0/3554552652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7e0074fd0 0x7fa7e0073430 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:40.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.214+0000 7fa7e5400700 1 --2- 172.21.15.131:0/3554552652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7e0073a00 0x7fa7e0073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:40.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.214+0000 7fa7e5400700 1 -- 172.21.15.131:0/3554552652 >> 172.21.15.131:0/3554552652 conn(0x7fa7e00fc5a0 msgr2=0x7fa7e00fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:40.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.215+0000 7fa7e5400700 1 -- 172.21.15.131:0/3554552652 shutdown_connections 2024-05-08T22:01:40.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.215+0000 7fa7e5400700 1 -- 172.21.15.131:0/3554552652 wait complete. 2024-05-08T22:01:40.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.216+0000 7fa7e5400700 1 Processor -- start 2024-05-08T22:01:40.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.216+0000 7fa7e5400700 1 -- start start 2024-05-08T22:01:40.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.217+0000 7fa7e5400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7e0073a00 0x7fa7e019d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:40.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.217+0000 7fa7e5400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7e0074fd0 0x7fa7e019da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:40.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.217+0000 7fa7e5400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7e010d5e0 0x7fa7e01a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:40.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.217+0000 7fa7e5400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e01a5090 con 0x7fa7e0074fd0 2024-05-08T22:01:40.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.217+0000 7fa7e5400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e01a51d0 con 0x7fa7e0073a00 2024-05-08T22:01:40.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.217+0000 7fa7e5400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e01a5360 con 0x7fa7e010d5e0 2024-05-08T22:01:40.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.217+0000 7fa7dde00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7e0074fd0 0x7fa7e019da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:40.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.217+0000 7fa7dde00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7e0074fd0 0x7fa7e019da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44690/0 (socket says 172.21.15.131:44690) 2024-05-08T22:01:40.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.217+0000 7fa7dde00700 1 -- 172.21.15.131:0/3354393679 learned_addr learned my addr 172.21.15.131:0/3354393679 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:01:40.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.217+0000 7fa7df200700 1 --2- 172.21.15.131:0/3354393679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7e010d5e0 0x7fa7e01a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:40.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.217+0000 7fa7de800700 1 --2- 172.21.15.131:0/3354393679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7e0073a00 0x7fa7e019d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:40.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.217+0000 7fa7dde00700 1 -- 172.21.15.131:0/3354393679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7e010d5e0 msgr2=0x7fa7e01a4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:40.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.217+0000 7fa7dde00700 1 --2- 172.21.15.131:0/3354393679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7e010d5e0 0x7fa7e01a4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:40.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.217+0000 7fa7dde00700 1 -- 172.21.15.131:0/3354393679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7e0073a00 msgr2=0x7fa7e019d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:40.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.217+0000 7fa7dde00700 1 --2- 172.21.15.131:0/3354393679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7e0073a00 0x7fa7e019d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:40.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.217+0000 7fa7dde00700 1 -- 172.21.15.131:0/3354393679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d8009c40 con 0x7fa7e0074fd0 2024-05-08T22:01:40.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.218+0000 7fa7dde00700 1 --2- 172.21.15.131:0/3354393679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7e0074fd0 0x7fa7e019da90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00e9a0 tx=0x7fa7cc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:40.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.218+0000 7fa7d7400700 1 -- 172.21.15.131:0/3354393679 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc009790 con 0x7fa7e0074fd0 2024-05-08T22:01:40.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.218+0000 7fa7d7400700 1 -- 172.21.15.131:0/3354393679 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7cc00c7c0 con 0x7fa7e0074fd0 2024-05-08T22:01:40.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.218+0000 7fa7d7400700 1 -- 172.21.15.131:0/3354393679 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc003c20 con 0x7fa7e0074fd0 2024-05-08T22:01:40.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.218+0000 7fa7e5400700 1 -- 172.21.15.131:0/3354393679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d00095f0 con 0x7fa7e0074fd0 2024-05-08T22:01:40.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.218+0000 7fa7e5400700 1 -- 172.21.15.131:0/3354393679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e01a5960 con 0x7fa7e0074fd0 2024-05-08T22:01:40.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.219+0000 7fa7e5400700 1 -- 172.21.15.131:0/3354393679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e0103840 con 0x7fa7e0074fd0 2024-05-08T22:01:40.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.222+0000 7fa7d7400700 1 -- 172.21.15.131:0/3354393679 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 15) v1 ==== 135818+0+0 (secure 0 0 0) 0x7fa7cc00cc30 con 0x7fa7e0074fd0 2024-05-08T22:01:40.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.223+0000 7fa7d7400700 1 --2- 172.21.15.131:0/3354393679 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa7c40a11e0 0x7fa7c40a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:01:40.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.223+0000 7fa7d7400700 1 -- 172.21.15.131:0/3354393679 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2571+0+0 (secure 0 0 0) 0x7fa7cc014070 con 0x7fa7e0074fd0 2024-05-08T22:01:40.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.223+0000 7fa7de800700 1 --2- 172.21.15.131:0/3354393679 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa7c40a11e0 0x7fa7c40a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:01:40.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.224+0000 7fa7de800700 1 --2- 172.21.15.131:0/3354393679 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa7c40a11e0 0x7fa7c40a36a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0000c00 tx=0x7fa7d0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:01:40.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.227+0000 7fa7d7400700 1 -- 172.21.15.131:0/3354393679 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa7cc0960a0 con 0x7fa7e0074fd0 2024-05-08T22:01:40.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:40 smithi157 ceph-mon[77746]: Detected new or changed devices on smithi157 2024-05-08T22:01:40.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:40 smithi157 ceph-mon[77746]: Adjusting osd_memory_target on smithi157 to 17114M 2024-05-08T22:01:40.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:40 smithi157 ceph-mon[77746]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:40.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:40 smithi157 ceph-mon[77746]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-08T22:01:40.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:40 smithi157 ceph-mon[77746]: osdmap e17: 3 total, 2 up, 3 in 2024-05-08T22:01:40.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:01:40.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:01:40.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:40 smithi157 ceph-mon[77746]: from='osd.2 ' entity='osd.2' 2024-05-08T22:01:40.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:40 smithi132 ceph-mon[78180]: Detected new or changed devices on smithi157 2024-05-08T22:01:40.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:40 smithi132 ceph-mon[78180]: Adjusting osd_memory_target on smithi157 to 17114M 2024-05-08T22:01:40.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:40 smithi132 ceph-mon[78180]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:40.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:40 smithi132 ceph-mon[78180]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-08T22:01:40.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:40 smithi132 ceph-mon[78180]: osdmap e17: 3 total, 2 up, 3 in 2024-05-08T22:01:40.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:01:40.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:01:40.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:40 smithi132 ceph-mon[78180]: from='osd.2 ' entity='osd.2' 2024-05-08T22:01:40.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:40 smithi131 ceph-mon[37260]: Detected new or changed devices on smithi157 2024-05-08T22:01:40.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:40 smithi131 ceph-mon[37260]: Adjusting osd_memory_target on smithi157 to 17114M 2024-05-08T22:01:40.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:40 smithi131 ceph-mon[37260]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:40.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:40 smithi131 ceph-mon[37260]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-08T22:01:40.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:40 smithi131 ceph-mon[37260]: osdmap e17: 3 total, 2 up, 3 in 2024-05-08T22:01:40.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:01:40.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:01:40.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:40 smithi131 ceph-mon[37260]: from='osd.2 ' entity='osd.2' 2024-05-08T22:01:40.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:40.443+0000 7fa7e5400700 1 -- 172.21.15.131:0/3354393679 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7e002ce10 con 0x7fa7c40a11e0 2024-05-08T22:01:41.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:41 smithi131 ceph-mon[37260]: purged_snaps scrub starts 2024-05-08T22:01:41.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:41 smithi131 ceph-mon[37260]: purged_snaps scrub ok 2024-05-08T22:01:41.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:01:41.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:01:41.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:01:41.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:41.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:41 smithi131 ceph-mon[37260]: osd.2 [v2:172.21.15.157:6800/3561264016,v1:172.21.15.157:6801/3561264016] boot 2024-05-08T22:01:41.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:41 smithi131 ceph-mon[37260]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T22:01:41.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:01:41.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:41 smithi157 ceph-mon[77746]: purged_snaps scrub starts 2024-05-08T22:01:41.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:41 smithi157 ceph-mon[77746]: purged_snaps scrub ok 2024-05-08T22:01:41.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:01:41.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:01:41.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:01:41.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:41.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:41 smithi157 ceph-mon[77746]: osd.2 [v2:172.21.15.157:6800/3561264016,v1:172.21.15.157:6801/3561264016] boot 2024-05-08T22:01:41.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:41 smithi157 ceph-mon[77746]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T22:01:41.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:01:41.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:41 smithi132 ceph-mon[78180]: purged_snaps scrub starts 2024-05-08T22:01:41.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:41 smithi132 ceph-mon[78180]: purged_snaps scrub ok 2024-05-08T22:01:41.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:01:41.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:01:41.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:01:41.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:41.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:41 smithi132 ceph-mon[78180]: osd.2 [v2:172.21.15.157:6800/3561264016,v1:172.21.15.157:6801/3561264016] boot 2024-05-08T22:01:41.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:41 smithi132 ceph-mon[78180]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T22:01:41.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:01:42.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:42 smithi157 ceph-mon[77746]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:01:42.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:42 smithi157 ceph-mon[77746]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:42.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:42 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:01:42.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:42 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T22:01:42.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:42 smithi157 ceph-mon[77746]: osdmap e19: 3 total, 3 up, 3 in 2024-05-08T22:01:42.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:42 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:01:42.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:42 smithi132 ceph-mon[78180]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:01:42.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:42 smithi132 ceph-mon[78180]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:42.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:42 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:01:42.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:42 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T22:01:42.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:42 smithi132 ceph-mon[78180]: osdmap e19: 3 total, 3 up, 3 in 2024-05-08T22:01:42.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:42 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:01:42.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:42 smithi131 ceph-mon[37260]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:01:42.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:42 smithi131 ceph-mon[37260]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:01:42.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:42 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:01:42.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:42 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T22:01:42.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:42 smithi131 ceph-mon[37260]: osdmap e19: 3 total, 3 up, 3 in 2024-05-08T22:01:42.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:42 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:01:43.503 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 08 22:01:43 smithi131 sudo[61115]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T22:01:43.503 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 08 22:01:43 smithi131 sudo[61115]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:01:43.503 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 08 22:01:43 smithi131 sudo[61115]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:01:43.504 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 08 22:01:43 smithi131 sudo[61115]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:01:43.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:43.510+0000 7fa7d7400700 1 -- 172.21.15.131:0/3354393679 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa7cc096470 con 0x7fa7e0074fd0 2024-05-08T22:01:43.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:43 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T22:01:43.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:43 smithi157 ceph-mon[77746]: osdmap e20: 3 total, 3 up, 3 in 2024-05-08T22:01:43.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:43 smithi157 ceph-mon[77746]: pgmap v71: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:01:43.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:43 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2026219337' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8281ef9-2390-4b3e-9ba8-0f7e71ca3616"}]: dispatch 2024-05-08T22:01:43.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:43 smithi157 ceph-mon[77746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8281ef9-2390-4b3e-9ba8-0f7e71ca3616"}]: dispatch 2024-05-08T22:01:43.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:43 smithi157 ceph-mon[77746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b8281ef9-2390-4b3e-9ba8-0f7e71ca3616"}]': finished 2024-05-08T22:01:43.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:43 smithi157 ceph-mon[77746]: osdmap e21: 4 total, 3 up, 4 in 2024-05-08T22:01:43.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:43 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:01:43.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:43 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T22:01:43.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:43 smithi132 ceph-mon[78180]: osdmap e20: 3 total, 3 up, 3 in 2024-05-08T22:01:43.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:43 smithi132 ceph-mon[78180]: pgmap v71: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:01:43.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:43 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2026219337' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8281ef9-2390-4b3e-9ba8-0f7e71ca3616"}]: dispatch 2024-05-08T22:01:43.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:43 smithi132 ceph-mon[78180]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8281ef9-2390-4b3e-9ba8-0f7e71ca3616"}]: dispatch 2024-05-08T22:01:43.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:43 smithi132 ceph-mon[78180]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b8281ef9-2390-4b3e-9ba8-0f7e71ca3616"}]': finished 2024-05-08T22:01:43.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:43 smithi132 ceph-mon[78180]: osdmap e21: 4 total, 3 up, 4 in 2024-05-08T22:01:43.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:43 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:01:43.890 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 08 22:01:43 smithi131 sudo[61147]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T22:01:43.891 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 08 22:01:43 smithi131 sudo[61147]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:01:43.891 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 08 22:01:43 smithi131 sudo[61147]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:01:43.891 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 08 22:01:43 smithi131 sudo[61147]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:01:43.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:43 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T22:01:43.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:43 smithi131 ceph-mon[37260]: osdmap e20: 3 total, 3 up, 3 in 2024-05-08T22:01:43.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:43 smithi131 ceph-mon[37260]: pgmap v71: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:01:43.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:43 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2026219337' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8281ef9-2390-4b3e-9ba8-0f7e71ca3616"}]: dispatch 2024-05-08T22:01:43.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:43 smithi131 ceph-mon[37260]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8281ef9-2390-4b3e-9ba8-0f7e71ca3616"}]: dispatch 2024-05-08T22:01:43.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:43 smithi131 ceph-mon[37260]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b8281ef9-2390-4b3e-9ba8-0f7e71ca3616"}]': finished 2024-05-08T22:01:43.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:43 smithi131 ceph-mon[37260]: osdmap e21: 4 total, 3 up, 4 in 2024-05-08T22:01:43.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:43 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:01:44.340 INFO:journalctl@ceph.osd.1.smithi132.stdout:May 08 22:01:43 smithi132 sudo[88756]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T22:01:44.340 INFO:journalctl@ceph.osd.1.smithi132.stdout:May 08 22:01:43 smithi132 sudo[88756]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:01:44.340 INFO:journalctl@ceph.osd.1.smithi132.stdout:May 08 22:01:43 smithi132 sudo[88756]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:01:44.340 INFO:journalctl@ceph.osd.1.smithi132.stdout:May 08 22:01:44 smithi132 sudo[88756]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:01:44.340 INFO:journalctl@ceph.osd.1.smithi132.stdout:May 08 22:01:44 smithi132 sudo[88759]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T22:01:44.340 INFO:journalctl@ceph.osd.1.smithi132.stdout:May 08 22:01:44 smithi132 sudo[88759]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:01:44.340 INFO:journalctl@ceph.osd.1.smithi132.stdout:May 08 22:01:44 smithi132 sudo[88759]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:01:44.340 INFO:journalctl@ceph.osd.1.smithi132.stdout:May 08 22:01:44 smithi132 sudo[88759]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:01:44.760 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:44 smithi157 ceph-mon[77746]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-08T22:01:44.760 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:44 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:44.760 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:44 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/908867661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:01:44.760 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:44 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:44.761 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 08 22:01:44 smithi157 sudo[88214]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T22:01:44.761 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 08 22:01:44 smithi157 sudo[88214]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:01:44.761 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 08 22:01:44 smithi157 sudo[88214]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:01:44.761 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 08 22:01:44 smithi157 sudo[88214]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:01:44.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:44 smithi132 ceph-mon[78180]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-08T22:01:44.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:44 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:44.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:44 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/908867661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:01:44.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:44 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:44.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:44 smithi131 ceph-mon[37260]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-08T22:01:44.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:44 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:44.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:44 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/908867661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:01:44.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:44 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:45.026 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 08 22:01:44 smithi157 sudo[88217]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T22:01:45.026 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 08 22:01:44 smithi157 sudo[88217]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:01:45.026 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 08 22:01:44 smithi157 sudo[88217]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:01:45.026 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 08 22:01:44 smithi157 sudo[88217]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:01:45.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:45 smithi131 sudo[61761]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T22:01:45.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:45 smithi131 sudo[61761]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:01:45.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:45 smithi131 sudo[61761]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:01:45.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:45 smithi131 sudo[61761]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:01:46.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:45 smithi131 sudo[61952]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T22:01:46.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:45 smithi131 sudo[61952]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:01:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:45 smithi131 sudo[61952]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:01:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:45 smithi131 sudo[61952]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:01:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:45 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:45 smithi131 ceph-mon[37260]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:01:46.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:45 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:46.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:45 smithi132 ceph-mon[78180]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:01:46.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:46 smithi157 sudo[88220]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T22:01:46.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:46 smithi157 sudo[88220]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:01:46.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:46 smithi157 sudo[88220]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:01:46.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:46 smithi157 sudo[88220]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:01:46.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:46 smithi157 sudo[88223]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T22:01:46.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:46 smithi157 sudo[88223]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:01:46.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:46 smithi157 sudo[88223]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:01:46.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:46 smithi157 sudo[88223]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:01:46.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:46 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:46.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:46 smithi157 ceph-mon[77746]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:01:46.905 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:46 smithi132 sudo[88762]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T22:01:46.906 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:46 smithi132 sudo[88762]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:01:46.906 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:46 smithi132 sudo[88762]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:01:46.906 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:46 smithi132 sudo[88762]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:01:46.906 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:46 smithi132 sudo[88765]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T22:01:46.906 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:46 smithi132 sudo[88765]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:01:46.906 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:46 smithi132 sudo[88765]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:01:46.906 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:46 smithi132 sudo[88765]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:01:47.214 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:46 smithi131 ceph-mon[37260]: pgmap v73: 1 pgs: 1 active+clean; 449 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:01:47.214 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:46 smithi131 ceph-mon[37260]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:01:47.214 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:46 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:01:47.214 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:46 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:01:47.214 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:46 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:01:47.214 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:46 smithi131 ceph-mon[37260]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:01:47.214 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:46 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:01:47.214 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:46 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:01:47.214 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:46 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:01:47.214 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:46 smithi131 ceph-mon[37260]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:01:47.214 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:46 smithi131 ceph-mon[37260]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:01:47.215 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:46 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:01:47.215 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:46 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:01:47.215 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:46 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:01:47.215 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:46 smithi131 ceph-mon[37260]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:01:47.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:46 smithi157 ceph-mon[77746]: pgmap v73: 1 pgs: 1 active+clean; 449 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:01:47.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:46 smithi157 ceph-mon[77746]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:01:47.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:46 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:01:47.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:46 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:01:47.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:46 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:01:47.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:46 smithi157 ceph-mon[77746]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:01:47.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:46 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:01:47.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:46 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:01:47.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:46 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:01:47.278 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:46 smithi157 ceph-mon[77746]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:01:47.278 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:46 smithi157 ceph-mon[77746]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:01:47.278 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:46 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:01:47.278 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:46 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:01:47.278 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:46 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:01:47.278 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:46 smithi157 ceph-mon[77746]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:01:47.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:46 smithi132 ceph-mon[78180]: pgmap v73: 1 pgs: 1 active+clean; 449 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:01:47.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:46 smithi132 ceph-mon[78180]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:01:47.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:46 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:01:47.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:46 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:01:47.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:46 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:01:47.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:46 smithi132 ceph-mon[78180]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:01:47.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:46 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:01:47.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:46 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:01:47.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:46 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:01:47.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:46 smithi132 ceph-mon[78180]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:01:47.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:46 smithi132 ceph-mon[78180]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:01:47.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:46 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:01:47.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:46 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:01:47.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:46 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:01:47.342 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:46 smithi132 ceph-mon[78180]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:01:48.176 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:47 smithi131 ceph-mon[37260]: pgmap v74: 1 pgs: 1 active+clean; 449 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:01:48.278 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:47 smithi157 ceph-mon[77746]: pgmap v74: 1 pgs: 1 active+clean; 449 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:01:48.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:47 smithi132 ceph-mon[78180]: pgmap v74: 1 pgs: 1 active+clean; 449 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:01:50.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:49 smithi131 ceph-mon[37260]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 10 KiB/s rd, 457 KiB/s wr, 31 op/s 2024-05-08T22:01:50.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:49 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T22:01:50.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:49 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:50.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:49 smithi157 ceph-mon[77746]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 10 KiB/s rd, 457 KiB/s wr, 31 op/s 2024-05-08T22:01:50.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:49 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T22:01:50.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:49 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:50.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:49 smithi132 ceph-mon[78180]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 10 KiB/s rd, 457 KiB/s wr, 31 op/s 2024-05-08T22:01:50.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:49 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T22:01:50.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:49 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:51.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:50 smithi157 ceph-mon[77746]: Deploying daemon osd.3 on smithi131 2024-05-08T22:01:51.339 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:50 smithi132 ceph-mon[78180]: Deploying daemon osd.3 on smithi131 2024-05-08T22:01:51.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:50 smithi131 ceph-mon[37260]: Deploying daemon osd.3 on smithi131 2024-05-08T22:01:52.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:51 smithi157 ceph-mon[77746]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 6.7 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-05-08T22:01:52.280 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:51 smithi131 ceph-mon[37260]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 6.7 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-05-08T22:01:52.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:51 smithi132 ceph-mon[78180]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 6.7 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-05-08T22:01:54.248 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:53 smithi131 ceph-mon[37260]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-05-08T22:01:54.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:53 smithi157 ceph-mon[77746]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-05-08T22:01:54.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:53 smithi132 ceph-mon[78180]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-05-08T22:01:56.085 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:55 smithi131 ceph-mon[37260]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-05-08T22:01:56.085 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:55 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:56.085 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:55 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:56.085 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:55 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:01:56.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:55 smithi157 ceph-mon[77746]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-05-08T22:01:56.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:55 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:56.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:55 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:56.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:55 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:01:56.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:55 smithi132 ceph-mon[78180]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-05-08T22:01:56.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:55 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:56.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:55 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:56.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:55 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:01:58.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:57 smithi131 ceph-mon[37260]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-05-08T22:01:58.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:57 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:58.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:57 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:58.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:57 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:58.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:57 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:01:58.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:57 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:58.275 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:57 smithi157 ceph-mon[77746]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-05-08T22:01:58.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:57 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:58.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:57 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:58.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:57 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:58.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:57 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:01:58.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:57 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:58.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:57 smithi132 ceph-mon[78180]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-05-08T22:01:58.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:57 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:58.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:57 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:58.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:57 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:01:58.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:57 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:01:58.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:57 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:58.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:58.493+0000 7fa7d7400700 1 -- 172.21.15.131:0/3354393679 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa7e002ce10 con 0x7fa7c40a11e0 2024-05-08T22:01:58.494 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 3 on host 'smithi131' 2024-05-08T22:01:58.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:58.496+0000 7fa7e5400700 1 -- 172.21.15.131:0/3354393679 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa7c40a11e0 msgr2=0x7fa7c40a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:58.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:58.496+0000 7fa7e5400700 1 --2- 172.21.15.131:0/3354393679 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa7c40a11e0 0x7fa7c40a36a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0000c00 tx=0x7fa7d0005fb0 comp rx=0 tx=0).stop 2024-05-08T22:01:58.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:58.496+0000 7fa7e5400700 1 -- 172.21.15.131:0/3354393679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7e0074fd0 msgr2=0x7fa7e019da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:01:58.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:58.496+0000 7fa7e5400700 1 --2- 172.21.15.131:0/3354393679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7e0074fd0 0x7fa7e019da90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00e9a0 tx=0x7fa7cc00ecb0 comp rx=0 tx=0).stop 2024-05-08T22:01:58.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:58.496+0000 7fa7e5400700 1 -- 172.21.15.131:0/3354393679 shutdown_connections 2024-05-08T22:01:58.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:58.496+0000 7fa7e5400700 1 --2- 172.21.15.131:0/3354393679 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa7c40a11e0 0x7fa7c40a36a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:58.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:58.496+0000 7fa7e5400700 1 --2- 172.21.15.131:0/3354393679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7e010d5e0 0x7fa7e01a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:58.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:58.496+0000 7fa7e5400700 1 --2- 172.21.15.131:0/3354393679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7e0073a00 0x7fa7e019d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:58.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:58.496+0000 7fa7e5400700 1 --2- 172.21.15.131:0/3354393679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7e0074fd0 0x7fa7e019da90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:01:58.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:58.496+0000 7fa7e5400700 1 -- 172.21.15.131:0/3354393679 >> 172.21.15.131:0/3354393679 conn(0x7fa7e00fc5a0 msgr2=0x7fa7e0100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:01:58.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:58.497+0000 7fa7e5400700 1 -- 172.21.15.131:0/3354393679 shutdown_connections 2024-05-08T22:01:58.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:01:58.497+0000 7fa7e5400700 1 -- 172.21.15.131:0/3354393679 wait complete. 2024-05-08T22:01:58.985 DEBUG:teuthology.orchestra.run.smithi131:osd.3> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.3.service 2024-05-08T22:01:58.989 INFO:tasks.cephadm:Deploying osd.4 on smithi132 with /dev/vg_nvme/lv_3... 2024-05-08T22:01:58.990 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T22:01:59.068 INFO:journalctl@ceph.osd.3.smithi131.stdout:-- Logs begin at Wed 2024-05-08 21:41:05 UTC. -- 2024-05-08T22:01:59.221 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:01:59.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:59.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:59.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:01:59.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:59 smithi132 ceph-mon[78180]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-05-08T22:01:59.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:01:59 smithi132 ceph-mon[78180]: from='osd.3 [v2:172.21.15.131:6810/1075134307,v1:172.21.15.131:6811/1075134307]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T22:01:59.620 INFO:journalctl@ceph.osd.3.smithi131.stdout:May 08 22:01:59 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-3[63378]: 2024-05-08T22:01:59.371+0000 7f46b399e7c0 -1 osd.3 0 log_to_monitors true 2024-05-08T22:01:59.620 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:59.620 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:59.620 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:01:59.621 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:59 smithi131 ceph-mon[37260]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-05-08T22:01:59.621 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:01:59 smithi131 ceph-mon[37260]: from='osd.3 [v2:172.21.15.131:6810/1075134307,v1:172.21.15.131:6811/1075134307]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T22:01:59.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:59.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:01:59.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:01:59.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:59 smithi157 ceph-mon[77746]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-05-08T22:01:59.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:01:59 smithi157 ceph-mon[77746]: from='osd.3 [v2:172.21.15.131:6810/1075134307,v1:172.21.15.131:6811/1075134307]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T22:02:00.608 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T22:02:00.609 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T22:02:00.609 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2024-05-08T22:02:00.609 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2024-05-08T22:02:00.609 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113717 s, 922 MB/s 2024-05-08T22:02:00.609 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2024-05-08T22:02:00.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:00 smithi157 ceph-mon[77746]: from='osd.3 [v2:172.21.15.131:6810/1075134307,v1:172.21.15.131:6811/1075134307]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T22:02:00.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:00 smithi157 ceph-mon[77746]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T22:02:00.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:00 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:02:00.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:00 smithi157 ceph-mon[77746]: from='osd.3 [v2:172.21.15.131:6810/1075134307,v1:172.21.15.131:6811/1075134307]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-08T22:02:00.830 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:00 smithi132 ceph-mon[78180]: from='osd.3 [v2:172.21.15.131:6810/1075134307,v1:172.21.15.131:6811/1075134307]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T22:02:00.831 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:00 smithi132 ceph-mon[78180]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T22:02:00.831 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:00 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:02:00.831 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:00 smithi132 ceph-mon[78180]: from='osd.3 [v2:172.21.15.131:6810/1075134307,v1:172.21.15.131:6811/1075134307]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-08T22:02:00.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:00 smithi131 ceph-mon[37260]: from='osd.3 [v2:172.21.15.131:6810/1075134307,v1:172.21.15.131:6811/1075134307]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T22:02:00.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:00 smithi131 ceph-mon[37260]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T22:02:00.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:00 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:02:00.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:00 smithi131 ceph-mon[37260]: from='osd.3 [v2:172.21.15.131:6810/1075134307,v1:172.21.15.131:6811/1075134307]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-08T22:02:00.892 INFO:journalctl@ceph.osd.3.smithi131.stdout:May 08 22:02:00 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-3[63378]: 2024-05-08T22:02:00.493+0000 7f46a8000700 -1 osd.3 0 waiting for initial osdmap 2024-05-08T22:02:00.892 INFO:journalctl@ceph.osd.3.smithi131.stdout:May 08 22:02:00 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-3[63378]: 2024-05-08T22:02:00.500+0000 7f46a2e00700 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:02:01.035 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi132:vg_nvme/lv_3 2024-05-08T22:02:01.252 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:02:01.722 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:01 smithi132 ceph-mon[78180]: from='osd.3 [v2:172.21.15.131:6810/1075134307,v1:172.21.15.131:6811/1075134307]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-08T22:02:01.722 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:01 smithi132 ceph-mon[78180]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T22:02:01.722 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:01 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:02:01.722 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:01 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:02:01.722 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:01 smithi132 ceph-mon[78180]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:02:01.745 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:01 smithi131 ceph-mon[37260]: from='osd.3 [v2:172.21.15.131:6810/1075134307,v1:172.21.15.131:6811/1075134307]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-08T22:02:01.745 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:01 smithi131 ceph-mon[37260]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T22:02:01.745 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:01 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:02:01.745 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:01 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:02:01.745 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:01 smithi131 ceph-mon[37260]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:02:01.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:01 smithi157 ceph-mon[77746]: from='osd.3 [v2:172.21.15.131:6810/1075134307,v1:172.21.15.131:6811/1075134307]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-08T22:02:01.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:01 smithi157 ceph-mon[77746]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T22:02:01.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:01 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:02:01.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:01 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:02:01.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:01 smithi157 ceph-mon[77746]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:02:02.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.377+0000 7f4160028700 1 -- 172.21.15.132:0/2830018705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4158101060 msgr2=0x7f41581014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:02.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.377+0000 7f4160028700 1 --2- 172.21.15.132:0/2830018705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4158101060 0x7f41581014e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4148009990 tx=0x7f4148009ca0 comp rx=0 tx=0).stop 2024-05-08T22:02:02.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.377+0000 7f4160028700 1 -- 172.21.15.132:0/2830018705 shutdown_connections 2024-05-08T22:02:02.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.377+0000 7f4160028700 1 --2- 172.21.15.132:0/2830018705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4158101a20 0x7f4158108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:02.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.377+0000 7f4160028700 1 --2- 172.21.15.132:0/2830018705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41580fff00 0x7f4158100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:02.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.377+0000 7f4160028700 1 --2- 172.21.15.132:0/2830018705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4158101060 0x7f41581014e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:02.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.377+0000 7f4160028700 1 -- 172.21.15.132:0/2830018705 >> 172.21.15.132:0/2830018705 conn(0x7f41580fb4a0 msgr2=0x7f41580fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:02.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.377+0000 7f4160028700 1 -- 172.21.15.132:0/2830018705 shutdown_connections 2024-05-08T22:02:02.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.377+0000 7f4160028700 1 -- 172.21.15.132:0/2830018705 wait complete. 2024-05-08T22:02:02.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.378+0000 7f4160028700 1 Processor -- start 2024-05-08T22:02:02.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.378+0000 7f4160028700 1 -- start start 2024-05-08T22:02:02.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.378+0000 7f4160028700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41580fff00 0x7f4158196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:02.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.378+0000 7f4160028700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4158101060 0x7f4158197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:02.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.378+0000 7f4160028700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4158101a20 0x7f415819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:02.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.378+0000 7f4160028700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415819ea60 con 0x7f4158101a20 2024-05-08T22:02:02.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.378+0000 7f4160028700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415819eba0 con 0x7f4158101060 2024-05-08T22:02:02.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.378+0000 7f4160028700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415819ed30 con 0x7f41580fff00 2024-05-08T22:02:02.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.378+0000 7f415d5c3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4158101060 0x7f4158197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:02.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.378+0000 7f415d5c3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4158101060 0x7f4158197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:52452/0 (socket says 172.21.15.132:52452) 2024-05-08T22:02:02.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.378+0000 7f415d5c3700 1 -- 172.21.15.132:0/258253033 learned_addr learned my addr 172.21.15.132:0/258253033 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-05-08T22:02:02.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.378+0000 7f415ddc4700 1 --2- 172.21.15.132:0/258253033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41580fff00 0x7f4158196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:02.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.378+0000 7f415e5c5700 1 --2- 172.21.15.132:0/258253033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4158101a20 0x7f415819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:02.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.378+0000 7f415d5c3700 1 -- 172.21.15.132:0/258253033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41580fff00 msgr2=0x7f4158196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:02.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.378+0000 7f415d5c3700 1 --2- 172.21.15.132:0/258253033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41580fff00 0x7f4158196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:02.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.378+0000 7f415d5c3700 1 -- 172.21.15.132:0/258253033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4158101a20 msgr2=0x7f415819e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:02.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.378+0000 7f415d5c3700 1 --2- 172.21.15.132:0/258253033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4158101a20 0x7f415819e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:02.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.378+0000 7f415d5c3700 1 -- 172.21.15.132:0/258253033 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41480095f0 con 0x7f4158101060 2024-05-08T22:02:02.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.378+0000 7f415ddc4700 1 --2- 172.21.15.132:0/258253033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41580fff00 0x7f4158196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:02:02.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.379+0000 7f415d5c3700 1 --2- 172.21.15.132:0/258253033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4158101060 0x7f4158197460 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f414800b5c0 tx=0x7f4148009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:02.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.379+0000 7f414effd700 1 -- 172.21.15.132:0/258253033 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414801d070 con 0x7f4158101060 2024-05-08T22:02:02.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.379+0000 7f4160028700 1 -- 172.21.15.132:0/258253033 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f415819efb0 con 0x7f4158101060 2024-05-08T22:02:02.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.379+0000 7f414effd700 1 -- 172.21.15.132:0/258253033 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41480041d0 con 0x7f4158101060 2024-05-08T22:02:02.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.379+0000 7f414effd700 1 -- 172.21.15.132:0/258253033 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414800f460 con 0x7f4158101060 2024-05-08T22:02:02.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.379+0000 7f4160028700 1 -- 172.21.15.132:0/258253033 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f415819f360 con 0x7f4158101060 2024-05-08T22:02:02.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.380+0000 7f4160028700 1 -- 172.21.15.132:0/258253033 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4158107300 con 0x7f4158101060 2024-05-08T22:02:02.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.381+0000 7f414effd700 1 -- 172.21.15.132:0/258253033 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f4148022700 con 0x7f4158101060 2024-05-08T22:02:02.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.381+0000 7f414effd700 1 --2- 172.21.15.132:0/258253033 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f41440a11a0 0x7f41440a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:02.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.381+0000 7f414effd700 1 -- 172.21.15.132:0/258253033 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f41480cd030 con 0x7f4158101060 2024-05-08T22:02:02.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.382+0000 7f415ddc4700 1 --2- 172.21.15.132:0/258253033 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f41440a11a0 0x7f41440a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:02.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.382+0000 7f415ddc4700 1 --2- 172.21.15.132:0/258253033 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f41440a11a0 0x7f41440a3660 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4154005950 tx=0x7f4154005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:02.386 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.385+0000 7f414effd700 1 -- 172.21.15.132:0/258253033 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f41480979b0 con 0x7f4158101060 2024-05-08T22:02:02.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:02.584+0000 7f4160028700 1 -- 172.21.15.132:0/258253033 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f415802ceb0 con 0x7f41440a11a0 2024-05-08T22:02:02.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:02 smithi157 ceph-mon[77746]: purged_snaps scrub starts 2024-05-08T22:02:02.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:02 smithi157 ceph-mon[77746]: purged_snaps scrub ok 2024-05-08T22:02:02.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:02 smithi157 ceph-mon[77746]: osd.3 [v2:172.21.15.131:6810/1075134307,v1:172.21.15.131:6811/1075134307] boot 2024-05-08T22:02:02.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:02 smithi157 ceph-mon[77746]: osdmap e24: 4 total, 4 up, 4 in 2024-05-08T22:02:02.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:02 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:02:02.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:02 smithi157 ceph-mon[77746]: Detected new or changed devices on smithi131 2024-05-08T22:02:02.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:02 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:02.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:02 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:02.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:02 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:02.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:02 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:02.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:02 smithi157 ceph-mon[77746]: Adjusting osd_memory_target on smithi131 to 8634M 2024-05-08T22:02:02.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:02 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:02.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:02 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:02.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:02 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:02:02.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:02 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:02.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:02 smithi157 ceph-mon[77746]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:02 smithi132 ceph-mon[78180]: purged_snaps scrub starts 2024-05-08T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:02 smithi132 ceph-mon[78180]: purged_snaps scrub ok 2024-05-08T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:02 smithi132 ceph-mon[78180]: osd.3 [v2:172.21.15.131:6810/1075134307,v1:172.21.15.131:6811/1075134307] boot 2024-05-08T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:02 smithi132 ceph-mon[78180]: osdmap e24: 4 total, 4 up, 4 in 2024-05-08T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:02 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:02 smithi132 ceph-mon[78180]: Detected new or changed devices on smithi131 2024-05-08T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:02 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:02 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:02 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:02 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:02 smithi132 ceph-mon[78180]: Adjusting osd_memory_target on smithi131 to 8634M 2024-05-08T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:02 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:02.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:02 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:02.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:02 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:02:02.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:02 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:02.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:02 smithi132 ceph-mon[78180]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T22:02:02.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:02 smithi131 ceph-mon[37260]: purged_snaps scrub starts 2024-05-08T22:02:02.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:02 smithi131 ceph-mon[37260]: purged_snaps scrub ok 2024-05-08T22:02:02.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:02 smithi131 ceph-mon[37260]: osd.3 [v2:172.21.15.131:6810/1075134307,v1:172.21.15.131:6811/1075134307] boot 2024-05-08T22:02:02.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:02 smithi131 ceph-mon[37260]: osdmap e24: 4 total, 4 up, 4 in 2024-05-08T22:02:02.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:02 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:02:02.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:02 smithi131 ceph-mon[37260]: Detected new or changed devices on smithi131 2024-05-08T22:02:02.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:02 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:02.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:02 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:02.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:02 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:02.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:02 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:02.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:02 smithi131 ceph-mon[37260]: Adjusting osd_memory_target on smithi131 to 8634M 2024-05-08T22:02:02.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:02 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:02.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:02 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:02.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:02 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:02:02.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:02 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:02.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:02 smithi131 ceph-mon[37260]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T22:02:03.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:03 smithi157 ceph-mon[77746]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:02:03.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:03 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:02:03.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:03 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:02:03.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:03 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:03.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:03 smithi157 ceph-mon[77746]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 122 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:02:03.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:03 smithi157 ceph-mon[77746]: osdmap e26: 4 total, 4 up, 4 in 2024-05-08T22:02:03.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:03 smithi132 ceph-mon[78180]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:02:03.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:03 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:02:03.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:03 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:02:03.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:03 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:03.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:03 smithi132 ceph-mon[78180]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 122 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:02:03.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:03 smithi132 ceph-mon[78180]: osdmap e26: 4 total, 4 up, 4 in 2024-05-08T22:02:03.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:03 smithi131 ceph-mon[37260]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:02:03.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:03 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:02:03.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:03 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:02:03.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:03 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:03.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:03 smithi131 ceph-mon[37260]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 122 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:02:03.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:03 smithi131 ceph-mon[37260]: osdmap e26: 4 total, 4 up, 4 in 2024-05-08T22:02:05.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:05 smithi132 ceph-mon[78180]: osdmap e27: 4 total, 4 up, 4 in 2024-05-08T22:02:05.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:05 smithi132 ceph-mon[78180]: from='client.? 172.21.15.132:0/2188159919' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1592d54-01e0-4201-905f-89664e2da391"}]: dispatch 2024-05-08T22:02:05.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:05 smithi132 ceph-mon[78180]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1592d54-01e0-4201-905f-89664e2da391"}]: dispatch 2024-05-08T22:02:05.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:05 smithi132 ceph-mon[78180]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1592d54-01e0-4201-905f-89664e2da391"}]': finished 2024-05-08T22:02:05.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:05 smithi132 ceph-mon[78180]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T22:02:05.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:05 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:02:05.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:05 smithi132 ceph-mon[78180]: pgmap v90: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 181 KiB/s, 0 objects/s recovering 2024-05-08T22:02:05.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:05 smithi132 ceph-mon[78180]: from='client.? 172.21.15.132:0/623374870' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:02:05.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:05 smithi131 ceph-mon[37260]: osdmap e27: 4 total, 4 up, 4 in 2024-05-08T22:02:05.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:05 smithi131 ceph-mon[37260]: from='client.? 172.21.15.132:0/2188159919' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1592d54-01e0-4201-905f-89664e2da391"}]: dispatch 2024-05-08T22:02:05.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:05 smithi131 ceph-mon[37260]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1592d54-01e0-4201-905f-89664e2da391"}]: dispatch 2024-05-08T22:02:05.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:05 smithi131 ceph-mon[37260]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1592d54-01e0-4201-905f-89664e2da391"}]': finished 2024-05-08T22:02:05.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:05 smithi131 ceph-mon[37260]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T22:02:05.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:05 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:02:05.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:05 smithi131 ceph-mon[37260]: pgmap v90: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 181 KiB/s, 0 objects/s recovering 2024-05-08T22:02:05.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:05 smithi131 ceph-mon[37260]: from='client.? 172.21.15.132:0/623374870' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:02:05.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:05 smithi157 ceph-mon[77746]: osdmap e27: 4 total, 4 up, 4 in 2024-05-08T22:02:05.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:05 smithi157 ceph-mon[77746]: from='client.? 172.21.15.132:0/2188159919' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1592d54-01e0-4201-905f-89664e2da391"}]: dispatch 2024-05-08T22:02:05.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:05 smithi157 ceph-mon[77746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1592d54-01e0-4201-905f-89664e2da391"}]: dispatch 2024-05-08T22:02:05.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:05 smithi157 ceph-mon[77746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1592d54-01e0-4201-905f-89664e2da391"}]': finished 2024-05-08T22:02:05.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:05 smithi157 ceph-mon[77746]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T22:02:05.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:05 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:02:05.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:05 smithi157 ceph-mon[77746]: pgmap v90: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 181 KiB/s, 0 objects/s recovering 2024-05-08T22:02:05.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:05 smithi157 ceph-mon[77746]: from='client.? 172.21.15.132:0/623374870' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:02:08.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:07 smithi157 ceph-mon[77746]: pgmap v91: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-05-08T22:02:08.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:07 smithi132 ceph-mon[78180]: pgmap v91: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-05-08T22:02:08.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:07 smithi131 ceph-mon[37260]: pgmap v91: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-05-08T22:02:09.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:09 smithi132 ceph-mon[78180]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T22:02:10.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:09 smithi157 ceph-mon[77746]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T22:02:10.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:09 smithi131 ceph-mon[37260]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T22:02:11.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:10 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T22:02:11.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:10 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:11.079 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:10 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T22:02:11.079 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:10 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:11.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:10 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T22:02:11.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:10 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:11.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:11 smithi132 ceph-mon[78180]: Deploying daemon osd.4 on smithi132 2024-05-08T22:02:11.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:11 smithi132 ceph-mon[78180]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-08T22:02:12.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:11 smithi157 ceph-mon[77746]: Deploying daemon osd.4 on smithi132 2024-05-08T22:02:12.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:11 smithi157 ceph-mon[77746]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-08T22:02:12.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:11 smithi131 ceph-mon[37260]: Deploying daemon osd.4 on smithi132 2024-05-08T22:02:12.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:11 smithi131 ceph-mon[37260]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-08T22:02:14.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:13 smithi157 ceph-mon[77746]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:02:14.054 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:13 smithi132 ceph-mon[78180]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:02:14.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:13 smithi131 ceph-mon[37260]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:02:15.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:15 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:15.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:15 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:15.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:15 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:02:15.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:15 smithi157 ceph-mon[77746]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:02:15.574 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:15 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:15.574 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:15 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:15.574 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:15 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:02:15.574 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:15 smithi132 ceph-mon[78180]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:02:15.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:15 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:15.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:15 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:15.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:15 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:02:15.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:15 smithi131 ceph-mon[37260]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:02:16.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:16.791+0000 7f414effd700 1 -- 172.21.15.132:0/258253033 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f415802ceb0 con 0x7f41440a11a0 2024-05-08T22:02:16.792 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 4 on host 'smithi132' 2024-05-08T22:02:16.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:16.794+0000 7f4160028700 1 -- 172.21.15.132:0/258253033 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f41440a11a0 msgr2=0x7f41440a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:16.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:16.794+0000 7f4160028700 1 --2- 172.21.15.132:0/258253033 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f41440a11a0 0x7f41440a3660 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4154005950 tx=0x7f4154005f50 comp rx=0 tx=0).stop 2024-05-08T22:02:16.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:16.794+0000 7f4160028700 1 -- 172.21.15.132:0/258253033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4158101060 msgr2=0x7f4158197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:16.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:16.794+0000 7f4160028700 1 --2- 172.21.15.132:0/258253033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4158101060 0x7f4158197460 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f414800b5c0 tx=0x7f4148009f70 comp rx=0 tx=0).stop 2024-05-08T22:02:16.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:16.794+0000 7f4160028700 1 -- 172.21.15.132:0/258253033 shutdown_connections 2024-05-08T22:02:16.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:16.794+0000 7f4160028700 1 --2- 172.21.15.132:0/258253033 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f41440a11a0 0x7f41440a3660 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:16.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:16.794+0000 7f4160028700 1 --2- 172.21.15.132:0/258253033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4158101a20 0x7f415819e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:16.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:16.794+0000 7f4160028700 1 --2- 172.21.15.132:0/258253033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41580fff00 0x7f4158196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:16.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:16.794+0000 7f4160028700 1 --2- 172.21.15.132:0/258253033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4158101060 0x7f4158197460 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:16.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:16.794+0000 7f4160028700 1 -- 172.21.15.132:0/258253033 >> 172.21.15.132:0/258253033 conn(0x7f41580fb4a0 msgr2=0x7f4158104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:16.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:16.794+0000 7f4160028700 1 -- 172.21.15.132:0/258253033 shutdown_connections 2024-05-08T22:02:16.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:02:16.794+0000 7f4160028700 1 -- 172.21.15.132:0/258253033 wait complete. 2024-05-08T22:02:16.970 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:16 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:16.970 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:16 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:16.970 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:16 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:16.970 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:16 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:02:16.970 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:16 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:17.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:16 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:17.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:16 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:17.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:16 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:17.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:16 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:02:17.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:16 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:17.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:16 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:17.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:16 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:17.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:16 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:17.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:16 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:02:17.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:16 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:17.203 DEBUG:teuthology.orchestra.run.smithi132:osd.4> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.4.service 2024-05-08T22:02:17.206 INFO:tasks.cephadm:Deploying osd.5 on smithi157 with /dev/vg_nvme/lv_3... 2024-05-08T22:02:17.206 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T22:02:17.237 INFO:journalctl@ceph.osd.4.smithi132.stdout:-- Logs begin at Wed 2024-05-08 21:34:20 UTC. -- 2024-05-08T22:02:17.422 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:02:17.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:17 smithi132 ceph-mon[78180]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:02:17.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:17 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:17.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:17 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:17.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:17 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:02:17.897 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:17 smithi157 ceph-mon[77746]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:02:17.897 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:17 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:17.897 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:17 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:17.897 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:17 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:02:18.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:17 smithi131 ceph-mon[37260]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:02:18.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:17 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:18.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:17 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:18.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:17 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:02:18.590 INFO:journalctl@ceph.osd.4.smithi132.stdout:May 08 22:02:18 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-4[91853]: 2024-05-08T22:02:18.211+0000 7f1faca117c0 -1 osd.4 0 log_to_monitors true 2024-05-08T22:02:18.783 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T22:02:18.783 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T22:02:18.783 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-05-08T22:02:18.783 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-05-08T22:02:18.783 INFO:teuthology.orchestra.run.smithi157.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.013637 s, 769 MB/s 2024-05-08T22:02:18.783 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-05-08T22:02:19.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:18 smithi157 ceph-mon[77746]: from='osd.4 [v2:172.21.15.132:6808/1317083238,v1:172.21.15.132:6809/1317083238]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T22:02:19.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:18 smithi157 ceph-mon[77746]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T22:02:19.057 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:18 smithi132 ceph-mon[78180]: from='osd.4 [v2:172.21.15.132:6808/1317083238,v1:172.21.15.132:6809/1317083238]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T22:02:19.057 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:18 smithi132 ceph-mon[78180]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T22:02:19.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:18 smithi131 ceph-mon[37260]: from='osd.4 [v2:172.21.15.132:6808/1317083238,v1:172.21.15.132:6809/1317083238]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T22:02:19.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:18 smithi131 ceph-mon[37260]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T22:02:19.150 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_3 2024-05-08T22:02:19.368 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:02:20.090 INFO:journalctl@ceph.osd.4.smithi132.stdout:May 08 22:02:19 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-4[91853]: 2024-05-08T22:02:19.788+0000 7f1fa2075700 -1 osd.4 0 waiting for initial osdmap 2024-05-08T22:02:20.090 INFO:journalctl@ceph.osd.4.smithi132.stdout:May 08 22:02:19 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-4[91853]: 2024-05-08T22:02:19.794+0000 7f1f9de6a700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:02:20.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:19 smithi132 ceph-mon[78180]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:02:20.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:19 smithi132 ceph-mon[78180]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T22:02:20.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:19 smithi132 ceph-mon[78180]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T22:02:20.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:19 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:02:20.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:19 smithi132 ceph-mon[78180]: from='osd.4 [v2:172.21.15.132:6808/1317083238,v1:172.21.15.132:6809/1317083238]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-05-08T22:02:20.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:19 smithi132 ceph-mon[78180]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-05-08T22:02:20.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:19 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:20.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:19 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:20.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:19 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:20.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:19 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:20.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:19 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:20.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:19 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:20.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:19 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:02:20.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:19 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:19 smithi131 ceph-mon[37260]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:02:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:19 smithi131 ceph-mon[37260]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T22:02:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:19 smithi131 ceph-mon[37260]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T22:02:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:02:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:19 smithi131 ceph-mon[37260]: from='osd.4 [v2:172.21.15.132:6808/1317083238,v1:172.21.15.132:6809/1317083238]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-05-08T22:02:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:19 smithi131 ceph-mon[37260]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-05-08T22:02:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:20.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:20.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:02:20.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:20.190 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:19 smithi157 ceph-mon[77746]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:02:20.190 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:19 smithi157 ceph-mon[77746]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T22:02:20.191 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:19 smithi157 ceph-mon[77746]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T22:02:20.191 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:02:20.191 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:19 smithi157 ceph-mon[77746]: from='osd.4 [v2:172.21.15.132:6808/1317083238,v1:172.21.15.132:6809/1317083238]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-05-08T22:02:20.191 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:19 smithi157 ceph-mon[77746]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-05-08T22:02:20.191 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:20.191 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:20.191 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:20.191 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:20.191 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:20.191 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:20.191 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:02:20.191 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:20.350 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.348+0000 7fc4e1af1700 1 -- 172.21.15.157:0/407343156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4dc106cf0 msgr2=0x7fc4dc109160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:20.350 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.348+0000 7fc4e1af1700 1 --2- 172.21.15.157:0/407343156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4dc106cf0 0x7fc4dc109160 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400b0a0 tx=0x7fc4d400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:02:20.350 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.348+0000 7fc4e1af1700 1 -- 172.21.15.157:0/407343156 shutdown_connections 2024-05-08T22:02:20.350 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.348+0000 7fc4e1af1700 1 --2- 172.21.15.157:0/407343156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4dc106cf0 0x7fc4dc109160 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:20.350 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.348+0000 7fc4e1af1700 1 --2- 172.21.15.157:0/407343156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4dc101a90 0x7fc4dc103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:20.350 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.348+0000 7fc4e1af1700 1 --2- 172.21.15.157:0/407343156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4dc1043c0 0x7fc4dc1067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:20.350 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.348+0000 7fc4e1af1700 1 -- 172.21.15.157:0/407343156 >> 172.21.15.157:0/407343156 conn(0x7fc4dc0fb4a0 msgr2=0x7fc4dc0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:20.350 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.348+0000 7fc4e1af1700 1 -- 172.21.15.157:0/407343156 shutdown_connections 2024-05-08T22:02:20.350 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.348+0000 7fc4e1af1700 1 -- 172.21.15.157:0/407343156 wait complete. 2024-05-08T22:02:20.350 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.349+0000 7fc4e1af1700 1 Processor -- start 2024-05-08T22:02:20.351 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.349+0000 7fc4e1af1700 1 -- start start 2024-05-08T22:02:20.352 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.349+0000 7fc4e1af1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4dc101a90 0x7fc4dc196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:20.352 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.349+0000 7fc4e1af1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4dc1043c0 0x7fc4dc197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:20.352 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.349+0000 7fc4e1af1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4dc106cf0 0x7fc4dc19e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:20.352 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.349+0000 7fc4e1af1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc19eb10 con 0x7fc4dc101a90 2024-05-08T22:02:20.352 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.349+0000 7fc4e1af1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc19ec50 con 0x7fc4dc106cf0 2024-05-08T22:02:20.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.349+0000 7fc4e1af1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc19ede0 con 0x7fc4dc1043c0 2024-05-08T22:02:20.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.349+0000 7fc4daffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4dc1043c0 0x7fc4dc197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:20.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.349+0000 7fc4daffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4dc1043c0 0x7fc4dc197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58332/0 (socket says 172.21.15.157:58332) 2024-05-08T22:02:20.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.349+0000 7fc4daffd700 1 -- 172.21.15.157:0/2937987520 learned_addr learned my addr 172.21.15.157:0/2937987520 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-08T22:02:20.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.350+0000 7fc4daffd700 1 -- 172.21.15.157:0/2937987520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4dc106cf0 msgr2=0x7fc4dc19e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:20.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.350+0000 7fc4daffd700 1 --2- 172.21.15.157:0/2937987520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4dc106cf0 0x7fc4dc19e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:20.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.350+0000 7fc4daffd700 1 -- 172.21.15.157:0/2937987520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4dc101a90 msgr2=0x7fc4dc196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:20.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.350+0000 7fc4db7fe700 1 --2- 172.21.15.157:0/2937987520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4dc101a90 0x7fc4dc196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:20.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.350+0000 7fc4daffd700 1 --2- 172.21.15.157:0/2937987520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4dc101a90 0x7fc4dc196fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:20.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.350+0000 7fc4daffd700 1 -- 172.21.15.157:0/2937987520 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d4009d00 con 0x7fc4dc1043c0 2024-05-08T22:02:20.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.350+0000 7fc4daffd700 1 --2- 172.21.15.157:0/2937987520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4dc1043c0 0x7fc4dc197510 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000c950 tx=0x7fc4d000cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:20.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.350+0000 7fc4d8ff9700 1 -- 172.21.15.157:0/2937987520 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0007ac0 con 0x7fc4dc1043c0 2024-05-08T22:02:20.354 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.350+0000 7fc4e1af1700 1 -- 172.21.15.157:0/2937987520 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4dc19f0c0 con 0x7fc4dc1043c0 2024-05-08T22:02:20.354 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.350+0000 7fc4d8ff9700 1 -- 172.21.15.157:0/2937987520 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4d0004cf0 con 0x7fc4dc1043c0 2024-05-08T22:02:20.354 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.350+0000 7fc4d8ff9700 1 -- 172.21.15.157:0/2937987520 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0005580 con 0x7fc4dc1043c0 2024-05-08T22:02:20.354 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.350+0000 7fc4e1af1700 1 -- 172.21.15.157:0/2937987520 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4dc19f5c0 con 0x7fc4dc1043c0 2024-05-08T22:02:20.354 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.351+0000 7fc4e1af1700 1 -- 172.21.15.157:0/2937987520 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4dc0fd020 con 0x7fc4dc1043c0 2024-05-08T22:02:20.355 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.352+0000 7fc4d8ff9700 1 -- 172.21.15.157:0/2937987520 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc4d0007c20 con 0x7fc4dc1043c0 2024-05-08T22:02:20.355 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.352+0000 7fc4d8ff9700 1 --2- 172.21.15.157:0/2937987520 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc4c40a11a0 0x7fc4c40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:20.355 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.353+0000 7fc4d8ff9700 1 -- 172.21.15.157:0/2937987520 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3821+0+0 (secure 0 0 0) 0x7fc4d00cb820 con 0x7fc4dc1043c0 2024-05-08T22:02:20.355 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.353+0000 7fc4db7fe700 1 --2- 172.21.15.157:0/2937987520 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc4c40a11a0 0x7fc4c40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:20.355 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.353+0000 7fc4db7fe700 1 --2- 172.21.15.157:0/2937987520 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc4c40a11a0 0x7fc4c40a3660 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc005950 tx=0x7fc4cc0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:20.357 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.356+0000 7fc4d8ff9700 1 -- 172.21.15.157:0/2937987520 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc4d00960a0 con 0x7fc4dc1043c0 2024-05-08T22:02:20.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:20.555+0000 7fc4e1af1700 1 -- 172.21.15.157:0/2937987520 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4dc02ce10 con 0x7fc4c40a11a0 2024-05-08T22:02:21.090 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:20 smithi157 ceph-mon[77746]: Detected new or changed devices on smithi132 2024-05-08T22:02:21.090 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:20 smithi157 ceph-mon[77746]: Adjusting osd_memory_target on smithi132 to 8557M 2024-05-08T22:02:21.090 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:20 smithi157 ceph-mon[77746]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-05-08T22:02:21.090 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:20 smithi157 ceph-mon[77746]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T22:02:21.090 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:20 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:02:21.090 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:20 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:02:21.090 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:20 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:02:21.090 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:20 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:02:21.090 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:20 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:21.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:20 smithi132 ceph-mon[78180]: Detected new or changed devices on smithi132 2024-05-08T22:02:21.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:20 smithi132 ceph-mon[78180]: Adjusting osd_memory_target on smithi132 to 8557M 2024-05-08T22:02:21.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:20 smithi132 ceph-mon[78180]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-05-08T22:02:21.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:20 smithi132 ceph-mon[78180]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T22:02:21.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:20 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:02:21.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:20 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:02:21.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:20 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:02:21.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:20 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:02:21.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:20 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:21.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:20 smithi131 ceph-mon[37260]: Detected new or changed devices on smithi132 2024-05-08T22:02:21.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:20 smithi131 ceph-mon[37260]: Adjusting osd_memory_target on smithi132 to 8557M 2024-05-08T22:02:21.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:20 smithi131 ceph-mon[37260]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-05-08T22:02:21.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:20 smithi131 ceph-mon[37260]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T22:02:21.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:20 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:02:21.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:20 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:02:21.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:20 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:02:21.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:20 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:02:21.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:20 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:22.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:21 smithi132 ceph-mon[78180]: purged_snaps scrub starts 2024-05-08T22:02:22.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:21 smithi132 ceph-mon[78180]: purged_snaps scrub ok 2024-05-08T22:02:22.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:21 smithi132 ceph-mon[78180]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:02:22.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:21 smithi132 ceph-mon[78180]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:02:22.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:21 smithi132 ceph-mon[78180]: osd.4 [v2:172.21.15.132:6808/1317083238,v1:172.21.15.132:6809/1317083238] boot 2024-05-08T22:02:22.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:21 smithi132 ceph-mon[78180]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T22:02:22.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:21 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:02:22.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:21 smithi131 ceph-mon[37260]: purged_snaps scrub starts 2024-05-08T22:02:22.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:21 smithi131 ceph-mon[37260]: purged_snaps scrub ok 2024-05-08T22:02:22.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:21 smithi131 ceph-mon[37260]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:02:22.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:21 smithi131 ceph-mon[37260]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:02:22.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:21 smithi131 ceph-mon[37260]: osd.4 [v2:172.21.15.132:6808/1317083238,v1:172.21.15.132:6809/1317083238] boot 2024-05-08T22:02:22.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:21 smithi131 ceph-mon[37260]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T22:02:22.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:21 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:02:22.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:21 smithi157 ceph-mon[77746]: purged_snaps scrub starts 2024-05-08T22:02:22.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:21 smithi157 ceph-mon[77746]: purged_snaps scrub ok 2024-05-08T22:02:22.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:21 smithi157 ceph-mon[77746]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:02:22.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:21 smithi157 ceph-mon[77746]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:02:22.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:21 smithi157 ceph-mon[77746]: osd.4 [v2:172.21.15.132:6808/1317083238,v1:172.21.15.132:6809/1317083238] boot 2024-05-08T22:02:22.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:21 smithi157 ceph-mon[77746]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T22:02:22.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:21 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:02:23.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:22 smithi132 ceph-mon[78180]: osdmap e32: 5 total, 5 up, 5 in 2024-05-08T22:02:23.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:22 smithi132 ceph-mon[78180]: from='client.? 172.21.15.157:0/209931262' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d54acce-6950-42eb-b65b-97931044a3f4"}]: dispatch 2024-05-08T22:02:23.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:22 smithi132 ceph-mon[78180]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d54acce-6950-42eb-b65b-97931044a3f4"}]: dispatch 2024-05-08T22:02:23.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:22 smithi132 ceph-mon[78180]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d54acce-6950-42eb-b65b-97931044a3f4"}]': finished 2024-05-08T22:02:23.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:22 smithi132 ceph-mon[78180]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T22:02:23.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:22 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:02:23.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:22 smithi131 ceph-mon[37260]: osdmap e32: 5 total, 5 up, 5 in 2024-05-08T22:02:23.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:22 smithi131 ceph-mon[37260]: from='client.? 172.21.15.157:0/209931262' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d54acce-6950-42eb-b65b-97931044a3f4"}]: dispatch 2024-05-08T22:02:23.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:22 smithi131 ceph-mon[37260]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d54acce-6950-42eb-b65b-97931044a3f4"}]: dispatch 2024-05-08T22:02:23.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:22 smithi131 ceph-mon[37260]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d54acce-6950-42eb-b65b-97931044a3f4"}]': finished 2024-05-08T22:02:23.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:22 smithi131 ceph-mon[37260]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T22:02:23.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:22 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:02:23.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:22 smithi157 ceph-mon[77746]: osdmap e32: 5 total, 5 up, 5 in 2024-05-08T22:02:23.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:22 smithi157 ceph-mon[77746]: from='client.? 172.21.15.157:0/209931262' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d54acce-6950-42eb-b65b-97931044a3f4"}]: dispatch 2024-05-08T22:02:23.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:22 smithi157 ceph-mon[77746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d54acce-6950-42eb-b65b-97931044a3f4"}]: dispatch 2024-05-08T22:02:23.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:22 smithi157 ceph-mon[77746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d54acce-6950-42eb-b65b-97931044a3f4"}]': finished 2024-05-08T22:02:23.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:22 smithi157 ceph-mon[77746]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T22:02:23.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:22 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:02:24.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:23 smithi132 ceph-mon[78180]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:24.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:23 smithi132 ceph-mon[78180]: from='client.? 172.21.15.157:0/1063026796' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:02:24.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:23 smithi131 ceph-mon[37260]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:24.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:23 smithi131 ceph-mon[37260]: from='client.? 172.21.15.157:0/1063026796' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:02:24.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:23 smithi157 ceph-mon[77746]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:24.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:23 smithi157 ceph-mon[77746]: from='client.? 172.21.15.157:0/1063026796' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:02:26.075 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:25 smithi157 ceph-mon[77746]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:26.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:25 smithi132 ceph-mon[78180]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:26.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:25 smithi131 ceph-mon[37260]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:28.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:27 smithi132 ceph-mon[78180]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:28.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:27 smithi131 ceph-mon[37260]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:28.196 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:27 smithi157 ceph-mon[77746]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:29.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:28 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T22:02:29.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:28 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:29.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:28 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T22:02:29.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:28 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:29.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:28 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T22:02:29.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:28 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:30.063 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:29 smithi157 ceph-mon[77746]: Deploying daemon osd.5 on smithi157 2024-05-08T22:02:30.064 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:29 smithi157 ceph-mon[77746]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:30.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:29 smithi132 ceph-mon[78180]: Deploying daemon osd.5 on smithi157 2024-05-08T22:02:30.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:29 smithi132 ceph-mon[78180]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:30.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:29 smithi131 ceph-mon[37260]: Deploying daemon osd.5 on smithi157 2024-05-08T22:02:30.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:29 smithi131 ceph-mon[37260]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:32.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:31 smithi132 ceph-mon[78180]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:32.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:31 smithi131 ceph-mon[37260]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:32.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:31 smithi157 ceph-mon[77746]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:33.202 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:33 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:33.202 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:33 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:33.202 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:33 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:02:33.202 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:33 smithi157 ceph-mon[77746]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:33.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:33 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:33.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:33 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:33.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:33 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:02:33.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:33 smithi132 ceph-mon[78180]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:33.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:33 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:33.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:33 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:33.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:33 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:02:33.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:33 smithi131 ceph-mon[37260]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:34.265 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:34.263+0000 7fc4d8ff9700 1 -- 172.21.15.157:0/2937987520 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc4dc02ce10 con 0x7fc4c40a11a0 2024-05-08T22:02:34.265 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 5 on host 'smithi157' 2024-05-08T22:02:34.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:34.266+0000 7fc4e1af1700 1 -- 172.21.15.157:0/2937987520 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc4c40a11a0 msgr2=0x7fc4c40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:34.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:34.266+0000 7fc4e1af1700 1 --2- 172.21.15.157:0/2937987520 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc4c40a11a0 0x7fc4c40a3660 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc005950 tx=0x7fc4cc0058e0 comp rx=0 tx=0).stop 2024-05-08T22:02:34.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:34.266+0000 7fc4e1af1700 1 -- 172.21.15.157:0/2937987520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4dc1043c0 msgr2=0x7fc4dc197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:34.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:34.266+0000 7fc4e1af1700 1 --2- 172.21.15.157:0/2937987520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4dc1043c0 0x7fc4dc197510 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000c950 tx=0x7fc4d000cd10 comp rx=0 tx=0).stop 2024-05-08T22:02:34.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:34.266+0000 7fc4e1af1700 1 -- 172.21.15.157:0/2937987520 shutdown_connections 2024-05-08T22:02:34.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:34.266+0000 7fc4e1af1700 1 --2- 172.21.15.157:0/2937987520 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc4c40a11a0 0x7fc4c40a3660 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:34.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:34.266+0000 7fc4e1af1700 1 --2- 172.21.15.157:0/2937987520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4dc106cf0 0x7fc4dc19e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:34.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:34.266+0000 7fc4e1af1700 1 --2- 172.21.15.157:0/2937987520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4dc101a90 0x7fc4dc196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:34.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:34.266+0000 7fc4e1af1700 1 --2- 172.21.15.157:0/2937987520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4dc1043c0 0x7fc4dc197510 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:34.267 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:34.266+0000 7fc4e1af1700 1 -- 172.21.15.157:0/2937987520 >> 172.21.15.157:0/2937987520 conn(0x7fc4dc0fb4a0 msgr2=0x7fc4dc1023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:34.268 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:34.266+0000 7fc4e1af1700 1 -- 172.21.15.157:0/2937987520 shutdown_connections 2024-05-08T22:02:34.268 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:02:34.266+0000 7fc4e1af1700 1 -- 172.21.15.157:0/2937987520 wait complete. 2024-05-08T22:02:34.648 DEBUG:teuthology.orchestra.run.smithi157:osd.5> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.5.service 2024-05-08T22:02:34.651 INFO:tasks.cephadm:Deploying osd.6 on smithi131 with /dev/vg_nvme/lv_2... 2024-05-08T22:02:34.651 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T22:02:34.681 INFO:journalctl@ceph.osd.5.smithi157.stdout:-- Logs begin at Wed 2024-05-08 21:35:00 UTC. -- 2024-05-08T22:02:35.014 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:02:35.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:34 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:35.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:34 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:35.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:34 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:35.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:34 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:02:35.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:34 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:35.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:34 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:35.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:34 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:02:35.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:34 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:35.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:34 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:35.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:34 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:35.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:34 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:35.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:34 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:02:35.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:34 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:35.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:34 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:35.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:34 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:02:35.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:34 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:35.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:34 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:35.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:34 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:35.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:34 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:35.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:34 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:02:35.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:34 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:35.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:34 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:35.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:34 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:02:35.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:34 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:35 smithi131 ceph-mon[37260]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:36.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:35 smithi132 ceph-mon[78180]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:36.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:35 smithi157 ceph-mon[77746]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:36.277 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 08 22:02:36 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-5[91506]: 2024-05-08T22:02:36.022+0000 7f174e1d27c0 -1 osd.5 0 log_to_monitors true 2024-05-08T22:02:36.903 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T22:02:36.904 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T22:02:36.904 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-05-08T22:02:36.904 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-05-08T22:02:36.904 INFO:teuthology.orchestra.run.smithi131.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0410088 s, 256 MB/s 2024-05-08T22:02:36.904 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 2024-05-08T22:02:36.904 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-05-08T22:02:37.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:36 smithi132 ceph-mon[78180]: from='osd.5 [v2:172.21.15.157:6808/1488005925,v1:172.21.15.157:6809/1488005925]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T22:02:37.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:36 smithi132 ceph-mon[78180]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T22:02:37.114 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:36 smithi131 ceph-mon[37260]: from='osd.5 [v2:172.21.15.157:6808/1488005925,v1:172.21.15.157:6809/1488005925]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T22:02:37.114 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:36 smithi131 ceph-mon[37260]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T22:02:37.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:36 smithi157 ceph-mon[77746]: from='osd.5 [v2:172.21.15.157:6808/1488005925,v1:172.21.15.157:6809/1488005925]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T22:02:37.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:36 smithi157 ceph-mon[77746]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T22:02:37.348 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_2 2024-05-08T22:02:37.701 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:02:38.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:37 smithi132 ceph-mon[78180]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:38.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:37 smithi132 ceph-mon[78180]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T22:02:38.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:37 smithi132 ceph-mon[78180]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T22:02:38.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:37 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:02:38.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:37 smithi132 ceph-mon[78180]: from='osd.5 [v2:172.21.15.157:6808/1488005925,v1:172.21.15.157:6809/1488005925]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-08T22:02:38.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:37 smithi132 ceph-mon[78180]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-08T22:02:38.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:37 smithi132 ceph-mon[78180]: Detected new or changed devices on smithi157 2024-05-08T22:02:38.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:37 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:38.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:37 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:38.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:37 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:38.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:37 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:38.092 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:37 smithi132 ceph-mon[78180]: Adjusting osd_memory_target on smithi157 to 8557M 2024-05-08T22:02:38.092 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:37 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:38.092 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:37 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:38.092 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:37 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:02:38.092 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:37 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:37 smithi131 ceph-mon[37260]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:37 smithi131 ceph-mon[37260]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T22:02:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:37 smithi131 ceph-mon[37260]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T22:02:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:37 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:02:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:37 smithi131 ceph-mon[37260]: from='osd.5 [v2:172.21.15.157:6808/1488005925,v1:172.21.15.157:6809/1488005925]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-08T22:02:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:37 smithi131 ceph-mon[37260]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-08T22:02:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:37 smithi131 ceph-mon[37260]: Detected new or changed devices on smithi157 2024-05-08T22:02:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:37 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:37 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:37 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:37 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:38.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:37 smithi131 ceph-mon[37260]: Adjusting osd_memory_target on smithi157 to 8557M 2024-05-08T22:02:38.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:37 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:38.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:37 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:38.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:37 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:02:38.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:37 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:38.264 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:37 smithi157 ceph-mon[77746]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:38.264 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:37 smithi157 ceph-mon[77746]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T22:02:38.265 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:37 smithi157 ceph-mon[77746]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T22:02:38.265 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:37 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:02:38.265 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:37 smithi157 ceph-mon[77746]: from='osd.5 [v2:172.21.15.157:6808/1488005925,v1:172.21.15.157:6809/1488005925]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-08T22:02:38.265 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:37 smithi157 ceph-mon[77746]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-08T22:02:38.265 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:37 smithi157 ceph-mon[77746]: Detected new or changed devices on smithi157 2024-05-08T22:02:38.265 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:37 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:38.265 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:37 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:38.266 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:37 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:38.266 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:37 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:02:38.266 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:37 smithi157 ceph-mon[77746]: Adjusting osd_memory_target on smithi157 to 8557M 2024-05-08T22:02:38.266 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:37 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:38.266 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:37 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:38.266 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:37 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:02:38.267 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:37 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:38.526 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 08 22:02:38 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-5[91506]: 2024-05-08T22:02:38.262+0000 7f1743836700 -1 osd.5 0 waiting for initial osdmap 2024-05-08T22:02:38.526 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 08 22:02:38 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-5[91506]: 2024-05-08T22:02:38.269+0000 7f173b623700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:02:38.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.980+0000 7ff839000700 1 -- 172.21.15.131:0/333590687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff834102200 msgr2=0x7ff834102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:38.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.980+0000 7ff839000700 1 --2- 172.21.15.131:0/333590687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff834102200 0x7ff834102660 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff8200099c0 tx=0x7ff820009cd0 comp rx=0 tx=0).stop 2024-05-08T22:02:38.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.981+0000 7ff839000700 1 -- 172.21.15.131:0/333590687 shutdown_connections 2024-05-08T22:02:38.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.981+0000 7ff839000700 1 --2- 172.21.15.131:0/333590687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff834102ba0 0x7ff834109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:38.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.981+0000 7ff839000700 1 --2- 172.21.15.131:0/333590687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff834101000 0x7ff834101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:38.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.981+0000 7ff839000700 1 --2- 172.21.15.131:0/333590687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff834102200 0x7ff834102660 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:38.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.981+0000 7ff839000700 1 -- 172.21.15.131:0/333590687 >> 172.21.15.131:0/333590687 conn(0x7ff8340fc5a0 msgr2=0x7ff8340fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:38.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.981+0000 7ff839000700 1 -- 172.21.15.131:0/333590687 shutdown_connections 2024-05-08T22:02:38.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.981+0000 7ff839000700 1 -- 172.21.15.131:0/333590687 wait complete. 2024-05-08T22:02:38.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.982+0000 7ff839000700 1 Processor -- start 2024-05-08T22:02:38.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.982+0000 7ff839000700 1 -- start start 2024-05-08T22:02:38.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.982+0000 7ff839000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff834101000 0x7ff834196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:38.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.982+0000 7ff839000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff834102200 0x7ff834197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:38.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.982+0000 7ff839000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff834102ba0 0x7ff83419e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:38.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.982+0000 7ff839000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83419ea60 con 0x7ff834102200 2024-05-08T22:02:38.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.982+0000 7ff839000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83419eba0 con 0x7ff834101000 2024-05-08T22:02:38.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.982+0000 7ff839000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83419ed30 con 0x7ff834102ba0 2024-05-08T22:02:38.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.982+0000 7ff831400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff834102200 0x7ff834197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:38.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.982+0000 7ff831e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff834101000 0x7ff834196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:38.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.982+0000 7ff831e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff834101000 0x7ff834196f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:55472/0 (socket says 172.21.15.131:55472) 2024-05-08T22:02:38.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.982+0000 7ff831e00700 1 -- 172.21.15.131:0/443778346 learned_addr learned my addr 172.21.15.131:0/443778346 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:02:38.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.982+0000 7ff832800700 1 --2- 172.21.15.131:0/443778346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff834102ba0 0x7ff83419e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:38.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.983+0000 7ff831e00700 1 -- 172.21.15.131:0/443778346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff834102ba0 msgr2=0x7ff83419e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:38.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.983+0000 7ff831e00700 1 --2- 172.21.15.131:0/443778346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff834102ba0 0x7ff83419e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:38.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.983+0000 7ff831e00700 1 -- 172.21.15.131:0/443778346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff834102200 msgr2=0x7ff834197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:38.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.983+0000 7ff831e00700 1 --2- 172.21.15.131:0/443778346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff834102200 0x7ff834197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:38.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.983+0000 7ff831e00700 1 -- 172.21.15.131:0/443778346 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8200095f0 con 0x7ff834101000 2024-05-08T22:02:38.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.983+0000 7ff831e00700 1 --2- 172.21.15.131:0/443778346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff834101000 0x7ff834196f20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff82400e970 tx=0x7ff82400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:38.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.983+0000 7ff82aa00700 1 -- 172.21.15.131:0/443778346 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff824009790 con 0x7ff834101000 2024-05-08T22:02:38.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.983+0000 7ff82aa00700 1 -- 172.21.15.131:0/443778346 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff82400c7e0 con 0x7ff834101000 2024-05-08T22:02:38.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.983+0000 7ff82aa00700 1 -- 172.21.15.131:0/443778346 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff824003c70 con 0x7ff834101000 2024-05-08T22:02:38.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.983+0000 7ff839000700 1 -- 172.21.15.131:0/443778346 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff83419efb0 con 0x7ff834101000 2024-05-08T22:02:38.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.983+0000 7ff839000700 1 -- 172.21.15.131:0/443778346 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff83419f3d0 con 0x7ff834101000 2024-05-08T22:02:38.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.984+0000 7ff839000700 1 -- 172.21.15.131:0/443778346 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8341084b0 con 0x7ff834101000 2024-05-08T22:02:38.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.986+0000 7ff82aa00700 1 -- 172.21.15.131:0/443778346 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff82400c950 con 0x7ff834101000 2024-05-08T22:02:38.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.987+0000 7ff82aa00700 1 --2- 172.21.15.131:0/443778346 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff8180a11a0 0x7ff8180a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:02:38.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.988+0000 7ff82aa00700 1 -- 172.21.15.131:0/443778346 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4566+0+0 (secure 0 0 0) 0x7ff824014070 con 0x7ff834101000 2024-05-08T22:02:38.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.988+0000 7ff831400700 1 --2- 172.21.15.131:0/443778346 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff8180a11a0 0x7ff8180a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:02:38.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.988+0000 7ff831400700 1 --2- 172.21.15.131:0/443778346 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff8180a11a0 0x7ff8180a3660 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff820006010 tx=0x7ff82000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:02:38.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:38.990+0000 7ff82aa00700 1 -- 172.21.15.131:0/443778346 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff8240968c0 con 0x7ff834101000 2024-05-08T22:02:39.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:38 smithi132 ceph-mon[78180]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-08T22:02:39.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:38 smithi132 ceph-mon[78180]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T22:02:39.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:38 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:02:39.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:38 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:02:39.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:38 smithi132 ceph-mon[78180]: from='osd.5 ' entity='osd.5' 2024-05-08T22:02:39.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:38 smithi131 ceph-mon[37260]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-08T22:02:39.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:38 smithi131 ceph-mon[37260]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T22:02:39.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:38 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:02:39.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:38 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:02:39.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:38 smithi131 ceph-mon[37260]: from='osd.5 ' entity='osd.5' 2024-05-08T22:02:39.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:39.205+0000 7ff839000700 1 -- 172.21.15.131:0/443778346 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff83402cf90 con 0x7ff8180a11a0 2024-05-08T22:02:39.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:38 smithi157 ceph-mon[77746]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-08T22:02:39.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:38 smithi157 ceph-mon[77746]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T22:02:39.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:38 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:02:39.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:38 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:02:39.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:38 smithi157 ceph-mon[77746]: from='osd.5 ' entity='osd.5' 2024-05-08T22:02:40.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:39 smithi132 ceph-mon[78180]: purged_snaps scrub starts 2024-05-08T22:02:40.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:39 smithi132 ceph-mon[78180]: purged_snaps scrub ok 2024-05-08T22:02:40.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:39 smithi132 ceph-mon[78180]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:40.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:39 smithi132 ceph-mon[78180]: osd.5 [v2:172.21.15.157:6808/1488005925,v1:172.21.15.157:6809/1488005925] boot 2024-05-08T22:02:40.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:39 smithi132 ceph-mon[78180]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T22:02:40.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:39 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:02:40.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:39 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:02:40.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:39 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:02:40.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:39 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:40.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:39 smithi131 ceph-mon[37260]: purged_snaps scrub starts 2024-05-08T22:02:40.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:39 smithi131 ceph-mon[37260]: purged_snaps scrub ok 2024-05-08T22:02:40.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:39 smithi131 ceph-mon[37260]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:40.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:39 smithi131 ceph-mon[37260]: osd.5 [v2:172.21.15.157:6808/1488005925,v1:172.21.15.157:6809/1488005925] boot 2024-05-08T22:02:40.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:39 smithi131 ceph-mon[37260]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T22:02:40.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:39 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:02:40.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:39 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:02:40.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:39 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:02:40.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:39 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:40.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:39 smithi157 ceph-mon[77746]: purged_snaps scrub starts 2024-05-08T22:02:40.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:39 smithi157 ceph-mon[77746]: purged_snaps scrub ok 2024-05-08T22:02:40.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:39 smithi157 ceph-mon[77746]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 149 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:02:40.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:39 smithi157 ceph-mon[77746]: osd.5 [v2:172.21.15.157:6808/1488005925,v1:172.21.15.157:6809/1488005925] boot 2024-05-08T22:02:40.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:39 smithi157 ceph-mon[77746]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T22:02:40.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:39 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:02:40.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:39 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:02:40.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:39 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:02:40.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:39 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:41.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:40 smithi132 ceph-mon[78180]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:02:41.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:40 smithi132 ceph-mon[78180]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T22:02:41.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:40 smithi131 ceph-mon[37260]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:02:41.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:40 smithi131 ceph-mon[37260]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T22:02:41.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:40 smithi157 ceph-mon[77746]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:02:41.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:40 smithi157 ceph-mon[77746]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T22:02:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:41 smithi132 ceph-mon[78180]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 184 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:02:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:41 smithi132 ceph-mon[78180]: osdmap e38: 6 total, 6 up, 6 in 2024-05-08T22:02:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:41 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/599889571' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fecf0f79-034b-4ad3-a194-eb7c3ed23776"}]: dispatch 2024-05-08T22:02:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:41 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/599889571' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fecf0f79-034b-4ad3-a194-eb7c3ed23776"}]': finished 2024-05-08T22:02:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:41 smithi132 ceph-mon[78180]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T22:02:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:02:42.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:41 smithi131 ceph-mon[37260]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 184 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:02:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:41 smithi131 ceph-mon[37260]: osdmap e38: 6 total, 6 up, 6 in 2024-05-08T22:02:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:41 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/599889571' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fecf0f79-034b-4ad3-a194-eb7c3ed23776"}]: dispatch 2024-05-08T22:02:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:41 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/599889571' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fecf0f79-034b-4ad3-a194-eb7c3ed23776"}]': finished 2024-05-08T22:02:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:41 smithi131 ceph-mon[37260]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T22:02:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:02:42.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:41 smithi157 ceph-mon[77746]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 184 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:02:42.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:41 smithi157 ceph-mon[77746]: osdmap e38: 6 total, 6 up, 6 in 2024-05-08T22:02:42.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:41 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/599889571' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fecf0f79-034b-4ad3-a194-eb7c3ed23776"}]: dispatch 2024-05-08T22:02:42.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:41 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/599889571' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fecf0f79-034b-4ad3-a194-eb7c3ed23776"}]': finished 2024-05-08T22:02:42.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:41 smithi157 ceph-mon[77746]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T22:02:42.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:02:43.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:42 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3873942845' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:02:43.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:42 smithi131 ceph-mon[37260]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T22:02:43.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:42 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:02:43.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:42 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3873942845' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:02:43.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:42 smithi157 ceph-mon[77746]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T22:02:43.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:42 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:02:43.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:42 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3873942845' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:02:43.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:42 smithi132 ceph-mon[78180]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T22:02:43.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:42 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:02:44.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:43 smithi131 ceph-mon[37260]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 184 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:02:44.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:43 smithi157 ceph-mon[77746]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 184 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:02:44.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:43 smithi132 ceph-mon[78180]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 184 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:02:46.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:45 smithi131 ceph-mon[37260]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 184 MiB used, 536 GiB / 536 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-05-08T22:02:46.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:45 smithi157 ceph-mon[77746]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 184 MiB used, 536 GiB / 536 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-05-08T22:02:46.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:45 smithi132 ceph-mon[78180]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 184 MiB used, 536 GiB / 536 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-05-08T22:02:48.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:47 smithi131 ceph-mon[37260]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T22:02:48.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:47 smithi157 ceph-mon[77746]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T22:02:48.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:47 smithi132 ceph-mon[78180]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T22:02:49.129 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:48 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T22:02:49.130 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:48 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:49.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:48 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T22:02:49.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:48 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:49.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:48 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T22:02:49.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:48 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:50.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:49 smithi131 ceph-mon[37260]: Deploying daemon osd.6 on smithi131 2024-05-08T22:02:50.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:49 smithi131 ceph-mon[37260]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-08T22:02:50.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:49 smithi157 ceph-mon[77746]: Deploying daemon osd.6 on smithi131 2024-05-08T22:02:50.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:49 smithi157 ceph-mon[77746]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-08T22:02:50.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:49 smithi132 ceph-mon[78180]: Deploying daemon osd.6 on smithi131 2024-05-08T22:02:50.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:49 smithi132 ceph-mon[78180]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-08T22:02:52.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:51 smithi131 ceph-mon[37260]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-08T22:02:52.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:51 smithi157 ceph-mon[77746]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-08T22:02:52.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:51 smithi132 ceph-mon[78180]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-08T22:02:54.138 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:53 smithi131 ceph-mon[37260]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T22:02:54.138 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:53 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:54.138 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:53 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:54.138 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:53 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:02:54.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:53 smithi157 ceph-mon[77746]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T22:02:54.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:53 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:54.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:53 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:54.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:53 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:02:54.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:53 smithi132 ceph-mon[78180]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T22:02:54.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:53 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:54.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:53 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:54.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:53 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:02:56.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:55 smithi131 ceph-mon[37260]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-08T22:02:56.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:55 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:56.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:55 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:56.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:55 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:56.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:55 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:02:56.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:55 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:56.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:55 smithi157 ceph-mon[77746]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-08T22:02:56.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:55 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:56.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:55 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:56.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:55 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:56.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:55 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:02:56.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:55 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:56.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:55 smithi132 ceph-mon[78180]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-08T22:02:56.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:55 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:56.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:55 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:56.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:55 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:02:56.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:55 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:02:56.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:55 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:56.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:56.707+0000 7ff82aa00700 1 -- 172.21.15.131:0/443778346 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff83402cf90 con 0x7ff8180a11a0 2024-05-08T22:02:56.708 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 6 on host 'smithi131' 2024-05-08T22:02:56.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:56.709+0000 7ff839000700 1 -- 172.21.15.131:0/443778346 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff8180a11a0 msgr2=0x7ff8180a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:56.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:56.710+0000 7ff839000700 1 --2- 172.21.15.131:0/443778346 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff8180a11a0 0x7ff8180a3660 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff820006010 tx=0x7ff82000b540 comp rx=0 tx=0).stop 2024-05-08T22:02:56.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:56.710+0000 7ff839000700 1 -- 172.21.15.131:0/443778346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff834101000 msgr2=0x7ff834196f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:02:56.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:56.710+0000 7ff839000700 1 --2- 172.21.15.131:0/443778346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff834101000 0x7ff834196f20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff82400e970 tx=0x7ff82400ec80 comp rx=0 tx=0).stop 2024-05-08T22:02:56.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:56.710+0000 7ff839000700 1 -- 172.21.15.131:0/443778346 shutdown_connections 2024-05-08T22:02:56.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:56.710+0000 7ff839000700 1 --2- 172.21.15.131:0/443778346 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff8180a11a0 0x7ff8180a3660 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:56.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:56.710+0000 7ff839000700 1 --2- 172.21.15.131:0/443778346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff834102ba0 0x7ff83419e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:56.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:56.710+0000 7ff839000700 1 --2- 172.21.15.131:0/443778346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff834101000 0x7ff834196f20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:56.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:56.710+0000 7ff839000700 1 --2- 172.21.15.131:0/443778346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff834102200 0x7ff834197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:02:56.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:56.710+0000 7ff839000700 1 -- 172.21.15.131:0/443778346 >> 172.21.15.131:0/443778346 conn(0x7ff8340fc5a0 msgr2=0x7ff8341055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:02:56.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:56.710+0000 7ff839000700 1 -- 172.21.15.131:0/443778346 shutdown_connections 2024-05-08T22:02:56.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:02:56.710+0000 7ff839000700 1 -- 172.21.15.131:0/443778346 wait complete. 2024-05-08T22:02:57.165 DEBUG:teuthology.orchestra.run.smithi131:osd.6> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.6.service 2024-05-08T22:02:57.169 INFO:tasks.cephadm:Deploying osd.7 on smithi132 with /dev/vg_nvme/lv_2... 2024-05-08T22:02:57.170 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T22:02:57.274 INFO:journalctl@ceph.osd.6.smithi131.stdout:-- Logs begin at Wed 2024-05-08 21:41:05 UTC. -- 2024-05-08T22:02:57.415 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:02:57.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:57 smithi132 ceph-mon[78180]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:02:57.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:57 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:57.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:57 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:02:57.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:57 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:57.891 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:02:57 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:02:57.717+0000 7ffa164727c0 -1 osd.6 0 log_to_monitors true 2024-05-08T22:02:57.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:57 smithi131 ceph-mon[37260]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:02:57.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:57 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:57.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:57 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:02:57.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:57 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:58.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:57 smithi157 ceph-mon[77746]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:02:58.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:57 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:58.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:57 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:02:58.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:57 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:02:58.884 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T22:02:58.884 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T22:02:58.884 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2024-05-08T22:02:58.884 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2024-05-08T22:02:58.885 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129705 s, 808 MB/s 2024-05-08T22:02:58.885 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2024-05-08T22:02:59.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:58 smithi157 ceph-mon[77746]: from='osd.6 [v2:172.21.15.131:6818/260285513,v1:172.21.15.131:6819/260285513]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T22:02:59.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:58 smithi157 ceph-mon[77746]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T22:02:59.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:58 smithi132 ceph-mon[78180]: from='osd.6 [v2:172.21.15.131:6818/260285513,v1:172.21.15.131:6819/260285513]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T22:02:59.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:58 smithi132 ceph-mon[78180]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T22:02:59.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:58 smithi131 ceph-mon[37260]: from='osd.6 [v2:172.21.15.131:6818/260285513,v1:172.21.15.131:6819/260285513]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T22:02:59.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:58 smithi131 ceph-mon[37260]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T22:02:59.318 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi132:vg_nvme/lv_2 2024-05-08T22:02:59.536 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:02:59.886 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:02:59 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:02:59.706+0000 7ffa0aa00700 -1 osd.6 0 waiting for initial osdmap 2024-05-08T22:02:59.886 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:02:59 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:02:59.765+0000 7ffa05800700 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:02:59.887 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:59 smithi131 ceph-mon[37260]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:02:59.887 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:59 smithi131 ceph-mon[37260]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T22:02:59.887 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:59 smithi131 ceph-mon[37260]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T22:02:59.887 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:02:59.887 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:59 smithi131 ceph-mon[37260]: from='osd.6 [v2:172.21.15.131:6818/260285513,v1:172.21.15.131:6819/260285513]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-08T22:02:59.887 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:02:59 smithi131 ceph-mon[37260]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-08T22:03:00.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:59 smithi157 ceph-mon[77746]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:03:00.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:59 smithi157 ceph-mon[77746]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T22:03:00.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:59 smithi157 ceph-mon[77746]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T22:03:00.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:03:00.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:59 smithi157 ceph-mon[77746]: from='osd.6 [v2:172.21.15.131:6818/260285513,v1:172.21.15.131:6819/260285513]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-08T22:03:00.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:02:59 smithi157 ceph-mon[77746]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-08T22:03:00.054 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:59 smithi132 ceph-mon[78180]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:03:00.055 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:59 smithi132 ceph-mon[78180]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T22:03:00.055 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:59 smithi132 ceph-mon[78180]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T22:03:00.055 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:03:00.055 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:59 smithi132 ceph-mon[78180]: from='osd.6 [v2:172.21.15.131:6818/260285513,v1:172.21.15.131:6819/260285513]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-08T22:03:00.055 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:02:59 smithi132 ceph-mon[78180]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-08T22:03:00.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.573+0000 7f7effb99700 1 -- 172.21.15.132:0/2887366916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ef8102200 msgr2=0x7f7ef8102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:00.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.573+0000 7f7effb99700 1 --2- 172.21.15.132:0/2887366916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ef8102200 0x7f7ef8102660 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80099c0 tx=0x7f7ee8009cd0 comp rx=0 tx=0).stop 2024-05-08T22:03:00.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.573+0000 7f7effb99700 1 -- 172.21.15.132:0/2887366916 shutdown_connections 2024-05-08T22:03:00.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.573+0000 7f7effb99700 1 --2- 172.21.15.132:0/2887366916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ef8102ba0 0x7f7ef8109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:00.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.573+0000 7f7effb99700 1 --2- 172.21.15.132:0/2887366916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ef8101000 0x7f7ef8101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:00.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.573+0000 7f7effb99700 1 --2- 172.21.15.132:0/2887366916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ef8102200 0x7f7ef8102660 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:00.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.573+0000 7f7effb99700 1 -- 172.21.15.132:0/2887366916 >> 172.21.15.132:0/2887366916 conn(0x7f7ef80fc5a0 msgr2=0x7f7ef80fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:00.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.573+0000 7f7effb99700 1 -- 172.21.15.132:0/2887366916 shutdown_connections 2024-05-08T22:03:00.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.573+0000 7f7effb99700 1 -- 172.21.15.132:0/2887366916 wait complete. 2024-05-08T22:03:00.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.573+0000 7f7effb99700 1 Processor -- start 2024-05-08T22:03:00.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.574+0000 7f7effb99700 1 -- start start 2024-05-08T22:03:00.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.574+0000 7f7effb99700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ef8101000 0x7f7ef8197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:00.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.574+0000 7f7effb99700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ef8102200 0x7f7ef8197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:00.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.574+0000 7f7effb99700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ef8102ba0 0x7f7ef819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:00.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.574+0000 7f7effb99700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef819eb70 con 0x7f7ef8101000 2024-05-08T22:03:00.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.574+0000 7f7effb99700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef819ecb0 con 0x7f7ef8102ba0 2024-05-08T22:03:00.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.574+0000 7f7effb99700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef819ee40 con 0x7f7ef8102200 2024-05-08T22:03:00.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.574+0000 7f7efe136700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ef8102ba0 0x7f7ef819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:00.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.574+0000 7f7efe136700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ef8102ba0 0x7f7ef819e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:55186/0 (socket says 172.21.15.132:55186) 2024-05-08T22:03:00.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.574+0000 7f7efe136700 1 -- 172.21.15.132:0/2895342256 learned_addr learned my addr 172.21.15.132:0/2895342256 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-05-08T22:03:00.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.574+0000 7f7efd935700 1 --2- 172.21.15.132:0/2895342256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ef8101000 0x7f7ef8197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:00.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.574+0000 7f7efe136700 1 -- 172.21.15.132:0/2895342256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ef8102200 msgr2=0x7f7ef8197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:00.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.574+0000 7f7efe136700 1 --2- 172.21.15.132:0/2895342256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ef8102200 0x7f7ef8197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:00.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.574+0000 7f7efe136700 1 -- 172.21.15.132:0/2895342256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ef8101000 msgr2=0x7f7ef8197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:00.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.574+0000 7f7efe136700 1 --2- 172.21.15.132:0/2895342256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ef8101000 0x7f7ef8197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:00.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.574+0000 7f7efe136700 1 -- 172.21.15.132:0/2895342256 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee80095f0 con 0x7f7ef8102ba0 2024-05-08T22:03:00.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.574+0000 7f7efe136700 1 --2- 172.21.15.132:0/2895342256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ef8102ba0 0x7f7ef819e5a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000b970 tx=0x7f7ef000bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:00.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.574+0000 7f7eeeffd700 1 -- 172.21.15.132:0/2895342256 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef000c9b0 con 0x7f7ef8102ba0 2024-05-08T22:03:00.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.574+0000 7f7eeeffd700 1 -- 172.21.15.132:0/2895342256 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ef0014440 con 0x7f7ef8102ba0 2024-05-08T22:03:00.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.574+0000 7f7effb99700 1 -- 172.21.15.132:0/2895342256 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef819f120 con 0x7f7ef8102ba0 2024-05-08T22:03:00.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.574+0000 7f7eeeffd700 1 -- 172.21.15.132:0/2895342256 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef00075d0 con 0x7f7ef8102ba0 2024-05-08T22:03:00.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.574+0000 7f7effb99700 1 -- 172.21.15.132:0/2895342256 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef819f620 con 0x7f7ef8102ba0 2024-05-08T22:03:00.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.575+0000 7f7effb99700 1 -- 172.21.15.132:0/2895342256 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ef81084b0 con 0x7f7ef8102ba0 2024-05-08T22:03:00.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.576+0000 7f7eeeffd700 1 -- 172.21.15.132:0/2895342256 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f7ef00148d0 con 0x7f7ef8102ba0 2024-05-08T22:03:00.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.577+0000 7f7eeeffd700 1 --2- 172.21.15.132:0/2895342256 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7ee40a1220 0x7f7ee40a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:00.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.577+0000 7f7eeeffd700 1 -- 172.21.15.132:0/2895342256 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4685+0+0 (secure 0 0 0) 0x7f7ef0006e40 con 0x7f7ef8102ba0 2024-05-08T22:03:00.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.577+0000 7f7efd935700 1 --2- 172.21.15.132:0/2895342256 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7ee40a1220 0x7f7ee40a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:00.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.578+0000 7f7efd935700 1 --2- 172.21.15.132:0/2895342256 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7ee40a1220 0x7f7ee40a36e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4006fd0 tx=0x7f7ef4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:00.582 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.581+0000 7f7eeeffd700 1 -- 172.21.15.132:0/2895342256 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7ef0095bc0 con 0x7f7ef8102ba0 2024-05-08T22:03:00.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:00.784+0000 7f7effb99700 1 -- 172.21.15.132:0/2895342256 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ef802ceb0 con 0x7f7ee40a1220 2024-05-08T22:03:01.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:00 smithi157 ceph-mon[77746]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-08T22:03:01.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:00 smithi157 ceph-mon[77746]: osdmap e42: 7 total, 6 up, 7 in 2024-05-08T22:03:01.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:00 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:03:01.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:00 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:03:01.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:00 smithi157 ceph-mon[77746]: Detected new or changed devices on smithi131 2024-05-08T22:03:01.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:00 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:01.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:00 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:01.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:00 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:01.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:00 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:01.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:00 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:01.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:00 smithi157 ceph-mon[77746]: Adjusting osd_memory_target on smithi131 to 5756M 2024-05-08T22:03:01.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:00 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:01.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:00 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:01.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:00 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:03:01.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:00 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:01.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:00 smithi132 ceph-mon[78180]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-08T22:03:01.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:00 smithi132 ceph-mon[78180]: osdmap e42: 7 total, 6 up, 7 in 2024-05-08T22:03:01.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:00 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:03:01.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:00 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:03:01.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:00 smithi132 ceph-mon[78180]: Detected new or changed devices on smithi131 2024-05-08T22:03:01.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:00 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:01.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:00 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:01.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:00 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:01.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:00 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:01.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:00 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:01.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:00 smithi132 ceph-mon[78180]: Adjusting osd_memory_target on smithi131 to 5756M 2024-05-08T22:03:01.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:00 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:01.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:00 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:01.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:00 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:03:01.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:00 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:01.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:00 smithi131 ceph-mon[37260]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-08T22:03:01.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:00 smithi131 ceph-mon[37260]: osdmap e42: 7 total, 6 up, 7 in 2024-05-08T22:03:01.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:00 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:03:01.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:00 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:03:01.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:00 smithi131 ceph-mon[37260]: Detected new or changed devices on smithi131 2024-05-08T22:03:01.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:00 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:01.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:00 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:01.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:00 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:01.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:00 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:01.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:00 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:01.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:00 smithi131 ceph-mon[37260]: Adjusting osd_memory_target on smithi131 to 5756M 2024-05-08T22:03:01.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:00 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:01.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:00 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:01.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:00 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:03:01.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:00 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:01.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:01 smithi132 ceph-mon[78180]: purged_snaps scrub starts 2024-05-08T22:03:01.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:01 smithi132 ceph-mon[78180]: purged_snaps scrub ok 2024-05-08T22:03:01.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:01 smithi132 ceph-mon[78180]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:03:01.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:01 smithi132 ceph-mon[78180]: osd.6 [v2:172.21.15.131:6818/260285513,v1:172.21.15.131:6819/260285513] boot 2024-05-08T22:03:01.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:01 smithi132 ceph-mon[78180]: osdmap e43: 7 total, 7 up, 7 in 2024-05-08T22:03:01.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:01 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:03:01.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:01 smithi132 ceph-mon[78180]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:03:01.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:01 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:03:01.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:01 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:03:01.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:01 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:02.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:01 smithi157 ceph-mon[77746]: purged_snaps scrub starts 2024-05-08T22:03:02.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:01 smithi157 ceph-mon[77746]: purged_snaps scrub ok 2024-05-08T22:03:02.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:01 smithi157 ceph-mon[77746]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:03:02.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:01 smithi157 ceph-mon[77746]: osd.6 [v2:172.21.15.131:6818/260285513,v1:172.21.15.131:6819/260285513] boot 2024-05-08T22:03:02.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:01 smithi157 ceph-mon[77746]: osdmap e43: 7 total, 7 up, 7 in 2024-05-08T22:03:02.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:01 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:03:02.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:01 smithi157 ceph-mon[77746]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:03:02.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:01 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:03:02.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:01 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:03:02.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:01 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:01 smithi131 ceph-mon[37260]: purged_snaps scrub starts 2024-05-08T22:03:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:01 smithi131 ceph-mon[37260]: purged_snaps scrub ok 2024-05-08T22:03:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:01 smithi131 ceph-mon[37260]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:03:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:01 smithi131 ceph-mon[37260]: osd.6 [v2:172.21.15.131:6818/260285513,v1:172.21.15.131:6819/260285513] boot 2024-05-08T22:03:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:01 smithi131 ceph-mon[37260]: osdmap e43: 7 total, 7 up, 7 in 2024-05-08T22:03:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:01 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:03:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:01 smithi131 ceph-mon[37260]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:03:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:01 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:03:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:01 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:03:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:01 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:03.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:02 smithi157 ceph-mon[77746]: osdmap e44: 7 total, 7 up, 7 in 2024-05-08T22:03:03.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:02 smithi157 ceph-mon[77746]: from='client.? 172.21.15.132:0/583719196' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fbdd862-cf84-4929-a74c-5c4be1ce7d5e"}]: dispatch 2024-05-08T22:03:03.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:02 smithi157 ceph-mon[77746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fbdd862-cf84-4929-a74c-5c4be1ce7d5e"}]: dispatch 2024-05-08T22:03:03.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:02 smithi157 ceph-mon[77746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fbdd862-cf84-4929-a74c-5c4be1ce7d5e"}]': finished 2024-05-08T22:03:03.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:02 smithi157 ceph-mon[77746]: osdmap e45: 8 total, 7 up, 8 in 2024-05-08T22:03:03.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:02 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:03:03.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:02 smithi132 ceph-mon[78180]: osdmap e44: 7 total, 7 up, 7 in 2024-05-08T22:03:03.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:02 smithi132 ceph-mon[78180]: from='client.? 172.21.15.132:0/583719196' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fbdd862-cf84-4929-a74c-5c4be1ce7d5e"}]: dispatch 2024-05-08T22:03:03.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:02 smithi132 ceph-mon[78180]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fbdd862-cf84-4929-a74c-5c4be1ce7d5e"}]: dispatch 2024-05-08T22:03:03.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:02 smithi132 ceph-mon[78180]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fbdd862-cf84-4929-a74c-5c4be1ce7d5e"}]': finished 2024-05-08T22:03:03.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:02 smithi132 ceph-mon[78180]: osdmap e45: 8 total, 7 up, 8 in 2024-05-08T22:03:03.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:02 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:03:03.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:02 smithi131 ceph-mon[37260]: osdmap e44: 7 total, 7 up, 7 in 2024-05-08T22:03:03.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:02 smithi131 ceph-mon[37260]: from='client.? 172.21.15.132:0/583719196' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fbdd862-cf84-4929-a74c-5c4be1ce7d5e"}]: dispatch 2024-05-08T22:03:03.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:02 smithi131 ceph-mon[37260]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fbdd862-cf84-4929-a74c-5c4be1ce7d5e"}]: dispatch 2024-05-08T22:03:03.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:02 smithi131 ceph-mon[37260]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fbdd862-cf84-4929-a74c-5c4be1ce7d5e"}]': finished 2024-05-08T22:03:03.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:02 smithi131 ceph-mon[37260]: osdmap e45: 8 total, 7 up, 8 in 2024-05-08T22:03:03.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:02 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:03:04.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:03 smithi157 ceph-mon[77746]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 211 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:03:04.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:03 smithi157 ceph-mon[77746]: from='client.? 172.21.15.132:0/1084460106' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:03:04.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:03 smithi132 ceph-mon[78180]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 211 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:03:04.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:03 smithi132 ceph-mon[78180]: from='client.? 172.21.15.132:0/1084460106' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:03:04.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:03 smithi131 ceph-mon[37260]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 211 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:03:04.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:03 smithi131 ceph-mon[37260]: from='client.? 172.21.15.132:0/1084460106' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:03:06.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:05 smithi157 ceph-mon[77746]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:03:06.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:05 smithi132 ceph-mon[78180]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:03:06.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:05 smithi131 ceph-mon[37260]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:03:08.266 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:07 smithi132 ceph-mon[78180]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:03:08.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:07 smithi157 ceph-mon[77746]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:03:08.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:07 smithi131 ceph-mon[37260]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:03:09.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:09 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T22:03:09.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:09 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:09.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:09 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T22:03:09.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:09 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:09.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:09 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T22:03:09.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:09 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:10.094 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:10 smithi132 ceph-mon[78180]: Deploying daemon osd.7 on smithi132 2024-05-08T22:03:10.094 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:10 smithi132 ceph-mon[78180]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:03:10.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:10 smithi157 ceph-mon[77746]: Deploying daemon osd.7 on smithi132 2024-05-08T22:03:10.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:10 smithi157 ceph-mon[77746]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:03:10.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:10 smithi131 ceph-mon[37260]: Deploying daemon osd.7 on smithi132 2024-05-08T22:03:10.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:10 smithi131 ceph-mon[37260]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:03:12.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:12 smithi157 ceph-mon[77746]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:03:12.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:12 smithi132 ceph-mon[78180]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:03:12.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:12 smithi131 ceph-mon[37260]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:03:13.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:13 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:13.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:13 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:13.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:13 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:03:13.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:13 smithi157 ceph-mon[77746]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:03:13.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:13 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:13.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:13 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:13.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:13 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:03:13.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:13 smithi132 ceph-mon[78180]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:03:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:13 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:13 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:13 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:03:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:13 smithi131 ceph-mon[37260]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:03:14.843 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 7 on host 'smithi132' 2024-05-08T22:03:14.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:14.842+0000 7f7eeeffd700 1 -- 172.21.15.132:0/2895342256 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7ef802ceb0 con 0x7f7ee40a1220 2024-05-08T22:03:14.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:14.845+0000 7f7effb99700 1 -- 172.21.15.132:0/2895342256 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7ee40a1220 msgr2=0x7f7ee40a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:14.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:14.845+0000 7f7effb99700 1 --2- 172.21.15.132:0/2895342256 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7ee40a1220 0x7f7ee40a36e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4006fd0 tx=0x7f7ef4008040 comp rx=0 tx=0).stop 2024-05-08T22:03:14.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:14.845+0000 7f7effb99700 1 -- 172.21.15.132:0/2895342256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ef8102ba0 msgr2=0x7f7ef819e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:14.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:14.845+0000 7f7effb99700 1 --2- 172.21.15.132:0/2895342256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ef8102ba0 0x7f7ef819e5a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000b970 tx=0x7f7ef000bd30 comp rx=0 tx=0).stop 2024-05-08T22:03:14.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:14.845+0000 7f7effb99700 1 -- 172.21.15.132:0/2895342256 shutdown_connections 2024-05-08T22:03:14.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:14.845+0000 7f7effb99700 1 --2- 172.21.15.132:0/2895342256 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7ee40a1220 0x7f7ee40a36e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:14.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:14.845+0000 7f7effb99700 1 --2- 172.21.15.132:0/2895342256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ef8102ba0 0x7f7ef819e5a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:14.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:14.845+0000 7f7effb99700 1 --2- 172.21.15.132:0/2895342256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ef8101000 0x7f7ef8197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:14.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:14.845+0000 7f7effb99700 1 --2- 172.21.15.132:0/2895342256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ef8102200 0x7f7ef8197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:14.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:14.845+0000 7f7effb99700 1 -- 172.21.15.132:0/2895342256 >> 172.21.15.132:0/2895342256 conn(0x7f7ef80fc5a0 msgr2=0x7f7ef81055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:14.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:14.845+0000 7f7effb99700 1 -- 172.21.15.132:0/2895342256 shutdown_connections 2024-05-08T22:03:14.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:14.845+0000 7f7effb99700 1 -- 172.21.15.132:0/2895342256 wait complete. 2024-05-08T22:03:15.291 DEBUG:teuthology.orchestra.run.smithi132:osd.7> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.7.service 2024-05-08T22:03:15.294 INFO:tasks.cephadm:Deploying osd.8 on smithi157 with /dev/vg_nvme/lv_2... 2024-05-08T22:03:15.294 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T22:03:15.318 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:15 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:15.318 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:15 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:15.318 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:15 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:15.318 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:15 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:03:15.318 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:15 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:15.318 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:15 smithi132 ceph-mon[78180]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:03:15.318 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:15 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:15.318 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:15 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:15.319 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:15 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:03:15.324 INFO:journalctl@ceph.osd.7.smithi132.stdout:-- Logs begin at Wed 2024-05-08 21:34:20 UTC. -- 2024-05-08T22:03:15.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:15 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:15.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:15 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:15.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:15 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:15.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:15 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:03:15.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:15 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:15.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:15 smithi157 ceph-mon[77746]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:03:15.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:15 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:15.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:15 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:15.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:15 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:03:15.539 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:03:15.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:15 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:15.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:15 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:15.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:15 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:15.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:15 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:03:15.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:15 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:15.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:15 smithi131 ceph-mon[37260]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:03:15.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:15 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:15.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:15 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:15.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:15 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:03:16.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:16 smithi157 ceph-mon[77746]: from='osd.7 [v2:172.21.15.132:6816/464003238,v1:172.21.15.132:6817/464003238]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T22:03:16.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:16 smithi157 ceph-mon[77746]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T22:03:16.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:16 smithi132 ceph-mon[78180]: from='osd.7 [v2:172.21.15.132:6816/464003238,v1:172.21.15.132:6817/464003238]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T22:03:16.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:16 smithi132 ceph-mon[78180]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T22:03:16.840 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:03:16 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:03:16.476+0000 7f5b095447c0 -1 osd.7 0 log_to_monitors true 2024-05-08T22:03:16.841 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T22:03:16.841 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T22:03:16.841 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-05-08T22:03:16.841 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-05-08T22:03:16.841 INFO:teuthology.orchestra.run.smithi157.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00825684 s, 1.3 GB/s 2024-05-08T22:03:16.841 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-05-08T22:03:16.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:16 smithi131 ceph-mon[37260]: from='osd.7 [v2:172.21.15.132:6816/464003238,v1:172.21.15.132:6817/464003238]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T22:03:16.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:16 smithi131 ceph-mon[37260]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T22:03:17.223 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_2 2024-05-08T22:03:17.440 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:03:17.840 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:03:17 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:03:17.547+0000 7f5afeba8700 -1 osd.7 0 waiting for initial osdmap 2024-05-08T22:03:17.840 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:03:17 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:03:17.559+0000 7f5afa99d700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:03:17.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:17 smithi132 ceph-mon[78180]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T22:03:17.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:17 smithi132 ceph-mon[78180]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T22:03:17.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:17 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:03:17.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:17 smithi132 ceph-mon[78180]: from='osd.7 [v2:172.21.15.132:6816/464003238,v1:172.21.15.132:6817/464003238]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-05-08T22:03:17.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:17 smithi132 ceph-mon[78180]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-05-08T22:03:17.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:17 smithi132 ceph-mon[78180]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:03:17.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:17 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:17.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:17 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:17.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:17 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:17.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:17 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:17.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:17 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:17.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:17 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:17.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:17 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:17.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:17 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:03:17.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:17 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:17 smithi131 ceph-mon[37260]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T22:03:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:17 smithi131 ceph-mon[37260]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T22:03:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:17 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:03:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:17 smithi131 ceph-mon[37260]: from='osd.7 [v2:172.21.15.132:6816/464003238,v1:172.21.15.132:6817/464003238]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-05-08T22:03:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:17 smithi131 ceph-mon[37260]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-05-08T22:03:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:17 smithi131 ceph-mon[37260]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:03:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:17 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:17 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:17 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:17 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:17.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:17 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:17.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:17 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:17.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:17 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:17.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:17 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:03:17.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:17 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:17.893 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:17 smithi157 ceph-mon[77746]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T22:03:17.893 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:17 smithi157 ceph-mon[77746]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T22:03:17.893 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:17 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:03:17.893 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:17 smithi157 ceph-mon[77746]: from='osd.7 [v2:172.21.15.132:6816/464003238,v1:172.21.15.132:6817/464003238]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-05-08T22:03:17.894 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:17 smithi157 ceph-mon[77746]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-05-08T22:03:17.894 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:17 smithi157 ceph-mon[77746]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:03:17.894 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:17 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:17.894 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:17 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:17.894 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:17 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:17.894 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:17 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:17.894 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:17 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:17.894 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:17 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:17.894 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:17 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:17.894 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:17 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:03:17.894 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:17 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:18.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.437+0000 7f49b275a700 1 -- 172.21.15.157:0/1471836581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49ac102ba0 msgr2=0x7f49ac109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:18.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.437+0000 7f49b275a700 1 --2- 172.21.15.157:0/1471836581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49ac102ba0 0x7f49ac109450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f49a400b0a0 tx=0x7f49a400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:03:18.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.437+0000 7f49b275a700 1 -- 172.21.15.157:0/1471836581 shutdown_connections 2024-05-08T22:03:18.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.437+0000 7f49b275a700 1 --2- 172.21.15.157:0/1471836581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49ac102ba0 0x7f49ac109450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:18.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.437+0000 7f49b275a700 1 --2- 172.21.15.157:0/1471836581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49ac101000 0x7f49ac101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:18.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.437+0000 7f49b275a700 1 --2- 172.21.15.157:0/1471836581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49ac102200 0x7f49ac102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:18.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.437+0000 7f49b275a700 1 -- 172.21.15.157:0/1471836581 >> 172.21.15.157:0/1471836581 conn(0x7f49ac0fc5a0 msgr2=0x7f49ac0fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:18.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.437+0000 7f49b275a700 1 -- 172.21.15.157:0/1471836581 shutdown_connections 2024-05-08T22:03:18.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.437+0000 7f49b275a700 1 -- 172.21.15.157:0/1471836581 wait complete. 2024-05-08T22:03:18.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.438+0000 7f49b275a700 1 Processor -- start 2024-05-08T22:03:18.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.438+0000 7f49b275a700 1 -- start start 2024-05-08T22:03:18.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.438+0000 7f49b275a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49ac101000 0x7f49ac197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:18.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.438+0000 7f49b275a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49ac102200 0x7f49ac197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:18.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.438+0000 7f49b275a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49ac102ba0 0x7f49ac19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:18.441 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.438+0000 7f49b275a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac19eb90 con 0x7f49ac101000 2024-05-08T22:03:18.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.438+0000 7f49b275a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac19ecd0 con 0x7f49ac102200 2024-05-08T22:03:18.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.438+0000 7f49b275a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac19ee60 con 0x7f49ac102ba0 2024-05-08T22:03:18.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.438+0000 7f49b0cf7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49ac102ba0 0x7f49ac19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:18.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.438+0000 7f49b0cf7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49ac102ba0 0x7f49ac19e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54294/0 (socket says 172.21.15.157:54294) 2024-05-08T22:03:18.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.438+0000 7f49b0cf7700 1 -- 172.21.15.157:0/3636296645 learned_addr learned my addr 172.21.15.157:0/3636296645 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-08T22:03:18.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.438+0000 7f49ab7fe700 1 --2- 172.21.15.157:0/3636296645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49ac102200 0x7f49ac197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:18.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.438+0000 7f49b0cf7700 1 -- 172.21.15.157:0/3636296645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49ac102200 msgr2=0x7f49ac197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:18.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.438+0000 7f49b0cf7700 1 --2- 172.21.15.157:0/3636296645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49ac102200 0x7f49ac197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:18.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.438+0000 7f49b0cf7700 1 -- 172.21.15.157:0/3636296645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49ac101000 msgr2=0x7f49ac197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:18.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.438+0000 7f49b0cf7700 1 --2- 172.21.15.157:0/3636296645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49ac101000 0x7f49ac197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:18.443 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.438+0000 7f49b0cf7700 1 -- 172.21.15.157:0/3636296645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a4009d00 con 0x7f49ac102ba0 2024-05-08T22:03:18.443 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.439+0000 7f49b0cf7700 1 --2- 172.21.15.157:0/3636296645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49ac102ba0 0x7f49ac19e5c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f49a4004d30 tx=0x7f49a4004d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:18.443 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.439+0000 7f49a97fa700 1 -- 172.21.15.157:0/3636296645 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a400e040 con 0x7f49ac102ba0 2024-05-08T22:03:18.443 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.439+0000 7f49b275a700 1 -- 172.21.15.157:0/3636296645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49ac19f0e0 con 0x7f49ac102ba0 2024-05-08T22:03:18.443 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.439+0000 7f49a97fa700 1 -- 172.21.15.157:0/3636296645 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49a400ce50 con 0x7f49ac102ba0 2024-05-08T22:03:18.443 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.439+0000 7f49a97fa700 1 -- 172.21.15.157:0/3636296645 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a401bb60 con 0x7f49ac102ba0 2024-05-08T22:03:18.443 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.439+0000 7f49b275a700 1 -- 172.21.15.157:0/3636296645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ac19f5a0 con 0x7f49ac102ba0 2024-05-08T22:03:18.444 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.440+0000 7f49b275a700 1 -- 172.21.15.157:0/3636296645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49ac108490 con 0x7f49ac102ba0 2024-05-08T22:03:18.444 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.441+0000 7f49a97fa700 1 -- 172.21.15.157:0/3636296645 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f49a401bcc0 con 0x7f49ac102ba0 2024-05-08T22:03:18.444 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.441+0000 7f49a97fa700 1 --2- 172.21.15.157:0/3636296645 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f49940a1120 0x7f49940a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:18.444 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.441+0000 7f49a97fa700 1 -- 172.21.15.157:0/3636296645 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5117+0+0 (secure 0 0 0) 0x7f49a40cd930 con 0x7f49ac102ba0 2024-05-08T22:03:18.444 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.441+0000 7f49abfff700 1 --2- 172.21.15.157:0/3636296645 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f49940a1120 0x7f49940a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:18.444 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.442+0000 7f49abfff700 1 --2- 172.21.15.157:0/3636296645 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f49940a1120 0x7f49940a35e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f49ac102060 tx=0x7f499c009470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:18.447 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.446+0000 7f49a97fa700 1 -- 172.21.15.157:0/3636296645 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f49a4097dd0 con 0x7f49ac102ba0 2024-05-08T22:03:18.652 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:18.651+0000 7f49b275a700 1 -- 172.21.15.157:0/3636296645 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f49ac02ce60 con 0x7f49940a1120 2024-05-08T22:03:18.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:18 smithi132 ceph-mon[78180]: Detected new or changed devices on smithi132 2024-05-08T22:03:18.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:18 smithi132 ceph-mon[78180]: Adjusting osd_memory_target on smithi132 to 5704M 2024-05-08T22:03:18.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:18 smithi132 ceph-mon[78180]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-05-08T22:03:18.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:18 smithi132 ceph-mon[78180]: osdmap e47: 8 total, 7 up, 8 in 2024-05-08T22:03:18.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:18 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:03:18.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:18 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:03:18.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:18 smithi131 ceph-mon[37260]: Detected new or changed devices on smithi132 2024-05-08T22:03:18.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:18 smithi131 ceph-mon[37260]: Adjusting osd_memory_target on smithi132 to 5704M 2024-05-08T22:03:18.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:18 smithi131 ceph-mon[37260]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-05-08T22:03:18.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:18 smithi131 ceph-mon[37260]: osdmap e47: 8 total, 7 up, 8 in 2024-05-08T22:03:18.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:18 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:03:18.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:18 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:03:19.016 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:18 smithi157 ceph-mon[77746]: Detected new or changed devices on smithi132 2024-05-08T22:03:19.016 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:18 smithi157 ceph-mon[77746]: Adjusting osd_memory_target on smithi132 to 5704M 2024-05-08T22:03:19.016 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:18 smithi157 ceph-mon[77746]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-05-08T22:03:19.017 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:18 smithi157 ceph-mon[77746]: osdmap e47: 8 total, 7 up, 8 in 2024-05-08T22:03:19.017 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:18 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:03:19.017 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:18 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:03:19.717 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:19 smithi157 ceph-mon[77746]: purged_snaps scrub starts 2024-05-08T22:03:19.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:19 smithi157 ceph-mon[77746]: purged_snaps scrub ok 2024-05-08T22:03:19.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:19 smithi157 ceph-mon[77746]: osd.7 [v2:172.21.15.132:6816/464003238,v1:172.21.15.132:6817/464003238] boot 2024-05-08T22:03:19.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:19 smithi157 ceph-mon[77746]: osdmap e48: 8 total, 8 up, 8 in 2024-05-08T22:03:19.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:03:19.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:19 smithi157 ceph-mon[77746]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:03:19.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:03:19.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:03:19.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:19.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:19 smithi157 ceph-mon[77746]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:19.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:19 smithi157 ceph-mon[77746]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T22:03:19.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:19 smithi132 ceph-mon[78180]: purged_snaps scrub starts 2024-05-08T22:03:19.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:19 smithi132 ceph-mon[78180]: purged_snaps scrub ok 2024-05-08T22:03:19.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:19 smithi132 ceph-mon[78180]: osd.7 [v2:172.21.15.132:6816/464003238,v1:172.21.15.132:6817/464003238] boot 2024-05-08T22:03:19.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:19 smithi132 ceph-mon[78180]: osdmap e48: 8 total, 8 up, 8 in 2024-05-08T22:03:19.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:19 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:03:19.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:19 smithi132 ceph-mon[78180]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:03:19.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:19 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:03:19.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:19 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:03:19.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:19 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:19.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:19 smithi132 ceph-mon[78180]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:19.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:19 smithi132 ceph-mon[78180]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T22:03:19.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:19 smithi131 ceph-mon[37260]: purged_snaps scrub starts 2024-05-08T22:03:19.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:19 smithi131 ceph-mon[37260]: purged_snaps scrub ok 2024-05-08T22:03:19.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:19 smithi131 ceph-mon[37260]: osd.7 [v2:172.21.15.132:6816/464003238,v1:172.21.15.132:6817/464003238] boot 2024-05-08T22:03:19.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:19 smithi131 ceph-mon[37260]: osdmap e48: 8 total, 8 up, 8 in 2024-05-08T22:03:19.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:03:19.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:19 smithi131 ceph-mon[37260]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:03:19.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:03:19.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:03:19.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:19.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:19 smithi131 ceph-mon[37260]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:19.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:19 smithi131 ceph-mon[37260]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T22:03:20.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:20 smithi132 ceph-mon[78180]: from='client.? 172.21.15.157:0/3358999763' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47077c11-3248-442f-b15c-692931137f8a"}]: dispatch 2024-05-08T22:03:20.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:20 smithi132 ceph-mon[78180]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47077c11-3248-442f-b15c-692931137f8a"}]: dispatch 2024-05-08T22:03:20.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:20 smithi132 ceph-mon[78180]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47077c11-3248-442f-b15c-692931137f8a"}]': finished 2024-05-08T22:03:20.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:20 smithi132 ceph-mon[78180]: osdmap e50: 9 total, 8 up, 9 in 2024-05-08T22:03:20.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:20 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:03:20.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:20 smithi131 ceph-mon[37260]: from='client.? 172.21.15.157:0/3358999763' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47077c11-3248-442f-b15c-692931137f8a"}]: dispatch 2024-05-08T22:03:20.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:20 smithi131 ceph-mon[37260]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47077c11-3248-442f-b15c-692931137f8a"}]: dispatch 2024-05-08T22:03:20.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:20 smithi131 ceph-mon[37260]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47077c11-3248-442f-b15c-692931137f8a"}]': finished 2024-05-08T22:03:20.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:20 smithi131 ceph-mon[37260]: osdmap e50: 9 total, 8 up, 9 in 2024-05-08T22:03:20.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:20 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:03:21.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:20 smithi157 ceph-mon[77746]: from='client.? 172.21.15.157:0/3358999763' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47077c11-3248-442f-b15c-692931137f8a"}]: dispatch 2024-05-08T22:03:21.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:20 smithi157 ceph-mon[77746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47077c11-3248-442f-b15c-692931137f8a"}]: dispatch 2024-05-08T22:03:21.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:20 smithi157 ceph-mon[77746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47077c11-3248-442f-b15c-692931137f8a"}]': finished 2024-05-08T22:03:21.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:20 smithi157 ceph-mon[77746]: osdmap e50: 9 total, 8 up, 9 in 2024-05-08T22:03:21.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:20 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:03:21.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:21 smithi132 ceph-mon[78180]: pgmap v150: 1 pgs: 1 remapped+peering; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:21.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:21 smithi132 ceph-mon[78180]: from='client.? 172.21.15.157:0/794957529' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:03:21.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:21 smithi132 ceph-mon[78180]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T22:03:21.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:21 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:03:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:21 smithi131 ceph-mon[37260]: pgmap v150: 1 pgs: 1 remapped+peering; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:21 smithi131 ceph-mon[37260]: from='client.? 172.21.15.157:0/794957529' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:03:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:21 smithi131 ceph-mon[37260]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T22:03:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:21 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:03:22.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:21 smithi157 ceph-mon[77746]: pgmap v150: 1 pgs: 1 remapped+peering; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:22.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:21 smithi157 ceph-mon[77746]: from='client.? 172.21.15.157:0/794957529' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:03:22.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:21 smithi157 ceph-mon[77746]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T22:03:22.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:21 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:03:23.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:23 smithi157 ceph-mon[77746]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T22:03:23.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:23 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:03:23.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:23 smithi157 ceph-mon[77746]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:23.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:23 smithi132 ceph-mon[78180]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T22:03:23.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:23 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:03:23.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:23 smithi132 ceph-mon[78180]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:23.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:23 smithi131 ceph-mon[37260]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T22:03:23.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:23 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:03:23.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:23 smithi131 ceph-mon[37260]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:26.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:25 smithi157 ceph-mon[77746]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:26.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:25 smithi132 ceph-mon[78180]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:26.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:25 smithi131 ceph-mon[37260]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:27.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:26 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T22:03:27.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:26 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:27.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:26 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T22:03:27.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:26 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:27.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:26 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T22:03:27.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:26 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:27.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:27 smithi157 ceph-mon[77746]: Deploying daemon osd.8 on smithi157 2024-05-08T22:03:27.982 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:27 smithi157 ceph-mon[77746]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:28.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:27 smithi132 ceph-mon[78180]: Deploying daemon osd.8 on smithi157 2024-05-08T22:03:28.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:27 smithi132 ceph-mon[78180]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:28.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:27 smithi131 ceph-mon[37260]: Deploying daemon osd.8 on smithi157 2024-05-08T22:03:28.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:27 smithi131 ceph-mon[37260]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:30.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:29 smithi157 ceph-mon[77746]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:30.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:29 smithi132 ceph-mon[78180]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:30.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:29 smithi131 ceph-mon[37260]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:31.494 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:31 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:31.494 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:31 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:31.495 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:31 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:03:31.495 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:31 smithi157 ceph-mon[77746]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:31.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:31 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:31.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:31 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:31.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:31 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:03:31.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:31 smithi132 ceph-mon[78180]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:31.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:31 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:31.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:31 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:31.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:31 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:03:31.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:31 smithi131 ceph-mon[37260]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:32.494 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:32 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:32.494 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:32 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:32.494 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:32 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:32.495 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:32 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:03:32.495 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:32 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:32.619 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:32.617+0000 7f49a97fa700 1 -- 172.21.15.157:0/3636296645 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f49ac02ce60 con 0x7f49940a1120 2024-05-08T22:03:32.619 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 8 on host 'smithi157' 2024-05-08T22:03:32.621 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:32.620+0000 7f49b275a700 1 -- 172.21.15.157:0/3636296645 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f49940a1120 msgr2=0x7f49940a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:32.621 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:32.620+0000 7f49b275a700 1 --2- 172.21.15.157:0/3636296645 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f49940a1120 0x7f49940a35e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f49ac102060 tx=0x7f499c009470 comp rx=0 tx=0).stop 2024-05-08T22:03:32.621 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:32.620+0000 7f49b275a700 1 -- 172.21.15.157:0/3636296645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49ac102ba0 msgr2=0x7f49ac19e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:32.621 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:32.620+0000 7f49b275a700 1 --2- 172.21.15.157:0/3636296645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49ac102ba0 0x7f49ac19e5c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f49a4004d30 tx=0x7f49a4004d60 comp rx=0 tx=0).stop 2024-05-08T22:03:32.621 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:32.620+0000 7f49b275a700 1 -- 172.21.15.157:0/3636296645 shutdown_connections 2024-05-08T22:03:32.621 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:32.620+0000 7f49b275a700 1 --2- 172.21.15.157:0/3636296645 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f49940a1120 0x7f49940a35e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:32.622 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:32.620+0000 7f49b275a700 1 --2- 172.21.15.157:0/3636296645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49ac102ba0 0x7f49ac19e5c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:32.622 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:32.620+0000 7f49b275a700 1 --2- 172.21.15.157:0/3636296645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49ac101000 0x7f49ac197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:32.622 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:32.620+0000 7f49b275a700 1 --2- 172.21.15.157:0/3636296645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49ac102200 0x7f49ac197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:32.622 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:32.620+0000 7f49b275a700 1 -- 172.21.15.157:0/3636296645 >> 172.21.15.157:0/3636296645 conn(0x7f49ac0fc5a0 msgr2=0x7f49ac1055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:32.622 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:32.620+0000 7f49b275a700 1 -- 172.21.15.157:0/3636296645 shutdown_connections 2024-05-08T22:03:32.622 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:03:32.620+0000 7f49b275a700 1 -- 172.21.15.157:0/3636296645 wait complete. 2024-05-08T22:03:32.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:32 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:32.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:32 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:32.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:32 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:32.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:32 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:03:32.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:32 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:32.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:32 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:32.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:32 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:32.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:32 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:32.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:32 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:03:32.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:32 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:32.993 DEBUG:teuthology.orchestra.run.smithi157:osd.8> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.8.service 2024-05-08T22:03:32.995 INFO:tasks.cephadm:Deploying osd.9 on smithi131 with /dev/vg_nvme/lv_1... 2024-05-08T22:03:32.996 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T22:03:33.028 INFO:journalctl@ceph.osd.8.smithi157.stdout:-- Logs begin at Wed 2024-05-08 21:35:00 UTC. -- 2024-05-08T22:03:33.362 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:03:33.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:33 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:33.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:33 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:33.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:33 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:03:33.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:33 smithi157 ceph-mon[77746]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:33.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:33 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:33.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:33 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:33.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:33 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:03:33.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:33 smithi132 ceph-mon[78180]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:33.888 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:33 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:33.889 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:33 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:33.889 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:33 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:03:33.889 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:33 smithi131 ceph-mon[37260]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:34.526 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:03:34 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:03:34.182+0000 7f0cdae9e7c0 -1 osd.8 0 log_to_monitors true 2024-05-08T22:03:34.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:34 smithi132 ceph-mon[78180]: from='osd.8 [v2:172.21.15.157:6816/3594097089,v1:172.21.15.157:6817/3594097089]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T22:03:34.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:34 smithi132 ceph-mon[78180]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T22:03:34.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:34 smithi131 ceph-mon[37260]: from='osd.8 [v2:172.21.15.157:6816/3594097089,v1:172.21.15.157:6817/3594097089]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T22:03:34.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:34 smithi131 ceph-mon[37260]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T22:03:34.999 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:34 smithi157 ceph-mon[77746]: from='osd.8 [v2:172.21.15.157:6816/3594097089,v1:172.21.15.157:6817/3594097089]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T22:03:34.999 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:34 smithi157 ceph-mon[77746]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T22:03:35.249 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T22:03:35.249 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T22:03:35.249 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-05-08T22:03:35.249 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-05-08T22:03:35.249 INFO:teuthology.orchestra.run.smithi131.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0533858 s, 196 MB/s 2024-05-08T22:03:35.249 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 2024-05-08T22:03:35.249 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-05-08T22:03:35.721 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_1 2024-05-08T22:03:35.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:35 smithi132 ceph-mon[78180]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T22:03:35.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:35 smithi132 ceph-mon[78180]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T22:03:35.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:03:35.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:35 smithi132 ceph-mon[78180]: from='osd.8 [v2:172.21.15.157:6816/3594097089,v1:172.21.15.157:6817/3594097089]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-08T22:03:35.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:35 smithi132 ceph-mon[78180]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-08T22:03:35.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:35 smithi132 ceph-mon[78180]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:35.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:35.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:35.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:35.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:35.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:35.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:35.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:35.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:03:35.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:35 smithi131 ceph-mon[37260]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T22:03:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:35 smithi131 ceph-mon[37260]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T22:03:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:03:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:35 smithi131 ceph-mon[37260]: from='osd.8 [v2:172.21.15.157:6816/3594097089,v1:172.21.15.157:6817/3594097089]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-08T22:03:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:35 smithi131 ceph-mon[37260]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-08T22:03:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:35 smithi131 ceph-mon[37260]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:35.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:03:35.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:36.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:03:35 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:03:35.591+0000 7f0cd0502700 -1 osd.8 0 waiting for initial osdmap 2024-05-08T22:03:36.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:03:35 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:03:35.602+0000 7f0ccc2f7700 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:03:36.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:35 smithi157 ceph-mon[77746]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T22:03:36.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:35 smithi157 ceph-mon[77746]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T22:03:36.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:03:36.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:35 smithi157 ceph-mon[77746]: from='osd.8 [v2:172.21.15.157:6816/3594097089,v1:172.21.15.157:6817/3594097089]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-08T22:03:36.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:35 smithi157 ceph-mon[77746]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-08T22:03:36.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:35 smithi157 ceph-mon[77746]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:36.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:36.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:36.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:36.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:36.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:36.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:36.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:36.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:03:36.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:36.076 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:03:36.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:36 smithi131 ceph-mon[37260]: Detected new or changed devices on smithi157 2024-05-08T22:03:36.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:36 smithi131 ceph-mon[37260]: Adjusting osd_memory_target on smithi157 to 5704M 2024-05-08T22:03:36.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:36 smithi131 ceph-mon[37260]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-08T22:03:36.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:36 smithi131 ceph-mon[37260]: osdmap e54: 9 total, 8 up, 9 in 2024-05-08T22:03:36.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:36 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:03:36.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:36 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:03:37.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:36 smithi157 ceph-mon[77746]: Detected new or changed devices on smithi157 2024-05-08T22:03:37.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:36 smithi157 ceph-mon[77746]: Adjusting osd_memory_target on smithi157 to 5704M 2024-05-08T22:03:37.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:36 smithi157 ceph-mon[77746]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-08T22:03:37.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:36 smithi157 ceph-mon[77746]: osdmap e54: 9 total, 8 up, 9 in 2024-05-08T22:03:37.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:36 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:03:37.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:36 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:03:37.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:36 smithi132 ceph-mon[78180]: Detected new or changed devices on smithi157 2024-05-08T22:03:37.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:36 smithi132 ceph-mon[78180]: Adjusting osd_memory_target on smithi157 to 5704M 2024-05-08T22:03:37.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:36 smithi132 ceph-mon[78180]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-08T22:03:37.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:36 smithi132 ceph-mon[78180]: osdmap e54: 9 total, 8 up, 9 in 2024-05-08T22:03:37.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:36 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:03:37.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:36 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:03:37.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.430+0000 7f1dde000700 1 -- 172.21.15.131:0/1051278342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dd810d5e0 msgr2=0x7f1dd810f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:37.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.430+0000 7f1dde000700 1 --2- 172.21.15.131:0/1051278342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dd810d5e0 0x7f1dd810f9d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1dd000b0a0 tx=0x7f1dd000b3b0 comp rx=0 tx=0).stop 2024-05-08T22:03:37.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.430+0000 7f1dde000700 1 -- 172.21.15.131:0/1051278342 shutdown_connections 2024-05-08T22:03:37.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.430+0000 7f1dde000700 1 --2- 172.21.15.131:0/1051278342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dd810d5e0 0x7f1dd810f9d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:37.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.430+0000 7f1dde000700 1 --2- 172.21.15.131:0/1051278342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dd8074fd0 0x7f1dd8073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:37.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.430+0000 7f1dde000700 1 --2- 172.21.15.131:0/1051278342 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dd8073a00 0x7f1dd8073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:37.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.430+0000 7f1dde000700 1 -- 172.21.15.131:0/1051278342 >> 172.21.15.131:0/1051278342 conn(0x7f1dd80fc5a0 msgr2=0x7f1dd80fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:37.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.430+0000 7f1dde000700 1 -- 172.21.15.131:0/1051278342 shutdown_connections 2024-05-08T22:03:37.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.431+0000 7f1dde000700 1 -- 172.21.15.131:0/1051278342 wait complete. 2024-05-08T22:03:37.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.432+0000 7f1dde000700 1 Processor -- start 2024-05-08T22:03:37.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.432+0000 7f1dde000700 1 -- start start 2024-05-08T22:03:37.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.432+0000 7f1dde000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dd8073a00 0x7f1dd819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:37.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.433+0000 7f1dde000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dd8074fd0 0x7f1dd819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:37.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.433+0000 7f1dde000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dd810d5e0 0x7f1dd81a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:37.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.433+0000 7f1dde000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd81a51c0 con 0x7f1dd810d5e0 2024-05-08T22:03:37.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.433+0000 7f1dde000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd81a5300 con 0x7f1dd8074fd0 2024-05-08T22:03:37.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.433+0000 7f1dde000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd81a5490 con 0x7f1dd8073a00 2024-05-08T22:03:37.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.433+0000 7f1dd7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dd8073a00 0x7f1dd819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:37.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.433+0000 7f1dd7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dd810d5e0 0x7f1dd81a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:37.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.433+0000 7f1dd7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dd8073a00 0x7f1dd819d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:58560/0 (socket says 172.21.15.131:58560) 2024-05-08T22:03:37.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.433+0000 7f1dd7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dd810d5e0 0x7f1dd81a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51988/0 (socket says 172.21.15.131:51988) 2024-05-08T22:03:37.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.433+0000 7f1dd7400700 1 -- 172.21.15.131:0/3747564753 learned_addr learned my addr 172.21.15.131:0/3747564753 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:03:37.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.433+0000 7f1dd6a00700 1 --2- 172.21.15.131:0/3747564753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dd8074fd0 0x7f1dd819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:37.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.433+0000 7f1dd7400700 1 -- 172.21.15.131:0/3747564753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dd8074fd0 msgr2=0x7f1dd819dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:37.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.433+0000 7f1dd7400700 1 --2- 172.21.15.131:0/3747564753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dd8074fd0 0x7f1dd819dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:37.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.433+0000 7f1dd7400700 1 -- 172.21.15.131:0/3747564753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dd810d5e0 msgr2=0x7f1dd81a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:37.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.433+0000 7f1dd7400700 1 --2- 172.21.15.131:0/3747564753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dd810d5e0 0x7f1dd81a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:37.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.433+0000 7f1dd7400700 1 -- 172.21.15.131:0/3747564753 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd0009d00 con 0x7f1dd8073a00 2024-05-08T22:03:37.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.433+0000 7f1dd7e00700 1 --2- 172.21.15.131:0/3747564753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dd810d5e0 0x7f1dd81a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:03:37.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.434+0000 7f1dd7400700 1 --2- 172.21.15.131:0/3747564753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dd8073a00 0x7f1dd819d680 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800e9a0 tx=0x7f1dc800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:37.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.434+0000 7f1dcfe00700 1 -- 172.21.15.131:0/3747564753 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8009790 con 0x7f1dd8073a00 2024-05-08T22:03:37.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.434+0000 7f1dde000700 1 -- 172.21.15.131:0/3747564753 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd81a5770 con 0x7f1dd8073a00 2024-05-08T22:03:37.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.434+0000 7f1dcfe00700 1 -- 172.21.15.131:0/3747564753 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dc800c810 con 0x7f1dd8073a00 2024-05-08T22:03:37.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.434+0000 7f1dcfe00700 1 -- 172.21.15.131:0/3747564753 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc80054c0 con 0x7f1dd8073a00 2024-05-08T22:03:37.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.434+0000 7f1dde000700 1 -- 172.21.15.131:0/3747564753 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd81a5c70 con 0x7f1dd8073a00 2024-05-08T22:03:37.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.435+0000 7f1dde000700 1 -- 172.21.15.131:0/3747564753 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd8103840 con 0x7f1dd8073a00 2024-05-08T22:03:37.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.437+0000 7f1dcfe00700 1 -- 172.21.15.131:0/3747564753 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1dc800cc80 con 0x7f1dd8073a00 2024-05-08T22:03:37.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.438+0000 7f1dcfe00700 1 --2- 172.21.15.131:0/3747564753 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1dbc0a1120 0x7f1dbc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:37.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.438+0000 7f1dcfe00700 1 -- 172.21.15.131:0/3747564753 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f1dc8014070 con 0x7f1dd8073a00 2024-05-08T22:03:37.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.438+0000 7f1dd6a00700 1 --2- 172.21.15.131:0/3747564753 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1dbc0a1120 0x7f1dbc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:37.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.439+0000 7f1dd6a00700 1 --2- 172.21.15.131:0/3747564753 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1dbc0a1120 0x7f1dbc0a35e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8074d00 tx=0x7f1dc4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:37.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.442+0000 7f1dcfe00700 1 -- 172.21.15.131:0/3747564753 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1dc8096290 con 0x7f1dd8073a00 2024-05-08T22:03:37.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:37.659+0000 7f1dde000700 1 -- 172.21.15.131:0/3747564753 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1dd802ce10 con 0x7f1dbc0a1120 2024-05-08T22:03:37.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:37 smithi131 ceph-mon[37260]: purged_snaps scrub starts 2024-05-08T22:03:37.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:37 smithi131 ceph-mon[37260]: purged_snaps scrub ok 2024-05-08T22:03:37.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:37 smithi131 ceph-mon[37260]: osd.8 [v2:172.21.15.157:6816/3594097089,v1:172.21.15.157:6817/3594097089] boot 2024-05-08T22:03:37.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:37 smithi131 ceph-mon[37260]: osdmap e55: 9 total, 9 up, 9 in 2024-05-08T22:03:37.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:37 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:03:37.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:37 smithi131 ceph-mon[37260]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:37.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:37 smithi131 ceph-mon[37260]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T22:03:38.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:37 smithi157 ceph-mon[77746]: purged_snaps scrub starts 2024-05-08T22:03:38.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:37 smithi157 ceph-mon[77746]: purged_snaps scrub ok 2024-05-08T22:03:38.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:37 smithi157 ceph-mon[77746]: osd.8 [v2:172.21.15.157:6816/3594097089,v1:172.21.15.157:6817/3594097089] boot 2024-05-08T22:03:38.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:37 smithi157 ceph-mon[77746]: osdmap e55: 9 total, 9 up, 9 in 2024-05-08T22:03:38.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:37 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:03:38.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:37 smithi157 ceph-mon[77746]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:38.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:37 smithi157 ceph-mon[77746]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T22:03:38.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:37 smithi132 ceph-mon[78180]: purged_snaps scrub starts 2024-05-08T22:03:38.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:37 smithi132 ceph-mon[78180]: purged_snaps scrub ok 2024-05-08T22:03:38.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:37 smithi132 ceph-mon[78180]: osd.8 [v2:172.21.15.157:6816/3594097089,v1:172.21.15.157:6817/3594097089] boot 2024-05-08T22:03:38.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:37 smithi132 ceph-mon[78180]: osdmap e55: 9 total, 9 up, 9 in 2024-05-08T22:03:38.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:37 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:03:38.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:37 smithi132 ceph-mon[78180]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:03:38.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:37 smithi132 ceph-mon[78180]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T22:03:38.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:38 smithi131 ceph-mon[37260]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:03:38.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:38 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:03:38.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:38 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:03:38.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:38 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:38.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:38 smithi131 ceph-mon[37260]: osdmap e57: 9 total, 9 up, 9 in 2024-05-08T22:03:39.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:38 smithi157 ceph-mon[77746]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:03:39.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:38 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:03:39.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:38 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:03:39.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:38 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:39.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:38 smithi157 ceph-mon[77746]: osdmap e57: 9 total, 9 up, 9 in 2024-05-08T22:03:39.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:38 smithi132 ceph-mon[78180]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:03:39.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:38 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:03:39.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:38 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:03:39.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:38 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:39.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:38 smithi132 ceph-mon[78180]: osdmap e57: 9 total, 9 up, 9 in 2024-05-08T22:03:39.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:39 smithi131 ceph-mon[37260]: pgmap v166: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T22:03:39.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:39 smithi131 ceph-mon[37260]: osdmap e58: 9 total, 9 up, 9 in 2024-05-08T22:03:40.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:39 smithi157 ceph-mon[77746]: pgmap v166: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T22:03:40.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:39 smithi157 ceph-mon[77746]: osdmap e58: 9 total, 9 up, 9 in 2024-05-08T22:03:40.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:39 smithi132 ceph-mon[78180]: pgmap v166: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T22:03:40.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:39 smithi132 ceph-mon[78180]: osdmap e58: 9 total, 9 up, 9 in 2024-05-08T22:03:40.891 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:03:40 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:03:40.664+0000 7f9d9a400700 -1 calc_pg_upmaps abort due to max <= 0 2024-05-08T22:03:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:40 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/304986406' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b027ba0a-8aec-45af-8137-b261c1588499"}]: dispatch 2024-05-08T22:03:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:40 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/304986406' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b027ba0a-8aec-45af-8137-b261c1588499"}]': finished 2024-05-08T22:03:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:40 smithi131 ceph-mon[37260]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T22:03:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:03:41.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:40 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/304986406' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b027ba0a-8aec-45af-8137-b261c1588499"}]: dispatch 2024-05-08T22:03:41.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:40 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/304986406' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b027ba0a-8aec-45af-8137-b261c1588499"}]': finished 2024-05-08T22:03:41.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:40 smithi157 ceph-mon[77746]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T22:03:41.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:03:41.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:40 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/304986406' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b027ba0a-8aec-45af-8137-b261c1588499"}]: dispatch 2024-05-08T22:03:41.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:40 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/304986406' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b027ba0a-8aec-45af-8137-b261c1588499"}]': finished 2024-05-08T22:03:41.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:40 smithi132 ceph-mon[78180]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T22:03:41.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:03:41.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:41 smithi131 ceph-mon[37260]: pgmap v169: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T22:03:41.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:41 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2019114392' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:03:42.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:41 smithi157 ceph-mon[77746]: pgmap v169: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T22:03:42.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:41 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2019114392' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:03:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:41 smithi132 ceph-mon[78180]: pgmap v169: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T22:03:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:41 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2019114392' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:03:44.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:43 smithi157 ceph-mon[77746]: pgmap v170: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-05-08T22:03:44.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:43 smithi132 ceph-mon[78180]: pgmap v170: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-05-08T22:03:44.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:43 smithi131 ceph-mon[37260]: pgmap v170: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-05-08T22:03:46.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:45 smithi157 ceph-mon[77746]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-05-08T22:03:46.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:45 smithi132 ceph-mon[78180]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-05-08T22:03:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:45 smithi131 ceph-mon[37260]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-05-08T22:03:48.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:47 smithi157 ceph-mon[77746]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:03:48.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:47 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T22:03:48.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:47 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:48.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:47 smithi132 ceph-mon[78180]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:03:48.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:47 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T22:03:48.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:47 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:48.120 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:47 smithi131 ceph-mon[37260]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:03:48.120 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:47 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T22:03:48.120 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:47 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:49.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:48 smithi157 ceph-mon[77746]: Deploying daemon osd.9 on smithi131 2024-05-08T22:03:49.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:48 smithi132 ceph-mon[78180]: Deploying daemon osd.9 on smithi131 2024-05-08T22:03:49.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:48 smithi131 ceph-mon[37260]: Deploying daemon osd.9 on smithi131 2024-05-08T22:03:49.793 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:49 smithi131 ceph-mon[37260]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:03:50.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:49 smithi157 ceph-mon[77746]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:03:50.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:49 smithi132 ceph-mon[78180]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:03:52.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:51 smithi157 ceph-mon[77746]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:03:52.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:51 smithi132 ceph-mon[78180]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:03:52.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:51 smithi131 ceph-mon[37260]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:03:53.694 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:53 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:53.694 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:53 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:53.694 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:53 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:03:53.694 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:53 smithi131 ceph-mon[37260]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:03:53.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:53 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:53.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:53 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:53.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:53 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:03:53.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:53 smithi157 ceph-mon[77746]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:03:53.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:53 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:53.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:53 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:53.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:53 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:03:53.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:53 smithi132 ceph-mon[78180]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:03:55.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:55.460+0000 7f1dcfe00700 1 -- 172.21.15.131:0/3747564753 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1dd802ce10 con 0x7f1dbc0a1120 2024-05-08T22:03:55.461 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 9 on host 'smithi131' 2024-05-08T22:03:55.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:55.462+0000 7f1dde000700 1 -- 172.21.15.131:0/3747564753 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1dbc0a1120 msgr2=0x7f1dbc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:55.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:55.462+0000 7f1dde000700 1 --2- 172.21.15.131:0/3747564753 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1dbc0a1120 0x7f1dbc0a35e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8074d00 tx=0x7f1dc4009260 comp rx=0 tx=0).stop 2024-05-08T22:03:55.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:55.462+0000 7f1dde000700 1 -- 172.21.15.131:0/3747564753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dd8073a00 msgr2=0x7f1dd819d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:55.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:55.462+0000 7f1dde000700 1 --2- 172.21.15.131:0/3747564753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dd8073a00 0x7f1dd819d680 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800e9a0 tx=0x7f1dc800ecb0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:55.463+0000 7f1dde000700 1 -- 172.21.15.131:0/3747564753 shutdown_connections 2024-05-08T22:03:55.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:55.463+0000 7f1dde000700 1 --2- 172.21.15.131:0/3747564753 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1dbc0a1120 0x7f1dbc0a35e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:55.463+0000 7f1dde000700 1 --2- 172.21.15.131:0/3747564753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dd810d5e0 0x7f1dd81a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:55.463+0000 7f1dde000700 1 --2- 172.21.15.131:0/3747564753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dd8073a00 0x7f1dd819d680 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:55.463+0000 7f1dde000700 1 --2- 172.21.15.131:0/3747564753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dd8074fd0 0x7f1dd819dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:55.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:55.463+0000 7f1dde000700 1 -- 172.21.15.131:0/3747564753 >> 172.21.15.131:0/3747564753 conn(0x7f1dd80fc5a0 msgr2=0x7f1dd8100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:55.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:55.463+0000 7f1dde000700 1 -- 172.21.15.131:0/3747564753 shutdown_connections 2024-05-08T22:03:55.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:03:55.463+0000 7f1dde000700 1 -- 172.21.15.131:0/3747564753 wait complete. 2024-05-08T22:03:55.675 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:55 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:55.675 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:55 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:55.675 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:55 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:55.675 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:55 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:03:55.675 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:55 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:55.676 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:55 smithi131 ceph-mon[37260]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:03:55.676 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:55 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:55.676 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:55 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:55.676 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:55 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:03:55.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:55 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:55.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:55 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:55.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:55 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:55.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:55 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:03:55.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:55 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:55.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:55 smithi132 ceph-mon[78180]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:03:55.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:55 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:55.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:55 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:55.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:55 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:03:55.850 DEBUG:teuthology.orchestra.run.smithi131:osd.9> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.9.service 2024-05-08T22:03:55.854 INFO:tasks.cephadm:Deploying osd.10 on smithi132 with /dev/vg_nvme/lv_1... 2024-05-08T22:03:55.854 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T22:03:55.930 INFO:journalctl@ceph.osd.9.smithi131.stdout:-- Logs begin at Wed 2024-05-08 21:41:05 UTC. -- 2024-05-08T22:03:56.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:55 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:56.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:55 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:56.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:55 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:56.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:55 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:03:56.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:55 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:56.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:55 smithi157 ceph-mon[77746]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:03:56.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:55 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:56.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:55 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:56.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:55 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:03:56.087 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:03:56.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:56 smithi132 ceph-mon[78180]: from='osd.9 [v2:172.21.15.131:6826/4217805693,v1:172.21.15.131:6827/4217805693]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T22:03:56.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:56 smithi131 ceph-mon[37260]: from='osd.9 [v2:172.21.15.131:6826/4217805693,v1:172.21.15.131:6827/4217805693]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T22:03:56.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:03:56 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:03:56.578+0000 7f400b3d87c0 -1 osd.9 0 log_to_monitors true 2024-05-08T22:03:57.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:56 smithi157 ceph-mon[77746]: from='osd.9 [v2:172.21.15.131:6826/4217805693,v1:172.21.15.131:6827/4217805693]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T22:03:57.446 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T22:03:57.446 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T22:03:57.446 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2024-05-08T22:03:57.446 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2024-05-08T22:03:57.446 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0141463 s, 741 MB/s 2024-05-08T22:03:57.446 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2024-05-08T22:03:57.774 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:57 smithi132 ceph-mon[78180]: from='osd.9 [v2:172.21.15.131:6826/4217805693,v1:172.21.15.131:6827/4217805693]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T22:03:57.774 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:57 smithi132 ceph-mon[78180]: osdmap e60: 10 total, 9 up, 10 in 2024-05-08T22:03:57.774 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:57 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:03:57.774 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:57 smithi132 ceph-mon[78180]: from='osd.9 [v2:172.21.15.131:6826/4217805693,v1:172.21.15.131:6827/4217805693]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-08T22:03:57.774 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:57 smithi132 ceph-mon[78180]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:03:57.774 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:57 smithi132 ceph-mon[78180]: from='osd.9 [v2:172.21.15.131:6826/4217805693,v1:172.21.15.131:6827/4217805693]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-08T22:03:57.774 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:57 smithi132 ceph-mon[78180]: osdmap e61: 10 total, 9 up, 10 in 2024-05-08T22:03:57.774 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:57 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:03:57.793 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi132:vg_nvme/lv_1 2024-05-08T22:03:58.013 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:03:58.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:57 smithi157 ceph-mon[77746]: from='osd.9 [v2:172.21.15.131:6826/4217805693,v1:172.21.15.131:6827/4217805693]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T22:03:58.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:57 smithi157 ceph-mon[77746]: osdmap e60: 10 total, 9 up, 10 in 2024-05-08T22:03:58.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:57 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:03:58.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:57 smithi157 ceph-mon[77746]: from='osd.9 [v2:172.21.15.131:6826/4217805693,v1:172.21.15.131:6827/4217805693]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-08T22:03:58.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:57 smithi157 ceph-mon[77746]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:03:58.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:57 smithi157 ceph-mon[77746]: from='osd.9 [v2:172.21.15.131:6826/4217805693,v1:172.21.15.131:6827/4217805693]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-08T22:03:58.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:57 smithi157 ceph-mon[77746]: osdmap e61: 10 total, 9 up, 10 in 2024-05-08T22:03:58.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:57 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:03:58.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:03:57 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:03:57.659+0000 7f3fffa00700 -1 osd.9 0 waiting for initial osdmap 2024-05-08T22:03:58.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:03:57 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:03:57.675+0000 7f3ffa800700 -1 osd.9 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:03:58.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:57 smithi131 ceph-mon[37260]: from='osd.9 [v2:172.21.15.131:6826/4217805693,v1:172.21.15.131:6827/4217805693]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T22:03:58.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:57 smithi131 ceph-mon[37260]: osdmap e60: 10 total, 9 up, 10 in 2024-05-08T22:03:58.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:57 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:03:58.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:57 smithi131 ceph-mon[37260]: from='osd.9 [v2:172.21.15.131:6826/4217805693,v1:172.21.15.131:6827/4217805693]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-08T22:03:58.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:57 smithi131 ceph-mon[37260]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:03:58.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:57 smithi131 ceph-mon[37260]: from='osd.9 [v2:172.21.15.131:6826/4217805693,v1:172.21.15.131:6827/4217805693]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-08T22:03:58.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:57 smithi131 ceph-mon[37260]: osdmap e61: 10 total, 9 up, 10 in 2024-05-08T22:03:58.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:57 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:03:58.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:58 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:03:59.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:58 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:03:59.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.058+0000 7faba4eac700 1 -- 172.21.15.132:0/4073947319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faba0102b80 msgr2=0x7faba0109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:59.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.058+0000 7faba4eac700 1 --2- 172.21.15.132:0/4073947319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faba0102b80 0x7faba0109430 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fab9800b0d0 tx=0x7fab9800b3e0 comp rx=0 tx=0).stop 2024-05-08T22:03:59.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.058+0000 7faba4eac700 1 -- 172.21.15.132:0/4073947319 shutdown_connections 2024-05-08T22:03:59.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.058+0000 7faba4eac700 1 --2- 172.21.15.132:0/4073947319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faba0102b80 0x7faba0109430 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:59.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.058+0000 7faba4eac700 1 --2- 172.21.15.132:0/4073947319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faba0100fe0 0x7faba0101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:59.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.058+0000 7faba4eac700 1 --2- 172.21.15.132:0/4073947319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faba01021e0 0x7faba0102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:59.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.058+0000 7faba4eac700 1 -- 172.21.15.132:0/4073947319 >> 172.21.15.132:0/4073947319 conn(0x7faba00fc580 msgr2=0x7faba00fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:03:59.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.058+0000 7faba4eac700 1 -- 172.21.15.132:0/4073947319 shutdown_connections 2024-05-08T22:03:59.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.058+0000 7faba4eac700 1 -- 172.21.15.132:0/4073947319 wait complete. 2024-05-08T22:03:59.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.059+0000 7faba4eac700 1 Processor -- start 2024-05-08T22:03:59.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.059+0000 7faba4eac700 1 -- start start 2024-05-08T22:03:59.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.059+0000 7faba4eac700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faba0100fe0 0x7faba0074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:59.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.059+0000 7faba4eac700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faba01021e0 0x7faba0073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:59.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.059+0000 7faba4eac700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faba0102b80 0x7faba00738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:59.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.059+0000 7faba4eac700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba0074070 con 0x7faba0102b80 2024-05-08T22:03:59.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.059+0000 7faba4eac700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba00741e0 con 0x7faba01021e0 2024-05-08T22:03:59.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.059+0000 7faba4eac700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba01a1020 con 0x7faba0100fe0 2024-05-08T22:03:59.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.059+0000 7fab9dd9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faba01021e0 0x7faba0073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:59.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.059+0000 7fab9dd9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faba01021e0 0x7faba0073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:44388/0 (socket says 172.21.15.132:44388) 2024-05-08T22:03:59.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.060+0000 7fab9dd9b700 1 -- 172.21.15.132:0/2817006444 learned_addr learned my addr 172.21.15.132:0/2817006444 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-05-08T22:03:59.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.060+0000 7fab9ed9d700 1 --2- 172.21.15.132:0/2817006444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faba0102b80 0x7faba00738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:59.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.060+0000 7fab9e59c700 1 --2- 172.21.15.132:0/2817006444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faba0100fe0 0x7faba0074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:59.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.060+0000 7fab9dd9b700 1 -- 172.21.15.132:0/2817006444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faba0100fe0 msgr2=0x7faba0074cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:59.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.060+0000 7fab9dd9b700 1 --2- 172.21.15.132:0/2817006444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faba0100fe0 0x7faba0074cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:59.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.060+0000 7fab9dd9b700 1 -- 172.21.15.132:0/2817006444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faba0102b80 msgr2=0x7faba00738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:03:59.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.060+0000 7fab9dd9b700 1 --2- 172.21.15.132:0/2817006444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faba0102b80 0x7faba00738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:03:59.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.060+0000 7fab9dd9b700 1 -- 172.21.15.132:0/2817006444 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab98009d00 con 0x7faba01021e0 2024-05-08T22:03:59.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.060+0000 7fab9e59c700 1 --2- 172.21.15.132:0/2817006444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faba0100fe0 0x7faba0074cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:03:59.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.060+0000 7fab9dd9b700 1 --2- 172.21.15.132:0/2817006444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faba01021e0 0x7faba0073390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fab9000d950 tx=0x7fab9000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:59.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.060+0000 7fab977fe700 1 -- 172.21.15.132:0/2817006444 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab90009940 con 0x7faba01021e0 2024-05-08T22:03:59.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.060+0000 7faba4eac700 1 -- 172.21.15.132:0/2817006444 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba01a12b0 con 0x7faba01021e0 2024-05-08T22:03:59.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.060+0000 7fab977fe700 1 -- 172.21.15.132:0/2817006444 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab90010460 con 0x7faba01021e0 2024-05-08T22:03:59.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.060+0000 7fab977fe700 1 -- 172.21.15.132:0/2817006444 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9000f620 con 0x7faba01021e0 2024-05-08T22:03:59.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.060+0000 7faba4eac700 1 -- 172.21.15.132:0/2817006444 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba01a1800 con 0x7faba01021e0 2024-05-08T22:03:59.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.061+0000 7faba4eac700 1 -- 172.21.15.132:0/2817006444 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba0108420 con 0x7faba01021e0 2024-05-08T22:03:59.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.062+0000 7fab977fe700 1 -- 172.21.15.132:0/2817006444 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fab900108d0 con 0x7faba01021e0 2024-05-08T22:03:59.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.063+0000 7fab977fe700 1 --2- 172.21.15.132:0/2817006444 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fab8c0a1120 0x7fab8c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:03:59.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.063+0000 7fab977fe700 1 -- 172.21.15.132:0/2817006444 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fab900cc3a0 con 0x7faba01021e0 2024-05-08T22:03:59.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.063+0000 7fab9e59c700 1 --2- 172.21.15.132:0/2817006444 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fab8c0a1120 0x7fab8c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:03:59.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.063+0000 7fab9e59c700 1 --2- 172.21.15.132:0/2817006444 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fab8c0a1120 0x7fab8c0a35e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fab88005fd0 tx=0x7fab88005f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:03:59.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.067+0000 7fab977fe700 1 -- 172.21.15.132:0/2817006444 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fab90096290 con 0x7faba01021e0 2024-05-08T22:03:59.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:58 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:03:59.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:03:59.266+0000 7faba4eac700 1 -- 172.21.15.132:0/2817006444 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7faba00613e0 con 0x7fab8c0a1120 2024-05-08T22:03:59.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:59 smithi132 ceph-mon[78180]: purged_snaps scrub starts 2024-05-08T22:03:59.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:59 smithi132 ceph-mon[78180]: purged_snaps scrub ok 2024-05-08T22:03:59.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:03:59.953 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:59 smithi132 ceph-mon[78180]: osd.9 [v2:172.21.15.131:6826/4217805693,v1:172.21.15.131:6827/4217805693] boot 2024-05-08T22:03:59.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:59 smithi132 ceph-mon[78180]: osdmap e62: 10 total, 10 up, 10 in 2024-05-08T22:03:59.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:03:59.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:59 smithi132 ceph-mon[78180]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:03:59.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:59 smithi132 ceph-mon[78180]: Detected new or changed devices on smithi131 2024-05-08T22:03:59.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:59.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:59.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:59.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:59.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:59.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:03:59.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:59 smithi132 ceph-mon[78180]: Adjusting osd_memory_target on smithi131 to 4317M 2024-05-08T22:03:59.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:59.954 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:03:59.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:03:59.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:03:59.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:03:59.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:03:59.955 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:03:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:00.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:59 smithi157 ceph-mon[77746]: purged_snaps scrub starts 2024-05-08T22:04:00.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:59 smithi157 ceph-mon[77746]: purged_snaps scrub ok 2024-05-08T22:04:00.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:04:00.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:59 smithi157 ceph-mon[77746]: osd.9 [v2:172.21.15.131:6826/4217805693,v1:172.21.15.131:6827/4217805693] boot 2024-05-08T22:04:00.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:59 smithi157 ceph-mon[77746]: osdmap e62: 10 total, 10 up, 10 in 2024-05-08T22:04:00.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:04:00.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:59 smithi157 ceph-mon[77746]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:04:00.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:59 smithi157 ceph-mon[77746]: Detected new or changed devices on smithi131 2024-05-08T22:04:00.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:00.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:00.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:00.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:00.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:00.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:00.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:59 smithi157 ceph-mon[77746]: Adjusting osd_memory_target on smithi131 to 4317M 2024-05-08T22:04:00.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:00.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:00.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:04:00.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:00.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:04:00.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:04:00.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:03:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:00.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:59 smithi131 ceph-mon[37260]: purged_snaps scrub starts 2024-05-08T22:04:00.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:59 smithi131 ceph-mon[37260]: purged_snaps scrub ok 2024-05-08T22:04:00.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:04:00.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:59 smithi131 ceph-mon[37260]: osd.9 [v2:172.21.15.131:6826/4217805693,v1:172.21.15.131:6827/4217805693] boot 2024-05-08T22:04:00.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:59 smithi131 ceph-mon[37260]: osdmap e62: 10 total, 10 up, 10 in 2024-05-08T22:04:00.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:04:00.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:59 smithi131 ceph-mon[37260]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:04:00.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:59 smithi131 ceph-mon[37260]: Detected new or changed devices on smithi131 2024-05-08T22:04:00.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:00.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:00.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:00.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:00.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:00.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:00.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:59 smithi131 ceph-mon[37260]: Adjusting osd_memory_target on smithi131 to 4317M 2024-05-08T22:04:00.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:00.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:00.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:04:00.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:00.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:04:00.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:04:00.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:03:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:01.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:00 smithi132 ceph-mon[78180]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:04:01.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:00 smithi132 ceph-mon[78180]: osdmap e63: 10 total, 10 up, 10 in 2024-05-08T22:04:01.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:00 smithi131 ceph-mon[37260]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:04:01.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:00 smithi131 ceph-mon[37260]: osdmap e63: 10 total, 10 up, 10 in 2024-05-08T22:04:01.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:00 smithi157 ceph-mon[77746]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:04:01.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:00 smithi157 ceph-mon[77746]: osdmap e63: 10 total, 10 up, 10 in 2024-05-08T22:04:02.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:01 smithi132 ceph-mon[78180]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:02.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:01 smithi132 ceph-mon[78180]: from='client.? 172.21.15.132:0/2975775413' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "430013a4-d468-49b2-9162-165ac5a4a7f8"}]: dispatch 2024-05-08T22:04:02.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:01 smithi132 ceph-mon[78180]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "430013a4-d468-49b2-9162-165ac5a4a7f8"}]: dispatch 2024-05-08T22:04:02.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:01 smithi132 ceph-mon[78180]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "430013a4-d468-49b2-9162-165ac5a4a7f8"}]': finished 2024-05-08T22:04:02.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:01 smithi132 ceph-mon[78180]: osdmap e64: 11 total, 10 up, 11 in 2024-05-08T22:04:02.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:01 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:04:02.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:01 smithi132 ceph-mon[78180]: from='client.? 172.21.15.132:0/3436190390' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:04:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:01 smithi131 ceph-mon[37260]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:01 smithi131 ceph-mon[37260]: from='client.? 172.21.15.132:0/2975775413' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "430013a4-d468-49b2-9162-165ac5a4a7f8"}]: dispatch 2024-05-08T22:04:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:01 smithi131 ceph-mon[37260]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "430013a4-d468-49b2-9162-165ac5a4a7f8"}]: dispatch 2024-05-08T22:04:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:01 smithi131 ceph-mon[37260]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "430013a4-d468-49b2-9162-165ac5a4a7f8"}]': finished 2024-05-08T22:04:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:01 smithi131 ceph-mon[37260]: osdmap e64: 11 total, 10 up, 11 in 2024-05-08T22:04:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:01 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:04:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:01 smithi131 ceph-mon[37260]: from='client.? 172.21.15.132:0/3436190390' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:04:02.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:01 smithi157 ceph-mon[77746]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:02.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:01 smithi157 ceph-mon[77746]: from='client.? 172.21.15.132:0/2975775413' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "430013a4-d468-49b2-9162-165ac5a4a7f8"}]: dispatch 2024-05-08T22:04:02.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:01 smithi157 ceph-mon[77746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "430013a4-d468-49b2-9162-165ac5a4a7f8"}]: dispatch 2024-05-08T22:04:02.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:01 smithi157 ceph-mon[77746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "430013a4-d468-49b2-9162-165ac5a4a7f8"}]': finished 2024-05-08T22:04:02.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:01 smithi157 ceph-mon[77746]: osdmap e64: 11 total, 10 up, 11 in 2024-05-08T22:04:02.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:01 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:04:02.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:01 smithi157 ceph-mon[77746]: from='client.? 172.21.15.132:0/3436190390' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:04:04.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:03 smithi132 ceph-mon[78180]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:04.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:03 smithi131 ceph-mon[37260]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:04.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:03 smithi157 ceph-mon[77746]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:05.961 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:05 smithi132 ceph-mon[78180]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:06.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:05 smithi131 ceph-mon[37260]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:06.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:05 smithi157 ceph-mon[77746]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:08.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:07 smithi132 ceph-mon[78180]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:08.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:07 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T22:04:08.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:07 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:08.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:07 smithi131 ceph-mon[37260]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:08.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:07 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T22:04:08.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:07 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:08.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:07 smithi157 ceph-mon[77746]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:08.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:07 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T22:04:08.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:07 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:08.889 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:08 smithi132 ceph-mon[78180]: Deploying daemon osd.10 on smithi132 2024-05-08T22:04:09.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:08 smithi131 ceph-mon[37260]: Deploying daemon osd.10 on smithi132 2024-05-08T22:04:09.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:08 smithi157 ceph-mon[77746]: Deploying daemon osd.10 on smithi132 2024-05-08T22:04:10.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:09 smithi132 ceph-mon[78180]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:10.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:09 smithi131 ceph-mon[37260]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:10.278 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:09 smithi157 ceph-mon[77746]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:12.261 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:11 smithi132 ceph-mon[78180]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:12.261 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:11 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:12.261 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:11 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:12.261 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:11 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:04:12.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:11 smithi157 ceph-mon[77746]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:12.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:11 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:12.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:11 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:12.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:11 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:04:12.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:11 smithi131 ceph-mon[37260]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:12.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:11 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:12.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:11 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:12.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:11 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:04:13.223 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 10 on host 'smithi132' 2024-05-08T22:04:13.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:04:13.221+0000 7fab977fe700 1 -- 172.21.15.132:0/2817006444 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7faba00613e0 con 0x7fab8c0a1120 2024-05-08T22:04:13.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:04:13.223+0000 7faba4eac700 1 -- 172.21.15.132:0/2817006444 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fab8c0a1120 msgr2=0x7fab8c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:13.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:04:13.223+0000 7faba4eac700 1 --2- 172.21.15.132:0/2817006444 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fab8c0a1120 0x7fab8c0a35e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fab88005fd0 tx=0x7fab88005f00 comp rx=0 tx=0).stop 2024-05-08T22:04:13.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:04:13.223+0000 7faba4eac700 1 -- 172.21.15.132:0/2817006444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faba01021e0 msgr2=0x7faba0073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:13.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:04:13.223+0000 7faba4eac700 1 --2- 172.21.15.132:0/2817006444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faba01021e0 0x7faba0073390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fab9000d950 tx=0x7fab9000dc60 comp rx=0 tx=0).stop 2024-05-08T22:04:13.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:04:13.223+0000 7faba4eac700 1 -- 172.21.15.132:0/2817006444 shutdown_connections 2024-05-08T22:04:13.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:04:13.223+0000 7faba4eac700 1 --2- 172.21.15.132:0/2817006444 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fab8c0a1120 0x7fab8c0a35e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:13.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:04:13.223+0000 7faba4eac700 1 --2- 172.21.15.132:0/2817006444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faba0102b80 0x7faba00738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:13.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:04:13.223+0000 7faba4eac700 1 --2- 172.21.15.132:0/2817006444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faba0100fe0 0x7faba0074cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:13.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:04:13.224+0000 7faba4eac700 1 --2- 172.21.15.132:0/2817006444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faba01021e0 0x7faba0073390 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:13.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:04:13.224+0000 7faba4eac700 1 -- 172.21.15.132:0/2817006444 >> 172.21.15.132:0/2817006444 conn(0x7faba00fc580 msgr2=0x7faba0105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:13.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:04:13.224+0000 7faba4eac700 1 -- 172.21.15.132:0/2817006444 shutdown_connections 2024-05-08T22:04:13.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-05-08T22:04:13.224+0000 7faba4eac700 1 -- 172.21.15.132:0/2817006444 wait complete. 2024-05-08T22:04:13.621 DEBUG:teuthology.orchestra.run.smithi132:osd.10> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.10.service 2024-05-08T22:04:13.624 INFO:tasks.cephadm:Deploying osd.11 on smithi157 with /dev/vg_nvme/lv_1... 2024-05-08T22:04:13.624 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T22:04:13.657 INFO:journalctl@ceph.osd.10.smithi132.stdout:-- Logs begin at Wed 2024-05-08 21:34:20 UTC. -- 2024-05-08T22:04:13.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:13 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:13.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:13 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:13.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:13 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:13.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:13 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:04:13.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:13 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:13.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:13 smithi157 ceph-mon[77746]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:13.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:13 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:13.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:13 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:13.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:13 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:04:13.783 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:13 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:13.783 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:13 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:13.783 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:13 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:13.783 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:13 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:04:13.783 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:13 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:13.783 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:13 smithi132 ceph-mon[78180]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:13.783 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:13 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:13.783 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:13 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:13.783 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:13 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:04:13.860 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:04:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:13 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:13 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:13 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:13 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:04:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:13 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:13 smithi131 ceph-mon[37260]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:13 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:13 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:13 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:04:15.195 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T22:04:15.195 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T22:04:15.195 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-05-08T22:04:15.195 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-05-08T22:04:15.195 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0133841 s, 783 MB/s 2024-05-08T22:04:15.195 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-05-08T22:04:15.340 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:04:14 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:04:14.922+0000 7f02f793a7c0 -1 osd.10 0 log_to_monitors true 2024-05-08T22:04:15.604 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_1 2024-05-08T22:04:15.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:15 smithi157 ceph-mon[77746]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:15.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:15 smithi157 ceph-mon[77746]: from='osd.10 [v2:172.21.15.132:6824/306651644,v1:172.21.15.132:6825/306651644]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T22:04:15.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:15 smithi157 ceph-mon[77746]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T22:04:15.824 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:04:16.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:15 smithi132 ceph-mon[78180]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:16.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:15 smithi132 ceph-mon[78180]: from='osd.10 [v2:172.21.15.132:6824/306651644,v1:172.21.15.132:6825/306651644]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T22:04:16.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:15 smithi132 ceph-mon[78180]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T22:04:16.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:15 smithi131 ceph-mon[37260]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:16.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:15 smithi131 ceph-mon[37260]: from='osd.10 [v2:172.21.15.132:6824/306651644,v1:172.21.15.132:6825/306651644]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T22:04:16.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:15 smithi131 ceph-mon[37260]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T22:04:16.848 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.847+0000 7fb1cca6d700 1 -- 172.21.15.157:0/130552874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1c8101060 msgr2=0x7fb1c81014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:16.849 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.847+0000 7fb1cca6d700 1 --2- 172.21.15.157:0/130552874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1c8101060 0x7fb1c81014e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009990 tx=0x7fb1b8009ca0 comp rx=0 tx=0).stop 2024-05-08T22:04:16.849 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.847+0000 7fb1cca6d700 1 -- 172.21.15.157:0/130552874 shutdown_connections 2024-05-08T22:04:16.849 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.847+0000 7fb1cca6d700 1 --2- 172.21.15.157:0/130552874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1c8101a20 0x7fb1c8108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:16.849 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.847+0000 7fb1cca6d700 1 --2- 172.21.15.157:0/130552874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1c80fff00 0x7fb1c8100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:16.849 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.847+0000 7fb1cca6d700 1 --2- 172.21.15.157:0/130552874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1c8101060 0x7fb1c81014e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:16.849 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.847+0000 7fb1cca6d700 1 -- 172.21.15.157:0/130552874 >> 172.21.15.157:0/130552874 conn(0x7fb1c80fb4c0 msgr2=0x7fb1c80fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:16.849 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.847+0000 7fb1cca6d700 1 -- 172.21.15.157:0/130552874 shutdown_connections 2024-05-08T22:04:16.849 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.847+0000 7fb1cca6d700 1 -- 172.21.15.157:0/130552874 wait complete. 2024-05-08T22:04:16.851 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.848+0000 7fb1cca6d700 1 Processor -- start 2024-05-08T22:04:16.851 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.848+0000 7fb1cca6d700 1 -- start start 2024-05-08T22:04:16.851 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.848+0000 7fb1cca6d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1c80fff00 0x7fb1c8196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:16.851 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.848+0000 7fb1cca6d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1c8101060 0x7fb1c8197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:16.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.848+0000 7fb1cca6d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1c8101a20 0x7fb1c819e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:16.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.848+0000 7fb1cca6d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c819ea70 con 0x7fb1c8101a20 2024-05-08T22:04:16.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.848+0000 7fb1cca6d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c819ebb0 con 0x7fb1c80fff00 2024-05-08T22:04:16.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.848+0000 7fb1cca6d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c819ed40 con 0x7fb1c8101060 2024-05-08T22:04:16.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.848+0000 7fb1c5d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1c8101060 0x7fb1c8197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:16.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.848+0000 7fb1c5d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1c8101060 0x7fb1c8197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34868/0 (socket says 172.21.15.157:34868) 2024-05-08T22:04:16.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.848+0000 7fb1c5d9b700 1 -- 172.21.15.157:0/1908309047 learned_addr learned my addr 172.21.15.157:0/1908309047 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-08T22:04:16.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.848+0000 7fb1c659c700 1 --2- 172.21.15.157:0/1908309047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1c80fff00 0x7fb1c8196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:16.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.849+0000 7fb1c6d9d700 1 --2- 172.21.15.157:0/1908309047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1c8101a20 0x7fb1c819e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:16.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.849+0000 7fb1c5d9b700 1 -- 172.21.15.157:0/1908309047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1c80fff00 msgr2=0x7fb1c8196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:16.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.849+0000 7fb1c5d9b700 1 --2- 172.21.15.157:0/1908309047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1c80fff00 0x7fb1c8196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:16.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.849+0000 7fb1c5d9b700 1 -- 172.21.15.157:0/1908309047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1c8101a20 msgr2=0x7fb1c819e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:16.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.849+0000 7fb1c5d9b700 1 --2- 172.21.15.157:0/1908309047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1c8101a20 0x7fb1c819e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:16.852 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.849+0000 7fb1c5d9b700 1 -- 172.21.15.157:0/1908309047 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b80095f0 con 0x7fb1c8101060 2024-05-08T22:04:16.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.849+0000 7fb1c6d9d700 1 --2- 172.21.15.157:0/1908309047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1c8101a20 0x7fb1c819e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:04:16.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.849+0000 7fb1c5d9b700 1 --2- 172.21.15.157:0/1908309047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1c8101060 0x7fb1c8197470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80059d0 tx=0x7fb1b800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:16.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.849+0000 7fb1b77fe700 1 -- 172.21.15.157:0/1908309047 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b801d070 con 0x7fb1c8101060 2024-05-08T22:04:16.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.849+0000 7fb1b77fe700 1 -- 172.21.15.157:0/1908309047 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1b8022470 con 0x7fb1c8101060 2024-05-08T22:04:16.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.849+0000 7fb1b77fe700 1 -- 172.21.15.157:0/1908309047 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8017650 con 0x7fb1c8101060 2024-05-08T22:04:16.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.849+0000 7fb1cca6d700 1 -- 172.21.15.157:0/1908309047 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c819efc0 con 0x7fb1c8101060 2024-05-08T22:04:16.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.849+0000 7fb1cca6d700 1 -- 172.21.15.157:0/1908309047 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c819f380 con 0x7fb1c8101060 2024-05-08T22:04:16.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.850+0000 7fb1cca6d700 1 -- 172.21.15.157:0/1908309047 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c804ec60 con 0x7fb1c8101060 2024-05-08T22:04:16.854 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.851+0000 7fb1b77fe700 1 -- 172.21.15.157:0/1908309047 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb1b80225e0 con 0x7fb1c8101060 2024-05-08T22:04:16.854 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.851+0000 7fb1b77fe700 1 --2- 172.21.15.157:0/1908309047 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb1b00a11a0 0x7fb1b00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:16.854 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.852+0000 7fb1b77fe700 1 -- 172.21.15.157:0/1908309047 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb1b80cd010 con 0x7fb1c8101060 2024-05-08T22:04:16.854 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.852+0000 7fb1c659c700 1 --2- 172.21.15.157:0/1908309047 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb1b00a11a0 0x7fb1b00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:16.854 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.852+0000 7fb1c659c700 1 --2- 172.21.15.157:0/1908309047 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb1b00a11a0 0x7fb1b00a3660 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009a10 tx=0x7fb1bc009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:16.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:16.856+0000 7fb1b77fe700 1 -- 172.21.15.157:0/1908309047 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb1b8096fa0 con 0x7fb1c8101060 2024-05-08T22:04:17.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:16 smithi157 ceph-mon[77746]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T22:04:17.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:16 smithi157 ceph-mon[77746]: osdmap e65: 11 total, 10 up, 11 in 2024-05-08T22:04:17.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:16 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:04:17.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:16 smithi157 ceph-mon[77746]: from='osd.10 [v2:172.21.15.132:6824/306651644,v1:172.21.15.132:6825/306651644]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-05-08T22:04:17.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:16 smithi157 ceph-mon[77746]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-05-08T22:04:17.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:16 smithi157 ceph-mon[77746]: Detected new or changed devices on smithi132 2024-05-08T22:04:17.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:16 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:17.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:16 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:17.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:16 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:17.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:16 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:17.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:16 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:17.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:16 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:17.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:16 smithi157 ceph-mon[77746]: Adjusting osd_memory_target on smithi132 to 4278M 2024-05-08T22:04:17.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:16 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:17.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:16 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:17.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:16 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:04:17.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:16 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:17.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:17.061+0000 7fb1cca6d700 1 -- 172.21.15.157:0/1908309047 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1c80fc2c0 con 0x7fb1b00a11a0 2024-05-08T22:04:17.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:16 smithi132 ceph-mon[78180]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T22:04:17.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:16 smithi132 ceph-mon[78180]: osdmap e65: 11 total, 10 up, 11 in 2024-05-08T22:04:17.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:16 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:04:17.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:16 smithi132 ceph-mon[78180]: from='osd.10 [v2:172.21.15.132:6824/306651644,v1:172.21.15.132:6825/306651644]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-05-08T22:04:17.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:16 smithi132 ceph-mon[78180]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-05-08T22:04:17.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:16 smithi132 ceph-mon[78180]: Detected new or changed devices on smithi132 2024-05-08T22:04:17.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:16 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:17.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:16 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:17.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:16 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:17.092 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:16 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:17.092 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:16 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:17.092 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:16 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:17.092 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:16 smithi132 ceph-mon[78180]: Adjusting osd_memory_target on smithi132 to 4278M 2024-05-08T22:04:17.092 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:16 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:17.092 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:16 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:17.093 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:16 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:04:17.093 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:16 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:17.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:16 smithi131 ceph-mon[37260]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T22:04:17.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:16 smithi131 ceph-mon[37260]: osdmap e65: 11 total, 10 up, 11 in 2024-05-08T22:04:17.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:16 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:04:17.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:16 smithi131 ceph-mon[37260]: from='osd.10 [v2:172.21.15.132:6824/306651644,v1:172.21.15.132:6825/306651644]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-05-08T22:04:17.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:16 smithi131 ceph-mon[37260]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-05-08T22:04:17.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:16 smithi131 ceph-mon[37260]: Detected new or changed devices on smithi132 2024-05-08T22:04:17.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:16 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:17.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:16 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:17.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:16 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:17.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:16 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:17.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:16 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:17.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:16 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:17.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:16 smithi131 ceph-mon[37260]: Adjusting osd_memory_target on smithi132 to 4278M 2024-05-08T22:04:17.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:16 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:17.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:16 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:17.144 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:16 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:04:17.144 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:16 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:17.590 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:04:17 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:04:17.225+0000 7f02ecf9e700 -1 osd.10 0 waiting for initial osdmap 2024-05-08T22:04:17.590 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:04:17 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:04:17.238+0000 7f02e4d8b700 -1 osd.10 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:04:18.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:17 smithi157 ceph-mon[77746]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:18.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:17 smithi157 ceph-mon[77746]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-05-08T22:04:18.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:17 smithi157 ceph-mon[77746]: osdmap e66: 11 total, 10 up, 11 in 2024-05-08T22:04:18.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:17 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:04:18.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:17 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:04:18.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:17 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:04:18.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:17 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:04:18.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:17 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:18.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:17 smithi157 ceph-mon[77746]: from='osd.10 ' entity='osd.10' 2024-05-08T22:04:18.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:17 smithi157 ceph-mon[77746]: osd.10 [v2:172.21.15.132:6824/306651644,v1:172.21.15.132:6825/306651644] boot 2024-05-08T22:04:18.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:17 smithi157 ceph-mon[77746]: osdmap e67: 11 total, 11 up, 11 in 2024-05-08T22:04:18.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:17 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:04:18.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:17 smithi132 ceph-mon[78180]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:18.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:17 smithi132 ceph-mon[78180]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-05-08T22:04:18.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:17 smithi132 ceph-mon[78180]: osdmap e66: 11 total, 10 up, 11 in 2024-05-08T22:04:18.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:17 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:04:18.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:17 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:04:18.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:17 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:04:18.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:17 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:04:18.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:17 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:18.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:17 smithi132 ceph-mon[78180]: from='osd.10 ' entity='osd.10' 2024-05-08T22:04:18.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:17 smithi132 ceph-mon[78180]: osd.10 [v2:172.21.15.132:6824/306651644,v1:172.21.15.132:6825/306651644] boot 2024-05-08T22:04:18.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:17 smithi132 ceph-mon[78180]: osdmap e67: 11 total, 11 up, 11 in 2024-05-08T22:04:18.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:17 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:04:18.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:17 smithi131 ceph-mon[37260]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:04:18.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:17 smithi131 ceph-mon[37260]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-05-08T22:04:18.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:17 smithi131 ceph-mon[37260]: osdmap e66: 11 total, 10 up, 11 in 2024-05-08T22:04:18.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:17 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:04:18.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:17 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:04:18.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:17 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:04:18.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:17 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:04:18.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:17 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:18.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:17 smithi131 ceph-mon[37260]: from='osd.10 ' entity='osd.10' 2024-05-08T22:04:18.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:17 smithi131 ceph-mon[37260]: osd.10 [v2:172.21.15.132:6824/306651644,v1:172.21.15.132:6825/306651644] boot 2024-05-08T22:04:18.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:17 smithi131 ceph-mon[37260]: osdmap e67: 11 total, 11 up, 11 in 2024-05-08T22:04:18.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:17 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:04:19.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:18 smithi157 ceph-mon[77746]: purged_snaps scrub starts 2024-05-08T22:04:19.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:18 smithi157 ceph-mon[77746]: purged_snaps scrub ok 2024-05-08T22:04:19.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:18 smithi157 ceph-mon[77746]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:04:19.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:18 smithi157 ceph-mon[77746]: osdmap e68: 11 total, 11 up, 11 in 2024-05-08T22:04:19.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:18 smithi132 ceph-mon[78180]: purged_snaps scrub starts 2024-05-08T22:04:19.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:18 smithi132 ceph-mon[78180]: purged_snaps scrub ok 2024-05-08T22:04:19.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:18 smithi132 ceph-mon[78180]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:04:19.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:18 smithi132 ceph-mon[78180]: osdmap e68: 11 total, 11 up, 11 in 2024-05-08T22:04:19.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:18 smithi131 ceph-mon[37260]: purged_snaps scrub starts 2024-05-08T22:04:19.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:18 smithi131 ceph-mon[37260]: purged_snaps scrub ok 2024-05-08T22:04:19.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:18 smithi131 ceph-mon[37260]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:04:19.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:18 smithi131 ceph-mon[37260]: osdmap e68: 11 total, 11 up, 11 in 2024-05-08T22:04:20.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:19 smithi157 ceph-mon[77746]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 347 MiB used, 983 GiB / 983 GiB avail 2024-05-08T22:04:20.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:19 smithi157 ceph-mon[77746]: from='client.? 172.21.15.157:0/650557817' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76622ed0-d3b9-4f55-9206-fc96c56e18ed"}]: dispatch 2024-05-08T22:04:20.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:19 smithi157 ceph-mon[77746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76622ed0-d3b9-4f55-9206-fc96c56e18ed"}]: dispatch 2024-05-08T22:04:20.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:19 smithi157 ceph-mon[77746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76622ed0-d3b9-4f55-9206-fc96c56e18ed"}]': finished 2024-05-08T22:04:20.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:19 smithi157 ceph-mon[77746]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T22:04:20.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:19 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:04:20.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:19 smithi157 ceph-mon[77746]: from='client.? 172.21.15.157:0/2904056862' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:04:20.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:19 smithi132 ceph-mon[78180]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 347 MiB used, 983 GiB / 983 GiB avail 2024-05-08T22:04:20.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:19 smithi132 ceph-mon[78180]: from='client.? 172.21.15.157:0/650557817' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76622ed0-d3b9-4f55-9206-fc96c56e18ed"}]: dispatch 2024-05-08T22:04:20.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:19 smithi132 ceph-mon[78180]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76622ed0-d3b9-4f55-9206-fc96c56e18ed"}]: dispatch 2024-05-08T22:04:20.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:19 smithi132 ceph-mon[78180]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76622ed0-d3b9-4f55-9206-fc96c56e18ed"}]': finished 2024-05-08T22:04:20.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:19 smithi132 ceph-mon[78180]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T22:04:20.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:19 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:04:20.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:19 smithi132 ceph-mon[78180]: from='client.? 172.21.15.157:0/2904056862' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:04:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:19 smithi131 ceph-mon[37260]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 347 MiB used, 983 GiB / 983 GiB avail 2024-05-08T22:04:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:19 smithi131 ceph-mon[37260]: from='client.? 172.21.15.157:0/650557817' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76622ed0-d3b9-4f55-9206-fc96c56e18ed"}]: dispatch 2024-05-08T22:04:20.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:19 smithi131 ceph-mon[37260]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76622ed0-d3b9-4f55-9206-fc96c56e18ed"}]: dispatch 2024-05-08T22:04:20.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:19 smithi131 ceph-mon[37260]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76622ed0-d3b9-4f55-9206-fc96c56e18ed"}]': finished 2024-05-08T22:04:20.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:19 smithi131 ceph-mon[37260]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T22:04:20.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:19 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:04:20.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:19 smithi131 ceph-mon[37260]: from='client.? 172.21.15.157:0/2904056862' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:04:21.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:20 smithi157 ceph-mon[77746]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T22:04:21.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:20 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:04:21.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:20 smithi132 ceph-mon[78180]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T22:04:21.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:20 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:04:21.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:20 smithi131 ceph-mon[37260]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T22:04:21.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:20 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:04:22.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:21 smithi157 ceph-mon[77746]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-05-08T22:04:22.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:21 smithi157 ceph-mon[77746]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T22:04:22.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:21 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:04:22.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:21 smithi132 ceph-mon[78180]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-05-08T22:04:22.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:21 smithi132 ceph-mon[78180]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T22:04:22.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:21 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:04:22.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:21 smithi131 ceph-mon[37260]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-05-08T22:04:22.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:21 smithi131 ceph-mon[37260]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T22:04:22.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:21 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:04:24.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:23 smithi157 ceph-mon[77746]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-05-08T22:04:24.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:23 smithi132 ceph-mon[78180]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-05-08T22:04:24.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:23 smithi131 ceph-mon[37260]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-05-08T22:04:25.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:24 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T22:04:25.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:24 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:25.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:24 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T22:04:25.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:24 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:25.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:24 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T22:04:25.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:24 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:26.255 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:25 smithi157 ceph-mon[77746]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T22:04:26.255 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:25 smithi157 ceph-mon[77746]: Deploying daemon osd.11 on smithi157 2024-05-08T22:04:26.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:25 smithi132 ceph-mon[78180]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T22:04:26.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:25 smithi132 ceph-mon[78180]: Deploying daemon osd.11 on smithi157 2024-05-08T22:04:26.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:25 smithi131 ceph-mon[37260]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T22:04:26.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:25 smithi131 ceph-mon[37260]: Deploying daemon osd.11 on smithi157 2024-05-08T22:04:28.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:27 smithi157 ceph-mon[77746]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-08T22:04:28.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:27 smithi132 ceph-mon[78180]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-08T22:04:28.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:27 smithi131 ceph-mon[37260]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-08T22:04:29.758 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:29 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:29.758 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:29 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:29.758 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:29 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:04:29.758 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:29 smithi157 ceph-mon[77746]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-08T22:04:29.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:29 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:29.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:29 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:29.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:29 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:04:29.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:29 smithi132 ceph-mon[78180]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-08T22:04:29.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:29 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:29.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:29 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:29.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:29 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:04:29.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:29 smithi131 ceph-mon[37260]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-08T22:04:30.820 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:30.819+0000 7fb1b77fe700 1 -- 172.21.15.157:0/1908309047 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fb1c80fc2c0 con 0x7fb1b00a11a0 2024-05-08T22:04:30.821 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 11 on host 'smithi157' 2024-05-08T22:04:30.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:30.821+0000 7fb1cca6d700 1 -- 172.21.15.157:0/1908309047 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb1b00a11a0 msgr2=0x7fb1b00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:30.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:30.821+0000 7fb1cca6d700 1 --2- 172.21.15.157:0/1908309047 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb1b00a11a0 0x7fb1b00a3660 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009a10 tx=0x7fb1bc009190 comp rx=0 tx=0).stop 2024-05-08T22:04:30.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:30.821+0000 7fb1cca6d700 1 -- 172.21.15.157:0/1908309047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1c8101060 msgr2=0x7fb1c8197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:30.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:30.821+0000 7fb1cca6d700 1 --2- 172.21.15.157:0/1908309047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1c8101060 0x7fb1c8197470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80059d0 tx=0x7fb1b800f690 comp rx=0 tx=0).stop 2024-05-08T22:04:30.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:30.821+0000 7fb1cca6d700 1 -- 172.21.15.157:0/1908309047 shutdown_connections 2024-05-08T22:04:30.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:30.821+0000 7fb1cca6d700 1 --2- 172.21.15.157:0/1908309047 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb1b00a11a0 0x7fb1b00a3660 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:30.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:30.821+0000 7fb1cca6d700 1 --2- 172.21.15.157:0/1908309047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1c8101a20 0x7fb1c819e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:30.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:30.821+0000 7fb1cca6d700 1 --2- 172.21.15.157:0/1908309047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1c80fff00 0x7fb1c8196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:30.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:30.821+0000 7fb1cca6d700 1 --2- 172.21.15.157:0/1908309047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1c8101060 0x7fb1c8197470 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:30.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:30.821+0000 7fb1cca6d700 1 -- 172.21.15.157:0/1908309047 >> 172.21.15.157:0/1908309047 conn(0x7fb1c80fb4c0 msgr2=0x7fb1c80fbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:30.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:30.822+0000 7fb1cca6d700 1 -- 172.21.15.157:0/1908309047 shutdown_connections 2024-05-08T22:04:30.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:30.822+0000 7fb1cca6d700 1 -- 172.21.15.157:0/1908309047 wait complete. 2024-05-08T22:04:31.239 DEBUG:teuthology.orchestra.run.smithi157:osd.11> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.11.service 2024-05-08T22:04:31.241 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-08T22:04:31.241 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-08T22:04:31.270 INFO:journalctl@ceph.osd.11.smithi157.stdout:-- Logs begin at Wed 2024-05-08 21:35:00 UTC. -- 2024-05-08T22:04:31.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:31 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:31.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:31 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:31.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:31 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:31.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:31 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:04:31.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:31 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:31.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:31 smithi132 ceph-mon[78180]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T22:04:31.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:31 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:31.592 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:31 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:31.592 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:31 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:04:31.612 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:04:31.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:31 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:31.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:31 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:31.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:31 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:31.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:31 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:04:31.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:31 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:31.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:31 smithi131 ceph-mon[37260]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T22:04:31.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:31 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:31.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:31 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:31.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:31 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:04:31.647 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:31 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:31.647 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:31 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:31.647 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:31 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:31.647 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:31 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:04:31.647 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:31 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:31.647 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:31 smithi157 ceph-mon[77746]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T22:04:31.647 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:31 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:31.648 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:31 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:31.648 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:31 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:04:32.489 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:04:32 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:04:32.486+0000 7f77ba4e87c0 -1 osd.11 0 log_to_monitors true 2024-05-08T22:04:32.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:32 smithi157 ceph-mon[77746]: from='osd.11 [v2:172.21.15.157:6824/2884744696,v1:172.21.15.157:6825/2884744696]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T22:04:32.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:32 smithi157 ceph-mon[77746]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T22:04:32.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:32 smithi132 ceph-mon[78180]: from='osd.11 [v2:172.21.15.157:6824/2884744696,v1:172.21.15.157:6825/2884744696]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T22:04:32.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:32 smithi132 ceph-mon[78180]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T22:04:32.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.859+0000 7fc6da200700 1 -- 172.21.15.131:0/2999673414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6d4073a00 msgr2=0x7fc6d4073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:32.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.859+0000 7fc6da200700 1 --2- 172.21.15.131:0/2999673414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6d4073a00 0x7fc6d4073e80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009990 tx=0x7fc6c0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:04:32.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.859+0000 7fc6da200700 1 -- 172.21.15.131:0/2999673414 shutdown_connections 2024-05-08T22:04:32.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.859+0000 7fc6da200700 1 --2- 172.21.15.131:0/2999673414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6d4108ed0 0x7fc6d410b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:32.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.859+0000 7fc6da200700 1 --2- 172.21.15.131:0/2999673414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6d4074fd0 0x7fc6d4073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:32.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.859+0000 7fc6da200700 1 --2- 172.21.15.131:0/2999673414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6d4073a00 0x7fc6d4073e80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:32.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.859+0000 7fc6da200700 1 -- 172.21.15.131:0/2999673414 >> 172.21.15.131:0/2999673414 conn(0x7fc6d40fc5a0 msgr2=0x7fc6d40fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:32.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.859+0000 7fc6da200700 1 -- 172.21.15.131:0/2999673414 shutdown_connections 2024-05-08T22:04:32.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.859+0000 7fc6da200700 1 -- 172.21.15.131:0/2999673414 wait complete. 2024-05-08T22:04:32.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.860+0000 7fc6da200700 1 Processor -- start 2024-05-08T22:04:32.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.860+0000 7fc6da200700 1 -- start start 2024-05-08T22:04:32.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.861+0000 7fc6da200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6d4073a00 0x7fc6d4199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:32.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.861+0000 7fc6da200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6d4074fd0 0x7fc6d41996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:32.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.861+0000 7fc6da200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6d4108ed0 0x7fc6d41a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:32.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.861+0000 7fc6da200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d41a0cd0 con 0x7fc6d4073a00 2024-05-08T22:04:32.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.861+0000 7fc6da200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d41a0e10 con 0x7fc6d4074fd0 2024-05-08T22:04:32.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.861+0000 7fc6da200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d41a0fa0 con 0x7fc6d4108ed0 2024-05-08T22:04:32.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.861+0000 7fc6d3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6d4108ed0 0x7fc6d41a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:32.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.861+0000 7fc6d3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6d4108ed0 0x7fc6d41a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37038/0 (socket says 172.21.15.131:37038) 2024-05-08T22:04:32.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.861+0000 7fc6d3e00700 1 -- 172.21.15.131:0/4148322870 learned_addr learned my addr 172.21.15.131:0/4148322870 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:04:32.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.861+0000 7fc6d3400700 1 --2- 172.21.15.131:0/4148322870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6d4073a00 0x7fc6d4199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:32.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.861+0000 7fc6d2a00700 1 --2- 172.21.15.131:0/4148322870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6d4074fd0 0x7fc6d41996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:32.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.861+0000 7fc6d3e00700 1 -- 172.21.15.131:0/4148322870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6d4074fd0 msgr2=0x7fc6d41996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:32.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.861+0000 7fc6d3e00700 1 --2- 172.21.15.131:0/4148322870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6d4074fd0 0x7fc6d41996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:32.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.861+0000 7fc6d3e00700 1 -- 172.21.15.131:0/4148322870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6d4073a00 msgr2=0x7fc6d4199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:32.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.861+0000 7fc6d3e00700 1 --2- 172.21.15.131:0/4148322870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6d4073a00 0x7fc6d4199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:32.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.861+0000 7fc6d3e00700 1 -- 172.21.15.131:0/4148322870 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6cc009c40 con 0x7fc6d4108ed0 2024-05-08T22:04:32.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.861+0000 7fc6d3400700 1 --2- 172.21.15.131:0/4148322870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6d4073a00 0x7fc6d4199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:04:32.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.862+0000 7fc6d3e00700 1 --2- 172.21.15.131:0/4148322870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6d4108ed0 0x7fc6d41a0700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc00c3a0 tx=0x7fc6cc00c6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:32.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.862+0000 7fc6cbe00700 1 -- 172.21.15.131:0/4148322870 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc017070 con 0x7fc6d4108ed0 2024-05-08T22:04:32.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.862+0000 7fc6da200700 1 -- 172.21.15.131:0/4148322870 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c00095f0 con 0x7fc6d4108ed0 2024-05-08T22:04:32.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.862+0000 7fc6cbe00700 1 -- 172.21.15.131:0/4148322870 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6cc00f040 con 0x7fc6d4108ed0 2024-05-08T22:04:32.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.862+0000 7fc6cbe00700 1 -- 172.21.15.131:0/4148322870 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc00e050 con 0x7fc6d4108ed0 2024-05-08T22:04:32.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.862+0000 7fc6da200700 1 -- 172.21.15.131:0/4148322870 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d41a1590 con 0x7fc6d4108ed0 2024-05-08T22:04:32.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.863+0000 7fc6da200700 1 -- 172.21.15.131:0/4148322870 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d4107c90 con 0x7fc6d4108ed0 2024-05-08T22:04:32.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.865+0000 7fc6cbe00700 1 -- 172.21.15.131:0/4148322870 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc6cc0044c0 con 0x7fc6d4108ed0 2024-05-08T22:04:32.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.866+0000 7fc6cbe00700 1 --2- 172.21.15.131:0/4148322870 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc6b80a1120 0x7fc6b80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:32.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.866+0000 7fc6cbe00700 1 -- 172.21.15.131:0/4148322870 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6831+0+0 (secure 0 0 0) 0x7fc6cc0cbc60 con 0x7fc6d4108ed0 2024-05-08T22:04:32.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.867+0000 7fc6d3400700 1 --2- 172.21.15.131:0/4148322870 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc6b80a1120 0x7fc6b80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:32.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.867+0000 7fc6d3400700 1 --2- 172.21.15.131:0/4148322870 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc6b80a1120 0x7fc6b80a35e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc6d40753b0 tx=0x7fc6c4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:32.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:32.873+0000 7fc6cbe00700 1 -- 172.21.15.131:0/4148322870 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc6cc095920 con 0x7fc6d4108ed0 2024-05-08T22:04:32.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:32 smithi131 ceph-mon[37260]: from='osd.11 [v2:172.21.15.157:6824/2884744696,v1:172.21.15.157:6825/2884744696]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T22:04:32.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:32 smithi131 ceph-mon[37260]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T22:04:33.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:33.079+0000 7fc6da200700 1 -- 172.21.15.131:0/4148322870 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fc6d404ec60 con 0x7fc6d4108ed0 2024-05-08T22:04:33.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:33.080+0000 7fc6cbe00700 1 -- 172.21.15.131:0/4148322870 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) v1 ==== 74+0+133 (secure 0 0 0) 0x7fc6cc098f40 con 0x7fc6d4108ed0 2024-05-08T22:04:33.080 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:04:33.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:33.082+0000 7fc6da200700 1 -- 172.21.15.131:0/4148322870 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc6b80a1120 msgr2=0x7fc6b80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:33.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:33.082+0000 7fc6da200700 1 --2- 172.21.15.131:0/4148322870 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc6b80a1120 0x7fc6b80a35e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc6d40753b0 tx=0x7fc6c4009260 comp rx=0 tx=0).stop 2024-05-08T22:04:33.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:33.082+0000 7fc6da200700 1 -- 172.21.15.131:0/4148322870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6d4108ed0 msgr2=0x7fc6d41a0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:33.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:33.082+0000 7fc6da200700 1 --2- 172.21.15.131:0/4148322870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6d4108ed0 0x7fc6d41a0700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc00c3a0 tx=0x7fc6cc00c6b0 comp rx=0 tx=0).stop 2024-05-08T22:04:33.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:33.082+0000 7fc6da200700 1 -- 172.21.15.131:0/4148322870 shutdown_connections 2024-05-08T22:04:33.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:33.082+0000 7fc6da200700 1 --2- 172.21.15.131:0/4148322870 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc6b80a1120 0x7fc6b80a35e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:33.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:33.082+0000 7fc6da200700 1 --2- 172.21.15.131:0/4148322870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6d4108ed0 0x7fc6d41a0700 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:33.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:33.082+0000 7fc6da200700 1 --2- 172.21.15.131:0/4148322870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6d4073a00 0x7fc6d4199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:33.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:33.082+0000 7fc6da200700 1 --2- 172.21.15.131:0/4148322870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6d4074fd0 0x7fc6d41996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:33.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:33.082+0000 7fc6da200700 1 -- 172.21.15.131:0/4148322870 >> 172.21.15.131:0/4148322870 conn(0x7fc6d40fc5a0 msgr2=0x7fc6d4104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:33.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:33.082+0000 7fc6da200700 1 -- 172.21.15.131:0/4148322870 shutdown_connections 2024-05-08T22:04:33.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:33.082+0000 7fc6da200700 1 -- 172.21.15.131:0/4148322870 wait complete. 2024-05-08T22:04:33.538 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1715205857,"num_in_osds":12,"osd_in_since":1715205858,"num_remapped_pgs":0} 2024-05-08T22:04:33.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:33 smithi132 ceph-mon[78180]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T22:04:33.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:33 smithi132 ceph-mon[78180]: osdmap e72: 12 total, 11 up, 12 in 2024-05-08T22:04:33.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:33 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:04:33.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:33 smithi132 ceph-mon[78180]: from='osd.11 [v2:172.21.15.157:6824/2884744696,v1:172.21.15.157:6825/2884744696]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-08T22:04:33.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:33 smithi132 ceph-mon[78180]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-08T22:04:33.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:33 smithi132 ceph-mon[78180]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T22:04:33.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:33 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/4148322870' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T22:04:33.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:33 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:33.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:33 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:33.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:33 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:33.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:33 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:33.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:33 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:33.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:33 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:33.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:33 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:33.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:33 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:33.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:33 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:04:33.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:33 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:33.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:33 smithi131 ceph-mon[37260]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T22:04:33.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:33 smithi131 ceph-mon[37260]: osdmap e72: 12 total, 11 up, 12 in 2024-05-08T22:04:33.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:33 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:04:33.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:33 smithi131 ceph-mon[37260]: from='osd.11 [v2:172.21.15.157:6824/2884744696,v1:172.21.15.157:6825/2884744696]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-08T22:04:33.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:33 smithi131 ceph-mon[37260]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-08T22:04:33.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:33 smithi131 ceph-mon[37260]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T22:04:33.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:33 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/4148322870' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T22:04:33.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:33 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:33.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:33 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:33.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:33 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:33.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:33 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:33.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:33 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:33.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:33 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:33.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:33 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:33.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:33 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:33.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:33 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:04:33.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:33 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:34.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:33 smithi157 ceph-mon[77746]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T22:04:34.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:33 smithi157 ceph-mon[77746]: osdmap e72: 12 total, 11 up, 12 in 2024-05-08T22:04:34.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:33 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:04:34.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:33 smithi157 ceph-mon[77746]: from='osd.11 [v2:172.21.15.157:6824/2884744696,v1:172.21.15.157:6825/2884744696]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-08T22:04:34.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:33 smithi157 ceph-mon[77746]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-08T22:04:34.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:33 smithi157 ceph-mon[77746]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T22:04:34.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:33 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/4148322870' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T22:04:34.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:33 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:34.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:33 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:34.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:33 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:34.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:33 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:34.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:33 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:34.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:33 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:04:34.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:33 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:34.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:33 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:34.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:33 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:04:34.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:33 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:34.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:04:34 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:04:34.104+0000 7f77afb4c700 -1 osd.11 0 waiting for initial osdmap 2024-05-08T22:04:34.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:04:34 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:04:34.123+0000 7f77a7939700 -1 osd.11 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:04:34.539 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-08T22:04:34.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:34 smithi132 ceph-mon[78180]: Detected new or changed devices on smithi157 2024-05-08T22:04:34.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:34 smithi132 ceph-mon[78180]: Adjusting osd_memory_target on smithi157 to 4278M 2024-05-08T22:04:34.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:34 smithi132 ceph-mon[78180]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-08T22:04:34.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:34 smithi132 ceph-mon[78180]: osdmap e73: 12 total, 11 up, 12 in 2024-05-08T22:04:34.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:34 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:04:34.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:34 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:04:34.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:34 smithi132 ceph-mon[78180]: from='osd.11 ' entity='osd.11' 2024-05-08T22:04:34.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:34 smithi131 ceph-mon[37260]: Detected new or changed devices on smithi157 2024-05-08T22:04:34.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:34 smithi131 ceph-mon[37260]: Adjusting osd_memory_target on smithi157 to 4278M 2024-05-08T22:04:34.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:34 smithi131 ceph-mon[37260]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-08T22:04:34.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:34 smithi131 ceph-mon[37260]: osdmap e73: 12 total, 11 up, 12 in 2024-05-08T22:04:34.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:34 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:04:34.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:34 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:04:34.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:34 smithi131 ceph-mon[37260]: from='osd.11 ' entity='osd.11' 2024-05-08T22:04:34.905 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:04:35.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:34 smithi157 ceph-mon[77746]: Detected new or changed devices on smithi157 2024-05-08T22:04:35.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:34 smithi157 ceph-mon[77746]: Adjusting osd_memory_target on smithi157 to 4278M 2024-05-08T22:04:35.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:34 smithi157 ceph-mon[77746]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-08T22:04:35.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:34 smithi157 ceph-mon[77746]: osdmap e73: 12 total, 11 up, 12 in 2024-05-08T22:04:35.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:34 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:04:35.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:34 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:04:35.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:34 smithi157 ceph-mon[77746]: from='osd.11 ' entity='osd.11' 2024-05-08T22:04:35.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:35 smithi132 ceph-mon[78180]: purged_snaps scrub starts 2024-05-08T22:04:35.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:35 smithi132 ceph-mon[78180]: purged_snaps scrub ok 2024-05-08T22:04:35.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:04:35.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:35 smithi132 ceph-mon[78180]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-08T22:04:35.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:35 smithi132 ceph-mon[78180]: osd.11 [v2:172.21.15.157:6824/2884744696,v1:172.21.15.157:6825/2884744696] boot 2024-05-08T22:04:35.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:35 smithi132 ceph-mon[78180]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T22:04:35.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:35 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:04:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:35 smithi131 ceph-mon[37260]: purged_snaps scrub starts 2024-05-08T22:04:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:35 smithi131 ceph-mon[37260]: purged_snaps scrub ok 2024-05-08T22:04:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:04:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:35 smithi131 ceph-mon[37260]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-08T22:04:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:35 smithi131 ceph-mon[37260]: osd.11 [v2:172.21.15.157:6824/2884744696,v1:172.21.15.157:6825/2884744696] boot 2024-05-08T22:04:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:35 smithi131 ceph-mon[37260]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T22:04:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:35 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:04:36.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:35 smithi157 ceph-mon[77746]: purged_snaps scrub starts 2024-05-08T22:04:36.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:35 smithi157 ceph-mon[77746]: purged_snaps scrub ok 2024-05-08T22:04:36.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:04:36.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:35 smithi157 ceph-mon[77746]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-08T22:04:36.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:35 smithi157 ceph-mon[77746]: osd.11 [v2:172.21.15.157:6824/2884744696,v1:172.21.15.157:6825/2884744696] boot 2024-05-08T22:04:36.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:35 smithi157 ceph-mon[77746]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T22:04:36.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:35 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:04:36.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.235+0000 7f33bd400700 1 -- 172.21.15.131:0/2106162798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33b8100e10 msgr2=0x7f33b8101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:36.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.235+0000 7f33bd400700 1 --2- 172.21.15.131:0/2106162798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33b8100e10 0x7f33b8101230 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f33a8009990 tx=0x7f33a8009ca0 comp rx=0 tx=0).stop 2024-05-08T22:04:36.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.235+0000 7f33bd400700 1 -- 172.21.15.131:0/2106162798 shutdown_connections 2024-05-08T22:04:36.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.235+0000 7f33bd400700 1 --2- 172.21.15.131:0/2106162798 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33b8102930 0x7f33b8109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:36.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.235+0000 7f33bd400700 1 --2- 172.21.15.131:0/2106162798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33b8100e10 0x7f33b8101230 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:36.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.235+0000 7f33bd400700 1 --2- 172.21.15.131:0/2106162798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b8101f70 0x7f33b81023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:36.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.235+0000 7f33bd400700 1 -- 172.21.15.131:0/2106162798 >> 172.21.15.131:0/2106162798 conn(0x7f33b80fc3b0 msgr2=0x7f33b80fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:36.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.235+0000 7f33bd400700 1 -- 172.21.15.131:0/2106162798 shutdown_connections 2024-05-08T22:04:36.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.236+0000 7f33bd400700 1 -- 172.21.15.131:0/2106162798 wait complete. 2024-05-08T22:04:36.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.237+0000 7f33bd400700 1 Processor -- start 2024-05-08T22:04:36.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.237+0000 7f33bd400700 1 -- start start 2024-05-08T22:04:36.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.237+0000 7f33bd400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33b8100e10 0x7f33b8198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:36.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.238+0000 7f33bd400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33b8101f70 0x7f33b8199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:36.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.238+0000 7f33bd400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b8102930 0x7f33b81a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:36.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.238+0000 7f33bd400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b81a0a20 con 0x7f33b8101f70 2024-05-08T22:04:36.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.238+0000 7f33bd400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b81a0b60 con 0x7f33b8100e10 2024-05-08T22:04:36.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.238+0000 7f33bd400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b81a0cf0 con 0x7f33b8102930 2024-05-08T22:04:36.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.238+0000 7f33b7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33b8101f70 0x7f33b8199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:36.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.238+0000 7f33b7e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33b8100e10 0x7f33b8198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:36.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.238+0000 7f33b7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33b8101f70 0x7f33b8199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50662/0 (socket says 172.21.15.131:50662) 2024-05-08T22:04:36.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.238+0000 7f33b7400700 1 -- 172.21.15.131:0/2290875515 learned_addr learned my addr 172.21.15.131:0/2290875515 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:04:36.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.238+0000 7f33bca00700 1 --2- 172.21.15.131:0/2290875515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b8102930 0x7f33b81a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:36.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.238+0000 7f33b7e00700 1 -- 172.21.15.131:0/2290875515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b8102930 msgr2=0x7f33b81a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:36.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.238+0000 7f33b7e00700 1 --2- 172.21.15.131:0/2290875515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b8102930 0x7f33b81a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:36.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.238+0000 7f33b7e00700 1 -- 172.21.15.131:0/2290875515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33b8101f70 msgr2=0x7f33b8199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:36.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.238+0000 7f33b7e00700 1 --2- 172.21.15.131:0/2290875515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33b8101f70 0x7f33b8199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:36.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.238+0000 7f33b7e00700 1 -- 172.21.15.131:0/2290875515 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a80095f0 con 0x7f33b8100e10 2024-05-08T22:04:36.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.238+0000 7f33b7400700 1 --2- 172.21.15.131:0/2290875515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33b8101f70 0x7f33b8199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:04:36.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.238+0000 7f33bca00700 1 --2- 172.21.15.131:0/2290875515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b8102930 0x7f33b81a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:04:36.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.238+0000 7f33b7e00700 1 --2- 172.21.15.131:0/2290875515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33b8100e10 0x7f33b8198f30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f33a8000c00 tx=0x7f33a8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:36.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.239+0000 7f33b4c00700 1 -- 172.21.15.131:0/2290875515 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a801d070 con 0x7f33b8100e10 2024-05-08T22:04:36.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.239+0000 7f33b4c00700 1 -- 172.21.15.131:0/2290875515 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33a8021410 con 0x7f33b8100e10 2024-05-08T22:04:36.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.239+0000 7f33bd400700 1 -- 172.21.15.131:0/2290875515 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b81a0f70 con 0x7f33b8100e10 2024-05-08T22:04:36.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.239+0000 7f33b4c00700 1 -- 172.21.15.131:0/2290875515 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a8017650 con 0x7f33b8100e10 2024-05-08T22:04:36.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.239+0000 7f33bd400700 1 -- 172.21.15.131:0/2290875515 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b81a12d0 con 0x7f33b8100e10 2024-05-08T22:04:36.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.240+0000 7f33bd400700 1 -- 172.21.15.131:0/2290875515 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b804ec60 con 0x7f33b8100e10 2024-05-08T22:04:36.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.242+0000 7f33b4c00700 1 -- 172.21.15.131:0/2290875515 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f33a8021580 con 0x7f33b8100e10 2024-05-08T22:04:36.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.243+0000 7f33b4c00700 1 --2- 172.21.15.131:0/2290875515 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f339c0a1120 0x7f339c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:36.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.243+0000 7f33b4c00700 1 -- 172.21.15.131:0/2290875515 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f33a80cd750 con 0x7f33b8100e10 2024-05-08T22:04:36.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.243+0000 7f33b7400700 1 --2- 172.21.15.131:0/2290875515 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f339c0a1120 0x7f339c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:36.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.244+0000 7f33b7400700 1 --2- 172.21.15.131:0/2290875515 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f339c0a1120 0x7f339c0a35e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f33a4011550 tx=0x7f33a4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:36.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.246+0000 7f33b4c00700 1 -- 172.21.15.131:0/2290875515 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f33a8097370 con 0x7f33b8100e10 2024-05-08T22:04:36.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.453+0000 7f33bd400700 1 -- 172.21.15.131:0/2290875515 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f33b810c620 con 0x7f33b8100e10 2024-05-08T22:04:36.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.454+0000 7f33b4c00700 1 -- 172.21.15.131:0/2290875515 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) v1 ==== 74+0+133 (secure 0 0 0) 0x7f33a809a990 con 0x7f33b8100e10 2024-05-08T22:04:36.455 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:04:36.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.457+0000 7f33bd400700 1 -- 172.21.15.131:0/2290875515 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f339c0a1120 msgr2=0x7f339c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:36.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.457+0000 7f33bd400700 1 --2- 172.21.15.131:0/2290875515 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f339c0a1120 0x7f339c0a35e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f33a4011550 tx=0x7f33a4009260 comp rx=0 tx=0).stop 2024-05-08T22:04:36.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.457+0000 7f33bd400700 1 -- 172.21.15.131:0/2290875515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33b8100e10 msgr2=0x7f33b8198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:36.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.457+0000 7f33bd400700 1 --2- 172.21.15.131:0/2290875515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33b8100e10 0x7f33b8198f30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f33a8000c00 tx=0x7f33a8004730 comp rx=0 tx=0).stop 2024-05-08T22:04:36.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.457+0000 7f33bd400700 1 -- 172.21.15.131:0/2290875515 shutdown_connections 2024-05-08T22:04:36.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.457+0000 7f33bd400700 1 --2- 172.21.15.131:0/2290875515 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f339c0a1120 0x7f339c0a35e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:36.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.457+0000 7f33bd400700 1 --2- 172.21.15.131:0/2290875515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b8102930 0x7f33b81a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:36.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.457+0000 7f33bd400700 1 --2- 172.21.15.131:0/2290875515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33b8100e10 0x7f33b8198f30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:36.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.457+0000 7f33bd400700 1 --2- 172.21.15.131:0/2290875515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33b8101f70 0x7f33b8199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:36.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.457+0000 7f33bd400700 1 -- 172.21.15.131:0/2290875515 >> 172.21.15.131:0/2290875515 conn(0x7f33b80fc3b0 msgr2=0x7f33b80731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:36.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.457+0000 7f33bd400700 1 -- 172.21.15.131:0/2290875515 shutdown_connections 2024-05-08T22:04:36.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:36.457+0000 7f33bd400700 1 -- 172.21.15.131:0/2290875515 wait complete. 2024-05-08T22:04:36.956 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":75,"num_osds":12,"num_up_osds":12,"osd_up_since":1715205875,"num_in_osds":12,"osd_in_since":1715205858,"num_remapped_pgs":0} 2024-05-08T22:04:36.956 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T22:04:37.318 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:04:37.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:37 smithi131 ceph-mon[37260]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T22:04:37.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:37 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2290875515' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T22:04:37.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:37 smithi131 ceph-mon[37260]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:37.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:37 smithi157 ceph-mon[77746]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T22:04:37.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:37 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2290875515' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T22:04:37.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:37 smithi157 ceph-mon[77746]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:37.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:37 smithi132 ceph-mon[78180]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T22:04:37.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:37 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2290875515' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T22:04:37.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:37 smithi132 ceph-mon[78180]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:38.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.553+0000 7fccec600700 1 -- 172.21.15.131:0/3542496128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce4100ff0 msgr2=0x7fcce4101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:38.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.553+0000 7fccec600700 1 --2- 172.21.15.131:0/3542496128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce4100ff0 0x7fcce4101450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009990 tx=0x7fccd0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:04:38.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.553+0000 7fccec600700 1 -- 172.21.15.131:0/3542496128 shutdown_connections 2024-05-08T22:04:38.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.553+0000 7fccec600700 1 --2- 172.21.15.131:0/3542496128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcce4101990 0x7fcce410de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:38.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.553+0000 7fccec600700 1 --2- 172.21.15.131:0/3542496128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcce4107030 0x7fcce4107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:38.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.553+0000 7fccec600700 1 --2- 172.21.15.131:0/3542496128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce4100ff0 0x7fcce4101450 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:38.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.553+0000 7fccec600700 1 -- 172.21.15.131:0/3542496128 >> 172.21.15.131:0/3542496128 conn(0x7fcce4078190 msgr2=0x7fcce40785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:38.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.553+0000 7fccec600700 1 -- 172.21.15.131:0/3542496128 shutdown_connections 2024-05-08T22:04:38.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.553+0000 7fccec600700 1 -- 172.21.15.131:0/3542496128 wait complete. 2024-05-08T22:04:38.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.554+0000 7fccec600700 1 Processor -- start 2024-05-08T22:04:38.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.554+0000 7fccec600700 1 -- start start 2024-05-08T22:04:38.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.554+0000 7fccec600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcce4100ff0 0x7fcce4199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:38.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.554+0000 7fccec600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce4101990 0x7fcce4199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:38.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.554+0000 7fccec600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcce4107030 0x7fcce419df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:38.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.554+0000 7fccec600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce419e550 con 0x7fcce4101990 2024-05-08T22:04:38.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.554+0000 7fccec600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce419e6c0 con 0x7fcce4107030 2024-05-08T22:04:38.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.554+0000 7fccec600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce419e880 con 0x7fcce4100ff0 2024-05-08T22:04:38.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.555+0000 7fcce9200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce4101990 0x7fcce4199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:38.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.555+0000 7fcce9200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce4101990 0x7fcce4199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50678/0 (socket says 172.21.15.131:50678) 2024-05-08T22:04:38.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.555+0000 7fcce9200700 1 -- 172.21.15.131:0/1847778959 learned_addr learned my addr 172.21.15.131:0/1847778959 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:04:38.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.555+0000 7fcce9c00700 1 --2- 172.21.15.131:0/1847778959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcce4100ff0 0x7fcce4199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:38.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.555+0000 7fccea600700 1 --2- 172.21.15.131:0/1847778959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcce4107030 0x7fcce419df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:38.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.555+0000 7fcce9200700 1 -- 172.21.15.131:0/1847778959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcce4100ff0 msgr2=0x7fcce4199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:38.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.555+0000 7fcce9200700 1 --2- 172.21.15.131:0/1847778959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcce4100ff0 0x7fcce4199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:38.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.555+0000 7fcce9200700 1 -- 172.21.15.131:0/1847778959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcce4107030 msgr2=0x7fcce419df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:38.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.555+0000 7fcce9200700 1 --2- 172.21.15.131:0/1847778959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcce4107030 0x7fcce419df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:38.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.555+0000 7fcce9200700 1 -- 172.21.15.131:0/1847778959 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd00095f0 con 0x7fcce4101990 2024-05-08T22:04:38.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.555+0000 7fcce9200700 1 --2- 172.21.15.131:0/1847778959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce4101990 0x7fcce4199b90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fccd00095c0 tx=0x7fccd000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:38.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.556+0000 7fcce2000700 1 -- 172.21.15.131:0/1847778959 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd001d070 con 0x7fcce4101990 2024-05-08T22:04:38.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.556+0000 7fcce2000700 1 -- 172.21.15.131:0/1847778959 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccd00054f0 con 0x7fcce4101990 2024-05-08T22:04:38.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.556+0000 7fccec600700 1 -- 172.21.15.131:0/1847778959 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce419eb00 con 0x7fcce4101990 2024-05-08T22:04:38.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.556+0000 7fcce2000700 1 -- 172.21.15.131:0/1847778959 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd0004bd0 con 0x7fcce4101990 2024-05-08T22:04:38.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.556+0000 7fccec600700 1 -- 172.21.15.131:0/1847778959 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce41a63c0 con 0x7fcce4101990 2024-05-08T22:04:38.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.557+0000 7fccec600700 1 -- 172.21.15.131:0/1847778959 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcce4104370 con 0x7fcce4101990 2024-05-08T22:04:38.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.559+0000 7fcce2000700 1 -- 172.21.15.131:0/1847778959 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fccd000fb80 con 0x7fcce4101990 2024-05-08T22:04:38.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.560+0000 7fcce2000700 1 --2- 172.21.15.131:0/1847778959 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fcccc0a11a0 0x7fcccc0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:38.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.560+0000 7fcce2000700 1 -- 172.21.15.131:0/1847778959 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fccd00cd5f0 con 0x7fcce4101990 2024-05-08T22:04:38.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.560+0000 7fcce9c00700 1 --2- 172.21.15.131:0/1847778959 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fcccc0a11a0 0x7fcccc0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:38.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.561+0000 7fcce9c00700 1 --2- 172.21.15.131:0/1847778959 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fcccc0a11a0 0x7fcccc0a3660 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fccd8005950 tx=0x7fccd800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:38.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.563+0000 7fcce2000700 1 -- 172.21.15.131:0/1847778959 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fccd0097210 con 0x7fcce4101990 2024-05-08T22:04:38.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.769+0000 7fccec600700 1 -- 172.21.15.131:0/1847778959 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcce419a960 con 0x7fcce4101990 2024-05-08T22:04:38.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.770+0000 7fcce2000700 1 -- 172.21.15.131:0/1847778959 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+18469 (secure 0 0 0) 0x7fccd009a830 con 0x7fcce4101990 2024-05-08T22:04:38.772 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:04:38.772 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":75,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:04:36.108486+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:04:38.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.773+0000 7fccec600700 1 -- 172.21.15.131:0/1847778959 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fcccc0a11a0 msgr2=0x7fcccc0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:38.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.773+0000 7fccec600700 1 --2- 172.21.15.131:0/1847778959 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fcccc0a11a0 0x7fcccc0a3660 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fccd8005950 tx=0x7fccd800b410 comp rx=0 tx=0).stop 2024-05-08T22:04:38.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.773+0000 7fccec600700 1 -- 172.21.15.131:0/1847778959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce4101990 msgr2=0x7fcce4199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:38.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.773+0000 7fccec600700 1 --2- 172.21.15.131:0/1847778959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce4101990 0x7fcce4199b90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fccd00095c0 tx=0x7fccd000f690 comp rx=0 tx=0).stop 2024-05-08T22:04:38.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.773+0000 7fccec600700 1 -- 172.21.15.131:0/1847778959 shutdown_connections 2024-05-08T22:04:38.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.773+0000 7fccec600700 1 --2- 172.21.15.131:0/1847778959 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fcccc0a11a0 0x7fcccc0a3660 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:38.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.773+0000 7fccec600700 1 --2- 172.21.15.131:0/1847778959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcce4107030 0x7fcce419df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:38.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.773+0000 7fccec600700 1 --2- 172.21.15.131:0/1847778959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcce4100ff0 0x7fcce4199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:38.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.773+0000 7fccec600700 1 --2- 172.21.15.131:0/1847778959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce4101990 0x7fcce4199b90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:38.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.773+0000 7fccec600700 1 -- 172.21.15.131:0/1847778959 >> 172.21.15.131:0/1847778959 conn(0x7fcce4078190 msgr2=0x7fcce410c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:38.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.773+0000 7fccec600700 1 -- 172.21.15.131:0/1847778959 shutdown_connections 2024-05-08T22:04:38.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:38.773+0000 7fccec600700 1 -- 172.21.15.131:0/1847778959 wait complete. 2024-05-08T22:04:39.203 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:01:41.271078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:04:39.204 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-08T22:04:39.561 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:04:40.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:39 smithi157 ceph-mon[77746]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:40.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:39 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1847778959' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:04:40.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:39 smithi132 ceph-mon[78180]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:40.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:39 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1847778959' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:04:40.094 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:39 smithi131 ceph-mon[37260]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:40.094 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:39 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1847778959' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:04:40.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.783+0000 7f8132c00700 1 -- 172.21.15.131:0/1688964504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f812c107030 msgr2=0x7f812c107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:40.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.783+0000 7f8132c00700 1 --2- 172.21.15.131:0/1688964504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f812c107030 0x7f812c107410 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f811c009990 tx=0x7f811c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:04:40.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.783+0000 7f8132c00700 1 -- 172.21.15.131:0/1688964504 shutdown_connections 2024-05-08T22:04:40.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.783+0000 7f8132c00700 1 --2- 172.21.15.131:0/1688964504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f812c101990 0x7f812c10de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:40.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.783+0000 7f8132c00700 1 --2- 172.21.15.131:0/1688964504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f812c107030 0x7f812c107410 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:40.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.783+0000 7f8132c00700 1 --2- 172.21.15.131:0/1688964504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f812c100ff0 0x7f812c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:40.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.783+0000 7f8132c00700 1 -- 172.21.15.131:0/1688964504 >> 172.21.15.131:0/1688964504 conn(0x7f812c078190 msgr2=0x7f812c0785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:40.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.783+0000 7f8132c00700 1 -- 172.21.15.131:0/1688964504 shutdown_connections 2024-05-08T22:04:40.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.783+0000 7f8132c00700 1 -- 172.21.15.131:0/1688964504 wait complete. 2024-05-08T22:04:40.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.784+0000 7f8132c00700 1 Processor -- start 2024-05-08T22:04:40.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.784+0000 7f8132c00700 1 -- start start 2024-05-08T22:04:40.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.785+0000 7f8132c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f812c100ff0 0x7f812c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:40.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.785+0000 7f8132c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f812c101990 0x7f812c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:40.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.785+0000 7f8132c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f812c107030 0x7f812c073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:40.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.785+0000 7f8132c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c1a5770 con 0x7f812c100ff0 2024-05-08T22:04:40.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.785+0000 7f8132c00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c1a58e0 con 0x7f812c101990 2024-05-08T22:04:40.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.785+0000 7f8132c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c1a5aa0 con 0x7f812c107030 2024-05-08T22:04:40.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.785+0000 7f812be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f812c100ff0 0x7f812c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:40.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.785+0000 7f812be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f812c100ff0 0x7f812c0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50696/0 (socket says 172.21.15.131:50696) 2024-05-08T22:04:40.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.785+0000 7f812be00700 1 -- 172.21.15.131:0/1379586405 learned_addr learned my addr 172.21.15.131:0/1379586405 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:04:40.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.785+0000 7f812b400700 1 --2- 172.21.15.131:0/1379586405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f812c101990 0x7f812c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:40.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.785+0000 7f8130c00700 1 --2- 172.21.15.131:0/1379586405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f812c107030 0x7f812c073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:40.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.785+0000 7f812be00700 1 -- 172.21.15.131:0/1379586405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f812c107030 msgr2=0x7f812c073d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:40.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.785+0000 7f812be00700 1 --2- 172.21.15.131:0/1379586405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f812c107030 0x7f812c073d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:40.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.785+0000 7f812be00700 1 -- 172.21.15.131:0/1379586405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f812c101990 msgr2=0x7f812c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:40.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.785+0000 7f812be00700 1 --2- 172.21.15.131:0/1379586405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f812c101990 0x7f812c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:40.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.785+0000 7f812be00700 1 -- 172.21.15.131:0/1379586405 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811c0095f0 con 0x7f812c100ff0 2024-05-08T22:04:40.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.786+0000 7f812be00700 1 --2- 172.21.15.131:0/1379586405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f812c100ff0 0x7f812c0732a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f811c0095c0 tx=0x7f811c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:40.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.786+0000 7f8128c00700 1 -- 172.21.15.131:0/1379586405 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c01d070 con 0x7f812c100ff0 2024-05-08T22:04:40.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.786+0000 7f8128c00700 1 -- 172.21.15.131:0/1379586405 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f811c005030 con 0x7f812c100ff0 2024-05-08T22:04:40.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.786+0000 7f8132c00700 1 -- 172.21.15.131:0/1379586405 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f812c1a5d20 con 0x7f812c100ff0 2024-05-08T22:04:40.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.786+0000 7f8128c00700 1 -- 172.21.15.131:0/1379586405 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c004a90 con 0x7f812c100ff0 2024-05-08T22:04:40.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.786+0000 7f8132c00700 1 -- 172.21.15.131:0/1379586405 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812c1a6190 con 0x7f812c100ff0 2024-05-08T22:04:40.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.787+0000 7f8132c00700 1 -- 172.21.15.131:0/1379586405 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f812c109550 con 0x7f812c100ff0 2024-05-08T22:04:40.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.789+0000 7f8128c00700 1 -- 172.21.15.131:0/1379586405 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f811c005410 con 0x7f812c100ff0 2024-05-08T22:04:40.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.790+0000 7f8128c00700 1 --2- 172.21.15.131:0/1379586405 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f81100a11a0 0x7f81100a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:40.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.790+0000 7f8128c00700 1 -- 172.21.15.131:0/1379586405 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f811c0cd5b0 con 0x7f812c100ff0 2024-05-08T22:04:40.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.791+0000 7f812b400700 1 --2- 172.21.15.131:0/1379586405 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f81100a11a0 0x7f81100a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:40.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.791+0000 7f812b400700 1 --2- 172.21.15.131:0/1379586405 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f81100a11a0 0x7f81100a3660 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f812c0749d0 tx=0x7f811800b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:40.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.793+0000 7f8128c00700 1 -- 172.21.15.131:0/1379586405 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f811c0971d0 con 0x7f812c100ff0 2024-05-08T22:04:40.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.997+0000 7f8132c00700 1 -- 172.21.15.131:0/1379586405 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f812c074780 con 0x7f812c100ff0 2024-05-08T22:04:40.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:40.997+0000 7f8128c00700 1 -- 172.21.15.131:0/1379586405 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f811c09a7f0 con 0x7f812c100ff0 2024-05-08T22:04:40.998 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-08T22:04:41.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:41.000+0000 7f8132c00700 1 -- 172.21.15.131:0/1379586405 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f81100a11a0 msgr2=0x7f81100a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:41.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:41.000+0000 7f8132c00700 1 --2- 172.21.15.131:0/1379586405 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f81100a11a0 0x7f81100a3660 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f812c0749d0 tx=0x7f811800b3c0 comp rx=0 tx=0).stop 2024-05-08T22:04:41.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:41.000+0000 7f8132c00700 1 -- 172.21.15.131:0/1379586405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f812c100ff0 msgr2=0x7f812c0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:41.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:41.000+0000 7f8132c00700 1 --2- 172.21.15.131:0/1379586405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f812c100ff0 0x7f812c0732a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f811c0095c0 tx=0x7f811c00f690 comp rx=0 tx=0).stop 2024-05-08T22:04:41.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:41.000+0000 7f8132c00700 1 -- 172.21.15.131:0/1379586405 shutdown_connections 2024-05-08T22:04:41.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:41.000+0000 7f8132c00700 1 --2- 172.21.15.131:0/1379586405 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f81100a11a0 0x7f81100a3660 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:41.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:41.000+0000 7f8132c00700 1 --2- 172.21.15.131:0/1379586405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f812c107030 0x7f812c073d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:41.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:41.000+0000 7f8132c00700 1 --2- 172.21.15.131:0/1379586405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f812c100ff0 0x7f812c0732a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:41.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:41.000+0000 7f8132c00700 1 --2- 172.21.15.131:0/1379586405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f812c101990 0x7f812c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:41.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:41.000+0000 7f8132c00700 1 -- 172.21.15.131:0/1379586405 >> 172.21.15.131:0/1379586405 conn(0x7f812c078190 msgr2=0x7f812c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:41.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:41.000+0000 7f8132c00700 1 -- 172.21.15.131:0/1379586405 shutdown_connections 2024-05-08T22:04:41.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:41.000+0000 7f8132c00700 1 -- 172.21.15.131:0/1379586405 wait complete. 2024-05-08T22:04:41.434 INFO:tasks.cephadm:Adding mds.a on smithi131 2024-05-08T22:04:41.435 INFO:tasks.cephadm:Adding mds.d on smithi131 2024-05-08T22:04:41.435 INFO:tasks.cephadm:Adding mds.g on smithi131 2024-05-08T22:04:41.435 INFO:tasks.cephadm:Adding mds.j on smithi131 2024-05-08T22:04:41.435 INFO:tasks.cephadm:Adding mds.b on smithi132 2024-05-08T22:04:41.435 INFO:tasks.cephadm:Adding mds.e on smithi132 2024-05-08T22:04:41.435 INFO:tasks.cephadm:Adding mds.h on smithi132 2024-05-08T22:04:41.435 INFO:tasks.cephadm:Adding mds.k on smithi132 2024-05-08T22:04:41.435 INFO:tasks.cephadm:Adding mds.c on smithi157 2024-05-08T22:04:41.435 INFO:tasks.cephadm:Adding mds.f on smithi157 2024-05-08T22:04:41.435 INFO:tasks.cephadm:Adding mds.i on smithi157 2024-05-08T22:04:41.435 INFO:tasks.cephadm:Adding mds.l on smithi157 2024-05-08T22:04:41.435 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l' 2024-05-08T22:04:41.672 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:04:42.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:41 smithi157 ceph-mon[77746]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:42.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:41 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1379586405' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:04:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:41 smithi132 ceph-mon[78180]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:41 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1379586405' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:04:42.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:41 smithi131 ceph-mon[37260]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:42.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:41 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1379586405' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:04:42.642 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.641+0000 7fc195b3e700 1 -- 172.21.15.157:0/2258351887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1901012b0 msgr2=0x7fc190107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:42.642 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.641+0000 7fc195b3e700 1 --2- 172.21.15.157:0/2258351887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1901012b0 0x7fc190107b60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc18800b0a0 tx=0x7fc18800b3b0 comp rx=0 tx=0).stop 2024-05-08T22:04:42.642 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.641+0000 7fc195b3e700 1 -- 172.21.15.157:0/2258351887 shutdown_connections 2024-05-08T22:04:42.642 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.641+0000 7fc195b3e700 1 --2- 172.21.15.157:0/2258351887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1901012b0 0x7fc190107b60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:42.642 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.641+0000 7fc195b3e700 1 --2- 172.21.15.157:0/2258351887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1900ff710 0x7fc1900ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:42.642 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.641+0000 7fc195b3e700 1 --2- 172.21.15.157:0/2258351887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc190100910 0x7fc190100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:42.642 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.641+0000 7fc195b3e700 1 -- 172.21.15.157:0/2258351887 >> 172.21.15.157:0/2258351887 conn(0x7fc1900facb0 msgr2=0x7fc1900fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:42.643 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.642+0000 7fc195b3e700 1 -- 172.21.15.157:0/2258351887 shutdown_connections 2024-05-08T22:04:42.643 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.642+0000 7fc195b3e700 1 -- 172.21.15.157:0/2258351887 wait complete. 2024-05-08T22:04:42.643 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.642+0000 7fc195b3e700 1 Processor -- start 2024-05-08T22:04:42.644 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.642+0000 7fc195b3e700 1 -- start start 2024-05-08T22:04:42.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.642+0000 7fc195b3e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1900ff710 0x7fc190197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:42.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.642+0000 7fc195b3e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc190100910 0x7fc190197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:42.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.642+0000 7fc195b3e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1901012b0 0x7fc19019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:42.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.642+0000 7fc195b3e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19019ec40 con 0x7fc190100910 2024-05-08T22:04:42.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.642+0000 7fc195b3e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19019ed80 con 0x7fc1901012b0 2024-05-08T22:04:42.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.642+0000 7fc195b3e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19019ef10 con 0x7fc1900ff710 2024-05-08T22:04:42.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.643+0000 7fc18f7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1900ff710 0x7fc190197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:42.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.643+0000 7fc18f7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1900ff710 0x7fc190197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46434/0 (socket says 172.21.15.157:46434) 2024-05-08T22:04:42.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.643+0000 7fc18f7fe700 1 -- 172.21.15.157:0/2017733341 learned_addr learned my addr 172.21.15.157:0/2017733341 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-08T22:04:42.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.643+0000 7fc18ffff700 1 --2- 172.21.15.157:0/2017733341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1901012b0 0x7fc19019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:42.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.643+0000 7fc18f7fe700 1 -- 172.21.15.157:0/2017733341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1901012b0 msgr2=0x7fc19019e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:42.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.643+0000 7fc18f7fe700 1 --2- 172.21.15.157:0/2017733341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1901012b0 0x7fc19019e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:42.646 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.643+0000 7fc18f7fe700 1 -- 172.21.15.157:0/2017733341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc190100910 msgr2=0x7fc190197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:42.646 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.643+0000 7fc18f7fe700 1 --2- 172.21.15.157:0/2017733341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc190100910 0x7fc190197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:42.646 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.643+0000 7fc18f7fe700 1 -- 172.21.15.157:0/2017733341 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc188009d00 con 0x7fc1900ff710 2024-05-08T22:04:42.646 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.643+0000 7fc18f7fe700 1 --2- 172.21.15.157:0/2017733341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1900ff710 0x7fc190197100 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc18000b530 tx=0x7fc18000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:42.646 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.643+0000 7fc18cff9700 1 -- 172.21.15.157:0/2017733341 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc180011830 con 0x7fc1900ff710 2024-05-08T22:04:42.646 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.643+0000 7fc18cff9700 1 -- 172.21.15.157:0/2017733341 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc180011e50 con 0x7fc1900ff710 2024-05-08T22:04:42.646 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.643+0000 7fc195b3e700 1 -- 172.21.15.157:0/2017733341 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc19019f1f0 con 0x7fc1900ff710 2024-05-08T22:04:42.646 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.643+0000 7fc18cff9700 1 -- 172.21.15.157:0/2017733341 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18000f2e0 con 0x7fc1900ff710 2024-05-08T22:04:42.646 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.643+0000 7fc195b3e700 1 -- 172.21.15.157:0/2017733341 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc19019f6f0 con 0x7fc1900ff710 2024-05-08T22:04:42.646 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.644+0000 7fc195b3e700 1 -- 172.21.15.157:0/2017733341 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1901074f0 con 0x7fc1900ff710 2024-05-08T22:04:42.647 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.645+0000 7fc18cff9700 1 -- 172.21.15.157:0/2017733341 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc180010580 con 0x7fc1900ff710 2024-05-08T22:04:42.647 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.646+0000 7fc18cff9700 1 --2- 172.21.15.157:0/2017733341 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc1780a1120 0x7fc1780a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:42.647 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.646+0000 7fc18cff9700 1 -- 172.21.15.157:0/2017733341 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc1800cbe90 con 0x7fc1900ff710 2024-05-08T22:04:42.647 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.646+0000 7fc18effd700 1 --2- 172.21.15.157:0/2017733341 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc1780a1120 0x7fc1780a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:42.647 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.646+0000 7fc18effd700 1 --2- 172.21.15.157:0/2017733341 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc1780a1120 0x7fc1780a35e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc18400a990 tx=0x7fc184005c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:42.650 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.650+0000 7fc18cff9700 1 -- 172.21.15.157:0/2017733341 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc180095ac0 con 0x7fc1900ff710 2024-05-08T22:04:42.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.852+0000 7fc195b3e700 1 -- 172.21.15.157:0/2017733341 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fc19002cf10 con 0x7fc1780a1120 2024-05-08T22:04:42.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.862+0000 7fc18cff9700 1 -- 172.21.15.157:0/2017733341 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fc19002cf10 con 0x7fc1780a1120 2024-05-08T22:04:42.864 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mds.all update... 2024-05-08T22:04:42.866 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.865+0000 7fc195b3e700 1 -- 172.21.15.157:0/2017733341 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc1780a1120 msgr2=0x7fc1780a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:42.866 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.865+0000 7fc195b3e700 1 --2- 172.21.15.157:0/2017733341 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc1780a1120 0x7fc1780a35e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc18400a990 tx=0x7fc184005c80 comp rx=0 tx=0).stop 2024-05-08T22:04:42.866 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.865+0000 7fc195b3e700 1 -- 172.21.15.157:0/2017733341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1900ff710 msgr2=0x7fc190197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:42.866 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.865+0000 7fc195b3e700 1 --2- 172.21.15.157:0/2017733341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1900ff710 0x7fc190197100 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc18000b530 tx=0x7fc18000b840 comp rx=0 tx=0).stop 2024-05-08T22:04:42.866 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.865+0000 7fc195b3e700 1 -- 172.21.15.157:0/2017733341 shutdown_connections 2024-05-08T22:04:42.866 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.865+0000 7fc195b3e700 1 --2- 172.21.15.157:0/2017733341 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc1780a1120 0x7fc1780a35e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:42.866 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.865+0000 7fc195b3e700 1 --2- 172.21.15.157:0/2017733341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1901012b0 0x7fc19019e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:42.867 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.865+0000 7fc195b3e700 1 --2- 172.21.15.157:0/2017733341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1900ff710 0x7fc190197100 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:42.867 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.865+0000 7fc195b3e700 1 --2- 172.21.15.157:0/2017733341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc190100910 0x7fc190197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:42.867 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.865+0000 7fc195b3e700 1 -- 172.21.15.157:0/2017733341 >> 172.21.15.157:0/2017733341 conn(0x7fc1900facb0 msgr2=0x7fc1900fbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:42.867 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.865+0000 7fc195b3e700 1 -- 172.21.15.157:0/2017733341 shutdown_connections 2024-05-08T22:04:42.867 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-08T22:04:42.865+0000 7fc195b3e700 1 -- 172.21.15.157:0/2017733341 wait complete. 2024-05-08T22:04:43.266 DEBUG:teuthology.orchestra.run.smithi131:mds.a> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.a.service 2024-05-08T22:04:43.270 DEBUG:teuthology.orchestra.run.smithi131:mds.d> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.d.service 2024-05-08T22:04:43.316 DEBUG:teuthology.orchestra.run.smithi131:mds.g> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.g.service 2024-05-08T22:04:43.341 INFO:journalctl@ceph.mds.a.smithi131.stdout:-- Logs begin at Wed 2024-05-08 21:41:05 UTC. -- 2024-05-08T22:04:43.344 DEBUG:teuthology.orchestra.run.smithi131:mds.j> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.j.service 2024-05-08T22:04:43.383 INFO:journalctl@ceph.mds.d.smithi131.stdout:-- Logs begin at Wed 2024-05-08 21:41:05 UTC. -- 2024-05-08T22:04:43.386 DEBUG:teuthology.orchestra.run.smithi132:mds.b> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.b.service 2024-05-08T22:04:43.389 DEBUG:teuthology.orchestra.run.smithi132:mds.e> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.e.service 2024-05-08T22:04:43.413 INFO:journalctl@ceph.mds.g.smithi131.stdout:-- Logs begin at Wed 2024-05-08 21:41:05 UTC. -- 2024-05-08T22:04:43.427 INFO:journalctl@ceph.mds.b.smithi132.stdout:-- Logs begin at Wed 2024-05-08 21:34:20 UTC. -- 2024-05-08T22:04:43.428 DEBUG:teuthology.orchestra.run.smithi132:mds.h> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.h.service 2024-05-08T22:04:43.450 INFO:journalctl@ceph.mds.j.smithi131.stdout:-- Logs begin at Wed 2024-05-08 21:41:05 UTC. -- 2024-05-08T22:04:43.457 INFO:journalctl@ceph.mds.e.smithi132.stdout:-- Logs begin at Wed 2024-05-08 21:34:20 UTC. -- 2024-05-08T22:04:43.458 DEBUG:teuthology.orchestra.run.smithi132:mds.k> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.k.service 2024-05-08T22:04:43.486 INFO:journalctl@ceph.mds.h.smithi132.stdout:-- Logs begin at Wed 2024-05-08 21:34:20 UTC. -- 2024-05-08T22:04:43.487 DEBUG:teuthology.orchestra.run.smithi157:mds.c> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.c.service 2024-05-08T22:04:43.490 DEBUG:teuthology.orchestra.run.smithi157:mds.f> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.f.service 2024-05-08T22:04:43.514 INFO:journalctl@ceph.mds.k.smithi132.stdout:-- Logs begin at Wed 2024-05-08 21:34:20 UTC. -- 2024-05-08T22:04:43.521 INFO:journalctl@ceph.mds.c.smithi157.stdout:-- Logs begin at Wed 2024-05-08 21:35:00 UTC. -- 2024-05-08T22:04:43.522 DEBUG:teuthology.orchestra.run.smithi157:mds.i> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.i.service 2024-05-08T22:04:43.549 INFO:journalctl@ceph.mds.f.smithi157.stdout:-- Logs begin at Wed 2024-05-08 21:35:00 UTC. -- 2024-05-08T22:04:43.550 DEBUG:teuthology.orchestra.run.smithi157:mds.l> sudo journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.l.service 2024-05-08T22:04:43.579 INFO:journalctl@ceph.mds.i.smithi157.stdout:-- Logs begin at Wed 2024-05-08 21:35:00 UTC. -- 2024-05-08T22:04:43.580 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-08T22:04:43.580 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:04:43.603 INFO:journalctl@ceph.mds.l.smithi157.stdout:-- Logs begin at Wed 2024-05-08 21:35:00 UTC. -- 2024-05-08T22:04:43.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.783+0000 7ff96e800700 1 -- 172.21.15.131:0/4055603985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff968101f90 msgr2=0x7ff968102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:43.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.783+0000 7ff96e800700 1 --2- 172.21.15.131:0/4055603985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff968101f90 0x7ff968102410 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff954009990 tx=0x7ff954009ca0 comp rx=0 tx=0).stop 2024-05-08T22:04:43.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.783+0000 7ff96e800700 1 -- 172.21.15.131:0/4055603985 shutdown_connections 2024-05-08T22:04:43.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.783+0000 7ff96e800700 1 --2- 172.21.15.131:0/4055603985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff968102950 0x7ff968109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:43.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.783+0000 7ff96e800700 1 --2- 172.21.15.131:0/4055603985 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff968100e40 0x7ff968101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:43.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.783+0000 7ff96e800700 1 --2- 172.21.15.131:0/4055603985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff968101f90 0x7ff968102410 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:43.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.783+0000 7ff96e800700 1 -- 172.21.15.131:0/4055603985 >> 172.21.15.131:0/4055603985 conn(0x7ff9680fc3b0 msgr2=0x7ff9680fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:43.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.784+0000 7ff96e800700 1 -- 172.21.15.131:0/4055603985 shutdown_connections 2024-05-08T22:04:43.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.784+0000 7ff96e800700 1 -- 172.21.15.131:0/4055603985 wait complete. 2024-05-08T22:04:43.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.784+0000 7ff96e800700 1 Processor -- start 2024-05-08T22:04:43.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.785+0000 7ff96e800700 1 -- start start 2024-05-08T22:04:43.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.785+0000 7ff96e800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff968100e40 0x7ff968198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:43.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.785+0000 7ff96e800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff968101f90 0x7ff968199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:43.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.785+0000 7ff96e800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff968102950 0x7ff9681a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:43.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.785+0000 7ff96e800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9681a0ad0 con 0x7ff968102950 2024-05-08T22:04:43.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.785+0000 7ff96e800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9681a0c10 con 0x7ff968100e40 2024-05-08T22:04:43.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.785+0000 7ff96e800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9681a0da0 con 0x7ff968101f90 2024-05-08T22:04:43.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.785+0000 7ff96ca00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff968101f90 0x7ff968199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:43.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.785+0000 7ff96de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff968102950 0x7ff9681a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:43.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.785+0000 7ff96ca00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff968101f90 0x7ff968199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:57666/0 (socket says 172.21.15.131:57666) 2024-05-08T22:04:43.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.785+0000 7ff96ca00700 1 -- 172.21.15.131:0/1056880758 learned_addr learned my addr 172.21.15.131:0/1056880758 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:04:43.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.785+0000 7ff96d400700 1 --2- 172.21.15.131:0/1056880758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff968100e40 0x7ff968198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:43.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.786+0000 7ff96ca00700 1 -- 172.21.15.131:0/1056880758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff968100e40 msgr2=0x7ff968198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:43.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.786+0000 7ff96ca00700 1 --2- 172.21.15.131:0/1056880758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff968100e40 0x7ff968198ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:43.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.786+0000 7ff96ca00700 1 -- 172.21.15.131:0/1056880758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff968102950 msgr2=0x7ff9681a0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:43.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.786+0000 7ff96ca00700 1 --2- 172.21.15.131:0/1056880758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff968102950 0x7ff9681a0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:43.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.786+0000 7ff96ca00700 1 -- 172.21.15.131:0/1056880758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff960009c40 con 0x7ff968101f90 2024-05-08T22:04:43.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.786+0000 7ff96ca00700 1 --2- 172.21.15.131:0/1056880758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff968101f90 0x7ff968199530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff95400b680 tx=0x7ff954009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:43.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.786+0000 7ff966000700 1 -- 172.21.15.131:0/1056880758 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95401d070 con 0x7ff968101f90 2024-05-08T22:04:43.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.786+0000 7ff96e800700 1 -- 172.21.15.131:0/1056880758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9540095f0 con 0x7ff968101f90 2024-05-08T22:04:43.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.786+0000 7ff966000700 1 -- 172.21.15.131:0/1056880758 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff954022470 con 0x7ff968101f90 2024-05-08T22:04:43.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.786+0000 7ff966000700 1 -- 172.21.15.131:0/1056880758 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff954017650 con 0x7ff968101f90 2024-05-08T22:04:43.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.786+0000 7ff96e800700 1 -- 172.21.15.131:0/1056880758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9681a1310 con 0x7ff968101f90 2024-05-08T22:04:43.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.787+0000 7ff96e800700 1 -- 172.21.15.131:0/1056880758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff968067ba0 con 0x7ff968101f90 2024-05-08T22:04:43.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.789+0000 7ff966000700 1 -- 172.21.15.131:0/1056880758 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff954022870 con 0x7ff968101f90 2024-05-08T22:04:43.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.790+0000 7ff966000700 1 --2- 172.21.15.131:0/1056880758 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff9500a11a0 0x7ff9500a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:43.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.790+0000 7ff966000700 1 -- 172.21.15.131:0/1056880758 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff9540cda50 con 0x7ff968101f90 2024-05-08T22:04:43.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.791+0000 7ff96d400700 1 --2- 172.21.15.131:0/1056880758 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff9500a11a0 0x7ff9500a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:43.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.791+0000 7ff96d400700 1 --2- 172.21.15.131:0/1056880758 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff9500a11a0 0x7ff9500a3660 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009c60 tx=0x7ff95c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:43.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:43.796+0000 7ff966000700 1 -- 172.21.15.131:0/1056880758 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff954097670 con 0x7ff968101f90 2024-05-08T22:04:43.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:43 smithi131 ceph-mon[37260]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:43.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:43 smithi131 ceph-mon[37260]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:04:43.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:43 smithi131 ceph-mon[37260]: Saving service mds.all spec with placement smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-05-08T22:04:43.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:43 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:43.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:43 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:04:43.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:43 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:43.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:43 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:04:43.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:43 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:43.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:43 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:04:43.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:43 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:04:43.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:43 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:44.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.011+0000 7ff96e800700 1 -- 172.21.15.131:0/1056880758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff96810c650 con 0x7ff968101f90 2024-05-08T22:04:44.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.012+0000 7ff966000700 1 -- 172.21.15.131:0/1056880758 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+18469 (secure 0 0 0) 0x7ff95409ac90 con 0x7ff968101f90 2024-05-08T22:04:44.013 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:04:44.014 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":75,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:04:36.108486+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:04:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.014+0000 7ff96e800700 1 -- 172.21.15.131:0/1056880758 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff9500a11a0 msgr2=0x7ff9500a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.014+0000 7ff96e800700 1 --2- 172.21.15.131:0/1056880758 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff9500a11a0 0x7ff9500a3660 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009c60 tx=0x7ff95c009260 comp rx=0 tx=0).stop 2024-05-08T22:04:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.014+0000 7ff96e800700 1 -- 172.21.15.131:0/1056880758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff968101f90 msgr2=0x7ff968199530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.014+0000 7ff96e800700 1 --2- 172.21.15.131:0/1056880758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff968101f90 0x7ff968199530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff95400b680 tx=0x7ff954009580 comp rx=0 tx=0).stop 2024-05-08T22:04:44.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.014+0000 7ff96e800700 1 -- 172.21.15.131:0/1056880758 shutdown_connections 2024-05-08T22:04:44.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.014+0000 7ff96e800700 1 --2- 172.21.15.131:0/1056880758 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff9500a11a0 0x7ff9500a3660 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.014+0000 7ff96e800700 1 --2- 172.21.15.131:0/1056880758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff968102950 0x7ff9681a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.014+0000 7ff96e800700 1 --2- 172.21.15.131:0/1056880758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff968100e40 0x7ff968198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.014+0000 7ff96e800700 1 --2- 172.21.15.131:0/1056880758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff968101f90 0x7ff968199530 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.014+0000 7ff96e800700 1 -- 172.21.15.131:0/1056880758 >> 172.21.15.131:0/1056880758 conn(0x7ff9680fc3b0 msgr2=0x7ff9680731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:44.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.014+0000 7ff96e800700 1 -- 172.21.15.131:0/1056880758 shutdown_connections 2024-05-08T22:04:44.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.014+0000 7ff96e800700 1 -- 172.21.15.131:0/1056880758 wait complete. 2024-05-08T22:04:44.036 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:01:41.271078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:04:44.036 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:04:44.155 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:43 smithi157 ceph-mon[77746]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:44.156 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:43 smithi157 ceph-mon[77746]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:04:44.156 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:43 smithi157 ceph-mon[77746]: Saving service mds.all spec with placement smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-05-08T22:04:44.156 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:43 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:44.156 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:43 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:04:44.156 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:43 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:44.156 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:43 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:04:44.156 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:43 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:44.156 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:43 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:04:44.156 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:43 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:04:44.156 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:43 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:44.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.231+0000 7f50d5800700 1 -- 172.21.15.131:0/1255125238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d0102170 msgr2=0x7f50d01025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:44.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.231+0000 7f50d5800700 1 --2- 172.21.15.131:0/1255125238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d0102170 0x7f50d01025d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0099c0 tx=0x7f50bc009cd0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.231+0000 7f50d5800700 1 -- 172.21.15.131:0/1255125238 shutdown_connections 2024-05-08T22:04:44.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.231+0000 7f50d5800700 1 --2- 172.21.15.131:0/1255125238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50d0102b10 0x7f50d01093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.231+0000 7f50d5800700 1 --2- 172.21.15.131:0/1255125238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50d0100f80 0x7f50d01013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.231+0000 7f50d5800700 1 --2- 172.21.15.131:0/1255125238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d0102170 0x7f50d01025d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.231+0000 7f50d5800700 1 -- 172.21.15.131:0/1255125238 >> 172.21.15.131:0/1255125238 conn(0x7f50d00fc4f0 msgr2=0x7f50d00fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:44.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.231+0000 7f50d5800700 1 -- 172.21.15.131:0/1255125238 shutdown_connections 2024-05-08T22:04:44.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.231+0000 7f50d5800700 1 -- 172.21.15.131:0/1255125238 wait complete. 2024-05-08T22:04:44.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.232+0000 7f50d5800700 1 Processor -- start 2024-05-08T22:04:44.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.233+0000 7f50d5800700 1 -- start start 2024-05-08T22:04:44.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.233+0000 7f50d5800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50d0100f80 0x7f50d0196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:44.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.233+0000 7f50d5800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50d0102170 0x7f50d0197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:44.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.233+0000 7f50d5800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d0102b10 0x7f50d019e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:44.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.233+0000 7f50d5800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d019ea40 con 0x7f50d0102b10 2024-05-08T22:04:44.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.233+0000 7f50d5800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d019eb80 con 0x7f50d0102170 2024-05-08T22:04:44.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.233+0000 7f50d5800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d019ed10 con 0x7f50d0100f80 2024-05-08T22:04:44.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.233+0000 7f50cea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50d0100f80 0x7f50d0196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:44.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.233+0000 7f50cf400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d0102b10 0x7f50d019e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:44.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.233+0000 7f50cea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50d0100f80 0x7f50d0196f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46018/0 (socket says 172.21.15.131:46018) 2024-05-08T22:04:44.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.233+0000 7f50cea00700 1 -- 172.21.15.131:0/422646424 learned_addr learned my addr 172.21.15.131:0/422646424 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:04:44.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.233+0000 7f50ce000700 1 --2- 172.21.15.131:0/422646424 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50d0102170 0x7f50d0197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:44.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.233+0000 7f50cea00700 1 -- 172.21.15.131:0/422646424 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50d0102170 msgr2=0x7f50d0197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:44.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.234+0000 7f50cea00700 1 --2- 172.21.15.131:0/422646424 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50d0102170 0x7f50d0197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.234+0000 7f50cea00700 1 -- 172.21.15.131:0/422646424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d0102b10 msgr2=0x7f50d019e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:44.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.234+0000 7f50cea00700 1 --2- 172.21.15.131:0/422646424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d0102b10 0x7f50d019e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.234+0000 7f50cea00700 1 -- 172.21.15.131:0/422646424 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50bc0095f0 con 0x7f50d0100f80 2024-05-08T22:04:44.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.234+0000 7f50cea00700 1 --2- 172.21.15.131:0/422646424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50d0100f80 0x7f50d0196f10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f50c000e970 tx=0x7f50c000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:44.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.234+0000 7f50c7400700 1 -- 172.21.15.131:0/422646424 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0009790 con 0x7f50d0100f80 2024-05-08T22:04:44.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.234+0000 7f50c7400700 1 -- 172.21.15.131:0/422646424 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50c000c7e0 con 0x7f50d0100f80 2024-05-08T22:04:44.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.234+0000 7f50d5800700 1 -- 172.21.15.131:0/422646424 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d019eff0 con 0x7f50d0100f80 2024-05-08T22:04:44.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.234+0000 7f50c7400700 1 -- 172.21.15.131:0/422646424 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0003c70 con 0x7f50d0100f80 2024-05-08T22:04:44.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.234+0000 7f50d5800700 1 -- 172.21.15.131:0/422646424 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d019f410 con 0x7f50d0100f80 2024-05-08T22:04:44.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.235+0000 7f50d5800700 1 -- 172.21.15.131:0/422646424 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d0108450 con 0x7f50d0100f80 2024-05-08T22:04:44.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.237+0000 7f50c7400700 1 -- 172.21.15.131:0/422646424 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f50c000cc50 con 0x7f50d0100f80 2024-05-08T22:04:44.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.238+0000 7f50c7400700 1 --2- 172.21.15.131:0/422646424 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f50b40a1120 0x7f50b40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:44.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.238+0000 7f50c7400700 1 -- 172.21.15.131:0/422646424 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f50c0014070 con 0x7f50d0100f80 2024-05-08T22:04:44.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.238+0000 7f50ce000700 1 --2- 172.21.15.131:0/422646424 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f50b40a1120 0x7f50b40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:44.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.239+0000 7f50ce000700 1 --2- 172.21.15.131:0/422646424 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f50b40a1120 0x7f50b40a35e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f50bc003730 tx=0x7f50bc00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:44.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.242+0000 7f50c7400700 1 -- 172.21.15.131:0/422646424 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f50c0096240 con 0x7f50d0100f80 2024-05-08T22:04:44.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:43 smithi132 ceph-mon[78180]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:44.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:43 smithi132 ceph-mon[78180]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:04:44.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:43 smithi132 ceph-mon[78180]: Saving service mds.all spec with placement smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-05-08T22:04:44.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:43 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:44.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:43 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:04:44.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:43 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:44.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:43 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:04:44.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:43 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:44.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:43 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:04:44.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:43 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:04:44.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:43 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:44.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.448+0000 7f50d5800700 1 -- 172.21.15.131:0/422646424 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f50d010c9e0 con 0x7f50d0100f80 2024-05-08T22:04:44.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.449+0000 7f50c7400700 1 -- 172.21.15.131:0/422646424 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f50c0003dd0 con 0x7f50d0100f80 2024-05-08T22:04:44.450 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-08T22:04:44.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.451+0000 7f50d5800700 1 -- 172.21.15.131:0/422646424 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f50b40a1120 msgr2=0x7f50b40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:44.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.451+0000 7f50d5800700 1 --2- 172.21.15.131:0/422646424 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f50b40a1120 0x7f50b40a35e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f50bc003730 tx=0x7f50bc00b540 comp rx=0 tx=0).stop 2024-05-08T22:04:44.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.451+0000 7f50d5800700 1 -- 172.21.15.131:0/422646424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50d0100f80 msgr2=0x7f50d0196f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:44.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.451+0000 7f50d5800700 1 --2- 172.21.15.131:0/422646424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50d0100f80 0x7f50d0196f10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f50c000e970 tx=0x7f50c000ec80 comp rx=0 tx=0).stop 2024-05-08T22:04:44.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.452+0000 7f50d5800700 1 -- 172.21.15.131:0/422646424 shutdown_connections 2024-05-08T22:04:44.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.452+0000 7f50d5800700 1 --2- 172.21.15.131:0/422646424 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f50b40a1120 0x7f50b40a35e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.452+0000 7f50d5800700 1 --2- 172.21.15.131:0/422646424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d0102b10 0x7f50d019e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.452+0000 7f50d5800700 1 --2- 172.21.15.131:0/422646424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50d0100f80 0x7f50d0196f10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.452+0000 7f50d5800700 1 --2- 172.21.15.131:0/422646424 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50d0102170 0x7f50d0197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.452+0000 7f50d5800700 1 -- 172.21.15.131:0/422646424 >> 172.21.15.131:0/422646424 conn(0x7f50d00fc4f0 msgr2=0x7f50d0105590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:44.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.452+0000 7f50d5800700 1 -- 172.21.15.131:0/422646424 shutdown_connections 2024-05-08T22:04:44.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.452+0000 7f50d5800700 1 -- 172.21.15.131:0/422646424 wait complete. 2024-05-08T22:04:44.472 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:04:44.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.664+0000 7ff41cc00700 1 -- 172.21.15.131:0/2305883444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff418101fc0 msgr2=0x7ff418102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:44.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.664+0000 7ff41cc00700 1 --2- 172.21.15.131:0/2305883444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff418101fc0 0x7ff418102440 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff404009990 tx=0x7ff404009ca0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.664+0000 7ff41cc00700 1 -- 172.21.15.131:0/2305883444 shutdown_connections 2024-05-08T22:04:44.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.664+0000 7ff41cc00700 1 --2- 172.21.15.131:0/2305883444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff418102980 0x7ff4181092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.664+0000 7ff41cc00700 1 --2- 172.21.15.131:0/2305883444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff418100e70 0x7ff418101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.664+0000 7ff41cc00700 1 --2- 172.21.15.131:0/2305883444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff418101fc0 0x7ff418102440 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.664+0000 7ff41cc00700 1 -- 172.21.15.131:0/2305883444 >> 172.21.15.131:0/2305883444 conn(0x7ff4180fc3e0 msgr2=0x7ff4180fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:44.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.664+0000 7ff41cc00700 1 -- 172.21.15.131:0/2305883444 shutdown_connections 2024-05-08T22:04:44.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.664+0000 7ff41cc00700 1 -- 172.21.15.131:0/2305883444 wait complete. 2024-05-08T22:04:44.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.665+0000 7ff41cc00700 1 Processor -- start 2024-05-08T22:04:44.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.665+0000 7ff41cc00700 1 -- start start 2024-05-08T22:04:44.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.666+0000 7ff41cc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff418100e70 0x7ff418074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:44.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.666+0000 7ff41cc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff418101fc0 0x7ff418073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:44.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.666+0000 7ff41cc00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff418102980 0x7ff4180738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:44.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.666+0000 7ff41cc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff418074070 con 0x7ff418101fc0 2024-05-08T22:04:44.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.666+0000 7ff41cc00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4180741e0 con 0x7ff418102980 2024-05-08T22:04:44.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.666+0000 7ff41cc00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4181a0eb0 con 0x7ff418100e70 2024-05-08T22:04:44.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.666+0000 7ff415e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff418100e70 0x7ff418074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:44.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.666+0000 7ff415e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff418100e70 0x7ff418074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46034/0 (socket says 172.21.15.131:46034) 2024-05-08T22:04:44.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.666+0000 7ff415e00700 1 -- 172.21.15.131:0/2392685325 learned_addr learned my addr 172.21.15.131:0/2392685325 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:04:44.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.666+0000 7ff415400700 1 --2- 172.21.15.131:0/2392685325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff418101fc0 0x7ff418073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:44.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.666+0000 7ff416800700 1 --2- 172.21.15.131:0/2392685325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff418102980 0x7ff4180738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:44.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.666+0000 7ff415e00700 1 -- 172.21.15.131:0/2392685325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff418102980 msgr2=0x7ff4180738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:44.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.666+0000 7ff415e00700 1 --2- 172.21.15.131:0/2392685325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff418102980 0x7ff4180738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.666+0000 7ff415e00700 1 -- 172.21.15.131:0/2392685325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff418101fc0 msgr2=0x7ff418073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:44.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.666+0000 7ff415e00700 1 --2- 172.21.15.131:0/2392685325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff418101fc0 0x7ff418073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.666+0000 7ff415e00700 1 -- 172.21.15.131:0/2392685325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4040095f0 con 0x7ff418100e70 2024-05-08T22:04:44.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.667+0000 7ff415e00700 1 --2- 172.21.15.131:0/2392685325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff418100e70 0x7ff418074d00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff40800e9c0 tx=0x7ff40800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:44.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.667+0000 7ff40ea00700 1 -- 172.21.15.131:0/2392685325 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4080097b0 con 0x7ff418100e70 2024-05-08T22:04:44.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.667+0000 7ff40ea00700 1 -- 172.21.15.131:0/2392685325 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff40800ee60 con 0x7ff418100e70 2024-05-08T22:04:44.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.667+0000 7ff41cc00700 1 -- 172.21.15.131:0/2392685325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4181a10e0 con 0x7ff418100e70 2024-05-08T22:04:44.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.667+0000 7ff40ea00700 1 -- 172.21.15.131:0/2392685325 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4080054f0 con 0x7ff418100e70 2024-05-08T22:04:44.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.667+0000 7ff41cc00700 1 -- 172.21.15.131:0/2392685325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4181a15f0 con 0x7ff418100e70 2024-05-08T22:04:44.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.668+0000 7ff41cc00700 1 -- 172.21.15.131:0/2392685325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff418108330 con 0x7ff418100e70 2024-05-08T22:04:44.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.670+0000 7ff40ea00700 1 -- 172.21.15.131:0/2392685325 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff408004750 con 0x7ff418100e70 2024-05-08T22:04:44.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.671+0000 7ff40ea00700 1 --2- 172.21.15.131:0/2392685325 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff3fc0a11a0 0x7ff3fc0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:44.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.671+0000 7ff40ea00700 1 -- 172.21.15.131:0/2392685325 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff408014070 con 0x7ff418100e70 2024-05-08T22:04:44.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.671+0000 7ff415400700 1 --2- 172.21.15.131:0/2392685325 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff3fc0a11a0 0x7ff3fc0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:44.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.672+0000 7ff415400700 1 --2- 172.21.15.131:0/2392685325 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff3fc0a11a0 0x7ff3fc0a3660 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff40400b2e0 tx=0x7ff404005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:44.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.675+0000 7ff40ea00700 1 -- 172.21.15.131:0/2392685325 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff408096290 con 0x7ff418100e70 2024-05-08T22:04:44.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.929+0000 7ff41cc00700 1 -- 172.21.15.131:0/2392685325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff41810c990 con 0x7ff418100e70 2024-05-08T22:04:44.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.929+0000 7ff40ea00700 1 -- 172.21.15.131:0/2392685325 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7ff4080998b0 con 0x7ff418100e70 2024-05-08T22:04:44.931 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:04:44.931 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-08T22:04:44.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.932+0000 7ff41cc00700 1 -- 172.21.15.131:0/2392685325 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff3fc0a11a0 msgr2=0x7ff3fc0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:44.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.932+0000 7ff41cc00700 1 --2- 172.21.15.131:0/2392685325 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff3fc0a11a0 0x7ff3fc0a3660 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff40400b2e0 tx=0x7ff404005fd0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.932+0000 7ff41cc00700 1 -- 172.21.15.131:0/2392685325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff418100e70 msgr2=0x7ff418074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:44.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.932+0000 7ff41cc00700 1 --2- 172.21.15.131:0/2392685325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff418100e70 0x7ff418074d00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff40800e9c0 tx=0x7ff40800ecd0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.932+0000 7ff41cc00700 1 -- 172.21.15.131:0/2392685325 shutdown_connections 2024-05-08T22:04:44.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.932+0000 7ff41cc00700 1 --2- 172.21.15.131:0/2392685325 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff3fc0a11a0 0x7ff3fc0a3660 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.932+0000 7ff41cc00700 1 --2- 172.21.15.131:0/2392685325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff418102980 0x7ff4180738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.932+0000 7ff41cc00700 1 --2- 172.21.15.131:0/2392685325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff418100e70 0x7ff418074d00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.932+0000 7ff41cc00700 1 --2- 172.21.15.131:0/2392685325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff418101fc0 0x7ff418073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:44.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.932+0000 7ff41cc00700 1 -- 172.21.15.131:0/2392685325 >> 172.21.15.131:0/2392685325 conn(0x7ff4180fc3e0 msgr2=0x7ff418105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:44.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.932+0000 7ff41cc00700 1 -- 172.21.15.131:0/2392685325 shutdown_connections 2024-05-08T22:04:44.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:44.932+0000 7ff41cc00700 1 -- 172.21.15.131:0/2392685325 wait complete. 2024-05-08T22:04:44.935 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 1 2024-05-08T22:04:45.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:44 smithi157 ceph-mon[77746]: Deploying daemon mds.l on smithi157 2024-05-08T22:04:45.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:44 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1056880758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:04:45.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:44 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/422646424' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:04:45.027 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 08 22:04:44 smithi157 systemd[1]: Starting Ceph mds.l for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:04:45.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:44 smithi131 ceph-mon[37260]: Deploying daemon mds.l on smithi157 2024-05-08T22:04:45.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:44 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1056880758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:04:45.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:44 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/422646424' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:04:45.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:44 smithi132 ceph-mon[78180]: Deploying daemon mds.l on smithi157 2024-05-08T22:04:45.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:44 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1056880758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:04:45.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:44 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/422646424' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:04:45.526 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 08 22:04:45 smithi157 bash[114549]: b1be67ca5ab41413368d694121f1419518d394fe2e564e9b41d1eaf96178e7f1 2024-05-08T22:04:45.526 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 08 22:04:45 smithi157 systemd[1]: Started Ceph mds.l for 06f1a696-0d86-11ef-bc97-c7b262605968. 2024-05-08T22:04:45.526 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 08 22:04:45 smithi157 ceph-mds[114587]: -- 172.21.15.157:0/1111332912 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556be0cd21a0 con 0x556be0ca8800 2024-05-08T22:04:45.526 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 08 22:04:45 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-l[114564]: starting mds.l at 2024-05-08T22:04:46.080 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:45 smithi132 ceph-mon[78180]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:46.080 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:45 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2392685325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:04:46.080 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:45 smithi132 ceph-mon[78180]: mds.? [v2:172.21.15.157:6832/1034338270,v1:172.21.15.157:6833/1034338270] up:boot 2024-05-08T22:04:46.080 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:45 smithi132 ceph-mon[78180]: fsmap 1 up:standby 2024-05-08T22:04:46.080 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:45 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T22:04:46.080 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:45 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:46.080 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:45 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:46.081 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:45 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:46.081 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:45 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:04:46.081 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:45 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:04:46.081 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:45 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:45 smithi131 ceph-mon[37260]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:45 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2392685325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:04:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:45 smithi131 ceph-mon[37260]: mds.? [v2:172.21.15.157:6832/1034338270,v1:172.21.15.157:6833/1034338270] up:boot 2024-05-08T22:04:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:45 smithi131 ceph-mon[37260]: fsmap 1 up:standby 2024-05-08T22:04:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:45 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T22:04:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:45 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:45 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:45 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:45 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:04:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:45 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:04:46.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:45 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:46.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:45 smithi157 ceph-mon[77746]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:46.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:45 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2392685325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:04:46.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:45 smithi157 ceph-mon[77746]: mds.? [v2:172.21.15.157:6832/1034338270,v1:172.21.15.157:6833/1034338270] up:boot 2024-05-08T22:04:46.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:45 smithi157 ceph-mon[77746]: fsmap 1 up:standby 2024-05-08T22:04:46.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:45 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T22:04:46.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:45 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:46.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:45 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:46.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:45 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:46.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:45 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:04:46.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:45 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:04:46.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:45 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:46.840 INFO:journalctl@ceph.mds.k.smithi132.stdout:May 08 22:04:46 smithi132 systemd[1]: Starting Ceph mds.k for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:04:46.953 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:04:47.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:46 smithi131 ceph-mon[37260]: Deploying daemon mds.k on smithi132 2024-05-08T22:04:47.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.163+0000 7f42a7800700 1 -- 172.21.15.131:0/2091675406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42a0102980 msgr2=0x7f42a01092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:47.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.163+0000 7f42a7800700 1 --2- 172.21.15.131:0/2091675406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42a0102980 0x7f42a01092a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4298009f90 tx=0x7f429800b330 comp rx=0 tx=0).stop 2024-05-08T22:04:47.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.163+0000 7f42a7800700 1 -- 172.21.15.131:0/2091675406 shutdown_connections 2024-05-08T22:04:47.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.163+0000 7f42a7800700 1 --2- 172.21.15.131:0/2091675406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42a0102980 0x7f42a01092a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:47.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.163+0000 7f42a7800700 1 --2- 172.21.15.131:0/2091675406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42a0100e70 0x7f42a0101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:47.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.163+0000 7f42a7800700 1 --2- 172.21.15.131:0/2091675406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42a0101fc0 0x7f42a0102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:47.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.163+0000 7f42a7800700 1 -- 172.21.15.131:0/2091675406 >> 172.21.15.131:0/2091675406 conn(0x7f42a00fc3e0 msgr2=0x7f42a00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:47.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.163+0000 7f42a7800700 1 -- 172.21.15.131:0/2091675406 shutdown_connections 2024-05-08T22:04:47.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.163+0000 7f42a7800700 1 -- 172.21.15.131:0/2091675406 wait complete. 2024-05-08T22:04:47.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.164+0000 7f42a7800700 1 Processor -- start 2024-05-08T22:04:47.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.164+0000 7f42a7800700 1 -- start start 2024-05-08T22:04:47.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.165+0000 7f42a7800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42a0100e70 0x7f42a0074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:47.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.165+0000 7f42a7800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42a0101fc0 0x7f42a0073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:47.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.165+0000 7f42a7800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42a0102980 0x7f42a00738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:47.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.165+0000 7f42a7800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a0074070 con 0x7f42a0101fc0 2024-05-08T22:04:47.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.165+0000 7f42a7800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a00741e0 con 0x7f42a0102980 2024-05-08T22:04:47.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.165+0000 7f42a7800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a01a0f60 con 0x7f42a0100e70 2024-05-08T22:04:47.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.165+0000 7f429fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42a0101fc0 0x7f42a0073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:47.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.165+0000 7f42a4e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42a0100e70 0x7f42a0074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:47.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.165+0000 7f429fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42a0101fc0 0x7f42a0073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54214/0 (socket says 172.21.15.131:54214) 2024-05-08T22:04:47.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.165+0000 7f429fe00700 1 -- 172.21.15.131:0/1072845105 learned_addr learned my addr 172.21.15.131:0/1072845105 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:04:47.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.165+0000 7f42a5800700 1 --2- 172.21.15.131:0/1072845105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42a0102980 0x7f42a00738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:47.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.165+0000 7f429fe00700 1 -- 172.21.15.131:0/1072845105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42a0100e70 msgr2=0x7f42a0074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:47.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.165+0000 7f429fe00700 1 --2- 172.21.15.131:0/1072845105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42a0100e70 0x7f42a0074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:47.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.165+0000 7f429fe00700 1 -- 172.21.15.131:0/1072845105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42a0102980 msgr2=0x7f42a00738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:47.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.165+0000 7f429fe00700 1 --2- 172.21.15.131:0/1072845105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42a0102980 0x7f42a00738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:47.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.165+0000 7f429fe00700 1 -- 172.21.15.131:0/1072845105 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4294009520 con 0x7f42a0101fc0 2024-05-08T22:04:47.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.165+0000 7f42a5800700 1 --2- 172.21.15.131:0/1072845105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42a0102980 0x7f42a00738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:04:47.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.165+0000 7f42a4e00700 1 --2- 172.21.15.131:0/1072845105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42a0100e70 0x7f42a0074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:04:47.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.166+0000 7f429fe00700 1 --2- 172.21.15.131:0/1072845105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42a0101fc0 0x7f42a0073390 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f428c00ec10 tx=0x7f428c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:47.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.166+0000 7f429d600700 1 -- 172.21.15.131:0/1072845105 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c00cc30 con 0x7f42a0101fc0 2024-05-08T22:04:47.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.166+0000 7f429d600700 1 -- 172.21.15.131:0/1072845105 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f428c0044e0 con 0x7f42a0101fc0 2024-05-08T22:04:47.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.166+0000 7f42a7800700 1 -- 172.21.15.131:0/1072845105 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f428c0095f0 con 0x7f42a0101fc0 2024-05-08T22:04:47.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.166+0000 7f429d600700 1 -- 172.21.15.131:0/1072845105 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c010430 con 0x7f42a0101fc0 2024-05-08T22:04:47.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.166+0000 7f42a7800700 1 -- 172.21.15.131:0/1072845105 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4298009c40 con 0x7f42a0101fc0 2024-05-08T22:04:47.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.167+0000 7f42a7800700 1 -- 172.21.15.131:0/1072845105 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a0108210 con 0x7f42a0101fc0 2024-05-08T22:04:47.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.170+0000 7f429d600700 1 -- 172.21.15.131:0/1072845105 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f428c003f20 con 0x7f42a0101fc0 2024-05-08T22:04:47.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.172+0000 7f429d600700 1 --2- 172.21.15.131:0/1072845105 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f42880a11a0 0x7f42880a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:47.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.172+0000 7f429d600700 1 -- 172.21.15.131:0/1072845105 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f428c014070 con 0x7f42a0101fc0 2024-05-08T22:04:47.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.172+0000 7f42a4e00700 1 --2- 172.21.15.131:0/1072845105 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f42880a11a0 0x7f42880a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:47.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.173+0000 7f42a4e00700 1 --2- 172.21.15.131:0/1072845105 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f42880a11a0 0x7f42880a3660 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4294009fd0 tx=0x7f4294009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:47.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.175+0000 7f429d600700 1 -- 172.21.15.131:0/1072845105 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f428c0965c0 con 0x7f42a0101fc0 2024-05-08T22:04:47.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:46 smithi157 ceph-mon[77746]: Deploying daemon mds.k on smithi132 2024-05-08T22:04:47.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:46 smithi132 ceph-mon[78180]: Deploying daemon mds.k on smithi132 2024-05-08T22:04:47.340 INFO:journalctl@ceph.mds.k.smithi132.stdout:May 08 22:04:46 smithi132 ceph-mds[113817]: -- 172.21.15.132:0/764461496 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55766eb161a0 con 0x55766eaed000 2024-05-08T22:04:47.340 INFO:journalctl@ceph.mds.k.smithi132.stdout:May 08 22:04:46 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-k[113794]: starting mds.k at 2024-05-08T22:04:47.340 INFO:journalctl@ceph.mds.k.smithi132.stdout:May 08 22:04:47 smithi132 bash[113780]: fdc5db421f7b322a0777b1486207f000ef10530aea0d4a98691afce6d909999c 2024-05-08T22:04:47.341 INFO:journalctl@ceph.mds.k.smithi132.stdout:May 08 22:04:47 smithi132 systemd[1]: Started Ceph mds.k for 06f1a696-0d86-11ef-bc97-c7b262605968. 2024-05-08T22:04:47.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.440+0000 7f42a7800700 1 -- 172.21.15.131:0/1072845105 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42a0067c10 con 0x7f42a0101fc0 2024-05-08T22:04:47.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.441+0000 7f429d600700 1 -- 172.21.15.131:0/1072845105 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7f428c01f340 con 0x7f42a0101fc0 2024-05-08T22:04:47.443 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:04:47.443 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-05-08T22:04:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.444+0000 7f42a7800700 1 -- 172.21.15.131:0/1072845105 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f42880a11a0 msgr2=0x7f42880a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.444+0000 7f42a7800700 1 --2- 172.21.15.131:0/1072845105 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f42880a11a0 0x7f42880a3660 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4294009fd0 tx=0x7f4294009260 comp rx=0 tx=0).stop 2024-05-08T22:04:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.444+0000 7f42a7800700 1 -- 172.21.15.131:0/1072845105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42a0101fc0 msgr2=0x7f42a0073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.444+0000 7f42a7800700 1 --2- 172.21.15.131:0/1072845105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42a0101fc0 0x7f42a0073390 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f428c00ec10 tx=0x7f428c00ef20 comp rx=0 tx=0).stop 2024-05-08T22:04:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.444+0000 7f42a7800700 1 -- 172.21.15.131:0/1072845105 shutdown_connections 2024-05-08T22:04:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.444+0000 7f42a7800700 1 --2- 172.21.15.131:0/1072845105 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f42880a11a0 0x7f42880a3660 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.444+0000 7f42a7800700 1 --2- 172.21.15.131:0/1072845105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42a0102980 0x7f42a00738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.444+0000 7f42a7800700 1 --2- 172.21.15.131:0/1072845105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42a0100e70 0x7f42a0074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.444+0000 7f42a7800700 1 --2- 172.21.15.131:0/1072845105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42a0101fc0 0x7f42a0073390 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.444+0000 7f42a7800700 1 -- 172.21.15.131:0/1072845105 >> 172.21.15.131:0/1072845105 conn(0x7f42a00fc3e0 msgr2=0x7f42a0105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.444+0000 7f42a7800700 1 -- 172.21.15.131:0/1072845105 shutdown_connections 2024-05-08T22:04:47.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:47.444+0000 7f42a7800700 1 -- 172.21.15.131:0/1072845105 wait complete. 2024-05-08T22:04:47.447 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 2 2024-05-08T22:04:48.119 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:47 smithi131 ceph-mon[37260]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:48.119 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:47 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:48.119 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:47 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:48.119 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:47 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:48.119 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:47 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:04:48.119 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:47 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:04:48.119 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:47 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:48.119 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:47 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1072845105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:04:48.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:47 smithi157 ceph-mon[77746]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:48.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:47 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:48.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:47 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:48.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:47 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:48.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:47 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:04:48.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:47 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:04:48.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:47 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:48.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:47 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1072845105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:04:48.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:47 smithi132 ceph-mon[78180]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:48.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:47 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:48.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:47 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:48.342 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:47 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:48.342 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:47 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:04:48.342 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:47 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:04:48.342 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:47 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:48.342 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:47 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1072845105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:04:49.013 INFO:journalctl@ceph.mds.j.smithi131.stdout:May 08 22:04:49 smithi131 systemd[1]: Starting Ceph mds.j for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:04:49.013 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:48 smithi131 ceph-mon[37260]: Deploying daemon mds.j on smithi131 2024-05-08T22:04:49.013 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:48 smithi131 ceph-mon[37260]: mds.? [v2:172.21.15.132:6832/505027564,v1:172.21.15.132:6833/505027564] up:boot 2024-05-08T22:04:49.013 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:48 smithi131 ceph-mon[37260]: fsmap 2 up:standby 2024-05-08T22:04:49.013 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:48 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T22:04:49.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:48 smithi157 ceph-mon[77746]: Deploying daemon mds.j on smithi131 2024-05-08T22:04:49.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:48 smithi157 ceph-mon[77746]: mds.? [v2:172.21.15.132:6832/505027564,v1:172.21.15.132:6833/505027564] up:boot 2024-05-08T22:04:49.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:48 smithi157 ceph-mon[77746]: fsmap 2 up:standby 2024-05-08T22:04:49.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:48 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T22:04:49.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:48 smithi132 ceph-mon[78180]: Deploying daemon mds.j on smithi131 2024-05-08T22:04:49.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:48 smithi132 ceph-mon[78180]: mds.? [v2:172.21.15.132:6832/505027564,v1:172.21.15.132:6833/505027564] up:boot 2024-05-08T22:04:49.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:48 smithi132 ceph-mon[78180]: fsmap 2 up:standby 2024-05-08T22:04:49.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:48 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T22:04:49.466 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:04:49.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.682+0000 7f18f5400700 1 -- 172.21.15.131:0/1858329950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f01018f0 msgr2=0x7f18f0103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:49.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.682+0000 7f18f5400700 1 --2- 172.21.15.131:0/1858329950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f01018f0 0x7f18f0103ce0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009990 tx=0x7f18e0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:04:49.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.682+0000 7f18f5400700 1 -- 172.21.15.131:0/1858329950 shutdown_connections 2024-05-08T22:04:49.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.682+0000 7f18f5400700 1 --2- 172.21.15.131:0/1858329950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18f0106b50 0x7f18f0108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:49.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.682+0000 7f18f5400700 1 --2- 172.21.15.131:0/1858329950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f01018f0 0x7f18f0103ce0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:49.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.682+0000 7f18f5400700 1 --2- 172.21.15.131:0/1858329950 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18f0104220 0x7f18f0106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:49.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.682+0000 7f18f5400700 1 -- 172.21.15.131:0/1858329950 >> 172.21.15.131:0/1858329950 conn(0x7f18f00fb2e0 msgr2=0x7f18f00fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:49.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.682+0000 7f18f5400700 1 -- 172.21.15.131:0/1858329950 shutdown_connections 2024-05-08T22:04:49.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.683+0000 7f18f5400700 1 -- 172.21.15.131:0/1858329950 wait complete. 2024-05-08T22:04:49.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.684+0000 7f18f5400700 1 Processor -- start 2024-05-08T22:04:49.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.684+0000 7f18f5400700 1 -- start start 2024-05-08T22:04:49.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.684+0000 7f18f5400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18f01018f0 0x7f18f0199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:49.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.684+0000 7f18f5400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18f0104220 0x7f18f0199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:49.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.684+0000 7f18f5400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f0106b50 0x7f18f01a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:49.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.684+0000 7f18f5400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f01a0b60 con 0x7f18f0106b50 2024-05-08T22:04:49.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.684+0000 7f18f5400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f01a0ca0 con 0x7f18f0104220 2024-05-08T22:04:49.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.684+0000 7f18f5400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f01a0e30 con 0x7f18f01018f0 2024-05-08T22:04:49.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.684+0000 7f18ee800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18f01018f0 0x7f18f0199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:49.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.684+0000 7f18ee800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18f01018f0 0x7f18f0199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46064/0 (socket says 172.21.15.131:46064) 2024-05-08T22:04:49.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.684+0000 7f18ee800700 1 -- 172.21.15.131:0/2797680381 learned_addr learned my addr 172.21.15.131:0/2797680381 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:04:49.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.684+0000 7f18ede00700 1 --2- 172.21.15.131:0/2797680381 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18f0104220 0x7f18f0199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:49.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.685+0000 7f18ef200700 1 --2- 172.21.15.131:0/2797680381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f0106b50 0x7f18f01a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:49.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.685+0000 7f18ee800700 1 -- 172.21.15.131:0/2797680381 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18f0104220 msgr2=0x7f18f0199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:49.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.685+0000 7f18ee800700 1 --2- 172.21.15.131:0/2797680381 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18f0104220 0x7f18f0199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:49.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.685+0000 7f18ee800700 1 -- 172.21.15.131:0/2797680381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f0106b50 msgr2=0x7f18f01a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:49.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.685+0000 7f18ee800700 1 --2- 172.21.15.131:0/2797680381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f0106b50 0x7f18f01a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:49.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.685+0000 7f18ee800700 1 -- 172.21.15.131:0/2797680381 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc009520 con 0x7f18f01018f0 2024-05-08T22:04:49.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.685+0000 7f18ee800700 1 --2- 172.21.15.131:0/2797680381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18f01018f0 0x7f18f0199030 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f18e0005910 tx=0x7f18e0009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:49.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.686+0000 7f18e7400700 1 -- 172.21.15.131:0/2797680381 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e001d070 con 0x7f18f01018f0 2024-05-08T22:04:49.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.686+0000 7f18e7400700 1 -- 172.21.15.131:0/2797680381 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18e0021410 con 0x7f18f01018f0 2024-05-08T22:04:49.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.686+0000 7f18e7400700 1 -- 172.21.15.131:0/2797680381 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e0005720 con 0x7f18f01018f0 2024-05-08T22:04:49.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.686+0000 7f18f5400700 1 -- 172.21.15.131:0/2797680381 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e00095f0 con 0x7f18f01018f0 2024-05-08T22:04:49.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.686+0000 7f18f5400700 1 -- 172.21.15.131:0/2797680381 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f01a12f0 con 0x7f18f01018f0 2024-05-08T22:04:49.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.687+0000 7f18f5400700 1 -- 172.21.15.131:0/2797680381 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f0108720 con 0x7f18f01018f0 2024-05-08T22:04:49.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.689+0000 7f18e7400700 1 -- 172.21.15.131:0/2797680381 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f18e0021580 con 0x7f18f01018f0 2024-05-08T22:04:49.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.691+0000 7f18e7400700 1 --2- 172.21.15.131:0/2797680381 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f18d40a11a0 0x7f18d40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:49.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.691+0000 7f18e7400700 1 -- 172.21.15.131:0/2797680381 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f18e00cd8f0 con 0x7f18f01018f0 2024-05-08T22:04:49.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.691+0000 7f18ede00700 1 --2- 172.21.15.131:0/2797680381 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f18d40a11a0 0x7f18d40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:49.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.691+0000 7f18ede00700 1 --2- 172.21.15.131:0/2797680381 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f18d40a11a0 0x7f18d40a3660 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009df0 tx=0x7f18dc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:49.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.693+0000 7f18e7400700 1 -- 172.21.15.131:0/2797680381 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f18e0097510 con 0x7f18f01018f0 2024-05-08T22:04:49.890 INFO:journalctl@ceph.mds.j.smithi131.stdout:May 08 22:04:49 smithi131 ceph-mds[86334]: -- 172.21.15.131:0/2520164813 <== mon.0 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562f741ce1a0 con 0x562f741a6c00 2024-05-08T22:04:49.891 INFO:journalctl@ceph.mds.j.smithi131.stdout:May 08 22:04:49 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-j[86291]: starting mds.j at 2024-05-08T22:04:49.891 INFO:journalctl@ceph.mds.j.smithi131.stdout:May 08 22:04:49 smithi131 bash[86276]: 1e7269324a2dd47cf5f328d3846d4ae2c8fdbbac01f3dda8b8fe2e3c0a60670b 2024-05-08T22:04:49.891 INFO:journalctl@ceph.mds.j.smithi131.stdout:May 08 22:04:49 smithi131 systemd[1]: Started Ceph mds.j for 06f1a696-0d86-11ef-bc97-c7b262605968. 2024-05-08T22:04:49.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.971+0000 7f18f5400700 1 -- 172.21.15.131:0/2797680381 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18f0068060 con 0x7f18f01018f0 2024-05-08T22:04:49.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.972+0000 7f18e7400700 1 -- 172.21.15.131:0/2797680381 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2777 (secure 0 0 0) 0x7f18e009ab30 con 0x7f18f01018f0 2024-05-08T22:04:49.974 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:04:49.974 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[]} 2024-05-08T22:04:49.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.975+0000 7f18f5400700 1 -- 172.21.15.131:0/2797680381 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f18d40a11a0 msgr2=0x7f18d40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:49.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.975+0000 7f18f5400700 1 --2- 172.21.15.131:0/2797680381 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f18d40a11a0 0x7f18d40a3660 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009df0 tx=0x7f18dc009260 comp rx=0 tx=0).stop 2024-05-08T22:04:49.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.975+0000 7f18f5400700 1 -- 172.21.15.131:0/2797680381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18f01018f0 msgr2=0x7f18f0199030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:49.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.975+0000 7f18f5400700 1 --2- 172.21.15.131:0/2797680381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18f01018f0 0x7f18f0199030 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f18e0005910 tx=0x7f18e0009580 comp rx=0 tx=0).stop 2024-05-08T22:04:49.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.975+0000 7f18f5400700 1 -- 172.21.15.131:0/2797680381 shutdown_connections 2024-05-08T22:04:49.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.975+0000 7f18f5400700 1 --2- 172.21.15.131:0/2797680381 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f18d40a11a0 0x7f18d40a3660 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:49.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.975+0000 7f18f5400700 1 --2- 172.21.15.131:0/2797680381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f0106b50 0x7f18f01a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:49.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.975+0000 7f18f5400700 1 --2- 172.21.15.131:0/2797680381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18f01018f0 0x7f18f0199030 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f18e0005910 tx=0x7f18e0009580 comp rx=0 tx=0).stop 2024-05-08T22:04:49.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.975+0000 7f18f5400700 1 --2- 172.21.15.131:0/2797680381 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18f0104220 0x7f18f0199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:49.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.975+0000 7f18f5400700 1 -- 172.21.15.131:0/2797680381 >> 172.21.15.131:0/2797680381 conn(0x7f18f00fb2e0 msgr2=0x7f18f01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:49.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.975+0000 7f18f5400700 1 -- 172.21.15.131:0/2797680381 shutdown_connections 2024-05-08T22:04:49.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:49.975+0000 7f18f5400700 1 -- 172.21.15.131:0/2797680381 wait complete. 2024-05-08T22:04:49.977 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 4 2024-05-08T22:04:50.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:49 smithi157 ceph-mon[77746]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:50.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:49 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:50.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:49 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:50.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:49 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:50.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:49 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:04:50.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:49 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:04:50.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:49 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:50.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:49 smithi132 ceph-mon[78180]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:50.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:49 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:50.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:49 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:50.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:49 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:50.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:49 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:04:50.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:49 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:04:50.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:49 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:50.345 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:49 smithi131 ceph-mon[37260]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:50.345 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:49 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:50.345 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:49 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:50.345 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:49 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:50.345 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:49 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:04:50.345 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:49 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:04:50.345 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:49 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:51.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:50 smithi131 ceph-mon[37260]: Deploying daemon mds.d on smithi131 2024-05-08T22:04:51.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:50 smithi131 ceph-mon[37260]: mds.? [v2:172.21.15.131:6834/2147999088,v1:172.21.15.131:6835/2147999088] up:boot 2024-05-08T22:04:51.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:50 smithi131 ceph-mon[37260]: fsmap 3 up:standby 2024-05-08T22:04:51.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:50 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T22:04:51.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:50 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2797680381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:04:51.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:50 smithi157 ceph-mon[77746]: Deploying daemon mds.d on smithi131 2024-05-08T22:04:51.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:50 smithi157 ceph-mon[77746]: mds.? [v2:172.21.15.131:6834/2147999088,v1:172.21.15.131:6835/2147999088] up:boot 2024-05-08T22:04:51.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:50 smithi157 ceph-mon[77746]: fsmap 3 up:standby 2024-05-08T22:04:51.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:50 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T22:04:51.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:50 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2797680381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:04:51.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:50 smithi132 ceph-mon[78180]: Deploying daemon mds.d on smithi131 2024-05-08T22:04:51.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:50 smithi132 ceph-mon[78180]: mds.? [v2:172.21.15.131:6834/2147999088,v1:172.21.15.131:6835/2147999088] up:boot 2024-05-08T22:04:51.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:50 smithi132 ceph-mon[78180]: fsmap 3 up:standby 2024-05-08T22:04:51.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:50 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T22:04:51.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:50 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2797680381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:04:51.641 INFO:journalctl@ceph.mds.d.smithi131.stdout:May 08 22:04:51 smithi131 systemd[1]: Starting Ceph mds.d for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:04:51.998 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:04:52.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.212+0000 7f0790600700 1 -- 172.21.15.131:0/3914528358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0788100e80 msgr2=0x7f07881012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:52.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.212+0000 7f0790600700 1 --2- 172.21.15.131:0/3914528358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0788100e80 0x7f07881012a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f077c009990 tx=0x7f077c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:04:52.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.212+0000 7f0790600700 1 -- 172.21.15.131:0/3914528358 shutdown_connections 2024-05-08T22:04:52.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.212+0000 7f0790600700 1 --2- 172.21.15.131:0/3914528358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0788102990 0x7f07881092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:52.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.212+0000 7f0790600700 1 --2- 172.21.15.131:0/3914528358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0788100e80 0x7f07881012a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:52.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.212+0000 7f0790600700 1 --2- 172.21.15.131:0/3914528358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0788101fd0 0x7f0788102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:52.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.212+0000 7f0790600700 1 -- 172.21.15.131:0/3914528358 >> 172.21.15.131:0/3914528358 conn(0x7f07880fc3f0 msgr2=0x7f07880fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:52.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.213+0000 7f0790600700 1 -- 172.21.15.131:0/3914528358 shutdown_connections 2024-05-08T22:04:52.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.213+0000 7f0790600700 1 -- 172.21.15.131:0/3914528358 wait complete. 2024-05-08T22:04:52.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.214+0000 7f0790600700 1 Processor -- start 2024-05-08T22:04:52.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.214+0000 7f0790600700 1 -- start start 2024-05-08T22:04:52.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.214+0000 7f0790600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0788100e80 0x7f0788074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:52.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.214+0000 7f0790600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0788101fd0 0x7f0788073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:52.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.214+0000 7f0790600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0788102990 0x7f07880738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:52.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.214+0000 7f0790600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0788074070 con 0x7f0788100e80 2024-05-08T22:04:52.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.214+0000 7f0790600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07880741e0 con 0x7f0788101fd0 2024-05-08T22:04:52.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.214+0000 7f0790600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07881a0ec0 con 0x7f0788102990 2024-05-08T22:04:52.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.214+0000 7f078dc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0788100e80 0x7f0788074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:52.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.214+0000 7f078dc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0788100e80 0x7f0788074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54304/0 (socket says 172.21.15.131:54304) 2024-05-08T22:04:52.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.214+0000 7f078dc00700 1 -- 172.21.15.131:0/1720385768 learned_addr learned my addr 172.21.15.131:0/1720385768 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:04:52.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.214+0000 7f078d200700 1 --2- 172.21.15.131:0/1720385768 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0788101fd0 0x7f0788073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:52.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.214+0000 7f078e600700 1 --2- 172.21.15.131:0/1720385768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0788102990 0x7f07880738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:52.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.215+0000 7f078dc00700 1 -- 172.21.15.131:0/1720385768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0788102990 msgr2=0x7f07880738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:52.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.215+0000 7f078dc00700 1 --2- 172.21.15.131:0/1720385768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0788102990 0x7f07880738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:52.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.215+0000 7f078dc00700 1 -- 172.21.15.131:0/1720385768 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0788101fd0 msgr2=0x7f0788073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:52.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.215+0000 7f078dc00700 1 --2- 172.21.15.131:0/1720385768 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0788101fd0 0x7f0788073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:52.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.215+0000 7f078dc00700 1 -- 172.21.15.131:0/1720385768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c0095f0 con 0x7f0788100e80 2024-05-08T22:04:52.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.215+0000 7f078dc00700 1 --2- 172.21.15.131:0/1720385768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0788100e80 0x7f0788074d00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f077c005910 tx=0x7f077c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:52.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.215+0000 7f0786000700 1 -- 172.21.15.131:0/1720385768 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c01d070 con 0x7f0788100e80 2024-05-08T22:04:52.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.215+0000 7f0786000700 1 -- 172.21.15.131:0/1720385768 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f077c021410 con 0x7f0788100e80 2024-05-08T22:04:52.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.215+0000 7f0786000700 1 -- 172.21.15.131:0/1720385768 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c017650 con 0x7f0788100e80 2024-05-08T22:04:52.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.215+0000 7f0790600700 1 -- 172.21.15.131:0/1720385768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07881a10f0 con 0x7f0788100e80 2024-05-08T22:04:52.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.216+0000 7f0790600700 1 -- 172.21.15.131:0/1720385768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07881a1460 con 0x7f0788100e80 2024-05-08T22:04:52.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.217+0000 7f0790600700 1 -- 172.21.15.131:0/1720385768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07881082f0 con 0x7f0788100e80 2024-05-08T22:04:52.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.219+0000 7f0786000700 1 -- 172.21.15.131:0/1720385768 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f077c021580 con 0x7f0788100e80 2024-05-08T22:04:52.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.220+0000 7f0786000700 1 --2- 172.21.15.131:0/1720385768 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f07700a11a0 0x7f07700a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:52.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.220+0000 7f0786000700 1 -- 172.21.15.131:0/1720385768 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f077c0cdeb0 con 0x7f0788100e80 2024-05-08T22:04:52.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.220+0000 7f078d200700 1 --2- 172.21.15.131:0/1720385768 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f07700a11a0 0x7f07700a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:52.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.221+0000 7f078d200700 1 --2- 172.21.15.131:0/1720385768 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f07700a11a0 0x7f07700a3660 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0774009c60 tx=0x7f0774009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:52.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.224+0000 7f0786000700 1 -- 172.21.15.131:0/1720385768 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f077c097ad0 con 0x7f0788100e80 2024-05-08T22:04:52.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:51 smithi157 ceph-mon[77746]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:52.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:51 smithi132 ceph-mon[78180]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:52.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:51 smithi131 ceph-mon[37260]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:52.391 INFO:journalctl@ceph.mds.d.smithi131.stdout:May 08 22:04:52 smithi131 ceph-mds[86717]: -- 172.21.15.131:0/846935616 <== mon.0 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557df44081a0 con 0x557df43e0800 2024-05-08T22:04:52.391 INFO:journalctl@ceph.mds.d.smithi131.stdout:May 08 22:04:52 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-d[86681]: starting mds.d at 2024-05-08T22:04:52.391 INFO:journalctl@ceph.mds.d.smithi131.stdout:May 08 22:04:52 smithi131 bash[86666]: 959420d43727a1552d6a1f575900a35a6eb6aa394ce92815be2417c32401f49d 2024-05-08T22:04:52.391 INFO:journalctl@ceph.mds.d.smithi131.stdout:May 08 22:04:52 smithi131 systemd[1]: Started Ceph mds.d for 06f1a696-0d86-11ef-bc97-c7b262605968. 2024-05-08T22:04:52.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.480+0000 7f0790600700 1 -- 172.21.15.131:0/1720385768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0788067380 con 0x7f0788100e80 2024-05-08T22:04:52.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.481+0000 7f0786000700 1 -- 172.21.15.131:0/1720385768 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2777 (secure 0 0 0) 0x7f077c09b0f0 con 0x7f0788100e80 2024-05-08T22:04:52.482 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:04:52.482 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[]} 2024-05-08T22:04:52.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.483+0000 7f0790600700 1 -- 172.21.15.131:0/1720385768 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f07700a11a0 msgr2=0x7f07700a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:52.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.483+0000 7f0790600700 1 --2- 172.21.15.131:0/1720385768 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f07700a11a0 0x7f07700a3660 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0774009c60 tx=0x7f0774009260 comp rx=0 tx=0).stop 2024-05-08T22:04:52.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.483+0000 7f0790600700 1 -- 172.21.15.131:0/1720385768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0788100e80 msgr2=0x7f0788074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:52.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.483+0000 7f0790600700 1 --2- 172.21.15.131:0/1720385768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0788100e80 0x7f0788074d00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f077c005910 tx=0x7f077c005740 comp rx=0 tx=0).stop 2024-05-08T22:04:52.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.483+0000 7f0790600700 1 -- 172.21.15.131:0/1720385768 shutdown_connections 2024-05-08T22:04:52.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.483+0000 7f0790600700 1 --2- 172.21.15.131:0/1720385768 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f07700a11a0 0x7f07700a3660 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:52.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.483+0000 7f0790600700 1 --2- 172.21.15.131:0/1720385768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0788102990 0x7f07880738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:52.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.483+0000 7f0790600700 1 --2- 172.21.15.131:0/1720385768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0788100e80 0x7f0788074d00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:52.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.483+0000 7f0790600700 1 --2- 172.21.15.131:0/1720385768 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0788101fd0 0x7f0788073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:52.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.483+0000 7f0790600700 1 -- 172.21.15.131:0/1720385768 >> 172.21.15.131:0/1720385768 conn(0x7f07880fc3f0 msgr2=0x7f0788105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:52.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.484+0000 7f0790600700 1 -- 172.21.15.131:0/1720385768 shutdown_connections 2024-05-08T22:04:52.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:52.484+0000 7f0790600700 1 -- 172.21.15.131:0/1720385768 wait complete. 2024-05-08T22:04:52.486 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 4 2024-05-08T22:04:53.328 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:53 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:53.328 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:53 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:53.328 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:53 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:53.328 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:53 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:04:53.328 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:53 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:04:53.329 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:53 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:53.329 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:53 smithi132 ceph-mon[78180]: Deploying daemon mds.e on smithi132 2024-05-08T22:04:53.329 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:53 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1720385768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:04:53.329 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:53 smithi132 ceph-mon[78180]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:53.329 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:53 smithi132 ceph-mon[78180]: mds.? [v2:172.21.15.131:6836/1062828950,v1:172.21.15.131:6837/1062828950] up:boot 2024-05-08T22:04:53.329 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:53 smithi132 ceph-mon[78180]: fsmap 4 up:standby 2024-05-08T22:04:53.329 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:53 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T22:04:53.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:53 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:53.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:53 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:53.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:53 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:53.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:53 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:04:53.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:53 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:04:53.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:53 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:53.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:53 smithi157 ceph-mon[77746]: Deploying daemon mds.e on smithi132 2024-05-08T22:04:53.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:53 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1720385768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:04:53.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:53 smithi157 ceph-mon[77746]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:53.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:53 smithi157 ceph-mon[77746]: mds.? [v2:172.21.15.131:6836/1062828950,v1:172.21.15.131:6837/1062828950] up:boot 2024-05-08T22:04:53.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:53 smithi157 ceph-mon[77746]: fsmap 4 up:standby 2024-05-08T22:04:53.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:53 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T22:04:53.590 INFO:journalctl@ceph.mds.e.smithi132.stdout:May 08 22:04:53 smithi132 systemd[1]: Starting Ceph mds.e for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:04:53.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:53 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:53.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:53 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:53.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:53 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:53.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:53 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:04:53.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:53 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:04:53.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:53 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:53.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:53 smithi131 ceph-mon[37260]: Deploying daemon mds.e on smithi132 2024-05-08T22:04:53.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:53 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1720385768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:04:53.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:53 smithi131 ceph-mon[37260]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:53.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:53 smithi131 ceph-mon[37260]: mds.? [v2:172.21.15.131:6836/1062828950,v1:172.21.15.131:6837/1062828950] up:boot 2024-05-08T22:04:53.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:53 smithi131 ceph-mon[37260]: fsmap 4 up:standby 2024-05-08T22:04:53.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:53 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T22:04:54.090 INFO:journalctl@ceph.mds.e.smithi132.stdout:May 08 22:04:53 smithi132 ceph-mds[114161]: -- 172.21.15.132:0/1324201534 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56338973a1a0 con 0x563389710c00 2024-05-08T22:04:54.090 INFO:journalctl@ceph.mds.e.smithi132.stdout:May 08 22:04:53 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-e[114138]: starting mds.e at 2024-05-08T22:04:54.090 INFO:journalctl@ceph.mds.e.smithi132.stdout:May 08 22:04:53 smithi132 bash[114123]: 7b23598426c833e2587f6d2360a49afee20e81460cefbae63510199ec4a03c2a 2024-05-08T22:04:54.090 INFO:journalctl@ceph.mds.e.smithi132.stdout:May 08 22:04:53 smithi132 systemd[1]: Started Ceph mds.e for 06f1a696-0d86-11ef-bc97-c7b262605968. 2024-05-08T22:04:54.505 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:04:54.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.708+0000 7f1a9b000700 1 -- 172.21.15.131:0/3380397343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a94101f90 msgr2=0x7f1a94102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:54.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.708+0000 7f1a9b000700 1 --2- 172.21.15.131:0/3380397343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a94101f90 0x7f1a94102410 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009990 tx=0x7f1a80009ca0 comp rx=0 tx=0).stop 2024-05-08T22:04:54.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.709+0000 7f1a9b000700 1 -- 172.21.15.131:0/3380397343 shutdown_connections 2024-05-08T22:04:54.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.709+0000 7f1a9b000700 1 --2- 172.21.15.131:0/3380397343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a94102950 0x7f1a94109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:54.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.709+0000 7f1a9b000700 1 --2- 172.21.15.131:0/3380397343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a94100e40 0x7f1a94101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:54.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.709+0000 7f1a9b000700 1 --2- 172.21.15.131:0/3380397343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a94101f90 0x7f1a94102410 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:54.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.709+0000 7f1a9b000700 1 -- 172.21.15.131:0/3380397343 >> 172.21.15.131:0/3380397343 conn(0x7f1a940fc3b0 msgr2=0x7f1a940fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:54.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.709+0000 7f1a9b000700 1 -- 172.21.15.131:0/3380397343 shutdown_connections 2024-05-08T22:04:54.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.709+0000 7f1a9b000700 1 -- 172.21.15.131:0/3380397343 wait complete. 2024-05-08T22:04:54.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.710+0000 7f1a9b000700 1 Processor -- start 2024-05-08T22:04:54.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.710+0000 7f1a9b000700 1 -- start start 2024-05-08T22:04:54.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.710+0000 7f1a9b000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a94100e40 0x7f1a94074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:54.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.710+0000 7f1a9b000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a94101f90 0x7f1a94073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:54.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.710+0000 7f1a9b000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a94102950 0x7f1a940738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:54.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.710+0000 7f1a9b000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a94074070 con 0x7f1a94101f90 2024-05-08T22:04:54.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.710+0000 7f1a9b000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a940741e0 con 0x7f1a94102950 2024-05-08T22:04:54.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.710+0000 7f1a9b000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a941a0e20 con 0x7f1a94100e40 2024-05-08T22:04:54.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.710+0000 7f1a99c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a94100e40 0x7f1a94074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:54.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.710+0000 7f1a99200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a94101f90 0x7f1a94073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:54.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.710+0000 7f1a99c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a94100e40 0x7f1a94074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51710/0 (socket says 172.21.15.131:51710) 2024-05-08T22:04:54.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.710+0000 7f1a99c00700 1 -- 172.21.15.131:0/3183275069 learned_addr learned my addr 172.21.15.131:0/3183275069 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:04:54.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.711+0000 7f1a9a600700 1 --2- 172.21.15.131:0/3183275069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a94102950 0x7f1a940738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:54.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.711+0000 7f1a99c00700 1 -- 172.21.15.131:0/3183275069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a94102950 msgr2=0x7f1a940738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:54.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.711+0000 7f1a99c00700 1 --2- 172.21.15.131:0/3183275069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a94102950 0x7f1a940738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:54.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.711+0000 7f1a99c00700 1 -- 172.21.15.131:0/3183275069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a94101f90 msgr2=0x7f1a94073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:54.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.711+0000 7f1a99c00700 1 --2- 172.21.15.131:0/3183275069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a94101f90 0x7f1a94073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:54.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.711+0000 7f1a99c00700 1 -- 172.21.15.131:0/3183275069 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a800095f0 con 0x7f1a94100e40 2024-05-08T22:04:54.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.711+0000 7f1a99c00700 1 --2- 172.21.15.131:0/3183275069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a94100e40 0x7f1a94074d00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1a8800e9a0 tx=0x7f1a8800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:54.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.711+0000 7f1a92000700 1 -- 172.21.15.131:0/3183275069 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a88009790 con 0x7f1a94100e40 2024-05-08T22:04:54.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.711+0000 7f1a92000700 1 -- 172.21.15.131:0/3183275069 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a8800cbd0 con 0x7f1a94100e40 2024-05-08T22:04:54.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.711+0000 7f1a9b000700 1 -- 172.21.15.131:0/3183275069 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a941a1050 con 0x7f1a94100e40 2024-05-08T22:04:54.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.711+0000 7f1a92000700 1 -- 172.21.15.131:0/3183275069 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a88018680 con 0x7f1a94100e40 2024-05-08T22:04:54.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.711+0000 7f1a9b000700 1 -- 172.21.15.131:0/3183275069 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a941a1500 con 0x7f1a94100e40 2024-05-08T22:04:54.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.712+0000 7f1a9b000700 1 -- 172.21.15.131:0/3183275069 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a94108260 con 0x7f1a94100e40 2024-05-08T22:04:54.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.714+0000 7f1a92000700 1 -- 172.21.15.131:0/3183275069 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1a8800cd40 con 0x7f1a94100e40 2024-05-08T22:04:54.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.716+0000 7f1a92000700 1 --2- 172.21.15.131:0/3183275069 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1a7c0a1120 0x7f1a7c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:54.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.716+0000 7f1a92000700 1 -- 172.21.15.131:0/3183275069 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1a88014070 con 0x7f1a94100e40 2024-05-08T22:04:54.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.716+0000 7f1a99200700 1 --2- 172.21.15.131:0/3183275069 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1a7c0a1120 0x7f1a7c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:54.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.716+0000 7f1a99200700 1 --2- 172.21.15.131:0/3183275069 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1a7c0a1120 0x7f1a7c0a35e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1a8000b2e0 tx=0x7f1a80005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:54.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.718+0000 7f1a92000700 1 -- 172.21.15.131:0/3183275069 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1a88052d60 con 0x7f1a94100e40 2024-05-08T22:04:54.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.975+0000 7f1a9b000700 1 -- 172.21.15.131:0/3183275069 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a9410c970 con 0x7f1a94100e40 2024-05-08T22:04:54.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.976+0000 7f1a92000700 1 -- 172.21.15.131:0/3183275069 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4297 (secure 0 0 0) 0x7f1a88096340 con 0x7f1a94100e40 2024-05-08T22:04:54.977 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:04:54.977 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2024-05-08T22:04:54.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.979+0000 7f1a9b000700 1 -- 172.21.15.131:0/3183275069 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1a7c0a1120 msgr2=0x7f1a7c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:54.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.979+0000 7f1a9b000700 1 --2- 172.21.15.131:0/3183275069 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1a7c0a1120 0x7f1a7c0a35e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1a8000b2e0 tx=0x7f1a80005fb0 comp rx=0 tx=0).stop 2024-05-08T22:04:54.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.979+0000 7f1a9b000700 1 -- 172.21.15.131:0/3183275069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a94100e40 msgr2=0x7f1a94074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:54.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.979+0000 7f1a9b000700 1 --2- 172.21.15.131:0/3183275069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a94100e40 0x7f1a94074d00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1a8800e9a0 tx=0x7f1a8800ecb0 comp rx=0 tx=0).stop 2024-05-08T22:04:54.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.979+0000 7f1a9b000700 1 -- 172.21.15.131:0/3183275069 shutdown_connections 2024-05-08T22:04:54.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.979+0000 7f1a9b000700 1 --2- 172.21.15.131:0/3183275069 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1a7c0a1120 0x7f1a7c0a35e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:54.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.979+0000 7f1a9b000700 1 --2- 172.21.15.131:0/3183275069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a94102950 0x7f1a940738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:54.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.979+0000 7f1a9b000700 1 --2- 172.21.15.131:0/3183275069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a94100e40 0x7f1a94074d00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:54.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.979+0000 7f1a9b000700 1 --2- 172.21.15.131:0/3183275069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a94101f90 0x7f1a94073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:54.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.979+0000 7f1a9b000700 1 -- 172.21.15.131:0/3183275069 >> 172.21.15.131:0/3183275069 conn(0x7f1a940fc3b0 msgr2=0x7f1a941053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:54.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.979+0000 7f1a9b000700 1 -- 172.21.15.131:0/3183275069 shutdown_connections 2024-05-08T22:04:54.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:54.979+0000 7f1a9b000700 1 -- 172.21.15.131:0/3183275069 wait complete. 2024-05-08T22:04:54.981 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 6 2024-05-08T22:04:55.011 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:54 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:55.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:54 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:55.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:54 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:55.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:54 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:04:55.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:54 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:04:55.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:54 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:55.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:54 smithi157 ceph-mon[77746]: Deploying daemon mds.f on smithi157 2024-05-08T22:04:55.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:54 smithi157 ceph-mon[77746]: mds.? [v2:172.21.15.132:6834/2610471069,v1:172.21.15.132:6835/2610471069] up:boot 2024-05-08T22:04:55.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:54 smithi157 ceph-mon[77746]: fsmap 5 up:standby 2024-05-08T22:04:55.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:54 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T22:04:55.276 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 08 22:04:55 smithi157 systemd[1]: Starting Ceph mds.f for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:04:55.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:54 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:55.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:54 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:55.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:54 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:55.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:54 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:04:55.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:54 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:04:55.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:54 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:55.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:54 smithi132 ceph-mon[78180]: Deploying daemon mds.f on smithi157 2024-05-08T22:04:55.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:54 smithi132 ceph-mon[78180]: mds.? [v2:172.21.15.132:6834/2610471069,v1:172.21.15.132:6835/2610471069] up:boot 2024-05-08T22:04:55.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:54 smithi132 ceph-mon[78180]: fsmap 5 up:standby 2024-05-08T22:04:55.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:54 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T22:04:55.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:54 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:55.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:54 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:55.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:54 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:55.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:54 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:04:55.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:54 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:04:55.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:54 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:55.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:54 smithi131 ceph-mon[37260]: Deploying daemon mds.f on smithi157 2024-05-08T22:04:55.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:54 smithi131 ceph-mon[37260]: mds.? [v2:172.21.15.132:6834/2610471069,v1:172.21.15.132:6835/2610471069] up:boot 2024-05-08T22:04:55.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:54 smithi131 ceph-mon[37260]: fsmap 5 up:standby 2024-05-08T22:04:55.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:54 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T22:04:55.960 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 08 22:04:55 smithi157 bash[114899]: cf1ee9d65125104f268e6fd5f28774a7d515cdfff5542265dac216204687ddf1 2024-05-08T22:04:55.961 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 08 22:04:55 smithi157 systemd[1]: Started Ceph mds.f for 06f1a696-0d86-11ef-bc97-c7b262605968. 2024-05-08T22:04:55.961 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 08 22:04:55 smithi157 ceph-mds[114936]: -- 172.21.15.157:0/3796948332 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fd166a01a0 con 0x55fd16676800 2024-05-08T22:04:55.961 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 08 22:04:55 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-f[114914]: starting mds.f at 2024-05-08T22:04:56.246 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:55 smithi157 ceph-mon[77746]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:56.246 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:55 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3183275069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:04:56.246 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:55 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:56.246 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:55 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:56.246 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:55 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:56.247 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:55 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:04:56.247 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:55 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:04:56.247 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:55 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:56.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:55 smithi132 ceph-mon[78180]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:56.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:55 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3183275069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:04:56.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:55 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:56.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:55 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:56.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:55 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:56.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:55 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:04:56.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:55 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:04:56.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:55 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:56.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:55 smithi131 ceph-mon[37260]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:56.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:55 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3183275069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:04:56.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:55 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:56.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:55 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:56.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:55 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:56.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:55 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:04:56.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:55 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:04:56.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:55 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:57.001 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:04:57.174 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:56 smithi157 ceph-mon[77746]: Deploying daemon mds.c on smithi157 2024-05-08T22:04:57.175 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:56 smithi157 ceph-mon[77746]: mds.? [v2:172.21.15.157:6834/1436814390,v1:172.21.15.157:6835/1436814390] up:boot 2024-05-08T22:04:57.175 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:56 smithi157 ceph-mon[77746]: fsmap 6 up:standby 2024-05-08T22:04:57.175 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:56 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T22:04:57.175 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 08 22:04:56 smithi157 systemd[1]: Starting Ceph mds.c for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:04:57.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.203+0000 7f8a83e00700 1 -- 172.21.15.131:0/585950540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a7c101fd0 msgr2=0x7f8a7c102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:57.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.203+0000 7f8a83e00700 1 --2- 172.21.15.131:0/585950540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a7c101fd0 0x7f8a7c102450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8a68009990 tx=0x7f8a68009ca0 comp rx=0 tx=0).stop 2024-05-08T22:04:57.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.203+0000 7f8a83e00700 1 -- 172.21.15.131:0/585950540 shutdown_connections 2024-05-08T22:04:57.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.203+0000 7f8a83e00700 1 --2- 172.21.15.131:0/585950540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a7c102990 0x7f8a7c1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:57.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.203+0000 7f8a83e00700 1 --2- 172.21.15.131:0/585950540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c100e80 0x7f8a7c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:57.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.203+0000 7f8a83e00700 1 --2- 172.21.15.131:0/585950540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a7c101fd0 0x7f8a7c102450 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:57.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.203+0000 7f8a83e00700 1 -- 172.21.15.131:0/585950540 >> 172.21.15.131:0/585950540 conn(0x7f8a7c0fc410 msgr2=0x7f8a7c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:57.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.203+0000 7f8a83e00700 1 -- 172.21.15.131:0/585950540 shutdown_connections 2024-05-08T22:04:57.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.203+0000 7f8a83e00700 1 -- 172.21.15.131:0/585950540 wait complete. 2024-05-08T22:04:57.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.204+0000 7f8a83e00700 1 Processor -- start 2024-05-08T22:04:57.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.204+0000 7f8a83e00700 1 -- start start 2024-05-08T22:04:57.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.204+0000 7f8a83e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a7c100e80 0x7f8a7c199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:57.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.204+0000 7f8a83e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c101fd0 0x7f8a7c199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:57.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.204+0000 7f8a83e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a7c102990 0x7f8a7c1a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:57.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.204+0000 7f8a83e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c1a0b40 con 0x7f8a7c102990 2024-05-08T22:04:57.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.204+0000 7f8a83e00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c1a0c80 con 0x7f8a7c100e80 2024-05-08T22:04:57.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.204+0000 7f8a83e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c1a0e10 con 0x7f8a7c101fd0 2024-05-08T22:04:57.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.204+0000 7f8a81e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a7c102990 0x7f8a7c1a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:57.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.205+0000 7f8a81e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a7c102990 0x7f8a7c1a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42272/0 (socket says 172.21.15.131:42272) 2024-05-08T22:04:57.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.205+0000 7f8a81e00700 1 -- 172.21.15.131:0/23623359 learned_addr learned my addr 172.21.15.131:0/23623359 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:04:57.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.205+0000 7f8a81400700 1 --2- 172.21.15.131:0/23623359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a7c100e80 0x7f8a7c199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:57.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.205+0000 7f8a80a00700 1 --2- 172.21.15.131:0/23623359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c101fd0 0x7f8a7c199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:57.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.205+0000 7f8a81e00700 1 -- 172.21.15.131:0/23623359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c101fd0 msgr2=0x7f8a7c199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:57.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.205+0000 7f8a81e00700 1 --2- 172.21.15.131:0/23623359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c101fd0 0x7f8a7c199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:57.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.205+0000 7f8a81e00700 1 -- 172.21.15.131:0/23623359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a7c100e80 msgr2=0x7f8a7c199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:57.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.205+0000 7f8a81e00700 1 --2- 172.21.15.131:0/23623359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a7c100e80 0x7f8a7c199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:57.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.205+0000 7f8a81e00700 1 -- 172.21.15.131:0/23623359 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a680095f0 con 0x7f8a7c102990 2024-05-08T22:04:57.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.205+0000 7f8a80a00700 1 --2- 172.21.15.131:0/23623359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c101fd0 0x7f8a7c199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:04:57.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.205+0000 7f8a81e00700 1 --2- 172.21.15.131:0/23623359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a7c102990 0x7f8a7c1a0570 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400ef30 tx=0x7f8a7400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:57.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.205+0000 7f8a7a000700 1 -- 172.21.15.131:0/23623359 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a74018070 con 0x7f8a7c102990 2024-05-08T22:04:57.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.205+0000 7f8a7a000700 1 -- 172.21.15.131:0/23623359 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a7400f040 con 0x7f8a7c102990 2024-05-08T22:04:57.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.205+0000 7f8a83e00700 1 -- 172.21.15.131:0/23623359 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a7c1a1090 con 0x7f8a7c102990 2024-05-08T22:04:57.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.205+0000 7f8a7a000700 1 -- 172.21.15.131:0/23623359 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7400cdb0 con 0x7f8a7c102990 2024-05-08T22:04:57.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.205+0000 7f8a83e00700 1 -- 172.21.15.131:0/23623359 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a7c1a15f0 con 0x7f8a7c102990 2024-05-08T22:04:57.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.206+0000 7f8a83e00700 1 -- 172.21.15.131:0/23623359 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a7c067c10 con 0x7f8a7c102990 2024-05-08T22:04:57.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.208+0000 7f8a7a000700 1 -- 172.21.15.131:0/23623359 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8a74003a00 con 0x7f8a7c102990 2024-05-08T22:04:57.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.211+0000 7f8a7a000700 1 --2- 172.21.15.131:0/23623359 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8a640a11a0 0x7f8a640a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:57.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.211+0000 7f8a7a000700 1 -- 172.21.15.131:0/23623359 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8a740cca00 con 0x7f8a7c102990 2024-05-08T22:04:57.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.211+0000 7f8a81400700 1 --2- 172.21.15.131:0/23623359 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8a640a11a0 0x7f8a640a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:57.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.211+0000 7f8a81400700 1 --2- 172.21.15.131:0/23623359 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8a640a11a0 0x7f8a640a3660 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8a70009c60 tx=0x7f8a70009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:57.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.213+0000 7f8a7a000700 1 -- 172.21.15.131:0/23623359 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8a74096620 con 0x7f8a7c102990 2024-05-08T22:04:57.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:56 smithi132 ceph-mon[78180]: Deploying daemon mds.c on smithi157 2024-05-08T22:04:57.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:56 smithi132 ceph-mon[78180]: mds.? [v2:172.21.15.157:6834/1436814390,v1:172.21.15.157:6835/1436814390] up:boot 2024-05-08T22:04:57.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:56 smithi132 ceph-mon[78180]: fsmap 6 up:standby 2024-05-08T22:04:57.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:56 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T22:04:57.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:56 smithi131 ceph-mon[37260]: Deploying daemon mds.c on smithi157 2024-05-08T22:04:57.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:56 smithi131 ceph-mon[37260]: mds.? [v2:172.21.15.157:6834/1436814390,v1:172.21.15.157:6835/1436814390] up:boot 2024-05-08T22:04:57.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:56 smithi131 ceph-mon[37260]: fsmap 6 up:standby 2024-05-08T22:04:57.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:56 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T22:04:57.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.471+0000 7f8a83e00700 1 -- 172.21.15.131:0/23623359 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a7c10c6d0 con 0x7f8a7c102990 2024-05-08T22:04:57.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.471+0000 7f8a7a000700 1 -- 172.21.15.131:0/23623359 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5057 (secure 0 0 0) 0x7f8a74099c40 con 0x7f8a7c102990 2024-05-08T22:04:57.472 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:04:57.472 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2024-05-08T22:04:57.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.474+0000 7f8a83e00700 1 -- 172.21.15.131:0/23623359 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8a640a11a0 msgr2=0x7f8a640a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:57.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.474+0000 7f8a83e00700 1 --2- 172.21.15.131:0/23623359 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8a640a11a0 0x7f8a640a3660 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8a70009c60 tx=0x7f8a70009260 comp rx=0 tx=0).stop 2024-05-08T22:04:57.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.474+0000 7f8a83e00700 1 -- 172.21.15.131:0/23623359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a7c102990 msgr2=0x7f8a7c1a0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:57.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.474+0000 7f8a83e00700 1 --2- 172.21.15.131:0/23623359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a7c102990 0x7f8a7c1a0570 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400ef30 tx=0x7f8a7400c3a0 comp rx=0 tx=0).stop 2024-05-08T22:04:57.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.474+0000 7f8a83e00700 1 -- 172.21.15.131:0/23623359 shutdown_connections 2024-05-08T22:04:57.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.474+0000 7f8a83e00700 1 --2- 172.21.15.131:0/23623359 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8a640a11a0 0x7f8a640a3660 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:57.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.474+0000 7f8a83e00700 1 --2- 172.21.15.131:0/23623359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a7c102990 0x7f8a7c1a0570 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:57.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.474+0000 7f8a83e00700 1 --2- 172.21.15.131:0/23623359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a7c100e80 0x7f8a7c199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:57.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.474+0000 7f8a83e00700 1 --2- 172.21.15.131:0/23623359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7c101fd0 0x7f8a7c199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:57.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.474+0000 7f8a83e00700 1 -- 172.21.15.131:0/23623359 >> 172.21.15.131:0/23623359 conn(0x7f8a7c0fc410 msgr2=0x7f8a7c0731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:57.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.474+0000 7f8a83e00700 1 -- 172.21.15.131:0/23623359 shutdown_connections 2024-05-08T22:04:57.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:57.474+0000 7f8a83e00700 1 -- 172.21.15.131:0/23623359 wait complete. 2024-05-08T22:04:57.476 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 7 2024-05-08T22:04:57.776 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 08 22:04:57 smithi157 ceph-mds[115277]: -- 172.21.15.157:0/4195300547 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55621a9941a0 con 0x55621a96b000 2024-05-08T22:04:57.776 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 08 22:04:57 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-c[115254]: starting mds.c at 2024-05-08T22:04:57.776 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 08 22:04:57 smithi157 bash[115239]: a6cd41bd7be5ccec4befb963db2713fc3fd22c136d6ea78b484e77f54a6c9b80 2024-05-08T22:04:57.776 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 08 22:04:57 smithi157 systemd[1]: Started Ceph mds.c for 06f1a696-0d86-11ef-bc97-c7b262605968. 2024-05-08T22:04:58.211 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:57 smithi132 ceph-mon[78180]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:58.211 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:57 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:58.211 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:57 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:58.211 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:57 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:58.211 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:57 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:04:58.211 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:57 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:04:58.212 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:57 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:58.212 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:57 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/23623359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:04:58.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:57 smithi157 ceph-mon[77746]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:58.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:57 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:58.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:57 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:58.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:57 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:58.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:57 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:04:58.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:57 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:04:58.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:57 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:58.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:57 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/23623359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:04:58.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:57 smithi131 ceph-mon[37260]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:04:58.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:57 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:58.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:57 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:58.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:57 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:04:58.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:57 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:04:58.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:57 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:04:58.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:57 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:04:58.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:57 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/23623359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:04:58.758 INFO:journalctl@ceph.mds.h.smithi132.stdout:May 08 22:04:58 smithi132 systemd[1]: Starting Ceph mds.h for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:04:59.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:58 smithi157 ceph-mon[77746]: Deploying daemon mds.h on smithi132 2024-05-08T22:04:59.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:58 smithi157 ceph-mon[77746]: mds.? [v2:172.21.15.157:6836/2693198274,v1:172.21.15.157:6837/2693198274] up:boot 2024-05-08T22:04:59.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:58 smithi157 ceph-mon[77746]: fsmap 7 up:standby 2024-05-08T22:04:59.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:58 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T22:04:59.340 INFO:journalctl@ceph.mds.h.smithi132.stdout:May 08 22:04:59 smithi132 ceph-mds[114505]: -- 172.21.15.132:0/2900427709 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56125ebd61a0 con 0x56125ebacc00 2024-05-08T22:04:59.340 INFO:journalctl@ceph.mds.h.smithi132.stdout:May 08 22:04:59 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-h[114482]: starting mds.h at 2024-05-08T22:04:59.340 INFO:journalctl@ceph.mds.h.smithi132.stdout:May 08 22:04:59 smithi132 bash[114467]: 6dfed167cf71f42dc38ff9ad56a6e8ad660ebfebae5b88f9626b4cfd060b03bb 2024-05-08T22:04:59.340 INFO:journalctl@ceph.mds.h.smithi132.stdout:May 08 22:04:59 smithi132 systemd[1]: Started Ceph mds.h for 06f1a696-0d86-11ef-bc97-c7b262605968. 2024-05-08T22:04:59.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:58 smithi132 ceph-mon[78180]: Deploying daemon mds.h on smithi132 2024-05-08T22:04:59.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:58 smithi132 ceph-mon[78180]: mds.? [v2:172.21.15.157:6836/2693198274,v1:172.21.15.157:6837/2693198274] up:boot 2024-05-08T22:04:59.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:58 smithi132 ceph-mon[78180]: fsmap 7 up:standby 2024-05-08T22:04:59.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:58 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T22:04:59.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:58 smithi131 ceph-mon[37260]: Deploying daemon mds.h on smithi132 2024-05-08T22:04:59.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:58 smithi131 ceph-mon[37260]: mds.? [v2:172.21.15.157:6836/2693198274,v1:172.21.15.157:6837/2693198274] up:boot 2024-05-08T22:04:59.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:58 smithi131 ceph-mon[37260]: fsmap 7 up:standby 2024-05-08T22:04:59.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:58 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T22:04:59.495 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:04:59.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.702+0000 7f3327800700 1 -- 172.21.15.131:0/1710611993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3320102980 msgr2=0x7f33201092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:59.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.702+0000 7f3327800700 1 --2- 172.21.15.131:0/1710611993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3320102980 0x7f33201092a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f331800b0a0 tx=0x7f331800b3b0 comp rx=0 tx=0).stop 2024-05-08T22:04:59.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.702+0000 7f3327800700 1 -- 172.21.15.131:0/1710611993 shutdown_connections 2024-05-08T22:04:59.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.702+0000 7f3327800700 1 --2- 172.21.15.131:0/1710611993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3320102980 0x7f33201092a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:59.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.702+0000 7f3327800700 1 --2- 172.21.15.131:0/1710611993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3320100e70 0x7f3320101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:59.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.702+0000 7f3327800700 1 --2- 172.21.15.131:0/1710611993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3320101fc0 0x7f3320102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:59.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.702+0000 7f3327800700 1 -- 172.21.15.131:0/1710611993 >> 172.21.15.131:0/1710611993 conn(0x7f33200fc3e0 msgr2=0x7f33200fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:59.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.702+0000 7f3327800700 1 -- 172.21.15.131:0/1710611993 shutdown_connections 2024-05-08T22:04:59.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.703+0000 7f3327800700 1 -- 172.21.15.131:0/1710611993 wait complete. 2024-05-08T22:04:59.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.703+0000 7f3327800700 1 Processor -- start 2024-05-08T22:04:59.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.704+0000 7f3327800700 1 -- start start 2024-05-08T22:04:59.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.704+0000 7f3327800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3320100e70 0x7f3320198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:59.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.704+0000 7f3327800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3320101fc0 0x7f3320199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:59.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.704+0000 7f3327800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3320102980 0x7f33201a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:59.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.704+0000 7f3327800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33201a0a20 con 0x7f3320102980 2024-05-08T22:04:59.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.704+0000 7f3327800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33201a0b60 con 0x7f3320100e70 2024-05-08T22:04:59.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.704+0000 7f3327800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33201a0cf0 con 0x7f3320101fc0 2024-05-08T22:04:59.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.704+0000 7f3324e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3320100e70 0x7f3320198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:59.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.704+0000 7f3325800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3320102980 0x7f33201a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:59.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.704+0000 7f3324e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3320100e70 0x7f3320198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:33164/0 (socket says 172.21.15.131:33164) 2024-05-08T22:04:59.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.704+0000 7f3324e00700 1 -- 172.21.15.131:0/3313520912 learned_addr learned my addr 172.21.15.131:0/3313520912 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:04:59.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.704+0000 7f331fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3320101fc0 0x7f3320199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:59.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.704+0000 7f3324e00700 1 -- 172.21.15.131:0/3313520912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3320101fc0 msgr2=0x7f3320199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:59.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.704+0000 7f3324e00700 1 --2- 172.21.15.131:0/3313520912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3320101fc0 0x7f3320199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:59.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.705+0000 7f3324e00700 1 -- 172.21.15.131:0/3313520912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3320102980 msgr2=0x7f33201a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:59.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.705+0000 7f3324e00700 1 --2- 172.21.15.131:0/3313520912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3320102980 0x7f33201a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:59.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.705+0000 7f3324e00700 1 -- 172.21.15.131:0/3313520912 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3318009d00 con 0x7f3320100e70 2024-05-08T22:04:59.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.705+0000 7f3324e00700 1 --2- 172.21.15.131:0/3313520912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3320100e70 0x7f3320198ef0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f331400e9a0 tx=0x7f331400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:59.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.705+0000 7f331d600700 1 -- 172.21.15.131:0/3313520912 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3314009790 con 0x7f3320100e70 2024-05-08T22:04:59.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.705+0000 7f331d600700 1 -- 172.21.15.131:0/3313520912 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3314010430 con 0x7f3320100e70 2024-05-08T22:04:59.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.705+0000 7f3327800700 1 -- 172.21.15.131:0/3313520912 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33201a0fd0 con 0x7f3320100e70 2024-05-08T22:04:59.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.705+0000 7f331d600700 1 -- 172.21.15.131:0/3313520912 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3314018680 con 0x7f3320100e70 2024-05-08T22:04:59.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.705+0000 7f3327800700 1 -- 172.21.15.131:0/3313520912 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33201a13f0 con 0x7f3320100e70 2024-05-08T22:04:59.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.706+0000 7f3327800700 1 -- 172.21.15.131:0/3313520912 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3320067c10 con 0x7f3320100e70 2024-05-08T22:04:59.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.708+0000 7f331d600700 1 -- 172.21.15.131:0/3313520912 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f33140105a0 con 0x7f3320100e70 2024-05-08T22:04:59.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.709+0000 7f331d600700 1 --2- 172.21.15.131:0/3313520912 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f33080a1120 0x7f33080a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:04:59.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.709+0000 7f331d600700 1 -- 172.21.15.131:0/3313520912 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3314014070 con 0x7f3320100e70 2024-05-08T22:04:59.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.709+0000 7f331fe00700 1 --2- 172.21.15.131:0/3313520912 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f33080a1120 0x7f33080a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:04:59.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.710+0000 7f331fe00700 1 --2- 172.21.15.131:0/3313520912 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f33080a1120 0x7f33080a35e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f330c009c60 tx=0x7f330c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:04:59.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.713+0000 7f331d600700 1 -- 172.21.15.131:0/3313520912 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3314052d60 con 0x7f3320100e70 2024-05-08T22:04:59.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.966+0000 7f3327800700 1 -- 172.21.15.131:0/3313520912 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33201a15e0 con 0x7f3320100e70 2024-05-08T22:04:59.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.974+0000 7f331d600700 1 -- 172.21.15.131:0/3313520912 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6577 (secure 0 0 0) 0x7f3314096230 con 0x7f3320100e70 2024-05-08T22:04:59.975 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:04:59.975 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2024-05-08T22:04:59.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.976+0000 7f3327800700 1 -- 172.21.15.131:0/3313520912 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f33080a1120 msgr2=0x7f33080a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:59.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.976+0000 7f3327800700 1 --2- 172.21.15.131:0/3313520912 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f33080a1120 0x7f33080a35e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f330c009c60 tx=0x7f330c009260 comp rx=0 tx=0).stop 2024-05-08T22:04:59.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.977+0000 7f3327800700 1 -- 172.21.15.131:0/3313520912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3320100e70 msgr2=0x7f3320198ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:04:59.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.977+0000 7f3327800700 1 --2- 172.21.15.131:0/3313520912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3320100e70 0x7f3320198ef0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f331400e9a0 tx=0x7f331400ecb0 comp rx=0 tx=0).stop 2024-05-08T22:04:59.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.977+0000 7f3327800700 1 -- 172.21.15.131:0/3313520912 shutdown_connections 2024-05-08T22:04:59.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.977+0000 7f3327800700 1 --2- 172.21.15.131:0/3313520912 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f33080a1120 0x7f33080a35e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:59.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.977+0000 7f3327800700 1 --2- 172.21.15.131:0/3313520912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3320102980 0x7f33201a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:59.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.977+0000 7f3327800700 1 --2- 172.21.15.131:0/3313520912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3320100e70 0x7f3320198ef0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:59.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.977+0000 7f3327800700 1 --2- 172.21.15.131:0/3313520912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3320101fc0 0x7f3320199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:04:59.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.977+0000 7f3327800700 1 -- 172.21.15.131:0/3313520912 >> 172.21.15.131:0/3313520912 conn(0x7f33200fc3e0 msgr2=0x7f33200731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:04:59.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.977+0000 7f3327800700 1 -- 172.21.15.131:0/3313520912 shutdown_connections 2024-05-08T22:04:59.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:04:59.977+0000 7f3327800700 1 -- 172.21.15.131:0/3313520912 wait complete. 2024-05-08T22:04:59.979 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 9 2024-05-08T22:05:00.177 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:59 smithi157 ceph-mon[77746]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:05:00.177 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:00.177 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:00.177 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:00.178 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:05:00.178 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:05:00.178 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:04:59 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:05:00.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:59 smithi132 ceph-mon[78180]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:05:00.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:00.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:00.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:00.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:05:00.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:05:00.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:04:59 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:05:00.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:59 smithi131 ceph-mon[37260]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:05:00.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:00.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:00.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:00.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:05:00.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:05:00.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:04:59 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:05:00.776 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 08 22:05:00 smithi157 systemd[1]: Starting Ceph mds.i for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:05:01.276 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 08 22:05:00 smithi157 ceph-mds[115614]: -- 172.21.15.157:0/727903371 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f6470281a0 con 0x55f646ffe800 2024-05-08T22:05:01.276 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 08 22:05:00 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-i[115591]: starting mds.i at 2024-05-08T22:05:01.276 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 08 22:05:00 smithi157 bash[115576]: ab16faefb86a42c5e58eec81fdd5c3b4ccb9cedd6093528b78315c3db31b7cb6 2024-05-08T22:05:01.276 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 08 22:05:00 smithi157 systemd[1]: Started Ceph mds.i for 06f1a696-0d86-11ef-bc97-c7b262605968. 2024-05-08T22:05:01.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:00 smithi157 ceph-mon[77746]: Deploying daemon mds.i on smithi157 2024-05-08T22:05:01.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:00 smithi157 ceph-mon[77746]: mds.? [v2:172.21.15.132:6836/2685833178,v1:172.21.15.132:6837/2685833178] up:boot 2024-05-08T22:05:01.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:00 smithi157 ceph-mon[77746]: fsmap 8 up:standby 2024-05-08T22:05:01.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:00 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T22:05:01.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:00 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3313520912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:05:01.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:00 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:01.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:00 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:01.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:00 smithi132 ceph-mon[78180]: Deploying daemon mds.i on smithi157 2024-05-08T22:05:01.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:00 smithi132 ceph-mon[78180]: mds.? [v2:172.21.15.132:6836/2685833178,v1:172.21.15.132:6837/2685833178] up:boot 2024-05-08T22:05:01.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:00 smithi132 ceph-mon[78180]: fsmap 8 up:standby 2024-05-08T22:05:01.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:00 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T22:05:01.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:00 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3313520912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:05:01.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:00 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:01.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:00 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:01.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:00 smithi131 ceph-mon[37260]: Deploying daemon mds.i on smithi157 2024-05-08T22:05:01.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:00 smithi131 ceph-mon[37260]: mds.? [v2:172.21.15.132:6836/2685833178,v1:172.21.15.132:6837/2685833178] up:boot 2024-05-08T22:05:01.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:00 smithi131 ceph-mon[37260]: fsmap 8 up:standby 2024-05-08T22:05:01.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:00 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T22:05:01.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:00 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3313520912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:05:01.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:00 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:01.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:00 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:01.998 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:05:02.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.200+0000 7fddbd800700 1 -- 172.21.15.131:0/2520057257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddb8100e80 msgr2=0x7fddb81012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:02.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.200+0000 7fddbd800700 1 --2- 172.21.15.131:0/2520057257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddb8100e80 0x7fddb81012a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdda8009990 tx=0x7fdda8009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:02.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.200+0000 7fddbd800700 1 -- 172.21.15.131:0/2520057257 shutdown_connections 2024-05-08T22:05:02.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.200+0000 7fddbd800700 1 --2- 172.21.15.131:0/2520057257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fddb8102990 0x7fddb81092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:02.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.200+0000 7fddbd800700 1 --2- 172.21.15.131:0/2520057257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddb8100e80 0x7fddb81012a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:02.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.200+0000 7fddbd800700 1 --2- 172.21.15.131:0/2520057257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddb8101fd0 0x7fddb8102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:02.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.200+0000 7fddbd800700 1 -- 172.21.15.131:0/2520057257 >> 172.21.15.131:0/2520057257 conn(0x7fddb80fc410 msgr2=0x7fddb80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:02.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.200+0000 7fddbd800700 1 -- 172.21.15.131:0/2520057257 shutdown_connections 2024-05-08T22:05:02.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.200+0000 7fddbd800700 1 -- 172.21.15.131:0/2520057257 wait complete. 2024-05-08T22:05:02.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.201+0000 7fddbd800700 1 Processor -- start 2024-05-08T22:05:02.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.202+0000 7fddbd800700 1 -- start start 2024-05-08T22:05:02.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.202+0000 7fddbd800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fddb8100e80 0x7fddb8074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:02.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.202+0000 7fddbd800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddb8101fd0 0x7fddb8073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:02.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.202+0000 7fddbd800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddb8102990 0x7fddb80738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:02.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.202+0000 7fddbd800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb8074070 con 0x7fddb8101fd0 2024-05-08T22:05:02.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.202+0000 7fddbd800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb80741e0 con 0x7fddb8100e80 2024-05-08T22:05:02.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.202+0000 7fddbd800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb81a0eb0 con 0x7fddb8102990 2024-05-08T22:05:02.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.202+0000 7fddb6000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddb8101fd0 0x7fddb8073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:02.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.202+0000 7fddb6000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddb8101fd0 0x7fddb8073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42306/0 (socket says 172.21.15.131:42306) 2024-05-08T22:05:02.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.202+0000 7fddb6000700 1 -- 172.21.15.131:0/2773124708 learned_addr learned my addr 172.21.15.131:0/2773124708 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:02.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.202+0000 7fddb6a00700 1 --2- 172.21.15.131:0/2773124708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fddb8100e80 0x7fddb8074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:02.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.202+0000 7fddb7400700 1 --2- 172.21.15.131:0/2773124708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddb8102990 0x7fddb80738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:02.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.202+0000 7fddb6000700 1 -- 172.21.15.131:0/2773124708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddb8102990 msgr2=0x7fddb80738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:02.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.203+0000 7fddb6000700 1 --2- 172.21.15.131:0/2773124708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddb8102990 0x7fddb80738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:02.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.203+0000 7fddb6000700 1 -- 172.21.15.131:0/2773124708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fddb8100e80 msgr2=0x7fddb8074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:02.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.203+0000 7fddb6000700 1 --2- 172.21.15.131:0/2773124708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fddb8100e80 0x7fddb8074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:02.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.203+0000 7fddb6000700 1 -- 172.21.15.131:0/2773124708 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda80095f0 con 0x7fddb8101fd0 2024-05-08T22:05:02.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.203+0000 7fddb6000700 1 --2- 172.21.15.131:0/2773124708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddb8101fd0 0x7fddb8073390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdda400e9c0 tx=0x7fdda400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:02.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.203+0000 7fddaf400700 1 -- 172.21.15.131:0/2773124708 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda40097b0 con 0x7fddb8101fd0 2024-05-08T22:05:02.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.203+0000 7fddaf400700 1 -- 172.21.15.131:0/2773124708 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdda400ee60 con 0x7fddb8101fd0 2024-05-08T22:05:02.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.203+0000 7fddbd800700 1 -- 172.21.15.131:0/2773124708 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb81a1140 con 0x7fddb8101fd0 2024-05-08T22:05:02.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.203+0000 7fddaf400700 1 -- 172.21.15.131:0/2773124708 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda40054a0 con 0x7fddb8101fd0 2024-05-08T22:05:02.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.203+0000 7fddbd800700 1 -- 172.21.15.131:0/2773124708 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb81a1650 con 0x7fddb8101fd0 2024-05-08T22:05:02.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.204+0000 7fddbd800700 1 -- 172.21.15.131:0/2773124708 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb81082e0 con 0x7fddb8101fd0 2024-05-08T22:05:02.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.206+0000 7fddaf400700 1 -- 172.21.15.131:0/2773124708 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fdda4004aa0 con 0x7fddb8101fd0 2024-05-08T22:05:02.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.207+0000 7fddaf400700 1 --2- 172.21.15.131:0/2773124708 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdd9c0a11a0 0x7fdd9c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:02.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.207+0000 7fddaf400700 1 -- 172.21.15.131:0/2773124708 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdda4014070 con 0x7fddb8101fd0 2024-05-08T22:05:02.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.207+0000 7fddb6a00700 1 --2- 172.21.15.131:0/2773124708 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdd9c0a11a0 0x7fdd9c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:02.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.208+0000 7fddb6a00700 1 --2- 172.21.15.131:0/2773124708 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdd9c0a11a0 0x7fdd9c0a3660 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdda800b2e0 tx=0x7fdda8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:02.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.212+0000 7fddaf400700 1 -- 172.21.15.131:0/2773124708 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdda40961d0 con 0x7fddb8101fd0 2024-05-08T22:05:02.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:01 smithi157 ceph-mon[77746]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:05:02.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:01 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:02.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:01 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:05:02.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:01 smithi157 ceph-mon[77746]: mds.? [v2:172.21.15.157:6838/3040193644,v1:172.21.15.157:6839/3040193644] up:boot 2024-05-08T22:05:02.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:01 smithi157 ceph-mon[77746]: fsmap 9 up:standby 2024-05-08T22:05:02.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:01 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:05:02.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:01 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T22:05:02.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:01 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:05:02.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:01 smithi157 ceph-mon[77746]: Deploying daemon mds.g on smithi131 2024-05-08T22:05:02.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:01 smithi132 ceph-mon[78180]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:05:02.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:01 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:02.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:01 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:05:02.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:01 smithi132 ceph-mon[78180]: mds.? [v2:172.21.15.157:6838/3040193644,v1:172.21.15.157:6839/3040193644] up:boot 2024-05-08T22:05:02.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:01 smithi132 ceph-mon[78180]: fsmap 9 up:standby 2024-05-08T22:05:02.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:01 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:05:02.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:01 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T22:05:02.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:01 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:05:02.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:01 smithi132 ceph-mon[78180]: Deploying daemon mds.g on smithi131 2024-05-08T22:05:02.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:01 smithi131 ceph-mon[37260]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:05:02.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:01 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:02.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:01 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:05:02.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:01 smithi131 ceph-mon[37260]: mds.? [v2:172.21.15.157:6838/3040193644,v1:172.21.15.157:6839/3040193644] up:boot 2024-05-08T22:05:02.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:01 smithi131 ceph-mon[37260]: fsmap 9 up:standby 2024-05-08T22:05:02.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:01 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:05:02.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:01 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T22:05:02.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:01 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:05:02.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:01 smithi131 ceph-mon[37260]: Deploying daemon mds.g on smithi131 2024-05-08T22:05:02.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.474+0000 7fddbd800700 1 -- 172.21.15.131:0/2773124708 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fddb8067c10 con 0x7fddb8101fd0 2024-05-08T22:05:02.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.475+0000 7fddaf400700 1 -- 172.21.15.131:0/2773124708 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7339 (secure 0 0 0) 0x7fdda40997f0 con 0x7fddb8101fd0 2024-05-08T22:05:02.476 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:02.476 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-05-08T22:05:02.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.478+0000 7fddbd800700 1 -- 172.21.15.131:0/2773124708 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdd9c0a11a0 msgr2=0x7fdd9c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:02.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.478+0000 7fddbd800700 1 --2- 172.21.15.131:0/2773124708 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdd9c0a11a0 0x7fdd9c0a3660 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdda800b2e0 tx=0x7fdda8005fb0 comp rx=0 tx=0).stop 2024-05-08T22:05:02.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.478+0000 7fddbd800700 1 -- 172.21.15.131:0/2773124708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddb8101fd0 msgr2=0x7fddb8073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:02.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.478+0000 7fddbd800700 1 --2- 172.21.15.131:0/2773124708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddb8101fd0 0x7fddb8073390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdda400e9c0 tx=0x7fdda400ecd0 comp rx=0 tx=0).stop 2024-05-08T22:05:02.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.478+0000 7fddbd800700 1 -- 172.21.15.131:0/2773124708 shutdown_connections 2024-05-08T22:05:02.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.478+0000 7fddbd800700 1 --2- 172.21.15.131:0/2773124708 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdd9c0a11a0 0x7fdd9c0a3660 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:02.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.478+0000 7fddbd800700 1 --2- 172.21.15.131:0/2773124708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddb8102990 0x7fddb80738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:02.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.478+0000 7fddbd800700 1 --2- 172.21.15.131:0/2773124708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fddb8100e80 0x7fddb8074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:02.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.478+0000 7fddbd800700 1 --2- 172.21.15.131:0/2773124708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddb8101fd0 0x7fddb8073390 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:02.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.478+0000 7fddbd800700 1 -- 172.21.15.131:0/2773124708 >> 172.21.15.131:0/2773124708 conn(0x7fddb80fc410 msgr2=0x7fddb8105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:02.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.478+0000 7fddbd800700 1 -- 172.21.15.131:0/2773124708 shutdown_connections 2024-05-08T22:05:02.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:02.478+0000 7fddbd800700 1 -- 172.21.15.131:0/2773124708 wait complete. 2024-05-08T22:05:02.480 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 10 2024-05-08T22:05:03.064 INFO:journalctl@ceph.mds.g.smithi131.stdout:May 08 22:05:03 smithi131 systemd[1]: Starting Ceph mds.g for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:05:03.064 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:02 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2773124708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:05:03.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:02 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2773124708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:05:03.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:02 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2773124708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:05:03.891 INFO:journalctl@ceph.mds.g.smithi131.stdout:May 08 22:05:03 smithi131 bash[87235]: 9bb6f1647f586de97be9379e68f813dedb45672bd390d57515904d5969c49fa1 2024-05-08T22:05:03.891 INFO:journalctl@ceph.mds.g.smithi131.stdout:May 08 22:05:03 smithi131 systemd[1]: Started Ceph mds.g for 06f1a696-0d86-11ef-bc97-c7b262605968. 2024-05-08T22:05:03.891 INFO:journalctl@ceph.mds.g.smithi131.stdout:May 08 22:05:03 smithi131 ceph-mds[87274]: -- 172.21.15.131:0/2263028716 <== mon.0 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5628d04da1a0 con 0x5628d04b2800 2024-05-08T22:05:03.891 INFO:journalctl@ceph.mds.g.smithi131.stdout:May 08 22:05:03 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-g[87250]: starting mds.g at 2024-05-08T22:05:04.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:03 smithi157 ceph-mon[77746]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:05:04.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:03 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:04.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:03 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:04.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:03 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:04.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:03 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:05:04.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:03 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:05:04.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:03 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:05:04.336 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:03 smithi131 ceph-mon[37260]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:05:04.337 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:03 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:04.337 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:03 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:04.337 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:03 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:04.337 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:03 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:05:04.337 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:03 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:05:04.337 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:03 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:05:04.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:03 smithi132 ceph-mon[78180]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:05:04.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:03 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:04.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:03 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:04.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:03 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:04.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:03 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:05:04.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:03 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:05:04.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:03 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:05:04.499 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:05:04.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.704+0000 7f8fafe00700 1 -- 172.21.15.131:0/317182647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fb0100e70 msgr2=0x7f8fb0101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:04.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.704+0000 7f8fafe00700 1 --2- 172.21.15.131:0/317182647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fb0100e70 0x7f8fb0101290 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009990 tx=0x7f8fa0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:04.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.704+0000 7f8fafe00700 1 -- 172.21.15.131:0/317182647 shutdown_connections 2024-05-08T22:05:04.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.704+0000 7f8fafe00700 1 --2- 172.21.15.131:0/317182647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb0102980 0x7f8fb01092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:04.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.704+0000 7f8fafe00700 1 --2- 172.21.15.131:0/317182647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fb0100e70 0x7f8fb0101290 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:04.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.704+0000 7f8fafe00700 1 --2- 172.21.15.131:0/317182647 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fb0101fc0 0x7f8fb0102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:04.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.704+0000 7f8fafe00700 1 -- 172.21.15.131:0/317182647 >> 172.21.15.131:0/317182647 conn(0x7f8fb00fc3e0 msgr2=0x7f8fb00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:04.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.705+0000 7f8fafe00700 1 -- 172.21.15.131:0/317182647 shutdown_connections 2024-05-08T22:05:04.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.705+0000 7f8fafe00700 1 -- 172.21.15.131:0/317182647 wait complete. 2024-05-08T22:05:04.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.706+0000 7f8fafe00700 1 Processor -- start 2024-05-08T22:05:04.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.706+0000 7f8fafe00700 1 -- start start 2024-05-08T22:05:04.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.706+0000 7f8fafe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fb0100e70 0x7f8fb0198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:04.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.706+0000 7f8fafe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb0101fc0 0x7f8fb0199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:04.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.706+0000 7f8fafe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fb0102980 0x7f8fb01a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:04.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.706+0000 7f8fafe00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb01a0af0 con 0x7f8fb0100e70 2024-05-08T22:05:04.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.706+0000 7f8fafe00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb01a0c30 con 0x7f8fb0102980 2024-05-08T22:05:04.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.706+0000 7f8fafe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb01a0dc0 con 0x7f8fb0101fc0 2024-05-08T22:05:04.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.706+0000 7f8faea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fb0100e70 0x7f8fb0198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:04.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.707+0000 7f8faea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fb0100e70 0x7f8fb0198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53192/0 (socket says 172.21.15.131:53192) 2024-05-08T22:05:04.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.707+0000 7f8faea00700 1 -- 172.21.15.131:0/1390445322 learned_addr learned my addr 172.21.15.131:0/1390445322 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:04.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.707+0000 7f8fae000700 1 --2- 172.21.15.131:0/1390445322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb0101fc0 0x7f8fb0199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:04.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.707+0000 7f8faf400700 1 --2- 172.21.15.131:0/1390445322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fb0102980 0x7f8fb01a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:04.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.707+0000 7f8faea00700 1 -- 172.21.15.131:0/1390445322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb0101fc0 msgr2=0x7f8fb0199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:04.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.707+0000 7f8faea00700 1 --2- 172.21.15.131:0/1390445322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb0101fc0 0x7f8fb0199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:04.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.707+0000 7f8faea00700 1 -- 172.21.15.131:0/1390445322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fb0102980 msgr2=0x7f8fb01a0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:04.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.707+0000 7f8faea00700 1 --2- 172.21.15.131:0/1390445322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fb0102980 0x7f8fb01a0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:04.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.707+0000 7f8faea00700 1 -- 172.21.15.131:0/1390445322 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa00095f0 con 0x7f8fb0100e70 2024-05-08T22:05:04.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.707+0000 7f8faf400700 1 --2- 172.21.15.131:0/1390445322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fb0102980 0x7f8fb01a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:05:04.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.707+0000 7f8faea00700 1 --2- 172.21.15.131:0/1390445322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fb0100e70 0x7f8fb0198fc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0000c00 tx=0x7f8fa00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:04.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.707+0000 7f8fa7400700 1 -- 172.21.15.131:0/1390445322 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa001d070 con 0x7f8fb0100e70 2024-05-08T22:05:04.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.708+0000 7f8fa7400700 1 -- 172.21.15.131:0/1390445322 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fa0021410 con 0x7f8fb0100e70 2024-05-08T22:05:04.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.708+0000 7f8fa7400700 1 -- 172.21.15.131:0/1390445322 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0017650 con 0x7f8fb0100e70 2024-05-08T22:05:04.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.708+0000 7f8fafe00700 1 -- 172.21.15.131:0/1390445322 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb01a1040 con 0x7f8fb0100e70 2024-05-08T22:05:04.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.708+0000 7f8fafe00700 1 -- 172.21.15.131:0/1390445322 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb01a1460 con 0x7f8fb0100e70 2024-05-08T22:05:04.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.709+0000 7f8fafe00700 1 -- 172.21.15.131:0/1390445322 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb0067ba0 con 0x7f8fb0100e70 2024-05-08T22:05:04.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.711+0000 7f8fa7400700 1 -- 172.21.15.131:0/1390445322 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8fa0021580 con 0x7f8fb0100e70 2024-05-08T22:05:04.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.712+0000 7f8fa7400700 1 --2- 172.21.15.131:0/1390445322 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8f940a11a0 0x7f8f940a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:04.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.712+0000 7f8fa7400700 1 -- 172.21.15.131:0/1390445322 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8fa00cd8f0 con 0x7f8fb0100e70 2024-05-08T22:05:04.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.712+0000 7f8fae000700 1 --2- 172.21.15.131:0/1390445322 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8f940a11a0 0x7f8f940a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:04.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.713+0000 7f8fae000700 1 --2- 172.21.15.131:0/1390445322 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8f940a11a0 0x7f8f940a3660 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c009bf0 tx=0x7f8f9c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:04.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.716+0000 7f8fa7400700 1 -- 172.21.15.131:0/1390445322 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8fa0097510 con 0x7f8fb0100e70 2024-05-08T22:05:04.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.991+0000 7f8fafe00700 1 -- 172.21.15.131:0/1390445322 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fb01a1740 con 0x7f8fb0100e70 2024-05-08T22:05:04.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.992+0000 7f8fa7400700 1 -- 172.21.15.131:0/1390445322 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8100 (secure 0 0 0) 0x7f8fa009ab30 con 0x7f8fb0100e70 2024-05-08T22:05:04.994 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:04.994 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24550,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[]} 2024-05-08T22:05:04.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.995+0000 7f8fafe00700 1 -- 172.21.15.131:0/1390445322 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8f940a11a0 msgr2=0x7f8f940a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:04.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.995+0000 7f8fafe00700 1 --2- 172.21.15.131:0/1390445322 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8f940a11a0 0x7f8f940a3660 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c009bf0 tx=0x7f8f9c009260 comp rx=0 tx=0).stop 2024-05-08T22:05:04.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.995+0000 7f8fafe00700 1 -- 172.21.15.131:0/1390445322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fb0100e70 msgr2=0x7f8fb0198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:04.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.995+0000 7f8fafe00700 1 --2- 172.21.15.131:0/1390445322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fb0100e70 0x7f8fb0198fc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0000c00 tx=0x7f8fa00057f0 comp rx=0 tx=0).stop 2024-05-08T22:05:04.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.995+0000 7f8fafe00700 1 -- 172.21.15.131:0/1390445322 shutdown_connections 2024-05-08T22:05:04.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.995+0000 7f8fafe00700 1 --2- 172.21.15.131:0/1390445322 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8f940a11a0 0x7f8f940a3660 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:04.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.995+0000 7f8fafe00700 1 --2- 172.21.15.131:0/1390445322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fb0102980 0x7f8fb01a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:04.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.995+0000 7f8fafe00700 1 --2- 172.21.15.131:0/1390445322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fb0100e70 0x7f8fb0198fc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:04.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.995+0000 7f8fafe00700 1 --2- 172.21.15.131:0/1390445322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb0101fc0 0x7f8fb0199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:04.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.995+0000 7f8fafe00700 1 -- 172.21.15.131:0/1390445322 >> 172.21.15.131:0/1390445322 conn(0x7f8fb00fc3e0 msgr2=0x7f8fb00731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:04.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.996+0000 7f8fafe00700 1 -- 172.21.15.131:0/1390445322 shutdown_connections 2024-05-08T22:05:04.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:04.996+0000 7f8fafe00700 1 -- 172.21.15.131:0/1390445322 wait complete. 2024-05-08T22:05:04.998 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 11 2024-05-08T22:05:05.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:05 smithi157 ceph-mon[77746]: Deploying daemon mds.a on smithi131 2024-05-08T22:05:05.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:05 smithi157 ceph-mon[77746]: mds.? [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] up:boot 2024-05-08T22:05:05.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:05 smithi157 ceph-mon[77746]: fsmap 10 up:standby 2024-05-08T22:05:05.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:05 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T22:05:05.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:05 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1390445322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:05:05.278 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:05 smithi131 ceph-mon[37260]: Deploying daemon mds.a on smithi131 2024-05-08T22:05:05.278 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:05 smithi131 ceph-mon[37260]: mds.? [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] up:boot 2024-05-08T22:05:05.278 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:05 smithi131 ceph-mon[37260]: fsmap 10 up:standby 2024-05-08T22:05:05.278 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:05 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T22:05:05.279 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:05 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1390445322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:05:05.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:05 smithi132 ceph-mon[78180]: Deploying daemon mds.a on smithi131 2024-05-08T22:05:05.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:05 smithi132 ceph-mon[78180]: mds.? [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] up:boot 2024-05-08T22:05:05.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:05 smithi132 ceph-mon[78180]: fsmap 10 up:standby 2024-05-08T22:05:05.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:05 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T22:05:05.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:05 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1390445322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:05:05.532 INFO:journalctl@ceph.mds.a.smithi131.stdout:May 08 22:05:05 smithi131 systemd[1]: Starting Ceph mds.a for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:05:06.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:06 smithi157 ceph-mon[77746]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:05:06.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:06 smithi132 ceph-mon[78180]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:05:06.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:06 smithi131 ceph-mon[37260]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:05:06.391 INFO:journalctl@ceph.mds.a.smithi131.stdout:May 08 22:05:05 smithi131 bash[87627]: 0271925cd6a715d95ef3a62a8cb99266be1982d9f12b65641813b6506b1bcee3 2024-05-08T22:05:06.391 INFO:journalctl@ceph.mds.a.smithi131.stdout:May 08 22:05:06 smithi131 systemd[1]: Started Ceph mds.a for 06f1a696-0d86-11ef-bc97-c7b262605968. 2024-05-08T22:05:06.391 INFO:journalctl@ceph.mds.a.smithi131.stdout:May 08 22:05:06 smithi131 ceph-mds[87665]: -- 172.21.15.131:0/2371613420 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562c510f41a0 con 0x562c510ccc00 2024-05-08T22:05:06.391 INFO:journalctl@ceph.mds.a.smithi131.stdout:May 08 22:05:06 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-a[87641]: starting mds.a at 2024-05-08T22:05:07.029 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:05:07.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.236+0000 7ffbc8400700 1 -- 172.21.15.131:0/4289075422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffbc0101fc0 msgr2=0x7ffbc0102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:07.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.236+0000 7ffbc8400700 1 --2- 172.21.15.131:0/4289075422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffbc0101fc0 0x7ffbc0102440 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ffbac009990 tx=0x7ffbac009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:07.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.236+0000 7ffbc8400700 1 -- 172.21.15.131:0/4289075422 shutdown_connections 2024-05-08T22:05:07.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.236+0000 7ffbc8400700 1 --2- 172.21.15.131:0/4289075422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbc0102980 0x7ffbc01092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:07.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.236+0000 7ffbc8400700 1 --2- 172.21.15.131:0/4289075422 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbc0100e70 0x7ffbc0101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:07.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.236+0000 7ffbc8400700 1 --2- 172.21.15.131:0/4289075422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffbc0101fc0 0x7ffbc0102440 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:07.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.236+0000 7ffbc8400700 1 -- 172.21.15.131:0/4289075422 >> 172.21.15.131:0/4289075422 conn(0x7ffbc00fc3e0 msgr2=0x7ffbc00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:07.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.236+0000 7ffbc8400700 1 -- 172.21.15.131:0/4289075422 shutdown_connections 2024-05-08T22:05:07.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.237+0000 7ffbc8400700 1 -- 172.21.15.131:0/4289075422 wait complete. 2024-05-08T22:05:07.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.237+0000 7ffbc8400700 1 Processor -- start 2024-05-08T22:05:07.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.238+0000 7ffbc8400700 1 -- start start 2024-05-08T22:05:07.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.238+0000 7ffbc8400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffbc0100e70 0x7ffbc0074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:07.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.238+0000 7ffbc8400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbc0101fc0 0x7ffbc0073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:07.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.238+0000 7ffbc8400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbc0102980 0x7ffbc00738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:07.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.238+0000 7ffbc8400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc0074070 con 0x7ffbc0100e70 2024-05-08T22:05:07.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.238+0000 7ffbc8400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc00741e0 con 0x7ffbc0101fc0 2024-05-08T22:05:07.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.238+0000 7ffbc8400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc01a0e60 con 0x7ffbc0102980 2024-05-08T22:05:07.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.238+0000 7ffbc5a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffbc0100e70 0x7ffbc0074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:07.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.238+0000 7ffbc5000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbc0101fc0 0x7ffbc0073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:07.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.238+0000 7ffbc5000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbc0101fc0 0x7ffbc0073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:54004/0 (socket says 172.21.15.131:54004) 2024-05-08T22:05:07.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.238+0000 7ffbc5000700 1 -- 172.21.15.131:0/149084663 learned_addr learned my addr 172.21.15.131:0/149084663 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:07.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.238+0000 7ffbc6400700 1 --2- 172.21.15.131:0/149084663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbc0102980 0x7ffbc00738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:07.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.238+0000 7ffbc5000700 1 -- 172.21.15.131:0/149084663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbc0102980 msgr2=0x7ffbc00738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:07.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.238+0000 7ffbc5000700 1 --2- 172.21.15.131:0/149084663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbc0102980 0x7ffbc00738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:07.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.238+0000 7ffbc5000700 1 -- 172.21.15.131:0/149084663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffbc0100e70 msgr2=0x7ffbc0074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:07.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.238+0000 7ffbc5000700 1 --2- 172.21.15.131:0/149084663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffbc0100e70 0x7ffbc0074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:07.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.239+0000 7ffbc5000700 1 -- 172.21.15.131:0/149084663 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb4009520 con 0x7ffbc0101fc0 2024-05-08T22:05:07.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.239+0000 7ffbc5a00700 1 --2- 172.21.15.131:0/149084663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffbc0100e70 0x7ffbc0074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:07.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.239+0000 7ffbc5000700 1 --2- 172.21.15.131:0/149084663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbc0101fc0 0x7ffbc0073390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffbac005910 tx=0x7ffbac005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:07.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.239+0000 7ffbbe000700 1 -- 172.21.15.131:0/149084663 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac01d070 con 0x7ffbc0101fc0 2024-05-08T22:05:07.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.239+0000 7ffbbe000700 1 -- 172.21.15.131:0/149084663 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbac021410 con 0x7ffbc0101fc0 2024-05-08T22:05:07.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.239+0000 7ffbbe000700 1 -- 172.21.15.131:0/149084663 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac017650 con 0x7ffbc0101fc0 2024-05-08T22:05:07.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.239+0000 7ffbc8400700 1 -- 172.21.15.131:0/149084663 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbac0095f0 con 0x7ffbc0101fc0 2024-05-08T22:05:07.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.239+0000 7ffbc8400700 1 -- 172.21.15.131:0/149084663 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc01a1270 con 0x7ffbc0101fc0 2024-05-08T22:05:07.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.240+0000 7ffbc8400700 1 -- 172.21.15.131:0/149084663 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc01082e0 con 0x7ffbc0101fc0 2024-05-08T22:05:07.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.242+0000 7ffbbe000700 1 -- 172.21.15.131:0/149084663 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ffbac021580 con 0x7ffbc0101fc0 2024-05-08T22:05:07.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.244+0000 7ffbbe000700 1 --2- 172.21.15.131:0/149084663 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ffba80a1120 0x7ffba80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:07.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.244+0000 7ffbbe000700 1 -- 172.21.15.131:0/149084663 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ffbac0cdf80 con 0x7ffbc0101fc0 2024-05-08T22:05:07.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.244+0000 7ffbc5a00700 1 --2- 172.21.15.131:0/149084663 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ffba80a1120 0x7ffba80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:07.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.244+0000 7ffbc5a00700 1 --2- 172.21.15.131:0/149084663 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ffba80a1120 0x7ffba80a35e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4009fd0 tx=0x7ffbb4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:07.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.246+0000 7ffbbe000700 1 -- 172.21.15.131:0/149084663 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ffbac097ba0 con 0x7ffbc0101fc0 2024-05-08T22:05:07.305 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:07 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:07.305 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:07 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:07.305 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:07 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:07.305 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:07 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:05:07.305 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:07 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:05:07.305 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:07 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:05:07.305 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:07 smithi132 ceph-mon[78180]: Deploying daemon mds.b on smithi132 2024-05-08T22:05:07.305 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:07 smithi132 ceph-mon[78180]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:05:07.306 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:07 smithi132 ceph-mon[78180]: mds.? [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] up:boot 2024-05-08T22:05:07.306 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:07 smithi132 ceph-mon[78180]: fsmap 11 up:standby 2024-05-08T22:05:07.306 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:07 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T22:05:07.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:07 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:07.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:07 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:07.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:07 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:07.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:07 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:05:07.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:07 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:05:07.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:07 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:05:07.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:07 smithi131 ceph-mon[37260]: Deploying daemon mds.b on smithi132 2024-05-08T22:05:07.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:07 smithi131 ceph-mon[37260]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:05:07.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:07 smithi131 ceph-mon[37260]: mds.? [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] up:boot 2024-05-08T22:05:07.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:07 smithi131 ceph-mon[37260]: fsmap 11 up:standby 2024-05-08T22:05:07.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:07 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T22:05:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.498+0000 7ffbc8400700 1 -- 172.21.15.131:0/149084663 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffbc0067c10 con 0x7ffbc0101fc0 2024-05-08T22:05:07.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.499+0000 7ffbbe000700 1 -- 172.21.15.131:0/149084663 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8861 (secure 0 0 0) 0x7ffbac09b1c0 con 0x7ffbc0101fc0 2024-05-08T22:05:07.500 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:07.500 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24550,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24562,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[]} 2024-05-08T22:05:07.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.501+0000 7ffbc8400700 1 -- 172.21.15.131:0/149084663 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ffba80a1120 msgr2=0x7ffba80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:07.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.501+0000 7ffbc8400700 1 --2- 172.21.15.131:0/149084663 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ffba80a1120 0x7ffba80a35e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4009fd0 tx=0x7ffbb4009260 comp rx=0 tx=0).stop 2024-05-08T22:05:07.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.501+0000 7ffbc8400700 1 -- 172.21.15.131:0/149084663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbc0101fc0 msgr2=0x7ffbc0073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:07.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.501+0000 7ffbc8400700 1 --2- 172.21.15.131:0/149084663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbc0101fc0 0x7ffbc0073390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffbac005910 tx=0x7ffbac005740 comp rx=0 tx=0).stop 2024-05-08T22:05:07.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.502+0000 7ffbc8400700 1 -- 172.21.15.131:0/149084663 shutdown_connections 2024-05-08T22:05:07.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.502+0000 7ffbc8400700 1 --2- 172.21.15.131:0/149084663 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ffba80a1120 0x7ffba80a35e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:07.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.502+0000 7ffbc8400700 1 --2- 172.21.15.131:0/149084663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbc0102980 0x7ffbc00738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:07.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.502+0000 7ffbc8400700 1 --2- 172.21.15.131:0/149084663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffbc0100e70 0x7ffbc0074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:07.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.502+0000 7ffbc8400700 1 --2- 172.21.15.131:0/149084663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbc0101fc0 0x7ffbc0073390 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:07.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.502+0000 7ffbc8400700 1 -- 172.21.15.131:0/149084663 >> 172.21.15.131:0/149084663 conn(0x7ffbc00fc3e0 msgr2=0x7ffbc0105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:07.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.502+0000 7ffbc8400700 1 -- 172.21.15.131:0/149084663 shutdown_connections 2024-05-08T22:05:07.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:07.502+0000 7ffbc8400700 1 -- 172.21.15.131:0/149084663 wait complete. 2024-05-08T22:05:07.504 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 12 2024-05-08T22:05:07.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:07 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:07.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:07 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:07.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:07 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:07.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:07 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:05:07.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:07 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:05:07.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:07 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:05:07.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:07 smithi157 ceph-mon[77746]: Deploying daemon mds.b on smithi132 2024-05-08T22:05:07.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:07 smithi157 ceph-mon[77746]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:05:07.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:07 smithi157 ceph-mon[77746]: mds.? [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] up:boot 2024-05-08T22:05:07.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:07 smithi157 ceph-mon[77746]: fsmap 11 up:standby 2024-05-08T22:05:07.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:07 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T22:05:07.578 INFO:journalctl@ceph.mds.b.smithi132.stdout:May 08 22:05:07 smithi132 systemd[1]: Starting Ceph mds.b for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:05:08.090 INFO:journalctl@ceph.mds.b.smithi132.stdout:May 08 22:05:07 smithi132 ceph-mds[114846]: -- 172.21.15.132:0/1533572236 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5597299c21a0 con 0x559729998c00 2024-05-08T22:05:08.090 INFO:journalctl@ceph.mds.b.smithi132.stdout:May 08 22:05:07 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-b[114823]: starting mds.b at 2024-05-08T22:05:08.090 INFO:journalctl@ceph.mds.b.smithi132.stdout:May 08 22:05:07 smithi132 bash[114808]: a9f281e5efa24dc88653b0b879ecbc9e8ffed4a04473f9fb518675444db53766 2024-05-08T22:05:08.090 INFO:journalctl@ceph.mds.b.smithi132.stdout:May 08 22:05:07 smithi132 systemd[1]: Started Ceph mds.b for 06f1a696-0d86-11ef-bc97-c7b262605968. 2024-05-08T22:05:08.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:08 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/149084663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:05:08.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:08 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:08.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:08 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:08.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:08 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:08.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:08 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:08.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:08 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:08.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:08 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/149084663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:05:08.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:08 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:08.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:08 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:08.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:08 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:08.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:08 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:08.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:08 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:08.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:08 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/149084663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:05:08.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:08 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:08.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:08 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:08.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:08 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:08.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:08 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:08.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:08 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:09.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:09 smithi131 ceph-mon[37260]: mds.? [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] up:boot 2024-05-08T22:05:09.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:09 smithi131 ceph-mon[37260]: fsmap 12 up:standby 2024-05-08T22:05:09.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:09 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T22:05:09.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:09 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:05:09.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:09 smithi131 ceph-mon[37260]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:05:09.523 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:05:09.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:09 smithi157 ceph-mon[77746]: mds.? [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] up:boot 2024-05-08T22:05:09.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:09 smithi157 ceph-mon[77746]: fsmap 12 up:standby 2024-05-08T22:05:09.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:09 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T22:05:09.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:09 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:05:09.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:09 smithi157 ceph-mon[77746]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:05:09.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:09 smithi132 ceph-mon[78180]: mds.? [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] up:boot 2024-05-08T22:05:09.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:09 smithi132 ceph-mon[78180]: fsmap 12 up:standby 2024-05-08T22:05:09.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:09 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T22:05:09.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:09 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:05:09.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:09 smithi132 ceph-mon[78180]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:05:09.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.731+0000 7f75d3000700 1 -- 172.21.15.131:0/1352008941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75cc102980 msgr2=0x7f75cc1092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:09.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.731+0000 7f75d3000700 1 --2- 172.21.15.131:0/1352008941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75cc102980 0x7f75cc1092a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f75c400b0a0 tx=0x7f75c400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:05:09.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.731+0000 7f75d3000700 1 -- 172.21.15.131:0/1352008941 shutdown_connections 2024-05-08T22:05:09.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.731+0000 7f75d3000700 1 --2- 172.21.15.131:0/1352008941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75cc102980 0x7f75cc1092a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:09.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.731+0000 7f75d3000700 1 --2- 172.21.15.131:0/1352008941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75cc100e70 0x7f75cc101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:09.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.731+0000 7f75d3000700 1 --2- 172.21.15.131:0/1352008941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75cc101fc0 0x7f75cc102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:09.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.731+0000 7f75d3000700 1 -- 172.21.15.131:0/1352008941 >> 172.21.15.131:0/1352008941 conn(0x7f75cc0fc400 msgr2=0x7f75cc0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:09.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.731+0000 7f75d3000700 1 -- 172.21.15.131:0/1352008941 shutdown_connections 2024-05-08T22:05:09.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.731+0000 7f75d3000700 1 -- 172.21.15.131:0/1352008941 wait complete. 2024-05-08T22:05:09.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.733+0000 7f75d3000700 1 Processor -- start 2024-05-08T22:05:09.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.733+0000 7f75d3000700 1 -- start start 2024-05-08T22:05:09.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.733+0000 7f75d3000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75cc100e70 0x7f75cc199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:09.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.733+0000 7f75d3000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75cc101fc0 0x7f75cc199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:09.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.734+0000 7f75d3000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75cc102980 0x7f75cc1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:09.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.734+0000 7f75d3000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc1a0c30 con 0x7f75cc100e70 2024-05-08T22:05:09.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.734+0000 7f75d3000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc1a0d70 con 0x7f75cc102980 2024-05-08T22:05:09.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.734+0000 7f75d3000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc1a0f00 con 0x7f75cc101fc0 2024-05-08T22:05:09.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.734+0000 7f75cbe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75cc100e70 0x7f75cc199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:09.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.734+0000 7f75cbe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75cc100e70 0x7f75cc199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53254/0 (socket says 172.21.15.131:53254) 2024-05-08T22:05:09.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.734+0000 7f75cbe00700 1 -- 172.21.15.131:0/2698354157 learned_addr learned my addr 172.21.15.131:0/2698354157 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:09.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.734+0000 7f75cb400700 1 --2- 172.21.15.131:0/2698354157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75cc101fc0 0x7f75cc199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:09.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.734+0000 7f75d1000700 1 --2- 172.21.15.131:0/2698354157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75cc102980 0x7f75cc1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:09.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.734+0000 7f75d1000700 1 -- 172.21.15.131:0/2698354157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75cc101fc0 msgr2=0x7f75cc199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:09.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.734+0000 7f75d1000700 1 --2- 172.21.15.131:0/2698354157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75cc101fc0 0x7f75cc199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:09.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.734+0000 7f75d1000700 1 -- 172.21.15.131:0/2698354157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75cc100e70 msgr2=0x7f75cc199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:09.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.734+0000 7f75d1000700 1 --2- 172.21.15.131:0/2698354157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75cc100e70 0x7f75cc199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:09.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.734+0000 7f75d1000700 1 -- 172.21.15.131:0/2698354157 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75bc009520 con 0x7f75cc102980 2024-05-08T22:05:09.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.734+0000 7f75cbe00700 1 --2- 172.21.15.131:0/2698354157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75cc100e70 0x7f75cc199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:09.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.735+0000 7f75d1000700 1 --2- 172.21.15.131:0/2698354157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75cc102980 0x7f75cc1a0660 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f75c4007b20 tx=0x7f75c4007f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:09.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.735+0000 7f75c8c00700 1 -- 172.21.15.131:0/2698354157 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c400e040 con 0x7f75cc102980 2024-05-08T22:05:09.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.735+0000 7f75c8c00700 1 -- 172.21.15.131:0/2698354157 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75c400bde0 con 0x7f75cc102980 2024-05-08T22:05:09.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.735+0000 7f75d3000700 1 -- 172.21.15.131:0/2698354157 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c4009d00 con 0x7f75cc102980 2024-05-08T22:05:09.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.735+0000 7f75c8c00700 1 -- 172.21.15.131:0/2698354157 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c401b450 con 0x7f75cc102980 2024-05-08T22:05:09.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.735+0000 7f75d3000700 1 -- 172.21.15.131:0/2698354157 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75cc1a14c0 con 0x7f75cc102980 2024-05-08T22:05:09.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.736+0000 7f75d3000700 1 -- 172.21.15.131:0/2698354157 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75cc068060 con 0x7f75cc102980 2024-05-08T22:05:09.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.738+0000 7f75c8c00700 1 -- 172.21.15.131:0/2698354157 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f75c401d480 con 0x7f75cc102980 2024-05-08T22:05:09.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.739+0000 7f75c8c00700 1 --2- 172.21.15.131:0/2698354157 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f75b00a11a0 0x7f75b00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:09.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.739+0000 7f75c8c00700 1 -- 172.21.15.131:0/2698354157 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f75c40cd600 con 0x7f75cc102980 2024-05-08T22:05:09.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.740+0000 7f75cbe00700 1 --2- 172.21.15.131:0/2698354157 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f75b00a11a0 0x7f75b00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:09.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.740+0000 7f75cbe00700 1 --2- 172.21.15.131:0/2698354157 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f75b00a11a0 0x7f75b00a3660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009fd0 tx=0x7f75bc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:09.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:09.743+0000 7f75c8c00700 1 -- 172.21.15.131:0/2698354157 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f75c40972a0 con 0x7f75cc102980 2024-05-08T22:05:10.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.039+0000 7f75d3000700 1 -- 172.21.15.131:0/2698354157 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75cc10c6a0 con 0x7f75cc102980 2024-05-08T22:05:10.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.040+0000 7f75c8c00700 1 -- 172.21.15.131:0/2698354157 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9622 (secure 0 0 0) 0x7f75c409a8c0 con 0x7f75cc102980 2024-05-08T22:05:10.041 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:10.041 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24550,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24562,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2024-05-08T22:05:10.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.043+0000 7f75d3000700 1 -- 172.21.15.131:0/2698354157 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f75b00a11a0 msgr2=0x7f75b00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:10.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.043+0000 7f75d3000700 1 --2- 172.21.15.131:0/2698354157 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f75b00a11a0 0x7f75b00a3660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009fd0 tx=0x7f75bc009260 comp rx=0 tx=0).stop 2024-05-08T22:05:10.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.043+0000 7f75d3000700 1 -- 172.21.15.131:0/2698354157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75cc102980 msgr2=0x7f75cc1a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:10.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.043+0000 7f75d3000700 1 --2- 172.21.15.131:0/2698354157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75cc102980 0x7f75cc1a0660 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f75c4007b20 tx=0x7f75c4007f20 comp rx=0 tx=0).stop 2024-05-08T22:05:10.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.043+0000 7f75d3000700 1 -- 172.21.15.131:0/2698354157 shutdown_connections 2024-05-08T22:05:10.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.043+0000 7f75d3000700 1 --2- 172.21.15.131:0/2698354157 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f75b00a11a0 0x7f75b00a3660 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.043+0000 7f75d3000700 1 --2- 172.21.15.131:0/2698354157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75cc102980 0x7f75cc1a0660 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.043+0000 7f75d3000700 1 --2- 172.21.15.131:0/2698354157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75cc100e70 0x7f75cc199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.043+0000 7f75d3000700 1 --2- 172.21.15.131:0/2698354157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75cc101fc0 0x7f75cc199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.043+0000 7f75d3000700 1 -- 172.21.15.131:0/2698354157 >> 172.21.15.131:0/2698354157 conn(0x7f75cc0fc400 msgr2=0x7f75cc0731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:10.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.043+0000 7f75d3000700 1 -- 172.21.15.131:0/2698354157 shutdown_connections 2024-05-08T22:05:10.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.043+0000 7f75d3000700 1 -- 172.21.15.131:0/2698354157 wait complete. 2024-05-08T22:05:10.046 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 13 2024-05-08T22:05:10.064 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:05:10.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.270+0000 7f9d4f200700 1 -- 172.21.15.131:0/800117368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d48100e80 msgr2=0x7f9d481012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:10.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.270+0000 7f9d4f200700 1 --2- 172.21.15.131:0/800117368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d48100e80 0x7f9d481012a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009990 tx=0x7f9d38009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.270+0000 7f9d4f200700 1 -- 172.21.15.131:0/800117368 shutdown_connections 2024-05-08T22:05:10.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.270+0000 7f9d4f200700 1 --2- 172.21.15.131:0/800117368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d48102990 0x7f9d481092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.270+0000 7f9d4f200700 1 --2- 172.21.15.131:0/800117368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d48100e80 0x7f9d481012a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.270+0000 7f9d4f200700 1 --2- 172.21.15.131:0/800117368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d48101fd0 0x7f9d48102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.270+0000 7f9d4f200700 1 -- 172.21.15.131:0/800117368 >> 172.21.15.131:0/800117368 conn(0x7f9d480fc410 msgr2=0x7f9d480fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:10.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.270+0000 7f9d4f200700 1 -- 172.21.15.131:0/800117368 shutdown_connections 2024-05-08T22:05:10.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.270+0000 7f9d4f200700 1 -- 172.21.15.131:0/800117368 wait complete. 2024-05-08T22:05:10.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.271+0000 7f9d4f200700 1 Processor -- start 2024-05-08T22:05:10.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.272+0000 7f9d4f200700 1 -- start start 2024-05-08T22:05:10.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.272+0000 7f9d4f200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d48100e80 0x7f9d48198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:10.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.272+0000 7f9d4f200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d48101fd0 0x7f9d48199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:10.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.272+0000 7f9d4f200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d48102990 0x7f9d481a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:10.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.272+0000 7f9d4f200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d481a0b10 con 0x7f9d48100e80 2024-05-08T22:05:10.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.272+0000 7f9d4f200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d481a0c50 con 0x7f9d48101fd0 2024-05-08T22:05:10.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.272+0000 7f9d4f200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d481a0de0 con 0x7f9d48102990 2024-05-08T22:05:10.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.272+0000 7f9d47400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d48101fd0 0x7f9d48199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:10.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.272+0000 7f9d47e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d48100e80 0x7f9d48198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:10.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.272+0000 7f9d47400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d48101fd0 0x7f9d48199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:54038/0 (socket says 172.21.15.131:54038) 2024-05-08T22:05:10.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.272+0000 7f9d47400700 1 -- 172.21.15.131:0/2330940493 learned_addr learned my addr 172.21.15.131:0/2330940493 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:10.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.273+0000 7f9d4d200700 1 --2- 172.21.15.131:0/2330940493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d48102990 0x7f9d481a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:10.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.273+0000 7f9d47400700 1 -- 172.21.15.131:0/2330940493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d48102990 msgr2=0x7f9d481a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:10.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.273+0000 7f9d47400700 1 --2- 172.21.15.131:0/2330940493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d48102990 0x7f9d481a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.273+0000 7f9d47400700 1 -- 172.21.15.131:0/2330940493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d48100e80 msgr2=0x7f9d48198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:10.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.273+0000 7f9d47400700 1 --2- 172.21.15.131:0/2330940493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d48100e80 0x7f9d48198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.273+0000 7f9d47400700 1 -- 172.21.15.131:0/2330940493 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d380095f0 con 0x7f9d48101fd0 2024-05-08T22:05:10.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.273+0000 7f9d47400700 1 --2- 172.21.15.131:0/2330940493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d48101fd0 0x7f9d48199520 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9d3400e9a0 tx=0x7f9d3400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:10.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.273+0000 7f9d44c00700 1 -- 172.21.15.131:0/2330940493 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d34009790 con 0x7f9d48101fd0 2024-05-08T22:05:10.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.273+0000 7f9d44c00700 1 -- 172.21.15.131:0/2330940493 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d34010430 con 0x7f9d48101fd0 2024-05-08T22:05:10.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.273+0000 7f9d44c00700 1 -- 172.21.15.131:0/2330940493 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d34018680 con 0x7f9d48101fd0 2024-05-08T22:05:10.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.273+0000 7f9d4f200700 1 -- 172.21.15.131:0/2330940493 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d481a10c0 con 0x7f9d48101fd0 2024-05-08T22:05:10.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.274+0000 7f9d4f200700 1 -- 172.21.15.131:0/2330940493 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d481a15d0 con 0x7f9d48101fd0 2024-05-08T22:05:10.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.275+0000 7f9d4f200700 1 -- 172.21.15.131:0/2330940493 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d48067c10 con 0x7f9d48101fd0 2024-05-08T22:05:10.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.276+0000 7f9d44c00700 1 -- 172.21.15.131:0/2330940493 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f9d340105a0 con 0x7f9d48101fd0 2024-05-08T22:05:10.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.278+0000 7f9d44c00700 1 --2- 172.21.15.131:0/2330940493 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f9d2c0a11a0 0x7f9d2c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:10.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.278+0000 7f9d44c00700 1 -- 172.21.15.131:0/2330940493 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9d34014070 con 0x7f9d48101fd0 2024-05-08T22:05:10.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.278+0000 7f9d47e00700 1 --2- 172.21.15.131:0/2330940493 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f9d2c0a11a0 0x7f9d2c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:10.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.279+0000 7f9d47e00700 1 --2- 172.21.15.131:0/2330940493 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f9d2c0a11a0 0x7f9d2c0a3660 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800b2e0 tx=0x7f9d38005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:10.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.282+0000 7f9d44c00700 1 -- 172.21.15.131:0/2330940493 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9d34052d60 con 0x7f9d48101fd0 2024-05-08T22:05:10.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:10 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:10.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:10 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:10.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:10 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:10.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:10 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:10.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:10 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:10.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:10 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:10.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:10 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:05:10.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:10 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:05:10.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:10 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:10.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:10 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2698354157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:05:10.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:10 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:05:10.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.503+0000 7f9d4f200700 1 -- 172.21.15.131:0/2330940493 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d4810c6d0 con 0x7f9d48101fd0 2024-05-08T22:05:10.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.504+0000 7f9d44c00700 1 -- 172.21.15.131:0/2330940493 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+18469 (secure 0 0 0) 0x7f9d340961b0 con 0x7f9d48101fd0 2024-05-08T22:05:10.505 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:10.506 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":75,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:04:36.108486+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:05:10.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.508+0000 7f9d4f200700 1 -- 172.21.15.131:0/2330940493 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f9d2c0a11a0 msgr2=0x7f9d2c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:10.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.508+0000 7f9d4f200700 1 --2- 172.21.15.131:0/2330940493 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f9d2c0a11a0 0x7f9d2c0a3660 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800b2e0 tx=0x7f9d38005fd0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.508+0000 7f9d4f200700 1 -- 172.21.15.131:0/2330940493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d48101fd0 msgr2=0x7f9d48199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:10.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.508+0000 7f9d4f200700 1 --2- 172.21.15.131:0/2330940493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d48101fd0 0x7f9d48199520 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9d3400e9a0 tx=0x7f9d3400ecb0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.508+0000 7f9d4f200700 1 -- 172.21.15.131:0/2330940493 shutdown_connections 2024-05-08T22:05:10.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.508+0000 7f9d4f200700 1 --2- 172.21.15.131:0/2330940493 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f9d2c0a11a0 0x7f9d2c0a3660 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.508+0000 7f9d4f200700 1 --2- 172.21.15.131:0/2330940493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d48102990 0x7f9d481a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.508+0000 7f9d4f200700 1 --2- 172.21.15.131:0/2330940493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d48100e80 0x7f9d48198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.508+0000 7f9d4f200700 1 --2- 172.21.15.131:0/2330940493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d48101fd0 0x7f9d48199520 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.508+0000 7f9d4f200700 1 -- 172.21.15.131:0/2330940493 >> 172.21.15.131:0/2330940493 conn(0x7f9d480fc410 msgr2=0x7f9d480731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:10.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.508+0000 7f9d4f200700 1 -- 172.21.15.131:0/2330940493 shutdown_connections 2024-05-08T22:05:10.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.508+0000 7f9d4f200700 1 -- 172.21.15.131:0/2330940493 wait complete. 2024-05-08T22:05:10.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:10 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:10.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:10 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:10.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:10 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:10.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:10 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:10.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:10 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:10.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:10 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:10.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:10 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:05:10.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:10 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:05:10.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:10 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:10.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:10 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2698354157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:05:10.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:10 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:05:10.535 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:01:41.271078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:05:10.536 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:05:10.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:10 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:10.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:10 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:10.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:10 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:10.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:10 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:10.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:10 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:10.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:10 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:10.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:10 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:05:10.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:10 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:05:10.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:10 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:10.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:10 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2698354157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:05:10.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:10 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:05:10.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.745+0000 7fa787600700 1 -- 172.21.15.131:0/2148864527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa780102ab0 msgr2=0x7fa7801093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:10.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.745+0000 7fa787600700 1 --2- 172.21.15.131:0/2148864527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa780102ab0 0x7fa7801093d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa77800b0a0 tx=0x7fa77800b3b0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.745+0000 7fa787600700 1 -- 172.21.15.131:0/2148864527 shutdown_connections 2024-05-08T22:05:10.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.746+0000 7fa787600700 1 --2- 172.21.15.131:0/2148864527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa780102ab0 0x7fa7801093d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.746+0000 7fa787600700 1 --2- 172.21.15.131:0/2148864527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa780100f20 0x7fa780101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.746+0000 7fa787600700 1 --2- 172.21.15.131:0/2148864527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7801020f0 0x7fa780102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.746+0000 7fa787600700 1 -- 172.21.15.131:0/2148864527 >> 172.21.15.131:0/2148864527 conn(0x7fa7800fc500 msgr2=0x7fa7800fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:10.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.746+0000 7fa787600700 1 -- 172.21.15.131:0/2148864527 shutdown_connections 2024-05-08T22:05:10.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.746+0000 7fa787600700 1 -- 172.21.15.131:0/2148864527 wait complete. 2024-05-08T22:05:10.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.747+0000 7fa787600700 1 Processor -- start 2024-05-08T22:05:10.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.747+0000 7fa787600700 1 -- start start 2024-05-08T22:05:10.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.747+0000 7fa787600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa780100f20 0x7fa780106cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:10.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.747+0000 7fa787600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7801020f0 0x7fa780107200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:10.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.747+0000 7fa787600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa780102ab0 0x7fa780103850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:10.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.747+0000 7fa787600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa780103ff0 con 0x7fa7801020f0 2024-05-08T22:05:10.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.747+0000 7fa787600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa780104160 con 0x7fa780100f20 2024-05-08T22:05:10.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.747+0000 7fa787600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa780104320 con 0x7fa780102ab0 2024-05-08T22:05:10.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.747+0000 7fa77fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7801020f0 0x7fa780107200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:10.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.748+0000 7fa784c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa780100f20 0x7fa780106cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:10.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.748+0000 7fa77fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7801020f0 0x7fa780107200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53298/0 (socket says 172.21.15.131:53298) 2024-05-08T22:05:10.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.748+0000 7fa77fe00700 1 -- 172.21.15.131:0/1548892891 learned_addr learned my addr 172.21.15.131:0/1548892891 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:10.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.748+0000 7fa785600700 1 --2- 172.21.15.131:0/1548892891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa780102ab0 0x7fa780103850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:10.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.748+0000 7fa784c00700 1 -- 172.21.15.131:0/1548892891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa780102ab0 msgr2=0x7fa780103850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:10.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.748+0000 7fa784c00700 1 --2- 172.21.15.131:0/1548892891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa780102ab0 0x7fa780103850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.748+0000 7fa784c00700 1 -- 172.21.15.131:0/1548892891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7801020f0 msgr2=0x7fa780107200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:10.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.748+0000 7fa784c00700 1 --2- 172.21.15.131:0/1548892891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7801020f0 0x7fa780107200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.748+0000 7fa784c00700 1 -- 172.21.15.131:0/1548892891 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa778009d00 con 0x7fa780100f20 2024-05-08T22:05:10.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.748+0000 7fa784c00700 1 --2- 172.21.15.131:0/1548892891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa780100f20 0x7fa780106cc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa77400e9c0 tx=0x7fa77400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:10.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.748+0000 7fa77d600700 1 -- 172.21.15.131:0/1548892891 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa774004220 con 0x7fa780100f20 2024-05-08T22:05:10.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.748+0000 7fa77d600700 1 -- 172.21.15.131:0/1548892891 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa774004380 con 0x7fa780100f20 2024-05-08T22:05:10.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.748+0000 7fa787600700 1 -- 172.21.15.131:0/1548892891 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa780198d00 con 0x7fa780100f20 2024-05-08T22:05:10.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.749+0000 7fa77d600700 1 -- 172.21.15.131:0/1548892891 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa774003be0 con 0x7fa780100f20 2024-05-08T22:05:10.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.749+0000 7fa787600700 1 -- 172.21.15.131:0/1548892891 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7801991e0 con 0x7fa780100f20 2024-05-08T22:05:10.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.750+0000 7fa787600700 1 -- 172.21.15.131:0/1548892891 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa780067cf0 con 0x7fa780100f20 2024-05-08T22:05:10.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.751+0000 7fa77d600700 1 -- 172.21.15.131:0/1548892891 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa774003d40 con 0x7fa780100f20 2024-05-08T22:05:10.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.753+0000 7fa77d600700 1 --2- 172.21.15.131:0/1548892891 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa7680a1120 0x7fa7680a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:10.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.753+0000 7fa77d600700 1 -- 172.21.15.131:0/1548892891 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa774014070 con 0x7fa780100f20 2024-05-08T22:05:10.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.753+0000 7fa77fe00700 1 --2- 172.21.15.131:0/1548892891 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa7680a1120 0x7fa7680a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:10.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.754+0000 7fa77fe00700 1 --2- 172.21.15.131:0/1548892891 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa7680a1120 0x7fa7680a35e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00bd30 tx=0x7fa76c00b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:10.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.757+0000 7fa77d600700 1 -- 172.21.15.131:0/1548892891 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa774096320 con 0x7fa780100f20 2024-05-08T22:05:10.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.976+0000 7fa787600700 1 -- 172.21.15.131:0/1548892891 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa780199460 con 0x7fa780100f20 2024-05-08T22:05:10.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.976+0000 7fa77d600700 1 -- 172.21.15.131:0/1548892891 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa774099940 con 0x7fa780100f20 2024-05-08T22:05:10.977 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-08T22:05:10.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.979+0000 7fa787600700 1 -- 172.21.15.131:0/1548892891 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa7680a1120 msgr2=0x7fa7680a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:10.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.979+0000 7fa787600700 1 --2- 172.21.15.131:0/1548892891 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa7680a1120 0x7fa7680a35e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00bd30 tx=0x7fa76c00b480 comp rx=0 tx=0).stop 2024-05-08T22:05:10.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.979+0000 7fa787600700 1 -- 172.21.15.131:0/1548892891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa780100f20 msgr2=0x7fa780106cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:10.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.979+0000 7fa787600700 1 --2- 172.21.15.131:0/1548892891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa780100f20 0x7fa780106cc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa77400e9c0 tx=0x7fa77400ecd0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.979+0000 7fa787600700 1 -- 172.21.15.131:0/1548892891 shutdown_connections 2024-05-08T22:05:10.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.979+0000 7fa787600700 1 --2- 172.21.15.131:0/1548892891 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa7680a1120 0x7fa7680a35e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.979+0000 7fa787600700 1 --2- 172.21.15.131:0/1548892891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa780102ab0 0x7fa780103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.979+0000 7fa787600700 1 --2- 172.21.15.131:0/1548892891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa780100f20 0x7fa780106cc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.979+0000 7fa787600700 1 --2- 172.21.15.131:0/1548892891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7801020f0 0x7fa780107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:10.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.979+0000 7fa787600700 1 -- 172.21.15.131:0/1548892891 >> 172.21.15.131:0/1548892891 conn(0x7fa7800fc500 msgr2=0x7fa7800731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:10.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.979+0000 7fa787600700 1 -- 172.21.15.131:0/1548892891 shutdown_connections 2024-05-08T22:05:10.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:10.979+0000 7fa787600700 1 -- 172.21.15.131:0/1548892891 wait complete. 2024-05-08T22:05:11.000 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T22:05:11.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.204+0000 7f05fc800700 1 -- 172.21.15.131:0/850576892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05f41018e0 msgr2=0x7f05f4103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:11.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.205+0000 7f05fc800700 1 --2- 172.21.15.131:0/850576892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05f41018e0 0x7f05f4103cd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009990 tx=0x7f05e8009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:11.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.205+0000 7f05fc800700 1 -- 172.21.15.131:0/850576892 shutdown_connections 2024-05-08T22:05:11.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.205+0000 7f05fc800700 1 --2- 172.21.15.131:0/850576892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05f4106b40 0x7f05f4108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:11.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.205+0000 7f05fc800700 1 --2- 172.21.15.131:0/850576892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05f41018e0 0x7f05f4103cd0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:11.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.205+0000 7f05fc800700 1 --2- 172.21.15.131:0/850576892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05f4104210 0x7f05f4106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:11.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.205+0000 7f05fc800700 1 -- 172.21.15.131:0/850576892 >> 172.21.15.131:0/850576892 conn(0x7f05f40fb2f0 msgr2=0x7f05f40fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:11.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.205+0000 7f05fc800700 1 -- 172.21.15.131:0/850576892 shutdown_connections 2024-05-08T22:05:11.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.205+0000 7f05fc800700 1 -- 172.21.15.131:0/850576892 wait complete. 2024-05-08T22:05:11.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.206+0000 7f05fc800700 1 Processor -- start 2024-05-08T22:05:11.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.206+0000 7f05fc800700 1 -- start start 2024-05-08T22:05:11.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.206+0000 7f05fc800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05f41018e0 0x7f05f4196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:11.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.206+0000 7f05fc800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05f4104210 0x7f05f4197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:11.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.206+0000 7f05fc800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05f4106b40 0x7f05f419e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:11.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.206+0000 7f05fc800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f419e8f0 con 0x7f05f4106b40 2024-05-08T22:05:11.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.206+0000 7f05fc800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f419ea60 con 0x7f05f41018e0 2024-05-08T22:05:11.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.206+0000 7f05fc800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f419ec20 con 0x7f05f4104210 2024-05-08T22:05:11.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.206+0000 7f05f9400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05f4104210 0x7f05f4197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:11.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.206+0000 7f05f9e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05f41018e0 0x7f05f4196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:11.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.206+0000 7f05f9400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05f4104210 0x7f05f4197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:33874/0 (socket says 172.21.15.131:33874) 2024-05-08T22:05:11.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.206+0000 7f05f9400700 1 -- 172.21.15.131:0/1071009583 learned_addr learned my addr 172.21.15.131:0/1071009583 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:11.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.207+0000 7f05f9400700 1 -- 172.21.15.131:0/1071009583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05f41018e0 msgr2=0x7f05f4196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:11.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.207+0000 7f05f9400700 1 --2- 172.21.15.131:0/1071009583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05f41018e0 0x7f05f4196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:11.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.207+0000 7f05f9400700 1 -- 172.21.15.131:0/1071009583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05f4106b40 msgr2=0x7f05f419e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:11.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.207+0000 7f05f9400700 1 --2- 172.21.15.131:0/1071009583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05f4106b40 0x7f05f419e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:11.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.207+0000 7f05f9400700 1 -- 172.21.15.131:0/1071009583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e80095f0 con 0x7f05f4104210 2024-05-08T22:05:11.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.207+0000 7f05f9400700 1 --2- 172.21.15.131:0/1071009583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05f4104210 0x7f05f4197340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f05e400e9c0 tx=0x7f05e400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:11.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.207+0000 7f05eea00700 1 -- 172.21.15.131:0/1071009583 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e40097b0 con 0x7f05f4104210 2024-05-08T22:05:11.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.207+0000 7f05eea00700 1 -- 172.21.15.131:0/1071009583 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05e4004cf0 con 0x7f05f4104210 2024-05-08T22:05:11.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.207+0000 7f05fc800700 1 -- 172.21.15.131:0/1071009583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f40ff730 con 0x7f05f4104210 2024-05-08T22:05:11.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.208+0000 7f05eea00700 1 -- 172.21.15.131:0/1071009583 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e40055d0 con 0x7f05f4104210 2024-05-08T22:05:11.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.208+0000 7f05fc800700 1 -- 172.21.15.131:0/1071009583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f40ffc40 con 0x7f05f4104210 2024-05-08T22:05:11.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.209+0000 7f05fc800700 1 -- 172.21.15.131:0/1071009583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f40fce70 con 0x7f05f4104210 2024-05-08T22:05:11.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.213+0000 7f05eea00700 1 -- 172.21.15.131:0/1071009583 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f05e4004830 con 0x7f05f4104210 2024-05-08T22:05:11.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.214+0000 7f05eea00700 1 --2- 172.21.15.131:0/1071009583 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f05dc0a11a0 0x7f05dc0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:11.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.214+0000 7f05eea00700 1 -- 172.21.15.131:0/1071009583 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f05e4014070 con 0x7f05f4104210 2024-05-08T22:05:11.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.214+0000 7f05f9e00700 1 --2- 172.21.15.131:0/1071009583 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f05dc0a11a0 0x7f05dc0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:11.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.215+0000 7f05f9e00700 1 --2- 172.21.15.131:0/1071009583 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f05dc0a11a0 0x7f05dc0a3660 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f05e800b680 tx=0x7f05e8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:11.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.218+0000 7f05eea00700 1 -- 172.21.15.131:0/1071009583 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f05e4096460 con 0x7f05f4104210 2024-05-08T22:05:11.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:11 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2330940493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:11.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:11 smithi131 ceph-mon[37260]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:05:11.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:11 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:11.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:11 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1548892891' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:05:11.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.448+0000 7f05fc800700 1 -- 172.21.15.131:0/1071009583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f05f410c7c0 con 0x7f05f4104210 2024-05-08T22:05:11.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.449+0000 7f05eea00700 1 -- 172.21.15.131:0/1071009583 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) v1 ==== 84+0+65 (secure 0 0 0) 0x7f05e4004e60 con 0x7f05f4104210 2024-05-08T22:05:11.450 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:11.450 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2024-05-08T22:05:11.450 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:05:11.450 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2024-05-08T22:05:11.450 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2024-05-08T22:05:11.450 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:05:11.450 INFO:teuthology.orchestra.run.smithi131.stdout:] 2024-05-08T22:05:11.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.452+0000 7f05fc800700 1 -- 172.21.15.131:0/1071009583 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f05dc0a11a0 msgr2=0x7f05dc0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:11.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.452+0000 7f05fc800700 1 --2- 172.21.15.131:0/1071009583 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f05dc0a11a0 0x7f05dc0a3660 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f05e800b680 tx=0x7f05e8005fd0 comp rx=0 tx=0).stop 2024-05-08T22:05:11.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.452+0000 7f05fc800700 1 -- 172.21.15.131:0/1071009583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05f4104210 msgr2=0x7f05f4197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:11.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.452+0000 7f05fc800700 1 --2- 172.21.15.131:0/1071009583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05f4104210 0x7f05f4197340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f05e400e9c0 tx=0x7f05e400ecd0 comp rx=0 tx=0).stop 2024-05-08T22:05:11.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.452+0000 7f05fc800700 1 -- 172.21.15.131:0/1071009583 shutdown_connections 2024-05-08T22:05:11.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.452+0000 7f05fc800700 1 --2- 172.21.15.131:0/1071009583 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f05dc0a11a0 0x7f05dc0a3660 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:11.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.452+0000 7f05fc800700 1 --2- 172.21.15.131:0/1071009583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05f4106b40 0x7f05f419e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:11.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.452+0000 7f05fc800700 1 --2- 172.21.15.131:0/1071009583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05f41018e0 0x7f05f4196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:11.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.452+0000 7f05fc800700 1 --2- 172.21.15.131:0/1071009583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05f4104210 0x7f05f4197340 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:11.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.452+0000 7f05fc800700 1 -- 172.21.15.131:0/1071009583 >> 172.21.15.131:0/1071009583 conn(0x7f05f40fb2f0 msgr2=0x7f05f4103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:11.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.452+0000 7f05fc800700 1 -- 172.21.15.131:0/1071009583 shutdown_connections 2024-05-08T22:05:11.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.452+0000 7f05fc800700 1 -- 172.21.15.131:0/1071009583 wait complete. 2024-05-08T22:05:11.473 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T22:05:11.474 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T22:05:11.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:11 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2330940493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:11.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:11 smithi157 ceph-mon[77746]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:05:11.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:11 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:11.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:11 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1548892891' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:05:11.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:11 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2330940493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:11.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:11 smithi132 ceph-mon[78180]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:05:11.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:11 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:11.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:11 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1548892891' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:05:11.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.690+0000 7f2311800700 1 -- 172.21.15.131:0/796964178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f230c101aa0 msgr2=0x7f230c103e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:11.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.690+0000 7f2311800700 1 --2- 172.21.15.131:0/796964178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f230c101aa0 0x7f230c103e90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009990 tx=0x7f22fc009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:11.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.691+0000 7f2311800700 1 -- 172.21.15.131:0/796964178 shutdown_connections 2024-05-08T22:05:11.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.691+0000 7f2311800700 1 --2- 172.21.15.131:0/796964178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f230c106d00 0x7f230c1090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:11.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.691+0000 7f2311800700 1 --2- 172.21.15.131:0/796964178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f230c101aa0 0x7f230c103e90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:11.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.691+0000 7f2311800700 1 --2- 172.21.15.131:0/796964178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f230c1043d0 0x7f230c1067c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:11.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.691+0000 7f2311800700 1 -- 172.21.15.131:0/796964178 >> 172.21.15.131:0/796964178 conn(0x7f230c0fb3f0 msgr2=0x7f230c0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:11.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.691+0000 7f2311800700 1 -- 172.21.15.131:0/796964178 shutdown_connections 2024-05-08T22:05:11.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.691+0000 7f2311800700 1 -- 172.21.15.131:0/796964178 wait complete. 2024-05-08T22:05:11.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.692+0000 7f2311800700 1 Processor -- start 2024-05-08T22:05:11.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.692+0000 7f2311800700 1 -- start start 2024-05-08T22:05:11.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.692+0000 7f2311800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f230c101aa0 0x7f230c199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:11.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.692+0000 7f2311800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f230c1043d0 0x7f230c199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:11.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.692+0000 7f2311800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f230c106d00 0x7f230c1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:11.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.692+0000 7f2311800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c1a0c30 con 0x7f230c1043d0 2024-05-08T22:05:11.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.692+0000 7f2311800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c1a0d70 con 0x7f230c106d00 2024-05-08T22:05:11.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.692+0000 7f2311800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c1a0f00 con 0x7f230c101aa0 2024-05-08T22:05:11.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.692+0000 7f230a000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f230c1043d0 0x7f230c199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:11.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.692+0000 7f230aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f230c101aa0 0x7f230c199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:11.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.693+0000 7f230aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f230c101aa0 0x7f230c199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:33888/0 (socket says 172.21.15.131:33888) 2024-05-08T22:05:11.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.693+0000 7f230b400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f230c106d00 0x7f230c1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:11.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.693+0000 7f230aa00700 1 -- 172.21.15.131:0/1969490121 learned_addr learned my addr 172.21.15.131:0/1969490121 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:11.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.693+0000 7f230b400700 1 -- 172.21.15.131:0/1969490121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f230c101aa0 msgr2=0x7f230c199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:11.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.693+0000 7f230b400700 1 --2- 172.21.15.131:0/1969490121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f230c101aa0 0x7f230c199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:11.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.693+0000 7f230b400700 1 -- 172.21.15.131:0/1969490121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f230c1043d0 msgr2=0x7f230c199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:11.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.693+0000 7f230b400700 1 --2- 172.21.15.131:0/1969490121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f230c1043d0 0x7f230c199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:11.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.693+0000 7f230b400700 1 -- 172.21.15.131:0/1969490121 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22fc0095f0 con 0x7f230c106d00 2024-05-08T22:05:11.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.693+0000 7f230b400700 1 --2- 172.21.15.131:0/1969490121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f230c106d00 0x7f230c1a0660 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f230400efc0 tx=0x7f230400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:11.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.693+0000 7f2303400700 1 -- 172.21.15.131:0/1969490121 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2304018070 con 0x7f230c106d00 2024-05-08T22:05:11.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.693+0000 7f2311800700 1 -- 172.21.15.131:0/1969490121 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f230c1a1180 con 0x7f230c106d00 2024-05-08T22:05:11.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.694+0000 7f2303400700 1 -- 172.21.15.131:0/1969490121 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f230400f040 con 0x7f230c106d00 2024-05-08T22:05:11.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.694+0000 7f2303400700 1 -- 172.21.15.131:0/1969490121 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230400cc70 con 0x7f230c106d00 2024-05-08T22:05:11.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.694+0000 7f2311800700 1 -- 172.21.15.131:0/1969490121 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f230c1a16e0 con 0x7f230c106d00 2024-05-08T22:05:11.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.695+0000 7f2311800700 1 -- 172.21.15.131:0/1969490121 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f230c108880 con 0x7f230c106d00 2024-05-08T22:05:11.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.697+0000 7f2303400700 1 -- 172.21.15.131:0/1969490121 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2304003a00 con 0x7f230c106d00 2024-05-08T22:05:11.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.698+0000 7f2303400700 1 --2- 172.21.15.131:0/1969490121 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f22f00a11a0 0x7f22f00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:11.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.698+0000 7f2303400700 1 -- 172.21.15.131:0/1969490121 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f23040cbf30 con 0x7f230c106d00 2024-05-08T22:05:11.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.699+0000 7f230aa00700 1 --2- 172.21.15.131:0/1969490121 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f22f00a11a0 0x7f22f00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:11.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.699+0000 7f230aa00700 1 --2- 172.21.15.131:0/1969490121 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f22f00a11a0 0x7f22f00a3660 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00b2e0 tx=0x7f22fc00b750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:11.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.702+0000 7f2303400700 1 -- 172.21.15.131:0/1969490121 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2304095b50 con 0x7f230c106d00 2024-05-08T22:05:11.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:11.959+0000 7f2311800700 1 -- 172.21.15.131:0/1969490121 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f230c108260 con 0x7f230c106d00 2024-05-08T22:05:12.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.527+0000 7f2303400700 1 -- 172.21.15.131:0/1969490121 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2304099170 con 0x7f230c106d00 2024-05-08T22:05:12.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.531+0000 7f2311800700 1 -- 172.21.15.131:0/1969490121 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f22f00a11a0 msgr2=0x7f22f00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:12.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.531+0000 7f2311800700 1 --2- 172.21.15.131:0/1969490121 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f22f00a11a0 0x7f22f00a3660 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00b2e0 tx=0x7f22fc00b750 comp rx=0 tx=0).stop 2024-05-08T22:05:12.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.531+0000 7f2311800700 1 -- 172.21.15.131:0/1969490121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f230c106d00 msgr2=0x7f230c1a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:12.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.531+0000 7f2311800700 1 --2- 172.21.15.131:0/1969490121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f230c106d00 0x7f230c1a0660 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f230400efc0 tx=0x7f230400c3a0 comp rx=0 tx=0).stop 2024-05-08T22:05:12.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.532+0000 7f2311800700 1 -- 172.21.15.131:0/1969490121 shutdown_connections 2024-05-08T22:05:12.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.532+0000 7f2311800700 1 --2- 172.21.15.131:0/1969490121 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f22f00a11a0 0x7f22f00a3660 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:12.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.532+0000 7f2311800700 1 --2- 172.21.15.131:0/1969490121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f230c106d00 0x7f230c1a0660 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:12.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.532+0000 7f2311800700 1 --2- 172.21.15.131:0/1969490121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f230c101aa0 0x7f230c199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:12.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.532+0000 7f2311800700 1 --2- 172.21.15.131:0/1969490121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f230c1043d0 0x7f230c199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:12.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.532+0000 7f2311800700 1 -- 172.21.15.131:0/1969490121 >> 172.21.15.131:0/1969490121 conn(0x7f230c0fb3f0 msgr2=0x7f230c101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:12.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.533+0000 7f2311800700 1 -- 172.21.15.131:0/1969490121 shutdown_connections 2024-05-08T22:05:12.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.533+0000 7f2311800700 1 -- 172.21.15.131:0/1969490121 wait complete. 2024-05-08T22:05:12.535 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2024-05-08T22:05:12.557 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-08T22:05:12.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.757+0000 7fe9b8600700 1 -- 172.21.15.131:0/1855680165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b0102ab0 msgr2=0x7fe9b01093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:12.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.757+0000 7fe9b8600700 1 --2- 172.21.15.131:0/1855680165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b0102ab0 0x7fe9b01093d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800b0a0 tx=0x7fe9a800b3b0 comp rx=0 tx=0).stop 2024-05-08T22:05:12.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.757+0000 7fe9b8600700 1 -- 172.21.15.131:0/1855680165 shutdown_connections 2024-05-08T22:05:12.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.757+0000 7fe9b8600700 1 --2- 172.21.15.131:0/1855680165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b0102ab0 0x7fe9b01093d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:12.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.757+0000 7fe9b8600700 1 --2- 172.21.15.131:0/1855680165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9b0100f20 0x7fe9b0101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:12.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.757+0000 7fe9b8600700 1 --2- 172.21.15.131:0/1855680165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9b01020f0 0x7fe9b0102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:12.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.757+0000 7fe9b8600700 1 -- 172.21.15.131:0/1855680165 >> 172.21.15.131:0/1855680165 conn(0x7fe9b00fc4e0 msgr2=0x7fe9b00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:12.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.757+0000 7fe9b8600700 1 -- 172.21.15.131:0/1855680165 shutdown_connections 2024-05-08T22:05:12.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.757+0000 7fe9b8600700 1 -- 172.21.15.131:0/1855680165 wait complete. 2024-05-08T22:05:12.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.758+0000 7fe9b8600700 1 Processor -- start 2024-05-08T22:05:12.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.758+0000 7fe9b8600700 1 -- start start 2024-05-08T22:05:12.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.759+0000 7fe9b8600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b0100f20 0x7fe9b0074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:12.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.759+0000 7fe9b8600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9b01020f0 0x7fe9b0073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:12.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.759+0000 7fe9b8600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9b0102ab0 0x7fe9b00738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:12.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.759+0000 7fe9b8600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b0074070 con 0x7fe9b0102ab0 2024-05-08T22:05:12.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.759+0000 7fe9b8600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b00741e0 con 0x7fe9b01020f0 2024-05-08T22:05:12.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.759+0000 7fe9b8600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b01a0e40 con 0x7fe9b0100f20 2024-05-08T22:05:12.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.759+0000 7fe9b5c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b0100f20 0x7fe9b0074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:12.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.759+0000 7fe9b5c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b0100f20 0x7fe9b0074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:33908/0 (socket says 172.21.15.131:33908) 2024-05-08T22:05:12.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.759+0000 7fe9b5c00700 1 -- 172.21.15.131:0/1404666656 learned_addr learned my addr 172.21.15.131:0/1404666656 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:12.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.759+0000 7fe9b5200700 1 --2- 172.21.15.131:0/1404666656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9b01020f0 0x7fe9b0073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:12.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.759+0000 7fe9b6600700 1 --2- 172.21.15.131:0/1404666656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9b0102ab0 0x7fe9b00738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:12.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.759+0000 7fe9b5c00700 1 -- 172.21.15.131:0/1404666656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9b01020f0 msgr2=0x7fe9b0073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:12.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.759+0000 7fe9b5c00700 1 --2- 172.21.15.131:0/1404666656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9b01020f0 0x7fe9b0073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:12.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.759+0000 7fe9b5c00700 1 -- 172.21.15.131:0/1404666656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9b0102ab0 msgr2=0x7fe9b00738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:12.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.759+0000 7fe9b5c00700 1 --2- 172.21.15.131:0/1404666656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9b0102ab0 0x7fe9b00738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:12.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.759+0000 7fe9b5c00700 1 -- 172.21.15.131:0/1404666656 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a8009d00 con 0x7fe9b0100f20 2024-05-08T22:05:12.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.760+0000 7fe9b5c00700 1 --2- 172.21.15.131:0/1404666656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b0100f20 0x7fe9b0074d00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400e970 tx=0x7fe9a400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:12.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.760+0000 7fe9ae000700 1 -- 172.21.15.131:0/1404666656 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4009790 con 0x7fe9b0100f20 2024-05-08T22:05:12.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.760+0000 7fe9ae000700 1 -- 172.21.15.131:0/1404666656 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9a400c7e0 con 0x7fe9b0100f20 2024-05-08T22:05:12.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.760+0000 7fe9b8600700 1 -- 172.21.15.131:0/1404666656 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b01a1070 con 0x7fe9b0100f20 2024-05-08T22:05:12.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.760+0000 7fe9ae000700 1 -- 172.21.15.131:0/1404666656 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4003c70 con 0x7fe9b0100f20 2024-05-08T22:05:12.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.760+0000 7fe9b8600700 1 -- 172.21.15.131:0/1404666656 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b01a1580 con 0x7fe9b0100f20 2024-05-08T22:05:12.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.761+0000 7fe9b8600700 1 -- 172.21.15.131:0/1404666656 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b01083c0 con 0x7fe9b0100f20 2024-05-08T22:05:12.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.763+0000 7fe9ae000700 1 -- 172.21.15.131:0/1404666656 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe9a400cc50 con 0x7fe9b0100f20 2024-05-08T22:05:12.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.764+0000 7fe9ae000700 1 --2- 172.21.15.131:0/1404666656 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe9980a1120 0x7fe9980a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:12.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.764+0000 7fe9ae000700 1 -- 172.21.15.131:0/1404666656 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fe9a4014070 con 0x7fe9b0100f20 2024-05-08T22:05:12.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.764+0000 7fe9b5200700 1 --2- 172.21.15.131:0/1404666656 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe9980a1120 0x7fe9980a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:12.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.765+0000 7fe9b5200700 1 --2- 172.21.15.131:0/1404666656 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe9980a1120 0x7fe9980a35e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009c60 tx=0x7fe99c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:12.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.766+0000 7fe9ae000700 1 -- 172.21.15.131:0/1404666656 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe9a4096360 con 0x7fe9b0100f20 2024-05-08T22:05:12.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:12 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:12.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:12 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:12.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:12 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:12.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:12 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:12.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:12 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1071009583' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T22:05:12.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:12 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:12.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:12 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:12.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:12 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:12.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:12 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:12.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:12 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1071009583' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T22:05:12.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:12 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:12.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:12 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:12.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:12 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:12.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:12 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:12.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:12 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1071009583' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T22:05:12.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:12.974+0000 7fe9b8600700 1 -- 172.21.15.131:0/1404666656 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fe9b010c950 con 0x7fe9b0100f20 2024-05-08T22:05:13.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.533+0000 7fe9ae000700 1 -- 172.21.15.131:0/1404666656 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v77) v1 ==== 171+0+0 (secure 0 0 0) 0x7fe9a4099980 con 0x7fe9b0100f20 2024-05-08T22:05:13.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.539+0000 7fe9b8600700 1 -- 172.21.15.131:0/1404666656 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe9980a1120 msgr2=0x7fe9980a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:13.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.539+0000 7fe9b8600700 1 --2- 172.21.15.131:0/1404666656 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe9980a1120 0x7fe9980a35e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009c60 tx=0x7fe99c009260 comp rx=0 tx=0).stop 2024-05-08T22:05:13.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.539+0000 7fe9b8600700 1 -- 172.21.15.131:0/1404666656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b0100f20 msgr2=0x7fe9b0074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:13.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.539+0000 7fe9b8600700 1 --2- 172.21.15.131:0/1404666656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b0100f20 0x7fe9b0074d00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400e970 tx=0x7fe9a400ec80 comp rx=0 tx=0).stop 2024-05-08T22:05:13.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.540+0000 7fe9b8600700 1 -- 172.21.15.131:0/1404666656 shutdown_connections 2024-05-08T22:05:13.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.540+0000 7fe9b8600700 1 --2- 172.21.15.131:0/1404666656 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe9980a1120 0x7fe9980a35e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:13.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.540+0000 7fe9b8600700 1 --2- 172.21.15.131:0/1404666656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9b0102ab0 0x7fe9b00738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:13.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.540+0000 7fe9b8600700 1 --2- 172.21.15.131:0/1404666656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b0100f20 0x7fe9b0074d00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:13.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.540+0000 7fe9b8600700 1 --2- 172.21.15.131:0/1404666656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9b01020f0 0x7fe9b0073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:13.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.540+0000 7fe9b8600700 1 -- 172.21.15.131:0/1404666656 >> 172.21.15.131:0/1404666656 conn(0x7fe9b00fc4e0 msgr2=0x7fe9b0105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:13.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.540+0000 7fe9b8600700 1 -- 172.21.15.131:0/1404666656 shutdown_connections 2024-05-08T22:05:13.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.540+0000 7fe9b8600700 1 -- 172.21.15.131:0/1404666656 wait complete. 2024-05-08T22:05:13.544 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2024-05-08T22:05:13.570 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T22:05:13.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.765+0000 7f05c8000700 1 -- 172.21.15.131:0/2849453130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05c0100f80 msgr2=0x7f05c01013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:13.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.765+0000 7f05c8000700 1 --2- 172.21.15.131:0/2849453130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05c0100f80 0x7f05c01013a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f05b40098f0 tx=0x7f05b4009c00 comp rx=0 tx=0).stop 2024-05-08T22:05:13.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.765+0000 7f05c8000700 1 -- 172.21.15.131:0/2849453130 shutdown_connections 2024-05-08T22:05:13.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.765+0000 7f05c8000700 1 --2- 172.21.15.131:0/2849453130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05c0102b10 0x7f05c01093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:13.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.765+0000 7f05c8000700 1 --2- 172.21.15.131:0/2849453130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05c0100f80 0x7f05c01013a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:13.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.765+0000 7f05c8000700 1 --2- 172.21.15.131:0/2849453130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05c0102170 0x7f05c01025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:13.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.765+0000 7f05c8000700 1 -- 172.21.15.131:0/2849453130 >> 172.21.15.131:0/2849453130 conn(0x7f05c00fc4f0 msgr2=0x7f05c00fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:13.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.765+0000 7f05c8000700 1 -- 172.21.15.131:0/2849453130 shutdown_connections 2024-05-08T22:05:13.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.765+0000 7f05c8000700 1 -- 172.21.15.131:0/2849453130 wait complete. 2024-05-08T22:05:13.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.766+0000 7f05c8000700 1 Processor -- start 2024-05-08T22:05:13.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.767+0000 7f05c8000700 1 -- start start 2024-05-08T22:05:13.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.767+0000 7f05c8000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05c0100f80 0x7f05c0074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:13.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.767+0000 7f05c8000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05c0102170 0x7f05c0073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:13.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.767+0000 7f05c8000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05c0102b10 0x7f05c00738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:13.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.767+0000 7f05c8000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c0074070 con 0x7f05c0102170 2024-05-08T22:05:13.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.767+0000 7f05c8000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c00741e0 con 0x7f05c0102b10 2024-05-08T22:05:13.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.767+0000 7f05c8000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c01a1060 con 0x7f05c0100f80 2024-05-08T22:05:13.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.767+0000 7f05c4c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05c0102170 0x7f05c0073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:13.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.767+0000 7f05c4c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05c0102170 0x7f05c0073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53350/0 (socket says 172.21.15.131:53350) 2024-05-08T22:05:13.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.767+0000 7f05c4c00700 1 -- 172.21.15.131:0/84215876 learned_addr learned my addr 172.21.15.131:0/84215876 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:13.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.767+0000 7f05c5600700 1 --2- 172.21.15.131:0/84215876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05c0100f80 0x7f05c0074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:13.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.767+0000 7f05c6000700 1 --2- 172.21.15.131:0/84215876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05c0102b10 0x7f05c00738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:13.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.768+0000 7f05c4c00700 1 -- 172.21.15.131:0/84215876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05c0100f80 msgr2=0x7f05c0074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:13.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.768+0000 7f05c4c00700 1 --2- 172.21.15.131:0/84215876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05c0100f80 0x7f05c0074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:13.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.768+0000 7f05c4c00700 1 -- 172.21.15.131:0/84215876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05c0102b10 msgr2=0x7f05c00738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:13.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.768+0000 7f05c4c00700 1 --2- 172.21.15.131:0/84215876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05c0102b10 0x7f05c00738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:13.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.768+0000 7f05c4c00700 1 -- 172.21.15.131:0/84215876 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b8009d00 con 0x7f05c0102170 2024-05-08T22:05:13.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.768+0000 7f05c5600700 1 --2- 172.21.15.131:0/84215876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05c0100f80 0x7f05c0074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:13.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.768+0000 7f05c4c00700 1 --2- 172.21.15.131:0/84215876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05c0102170 0x7f05c0073390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00e9c0 tx=0x7f05ac00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:13.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.768+0000 7f05be000700 1 -- 172.21.15.131:0/84215876 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac0097b0 con 0x7f05c0102170 2024-05-08T22:05:13.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.768+0000 7f05be000700 1 -- 172.21.15.131:0/84215876 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05ac00ee60 con 0x7f05c0102170 2024-05-08T22:05:13.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.768+0000 7f05c8000700 1 -- 172.21.15.131:0/84215876 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b4009520 con 0x7f05c0102170 2024-05-08T22:05:13.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.768+0000 7f05be000700 1 -- 172.21.15.131:0/84215876 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac0054a0 con 0x7f05c0102170 2024-05-08T22:05:13.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.768+0000 7f05c8000700 1 -- 172.21.15.131:0/84215876 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c01a1620 con 0x7f05c0102170 2024-05-08T22:05:13.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.769+0000 7f05c8000700 1 -- 172.21.15.131:0/84215876 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05c0108470 con 0x7f05c0102170 2024-05-08T22:05:13.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.771+0000 7f05be000700 1 -- 172.21.15.131:0/84215876 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f05ac004aa0 con 0x7f05c0102170 2024-05-08T22:05:13.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.774+0000 7f05be000700 1 --2- 172.21.15.131:0/84215876 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f05a80a1120 0x7f05a80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:13.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.774+0000 7f05be000700 1 -- 172.21.15.131:0/84215876 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f05ac014070 con 0x7f05c0102170 2024-05-08T22:05:13.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.774+0000 7f05c5600700 1 --2- 172.21.15.131:0/84215876 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f05a80a1120 0x7f05a80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:13.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.774+0000 7f05c5600700 1 --2- 172.21.15.131:0/84215876 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f05a80a1120 0x7f05a80a35e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f05b40098c0 tx=0x7f05b4019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:13.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:13 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1969490121' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T22:05:13.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:13 smithi157 ceph-mon[77746]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T22:05:13.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:13 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:13.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:13 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:13.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:13 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:05:13.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:13 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:05:13.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:13 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:13.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:13 smithi157 ceph-mon[77746]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T22:05:13.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:13 smithi157 ceph-mon[77746]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T22:05:13.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:13 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1404666656' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-08T22:05:13.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:13.777+0000 7f05be000700 1 -- 172.21.15.131:0/84215876 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f05ac096490 con 0x7f05c0102170 2024-05-08T22:05:13.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:13 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1969490121' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T22:05:13.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:13 smithi132 ceph-mon[78180]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T22:05:13.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:13 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:13.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:13 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:13.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:13 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:05:13.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:13 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:05:13.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:13 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:13.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:13 smithi132 ceph-mon[78180]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T22:05:13.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:13 smithi132 ceph-mon[78180]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T22:05:13.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:13 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1404666656' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-08T22:05:13.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:13 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1969490121' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T22:05:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:13 smithi131 ceph-mon[37260]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T22:05:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:13 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:13 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:13 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:05:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:13 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:05:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:13 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:05:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:13 smithi131 ceph-mon[37260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T22:05:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:13 smithi131 ceph-mon[37260]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T22:05:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:13 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1404666656' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-08T22:05:14.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.033+0000 7f05c8000700 1 -- 172.21.15.131:0/84215876 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f05c010c980 con 0x7f05c0102170 2024-05-08T22:05:14.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.040+0000 7f05be000700 1 -- 172.21.15.131:0/84215876 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f05ac099ab0 con 0x7f05c0102170 2024-05-08T22:05:14.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.045+0000 7f05c8000700 1 -- 172.21.15.131:0/84215876 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f05a80a1120 msgr2=0x7f05a80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:14.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.045+0000 7f05c8000700 1 --2- 172.21.15.131:0/84215876 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f05a80a1120 0x7f05a80a35e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f05b40098c0 tx=0x7f05b4019040 comp rx=0 tx=0).stop 2024-05-08T22:05:14.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.045+0000 7f05c8000700 1 -- 172.21.15.131:0/84215876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05c0102170 msgr2=0x7f05c0073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:14.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.045+0000 7f05c8000700 1 --2- 172.21.15.131:0/84215876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05c0102170 0x7f05c0073390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00e9c0 tx=0x7f05ac00ecd0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.045+0000 7f05c8000700 1 -- 172.21.15.131:0/84215876 shutdown_connections 2024-05-08T22:05:14.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.045+0000 7f05c8000700 1 --2- 172.21.15.131:0/84215876 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f05a80a1120 0x7f05a80a35e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.045+0000 7f05c8000700 1 --2- 172.21.15.131:0/84215876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05c0102b10 0x7f05c00738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.045+0000 7f05c8000700 1 --2- 172.21.15.131:0/84215876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05c0100f80 0x7f05c0074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.045+0000 7f05c8000700 1 --2- 172.21.15.131:0/84215876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05c0102170 0x7f05c0073390 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.045+0000 7f05c8000700 1 -- 172.21.15.131:0/84215876 >> 172.21.15.131:0/84215876 conn(0x7f05c00fc4f0 msgr2=0x7f05c01055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:14.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.046+0000 7f05c8000700 1 -- 172.21.15.131:0/84215876 shutdown_connections 2024-05-08T22:05:14.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.047+0000 7f05c8000700 1 -- 172.21.15.131:0/84215876 wait complete. 2024-05-08T22:05:14.051 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 2 and data pool 3 2024-05-08T22:05:14.080 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-08T22:05:14.081 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-08T22:05:14.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.278+0000 7fa9f3e00700 1 -- 172.21.15.131:0/290147529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9f4101fb0 msgr2=0x7fa9f4102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:14.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.278+0000 7fa9f3e00700 1 --2- 172.21.15.131:0/290147529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9f4101fb0 0x7fa9f4102410 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009920 tx=0x7fa9e0009c30 comp rx=0 tx=0).stop 2024-05-08T22:05:14.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.278+0000 7fa9f3e00700 1 -- 172.21.15.131:0/290147529 shutdown_connections 2024-05-08T22:05:14.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.278+0000 7fa9f3e00700 1 --2- 172.21.15.131:0/290147529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f4102950 0x7fa9f4109200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.278+0000 7fa9f3e00700 1 --2- 172.21.15.131:0/290147529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9f4100dc0 0x7fa9f41011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.278+0000 7fa9f3e00700 1 --2- 172.21.15.131:0/290147529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9f4101fb0 0x7fa9f4102410 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.278+0000 7fa9f3e00700 1 -- 172.21.15.131:0/290147529 >> 172.21.15.131:0/290147529 conn(0x7fa9f40fc330 msgr2=0x7fa9f40fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:14.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.278+0000 7fa9f3e00700 1 -- 172.21.15.131:0/290147529 shutdown_connections 2024-05-08T22:05:14.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.278+0000 7fa9f3e00700 1 -- 172.21.15.131:0/290147529 wait complete. 2024-05-08T22:05:14.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.279+0000 7fa9f3e00700 1 Processor -- start 2024-05-08T22:05:14.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.279+0000 7fa9f3e00700 1 -- start start 2024-05-08T22:05:14.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.280+0000 7fa9f3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9f4100dc0 0x7fa9f4199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:14.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.280+0000 7fa9f3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f4101fb0 0x7fa9f4199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:14.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.280+0000 7fa9f3e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9f4102950 0x7fa9f41a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:14.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.280+0000 7fa9f3e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f41a0b10 con 0x7fa9f4100dc0 2024-05-08T22:05:14.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.280+0000 7fa9f3e00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f41a0c50 con 0x7fa9f4102950 2024-05-08T22:05:14.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.280+0000 7fa9f3e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f41a0de0 con 0x7fa9f4101fb0 2024-05-08T22:05:14.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.280+0000 7fa9f2a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9f4100dc0 0x7fa9f4199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:14.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.280+0000 7fa9f2a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9f4100dc0 0x7fa9f4199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56544/0 (socket says 172.21.15.131:56544) 2024-05-08T22:05:14.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.280+0000 7fa9f2a00700 1 -- 172.21.15.131:0/2912226494 learned_addr learned my addr 172.21.15.131:0/2912226494 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.280+0000 7fa9f2000700 1 --2- 172.21.15.131:0/2912226494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f4101fb0 0x7fa9f4199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.280+0000 7fa9f3400700 1 --2- 172.21.15.131:0/2912226494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9f4102950 0x7fa9f41a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.280+0000 7fa9f2a00700 1 -- 172.21.15.131:0/2912226494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f4101fb0 msgr2=0x7fa9f4199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.280+0000 7fa9f2a00700 1 --2- 172.21.15.131:0/2912226494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f4101fb0 0x7fa9f4199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.280+0000 7fa9f2a00700 1 -- 172.21.15.131:0/2912226494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9f4102950 msgr2=0x7fa9f41a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.280+0000 7fa9f2a00700 1 --2- 172.21.15.131:0/2912226494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9f4102950 0x7fa9f41a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.280+0000 7fa9f2a00700 1 -- 172.21.15.131:0/2912226494 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e40095f0 con 0x7fa9f4100dc0 2024-05-08T22:05:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.281+0000 7fa9f2a00700 1 --2- 172.21.15.131:0/2912226494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9f4100dc0 0x7fa9f4199030 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400ebf0 tx=0x7fa9e400ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.281+0000 7fa9eb400700 1 -- 172.21.15.131:0/2912226494 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4009a10 con 0x7fa9f4100dc0 2024-05-08T22:05:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.281+0000 7fa9eb400700 1 -- 172.21.15.131:0/2912226494 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9e400cc90 con 0x7fa9f4100dc0 2024-05-08T22:05:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.281+0000 7fa9f3e00700 1 -- 172.21.15.131:0/2912226494 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e0009520 con 0x7fa9f4100dc0 2024-05-08T22:05:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.281+0000 7fa9eb400700 1 -- 172.21.15.131:0/2912226494 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4018750 con 0x7fa9f4100dc0 2024-05-08T22:05:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.281+0000 7fa9f3e00700 1 -- 172.21.15.131:0/2912226494 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f41a1440 con 0x7fa9f4100dc0 2024-05-08T22:05:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.282+0000 7fa9f3e00700 1 -- 172.21.15.131:0/2912226494 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f4067ba0 con 0x7fa9f4100dc0 2024-05-08T22:05:14.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.284+0000 7fa9eb400700 1 -- 172.21.15.131:0/2912226494 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa9e40188b0 con 0x7fa9f4100dc0 2024-05-08T22:05:14.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.285+0000 7fa9eb400700 1 --2- 172.21.15.131:0/2912226494 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa9d80a11a0 0x7fa9d80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:14.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.285+0000 7fa9eb400700 1 -- 172.21.15.131:0/2912226494 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa9e4014070 con 0x7fa9f4100dc0 2024-05-08T22:05:14.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.285+0000 7fa9f2000700 1 --2- 172.21.15.131:0/2912226494 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa9d80a11a0 0x7fa9d80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:14.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.286+0000 7fa9f2000700 1 --2- 172.21.15.131:0/2912226494 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa9d80a11a0 0x7fa9d80a3660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa9e000fa40 tx=0x7fa9e0019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:14.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.289+0000 7fa9eb400700 1 -- 172.21.15.131:0/2912226494 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa9e4096650 con 0x7fa9f4100dc0 2024-05-08T22:05:14.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:14.512+0000 7fa9f3e00700 1 -- 172.21.15.131:0/2912226494 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fa9f410c650 con 0x7fa9f4100dc0 2024-05-08T22:05:14.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:14 smithi157 ceph-mon[77746]: pgmap v232: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:05:14.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:14 smithi157 ceph-mon[77746]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-08T22:05:14.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:14 smithi157 ceph-mon[77746]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T22:05:14.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:14 smithi157 ceph-mon[77746]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-08T22:05:14.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:14 smithi157 ceph-mon[77746]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T22:05:14.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:14 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/84215876' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T22:05:14.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:14 smithi157 ceph-mon[77746]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T22:05:14.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:14 smithi157 ceph-mon[77746]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T22:05:14.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:14 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/84215876' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T22:05:14.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:14 smithi157 ceph-mon[77746]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T22:05:14.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:14 smithi157 ceph-mon[77746]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T22:05:14.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:14 smithi132 ceph-mon[78180]: pgmap v232: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:05:14.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:14 smithi132 ceph-mon[78180]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-08T22:05:14.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:14 smithi132 ceph-mon[78180]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T22:05:14.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:14 smithi132 ceph-mon[78180]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-08T22:05:14.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:14 smithi132 ceph-mon[78180]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T22:05:14.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:14 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/84215876' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T22:05:14.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:14 smithi132 ceph-mon[78180]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T22:05:14.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:14 smithi132 ceph-mon[78180]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T22:05:14.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:14 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/84215876' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T22:05:14.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:14 smithi132 ceph-mon[78180]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T22:05:14.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:14 smithi132 ceph-mon[78180]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T22:05:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:14 smithi131 ceph-mon[37260]: pgmap v232: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:05:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:14 smithi131 ceph-mon[37260]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-08T22:05:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:14 smithi131 ceph-mon[37260]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T22:05:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:14 smithi131 ceph-mon[37260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-08T22:05:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:14 smithi131 ceph-mon[37260]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T22:05:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:14 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/84215876' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T22:05:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:14 smithi131 ceph-mon[37260]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T22:05:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:14 smithi131 ceph-mon[37260]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T22:05:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:14 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/84215876' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T22:05:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:14 smithi131 ceph-mon[37260]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T22:05:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:14 smithi131 ceph-mon[37260]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T22:05:15.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.050+0000 7fa9eb400700 1 -- 172.21.15.131:0/2912226494 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v79) v1 ==== 157+0+0 (secure 0 0 0) 0x7fa9e4099c70 con 0x7fa9f4100dc0 2024-05-08T22:05:15.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.053+0000 7fa9f3e00700 1 -- 172.21.15.131:0/2912226494 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa9d80a11a0 msgr2=0x7fa9d80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:15.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.053+0000 7fa9f3e00700 1 --2- 172.21.15.131:0/2912226494 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa9d80a11a0 0x7fa9d80a3660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa9e000fa40 tx=0x7fa9e0019040 comp rx=0 tx=0).stop 2024-05-08T22:05:15.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.053+0000 7fa9f3e00700 1 -- 172.21.15.131:0/2912226494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9f4100dc0 msgr2=0x7fa9f4199030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:15.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.053+0000 7fa9f3e00700 1 --2- 172.21.15.131:0/2912226494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9f4100dc0 0x7fa9f4199030 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400ebf0 tx=0x7fa9e400ef00 comp rx=0 tx=0).stop 2024-05-08T22:05:15.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.053+0000 7fa9f3e00700 1 -- 172.21.15.131:0/2912226494 shutdown_connections 2024-05-08T22:05:15.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.053+0000 7fa9f3e00700 1 --2- 172.21.15.131:0/2912226494 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa9d80a11a0 0x7fa9d80a3660 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:15.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.053+0000 7fa9f3e00700 1 --2- 172.21.15.131:0/2912226494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9f4102950 0x7fa9f41a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:15.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.053+0000 7fa9f3e00700 1 --2- 172.21.15.131:0/2912226494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9f4100dc0 0x7fa9f4199030 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:15.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.053+0000 7fa9f3e00700 1 --2- 172.21.15.131:0/2912226494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f4101fb0 0x7fa9f4199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:15.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.053+0000 7fa9f3e00700 1 -- 172.21.15.131:0/2912226494 >> 172.21.15.131:0/2912226494 conn(0x7fa9f40fc330 msgr2=0x7fa9f4107780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:15.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.053+0000 7fa9f3e00700 1 -- 172.21.15.131:0/2912226494 shutdown_connections 2024-05-08T22:05:15.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.053+0000 7fa9f3e00700 1 -- 172.21.15.131:0/2912226494 wait complete. 2024-05-08T22:05:15.073 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T22:05:15.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.273+0000 7f77b6600700 1 -- 172.21.15.131:0/322317137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77b0100eb0 msgr2=0x7f77b01012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:15.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.273+0000 7f77b6600700 1 --2- 172.21.15.131:0/322317137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77b0100eb0 0x7f77b01012d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009990 tx=0x7f77a0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:15.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.273+0000 7f77b6600700 1 -- 172.21.15.131:0/322317137 shutdown_connections 2024-05-08T22:05:15.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.273+0000 7f77b6600700 1 --2- 172.21.15.131:0/322317137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77b0102a50 0x7f77b0109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:15.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.273+0000 7f77b6600700 1 --2- 172.21.15.131:0/322317137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77b0100eb0 0x7f77b01012d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:15.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.273+0000 7f77b6600700 1 --2- 172.21.15.131:0/322317137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77b0102090 0x7f77b0102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:15.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.273+0000 7f77b6600700 1 -- 172.21.15.131:0/322317137 >> 172.21.15.131:0/322317137 conn(0x7f77b00fc470 msgr2=0x7f77b00fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:15.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.273+0000 7f77b6600700 1 -- 172.21.15.131:0/322317137 shutdown_connections 2024-05-08T22:05:15.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.273+0000 7f77b6600700 1 -- 172.21.15.131:0/322317137 wait complete. 2024-05-08T22:05:15.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.274+0000 7f77b6600700 1 Processor -- start 2024-05-08T22:05:15.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.274+0000 7f77b6600700 1 -- start start 2024-05-08T22:05:15.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.274+0000 7f77b6600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77b0100eb0 0x7f77b0199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:15.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.274+0000 7f77b6600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77b0102090 0x7f77b01995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:15.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.275+0000 7f77b6600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77b0102a50 0x7f77b01a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:15.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.275+0000 7f77b6600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b01a0b70 con 0x7f77b0102090 2024-05-08T22:05:15.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.275+0000 7f77b6600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b01a0cb0 con 0x7f77b0102a50 2024-05-08T22:05:15.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.275+0000 7f77b6600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b01a0e40 con 0x7f77b0100eb0 2024-05-08T22:05:15.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.275+0000 7f77aea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77b0102090 0x7f77b01995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:15.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.275+0000 7f77af400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77b0100eb0 0x7f77b0199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:15.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.275+0000 7f77aea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77b0102090 0x7f77b01995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56570/0 (socket says 172.21.15.131:56570) 2024-05-08T22:05:15.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.275+0000 7f77af400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77b0100eb0 0x7f77b0199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55516/0 (socket says 172.21.15.131:55516) 2024-05-08T22:05:15.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.275+0000 7f77aea00700 1 -- 172.21.15.131:0/3475130493 learned_addr learned my addr 172.21.15.131:0/3475130493 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:15.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.275+0000 7f77afe00700 1 --2- 172.21.15.131:0/3475130493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77b0102a50 0x7f77b01a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:15.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.275+0000 7f77aea00700 1 -- 172.21.15.131:0/3475130493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77b0100eb0 msgr2=0x7f77b0199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:15.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.275+0000 7f77aea00700 1 --2- 172.21.15.131:0/3475130493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77b0100eb0 0x7f77b0199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:15.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.275+0000 7f77aea00700 1 -- 172.21.15.131:0/3475130493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77b0102a50 msgr2=0x7f77b01a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:15.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.275+0000 7f77aea00700 1 --2- 172.21.15.131:0/3475130493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77b0102a50 0x7f77b01a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:15.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.275+0000 7f77aea00700 1 -- 172.21.15.131:0/3475130493 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a00095f0 con 0x7f77b0102090 2024-05-08T22:05:15.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.275+0000 7f77afe00700 1 --2- 172.21.15.131:0/3475130493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77b0102a50 0x7f77b01a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:05:15.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.275+0000 7f77aea00700 1 --2- 172.21.15.131:0/3475130493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77b0102090 0x7f77b01995d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f779c00e970 tx=0x7f779c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:15.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.276+0000 7f77a7e00700 1 -- 172.21.15.131:0/3475130493 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c009790 con 0x7f77b0102090 2024-05-08T22:05:15.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.276+0000 7f77a7e00700 1 -- 172.21.15.131:0/3475130493 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f779c00eec0 con 0x7f77b0102090 2024-05-08T22:05:15.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.276+0000 7f77a7e00700 1 -- 172.21.15.131:0/3475130493 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c003cf0 con 0x7f77b0102090 2024-05-08T22:05:15.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.276+0000 7f77b6600700 1 -- 172.21.15.131:0/3475130493 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b01a1120 con 0x7f77b0102090 2024-05-08T22:05:15.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.276+0000 7f77b6600700 1 -- 172.21.15.131:0/3475130493 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b01a15e0 con 0x7f77b0102090 2024-05-08T22:05:15.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.277+0000 7f77b6600700 1 -- 172.21.15.131:0/3475130493 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b0067c10 con 0x7f77b0102090 2024-05-08T22:05:15.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.279+0000 7f77a7e00700 1 -- 172.21.15.131:0/3475130493 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f779c00ccd0 con 0x7f77b0102090 2024-05-08T22:05:15.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.280+0000 7f77a7e00700 1 --2- 172.21.15.131:0/3475130493 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f77940a1190 0x7f77940a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:15.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.280+0000 7f77a7e00700 1 -- 172.21.15.131:0/3475130493 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f779c014070 con 0x7f77b0102090 2024-05-08T22:05:15.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.280+0000 7f77af400700 1 --2- 172.21.15.131:0/3475130493 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f77940a1190 0x7f77940a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:15.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.281+0000 7f77af400700 1 --2- 172.21.15.131:0/3475130493 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f77940a1190 0x7f77940a3650 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f77a000b680 tx=0x7f77a0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:15.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.284+0000 7f77a7e00700 1 -- 172.21.15.131:0/3475130493 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f779c096260 con 0x7f77b0102090 2024-05-08T22:05:15.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:15.496+0000 7f77b6600700 1 -- 172.21.15.131:0/3475130493 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f77b010c720 con 0x7f77b0102090 2024-05-08T22:05:15.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:15 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2912226494' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-08T22:05:15.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:15 smithi157 ceph-mon[77746]: pgmap v235: 97 pgs: 17 active+clean, 18 creating+peering, 62 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-08T22:05:15.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:15 smithi157 ceph-mon[77746]: mds.? [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] up:active 2024-05-08T22:05:15.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:15 smithi157 ceph-mon[77746]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T22:05:15.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:15 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2912226494' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-08T22:05:15.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:15 smithi157 ceph-mon[77746]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T22:05:15.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:15 smithi157 ceph-mon[77746]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T22:05:15.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:15 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2912226494' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-08T22:05:15.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:15 smithi132 ceph-mon[78180]: pgmap v235: 97 pgs: 17 active+clean, 18 creating+peering, 62 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-08T22:05:15.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:15 smithi132 ceph-mon[78180]: mds.? [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] up:active 2024-05-08T22:05:15.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:15 smithi132 ceph-mon[78180]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T22:05:15.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:15 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2912226494' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-08T22:05:15.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:15 smithi132 ceph-mon[78180]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T22:05:15.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:15 smithi132 ceph-mon[78180]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T22:05:15.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:15 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2912226494' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-08T22:05:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:15 smithi131 ceph-mon[37260]: pgmap v235: 97 pgs: 17 active+clean, 18 creating+peering, 62 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-08T22:05:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:15 smithi131 ceph-mon[37260]: mds.? [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] up:active 2024-05-08T22:05:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:15 smithi131 ceph-mon[37260]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T22:05:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:15 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2912226494' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-08T22:05:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:15 smithi131 ceph-mon[37260]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T22:05:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:15 smithi131 ceph-mon[37260]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T22:05:17.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:17 smithi157 ceph-mon[77746]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T22:05:17.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:17 smithi157 ceph-mon[77746]: Cluster is now healthy 2024-05-08T22:05:17.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:17 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3475130493' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T22:05:17.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:17 smithi132 ceph-mon[78180]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T22:05:17.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:17 smithi132 ceph-mon[78180]: Cluster is now healthy 2024-05-08T22:05:17.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:17 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3475130493' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T22:05:17.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:17 smithi131 ceph-mon[37260]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T22:05:17.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:17 smithi131 ceph-mon[37260]: Cluster is now healthy 2024-05-08T22:05:17.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:17 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3475130493' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T22:05:17.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.505+0000 7f77a7e00700 1 -- 172.21.15.131:0/3475130493 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v81) v1 ==== 228+0+0 (secure 0 0 0) 0x7f779c099880 con 0x7f77b0102090 2024-05-08T22:05:17.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.511+0000 7f77b6600700 1 -- 172.21.15.131:0/3475130493 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f77940a1190 msgr2=0x7f77940a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:17.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.511+0000 7f77b6600700 1 --2- 172.21.15.131:0/3475130493 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f77940a1190 0x7f77940a3650 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f77a000b680 tx=0x7f77a0005fd0 comp rx=0 tx=0).stop 2024-05-08T22:05:17.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.511+0000 7f77b6600700 1 -- 172.21.15.131:0/3475130493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77b0102090 msgr2=0x7f77b01995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:17.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.511+0000 7f77b6600700 1 --2- 172.21.15.131:0/3475130493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77b0102090 0x7f77b01995d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f779c00e970 tx=0x7f779c00ec80 comp rx=0 tx=0).stop 2024-05-08T22:05:17.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.512+0000 7f77b6600700 1 -- 172.21.15.131:0/3475130493 shutdown_connections 2024-05-08T22:05:17.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.512+0000 7f77b6600700 1 --2- 172.21.15.131:0/3475130493 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f77940a1190 0x7f77940a3650 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:17.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.512+0000 7f77b6600700 1 --2- 172.21.15.131:0/3475130493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77b0102a50 0x7f77b01a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:17.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.512+0000 7f77b6600700 1 --2- 172.21.15.131:0/3475130493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77b0100eb0 0x7f77b0199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:17.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.512+0000 7f77b6600700 1 --2- 172.21.15.131:0/3475130493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77b0102090 0x7f77b01995d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:17.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.512+0000 7f77b6600700 1 -- 172.21.15.131:0/3475130493 >> 172.21.15.131:0/3475130493 conn(0x7f77b00fc470 msgr2=0x7f77b00731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:17.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.512+0000 7f77b6600700 1 -- 172.21.15.131:0/3475130493 shutdown_connections 2024-05-08T22:05:17.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.512+0000 7f77b6600700 1 -- 172.21.15.131:0/3475130493 wait complete. 2024-05-08T22:05:17.515 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data_ec' created 2024-05-08T22:05:17.535 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-08T22:05:17.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.732+0000 7fab6d200700 1 -- 172.21.15.131:0/2104118578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab681020f0 msgr2=0x7fab68102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:17.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.732+0000 7fab6d200700 1 --2- 172.21.15.131:0/2104118578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab681020f0 0x7fab68102570 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fab54009990 tx=0x7fab54009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:17.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.732+0000 7fab6d200700 1 -- 172.21.15.131:0/2104118578 shutdown_connections 2024-05-08T22:05:17.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.732+0000 7fab6d200700 1 --2- 172.21.15.131:0/2104118578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab68102ab0 0x7fab681093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:17.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.732+0000 7fab6d200700 1 --2- 172.21.15.131:0/2104118578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab68100f20 0x7fab68101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:17.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.732+0000 7fab6d200700 1 --2- 172.21.15.131:0/2104118578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab681020f0 0x7fab68102570 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:17.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.732+0000 7fab6d200700 1 -- 172.21.15.131:0/2104118578 >> 172.21.15.131:0/2104118578 conn(0x7fab680fc500 msgr2=0x7fab680fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:17.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.732+0000 7fab6d200700 1 -- 172.21.15.131:0/2104118578 shutdown_connections 2024-05-08T22:05:17.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.732+0000 7fab6d200700 1 -- 172.21.15.131:0/2104118578 wait complete. 2024-05-08T22:05:17.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.733+0000 7fab6d200700 1 Processor -- start 2024-05-08T22:05:17.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.733+0000 7fab6d200700 1 -- start start 2024-05-08T22:05:17.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.734+0000 7fab6d200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab68100f20 0x7fab68074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:17.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.734+0000 7fab6d200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab681020f0 0x7fab68073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:17.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.734+0000 7fab6d200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab68102ab0 0x7fab680738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:17.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.734+0000 7fab6d200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab68074070 con 0x7fab68102ab0 2024-05-08T22:05:17.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.734+0000 7fab6d200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab680741e0 con 0x7fab681020f0 2024-05-08T22:05:17.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.734+0000 7fab6d200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab681a0ee0 con 0x7fab68100f20 2024-05-08T22:05:17.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.734+0000 7fab66800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab68102ab0 0x7fab680738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:17.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.734+0000 7fab65400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab681020f0 0x7fab68073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:17.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.734+0000 7fab66800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab68102ab0 0x7fab680738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56580/0 (socket says 172.21.15.131:56580) 2024-05-08T22:05:17.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.734+0000 7fab66800700 1 -- 172.21.15.131:0/1143778180 learned_addr learned my addr 172.21.15.131:0/1143778180 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:17.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.734+0000 7fab65e00700 1 --2- 172.21.15.131:0/1143778180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab68100f20 0x7fab68074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:17.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.735+0000 7fab65400700 1 -- 172.21.15.131:0/1143778180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab68100f20 msgr2=0x7fab68074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:17.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.735+0000 7fab65400700 1 --2- 172.21.15.131:0/1143778180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab68100f20 0x7fab68074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:17.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.735+0000 7fab65400700 1 -- 172.21.15.131:0/1143778180 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab68102ab0 msgr2=0x7fab680738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:17.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.735+0000 7fab65400700 1 --2- 172.21.15.131:0/1143778180 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab68102ab0 0x7fab680738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:17.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.735+0000 7fab65400700 1 -- 172.21.15.131:0/1143778180 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab540095f0 con 0x7fab681020f0 2024-05-08T22:05:17.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.735+0000 7fab66800700 1 --2- 172.21.15.131:0/1143778180 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab68102ab0 0x7fab680738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:17.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.735+0000 7fab65400700 1 --2- 172.21.15.131:0/1143778180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab681020f0 0x7fab68073390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fab54000c00 tx=0x7fab54004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:17.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.735+0000 7fab5ea00700 1 -- 172.21.15.131:0/1143778180 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5401d070 con 0x7fab681020f0 2024-05-08T22:05:17.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.735+0000 7fab6d200700 1 -- 172.21.15.131:0/1143778180 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab681a1110 con 0x7fab681020f0 2024-05-08T22:05:17.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.735+0000 7fab5ea00700 1 -- 172.21.15.131:0/1143778180 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab54021410 con 0x7fab681020f0 2024-05-08T22:05:17.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.735+0000 7fab5ea00700 1 -- 172.21.15.131:0/1143778180 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab54017650 con 0x7fab681020f0 2024-05-08T22:05:17.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.735+0000 7fab6d200700 1 -- 172.21.15.131:0/1143778180 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab681a1590 con 0x7fab681020f0 2024-05-08T22:05:17.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.736+0000 7fab6d200700 1 -- 172.21.15.131:0/1143778180 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab68108460 con 0x7fab681020f0 2024-05-08T22:05:17.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.738+0000 7fab5ea00700 1 -- 172.21.15.131:0/1143778180 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fab54021580 con 0x7fab681020f0 2024-05-08T22:05:17.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.739+0000 7fab5ea00700 1 --2- 172.21.15.131:0/1143778180 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fab4c0a1120 0x7fab4c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:17.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.740+0000 7fab5ea00700 1 -- 172.21.15.131:0/1143778180 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fab540ce1f0 con 0x7fab681020f0 2024-05-08T22:05:17.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.740+0000 7fab65e00700 1 --2- 172.21.15.131:0/1143778180 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fab4c0a1120 0x7fab4c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:17.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.740+0000 7fab65e00700 1 --2- 172.21.15.131:0/1143778180 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fab4c0a1120 0x7fab4c0a35e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fab68100c40 tx=0x7fab58009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:17.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.742+0000 7fab5ea00700 1 -- 172.21.15.131:0/1143778180 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fab54097870 con 0x7fab681020f0 2024-05-08T22:05:17.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:17.950+0000 7fab6d200700 1 -- 172.21.15.131:0/1143778180 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fab6810c8b0 con 0x7fab681020f0 2024-05-08T22:05:18.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:18 smithi157 ceph-mon[77746]: pgmap v238: 97 pgs: 51 active+clean, 18 creating+peering, 28 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 2 op/s 2024-05-08T22:05:18.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:18 smithi157 ceph-mon[77746]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T22:05:18.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:18 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3475130493' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T22:05:18.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:18 smithi157 ceph-mon[77746]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T22:05:18.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:18 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1143778180' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-08T22:05:18.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:18 smithi157 ceph-mon[77746]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-08T22:05:18.340 INFO:journalctl@ceph.mds.b.smithi132.stdout:May 08 22:05:18 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-b[114823]: 2024-05-08T22:05:18.103+0000 7f6b8ff30700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:05:18.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:18 smithi132 ceph-mon[78180]: pgmap v238: 97 pgs: 51 active+clean, 18 creating+peering, 28 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 2 op/s 2024-05-08T22:05:18.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:18 smithi132 ceph-mon[78180]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T22:05:18.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:18 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3475130493' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T22:05:18.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:18 smithi132 ceph-mon[78180]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T22:05:18.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:18 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1143778180' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-08T22:05:18.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:18 smithi132 ceph-mon[78180]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-08T22:05:18.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:18 smithi131 ceph-mon[37260]: pgmap v238: 97 pgs: 51 active+clean, 18 creating+peering, 28 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 2 op/s 2024-05-08T22:05:18.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:18 smithi131 ceph-mon[37260]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T22:05:18.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:18 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3475130493' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T22:05:18.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:18 smithi131 ceph-mon[37260]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T22:05:18.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:18 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1143778180' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-08T22:05:18.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:18 smithi131 ceph-mon[37260]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-08T22:05:18.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.513+0000 7fab5ea00700 1 -- 172.21.15.131:0/1143778180 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v82) v1 ==== 169+0+0 (secure 0 0 0) 0x7fab6810c8b0 con 0x7fab681020f0 2024-05-08T22:05:18.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.519+0000 7fab6d200700 1 -- 172.21.15.131:0/1143778180 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fab4c0a1120 msgr2=0x7fab4c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.519+0000 7fab6d200700 1 --2- 172.21.15.131:0/1143778180 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fab4c0a1120 0x7fab4c0a35e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fab68100c40 tx=0x7fab58009260 comp rx=0 tx=0).stop 2024-05-08T22:05:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.519+0000 7fab6d200700 1 -- 172.21.15.131:0/1143778180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab681020f0 msgr2=0x7fab68073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.519+0000 7fab6d200700 1 --2- 172.21.15.131:0/1143778180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab681020f0 0x7fab68073390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fab54000c00 tx=0x7fab54004730 comp rx=0 tx=0).stop 2024-05-08T22:05:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.519+0000 7fab6d200700 1 -- 172.21.15.131:0/1143778180 shutdown_connections 2024-05-08T22:05:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.519+0000 7fab6d200700 1 --2- 172.21.15.131:0/1143778180 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fab4c0a1120 0x7fab4c0a35e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.519+0000 7fab6d200700 1 --2- 172.21.15.131:0/1143778180 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab68102ab0 0x7fab680738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.519+0000 7fab6d200700 1 --2- 172.21.15.131:0/1143778180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab68100f20 0x7fab68074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.519+0000 7fab6d200700 1 --2- 172.21.15.131:0/1143778180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab681020f0 0x7fab68073390 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.519+0000 7fab6d200700 1 -- 172.21.15.131:0/1143778180 >> 172.21.15.131:0/1143778180 conn(0x7fab680fc500 msgr2=0x7fab681055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.520+0000 7fab6d200700 1 -- 172.21.15.131:0/1143778180 shutdown_connections 2024-05-08T22:05:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.520+0000 7fab6d200700 1 -- 172.21.15.131:0/1143778180 wait complete. 2024-05-08T22:05:18.529 INFO:teuthology.orchestra.run.smithi131.stderr:set pool 4 allow_ec_overwrites to true 2024-05-08T22:05:18.552 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-08T22:05:18.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.749+0000 7fe289600700 1 -- 172.21.15.131:0/3108609942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2841020f0 msgr2=0x7fe284102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:18.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.749+0000 7fe289600700 1 --2- 172.21.15.131:0/3108609942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2841020f0 0x7fe284102570 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009990 tx=0x7fe26c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:18.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.749+0000 7fe289600700 1 -- 172.21.15.131:0/3108609942 shutdown_connections 2024-05-08T22:05:18.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.749+0000 7fe289600700 1 --2- 172.21.15.131:0/3108609942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe284102ab0 0x7fe2841093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:18.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.749+0000 7fe289600700 1 --2- 172.21.15.131:0/3108609942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe284100f20 0x7fe284101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:18.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.749+0000 7fe289600700 1 --2- 172.21.15.131:0/3108609942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2841020f0 0x7fe284102570 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:18.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.749+0000 7fe289600700 1 -- 172.21.15.131:0/3108609942 >> 172.21.15.131:0/3108609942 conn(0x7fe2840fc500 msgr2=0x7fe2840fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:18.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.749+0000 7fe289600700 1 -- 172.21.15.131:0/3108609942 shutdown_connections 2024-05-08T22:05:18.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.749+0000 7fe289600700 1 -- 172.21.15.131:0/3108609942 wait complete. 2024-05-08T22:05:18.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.750+0000 7fe289600700 1 Processor -- start 2024-05-08T22:05:18.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.750+0000 7fe289600700 1 -- start start 2024-05-08T22:05:18.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.751+0000 7fe289600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe284100f20 0x7fe284074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:18.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.751+0000 7fe289600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2841020f0 0x7fe284073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:18.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.751+0000 7fe289600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe284102ab0 0x7fe2840738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:18.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.751+0000 7fe289600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe284074070 con 0x7fe284102ab0 2024-05-08T22:05:18.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.751+0000 7fe289600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2840741e0 con 0x7fe2841020f0 2024-05-08T22:05:18.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.751+0000 7fe289600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2841a0f60 con 0x7fe284100f20 2024-05-08T22:05:18.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.751+0000 7fe282a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe284100f20 0x7fe284074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:18.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.751+0000 7fe282a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe284100f20 0x7fe284074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55534/0 (socket says 172.21.15.131:55534) 2024-05-08T22:05:18.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.751+0000 7fe283400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe284102ab0 0x7fe2840738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:18.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.751+0000 7fe282a00700 1 -- 172.21.15.131:0/1533106616 learned_addr learned my addr 172.21.15.131:0/1533106616 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:18.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.751+0000 7fe27be00700 1 --2- 172.21.15.131:0/1533106616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2841020f0 0x7fe284073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:18.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.752+0000 7fe282a00700 1 -- 172.21.15.131:0/1533106616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2841020f0 msgr2=0x7fe284073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:18.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.752+0000 7fe282a00700 1 --2- 172.21.15.131:0/1533106616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2841020f0 0x7fe284073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:18.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.752+0000 7fe282a00700 1 -- 172.21.15.131:0/1533106616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe284102ab0 msgr2=0x7fe2840738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:18.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.752+0000 7fe282a00700 1 --2- 172.21.15.131:0/1533106616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe284102ab0 0x7fe2840738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:18.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.752+0000 7fe282a00700 1 -- 172.21.15.131:0/1533106616 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe274009520 con 0x7fe284100f20 2024-05-08T22:05:18.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.752+0000 7fe283400700 1 --2- 172.21.15.131:0/1533106616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe284102ab0 0x7fe2840738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:18.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.752+0000 7fe27be00700 1 --2- 172.21.15.131:0/1533106616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2841020f0 0x7fe284073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:18.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.752+0000 7fe282a00700 1 --2- 172.21.15.131:0/1533106616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe284100f20 0x7fe284074d00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe27400ea50 tx=0x7fe27400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:18.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.752+0000 7fe27b400700 1 -- 172.21.15.131:0/1533106616 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27400cbd0 con 0x7fe284100f20 2024-05-08T22:05:18.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.752+0000 7fe289600700 1 -- 172.21.15.131:0/1533106616 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe26c0095f0 con 0x7fe284100f20 2024-05-08T22:05:18.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.752+0000 7fe27b400700 1 -- 172.21.15.131:0/1533106616 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe27400cd30 con 0x7fe284100f20 2024-05-08T22:05:18.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.752+0000 7fe27b400700 1 -- 172.21.15.131:0/1533106616 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe274018680 con 0x7fe284100f20 2024-05-08T22:05:18.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.752+0000 7fe289600700 1 -- 172.21.15.131:0/1533106616 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2841a1500 con 0x7fe284100f20 2024-05-08T22:05:18.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.753+0000 7fe289600700 1 -- 172.21.15.131:0/1533106616 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe284108430 con 0x7fe284100f20 2024-05-08T22:05:18.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.755+0000 7fe27b400700 1 -- 172.21.15.131:0/1533106616 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe274010430 con 0x7fe284100f20 2024-05-08T22:05:18.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.756+0000 7fe27b400700 1 --2- 172.21.15.131:0/1533106616 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe2680a1120 0x7fe2680a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:18.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.757+0000 7fe27b400700 1 -- 172.21.15.131:0/1533106616 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fe274014070 con 0x7fe284100f20 2024-05-08T22:05:18.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.757+0000 7fe27be00700 1 --2- 172.21.15.131:0/1533106616 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe2680a1120 0x7fe2680a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:18.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.757+0000 7fe27be00700 1 --2- 172.21.15.131:0/1533106616 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe2680a1120 0x7fe2680a35e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe26c00b2e0 tx=0x7fe26c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:18.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:18.759+0000 7fe27b400700 1 -- 172.21.15.131:0/1533106616 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe274096a40 con 0x7fe284100f20 2024-05-08T22:05:19.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.011+0000 7fe289600700 1 -- 172.21.15.131:0/1533106616 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fe28410c800 con 0x7fe284100f20 2024-05-08T22:05:19.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.018+0000 7fe27b400700 1 -- 172.21.15.131:0/1533106616 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7fe27409a060 con 0x7fe284100f20 2024-05-08T22:05:19.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.021+0000 7fe289600700 1 -- 172.21.15.131:0/1533106616 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe2680a1120 msgr2=0x7fe2680a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:19.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.021+0000 7fe289600700 1 --2- 172.21.15.131:0/1533106616 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe2680a1120 0x7fe2680a35e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe26c00b2e0 tx=0x7fe26c005fd0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.021+0000 7fe289600700 1 -- 172.21.15.131:0/1533106616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe284100f20 msgr2=0x7fe284074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:19.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.021+0000 7fe289600700 1 --2- 172.21.15.131:0/1533106616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe284100f20 0x7fe284074d00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe27400ea50 tx=0x7fe27400ed60 comp rx=0 tx=0).stop 2024-05-08T22:05:19.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.021+0000 7fe289600700 1 -- 172.21.15.131:0/1533106616 shutdown_connections 2024-05-08T22:05:19.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.021+0000 7fe289600700 1 --2- 172.21.15.131:0/1533106616 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe2680a1120 0x7fe2680a35e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.021+0000 7fe289600700 1 --2- 172.21.15.131:0/1533106616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe284102ab0 0x7fe2840738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.021+0000 7fe289600700 1 --2- 172.21.15.131:0/1533106616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe284100f20 0x7fe284074d00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.021+0000 7fe289600700 1 --2- 172.21.15.131:0/1533106616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2841020f0 0x7fe284073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.021+0000 7fe289600700 1 -- 172.21.15.131:0/1533106616 >> 172.21.15.131:0/1533106616 conn(0x7fe2840fc500 msgr2=0x7fe284105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:19.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.021+0000 7fe289600700 1 -- 172.21.15.131:0/1533106616 shutdown_connections 2024-05-08T22:05:19.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.022+0000 7fe289600700 1 -- 172.21.15.131:0/1533106616 wait complete. 2024-05-08T22:05:19.023 INFO:teuthology.orchestra.run.smithi131.stderr:added data pool 4 to fsmap 2024-05-08T22:05:19.041 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:05:19.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.235+0000 7f97ffe00700 1 -- 172.21.15.131:0/3142692984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9800102980 msgr2=0x7f98001092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:19.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.235+0000 7f97ffe00700 1 --2- 172.21.15.131:0/3142692984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9800102980 0x7f98001092a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f97f800b0a0 tx=0x7f97f800b3b0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.235+0000 7f97ffe00700 1 -- 172.21.15.131:0/3142692984 shutdown_connections 2024-05-08T22:05:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.235+0000 7f97ffe00700 1 --2- 172.21.15.131:0/3142692984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9800102980 0x7f98001092a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.235+0000 7f97ffe00700 1 --2- 172.21.15.131:0/3142692984 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9800100e70 0x7f9800101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.235+0000 7f97ffe00700 1 --2- 172.21.15.131:0/3142692984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9800101fc0 0x7f9800102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.235+0000 7f97ffe00700 1 -- 172.21.15.131:0/3142692984 >> 172.21.15.131:0/3142692984 conn(0x7f98000fc3e0 msgr2=0x7f98000fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.235+0000 7f97ffe00700 1 -- 172.21.15.131:0/3142692984 shutdown_connections 2024-05-08T22:05:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.235+0000 7f97ffe00700 1 -- 172.21.15.131:0/3142692984 wait complete. 2024-05-08T22:05:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.236+0000 7f97ffe00700 1 Processor -- start 2024-05-08T22:05:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.236+0000 7f97ffe00700 1 -- start start 2024-05-08T22:05:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.237+0000 7f97ffe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9800100e70 0x7f9800074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.237+0000 7f97ffe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9800101fc0 0x7f9800073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.237+0000 7f97ffe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9800102980 0x7f98000738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.237+0000 7f97ffe00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9800074070 con 0x7f9800102980 2024-05-08T22:05:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.237+0000 7f97ffe00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98000741e0 con 0x7f9800101fc0 2024-05-08T22:05:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.237+0000 7f97ffe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98001a0f20 con 0x7f9800100e70 2024-05-08T22:05:19.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.237+0000 7f97ff400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9800102980 0x7f98000738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:19.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.237+0000 7f97ff400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9800102980 0x7f98000738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56620/0 (socket says 172.21.15.131:56620) 2024-05-08T22:05:19.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.237+0000 7f97ff400700 1 -- 172.21.15.131:0/3630423812 learned_addr learned my addr 172.21.15.131:0/3630423812 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:19.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.237+0000 7f97fea00700 1 --2- 172.21.15.131:0/3630423812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9800100e70 0x7f9800074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:19.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.237+0000 7f97fe000700 1 --2- 172.21.15.131:0/3630423812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9800101fc0 0x7f9800073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:19.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.237+0000 7f97fea00700 1 -- 172.21.15.131:0/3630423812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9800101fc0 msgr2=0x7f9800073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:19.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.237+0000 7f97fea00700 1 --2- 172.21.15.131:0/3630423812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9800101fc0 0x7f9800073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.237+0000 7f97fea00700 1 -- 172.21.15.131:0/3630423812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9800102980 msgr2=0x7f98000738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:19.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.237+0000 7f97fea00700 1 --2- 172.21.15.131:0/3630423812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9800102980 0x7f98000738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.237+0000 7f97fea00700 1 -- 172.21.15.131:0/3630423812 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f8009d00 con 0x7f9800100e70 2024-05-08T22:05:19.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.237+0000 7f97ff400700 1 --2- 172.21.15.131:0/3630423812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9800102980 0x7f98000738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:19.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.238+0000 7f97fea00700 1 --2- 172.21.15.131:0/3630423812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9800100e70 0x7f9800074d00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f97f000e9f0 tx=0x7f97f000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:19.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.238+0000 7f97f7400700 1 -- 172.21.15.131:0/3630423812 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f000c9c0 con 0x7f9800100e70 2024-05-08T22:05:19.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.238+0000 7f97ffe00700 1 -- 172.21.15.131:0/3630423812 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98001a1150 con 0x7f9800100e70 2024-05-08T22:05:19.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.238+0000 7f97ffe00700 1 -- 172.21.15.131:0/3630423812 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98001a1650 con 0x7f9800100e70 2024-05-08T22:05:19.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.238+0000 7f97f7400700 1 -- 172.21.15.131:0/3630423812 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97f0010430 con 0x7f9800100e70 2024-05-08T22:05:19.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.238+0000 7f97f7400700 1 -- 172.21.15.131:0/3630423812 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0018680 con 0x7f9800100e70 2024-05-08T22:05:19.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.239+0000 7f97ffe00700 1 -- 172.21.15.131:0/3630423812 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e0005320 con 0x7f9800100e70 2024-05-08T22:05:19.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.241+0000 7f97f7400700 1 -- 172.21.15.131:0/3630423812 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f97f000cb20 con 0x7f9800100e70 2024-05-08T22:05:19.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.242+0000 7f97f7400700 1 --2- 172.21.15.131:0/3630423812 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f97e40a11a0 0x7f97e40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:19.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.242+0000 7f97f7400700 1 -- 172.21.15.131:0/3630423812 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f97f0014070 con 0x7f9800100e70 2024-05-08T22:05:19.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.243+0000 7f97fe000700 1 --2- 172.21.15.131:0/3630423812 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f97e40a11a0 0x7f97e40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:19.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.243+0000 7f97fe000700 1 --2- 172.21.15.131:0/3630423812 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f97e40a11a0 0x7f97e40a3660 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f97ec009c60 tx=0x7f97ec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:19.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.244+0000 7f97f7400700 1 -- 172.21.15.131:0/3630423812 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f97f0052d60 con 0x7f9800100e70 2024-05-08T22:05:19.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.497+0000 7f97ffe00700 1 -- 172.21.15.131:0/3630423812 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97e0005190 con 0x7f9800100e70 2024-05-08T22:05:19.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.498+0000 7f97f7400700 1 -- 172.21.15.131:0/3630423812 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10831 (secure 0 0 0) 0x7f97f0099c30 con 0x7f9800100e70 2024-05-08T22:05:19.499 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:19.499 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24550,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24562,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:05:14.034908+0000","modified":"2024-05-08T22:05:19.013938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24566},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-08T22:05:19.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.501+0000 7f97ffe00700 1 -- 172.21.15.131:0/3630423812 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f97e40a11a0 msgr2=0x7f97e40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:19.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.501+0000 7f97ffe00700 1 --2- 172.21.15.131:0/3630423812 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f97e40a11a0 0x7f97e40a3660 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f97ec009c60 tx=0x7f97ec009260 comp rx=0 tx=0).stop 2024-05-08T22:05:19.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.501+0000 7f97ffe00700 1 -- 172.21.15.131:0/3630423812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9800100e70 msgr2=0x7f9800074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:19.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.501+0000 7f97ffe00700 1 --2- 172.21.15.131:0/3630423812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9800100e70 0x7f9800074d00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f97f000e9f0 tx=0x7f97f000ed00 comp rx=0 tx=0).stop 2024-05-08T22:05:19.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.501+0000 7f97ffe00700 1 -- 172.21.15.131:0/3630423812 shutdown_connections 2024-05-08T22:05:19.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.501+0000 7f97ffe00700 1 --2- 172.21.15.131:0/3630423812 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f97e40a11a0 0x7f97e40a3660 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.501+0000 7f97ffe00700 1 --2- 172.21.15.131:0/3630423812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9800102980 0x7f98000738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.501+0000 7f97ffe00700 1 --2- 172.21.15.131:0/3630423812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9800100e70 0x7f9800074d00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.501+0000 7f97ffe00700 1 --2- 172.21.15.131:0/3630423812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9800101fc0 0x7f9800073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.501+0000 7f97ffe00700 1 -- 172.21.15.131:0/3630423812 >> 172.21.15.131:0/3630423812 conn(0x7f98000fc3e0 msgr2=0x7f9800105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:19.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.501+0000 7f97ffe00700 1 -- 172.21.15.131:0/3630423812 shutdown_connections 2024-05-08T22:05:19.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.501+0000 7f97ffe00700 1 -- 172.21.15.131:0/3630423812 wait complete. 2024-05-08T22:05:19.503 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 17 2024-05-08T22:05:19.525 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:05:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.719+0000 7f8ec2a00700 1 -- 172.21.15.131:0/3358833946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ebc100e90 msgr2=0x7f8ebc1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.719+0000 7f8ec2a00700 1 --2- 172.21.15.131:0/3358833946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ebc100e90 0x7f8ebc1012b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8eac009990 tx=0x7f8eac009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.719+0000 7f8ec2a00700 1 -- 172.21.15.131:0/3358833946 shutdown_connections 2024-05-08T22:05:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.719+0000 7f8ec2a00700 1 --2- 172.21.15.131:0/3358833946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ebc1029a0 0x7f8ebc1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.719+0000 7f8ec2a00700 1 --2- 172.21.15.131:0/3358833946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ebc100e90 0x7f8ebc1012b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.719+0000 7f8ec2a00700 1 --2- 172.21.15.131:0/3358833946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ebc101fe0 0x7f8ebc102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.719+0000 7f8ec2a00700 1 -- 172.21.15.131:0/3358833946 >> 172.21.15.131:0/3358833946 conn(0x7f8ebc0fc400 msgr2=0x7f8ebc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:19.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.719+0000 7f8ec2a00700 1 -- 172.21.15.131:0/3358833946 shutdown_connections 2024-05-08T22:05:19.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.719+0000 7f8ec2a00700 1 -- 172.21.15.131:0/3358833946 wait complete. 2024-05-08T22:05:19.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.720+0000 7f8ec2a00700 1 Processor -- start 2024-05-08T22:05:19.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.720+0000 7f8ec2a00700 1 -- start start 2024-05-08T22:05:19.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.720+0000 7f8ec2a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ebc100e90 0x7f8ebc196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:19.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.720+0000 7f8ec2a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ebc101fe0 0x7f8ebc197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:19.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.720+0000 7f8ec2a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ebc1029a0 0x7f8ebc19e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:19.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.721+0000 7f8ec2a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc19e950 con 0x7f8ebc100e90 2024-05-08T22:05:19.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.721+0000 7f8ec2a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc19ea90 con 0x7f8ebc101fe0 2024-05-08T22:05:19.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.721+0000 7f8ec2a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc19ec20 con 0x7f8ebc1029a0 2024-05-08T22:05:19.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.721+0000 7f8ebb400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ebc101fe0 0x7f8ebc197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:19.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.721+0000 7f8ebbe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ebc100e90 0x7f8ebc196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:19.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.721+0000 7f8ebb400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ebc101fe0 0x7f8ebc197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:43728/0 (socket says 172.21.15.131:43728) 2024-05-08T22:05:19.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.721+0000 7f8ebb400700 1 -- 172.21.15.131:0/3554700259 learned_addr learned my addr 172.21.15.131:0/3554700259 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:19.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.721+0000 7f8ec0a00700 1 --2- 172.21.15.131:0/3554700259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ebc1029a0 0x7f8ebc19e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:19.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.721+0000 7f8ebb400700 1 -- 172.21.15.131:0/3554700259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ebc1029a0 msgr2=0x7f8ebc19e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:19.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.721+0000 7f8ebb400700 1 --2- 172.21.15.131:0/3554700259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ebc1029a0 0x7f8ebc19e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.721+0000 7f8ebb400700 1 -- 172.21.15.131:0/3554700259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ebc100e90 msgr2=0x7f8ebc196e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:19.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.721+0000 7f8ebb400700 1 --2- 172.21.15.131:0/3554700259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ebc100e90 0x7f8ebc196e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.721+0000 7f8ebb400700 1 -- 172.21.15.131:0/3554700259 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eac0095f0 con 0x7f8ebc101fe0 2024-05-08T22:05:19.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.721+0000 7f8ebbe00700 1 --2- 172.21.15.131:0/3554700259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ebc100e90 0x7f8ebc196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:19.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.721+0000 7f8ebb400700 1 --2- 172.21.15.131:0/3554700259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ebc101fe0 0x7f8ebc197360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800e9a0 tx=0x7f8ea800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:19.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.722+0000 7f8eb8c00700 1 -- 172.21.15.131:0/3554700259 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8009790 con 0x7f8ebc101fe0 2024-05-08T22:05:19.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.722+0000 7f8eb8c00700 1 -- 172.21.15.131:0/3554700259 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ea8010430 con 0x7f8ebc101fe0 2024-05-08T22:05:19.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.722+0000 7f8ec2a00700 1 -- 172.21.15.131:0/3554700259 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ebc19ef00 con 0x7f8ebc101fe0 2024-05-08T22:05:19.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.722+0000 7f8eb8c00700 1 -- 172.21.15.131:0/3554700259 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8018680 con 0x7f8ebc101fe0 2024-05-08T22:05:19.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.722+0000 7f8ec2a00700 1 -- 172.21.15.131:0/3554700259 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ebc19f380 con 0x7f8ebc101fe0 2024-05-08T22:05:19.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.723+0000 7f8ec2a00700 1 -- 172.21.15.131:0/3554700259 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ebc1082a0 con 0x7f8ebc101fe0 2024-05-08T22:05:19.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.725+0000 7f8eb8c00700 1 -- 172.21.15.131:0/3554700259 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8ea80105a0 con 0x7f8ebc101fe0 2024-05-08T22:05:19.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.726+0000 7f8eb8c00700 1 --2- 172.21.15.131:0/3554700259 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8ea00a1120 0x7f8ea00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:19.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.726+0000 7f8eb8c00700 1 -- 172.21.15.131:0/3554700259 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8ea8014070 con 0x7f8ebc101fe0 2024-05-08T22:05:19.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.726+0000 7f8ebbe00700 1 --2- 172.21.15.131:0/3554700259 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8ea00a1120 0x7f8ea00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:19.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.727+0000 7f8ebbe00700 1 --2- 172.21.15.131:0/3554700259 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8ea00a1120 0x7f8ea00a35e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00b2e0 tx=0x7f8eac005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:19.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.730+0000 7f8eb8c00700 1 -- 172.21.15.131:0/3554700259 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8ea8052d60 con 0x7f8ebc101fe0 2024-05-08T22:05:19.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:19 smithi157 ceph-mon[77746]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-08T22:05:19.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:19 smithi157 ceph-mon[77746]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T22:05:19.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:19 smithi157 ceph-mon[77746]: pgmap v241: 129 pgs: 98 active+clean, 2 creating+peering, 29 unknown; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-05-08T22:05:19.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:19 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1533106616' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-08T22:05:19.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:19 smithi157 ceph-mon[77746]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-08T22:05:19.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:19 smithi157 ceph-mon[77746]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T22:05:19.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:19 smithi157 ceph-mon[77746]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-08T22:05:19.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:19 smithi157 ceph-mon[77746]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T22:05:19.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:19 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3630423812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:05:19.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:19 smithi132 ceph-mon[78180]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-08T22:05:19.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:19 smithi132 ceph-mon[78180]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T22:05:19.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:19 smithi132 ceph-mon[78180]: pgmap v241: 129 pgs: 98 active+clean, 2 creating+peering, 29 unknown; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-05-08T22:05:19.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:19 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1533106616' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-08T22:05:19.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:19 smithi132 ceph-mon[78180]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-08T22:05:19.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:19 smithi132 ceph-mon[78180]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T22:05:19.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:19 smithi132 ceph-mon[78180]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-08T22:05:19.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:19 smithi132 ceph-mon[78180]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T22:05:19.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:19 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3630423812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:05:19.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:19 smithi131 ceph-mon[37260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-08T22:05:19.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:19 smithi131 ceph-mon[37260]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T22:05:19.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:19 smithi131 ceph-mon[37260]: pgmap v241: 129 pgs: 98 active+clean, 2 creating+peering, 29 unknown; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-05-08T22:05:19.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:19 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1533106616' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-08T22:05:19.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:19 smithi131 ceph-mon[37260]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-08T22:05:19.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:19 smithi131 ceph-mon[37260]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T22:05:19.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:19 smithi131 ceph-mon[37260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-08T22:05:19.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:19 smithi131 ceph-mon[37260]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T22:05:19.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:19 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3630423812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:05:19.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.940+0000 7f8ec2a00700 1 -- 172.21.15.131:0/3554700259 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ebc067c10 con 0x7f8ebc101fe0 2024-05-08T22:05:19.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.942+0000 7f8eb8c00700 1 -- 172.21.15.131:0/3554700259 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+24007 (secure 0 0 0) 0x7f8ea8096230 con 0x7f8ebc101fe0 2024-05-08T22:05:19.943 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:19.943 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":83,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:05:19.013678+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:05:12.469858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:05:13.484446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:05:17.016186+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:05:19.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.944+0000 7f8ec2a00700 1 -- 172.21.15.131:0/3554700259 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8ea00a1120 msgr2=0x7f8ea00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:19.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.944+0000 7f8ec2a00700 1 --2- 172.21.15.131:0/3554700259 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8ea00a1120 0x7f8ea00a35e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00b2e0 tx=0x7f8eac005fb0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.944+0000 7f8ec2a00700 1 -- 172.21.15.131:0/3554700259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ebc101fe0 msgr2=0x7f8ebc197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:19.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.944+0000 7f8ec2a00700 1 --2- 172.21.15.131:0/3554700259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ebc101fe0 0x7f8ebc197360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800e9a0 tx=0x7f8ea800ecb0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.945+0000 7f8ec2a00700 1 -- 172.21.15.131:0/3554700259 shutdown_connections 2024-05-08T22:05:19.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.945+0000 7f8ec2a00700 1 --2- 172.21.15.131:0/3554700259 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f8ea00a1120 0x7f8ea00a35e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.945+0000 7f8ec2a00700 1 --2- 172.21.15.131:0/3554700259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ebc1029a0 0x7f8ebc19e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.945+0000 7f8ec2a00700 1 --2- 172.21.15.131:0/3554700259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ebc100e90 0x7f8ebc196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.945+0000 7f8ec2a00700 1 --2- 172.21.15.131:0/3554700259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ebc101fe0 0x7f8ebc197360 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:19.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.945+0000 7f8ec2a00700 1 -- 172.21.15.131:0/3554700259 >> 172.21.15.131:0/3554700259 conn(0x7f8ebc0fc400 msgr2=0x7f8ebc1053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:19.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.945+0000 7f8ec2a00700 1 -- 172.21.15.131:0/3554700259 shutdown_connections 2024-05-08T22:05:19.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:19.945+0000 7f8ec2a00700 1 -- 172.21.15.131:0/3554700259 wait complete. 2024-05-08T22:05:19.965 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:05:20.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.157+0000 7fea91a00700 1 -- 172.21.15.131:0/3001932918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea8c101fc0 msgr2=0x7fea8c102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:20.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.157+0000 7fea91a00700 1 --2- 172.21.15.131:0/3001932918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea8c101fc0 0x7fea8c102440 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fea78009990 tx=0x7fea78009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:20.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.157+0000 7fea91a00700 1 -- 172.21.15.131:0/3001932918 shutdown_connections 2024-05-08T22:05:20.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.157+0000 7fea91a00700 1 --2- 172.21.15.131:0/3001932918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea8c102980 0x7fea8c1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:20.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.157+0000 7fea91a00700 1 --2- 172.21.15.131:0/3001932918 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea8c100e70 0x7fea8c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:20.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.157+0000 7fea91a00700 1 --2- 172.21.15.131:0/3001932918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea8c101fc0 0x7fea8c102440 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:20.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.157+0000 7fea91a00700 1 -- 172.21.15.131:0/3001932918 >> 172.21.15.131:0/3001932918 conn(0x7fea8c0fc3e0 msgr2=0x7fea8c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:20.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.157+0000 7fea91a00700 1 -- 172.21.15.131:0/3001932918 shutdown_connections 2024-05-08T22:05:20.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.157+0000 7fea91a00700 1 -- 172.21.15.131:0/3001932918 wait complete. 2024-05-08T22:05:20.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.158+0000 7fea91a00700 1 Processor -- start 2024-05-08T22:05:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.158+0000 7fea91a00700 1 -- start start 2024-05-08T22:05:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.158+0000 7fea91a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea8c100e70 0x7fea8c074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.158+0000 7fea91a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea8c101fc0 0x7fea8c073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.158+0000 7fea91a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea8c102980 0x7fea8c0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.158+0000 7fea91a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c074070 con 0x7fea8c100e70 2024-05-08T22:05:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.158+0000 7fea91a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c0741e0 con 0x7fea8c102980 2024-05-08T22:05:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.158+0000 7fea91a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c1a0e10 con 0x7fea8c101fc0 2024-05-08T22:05:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.159+0000 7fea8aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea8c100e70 0x7fea8c074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.159+0000 7fea8aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea8c100e70 0x7fea8c074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56660/0 (socket says 172.21.15.131:56660) 2024-05-08T22:05:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.159+0000 7fea8aa00700 1 -- 172.21.15.131:0/814296667 learned_addr learned my addr 172.21.15.131:0/814296667 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.159+0000 7fea8a000700 1 --2- 172.21.15.131:0/814296667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea8c101fc0 0x7fea8c073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.159+0000 7fea8b400700 1 --2- 172.21.15.131:0/814296667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea8c102980 0x7fea8c0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.159+0000 7fea8aa00700 1 -- 172.21.15.131:0/814296667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea8c101fc0 msgr2=0x7fea8c073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.159+0000 7fea8aa00700 1 --2- 172.21.15.131:0/814296667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea8c101fc0 0x7fea8c073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.159+0000 7fea8aa00700 1 -- 172.21.15.131:0/814296667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea8c102980 msgr2=0x7fea8c0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.159+0000 7fea8aa00700 1 --2- 172.21.15.131:0/814296667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea8c102980 0x7fea8c0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.159+0000 7fea8aa00700 1 -- 172.21.15.131:0/814296667 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea780095f0 con 0x7fea8c100e70 2024-05-08T22:05:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.159+0000 7fea8aa00700 1 --2- 172.21.15.131:0/814296667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea8c100e70 0x7fea8c074d00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00d980 tx=0x7fea7c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.160+0000 7fea83400700 1 -- 172.21.15.131:0/814296667 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c004220 con 0x7fea8c100e70 2024-05-08T22:05:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.160+0000 7fea83400700 1 -- 172.21.15.131:0/814296667 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea7c007640 con 0x7fea8c100e70 2024-05-08T22:05:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.160+0000 7fea91a00700 1 -- 172.21.15.131:0/814296667 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea8c1a10a0 con 0x7fea8c100e70 2024-05-08T22:05:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.160+0000 7fea83400700 1 -- 172.21.15.131:0/814296667 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c005690 con 0x7fea8c100e70 2024-05-08T22:05:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.160+0000 7fea91a00700 1 -- 172.21.15.131:0/814296667 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea8c1a1600 con 0x7fea8c100e70 2024-05-08T22:05:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.161+0000 7fea91a00700 1 -- 172.21.15.131:0/814296667 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea8c108230 con 0x7fea8c100e70 2024-05-08T22:05:20.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.163+0000 7fea83400700 1 -- 172.21.15.131:0/814296667 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fea7c004750 con 0x7fea8c100e70 2024-05-08T22:05:20.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.164+0000 7fea83400700 1 --2- 172.21.15.131:0/814296667 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fea700a11a0 0x7fea700a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:20.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.164+0000 7fea83400700 1 -- 172.21.15.131:0/814296667 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fea7c0ccda0 con 0x7fea8c100e70 2024-05-08T22:05:20.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.164+0000 7fea8a000700 1 --2- 172.21.15.131:0/814296667 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fea700a11a0 0x7fea700a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:20.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.165+0000 7fea8a000700 1 --2- 172.21.15.131:0/814296667 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fea700a11a0 0x7fea700a3660 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fea7800b5c0 tx=0x7fea78003790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:20.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.167+0000 7fea83400700 1 -- 172.21.15.131:0/814296667 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fea7c0962e0 con 0x7fea8c100e70 2024-05-08T22:05:20.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.383+0000 7fea91a00700 1 -- 172.21.15.131:0/814296667 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea8c067c10 con 0x7fea8c100e70 2024-05-08T22:05:20.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.385+0000 7fea83400700 1 -- 172.21.15.131:0/814296667 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+24007 (secure 0 0 0) 0x7fea7c016070 con 0x7fea8c100e70 2024-05-08T22:05:20.388 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:20.388 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":83,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:05:19.013678+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:05:12.469858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:05:13.484446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:05:17.016186+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:05:20.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.387+0000 7fea91a00700 1 -- 172.21.15.131:0/814296667 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fea700a11a0 msgr2=0x7fea700a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:20.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.387+0000 7fea91a00700 1 --2- 172.21.15.131:0/814296667 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fea700a11a0 0x7fea700a3660 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fea7800b5c0 tx=0x7fea78003790 comp rx=0 tx=0).stop 2024-05-08T22:05:20.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.387+0000 7fea91a00700 1 -- 172.21.15.131:0/814296667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea8c100e70 msgr2=0x7fea8c074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:20.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.387+0000 7fea91a00700 1 --2- 172.21.15.131:0/814296667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea8c100e70 0x7fea8c074d00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00d980 tx=0x7fea7c00dc90 comp rx=0 tx=0).stop 2024-05-08T22:05:20.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.388+0000 7fea91a00700 1 -- 172.21.15.131:0/814296667 shutdown_connections 2024-05-08T22:05:20.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.388+0000 7fea91a00700 1 --2- 172.21.15.131:0/814296667 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fea700a11a0 0x7fea700a3660 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:20.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.388+0000 7fea91a00700 1 --2- 172.21.15.131:0/814296667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea8c102980 0x7fea8c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:20.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.388+0000 7fea91a00700 1 --2- 172.21.15.131:0/814296667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea8c100e70 0x7fea8c074d00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:20.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.388+0000 7fea91a00700 1 --2- 172.21.15.131:0/814296667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea8c101fc0 0x7fea8c073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:20.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.388+0000 7fea91a00700 1 -- 172.21.15.131:0/814296667 >> 172.21.15.131:0/814296667 conn(0x7fea8c0fc3e0 msgr2=0x7fea8c105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:20.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.388+0000 7fea91a00700 1 -- 172.21.15.131:0/814296667 shutdown_connections 2024-05-08T22:05:20.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.388+0000 7fea91a00700 1 -- 172.21.15.131:0/814296667 wait complete. 2024-05-08T22:05:20.423 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:05:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.616+0000 7fa4e4c00700 1 -- 172.21.15.131:0/433899218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4e0102980 msgr2=0x7fa4e01092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.616+0000 7fa4e4c00700 1 --2- 172.21.15.131:0/433899218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4e0102980 0x7fa4e01092a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800b0a0 tx=0x7fa4d800b3b0 comp rx=0 tx=0).stop 2024-05-08T22:05:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.616+0000 7fa4e4c00700 1 -- 172.21.15.131:0/433899218 shutdown_connections 2024-05-08T22:05:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.616+0000 7fa4e4c00700 1 --2- 172.21.15.131:0/433899218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4e0102980 0x7fa4e01092a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.617+0000 7fa4e4c00700 1 --2- 172.21.15.131:0/433899218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e0100e70 0x7fa4e0101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.617+0000 7fa4e4c00700 1 --2- 172.21.15.131:0/433899218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4e0101fc0 0x7fa4e0102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.617+0000 7fa4e4c00700 1 -- 172.21.15.131:0/433899218 >> 172.21.15.131:0/433899218 conn(0x7fa4e00fc3e0 msgr2=0x7fa4e00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.617+0000 7fa4e4c00700 1 -- 172.21.15.131:0/433899218 shutdown_connections 2024-05-08T22:05:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.617+0000 7fa4e4c00700 1 -- 172.21.15.131:0/433899218 wait complete. 2024-05-08T22:05:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.618+0000 7fa4e4c00700 1 Processor -- start 2024-05-08T22:05:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.618+0000 7fa4e4c00700 1 -- start start 2024-05-08T22:05:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.618+0000 7fa4e4c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4e0100e70 0x7fa4e0196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.618+0000 7fa4e4c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4e0101fc0 0x7fa4e0197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.618+0000 7fa4e4c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e0102980 0x7fa4e019e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.618+0000 7fa4e4c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e019ea50 con 0x7fa4e0101fc0 2024-05-08T22:05:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.618+0000 7fa4e4c00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e019eb90 con 0x7fa4e0100e70 2024-05-08T22:05:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.618+0000 7fa4e4c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e019ed20 con 0x7fa4e0102980 2024-05-08T22:05:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.618+0000 7fa4dd400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4e0101fc0 0x7fa4e0197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.618+0000 7fa4dd400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4e0101fc0 0x7fa4e0197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56676/0 (socket says 172.21.15.131:56676) 2024-05-08T22:05:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.618+0000 7fa4dd400700 1 -- 172.21.15.131:0/1324170774 learned_addr learned my addr 172.21.15.131:0/1324170774 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.618+0000 7fa4dde00700 1 --2- 172.21.15.131:0/1324170774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4e0100e70 0x7fa4e0196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.618+0000 7fa4de800700 1 --2- 172.21.15.131:0/1324170774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e0102980 0x7fa4e019e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.619+0000 7fa4dde00700 1 -- 172.21.15.131:0/1324170774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e0102980 msgr2=0x7fa4e019e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.619+0000 7fa4dde00700 1 --2- 172.21.15.131:0/1324170774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e0102980 0x7fa4e019e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.619+0000 7fa4dde00700 1 -- 172.21.15.131:0/1324170774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4e0101fc0 msgr2=0x7fa4e0197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:20.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.619+0000 7fa4dde00700 1 --2- 172.21.15.131:0/1324170774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4e0101fc0 0x7fa4e0197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:20.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.619+0000 7fa4dde00700 1 -- 172.21.15.131:0/1324170774 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d8009d00 con 0x7fa4e0100e70 2024-05-08T22:05:20.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.619+0000 7fa4dde00700 1 --2- 172.21.15.131:0/1324170774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4e0100e70 0x7fa4e0196f20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000e9f0 tx=0x7fa4d000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:20.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.619+0000 7fa4d6a00700 1 -- 172.21.15.131:0/1324170774 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0004220 con 0x7fa4e0100e70 2024-05-08T22:05:20.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.619+0000 7fa4d6a00700 1 -- 172.21.15.131:0/1324170774 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4d0004cf0 con 0x7fa4e0100e70 2024-05-08T22:05:20.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.619+0000 7fa4e4c00700 1 -- 172.21.15.131:0/1324170774 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e019efa0 con 0x7fa4e0100e70 2024-05-08T22:05:20.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.619+0000 7fa4d6a00700 1 -- 172.21.15.131:0/1324170774 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d000ce70 con 0x7fa4e0100e70 2024-05-08T22:05:20.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.620+0000 7fa4e4c00700 1 -- 172.21.15.131:0/1324170774 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e019f3d0 con 0x7fa4e0100e70 2024-05-08T22:05:20.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.620+0000 7fa4e4c00700 1 -- 172.21.15.131:0/1324170774 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e0108290 con 0x7fa4e0100e70 2024-05-08T22:05:20.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.622+0000 7fa4d6a00700 1 -- 172.21.15.131:0/1324170774 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa4d0004750 con 0x7fa4e0100e70 2024-05-08T22:05:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.623+0000 7fa4d6a00700 1 --2- 172.21.15.131:0/1324170774 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa4c40a1120 0x7fa4c40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.624+0000 7fa4d6a00700 1 -- 172.21.15.131:0/1324170774 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa4d0014070 con 0x7fa4e0100e70 2024-05-08T22:05:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.624+0000 7fa4dd400700 1 --2- 172.21.15.131:0/1324170774 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa4c40a1120 0x7fa4c40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:20.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.624+0000 7fa4dd400700 1 --2- 172.21.15.131:0/1324170774 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa4c40a1120 0x7fa4c40a35e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc009d20 tx=0x7fa4cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:20.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.626+0000 7fa4d6a00700 1 -- 172.21.15.131:0/1324170774 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa4d0096620 con 0x7fa4e0100e70 2024-05-08T22:05:20.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:20 smithi157 ceph-mon[77746]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T22:05:20.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:20 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3554700259' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:20.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:20 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/814296667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:20.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:20 smithi132 ceph-mon[78180]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T22:05:20.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:20 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3554700259' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:20.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:20 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/814296667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:20.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.884+0000 7fa4e4c00700 1 -- 172.21.15.131:0/1324170774 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4e0067c10 con 0x7fa4e0100e70 2024-05-08T22:05:20.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.885+0000 7fa4d6a00700 1 -- 172.21.15.131:0/1324170774 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10831 (secure 0 0 0) 0x7fa4d0004e60 con 0x7fa4e0100e70 2024-05-08T22:05:20.886 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:20.886 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24550,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24562,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:05:14.034908+0000","modified":"2024-05-08T22:05:19.013938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24566},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-08T22:05:20.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.887+0000 7fa4e4c00700 1 -- 172.21.15.131:0/1324170774 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa4c40a1120 msgr2=0x7fa4c40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:20.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.887+0000 7fa4e4c00700 1 --2- 172.21.15.131:0/1324170774 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa4c40a1120 0x7fa4c40a35e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc009d20 tx=0x7fa4cc009260 comp rx=0 tx=0).stop 2024-05-08T22:05:20.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.887+0000 7fa4e4c00700 1 -- 172.21.15.131:0/1324170774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4e0100e70 msgr2=0x7fa4e0196f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:20.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.887+0000 7fa4e4c00700 1 --2- 172.21.15.131:0/1324170774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4e0100e70 0x7fa4e0196f20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000e9f0 tx=0x7fa4d000ed00 comp rx=0 tx=0).stop 2024-05-08T22:05:20.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.887+0000 7fa4e4c00700 1 -- 172.21.15.131:0/1324170774 shutdown_connections 2024-05-08T22:05:20.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.887+0000 7fa4e4c00700 1 --2- 172.21.15.131:0/1324170774 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa4c40a1120 0x7fa4c40a35e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:20.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.887+0000 7fa4e4c00700 1 --2- 172.21.15.131:0/1324170774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e0102980 0x7fa4e019e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:20.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.887+0000 7fa4e4c00700 1 --2- 172.21.15.131:0/1324170774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4e0100e70 0x7fa4e0196f20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:20.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.887+0000 7fa4e4c00700 1 --2- 172.21.15.131:0/1324170774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4e0101fc0 0x7fa4e0197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:20.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.887+0000 7fa4e4c00700 1 -- 172.21.15.131:0/1324170774 >> 172.21.15.131:0/1324170774 conn(0x7fa4e00fc3e0 msgr2=0x7fa4e01053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:20.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.887+0000 7fa4e4c00700 1 -- 172.21.15.131:0/1324170774 shutdown_connections 2024-05-08T22:05:20.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:20.888+0000 7fa4e4c00700 1 -- 172.21.15.131:0/1324170774 wait complete. 2024-05-08T22:05:20.889 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 17 2024-05-08T22:05:20.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:20 smithi131 ceph-mon[37260]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T22:05:20.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:20 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3554700259' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:20.894 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:20 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/814296667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:20.909 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:05:21.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.102+0000 7fe369e00700 1 -- 172.21.15.131:0/69470319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe364101fd0 msgr2=0x7fe364102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:21.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.102+0000 7fe369e00700 1 --2- 172.21.15.131:0/69470319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe364101fd0 0x7fe364102450 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe350009990 tx=0x7fe350009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.102+0000 7fe369e00700 1 -- 172.21.15.131:0/69470319 shutdown_connections 2024-05-08T22:05:21.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.102+0000 7fe369e00700 1 --2- 172.21.15.131:0/69470319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe364102990 0x7fe3641092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.102+0000 7fe369e00700 1 --2- 172.21.15.131:0/69470319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe364100e80 0x7fe3641012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.102+0000 7fe369e00700 1 --2- 172.21.15.131:0/69470319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe364101fd0 0x7fe364102450 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.102+0000 7fe369e00700 1 -- 172.21.15.131:0/69470319 >> 172.21.15.131:0/69470319 conn(0x7fe3640fc410 msgr2=0x7fe3640fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:21.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.102+0000 7fe369e00700 1 -- 172.21.15.131:0/69470319 shutdown_connections 2024-05-08T22:05:21.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.102+0000 7fe369e00700 1 -- 172.21.15.131:0/69470319 wait complete. 2024-05-08T22:05:21.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.103+0000 7fe369e00700 1 Processor -- start 2024-05-08T22:05:21.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.103+0000 7fe369e00700 1 -- start start 2024-05-08T22:05:21.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.103+0000 7fe369e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe364100e80 0x7fe364199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:21.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.104+0000 7fe369e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe364101fd0 0x7fe364199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:21.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.104+0000 7fe369e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe364102990 0x7fe3641a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:21.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.104+0000 7fe369e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3641a0b20 con 0x7fe364102990 2024-05-08T22:05:21.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.104+0000 7fe369e00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3641a0c60 con 0x7fe364100e80 2024-05-08T22:05:21.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.104+0000 7fe369e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3641a0df0 con 0x7fe364101fd0 2024-05-08T22:05:21.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.104+0000 7fe363400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe364102990 0x7fe3641a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:21.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.104+0000 7fe363400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe364102990 0x7fe3641a0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56684/0 (socket says 172.21.15.131:56684) 2024-05-08T22:05:21.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.104+0000 7fe362a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe364100e80 0x7fe364199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:21.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.104+0000 7fe363400700 1 -- 172.21.15.131:0/2132365642 learned_addr learned my addr 172.21.15.131:0/2132365642 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:21.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.104+0000 7fe362000700 1 --2- 172.21.15.131:0/2132365642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe364101fd0 0x7fe364199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:21.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.104+0000 7fe363400700 1 -- 172.21.15.131:0/2132365642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe364101fd0 msgr2=0x7fe364199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:21.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.104+0000 7fe363400700 1 --2- 172.21.15.131:0/2132365642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe364101fd0 0x7fe364199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.104+0000 7fe363400700 1 -- 172.21.15.131:0/2132365642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe364100e80 msgr2=0x7fe364199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:21.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.104+0000 7fe363400700 1 --2- 172.21.15.131:0/2132365642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe364100e80 0x7fe364199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.104+0000 7fe363400700 1 -- 172.21.15.131:0/2132365642 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c009c40 con 0x7fe364102990 2024-05-08T22:05:21.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.104+0000 7fe362000700 1 --2- 172.21.15.131:0/2132365642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe364101fd0 0x7fe364199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:05:21.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.104+0000 7fe363400700 1 --2- 172.21.15.131:0/2132365642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe364102990 0x7fe3641a0550 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00c370 tx=0x7fe35c00c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:21.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.105+0000 7fe368a00700 1 -- 172.21.15.131:0/2132365642 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c017070 con 0x7fe364102990 2024-05-08T22:05:21.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.105+0000 7fe369e00700 1 -- 172.21.15.131:0/2132365642 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3500095f0 con 0x7fe364102990 2024-05-08T22:05:21.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.105+0000 7fe368a00700 1 -- 172.21.15.131:0/2132365642 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe35c00f040 con 0x7fe364102990 2024-05-08T22:05:21.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.105+0000 7fe368a00700 1 -- 172.21.15.131:0/2132365642 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c00e050 con 0x7fe364102990 2024-05-08T22:05:21.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.105+0000 7fe369e00700 1 -- 172.21.15.131:0/2132365642 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3641a13f0 con 0x7fe364102990 2024-05-08T22:05:21.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.106+0000 7fe369e00700 1 -- 172.21.15.131:0/2132365642 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe364067c10 con 0x7fe364102990 2024-05-08T22:05:21.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.108+0000 7fe368a00700 1 -- 172.21.15.131:0/2132365642 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe35c004520 con 0x7fe364102990 2024-05-08T22:05:21.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.109+0000 7fe368a00700 1 --2- 172.21.15.131:0/2132365642 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe3480a1120 0x7fe3480a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:21.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.109+0000 7fe368a00700 1 -- 172.21.15.131:0/2132365642 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe35c0ccb40 con 0x7fe364102990 2024-05-08T22:05:21.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.109+0000 7fe362a00700 1 --2- 172.21.15.131:0/2132365642 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe3480a1120 0x7fe3480a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:21.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.110+0000 7fe362a00700 1 --2- 172.21.15.131:0/2132365642 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe3480a1120 0x7fe3480a35e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe354009c60 tx=0x7fe354009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.111+0000 7fe368a00700 1 -- 172.21.15.131:0/2132365642 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe35c096080 con 0x7fe364102990 2024-05-08T22:05:21.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.321+0000 7fe369e00700 1 -- 172.21.15.131:0/2132365642 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe36410c6f0 con 0x7fe364102990 2024-05-08T22:05:21.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.323+0000 7fe368a00700 1 -- 172.21.15.131:0/2132365642 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+24007 (secure 0 0 0) 0x7fe35c0996a0 con 0x7fe364102990 2024-05-08T22:05:21.324 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:21.325 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":83,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:05:19.013678+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:05:12.469858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:05:13.484446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:05:17.016186+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:05:21.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.325+0000 7fe369e00700 1 -- 172.21.15.131:0/2132365642 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe3480a1120 msgr2=0x7fe3480a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:21.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.325+0000 7fe369e00700 1 --2- 172.21.15.131:0/2132365642 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe3480a1120 0x7fe3480a35e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe354009c60 tx=0x7fe354009260 comp rx=0 tx=0).stop 2024-05-08T22:05:21.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.325+0000 7fe369e00700 1 -- 172.21.15.131:0/2132365642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe364102990 msgr2=0x7fe3641a0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:21.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.325+0000 7fe369e00700 1 --2- 172.21.15.131:0/2132365642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe364102990 0x7fe3641a0550 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00c370 tx=0x7fe35c00c680 comp rx=0 tx=0).stop 2024-05-08T22:05:21.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.325+0000 7fe369e00700 1 -- 172.21.15.131:0/2132365642 shutdown_connections 2024-05-08T22:05:21.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.325+0000 7fe369e00700 1 --2- 172.21.15.131:0/2132365642 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe3480a1120 0x7fe3480a35e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.325+0000 7fe369e00700 1 --2- 172.21.15.131:0/2132365642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe364102990 0x7fe3641a0550 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.325+0000 7fe369e00700 1 --2- 172.21.15.131:0/2132365642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe364100e80 0x7fe364199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.325+0000 7fe369e00700 1 --2- 172.21.15.131:0/2132365642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe364101fd0 0x7fe364199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.325+0000 7fe369e00700 1 -- 172.21.15.131:0/2132365642 >> 172.21.15.131:0/2132365642 conn(0x7fe3640fc410 msgr2=0x7fe364107830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:21.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.325+0000 7fe369e00700 1 -- 172.21.15.131:0/2132365642 shutdown_connections 2024-05-08T22:05:21.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.326+0000 7fe369e00700 1 -- 172.21.15.131:0/2132365642 wait complete. 2024-05-08T22:05:21.346 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:05:14.034908+0000', 'modified': '2024-05-08T22:05:19.013938+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24566}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24566': {'gid': 24566, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.132:6839/4205858754', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6838', 'nonce': 4205858754}, {'type': 'v1', 'addr': '172.21.15.132:6839', 'nonce': 4205858754}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-05-08T22:05:21.346 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T22:05:21.347 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T22:05:21.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.544+0000 7f1ddf000700 1 -- 172.21.15.131:0/3375054399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dd81020f0 msgr2=0x7f1dd8102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:21.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.544+0000 7f1ddf000700 1 --2- 172.21.15.131:0/3375054399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dd81020f0 0x7f1dd8102570 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40098f0 tx=0x7f1dc4009c00 comp rx=0 tx=0).stop 2024-05-08T22:05:21.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.544+0000 7f1ddf000700 1 -- 172.21.15.131:0/3375054399 shutdown_connections 2024-05-08T22:05:21.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.544+0000 7f1ddf000700 1 --2- 172.21.15.131:0/3375054399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dd8102ab0 0x7f1dd81093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.544+0000 7f1ddf000700 1 --2- 172.21.15.131:0/3375054399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dd8100f20 0x7f1dd8101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.544+0000 7f1ddf000700 1 --2- 172.21.15.131:0/3375054399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dd81020f0 0x7f1dd8102570 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.544+0000 7f1ddf000700 1 -- 172.21.15.131:0/3375054399 >> 172.21.15.131:0/3375054399 conn(0x7f1dd80fc500 msgr2=0x7f1dd80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:21.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.544+0000 7f1ddf000700 1 -- 172.21.15.131:0/3375054399 shutdown_connections 2024-05-08T22:05:21.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.544+0000 7f1ddf000700 1 -- 172.21.15.131:0/3375054399 wait complete. 2024-05-08T22:05:21.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.546+0000 7f1ddf000700 1 Processor -- start 2024-05-08T22:05:21.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.546+0000 7f1ddf000700 1 -- start start 2024-05-08T22:05:21.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.546+0000 7f1ddf000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dd8100f20 0x7f1dd81991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:21.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.546+0000 7f1ddf000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dd81020f0 0x7f1dd8199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:21.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.546+0000 7f1ddf000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dd8102ab0 0x7f1dd81a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:21.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.546+0000 7f1ddf000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd81a0d00 con 0x7f1dd81020f0 2024-05-08T22:05:21.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.546+0000 7f1ddf000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd81a0e40 con 0x7f1dd8102ab0 2024-05-08T22:05:21.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.546+0000 7f1ddf000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd81a0fd0 con 0x7f1dd8100f20 2024-05-08T22:05:21.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.547+0000 7f1dd7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dd8100f20 0x7f1dd81991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:21.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.547+0000 7f1dd7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dd81020f0 0x7f1dd8199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:21.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.547+0000 7f1dd7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dd8100f20 0x7f1dd81991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55620/0 (socket says 172.21.15.131:55620) 2024-05-08T22:05:21.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.547+0000 7f1dd7e00700 1 -- 172.21.15.131:0/532746452 learned_addr learned my addr 172.21.15.131:0/532746452 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:21.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.547+0000 7f1ddd000700 1 --2- 172.21.15.131:0/532746452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dd8102ab0 0x7f1dd81a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:21.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.547+0000 7f1dd7e00700 1 -- 172.21.15.131:0/532746452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dd8102ab0 msgr2=0x7f1dd81a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:21.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.547+0000 7f1dd7e00700 1 --2- 172.21.15.131:0/532746452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dd8102ab0 0x7f1dd81a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.547+0000 7f1dd7e00700 1 -- 172.21.15.131:0/532746452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dd81020f0 msgr2=0x7f1dd8199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:21.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.547+0000 7f1dd7e00700 1 --2- 172.21.15.131:0/532746452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dd81020f0 0x7f1dd8199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.547+0000 7f1dd7e00700 1 -- 172.21.15.131:0/532746452 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd0009c40 con 0x7f1dd8100f20 2024-05-08T22:05:21.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.547+0000 7f1ddd000700 1 --2- 172.21.15.131:0/532746452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dd8102ab0 0x7f1dd81a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:21.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.547+0000 7f1dd7400700 1 --2- 172.21.15.131:0/532746452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dd81020f0 0x7f1dd8199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:21.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.547+0000 7f1dd7e00700 1 --2- 172.21.15.131:0/532746452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dd8100f20 0x7f1dd81991d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800e9f0 tx=0x7f1dc800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:21.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.548+0000 7f1dd4c00700 1 -- 172.21.15.131:0/532746452 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8004230 con 0x7f1dd8100f20 2024-05-08T22:05:21.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.548+0000 7f1ddf000700 1 -- 172.21.15.131:0/532746452 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc4009520 con 0x7f1dd8100f20 2024-05-08T22:05:21.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.548+0000 7f1dd4c00700 1 -- 172.21.15.131:0/532746452 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dc8004390 con 0x7f1dd8100f20 2024-05-08T22:05:21.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.548+0000 7f1dd4c00700 1 -- 172.21.15.131:0/532746452 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8003c60 con 0x7f1dd8100f20 2024-05-08T22:05:21.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.548+0000 7f1ddf000700 1 -- 172.21.15.131:0/532746452 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd81a1630 con 0x7f1dd8100f20 2024-05-08T22:05:21.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.551+0000 7f1dd4c00700 1 -- 172.21.15.131:0/532746452 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1dc800c7c0 con 0x7f1dd8100f20 2024-05-08T22:05:21.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.553+0000 7f1df2a32b80 1 Processor -- start 2024-05-08T22:05:21.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.553+0000 7f1df2a32b80 1 -- start start 2024-05-08T22:05:21.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.553+0000 7f1dd4c00700 1 --2- 172.21.15.131:0/532746452 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1dbc0a11a0 0x7f1dbc0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:21.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.553+0000 7f1df2a32b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7adba74f0 0x55b7adba7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:21.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.553+0000 7f1df2a32b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba7e50 0x55b7adba82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:21.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.553+0000 7f1dd4c00700 1 -- 172.21.15.131:0/532746452 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1dc8014070 con 0x7f1dd8100f20 2024-05-08T22:05:21.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.553+0000 7f1df2a32b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7adba8810 0x55b7adba8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:21.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.553+0000 7f1df2a32b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7adafe100 con 0x55b7adba74f0 2024-05-08T22:05:21.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.553+0000 7f1df2a32b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7adafbac0 con 0x55b7adba7e50 2024-05-08T22:05:21.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.553+0000 7f1df2a32b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7adaf73b0 con 0x55b7adba8810 2024-05-08T22:05:21.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.553+0000 7f1ddd000700 1 --2- 172.21.15.131:0/532746452 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1dbc0a11a0 0x7f1dbc0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:21.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.554+0000 7f1ddd000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7adba8810 0x55b7adba8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:21.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.554+0000 7f1dd7400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba7e50 0x55b7adba82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:21.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.554+0000 7f1dd7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7adba74f0 0x55b7adba7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:21.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.554+0000 7f1dd7400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba7e50 0x55b7adba82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:43816/0 (socket says 172.21.15.131:43816) 2024-05-08T22:05:21.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.554+0000 7f1ddd000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7adba8810 0x55b7adba8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55636/0 (socket says 172.21.15.131:55636) 2024-05-08T22:05:21.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.554+0000 7f1dd7400700 1 -- 172.21.15.131:0/2844879815 learned_addr learned my addr 172.21.15.131:0/2844879815 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:21.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.554+0000 7f1dd7400700 1 -- 172.21.15.131:0/2844879815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7adba8810 msgr2=0x55b7adba8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:21.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.554+0000 7f1dd7400700 1 --2- 172.21.15.131:0/2844879815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7adba8810 0x55b7adba8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.554+0000 7f1dd7400700 1 -- 172.21.15.131:0/2844879815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7adba74f0 msgr2=0x55b7adba7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:21.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.554+0000 7f1dd7400700 1 --2- 172.21.15.131:0/2844879815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7adba74f0 0x55b7adba7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.554+0000 7f1dd7400700 1 -- 172.21.15.131:0/2844879815 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7ad978360 con 0x55b7adba7e50 2024-05-08T22:05:21.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.555+0000 7f1dd7400700 1 --2- 172.21.15.131:0/2844879815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba7e50 0x55b7adba82d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40098f0 tx=0x7f1dc4009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d61ad7ff204d58d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:21.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.555+0000 7f1ddd000700 1 --2- 172.21.15.131:0/532746452 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1dbc0a11a0 0x7f1dbc0a3660 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55b7adba2cf0 tx=0x7f1dd0009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:21.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.555+0000 7f1dcd600700 1 -- 172.21.15.131:0/2844879815 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc401c050 con 0x55b7adba7e50 2024-05-08T22:05:21.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.555+0000 7f1dcd600700 1 -- 172.21.15.131:0/2844879815 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dc401d040 con 0x55b7adba7e50 2024-05-08T22:05:21.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.555+0000 7f1dcd600700 1 -- 172.21.15.131:0/2844879815 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4005330 con 0x55b7adba7e50 2024-05-08T22:05:21.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.556+0000 7f1df2a32b80 1 -- 172.21.15.131:0/2844879815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba7e50 msgr2=0x55b7adba82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:21.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.556+0000 7f1df2a32b80 1 --2- 172.21.15.131:0/2844879815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba7e50 0x55b7adba82d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40098f0 tx=0x7f1dc4009f50 comp rx=0 tx=0).stop 2024-05-08T22:05:21.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.556+0000 7f1df2a32b80 1 -- 172.21.15.131:0/2844879815 shutdown_connections 2024-05-08T22:05:21.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.556+0000 7f1df2a32b80 1 --2- 172.21.15.131:0/2844879815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7adba8810 0x55b7adba8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.556+0000 7f1df2a32b80 1 --2- 172.21.15.131:0/2844879815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7adba74f0 0x55b7adba7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.556+0000 7f1df2a32b80 1 --2- 172.21.15.131:0/2844879815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba7e50 0x55b7adba82d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.556+0000 7f1df2a32b80 1 -- 172.21.15.131:0/2844879815 >> 172.21.15.131:0/2844879815 conn(0x55b7adb051c0 msgr2=0x55b7adba1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:21.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.556+0000 7f1df2a32b80 1 -- 172.21.15.131:0/2844879815 shutdown_connections 2024-05-08T22:05:21.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.556+0000 7f1df2a32b80 1 -- 172.21.15.131:0/2844879815 wait complete. 2024-05-08T22:05:21.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.556+0000 7f1df2a32b80 1 -- start start 2024-05-08T22:05:21.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.556+0000 7f1df2a32b80 1 Processor -- start 2024-05-08T22:05:21.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.557+0000 7f1df2a32b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7adba74f0 0x55b7adbb0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:21.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.557+0000 7f1df2a32b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba7e50 0x55b7adbb12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:21.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.557+0000 7f1df2a32b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7adba8810 0x55b7adbb82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:21.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.557+0000 7f1df2a32b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ada93510 con 0x55b7adba74f0 2024-05-08T22:05:21.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.557+0000 7f1df2a32b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ada931b0 con 0x55b7adba7e50 2024-05-08T22:05:21.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.557+0000 7f1df2a32b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ad97a650 con 0x55b7adba8810 2024-05-08T22:05:21.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.557+0000 7f1df2a32b80 10 client.0 _ll_register_callbacks cb 0x55b7ada9cae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:05:21.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.557+0000 7f1dd7400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba7e50 0x55b7adbb12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:21.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.557+0000 7f1ddd000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7adba8810 0x55b7adbb82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:21.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.557+0000 7f1dd7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7adba74f0 0x55b7adbb0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:21.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.557+0000 7f1dd7400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba7e50 0x55b7adbb12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:43822/0 (socket says 172.21.15.131:43822) 2024-05-08T22:05:21.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.557+0000 7f1dd7400700 1 -- 172.21.15.131:0/2570108935 learned_addr learned my addr 172.21.15.131:0/2570108935 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:21.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.558+0000 7f1dd7400700 1 -- 172.21.15.131:0/2570108935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7adba8810 msgr2=0x55b7adbb82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:21.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.558+0000 7f1dd7400700 1 --2- 172.21.15.131:0/2570108935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7adba8810 0x55b7adbb82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.558+0000 7f1dd7400700 1 -- 172.21.15.131:0/2570108935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7adba74f0 msgr2=0x55b7adbb0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:21.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.558+0000 7f1dd7400700 1 --2- 172.21.15.131:0/2570108935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7adba74f0 0x55b7adbb0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.558+0000 7f1dd7400700 1 -- 172.21.15.131:0/2570108935 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd8109d00 con 0x55b7adba7e50 2024-05-08T22:05:21.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.558+0000 7f1ddd000700 1 --2- 172.21.15.131:0/2570108935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7adba8810 0x55b7adbb82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:21.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.558+0000 7f1dd7400700 1 --2- 172.21.15.131:0/2570108935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba7e50 0x55b7adbb12b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009c40 tx=0x7f1dc400b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe5819709a80c9b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:21.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.558+0000 7f1dc2000700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:05:21.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.558+0000 7f1dc2000700 1 -- 172.21.15.131:0/2570108935 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc40054c0 con 0x55b7adba7e50 2024-05-08T22:05:21.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.558+0000 7f1df2a32b80 1 -- 172.21.15.131:0/2570108935 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7ada50220 con 0x55b7adba7e50 2024-05-08T22:05:21.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.558+0000 7f1dc2000700 1 -- 172.21.15.131:0/2570108935 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1dc4005620 con 0x55b7adba7e50 2024-05-08T22:05:21.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.558+0000 7f1dc2000700 1 -- 172.21.15.131:0/2570108935 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1dc400be90 con 0x55b7adba7e50 2024-05-08T22:05:21.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.558+0000 7f1df2a32b80 10 client.24674 fetch_fsmap learned FSMap version 17 2024-05-08T22:05:21.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.558+0000 7f1df2a32b80 1 -- 172.21.15.131:0/2570108935 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55b7ad924b10 con 0x55b7adba7e50 2024-05-08T22:05:21.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.559+0000 7f1dc2000700 1 -- 172.21.15.131:0/2570108935 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b7ada50220 con 0x55b7adba7e50 2024-05-08T22:05:21.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.559+0000 7f1dc2000700 1 -- 172.21.15.131:0/2570108935 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f1dc402a070 con 0x55b7adba7e50 2024-05-08T22:05:21.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.559+0000 7f1df2a32b80 10 client.24674 fetch_fsmap finished waiting for FSMap version 17 2024-05-08T22:05:21.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.559+0000 7f1df2a32b80 10 client.24674 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T22:05:21.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.559+0000 7f1df2a32b80 20 client.24674 populate_metadata read hostname 'smithi131' 2024-05-08T22:05:21.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.559+0000 7f1df2a32b80 1 --2- 172.21.15.131:0/2570108935 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b7adbbba50 0x55b7adbc0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:21.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.559+0000 7f1df2a32b80 4 client.24674 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:05:21.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.559+0000 7f1df2a32b80 1 -- 172.21.15.131:0/2570108935 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b7adb767c0 con 0x55b7adbbba50 2024-05-08T22:05:21.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.559+0000 7f1dd7e00700 1 --2- 172.21.15.131:0/2570108935 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b7adbbba50 0x55b7adbc0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:21.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.560+0000 7f1dc2000700 10 client.24674 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:05:21.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.560+0000 7f1dd7e00700 1 --2- 172.21.15.131:0/2570108935 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b7adbbba50 0x55b7adbc0d70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97312c69ddf37b45 server_cookie=91979a27a9b726cd in_seq=0 out_seq=0 2024-05-08T22:05:21.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.560+0000 7f1dc2000700 1 -- 172.21.15.131:0/2570108935 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b7adb767c0 con 0x55b7adbbba50 2024-05-08T22:05:21.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.560+0000 7f1dc2000700 10 client.24674 handle_command_reply: tid=0 2024-05-08T22:05:21.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.561+0000 7f1df2a32b80 1 client.24674 shutdown 2024-05-08T22:05:21.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.561+0000 7f1df2a32b80 1 -- 172.21.15.131:0/2570108935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba7e50 msgr2=0x55b7adbb12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:21.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.561+0000 7f1df2a32b80 1 --2- 172.21.15.131:0/2570108935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba7e50 0x55b7adbb12b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009c40 tx=0x7f1dc400b7e0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.561+0000 7f1df2a32b80 1 -- 172.21.15.131:0/2570108935 shutdown_connections 2024-05-08T22:05:21.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.561+0000 7f1df2a32b80 1 --2- 172.21.15.131:0/2570108935 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b7adbbba50 0x55b7adbc0d70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.561+0000 7f1df2a32b80 1 --2- 172.21.15.131:0/2570108935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7adba8810 0x55b7adbb82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.561+0000 7f1df2a32b80 1 --2- 172.21.15.131:0/2570108935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7adba74f0 0x55b7adbb0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.561+0000 7f1dc2000700 0 client.24674 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:05:21.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.561+0000 7f1df2a32b80 1 --2- 172.21.15.131:0/2570108935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba7e50 0x55b7adbb12b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.561+0000 7f1df2a32b80 1 -- 172.21.15.131:0/2570108935 >> 172.21.15.131:0/2570108935 conn(0x55b7adb051c0 msgr2=0x55b7adba1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:21.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.561+0000 7f1df2a32b80 1 -- 172.21.15.131:0/2570108935 shutdown_connections 2024-05-08T22:05:21.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.561+0000 7f1df2a32b80 1 -- 172.21.15.131:0/2570108935 wait complete. 2024-05-08T22:05:21.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.561+0000 7f1df2a32b80 20 client.24674 trim_cache size 0 max 16384 2024-05-08T22:05:21.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.579+0000 7f1df2a32b80 1 Processor -- start 2024-05-08T22:05:21.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.579+0000 7f1df2a32b80 1 -- start start 2024-05-08T22:05:21.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.580+0000 7f1df2a32b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7adba74f0 0x55b7adba78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:21.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.580+0000 7f1df2a32b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7adba7e50 0x55b7adbb82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:21.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.580+0000 7f1df2a32b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba8810 0x55b7adbaeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:21.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.580+0000 7f1df2a32b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7adb767c0 con 0x55b7adba7e50 2024-05-08T22:05:21.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.580+0000 7f1df2a32b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ada50220 con 0x55b7adba8810 2024-05-08T22:05:21.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.580+0000 7f1df2a32b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7adab52b0 con 0x55b7adba74f0 2024-05-08T22:05:21.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.580+0000 7f1dd7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7adba7e50 0x55b7adbb82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:21.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.580+0000 7f1dd7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7adba7e50 0x55b7adbb82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56728/0 (socket says 172.21.15.131:56728) 2024-05-08T22:05:21.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.580+0000 7f1dd7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7adba74f0 0x55b7adba78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:21.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.580+0000 7f1dd7400700 1 -- 172.21.15.131:0/3474137755 learned_addr learned my addr 172.21.15.131:0/3474137755 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:21.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.580+0000 7f1ddd000700 1 --2- 172.21.15.131:0/3474137755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba8810 0x55b7adbaeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:21.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.580+0000 7f1dd7400700 1 -- 172.21.15.131:0/3474137755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7adba74f0 msgr2=0x55b7adba78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:21.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.580+0000 7f1dd7400700 1 --2- 172.21.15.131:0/3474137755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7adba74f0 0x55b7adba78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.580+0000 7f1dd7400700 1 -- 172.21.15.131:0/3474137755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba8810 msgr2=0x55b7adbaeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:21.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.580+0000 7f1dd7400700 1 --2- 172.21.15.131:0/3474137755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba8810 0x55b7adbaeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.580+0000 7f1dd7400700 1 -- 172.21.15.131:0/3474137755 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7adabd510 con 0x55b7adba7e50 2024-05-08T22:05:21.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.580+0000 7f1ddd000700 1 --2- 172.21.15.131:0/3474137755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba8810 0x55b7adbaeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:05:21.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.581+0000 7f1dd7400700 1 --2- 172.21.15.131:0/3474137755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7adba7e50 0x55b7adbb82b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009810 tx=0x7f1dc4005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73950cb72e9910b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:21.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.581+0000 7f1dce000700 1 -- 172.21.15.131:0/3474137755 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc401c030 con 0x55b7adba7e50 2024-05-08T22:05:21.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.581+0000 7f1dce000700 1 -- 172.21.15.131:0/3474137755 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dc400be90 con 0x55b7adba7e50 2024-05-08T22:05:21.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.581+0000 7f1dce000700 1 -- 172.21.15.131:0/3474137755 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4003680 con 0x55b7adba7e50 2024-05-08T22:05:21.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.581+0000 7f1df2a32b80 1 -- 172.21.15.131:0/3474137755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7adba7e50 msgr2=0x55b7adbb82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:21.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.581+0000 7f1df2a32b80 1 --2- 172.21.15.131:0/3474137755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7adba7e50 0x55b7adbb82b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009810 tx=0x7f1dc4005e20 comp rx=0 tx=0).stop 2024-05-08T22:05:21.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.581+0000 7f1df2a32b80 1 -- 172.21.15.131:0/3474137755 shutdown_connections 2024-05-08T22:05:21.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.581+0000 7f1df2a32b80 1 --2- 172.21.15.131:0/3474137755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba8810 0x55b7adbaeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.581+0000 7f1df2a32b80 1 --2- 172.21.15.131:0/3474137755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7adba74f0 0x55b7adba78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.581+0000 7f1df2a32b80 1 --2- 172.21.15.131:0/3474137755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7adba7e50 0x55b7adbb82b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.582+0000 7f1df2a32b80 1 -- 172.21.15.131:0/3474137755 >> 172.21.15.131:0/3474137755 conn(0x55b7adb051c0 msgr2=0x55b7adba8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:21.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.582+0000 7f1df2a32b80 1 -- 172.21.15.131:0/3474137755 shutdown_connections 2024-05-08T22:05:21.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.582+0000 7f1df2a32b80 1 -- 172.21.15.131:0/3474137755 wait complete. 2024-05-08T22:05:21.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.582+0000 7f1df2a32b80 1 -- start start 2024-05-08T22:05:21.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.582+0000 7f1df2a32b80 1 Processor -- start 2024-05-08T22:05:21.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.583+0000 7f1df2a32b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7adba74f0 0x55b7adba0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:21.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.583+0000 7f1df2a32b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba7e50 0x55b7adba1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:21.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.583+0000 7f1df2a32b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7adba8810 0x55b7adbb7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:21.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.583+0000 7f1df2a32b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7adbb7a70 con 0x55b7adba8810 2024-05-08T22:05:21.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.583+0000 7f1df2a32b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7adbb7be0 con 0x55b7adba7e50 2024-05-08T22:05:21.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.583+0000 7f1df2a32b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7adbb7da0 con 0x55b7adba74f0 2024-05-08T22:05:21.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.583+0000 7f1df2a32b80 10 client.0 _ll_register_callbacks cb 0x55b7ada9cae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:05:21.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.583+0000 7f1dd7400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba7e50 0x55b7adba1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:21.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.583+0000 7f1ddd000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7adba8810 0x55b7adbb7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:21.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.583+0000 7f1dd7400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba7e50 0x55b7adba1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:43840/0 (socket says 172.21.15.131:43840) 2024-05-08T22:05:21.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.583+0000 7f1dd7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7adba74f0 0x55b7adba0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:21.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.583+0000 7f1dd7400700 1 -- 172.21.15.131:0/3542146593 learned_addr learned my addr 172.21.15.131:0/3542146593 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:21.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.584+0000 7f1dd7e00700 1 -- 172.21.15.131:0/3542146593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba7e50 msgr2=0x55b7adba1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:21.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.584+0000 7f1dd7e00700 1 --2- 172.21.15.131:0/3542146593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba7e50 0x55b7adba1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.584+0000 7f1dd7e00700 1 -- 172.21.15.131:0/3542146593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7adba8810 msgr2=0x55b7adbb7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:05:21.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.584+0000 7f1dd7e00700 1 --2- 172.21.15.131:0/3542146593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7adba8810 0x55b7adbb7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.584+0000 7f1dd7e00700 1 -- 172.21.15.131:0/3542146593 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7ad924b10 con 0x55b7adba74f0 2024-05-08T22:05:21.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.584+0000 7f1dd7e00700 1 --2- 172.21.15.131:0/3542146593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7adba74f0 0x55b7adba0f10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80ca6a0 tx=0x7f1dc809c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4ab8428fe38fbf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:21.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.584+0000 7f1dc2000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:05:21.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.584+0000 7f1dc2000700 1 -- 172.21.15.131:0/3542146593 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc809de20 con 0x55b7adba74f0 2024-05-08T22:05:21.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.584+0000 7f1dc2000700 1 -- 172.21.15.131:0/3542146593 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1dc80a2470 con 0x55b7adba74f0 2024-05-08T22:05:21.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.584+0000 7f1df2a32b80 1 -- 172.21.15.131:0/3542146593 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1dc401d040 con 0x55b7adba74f0 2024-05-08T22:05:21.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.584+0000 7f1dc2000700 1 -- 172.21.15.131:0/3542146593 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1dc8004500 con 0x55b7adba74f0 2024-05-08T22:05:21.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.584+0000 7f1df2a32b80 10 client.24688 fetch_fsmap learned FSMap version 17 2024-05-08T22:05:21.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.584+0000 7f1df2a32b80 1 -- 172.21.15.131:0/3542146593 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55b7ad97cc80 con 0x55b7adba74f0 2024-05-08T22:05:21.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.584+0000 7f1dc2000700 1 -- 172.21.15.131:0/3542146593 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dc401d040 con 0x55b7adba74f0 2024-05-08T22:05:21.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.585+0000 7f1dc2000700 1 -- 172.21.15.131:0/3542146593 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f1dc807ebe0 con 0x55b7adba74f0 2024-05-08T22:05:21.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.585+0000 7f1df2a32b80 10 client.24688 fetch_fsmap finished waiting for FSMap version 17 2024-05-08T22:05:21.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.585+0000 7f1df2a32b80 10 client.24688 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T22:05:21.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.585+0000 7f1df2a32b80 20 client.24688 populate_metadata read hostname 'smithi131' 2024-05-08T22:05:21.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.585+0000 7f1df2a32b80 1 --2- 172.21.15.131:0/3542146593 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b7adc16cc0 0x55b7adc13680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:21.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.585+0000 7f1df2a32b80 4 client.24688 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:05:21.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.585+0000 7f1df2a32b80 1 -- 172.21.15.131:0/3542146593 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b7adaefe70 con 0x55b7adc16cc0 2024-05-08T22:05:21.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.585+0000 7f1dd7400700 1 --2- 172.21.15.131:0/3542146593 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b7adc16cc0 0x55b7adc13680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:21.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.586+0000 7f1dc2000700 10 client.24688 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:05:21.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.586+0000 7f1dd7400700 1 --2- 172.21.15.131:0/3542146593 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b7adc16cc0 0x55b7adc13680 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8efb3387e0378e7 server_cookie=7d6316cdf35188d in_seq=0 out_seq=0 2024-05-08T22:05:21.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.586+0000 7f1dc2000700 1 -- 172.21.15.131:0/3542146593 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55b7adaefe70 con 0x55b7adc16cc0 2024-05-08T22:05:21.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.586+0000 7f1dc2000700 10 client.24688 handle_command_reply: tid=0 2024-05-08T22:05:21.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.586+0000 7f1df2a32b80 1 client.24688 shutdown 2024-05-08T22:05:21.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.586+0000 7f1df2a32b80 1 -- 172.21.15.131:0/3542146593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7adba74f0 msgr2=0x55b7adba0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:21.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.586+0000 7f1df2a32b80 1 --2- 172.21.15.131:0/3542146593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7adba74f0 0x55b7adba0f10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80ca6a0 tx=0x7f1dc809c040 comp rx=0 tx=0).stop 2024-05-08T22:05:21.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.586+0000 7f1df2a32b80 1 -- 172.21.15.131:0/3542146593 shutdown_connections 2024-05-08T22:05:21.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.586+0000 7f1df2a32b80 1 --2- 172.21.15.131:0/3542146593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7adba8810 0x55b7adbb7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.586+0000 7f1df2a32b80 1 --2- 172.21.15.131:0/3542146593 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b7adc16cc0 0x55b7adc13680 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.586+0000 7f1df2a32b80 1 --2- 172.21.15.131:0/3542146593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7adba74f0 0x55b7adba0f10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.586+0000 7f1df2a32b80 1 --2- 172.21.15.131:0/3542146593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7adba7e50 0x55b7adba1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.586+0000 7f1dc2000700 0 client.24688 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:05:21.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.586+0000 7f1df2a32b80 1 -- 172.21.15.131:0/3542146593 >> 172.21.15.131:0/3542146593 conn(0x55b7adb051c0 msgr2=0x55b7adbaec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:21.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.586+0000 7f1df2a32b80 1 -- 172.21.15.131:0/3542146593 shutdown_connections 2024-05-08T22:05:21.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.586+0000 7f1df2a32b80 1 -- 172.21.15.131:0/3542146593 wait complete. 2024-05-08T22:05:21.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.586+0000 7f1df2a32b80 20 client.24688 trim_cache size 0 max 16384 2024-05-08T22:05:21.592 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:05:21.592 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "06f1a696-0d86-11ef-bc97-c7b262605968", 2024-05-08T22:05:21.593 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-05-08T22:05:21.593 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24566, 2024-05-08T22:05:21.593 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-08T22:05:21.593 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-08T22:05:21.593 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-08T22:05:21.593 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 7.5488373390000003, 2024-05-08T22:05:21.593 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 17, 2024-05-08T22:05:21.593 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 83, 2024-05-08T22:05:21.593 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 82, 2024-05-08T22:05:21.593 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 13.805347230000001 2024-05-08T22:05:21.593 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:05:21.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.587+0000 7f1ddf000700 1 -- 172.21.15.131:0/532746452 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1dbc0a11a0 msgr2=0x7f1dbc0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:21.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.587+0000 7f1ddf000700 1 --2- 172.21.15.131:0/532746452 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1dbc0a11a0 0x7f1dbc0a3660 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55b7adba2cf0 tx=0x7f1dd0009920 comp rx=0 tx=0).stop 2024-05-08T22:05:21.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.587+0000 7f1ddf000700 1 -- 172.21.15.131:0/532746452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dd8100f20 msgr2=0x7f1dd81991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:21.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.587+0000 7f1ddf000700 1 --2- 172.21.15.131:0/532746452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dd8100f20 0x7f1dd81991d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800e9f0 tx=0x7f1dc800ed00 comp rx=0 tx=0).stop 2024-05-08T22:05:21.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.587+0000 7f1ddf000700 1 -- 172.21.15.131:0/532746452 shutdown_connections 2024-05-08T22:05:21.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.587+0000 7f1ddf000700 1 --2- 172.21.15.131:0/532746452 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1dbc0a11a0 0x7f1dbc0a3660 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.587+0000 7f1ddf000700 1 --2- 172.21.15.131:0/532746452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dd8102ab0 0x7f1dd81a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.587+0000 7f1ddf000700 1 --2- 172.21.15.131:0/532746452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dd8100f20 0x7f1dd81991d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.587+0000 7f1ddf000700 1 --2- 172.21.15.131:0/532746452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dd81020f0 0x7f1dd8199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:21.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.587+0000 7f1ddf000700 1 -- 172.21.15.131:0/532746452 >> 172.21.15.131:0/532746452 conn(0x7f1dd80fc500 msgr2=0x7f1dd8107950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:21.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.587+0000 7f1ddf000700 1 -- 172.21.15.131:0/532746452 shutdown_connections 2024-05-08T22:05:21.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:21.587+0000 7f1ddf000700 1 -- 172.21.15.131:0/532746452 wait complete. 2024-05-08T22:05:21.605 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:21 smithi131 ceph-mon[37260]: pgmap v243: 129 pgs: 111 active+clean, 2 creating+peering, 16 unknown; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 972 B/s wr, 7 op/s 2024-05-08T22:05:21.605 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:21 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1324170774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:05:21.605 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:21 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2132365642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:21.607 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T22:05:21.607 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-08T22:05:21.607 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-08T22:05:21.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:21 smithi157 ceph-mon[77746]: pgmap v243: 129 pgs: 111 active+clean, 2 creating+peering, 16 unknown; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 972 B/s wr, 7 op/s 2024-05-08T22:05:21.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:21 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1324170774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:05:21.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:21 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2132365642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:21.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:21 smithi132 ceph-mon[78180]: pgmap v243: 129 pgs: 111 active+clean, 2 creating+peering, 16 unknown; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 972 B/s wr, 7 op/s 2024-05-08T22:05:21.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:21 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1324170774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:05:21.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:21 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2132365642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:21.967 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:05:22.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:22 smithi157 ceph-mon[77746]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T22:05:22.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:22 smithi157 ceph-mon[77746]: Cluster is now healthy 2024-05-08T22:05:22.782 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:22 smithi131 ceph-mon[37260]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T22:05:22.782 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:22 smithi131 ceph-mon[37260]: Cluster is now healthy 2024-05-08T22:05:22.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:22 smithi132 ceph-mon[78180]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T22:05:22.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:22 smithi132 ceph-mon[78180]: Cluster is now healthy 2024-05-08T22:05:23.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.293+0000 7fd875400700 1 -- 172.21.15.131:0/2420098368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd870107030 msgr2=0x7fd870107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:23.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.293+0000 7fd875400700 1 --2- 172.21.15.131:0/2420098368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd870107030 0x7fd870107410 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd860009990 tx=0x7fd860009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:23.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.293+0000 7fd875400700 1 -- 172.21.15.131:0/2420098368 shutdown_connections 2024-05-08T22:05:23.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.293+0000 7fd875400700 1 --2- 172.21.15.131:0/2420098368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd870101990 0x7fd87010de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:23.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.293+0000 7fd875400700 1 --2- 172.21.15.131:0/2420098368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd870107030 0x7fd870107410 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:23.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.293+0000 7fd875400700 1 --2- 172.21.15.131:0/2420098368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd870100ff0 0x7fd870101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:23.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.293+0000 7fd875400700 1 -- 172.21.15.131:0/2420098368 >> 172.21.15.131:0/2420098368 conn(0x7fd870078190 msgr2=0x7fd8700785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:23.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.293+0000 7fd875400700 1 -- 172.21.15.131:0/2420098368 shutdown_connections 2024-05-08T22:05:23.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.293+0000 7fd875400700 1 -- 172.21.15.131:0/2420098368 wait complete. 2024-05-08T22:05:23.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.294+0000 7fd875400700 1 Processor -- start 2024-05-08T22:05:23.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.294+0000 7fd875400700 1 -- start start 2024-05-08T22:05:23.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.295+0000 7fd875400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd870100ff0 0x7fd870199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:23.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.295+0000 7fd875400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd870101990 0x7fd870199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:23.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.295+0000 7fd875400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd870107030 0x7fd87019df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:23.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.295+0000 7fd875400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87019e530 con 0x7fd870100ff0 2024-05-08T22:05:23.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.295+0000 7fd875400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87019e6a0 con 0x7fd870107030 2024-05-08T22:05:23.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.295+0000 7fd875400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87019e860 con 0x7fd870101990 2024-05-08T22:05:23.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.295+0000 7fd86e800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd870100ff0 0x7fd870199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:23.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.295+0000 7fd86e800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd870100ff0 0x7fd870199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56756/0 (socket says 172.21.15.131:56756) 2024-05-08T22:05:23.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.295+0000 7fd86e800700 1 -- 172.21.15.131:0/3784369257 learned_addr learned my addr 172.21.15.131:0/3784369257 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:23.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.295+0000 7fd86f200700 1 --2- 172.21.15.131:0/3784369257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd870107030 0x7fd87019df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:23.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.295+0000 7fd86de00700 1 --2- 172.21.15.131:0/3784369257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd870101990 0x7fd870199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:23.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.295+0000 7fd86f200700 1 -- 172.21.15.131:0/3784369257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd870101990 msgr2=0x7fd870199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:23.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.295+0000 7fd86f200700 1 --2- 172.21.15.131:0/3784369257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd870101990 0x7fd870199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:23.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.295+0000 7fd86f200700 1 -- 172.21.15.131:0/3784369257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd870100ff0 msgr2=0x7fd870199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:23.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.295+0000 7fd86f200700 1 --2- 172.21.15.131:0/3784369257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd870100ff0 0x7fd870199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:23.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.295+0000 7fd86f200700 1 -- 172.21.15.131:0/3784369257 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8600095f0 con 0x7fd870107030 2024-05-08T22:05:23.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.296+0000 7fd86e800700 1 --2- 172.21.15.131:0/3784369257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd870100ff0 0x7fd870199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:23.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.296+0000 7fd86f200700 1 --2- 172.21.15.131:0/3784369257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd870107030 0x7fd87019df20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd86800efc0 tx=0x7fd86800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:23.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.296+0000 7fd867400700 1 -- 172.21.15.131:0/3784369257 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd868018070 con 0x7fd870107030 2024-05-08T22:05:23.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.296+0000 7fd867400700 1 -- 172.21.15.131:0/3784369257 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd86800f040 con 0x7fd870107030 2024-05-08T22:05:23.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.296+0000 7fd875400700 1 -- 172.21.15.131:0/3784369257 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd87019eae0 con 0x7fd870107030 2024-05-08T22:05:23.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.296+0000 7fd867400700 1 -- 172.21.15.131:0/3784369257 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86800cc80 con 0x7fd870107030 2024-05-08T22:05:23.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.296+0000 7fd875400700 1 -- 172.21.15.131:0/3784369257 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8701a63a0 con 0x7fd870107030 2024-05-08T22:05:23.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.297+0000 7fd875400700 1 -- 172.21.15.131:0/3784369257 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd870104370 con 0x7fd870107030 2024-05-08T22:05:23.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.299+0000 7fd867400700 1 -- 172.21.15.131:0/3784369257 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fd868003a00 con 0x7fd870107030 2024-05-08T22:05:23.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.300+0000 7fd867400700 1 --2- 172.21.15.131:0/3784369257 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd8540a1120 0x7fd8540a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:23.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.301+0000 7fd867400700 1 -- 172.21.15.131:0/3784369257 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd8680ccf80 con 0x7fd870107030 2024-05-08T22:05:23.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.301+0000 7fd86e800700 1 --2- 172.21.15.131:0/3784369257 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd8540a1120 0x7fd8540a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:23.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.301+0000 7fd86e800700 1 --2- 172.21.15.131:0/3784369257 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd8540a1120 0x7fd8540a35e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd860006010 tx=0x7fd86000b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:23.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.303+0000 7fd867400700 1 -- 172.21.15.131:0/3784369257 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd868096550 con 0x7fd870107030 2024-05-08T22:05:23.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.550+0000 7fd875400700 1 -- 172.21.15.131:0/3784369257 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd8701026e0 con 0x7fd870107030 2024-05-08T22:05:23.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.551+0000 7fd867400700 1 -- 172.21.15.131:0/3784369257 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7fd868099b70 con 0x7fd870107030 2024-05-08T22:05:23.552 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-08T22:05:23.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.554+0000 7fd875400700 1 -- 172.21.15.131:0/3784369257 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd8540a1120 msgr2=0x7fd8540a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:23.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.554+0000 7fd875400700 1 --2- 172.21.15.131:0/3784369257 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd8540a1120 0x7fd8540a35e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd860006010 tx=0x7fd86000b260 comp rx=0 tx=0).stop 2024-05-08T22:05:23.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.554+0000 7fd875400700 1 -- 172.21.15.131:0/3784369257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd870107030 msgr2=0x7fd87019df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:23.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.554+0000 7fd875400700 1 --2- 172.21.15.131:0/3784369257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd870107030 0x7fd87019df20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd86800efc0 tx=0x7fd86800c3a0 comp rx=0 tx=0).stop 2024-05-08T22:05:23.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.554+0000 7fd875400700 1 -- 172.21.15.131:0/3784369257 shutdown_connections 2024-05-08T22:05:23.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.554+0000 7fd875400700 1 --2- 172.21.15.131:0/3784369257 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd8540a1120 0x7fd8540a35e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:23.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.554+0000 7fd875400700 1 --2- 172.21.15.131:0/3784369257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd870107030 0x7fd87019df20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:23.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.554+0000 7fd875400700 1 --2- 172.21.15.131:0/3784369257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd870100ff0 0x7fd870199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:23.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.554+0000 7fd875400700 1 --2- 172.21.15.131:0/3784369257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd870101990 0x7fd870199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:23.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.554+0000 7fd875400700 1 -- 172.21.15.131:0/3784369257 >> 172.21.15.131:0/3784369257 conn(0x7fd870078190 msgr2=0x7fd87010c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:23.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.554+0000 7fd875400700 1 -- 172.21.15.131:0/3784369257 shutdown_connections 2024-05-08T22:05:23.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:23.554+0000 7fd875400700 1 -- 172.21.15.131:0/3784369257 wait complete. 2024-05-08T22:05:23.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:23 smithi132 ceph-mon[78180]: pgmap v244: 129 pgs: 128 active+clean, 1 creating+peering; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 5 op/s 2024-05-08T22:05:23.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:23 smithi131 ceph-mon[37260]: pgmap v244: 129 pgs: 128 active+clean, 1 creating+peering; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 5 op/s 2024-05-08T22:05:23.937 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T22:05:23.937 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T22:05:23.937 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-05-08T22:05:23.937 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-08T22:05:23.937 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-08T22:05:23.938 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T22:05:23.938 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T22:05:23.938 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T22:05:23.938 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-08T22:05:23.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:05:23.938 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-05-08T22:05:23.983 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:05:23.983 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:05:23.984 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:05:23.984 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:05:23.984 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-05-08T22:05:23.984 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:05:23.984 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T22:05:23.984 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:05:23.985 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-05-08T22:05:23.985 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T22:05:23.985 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:05:23.985 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-05-08T22:05:23.985 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T22:05:23.985 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:05:23.986 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T22:05:23.986 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:05:23.986 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T22:05:23.986 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 41761sec preferred_lft 41761sec 2024-05-08T22:05:23.986 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-05-08T22:05:23.986 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:05:23.987 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T22:05:23.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:05:23.988 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2024-05-08T22:05:23.988 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-08T22:05:23.988 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add name ceph-brx type bridge 2024-05-08T22:05:23.988 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip addr flush dev ceph-brx 2024-05-08T22:05:23.989 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set ceph-brx up 2024-05-08T22:05:23.989 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T22:05:23.989 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-08T22:05:24.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:23 smithi157 ceph-mon[77746]: pgmap v244: 129 pgs: 128 active+clean, 1 creating+peering; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 5 op/s 2024-05-08T22:05:24.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:05:24.467 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T22:05:24.533 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-05-08T22:05:24.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:05:24.537 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-05-08T22:05:24.595 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-05-08T22:05:24.595 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T22:05:24.595 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T22:05:24.595 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T22:05:24.595 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T22:05:24.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:05:24.596 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2024-05-08T22:05:24.596 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-08T22:05:24.596 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T22:05:24.596 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T22:05:24.596 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T22:05:24.596 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-08T22:05:24.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:24 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3784369257' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T22:05:24.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:24 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3784369257' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T22:05:25.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:24 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3784369257' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T22:05:25.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:05:25.316 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-05-08T22:05:25.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:05:25.402 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list-id 2024-05-08T22:05:25.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:05:25.485 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2024-05-08T22:05:25.485 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-08T22:05:25.485 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T22:05:25.485 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-08T22:05:25.485 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-08T22:05:25.624 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:25 smithi131 ceph-mon[37260]: pgmap v245: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 568 B/s wr, 3 op/s 2024-05-08T22:05:25.665 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-08T22:05:25.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:05:25.665 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2024-05-08T22:05:25.665 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-08T22:05:25.665 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-08T22:05:25.665 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T22:05:25.665 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-08T22:05:25.665 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-08T22:05:25.665 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-08T22:05:25.665 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-08T22:05:25.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:25 smithi132 ceph-mon[78180]: pgmap v245: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 568 B/s wr, 3 op/s 2024-05-08T22:05:25.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:05:25.925 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2024-05-08T22:05:25.926 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-08T22:05:25.926 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-05-08T22:05:25.926 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T22:05:25.926 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-08T22:05:26.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:25 smithi157 ceph-mon[77746]: pgmap v245: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 568 B/s wr, 3 op/s 2024-05-08T22:05:26.093 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-08T22:05:26.093 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:05:26.093 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-08T22:05:26.135 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-08T22:05:26.135 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:05:26.136 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-08T22:05:26.218 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-05-08T22:05:26.325 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:05:26.325 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:05:26.390 INFO:teuthology.orchestra.run.smithi131.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:05:26.393 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:05:26.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:05:26.394 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-05-08T22:05:26.436 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T22:05:26.436 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-08T22:05:26.481 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-05-08T22:05:26.551 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:05:26.551 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:05:26.617 INFO:teuthology.orchestra.run.smithi131.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:05:26.620 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:05:26.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:05:26.621 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-05-08T22:05:26.678 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi131.stderr:2024-05-08T22:05:26.677+0000 7f9f75fff5c0 -1 init, newargv = 0x56091a32ea90 newargc=15 2024-05-08T22:05:26.678 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi131.stderr:ceph-fuse[90084]: starting ceph client 2024-05-08T22:05:26.687 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi131.stderr:ceph-fuse[90084]: starting fuse 2024-05-08T22:05:27.766 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-05-08T22:05:27.840 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:05:27.840 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:05:27.905 INFO:teuthology.orchestra.run.smithi131.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:05:27.908 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:05:27.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:05:27.908 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-05-08T22:05:27.950 INFO:teuthology.orchestra.run.smithi131.stdout:123 2024-05-08T22:05:27.950 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [123] 2024-05-08T22:05:27.951 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:05:27.951 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2024-05-08T22:05:27.951 DEBUG:teuthology.orchestra.run.smithi131:> import re 2024-05-08T22:05:27.951 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-05-08T22:05:27.951 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2024-05-08T22:05:27.951 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-08T22:05:27.951 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2024-05-08T22:05:27.951 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-08T22:05:27.951 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2024-05-08T22:05:27.951 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-08T22:05:27.951 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-08T22:05:27.951 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2024-05-08T22:05:27.952 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2024-05-08T22:05:27.952 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2024-05-08T22:05:27.952 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2024-05-08T22:05:27.952 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-08T22:05:27.952 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2024-05-08T22:05:27.952 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T22:05:27.952 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T22:05:27.952 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T22:05:27.952 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2024-05-08T22:05:27.952 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2024-05-08T22:05:27.952 DEBUG:teuthology.orchestra.run.smithi131:> return f 2024-05-08T22:05:27.952 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T22:05:27.952 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-08T22:05:27.952 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.admin")) 2024-05-08T22:05:27.952 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-05-08T22:05:28.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:27 smithi157 ceph-mon[77746]: pgmap v246: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-08T22:05:28.047 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:27 smithi131 ceph-mon[37260]: pgmap v246: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-08T22:05:28.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:27 smithi132 ceph-mon[78180]: pgmap v246: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-08T22:05:28.131 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.admin.90084.asok 2024-05-08T22:05:28.309 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.90084.asok 2024-05-08T22:05:28.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:05:28.310 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.90084.asok status 2024-05-08T22:05:28.532 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:05:28.532 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2024-05-08T22:05:28.532 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T22:05:28.532 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:05:28.532 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "admin", 2024-05-08T22:05:28.532 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2024-05-08T22:05:28.532 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-08T22:05:28.532 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "90084", 2024-05-08T22:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2024-05-08T22:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2024-05-08T22:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2024-05-08T22:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24700, 2024-05-08T22:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2024-05-08T22:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2024-05-08T22:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2024-05-08T22:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 24700 2024-05-08T22:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-05-08T22:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-05-08T22:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "172.21.15.131:0", 2024-05-08T22:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 3010035616 2024-05-08T22:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-05-08T22:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-05-08T22:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "172.21.15.131:0", 2024-05-08T22:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 3010035616 2024-05-08T22:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.24700 172.21.15.131:0/3010035616", 2024-05-08T22:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "172.21.15.131:0/3010035616", 2024-05-08T22:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2024-05-08T22:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 17, 2024-05-08T22:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 83, 2024-05-08T22:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2024-05-08T22:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2024-05-08T22:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2024-05-08T22:05:28.535 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:05:28.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:05:28.547 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2024-05-08T22:05:28.547 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-08T22:05:28.591 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2024-05-08T22:05:28.591 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-08T22:05:28.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:05:28.591 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-08T22:05:28.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:05:28.700 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-08T22:05:28.831 INFO:teuthology.orchestra.run.smithi131.stdout:# file: . 2024-05-08T22:05:28.831 INFO:teuthology.orchestra.run.smithi131.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-08T22:05:28.831 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:28.832 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-08T22:05:28.832 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:05:28.875 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:05:29.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.108+0000 7f6fc9000700 1 -- 172.21.15.131:0/1338578788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc4100e80 msgr2=0x7f6fc41012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:29.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.108+0000 7f6fc9000700 1 --2- 172.21.15.131:0/1338578788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc4100e80 0x7f6fc41012a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4009990 tx=0x7f6fb4009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.108+0000 7f6fc9000700 1 -- 172.21.15.131:0/1338578788 shutdown_connections 2024-05-08T22:05:29.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.108+0000 7f6fc9000700 1 --2- 172.21.15.131:0/1338578788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fc4102990 0x7f6fc41092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.108+0000 7f6fc9000700 1 --2- 172.21.15.131:0/1338578788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc4100e80 0x7f6fc41012a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.108+0000 7f6fc9000700 1 --2- 172.21.15.131:0/1338578788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fc4101fd0 0x7f6fc4102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.108+0000 7f6fc9000700 1 -- 172.21.15.131:0/1338578788 >> 172.21.15.131:0/1338578788 conn(0x7f6fc40fc410 msgr2=0x7f6fc40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:29.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.108+0000 7f6fc9000700 1 -- 172.21.15.131:0/1338578788 shutdown_connections 2024-05-08T22:05:29.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.108+0000 7f6fc9000700 1 -- 172.21.15.131:0/1338578788 wait complete. 2024-05-08T22:05:29.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.109+0000 7f6fc9000700 1 Processor -- start 2024-05-08T22:05:29.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.109+0000 7f6fc9000700 1 -- start start 2024-05-08T22:05:29.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.110+0000 7f6fc9000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fc4100e80 0x7f6fc41990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:29.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.110+0000 7f6fc9000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fc4101fd0 0x7f6fc4199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:29.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.110+0000 7f6fc9000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc4102990 0x7f6fc41a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:29.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.110+0000 7f6fc9000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc41a0bb0 con 0x7f6fc4102990 2024-05-08T22:05:29.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.110+0000 7f6fc9000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc41a0cf0 con 0x7f6fc4101fd0 2024-05-08T22:05:29.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.110+0000 7f6fc9000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc41a0e80 con 0x7f6fc4100e80 2024-05-08T22:05:29.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.110+0000 7f6fc2800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc4102990 0x7f6fc41a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:29.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.110+0000 7f6fc2800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc4102990 0x7f6fc41a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57222/0 (socket says 172.21.15.131:57222) 2024-05-08T22:05:29.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.110+0000 7f6fc2800700 1 -- 172.21.15.131:0/1887566051 learned_addr learned my addr 172.21.15.131:0/1887566051 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:29.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.110+0000 7f6fc1400700 1 --2- 172.21.15.131:0/1887566051 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fc4101fd0 0x7f6fc4199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:29.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.110+0000 7f6fc1e00700 1 --2- 172.21.15.131:0/1887566051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fc4100e80 0x7f6fc41990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:29.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.110+0000 7f6fc2800700 1 -- 172.21.15.131:0/1887566051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fc4100e80 msgr2=0x7f6fc41990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:29.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.110+0000 7f6fc2800700 1 --2- 172.21.15.131:0/1887566051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fc4100e80 0x7f6fc41990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.110+0000 7f6fc2800700 1 -- 172.21.15.131:0/1887566051 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fc4101fd0 msgr2=0x7f6fc4199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:29.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.110+0000 7f6fc2800700 1 --2- 172.21.15.131:0/1887566051 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fc4101fd0 0x7f6fc4199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.110+0000 7f6fc2800700 1 -- 172.21.15.131:0/1887566051 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb40095f0 con 0x7f6fc4102990 2024-05-08T22:05:29.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.111+0000 7f6fc2800700 1 --2- 172.21.15.131:0/1887566051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc4102990 0x7f6fc41a05e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00ef30 tx=0x7f6fbc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:29.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.111+0000 7f6fbaa00700 1 -- 172.21.15.131:0/1887566051 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc018070 con 0x7f6fc4102990 2024-05-08T22:05:29.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.111+0000 7f6fc9000700 1 -- 172.21.15.131:0/1887566051 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc41a1100 con 0x7f6fc4102990 2024-05-08T22:05:29.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.111+0000 7f6fbaa00700 1 -- 172.21.15.131:0/1887566051 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fbc00f040 con 0x7f6fc4102990 2024-05-08T22:05:29.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.111+0000 7f6fbaa00700 1 -- 172.21.15.131:0/1887566051 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc00ccc0 con 0x7f6fc4102990 2024-05-08T22:05:29.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.111+0000 7f6fc9000700 1 -- 172.21.15.131:0/1887566051 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc41a1610 con 0x7f6fc4102990 2024-05-08T22:05:29.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.112+0000 7f6fc9000700 1 -- 172.21.15.131:0/1887566051 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc4067c10 con 0x7f6fc4102990 2024-05-08T22:05:29.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.114+0000 7f6fbaa00700 1 -- 172.21.15.131:0/1887566051 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f6fbc003a00 con 0x7f6fc4102990 2024-05-08T22:05:29.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.116+0000 7f6fbaa00700 1 --2- 172.21.15.131:0/1887566051 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f6fa80a1120 0x7f6fa80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:29.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.116+0000 7f6fbaa00700 1 -- 172.21.15.131:0/1887566051 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6fbc0cc620 con 0x7f6fc4102990 2024-05-08T22:05:29.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.116+0000 7f6fc1e00700 1 --2- 172.21.15.131:0/1887566051 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f6fa80a1120 0x7f6fa80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:29.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.117+0000 7f6fc1e00700 1 --2- 172.21.15.131:0/1887566051 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f6fa80a1120 0x7f6fa80a35e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400b2e0 tx=0x7f6fb4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:29.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.121+0000 7f6fbaa00700 1 -- 172.21.15.131:0/1887566051 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6fbc095bf0 con 0x7f6fc4102990 2024-05-08T22:05:29.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.331+0000 7f6fc9000700 1 -- 172.21.15.131:0/1887566051 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fc410c6f0 con 0x7f6fc4102990 2024-05-08T22:05:29.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.333+0000 7f6fbaa00700 1 -- 172.21.15.131:0/1887566051 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+24007 (secure 0 0 0) 0x7f6fbc099210 con 0x7f6fc4102990 2024-05-08T22:05:29.334 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:29.334 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":83,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:05:19.013678+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:05:12.469858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:05:13.484446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:05:17.016186+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:05:29.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.335+0000 7f6fc9000700 1 -- 172.21.15.131:0/1887566051 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f6fa80a1120 msgr2=0x7f6fa80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:29.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.335+0000 7f6fc9000700 1 --2- 172.21.15.131:0/1887566051 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f6fa80a1120 0x7f6fa80a35e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400b2e0 tx=0x7f6fb4005fd0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.335+0000 7f6fc9000700 1 -- 172.21.15.131:0/1887566051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc4102990 msgr2=0x7f6fc41a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:29.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.335+0000 7f6fc9000700 1 --2- 172.21.15.131:0/1887566051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc4102990 0x7f6fc41a05e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00ef30 tx=0x7f6fbc00c3a0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.335+0000 7f6fc9000700 1 -- 172.21.15.131:0/1887566051 shutdown_connections 2024-05-08T22:05:29.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.335+0000 7f6fc9000700 1 --2- 172.21.15.131:0/1887566051 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f6fa80a1120 0x7f6fa80a35e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.335+0000 7f6fc9000700 1 --2- 172.21.15.131:0/1887566051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc4102990 0x7f6fc41a05e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.335+0000 7f6fc9000700 1 --2- 172.21.15.131:0/1887566051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fc4100e80 0x7f6fc41990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.335+0000 7f6fc9000700 1 --2- 172.21.15.131:0/1887566051 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fc4101fd0 0x7f6fc4199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.335+0000 7f6fc9000700 1 -- 172.21.15.131:0/1887566051 >> 172.21.15.131:0/1887566051 conn(0x7f6fc40fc410 msgr2=0x7f6fc40731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:29.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.336+0000 7f6fc9000700 1 -- 172.21.15.131:0/1887566051 shutdown_connections 2024-05-08T22:05:29.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.336+0000 7f6fc9000700 1 -- 172.21.15.131:0/1887566051 wait complete. 2024-05-08T22:05:29.356 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:01:41.271078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:05:12.469858+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:05:13.484446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:05:17.016186+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:05:29.357 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:05:29.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.553+0000 7f1019000700 1 -- 172.21.15.131:0/3092043906 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10140739a0 msgr2=0x7f1014073e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:29.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.553+0000 7f1019000700 1 --2- 172.21.15.131:0/3092043906 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10140739a0 0x7f1014073e20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1000009990 tx=0x7f1000009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.553+0000 7f1019000700 1 -- 172.21.15.131:0/3092043906 shutdown_connections 2024-05-08T22:05:29.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.553+0000 7f1019000700 1 --2- 172.21.15.131:0/3092043906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1014108e50 0x7f101410b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.553+0000 7f1019000700 1 --2- 172.21.15.131:0/3092043906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1014074f80 0x7f10140733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.553+0000 7f1019000700 1 --2- 172.21.15.131:0/3092043906 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10140739a0 0x7f1014073e20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.553+0000 7f1019000700 1 -- 172.21.15.131:0/3092043906 >> 172.21.15.131:0/3092043906 conn(0x7f10140fc4f0 msgr2=0x7f10140fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:29.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.553+0000 7f1019000700 1 -- 172.21.15.131:0/3092043906 shutdown_connections 2024-05-08T22:05:29.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.553+0000 7f1019000700 1 -- 172.21.15.131:0/3092043906 wait complete. 2024-05-08T22:05:29.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.554+0000 7f1019000700 1 Processor -- start 2024-05-08T22:05:29.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.554+0000 7f1019000700 1 -- start start 2024-05-08T22:05:29.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.555+0000 7f1019000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10140739a0 0x7f10141990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:29.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.555+0000 7f1019000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1014074f80 0x7f1014199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:29.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.555+0000 7f1019000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1014108e50 0x7f10141a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:29.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.555+0000 7f1019000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10141a0c10 con 0x7f1014074f80 2024-05-08T22:05:29.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.555+0000 7f1019000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10141a0d50 con 0x7f10140739a0 2024-05-08T22:05:29.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.555+0000 7f1019000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10141a0ee0 con 0x7f1014108e50 2024-05-08T22:05:29.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.555+0000 7f1011e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10140739a0 0x7f10141990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:29.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.555+0000 7f1011400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1014074f80 0x7f1014199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:29.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.555+0000 7f1011e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10140739a0 0x7f10141990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:59852/0 (socket says 172.21.15.131:59852) 2024-05-08T22:05:29.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.555+0000 7f1011e00700 1 -- 172.21.15.131:0/3274160572 learned_addr learned my addr 172.21.15.131:0/3274160572 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:29.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.555+0000 7f1012800700 1 --2- 172.21.15.131:0/3274160572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1014108e50 0x7f10141a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:29.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.555+0000 7f1011e00700 1 -- 172.21.15.131:0/3274160572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1014108e50 msgr2=0x7f10141a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:29.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.555+0000 7f1011e00700 1 --2- 172.21.15.131:0/3274160572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1014108e50 0x7f10141a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.555+0000 7f1011e00700 1 -- 172.21.15.131:0/3274160572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1014074f80 msgr2=0x7f1014199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:29.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.555+0000 7f1011e00700 1 --2- 172.21.15.131:0/3274160572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1014074f80 0x7f1014199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.555+0000 7f1011e00700 1 -- 172.21.15.131:0/3274160572 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10000095f0 con 0x7f10140739a0 2024-05-08T22:05:29.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.556+0000 7f1011e00700 1 --2- 172.21.15.131:0/3274160572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10140739a0 0x7f10141990e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f100400e9a0 tx=0x7f100400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:29.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.556+0000 7f100aa00700 1 -- 172.21.15.131:0/3274160572 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1004009790 con 0x7f10140739a0 2024-05-08T22:05:29.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.556+0000 7f1019000700 1 -- 172.21.15.131:0/3274160572 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10141a11c0 con 0x7f10140739a0 2024-05-08T22:05:29.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.556+0000 7f100aa00700 1 -- 172.21.15.131:0/3274160572 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1004010430 con 0x7f10140739a0 2024-05-08T22:05:29.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.556+0000 7f100aa00700 1 -- 172.21.15.131:0/3274160572 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1004018680 con 0x7f10140739a0 2024-05-08T22:05:29.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.556+0000 7f1019000700 1 -- 172.21.15.131:0/3274160572 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10141a1720 con 0x7f10140739a0 2024-05-08T22:05:29.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.557+0000 7f1019000700 1 -- 172.21.15.131:0/3274160572 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1014107c60 con 0x7f10140739a0 2024-05-08T22:05:29.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.559+0000 7f100aa00700 1 -- 172.21.15.131:0/3274160572 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f10040105a0 con 0x7f10140739a0 2024-05-08T22:05:29.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.561+0000 7f100aa00700 1 --2- 172.21.15.131:0/3274160572 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f0ff80a1120 0x7f0ff80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:29.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.561+0000 7f100aa00700 1 -- 172.21.15.131:0/3274160572 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1004014070 con 0x7f10140739a0 2024-05-08T22:05:29.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.561+0000 7f1011400700 1 --2- 172.21.15.131:0/3274160572 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f0ff80a1120 0x7f0ff80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:29.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.562+0000 7f1011400700 1 --2- 172.21.15.131:0/3274160572 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f0ff80a1120 0x7f0ff80a35e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f100000b2e0 tx=0x7f1000005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:29.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.563+0000 7f100aa00700 1 -- 172.21.15.131:0/3274160572 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1004096210 con 0x7f10140739a0 2024-05-08T22:05:29.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.771+0000 7f1019000700 1 -- 172.21.15.131:0/3274160572 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1014107640 con 0x7f10140739a0 2024-05-08T22:05:29.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.773+0000 7f100aa00700 1 -- 172.21.15.131:0/3274160572 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1004099830 con 0x7f10140739a0 2024-05-08T22:05:29.774 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-08T22:05:29.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.776+0000 7f1019000700 1 -- 172.21.15.131:0/3274160572 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f0ff80a1120 msgr2=0x7f0ff80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:29.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.776+0000 7f1019000700 1 --2- 172.21.15.131:0/3274160572 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f0ff80a1120 0x7f0ff80a35e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f100000b2e0 tx=0x7f1000005fd0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.776+0000 7f1019000700 1 -- 172.21.15.131:0/3274160572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10140739a0 msgr2=0x7f10141990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:29.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.776+0000 7f1019000700 1 --2- 172.21.15.131:0/3274160572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10140739a0 0x7f10141990e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f100400e9a0 tx=0x7f100400ecb0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.776+0000 7f1019000700 1 -- 172.21.15.131:0/3274160572 shutdown_connections 2024-05-08T22:05:29.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.776+0000 7f1019000700 1 --2- 172.21.15.131:0/3274160572 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f0ff80a1120 0x7f0ff80a35e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.776+0000 7f1019000700 1 --2- 172.21.15.131:0/3274160572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1014108e50 0x7f10141a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.776+0000 7f1019000700 1 --2- 172.21.15.131:0/3274160572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10140739a0 0x7f10141990e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.776+0000 7f1019000700 1 --2- 172.21.15.131:0/3274160572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1014074f80 0x7f1014199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.776+0000 7f1019000700 1 -- 172.21.15.131:0/3274160572 >> 172.21.15.131:0/3274160572 conn(0x7f10140fc4f0 msgr2=0x7f1014104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:29.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.776+0000 7f1019000700 1 -- 172.21.15.131:0/3274160572 shutdown_connections 2024-05-08T22:05:29.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.776+0000 7f1019000700 1 -- 172.21.15.131:0/3274160572 wait complete. 2024-05-08T22:05:29.796 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:05:29.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.992+0000 7fe3f2c00700 1 -- 172.21.15.131:0/698958606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3ec102ab0 msgr2=0x7fe3ec1093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:29.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.992+0000 7fe3f2c00700 1 --2- 172.21.15.131:0/698958606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3ec102ab0 0x7fe3ec1093d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400b0a0 tx=0x7fe3e400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.992+0000 7fe3f2c00700 1 -- 172.21.15.131:0/698958606 shutdown_connections 2024-05-08T22:05:29.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.992+0000 7fe3f2c00700 1 --2- 172.21.15.131:0/698958606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3ec102ab0 0x7fe3ec1093d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.992+0000 7fe3f2c00700 1 --2- 172.21.15.131:0/698958606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3ec100f20 0x7fe3ec101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.992+0000 7fe3f2c00700 1 --2- 172.21.15.131:0/698958606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3ec1020f0 0x7fe3ec102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:29.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.992+0000 7fe3f2c00700 1 -- 172.21.15.131:0/698958606 >> 172.21.15.131:0/698958606 conn(0x7fe3ec0fc500 msgr2=0x7fe3ec0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:29.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.992+0000 7fe3f2c00700 1 -- 172.21.15.131:0/698958606 shutdown_connections 2024-05-08T22:05:29.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.992+0000 7fe3f2c00700 1 -- 172.21.15.131:0/698958606 wait complete. 2024-05-08T22:05:29.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.993+0000 7fe3f2c00700 1 Processor -- start 2024-05-08T22:05:29.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.993+0000 7fe3f2c00700 1 -- start start 2024-05-08T22:05:29.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.994+0000 7fe3f2c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3ec100f20 0x7fe3ec199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:29.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.994+0000 7fe3f2c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3ec1020f0 0x7fe3ec199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:29.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.994+0000 7fe3f2c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3ec102ab0 0x7fe3ec1a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:29.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.994+0000 7fe3f2c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec1a0d70 con 0x7fe3ec1020f0 2024-05-08T22:05:29.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.994+0000 7fe3f2c00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec1a0eb0 con 0x7fe3ec102ab0 2024-05-08T22:05:29.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.994+0000 7fe3f2c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec1a1040 con 0x7fe3ec100f20 2024-05-08T22:05:30.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.994+0000 7fe3eb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3ec1020f0 0x7fe3ec199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:30.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.994+0000 7fe3ebe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3ec100f20 0x7fe3ec199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:30.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.994+0000 7fe3eb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3ec1020f0 0x7fe3ec199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57256/0 (socket says 172.21.15.131:57256) 2024-05-08T22:05:30.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.994+0000 7fe3eb400700 1 -- 172.21.15.131:0/4004524640 learned_addr learned my addr 172.21.15.131:0/4004524640 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:30.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.994+0000 7fe3f0c00700 1 --2- 172.21.15.131:0/4004524640 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3ec102ab0 0x7fe3ec1a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:30.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.994+0000 7fe3eb400700 1 -- 172.21.15.131:0/4004524640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3ec100f20 msgr2=0x7fe3ec199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:30.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.994+0000 7fe3eb400700 1 --2- 172.21.15.131:0/4004524640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3ec100f20 0x7fe3ec199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:30.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.994+0000 7fe3eb400700 1 -- 172.21.15.131:0/4004524640 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3ec102ab0 msgr2=0x7fe3ec1a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:30.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.994+0000 7fe3eb400700 1 --2- 172.21.15.131:0/4004524640 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3ec102ab0 0x7fe3ec1a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:30.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.994+0000 7fe3eb400700 1 -- 172.21.15.131:0/4004524640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e4009d00 con 0x7fe3ec1020f0 2024-05-08T22:05:30.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.994+0000 7fe3ebe00700 1 --2- 172.21.15.131:0/4004524640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3ec100f20 0x7fe3ec199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:30.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.995+0000 7fe3eb400700 1 --2- 172.21.15.131:0/4004524640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3ec1020f0 0x7fe3ec199780 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800e9c0 tx=0x7fe3d800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:30.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.995+0000 7fe3e8c00700 1 -- 172.21.15.131:0/4004524640 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d80097b0 con 0x7fe3ec1020f0 2024-05-08T22:05:30.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.995+0000 7fe3e8c00700 1 -- 172.21.15.131:0/4004524640 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3d800ee60 con 0x7fe3ec1020f0 2024-05-08T22:05:30.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.995+0000 7fe3f2c00700 1 -- 172.21.15.131:0/4004524640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3ec1a1320 con 0x7fe3ec1020f0 2024-05-08T22:05:30.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.995+0000 7fe3e8c00700 1 -- 172.21.15.131:0/4004524640 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d80054f0 con 0x7fe3ec1020f0 2024-05-08T22:05:30.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.995+0000 7fe3f2c00700 1 -- 172.21.15.131:0/4004524640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3ec1a1830 con 0x7fe3ec1020f0 2024-05-08T22:05:30.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.996+0000 7fe3f2c00700 1 -- 172.21.15.131:0/4004524640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3ec067cf0 con 0x7fe3ec1020f0 2024-05-08T22:05:30.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.998+0000 7fe3e8c00700 1 -- 172.21.15.131:0/4004524640 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe3d8004750 con 0x7fe3ec1020f0 2024-05-08T22:05:30.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.999+0000 7fe3e8c00700 1 --2- 172.21.15.131:0/4004524640 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe3d00a11a0 0x7fe3d00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:30.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:29.999+0000 7fe3e8c00700 1 -- 172.21.15.131:0/4004524640 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe3d8014070 con 0x7fe3ec1020f0 2024-05-08T22:05:30.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.000+0000 7fe3ebe00700 1 --2- 172.21.15.131:0/4004524640 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe3d00a11a0 0x7fe3d00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:30.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.000+0000 7fe3ebe00700 1 --2- 172.21.15.131:0/4004524640 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe3d00a11a0 0x7fe3d00a3660 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec100c40 tx=0x7fe3dc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:30.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.004+0000 7fe3e8c00700 1 -- 172.21.15.131:0/4004524640 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe3d8096240 con 0x7fe3ec1020f0 2024-05-08T22:05:30.028 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:29 smithi157 ceph-mon[77746]: pgmap v247: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 0 op/s 2024-05-08T22:05:30.108 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:29 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1887566051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:30.109 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:29 smithi132 ceph-mon[78180]: pgmap v247: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 0 op/s 2024-05-08T22:05:30.109 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:29 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1887566051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:30.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:29 smithi131 ceph-mon[37260]: pgmap v247: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 0 op/s 2024-05-08T22:05:30.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:29 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1887566051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:30.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.219+0000 7fe3f2c00700 1 -- 172.21.15.131:0/4004524640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe3ec1a1bb0 con 0x7fe3ec1020f0 2024-05-08T22:05:30.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.220+0000 7fe3e8c00700 1 -- 172.21.15.131:0/4004524640 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe3d8099860 con 0x7fe3ec1020f0 2024-05-08T22:05:30.221 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-08T22:05:30.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.222+0000 7fe3f2c00700 1 -- 172.21.15.131:0/4004524640 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe3d00a11a0 msgr2=0x7fe3d00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:30.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.222+0000 7fe3f2c00700 1 --2- 172.21.15.131:0/4004524640 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe3d00a11a0 0x7fe3d00a3660 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec100c40 tx=0x7fe3dc009260 comp rx=0 tx=0).stop 2024-05-08T22:05:30.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.222+0000 7fe3f2c00700 1 -- 172.21.15.131:0/4004524640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3ec1020f0 msgr2=0x7fe3ec199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:30.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.222+0000 7fe3f2c00700 1 --2- 172.21.15.131:0/4004524640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3ec1020f0 0x7fe3ec199780 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800e9c0 tx=0x7fe3d800ecd0 comp rx=0 tx=0).stop 2024-05-08T22:05:30.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.222+0000 7fe3f2c00700 1 -- 172.21.15.131:0/4004524640 shutdown_connections 2024-05-08T22:05:30.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.222+0000 7fe3f2c00700 1 --2- 172.21.15.131:0/4004524640 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe3d00a11a0 0x7fe3d00a3660 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:30.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.222+0000 7fe3f2c00700 1 --2- 172.21.15.131:0/4004524640 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3ec102ab0 0x7fe3ec1a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:30.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.222+0000 7fe3f2c00700 1 --2- 172.21.15.131:0/4004524640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3ec100f20 0x7fe3ec199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:30.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.222+0000 7fe3f2c00700 1 --2- 172.21.15.131:0/4004524640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3ec1020f0 0x7fe3ec199780 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:30.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.222+0000 7fe3f2c00700 1 -- 172.21.15.131:0/4004524640 >> 172.21.15.131:0/4004524640 conn(0x7fe3ec0fc500 msgr2=0x7fe3ec0731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:30.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.222+0000 7fe3f2c00700 1 -- 172.21.15.131:0/4004524640 shutdown_connections 2024-05-08T22:05:30.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.222+0000 7fe3f2c00700 1 -- 172.21.15.131:0/4004524640 wait complete. 2024-05-08T22:05:30.241 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:05:30.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.435+0000 7fce7b200700 1 -- 172.21.15.131:0/17238786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce7c100dc0 msgr2=0x7fce7c1011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:30.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.435+0000 7fce7b200700 1 --2- 172.21.15.131:0/17238786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce7c100dc0 0x7fce7c1011e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0099c0 tx=0x7fce6c009cd0 comp rx=0 tx=0).stop 2024-05-08T22:05:30.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.435+0000 7fce7b200700 1 -- 172.21.15.131:0/17238786 shutdown_connections 2024-05-08T22:05:30.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.435+0000 7fce7b200700 1 --2- 172.21.15.131:0/17238786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce7c102950 0x7fce7c109200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:30.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.435+0000 7fce7b200700 1 --2- 172.21.15.131:0/17238786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce7c100dc0 0x7fce7c1011e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:30.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.435+0000 7fce7b200700 1 --2- 172.21.15.131:0/17238786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c101fb0 0x7fce7c102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:30.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.435+0000 7fce7b200700 1 -- 172.21.15.131:0/17238786 >> 172.21.15.131:0/17238786 conn(0x7fce7c0fc330 msgr2=0x7fce7c0fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:30.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.436+0000 7fce7b200700 1 -- 172.21.15.131:0/17238786 shutdown_connections 2024-05-08T22:05:30.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.436+0000 7fce7b200700 1 -- 172.21.15.131:0/17238786 wait complete. 2024-05-08T22:05:30.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.436+0000 7fce7b200700 1 Processor -- start 2024-05-08T22:05:30.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.437+0000 7fce7b200700 1 -- start start 2024-05-08T22:05:30.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.437+0000 7fce7b200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce7c100dc0 0x7fce7c198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:30.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.437+0000 7fce7b200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce7c101fb0 0x7fce7c199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:30.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.437+0000 7fce7b200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c102950 0x7fce7c1a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:30.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.437+0000 7fce7b200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c1a0af0 con 0x7fce7c101fb0 2024-05-08T22:05:30.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.437+0000 7fce7b200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c1a0c30 con 0x7fce7c100dc0 2024-05-08T22:05:30.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.437+0000 7fce7b200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c1a0dc0 con 0x7fce7c102950 2024-05-08T22:05:30.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.437+0000 7fce79400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce7c101fb0 0x7fce7c199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:30.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.437+0000 7fce79e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce7c100dc0 0x7fce7c198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:30.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.437+0000 7fce79400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce7c101fb0 0x7fce7c199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57272/0 (socket says 172.21.15.131:57272) 2024-05-08T22:05:30.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.437+0000 7fce79400700 1 -- 172.21.15.131:0/1394432707 learned_addr learned my addr 172.21.15.131:0/1394432707 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:30.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.437+0000 7fce7a800700 1 --2- 172.21.15.131:0/1394432707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c102950 0x7fce7c1a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:30.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.438+0000 7fce79400700 1 -- 172.21.15.131:0/1394432707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c102950 msgr2=0x7fce7c1a0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:30.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.438+0000 7fce79400700 1 --2- 172.21.15.131:0/1394432707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c102950 0x7fce7c1a0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:30.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.438+0000 7fce79400700 1 -- 172.21.15.131:0/1394432707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce7c100dc0 msgr2=0x7fce7c198fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:30.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.438+0000 7fce79400700 1 --2- 172.21.15.131:0/1394432707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce7c100dc0 0x7fce7c198fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:30.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.438+0000 7fce79400700 1 -- 172.21.15.131:0/1394432707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c0095f0 con 0x7fce7c101fb0 2024-05-08T22:05:30.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.438+0000 7fce79e00700 1 --2- 172.21.15.131:0/1394432707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce7c100dc0 0x7fce7c198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:30.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.438+0000 7fce79400700 1 --2- 172.21.15.131:0/1394432707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce7c101fb0 0x7fce7c199500 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fce6800e9c0 tx=0x7fce6800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:30.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.438+0000 7fce72a00700 1 -- 172.21.15.131:0/1394432707 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce680097b0 con 0x7fce7c101fb0 2024-05-08T22:05:30.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.438+0000 7fce7b200700 1 -- 172.21.15.131:0/1394432707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce7c1a10a0 con 0x7fce7c101fb0 2024-05-08T22:05:30.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.438+0000 7fce72a00700 1 -- 172.21.15.131:0/1394432707 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce6800ee60 con 0x7fce7c101fb0 2024-05-08T22:05:30.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.438+0000 7fce72a00700 1 -- 172.21.15.131:0/1394432707 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce680054f0 con 0x7fce7c101fb0 2024-05-08T22:05:30.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.438+0000 7fce7b200700 1 -- 172.21.15.131:0/1394432707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7c1a1560 con 0x7fce7c101fb0 2024-05-08T22:05:30.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.439+0000 7fce7b200700 1 -- 172.21.15.131:0/1394432707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce7c067ba0 con 0x7fce7c101fb0 2024-05-08T22:05:30.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.441+0000 7fce72a00700 1 -- 172.21.15.131:0/1394432707 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fce68004750 con 0x7fce7c101fb0 2024-05-08T22:05:30.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.443+0000 7fce72a00700 1 --2- 172.21.15.131:0/1394432707 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fce600a1120 0x7fce600a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:30.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.443+0000 7fce72a00700 1 -- 172.21.15.131:0/1394432707 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fce68014070 con 0x7fce7c101fb0 2024-05-08T22:05:30.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.443+0000 7fce79e00700 1 --2- 172.21.15.131:0/1394432707 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fce600a1120 0x7fce600a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:30.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.443+0000 7fce79e00700 1 --2- 172.21.15.131:0/1394432707 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fce600a1120 0x7fce600a35e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009990 tx=0x7fce6c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:30.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.446+0000 7fce72a00700 1 -- 172.21.15.131:0/1394432707 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fce680962d0 con 0x7fce7c101fb0 2024-05-08T22:05:30.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.655+0000 7fce7b200700 1 -- 172.21.15.131:0/1394432707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fce7c1a17e0 con 0x7fce7c101fb0 2024-05-08T22:05:30.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.656+0000 7fce72a00700 1 -- 172.21.15.131:0/1394432707 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7fce680998f0 con 0x7fce7c101fb0 2024-05-08T22:05:30.657 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-08T22:05:30.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.658+0000 7fce7b200700 1 -- 172.21.15.131:0/1394432707 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fce600a1120 msgr2=0x7fce600a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:30.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.658+0000 7fce7b200700 1 --2- 172.21.15.131:0/1394432707 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fce600a1120 0x7fce600a35e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009990 tx=0x7fce6c01a040 comp rx=0 tx=0).stop 2024-05-08T22:05:30.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.658+0000 7fce7b200700 1 -- 172.21.15.131:0/1394432707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce7c101fb0 msgr2=0x7fce7c199500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:30.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.658+0000 7fce7b200700 1 --2- 172.21.15.131:0/1394432707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce7c101fb0 0x7fce7c199500 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fce6800e9c0 tx=0x7fce6800ecd0 comp rx=0 tx=0).stop 2024-05-08T22:05:30.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.658+0000 7fce7b200700 1 -- 172.21.15.131:0/1394432707 shutdown_connections 2024-05-08T22:05:30.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.658+0000 7fce7b200700 1 --2- 172.21.15.131:0/1394432707 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fce600a1120 0x7fce600a35e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:30.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.658+0000 7fce7b200700 1 --2- 172.21.15.131:0/1394432707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c102950 0x7fce7c1a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:30.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.658+0000 7fce7b200700 1 --2- 172.21.15.131:0/1394432707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce7c100dc0 0x7fce7c198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:30.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.658+0000 7fce7b200700 1 --2- 172.21.15.131:0/1394432707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce7c101fb0 0x7fce7c199500 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:30.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.658+0000 7fce7b200700 1 -- 172.21.15.131:0/1394432707 >> 172.21.15.131:0/1394432707 conn(0x7fce7c0fc330 msgr2=0x7fce7c107780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:30.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.658+0000 7fce7b200700 1 -- 172.21.15.131:0/1394432707 shutdown_connections 2024-05-08T22:05:30.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.658+0000 7fce7b200700 1 -- 172.21.15.131:0/1394432707 wait complete. 2024-05-08T22:05:30.678 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:05:30.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.877+0000 7f743ac00700 1 -- 172.21.15.131:0/1430185174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74341020f0 msgr2=0x7f7434102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:30.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.877+0000 7f743ac00700 1 --2- 172.21.15.131:0/1430185174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74341020f0 0x7f7434102570 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7420009990 tx=0x7f7420009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:30.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.877+0000 7f743ac00700 1 -- 172.21.15.131:0/1430185174 shutdown_connections 2024-05-08T22:05:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.877+0000 7f743ac00700 1 --2- 172.21.15.131:0/1430185174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7434102ab0 0x7f74341093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.877+0000 7f743ac00700 1 --2- 172.21.15.131:0/1430185174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7434100f20 0x7f7434101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.877+0000 7f743ac00700 1 --2- 172.21.15.131:0/1430185174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74341020f0 0x7f7434102570 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.877+0000 7f743ac00700 1 -- 172.21.15.131:0/1430185174 >> 172.21.15.131:0/1430185174 conn(0x7f74340fc500 msgr2=0x7f74340fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.877+0000 7f743ac00700 1 -- 172.21.15.131:0/1430185174 shutdown_connections 2024-05-08T22:05:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.877+0000 7f743ac00700 1 -- 172.21.15.131:0/1430185174 wait complete. 2024-05-08T22:05:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.878+0000 7f743ac00700 1 Processor -- start 2024-05-08T22:05:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.879+0000 7f743ac00700 1 -- start start 2024-05-08T22:05:30.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.879+0000 7f743ac00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7434100f20 0x7f7434074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:30.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.879+0000 7f743ac00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74341020f0 0x7f7434073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:30.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.879+0000 7f743ac00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7434102ab0 0x7f74340738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.879+0000 7f743ac00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7434074070 con 0x7f7434102ab0 2024-05-08T22:05:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.879+0000 7f743ac00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74340741e0 con 0x7f74341020f0 2024-05-08T22:05:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.879+0000 7f743ac00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74341a0f60 con 0x7f7434100f20 2024-05-08T22:05:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.879+0000 7f7438c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7434102ab0 0x7f74340738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.879+0000 7f7433e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7434100f20 0x7f7434074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.879+0000 7f7438c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7434102ab0 0x7f74340738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57304/0 (socket says 172.21.15.131:57304) 2024-05-08T22:05:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.879+0000 7f7433e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7434100f20 0x7f7434074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46864/0 (socket says 172.21.15.131:46864) 2024-05-08T22:05:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.879+0000 7f7438c00700 1 -- 172.21.15.131:0/3479686185 learned_addr learned my addr 172.21.15.131:0/3479686185 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.879+0000 7f7433400700 1 --2- 172.21.15.131:0/3479686185 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74341020f0 0x7f7434073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.880+0000 7f7438c00700 1 -- 172.21.15.131:0/3479686185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7434100f20 msgr2=0x7f7434074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.880+0000 7f7438c00700 1 --2- 172.21.15.131:0/3479686185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7434100f20 0x7f7434074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.880+0000 7f7438c00700 1 -- 172.21.15.131:0/3479686185 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74341020f0 msgr2=0x7f7434073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.880+0000 7f7438c00700 1 --2- 172.21.15.131:0/3479686185 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74341020f0 0x7f7434073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.880+0000 7f7438c00700 1 -- 172.21.15.131:0/3479686185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74200095f0 con 0x7f7434102ab0 2024-05-08T22:05:30.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.880+0000 7f7438c00700 1 --2- 172.21.15.131:0/3479686185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7434102ab0 0x7f74340738d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f742c00ef30 tx=0x7f742c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:30.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.880+0000 7f7430c00700 1 -- 172.21.15.131:0/3479686185 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c018070 con 0x7f7434102ab0 2024-05-08T22:05:30.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.880+0000 7f7430c00700 1 -- 172.21.15.131:0/3479686185 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f742c00f040 con 0x7f7434102ab0 2024-05-08T22:05:30.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.880+0000 7f743ac00700 1 -- 172.21.15.131:0/3479686185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74341a1190 con 0x7f7434102ab0 2024-05-08T22:05:30.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.880+0000 7f7430c00700 1 -- 172.21.15.131:0/3479686185 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c00cca0 con 0x7f7434102ab0 2024-05-08T22:05:30.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.880+0000 7f743ac00700 1 -- 172.21.15.131:0/3479686185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74341a16f0 con 0x7f7434102ab0 2024-05-08T22:05:30.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.882+0000 7f743ac00700 1 -- 172.21.15.131:0/3479686185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7434108430 con 0x7f7434102ab0 2024-05-08T22:05:30.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.883+0000 7f7430c00700 1 -- 172.21.15.131:0/3479686185 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f742c003a00 con 0x7f7434102ab0 2024-05-08T22:05:30.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.884+0000 7f7430c00700 1 --2- 172.21.15.131:0/3479686185 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f74180a11a0 0x7f74180a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:30.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.885+0000 7f7430c00700 1 -- 172.21.15.131:0/3479686185 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f742c0cc560 con 0x7f7434102ab0 2024-05-08T22:05:30.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.885+0000 7f7433e00700 1 --2- 172.21.15.131:0/3479686185 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f74180a11a0 0x7f74180a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:30.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.885+0000 7f7433e00700 1 --2- 172.21.15.131:0/3479686185 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f74180a11a0 0x7f74180a3660 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7434100c40 tx=0x7f7424009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:30.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:30.889+0000 7f7430c00700 1 -- 172.21.15.131:0/3479686185 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f742c095b30 con 0x7f7434102ab0 2024-05-08T22:05:31.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:30 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3274160572' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:05:31.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:30 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/4004524640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:05:31.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:30 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1394432707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:05:31.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.097+0000 7f743ac00700 1 -- 172.21.15.131:0/3479686185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f743410ca00 con 0x7f7434102ab0 2024-05-08T22:05:31.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.097+0000 7f7430c00700 1 -- 172.21.15.131:0/3479686185 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v83) v1 ==== 103+0+11 (secure 0 0 0) 0x7f742c099150 con 0x7f7434102ab0 2024-05-08T22:05:31.099 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-08T22:05:31.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.100+0000 7f743ac00700 1 -- 172.21.15.131:0/3479686185 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f74180a11a0 msgr2=0x7f74180a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:31.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.100+0000 7f743ac00700 1 --2- 172.21.15.131:0/3479686185 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f74180a11a0 0x7f74180a3660 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7434100c40 tx=0x7f7424009260 comp rx=0 tx=0).stop 2024-05-08T22:05:31.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.100+0000 7f743ac00700 1 -- 172.21.15.131:0/3479686185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7434102ab0 msgr2=0x7f74340738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:31.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.100+0000 7f743ac00700 1 --2- 172.21.15.131:0/3479686185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7434102ab0 0x7f74340738d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f742c00ef30 tx=0x7f742c00c3a0 comp rx=0 tx=0).stop 2024-05-08T22:05:31.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.100+0000 7f743ac00700 1 -- 172.21.15.131:0/3479686185 shutdown_connections 2024-05-08T22:05:31.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.100+0000 7f743ac00700 1 --2- 172.21.15.131:0/3479686185 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f74180a11a0 0x7f74180a3660 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:31.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.100+0000 7f743ac00700 1 --2- 172.21.15.131:0/3479686185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7434102ab0 0x7f74340738d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:31.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.100+0000 7f743ac00700 1 --2- 172.21.15.131:0/3479686185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7434100f20 0x7f7434074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:31.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.100+0000 7f743ac00700 1 --2- 172.21.15.131:0/3479686185 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74341020f0 0x7f7434073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:31.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.100+0000 7f743ac00700 1 -- 172.21.15.131:0/3479686185 >> 172.21.15.131:0/3479686185 conn(0x7f74340fc500 msgr2=0x7f7434105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:31.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.100+0000 7f743ac00700 1 -- 172.21.15.131:0/3479686185 shutdown_connections 2024-05-08T22:05:31.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.101+0000 7f743ac00700 1 -- 172.21.15.131:0/3479686185 wait complete. 2024-05-08T22:05:31.118 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:30 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3274160572' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:05:31.118 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:30 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/4004524640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:05:31.118 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:30 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1394432707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:05:31.121 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:05:31.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:30 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3274160572' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:05:31.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:30 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/4004524640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:05:31.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:30 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1394432707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:05:31.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.313+0000 7f3813200700 1 -- 172.21.15.131:0/1968379950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3814106aa0 msgr2=0x7f3814108f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:31.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.313+0000 7f3813200700 1 --2- 172.21.15.131:0/1968379950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3814106aa0 0x7f3814108f10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f380c00b0a0 tx=0x7f380c00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:05:31.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.313+0000 7f3813200700 1 -- 172.21.15.131:0/1968379950 shutdown_connections 2024-05-08T22:05:31.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.313+0000 7f3813200700 1 --2- 172.21.15.131:0/1968379950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3814106aa0 0x7f3814108f10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:31.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.313+0000 7f3813200700 1 --2- 172.21.15.131:0/1968379950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3814101840 0x7f3814103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:31.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.313+0000 7f3813200700 1 --2- 172.21.15.131:0/1968379950 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3814104170 0x7f3814106560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:31.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.313+0000 7f3813200700 1 -- 172.21.15.131:0/1968379950 >> 172.21.15.131:0/1968379950 conn(0x7f38140fb1f0 msgr2=0x7f38140fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:31.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.313+0000 7f3813200700 1 -- 172.21.15.131:0/1968379950 shutdown_connections 2024-05-08T22:05:31.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.313+0000 7f3813200700 1 -- 172.21.15.131:0/1968379950 wait complete. 2024-05-08T22:05:31.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.314+0000 7f3813200700 1 Processor -- start 2024-05-08T22:05:31.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.314+0000 7f3813200700 1 -- start start 2024-05-08T22:05:31.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.315+0000 7f3813200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3814101840 0x7f3814101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:31.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.315+0000 7f3813200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3814104170 0x7f38140ff7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:31.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.315+0000 7f3813200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3814106aa0 0x7f38140ffd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:31.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.315+0000 7f3813200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38141004d0 con 0x7f3814101840 2024-05-08T22:05:31.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.315+0000 7f3813200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3814100640 con 0x7f3814106aa0 2024-05-08T22:05:31.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.315+0000 7f3813200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38141a0e90 con 0x7f3814104170 2024-05-08T22:05:31.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.315+0000 7f3811e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3814101840 0x7f3814101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:31.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.315+0000 7f3811e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3814101840 0x7f3814101160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57314/0 (socket says 172.21.15.131:57314) 2024-05-08T22:05:31.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.315+0000 7f3811e00700 1 -- 172.21.15.131:0/22193923 learned_addr learned my addr 172.21.15.131:0/22193923 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:31.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.315+0000 7f3811400700 1 --2- 172.21.15.131:0/22193923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3814104170 0x7f38140ff7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:31.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.315+0000 7f3812800700 1 --2- 172.21.15.131:0/22193923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3814106aa0 0x7f38140ffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:31.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.315+0000 7f3811e00700 1 -- 172.21.15.131:0/22193923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3814104170 msgr2=0x7f38140ff7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:31.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.315+0000 7f3811e00700 1 --2- 172.21.15.131:0/22193923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3814104170 0x7f38140ff7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:31.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.315+0000 7f3811e00700 1 -- 172.21.15.131:0/22193923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3814106aa0 msgr2=0x7f38140ffd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:31.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.315+0000 7f3811e00700 1 --2- 172.21.15.131:0/22193923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3814106aa0 0x7f38140ffd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:31.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.315+0000 7f3811e00700 1 -- 172.21.15.131:0/22193923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3800009520 con 0x7f3814101840 2024-05-08T22:05:31.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.315+0000 7f3812800700 1 --2- 172.21.15.131:0/22193923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3814106aa0 0x7f38140ffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:05:31.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.316+0000 7f3811e00700 1 --2- 172.21.15.131:0/22193923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3814101840 0x7f3814101160 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f380400e9a0 tx=0x7f380400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:31.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.316+0000 7f380aa00700 1 -- 172.21.15.131:0/22193923 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3804009790 con 0x7f3814101840 2024-05-08T22:05:31.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.316+0000 7f380aa00700 1 -- 172.21.15.131:0/22193923 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f380400cb70 con 0x7f3814101840 2024-05-08T22:05:31.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.316+0000 7f3813200700 1 -- 172.21.15.131:0/22193923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f380c009d00 con 0x7f3814101840 2024-05-08T22:05:31.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.316+0000 7f380aa00700 1 -- 172.21.15.131:0/22193923 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3804010ea0 con 0x7f3814101840 2024-05-08T22:05:31.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.316+0000 7f3813200700 1 -- 172.21.15.131:0/22193923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38141a14a0 con 0x7f3814101840 2024-05-08T22:05:31.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.317+0000 7f3813200700 1 -- 172.21.15.131:0/22193923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38140fcd70 con 0x7f3814101840 2024-05-08T22:05:31.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.319+0000 7f380aa00700 1 -- 172.21.15.131:0/22193923 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f380400cce0 con 0x7f3814101840 2024-05-08T22:05:31.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.320+0000 7f380aa00700 1 --2- 172.21.15.131:0/22193923 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f37f80a1120 0x7f37f80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:31.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.320+0000 7f380aa00700 1 -- 172.21.15.131:0/22193923 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3804014070 con 0x7f3814101840 2024-05-08T22:05:31.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.320+0000 7f3811400700 1 --2- 172.21.15.131:0/22193923 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f37f80a1120 0x7f37f80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:31.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.321+0000 7f3811400700 1 --2- 172.21.15.131:0/22193923 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f37f80a1120 0x7f37f80a35e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3800009df0 tx=0x7f3800009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:31.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.322+0000 7f380aa00700 1 -- 172.21.15.131:0/22193923 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f38040967d0 con 0x7f3814101840 2024-05-08T22:05:31.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.544+0000 7f3813200700 1 -- 172.21.15.131:0/22193923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f381410c8a0 con 0x7f3814101840 2024-05-08T22:05:31.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.544+0000 7f380aa00700 1 -- 172.21.15.131:0/22193923 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v83) v1 ==== 81+0+601 (secure 0 0 0) 0x7f3804099df0 con 0x7f3814101840 2024-05-08T22:05:31.546 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6801/1418340798 2024-05-09T21:59:40.641373+0000 2024-05-08T22:05:31.546 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6800/1418340798 2024-05-09T21:59:40.641373+0000 2024-05-08T22:05:31.546 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/1241655400 2024-05-09T21:59:40.641373+0000 2024-05-08T22:05:31.546 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/1716895483 2024-05-09T21:59:40.641373+0000 2024-05-08T22:05:31.546 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/1981506111 2024-05-09T21:59:15.548383+0000 2024-05-08T22:05:31.546 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/976983862 2024-05-09T21:59:15.548383+0000 2024-05-08T22:05:31.546 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6801/1564023164 2024-05-09T21:59:15.548383+0000 2024-05-08T22:05:31.546 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/3309822030 2024-05-09T21:59:15.548383+0000 2024-05-08T22:05:31.546 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/3082367829 2024-05-09T21:59:40.641373+0000 2024-05-08T22:05:31.546 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6800/1564023164 2024-05-09T21:59:15.548383+0000 2024-05-08T22:05:31.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.547+0000 7f3813200700 1 -- 172.21.15.131:0/22193923 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f37f80a1120 msgr2=0x7f37f80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:31.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.547+0000 7f3813200700 1 --2- 172.21.15.131:0/22193923 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f37f80a1120 0x7f37f80a35e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3800009df0 tx=0x7f3800009260 comp rx=0 tx=0).stop 2024-05-08T22:05:31.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.547+0000 7f3813200700 1 -- 172.21.15.131:0/22193923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3814101840 msgr2=0x7f3814101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:31.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.547+0000 7f3813200700 1 --2- 172.21.15.131:0/22193923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3814101840 0x7f3814101160 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f380400e9a0 tx=0x7f380400ecb0 comp rx=0 tx=0).stop 2024-05-08T22:05:31.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.547+0000 7f3813200700 1 -- 172.21.15.131:0/22193923 shutdown_connections 2024-05-08T22:05:31.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.547+0000 7f3813200700 1 --2- 172.21.15.131:0/22193923 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f37f80a1120 0x7f37f80a35e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:31.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.547+0000 7f3813200700 1 --2- 172.21.15.131:0/22193923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3814106aa0 0x7f38140ffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:31.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.547+0000 7f3813200700 1 --2- 172.21.15.131:0/22193923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3814101840 0x7f3814101160 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:31.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.547+0000 7f3813200700 1 --2- 172.21.15.131:0/22193923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3814104170 0x7f38140ff7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:31.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.547+0000 7f3813200700 1 -- 172.21.15.131:0/22193923 >> 172.21.15.131:0/22193923 conn(0x7f38140fb1f0 msgr2=0x7f3814102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:31.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.547+0000 7f3813200700 1 -- 172.21.15.131:0/22193923 shutdown_connections 2024-05-08T22:05:31.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.547+0000 7f3813200700 1 -- 172.21.15.131:0/22193923 wait complete. 2024-05-08T22:05:31.549 INFO:teuthology.orchestra.run.smithi131.stderr:listed 10 entries 2024-05-08T22:05:31.567 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-08T22:05:31.567 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:05:31.610 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:05:31.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.842+0000 7ff804000700 1 -- 172.21.15.131:0/3515237734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7fc073a00 msgr2=0x7ff7fc073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:31.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.842+0000 7ff804000700 1 --2- 172.21.15.131:0/3515237734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7fc073a00 0x7ff7fc073e80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009990 tx=0x7ff7e8009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:31.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.843+0000 7ff804000700 1 -- 172.21.15.131:0/3515237734 shutdown_connections 2024-05-08T22:05:31.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.843+0000 7ff804000700 1 --2- 172.21.15.131:0/3515237734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7fc108d30 0x7ff7fc10b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:31.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.843+0000 7ff804000700 1 --2- 172.21.15.131:0/3515237734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7fc074fe0 0x7ff7fc073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:31.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.843+0000 7ff804000700 1 --2- 172.21.15.131:0/3515237734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7fc073a00 0x7ff7fc073e80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:31.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.843+0000 7ff804000700 1 -- 172.21.15.131:0/3515237734 >> 172.21.15.131:0/3515237734 conn(0x7ff7fc0fc3e0 msgr2=0x7ff7fc0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:31.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.843+0000 7ff804000700 1 -- 172.21.15.131:0/3515237734 shutdown_connections 2024-05-08T22:05:31.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.843+0000 7ff804000700 1 -- 172.21.15.131:0/3515237734 wait complete. 2024-05-08T22:05:31.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.844+0000 7ff804000700 1 Processor -- start 2024-05-08T22:05:31.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.844+0000 7ff804000700 1 -- start start 2024-05-08T22:05:31.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.844+0000 7ff804000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7fc073a00 0x7ff7fc199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:31.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.844+0000 7ff804000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7fc074fe0 0x7ff7fc199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:31.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.844+0000 7ff804000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7fc108d30 0x7ff7fc1a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:31.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.844+0000 7ff804000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc1a0b50 con 0x7ff7fc108d30 2024-05-08T22:05:31.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.844+0000 7ff804000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc1a0c90 con 0x7ff7fc073a00 2024-05-08T22:05:31.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.844+0000 7ff804000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc1a0e20 con 0x7ff7fc074fe0 2024-05-08T22:05:31.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.844+0000 7ff802000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7fc108d30 0x7ff7fc1a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.844+0000 7ff801600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7fc073a00 0x7ff7fc199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.844+0000 7ff801600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7fc073a00 0x7ff7fc199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:59944/0 (socket says 172.21.15.131:59944) 2024-05-08T22:05:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.844+0000 7ff801600700 1 -- 172.21.15.131:0/2174829708 learned_addr learned my addr 172.21.15.131:0/2174829708 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.844+0000 7ff800c00700 1 --2- 172.21.15.131:0/2174829708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7fc074fe0 0x7ff7fc199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.845+0000 7ff801600700 1 -- 172.21.15.131:0/2174829708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7fc074fe0 msgr2=0x7ff7fc199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.845+0000 7ff801600700 1 --2- 172.21.15.131:0/2174829708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7fc074fe0 0x7ff7fc199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.845+0000 7ff801600700 1 -- 172.21.15.131:0/2174829708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7fc108d30 msgr2=0x7ff7fc1a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.845+0000 7ff801600700 1 --2- 172.21.15.131:0/2174829708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7fc108d30 0x7ff7fc1a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.845+0000 7ff801600700 1 -- 172.21.15.131:0/2174829708 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f0009520 con 0x7ff7fc073a00 2024-05-08T22:05:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.845+0000 7ff801600700 1 --2- 172.21.15.131:0/2174829708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7fc073a00 0x7ff7fc199020 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff7f000eb30 tx=0x7ff7f000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.845+0000 7ff7fa000700 1 -- 172.21.15.131:0/2174829708 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0009880 con 0x7ff7fc073a00 2024-05-08T22:05:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.845+0000 7ff7fa000700 1 -- 172.21.15.131:0/2174829708 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7f00044e0 con 0x7ff7fc073a00 2024-05-08T22:05:31.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.845+0000 7ff7fa000700 1 -- 172.21.15.131:0/2174829708 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0003d80 con 0x7ff7fc073a00 2024-05-08T22:05:31.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.845+0000 7ff804000700 1 -- 172.21.15.131:0/2174829708 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e80095f0 con 0x7ff7fc073a00 2024-05-08T22:05:31.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.845+0000 7ff804000700 1 -- 172.21.15.131:0/2174829708 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7fc1a1480 con 0x7ff7fc073a00 2024-05-08T22:05:31.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.846+0000 7ff804000700 1 -- 172.21.15.131:0/2174829708 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7fc107af0 con 0x7ff7fc073a00 2024-05-08T22:05:31.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.848+0000 7ff7fa000700 1 -- 172.21.15.131:0/2174829708 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff7f0003f20 con 0x7ff7fc073a00 2024-05-08T22:05:31.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.849+0000 7ff7fa000700 1 --2- 172.21.15.131:0/2174829708 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff7e40a11a0 0x7ff7e40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:31.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.850+0000 7ff7fa000700 1 -- 172.21.15.131:0/2174829708 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff7f0014070 con 0x7ff7fc073a00 2024-05-08T22:05:31.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.850+0000 7ff800c00700 1 --2- 172.21.15.131:0/2174829708 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff7e40a11a0 0x7ff7e40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:31.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.850+0000 7ff800c00700 1 --2- 172.21.15.131:0/2174829708 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff7e40a11a0 0x7ff7e40a3660 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800b680 tx=0x7ff7e8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:31.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:31.854+0000 7ff7fa000700 1 -- 172.21.15.131:0/2174829708 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff7f0096530 con 0x7ff7fc073a00 2024-05-08T22:05:31.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:31 smithi131 ceph-mon[37260]: pgmap v248: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 87 B/s wr, 0 op/s 2024-05-08T22:05:31.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:31 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3479686185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:05:31.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:31 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/22193923' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T22:05:32.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.065+0000 7ff804000700 1 -- 172.21.15.131:0/2174829708 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7fc067c10 con 0x7ff7fc073a00 2024-05-08T22:05:32.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.067+0000 7ff7fa000700 1 -- 172.21.15.131:0/2174829708 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+24007 (secure 0 0 0) 0x7ff7f0099b50 con 0x7ff7fc073a00 2024-05-08T22:05:32.068 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:32.069 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":83,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:05:19.013678+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:05:12.469858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:05:13.484446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:05:17.016186+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:05:32.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.070+0000 7ff804000700 1 -- 172.21.15.131:0/2174829708 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff7e40a11a0 msgr2=0x7ff7e40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:32.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.070+0000 7ff804000700 1 --2- 172.21.15.131:0/2174829708 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff7e40a11a0 0x7ff7e40a3660 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800b680 tx=0x7ff7e8005fd0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.070+0000 7ff804000700 1 -- 172.21.15.131:0/2174829708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7fc073a00 msgr2=0x7ff7fc199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:32.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.070+0000 7ff804000700 1 --2- 172.21.15.131:0/2174829708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7fc073a00 0x7ff7fc199020 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff7f000eb30 tx=0x7ff7f000ee40 comp rx=0 tx=0).stop 2024-05-08T22:05:32.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.070+0000 7ff804000700 1 -- 172.21.15.131:0/2174829708 shutdown_connections 2024-05-08T22:05:32.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.070+0000 7ff804000700 1 --2- 172.21.15.131:0/2174829708 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff7e40a11a0 0x7ff7e40a3660 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.070+0000 7ff804000700 1 --2- 172.21.15.131:0/2174829708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7fc108d30 0x7ff7fc1a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.070+0000 7ff804000700 1 --2- 172.21.15.131:0/2174829708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7fc073a00 0x7ff7fc199020 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.070+0000 7ff804000700 1 --2- 172.21.15.131:0/2174829708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7fc074fe0 0x7ff7fc199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.070+0000 7ff804000700 1 -- 172.21.15.131:0/2174829708 >> 172.21.15.131:0/2174829708 conn(0x7ff7fc0fc3e0 msgr2=0x7ff7fc104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:32.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.070+0000 7ff804000700 1 -- 172.21.15.131:0/2174829708 shutdown_connections 2024-05-08T22:05:32.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.070+0000 7ff804000700 1 -- 172.21.15.131:0/2174829708 wait complete. 2024-05-08T22:05:32.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:31 smithi132 ceph-mon[78180]: pgmap v248: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 87 B/s wr, 0 op/s 2024-05-08T22:05:32.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:31 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3479686185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:05:32.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:31 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/22193923' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T22:05:32.092 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:01:41.271078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:05:12.469858+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:05:13.484446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:05:17.016186+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:05:32.092 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:05:32.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:31 smithi157 ceph-mon[77746]: pgmap v248: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 87 B/s wr, 0 op/s 2024-05-08T22:05:32.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:31 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3479686185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:05:32.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:31 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/22193923' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T22:05:32.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.286+0000 7f944c600700 1 -- 172.21.15.131:0/3692461930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9444104310 msgr2=0x7f9444106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:32.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.286+0000 7f944c600700 1 --2- 172.21.15.131:0/3692461930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9444104310 0x7f9444106700 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9430009990 tx=0x7f9430009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.286+0000 7f944c600700 1 -- 172.21.15.131:0/3692461930 shutdown_connections 2024-05-08T22:05:32.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.286+0000 7f944c600700 1 --2- 172.21.15.131:0/3692461930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9444106c40 0x7f94441090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.286+0000 7f944c600700 1 --2- 172.21.15.131:0/3692461930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94441019e0 0x7f9444103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.286+0000 7f944c600700 1 --2- 172.21.15.131:0/3692461930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9444104310 0x7f9444106700 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.286+0000 7f944c600700 1 -- 172.21.15.131:0/3692461930 >> 172.21.15.131:0/3692461930 conn(0x7f94440fb3f0 msgr2=0x7f94440fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:32.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.286+0000 7f944c600700 1 -- 172.21.15.131:0/3692461930 shutdown_connections 2024-05-08T22:05:32.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.286+0000 7f944c600700 1 -- 172.21.15.131:0/3692461930 wait complete. 2024-05-08T22:05:32.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.287+0000 7f944c600700 1 Processor -- start 2024-05-08T22:05:32.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.287+0000 7f944c600700 1 -- start start 2024-05-08T22:05:32.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.288+0000 7f944c600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94441019e0 0x7f94441990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:32.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.288+0000 7f944c600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9444104310 0x7f9444199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:32.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.288+0000 7f944c600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9444106c40 0x7f94441a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:32.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.288+0000 7f944c600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94441a0c10 con 0x7f94441019e0 2024-05-08T22:05:32.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.288+0000 7f944c600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94441a0d50 con 0x7f9444106c40 2024-05-08T22:05:32.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.288+0000 7f944c600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94441a0ee0 con 0x7f9444104310 2024-05-08T22:05:32.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.288+0000 7f9449c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94441019e0 0x7f94441990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:32.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.288+0000 7f9449c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94441019e0 0x7f94441990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57332/0 (socket says 172.21.15.131:57332) 2024-05-08T22:05:32.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.288+0000 7f9449c00700 1 -- 172.21.15.131:0/4292574319 learned_addr learned my addr 172.21.15.131:0/4292574319 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:32.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.288+0000 7f9449200700 1 --2- 172.21.15.131:0/4292574319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9444104310 0x7f9444199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:32.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.288+0000 7f944a600700 1 --2- 172.21.15.131:0/4292574319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9444106c40 0x7f94441a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:32.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.288+0000 7f9449c00700 1 -- 172.21.15.131:0/4292574319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9444104310 msgr2=0x7f9444199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:32.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.288+0000 7f9449c00700 1 --2- 172.21.15.131:0/4292574319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9444104310 0x7f9444199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.288+0000 7f9449c00700 1 -- 172.21.15.131:0/4292574319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9444106c40 msgr2=0x7f94441a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:32.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.288+0000 7f9449c00700 1 --2- 172.21.15.131:0/4292574319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9444106c40 0x7f94441a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.288+0000 7f9449c00700 1 -- 172.21.15.131:0/4292574319 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94300095f0 con 0x7f94441019e0 2024-05-08T22:05:32.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.289+0000 7f9449c00700 1 --2- 172.21.15.131:0/4292574319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94441019e0 0x7f94441990e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f943800e9a0 tx=0x7f943800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:32.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.289+0000 7f9442000700 1 -- 172.21.15.131:0/4292574319 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438009790 con 0x7f94441019e0 2024-05-08T22:05:32.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.289+0000 7f9442000700 1 -- 172.21.15.131:0/4292574319 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9438010430 con 0x7f94441019e0 2024-05-08T22:05:32.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.289+0000 7f9442000700 1 -- 172.21.15.131:0/4292574319 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438018680 con 0x7f94441019e0 2024-05-08T22:05:32.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.289+0000 7f944c600700 1 -- 172.21.15.131:0/4292574319 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94441a11c0 con 0x7f94441019e0 2024-05-08T22:05:32.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.289+0000 7f944c600700 1 -- 172.21.15.131:0/4292574319 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94441a1680 con 0x7f94441019e0 2024-05-08T22:05:32.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.290+0000 7f944c600700 1 -- 172.21.15.131:0/4292574319 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9444108810 con 0x7f94441019e0 2024-05-08T22:05:32.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.292+0000 7f9442000700 1 -- 172.21.15.131:0/4292574319 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f94380105a0 con 0x7f94441019e0 2024-05-08T22:05:32.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.293+0000 7f9442000700 1 --2- 172.21.15.131:0/4292574319 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f942c0a1120 0x7f942c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:32.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.293+0000 7f9442000700 1 -- 172.21.15.131:0/4292574319 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9438014070 con 0x7f94441019e0 2024-05-08T22:05:32.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.294+0000 7f9449200700 1 --2- 172.21.15.131:0/4292574319 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f942c0a1120 0x7f942c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:32.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.294+0000 7f9449200700 1 --2- 172.21.15.131:0/4292574319 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f942c0a1120 0x7f942c0a35e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f943000b2e0 tx=0x7f9430005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:32.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.297+0000 7f9442000700 1 -- 172.21.15.131:0/4292574319 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9438052d60 con 0x7f94441019e0 2024-05-08T22:05:32.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.504+0000 7f944c600700 1 -- 172.21.15.131:0/4292574319 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f944410ca50 con 0x7f94441019e0 2024-05-08T22:05:32.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.505+0000 7f9442000700 1 -- 172.21.15.131:0/4292574319 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9438096230 con 0x7f94441019e0 2024-05-08T22:05:32.506 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-08T22:05:32.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.507+0000 7f944c600700 1 -- 172.21.15.131:0/4292574319 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f942c0a1120 msgr2=0x7f942c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:32.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.507+0000 7f944c600700 1 --2- 172.21.15.131:0/4292574319 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f942c0a1120 0x7f942c0a35e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f943000b2e0 tx=0x7f9430005fd0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.507+0000 7f944c600700 1 -- 172.21.15.131:0/4292574319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94441019e0 msgr2=0x7f94441990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:32.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.507+0000 7f944c600700 1 --2- 172.21.15.131:0/4292574319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94441019e0 0x7f94441990e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f943800e9a0 tx=0x7f943800ecb0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.507+0000 7f944c600700 1 -- 172.21.15.131:0/4292574319 shutdown_connections 2024-05-08T22:05:32.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.507+0000 7f944c600700 1 --2- 172.21.15.131:0/4292574319 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f942c0a1120 0x7f942c0a35e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.507+0000 7f944c600700 1 --2- 172.21.15.131:0/4292574319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9444106c40 0x7f94441a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.507+0000 7f944c600700 1 --2- 172.21.15.131:0/4292574319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94441019e0 0x7f94441990e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.507+0000 7f944c600700 1 --2- 172.21.15.131:0/4292574319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9444104310 0x7f9444199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.507+0000 7f944c600700 1 -- 172.21.15.131:0/4292574319 >> 172.21.15.131:0/4292574319 conn(0x7f94440fb3f0 msgr2=0x7f9444101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:32.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.507+0000 7f944c600700 1 -- 172.21.15.131:0/4292574319 shutdown_connections 2024-05-08T22:05:32.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.507+0000 7f944c600700 1 -- 172.21.15.131:0/4292574319 wait complete. 2024-05-08T22:05:32.527 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:05:32.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.723+0000 7fdff8e00700 1 -- 172.21.15.131:0/2925923255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdff4100db0 msgr2=0x7fdff41011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:32.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.723+0000 7fdff8e00700 1 --2- 172.21.15.131:0/2925923255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdff4100db0 0x7fdff41011d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40099c0 tx=0x7fdfe4009cd0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.724+0000 7fdff8e00700 1 -- 172.21.15.131:0/2925923255 shutdown_connections 2024-05-08T22:05:32.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.724+0000 7fdff8e00700 1 --2- 172.21.15.131:0/2925923255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdff4102940 0x7fdff41091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.724+0000 7fdff8e00700 1 --2- 172.21.15.131:0/2925923255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdff4100db0 0x7fdff41011d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.724+0000 7fdff8e00700 1 --2- 172.21.15.131:0/2925923255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdff4101fa0 0x7fdff4102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.724+0000 7fdff8e00700 1 -- 172.21.15.131:0/2925923255 >> 172.21.15.131:0/2925923255 conn(0x7fdff40fc2e0 msgr2=0x7fdff40fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:32.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.724+0000 7fdff8e00700 1 -- 172.21.15.131:0/2925923255 shutdown_connections 2024-05-08T22:05:32.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.724+0000 7fdff8e00700 1 -- 172.21.15.131:0/2925923255 wait complete. 2024-05-08T22:05:32.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.725+0000 7fdff8e00700 1 Processor -- start 2024-05-08T22:05:32.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.725+0000 7fdff8e00700 1 -- start start 2024-05-08T22:05:32.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.725+0000 7fdff8e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdff4100db0 0x7fdff4198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:32.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.725+0000 7fdff8e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdff4101fa0 0x7fdff4199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:32.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.725+0000 7fdff8e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdff4102940 0x7fdff41a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:32.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.725+0000 7fdff8e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff41a0ac0 con 0x7fdff4101fa0 2024-05-08T22:05:32.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.725+0000 7fdff8e00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff41a0c00 con 0x7fdff4100db0 2024-05-08T22:05:32.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.725+0000 7fdff8e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff41a0d90 con 0x7fdff4102940 2024-05-08T22:05:32.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.725+0000 7fdff2a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdff4101fa0 0x7fdff4199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:32.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.726+0000 7fdff2a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdff4101fa0 0x7fdff4199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57348/0 (socket says 172.21.15.131:57348) 2024-05-08T22:05:32.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.726+0000 7fdff2a00700 1 -- 172.21.15.131:0/1647920522 learned_addr learned my addr 172.21.15.131:0/1647920522 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:32.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.726+0000 7fdff3e00700 1 --2- 172.21.15.131:0/1647920522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdff4102940 0x7fdff41a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:32.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.726+0000 7fdff3400700 1 --2- 172.21.15.131:0/1647920522 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdff4100db0 0x7fdff4198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:32.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.726+0000 7fdff2a00700 1 -- 172.21.15.131:0/1647920522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdff4102940 msgr2=0x7fdff41a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:32.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.726+0000 7fdff2a00700 1 --2- 172.21.15.131:0/1647920522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdff4102940 0x7fdff41a04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.726+0000 7fdff2a00700 1 -- 172.21.15.131:0/1647920522 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdff4100db0 msgr2=0x7fdff4198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:32.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.726+0000 7fdff2a00700 1 --2- 172.21.15.131:0/1647920522 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdff4100db0 0x7fdff4198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.726+0000 7fdff2a00700 1 -- 172.21.15.131:0/1647920522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec009c40 con 0x7fdff4101fa0 2024-05-08T22:05:32.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.726+0000 7fdff2a00700 1 --2- 172.21.15.131:0/1647920522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdff4101fa0 0x7fdff4199520 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000e9a0 tx=0x7fdfe000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:32.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.726+0000 7fdfebe00700 1 -- 172.21.15.131:0/1647920522 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0009790 con 0x7fdff4101fa0 2024-05-08T22:05:32.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.726+0000 7fdfebe00700 1 -- 172.21.15.131:0/1647920522 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdfe000ce80 con 0x7fdff4101fa0 2024-05-08T22:05:32.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.726+0000 7fdff8e00700 1 -- 172.21.15.131:0/1647920522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe40095f0 con 0x7fdff4101fa0 2024-05-08T22:05:32.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.726+0000 7fdfebe00700 1 -- 172.21.15.131:0/1647920522 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0018680 con 0x7fdff4101fa0 2024-05-08T22:05:32.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.727+0000 7fdff8e00700 1 -- 172.21.15.131:0/1647920522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff41a13f0 con 0x7fdff4101fa0 2024-05-08T22:05:32.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.728+0000 7fdff8e00700 1 -- 172.21.15.131:0/1647920522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff4067ba0 con 0x7fdff4101fa0 2024-05-08T22:05:32.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.729+0000 7fdfebe00700 1 -- 172.21.15.131:0/1647920522 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fdfe0010430 con 0x7fdff4101fa0 2024-05-08T22:05:32.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.731+0000 7fdfebe00700 1 --2- 172.21.15.131:0/1647920522 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdfd80a1120 0x7fdfd80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:32.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.731+0000 7fdfebe00700 1 -- 172.21.15.131:0/1647920522 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fdfe0014070 con 0x7fdff4101fa0 2024-05-08T22:05:32.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.731+0000 7fdff3400700 1 --2- 172.21.15.131:0/1647920522 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdfd80a1120 0x7fdfd80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:32.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.732+0000 7fdff3400700 1 --2- 172.21.15.131:0/1647920522 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdfd80a1120 0x7fdfd80a35e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4003730 tx=0x7fdfe4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:32.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.735+0000 7fdfebe00700 1 -- 172.21.15.131:0/1647920522 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdfe0052d60 con 0x7fdff4101fa0 2024-05-08T22:05:32.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.949+0000 7fdff8e00700 1 -- 172.21.15.131:0/1647920522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdff41a1640 con 0x7fdff4101fa0 2024-05-08T22:05:32.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.949+0000 7fdfebe00700 1 -- 172.21.15.131:0/1647920522 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdfe0099cd0 con 0x7fdff4101fa0 2024-05-08T22:05:32.950 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-08T22:05:32.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.952+0000 7fdff8e00700 1 -- 172.21.15.131:0/1647920522 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdfd80a1120 msgr2=0x7fdfd80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:32.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.952+0000 7fdff8e00700 1 --2- 172.21.15.131:0/1647920522 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdfd80a1120 0x7fdfd80a35e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4003730 tx=0x7fdfe4000bc0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.952+0000 7fdff8e00700 1 -- 172.21.15.131:0/1647920522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdff4101fa0 msgr2=0x7fdff4199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:32.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.952+0000 7fdff8e00700 1 --2- 172.21.15.131:0/1647920522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdff4101fa0 0x7fdff4199520 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000e9a0 tx=0x7fdfe000ecb0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.952+0000 7fdff8e00700 1 -- 172.21.15.131:0/1647920522 shutdown_connections 2024-05-08T22:05:32.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.952+0000 7fdff8e00700 1 --2- 172.21.15.131:0/1647920522 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdfd80a1120 0x7fdfd80a35e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.952+0000 7fdff8e00700 1 --2- 172.21.15.131:0/1647920522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdff4102940 0x7fdff41a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.952+0000 7fdff8e00700 1 --2- 172.21.15.131:0/1647920522 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdff4100db0 0x7fdff4198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.952+0000 7fdff8e00700 1 --2- 172.21.15.131:0/1647920522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdff4101fa0 0x7fdff4199520 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:32.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.952+0000 7fdff8e00700 1 -- 172.21.15.131:0/1647920522 >> 172.21.15.131:0/1647920522 conn(0x7fdff40fc2e0 msgr2=0x7fdff4107810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:32.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.952+0000 7fdff8e00700 1 -- 172.21.15.131:0/1647920522 shutdown_connections 2024-05-08T22:05:32.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:32.952+0000 7fdff8e00700 1 -- 172.21.15.131:0/1647920522 wait complete. 2024-05-08T22:05:32.971 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:05:33.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:32 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2174829708' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:33.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:32 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/4292574319' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:05:33.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:32 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2174829708' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:33.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:32 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/4292574319' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:05:33.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.167+0000 7fdbc5e00700 1 -- 172.21.15.131:0/4171134776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbc0102ab0 msgr2=0x7fdbc01093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:33.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.167+0000 7fdbc5e00700 1 --2- 172.21.15.131:0/4171134776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbc0102ab0 0x7fdbc01093d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8009f90 tx=0x7fdbb800b330 comp rx=0 tx=0).stop 2024-05-08T22:05:33.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.167+0000 7fdbc5e00700 1 -- 172.21.15.131:0/4171134776 shutdown_connections 2024-05-08T22:05:33.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.167+0000 7fdbc5e00700 1 --2- 172.21.15.131:0/4171134776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbc0102ab0 0x7fdbc01093d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:33.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.167+0000 7fdbc5e00700 1 --2- 172.21.15.131:0/4171134776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbc0100f20 0x7fdbc0101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:33.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.167+0000 7fdbc5e00700 1 --2- 172.21.15.131:0/4171134776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbc01020f0 0x7fdbc0102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:33.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.167+0000 7fdbc5e00700 1 -- 172.21.15.131:0/4171134776 >> 172.21.15.131:0/4171134776 conn(0x7fdbc00fc500 msgr2=0x7fdbc00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:33.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.167+0000 7fdbc5e00700 1 -- 172.21.15.131:0/4171134776 shutdown_connections 2024-05-08T22:05:33.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.167+0000 7fdbc5e00700 1 -- 172.21.15.131:0/4171134776 wait complete. 2024-05-08T22:05:33.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.168+0000 7fdbc5e00700 1 Processor -- start 2024-05-08T22:05:33.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.168+0000 7fdbc5e00700 1 -- start start 2024-05-08T22:05:33.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.168+0000 7fdbc5e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbc0100f20 0x7fdbc0074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:33.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.168+0000 7fdbc5e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbc01020f0 0x7fdbc0073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:33.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.168+0000 7fdbc5e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbc0102ab0 0x7fdbc00738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:33.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.168+0000 7fdbc5e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc0074070 con 0x7fdbc01020f0 2024-05-08T22:05:33.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.168+0000 7fdbc5e00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc00741e0 con 0x7fdbc0102ab0 2024-05-08T22:05:33.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.168+0000 7fdbc5e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc01a1040 con 0x7fdbc0100f20 2024-05-08T22:05:33.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.169+0000 7fdbbe000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbc01020f0 0x7fdbc0073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:33.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.169+0000 7fdbbe000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbc01020f0 0x7fdbc0073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57358/0 (socket says 172.21.15.131:57358) 2024-05-08T22:05:33.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.169+0000 7fdbbe000700 1 -- 172.21.15.131:0/2341058764 learned_addr learned my addr 172.21.15.131:0/2341058764 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:33.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.169+0000 7fdbbea00700 1 --2- 172.21.15.131:0/2341058764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbc0100f20 0x7fdbc0074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:33.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.169+0000 7fdbbf400700 1 --2- 172.21.15.131:0/2341058764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbc0102ab0 0x7fdbc00738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:33.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.169+0000 7fdbbe000700 1 -- 172.21.15.131:0/2341058764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbc0100f20 msgr2=0x7fdbc0074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:33.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.169+0000 7fdbbe000700 1 --2- 172.21.15.131:0/2341058764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbc0100f20 0x7fdbc0074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:33.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.169+0000 7fdbbe000700 1 -- 172.21.15.131:0/2341058764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbc0102ab0 msgr2=0x7fdbc00738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:33.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.169+0000 7fdbbe000700 1 --2- 172.21.15.131:0/2341058764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbc0102ab0 0x7fdbc00738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:33.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.169+0000 7fdbbe000700 1 -- 172.21.15.131:0/2341058764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb8009c40 con 0x7fdbc01020f0 2024-05-08T22:05:33.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.169+0000 7fdbbe000700 1 --2- 172.21.15.131:0/2341058764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbc01020f0 0x7fdbc0073390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00e9c0 tx=0x7fdbac00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:33.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.169+0000 7fdbc4a00700 1 -- 172.21.15.131:0/2341058764 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac004a30 con 0x7fdbc01020f0 2024-05-08T22:05:33.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.169+0000 7fdbc4a00700 1 -- 172.21.15.131:0/2341058764 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbac0054e0 con 0x7fdbc01020f0 2024-05-08T22:05:33.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.170+0000 7fdbc5e00700 1 -- 172.21.15.131:0/2341058764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbc01a12d0 con 0x7fdbc01020f0 2024-05-08T22:05:33.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.170+0000 7fdbc4a00700 1 -- 172.21.15.131:0/2341058764 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac00cbf0 con 0x7fdbc01020f0 2024-05-08T22:05:33.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.170+0000 7fdbc5e00700 1 -- 172.21.15.131:0/2341058764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbc01a17e0 con 0x7fdbc01020f0 2024-05-08T22:05:33.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.171+0000 7fdbc5e00700 1 -- 172.21.15.131:0/2341058764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbc0108340 con 0x7fdbc01020f0 2024-05-08T22:05:33.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.172+0000 7fdbc4a00700 1 -- 172.21.15.131:0/2341058764 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fdbac00cd50 con 0x7fdbc01020f0 2024-05-08T22:05:33.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.174+0000 7fdbc4a00700 1 --2- 172.21.15.131:0/2341058764 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdba40a1120 0x7fdba40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:33.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.174+0000 7fdbc4a00700 1 -- 172.21.15.131:0/2341058764 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fdbac014070 con 0x7fdbc01020f0 2024-05-08T22:05:33.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.174+0000 7fdbbea00700 1 --2- 172.21.15.131:0/2341058764 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdba40a1120 0x7fdba40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:33.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.174+0000 7fdbbea00700 1 --2- 172.21.15.131:0/2341058764 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdba40a1120 0x7fdba40a35e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0100c40 tx=0x7fdbb0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:33.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.178+0000 7fdbc4a00700 1 -- 172.21.15.131:0/2341058764 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdbac096170 con 0x7fdbc01020f0 2024-05-08T22:05:33.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:32 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2174829708' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:33.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:32 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/4292574319' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:05:33.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.388+0000 7fdbc5e00700 1 -- 172.21.15.131:0/2341058764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdbc0107d20 con 0x7fdbc01020f0 2024-05-08T22:05:33.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.389+0000 7fdbc4a00700 1 -- 172.21.15.131:0/2341058764 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdbac099790 con 0x7fdbc01020f0 2024-05-08T22:05:33.390 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-08T22:05:33.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.392+0000 7fdbc5e00700 1 -- 172.21.15.131:0/2341058764 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdba40a1120 msgr2=0x7fdba40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:33.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.392+0000 7fdbc5e00700 1 --2- 172.21.15.131:0/2341058764 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdba40a1120 0x7fdba40a35e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0100c40 tx=0x7fdbb0009260 comp rx=0 tx=0).stop 2024-05-08T22:05:33.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.392+0000 7fdbc5e00700 1 -- 172.21.15.131:0/2341058764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbc01020f0 msgr2=0x7fdbc0073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:33.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.392+0000 7fdbc5e00700 1 --2- 172.21.15.131:0/2341058764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbc01020f0 0x7fdbc0073390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00e9c0 tx=0x7fdbac00ecd0 comp rx=0 tx=0).stop 2024-05-08T22:05:33.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.392+0000 7fdbc5e00700 1 -- 172.21.15.131:0/2341058764 shutdown_connections 2024-05-08T22:05:33.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.392+0000 7fdbc5e00700 1 --2- 172.21.15.131:0/2341058764 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdba40a1120 0x7fdba40a35e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:33.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.392+0000 7fdbc5e00700 1 --2- 172.21.15.131:0/2341058764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbc0102ab0 0x7fdbc00738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:33.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.392+0000 7fdbc5e00700 1 --2- 172.21.15.131:0/2341058764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbc0100f20 0x7fdbc0074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:33.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.392+0000 7fdbc5e00700 1 --2- 172.21.15.131:0/2341058764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbc01020f0 0x7fdbc0073390 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:33.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.392+0000 7fdbc5e00700 1 -- 172.21.15.131:0/2341058764 >> 172.21.15.131:0/2341058764 conn(0x7fdbc00fc500 msgr2=0x7fdbc0105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:33.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.392+0000 7fdbc5e00700 1 -- 172.21.15.131:0/2341058764 shutdown_connections 2024-05-08T22:05:33.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.392+0000 7fdbc5e00700 1 -- 172.21.15.131:0/2341058764 wait complete. 2024-05-08T22:05:33.411 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:05:33.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.605+0000 7f413da00700 1 -- 172.21.15.131:0/102442282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4138102940 msgr2=0x7f41381091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:33.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.605+0000 7f413da00700 1 --2- 172.21.15.131:0/102442282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4138102940 0x7f41381091f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f413000b0d0 tx=0x7f413000b3e0 comp rx=0 tx=0).stop 2024-05-08T22:05:33.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.605+0000 7f413da00700 1 -- 172.21.15.131:0/102442282 shutdown_connections 2024-05-08T22:05:33.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.605+0000 7f413da00700 1 --2- 172.21.15.131:0/102442282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4138102940 0x7f41381091f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:33.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.605+0000 7f413da00700 1 --2- 172.21.15.131:0/102442282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4138100db0 0x7f41381011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:33.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.605+0000 7f413da00700 1 --2- 172.21.15.131:0/102442282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4138101fa0 0x7f4138102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:33.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.605+0000 7f413da00700 1 -- 172.21.15.131:0/102442282 >> 172.21.15.131:0/102442282 conn(0x7f41380fc2e0 msgr2=0x7f41380fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:33.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.605+0000 7f413da00700 1 -- 172.21.15.131:0/102442282 shutdown_connections 2024-05-08T22:05:33.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.606+0000 7f413da00700 1 -- 172.21.15.131:0/102442282 wait complete. 2024-05-08T22:05:33.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.606+0000 7f413da00700 1 Processor -- start 2024-05-08T22:05:33.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.607+0000 7f413da00700 1 -- start start 2024-05-08T22:05:33.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.607+0000 7f413da00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4138100db0 0x7f4138196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:33.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.607+0000 7f413da00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4138101fa0 0x7f4138197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:33.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.607+0000 7f413da00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4138102940 0x7f413819e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:33.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.607+0000 7f413da00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413819e980 con 0x7f4138100db0 2024-05-08T22:05:33.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.607+0000 7f413da00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413819eac0 con 0x7f4138102940 2024-05-08T22:05:33.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.607+0000 7f413da00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413819ec50 con 0x7f4138101fa0 2024-05-08T22:05:33.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.607+0000 7f4137e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4138100db0 0x7f4138196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:33.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.607+0000 7f4137e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4138100db0 0x7f4138196e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57388/0 (socket says 172.21.15.131:57388) 2024-05-08T22:05:33.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.607+0000 7f4137e00700 1 -- 172.21.15.131:0/2068826734 learned_addr learned my addr 172.21.15.131:0/2068826734 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:33.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.607+0000 7f4137400700 1 --2- 172.21.15.131:0/2068826734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4138101fa0 0x7f4138197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:33.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.607+0000 7f413d000700 1 --2- 172.21.15.131:0/2068826734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4138102940 0x7f413819e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:33.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.607+0000 7f4137e00700 1 -- 172.21.15.131:0/2068826734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4138101fa0 msgr2=0x7f4138197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:33.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.607+0000 7f4137e00700 1 --2- 172.21.15.131:0/2068826734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4138101fa0 0x7f4138197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:33.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.607+0000 7f4137e00700 1 -- 172.21.15.131:0/2068826734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4138102940 msgr2=0x7f413819e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:33.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.607+0000 7f4137e00700 1 --2- 172.21.15.131:0/2068826734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4138102940 0x7f413819e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:33.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.607+0000 7f4137e00700 1 -- 172.21.15.131:0/2068826734 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4128009520 con 0x7f4138100db0 2024-05-08T22:05:33.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.607+0000 7f413d000700 1 --2- 172.21.15.131:0/2068826734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4138102940 0x7f413819e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:05:33.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.608+0000 7f4137e00700 1 --2- 172.21.15.131:0/2068826734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4138100db0 0x7f4138196e50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f412800ea50 tx=0x7f412800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:33.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.608+0000 7f4134c00700 1 -- 172.21.15.131:0/2068826734 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412800cbd0 con 0x7f4138100db0 2024-05-08T22:05:33.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.608+0000 7f4134c00700 1 -- 172.21.15.131:0/2068826734 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f412800cd30 con 0x7f4138100db0 2024-05-08T22:05:33.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.608+0000 7f4134c00700 1 -- 172.21.15.131:0/2068826734 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41280186c0 con 0x7f4138100db0 2024-05-08T22:05:33.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.608+0000 7f413da00700 1 -- 172.21.15.131:0/2068826734 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4130009d00 con 0x7f4138100db0 2024-05-08T22:05:33.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.608+0000 7f413da00700 1 -- 172.21.15.131:0/2068826734 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f413819f1f0 con 0x7f4138100db0 2024-05-08T22:05:33.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.609+0000 7f413da00700 1 -- 172.21.15.131:0/2068826734 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4138108160 con 0x7f4138100db0 2024-05-08T22:05:33.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.611+0000 7f4134c00700 1 -- 172.21.15.131:0/2068826734 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f4128018820 con 0x7f4138100db0 2024-05-08T22:05:33.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.612+0000 7f4134c00700 1 --2- 172.21.15.131:0/2068826734 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f411c0a11a0 0x7f411c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:33.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.612+0000 7f4134c00700 1 -- 172.21.15.131:0/2068826734 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4128014070 con 0x7f4138100db0 2024-05-08T22:05:33.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.613+0000 7f4137400700 1 --2- 172.21.15.131:0/2068826734 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f411c0a11a0 0x7f411c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:33.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.613+0000 7f4137400700 1 --2- 172.21.15.131:0/2068826734 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f411c0a11a0 0x7f411c0a3660 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4124009c60 tx=0x7f4124009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:33.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.616+0000 7f4134c00700 1 -- 172.21.15.131:0/2068826734 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4128096520 con 0x7f4138100db0 2024-05-08T22:05:33.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.825+0000 7f413da00700 1 -- 172.21.15.131:0/2068826734 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f413810c850 con 0x7f4138100db0 2024-05-08T22:05:33.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.826+0000 7f4134c00700 1 -- 172.21.15.131:0/2068826734 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v83) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4128099b40 con 0x7f4138100db0 2024-05-08T22:05:33.827 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-08T22:05:33.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.828+0000 7f413da00700 1 -- 172.21.15.131:0/2068826734 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f411c0a11a0 msgr2=0x7f411c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:33.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.828+0000 7f413da00700 1 --2- 172.21.15.131:0/2068826734 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f411c0a11a0 0x7f411c0a3660 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4124009c60 tx=0x7f4124009260 comp rx=0 tx=0).stop 2024-05-08T22:05:33.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.828+0000 7f413da00700 1 -- 172.21.15.131:0/2068826734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4138100db0 msgr2=0x7f4138196e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:33.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.828+0000 7f413da00700 1 --2- 172.21.15.131:0/2068826734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4138100db0 0x7f4138196e50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f412800ea50 tx=0x7f412800ed60 comp rx=0 tx=0).stop 2024-05-08T22:05:33.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.828+0000 7f413da00700 1 -- 172.21.15.131:0/2068826734 shutdown_connections 2024-05-08T22:05:33.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.828+0000 7f413da00700 1 --2- 172.21.15.131:0/2068826734 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f411c0a11a0 0x7f411c0a3660 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:33.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.828+0000 7f413da00700 1 --2- 172.21.15.131:0/2068826734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4138102940 0x7f413819e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:33.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.828+0000 7f413da00700 1 --2- 172.21.15.131:0/2068826734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4138100db0 0x7f4138196e50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:33.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.828+0000 7f413da00700 1 --2- 172.21.15.131:0/2068826734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4138101fa0 0x7f4138197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:33.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.828+0000 7f413da00700 1 -- 172.21.15.131:0/2068826734 >> 172.21.15.131:0/2068826734 conn(0x7f41380fc2e0 msgr2=0x7f41381052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:33.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.829+0000 7f413da00700 1 -- 172.21.15.131:0/2068826734 shutdown_connections 2024-05-08T22:05:33.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:33.829+0000 7f413da00700 1 -- 172.21.15.131:0/2068826734 wait complete. 2024-05-08T22:05:33.848 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:05:34.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.042+0000 7f43fe200700 1 -- 172.21.15.131:0/3350835165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43f8102170 msgr2=0x7f43f81025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:34.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.042+0000 7f43fe200700 1 --2- 172.21.15.131:0/3350835165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43f8102170 0x7f43f81025d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009990 tx=0x7f43e4009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:34.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.042+0000 7f43fe200700 1 -- 172.21.15.131:0/3350835165 shutdown_connections 2024-05-08T22:05:34.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.042+0000 7f43fe200700 1 --2- 172.21.15.131:0/3350835165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43f8102b10 0x7f43f81093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:34.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.042+0000 7f43fe200700 1 --2- 172.21.15.131:0/3350835165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43f8100f80 0x7f43f81013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:34.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.042+0000 7f43fe200700 1 --2- 172.21.15.131:0/3350835165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43f8102170 0x7f43f81025d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:34.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.042+0000 7f43fe200700 1 -- 172.21.15.131:0/3350835165 >> 172.21.15.131:0/3350835165 conn(0x7f43f80fc4f0 msgr2=0x7f43f80fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:34.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.042+0000 7f43fe200700 1 -- 172.21.15.131:0/3350835165 shutdown_connections 2024-05-08T22:05:34.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.042+0000 7f43fe200700 1 -- 172.21.15.131:0/3350835165 wait complete. 2024-05-08T22:05:34.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.043+0000 7f43fe200700 1 Processor -- start 2024-05-08T22:05:34.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.043+0000 7f43fe200700 1 -- start start 2024-05-08T22:05:34.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.044+0000 7f43fe200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43f8100f80 0x7f43f8199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:34.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.044+0000 7f43fe200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43f8102170 0x7f43f81996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:34.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.044+0000 7f43fe200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43f8102b10 0x7f43f81a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:34.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.044+0000 7f43fe200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f81a0c60 con 0x7f43f8100f80 2024-05-08T22:05:34.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.044+0000 7f43fe200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f81a0da0 con 0x7f43f8102170 2024-05-08T22:05:34.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.044+0000 7f43fe200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f81a0f30 con 0x7f43f8102b10 2024-05-08T22:05:34.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.044+0000 7f43f7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43f8100f80 0x7f43f8199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:34.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.044+0000 7f43f6a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43f8102170 0x7f43f81996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:34.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.044+0000 7f43f7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43f8100f80 0x7f43f8199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41970/0 (socket says 172.21.15.131:41970) 2024-05-08T22:05:34.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.044+0000 7f43f7400700 1 -- 172.21.15.131:0/4163198552 learned_addr learned my addr 172.21.15.131:0/4163198552 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:34.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.044+0000 7f43f7e00700 1 --2- 172.21.15.131:0/4163198552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43f8102b10 0x7f43f81a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:34.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.044+0000 7f43f6a00700 1 -- 172.21.15.131:0/4163198552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43f8102b10 msgr2=0x7f43f81a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:34.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.044+0000 7f43f6a00700 1 --2- 172.21.15.131:0/4163198552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43f8102b10 0x7f43f81a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:34.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.044+0000 7f43f6a00700 1 -- 172.21.15.131:0/4163198552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43f8100f80 msgr2=0x7f43f8199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:34.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.044+0000 7f43f6a00700 1 --2- 172.21.15.131:0/4163198552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43f8100f80 0x7f43f8199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:34.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.044+0000 7f43f6a00700 1 -- 172.21.15.131:0/4163198552 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e40095f0 con 0x7f43f8102170 2024-05-08T22:05:34.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.044+0000 7f43f7400700 1 --2- 172.21.15.131:0/4163198552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43f8100f80 0x7f43f8199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:34.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.045+0000 7f43f6a00700 1 --2- 172.21.15.131:0/4163198552 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43f8102170 0x7f43f81996c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f43e400b680 tx=0x7f43e4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:34.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.045+0000 7f43efe00700 1 -- 172.21.15.131:0/4163198552 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e401d070 con 0x7f43f8102170 2024-05-08T22:05:34.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.045+0000 7f43fe200700 1 -- 172.21.15.131:0/4163198552 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f81a11b0 con 0x7f43f8102170 2024-05-08T22:05:34.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.045+0000 7f43efe00700 1 -- 172.21.15.131:0/4163198552 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43e4021410 con 0x7f43f8102170 2024-05-08T22:05:34.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.045+0000 7f43efe00700 1 -- 172.21.15.131:0/4163198552 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e4017650 con 0x7f43f8102170 2024-05-08T22:05:34.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.045+0000 7f43fe200700 1 -- 172.21.15.131:0/4163198552 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f81a15d0 con 0x7f43f8102170 2024-05-08T22:05:34.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.046+0000 7f43fe200700 1 -- 172.21.15.131:0/4163198552 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f8067cf0 con 0x7f43f8102170 2024-05-08T22:05:34.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.048+0000 7f43efe00700 1 -- 172.21.15.131:0/4163198552 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f43e400fb10 con 0x7f43f8102170 2024-05-08T22:05:34.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.049+0000 7f43efe00700 1 --2- 172.21.15.131:0/4163198552 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f43dc0a1120 0x7f43dc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:34.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.049+0000 7f43efe00700 1 -- 172.21.15.131:0/4163198552 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f43e40ce3e0 con 0x7f43f8102170 2024-05-08T22:05:34.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.049+0000 7f43f7400700 1 --2- 172.21.15.131:0/4163198552 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f43dc0a1120 0x7f43dc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:34.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.050+0000 7f43f7400700 1 --2- 172.21.15.131:0/4163198552 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f43dc0a1120 0x7f43dc0a35e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f43e8005950 tx=0x7f43e8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:34.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.052+0000 7f43efe00700 1 -- 172.21.15.131:0/4163198552 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f43e40979b0 con 0x7f43f8102170 2024-05-08T22:05:34.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:33 smithi132 ceph-mon[78180]: pgmap v249: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-08T22:05:34.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:33 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1647920522' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:05:34.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:33 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2341058764' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:05:34.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:33 smithi131 ceph-mon[37260]: pgmap v249: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-08T22:05:34.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:33 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1647920522' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:05:34.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:33 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2341058764' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:05:34.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:33 smithi157 ceph-mon[77746]: pgmap v249: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-08T22:05:34.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:33 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1647920522' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:05:34.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:33 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2341058764' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:05:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.276+0000 7f43fe200700 1 -- 172.21.15.131:0/4163198552 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f43f810c800 con 0x7f43f8102170 2024-05-08T22:05:34.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.277+0000 7f43efe00700 1 -- 172.21.15.131:0/4163198552 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v83) v1 ==== 81+0+601 (secure 0 0 0) 0x7f43e4021e60 con 0x7f43f8102170 2024-05-08T22:05:34.278 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6801/1418340798 2024-05-09T21:59:40.641373+0000 2024-05-08T22:05:34.278 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6800/1418340798 2024-05-09T21:59:40.641373+0000 2024-05-08T22:05:34.279 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/1241655400 2024-05-09T21:59:40.641373+0000 2024-05-08T22:05:34.279 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/1716895483 2024-05-09T21:59:40.641373+0000 2024-05-08T22:05:34.279 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/1981506111 2024-05-09T21:59:15.548383+0000 2024-05-08T22:05:34.279 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/976983862 2024-05-09T21:59:15.548383+0000 2024-05-08T22:05:34.279 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6801/1564023164 2024-05-09T21:59:15.548383+0000 2024-05-08T22:05:34.279 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/3309822030 2024-05-09T21:59:15.548383+0000 2024-05-08T22:05:34.279 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/3082367829 2024-05-09T21:59:40.641373+0000 2024-05-08T22:05:34.279 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6800/1564023164 2024-05-09T21:59:15.548383+0000 2024-05-08T22:05:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.279+0000 7f43fe200700 1 -- 172.21.15.131:0/4163198552 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f43dc0a1120 msgr2=0x7f43dc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.280+0000 7f43fe200700 1 --2- 172.21.15.131:0/4163198552 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f43dc0a1120 0x7f43dc0a35e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f43e8005950 tx=0x7f43e8009260 comp rx=0 tx=0).stop 2024-05-08T22:05:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.280+0000 7f43fe200700 1 -- 172.21.15.131:0/4163198552 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43f8102170 msgr2=0x7f43f81996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.280+0000 7f43fe200700 1 --2- 172.21.15.131:0/4163198552 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43f8102170 0x7f43f81996c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f43e400b680 tx=0x7f43e4004730 comp rx=0 tx=0).stop 2024-05-08T22:05:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.280+0000 7f43fe200700 1 -- 172.21.15.131:0/4163198552 shutdown_connections 2024-05-08T22:05:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.280+0000 7f43fe200700 1 --2- 172.21.15.131:0/4163198552 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f43dc0a1120 0x7f43dc0a35e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.280+0000 7f43fe200700 1 --2- 172.21.15.131:0/4163198552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43f8102b10 0x7f43f81a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.280+0000 7f43fe200700 1 --2- 172.21.15.131:0/4163198552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43f8100f80 0x7f43f8199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.280+0000 7f43fe200700 1 --2- 172.21.15.131:0/4163198552 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43f8102170 0x7f43f81996c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.280+0000 7f43fe200700 1 -- 172.21.15.131:0/4163198552 >> 172.21.15.131:0/4163198552 conn(0x7f43f80fc4f0 msgr2=0x7f43f80731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.280+0000 7f43fe200700 1 -- 172.21.15.131:0/4163198552 shutdown_connections 2024-05-08T22:05:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.280+0000 7f43fe200700 1 -- 172.21.15.131:0/4163198552 wait complete. 2024-05-08T22:05:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:listed 10 entries 2024-05-08T22:05:34.299 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2024-05-08T22:05:34.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:05:34.299 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-08T22:05:34.362 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi131.stderr:ceph-fuse[90084]: fuse finished with error 0 and tester_r 0 2024-05-08T22:05:34.366 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:05:34.444 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-05-08T22:05:34.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:05:34.445 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-08T22:05:34.502 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:05:34.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.698+0000 7fa79cc00700 1 -- 172.21.15.131:0/149908173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa798074f60 msgr2=0x7fa7980733b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:34.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.698+0000 7fa79cc00700 1 --2- 172.21.15.131:0/149908173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa798074f60 0x7fa7980733b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa7880098f0 tx=0x7fa788009c00 comp rx=0 tx=0).stop 2024-05-08T22:05:34.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.698+0000 7fa79cc00700 1 -- 172.21.15.131:0/149908173 shutdown_connections 2024-05-08T22:05:34.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.698+0000 7fa79cc00700 1 --2- 172.21.15.131:0/149908173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa798108d80 0x7fa79810b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:34.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.698+0000 7fa79cc00700 1 --2- 172.21.15.131:0/149908173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa798074f60 0x7fa7980733b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:34.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.698+0000 7fa79cc00700 1 --2- 172.21.15.131:0/149908173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa798073980 0x7fa798073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:34.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.698+0000 7fa79cc00700 1 -- 172.21.15.131:0/149908173 >> 172.21.15.131:0/149908173 conn(0x7fa7980fc400 msgr2=0x7fa7980fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:34.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.698+0000 7fa79cc00700 1 -- 172.21.15.131:0/149908173 shutdown_connections 2024-05-08T22:05:34.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.698+0000 7fa79cc00700 1 -- 172.21.15.131:0/149908173 wait complete. 2024-05-08T22:05:34.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.699+0000 7fa79cc00700 1 Processor -- start 2024-05-08T22:05:34.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.699+0000 7fa79cc00700 1 -- start start 2024-05-08T22:05:34.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.699+0000 7fa79cc00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa798073980 0x7fa7981990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:34.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.699+0000 7fa79cc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa798074f60 0x7fa7981995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:34.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.699+0000 7fa79cc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa798108d80 0x7fa7981a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:34.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.699+0000 7fa79cc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7981a0bd0 con 0x7fa798108d80 2024-05-08T22:05:34.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.699+0000 7fa79cc00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7981a0d10 con 0x7fa798073980 2024-05-08T22:05:34.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.699+0000 7fa79cc00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7981a0ea0 con 0x7fa798074f60 2024-05-08T22:05:34.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.700+0000 7fa796800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa798108d80 0x7fa7981a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:34.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.700+0000 7fa796800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa798108d80 0x7fa7981a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41980/0 (socket says 172.21.15.131:41980) 2024-05-08T22:05:34.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.700+0000 7fa796800700 1 -- 172.21.15.131:0/1725031878 learned_addr learned my addr 172.21.15.131:0/1725031878 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:34.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.700+0000 7fa795400700 1 --2- 172.21.15.131:0/1725031878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa798074f60 0x7fa7981995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:34.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.700+0000 7fa795e00700 1 --2- 172.21.15.131:0/1725031878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa798073980 0x7fa7981990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:34.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.700+0000 7fa796800700 1 -- 172.21.15.131:0/1725031878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa798074f60 msgr2=0x7fa7981995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:34.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.700+0000 7fa796800700 1 --2- 172.21.15.131:0/1725031878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa798074f60 0x7fa7981995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:34.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.700+0000 7fa796800700 1 -- 172.21.15.131:0/1725031878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa798073980 msgr2=0x7fa7981990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:34.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.700+0000 7fa796800700 1 --2- 172.21.15.131:0/1725031878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa798073980 0x7fa7981990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:34.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.700+0000 7fa796800700 1 -- 172.21.15.131:0/1725031878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa790009d00 con 0x7fa798108d80 2024-05-08T22:05:34.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.700+0000 7fa796800700 1 --2- 172.21.15.131:0/1725031878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa798108d80 0x7fa7981a0600 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa79000e3c0 tx=0x7fa79000e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:34.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.701+0000 7fa78ea00700 1 -- 172.21.15.131:0/1725031878 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7900090d0 con 0x7fa798108d80 2024-05-08T22:05:34.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.701+0000 7fa78ea00700 1 -- 172.21.15.131:0/1725031878 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa79000f040 con 0x7fa798108d80 2024-05-08T22:05:34.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.701+0000 7fa79cc00700 1 -- 172.21.15.131:0/1725031878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa788009520 con 0x7fa798108d80 2024-05-08T22:05:34.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.701+0000 7fa78ea00700 1 -- 172.21.15.131:0/1725031878 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa790007c20 con 0x7fa798108d80 2024-05-08T22:05:34.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.701+0000 7fa79cc00700 1 -- 172.21.15.131:0/1725031878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7981a14b0 con 0x7fa798108d80 2024-05-08T22:05:34.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.702+0000 7fa79cc00700 1 -- 172.21.15.131:0/1725031878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa798107b90 con 0x7fa798108d80 2024-05-08T22:05:34.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.704+0000 7fa78ea00700 1 -- 172.21.15.131:0/1725031878 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa790003a00 con 0x7fa798108d80 2024-05-08T22:05:34.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.705+0000 7fa78ea00700 1 --2- 172.21.15.131:0/1725031878 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa77c0a1120 0x7fa77c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:34.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.705+0000 7fa78ea00700 1 -- 172.21.15.131:0/1725031878 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa7900cc910 con 0x7fa798108d80 2024-05-08T22:05:34.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.705+0000 7fa795e00700 1 --2- 172.21.15.131:0/1725031878 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa77c0a1120 0x7fa77c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:34.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.706+0000 7fa795e00700 1 --2- 172.21.15.131:0/1725031878 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa77c0a1120 0x7fa77c0a35e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa78800fa40 tx=0x7fa788019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:34.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.709+0000 7fa78ea00700 1 -- 172.21.15.131:0/1725031878 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa790095ee0 con 0x7fa798108d80 2024-05-08T22:05:34.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:34 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2068826734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:05:34.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:34 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/4163198552' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T22:05:34.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.920+0000 7fa79cc00700 1 -- 172.21.15.131:0/1725031878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa798067c10 con 0x7fa798108d80 2024-05-08T22:05:34.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.922+0000 7fa78ea00700 1 -- 172.21.15.131:0/1725031878 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+24007 (secure 0 0 0) 0x7fa790099500 con 0x7fa798108d80 2024-05-08T22:05:34.923 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:34.923 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":83,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:05:19.013678+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:05:12.469858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:05:13.484446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:05:17.016186+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:05:34.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.924+0000 7fa79cc00700 1 -- 172.21.15.131:0/1725031878 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa77c0a1120 msgr2=0x7fa77c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:34.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.924+0000 7fa79cc00700 1 --2- 172.21.15.131:0/1725031878 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa77c0a1120 0x7fa77c0a35e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa78800fa40 tx=0x7fa788019040 comp rx=0 tx=0).stop 2024-05-08T22:05:34.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.924+0000 7fa79cc00700 1 -- 172.21.15.131:0/1725031878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa798108d80 msgr2=0x7fa7981a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:34.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.924+0000 7fa79cc00700 1 --2- 172.21.15.131:0/1725031878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa798108d80 0x7fa7981a0600 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa79000e3c0 tx=0x7fa79000e6d0 comp rx=0 tx=0).stop 2024-05-08T22:05:34.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.924+0000 7fa79cc00700 1 -- 172.21.15.131:0/1725031878 shutdown_connections 2024-05-08T22:05:34.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.924+0000 7fa79cc00700 1 --2- 172.21.15.131:0/1725031878 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa77c0a1120 0x7fa77c0a35e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:34.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.924+0000 7fa79cc00700 1 --2- 172.21.15.131:0/1725031878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa798108d80 0x7fa7981a0600 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:34.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.924+0000 7fa79cc00700 1 --2- 172.21.15.131:0/1725031878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa798073980 0x7fa7981990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:34.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.924+0000 7fa79cc00700 1 --2- 172.21.15.131:0/1725031878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa798074f60 0x7fa7981995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:34.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.924+0000 7fa79cc00700 1 -- 172.21.15.131:0/1725031878 >> 172.21.15.131:0/1725031878 conn(0x7fa7980fc400 msgr2=0x7fa798104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:34.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.925+0000 7fa79cc00700 1 -- 172.21.15.131:0/1725031878 shutdown_connections 2024-05-08T22:05:34.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:34.925+0000 7fa79cc00700 1 -- 172.21.15.131:0/1725031878 wait complete. 2024-05-08T22:05:34.945 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:05:35.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:34 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2068826734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:05:35.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:34 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/4163198552' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T22:05:35.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.137+0000 7fb038800700 1 -- 172.21.15.131:0/3466251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb030100e80 msgr2=0x7fb0301012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:35.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.137+0000 7fb038800700 1 --2- 172.21.15.131:0/3466251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb030100e80 0x7fb0301012a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb024009990 tx=0x7fb024009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:35.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.137+0000 7fb038800700 1 -- 172.21.15.131:0/3466251 shutdown_connections 2024-05-08T22:05:35.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.137+0000 7fb038800700 1 --2- 172.21.15.131:0/3466251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb030102990 0x7fb0301092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:35.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.137+0000 7fb038800700 1 --2- 172.21.15.131:0/3466251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb030100e80 0x7fb0301012a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:35.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.137+0000 7fb038800700 1 --2- 172.21.15.131:0/3466251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb030101fd0 0x7fb030102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:35.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.137+0000 7fb038800700 1 -- 172.21.15.131:0/3466251 >> 172.21.15.131:0/3466251 conn(0x7fb0300fc410 msgr2=0x7fb0300fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:35.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.137+0000 7fb038800700 1 -- 172.21.15.131:0/3466251 shutdown_connections 2024-05-08T22:05:35.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.137+0000 7fb038800700 1 -- 172.21.15.131:0/3466251 wait complete. 2024-05-08T22:05:35.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.138+0000 7fb038800700 1 Processor -- start 2024-05-08T22:05:35.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.139+0000 7fb038800700 1 -- start start 2024-05-08T22:05:35.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.139+0000 7fb038800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb030100e80 0x7fb030196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:35.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.139+0000 7fb038800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb030101fd0 0x7fb030197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:35.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.139+0000 7fb038800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb030102990 0x7fb03019e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:35.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.139+0000 7fb038800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03019e980 con 0x7fb030100e80 2024-05-08T22:05:35.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.139+0000 7fb038800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03019eac0 con 0x7fb030102990 2024-05-08T22:05:35.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.139+0000 7fb038800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03019ec50 con 0x7fb030101fd0 2024-05-08T22:05:35.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.139+0000 7fb035e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb030100e80 0x7fb030196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:35.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.139+0000 7fb035e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb030100e80 0x7fb030196e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41988/0 (socket says 172.21.15.131:41988) 2024-05-08T22:05:35.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.139+0000 7fb035e00700 1 -- 172.21.15.131:0/2803778244 learned_addr learned my addr 172.21.15.131:0/2803778244 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:35.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.139+0000 7fb036800700 1 --2- 172.21.15.131:0/2803778244 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb030102990 0x7fb03019e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:35.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.139+0000 7fb035400700 1 --2- 172.21.15.131:0/2803778244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb030101fd0 0x7fb030197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:35.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.140+0000 7fb035e00700 1 -- 172.21.15.131:0/2803778244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb030101fd0 msgr2=0x7fb030197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:35.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.140+0000 7fb035e00700 1 --2- 172.21.15.131:0/2803778244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb030101fd0 0x7fb030197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:35.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.140+0000 7fb035e00700 1 -- 172.21.15.131:0/2803778244 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb030102990 msgr2=0x7fb03019e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:35.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.140+0000 7fb035e00700 1 --2- 172.21.15.131:0/2803778244 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb030102990 0x7fb03019e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:35.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.140+0000 7fb035e00700 1 -- 172.21.15.131:0/2803778244 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0240095f0 con 0x7fb030100e80 2024-05-08T22:05:35.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.140+0000 7fb035e00700 1 --2- 172.21.15.131:0/2803778244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb030100e80 0x7fb030196e50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb02400b620 tx=0x7fb024009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:35.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.140+0000 7fb02aa00700 1 -- 172.21.15.131:0/2803778244 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02401d070 con 0x7fb030100e80 2024-05-08T22:05:35.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.140+0000 7fb02aa00700 1 -- 172.21.15.131:0/2803778244 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb024022470 con 0x7fb030100e80 2024-05-08T22:05:35.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.140+0000 7fb038800700 1 -- 172.21.15.131:0/2803778244 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03019eed0 con 0x7fb030100e80 2024-05-08T22:05:35.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.140+0000 7fb02aa00700 1 -- 172.21.15.131:0/2803778244 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024017650 con 0x7fb030100e80 2024-05-08T22:05:35.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.140+0000 7fb038800700 1 -- 172.21.15.131:0/2803778244 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03019f240 con 0x7fb030100e80 2024-05-08T22:05:35.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.141+0000 7fb038800700 1 -- 172.21.15.131:0/2803778244 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb030067c80 con 0x7fb030100e80 2024-05-08T22:05:35.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.143+0000 7fb02aa00700 1 -- 172.21.15.131:0/2803778244 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb024022870 con 0x7fb030100e80 2024-05-08T22:05:35.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.144+0000 7fb02aa00700 1 --2- 172.21.15.131:0/2803778244 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb0180a1220 0x7fb0180a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:35.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.145+0000 7fb02aa00700 1 -- 172.21.15.131:0/2803778244 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb0240ce500 con 0x7fb030100e80 2024-05-08T22:05:35.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.145+0000 7fb035400700 1 --2- 172.21.15.131:0/2803778244 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb0180a1220 0x7fb0180a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:35.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.145+0000 7fb035400700 1 --2- 172.21.15.131:0/2803778244 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb0180a1220 0x7fb0180a36e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb020009bf0 tx=0x7fb020009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:35.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.147+0000 7fb02aa00700 1 -- 172.21.15.131:0/2803778244 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb024097ad0 con 0x7fb030100e80 2024-05-08T22:05:35.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:34 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2068826734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:05:35.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:34 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/4163198552' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T22:05:35.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.356+0000 7fb038800700 1 -- 172.21.15.131:0/2803778244 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb030067c10 con 0x7fb030100e80 2024-05-08T22:05:35.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.358+0000 7fb02aa00700 1 -- 172.21.15.131:0/2803778244 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+24007 (secure 0 0 0) 0x7fb02409b0f0 con 0x7fb030100e80 2024-05-08T22:05:35.361 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":83,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:05:19.013678+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:05:12.469858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:05:13.484446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:05:17.016186+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:05:35.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.360+0000 7fb038800700 1 -- 172.21.15.131:0/2803778244 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb0180a1220 msgr2=0x7fb0180a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:35.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.360+0000 7fb038800700 1 --2- 172.21.15.131:0/2803778244 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb0180a1220 0x7fb0180a36e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb020009bf0 tx=0x7fb020009260 comp rx=0 tx=0).stop 2024-05-08T22:05:35.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.360+0000 7fb038800700 1 -- 172.21.15.131:0/2803778244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb030100e80 msgr2=0x7fb030196e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:35.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.360+0000 7fb038800700 1 --2- 172.21.15.131:0/2803778244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb030100e80 0x7fb030196e50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb02400b620 tx=0x7fb024009510 comp rx=0 tx=0).stop 2024-05-08T22:05:35.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.360+0000 7fb038800700 1 -- 172.21.15.131:0/2803778244 shutdown_connections 2024-05-08T22:05:35.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.360+0000 7fb038800700 1 --2- 172.21.15.131:0/2803778244 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb0180a1220 0x7fb0180a36e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:35.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.360+0000 7fb038800700 1 --2- 172.21.15.131:0/2803778244 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb030102990 0x7fb03019e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:35.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.360+0000 7fb038800700 1 --2- 172.21.15.131:0/2803778244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb030100e80 0x7fb030196e50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:35.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.360+0000 7fb038800700 1 --2- 172.21.15.131:0/2803778244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb030101fd0 0x7fb030197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:35.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.360+0000 7fb038800700 1 -- 172.21.15.131:0/2803778244 >> 172.21.15.131:0/2803778244 conn(0x7fb0300fc410 msgr2=0x7fb030105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:35.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.360+0000 7fb038800700 1 -- 172.21.15.131:0/2803778244 shutdown_connections 2024-05-08T22:05:35.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.360+0000 7fb038800700 1 -- 172.21.15.131:0/2803778244 wait complete. 2024-05-08T22:05:35.382 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T22:05:35.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.579+0000 7f986b200700 1 -- 172.21.15.131:0/2350348933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9864073a00 msgr2=0x7f9864073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:35.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.579+0000 7f986b200700 1 --2- 172.21.15.131:0/2350348933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9864073a00 0x7f9864073e80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9850009990 tx=0x7f9850009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:35.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.579+0000 7f986b200700 1 -- 172.21.15.131:0/2350348933 shutdown_connections 2024-05-08T22:05:35.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.579+0000 7f986b200700 1 --2- 172.21.15.131:0/2350348933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f986410d540 0x7f986410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:35.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.579+0000 7f986b200700 1 --2- 172.21.15.131:0/2350348933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9864074fe0 0x7f9864073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:35.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.579+0000 7f986b200700 1 --2- 172.21.15.131:0/2350348933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9864073a00 0x7f9864073e80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:35.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.579+0000 7f986b200700 1 -- 172.21.15.131:0/2350348933 >> 172.21.15.131:0/2350348933 conn(0x7f98640fc500 msgr2=0x7f98640fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:35.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.580+0000 7f986b200700 1 -- 172.21.15.131:0/2350348933 shutdown_connections 2024-05-08T22:05:35.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.580+0000 7f986b200700 1 -- 172.21.15.131:0/2350348933 wait complete. 2024-05-08T22:05:35.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.581+0000 7f986b200700 1 Processor -- start 2024-05-08T22:05:35.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.581+0000 7f986b200700 1 -- start start 2024-05-08T22:05:35.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.581+0000 7f986b200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9864073a00 0x7f986419d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:35.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.581+0000 7f986b200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9864074fe0 0x7f986419da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:35.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.581+0000 7f986b200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f986410d540 0x7f98641a4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:35.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.581+0000 7f986b200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98641a5070 con 0x7f9864073a00 2024-05-08T22:05:35.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.581+0000 7f986b200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98641a51b0 con 0x7f986410d540 2024-05-08T22:05:35.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.581+0000 7f986b200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98641a5340 con 0x7f9864074fe0 2024-05-08T22:05:35.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.581+0000 7f9863e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9864073a00 0x7f986419d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:35.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.581+0000 7f9863e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9864073a00 0x7f986419d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41994/0 (socket says 172.21.15.131:41994) 2024-05-08T22:05:35.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.581+0000 7f9863e00700 1 -- 172.21.15.131:0/454714757 learned_addr learned my addr 172.21.15.131:0/454714757 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:35.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.581+0000 7f9863400700 1 --2- 172.21.15.131:0/454714757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9864074fe0 0x7f986419da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:35.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.581+0000 7f9869200700 1 --2- 172.21.15.131:0/454714757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f986410d540 0x7f98641a4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:35.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.582+0000 7f9863e00700 1 -- 172.21.15.131:0/454714757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9864074fe0 msgr2=0x7f986419da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:35.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.582+0000 7f9863e00700 1 --2- 172.21.15.131:0/454714757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9864074fe0 0x7f986419da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:35.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.582+0000 7f9863e00700 1 -- 172.21.15.131:0/454714757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f986410d540 msgr2=0x7f98641a4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:35.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.582+0000 7f9863e00700 1 --2- 172.21.15.131:0/454714757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f986410d540 0x7f98641a4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:35.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.582+0000 7f9863e00700 1 -- 172.21.15.131:0/454714757 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98500095f0 con 0x7f9864073a00 2024-05-08T22:05:35.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.582+0000 7f9863e00700 1 --2- 172.21.15.131:0/454714757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9864073a00 0x7f986419d540 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f985400e9a0 tx=0x7f985400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:35.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.582+0000 7f9860c00700 1 -- 172.21.15.131:0/454714757 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9854009790 con 0x7f9864073a00 2024-05-08T22:05:35.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.582+0000 7f9860c00700 1 -- 172.21.15.131:0/454714757 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9854010430 con 0x7f9864073a00 2024-05-08T22:05:35.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.582+0000 7f986b200700 1 -- 172.21.15.131:0/454714757 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98641a5620 con 0x7f9864073a00 2024-05-08T22:05:35.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.582+0000 7f9860c00700 1 -- 172.21.15.131:0/454714757 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9854018680 con 0x7f9864073a00 2024-05-08T22:05:35.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.582+0000 7f986b200700 1 -- 172.21.15.131:0/454714757 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98641a5ae0 con 0x7f9864073a00 2024-05-08T22:05:35.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.583+0000 7f986b200700 1 -- 172.21.15.131:0/454714757 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98641037a0 con 0x7f9864073a00 2024-05-08T22:05:35.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.585+0000 7f9860c00700 1 -- 172.21.15.131:0/454714757 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f98540105a0 con 0x7f9864073a00 2024-05-08T22:05:35.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.586+0000 7f9860c00700 1 --2- 172.21.15.131:0/454714757 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f98480a1120 0x7f98480a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:35.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.587+0000 7f9860c00700 1 -- 172.21.15.131:0/454714757 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9854014070 con 0x7f9864073a00 2024-05-08T22:05:35.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.587+0000 7f9863400700 1 --2- 172.21.15.131:0/454714757 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f98480a1120 0x7f98480a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:35.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.587+0000 7f9863400700 1 --2- 172.21.15.131:0/454714757 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f98480a1120 0x7f98480a35e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9850000c00 tx=0x7f9850005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:35.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.590+0000 7f9860c00700 1 -- 172.21.15.131:0/454714757 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9854052d60 con 0x7f9864073a00 2024-05-08T22:05:35.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:35.848+0000 7f986b200700 1 -- 172.21.15.131:0/454714757 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9864113090 con 0x7f9864073a00 2024-05-08T22:05:36.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:35 smithi132 ceph-mon[78180]: pgmap v250: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-08T22:05:36.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:35 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1725031878' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:36.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:35 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2803778244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:36.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:35 smithi131 ceph-mon[37260]: pgmap v250: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-08T22:05:36.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:35 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1725031878' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:36.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:35 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2803778244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:36.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:35 smithi157 ceph-mon[77746]: pgmap v250: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-08T22:05:36.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:35 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1725031878' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:36.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:35 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2803778244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:36.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:36.803+0000 7f9860c00700 1 -- 172.21.15.131:0/454714757 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9854096230 con 0x7f9864073a00 2024-05-08T22:05:36.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:36.807+0000 7f986b200700 1 -- 172.21.15.131:0/454714757 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f98480a1120 msgr2=0x7f98480a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:36.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:36.807+0000 7f986b200700 1 --2- 172.21.15.131:0/454714757 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f98480a1120 0x7f98480a35e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9850000c00 tx=0x7f9850005fb0 comp rx=0 tx=0).stop 2024-05-08T22:05:36.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:36.807+0000 7f986b200700 1 -- 172.21.15.131:0/454714757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9864073a00 msgr2=0x7f986419d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:36.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:36.807+0000 7f986b200700 1 --2- 172.21.15.131:0/454714757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9864073a00 0x7f986419d540 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f985400e9a0 tx=0x7f985400ecb0 comp rx=0 tx=0).stop 2024-05-08T22:05:36.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:36.807+0000 7f986b200700 1 -- 172.21.15.131:0/454714757 shutdown_connections 2024-05-08T22:05:36.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:36.807+0000 7f986b200700 1 --2- 172.21.15.131:0/454714757 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f98480a1120 0x7f98480a35e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:36.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:36.807+0000 7f986b200700 1 --2- 172.21.15.131:0/454714757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f986410d540 0x7f98641a4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:36.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:36.807+0000 7f986b200700 1 --2- 172.21.15.131:0/454714757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9864073a00 0x7f986419d540 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:36.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:36.807+0000 7f986b200700 1 --2- 172.21.15.131:0/454714757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9864074fe0 0x7f986419da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:36.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:36.807+0000 7f986b200700 1 -- 172.21.15.131:0/454714757 >> 172.21.15.131:0/454714757 conn(0x7f98640fc500 msgr2=0x7f98641008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:36.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:36.807+0000 7f986b200700 1 -- 172.21.15.131:0/454714757 shutdown_connections 2024-05-08T22:05:36.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:36.807+0000 7f986b200700 1 -- 172.21.15.131:0/454714757 wait complete. 2024-05-08T22:05:36.830 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-05-08T22:05:36.830 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-05-08T22:05:37.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.029+0000 7fdc4ee00700 1 -- 172.21.15.131:0/1830546745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc48074fe0 msgr2=0x7fdc48073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:37.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.029+0000 7fdc4ee00700 1 --2- 172.21.15.131:0/1830546745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc48074fe0 0x7fdc48073430 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009990 tx=0x7fdc38009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:37.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.029+0000 7fdc4ee00700 1 -- 172.21.15.131:0/1830546745 shutdown_connections 2024-05-08T22:05:37.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.029+0000 7fdc4ee00700 1 --2- 172.21.15.131:0/1830546745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc48108e30 0x7fdc4810b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:37.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.029+0000 7fdc4ee00700 1 --2- 172.21.15.131:0/1830546745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc48074fe0 0x7fdc48073430 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:37.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.029+0000 7fdc4ee00700 1 --2- 172.21.15.131:0/1830546745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc48073a00 0x7fdc48073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:37.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.029+0000 7fdc4ee00700 1 -- 172.21.15.131:0/1830546745 >> 172.21.15.131:0/1830546745 conn(0x7fdc480fc500 msgr2=0x7fdc480fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:37.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.029+0000 7fdc4ee00700 1 -- 172.21.15.131:0/1830546745 shutdown_connections 2024-05-08T22:05:37.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.029+0000 7fdc4ee00700 1 -- 172.21.15.131:0/1830546745 wait complete. 2024-05-08T22:05:37.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.030+0000 7fdc4ee00700 1 Processor -- start 2024-05-08T22:05:37.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.030+0000 7fdc4ee00700 1 -- start start 2024-05-08T22:05:37.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.030+0000 7fdc4ee00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc48073a00 0x7fdc48199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:37.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.030+0000 7fdc4ee00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc48074fe0 0x7fdc48199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:37.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.030+0000 7fdc4ee00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc48108e30 0x7fdc481a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:37.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.030+0000 7fdc4ee00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc481a0c40 con 0x7fdc48073a00 2024-05-08T22:05:37.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.030+0000 7fdc4ee00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc481a0d80 con 0x7fdc48108e30 2024-05-08T22:05:37.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.030+0000 7fdc4ee00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc481a0f10 con 0x7fdc48074fe0 2024-05-08T22:05:37.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.031+0000 7fdc47e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc48073a00 0x7fdc48199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:37.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.031+0000 7fdc47e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc48073a00 0x7fdc48199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42018/0 (socket says 172.21.15.131:42018) 2024-05-08T22:05:37.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.031+0000 7fdc47e00700 1 -- 172.21.15.131:0/3157487769 learned_addr learned my addr 172.21.15.131:0/3157487769 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:37.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.031+0000 7fdc47400700 1 --2- 172.21.15.131:0/3157487769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc48074fe0 0x7fdc48199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:37.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.031+0000 7fdc4ce00700 1 --2- 172.21.15.131:0/3157487769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc48108e30 0x7fdc481a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:37.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.031+0000 7fdc47e00700 1 -- 172.21.15.131:0/3157487769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc48074fe0 msgr2=0x7fdc48199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:37.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.031+0000 7fdc47e00700 1 --2- 172.21.15.131:0/3157487769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc48074fe0 0x7fdc48199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:37.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.031+0000 7fdc47e00700 1 -- 172.21.15.131:0/3157487769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc48108e30 msgr2=0x7fdc481a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:37.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.031+0000 7fdc47e00700 1 --2- 172.21.15.131:0/3157487769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc48108e30 0x7fdc481a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:37.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.031+0000 7fdc47e00700 1 -- 172.21.15.131:0/3157487769 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc380095f0 con 0x7fdc48073a00 2024-05-08T22:05:37.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.031+0000 7fdc47e00700 1 --2- 172.21.15.131:0/3157487769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc48073a00 0x7fdc48199110 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdc380095c0 tx=0x7fdc3800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:37.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.032+0000 7fdc44c00700 1 -- 172.21.15.131:0/3157487769 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3801d070 con 0x7fdc48073a00 2024-05-08T22:05:37.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.032+0000 7fdc4ee00700 1 -- 172.21.15.131:0/3157487769 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc481a1190 con 0x7fdc48073a00 2024-05-08T22:05:37.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.032+0000 7fdc44c00700 1 -- 172.21.15.131:0/3157487769 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc38005030 con 0x7fdc48073a00 2024-05-08T22:05:37.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.032+0000 7fdc44c00700 1 -- 172.21.15.131:0/3157487769 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc38004a50 con 0x7fdc48073a00 2024-05-08T22:05:37.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.032+0000 7fdc4ee00700 1 -- 172.21.15.131:0/3157487769 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc481a1610 con 0x7fdc48073a00 2024-05-08T22:05:37.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.033+0000 7fdc4ee00700 1 -- 172.21.15.131:0/3157487769 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc48107bf0 con 0x7fdc48073a00 2024-05-08T22:05:37.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.035+0000 7fdc44c00700 1 -- 172.21.15.131:0/3157487769 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fdc38005450 con 0x7fdc48073a00 2024-05-08T22:05:37.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.036+0000 7fdc44c00700 1 --2- 172.21.15.131:0/3157487769 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdc2c0a1120 0x7fdc2c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:37.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.036+0000 7fdc44c00700 1 -- 172.21.15.131:0/3157487769 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fdc380cd990 con 0x7fdc48073a00 2024-05-08T22:05:37.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.036+0000 7fdc47400700 1 --2- 172.21.15.131:0/3157487769 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdc2c0a1120 0x7fdc2c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:37.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.037+0000 7fdc47400700 1 --2- 172.21.15.131:0/3157487769 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdc2c0a1120 0x7fdc2c0a35e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdc48074d00 tx=0x7fdc3400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:37.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.040+0000 7fdc44c00700 1 -- 172.21.15.131:0/3157487769 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdc38096f60 con 0x7fdc48073a00 2024-05-08T22:05:37.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:36 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/454714757' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T22:05:37.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:36 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/454714757' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T22:05:37.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:36 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/454714757' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T22:05:37.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.292+0000 7fdc4ee00700 1 -- 172.21.15.131:0/3157487769 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7fdc481075d0 con 0x7fdc48073a00 2024-05-08T22:05:37.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.481+0000 7fdc44c00700 1 -- 172.21.15.131:0/3157487769 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7fdc3809a580 con 0x7fdc48073a00 2024-05-08T22:05:37.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.485+0000 7fdc4ee00700 1 -- 172.21.15.131:0/3157487769 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdc2c0a1120 msgr2=0x7fdc2c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:37.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.485+0000 7fdc4ee00700 1 --2- 172.21.15.131:0/3157487769 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdc2c0a1120 0x7fdc2c0a35e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdc48074d00 tx=0x7fdc3400b410 comp rx=0 tx=0).stop 2024-05-08T22:05:37.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.485+0000 7fdc4ee00700 1 -- 172.21.15.131:0/3157487769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc48073a00 msgr2=0x7fdc48199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:37.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.485+0000 7fdc4ee00700 1 --2- 172.21.15.131:0/3157487769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc48073a00 0x7fdc48199110 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdc380095c0 tx=0x7fdc3800f690 comp rx=0 tx=0).stop 2024-05-08T22:05:37.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.485+0000 7fdc4ee00700 1 -- 172.21.15.131:0/3157487769 shutdown_connections 2024-05-08T22:05:37.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.485+0000 7fdc4ee00700 1 --2- 172.21.15.131:0/3157487769 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdc2c0a1120 0x7fdc2c0a35e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:37.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.485+0000 7fdc4ee00700 1 --2- 172.21.15.131:0/3157487769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc48108e30 0x7fdc481a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:37.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.485+0000 7fdc4ee00700 1 --2- 172.21.15.131:0/3157487769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc48073a00 0x7fdc48199110 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:37.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.485+0000 7fdc4ee00700 1 --2- 172.21.15.131:0/3157487769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc48074fe0 0x7fdc48199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:37.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.485+0000 7fdc4ee00700 1 -- 172.21.15.131:0/3157487769 >> 172.21.15.131:0/3157487769 conn(0x7fdc480fc500 msgr2=0x7fdc48104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:37.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.486+0000 7fdc4ee00700 1 -- 172.21.15.131:0/3157487769 shutdown_connections 2024-05-08T22:05:37.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.486+0000 7fdc4ee00700 1 -- 172.21.15.131:0/3157487769 wait complete. 2024-05-08T22:05:37.511 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-05-08T22:05:37.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.708+0000 7f13b8800700 1 -- 172.21.15.131:0/550887477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13b0102ab0 msgr2=0x7f13b01093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:37.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.708+0000 7f13b8800700 1 --2- 172.21.15.131:0/550887477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13b0102ab0 0x7f13b01093d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00b0a0 tx=0x7f13ac00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:05:37.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.708+0000 7f13b8800700 1 -- 172.21.15.131:0/550887477 shutdown_connections 2024-05-08T22:05:37.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.708+0000 7f13b8800700 1 --2- 172.21.15.131:0/550887477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13b0102ab0 0x7f13b01093d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:37.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.708+0000 7f13b8800700 1 --2- 172.21.15.131:0/550887477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13b0100f20 0x7f13b0101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:37.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.708+0000 7f13b8800700 1 --2- 172.21.15.131:0/550887477 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13b01020f0 0x7f13b0102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:37.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.708+0000 7f13b8800700 1 -- 172.21.15.131:0/550887477 >> 172.21.15.131:0/550887477 conn(0x7f13b00fc500 msgr2=0x7f13b00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:37.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.708+0000 7f13b8800700 1 -- 172.21.15.131:0/550887477 shutdown_connections 2024-05-08T22:05:37.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.708+0000 7f13b8800700 1 -- 172.21.15.131:0/550887477 wait complete. 2024-05-08T22:05:37.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.709+0000 7f13b8800700 1 Processor -- start 2024-05-08T22:05:37.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.709+0000 7f13b8800700 1 -- start start 2024-05-08T22:05:37.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.709+0000 7f13b8800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13b0100f20 0x7f13b0199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:37.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.709+0000 7f13b8800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13b01020f0 0x7f13b0199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:37.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.709+0000 7f13b8800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13b0102ab0 0x7f13b01a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:37.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.709+0000 7f13b8800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b01a0d50 con 0x7f13b01020f0 2024-05-08T22:05:37.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.709+0000 7f13b8800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b01a0e90 con 0x7f13b0100f20 2024-05-08T22:05:37.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.709+0000 7f13b8800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b01a1020 con 0x7f13b0102ab0 2024-05-08T22:05:37.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.710+0000 7f13b5400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13b01020f0 0x7f13b0199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:37.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.710+0000 7f13b5e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13b0100f20 0x7f13b0199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:37.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.710+0000 7f13b5e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13b0100f20 0x7f13b0199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:52552/0 (socket says 172.21.15.131:52552) 2024-05-08T22:05:37.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.710+0000 7f13b5e00700 1 -- 172.21.15.131:0/2234635657 learned_addr learned my addr 172.21.15.131:0/2234635657 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:37.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.710+0000 7f13b6800700 1 --2- 172.21.15.131:0/2234635657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13b0102ab0 0x7f13b01a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:37.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.710+0000 7f13b5400700 1 -- 172.21.15.131:0/2234635657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13b0102ab0 msgr2=0x7f13b01a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:37.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.710+0000 7f13b5400700 1 --2- 172.21.15.131:0/2234635657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13b0102ab0 0x7f13b01a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:37.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.710+0000 7f13b5400700 1 -- 172.21.15.131:0/2234635657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13b0100f20 msgr2=0x7f13b0199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:37.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.710+0000 7f13b5400700 1 --2- 172.21.15.131:0/2234635657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13b0100f20 0x7f13b0199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:37.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.710+0000 7f13b5400700 1 -- 172.21.15.131:0/2234635657 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a0009520 con 0x7f13b01020f0 2024-05-08T22:05:37.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.710+0000 7f13b6800700 1 --2- 172.21.15.131:0/2234635657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13b0102ab0 0x7f13b01a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:37.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.710+0000 7f13b5e00700 1 --2- 172.21.15.131:0/2234635657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13b0100f20 0x7f13b0199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:37.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.710+0000 7f13b5400700 1 --2- 172.21.15.131:0/2234635657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13b01020f0 0x7f13b0199760 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f13a000e9f0 tx=0x7f13a000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:37.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.711+0000 7f13aaa00700 1 -- 172.21.15.131:0/2234635657 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a000cb70 con 0x7f13b01020f0 2024-05-08T22:05:37.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.711+0000 7f13aaa00700 1 -- 172.21.15.131:0/2234635657 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13a000ee90 con 0x7f13b01020f0 2024-05-08T22:05:37.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.711+0000 7f13b8800700 1 -- 172.21.15.131:0/2234635657 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ac009d00 con 0x7f13b01020f0 2024-05-08T22:05:37.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.711+0000 7f13aaa00700 1 -- 172.21.15.131:0/2234635657 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a0003cf0 con 0x7f13b01020f0 2024-05-08T22:05:37.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.711+0000 7f13b8800700 1 -- 172.21.15.131:0/2234635657 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b01a1680 con 0x7f13b01020f0 2024-05-08T22:05:37.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.712+0000 7f13b8800700 1 -- 172.21.15.131:0/2234635657 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b0067cf0 con 0x7f13b01020f0 2024-05-08T22:05:37.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.714+0000 7f13aaa00700 1 -- 172.21.15.131:0/2234635657 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f13a000ccd0 con 0x7f13b01020f0 2024-05-08T22:05:37.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.715+0000 7f13aaa00700 1 --2- 172.21.15.131:0/2234635657 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f13980a1120 0x7f13980a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:37.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.716+0000 7f13aaa00700 1 -- 172.21.15.131:0/2234635657 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f13a0014070 con 0x7f13b01020f0 2024-05-08T22:05:37.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.716+0000 7f13b5e00700 1 --2- 172.21.15.131:0/2234635657 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f13980a1120 0x7f13980a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:37.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.716+0000 7f13b5e00700 1 --2- 172.21.15.131:0/2234635657 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f13980a1120 0x7f13980a35e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f13a4005950 tx=0x7f13a4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:37.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.720+0000 7f13aaa00700 1 -- 172.21.15.131:0/2234635657 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f13a0096330 con 0x7f13b01020f0 2024-05-08T22:05:37.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:37.974+0000 7f13b8800700 1 -- 172.21.15.131:0/2234635657 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f13b010c7e0 con 0x7f13b01020f0 2024-05-08T22:05:38.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:37 smithi132 ceph-mon[78180]: pgmap v251: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-08T22:05:38.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:37 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/454714757' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T22:05:38.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:37 smithi132 ceph-mon[78180]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T22:05:38.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:37 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3157487769' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-08T22:05:38.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:37 smithi132 ceph-mon[78180]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T22:05:38.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:37 smithi132 ceph-mon[78180]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T22:05:38.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:37 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3157487769' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-08T22:05:38.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:37 smithi132 ceph-mon[78180]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-05-08T22:05:38.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:37 smithi132 ceph-mon[78180]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-05-08T22:05:38.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:37 smithi131 ceph-mon[37260]: pgmap v251: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-08T22:05:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:37 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/454714757' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T22:05:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:37 smithi131 ceph-mon[37260]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T22:05:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:37 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3157487769' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-08T22:05:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:37 smithi131 ceph-mon[37260]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T22:05:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:37 smithi131 ceph-mon[37260]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T22:05:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:37 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3157487769' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-08T22:05:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:37 smithi131 ceph-mon[37260]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-05-08T22:05:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:37 smithi131 ceph-mon[37260]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-05-08T22:05:38.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:37 smithi157 ceph-mon[77746]: pgmap v251: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-08T22:05:38.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:37 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/454714757' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T22:05:38.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:37 smithi157 ceph-mon[77746]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T22:05:38.278 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:37 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3157487769' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-08T22:05:38.278 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:37 smithi157 ceph-mon[77746]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T22:05:38.278 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:37 smithi157 ceph-mon[77746]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T22:05:38.278 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:37 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3157487769' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-08T22:05:38.278 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:37 smithi157 ceph-mon[77746]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-05-08T22:05:38.278 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:37 smithi157 ceph-mon[77746]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-05-08T22:05:38.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.483+0000 7f13aaa00700 1 -- 172.21.15.131:0/2234635657 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f13a0099950 con 0x7f13b01020f0 2024-05-08T22:05:38.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.486+0000 7f13b8800700 1 -- 172.21.15.131:0/2234635657 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f13980a1120 msgr2=0x7f13980a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:38.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.486+0000 7f13b8800700 1 --2- 172.21.15.131:0/2234635657 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f13980a1120 0x7f13980a35e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f13a4005950 tx=0x7f13a4009260 comp rx=0 tx=0).stop 2024-05-08T22:05:38.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.486+0000 7f13b8800700 1 -- 172.21.15.131:0/2234635657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13b01020f0 msgr2=0x7f13b0199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:38.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.486+0000 7f13b8800700 1 --2- 172.21.15.131:0/2234635657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13b01020f0 0x7f13b0199760 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f13a000e9f0 tx=0x7f13a000ed00 comp rx=0 tx=0).stop 2024-05-08T22:05:38.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.486+0000 7f13b8800700 1 -- 172.21.15.131:0/2234635657 shutdown_connections 2024-05-08T22:05:38.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.486+0000 7f13b8800700 1 --2- 172.21.15.131:0/2234635657 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f13980a1120 0x7f13980a35e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:38.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.486+0000 7f13b8800700 1 --2- 172.21.15.131:0/2234635657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13b0102ab0 0x7f13b01a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:38.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.486+0000 7f13b8800700 1 --2- 172.21.15.131:0/2234635657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13b0100f20 0x7f13b0199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:38.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.486+0000 7f13b8800700 1 --2- 172.21.15.131:0/2234635657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13b01020f0 0x7f13b0199760 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:38.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.486+0000 7f13b8800700 1 -- 172.21.15.131:0/2234635657 >> 172.21.15.131:0/2234635657 conn(0x7f13b00fc500 msgr2=0x7f13b00731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:38.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.486+0000 7f13b8800700 1 -- 172.21.15.131:0/2234635657 shutdown_connections 2024-05-08T22:05:38.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.486+0000 7f13b8800700 1 -- 172.21.15.131:0/2234635657 wait complete. 2024-05-08T22:05:38.514 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-05-08T22:05:38.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.709+0000 7fc579200700 1 -- 172.21.15.131:0/3287653935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc574101fa0 msgr2=0x7fc574102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:38.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.709+0000 7fc579200700 1 --2- 172.21.15.131:0/3287653935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc574101fa0 0x7fc574102400 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc5600099c0 tx=0x7fc560009cd0 comp rx=0 tx=0).stop 2024-05-08T22:05:38.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.709+0000 7fc579200700 1 -- 172.21.15.131:0/3287653935 shutdown_connections 2024-05-08T22:05:38.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.709+0000 7fc579200700 1 --2- 172.21.15.131:0/3287653935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc574102940 0x7fc5741091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:38.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.709+0000 7fc579200700 1 --2- 172.21.15.131:0/3287653935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc574100db0 0x7fc5741011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:38.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.709+0000 7fc579200700 1 --2- 172.21.15.131:0/3287653935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc574101fa0 0x7fc574102400 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:38.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.709+0000 7fc579200700 1 -- 172.21.15.131:0/3287653935 >> 172.21.15.131:0/3287653935 conn(0x7fc5740fc2e0 msgr2=0x7fc5740fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:38.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.709+0000 7fc579200700 1 -- 172.21.15.131:0/3287653935 shutdown_connections 2024-05-08T22:05:38.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.709+0000 7fc579200700 1 -- 172.21.15.131:0/3287653935 wait complete. 2024-05-08T22:05:38.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.710+0000 7fc579200700 1 Processor -- start 2024-05-08T22:05:38.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.710+0000 7fc579200700 1 -- start start 2024-05-08T22:05:38.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.711+0000 7fc579200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc574100db0 0x7fc574198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:38.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.711+0000 7fc579200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc574101fa0 0x7fc5741994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:38.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.711+0000 7fc579200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc574102940 0x7fc5741a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:38.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.711+0000 7fc579200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5741a0ae0 con 0x7fc574102940 2024-05-08T22:05:38.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.711+0000 7fc579200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5741a0c20 con 0x7fc574100db0 2024-05-08T22:05:38.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.711+0000 7fc579200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5741a0db0 con 0x7fc574101fa0 2024-05-08T22:05:38.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.711+0000 7fc573e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc574102940 0x7fc5741a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.711+0000 7fc572a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc574101fa0 0x7fc5741994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.711+0000 7fc573e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc574102940 0x7fc5741a0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42060/0 (socket says 172.21.15.131:42060) 2024-05-08T22:05:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.711+0000 7fc573e00700 1 -- 172.21.15.131:0/521772639 learned_addr learned my addr 172.21.15.131:0/521772639 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.711+0000 7fc573400700 1 --2- 172.21.15.131:0/521772639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc574100db0 0x7fc574198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.711+0000 7fc572a00700 1 -- 172.21.15.131:0/521772639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc574100db0 msgr2=0x7fc574198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.711+0000 7fc572a00700 1 --2- 172.21.15.131:0/521772639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc574100db0 0x7fc574198fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.711+0000 7fc572a00700 1 -- 172.21.15.131:0/521772639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc574102940 msgr2=0x7fc5741a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.711+0000 7fc572a00700 1 --2- 172.21.15.131:0/521772639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc574102940 0x7fc5741a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.711+0000 7fc572a00700 1 -- 172.21.15.131:0/521772639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5600095f0 con 0x7fc574101fa0 2024-05-08T22:05:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.711+0000 7fc573400700 1 --2- 172.21.15.131:0/521772639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc574100db0 0x7fc574198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:05:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.712+0000 7fc573e00700 1 --2- 172.21.15.131:0/521772639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc574102940 0x7fc5741a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:38.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.712+0000 7fc572a00700 1 --2- 172.21.15.131:0/521772639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc574101fa0 0x7fc5741994f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc5600098e0 tx=0x7fc56000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:38.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.712+0000 7fc56be00700 1 -- 172.21.15.131:0/521772639 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56001d070 con 0x7fc574101fa0 2024-05-08T22:05:38.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.712+0000 7fc579200700 1 -- 172.21.15.131:0/521772639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5741a1030 con 0x7fc574101fa0 2024-05-08T22:05:38.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.712+0000 7fc56be00700 1 -- 172.21.15.131:0/521772639 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc560005020 con 0x7fc574101fa0 2024-05-08T22:05:38.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.712+0000 7fc56be00700 1 -- 172.21.15.131:0/521772639 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc560004ac0 con 0x7fc574101fa0 2024-05-08T22:05:38.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.712+0000 7fc579200700 1 -- 172.21.15.131:0/521772639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5741a1400 con 0x7fc574101fa0 2024-05-08T22:05:38.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.713+0000 7fc579200700 1 -- 172.21.15.131:0/521772639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc574067ba0 con 0x7fc574101fa0 2024-05-08T22:05:38.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.715+0000 7fc56be00700 1 -- 172.21.15.131:0/521772639 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc560005190 con 0x7fc574101fa0 2024-05-08T22:05:38.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.716+0000 7fc56be00700 1 --2- 172.21.15.131:0/521772639 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc5580a1120 0x7fc5580a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:38.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.716+0000 7fc56be00700 1 -- 172.21.15.131:0/521772639 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc5600cd740 con 0x7fc574101fa0 2024-05-08T22:05:38.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.717+0000 7fc573400700 1 --2- 172.21.15.131:0/521772639 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc5580a1120 0x7fc5580a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.717+0000 7fc573400700 1 --2- 172.21.15.131:0/521772639 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc5580a1120 0x7fc5580a35e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc574101e00 tx=0x7fc564009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.719+0000 7fc56be00700 1 -- 172.21.15.131:0/521772639 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc560096d10 con 0x7fc574101fa0 2024-05-08T22:05:38.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:38.974+0000 7fc579200700 1 -- 172.21.15.131:0/521772639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fc57410c630 con 0x7fc574101fa0 2024-05-08T22:05:39.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:38 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2234635657' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T22:05:39.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:38 smithi132 ceph-mon[78180]: mds.? [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] up:active 2024-05-08T22:05:39.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:38 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2234635657' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T22:05:39.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:38 smithi132 ceph-mon[78180]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T22:05:39.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:38 smithi132 ceph-mon[78180]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T22:05:39.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:38 smithi132 ceph-mon[78180]: Cluster is now healthy 2024-05-08T22:05:39.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:38 smithi132 ceph-mon[78180]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-05-08T22:05:39.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:38 smithi132 ceph-mon[78180]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:creating} 9 up:standby 2024-05-08T22:05:39.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:38 smithi132 ceph-mon[78180]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-05-08T22:05:39.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:38 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2234635657' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T22:05:39.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:38 smithi131 ceph-mon[37260]: mds.? [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] up:active 2024-05-08T22:05:39.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:38 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2234635657' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T22:05:39.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:38 smithi131 ceph-mon[37260]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T22:05:39.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:38 smithi131 ceph-mon[37260]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T22:05:39.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:38 smithi131 ceph-mon[37260]: Cluster is now healthy 2024-05-08T22:05:39.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:38 smithi131 ceph-mon[37260]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-05-08T22:05:39.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:38 smithi131 ceph-mon[37260]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:creating} 9 up:standby 2024-05-08T22:05:39.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:38 smithi131 ceph-mon[37260]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-05-08T22:05:39.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:38 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2234635657' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T22:05:39.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:38 smithi157 ceph-mon[77746]: mds.? [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] up:active 2024-05-08T22:05:39.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:38 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2234635657' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T22:05:39.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:38 smithi157 ceph-mon[77746]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T22:05:39.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:38 smithi157 ceph-mon[77746]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T22:05:39.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:38 smithi157 ceph-mon[77746]: Cluster is now healthy 2024-05-08T22:05:39.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:38 smithi157 ceph-mon[77746]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-05-08T22:05:39.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:38 smithi157 ceph-mon[77746]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:creating} 9 up:standby 2024-05-08T22:05:39.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:38 smithi157 ceph-mon[77746]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-05-08T22:05:39.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.494+0000 7fc56be00700 1 -- 172.21.15.131:0/521772639 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7fc56009a330 con 0x7fc574101fa0 2024-05-08T22:05:39.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.497+0000 7fc579200700 1 -- 172.21.15.131:0/521772639 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc5580a1120 msgr2=0x7fc5580a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:39.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.497+0000 7fc579200700 1 --2- 172.21.15.131:0/521772639 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc5580a1120 0x7fc5580a35e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc574101e00 tx=0x7fc564009260 comp rx=0 tx=0).stop 2024-05-08T22:05:39.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.497+0000 7fc579200700 1 -- 172.21.15.131:0/521772639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc574101fa0 msgr2=0x7fc5741994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:39.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.497+0000 7fc579200700 1 --2- 172.21.15.131:0/521772639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc574101fa0 0x7fc5741994f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc5600098e0 tx=0x7fc56000f690 comp rx=0 tx=0).stop 2024-05-08T22:05:39.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.497+0000 7fc579200700 1 -- 172.21.15.131:0/521772639 shutdown_connections 2024-05-08T22:05:39.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.497+0000 7fc579200700 1 --2- 172.21.15.131:0/521772639 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc5580a1120 0x7fc5580a35e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:39.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.497+0000 7fc579200700 1 --2- 172.21.15.131:0/521772639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc574102940 0x7fc5741a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:39.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.497+0000 7fc579200700 1 --2- 172.21.15.131:0/521772639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc574100db0 0x7fc574198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:39.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.497+0000 7fc579200700 1 --2- 172.21.15.131:0/521772639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc574101fa0 0x7fc5741994f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:39.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.497+0000 7fc579200700 1 -- 172.21.15.131:0/521772639 >> 172.21.15.131:0/521772639 conn(0x7fc5740fc2e0 msgr2=0x7fc5740731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:39.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.497+0000 7fc579200700 1 -- 172.21.15.131:0/521772639 shutdown_connections 2024-05-08T22:05:39.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.497+0000 7fc579200700 1 -- 172.21.15.131:0/521772639 wait complete. 2024-05-08T22:05:39.519 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-05-08T22:05:39.519 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-05-08T22:05:39.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.717+0000 7ff7ff400700 1 -- 172.21.15.131:0/1348540935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f8102ab0 msgr2=0x7ff7f81093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:39.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.717+0000 7ff7ff400700 1 --2- 172.21.15.131:0/1348540935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f8102ab0 0x7ff7f81093d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009f90 tx=0x7ff7f000b330 comp rx=0 tx=0).stop 2024-05-08T22:05:39.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.717+0000 7ff7ff400700 1 -- 172.21.15.131:0/1348540935 shutdown_connections 2024-05-08T22:05:39.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.717+0000 7ff7ff400700 1 --2- 172.21.15.131:0/1348540935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f8102ab0 0x7ff7f81093d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:39.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.717+0000 7ff7ff400700 1 --2- 172.21.15.131:0/1348540935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7f8100f20 0x7ff7f8101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:39.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.717+0000 7ff7ff400700 1 --2- 172.21.15.131:0/1348540935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7f81020f0 0x7ff7f8102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:39.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.717+0000 7ff7ff400700 1 -- 172.21.15.131:0/1348540935 >> 172.21.15.131:0/1348540935 conn(0x7ff7f80fc500 msgr2=0x7ff7f80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:39.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.717+0000 7ff7ff400700 1 -- 172.21.15.131:0/1348540935 shutdown_connections 2024-05-08T22:05:39.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.717+0000 7ff7ff400700 1 -- 172.21.15.131:0/1348540935 wait complete. 2024-05-08T22:05:39.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.718+0000 7ff7ff400700 1 Processor -- start 2024-05-08T22:05:39.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.718+0000 7ff7ff400700 1 -- start start 2024-05-08T22:05:39.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.719+0000 7ff7ff400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f8100f20 0x7ff7f8199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:39.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.719+0000 7ff7ff400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7f81020f0 0x7ff7f8199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:39.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.719+0000 7ff7ff400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7f8102ab0 0x7ff7f81a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:39.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.719+0000 7ff7ff400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f81a0d70 con 0x7ff7f81020f0 2024-05-08T22:05:39.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.719+0000 7ff7ff400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f81a0eb0 con 0x7ff7f8100f20 2024-05-08T22:05:39.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.719+0000 7ff7ff400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f81a1040 con 0x7ff7f8102ab0 2024-05-08T22:05:39.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.719+0000 7ff7fca00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f8100f20 0x7ff7f8199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:39.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.719+0000 7ff7fca00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f8100f20 0x7ff7f8199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:52598/0 (socket says 172.21.15.131:52598) 2024-05-08T22:05:39.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.719+0000 7ff7fca00700 1 -- 172.21.15.131:0/895608911 learned_addr learned my addr 172.21.15.131:0/895608911 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:39.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.719+0000 7ff7f7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7f81020f0 0x7ff7f8199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:39.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.719+0000 7ff7fd400700 1 --2- 172.21.15.131:0/895608911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7f8102ab0 0x7ff7f81a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:39.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.719+0000 7ff7fca00700 1 -- 172.21.15.131:0/895608911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7f8102ab0 msgr2=0x7ff7f81a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:39.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.719+0000 7ff7fca00700 1 --2- 172.21.15.131:0/895608911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7f8102ab0 0x7ff7f81a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:39.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.719+0000 7ff7fca00700 1 -- 172.21.15.131:0/895608911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7f81020f0 msgr2=0x7ff7f8199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:39.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.719+0000 7ff7fca00700 1 --2- 172.21.15.131:0/895608911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7f81020f0 0x7ff7f8199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:39.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.719+0000 7ff7fca00700 1 -- 172.21.15.131:0/895608911 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e4009520 con 0x7ff7f8100f20 2024-05-08T22:05:39.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.720+0000 7ff7fca00700 1 --2- 172.21.15.131:0/895608911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f8100f20 0x7ff7f8199240 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00e9f0 tx=0x7ff7ec00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:39.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.720+0000 7ff7f5600700 1 -- 172.21.15.131:0/895608911 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec004220 con 0x7ff7f8100f20 2024-05-08T22:05:39.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.720+0000 7ff7ff400700 1 -- 172.21.15.131:0/895608911 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f0009c40 con 0x7ff7f8100f20 2024-05-08T22:05:39.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.720+0000 7ff7f5600700 1 -- 172.21.15.131:0/895608911 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7ec004380 con 0x7ff7f8100f20 2024-05-08T22:05:39.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.720+0000 7ff7f5600700 1 -- 172.21.15.131:0/895608911 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec003be0 con 0x7ff7f8100f20 2024-05-08T22:05:39.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.720+0000 7ff7ff400700 1 -- 172.21.15.131:0/895608911 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f81a16a0 con 0x7ff7f8100f20 2024-05-08T22:05:39.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.721+0000 7ff7ff400700 1 -- 172.21.15.131:0/895608911 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f8067cf0 con 0x7ff7f8100f20 2024-05-08T22:05:39.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.723+0000 7ff7f5600700 1 -- 172.21.15.131:0/895608911 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff7ec003d40 con 0x7ff7f8100f20 2024-05-08T22:05:39.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.724+0000 7ff7f5600700 1 --2- 172.21.15.131:0/895608911 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff7e00a1120 0x7ff7e00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:39.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.724+0000 7ff7f5600700 1 -- 172.21.15.131:0/895608911 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff7ec014070 con 0x7ff7f8100f20 2024-05-08T22:05:39.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.725+0000 7ff7f7e00700 1 --2- 172.21.15.131:0/895608911 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff7e00a1120 0x7ff7e00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:39.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.725+0000 7ff7f7e00700 1 --2- 172.21.15.131:0/895608911 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff7e00a1120 0x7ff7e00a35e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009df0 tx=0x7ff7e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:39.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.727+0000 7ff7f5600700 1 -- 172.21.15.131:0/895608911 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff7ec096210 con 0x7ff7f8100f20 2024-05-08T22:05:39.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.967+0000 7ff7ff400700 1 -- 172.21.15.131:0/895608911 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7f80748f0 con 0x7ff7e00a1120 2024-05-08T22:05:39.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.993+0000 7ff7f5600700 1 -- 172.21.15.131:0/895608911 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff7f80748f0 con 0x7ff7e00a1120 2024-05-08T22:05:39.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.996+0000 7ff7ff400700 1 -- 172.21.15.131:0/895608911 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff7e00a1120 msgr2=0x7ff7e00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:39.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.996+0000 7ff7ff400700 1 --2- 172.21.15.131:0/895608911 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff7e00a1120 0x7ff7e00a35e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009df0 tx=0x7ff7e4009260 comp rx=0 tx=0).stop 2024-05-08T22:05:39.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.996+0000 7ff7ff400700 1 -- 172.21.15.131:0/895608911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f8100f20 msgr2=0x7ff7f8199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:39.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.996+0000 7ff7ff400700 1 --2- 172.21.15.131:0/895608911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f8100f20 0x7ff7f8199240 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00e9f0 tx=0x7ff7ec00ed00 comp rx=0 tx=0).stop 2024-05-08T22:05:39.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.996+0000 7ff7ff400700 1 -- 172.21.15.131:0/895608911 shutdown_connections 2024-05-08T22:05:39.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.996+0000 7ff7ff400700 1 --2- 172.21.15.131:0/895608911 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff7e00a1120 0x7ff7e00a35e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:39.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.996+0000 7ff7ff400700 1 --2- 172.21.15.131:0/895608911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7f8102ab0 0x7ff7f81a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:39.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.996+0000 7ff7ff400700 1 --2- 172.21.15.131:0/895608911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f8100f20 0x7ff7f8199240 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:39.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.996+0000 7ff7ff400700 1 --2- 172.21.15.131:0/895608911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7f81020f0 0x7ff7f8199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:39.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.996+0000 7ff7ff400700 1 -- 172.21.15.131:0/895608911 >> 172.21.15.131:0/895608911 conn(0x7ff7f80fc500 msgr2=0x7ff7f80731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:39.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.996+0000 7ff7ff400700 1 -- 172.21.15.131:0/895608911 shutdown_connections 2024-05-08T22:05:39.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:39.996+0000 7ff7ff400700 1 -- 172.21.15.131:0/895608911 wait complete. 2024-05-08T22:05:40.015 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-08T22:05:40.072 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:05:39 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:05:39.974+0000 7f9d61c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:05:40.073 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:05:39 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:05:39.974+0000 7f9d61c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:05:40.073 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:05:39 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:05:39.974+0000 7f9d61c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:05:40.073 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:05:39 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:05:39.974+0000 7f9d61c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:05:40.073 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:05:39 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:05:39.974+0000 7f9d61c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:05:40.073 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:39 smithi131 ceph-mon[37260]: pgmap v252: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 2 op/s 2024-05-08T22:05:40.073 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:39 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/521772639' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T22:05:40.073 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:39 smithi131 ceph-mon[37260]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T22:05:40.073 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:39 smithi131 ceph-mon[37260]: mds.? [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] up:active 2024-05-08T22:05:40.073 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:39 smithi131 ceph-mon[37260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T22:05:40.074 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:39 smithi131 ceph-mon[37260]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-05-08T22:05:40.074 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:39 smithi131 ceph-mon[37260]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T22:05:40.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:39 smithi132 ceph-mon[78180]: pgmap v252: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 2 op/s 2024-05-08T22:05:40.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:39 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/521772639' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T22:05:40.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:39 smithi132 ceph-mon[78180]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T22:05:40.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:39 smithi132 ceph-mon[78180]: mds.? [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] up:active 2024-05-08T22:05:40.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:39 smithi132 ceph-mon[78180]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T22:05:40.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:39 smithi132 ceph-mon[78180]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-05-08T22:05:40.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:39 smithi132 ceph-mon[78180]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T22:05:40.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.213+0000 7f5c73200700 1 -- 172.21.15.131:0/2650099302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c6c100dc0 msgr2=0x7f5c6c1011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:40.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.213+0000 7f5c73200700 1 --2- 172.21.15.131:0/2650099302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c6c100dc0 0x7f5c6c1011e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009990 tx=0x7f5c60009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:40.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.213+0000 7f5c73200700 1 -- 172.21.15.131:0/2650099302 shutdown_connections 2024-05-08T22:05:40.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.213+0000 7f5c73200700 1 --2- 172.21.15.131:0/2650099302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c6c102950 0x7f5c6c109200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:40.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.213+0000 7f5c73200700 1 --2- 172.21.15.131:0/2650099302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c6c100dc0 0x7f5c6c1011e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:40.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.213+0000 7f5c73200700 1 --2- 172.21.15.131:0/2650099302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c6c101fb0 0x7f5c6c102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:40.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.213+0000 7f5c73200700 1 -- 172.21.15.131:0/2650099302 >> 172.21.15.131:0/2650099302 conn(0x7f5c6c0fc350 msgr2=0x7f5c6c0fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:40.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.213+0000 7f5c73200700 1 -- 172.21.15.131:0/2650099302 shutdown_connections 2024-05-08T22:05:40.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.213+0000 7f5c73200700 1 -- 172.21.15.131:0/2650099302 wait complete. 2024-05-08T22:05:40.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.215+0000 7f5c73200700 1 Processor -- start 2024-05-08T22:05:40.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.215+0000 7f5c73200700 1 -- start start 2024-05-08T22:05:40.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.215+0000 7f5c73200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c6c100dc0 0x7f5c6c19d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:40.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.216+0000 7f5c73200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c6c101fb0 0x7f5c6c19d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:40.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.216+0000 7f5c73200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c6c102950 0x7f5c6c1a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:40.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.216+0000 7f5c73200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c1a4f30 con 0x7f5c6c100dc0 2024-05-08T22:05:40.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.216+0000 7f5c73200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c1a5070 con 0x7f5c6c101fb0 2024-05-08T22:05:40.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.216+0000 7f5c73200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c1a5200 con 0x7f5c6c102950 2024-05-08T22:05:40.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.216+0000 7f5c71e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c6c100dc0 0x7f5c6c19d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:40.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.216+0000 7f5c71e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c6c100dc0 0x7f5c6c19d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42096/0 (socket says 172.21.15.131:42096) 2024-05-08T22:05:40.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.216+0000 7f5c71e00700 1 -- 172.21.15.131:0/1390077779 learned_addr learned my addr 172.21.15.131:0/1390077779 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:40.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.216+0000 7f5c72800700 1 --2- 172.21.15.131:0/1390077779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c6c102950 0x7f5c6c1a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:40.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.216+0000 7f5c71400700 1 --2- 172.21.15.131:0/1390077779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c6c101fb0 0x7f5c6c19d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:40.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.216+0000 7f5c71e00700 1 -- 172.21.15.131:0/1390077779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c6c102950 msgr2=0x7f5c6c1a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:40.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.216+0000 7f5c71e00700 1 --2- 172.21.15.131:0/1390077779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c6c102950 0x7f5c6c1a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:40.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.216+0000 7f5c71e00700 1 -- 172.21.15.131:0/1390077779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c6c101fb0 msgr2=0x7f5c6c19d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:40.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.216+0000 7f5c71e00700 1 --2- 172.21.15.131:0/1390077779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c6c101fb0 0x7f5c6c19d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:40.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.216+0000 7f5c71e00700 1 -- 172.21.15.131:0/1390077779 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c600095f0 con 0x7f5c6c100dc0 2024-05-08T22:05:40.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.216+0000 7f5c72800700 1 --2- 172.21.15.131:0/1390077779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c6c102950 0x7f5c6c1a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:40.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.217+0000 7f5c71e00700 1 --2- 172.21.15.131:0/1390077779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c6c100dc0 0x7f5c6c19d400 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000b680 tx=0x7f5c60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:40.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.217+0000 7f5c66a00700 1 -- 172.21.15.131:0/1390077779 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6001d070 con 0x7f5c6c100dc0 2024-05-08T22:05:40.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.217+0000 7f5c73200700 1 -- 172.21.15.131:0/1390077779 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c6c1a5480 con 0x7f5c6c100dc0 2024-05-08T22:05:40.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.217+0000 7f5c66a00700 1 -- 172.21.15.131:0/1390077779 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c60021410 con 0x7f5c6c100dc0 2024-05-08T22:05:40.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.217+0000 7f5c73200700 1 -- 172.21.15.131:0/1390077779 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6c1a5900 con 0x7f5c6c100dc0 2024-05-08T22:05:40.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.217+0000 7f5c66a00700 1 -- 172.21.15.131:0/1390077779 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c60005220 con 0x7f5c6c100dc0 2024-05-08T22:05:40.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.218+0000 7f5c73200700 1 -- 172.21.15.131:0/1390077779 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c6c067ff0 con 0x7f5c6c100dc0 2024-05-08T22:05:40.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.220+0000 7f5c66a00700 1 -- 172.21.15.131:0/1390077779 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f5c60021580 con 0x7f5c6c100dc0 2024-05-08T22:05:40.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.221+0000 7f5c66a00700 1 --2- 172.21.15.131:0/1390077779 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f5c540a11a0 0x7f5c540a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:40.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.221+0000 7f5c66a00700 1 -- 172.21.15.131:0/1390077779 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5c600ce460 con 0x7f5c6c100dc0 2024-05-08T22:05:40.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.222+0000 7f5c71400700 1 --2- 172.21.15.131:0/1390077779 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f5c540a11a0 0x7f5c540a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:40.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.222+0000 7f5c71400700 1 --2- 172.21.15.131:0/1390077779 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f5c540a11a0 0x7f5c540a3660 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c009c60 tx=0x7f5c5c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:40.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.224+0000 7f5c66a00700 1 -- 172.21.15.131:0/1390077779 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5c60097a30 con 0x7f5c6c100dc0 2024-05-08T22:05:40.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:39 smithi157 ceph-mon[77746]: pgmap v252: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 2 op/s 2024-05-08T22:05:40.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:39 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/521772639' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T22:05:40.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:39 smithi157 ceph-mon[77746]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T22:05:40.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:39 smithi157 ceph-mon[77746]: mds.? [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] up:active 2024-05-08T22:05:40.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:39 smithi157 ceph-mon[77746]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T22:05:40.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:39 smithi157 ceph-mon[77746]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-05-08T22:05:40.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:39 smithi157 ceph-mon[77746]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T22:05:40.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.465+0000 7f5c73200700 1 -- 172.21.15.131:0/1390077779 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c6c0748f0 con 0x7f5c540a11a0 2024-05-08T22:05:40.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.489+0000 7f5c66a00700 1 -- 172.21.15.131:0/1390077779 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5c6c0748f0 con 0x7f5c540a11a0 2024-05-08T22:05:40.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.492+0000 7f5c73200700 1 -- 172.21.15.131:0/1390077779 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f5c540a11a0 msgr2=0x7f5c540a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:40.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.492+0000 7f5c73200700 1 --2- 172.21.15.131:0/1390077779 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f5c540a11a0 0x7f5c540a3660 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c009c60 tx=0x7f5c5c009260 comp rx=0 tx=0).stop 2024-05-08T22:05:40.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.492+0000 7f5c73200700 1 -- 172.21.15.131:0/1390077779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c6c100dc0 msgr2=0x7f5c6c19d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:40.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.492+0000 7f5c73200700 1 --2- 172.21.15.131:0/1390077779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c6c100dc0 0x7f5c6c19d400 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000b680 tx=0x7f5c60004730 comp rx=0 tx=0).stop 2024-05-08T22:05:40.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.492+0000 7f5c73200700 1 -- 172.21.15.131:0/1390077779 shutdown_connections 2024-05-08T22:05:40.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.492+0000 7f5c73200700 1 --2- 172.21.15.131:0/1390077779 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f5c540a11a0 0x7f5c540a3660 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:40.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.492+0000 7f5c73200700 1 --2- 172.21.15.131:0/1390077779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c6c102950 0x7f5c6c1a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:40.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.492+0000 7f5c73200700 1 --2- 172.21.15.131:0/1390077779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c6c100dc0 0x7f5c6c19d400 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:40.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.492+0000 7f5c73200700 1 --2- 172.21.15.131:0/1390077779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c6c101fb0 0x7f5c6c19d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:40.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.492+0000 7f5c73200700 1 -- 172.21.15.131:0/1390077779 >> 172.21.15.131:0/1390077779 conn(0x7f5c6c0fc350 msgr2=0x7f5c6c0731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:40.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.492+0000 7f5c73200700 1 -- 172.21.15.131:0/1390077779 shutdown_connections 2024-05-08T22:05:40.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.492+0000 7f5c73200700 1 -- 172.21.15.131:0/1390077779 wait complete. 2024-05-08T22:05:40.512 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-08T22:05:40.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.712+0000 7fb184600700 1 -- 172.21.15.131:0/349164496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb17c102b10 msgr2=0x7fb17c1093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:40.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.712+0000 7fb184600700 1 --2- 172.21.15.131:0/349164496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb17c102b10 0x7fb17c1093c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb17400b0d0 tx=0x7fb17400b3e0 comp rx=0 tx=0).stop 2024-05-08T22:05:40.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.713+0000 7fb184600700 1 -- 172.21.15.131:0/349164496 shutdown_connections 2024-05-08T22:05:40.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.713+0000 7fb184600700 1 --2- 172.21.15.131:0/349164496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb17c102b10 0x7fb17c1093c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:40.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.713+0000 7fb184600700 1 --2- 172.21.15.131:0/349164496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb17c100f80 0x7fb17c1013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:40.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.713+0000 7fb184600700 1 --2- 172.21.15.131:0/349164496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb17c102170 0x7fb17c1025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:40.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.713+0000 7fb184600700 1 -- 172.21.15.131:0/349164496 >> 172.21.15.131:0/349164496 conn(0x7fb17c0fc4f0 msgr2=0x7fb17c0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:40.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.713+0000 7fb184600700 1 -- 172.21.15.131:0/349164496 shutdown_connections 2024-05-08T22:05:40.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.713+0000 7fb184600700 1 -- 172.21.15.131:0/349164496 wait complete. 2024-05-08T22:05:40.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.714+0000 7fb184600700 1 Processor -- start 2024-05-08T22:05:40.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.714+0000 7fb184600700 1 -- start start 2024-05-08T22:05:40.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.714+0000 7fb184600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb17c100f80 0x7fb17c196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:40.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.714+0000 7fb184600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb17c102170 0x7fb17c197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:40.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.714+0000 7fb184600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb17c102b10 0x7fb17c19e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:40.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.714+0000 7fb184600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c19ea60 con 0x7fb17c102170 2024-05-08T22:05:40.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.714+0000 7fb184600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c19eba0 con 0x7fb17c100f80 2024-05-08T22:05:40.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.714+0000 7fb184600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c19ed30 con 0x7fb17c102b10 2024-05-08T22:05:40.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.715+0000 7fb181200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb17c102170 0x7fb17c197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:40.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.715+0000 7fb181200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb17c102170 0x7fb17c197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42122/0 (socket says 172.21.15.131:42122) 2024-05-08T22:05:40.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.715+0000 7fb181200700 1 -- 172.21.15.131:0/2591930624 learned_addr learned my addr 172.21.15.131:0/2591930624 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:40.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.715+0000 7fb181c00700 1 --2- 172.21.15.131:0/2591930624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb17c100f80 0x7fb17c196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:40.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.715+0000 7fb182600700 1 --2- 172.21.15.131:0/2591930624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb17c102b10 0x7fb17c19e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:40.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.715+0000 7fb181c00700 1 -- 172.21.15.131:0/2591930624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb17c102b10 msgr2=0x7fb17c19e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:40.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.715+0000 7fb181c00700 1 --2- 172.21.15.131:0/2591930624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb17c102b10 0x7fb17c19e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:40.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.715+0000 7fb181c00700 1 -- 172.21.15.131:0/2591930624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb17c102170 msgr2=0x7fb17c197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:40.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.715+0000 7fb181c00700 1 --2- 172.21.15.131:0/2591930624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb17c102170 0x7fb17c197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:40.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.715+0000 7fb181c00700 1 -- 172.21.15.131:0/2591930624 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb174009d00 con 0x7fb17c100f80 2024-05-08T22:05:40.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.715+0000 7fb181200700 1 --2- 172.21.15.131:0/2591930624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb17c102170 0x7fb17c197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:40.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.715+0000 7fb181c00700 1 --2- 172.21.15.131:0/2591930624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb17c100f80 0x7fb17c196f30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb17000e9c0 tx=0x7fb17000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:40.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.716+0000 7fb17a000700 1 -- 172.21.15.131:0/2591930624 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb170004220 con 0x7fb17c100f80 2024-05-08T22:05:40.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.716+0000 7fb17a000700 1 -- 172.21.15.131:0/2591930624 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb170004380 con 0x7fb17c100f80 2024-05-08T22:05:40.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.716+0000 7fb184600700 1 -- 172.21.15.131:0/2591930624 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb17c19efb0 con 0x7fb17c100f80 2024-05-08T22:05:40.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.716+0000 7fb17a000700 1 -- 172.21.15.131:0/2591930624 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb170003be0 con 0x7fb17c100f80 2024-05-08T22:05:40.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.716+0000 7fb184600700 1 -- 172.21.15.131:0/2591930624 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb17c19f430 con 0x7fb17c100f80 2024-05-08T22:05:40.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.717+0000 7fb184600700 1 -- 172.21.15.131:0/2591930624 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb17c1083b0 con 0x7fb17c100f80 2024-05-08T22:05:40.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.719+0000 7fb17a000700 1 -- 172.21.15.131:0/2591930624 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb170003d40 con 0x7fb17c100f80 2024-05-08T22:05:40.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.721+0000 7fb17a000700 1 --2- 172.21.15.131:0/2591930624 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb1640a1120 0x7fb1640a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:40.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.721+0000 7fb17a000700 1 -- 172.21.15.131:0/2591930624 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb170014070 con 0x7fb17c100f80 2024-05-08T22:05:40.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.721+0000 7fb181200700 1 --2- 172.21.15.131:0/2591930624 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb1640a1120 0x7fb1640a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:40.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.721+0000 7fb181200700 1 --2- 172.21.15.131:0/2591930624 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb1640a1120 0x7fb1640a35e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb168009ea0 tx=0x7fb168009210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.725+0000 7fb17a000700 1 -- 172.21.15.131:0/2591930624 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb1700962f0 con 0x7fb17c100f80 2024-05-08T22:05:40.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.828+0000 7fb17a000700 1 -- 172.21.15.131:0/2591930624 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fb1700966c0 con 0x7fb17c100f80 2024-05-08T22:05:40.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.981+0000 7fb184600700 1 -- 172.21.15.131:0/2591930624 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fb17c02ce40 con 0x7fb1640a1120 2024-05-08T22:05:40.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.985+0000 7fb17a000700 1 -- 172.21.15.131:0/2591930624 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fb17c02ce40 con 0x7fb1640a1120 2024-05-08T22:05:40.987 INFO:teuthology.orchestra.run.smithi131.stdout:/volumes/qa/sv_0/4330715f-67ae-4ed8-b1a3-5f7ef9b6899d 2024-05-08T22:05:40.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.988+0000 7fb184600700 1 -- 172.21.15.131:0/2591930624 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb1640a1120 msgr2=0x7fb1640a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:40.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.988+0000 7fb184600700 1 --2- 172.21.15.131:0/2591930624 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb1640a1120 0x7fb1640a35e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb168009ea0 tx=0x7fb168009210 comp rx=0 tx=0).stop 2024-05-08T22:05:40.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.988+0000 7fb184600700 1 -- 172.21.15.131:0/2591930624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb17c100f80 msgr2=0x7fb17c196f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:40.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.988+0000 7fb184600700 1 --2- 172.21.15.131:0/2591930624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb17c100f80 0x7fb17c196f30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb17000e9c0 tx=0x7fb17000ecd0 comp rx=0 tx=0).stop 2024-05-08T22:05:40.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.988+0000 7fb184600700 1 -- 172.21.15.131:0/2591930624 shutdown_connections 2024-05-08T22:05:40.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.988+0000 7fb184600700 1 --2- 172.21.15.131:0/2591930624 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb1640a1120 0x7fb1640a35e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:40.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.988+0000 7fb184600700 1 --2- 172.21.15.131:0/2591930624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb17c102b10 0x7fb17c19e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:40.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.988+0000 7fb184600700 1 --2- 172.21.15.131:0/2591930624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb17c100f80 0x7fb17c196f30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:40.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.988+0000 7fb184600700 1 --2- 172.21.15.131:0/2591930624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb17c102170 0x7fb17c197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:40.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.988+0000 7fb184600700 1 -- 172.21.15.131:0/2591930624 >> 172.21.15.131:0/2591930624 conn(0x7fb17c0fc4f0 msgr2=0x7fb17c1054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:40.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.988+0000 7fb184600700 1 -- 172.21.15.131:0/2591930624 shutdown_connections 2024-05-08T22:05:40.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:40.988+0000 7fb184600700 1 -- 172.21.15.131:0/2591930624 wait complete. 2024-05-08T22:05:41.008 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-08T22:05:41.066 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:40 smithi131 ceph-mon[37260]: from='client.24806 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:05:41.066 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-05-08T22:05:41.066 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-08T22:05:41.067 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-05-08T22:05:41.067 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-08T22:05:41.067 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-08T22:05:41.067 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-05-08T22:05:41.067 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-05-08T22:05:41.067 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [5, 8]}]: dispatch 2024-05-08T22:05:41.067 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-05-08T22:05:41.067 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-05-08T22:05:41.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:40 smithi132 ceph-mon[78180]: from='client.24806 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:05:41.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-05-08T22:05:41.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-08T22:05:41.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-05-08T22:05:41.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-08T22:05:41.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-08T22:05:41.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-05-08T22:05:41.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-05-08T22:05:41.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [5, 8]}]: dispatch 2024-05-08T22:05:41.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-05-08T22:05:41.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-05-08T22:05:41.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.207+0000 7f040c000700 1 -- 172.21.15.131:0/648273138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0404100f80 msgr2=0x7f04041013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:41.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.207+0000 7f040c000700 1 --2- 172.21.15.131:0/648273138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0404100f80 0x7f04041013a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f03f80099c0 tx=0x7f03f8009cd0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.207+0000 7f040c000700 1 -- 172.21.15.131:0/648273138 shutdown_connections 2024-05-08T22:05:41.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.207+0000 7f040c000700 1 --2- 172.21.15.131:0/648273138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0404102b10 0x7f04041093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.207+0000 7f040c000700 1 --2- 172.21.15.131:0/648273138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0404100f80 0x7f04041013a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.207+0000 7f040c000700 1 --2- 172.21.15.131:0/648273138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0404102170 0x7f04041025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.207+0000 7f040c000700 1 -- 172.21.15.131:0/648273138 >> 172.21.15.131:0/648273138 conn(0x7f04040fc4f0 msgr2=0x7f04040fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:41.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.207+0000 7f040c000700 1 -- 172.21.15.131:0/648273138 shutdown_connections 2024-05-08T22:05:41.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.207+0000 7f040c000700 1 -- 172.21.15.131:0/648273138 wait complete. 2024-05-08T22:05:41.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.208+0000 7f040c000700 1 Processor -- start 2024-05-08T22:05:41.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.208+0000 7f040c000700 1 -- start start 2024-05-08T22:05:41.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.209+0000 7f040c000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0404100f80 0x7f0404074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.209+0000 7f040c000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0404102170 0x7f0404073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.209+0000 7f040c000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0404102b10 0x7f04040738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.209+0000 7f040c000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0404074070 con 0x7f0404102b10 2024-05-08T22:05:41.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.209+0000 7f040c000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04040741e0 con 0x7f0404100f80 2024-05-08T22:05:41.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.209+0000 7f040c000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04041a0fa0 con 0x7f0404102170 2024-05-08T22:05:41.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.209+0000 7f0408c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0404102170 0x7f0404073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:41.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.209+0000 7f040a000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0404102b10 0x7f04040738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:41.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.209+0000 7f0408c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0404102170 0x7f0404073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45962/0 (socket says 172.21.15.131:45962) 2024-05-08T22:05:41.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.209+0000 7f0408c00700 1 -- 172.21.15.131:0/3335621750 learned_addr learned my addr 172.21.15.131:0/3335621750 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:41.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.209+0000 7f0409600700 1 --2- 172.21.15.131:0/3335621750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0404100f80 0x7f0404074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:41.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.209+0000 7f0408c00700 1 -- 172.21.15.131:0/3335621750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0404100f80 msgr2=0x7f0404074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:41.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.209+0000 7f0408c00700 1 --2- 172.21.15.131:0/3335621750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0404100f80 0x7f0404074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.209+0000 7f0408c00700 1 -- 172.21.15.131:0/3335621750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0404102b10 msgr2=0x7f04040738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:41.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.209+0000 7f0408c00700 1 --2- 172.21.15.131:0/3335621750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0404102b10 0x7f04040738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.209+0000 7f0408c00700 1 -- 172.21.15.131:0/3335621750 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f80095f0 con 0x7f0404102170 2024-05-08T22:05:41.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.209+0000 7f0408c00700 1 --2- 172.21.15.131:0/3335621750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0404102170 0x7f0404073390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f03f000e9a0 tx=0x7f03f000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:41.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.210+0000 7f0402000700 1 -- 172.21.15.131:0/3335621750 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f0009790 con 0x7f0404102170 2024-05-08T22:05:41.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.210+0000 7f040c000700 1 -- 172.21.15.131:0/3335621750 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04041a11d0 con 0x7f0404102170 2024-05-08T22:05:41.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.210+0000 7f0402000700 1 -- 172.21.15.131:0/3335621750 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03f000cbd0 con 0x7f0404102170 2024-05-08T22:05:41.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.210+0000 7f0402000700 1 -- 172.21.15.131:0/3335621750 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f0018680 con 0x7f0404102170 2024-05-08T22:05:41.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.210+0000 7f040c000700 1 -- 172.21.15.131:0/3335621750 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04041a1680 con 0x7f0404102170 2024-05-08T22:05:41.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.211+0000 7f040c000700 1 -- 172.21.15.131:0/3335621750 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0404108400 con 0x7f0404102170 2024-05-08T22:05:41.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.213+0000 7f0402000700 1 -- 172.21.15.131:0/3335621750 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f03f000cd40 con 0x7f0404102170 2024-05-08T22:05:41.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.214+0000 7f0402000700 1 --2- 172.21.15.131:0/3335621750 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f03ec0a11b0 0x7f03ec0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.214+0000 7f0402000700 1 -- 172.21.15.131:0/3335621750 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f03f0014070 con 0x7f0404102170 2024-05-08T22:05:41.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.214+0000 7f0409600700 1 --2- 172.21.15.131:0/3335621750 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f03ec0a11b0 0x7f03ec0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:41.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.215+0000 7f0409600700 1 --2- 172.21.15.131:0/3335621750 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f03ec0a11b0 0x7f03ec0a3670 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009990 tx=0x7f03f801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:41.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.216+0000 7f0402000700 1 -- 172.21.15.131:0/3335621750 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f03f0052d60 con 0x7f0404102170 2024-05-08T22:05:41.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:40 smithi157 ceph-mon[77746]: from='client.24806 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:05:41.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-05-08T22:05:41.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-08T22:05:41.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-05-08T22:05:41.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-08T22:05:41.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-08T22:05:41.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-05-08T22:05:41.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-05-08T22:05:41.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [5, 8]}]: dispatch 2024-05-08T22:05:41.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-05-08T22:05:41.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-05-08T22:05:41.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.456+0000 7f040c000700 1 -- 172.21.15.131:0/3335621750 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f04040613e0 con 0x7f03ec0a11b0 2024-05-08T22:05:41.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.479+0000 7f0402000700 1 -- 172.21.15.131:0/3335621750 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f04040613e0 con 0x7f03ec0a11b0 2024-05-08T22:05:41.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.481+0000 7f040c000700 1 -- 172.21.15.131:0/3335621750 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f03ec0a11b0 msgr2=0x7f03ec0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:41.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.481+0000 7f040c000700 1 --2- 172.21.15.131:0/3335621750 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f03ec0a11b0 0x7f03ec0a3670 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009990 tx=0x7f03f801a040 comp rx=0 tx=0).stop 2024-05-08T22:05:41.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.481+0000 7f040c000700 1 -- 172.21.15.131:0/3335621750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0404102170 msgr2=0x7f0404073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:41.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.481+0000 7f040c000700 1 --2- 172.21.15.131:0/3335621750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0404102170 0x7f0404073390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f03f000e9a0 tx=0x7f03f000ecb0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.482+0000 7f040c000700 1 -- 172.21.15.131:0/3335621750 shutdown_connections 2024-05-08T22:05:41.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.482+0000 7f040c000700 1 --2- 172.21.15.131:0/3335621750 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f03ec0a11b0 0x7f03ec0a3670 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.482+0000 7f040c000700 1 --2- 172.21.15.131:0/3335621750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0404102b10 0x7f04040738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.482+0000 7f040c000700 1 --2- 172.21.15.131:0/3335621750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0404100f80 0x7f0404074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.482+0000 7f040c000700 1 --2- 172.21.15.131:0/3335621750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0404102170 0x7f0404073390 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.482+0000 7f040c000700 1 -- 172.21.15.131:0/3335621750 >> 172.21.15.131:0/3335621750 conn(0x7f04040fc4f0 msgr2=0x7f0404105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:41.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.482+0000 7f040c000700 1 -- 172.21.15.131:0/3335621750 shutdown_connections 2024-05-08T22:05:41.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.482+0000 7f040c000700 1 -- 172.21.15.131:0/3335621750 wait complete. 2024-05-08T22:05:41.502 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-08T22:05:41.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.700+0000 7f7cdc400700 1 -- 172.21.15.131:0/428679613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cd41020f0 msgr2=0x7f7cd4102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:41.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.700+0000 7f7cdc400700 1 --2- 172.21.15.131:0/428679613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cd41020f0 0x7f7cd4102570 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7cc00098f0 tx=0x7f7cc0009c00 comp rx=0 tx=0).stop 2024-05-08T22:05:41.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.700+0000 7f7cdc400700 1 -- 172.21.15.131:0/428679613 shutdown_connections 2024-05-08T22:05:41.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.700+0000 7f7cdc400700 1 --2- 172.21.15.131:0/428679613 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7cd4102ab0 0x7f7cd41093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.700+0000 7f7cdc400700 1 --2- 172.21.15.131:0/428679613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cd4100f20 0x7f7cd4101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.700+0000 7f7cdc400700 1 --2- 172.21.15.131:0/428679613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cd41020f0 0x7f7cd4102570 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.700+0000 7f7cdc400700 1 -- 172.21.15.131:0/428679613 >> 172.21.15.131:0/428679613 conn(0x7f7cd40fc500 msgr2=0x7f7cd40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:41.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.701+0000 7f7cdc400700 1 -- 172.21.15.131:0/428679613 shutdown_connections 2024-05-08T22:05:41.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.701+0000 7f7cdc400700 1 -- 172.21.15.131:0/428679613 wait complete. 2024-05-08T22:05:41.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.702+0000 7f7cdc400700 1 Processor -- start 2024-05-08T22:05:41.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.702+0000 7f7cdc400700 1 -- start start 2024-05-08T22:05:41.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.703+0000 7f7cdc400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cd4100f20 0x7f7cd41991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.703+0000 7f7cdc400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7cd41020f0 0x7f7cd4199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.703+0000 7f7cdc400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cd4102ab0 0x7f7cd41a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.703+0000 7f7cdc400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd41a0cc0 con 0x7f7cd4102ab0 2024-05-08T22:05:41.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.703+0000 7f7cdc400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd41a0e00 con 0x7f7cd41020f0 2024-05-08T22:05:41.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.703+0000 7f7cdc400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd41a0f90 con 0x7f7cd4100f20 2024-05-08T22:05:41.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.703+0000 7f7cda400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cd4102ab0 0x7f7cd41a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:41.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.703+0000 7f7cda400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cd4102ab0 0x7f7cd41a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42160/0 (socket says 172.21.15.131:42160) 2024-05-08T22:05:41.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.703+0000 7f7cda400700 1 -- 172.21.15.131:0/3197799951 learned_addr learned my addr 172.21.15.131:0/3197799951 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:41.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.703+0000 7f7cd9000700 1 --2- 172.21.15.131:0/3197799951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7cd41020f0 0x7f7cd4199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:41.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.703+0000 7f7cd9a00700 1 --2- 172.21.15.131:0/3197799951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cd4100f20 0x7f7cd41991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:41.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.703+0000 7f7cda400700 1 -- 172.21.15.131:0/3197799951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cd4100f20 msgr2=0x7f7cd41991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:41.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.703+0000 7f7cda400700 1 --2- 172.21.15.131:0/3197799951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cd4100f20 0x7f7cd41991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.703+0000 7f7cda400700 1 -- 172.21.15.131:0/3197799951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7cd41020f0 msgr2=0x7f7cd4199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:41.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.703+0000 7f7cda400700 1 --2- 172.21.15.131:0/3197799951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7cd41020f0 0x7f7cd4199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.703+0000 7f7cda400700 1 -- 172.21.15.131:0/3197799951 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc80095f0 con 0x7f7cd4102ab0 2024-05-08T22:05:41.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.703+0000 7f7cda400700 1 --2- 172.21.15.131:0/3197799951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cd4102ab0 0x7f7cd41a06f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00c370 tx=0x7f7ccc00c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:41.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.704+0000 7f7cd2000700 1 -- 172.21.15.131:0/3197799951 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc017070 con 0x7f7cd4102ab0 2024-05-08T22:05:41.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.704+0000 7f7cd2000700 1 -- 172.21.15.131:0/3197799951 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ccc00f040 con 0x7f7cd4102ab0 2024-05-08T22:05:41.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.704+0000 7f7cdc400700 1 -- 172.21.15.131:0/3197799951 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ccc009c40 con 0x7f7cd4102ab0 2024-05-08T22:05:41.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.704+0000 7f7cd2000700 1 -- 172.21.15.131:0/3197799951 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc00e050 con 0x7f7cd4102ab0 2024-05-08T22:05:41.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.704+0000 7f7cdc400700 1 -- 172.21.15.131:0/3197799951 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc0009520 con 0x7f7cd4102ab0 2024-05-08T22:05:41.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.705+0000 7f7cdc400700 1 -- 172.21.15.131:0/3197799951 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cd4067cf0 con 0x7f7cd4102ab0 2024-05-08T22:05:41.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.708+0000 7f7cd2000700 1 -- 172.21.15.131:0/3197799951 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f7ccc00cb60 con 0x7f7cd4102ab0 2024-05-08T22:05:41.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.709+0000 7f7cd2000700 1 --2- 172.21.15.131:0/3197799951 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7cbc0a11b0 0x7f7cbc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:41.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.710+0000 7f7cd2000700 1 -- 172.21.15.131:0/3197799951 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7ccc0cc4d0 con 0x7f7cd4102ab0 2024-05-08T22:05:41.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.710+0000 7f7cd9a00700 1 --2- 172.21.15.131:0/3197799951 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7cbc0a11b0 0x7f7cbc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:41.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.710+0000 7f7cd9a00700 1 --2- 172.21.15.131:0/3197799951 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7cbc0a11b0 0x7f7cbc0a3670 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4100c40 tx=0x7f7cc8009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:41.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.712+0000 7f7cd2000700 1 -- 172.21.15.131:0/3197799951 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7ccc0957f0 con 0x7f7cd4102ab0 2024-05-08T22:05:41.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.961+0000 7f7cdc400700 1 -- 172.21.15.131:0/3197799951 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7cd40748f0 con 0x7f7cbc0a11b0 2024-05-08T22:05:41.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.965+0000 7f7cd2000700 1 -- 172.21.15.131:0/3197799951 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f7cd40748f0 con 0x7f7cbc0a11b0 2024-05-08T22:05:41.966 INFO:teuthology.orchestra.run.smithi131.stdout:/volumes/qa/sv_1/8d0ca3e4-ef2e-4394-b208-510ba6b717e9 2024-05-08T22:05:41.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.968+0000 7f7cdc400700 1 -- 172.21.15.131:0/3197799951 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7cbc0a11b0 msgr2=0x7f7cbc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:41.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.968+0000 7f7cdc400700 1 --2- 172.21.15.131:0/3197799951 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7cbc0a11b0 0x7f7cbc0a3670 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4100c40 tx=0x7f7cc8009310 comp rx=0 tx=0).stop 2024-05-08T22:05:41.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.968+0000 7f7cdc400700 1 -- 172.21.15.131:0/3197799951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cd4102ab0 msgr2=0x7f7cd41a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:41.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.968+0000 7f7cdc400700 1 --2- 172.21.15.131:0/3197799951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cd4102ab0 0x7f7cd41a06f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00c370 tx=0x7f7ccc00c680 comp rx=0 tx=0).stop 2024-05-08T22:05:41.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.968+0000 7f7cdc400700 1 -- 172.21.15.131:0/3197799951 shutdown_connections 2024-05-08T22:05:41.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.968+0000 7f7cdc400700 1 --2- 172.21.15.131:0/3197799951 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7cbc0a11b0 0x7f7cbc0a3670 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.968+0000 7f7cdc400700 1 --2- 172.21.15.131:0/3197799951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cd4102ab0 0x7f7cd41a06f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.968+0000 7f7cdc400700 1 --2- 172.21.15.131:0/3197799951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cd4100f20 0x7f7cd41991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.968+0000 7f7cdc400700 1 --2- 172.21.15.131:0/3197799951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7cd41020f0 0x7f7cd4199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:41.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.968+0000 7f7cdc400700 1 -- 172.21.15.131:0/3197799951 >> 172.21.15.131:0/3197799951 conn(0x7f7cd40fc500 msgr2=0x7f7cd40731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:41.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.968+0000 7f7cdc400700 1 -- 172.21.15.131:0/3197799951 shutdown_connections 2024-05-08T22:05:41.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:41.968+0000 7f7cdc400700 1 -- 172.21.15.131:0/3197799951 wait complete. 2024-05-08T22:05:41.988 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:05:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:41 smithi132 ceph-mon[78180]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:05:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:41 smithi132 ceph-mon[78180]: pgmap v253: 129 pgs: 129 active+clean; 582 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 767 B/s wr, 3 op/s 2024-05-08T22:05:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-05-08T22:05:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-08T22:05:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-05-08T22:05:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-08T22:05:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-08T22:05:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-05-08T22:05:42.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-05-08T22:05:42.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [5, 8]}]': finished 2024-05-08T22:05:42.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-05-08T22:05:42.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-05-08T22:05:42.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:41 smithi132 ceph-mon[78180]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T22:05:42.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:41 smithi132 ceph-mon[78180]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-05-08T22:05:42.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:41 smithi132 ceph-mon[78180]: from='client.24824 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:05:42.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:41 smithi131 ceph-mon[37260]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:05:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:41 smithi131 ceph-mon[37260]: pgmap v253: 129 pgs: 129 active+clean; 582 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 767 B/s wr, 3 op/s 2024-05-08T22:05:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-05-08T22:05:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-08T22:05:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-05-08T22:05:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-08T22:05:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-08T22:05:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-05-08T22:05:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-05-08T22:05:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [5, 8]}]': finished 2024-05-08T22:05:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-05-08T22:05:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-05-08T22:05:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:41 smithi131 ceph-mon[37260]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T22:05:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:41 smithi131 ceph-mon[37260]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-05-08T22:05:42.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:41 smithi131 ceph-mon[37260]: from='client.24824 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:05:42.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.183+0000 7f4202000700 1 -- 172.21.15.131:0/1270675857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41fc100e70 msgr2=0x7f41fc101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:42.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.183+0000 7f4202000700 1 --2- 172.21.15.131:0/1270675857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41fc100e70 0x7f41fc101290 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009990 tx=0x7f41ec009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:42.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.183+0000 7f4202000700 1 -- 172.21.15.131:0/1270675857 shutdown_connections 2024-05-08T22:05:42.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.183+0000 7f4202000700 1 --2- 172.21.15.131:0/1270675857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41fc102980 0x7f41fc1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:42.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.183+0000 7f4202000700 1 --2- 172.21.15.131:0/1270675857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41fc100e70 0x7f41fc101290 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:42.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.183+0000 7f4202000700 1 --2- 172.21.15.131:0/1270675857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41fc101fc0 0x7f41fc102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:42.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.183+0000 7f4202000700 1 -- 172.21.15.131:0/1270675857 >> 172.21.15.131:0/1270675857 conn(0x7f41fc0fc3e0 msgr2=0x7f41fc0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:42.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.183+0000 7f4202000700 1 -- 172.21.15.131:0/1270675857 shutdown_connections 2024-05-08T22:05:42.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.183+0000 7f4202000700 1 -- 172.21.15.131:0/1270675857 wait complete. 2024-05-08T22:05:42.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.184+0000 7f4202000700 1 Processor -- start 2024-05-08T22:05:42.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.184+0000 7f4202000700 1 -- start start 2024-05-08T22:05:42.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.184+0000 7f4202000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41fc100e70 0x7f41fc074cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:42.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.184+0000 7f4202000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41fc101fc0 0x7f41fc073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:42.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.184+0000 7f4202000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41fc102980 0x7f41fc0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:42.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.184+0000 7f4202000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc073fd0 con 0x7f41fc100e70 2024-05-08T22:05:42.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.184+0000 7f4202000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc074140 con 0x7f41fc101fc0 2024-05-08T22:05:42.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.184+0000 7f4202000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc1a0e40 con 0x7f41fc102980 2024-05-08T22:05:42.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.185+0000 7f41fb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41fc100e70 0x7f41fc074cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:42.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.185+0000 7f41fb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41fc100e70 0x7f41fc074cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42186/0 (socket says 172.21.15.131:42186) 2024-05-08T22:05:42.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.185+0000 7f41fb400700 1 -- 172.21.15.131:0/1670488334 learned_addr learned my addr 172.21.15.131:0/1670488334 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:42.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.185+0000 7f41fbe00700 1 --2- 172.21.15.131:0/1670488334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41fc102980 0x7f41fc0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:42.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.185+0000 7f41faa00700 1 --2- 172.21.15.131:0/1670488334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41fc101fc0 0x7f41fc073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:42.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.185+0000 7f41fb400700 1 -- 172.21.15.131:0/1670488334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41fc102980 msgr2=0x7f41fc0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:42.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.185+0000 7f41fb400700 1 --2- 172.21.15.131:0/1670488334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41fc102980 0x7f41fc0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:42.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.185+0000 7f41fb400700 1 -- 172.21.15.131:0/1670488334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41fc101fc0 msgr2=0x7f41fc073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:42.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.185+0000 7f41fb400700 1 --2- 172.21.15.131:0/1670488334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41fc101fc0 0x7f41fc073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:42.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.185+0000 7f41fb400700 1 -- 172.21.15.131:0/1670488334 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ec0095f0 con 0x7f41fc100e70 2024-05-08T22:05:42.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.185+0000 7f41fbe00700 1 --2- 172.21.15.131:0/1670488334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41fc102980 0x7f41fc0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:05:42.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.185+0000 7f41faa00700 1 --2- 172.21.15.131:0/1670488334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41fc101fc0 0x7f41fc073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:05:42.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.185+0000 7f41fb400700 1 --2- 172.21.15.131:0/1670488334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41fc100e70 0x7f41fc074cb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f41ec005910 tx=0x7f41ec009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:42.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.186+0000 7f41f3e00700 1 -- 172.21.15.131:0/1670488334 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec01d070 con 0x7f41fc100e70 2024-05-08T22:05:42.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.186+0000 7f41f3e00700 1 -- 172.21.15.131:0/1670488334 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41ec021410 con 0x7f41fc100e70 2024-05-08T22:05:42.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.186+0000 7f4202000700 1 -- 172.21.15.131:0/1670488334 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41fc1a1040 con 0x7f41fc100e70 2024-05-08T22:05:42.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.186+0000 7f41f3e00700 1 -- 172.21.15.131:0/1670488334 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec017650 con 0x7f41fc100e70 2024-05-08T22:05:42.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.186+0000 7f4202000700 1 -- 172.21.15.131:0/1670488334 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41fc1a14c0 con 0x7f41fc100e70 2024-05-08T22:05:42.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.187+0000 7f4202000700 1 -- 172.21.15.131:0/1670488334 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41fc108230 con 0x7f41fc100e70 2024-05-08T22:05:42.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.189+0000 7f41f3e00700 1 -- 172.21.15.131:0/1670488334 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f41ec021580 con 0x7f41fc100e70 2024-05-08T22:05:42.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.191+0000 7f41f3e00700 1 --2- 172.21.15.131:0/1670488334 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f41e00a11b0 0x7f41e00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:42.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.191+0000 7f41f3e00700 1 -- 172.21.15.131:0/1670488334 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f41ec0ce890 con 0x7f41fc100e70 2024-05-08T22:05:42.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.191+0000 7f41faa00700 1 --2- 172.21.15.131:0/1670488334 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f41e00a11b0 0x7f41e00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:42.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.192+0000 7f41faa00700 1 --2- 172.21.15.131:0/1670488334 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f41e00a11b0 0x7f41e00a3670 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009cb0 tx=0x7f41e8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:42.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.193+0000 7f41f3e00700 1 -- 172.21.15.131:0/1670488334 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f41ec097be0 con 0x7f41fc100e70 2024-05-08T22:05:42.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:41 smithi157 ceph-mon[77746]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:05:42.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:41 smithi157 ceph-mon[77746]: pgmap v253: 129 pgs: 129 active+clean; 582 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 767 B/s wr, 3 op/s 2024-05-08T22:05:42.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-05-08T22:05:42.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-08T22:05:42.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-05-08T22:05:42.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-08T22:05:42.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-08T22:05:42.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-05-08T22:05:42.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-05-08T22:05:42.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [5, 8]}]': finished 2024-05-08T22:05:42.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-05-08T22:05:42.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-05-08T22:05:42.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:41 smithi157 ceph-mon[77746]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T22:05:42.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:41 smithi157 ceph-mon[77746]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-05-08T22:05:42.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:41 smithi157 ceph-mon[77746]: from='client.24824 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:05:42.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.444+0000 7f4202000700 1 -- 172.21.15.131:0/1670488334 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41fc067c10 con 0x7f41fc100e70 2024-05-08T22:05:42.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.445+0000 7f41f3e00700 1 -- 172.21.15.131:0/1670488334 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10889 (secure 0 0 0) 0x7f41ec09b200 con 0x7f41fc100e70 2024-05-08T22:05:42.445 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:42.446 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:05:14.034908+0000","modified":"2024-05-08T22:05:39.494566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:05:42.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.447+0000 7f4202000700 1 -- 172.21.15.131:0/1670488334 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f41e00a11b0 msgr2=0x7f41e00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:42.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.447+0000 7f4202000700 1 --2- 172.21.15.131:0/1670488334 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f41e00a11b0 0x7f41e00a3670 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009cb0 tx=0x7f41e8009260 comp rx=0 tx=0).stop 2024-05-08T22:05:42.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.447+0000 7f4202000700 1 -- 172.21.15.131:0/1670488334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41fc100e70 msgr2=0x7f41fc074cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:42.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.447+0000 7f4202000700 1 --2- 172.21.15.131:0/1670488334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41fc100e70 0x7f41fc074cb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f41ec005910 tx=0x7f41ec009580 comp rx=0 tx=0).stop 2024-05-08T22:05:42.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.447+0000 7f4202000700 1 -- 172.21.15.131:0/1670488334 shutdown_connections 2024-05-08T22:05:42.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.447+0000 7f4202000700 1 --2- 172.21.15.131:0/1670488334 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f41e00a11b0 0x7f41e00a3670 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:42.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.447+0000 7f4202000700 1 --2- 172.21.15.131:0/1670488334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41fc102980 0x7f41fc0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:42.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.447+0000 7f4202000700 1 --2- 172.21.15.131:0/1670488334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41fc100e70 0x7f41fc074cb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:42.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.447+0000 7f4202000700 1 --2- 172.21.15.131:0/1670488334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41fc101fc0 0x7f41fc073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:42.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.447+0000 7f4202000700 1 -- 172.21.15.131:0/1670488334 >> 172.21.15.131:0/1670488334 conn(0x7f41fc0fc3e0 msgr2=0x7f41fc105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:42.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.447+0000 7f4202000700 1 -- 172.21.15.131:0/1670488334 shutdown_connections 2024-05-08T22:05:42.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.447+0000 7f4202000700 1 -- 172.21.15.131:0/1670488334 wait complete. 2024-05-08T22:05:42.449 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 23 2024-05-08T22:05:42.468 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:05:42.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.663+0000 7f7e82400700 1 -- 172.21.15.131:0/2350874784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e7c100e70 msgr2=0x7f7e7c101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:42.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.663+0000 7f7e82400700 1 --2- 172.21.15.131:0/2350874784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e7c100e70 0x7f7e7c101290 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009990 tx=0x7f7e6c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:42.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.663+0000 7f7e82400700 1 -- 172.21.15.131:0/2350874784 shutdown_connections 2024-05-08T22:05:42.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.663+0000 7f7e82400700 1 --2- 172.21.15.131:0/2350874784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e7c102980 0x7f7e7c1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:42.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.663+0000 7f7e82400700 1 --2- 172.21.15.131:0/2350874784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e7c100e70 0x7f7e7c101290 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:42.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.663+0000 7f7e82400700 1 --2- 172.21.15.131:0/2350874784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e7c101fc0 0x7f7e7c102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:42.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.663+0000 7f7e82400700 1 -- 172.21.15.131:0/2350874784 >> 172.21.15.131:0/2350874784 conn(0x7f7e7c0fc3e0 msgr2=0x7f7e7c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:42.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.663+0000 7f7e82400700 1 -- 172.21.15.131:0/2350874784 shutdown_connections 2024-05-08T22:05:42.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.663+0000 7f7e82400700 1 -- 172.21.15.131:0/2350874784 wait complete. 2024-05-08T22:05:42.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.664+0000 7f7e82400700 1 Processor -- start 2024-05-08T22:05:42.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.664+0000 7f7e82400700 1 -- start start 2024-05-08T22:05:42.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.664+0000 7f7e82400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e7c100e70 0x7f7e7c199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:42.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.664+0000 7f7e82400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e7c101fc0 0x7f7e7c1995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:42.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.664+0000 7f7e82400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e7c102980 0x7f7e7c1a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:42.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.664+0000 7f7e82400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c1a0b60 con 0x7f7e7c101fc0 2024-05-08T22:05:42.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.664+0000 7f7e82400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c1a0ca0 con 0x7f7e7c100e70 2024-05-08T22:05:42.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.664+0000 7f7e82400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c1a0e30 con 0x7f7e7c102980 2024-05-08T22:05:42.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.665+0000 7f7e7aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e7c101fc0 0x7f7e7c1995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:42.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.665+0000 7f7e7aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e7c101fc0 0x7f7e7c1995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42208/0 (socket says 172.21.15.131:42208) 2024-05-08T22:05:42.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.665+0000 7f7e7aa00700 1 -- 172.21.15.131:0/2299669838 learned_addr learned my addr 172.21.15.131:0/2299669838 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:42.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.665+0000 7f7e7b400700 1 --2- 172.21.15.131:0/2299669838 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e7c100e70 0x7f7e7c199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:42.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.665+0000 7f7e7be00700 1 --2- 172.21.15.131:0/2299669838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e7c102980 0x7f7e7c1a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:42.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.665+0000 7f7e7aa00700 1 -- 172.21.15.131:0/2299669838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e7c102980 msgr2=0x7f7e7c1a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:42.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.665+0000 7f7e7aa00700 1 --2- 172.21.15.131:0/2299669838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e7c102980 0x7f7e7c1a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:42.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.665+0000 7f7e7aa00700 1 -- 172.21.15.131:0/2299669838 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e7c100e70 msgr2=0x7f7e7c199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:42.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.665+0000 7f7e7aa00700 1 --2- 172.21.15.131:0/2299669838 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e7c100e70 0x7f7e7c199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:42.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.665+0000 7f7e7aa00700 1 -- 172.21.15.131:0/2299669838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6c0095f0 con 0x7f7e7c101fc0 2024-05-08T22:05:42.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.665+0000 7f7e7aa00700 1 --2- 172.21.15.131:0/2299669838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e7c101fc0 0x7f7e7c1995c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800d980 tx=0x7f7e6800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:42.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.665+0000 7f7e73e00700 1 -- 172.21.15.131:0/2299669838 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68004220 con 0x7f7e7c101fc0 2024-05-08T22:05:42.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.666+0000 7f7e82400700 1 -- 172.21.15.131:0/2299669838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e7c1a1110 con 0x7f7e7c101fc0 2024-05-08T22:05:42.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.666+0000 7f7e73e00700 1 -- 172.21.15.131:0/2299669838 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e68007640 con 0x7f7e7c101fc0 2024-05-08T22:05:42.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.666+0000 7f7e73e00700 1 -- 172.21.15.131:0/2299669838 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68005690 con 0x7f7e7c101fc0 2024-05-08T22:05:42.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.666+0000 7f7e82400700 1 -- 172.21.15.131:0/2299669838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e7c1a1670 con 0x7f7e7c101fc0 2024-05-08T22:05:42.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.667+0000 7f7e82400700 1 -- 172.21.15.131:0/2299669838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e7c067c10 con 0x7f7e7c101fc0 2024-05-08T22:05:42.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.669+0000 7f7e73e00700 1 -- 172.21.15.131:0/2299669838 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f7e68004750 con 0x7f7e7c101fc0 2024-05-08T22:05:42.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.670+0000 7f7e73e00700 1 --2- 172.21.15.131:0/2299669838 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7e600a1230 0x7f7e600a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:42.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.670+0000 7f7e73e00700 1 -- 172.21.15.131:0/2299669838 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7e680ccb10 con 0x7f7e7c101fc0 2024-05-08T22:05:42.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.670+0000 7f7e7b400700 1 --2- 172.21.15.131:0/2299669838 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7e600a1230 0x7f7e600a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:42.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.671+0000 7f7e7b400700 1 --2- 172.21.15.131:0/2299669838 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7e600a1230 0x7f7e600a36f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00b2e0 tx=0x7f7e6c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:42.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.674+0000 7f7e73e00700 1 -- 172.21.15.131:0/2299669838 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7e68095e30 con 0x7f7e7c101fc0 2024-05-08T22:05:42.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.889+0000 7f7e82400700 1 -- 172.21.15.131:0/2299669838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e7c10c8b0 con 0x7f7e7c101fc0 2024-05-08T22:05:42.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.891+0000 7f7e73e00700 1 -- 172.21.15.131:0/2299669838 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+24534 (secure 0 0 0) 0x7f7e68016020 con 0x7f7e7c101fc0 2024-05-08T22:05:42.892 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:42.893 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":85,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:05:41.813233+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:05:12.469858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:05:13.484446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:05:17.016186+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:05:42.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.894+0000 7f7e82400700 1 -- 172.21.15.131:0/2299669838 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7e600a1230 msgr2=0x7f7e600a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:42.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.894+0000 7f7e82400700 1 --2- 172.21.15.131:0/2299669838 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7e600a1230 0x7f7e600a36f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00b2e0 tx=0x7f7e6c005fd0 comp rx=0 tx=0).stop 2024-05-08T22:05:42.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.894+0000 7f7e82400700 1 -- 172.21.15.131:0/2299669838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e7c101fc0 msgr2=0x7f7e7c1995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:42.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.894+0000 7f7e82400700 1 --2- 172.21.15.131:0/2299669838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e7c101fc0 0x7f7e7c1995c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800d980 tx=0x7f7e6800dc90 comp rx=0 tx=0).stop 2024-05-08T22:05:42.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.894+0000 7f7e82400700 1 -- 172.21.15.131:0/2299669838 shutdown_connections 2024-05-08T22:05:42.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.894+0000 7f7e82400700 1 --2- 172.21.15.131:0/2299669838 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7e600a1230 0x7f7e600a36f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:42.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.894+0000 7f7e82400700 1 --2- 172.21.15.131:0/2299669838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e7c102980 0x7f7e7c1a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:42.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.894+0000 7f7e82400700 1 --2- 172.21.15.131:0/2299669838 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e7c100e70 0x7f7e7c199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:42.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.894+0000 7f7e82400700 1 --2- 172.21.15.131:0/2299669838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e7c101fc0 0x7f7e7c1995c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:42.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.894+0000 7f7e82400700 1 -- 172.21.15.131:0/2299669838 >> 172.21.15.131:0/2299669838 conn(0x7f7e7c0fc3e0 msgr2=0x7f7e7c0731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:42.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.894+0000 7f7e82400700 1 -- 172.21.15.131:0/2299669838 shutdown_connections 2024-05-08T22:05:42.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:42.894+0000 7f7e82400700 1 -- 172.21.15.131:0/2299669838 wait complete. 2024-05-08T22:05:42.914 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T22:05:42.914 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:05:43.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:42 smithi132 ceph-mon[78180]: from='client.24835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:05:43.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:42 smithi132 ceph-mon[78180]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T22:05:43.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:42 smithi132 ceph-mon[78180]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:05:43.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:42 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1670488334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:05:43.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.106+0000 7f67dbc00700 1 -- 172.21.15.131:0/1739076775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67d4102990 msgr2=0x7f67d41092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:43.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.106+0000 7f67dbc00700 1 --2- 172.21.15.131:0/1739076775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67d4102990 0x7f67d41092b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009f90 tx=0x7f67cc00b330 comp rx=0 tx=0).stop 2024-05-08T22:05:43.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.107+0000 7f67dbc00700 1 -- 172.21.15.131:0/1739076775 shutdown_connections 2024-05-08T22:05:43.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.107+0000 7f67dbc00700 1 --2- 172.21.15.131:0/1739076775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67d4102990 0x7f67d41092b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:43.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.107+0000 7f67dbc00700 1 --2- 172.21.15.131:0/1739076775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67d4100e80 0x7f67d41012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:43.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.107+0000 7f67dbc00700 1 --2- 172.21.15.131:0/1739076775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67d4101fd0 0x7f67d4102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:43.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.107+0000 7f67dbc00700 1 -- 172.21.15.131:0/1739076775 >> 172.21.15.131:0/1739076775 conn(0x7f67d40fc410 msgr2=0x7f67d40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:43.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.107+0000 7f67dbc00700 1 -- 172.21.15.131:0/1739076775 shutdown_connections 2024-05-08T22:05:43.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.107+0000 7f67dbc00700 1 -- 172.21.15.131:0/1739076775 wait complete. 2024-05-08T22:05:43.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.108+0000 7f67dbc00700 1 Processor -- start 2024-05-08T22:05:43.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.108+0000 7f67dbc00700 1 -- start start 2024-05-08T22:05:43.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.108+0000 7f67dbc00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67d4100e80 0x7f67d4196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:43.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.108+0000 7f67dbc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67d4101fd0 0x7f67d4197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:43.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.108+0000 7f67dbc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67d4102990 0x7f67d419e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:43.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.108+0000 7f67dbc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d419ea70 con 0x7f67d4101fd0 2024-05-08T22:05:43.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.108+0000 7f67dbc00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d419ebb0 con 0x7f67d4100e80 2024-05-08T22:05:43.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.108+0000 7f67dbc00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d419ed40 con 0x7f67d4102990 2024-05-08T22:05:43.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.108+0000 7f67d3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67d4101fd0 0x7f67d4197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:43.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.108+0000 7f67d9200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67d4100e80 0x7f67d4196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:43.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.108+0000 7f67d3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67d4101fd0 0x7f67d4197480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42230/0 (socket says 172.21.15.131:42230) 2024-05-08T22:05:43.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.108+0000 7f67d3e00700 1 -- 172.21.15.131:0/2925178481 learned_addr learned my addr 172.21.15.131:0/2925178481 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:43.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.109+0000 7f67d9c00700 1 --2- 172.21.15.131:0/2925178481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67d4102990 0x7f67d419e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:43.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.109+0000 7f67d9200700 1 -- 172.21.15.131:0/2925178481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67d4102990 msgr2=0x7f67d419e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:43.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.109+0000 7f67d9200700 1 --2- 172.21.15.131:0/2925178481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67d4102990 0x7f67d419e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:43.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.109+0000 7f67d9200700 1 -- 172.21.15.131:0/2925178481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67d4101fd0 msgr2=0x7f67d4197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:43.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.109+0000 7f67d9200700 1 --2- 172.21.15.131:0/2925178481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67d4101fd0 0x7f67d4197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:43.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.109+0000 7f67d9200700 1 -- 172.21.15.131:0/2925178481 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c80095f0 con 0x7f67d4100e80 2024-05-08T22:05:43.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.109+0000 7f67d9200700 1 --2- 172.21.15.131:0/2925178481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67d4100e80 0x7f67d4196f40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f67c800eb90 tx=0x7f67c800eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:43.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.109+0000 7f67d1600700 1 -- 172.21.15.131:0/2925178481 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c8009970 con 0x7f67d4100e80 2024-05-08T22:05:43.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.109+0000 7f67d1600700 1 -- 172.21.15.131:0/2925178481 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67c80044e0 con 0x7f67d4100e80 2024-05-08T22:05:43.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.109+0000 7f67dbc00700 1 -- 172.21.15.131:0/2925178481 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67cc009c40 con 0x7f67d4100e80 2024-05-08T22:05:43.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.109+0000 7f67d1600700 1 -- 172.21.15.131:0/2925178481 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c8003e50 con 0x7f67d4100e80 2024-05-08T22:05:43.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.109+0000 7f67dbc00700 1 -- 172.21.15.131:0/2925178481 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d419f2b0 con 0x7f67d4100e80 2024-05-08T22:05:43.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.110+0000 7f67dbc00700 1 -- 172.21.15.131:0/2925178481 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d4108240 con 0x7f67d4100e80 2024-05-08T22:05:43.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.112+0000 7f67d1600700 1 -- 172.21.15.131:0/2925178481 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f67c8003fb0 con 0x7f67d4100e80 2024-05-08T22:05:43.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.113+0000 7f67d1600700 1 --2- 172.21.15.131:0/2925178481 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f67bc0a11b0 0x7f67bc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:43.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.114+0000 7f67d1600700 1 -- 172.21.15.131:0/2925178481 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f67c8014070 con 0x7f67d4100e80 2024-05-08T22:05:43.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.114+0000 7f67d3e00700 1 --2- 172.21.15.131:0/2925178481 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f67bc0a11b0 0x7f67bc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:43.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.114+0000 7f67d3e00700 1 --2- 172.21.15.131:0/2925178481 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f67bc0a11b0 0x7f67bc0a3670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f67c0009d60 tx=0x7f67c0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:43.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.116+0000 7f67d1600700 1 -- 172.21.15.131:0/2925178481 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f67c8096ce0 con 0x7f67d4100e80 2024-05-08T22:05:43.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:42 smithi131 ceph-mon[37260]: from='client.24835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:05:43.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:42 smithi131 ceph-mon[37260]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T22:05:43.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:42 smithi131 ceph-mon[37260]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:05:43.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:42 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1670488334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:05:43.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:42 smithi157 ceph-mon[77746]: from='client.24835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:05:43.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:42 smithi157 ceph-mon[77746]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T22:05:43.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:42 smithi157 ceph-mon[77746]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:05:43.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:42 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1670488334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:05:43.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.319+0000 7f67dbc00700 1 -- 172.21.15.131:0/2925178481 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f67d402cee0 con 0x7f67bc0a11b0 2024-05-08T22:05:43.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.334+0000 7f67d1600700 1 -- 172.21.15.131:0/2925178481 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+421790 (secure 0 0 0) 0x7f67d402cee0 con 0x7f67bc0a11b0 2024-05-08T22:05:43.341 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:43.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.339+0000 7f67dbc00700 1 -- 172.21.15.131:0/2925178481 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f67bc0a11b0 msgr2=0x7f67bc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:43.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.339+0000 7f67dbc00700 1 --2- 172.21.15.131:0/2925178481 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f67bc0a11b0 0x7f67bc0a3670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f67c0009d60 tx=0x7f67c0009260 comp rx=0 tx=0).stop 2024-05-08T22:05:43.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.340+0000 7f67dbc00700 1 -- 172.21.15.131:0/2925178481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67d4100e80 msgr2=0x7f67d4196f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:43.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.340+0000 7f67dbc00700 1 --2- 172.21.15.131:0/2925178481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67d4100e80 0x7f67d4196f40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f67c800eb90 tx=0x7f67c800eea0 comp rx=0 tx=0).stop 2024-05-08T22:05:43.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.340+0000 7f67dbc00700 1 -- 172.21.15.131:0/2925178481 shutdown_connections 2024-05-08T22:05:43.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.340+0000 7f67dbc00700 1 --2- 172.21.15.131:0/2925178481 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f67bc0a11b0 0x7f67bc0a3670 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:43.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.340+0000 7f67dbc00700 1 --2- 172.21.15.131:0/2925178481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67d4102990 0x7f67d419e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:43.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.340+0000 7f67dbc00700 1 --2- 172.21.15.131:0/2925178481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67d4100e80 0x7f67d4196f40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:43.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.340+0000 7f67dbc00700 1 --2- 172.21.15.131:0/2925178481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67d4101fd0 0x7f67d4197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:43.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.340+0000 7f67dbc00700 1 -- 172.21.15.131:0/2925178481 >> 172.21.15.131:0/2925178481 conn(0x7f67d40fc410 msgr2=0x7f67d4105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:43.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.340+0000 7f67dbc00700 1 -- 172.21.15.131:0/2925178481 shutdown_connections 2024-05-08T22:05:43.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.340+0000 7f67dbc00700 1 -- 172.21.15.131:0/2925178481 wait complete. 2024-05-08T22:05:43.346 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-08T22:05:43.359 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-05-08T22:05:42.717959+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597213,"num_objects":57,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":159,"num_read_kb":138,"num_write":300,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":242,"ondisk_log_size":242,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371992,"kb_used_data":10916,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124569832,"statfs":{"total":1151940427776,"available":1151559507968,"internally_reserved":0,"allocated":11177984,"data_stored":7222109,"data_compressed":262820,"data_compressed_allocated":352256,"data_compressed_original":2203648,"omap_allocated":19131,"internal_metadata":369603909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2392,"num_objects":33,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":23,"num_read_kb":19,"num_write":72,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002287"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518839+0000","last_change":"2024-05-08T22:05:14.061628+0000","last_active":"2024-05-08T22:05:18.518839+0000","last_peered":"2024-05-08T22:05:18.518839+0000","last_clean":"2024-05-08T22:05:18.518839+0000","last_became_active":"2024-05-08T22:05:14.060986+0000","last_became_peered":"2024-05-08T22:05:14.060986+0000","last_unstale":"2024-05-08T22:05:18.518839+0000","last_undegraded":"2024-05-08T22:05:18.518839+0000","last_fullsized":"2024-05-08T22:05:18.518839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:51.643118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.523337+0000","last_change":"2024-05-08T22:05:14.073300+0000","last_active":"2024-05-08T22:05:18.523337+0000","last_peered":"2024-05-08T22:05:18.523337+0000","last_clean":"2024-05-08T22:05:18.523337+0000","last_became_active":"2024-05-08T22:05:14.069058+0000","last_became_peered":"2024-05-08T22:05:14.069058+0000","last_unstale":"2024-05-08T22:05:18.523337+0000","last_undegraded":"2024-05-08T22:05:18.523337+0000","last_fullsized":"2024-05-08T22:05:18.523337+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:15.834956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.523429+0000","last_change":"2024-05-08T22:05:14.071676+0000","last_active":"2024-05-08T22:05:18.523429+0000","last_peered":"2024-05-08T22:05:18.523429+0000","last_clean":"2024-05-08T22:05:18.523429+0000","last_became_active":"2024-05-08T22:05:14.058345+0000","last_became_peered":"2024-05-08T22:05:14.058345+0000","last_unstale":"2024-05-08T22:05:18.523429+0000","last_undegraded":"2024-05-08T22:05:18.523429+0000","last_fullsized":"2024-05-08T22:05:18.523429+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:09.239058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.019162+0000","last_change":"2024-05-08T22:05:14.067170+0000","last_active":"2024-05-08T22:05:19.019162+0000","last_peered":"2024-05-08T22:05:19.019162+0000","last_clean":"2024-05-08T22:05:19.019162+0000","last_became_active":"2024-05-08T22:05:14.066850+0000","last_became_peered":"2024-05-08T22:05:14.066850+0000","last_unstale":"2024-05-08T22:05:19.019162+0000","last_undegraded":"2024-05-08T22:05:19.019162+0000","last_fullsized":"2024-05-08T22:05:19.019162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:28.439786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829281+0000","last_change":"2024-05-08T22:05:14.065177+0000","last_active":"2024-05-08T22:05:40.829281+0000","last_peered":"2024-05-08T22:05:40.829281+0000","last_clean":"2024-05-08T22:05:40.829281+0000","last_became_active":"2024-05-08T22:05:14.064833+0000","last_became_peered":"2024-05-08T22:05:14.064833+0000","last_unstale":"2024-05-08T22:05:40.829281+0000","last_undegraded":"2024-05-08T22:05:40.829281+0000","last_fullsized":"2024-05-08T22:05:40.829281+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:11.764154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.615458+0000","last_change":"2024-05-08T22:05:14.061610+0000","last_active":"2024-05-08T22:05:18.615458+0000","last_peered":"2024-05-08T22:05:18.615458+0000","last_clean":"2024-05-08T22:05:18.615458+0000","last_became_active":"2024-05-08T22:05:14.061277+0000","last_became_peered":"2024-05-08T22:05:14.061277+0000","last_unstale":"2024-05-08T22:05:18.615458+0000","last_undegraded":"2024-05-08T22:05:18.615458+0000","last_fullsized":"2024-05-08T22:05:18.615458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:19.154980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.516423+0000","last_change":"2024-05-08T22:05:14.063941+0000","last_active":"2024-05-08T22:05:18.516423+0000","last_peered":"2024-05-08T22:05:18.516423+0000","last_clean":"2024-05-08T22:05:18.516423+0000","last_became_active":"2024-05-08T22:05:14.063579+0000","last_became_peered":"2024-05-08T22:05:14.063579+0000","last_unstale":"2024-05-08T22:05:18.516423+0000","last_undegraded":"2024-05-08T22:05:18.516423+0000","last_fullsized":"2024-05-08T22:05:18.516423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:27.679067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714348+0000","last_change":"2024-05-08T22:05:14.060366+0000","last_active":"2024-05-08T22:05:19.714348+0000","last_peered":"2024-05-08T22:05:19.714348+0000","last_clean":"2024-05-08T22:05:19.714348+0000","last_became_active":"2024-05-08T22:05:14.059818+0000","last_became_peered":"2024-05-08T22:05:14.059818+0000","last_unstale":"2024-05-08T22:05:19.714348+0000","last_undegraded":"2024-05-08T22:05:19.714348+0000","last_fullsized":"2024-05-08T22:05:19.714348+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:53.118508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518750+0000","last_change":"2024-05-08T22:05:14.057185+0000","last_active":"2024-05-08T22:05:18.518750+0000","last_peered":"2024-05-08T22:05:18.518750+0000","last_clean":"2024-05-08T22:05:18.518750+0000","last_became_active":"2024-05-08T22:05:14.056889+0000","last_became_peered":"2024-05-08T22:05:14.056889+0000","last_unstale":"2024-05-08T22:05:18.518750+0000","last_undegraded":"2024-05-08T22:05:18.518750+0000","last_fullsized":"2024-05-08T22:05:18.518750+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:35.372705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.616206+0000","last_change":"2024-05-08T22:05:14.062282+0000","last_active":"2024-05-08T22:05:18.616206+0000","last_peered":"2024-05-08T22:05:18.616206+0000","last_clean":"2024-05-08T22:05:18.616206+0000","last_became_active":"2024-05-08T22:05:14.061574+0000","last_became_peered":"2024-05-08T22:05:14.061574+0000","last_unstale":"2024-05-08T22:05:18.616206+0000","last_undegraded":"2024-05-08T22:05:18.616206+0000","last_fullsized":"2024-05-08T22:05:18.616206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:16:32.873590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.516691+0000","last_change":"2024-05-08T22:05:14.061267+0000","last_active":"2024-05-08T22:05:18.516691+0000","last_peered":"2024-05-08T22:05:18.516691+0000","last_clean":"2024-05-08T22:05:18.516691+0000","last_became_active":"2024-05-08T22:05:14.060943+0000","last_became_peered":"2024-05-08T22:05:14.060943+0000","last_unstale":"2024-05-08T22:05:18.516691+0000","last_undegraded":"2024-05-08T22:05:18.516691+0000","last_fullsized":"2024-05-08T22:05:18.516691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:03.831138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.019220+0000","last_change":"2024-05-08T22:05:14.067136+0000","last_active":"2024-05-08T22:05:19.019220+0000","last_peered":"2024-05-08T22:05:19.019220+0000","last_clean":"2024-05-08T22:05:19.019220+0000","last_became_active":"2024-05-08T22:05:14.066810+0000","last_became_peered":"2024-05-08T22:05:14.066810+0000","last_unstale":"2024-05-08T22:05:19.019220+0000","last_undegraded":"2024-05-08T22:05:19.019220+0000","last_fullsized":"2024-05-08T22:05:19.019220+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:43.001024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.154187+0000","last_change":"2024-05-08T22:05:14.062119+0000","last_active":"2024-05-08T22:05:19.154187+0000","last_peered":"2024-05-08T22:05:19.154187+0000","last_clean":"2024-05-08T22:05:19.154187+0000","last_became_active":"2024-05-08T22:05:14.058977+0000","last_became_peered":"2024-05-08T22:05:14.058977+0000","last_unstale":"2024-05-08T22:05:19.154187+0000","last_undegraded":"2024-05-08T22:05:19.154187+0000","last_fullsized":"2024-05-08T22:05:19.154187+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:03.657985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.519879+0000","last_change":"2024-05-08T22:05:14.067363+0000","last_active":"2024-05-08T22:05:18.519879+0000","last_peered":"2024-05-08T22:05:18.519879+0000","last_clean":"2024-05-08T22:05:18.519879+0000","last_became_active":"2024-05-08T22:05:14.066119+0000","last_became_peered":"2024-05-08T22:05:14.066119+0000","last_unstale":"2024-05-08T22:05:18.519879+0000","last_undegraded":"2024-05-08T22:05:18.519879+0000","last_fullsized":"2024-05-08T22:05:18.519879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:43.426750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.517625+0000","last_change":"2024-05-08T22:05:14.073925+0000","last_active":"2024-05-08T22:05:18.517625+0000","last_peered":"2024-05-08T22:05:18.517625+0000","last_clean":"2024-05-08T22:05:18.517625+0000","last_became_active":"2024-05-08T22:05:14.072287+0000","last_became_peered":"2024-05-08T22:05:14.072287+0000","last_unstale":"2024-05-08T22:05:18.517625+0000","last_undegraded":"2024-05-08T22:05:18.517625+0000","last_fullsized":"2024-05-08T22:05:18.517625+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:13.537938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.517166+0000","last_change":"2024-05-08T22:05:14.057274+0000","last_active":"2024-05-08T22:05:18.517166+0000","last_peered":"2024-05-08T22:05:18.517166+0000","last_clean":"2024-05-08T22:05:18.517166+0000","last_became_active":"2024-05-08T22:05:14.056989+0000","last_became_peered":"2024-05-08T22:05:14.056989+0000","last_unstale":"2024-05-08T22:05:18.517166+0000","last_undegraded":"2024-05-08T22:05:18.517166+0000","last_fullsized":"2024-05-08T22:05:18.517166+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:32.316406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518749+0000","last_change":"2024-05-08T22:05:14.069263+0000","last_active":"2024-05-08T22:05:18.518749+0000","last_peered":"2024-05-08T22:05:18.518749+0000","last_clean":"2024-05-08T22:05:18.518749+0000","last_became_active":"2024-05-08T22:05:14.068960+0000","last_became_peered":"2024-05-08T22:05:14.068960+0000","last_unstale":"2024-05-08T22:05:18.518749+0000","last_undegraded":"2024-05-08T22:05:18.518749+0000","last_fullsized":"2024-05-08T22:05:18.518749+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:59:42.829110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.514007+0000","last_change":"2024-05-08T22:05:14.049393+0000","last_active":"2024-05-08T22:05:18.514007+0000","last_peered":"2024-05-08T22:05:18.514007+0000","last_clean":"2024-05-08T22:05:18.514007+0000","last_became_active":"2024-05-08T22:05:14.049094+0000","last_became_peered":"2024-05-08T22:05:14.049094+0000","last_unstale":"2024-05-08T22:05:18.514007+0000","last_undegraded":"2024-05-08T22:05:18.514007+0000","last_fullsized":"2024-05-08T22:05:18.514007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:42.106568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.039735+0000","last_change":"2024-05-08T22:05:14.055436+0000","last_active":"2024-05-08T22:05:19.039735+0000","last_peered":"2024-05-08T22:05:19.039735+0000","last_clean":"2024-05-08T22:05:19.039735+0000","last_became_active":"2024-05-08T22:05:14.055100+0000","last_became_peered":"2024-05-08T22:05:14.055100+0000","last_unstale":"2024-05-08T22:05:19.039735+0000","last_undegraded":"2024-05-08T22:05:19.039735+0000","last_fullsized":"2024-05-08T22:05:19.039735+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:52.833780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.022998+0000","last_change":"2024-05-08T22:05:18.532086+0000","last_active":"2024-05-08T22:05:21.022998+0000","last_peered":"2024-05-08T22:05:21.022998+0000","last_clean":"2024-05-08T22:05:21.022998+0000","last_became_active":"2024-05-08T22:05:18.531485+0000","last_became_peered":"2024-05-08T22:05:18.531485+0000","last_unstale":"2024-05-08T22:05:21.022998+0000","last_undegraded":"2024-05-08T22:05:21.022998+0000","last_fullsized":"2024-05-08T22:05:21.022998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:40:36.050367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:38.499417+0000","last_change":"2024-05-08T22:05:13.549611+0000","last_active":"2024-05-08T22:05:38.499417+0000","last_peered":"2024-05-08T22:05:38.499417+0000","last_clean":"2024-05-08T22:05:38.499417+0000","last_became_active":"2024-05-08T22:05:13.549140+0000","last_became_peered":"2024-05-08T22:05:13.549140+0000","last_unstale":"2024-05-08T22:05:38.499417+0000","last_undegraded":"2024-05-08T22:05:38.499417+0000","last_fullsized":"2024-05-08T22:05:38.499417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:54.785123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714719+0000","last_change":"2024-05-08T22:05:14.057570+0000","last_active":"2024-05-08T22:05:19.714719+0000","last_peered":"2024-05-08T22:05:19.714719+0000","last_clean":"2024-05-08T22:05:19.714719+0000","last_became_active":"2024-05-08T22:05:14.057291+0000","last_became_peered":"2024-05-08T22:05:14.057291+0000","last_unstale":"2024-05-08T22:05:19.714719+0000","last_undegraded":"2024-05-08T22:05:19.714719+0000","last_fullsized":"2024-05-08T22:05:19.714719+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:15:38.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"peering","last_fresh":"2024-05-08T22:05:40.819123+0000","last_change":"2024-05-08T22:05:40.819370+0000","last_active":"2024-05-08T22:05:40.819123+0000","last_peered":"2024-05-08T22:05:21.804836+0000","last_clean":"2024-05-08T22:05:21.804836+0000","last_became_active":"2024-05-08T22:05:18.529966+0000","last_became_peered":"2024-05-08T22:05:18.529966+0000","last_unstale":"2024-05-08T22:05:40.819123+0000","last_undegraded":"2024-05-08T22:05:40.819123+0000","last_fullsized":"2024-05-08T22:05:40.819123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:28:51.858677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6,8,9],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:38.500116+0000","last_change":"2024-05-08T22:05:13.551351+0000","last_active":"2024-05-08T22:05:38.500116+0000","last_peered":"2024-05-08T22:05:38.500116+0000","last_clean":"2024-05-08T22:05:38.500116+0000","last_became_active":"2024-05-08T22:05:13.551014+0000","last_became_peered":"2024-05-08T22:05:13.551014+0000","last_unstale":"2024-05-08T22:05:38.500116+0000","last_undegraded":"2024-05-08T22:05:38.500116+0000","last_fullsized":"2024-05-08T22:05:38.500116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:07:22.405251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.525219+0000","last_change":"2024-05-08T22:05:14.067270+0000","last_active":"2024-05-08T22:05:18.525219+0000","last_peered":"2024-05-08T22:05:18.525219+0000","last_clean":"2024-05-08T22:05:18.525219+0000","last_became_active":"2024-05-08T22:05:14.065496+0000","last_became_peered":"2024-05-08T22:05:14.065496+0000","last_unstale":"2024-05-08T22:05:18.525219+0000","last_undegraded":"2024-05-08T22:05:18.525219+0000","last_fullsized":"2024-05-08T22:05:18.525219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:07.087880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.804877+0000","last_change":"2024-05-08T22:05:18.526557+0000","last_active":"2024-05-08T22:05:21.804877+0000","last_peered":"2024-05-08T22:05:21.804877+0000","last_clean":"2024-05-08T22:05:21.804877+0000","last_became_active":"2024-05-08T22:05:18.526129+0000","last_became_peered":"2024-05-08T22:05:18.526129+0000","last_unstale":"2024-05-08T22:05:21.804877+0000","last_undegraded":"2024-05-08T22:05:21.804877+0000","last_fullsized":"2024-05-08T22:05:21.804877+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:36.376579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"78'2","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.510733+0000","last_change":"2024-05-08T22:05:13.553630+0000","last_active":"2024-05-08T22:05:18.510733+0000","last_peered":"2024-05-08T22:05:18.510733+0000","last_clean":"2024-05-08T22:05:18.510733+0000","last_became_active":"2024-05-08T22:05:13.552879+0000","last_became_peered":"2024-05-08T22:05:13.552879+0000","last_unstale":"2024-05-08T22:05:18.510733+0000","last_undegraded":"2024-05-08T22:05:18.510733+0000","last_fullsized":"2024-05-08T22:05:18.510733+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:39:58.253305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.517225+0000","last_change":"2024-05-08T22:05:14.060089+0000","last_active":"2024-05-08T22:05:18.517225+0000","last_peered":"2024-05-08T22:05:18.517225+0000","last_clean":"2024-05-08T22:05:18.517225+0000","last_became_active":"2024-05-08T22:05:14.057620+0000","last_became_peered":"2024-05-08T22:05:14.057620+0000","last_unstale":"2024-05-08T22:05:18.517225+0000","last_undegraded":"2024-05-08T22:05:18.517225+0000","last_fullsized":"2024-05-08T22:05:18.517225+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:29:33.127510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.804864+0000","last_change":"2024-05-08T22:05:18.529746+0000","last_active":"2024-05-08T22:05:21.804864+0000","last_peered":"2024-05-08T22:05:21.804864+0000","last_clean":"2024-05-08T22:05:21.804864+0000","last_became_active":"2024-05-08T22:05:18.529097+0000","last_became_peered":"2024-05-08T22:05:18.529097+0000","last_unstale":"2024-05-08T22:05:21.804864+0000","last_undegraded":"2024-05-08T22:05:21.804864+0000","last_fullsized":"2024-05-08T22:05:21.804864+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:59:36.249787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.510546+0000","last_change":"2024-05-08T22:05:13.547070+0000","last_active":"2024-05-08T22:05:18.510546+0000","last_peered":"2024-05-08T22:05:18.510546+0000","last_clean":"2024-05-08T22:05:18.510546+0000","last_became_active":"2024-05-08T22:05:13.546708+0000","last_became_peered":"2024-05-08T22:05:13.546708+0000","last_unstale":"2024-05-08T22:05:18.510546+0000","last_undegraded":"2024-05-08T22:05:18.510546+0000","last_fullsized":"2024-05-08T22:05:18.510546+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:10.577900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.526371+0000","last_change":"2024-05-08T22:05:14.064695+0000","last_active":"2024-05-08T22:05:18.526371+0000","last_peered":"2024-05-08T22:05:18.526371+0000","last_clean":"2024-05-08T22:05:18.526371+0000","last_became_active":"2024-05-08T22:05:14.064283+0000","last_became_peered":"2024-05-08T22:05:14.064283+0000","last_unstale":"2024-05-08T22:05:18.526371+0000","last_undegraded":"2024-05-08T22:05:18.526371+0000","last_fullsized":"2024-05-08T22:05:18.526371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:06.980594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.023095+0000","last_change":"2024-05-08T22:05:18.528475+0000","last_active":"2024-05-08T22:05:21.023095+0000","last_peered":"2024-05-08T22:05:21.023095+0000","last_clean":"2024-05-08T22:05:21.023095+0000","last_became_active":"2024-05-08T22:05:18.528089+0000","last_became_peered":"2024-05-08T22:05:18.528089+0000","last_unstale":"2024-05-08T22:05:21.023095+0000","last_undegraded":"2024-05-08T22:05:21.023095+0000","last_fullsized":"2024-05-08T22:05:21.023095+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:01.508984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:39.502402+0000","last_change":"2024-05-08T22:05:13.549191+0000","last_active":"2024-05-08T22:05:39.502402+0000","last_peered":"2024-05-08T22:05:39.502402+0000","last_clean":"2024-05-08T22:05:39.502402+0000","last_became_active":"2024-05-08T22:05:13.548737+0000","last_became_peered":"2024-05-08T22:05:13.548737+0000","last_unstale":"2024-05-08T22:05:39.502402+0000","last_undegraded":"2024-05-08T22:05:39.502402+0000","last_fullsized":"2024-05-08T22:05:39.502402+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:07.355656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830198+0000","last_change":"2024-05-08T22:05:14.065910+0000","last_active":"2024-05-08T22:05:40.830198+0000","last_peered":"2024-05-08T22:05:40.830198+0000","last_clean":"2024-05-08T22:05:40.830198+0000","last_became_active":"2024-05-08T22:05:14.065504+0000","last_became_peered":"2024-05-08T22:05:14.065504+0000","last_unstale":"2024-05-08T22:05:40.830198+0000","last_undegraded":"2024-05-08T22:05:40.830198+0000","last_fullsized":"2024-05-08T22:05:40.830198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:12.447292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.023073+0000","last_change":"2024-05-08T22:05:18.536861+0000","last_active":"2024-05-08T22:05:21.023073+0000","last_peered":"2024-05-08T22:05:21.023073+0000","last_clean":"2024-05-08T22:05:21.023073+0000","last_became_active":"2024-05-08T22:05:18.536472+0000","last_became_peered":"2024-05-08T22:05:18.536472+0000","last_unstale":"2024-05-08T22:05:21.023073+0000","last_undegraded":"2024-05-08T22:05:21.023073+0000","last_fullsized":"2024-05-08T22:05:21.023073+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:45:21.826812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"78'1","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.615381+0000","last_change":"2024-05-08T22:05:13.553326+0000","last_active":"2024-05-08T22:05:18.615381+0000","last_peered":"2024-05-08T22:05:18.615381+0000","last_clean":"2024-05-08T22:05:18.615381+0000","last_became_active":"2024-05-08T22:05:13.552983+0000","last_became_peered":"2024-05-08T22:05:13.552983+0000","last_unstale":"2024-05-08T22:05:18.615381+0000","last_undegraded":"2024-05-08T22:05:18.615381+0000","last_fullsized":"2024-05-08T22:05:18.615381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:17.195253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.526142+0000","last_change":"2024-05-08T22:05:14.068617+0000","last_active":"2024-05-08T22:05:18.526142+0000","last_peered":"2024-05-08T22:05:18.526142+0000","last_clean":"2024-05-08T22:05:18.526142+0000","last_became_active":"2024-05-08T22:05:14.067400+0000","last_became_peered":"2024-05-08T22:05:14.067400+0000","last_unstale":"2024-05-08T22:05:18.526142+0000","last_undegraded":"2024-05-08T22:05:18.526142+0000","last_fullsized":"2024-05-08T22:05:18.526142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:25.488212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.018557+0000","last_change":"2024-05-08T22:05:18.538294+0000","last_active":"2024-05-08T22:05:19.018557+0000","last_peered":"2024-05-08T22:05:19.018557+0000","last_clean":"2024-05-08T22:05:19.018557+0000","last_became_active":"2024-05-08T22:05:18.537984+0000","last_became_peered":"2024-05-08T22:05:18.537984+0000","last_unstale":"2024-05-08T22:05:19.018557+0000","last_undegraded":"2024-05-08T22:05:19.018557+0000","last_fullsized":"2024-05-08T22:05:19.018557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:32.117439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:38.500252+0000","last_change":"2024-05-08T22:05:13.557669+0000","last_active":"2024-05-08T22:05:38.500252+0000","last_peered":"2024-05-08T22:05:38.500252+0000","last_clean":"2024-05-08T22:05:38.500252+0000","last_became_active":"2024-05-08T22:05:13.555835+0000","last_became_peered":"2024-05-08T22:05:13.555835+0000","last_unstale":"2024-05-08T22:05:38.500252+0000","last_undegraded":"2024-05-08T22:05:38.500252+0000","last_fullsized":"2024-05-08T22:05:38.500252+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:02.377635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714321+0000","last_change":"2024-05-08T22:05:14.060144+0000","last_active":"2024-05-08T22:05:19.714321+0000","last_peered":"2024-05-08T22:05:19.714321+0000","last_clean":"2024-05-08T22:05:19.714321+0000","last_became_active":"2024-05-08T22:05:14.059387+0000","last_became_peered":"2024-05-08T22:05:14.059387+0000","last_unstale":"2024-05-08T22:05:19.714321+0000","last_undegraded":"2024-05-08T22:05:19.714321+0000","last_fullsized":"2024-05-08T22:05:19.714321+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:32:28.057831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.154473+0000","last_change":"2024-05-08T22:05:18.527293+0000","last_active":"2024-05-08T22:05:19.154473+0000","last_peered":"2024-05-08T22:05:19.154473+0000","last_clean":"2024-05-08T22:05:19.154473+0000","last_became_active":"2024-05-08T22:05:18.526690+0000","last_became_peered":"2024-05-08T22:05:18.526690+0000","last_unstale":"2024-05-08T22:05:19.154473+0000","last_undegraded":"2024-05-08T22:05:19.154473+0000","last_fullsized":"2024-05-08T22:05:19.154473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:50.343536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:20.445763+0000","last_change":"2024-05-08T22:05:13.549774+0000","last_active":"2024-05-08T22:05:20.445763+0000","last_peered":"2024-05-08T22:05:20.445763+0000","last_clean":"2024-05-08T22:05:20.445763+0000","last_became_active":"2024-05-08T22:05:13.549531+0000","last_became_peered":"2024-05-08T22:05:13.549531+0000","last_unstale":"2024-05-08T22:05:20.445763+0000","last_undegraded":"2024-05-08T22:05:20.445763+0000","last_fullsized":"2024-05-08T22:05:20.445763+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:59:23.798728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.513387+0000","last_change":"2024-05-08T22:05:14.054232+0000","last_active":"2024-05-08T22:05:18.513387+0000","last_peered":"2024-05-08T22:05:18.513387+0000","last_clean":"2024-05-08T22:05:18.513387+0000","last_became_active":"2024-05-08T22:05:14.053806+0000","last_became_peered":"2024-05-08T22:05:14.053806+0000","last_unstale":"2024-05-08T22:05:18.513387+0000","last_undegraded":"2024-05-08T22:05:18.513387+0000","last_fullsized":"2024-05-08T22:05:18.513387+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:32:21.678792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.504963+0000","last_change":"2024-05-08T22:05:18.535241+0000","last_active":"2024-05-08T22:05:21.504963+0000","last_peered":"2024-05-08T22:05:21.504963+0000","last_clean":"2024-05-08T22:05:21.504963+0000","last_became_active":"2024-05-08T22:05:18.534753+0000","last_became_peered":"2024-05-08T22:05:18.534753+0000","last_unstale":"2024-05-08T22:05:21.504963+0000","last_undegraded":"2024-05-08T22:05:21.504963+0000","last_fullsized":"2024-05-08T22:05:21.504963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:46:15.138437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.493918+0000","last_change":"2024-05-08T22:05:13.544899+0000","last_active":"2024-05-08T22:05:37.493918+0000","last_peered":"2024-05-08T22:05:37.493918+0000","last_clean":"2024-05-08T22:05:37.493918+0000","last_became_active":"2024-05-08T22:05:13.544498+0000","last_became_peered":"2024-05-08T22:05:13.544498+0000","last_unstale":"2024-05-08T22:05:37.493918+0000","last_undegraded":"2024-05-08T22:05:37.493918+0000","last_fullsized":"2024-05-08T22:05:37.493918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:58.169648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518687+0000","last_change":"2024-05-08T22:05:14.049377+0000","last_active":"2024-05-08T22:05:18.518687+0000","last_peered":"2024-05-08T22:05:18.518687+0000","last_clean":"2024-05-08T22:05:18.518687+0000","last_became_active":"2024-05-08T22:05:14.049039+0000","last_became_peered":"2024-05-08T22:05:14.049039+0000","last_unstale":"2024-05-08T22:05:18.518687+0000","last_undegraded":"2024-05-08T22:05:18.518687+0000","last_fullsized":"2024-05-08T22:05:18.518687+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:24.415392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.017866+0000","last_change":"2024-05-08T22:05:18.528912+0000","last_active":"2024-05-08T22:05:19.017866+0000","last_peered":"2024-05-08T22:05:19.017866+0000","last_clean":"2024-05-08T22:05:19.017866+0000","last_became_active":"2024-05-08T22:05:18.528433+0000","last_became_peered":"2024-05-08T22:05:18.528433+0000","last_unstale":"2024-05-08T22:05:19.017866+0000","last_undegraded":"2024-05-08T22:05:19.017866+0000","last_fullsized":"2024-05-08T22:05:19.017866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:05.574728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.492321+0000","last_change":"2024-05-08T22:05:13.550214+0000","last_active":"2024-05-08T22:05:37.492321+0000","last_peered":"2024-05-08T22:05:37.492321+0000","last_clean":"2024-05-08T22:05:37.492321+0000","last_became_active":"2024-05-08T22:05:13.549717+0000","last_became_peered":"2024-05-08T22:05:13.549717+0000","last_unstale":"2024-05-08T22:05:37.492321+0000","last_undegraded":"2024-05-08T22:05:37.492321+0000","last_fullsized":"2024-05-08T22:05:37.492321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:12:20.406333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.525310+0000","last_change":"2024-05-08T22:05:14.071214+0000","last_active":"2024-05-08T22:05:18.525310+0000","last_peered":"2024-05-08T22:05:18.525310+0000","last_clean":"2024-05-08T22:05:18.525310+0000","last_became_active":"2024-05-08T22:05:14.066777+0000","last_became_peered":"2024-05-08T22:05:14.066777+0000","last_unstale":"2024-05-08T22:05:18.525310+0000","last_undegraded":"2024-05-08T22:05:18.525310+0000","last_fullsized":"2024-05-08T22:05:18.525310+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:53.360911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.804896+0000","last_change":"2024-05-08T22:05:18.523688+0000","last_active":"2024-05-08T22:05:21.804896+0000","last_peered":"2024-05-08T22:05:21.804896+0000","last_clean":"2024-05-08T22:05:21.804896+0000","last_became_active":"2024-05-08T22:05:18.523236+0000","last_became_peered":"2024-05-08T22:05:18.523236+0000","last_unstale":"2024-05-08T22:05:21.804896+0000","last_undegraded":"2024-05-08T22:05:21.804896+0000","last_fullsized":"2024-05-08T22:05:21.804896+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:34.531699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.494031+0000","last_change":"2024-05-08T22:05:13.551027+0000","last_active":"2024-05-08T22:05:37.494031+0000","last_peered":"2024-05-08T22:05:37.494031+0000","last_clean":"2024-05-08T22:05:37.494031+0000","last_became_active":"2024-05-08T22:05:13.550471+0000","last_became_peered":"2024-05-08T22:05:13.550471+0000","last_unstale":"2024-05-08T22:05:37.494031+0000","last_undegraded":"2024-05-08T22:05:37.494031+0000","last_fullsized":"2024-05-08T22:05:37.494031+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:50.900419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.517424+0000","last_change":"2024-05-08T22:05:14.065191+0000","last_active":"2024-05-08T22:05:18.517424+0000","last_peered":"2024-05-08T22:05:18.517424+0000","last_clean":"2024-05-08T22:05:18.517424+0000","last_became_active":"2024-05-08T22:05:14.064893+0000","last_became_peered":"2024-05-08T22:05:14.064893+0000","last_unstale":"2024-05-08T22:05:18.517424+0000","last_undegraded":"2024-05-08T22:05:18.517424+0000","last_fullsized":"2024-05-08T22:05:18.517424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:11.344646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.017892+0000","last_change":"2024-05-08T22:05:18.530447+0000","last_active":"2024-05-08T22:05:19.017892+0000","last_peered":"2024-05-08T22:05:19.017892+0000","last_clean":"2024-05-08T22:05:19.017892+0000","last_became_active":"2024-05-08T22:05:18.528171+0000","last_became_peered":"2024-05-08T22:05:18.528171+0000","last_unstale":"2024-05-08T22:05:19.017892+0000","last_undegraded":"2024-05-08T22:05:19.017892+0000","last_fullsized":"2024-05-08T22:05:19.017892+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:09:43.347366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"83'12","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.493215+0000","last_change":"2024-05-08T22:05:13.547289+0000","last_active":"2024-05-08T22:05:37.493215+0000","last_peered":"2024-05-08T22:05:37.493215+0000","last_clean":"2024-05-08T22:05:37.493215+0000","last_became_active":"2024-05-08T22:05:13.546860+0000","last_became_peered":"2024-05-08T22:05:13.546860+0000","last_unstale":"2024-05-08T22:05:37.493215+0000","last_undegraded":"2024-05-08T22:05:37.493215+0000","last_fullsized":"2024-05-08T22:05:37.493215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:08:50.339361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831151+0000","last_change":"2024-05-08T22:05:14.057925+0000","last_active":"2024-05-08T22:05:40.831151+0000","last_peered":"2024-05-08T22:05:40.831151+0000","last_clean":"2024-05-08T22:05:40.831151+0000","last_became_active":"2024-05-08T22:05:14.057662+0000","last_became_peered":"2024-05-08T22:05:14.057662+0000","last_unstale":"2024-05-08T22:05:40.831151+0000","last_undegraded":"2024-05-08T22:05:40.831151+0000","last_fullsized":"2024-05-08T22:05:40.831151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:49.929610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.834134+0000","last_change":"2024-05-08T22:05:18.535509+0000","last_active":"2024-05-08T22:05:19.834134+0000","last_peered":"2024-05-08T22:05:19.834134+0000","last_clean":"2024-05-08T22:05:19.834134+0000","last_became_active":"2024-05-08T22:05:18.535024+0000","last_became_peered":"2024-05-08T22:05:18.535024+0000","last_unstale":"2024-05-08T22:05:19.834134+0000","last_undegraded":"2024-05-08T22:05:19.834134+0000","last_fullsized":"2024-05-08T22:05:19.834134+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:28.098737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:39.501569+0000","last_change":"2024-05-08T22:05:13.552096+0000","last_active":"2024-05-08T22:05:39.501569+0000","last_peered":"2024-05-08T22:05:39.501569+0000","last_clean":"2024-05-08T22:05:39.501569+0000","last_became_active":"2024-05-08T22:05:13.551655+0000","last_became_peered":"2024-05-08T22:05:13.551655+0000","last_unstale":"2024-05-08T22:05:39.501569+0000","last_undegraded":"2024-05-08T22:05:39.501569+0000","last_fullsized":"2024-05-08T22:05:39.501569+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:24.371008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.153911+0000","last_change":"2024-05-08T22:05:14.066075+0000","last_active":"2024-05-08T22:05:19.153911+0000","last_peered":"2024-05-08T22:05:19.153911+0000","last_clean":"2024-05-08T22:05:19.153911+0000","last_became_active":"2024-05-08T22:05:14.065581+0000","last_became_peered":"2024-05-08T22:05:14.065581+0000","last_unstale":"2024-05-08T22:05:19.153911+0000","last_undegraded":"2024-05-08T22:05:19.153911+0000","last_fullsized":"2024-05-08T22:05:19.153911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:15.403714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.018254+0000","last_change":"2024-05-08T22:05:18.534033+0000","last_active":"2024-05-08T22:05:19.018254+0000","last_peered":"2024-05-08T22:05:19.018254+0000","last_clean":"2024-05-08T22:05:19.018254+0000","last_became_active":"2024-05-08T22:05:18.533563+0000","last_became_peered":"2024-05-08T22:05:18.533563+0000","last_unstale":"2024-05-08T22:05:19.018254+0000","last_undegraded":"2024-05-08T22:05:19.018254+0000","last_fullsized":"2024-05-08T22:05:19.018254+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:37:52.657794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.493496+0000","last_change":"2024-05-08T22:05:13.549112+0000","last_active":"2024-05-08T22:05:37.493496+0000","last_peered":"2024-05-08T22:05:37.493496+0000","last_clean":"2024-05-08T22:05:37.493496+0000","last_became_active":"2024-05-08T22:05:13.548763+0000","last_became_peered":"2024-05-08T22:05:13.548763+0000","last_unstale":"2024-05-08T22:05:37.493496+0000","last_undegraded":"2024-05-08T22:05:37.493496+0000","last_fullsized":"2024-05-08T22:05:37.493496+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:26:56.448611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.516056+0000","last_change":"2024-05-08T22:05:14.067116+0000","last_active":"2024-05-08T22:05:18.516056+0000","last_peered":"2024-05-08T22:05:18.516056+0000","last_clean":"2024-05-08T22:05:18.516056+0000","last_became_active":"2024-05-08T22:05:14.064775+0000","last_became_peered":"2024-05-08T22:05:14.064775+0000","last_unstale":"2024-05-08T22:05:18.516056+0000","last_undegraded":"2024-05-08T22:05:18.516056+0000","last_fullsized":"2024-05-08T22:05:18.516056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:28:32.901405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.018404+0000","last_change":"2024-05-08T22:05:18.539546+0000","last_active":"2024-05-08T22:05:19.018404+0000","last_peered":"2024-05-08T22:05:19.018404+0000","last_clean":"2024-05-08T22:05:19.018404+0000","last_became_active":"2024-05-08T22:05:18.538998+0000","last_became_peered":"2024-05-08T22:05:18.538998+0000","last_unstale":"2024-05-08T22:05:19.018404+0000","last_undegraded":"2024-05-08T22:05:19.018404+0000","last_fullsized":"2024-05-08T22:05:19.018404+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:30.453881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"83'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819019+0000","last_change":"2024-05-08T22:05:13.549053+0000","last_active":"2024-05-08T22:05:40.819019+0000","last_peered":"2024-05-08T22:05:40.819019+0000","last_clean":"2024-05-08T22:05:40.819019+0000","last_became_active":"2024-05-08T22:05:13.548739+0000","last_became_peered":"2024-05-08T22:05:13.548739+0000","last_unstale":"2024-05-08T22:05:40.819019+0000","last_undegraded":"2024-05-08T22:05:40.819019+0000","last_fullsized":"2024-05-08T22:05:40.819019+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:07:10.298084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.516222+0000","last_change":"2024-05-08T22:05:14.061753+0000","last_active":"2024-05-08T22:05:18.516222+0000","last_peered":"2024-05-08T22:05:18.516222+0000","last_clean":"2024-05-08T22:05:18.516222+0000","last_became_active":"2024-05-08T22:05:14.061412+0000","last_became_peered":"2024-05-08T22:05:14.061412+0000","last_unstale":"2024-05-08T22:05:18.516222+0000","last_undegraded":"2024-05-08T22:05:18.516222+0000","last_fullsized":"2024-05-08T22:05:18.516222+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:54.380181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.718981+0000","last_change":"2024-05-08T22:05:18.532627+0000","last_active":"2024-05-08T22:05:21.718981+0000","last_peered":"2024-05-08T22:05:21.718981+0000","last_clean":"2024-05-08T22:05:21.718981+0000","last_became_active":"2024-05-08T22:05:18.532205+0000","last_became_peered":"2024-05-08T22:05:18.532205+0000","last_unstale":"2024-05-08T22:05:21.718981+0000","last_undegraded":"2024-05-08T22:05:21.718981+0000","last_fullsized":"2024-05-08T22:05:21.718981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:20:11.028645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":61,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.841832+0000","last_change":"2024-05-08T22:05:41.841832+0000","last_active":"2024-05-08T22:05:41.841832+0000","last_peered":"2024-05-08T22:05:41.841832+0000","last_clean":"2024-05-08T22:05:41.841832+0000","last_became_active":"2024-05-08T22:05:41.829584+0000","last_became_peered":"2024-05-08T22:05:41.829584+0000","last_unstale":"2024-05-08T22:05:41.841832+0000","last_undegraded":"2024-05-08T22:05:41.841832+0000","last_fullsized":"2024-05-08T22:05:41.841832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:52:19.876215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.525104+0000","last_change":"2024-05-08T22:05:14.063027+0000","last_active":"2024-05-08T22:05:18.525104+0000","last_peered":"2024-05-08T22:05:18.525104+0000","last_clean":"2024-05-08T22:05:18.525104+0000","last_became_active":"2024-05-08T22:05:14.062682+0000","last_became_peered":"2024-05-08T22:05:14.062682+0000","last_unstale":"2024-05-08T22:05:18.525104+0000","last_undegraded":"2024-05-08T22:05:18.525104+0000","last_fullsized":"2024-05-08T22:05:18.525104+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:47.248832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.834071+0000","last_change":"2024-05-08T22:05:18.528370+0000","last_active":"2024-05-08T22:05:19.834071+0000","last_peered":"2024-05-08T22:05:19.834071+0000","last_clean":"2024-05-08T22:05:19.834071+0000","last_became_active":"2024-05-08T22:05:18.527990+0000","last_became_peered":"2024-05-08T22:05:18.527990+0000","last_unstale":"2024-05-08T22:05:19.834071+0000","last_undegraded":"2024-05-08T22:05:19.834071+0000","last_fullsized":"2024-05-08T22:05:19.834071+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:07.524175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829860+0000","last_change":"2024-05-08T22:05:13.547543+0000","last_active":"2024-05-08T22:05:40.829860+0000","last_peered":"2024-05-08T22:05:40.829860+0000","last_clean":"2024-05-08T22:05:40.829860+0000","last_became_active":"2024-05-08T22:05:13.547191+0000","last_became_peered":"2024-05-08T22:05:13.547191+0000","last_unstale":"2024-05-08T22:05:40.829860+0000","last_undegraded":"2024-05-08T22:05:40.829860+0000","last_fullsized":"2024-05-08T22:05:40.829860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:53.213239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518149+0000","last_change":"2024-05-08T22:05:14.062677+0000","last_active":"2024-05-08T22:05:18.518149+0000","last_peered":"2024-05-08T22:05:18.518149+0000","last_clean":"2024-05-08T22:05:18.518149+0000","last_became_active":"2024-05-08T22:05:14.062306+0000","last_became_peered":"2024-05-08T22:05:14.062306+0000","last_unstale":"2024-05-08T22:05:18.518149+0000","last_undegraded":"2024-05-08T22:05:18.518149+0000","last_fullsized":"2024-05-08T22:05:18.518149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:00.356505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830129+0000","last_change":"2024-05-08T22:05:18.617075+0000","last_active":"2024-05-08T22:05:40.830129+0000","last_peered":"2024-05-08T22:05:40.830129+0000","last_clean":"2024-05-08T22:05:40.830129+0000","last_became_active":"2024-05-08T22:05:18.616653+0000","last_became_peered":"2024-05-08T22:05:18.616653+0000","last_unstale":"2024-05-08T22:05:40.830129+0000","last_undegraded":"2024-05-08T22:05:40.830129+0000","last_fullsized":"2024-05-08T22:05:40.830129+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:30.537064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.513484+0000","last_change":"2024-05-08T22:05:13.553457+0000","last_active":"2024-05-08T22:05:18.513484+0000","last_peered":"2024-05-08T22:05:18.513484+0000","last_clean":"2024-05-08T22:05:18.513484+0000","last_became_active":"2024-05-08T22:05:13.551583+0000","last_became_peered":"2024-05-08T22:05:13.551583+0000","last_unstale":"2024-05-08T22:05:18.513484+0000","last_undegraded":"2024-05-08T22:05:18.513484+0000","last_fullsized":"2024-05-08T22:05:18.513484+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:07:05.971413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.616101+0000","last_change":"2024-05-08T22:05:14.057476+0000","last_active":"2024-05-08T22:05:18.616101+0000","last_peered":"2024-05-08T22:05:18.616101+0000","last_clean":"2024-05-08T22:05:18.616101+0000","last_became_active":"2024-05-08T22:05:14.056550+0000","last_became_peered":"2024-05-08T22:05:14.056550+0000","last_unstale":"2024-05-08T22:05:18.616101+0000","last_undegraded":"2024-05-08T22:05:18.616101+0000","last_fullsized":"2024-05-08T22:05:18.616101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:41:29.264096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.718898+0000","last_change":"2024-05-08T22:05:18.530719+0000","last_active":"2024-05-08T22:05:21.718898+0000","last_peered":"2024-05-08T22:05:21.718898+0000","last_clean":"2024-05-08T22:05:21.718898+0000","last_became_active":"2024-05-08T22:05:18.529659+0000","last_became_peered":"2024-05-08T22:05:18.529659+0000","last_unstale":"2024-05-08T22:05:21.718898+0000","last_undegraded":"2024-05-08T22:05:21.718898+0000","last_fullsized":"2024-05-08T22:05:21.718898+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:13:06.514460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:20.446632+0000","last_change":"2024-05-08T22:05:13.556191+0000","last_active":"2024-05-08T22:05:20.446632+0000","last_peered":"2024-05-08T22:05:20.446632+0000","last_clean":"2024-05-08T22:05:20.446632+0000","last_became_active":"2024-05-08T22:05:13.555888+0000","last_became_peered":"2024-05-08T22:05:13.555888+0000","last_unstale":"2024-05-08T22:05:20.446632+0000","last_undegraded":"2024-05-08T22:05:20.446632+0000","last_fullsized":"2024-05-08T22:05:20.446632+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:05.637980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.615673+0000","last_change":"2024-05-08T22:05:14.069974+0000","last_active":"2024-05-08T22:05:18.615673+0000","last_peered":"2024-05-08T22:05:18.615673+0000","last_clean":"2024-05-08T22:05:18.615673+0000","last_became_active":"2024-05-08T22:05:14.069637+0000","last_became_peered":"2024-05-08T22:05:14.069637+0000","last_unstale":"2024-05-08T22:05:18.615673+0000","last_undegraded":"2024-05-08T22:05:18.615673+0000","last_fullsized":"2024-05-08T22:05:18.615673+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:15:03.030292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"peering","last_fresh":"2024-05-08T22:05:40.825443+0000","last_change":"2024-05-08T22:05:40.825855+0000","last_active":"2024-05-08T22:05:40.825443+0000","last_peered":"2024-05-08T22:05:21.719003+0000","last_clean":"2024-05-08T22:05:21.719003+0000","last_became_active":"2024-05-08T22:05:18.616536+0000","last_became_peered":"2024-05-08T22:05:18.616536+0000","last_unstale":"2024-05-08T22:05:40.825443+0000","last_undegraded":"2024-05-08T22:05:40.825443+0000","last_fullsized":"2024-05-08T22:05:40.825443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:30:44.106162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5,8,10],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.525034+0000","last_change":"2024-05-08T22:05:14.069096+0000","last_active":"2024-05-08T22:05:18.525034+0000","last_peered":"2024-05-08T22:05:18.525034+0000","last_clean":"2024-05-08T22:05:18.525034+0000","last_became_active":"2024-05-08T22:05:14.068796+0000","last_became_peered":"2024-05-08T22:05:14.068796+0000","last_unstale":"2024-05-08T22:05:18.525034+0000","last_undegraded":"2024-05-08T22:05:18.525034+0000","last_fullsized":"2024-05-08T22:05:18.525034+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:52:52.391196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.821076+0000","last_change":"2024-05-08T22:05:13.559286+0000","last_active":"2024-05-08T22:05:40.821076+0000","last_peered":"2024-05-08T22:05:40.821076+0000","last_clean":"2024-05-08T22:05:40.821076+0000","last_became_active":"2024-05-08T22:05:13.558922+0000","last_became_peered":"2024-05-08T22:05:13.558922+0000","last_unstale":"2024-05-08T22:05:40.821076+0000","last_undegraded":"2024-05-08T22:05:40.821076+0000","last_fullsized":"2024-05-08T22:05:40.821076+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:25:24.732422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.566174+0000","last_change":"2024-05-08T22:05:18.527031+0000","last_active":"2024-05-08T22:05:18.566174+0000","last_peered":"2024-05-08T22:05:18.566174+0000","last_clean":"2024-05-08T22:05:18.566174+0000","last_became_active":"2024-05-08T22:05:18.526573+0000","last_became_peered":"2024-05-08T22:05:18.526573+0000","last_unstale":"2024-05-08T22:05:18.566174+0000","last_undegraded":"2024-05-08T22:05:18.566174+0000","last_fullsized":"2024-05-08T22:05:18.566174+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:08.492359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714236+0000","last_change":"2024-05-08T22:05:14.053930+0000","last_active":"2024-05-08T22:05:19.714236+0000","last_peered":"2024-05-08T22:05:19.714236+0000","last_clean":"2024-05-08T22:05:19.714236+0000","last_became_active":"2024-05-08T22:05:14.052968+0000","last_became_peered":"2024-05-08T22:05:14.052968+0000","last_unstale":"2024-05-08T22:05:19.714236+0000","last_undegraded":"2024-05-08T22:05:19.714236+0000","last_fullsized":"2024-05-08T22:05:19.714236+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:48.748323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:39.501541+0000","last_change":"2024-05-08T22:05:13.545709+0000","last_active":"2024-05-08T22:05:39.501541+0000","last_peered":"2024-05-08T22:05:39.501541+0000","last_clean":"2024-05-08T22:05:39.501541+0000","last_became_active":"2024-05-08T22:05:13.545419+0000","last_became_peered":"2024-05-08T22:05:13.545419+0000","last_unstale":"2024-05-08T22:05:39.501541+0000","last_undegraded":"2024-05-08T22:05:39.501541+0000","last_fullsized":"2024-05-08T22:05:39.501541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:44.903903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":335,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:20.446030+0000","last_change":"2024-05-08T22:04:20.948328+0000","last_active":"2024-05-08T22:05:20.446030+0000","last_peered":"2024-05-08T22:05:20.446030+0000","last_clean":"2024-05-08T22:05:20.446030+0000","last_became_active":"2024-05-08T22:04:20.947887+0000","last_became_peered":"2024-05-08T22:04:20.947887+0000","last_unstale":"2024-05-08T22:05:20.446030+0000","last_undegraded":"2024-05-08T22:05:20.446030+0000","last_fullsized":"2024-05-08T22:05:20.446030+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:01:41.485277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:01:41.485277+0000","last_clean_scrub_stamp":"2024-05-08T22:01:41.485277+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:43:59.208120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.834146+0000","last_change":"2024-05-08T22:05:18.616989+0000","last_active":"2024-05-08T22:05:19.834146+0000","last_peered":"2024-05-08T22:05:19.834146+0000","last_clean":"2024-05-08T22:05:19.834146+0000","last_became_active":"2024-05-08T22:05:18.616560+0000","last_became_peered":"2024-05-08T22:05:18.616560+0000","last_unstale":"2024-05-08T22:05:19.834146+0000","last_undegraded":"2024-05-08T22:05:19.834146+0000","last_fullsized":"2024-05-08T22:05:19.834146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:45:31.141303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.153692+0000","last_change":"2024-05-08T22:05:14.066818+0000","last_active":"2024-05-08T22:05:19.153692+0000","last_peered":"2024-05-08T22:05:19.153692+0000","last_clean":"2024-05-08T22:05:19.153692+0000","last_became_active":"2024-05-08T22:05:14.065152+0000","last_became_peered":"2024-05-08T22:05:14.065152+0000","last_unstale":"2024-05-08T22:05:19.153692+0000","last_undegraded":"2024-05-08T22:05:19.153692+0000","last_fullsized":"2024-05-08T22:05:19.153692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:53:19.018852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:38.498880+0000","last_change":"2024-05-08T22:05:13.550365+0000","last_active":"2024-05-08T22:05:38.498880+0000","last_peered":"2024-05-08T22:05:38.498880+0000","last_clean":"2024-05-08T22:05:38.498880+0000","last_became_active":"2024-05-08T22:05:13.549677+0000","last_became_peered":"2024-05-08T22:05:13.549677+0000","last_unstale":"2024-05-08T22:05:38.498880+0000","last_undegraded":"2024-05-08T22:05:38.498880+0000","last_fullsized":"2024-05-08T22:05:38.498880+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:10.842838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.018502+0000","last_change":"2024-05-08T22:05:18.535777+0000","last_active":"2024-05-08T22:05:19.018502+0000","last_peered":"2024-05-08T22:05:19.018502+0000","last_clean":"2024-05-08T22:05:19.018502+0000","last_became_active":"2024-05-08T22:05:18.535327+0000","last_became_peered":"2024-05-08T22:05:18.535327+0000","last_unstale":"2024-05-08T22:05:19.018502+0000","last_undegraded":"2024-05-08T22:05:19.018502+0000","last_fullsized":"2024-05-08T22:05:19.018502+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:13.886989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831170+0000","last_change":"2024-05-08T22:05:13.554045+0000","last_active":"2024-05-08T22:05:40.831170+0000","last_peered":"2024-05-08T22:05:40.831170+0000","last_clean":"2024-05-08T22:05:40.831170+0000","last_became_active":"2024-05-08T22:05:13.553315+0000","last_became_peered":"2024-05-08T22:05:13.553315+0000","last_unstale":"2024-05-08T22:05:40.831170+0000","last_undegraded":"2024-05-08T22:05:40.831170+0000","last_fullsized":"2024-05-08T22:05:40.831170+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:13.131267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.154279+0000","last_change":"2024-05-08T22:05:14.065018+0000","last_active":"2024-05-08T22:05:19.154279+0000","last_peered":"2024-05-08T22:05:19.154279+0000","last_clean":"2024-05-08T22:05:19.154279+0000","last_became_active":"2024-05-08T22:05:14.064701+0000","last_became_peered":"2024-05-08T22:05:14.064701+0000","last_unstale":"2024-05-08T22:05:19.154279+0000","last_undegraded":"2024-05-08T22:05:19.154279+0000","last_fullsized":"2024-05-08T22:05:19.154279+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:48:43.468373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.504924+0000","last_change":"2024-05-08T22:05:18.617853+0000","last_active":"2024-05-08T22:05:21.504924+0000","last_peered":"2024-05-08T22:05:21.504924+0000","last_clean":"2024-05-08T22:05:21.504924+0000","last_became_active":"2024-05-08T22:05:18.616255+0000","last_became_peered":"2024-05-08T22:05:18.616255+0000","last_unstale":"2024-05-08T22:05:21.504924+0000","last_undegraded":"2024-05-08T22:05:21.504924+0000","last_fullsized":"2024-05-08T22:05:21.504924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:13.233425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:38.499114+0000","last_change":"2024-05-08T22:05:13.546274+0000","last_active":"2024-05-08T22:05:38.499114+0000","last_peered":"2024-05-08T22:05:38.499114+0000","last_clean":"2024-05-08T22:05:38.499114+0000","last_became_active":"2024-05-08T22:05:13.543651+0000","last_became_peered":"2024-05-08T22:05:13.543651+0000","last_unstale":"2024-05-08T22:05:38.499114+0000","last_undegraded":"2024-05-08T22:05:38.499114+0000","last_fullsized":"2024-05-08T22:05:38.499114+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:21:58.601261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714604+0000","last_change":"2024-05-08T22:05:14.056535+0000","last_active":"2024-05-08T22:05:19.714604+0000","last_peered":"2024-05-08T22:05:19.714604+0000","last_clean":"2024-05-08T22:05:19.714604+0000","last_became_active":"2024-05-08T22:05:14.056211+0000","last_became_peered":"2024-05-08T22:05:14.056211+0000","last_unstale":"2024-05-08T22:05:19.714604+0000","last_undegraded":"2024-05-08T22:05:19.714604+0000","last_fullsized":"2024-05-08T22:05:19.714604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:46:30.744490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.022982+0000","last_change":"2024-05-08T22:05:18.524939+0000","last_active":"2024-05-08T22:05:21.022982+0000","last_peered":"2024-05-08T22:05:21.022982+0000","last_clean":"2024-05-08T22:05:21.022982+0000","last_became_active":"2024-05-08T22:05:18.524518+0000","last_became_peered":"2024-05-08T22:05:18.524518+0000","last_unstale":"2024-05-08T22:05:21.022982+0000","last_undegraded":"2024-05-08T22:05:21.022982+0000","last_fullsized":"2024-05-08T22:05:21.022982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:50.192400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.493290+0000","last_change":"2024-05-08T22:05:13.554346+0000","last_active":"2024-05-08T22:05:37.493290+0000","last_peered":"2024-05-08T22:05:37.493290+0000","last_clean":"2024-05-08T22:05:37.493290+0000","last_became_active":"2024-05-08T22:05:13.554080+0000","last_became_peered":"2024-05-08T22:05:13.554080+0000","last_unstale":"2024-05-08T22:05:37.493290+0000","last_undegraded":"2024-05-08T22:05:37.493290+0000","last_fullsized":"2024-05-08T22:05:37.493290+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:29:49.695682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.523372+0000","last_change":"2024-05-08T22:05:14.071422+0000","last_active":"2024-05-08T22:05:18.523372+0000","last_peered":"2024-05-08T22:05:18.523372+0000","last_clean":"2024-05-08T22:05:18.523372+0000","last_became_active":"2024-05-08T22:05:14.070396+0000","last_became_peered":"2024-05-08T22:05:14.070396+0000","last_unstale":"2024-05-08T22:05:18.523372+0000","last_undegraded":"2024-05-08T22:05:18.523372+0000","last_fullsized":"2024-05-08T22:05:18.523372+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:37.957983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.804960+0000","last_change":"2024-05-08T22:05:18.617149+0000","last_active":"2024-05-08T22:05:21.804960+0000","last_peered":"2024-05-08T22:05:21.804960+0000","last_clean":"2024-05-08T22:05:21.804960+0000","last_became_active":"2024-05-08T22:05:18.616699+0000","last_became_peered":"2024-05-08T22:05:18.616699+0000","last_unstale":"2024-05-08T22:05:21.804960+0000","last_undegraded":"2024-05-08T22:05:21.804960+0000","last_fullsized":"2024-05-08T22:05:21.804960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:51.167142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.492588+0000","last_change":"2024-05-08T22:05:13.551212+0000","last_active":"2024-05-08T22:05:37.492588+0000","last_peered":"2024-05-08T22:05:37.492588+0000","last_clean":"2024-05-08T22:05:37.492588+0000","last_became_active":"2024-05-08T22:05:13.550909+0000","last_became_peered":"2024-05-08T22:05:13.550909+0000","last_unstale":"2024-05-08T22:05:37.492588+0000","last_undegraded":"2024-05-08T22:05:37.492588+0000","last_fullsized":"2024-05-08T22:05:37.492588+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:48.657928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829703+0000","last_change":"2024-05-08T22:05:14.053976+0000","last_active":"2024-05-08T22:05:40.829703+0000","last_peered":"2024-05-08T22:05:40.829703+0000","last_clean":"2024-05-08T22:05:40.829703+0000","last_became_active":"2024-05-08T22:05:14.053664+0000","last_became_peered":"2024-05-08T22:05:14.053664+0000","last_unstale":"2024-05-08T22:05:40.829703+0000","last_undegraded":"2024-05-08T22:05:40.829703+0000","last_fullsized":"2024-05-08T22:05:40.829703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:25:25.642768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.018493+0000","last_change":"2024-05-08T22:05:18.536771+0000","last_active":"2024-05-08T22:05:19.018493+0000","last_peered":"2024-05-08T22:05:19.018493+0000","last_clean":"2024-05-08T22:05:19.018493+0000","last_became_active":"2024-05-08T22:05:18.536260+0000","last_became_peered":"2024-05-08T22:05:18.536260+0000","last_unstale":"2024-05-08T22:05:19.018493+0000","last_undegraded":"2024-05-08T22:05:19.018493+0000","last_fullsized":"2024-05-08T22:05:19.018493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:34.414381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.494220+0000","last_change":"2024-05-08T22:05:13.540679+0000","last_active":"2024-05-08T22:05:37.494220+0000","last_peered":"2024-05-08T22:05:37.494220+0000","last_clean":"2024-05-08T22:05:37.494220+0000","last_became_active":"2024-05-08T22:05:13.539428+0000","last_became_peered":"2024-05-08T22:05:13.539428+0000","last_unstale":"2024-05-08T22:05:37.494220+0000","last_undegraded":"2024-05-08T22:05:37.494220+0000","last_fullsized":"2024-05-08T22:05:37.494220+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:59:34.455411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518423+0000","last_change":"2024-05-08T22:05:14.061849+0000","last_active":"2024-05-08T22:05:18.518423+0000","last_peered":"2024-05-08T22:05:18.518423+0000","last_clean":"2024-05-08T22:05:18.518423+0000","last_became_active":"2024-05-08T22:05:14.061028+0000","last_became_peered":"2024-05-08T22:05:14.061028+0000","last_unstale":"2024-05-08T22:05:18.518423+0000","last_undegraded":"2024-05-08T22:05:18.518423+0000","last_fullsized":"2024-05-08T22:05:18.518423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:11:59.489376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819278+0000","last_change":"2024-05-08T22:05:18.517994+0000","last_active":"2024-05-08T22:05:40.819278+0000","last_peered":"2024-05-08T22:05:40.819278+0000","last_clean":"2024-05-08T22:05:40.819278+0000","last_became_active":"2024-05-08T22:05:18.517655+0000","last_became_peered":"2024-05-08T22:05:18.517655+0000","last_unstale":"2024-05-08T22:05:40.819278+0000","last_undegraded":"2024-05-08T22:05:40.819278+0000","last_fullsized":"2024-05-08T22:05:40.819278+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:50:08.956277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.039182+0000","last_change":"2024-05-08T22:05:13.552851+0000","last_active":"2024-05-08T22:05:19.039182+0000","last_peered":"2024-05-08T22:05:19.039182+0000","last_clean":"2024-05-08T22:05:19.039182+0000","last_became_active":"2024-05-08T22:05:13.552507+0000","last_became_peered":"2024-05-08T22:05:13.552507+0000","last_unstale":"2024-05-08T22:05:19.039182+0000","last_undegraded":"2024-05-08T22:05:19.039182+0000","last_fullsized":"2024-05-08T22:05:19.039182+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:29.248623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714486+0000","last_change":"2024-05-08T22:05:14.062618+0000","last_active":"2024-05-08T22:05:19.714486+0000","last_peered":"2024-05-08T22:05:19.714486+0000","last_clean":"2024-05-08T22:05:19.714486+0000","last_became_active":"2024-05-08T22:05:14.062308+0000","last_became_peered":"2024-05-08T22:05:14.062308+0000","last_unstale":"2024-05-08T22:05:19.714486+0000","last_undegraded":"2024-05-08T22:05:19.714486+0000","last_fullsized":"2024-05-08T22:05:19.714486+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:34.195343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.804949+0000","last_change":"2024-05-08T22:05:18.616846+0000","last_active":"2024-05-08T22:05:21.804949+0000","last_peered":"2024-05-08T22:05:21.804949+0000","last_clean":"2024-05-08T22:05:21.804949+0000","last_became_active":"2024-05-08T22:05:18.616369+0000","last_became_peered":"2024-05-08T22:05:18.616369+0000","last_unstale":"2024-05-08T22:05:21.804949+0000","last_undegraded":"2024-05-08T22:05:21.804949+0000","last_fullsized":"2024-05-08T22:05:21.804949+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:26.128949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829370+0000","last_change":"2024-05-08T22:05:13.549712+0000","last_active":"2024-05-08T22:05:40.829370+0000","last_peered":"2024-05-08T22:05:40.829370+0000","last_clean":"2024-05-08T22:05:40.829370+0000","last_became_active":"2024-05-08T22:05:13.549508+0000","last_became_peered":"2024-05-08T22:05:13.549508+0000","last_unstale":"2024-05-08T22:05:40.829370+0000","last_undegraded":"2024-05-08T22:05:40.829370+0000","last_fullsized":"2024-05-08T22:05:40.829370+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:30:22.706389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.516692+0000","last_change":"2024-05-08T22:05:14.060920+0000","last_active":"2024-05-08T22:05:18.516692+0000","last_peered":"2024-05-08T22:05:18.516692+0000","last_clean":"2024-05-08T22:05:18.516692+0000","last_became_active":"2024-05-08T22:05:14.059914+0000","last_became_peered":"2024-05-08T22:05:14.059914+0000","last_unstale":"2024-05-08T22:05:18.516692+0000","last_undegraded":"2024-05-08T22:05:18.516692+0000","last_fullsized":"2024-05-08T22:05:18.516692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:15.513600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.534473+0000","last_change":"2024-05-08T22:05:18.534076+0000","last_active":"2024-05-08T22:05:18.534473+0000","last_peered":"2024-05-08T22:05:18.534473+0000","last_clean":"2024-05-08T22:05:18.534473+0000","last_became_active":"2024-05-08T22:05:18.533679+0000","last_became_peered":"2024-05-08T22:05:18.533679+0000","last_unstale":"2024-05-08T22:05:18.534473+0000","last_undegraded":"2024-05-08T22:05:18.534473+0000","last_fullsized":"2024-05-08T22:05:18.534473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:08.979193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.521141+0000","last_change":"2024-05-08T22:05:13.559282+0000","last_active":"2024-05-08T22:05:18.521141+0000","last_peered":"2024-05-08T22:05:18.521141+0000","last_clean":"2024-05-08T22:05:18.521141+0000","last_became_active":"2024-05-08T22:05:13.558876+0000","last_became_peered":"2024-05-08T22:05:13.558876+0000","last_unstale":"2024-05-08T22:05:18.521141+0000","last_undegraded":"2024-05-08T22:05:18.521141+0000","last_fullsized":"2024-05-08T22:05:18.521141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:56.062332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.516801+0000","last_change":"2024-05-08T22:05:14.065868+0000","last_active":"2024-05-08T22:05:18.516801+0000","last_peered":"2024-05-08T22:05:18.516801+0000","last_clean":"2024-05-08T22:05:18.516801+0000","last_became_active":"2024-05-08T22:05:14.065545+0000","last_became_peered":"2024-05-08T22:05:14.065545+0000","last_unstale":"2024-05-08T22:05:18.516801+0000","last_undegraded":"2024-05-08T22:05:18.516801+0000","last_fullsized":"2024-05-08T22:05:18.516801+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:56:34.309095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.154428+0000","last_change":"2024-05-08T22:05:18.534261+0000","last_active":"2024-05-08T22:05:19.154428+0000","last_peered":"2024-05-08T22:05:19.154428+0000","last_clean":"2024-05-08T22:05:19.154428+0000","last_became_active":"2024-05-08T22:05:18.531814+0000","last_became_peered":"2024-05-08T22:05:18.531814+0000","last_unstale":"2024-05-08T22:05:19.154428+0000","last_undegraded":"2024-05-08T22:05:19.154428+0000","last_fullsized":"2024-05-08T22:05:19.154428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:38.344546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:20.446341+0000","last_change":"2024-05-08T22:05:13.549344+0000","last_active":"2024-05-08T22:05:20.446341+0000","last_peered":"2024-05-08T22:05:20.446341+0000","last_clean":"2024-05-08T22:05:20.446341+0000","last_became_active":"2024-05-08T22:05:13.548993+0000","last_became_peered":"2024-05-08T22:05:13.548993+0000","last_unstale":"2024-05-08T22:05:20.446341+0000","last_undegraded":"2024-05-08T22:05:20.446341+0000","last_fullsized":"2024-05-08T22:05:20.446341+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:50.096602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.515948+0000","last_change":"2024-05-08T22:05:14.065758+0000","last_active":"2024-05-08T22:05:18.515948+0000","last_peered":"2024-05-08T22:05:18.515948+0000","last_clean":"2024-05-08T22:05:18.515948+0000","last_became_active":"2024-05-08T22:05:14.065442+0000","last_became_peered":"2024-05-08T22:05:14.065442+0000","last_unstale":"2024-05-08T22:05:18.515948+0000","last_undegraded":"2024-05-08T22:05:18.515948+0000","last_fullsized":"2024-05-08T22:05:18.515948+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:43.047796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.834022+0000","last_change":"2024-05-08T22:05:18.529804+0000","last_active":"2024-05-08T22:05:19.834022+0000","last_peered":"2024-05-08T22:05:19.834022+0000","last_clean":"2024-05-08T22:05:19.834022+0000","last_became_active":"2024-05-08T22:05:18.529346+0000","last_became_peered":"2024-05-08T22:05:18.529346+0000","last_unstale":"2024-05-08T22:05:19.834022+0000","last_undegraded":"2024-05-08T22:05:19.834022+0000","last_fullsized":"2024-05-08T22:05:19.834022+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:13.565707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.154118+0000","last_change":"2024-05-08T22:05:13.551818+0000","last_active":"2024-05-08T22:05:19.154118+0000","last_peered":"2024-05-08T22:05:19.154118+0000","last_clean":"2024-05-08T22:05:19.154118+0000","last_became_active":"2024-05-08T22:05:13.551080+0000","last_became_peered":"2024-05-08T22:05:13.551080+0000","last_unstale":"2024-05-08T22:05:19.154118+0000","last_undegraded":"2024-05-08T22:05:19.154118+0000","last_fullsized":"2024-05-08T22:05:19.154118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:31.636482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.517095+0000","last_change":"2024-05-08T22:05:14.066572+0000","last_active":"2024-05-08T22:05:18.517095+0000","last_peered":"2024-05-08T22:05:18.517095+0000","last_clean":"2024-05-08T22:05:18.517095+0000","last_became_active":"2024-05-08T22:05:14.066272+0000","last_became_peered":"2024-05-08T22:05:14.066272+0000","last_unstale":"2024-05-08T22:05:18.517095+0000","last_undegraded":"2024-05-08T22:05:18.517095+0000","last_fullsized":"2024-05-08T22:05:18.517095+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:20:27.423464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518519+0000","last_change":"2024-05-08T22:05:14.061435+0000","last_active":"2024-05-08T22:05:18.518519+0000","last_peered":"2024-05-08T22:05:18.518519+0000","last_clean":"2024-05-08T22:05:18.518519+0000","last_became_active":"2024-05-08T22:05:14.061079+0000","last_became_peered":"2024-05-08T22:05:14.061079+0000","last_unstale":"2024-05-08T22:05:18.518519+0000","last_undegraded":"2024-05-08T22:05:18.518519+0000","last_fullsized":"2024-05-08T22:05:18.518519+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:42.684743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"peering","last_fresh":"2024-05-08T22:05:40.817668+0000","last_change":"2024-05-08T22:05:40.817845+0000","last_active":"2024-05-08T22:05:40.817668+0000","last_peered":"2024-05-08T22:05:19.039319+0000","last_clean":"2024-05-08T22:05:19.039319+0000","last_became_active":"2024-05-08T22:05:14.071184+0000","last_became_peered":"2024-05-08T22:05:14.071184+0000","last_unstale":"2024-05-08T22:05:40.817668+0000","last_undegraded":"2024-05-08T22:05:40.817668+0000","last_fullsized":"2024-05-08T22:05:40.817668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:12:55.547522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5,8],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.039153+0000","last_change":"2024-05-08T22:05:14.066697+0000","last_active":"2024-05-08T22:05:19.039153+0000","last_peered":"2024-05-08T22:05:19.039153+0000","last_clean":"2024-05-08T22:05:19.039153+0000","last_became_active":"2024-05-08T22:05:14.065952+0000","last_became_peered":"2024-05-08T22:05:14.065952+0000","last_unstale":"2024-05-08T22:05:19.039153+0000","last_undegraded":"2024-05-08T22:05:19.039153+0000","last_fullsized":"2024-05-08T22:05:19.039153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:03:19.006644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.521494+0000","last_change":"2024-05-08T22:05:14.072295+0000","last_active":"2024-05-08T22:05:18.521494+0000","last_peered":"2024-05-08T22:05:18.521494+0000","last_clean":"2024-05-08T22:05:18.521494+0000","last_became_active":"2024-05-08T22:05:14.065034+0000","last_became_peered":"2024-05-08T22:05:14.065034+0000","last_unstale":"2024-05-08T22:05:18.521494+0000","last_undegraded":"2024-05-08T22:05:18.521494+0000","last_fullsized":"2024-05-08T22:05:18.521494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:08:41.380814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.154280+0000","last_change":"2024-05-08T22:05:14.067263+0000","last_active":"2024-05-08T22:05:19.154280+0000","last_peered":"2024-05-08T22:05:19.154280+0000","last_clean":"2024-05-08T22:05:19.154280+0000","last_became_active":"2024-05-08T22:05:14.066957+0000","last_became_peered":"2024-05-08T22:05:14.066957+0000","last_unstale":"2024-05-08T22:05:19.154280+0000","last_undegraded":"2024-05-08T22:05:19.154280+0000","last_fullsized":"2024-05-08T22:05:19.154280+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:29:35.638489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518101+0000","last_change":"2024-05-08T22:05:14.060561+0000","last_active":"2024-05-08T22:05:18.518101+0000","last_peered":"2024-05-08T22:05:18.518101+0000","last_clean":"2024-05-08T22:05:18.518101+0000","last_became_active":"2024-05-08T22:05:14.060150+0000","last_became_peered":"2024-05-08T22:05:14.060150+0000","last_unstale":"2024-05-08T22:05:18.518101+0000","last_undegraded":"2024-05-08T22:05:18.518101+0000","last_fullsized":"2024-05-08T22:05:18.518101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:22.710776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.039533+0000","last_change":"2024-05-08T22:05:14.056394+0000","last_active":"2024-05-08T22:05:19.039533+0000","last_peered":"2024-05-08T22:05:19.039533+0000","last_clean":"2024-05-08T22:05:19.039533+0000","last_became_active":"2024-05-08T22:05:14.055977+0000","last_became_peered":"2024-05-08T22:05:14.055977+0000","last_unstale":"2024-05-08T22:05:19.039533+0000","last_undegraded":"2024-05-08T22:05:19.039533+0000","last_fullsized":"2024-05-08T22:05:19.039533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:56:48.249379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714720+0000","last_change":"2024-05-08T22:05:14.061934+0000","last_active":"2024-05-08T22:05:19.714720+0000","last_peered":"2024-05-08T22:05:19.714720+0000","last_clean":"2024-05-08T22:05:19.714720+0000","last_became_active":"2024-05-08T22:05:14.061635+0000","last_became_peered":"2024-05-08T22:05:14.061635+0000","last_unstale":"2024-05-08T22:05:19.714720+0000","last_undegraded":"2024-05-08T22:05:19.714720+0000","last_fullsized":"2024-05-08T22:05:19.714720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:02.423338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.524933+0000","last_change":"2024-05-08T22:05:14.069982+0000","last_active":"2024-05-08T22:05:18.524933+0000","last_peered":"2024-05-08T22:05:18.524933+0000","last_clean":"2024-05-08T22:05:18.524933+0000","last_became_active":"2024-05-08T22:05:14.068293+0000","last_became_peered":"2024-05-08T22:05:14.068293+0000","last_unstale":"2024-05-08T22:05:18.524933+0000","last_undegraded":"2024-05-08T22:05:18.524933+0000","last_fullsized":"2024-05-08T22:05:18.524933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:36.396983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714278+0000","last_change":"2024-05-08T22:05:14.058393+0000","last_active":"2024-05-08T22:05:19.714278+0000","last_peered":"2024-05-08T22:05:19.714278+0000","last_clean":"2024-05-08T22:05:19.714278+0000","last_became_active":"2024-05-08T22:05:14.058083+0000","last_became_peered":"2024-05-08T22:05:14.058083+0000","last_unstale":"2024-05-08T22:05:19.714278+0000","last_undegraded":"2024-05-08T22:05:19.714278+0000","last_fullsized":"2024-05-08T22:05:19.714278+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:10:47.074957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518092+0000","last_change":"2024-05-08T22:05:14.056989+0000","last_active":"2024-05-08T22:05:18.518092+0000","last_peered":"2024-05-08T22:05:18.518092+0000","last_clean":"2024-05-08T22:05:18.518092+0000","last_became_active":"2024-05-08T22:05:14.056669+0000","last_became_peered":"2024-05-08T22:05:14.056669+0000","last_unstale":"2024-05-08T22:05:18.518092+0000","last_undegraded":"2024-05-08T22:05:18.518092+0000","last_fullsized":"2024-05-08T22:05:18.518092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:56:51.173885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.153888+0000","last_change":"2024-05-08T22:05:14.058635+0000","last_active":"2024-05-08T22:05:19.153888+0000","last_peered":"2024-05-08T22:05:19.153888+0000","last_clean":"2024-05-08T22:05:19.153888+0000","last_became_active":"2024-05-08T22:05:14.058243+0000","last_became_peered":"2024-05-08T22:05:14.058243+0000","last_unstale":"2024-05-08T22:05:19.153888+0000","last_undegraded":"2024-05-08T22:05:19.153888+0000","last_fullsized":"2024-05-08T22:05:19.153888+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:18.142563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518557+0000","last_change":"2024-05-08T22:05:14.060735+0000","last_active":"2024-05-08T22:05:18.518557+0000","last_peered":"2024-05-08T22:05:18.518557+0000","last_clean":"2024-05-08T22:05:18.518557+0000","last_became_active":"2024-05-08T22:05:14.060391+0000","last_became_peered":"2024-05-08T22:05:14.060391+0000","last_unstale":"2024-05-08T22:05:18.518557+0000","last_undegraded":"2024-05-08T22:05:18.518557+0000","last_fullsized":"2024-05-08T22:05:18.518557+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:30.528705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283045,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6728,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":23,"num_read_kb":19,"num_write":118,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":41050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579920,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36088,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709064,"statfs":{"total":95995035648,"available":95958081536,"internally_reserved":0,"allocated":901120,"data_stored":447161,"data_compressed":11671,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36043205},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.36899999999999999}]},{"osd":1,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.46000000000000002}]},{"osd":2,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.47299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":4,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.504}]},{"osd":5,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.622}]},{"osd":6,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.68200000000000005}]},{"osd":7,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.60199999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":9,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.73499999999999999}]},{"osd":10,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.748}]}]},{"osd":10,"up_from":67,"seq":287762808850,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":966656,"data_stored":1040606,"data_compressed":47667,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":1,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.442}]},{"osd":2,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.54100000000000004}]},{"osd":3,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.502},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.64100000000000001}]},{"osd":4,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.68300000000000005}]},{"osd":5,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.65700000000000003}]},{"osd":6,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.70099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":8,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.747},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.78500000000000003}]},{"osd":9,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.80400000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]}]},{"osd":9,"up_from":62,"seq":266287972375,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":447397,"data_compressed":11671,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.38600000000000001}]},{"osd":1,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.63500000000000001}]},{"osd":2,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.77600000000000002}]},{"osd":3,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.46000000000000002}]},{"osd":4,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.73199999999999998}]},{"osd":5,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.66300000000000003}]},{"osd":6,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.71199999999999997}]},{"osd":7,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.80200000000000005}]},{"osd":8,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.498},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.67900000000000005}]},{"osd":10,"last update":"Wed May 8 22:05:22 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.746},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.76400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]}]},{"osd":8,"up_from":55,"seq":236223201308,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":469876,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":0.40799999999999997}]},{"osd":1,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.47199999999999998}]},{"osd":2,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.42499999999999999}]},{"osd":3,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":4.8920000000000003,"5min":4.8920000000000003,"15min":4.8920000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.67100000000000004}]},{"osd":4,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.627}]},{"osd":5,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.52500000000000002}]},{"osd":6,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.9210000000000003,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.77900000000000003}]},{"osd":7,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.748},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.64700000000000002}]},{"osd":9,"last update":"Wed May 8 22:05:01 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.80600000000000005}]},{"osd":10,"last update":"Wed May 8 22:05:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.79100000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]}]},{"osd":7,"up_from":48,"seq":206158430237,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":1038103,"data_compressed":47667,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.41199999999999998}]},{"osd":1,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.39500000000000002,"15min":0.38700000000000001},"min":{"1min":0.30599999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.443},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40600000000000003,"15min":0.40300000000000002},"min":{"1min":0.29199999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.36699999999999999}]},{"osd":2,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54100000000000004,"15min":0.53600000000000003},"min":{"1min":0.34899999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.38400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70299999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.68000000000000005}]},{"osd":3,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.41599999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79800000000000004,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.501,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80200000000000005,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.52900000000000003}]},{"osd":4,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.33800000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71999999999999997,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.754,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.51000000000000001}]},{"osd":5,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64400000000000002,"15min":0.64500000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89900000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.625,"15min":0.626},"min":{"1min":0.44500000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.77700000000000002,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.63200000000000001}]},{"osd":6,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85099999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69699999999999995,"15min":0.69899999999999995},"min":{"1min":0.48199999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89000000000000001,"5min":1.002,"15min":1.002},"last":0.46800000000000003}]},{"osd":8,"last update":"Wed May 8 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.68700000000000006,"15min":0.69399999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.78000000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.72099999999999997,"15min":0.72699999999999998},"min":{"1min":0.55300000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.85899999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.74099999999999999}]},{"osd":9,"last update":"Wed May 8 22:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.61699999999999999}]},{"osd":10,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.70799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]}]},{"osd":6,"up_from":43,"seq":184683593762,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":455906,"data_compressed":14627,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.46200000000000002,"15min":0.45100000000000001},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.66300000000000003,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.51000000000000001,"15min":0.51300000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.64400000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.45900000000000002}]},{"osd":1,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57299999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.61899999999999999,"15min":0.60599999999999998},"min":{"1min":0.47399999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.77800000000000002}]},{"osd":2,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.66200000000000003,"15min":0.64800000000000002},"min":{"1min":0.53200000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65200000000000002,"15min":0.64700000000000002},"min":{"1min":0.51400000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89300000000000002,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.65300000000000002}]},{"osd":3,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60999999999999999,"15min":0.60699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.141,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.44},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.495,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.71499999999999997}]},{"osd":4,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69699999999999995,"15min":0.69299999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0569999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89700000000000002,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.68700000000000006}]},{"osd":5,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.754,"15min":0.74399999999999999},"min":{"1min":0.57999999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.995,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.75800000000000001,"15min":0.752},"min":{"1min":0.56399999999999995,"5min":0.438,"15min":0.438},"max":{"1min":1.052,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.64200000000000002}]},{"osd":7,"last update":"Wed May 8 22:05:23 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.99299999999999999,"5min":1.028,"15min":1.028},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76300000000000001,"15min":0.76000000000000001},"min":{"1min":0.56999999999999995,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.98499999999999999}]},{"osd":8,"last update":"Wed May 8 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.83199999999999996,"15min":0.83999999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0289999999999999,"5min":1.093,"15min":1.093},"last":0.871},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.79300000000000004,"15min":0.79600000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0149999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.67100000000000004}]},{"osd":9,"last update":"Wed May 8 22:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.73599999999999999}]},{"osd":10,"last update":"Wed May 8 22:05:19 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.88200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":905216,"data_stored":450102,"data_compressed":11671,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.52700000000000002,"15min":0.49299999999999999},"min":{"1min":0.38900000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.89900000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54000000000000004,"15min":0.51400000000000001},"min":{"1min":0.371,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81200000000000006,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.58299999999999996}]},{"osd":2,"last update":"Wed May 8 22:04:46 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.51300000000000001,"15min":0.48499999999999999},"min":{"1min":0.378,"5min":0.316,"15min":0.316},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.50600000000000001,"15min":0.48399999999999999},"min":{"1min":0.315,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.621}]},{"osd":3,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.53300000000000003,"15min":0.505},"min":{"1min":0.40000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.53300000000000003,"15min":0.51200000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.56899999999999995}]},{"osd":4,"last update":"Wed May 8 22:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.39900000000000002,"15min":0.376},"min":{"1min":0.307,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.64000000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.42499999999999999,"15min":0.39300000000000002},"min":{"1min":0.32400000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.76700000000000002,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.497}]},{"osd":5,"last update":"Wed May 8 22:04:43 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.56399999999999995,"15min":0.54800000000000004},"min":{"1min":0.39400000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58499999999999996,"15min":0.57299999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.45400000000000001}]},{"osd":6,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.63,"15min":0.61599999999999999},"min":{"1min":0.55100000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.46500000000000002}]},{"osd":7,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.36499999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86299999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.438}]},{"osd":8,"last update":"Wed May 8 22:04:37 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.60499999999999998}]},{"osd":9,"last update":"Wed May 8 22:05:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.70999999999999996}]},{"osd":10,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.65900000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]}]},{"osd":0,"up_from":8,"seq":34359738425,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":905216,"data_stored":447000,"data_compressed":11671,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57899999999999996,"15min":0.54900000000000004},"min":{"1min":0.42499999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.94099999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.55400000000000005,"15min":0.52400000000000002},"min":{"1min":0.40100000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83199999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.55500000000000005}]},{"osd":2,"last update":"Wed May 8 22:04:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.56399999999999995,"15min":0.54000000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.56100000000000005,"15min":0.54100000000000004},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.59099999999999997}]},{"osd":3,"last update":"Wed May 8 22:04:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.436,"15min":0.41799999999999998},"min":{"1min":0.34799999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.48799999999999999,"15min":0.48199999999999998},"min":{"1min":0.35499999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.877,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.82899999999999996}]},{"osd":4,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.57099999999999995,"15min":0.54900000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69399999999999995,"15min":0.69599999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96999999999999997,"5min":2.984,"15min":2.984},"last":0.78200000000000003}]},{"osd":5,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.65500000000000003,"15min":0.63900000000000001},"min":{"1min":0.48399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67100000000000004,"15min":0.66200000000000003},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.86599999999999999}]},{"osd":6,"last update":"Wed May 8 22:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.68600000000000005,"15min":0.66900000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.97599999999999998}]},{"osd":7,"last update":"Wed May 8 22:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.92000000000000004}]},{"osd":8,"last update":"Wed May 8 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.96099999999999997}]},{"osd":9,"last update":"Wed May 8 22:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.014}]},{"osd":10,"last update":"Wed May 8 22:05:22 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":1},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.73899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36132,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709020,"statfs":{"total":95995035648,"available":95958036480,"internally_reserved":0,"allocated":946176,"data_stored":470429,"data_compressed":11671,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.41999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.51200000000000001,"15min":0.504},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76100000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.55700000000000005}]},{"osd":1,"last update":"Wed May 8 22:04:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52600000000000002,"15min":0.52500000000000002},"min":{"1min":0.34100000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.74199999999999999,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.35499999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.77500000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.66500000000000004}]},{"osd":3,"last update":"Wed May 8 22:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.372,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65900000000000003,"5min":0.872,"15min":0.872},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56599999999999995,"15min":0.55900000000000005},"min":{"1min":0.40100000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72199999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.70499999999999996}]},{"osd":4,"last update":"Wed May 8 22:04:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.35699999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73399999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55700000000000005,"15min":0.55200000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.746}]},{"osd":5,"last update":"Wed May 8 22:04:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.436,"15min":0.433},"min":{"1min":0.34200000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.59999999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44400000000000001,"15min":0.441},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.72399999999999998}]},{"osd":6,"last update":"Wed May 8 22:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.76400000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66600000000000004,"15min":0.66700000000000004},"min":{"1min":0.50800000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.81499999999999995,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.89500000000000002}]},{"osd":7,"last update":"Wed May 8 22:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.94499999999999995}]},{"osd":8,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.97299999999999998}]},{"osd":9,"last update":"Wed May 8 22:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.83199999999999996}]},{"osd":10,"last update":"Wed May 8 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.877}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]}]},{"osd":3,"up_from":24,"seq":103079215149,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":456373,"data_compressed":14627,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.41699999999999998,"15min":0.39200000000000002},"min":{"1min":0.33500000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.504},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.42399999999999999,"15min":0.40200000000000002},"min":{"1min":0.35699999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.71299999999999997}]},{"osd":1,"last update":"Wed May 8 22:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.59899999999999998,"15min":0.55700000000000005},"min":{"1min":0.48599999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.56399999999999995,"15min":0.53000000000000003},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.77200000000000002}]},{"osd":2,"last update":"Wed May 8 22:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.61499999999999999,"15min":0.58899999999999997},"min":{"1min":0.51800000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.63900000000000001,"15min":0.60899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.55500000000000005}]},{"osd":4,"last update":"Wed May 8 22:05:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.62,"15min":0.59399999999999997},"min":{"1min":0.42399999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.66300000000000003,"15min":0.62},"min":{"1min":0.61299999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.98599999999999999}]},{"osd":5,"last update":"Wed May 8 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.67800000000000005,"15min":0.64500000000000002},"min":{"1min":0.46500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.64400000000000002,"15min":0.60899999999999999},"min":{"1min":0.46500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.76300000000000001}]},{"osd":6,"last update":"Wed May 8 22:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.63100000000000001,"15min":0.61499999999999999},"min":{"1min":0.46400000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.64600000000000002,"15min":0.624},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.874}]},{"osd":7,"last update":"Wed May 8 22:05:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.65700000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":1.008}]},{"osd":8,"last update":"Wed May 8 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.53100000000000003}]},{"osd":9,"last update":"Wed May 8 22:05:01 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.84999999999999998}]},{"osd":10,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.88400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986219,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":460781,"data_compressed":14627,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.44700000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.51900000000000002,"15min":0.50900000000000001},"min":{"1min":0.45000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.629}]},{"osd":1,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.374,"15min":0.35099999999999998},"min":{"1min":0.27200000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.66800000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.377,"15min":0.35499999999999998},"min":{"1min":0.28199999999999997,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.51000000000000001}]},{"osd":2,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.52600000000000002,"15min":0.50700000000000001},"min":{"1min":0.35499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71899999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.80900000000000005}]},{"osd":3,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.58799999999999997,"15min":0.56399999999999995},"min":{"1min":0.5,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84799999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56699999999999995,"15min":0.55300000000000005},"min":{"1min":0.39600000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.80000000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.67300000000000004}]},{"osd":5,"last update":"Wed May 8 22:05:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.36399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76200000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.57199999999999995,"15min":0.55000000000000004},"min":{"1min":0.42799999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.82199999999999995}]},{"osd":6,"last update":"Wed May 8 22:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.64800000000000002,"15min":0.63600000000000001},"min":{"1min":0.371,"5min":0.316,"15min":0.316},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.748},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.41799999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80500000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.60499999999999998}]},{"osd":7,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76800000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57299999999999995}]},{"osd":8,"last update":"Wed May 8 22:05:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68400000000000005,"15min":0.68600000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70799999999999996,"15min":0.70899999999999996},"min":{"1min":0.54100000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.79300000000000004}]},{"osd":9,"last update":"Wed May 8 22:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.624}]},{"osd":10,"last update":"Wed May 8 22:05:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.83499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":5,"up_from":36,"seq":154618822695,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":974848,"data_stored":1038375,"data_compressed":47667,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.378,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72899999999999998,"5min":0.749,"15min":0.749},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.48999999999999999}]},{"osd":1,"last update":"Wed May 8 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.41299999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.71699999999999997,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.622},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.37,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.58199999999999996}]},{"osd":2,"last update":"Wed May 8 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.254,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.52400000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.376,"15min":0.379},"min":{"1min":0.26600000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.46500000000000002,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.36199999999999999}]},{"osd":3,"last update":"Wed May 8 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57299999999999995,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.438},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.51800000000000002}]},{"osd":4,"last update":"Wed May 8 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.53300000000000003},"min":{"1min":0.38400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.39400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.751,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.64900000000000002}]},{"osd":6,"last update":"Wed May 8 22:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58599999999999997,"15min":0.58399999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.60699999999999998}]},{"osd":7,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.57399999999999995,"15min":0.56499999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.58799999999999997,"15min":0.57699999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.63700000000000001}]},{"osd":8,"last update":"Wed May 8 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.69399999999999995}]},{"osd":9,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.70599999999999996}]},{"osd":10,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.67400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":24792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:05:43.368 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:05:43.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.592+0000 7f9390a00700 1 -- 172.21.15.131:0/1820966975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f938c101fc0 msgr2=0x7f938c102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:43.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.592+0000 7f9390a00700 1 --2- 172.21.15.131:0/1820966975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f938c101fc0 0x7f938c102440 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f93780098f0 tx=0x7f9378009c00 comp rx=0 tx=0).stop 2024-05-08T22:05:43.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.592+0000 7f9390a00700 1 -- 172.21.15.131:0/1820966975 shutdown_connections 2024-05-08T22:05:43.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.592+0000 7f9390a00700 1 --2- 172.21.15.131:0/1820966975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f938c102980 0x7f938c1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:43.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.592+0000 7f9390a00700 1 --2- 172.21.15.131:0/1820966975 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f938c100e70 0x7f938c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:43.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.592+0000 7f9390a00700 1 --2- 172.21.15.131:0/1820966975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f938c101fc0 0x7f938c102440 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:43.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.592+0000 7f9390a00700 1 -- 172.21.15.131:0/1820966975 >> 172.21.15.131:0/1820966975 conn(0x7f938c0fc3e0 msgr2=0x7f938c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:43.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.592+0000 7f9390a00700 1 -- 172.21.15.131:0/1820966975 shutdown_connections 2024-05-08T22:05:43.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.592+0000 7f9390a00700 1 -- 172.21.15.131:0/1820966975 wait complete. 2024-05-08T22:05:43.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.593+0000 7f9390a00700 1 Processor -- start 2024-05-08T22:05:43.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.593+0000 7f9390a00700 1 -- start start 2024-05-08T22:05:43.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.593+0000 7f9390a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f938c100e70 0x7f938c074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:43.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.593+0000 7f9390a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f938c101fc0 0x7f938c073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:43.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.593+0000 7f9390a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f938c102980 0x7f938c0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:43.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.593+0000 7f9390a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c074070 con 0x7f938c100e70 2024-05-08T22:05:43.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.593+0000 7f9390a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c0741e0 con 0x7f938c102980 2024-05-08T22:05:43.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.593+0000 7f9390a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c1a0e80 con 0x7f938c101fc0 2024-05-08T22:05:43.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.594+0000 7f9389e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f938c100e70 0x7f938c074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:43.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.594+0000 7f938a800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f938c102980 0x7f938c0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:43.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.594+0000 7f9389e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f938c100e70 0x7f938c074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42250/0 (socket says 172.21.15.131:42250) 2024-05-08T22:05:43.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.594+0000 7f9389e00700 1 -- 172.21.15.131:0/2643129945 learned_addr learned my addr 172.21.15.131:0/2643129945 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:43.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.594+0000 7f9389400700 1 --2- 172.21.15.131:0/2643129945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f938c101fc0 0x7f938c073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:43.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.594+0000 7f938a800700 1 -- 172.21.15.131:0/2643129945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f938c101fc0 msgr2=0x7f938c073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:43.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.594+0000 7f938a800700 1 --2- 172.21.15.131:0/2643129945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f938c101fc0 0x7f938c073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:43.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.594+0000 7f938a800700 1 -- 172.21.15.131:0/2643129945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f938c100e70 msgr2=0x7f938c074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:43.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.594+0000 7f938a800700 1 --2- 172.21.15.131:0/2643129945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f938c100e70 0x7f938c074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:43.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.594+0000 7f938a800700 1 -- 172.21.15.131:0/2643129945 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9384009d00 con 0x7f938c102980 2024-05-08T22:05:43.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.594+0000 7f9389e00700 1 --2- 172.21.15.131:0/2643129945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f938c100e70 0x7f938c074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:43.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.594+0000 7f938a800700 1 --2- 172.21.15.131:0/2643129945 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f938c102980 0x7f938c0738d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9384009cd0 tx=0x7f938400c620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:43.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.595+0000 7f9382a00700 1 -- 172.21.15.131:0/2643129945 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384017070 con 0x7f938c102980 2024-05-08T22:05:43.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.595+0000 7f9390a00700 1 -- 172.21.15.131:0/2643129945 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9378009520 con 0x7f938c102980 2024-05-08T22:05:43.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.595+0000 7f9382a00700 1 -- 172.21.15.131:0/2643129945 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f938400f040 con 0x7f938c102980 2024-05-08T22:05:43.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.595+0000 7f9382a00700 1 -- 172.21.15.131:0/2643129945 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938400e050 con 0x7f938c102980 2024-05-08T22:05:43.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.595+0000 7f9390a00700 1 -- 172.21.15.131:0/2643129945 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f938c1a1480 con 0x7f938c102980 2024-05-08T22:05:43.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.596+0000 7f9390a00700 1 -- 172.21.15.131:0/2643129945 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f938c1082e0 con 0x7f938c102980 2024-05-08T22:05:43.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.598+0000 7f9382a00700 1 -- 172.21.15.131:0/2643129945 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f938400ccb0 con 0x7f938c102980 2024-05-08T22:05:43.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.599+0000 7f9382a00700 1 --2- 172.21.15.131:0/2643129945 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f93700a11b0 0x7f93700a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:43.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.599+0000 7f9382a00700 1 -- 172.21.15.131:0/2643129945 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f93840cd130 con 0x7f938c102980 2024-05-08T22:05:43.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.599+0000 7f9389e00700 1 --2- 172.21.15.131:0/2643129945 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f93700a11b0 0x7f93700a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:43.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.600+0000 7f9389e00700 1 --2- 172.21.15.131:0/2643129945 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f93700a11b0 0x7f93700a3670 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f937c011590 tx=0x7f937c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:43.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.601+0000 7f9382a00700 1 -- 172.21.15.131:0/2643129945 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f93840963f0 con 0x7f938c102980 2024-05-08T22:05:43.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.804+0000 7f9390a00700 1 -- 172.21.15.131:0/2643129945 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f938c0613e0 con 0x7f93700a11b0 2024-05-08T22:05:43.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.816+0000 7f9382a00700 1 -- 172.21.15.131:0/2643129945 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+421790 (secure 0 0 0) 0x7f938c0613e0 con 0x7f93700a11b0 2024-05-08T22:05:43.823 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:43.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.823+0000 7f9390a00700 1 -- 172.21.15.131:0/2643129945 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f93700a11b0 msgr2=0x7f93700a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:43.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.823+0000 7f9390a00700 1 --2- 172.21.15.131:0/2643129945 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f93700a11b0 0x7f93700a3670 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f937c011590 tx=0x7f937c009260 comp rx=0 tx=0).stop 2024-05-08T22:05:43.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.823+0000 7f9390a00700 1 -- 172.21.15.131:0/2643129945 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f938c102980 msgr2=0x7f938c0738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:43.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.823+0000 7f9390a00700 1 --2- 172.21.15.131:0/2643129945 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f938c102980 0x7f938c0738d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9384009cd0 tx=0x7f938400c620 comp rx=0 tx=0).stop 2024-05-08T22:05:43.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.823+0000 7f9390a00700 1 -- 172.21.15.131:0/2643129945 shutdown_connections 2024-05-08T22:05:43.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.823+0000 7f9390a00700 1 --2- 172.21.15.131:0/2643129945 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f93700a11b0 0x7f93700a3670 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:43.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.823+0000 7f9390a00700 1 --2- 172.21.15.131:0/2643129945 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f938c102980 0x7f938c0738d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:43.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.823+0000 7f9390a00700 1 --2- 172.21.15.131:0/2643129945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f938c100e70 0x7f938c074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:43.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.823+0000 7f9390a00700 1 --2- 172.21.15.131:0/2643129945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f938c101fc0 0x7f938c073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:43.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.823+0000 7f9390a00700 1 -- 172.21.15.131:0/2643129945 >> 172.21.15.131:0/2643129945 conn(0x7f938c0fc3e0 msgr2=0x7f938c105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:43.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.823+0000 7f9390a00700 1 -- 172.21.15.131:0/2643129945 shutdown_connections 2024-05-08T22:05:43.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:43.823+0000 7f9390a00700 1 -- 172.21.15.131:0/2643129945 wait complete. 2024-05-08T22:05:43.827 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-08T22:05:43.843 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-05-08T22:05:42.717959+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597213,"num_objects":57,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":159,"num_read_kb":138,"num_write":300,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":242,"ondisk_log_size":242,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371992,"kb_used_data":10916,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124569832,"statfs":{"total":1151940427776,"available":1151559507968,"internally_reserved":0,"allocated":11177984,"data_stored":7222109,"data_compressed":262820,"data_compressed_allocated":352256,"data_compressed_original":2203648,"omap_allocated":19131,"internal_metadata":369603909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2392,"num_objects":33,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":23,"num_read_kb":19,"num_write":72,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002287"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518839+0000","last_change":"2024-05-08T22:05:14.061628+0000","last_active":"2024-05-08T22:05:18.518839+0000","last_peered":"2024-05-08T22:05:18.518839+0000","last_clean":"2024-05-08T22:05:18.518839+0000","last_became_active":"2024-05-08T22:05:14.060986+0000","last_became_peered":"2024-05-08T22:05:14.060986+0000","last_unstale":"2024-05-08T22:05:18.518839+0000","last_undegraded":"2024-05-08T22:05:18.518839+0000","last_fullsized":"2024-05-08T22:05:18.518839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:51.643118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.523337+0000","last_change":"2024-05-08T22:05:14.073300+0000","last_active":"2024-05-08T22:05:18.523337+0000","last_peered":"2024-05-08T22:05:18.523337+0000","last_clean":"2024-05-08T22:05:18.523337+0000","last_became_active":"2024-05-08T22:05:14.069058+0000","last_became_peered":"2024-05-08T22:05:14.069058+0000","last_unstale":"2024-05-08T22:05:18.523337+0000","last_undegraded":"2024-05-08T22:05:18.523337+0000","last_fullsized":"2024-05-08T22:05:18.523337+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:15.834956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.523429+0000","last_change":"2024-05-08T22:05:14.071676+0000","last_active":"2024-05-08T22:05:18.523429+0000","last_peered":"2024-05-08T22:05:18.523429+0000","last_clean":"2024-05-08T22:05:18.523429+0000","last_became_active":"2024-05-08T22:05:14.058345+0000","last_became_peered":"2024-05-08T22:05:14.058345+0000","last_unstale":"2024-05-08T22:05:18.523429+0000","last_undegraded":"2024-05-08T22:05:18.523429+0000","last_fullsized":"2024-05-08T22:05:18.523429+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:09.239058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.019162+0000","last_change":"2024-05-08T22:05:14.067170+0000","last_active":"2024-05-08T22:05:19.019162+0000","last_peered":"2024-05-08T22:05:19.019162+0000","last_clean":"2024-05-08T22:05:19.019162+0000","last_became_active":"2024-05-08T22:05:14.066850+0000","last_became_peered":"2024-05-08T22:05:14.066850+0000","last_unstale":"2024-05-08T22:05:19.019162+0000","last_undegraded":"2024-05-08T22:05:19.019162+0000","last_fullsized":"2024-05-08T22:05:19.019162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:28.439786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829281+0000","last_change":"2024-05-08T22:05:14.065177+0000","last_active":"2024-05-08T22:05:40.829281+0000","last_peered":"2024-05-08T22:05:40.829281+0000","last_clean":"2024-05-08T22:05:40.829281+0000","last_became_active":"2024-05-08T22:05:14.064833+0000","last_became_peered":"2024-05-08T22:05:14.064833+0000","last_unstale":"2024-05-08T22:05:40.829281+0000","last_undegraded":"2024-05-08T22:05:40.829281+0000","last_fullsized":"2024-05-08T22:05:40.829281+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:11.764154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.615458+0000","last_change":"2024-05-08T22:05:14.061610+0000","last_active":"2024-05-08T22:05:18.615458+0000","last_peered":"2024-05-08T22:05:18.615458+0000","last_clean":"2024-05-08T22:05:18.615458+0000","last_became_active":"2024-05-08T22:05:14.061277+0000","last_became_peered":"2024-05-08T22:05:14.061277+0000","last_unstale":"2024-05-08T22:05:18.615458+0000","last_undegraded":"2024-05-08T22:05:18.615458+0000","last_fullsized":"2024-05-08T22:05:18.615458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:19.154980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.516423+0000","last_change":"2024-05-08T22:05:14.063941+0000","last_active":"2024-05-08T22:05:18.516423+0000","last_peered":"2024-05-08T22:05:18.516423+0000","last_clean":"2024-05-08T22:05:18.516423+0000","last_became_active":"2024-05-08T22:05:14.063579+0000","last_became_peered":"2024-05-08T22:05:14.063579+0000","last_unstale":"2024-05-08T22:05:18.516423+0000","last_undegraded":"2024-05-08T22:05:18.516423+0000","last_fullsized":"2024-05-08T22:05:18.516423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:27.679067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714348+0000","last_change":"2024-05-08T22:05:14.060366+0000","last_active":"2024-05-08T22:05:19.714348+0000","last_peered":"2024-05-08T22:05:19.714348+0000","last_clean":"2024-05-08T22:05:19.714348+0000","last_became_active":"2024-05-08T22:05:14.059818+0000","last_became_peered":"2024-05-08T22:05:14.059818+0000","last_unstale":"2024-05-08T22:05:19.714348+0000","last_undegraded":"2024-05-08T22:05:19.714348+0000","last_fullsized":"2024-05-08T22:05:19.714348+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:53.118508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518750+0000","last_change":"2024-05-08T22:05:14.057185+0000","last_active":"2024-05-08T22:05:18.518750+0000","last_peered":"2024-05-08T22:05:18.518750+0000","last_clean":"2024-05-08T22:05:18.518750+0000","last_became_active":"2024-05-08T22:05:14.056889+0000","last_became_peered":"2024-05-08T22:05:14.056889+0000","last_unstale":"2024-05-08T22:05:18.518750+0000","last_undegraded":"2024-05-08T22:05:18.518750+0000","last_fullsized":"2024-05-08T22:05:18.518750+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:35.372705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.616206+0000","last_change":"2024-05-08T22:05:14.062282+0000","last_active":"2024-05-08T22:05:18.616206+0000","last_peered":"2024-05-08T22:05:18.616206+0000","last_clean":"2024-05-08T22:05:18.616206+0000","last_became_active":"2024-05-08T22:05:14.061574+0000","last_became_peered":"2024-05-08T22:05:14.061574+0000","last_unstale":"2024-05-08T22:05:18.616206+0000","last_undegraded":"2024-05-08T22:05:18.616206+0000","last_fullsized":"2024-05-08T22:05:18.616206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:16:32.873590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.516691+0000","last_change":"2024-05-08T22:05:14.061267+0000","last_active":"2024-05-08T22:05:18.516691+0000","last_peered":"2024-05-08T22:05:18.516691+0000","last_clean":"2024-05-08T22:05:18.516691+0000","last_became_active":"2024-05-08T22:05:14.060943+0000","last_became_peered":"2024-05-08T22:05:14.060943+0000","last_unstale":"2024-05-08T22:05:18.516691+0000","last_undegraded":"2024-05-08T22:05:18.516691+0000","last_fullsized":"2024-05-08T22:05:18.516691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:03.831138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.019220+0000","last_change":"2024-05-08T22:05:14.067136+0000","last_active":"2024-05-08T22:05:19.019220+0000","last_peered":"2024-05-08T22:05:19.019220+0000","last_clean":"2024-05-08T22:05:19.019220+0000","last_became_active":"2024-05-08T22:05:14.066810+0000","last_became_peered":"2024-05-08T22:05:14.066810+0000","last_unstale":"2024-05-08T22:05:19.019220+0000","last_undegraded":"2024-05-08T22:05:19.019220+0000","last_fullsized":"2024-05-08T22:05:19.019220+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:43.001024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.154187+0000","last_change":"2024-05-08T22:05:14.062119+0000","last_active":"2024-05-08T22:05:19.154187+0000","last_peered":"2024-05-08T22:05:19.154187+0000","last_clean":"2024-05-08T22:05:19.154187+0000","last_became_active":"2024-05-08T22:05:14.058977+0000","last_became_peered":"2024-05-08T22:05:14.058977+0000","last_unstale":"2024-05-08T22:05:19.154187+0000","last_undegraded":"2024-05-08T22:05:19.154187+0000","last_fullsized":"2024-05-08T22:05:19.154187+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:03.657985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.519879+0000","last_change":"2024-05-08T22:05:14.067363+0000","last_active":"2024-05-08T22:05:18.519879+0000","last_peered":"2024-05-08T22:05:18.519879+0000","last_clean":"2024-05-08T22:05:18.519879+0000","last_became_active":"2024-05-08T22:05:14.066119+0000","last_became_peered":"2024-05-08T22:05:14.066119+0000","last_unstale":"2024-05-08T22:05:18.519879+0000","last_undegraded":"2024-05-08T22:05:18.519879+0000","last_fullsized":"2024-05-08T22:05:18.519879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:43.426750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.517625+0000","last_change":"2024-05-08T22:05:14.073925+0000","last_active":"2024-05-08T22:05:18.517625+0000","last_peered":"2024-05-08T22:05:18.517625+0000","last_clean":"2024-05-08T22:05:18.517625+0000","last_became_active":"2024-05-08T22:05:14.072287+0000","last_became_peered":"2024-05-08T22:05:14.072287+0000","last_unstale":"2024-05-08T22:05:18.517625+0000","last_undegraded":"2024-05-08T22:05:18.517625+0000","last_fullsized":"2024-05-08T22:05:18.517625+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:13.537938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.517166+0000","last_change":"2024-05-08T22:05:14.057274+0000","last_active":"2024-05-08T22:05:18.517166+0000","last_peered":"2024-05-08T22:05:18.517166+0000","last_clean":"2024-05-08T22:05:18.517166+0000","last_became_active":"2024-05-08T22:05:14.056989+0000","last_became_peered":"2024-05-08T22:05:14.056989+0000","last_unstale":"2024-05-08T22:05:18.517166+0000","last_undegraded":"2024-05-08T22:05:18.517166+0000","last_fullsized":"2024-05-08T22:05:18.517166+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:32.316406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518749+0000","last_change":"2024-05-08T22:05:14.069263+0000","last_active":"2024-05-08T22:05:18.518749+0000","last_peered":"2024-05-08T22:05:18.518749+0000","last_clean":"2024-05-08T22:05:18.518749+0000","last_became_active":"2024-05-08T22:05:14.068960+0000","last_became_peered":"2024-05-08T22:05:14.068960+0000","last_unstale":"2024-05-08T22:05:18.518749+0000","last_undegraded":"2024-05-08T22:05:18.518749+0000","last_fullsized":"2024-05-08T22:05:18.518749+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:59:42.829110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.514007+0000","last_change":"2024-05-08T22:05:14.049393+0000","last_active":"2024-05-08T22:05:18.514007+0000","last_peered":"2024-05-08T22:05:18.514007+0000","last_clean":"2024-05-08T22:05:18.514007+0000","last_became_active":"2024-05-08T22:05:14.049094+0000","last_became_peered":"2024-05-08T22:05:14.049094+0000","last_unstale":"2024-05-08T22:05:18.514007+0000","last_undegraded":"2024-05-08T22:05:18.514007+0000","last_fullsized":"2024-05-08T22:05:18.514007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:42.106568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.039735+0000","last_change":"2024-05-08T22:05:14.055436+0000","last_active":"2024-05-08T22:05:19.039735+0000","last_peered":"2024-05-08T22:05:19.039735+0000","last_clean":"2024-05-08T22:05:19.039735+0000","last_became_active":"2024-05-08T22:05:14.055100+0000","last_became_peered":"2024-05-08T22:05:14.055100+0000","last_unstale":"2024-05-08T22:05:19.039735+0000","last_undegraded":"2024-05-08T22:05:19.039735+0000","last_fullsized":"2024-05-08T22:05:19.039735+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:52.833780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.022998+0000","last_change":"2024-05-08T22:05:18.532086+0000","last_active":"2024-05-08T22:05:21.022998+0000","last_peered":"2024-05-08T22:05:21.022998+0000","last_clean":"2024-05-08T22:05:21.022998+0000","last_became_active":"2024-05-08T22:05:18.531485+0000","last_became_peered":"2024-05-08T22:05:18.531485+0000","last_unstale":"2024-05-08T22:05:21.022998+0000","last_undegraded":"2024-05-08T22:05:21.022998+0000","last_fullsized":"2024-05-08T22:05:21.022998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:40:36.050367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:38.499417+0000","last_change":"2024-05-08T22:05:13.549611+0000","last_active":"2024-05-08T22:05:38.499417+0000","last_peered":"2024-05-08T22:05:38.499417+0000","last_clean":"2024-05-08T22:05:38.499417+0000","last_became_active":"2024-05-08T22:05:13.549140+0000","last_became_peered":"2024-05-08T22:05:13.549140+0000","last_unstale":"2024-05-08T22:05:38.499417+0000","last_undegraded":"2024-05-08T22:05:38.499417+0000","last_fullsized":"2024-05-08T22:05:38.499417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:54.785123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714719+0000","last_change":"2024-05-08T22:05:14.057570+0000","last_active":"2024-05-08T22:05:19.714719+0000","last_peered":"2024-05-08T22:05:19.714719+0000","last_clean":"2024-05-08T22:05:19.714719+0000","last_became_active":"2024-05-08T22:05:14.057291+0000","last_became_peered":"2024-05-08T22:05:14.057291+0000","last_unstale":"2024-05-08T22:05:19.714719+0000","last_undegraded":"2024-05-08T22:05:19.714719+0000","last_fullsized":"2024-05-08T22:05:19.714719+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:15:38.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"peering","last_fresh":"2024-05-08T22:05:40.819123+0000","last_change":"2024-05-08T22:05:40.819370+0000","last_active":"2024-05-08T22:05:40.819123+0000","last_peered":"2024-05-08T22:05:21.804836+0000","last_clean":"2024-05-08T22:05:21.804836+0000","last_became_active":"2024-05-08T22:05:18.529966+0000","last_became_peered":"2024-05-08T22:05:18.529966+0000","last_unstale":"2024-05-08T22:05:40.819123+0000","last_undegraded":"2024-05-08T22:05:40.819123+0000","last_fullsized":"2024-05-08T22:05:40.819123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:28:51.858677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6,8,9],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:38.500116+0000","last_change":"2024-05-08T22:05:13.551351+0000","last_active":"2024-05-08T22:05:38.500116+0000","last_peered":"2024-05-08T22:05:38.500116+0000","last_clean":"2024-05-08T22:05:38.500116+0000","last_became_active":"2024-05-08T22:05:13.551014+0000","last_became_peered":"2024-05-08T22:05:13.551014+0000","last_unstale":"2024-05-08T22:05:38.500116+0000","last_undegraded":"2024-05-08T22:05:38.500116+0000","last_fullsized":"2024-05-08T22:05:38.500116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:07:22.405251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.525219+0000","last_change":"2024-05-08T22:05:14.067270+0000","last_active":"2024-05-08T22:05:18.525219+0000","last_peered":"2024-05-08T22:05:18.525219+0000","last_clean":"2024-05-08T22:05:18.525219+0000","last_became_active":"2024-05-08T22:05:14.065496+0000","last_became_peered":"2024-05-08T22:05:14.065496+0000","last_unstale":"2024-05-08T22:05:18.525219+0000","last_undegraded":"2024-05-08T22:05:18.525219+0000","last_fullsized":"2024-05-08T22:05:18.525219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:07.087880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.804877+0000","last_change":"2024-05-08T22:05:18.526557+0000","last_active":"2024-05-08T22:05:21.804877+0000","last_peered":"2024-05-08T22:05:21.804877+0000","last_clean":"2024-05-08T22:05:21.804877+0000","last_became_active":"2024-05-08T22:05:18.526129+0000","last_became_peered":"2024-05-08T22:05:18.526129+0000","last_unstale":"2024-05-08T22:05:21.804877+0000","last_undegraded":"2024-05-08T22:05:21.804877+0000","last_fullsized":"2024-05-08T22:05:21.804877+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:36.376579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"78'2","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.510733+0000","last_change":"2024-05-08T22:05:13.553630+0000","last_active":"2024-05-08T22:05:18.510733+0000","last_peered":"2024-05-08T22:05:18.510733+0000","last_clean":"2024-05-08T22:05:18.510733+0000","last_became_active":"2024-05-08T22:05:13.552879+0000","last_became_peered":"2024-05-08T22:05:13.552879+0000","last_unstale":"2024-05-08T22:05:18.510733+0000","last_undegraded":"2024-05-08T22:05:18.510733+0000","last_fullsized":"2024-05-08T22:05:18.510733+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:39:58.253305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.517225+0000","last_change":"2024-05-08T22:05:14.060089+0000","last_active":"2024-05-08T22:05:18.517225+0000","last_peered":"2024-05-08T22:05:18.517225+0000","last_clean":"2024-05-08T22:05:18.517225+0000","last_became_active":"2024-05-08T22:05:14.057620+0000","last_became_peered":"2024-05-08T22:05:14.057620+0000","last_unstale":"2024-05-08T22:05:18.517225+0000","last_undegraded":"2024-05-08T22:05:18.517225+0000","last_fullsized":"2024-05-08T22:05:18.517225+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:29:33.127510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.804864+0000","last_change":"2024-05-08T22:05:18.529746+0000","last_active":"2024-05-08T22:05:21.804864+0000","last_peered":"2024-05-08T22:05:21.804864+0000","last_clean":"2024-05-08T22:05:21.804864+0000","last_became_active":"2024-05-08T22:05:18.529097+0000","last_became_peered":"2024-05-08T22:05:18.529097+0000","last_unstale":"2024-05-08T22:05:21.804864+0000","last_undegraded":"2024-05-08T22:05:21.804864+0000","last_fullsized":"2024-05-08T22:05:21.804864+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:59:36.249787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.510546+0000","last_change":"2024-05-08T22:05:13.547070+0000","last_active":"2024-05-08T22:05:18.510546+0000","last_peered":"2024-05-08T22:05:18.510546+0000","last_clean":"2024-05-08T22:05:18.510546+0000","last_became_active":"2024-05-08T22:05:13.546708+0000","last_became_peered":"2024-05-08T22:05:13.546708+0000","last_unstale":"2024-05-08T22:05:18.510546+0000","last_undegraded":"2024-05-08T22:05:18.510546+0000","last_fullsized":"2024-05-08T22:05:18.510546+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:10.577900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.526371+0000","last_change":"2024-05-08T22:05:14.064695+0000","last_active":"2024-05-08T22:05:18.526371+0000","last_peered":"2024-05-08T22:05:18.526371+0000","last_clean":"2024-05-08T22:05:18.526371+0000","last_became_active":"2024-05-08T22:05:14.064283+0000","last_became_peered":"2024-05-08T22:05:14.064283+0000","last_unstale":"2024-05-08T22:05:18.526371+0000","last_undegraded":"2024-05-08T22:05:18.526371+0000","last_fullsized":"2024-05-08T22:05:18.526371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:06.980594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.023095+0000","last_change":"2024-05-08T22:05:18.528475+0000","last_active":"2024-05-08T22:05:21.023095+0000","last_peered":"2024-05-08T22:05:21.023095+0000","last_clean":"2024-05-08T22:05:21.023095+0000","last_became_active":"2024-05-08T22:05:18.528089+0000","last_became_peered":"2024-05-08T22:05:18.528089+0000","last_unstale":"2024-05-08T22:05:21.023095+0000","last_undegraded":"2024-05-08T22:05:21.023095+0000","last_fullsized":"2024-05-08T22:05:21.023095+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:01.508984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:39.502402+0000","last_change":"2024-05-08T22:05:13.549191+0000","last_active":"2024-05-08T22:05:39.502402+0000","last_peered":"2024-05-08T22:05:39.502402+0000","last_clean":"2024-05-08T22:05:39.502402+0000","last_became_active":"2024-05-08T22:05:13.548737+0000","last_became_peered":"2024-05-08T22:05:13.548737+0000","last_unstale":"2024-05-08T22:05:39.502402+0000","last_undegraded":"2024-05-08T22:05:39.502402+0000","last_fullsized":"2024-05-08T22:05:39.502402+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:07.355656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830198+0000","last_change":"2024-05-08T22:05:14.065910+0000","last_active":"2024-05-08T22:05:40.830198+0000","last_peered":"2024-05-08T22:05:40.830198+0000","last_clean":"2024-05-08T22:05:40.830198+0000","last_became_active":"2024-05-08T22:05:14.065504+0000","last_became_peered":"2024-05-08T22:05:14.065504+0000","last_unstale":"2024-05-08T22:05:40.830198+0000","last_undegraded":"2024-05-08T22:05:40.830198+0000","last_fullsized":"2024-05-08T22:05:40.830198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:12.447292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.023073+0000","last_change":"2024-05-08T22:05:18.536861+0000","last_active":"2024-05-08T22:05:21.023073+0000","last_peered":"2024-05-08T22:05:21.023073+0000","last_clean":"2024-05-08T22:05:21.023073+0000","last_became_active":"2024-05-08T22:05:18.536472+0000","last_became_peered":"2024-05-08T22:05:18.536472+0000","last_unstale":"2024-05-08T22:05:21.023073+0000","last_undegraded":"2024-05-08T22:05:21.023073+0000","last_fullsized":"2024-05-08T22:05:21.023073+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:45:21.826812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"78'1","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.615381+0000","last_change":"2024-05-08T22:05:13.553326+0000","last_active":"2024-05-08T22:05:18.615381+0000","last_peered":"2024-05-08T22:05:18.615381+0000","last_clean":"2024-05-08T22:05:18.615381+0000","last_became_active":"2024-05-08T22:05:13.552983+0000","last_became_peered":"2024-05-08T22:05:13.552983+0000","last_unstale":"2024-05-08T22:05:18.615381+0000","last_undegraded":"2024-05-08T22:05:18.615381+0000","last_fullsized":"2024-05-08T22:05:18.615381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:17.195253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.526142+0000","last_change":"2024-05-08T22:05:14.068617+0000","last_active":"2024-05-08T22:05:18.526142+0000","last_peered":"2024-05-08T22:05:18.526142+0000","last_clean":"2024-05-08T22:05:18.526142+0000","last_became_active":"2024-05-08T22:05:14.067400+0000","last_became_peered":"2024-05-08T22:05:14.067400+0000","last_unstale":"2024-05-08T22:05:18.526142+0000","last_undegraded":"2024-05-08T22:05:18.526142+0000","last_fullsized":"2024-05-08T22:05:18.526142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:25.488212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.018557+0000","last_change":"2024-05-08T22:05:18.538294+0000","last_active":"2024-05-08T22:05:19.018557+0000","last_peered":"2024-05-08T22:05:19.018557+0000","last_clean":"2024-05-08T22:05:19.018557+0000","last_became_active":"2024-05-08T22:05:18.537984+0000","last_became_peered":"2024-05-08T22:05:18.537984+0000","last_unstale":"2024-05-08T22:05:19.018557+0000","last_undegraded":"2024-05-08T22:05:19.018557+0000","last_fullsized":"2024-05-08T22:05:19.018557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:32.117439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:38.500252+0000","last_change":"2024-05-08T22:05:13.557669+0000","last_active":"2024-05-08T22:05:38.500252+0000","last_peered":"2024-05-08T22:05:38.500252+0000","last_clean":"2024-05-08T22:05:38.500252+0000","last_became_active":"2024-05-08T22:05:13.555835+0000","last_became_peered":"2024-05-08T22:05:13.555835+0000","last_unstale":"2024-05-08T22:05:38.500252+0000","last_undegraded":"2024-05-08T22:05:38.500252+0000","last_fullsized":"2024-05-08T22:05:38.500252+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:02.377635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714321+0000","last_change":"2024-05-08T22:05:14.060144+0000","last_active":"2024-05-08T22:05:19.714321+0000","last_peered":"2024-05-08T22:05:19.714321+0000","last_clean":"2024-05-08T22:05:19.714321+0000","last_became_active":"2024-05-08T22:05:14.059387+0000","last_became_peered":"2024-05-08T22:05:14.059387+0000","last_unstale":"2024-05-08T22:05:19.714321+0000","last_undegraded":"2024-05-08T22:05:19.714321+0000","last_fullsized":"2024-05-08T22:05:19.714321+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:32:28.057831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.154473+0000","last_change":"2024-05-08T22:05:18.527293+0000","last_active":"2024-05-08T22:05:19.154473+0000","last_peered":"2024-05-08T22:05:19.154473+0000","last_clean":"2024-05-08T22:05:19.154473+0000","last_became_active":"2024-05-08T22:05:18.526690+0000","last_became_peered":"2024-05-08T22:05:18.526690+0000","last_unstale":"2024-05-08T22:05:19.154473+0000","last_undegraded":"2024-05-08T22:05:19.154473+0000","last_fullsized":"2024-05-08T22:05:19.154473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:50.343536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:20.445763+0000","last_change":"2024-05-08T22:05:13.549774+0000","last_active":"2024-05-08T22:05:20.445763+0000","last_peered":"2024-05-08T22:05:20.445763+0000","last_clean":"2024-05-08T22:05:20.445763+0000","last_became_active":"2024-05-08T22:05:13.549531+0000","last_became_peered":"2024-05-08T22:05:13.549531+0000","last_unstale":"2024-05-08T22:05:20.445763+0000","last_undegraded":"2024-05-08T22:05:20.445763+0000","last_fullsized":"2024-05-08T22:05:20.445763+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:59:23.798728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.513387+0000","last_change":"2024-05-08T22:05:14.054232+0000","last_active":"2024-05-08T22:05:18.513387+0000","last_peered":"2024-05-08T22:05:18.513387+0000","last_clean":"2024-05-08T22:05:18.513387+0000","last_became_active":"2024-05-08T22:05:14.053806+0000","last_became_peered":"2024-05-08T22:05:14.053806+0000","last_unstale":"2024-05-08T22:05:18.513387+0000","last_undegraded":"2024-05-08T22:05:18.513387+0000","last_fullsized":"2024-05-08T22:05:18.513387+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:32:21.678792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.504963+0000","last_change":"2024-05-08T22:05:18.535241+0000","last_active":"2024-05-08T22:05:21.504963+0000","last_peered":"2024-05-08T22:05:21.504963+0000","last_clean":"2024-05-08T22:05:21.504963+0000","last_became_active":"2024-05-08T22:05:18.534753+0000","last_became_peered":"2024-05-08T22:05:18.534753+0000","last_unstale":"2024-05-08T22:05:21.504963+0000","last_undegraded":"2024-05-08T22:05:21.504963+0000","last_fullsized":"2024-05-08T22:05:21.504963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:46:15.138437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.493918+0000","last_change":"2024-05-08T22:05:13.544899+0000","last_active":"2024-05-08T22:05:37.493918+0000","last_peered":"2024-05-08T22:05:37.493918+0000","last_clean":"2024-05-08T22:05:37.493918+0000","last_became_active":"2024-05-08T22:05:13.544498+0000","last_became_peered":"2024-05-08T22:05:13.544498+0000","last_unstale":"2024-05-08T22:05:37.493918+0000","last_undegraded":"2024-05-08T22:05:37.493918+0000","last_fullsized":"2024-05-08T22:05:37.493918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:58.169648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518687+0000","last_change":"2024-05-08T22:05:14.049377+0000","last_active":"2024-05-08T22:05:18.518687+0000","last_peered":"2024-05-08T22:05:18.518687+0000","last_clean":"2024-05-08T22:05:18.518687+0000","last_became_active":"2024-05-08T22:05:14.049039+0000","last_became_peered":"2024-05-08T22:05:14.049039+0000","last_unstale":"2024-05-08T22:05:18.518687+0000","last_undegraded":"2024-05-08T22:05:18.518687+0000","last_fullsized":"2024-05-08T22:05:18.518687+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:24.415392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.017866+0000","last_change":"2024-05-08T22:05:18.528912+0000","last_active":"2024-05-08T22:05:19.017866+0000","last_peered":"2024-05-08T22:05:19.017866+0000","last_clean":"2024-05-08T22:05:19.017866+0000","last_became_active":"2024-05-08T22:05:18.528433+0000","last_became_peered":"2024-05-08T22:05:18.528433+0000","last_unstale":"2024-05-08T22:05:19.017866+0000","last_undegraded":"2024-05-08T22:05:19.017866+0000","last_fullsized":"2024-05-08T22:05:19.017866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:05.574728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.492321+0000","last_change":"2024-05-08T22:05:13.550214+0000","last_active":"2024-05-08T22:05:37.492321+0000","last_peered":"2024-05-08T22:05:37.492321+0000","last_clean":"2024-05-08T22:05:37.492321+0000","last_became_active":"2024-05-08T22:05:13.549717+0000","last_became_peered":"2024-05-08T22:05:13.549717+0000","last_unstale":"2024-05-08T22:05:37.492321+0000","last_undegraded":"2024-05-08T22:05:37.492321+0000","last_fullsized":"2024-05-08T22:05:37.492321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:12:20.406333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.525310+0000","last_change":"2024-05-08T22:05:14.071214+0000","last_active":"2024-05-08T22:05:18.525310+0000","last_peered":"2024-05-08T22:05:18.525310+0000","last_clean":"2024-05-08T22:05:18.525310+0000","last_became_active":"2024-05-08T22:05:14.066777+0000","last_became_peered":"2024-05-08T22:05:14.066777+0000","last_unstale":"2024-05-08T22:05:18.525310+0000","last_undegraded":"2024-05-08T22:05:18.525310+0000","last_fullsized":"2024-05-08T22:05:18.525310+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:53.360911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.804896+0000","last_change":"2024-05-08T22:05:18.523688+0000","last_active":"2024-05-08T22:05:21.804896+0000","last_peered":"2024-05-08T22:05:21.804896+0000","last_clean":"2024-05-08T22:05:21.804896+0000","last_became_active":"2024-05-08T22:05:18.523236+0000","last_became_peered":"2024-05-08T22:05:18.523236+0000","last_unstale":"2024-05-08T22:05:21.804896+0000","last_undegraded":"2024-05-08T22:05:21.804896+0000","last_fullsized":"2024-05-08T22:05:21.804896+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:34.531699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.494031+0000","last_change":"2024-05-08T22:05:13.551027+0000","last_active":"2024-05-08T22:05:37.494031+0000","last_peered":"2024-05-08T22:05:37.494031+0000","last_clean":"2024-05-08T22:05:37.494031+0000","last_became_active":"2024-05-08T22:05:13.550471+0000","last_became_peered":"2024-05-08T22:05:13.550471+0000","last_unstale":"2024-05-08T22:05:37.494031+0000","last_undegraded":"2024-05-08T22:05:37.494031+0000","last_fullsized":"2024-05-08T22:05:37.494031+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:50.900419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.517424+0000","last_change":"2024-05-08T22:05:14.065191+0000","last_active":"2024-05-08T22:05:18.517424+0000","last_peered":"2024-05-08T22:05:18.517424+0000","last_clean":"2024-05-08T22:05:18.517424+0000","last_became_active":"2024-05-08T22:05:14.064893+0000","last_became_peered":"2024-05-08T22:05:14.064893+0000","last_unstale":"2024-05-08T22:05:18.517424+0000","last_undegraded":"2024-05-08T22:05:18.517424+0000","last_fullsized":"2024-05-08T22:05:18.517424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:11.344646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.017892+0000","last_change":"2024-05-08T22:05:18.530447+0000","last_active":"2024-05-08T22:05:19.017892+0000","last_peered":"2024-05-08T22:05:19.017892+0000","last_clean":"2024-05-08T22:05:19.017892+0000","last_became_active":"2024-05-08T22:05:18.528171+0000","last_became_peered":"2024-05-08T22:05:18.528171+0000","last_unstale":"2024-05-08T22:05:19.017892+0000","last_undegraded":"2024-05-08T22:05:19.017892+0000","last_fullsized":"2024-05-08T22:05:19.017892+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:09:43.347366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"83'12","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.493215+0000","last_change":"2024-05-08T22:05:13.547289+0000","last_active":"2024-05-08T22:05:37.493215+0000","last_peered":"2024-05-08T22:05:37.493215+0000","last_clean":"2024-05-08T22:05:37.493215+0000","last_became_active":"2024-05-08T22:05:13.546860+0000","last_became_peered":"2024-05-08T22:05:13.546860+0000","last_unstale":"2024-05-08T22:05:37.493215+0000","last_undegraded":"2024-05-08T22:05:37.493215+0000","last_fullsized":"2024-05-08T22:05:37.493215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:08:50.339361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831151+0000","last_change":"2024-05-08T22:05:14.057925+0000","last_active":"2024-05-08T22:05:40.831151+0000","last_peered":"2024-05-08T22:05:40.831151+0000","last_clean":"2024-05-08T22:05:40.831151+0000","last_became_active":"2024-05-08T22:05:14.057662+0000","last_became_peered":"2024-05-08T22:05:14.057662+0000","last_unstale":"2024-05-08T22:05:40.831151+0000","last_undegraded":"2024-05-08T22:05:40.831151+0000","last_fullsized":"2024-05-08T22:05:40.831151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:49.929610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.834134+0000","last_change":"2024-05-08T22:05:18.535509+0000","last_active":"2024-05-08T22:05:19.834134+0000","last_peered":"2024-05-08T22:05:19.834134+0000","last_clean":"2024-05-08T22:05:19.834134+0000","last_became_active":"2024-05-08T22:05:18.535024+0000","last_became_peered":"2024-05-08T22:05:18.535024+0000","last_unstale":"2024-05-08T22:05:19.834134+0000","last_undegraded":"2024-05-08T22:05:19.834134+0000","last_fullsized":"2024-05-08T22:05:19.834134+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:28.098737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:39.501569+0000","last_change":"2024-05-08T22:05:13.552096+0000","last_active":"2024-05-08T22:05:39.501569+0000","last_peered":"2024-05-08T22:05:39.501569+0000","last_clean":"2024-05-08T22:05:39.501569+0000","last_became_active":"2024-05-08T22:05:13.551655+0000","last_became_peered":"2024-05-08T22:05:13.551655+0000","last_unstale":"2024-05-08T22:05:39.501569+0000","last_undegraded":"2024-05-08T22:05:39.501569+0000","last_fullsized":"2024-05-08T22:05:39.501569+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:24.371008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.153911+0000","last_change":"2024-05-08T22:05:14.066075+0000","last_active":"2024-05-08T22:05:19.153911+0000","last_peered":"2024-05-08T22:05:19.153911+0000","last_clean":"2024-05-08T22:05:19.153911+0000","last_became_active":"2024-05-08T22:05:14.065581+0000","last_became_peered":"2024-05-08T22:05:14.065581+0000","last_unstale":"2024-05-08T22:05:19.153911+0000","last_undegraded":"2024-05-08T22:05:19.153911+0000","last_fullsized":"2024-05-08T22:05:19.153911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:15.403714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.018254+0000","last_change":"2024-05-08T22:05:18.534033+0000","last_active":"2024-05-08T22:05:19.018254+0000","last_peered":"2024-05-08T22:05:19.018254+0000","last_clean":"2024-05-08T22:05:19.018254+0000","last_became_active":"2024-05-08T22:05:18.533563+0000","last_became_peered":"2024-05-08T22:05:18.533563+0000","last_unstale":"2024-05-08T22:05:19.018254+0000","last_undegraded":"2024-05-08T22:05:19.018254+0000","last_fullsized":"2024-05-08T22:05:19.018254+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:37:52.657794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.493496+0000","last_change":"2024-05-08T22:05:13.549112+0000","last_active":"2024-05-08T22:05:37.493496+0000","last_peered":"2024-05-08T22:05:37.493496+0000","last_clean":"2024-05-08T22:05:37.493496+0000","last_became_active":"2024-05-08T22:05:13.548763+0000","last_became_peered":"2024-05-08T22:05:13.548763+0000","last_unstale":"2024-05-08T22:05:37.493496+0000","last_undegraded":"2024-05-08T22:05:37.493496+0000","last_fullsized":"2024-05-08T22:05:37.493496+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:26:56.448611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.516056+0000","last_change":"2024-05-08T22:05:14.067116+0000","last_active":"2024-05-08T22:05:18.516056+0000","last_peered":"2024-05-08T22:05:18.516056+0000","last_clean":"2024-05-08T22:05:18.516056+0000","last_became_active":"2024-05-08T22:05:14.064775+0000","last_became_peered":"2024-05-08T22:05:14.064775+0000","last_unstale":"2024-05-08T22:05:18.516056+0000","last_undegraded":"2024-05-08T22:05:18.516056+0000","last_fullsized":"2024-05-08T22:05:18.516056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:28:32.901405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.018404+0000","last_change":"2024-05-08T22:05:18.539546+0000","last_active":"2024-05-08T22:05:19.018404+0000","last_peered":"2024-05-08T22:05:19.018404+0000","last_clean":"2024-05-08T22:05:19.018404+0000","last_became_active":"2024-05-08T22:05:18.538998+0000","last_became_peered":"2024-05-08T22:05:18.538998+0000","last_unstale":"2024-05-08T22:05:19.018404+0000","last_undegraded":"2024-05-08T22:05:19.018404+0000","last_fullsized":"2024-05-08T22:05:19.018404+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:30.453881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"83'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819019+0000","last_change":"2024-05-08T22:05:13.549053+0000","last_active":"2024-05-08T22:05:40.819019+0000","last_peered":"2024-05-08T22:05:40.819019+0000","last_clean":"2024-05-08T22:05:40.819019+0000","last_became_active":"2024-05-08T22:05:13.548739+0000","last_became_peered":"2024-05-08T22:05:13.548739+0000","last_unstale":"2024-05-08T22:05:40.819019+0000","last_undegraded":"2024-05-08T22:05:40.819019+0000","last_fullsized":"2024-05-08T22:05:40.819019+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:07:10.298084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.516222+0000","last_change":"2024-05-08T22:05:14.061753+0000","last_active":"2024-05-08T22:05:18.516222+0000","last_peered":"2024-05-08T22:05:18.516222+0000","last_clean":"2024-05-08T22:05:18.516222+0000","last_became_active":"2024-05-08T22:05:14.061412+0000","last_became_peered":"2024-05-08T22:05:14.061412+0000","last_unstale":"2024-05-08T22:05:18.516222+0000","last_undegraded":"2024-05-08T22:05:18.516222+0000","last_fullsized":"2024-05-08T22:05:18.516222+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:54.380181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.718981+0000","last_change":"2024-05-08T22:05:18.532627+0000","last_active":"2024-05-08T22:05:21.718981+0000","last_peered":"2024-05-08T22:05:21.718981+0000","last_clean":"2024-05-08T22:05:21.718981+0000","last_became_active":"2024-05-08T22:05:18.532205+0000","last_became_peered":"2024-05-08T22:05:18.532205+0000","last_unstale":"2024-05-08T22:05:21.718981+0000","last_undegraded":"2024-05-08T22:05:21.718981+0000","last_fullsized":"2024-05-08T22:05:21.718981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:20:11.028645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":61,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.841832+0000","last_change":"2024-05-08T22:05:41.841832+0000","last_active":"2024-05-08T22:05:41.841832+0000","last_peered":"2024-05-08T22:05:41.841832+0000","last_clean":"2024-05-08T22:05:41.841832+0000","last_became_active":"2024-05-08T22:05:41.829584+0000","last_became_peered":"2024-05-08T22:05:41.829584+0000","last_unstale":"2024-05-08T22:05:41.841832+0000","last_undegraded":"2024-05-08T22:05:41.841832+0000","last_fullsized":"2024-05-08T22:05:41.841832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:52:19.876215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.525104+0000","last_change":"2024-05-08T22:05:14.063027+0000","last_active":"2024-05-08T22:05:18.525104+0000","last_peered":"2024-05-08T22:05:18.525104+0000","last_clean":"2024-05-08T22:05:18.525104+0000","last_became_active":"2024-05-08T22:05:14.062682+0000","last_became_peered":"2024-05-08T22:05:14.062682+0000","last_unstale":"2024-05-08T22:05:18.525104+0000","last_undegraded":"2024-05-08T22:05:18.525104+0000","last_fullsized":"2024-05-08T22:05:18.525104+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:47.248832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.834071+0000","last_change":"2024-05-08T22:05:18.528370+0000","last_active":"2024-05-08T22:05:19.834071+0000","last_peered":"2024-05-08T22:05:19.834071+0000","last_clean":"2024-05-08T22:05:19.834071+0000","last_became_active":"2024-05-08T22:05:18.527990+0000","last_became_peered":"2024-05-08T22:05:18.527990+0000","last_unstale":"2024-05-08T22:05:19.834071+0000","last_undegraded":"2024-05-08T22:05:19.834071+0000","last_fullsized":"2024-05-08T22:05:19.834071+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:07.524175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829860+0000","last_change":"2024-05-08T22:05:13.547543+0000","last_active":"2024-05-08T22:05:40.829860+0000","last_peered":"2024-05-08T22:05:40.829860+0000","last_clean":"2024-05-08T22:05:40.829860+0000","last_became_active":"2024-05-08T22:05:13.547191+0000","last_became_peered":"2024-05-08T22:05:13.547191+0000","last_unstale":"2024-05-08T22:05:40.829860+0000","last_undegraded":"2024-05-08T22:05:40.829860+0000","last_fullsized":"2024-05-08T22:05:40.829860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:53.213239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518149+0000","last_change":"2024-05-08T22:05:14.062677+0000","last_active":"2024-05-08T22:05:18.518149+0000","last_peered":"2024-05-08T22:05:18.518149+0000","last_clean":"2024-05-08T22:05:18.518149+0000","last_became_active":"2024-05-08T22:05:14.062306+0000","last_became_peered":"2024-05-08T22:05:14.062306+0000","last_unstale":"2024-05-08T22:05:18.518149+0000","last_undegraded":"2024-05-08T22:05:18.518149+0000","last_fullsized":"2024-05-08T22:05:18.518149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:00.356505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830129+0000","last_change":"2024-05-08T22:05:18.617075+0000","last_active":"2024-05-08T22:05:40.830129+0000","last_peered":"2024-05-08T22:05:40.830129+0000","last_clean":"2024-05-08T22:05:40.830129+0000","last_became_active":"2024-05-08T22:05:18.616653+0000","last_became_peered":"2024-05-08T22:05:18.616653+0000","last_unstale":"2024-05-08T22:05:40.830129+0000","last_undegraded":"2024-05-08T22:05:40.830129+0000","last_fullsized":"2024-05-08T22:05:40.830129+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:30.537064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.513484+0000","last_change":"2024-05-08T22:05:13.553457+0000","last_active":"2024-05-08T22:05:18.513484+0000","last_peered":"2024-05-08T22:05:18.513484+0000","last_clean":"2024-05-08T22:05:18.513484+0000","last_became_active":"2024-05-08T22:05:13.551583+0000","last_became_peered":"2024-05-08T22:05:13.551583+0000","last_unstale":"2024-05-08T22:05:18.513484+0000","last_undegraded":"2024-05-08T22:05:18.513484+0000","last_fullsized":"2024-05-08T22:05:18.513484+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:07:05.971413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.616101+0000","last_change":"2024-05-08T22:05:14.057476+0000","last_active":"2024-05-08T22:05:18.616101+0000","last_peered":"2024-05-08T22:05:18.616101+0000","last_clean":"2024-05-08T22:05:18.616101+0000","last_became_active":"2024-05-08T22:05:14.056550+0000","last_became_peered":"2024-05-08T22:05:14.056550+0000","last_unstale":"2024-05-08T22:05:18.616101+0000","last_undegraded":"2024-05-08T22:05:18.616101+0000","last_fullsized":"2024-05-08T22:05:18.616101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:41:29.264096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.718898+0000","last_change":"2024-05-08T22:05:18.530719+0000","last_active":"2024-05-08T22:05:21.718898+0000","last_peered":"2024-05-08T22:05:21.718898+0000","last_clean":"2024-05-08T22:05:21.718898+0000","last_became_active":"2024-05-08T22:05:18.529659+0000","last_became_peered":"2024-05-08T22:05:18.529659+0000","last_unstale":"2024-05-08T22:05:21.718898+0000","last_undegraded":"2024-05-08T22:05:21.718898+0000","last_fullsized":"2024-05-08T22:05:21.718898+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:13:06.514460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:20.446632+0000","last_change":"2024-05-08T22:05:13.556191+0000","last_active":"2024-05-08T22:05:20.446632+0000","last_peered":"2024-05-08T22:05:20.446632+0000","last_clean":"2024-05-08T22:05:20.446632+0000","last_became_active":"2024-05-08T22:05:13.555888+0000","last_became_peered":"2024-05-08T22:05:13.555888+0000","last_unstale":"2024-05-08T22:05:20.446632+0000","last_undegraded":"2024-05-08T22:05:20.446632+0000","last_fullsized":"2024-05-08T22:05:20.446632+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:05.637980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.615673+0000","last_change":"2024-05-08T22:05:14.069974+0000","last_active":"2024-05-08T22:05:18.615673+0000","last_peered":"2024-05-08T22:05:18.615673+0000","last_clean":"2024-05-08T22:05:18.615673+0000","last_became_active":"2024-05-08T22:05:14.069637+0000","last_became_peered":"2024-05-08T22:05:14.069637+0000","last_unstale":"2024-05-08T22:05:18.615673+0000","last_undegraded":"2024-05-08T22:05:18.615673+0000","last_fullsized":"2024-05-08T22:05:18.615673+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:15:03.030292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"peering","last_fresh":"2024-05-08T22:05:40.825443+0000","last_change":"2024-05-08T22:05:40.825855+0000","last_active":"2024-05-08T22:05:40.825443+0000","last_peered":"2024-05-08T22:05:21.719003+0000","last_clean":"2024-05-08T22:05:21.719003+0000","last_became_active":"2024-05-08T22:05:18.616536+0000","last_became_peered":"2024-05-08T22:05:18.616536+0000","last_unstale":"2024-05-08T22:05:40.825443+0000","last_undegraded":"2024-05-08T22:05:40.825443+0000","last_fullsized":"2024-05-08T22:05:40.825443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:30:44.106162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5,8,10],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.525034+0000","last_change":"2024-05-08T22:05:14.069096+0000","last_active":"2024-05-08T22:05:18.525034+0000","last_peered":"2024-05-08T22:05:18.525034+0000","last_clean":"2024-05-08T22:05:18.525034+0000","last_became_active":"2024-05-08T22:05:14.068796+0000","last_became_peered":"2024-05-08T22:05:14.068796+0000","last_unstale":"2024-05-08T22:05:18.525034+0000","last_undegraded":"2024-05-08T22:05:18.525034+0000","last_fullsized":"2024-05-08T22:05:18.525034+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:52:52.391196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.821076+0000","last_change":"2024-05-08T22:05:13.559286+0000","last_active":"2024-05-08T22:05:40.821076+0000","last_peered":"2024-05-08T22:05:40.821076+0000","last_clean":"2024-05-08T22:05:40.821076+0000","last_became_active":"2024-05-08T22:05:13.558922+0000","last_became_peered":"2024-05-08T22:05:13.558922+0000","last_unstale":"2024-05-08T22:05:40.821076+0000","last_undegraded":"2024-05-08T22:05:40.821076+0000","last_fullsized":"2024-05-08T22:05:40.821076+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:25:24.732422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.566174+0000","last_change":"2024-05-08T22:05:18.527031+0000","last_active":"2024-05-08T22:05:18.566174+0000","last_peered":"2024-05-08T22:05:18.566174+0000","last_clean":"2024-05-08T22:05:18.566174+0000","last_became_active":"2024-05-08T22:05:18.526573+0000","last_became_peered":"2024-05-08T22:05:18.526573+0000","last_unstale":"2024-05-08T22:05:18.566174+0000","last_undegraded":"2024-05-08T22:05:18.566174+0000","last_fullsized":"2024-05-08T22:05:18.566174+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:08.492359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714236+0000","last_change":"2024-05-08T22:05:14.053930+0000","last_active":"2024-05-08T22:05:19.714236+0000","last_peered":"2024-05-08T22:05:19.714236+0000","last_clean":"2024-05-08T22:05:19.714236+0000","last_became_active":"2024-05-08T22:05:14.052968+0000","last_became_peered":"2024-05-08T22:05:14.052968+0000","last_unstale":"2024-05-08T22:05:19.714236+0000","last_undegraded":"2024-05-08T22:05:19.714236+0000","last_fullsized":"2024-05-08T22:05:19.714236+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:48.748323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:39.501541+0000","last_change":"2024-05-08T22:05:13.545709+0000","last_active":"2024-05-08T22:05:39.501541+0000","last_peered":"2024-05-08T22:05:39.501541+0000","last_clean":"2024-05-08T22:05:39.501541+0000","last_became_active":"2024-05-08T22:05:13.545419+0000","last_became_peered":"2024-05-08T22:05:13.545419+0000","last_unstale":"2024-05-08T22:05:39.501541+0000","last_undegraded":"2024-05-08T22:05:39.501541+0000","last_fullsized":"2024-05-08T22:05:39.501541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:44.903903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":335,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:20.446030+0000","last_change":"2024-05-08T22:04:20.948328+0000","last_active":"2024-05-08T22:05:20.446030+0000","last_peered":"2024-05-08T22:05:20.446030+0000","last_clean":"2024-05-08T22:05:20.446030+0000","last_became_active":"2024-05-08T22:04:20.947887+0000","last_became_peered":"2024-05-08T22:04:20.947887+0000","last_unstale":"2024-05-08T22:05:20.446030+0000","last_undegraded":"2024-05-08T22:05:20.446030+0000","last_fullsized":"2024-05-08T22:05:20.446030+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:01:41.485277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:01:41.485277+0000","last_clean_scrub_stamp":"2024-05-08T22:01:41.485277+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:43:59.208120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.834146+0000","last_change":"2024-05-08T22:05:18.616989+0000","last_active":"2024-05-08T22:05:19.834146+0000","last_peered":"2024-05-08T22:05:19.834146+0000","last_clean":"2024-05-08T22:05:19.834146+0000","last_became_active":"2024-05-08T22:05:18.616560+0000","last_became_peered":"2024-05-08T22:05:18.616560+0000","last_unstale":"2024-05-08T22:05:19.834146+0000","last_undegraded":"2024-05-08T22:05:19.834146+0000","last_fullsized":"2024-05-08T22:05:19.834146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:45:31.141303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.153692+0000","last_change":"2024-05-08T22:05:14.066818+0000","last_active":"2024-05-08T22:05:19.153692+0000","last_peered":"2024-05-08T22:05:19.153692+0000","last_clean":"2024-05-08T22:05:19.153692+0000","last_became_active":"2024-05-08T22:05:14.065152+0000","last_became_peered":"2024-05-08T22:05:14.065152+0000","last_unstale":"2024-05-08T22:05:19.153692+0000","last_undegraded":"2024-05-08T22:05:19.153692+0000","last_fullsized":"2024-05-08T22:05:19.153692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:53:19.018852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:38.498880+0000","last_change":"2024-05-08T22:05:13.550365+0000","last_active":"2024-05-08T22:05:38.498880+0000","last_peered":"2024-05-08T22:05:38.498880+0000","last_clean":"2024-05-08T22:05:38.498880+0000","last_became_active":"2024-05-08T22:05:13.549677+0000","last_became_peered":"2024-05-08T22:05:13.549677+0000","last_unstale":"2024-05-08T22:05:38.498880+0000","last_undegraded":"2024-05-08T22:05:38.498880+0000","last_fullsized":"2024-05-08T22:05:38.498880+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:10.842838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.018502+0000","last_change":"2024-05-08T22:05:18.535777+0000","last_active":"2024-05-08T22:05:19.018502+0000","last_peered":"2024-05-08T22:05:19.018502+0000","last_clean":"2024-05-08T22:05:19.018502+0000","last_became_active":"2024-05-08T22:05:18.535327+0000","last_became_peered":"2024-05-08T22:05:18.535327+0000","last_unstale":"2024-05-08T22:05:19.018502+0000","last_undegraded":"2024-05-08T22:05:19.018502+0000","last_fullsized":"2024-05-08T22:05:19.018502+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:13.886989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831170+0000","last_change":"2024-05-08T22:05:13.554045+0000","last_active":"2024-05-08T22:05:40.831170+0000","last_peered":"2024-05-08T22:05:40.831170+0000","last_clean":"2024-05-08T22:05:40.831170+0000","last_became_active":"2024-05-08T22:05:13.553315+0000","last_became_peered":"2024-05-08T22:05:13.553315+0000","last_unstale":"2024-05-08T22:05:40.831170+0000","last_undegraded":"2024-05-08T22:05:40.831170+0000","last_fullsized":"2024-05-08T22:05:40.831170+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:13.131267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.154279+0000","last_change":"2024-05-08T22:05:14.065018+0000","last_active":"2024-05-08T22:05:19.154279+0000","last_peered":"2024-05-08T22:05:19.154279+0000","last_clean":"2024-05-08T22:05:19.154279+0000","last_became_active":"2024-05-08T22:05:14.064701+0000","last_became_peered":"2024-05-08T22:05:14.064701+0000","last_unstale":"2024-05-08T22:05:19.154279+0000","last_undegraded":"2024-05-08T22:05:19.154279+0000","last_fullsized":"2024-05-08T22:05:19.154279+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:48:43.468373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.504924+0000","last_change":"2024-05-08T22:05:18.617853+0000","last_active":"2024-05-08T22:05:21.504924+0000","last_peered":"2024-05-08T22:05:21.504924+0000","last_clean":"2024-05-08T22:05:21.504924+0000","last_became_active":"2024-05-08T22:05:18.616255+0000","last_became_peered":"2024-05-08T22:05:18.616255+0000","last_unstale":"2024-05-08T22:05:21.504924+0000","last_undegraded":"2024-05-08T22:05:21.504924+0000","last_fullsized":"2024-05-08T22:05:21.504924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:13.233425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:38.499114+0000","last_change":"2024-05-08T22:05:13.546274+0000","last_active":"2024-05-08T22:05:38.499114+0000","last_peered":"2024-05-08T22:05:38.499114+0000","last_clean":"2024-05-08T22:05:38.499114+0000","last_became_active":"2024-05-08T22:05:13.543651+0000","last_became_peered":"2024-05-08T22:05:13.543651+0000","last_unstale":"2024-05-08T22:05:38.499114+0000","last_undegraded":"2024-05-08T22:05:38.499114+0000","last_fullsized":"2024-05-08T22:05:38.499114+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:21:58.601261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714604+0000","last_change":"2024-05-08T22:05:14.056535+0000","last_active":"2024-05-08T22:05:19.714604+0000","last_peered":"2024-05-08T22:05:19.714604+0000","last_clean":"2024-05-08T22:05:19.714604+0000","last_became_active":"2024-05-08T22:05:14.056211+0000","last_became_peered":"2024-05-08T22:05:14.056211+0000","last_unstale":"2024-05-08T22:05:19.714604+0000","last_undegraded":"2024-05-08T22:05:19.714604+0000","last_fullsized":"2024-05-08T22:05:19.714604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:46:30.744490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.022982+0000","last_change":"2024-05-08T22:05:18.524939+0000","last_active":"2024-05-08T22:05:21.022982+0000","last_peered":"2024-05-08T22:05:21.022982+0000","last_clean":"2024-05-08T22:05:21.022982+0000","last_became_active":"2024-05-08T22:05:18.524518+0000","last_became_peered":"2024-05-08T22:05:18.524518+0000","last_unstale":"2024-05-08T22:05:21.022982+0000","last_undegraded":"2024-05-08T22:05:21.022982+0000","last_fullsized":"2024-05-08T22:05:21.022982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:50.192400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.493290+0000","last_change":"2024-05-08T22:05:13.554346+0000","last_active":"2024-05-08T22:05:37.493290+0000","last_peered":"2024-05-08T22:05:37.493290+0000","last_clean":"2024-05-08T22:05:37.493290+0000","last_became_active":"2024-05-08T22:05:13.554080+0000","last_became_peered":"2024-05-08T22:05:13.554080+0000","last_unstale":"2024-05-08T22:05:37.493290+0000","last_undegraded":"2024-05-08T22:05:37.493290+0000","last_fullsized":"2024-05-08T22:05:37.493290+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:29:49.695682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.523372+0000","last_change":"2024-05-08T22:05:14.071422+0000","last_active":"2024-05-08T22:05:18.523372+0000","last_peered":"2024-05-08T22:05:18.523372+0000","last_clean":"2024-05-08T22:05:18.523372+0000","last_became_active":"2024-05-08T22:05:14.070396+0000","last_became_peered":"2024-05-08T22:05:14.070396+0000","last_unstale":"2024-05-08T22:05:18.523372+0000","last_undegraded":"2024-05-08T22:05:18.523372+0000","last_fullsized":"2024-05-08T22:05:18.523372+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:37.957983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.804960+0000","last_change":"2024-05-08T22:05:18.617149+0000","last_active":"2024-05-08T22:05:21.804960+0000","last_peered":"2024-05-08T22:05:21.804960+0000","last_clean":"2024-05-08T22:05:21.804960+0000","last_became_active":"2024-05-08T22:05:18.616699+0000","last_became_peered":"2024-05-08T22:05:18.616699+0000","last_unstale":"2024-05-08T22:05:21.804960+0000","last_undegraded":"2024-05-08T22:05:21.804960+0000","last_fullsized":"2024-05-08T22:05:21.804960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:51.167142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.492588+0000","last_change":"2024-05-08T22:05:13.551212+0000","last_active":"2024-05-08T22:05:37.492588+0000","last_peered":"2024-05-08T22:05:37.492588+0000","last_clean":"2024-05-08T22:05:37.492588+0000","last_became_active":"2024-05-08T22:05:13.550909+0000","last_became_peered":"2024-05-08T22:05:13.550909+0000","last_unstale":"2024-05-08T22:05:37.492588+0000","last_undegraded":"2024-05-08T22:05:37.492588+0000","last_fullsized":"2024-05-08T22:05:37.492588+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:48.657928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829703+0000","last_change":"2024-05-08T22:05:14.053976+0000","last_active":"2024-05-08T22:05:40.829703+0000","last_peered":"2024-05-08T22:05:40.829703+0000","last_clean":"2024-05-08T22:05:40.829703+0000","last_became_active":"2024-05-08T22:05:14.053664+0000","last_became_peered":"2024-05-08T22:05:14.053664+0000","last_unstale":"2024-05-08T22:05:40.829703+0000","last_undegraded":"2024-05-08T22:05:40.829703+0000","last_fullsized":"2024-05-08T22:05:40.829703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:25:25.642768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.018493+0000","last_change":"2024-05-08T22:05:18.536771+0000","last_active":"2024-05-08T22:05:19.018493+0000","last_peered":"2024-05-08T22:05:19.018493+0000","last_clean":"2024-05-08T22:05:19.018493+0000","last_became_active":"2024-05-08T22:05:18.536260+0000","last_became_peered":"2024-05-08T22:05:18.536260+0000","last_unstale":"2024-05-08T22:05:19.018493+0000","last_undegraded":"2024-05-08T22:05:19.018493+0000","last_fullsized":"2024-05-08T22:05:19.018493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:34.414381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.494220+0000","last_change":"2024-05-08T22:05:13.540679+0000","last_active":"2024-05-08T22:05:37.494220+0000","last_peered":"2024-05-08T22:05:37.494220+0000","last_clean":"2024-05-08T22:05:37.494220+0000","last_became_active":"2024-05-08T22:05:13.539428+0000","last_became_peered":"2024-05-08T22:05:13.539428+0000","last_unstale":"2024-05-08T22:05:37.494220+0000","last_undegraded":"2024-05-08T22:05:37.494220+0000","last_fullsized":"2024-05-08T22:05:37.494220+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:59:34.455411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518423+0000","last_change":"2024-05-08T22:05:14.061849+0000","last_active":"2024-05-08T22:05:18.518423+0000","last_peered":"2024-05-08T22:05:18.518423+0000","last_clean":"2024-05-08T22:05:18.518423+0000","last_became_active":"2024-05-08T22:05:14.061028+0000","last_became_peered":"2024-05-08T22:05:14.061028+0000","last_unstale":"2024-05-08T22:05:18.518423+0000","last_undegraded":"2024-05-08T22:05:18.518423+0000","last_fullsized":"2024-05-08T22:05:18.518423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:11:59.489376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819278+0000","last_change":"2024-05-08T22:05:18.517994+0000","last_active":"2024-05-08T22:05:40.819278+0000","last_peered":"2024-05-08T22:05:40.819278+0000","last_clean":"2024-05-08T22:05:40.819278+0000","last_became_active":"2024-05-08T22:05:18.517655+0000","last_became_peered":"2024-05-08T22:05:18.517655+0000","last_unstale":"2024-05-08T22:05:40.819278+0000","last_undegraded":"2024-05-08T22:05:40.819278+0000","last_fullsized":"2024-05-08T22:05:40.819278+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:50:08.956277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.039182+0000","last_change":"2024-05-08T22:05:13.552851+0000","last_active":"2024-05-08T22:05:19.039182+0000","last_peered":"2024-05-08T22:05:19.039182+0000","last_clean":"2024-05-08T22:05:19.039182+0000","last_became_active":"2024-05-08T22:05:13.552507+0000","last_became_peered":"2024-05-08T22:05:13.552507+0000","last_unstale":"2024-05-08T22:05:19.039182+0000","last_undegraded":"2024-05-08T22:05:19.039182+0000","last_fullsized":"2024-05-08T22:05:19.039182+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:29.248623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714486+0000","last_change":"2024-05-08T22:05:14.062618+0000","last_active":"2024-05-08T22:05:19.714486+0000","last_peered":"2024-05-08T22:05:19.714486+0000","last_clean":"2024-05-08T22:05:19.714486+0000","last_became_active":"2024-05-08T22:05:14.062308+0000","last_became_peered":"2024-05-08T22:05:14.062308+0000","last_unstale":"2024-05-08T22:05:19.714486+0000","last_undegraded":"2024-05-08T22:05:19.714486+0000","last_fullsized":"2024-05-08T22:05:19.714486+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:34.195343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.804949+0000","last_change":"2024-05-08T22:05:18.616846+0000","last_active":"2024-05-08T22:05:21.804949+0000","last_peered":"2024-05-08T22:05:21.804949+0000","last_clean":"2024-05-08T22:05:21.804949+0000","last_became_active":"2024-05-08T22:05:18.616369+0000","last_became_peered":"2024-05-08T22:05:18.616369+0000","last_unstale":"2024-05-08T22:05:21.804949+0000","last_undegraded":"2024-05-08T22:05:21.804949+0000","last_fullsized":"2024-05-08T22:05:21.804949+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:26.128949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829370+0000","last_change":"2024-05-08T22:05:13.549712+0000","last_active":"2024-05-08T22:05:40.829370+0000","last_peered":"2024-05-08T22:05:40.829370+0000","last_clean":"2024-05-08T22:05:40.829370+0000","last_became_active":"2024-05-08T22:05:13.549508+0000","last_became_peered":"2024-05-08T22:05:13.549508+0000","last_unstale":"2024-05-08T22:05:40.829370+0000","last_undegraded":"2024-05-08T22:05:40.829370+0000","last_fullsized":"2024-05-08T22:05:40.829370+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:30:22.706389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.516692+0000","last_change":"2024-05-08T22:05:14.060920+0000","last_active":"2024-05-08T22:05:18.516692+0000","last_peered":"2024-05-08T22:05:18.516692+0000","last_clean":"2024-05-08T22:05:18.516692+0000","last_became_active":"2024-05-08T22:05:14.059914+0000","last_became_peered":"2024-05-08T22:05:14.059914+0000","last_unstale":"2024-05-08T22:05:18.516692+0000","last_undegraded":"2024-05-08T22:05:18.516692+0000","last_fullsized":"2024-05-08T22:05:18.516692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:15.513600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.534473+0000","last_change":"2024-05-08T22:05:18.534076+0000","last_active":"2024-05-08T22:05:18.534473+0000","last_peered":"2024-05-08T22:05:18.534473+0000","last_clean":"2024-05-08T22:05:18.534473+0000","last_became_active":"2024-05-08T22:05:18.533679+0000","last_became_peered":"2024-05-08T22:05:18.533679+0000","last_unstale":"2024-05-08T22:05:18.534473+0000","last_undegraded":"2024-05-08T22:05:18.534473+0000","last_fullsized":"2024-05-08T22:05:18.534473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:08.979193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.521141+0000","last_change":"2024-05-08T22:05:13.559282+0000","last_active":"2024-05-08T22:05:18.521141+0000","last_peered":"2024-05-08T22:05:18.521141+0000","last_clean":"2024-05-08T22:05:18.521141+0000","last_became_active":"2024-05-08T22:05:13.558876+0000","last_became_peered":"2024-05-08T22:05:13.558876+0000","last_unstale":"2024-05-08T22:05:18.521141+0000","last_undegraded":"2024-05-08T22:05:18.521141+0000","last_fullsized":"2024-05-08T22:05:18.521141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:56.062332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.516801+0000","last_change":"2024-05-08T22:05:14.065868+0000","last_active":"2024-05-08T22:05:18.516801+0000","last_peered":"2024-05-08T22:05:18.516801+0000","last_clean":"2024-05-08T22:05:18.516801+0000","last_became_active":"2024-05-08T22:05:14.065545+0000","last_became_peered":"2024-05-08T22:05:14.065545+0000","last_unstale":"2024-05-08T22:05:18.516801+0000","last_undegraded":"2024-05-08T22:05:18.516801+0000","last_fullsized":"2024-05-08T22:05:18.516801+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:56:34.309095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.154428+0000","last_change":"2024-05-08T22:05:18.534261+0000","last_active":"2024-05-08T22:05:19.154428+0000","last_peered":"2024-05-08T22:05:19.154428+0000","last_clean":"2024-05-08T22:05:19.154428+0000","last_became_active":"2024-05-08T22:05:18.531814+0000","last_became_peered":"2024-05-08T22:05:18.531814+0000","last_unstale":"2024-05-08T22:05:19.154428+0000","last_undegraded":"2024-05-08T22:05:19.154428+0000","last_fullsized":"2024-05-08T22:05:19.154428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:38.344546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:20.446341+0000","last_change":"2024-05-08T22:05:13.549344+0000","last_active":"2024-05-08T22:05:20.446341+0000","last_peered":"2024-05-08T22:05:20.446341+0000","last_clean":"2024-05-08T22:05:20.446341+0000","last_became_active":"2024-05-08T22:05:13.548993+0000","last_became_peered":"2024-05-08T22:05:13.548993+0000","last_unstale":"2024-05-08T22:05:20.446341+0000","last_undegraded":"2024-05-08T22:05:20.446341+0000","last_fullsized":"2024-05-08T22:05:20.446341+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:50.096602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.515948+0000","last_change":"2024-05-08T22:05:14.065758+0000","last_active":"2024-05-08T22:05:18.515948+0000","last_peered":"2024-05-08T22:05:18.515948+0000","last_clean":"2024-05-08T22:05:18.515948+0000","last_became_active":"2024-05-08T22:05:14.065442+0000","last_became_peered":"2024-05-08T22:05:14.065442+0000","last_unstale":"2024-05-08T22:05:18.515948+0000","last_undegraded":"2024-05-08T22:05:18.515948+0000","last_fullsized":"2024-05-08T22:05:18.515948+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:43.047796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.834022+0000","last_change":"2024-05-08T22:05:18.529804+0000","last_active":"2024-05-08T22:05:19.834022+0000","last_peered":"2024-05-08T22:05:19.834022+0000","last_clean":"2024-05-08T22:05:19.834022+0000","last_became_active":"2024-05-08T22:05:18.529346+0000","last_became_peered":"2024-05-08T22:05:18.529346+0000","last_unstale":"2024-05-08T22:05:19.834022+0000","last_undegraded":"2024-05-08T22:05:19.834022+0000","last_fullsized":"2024-05-08T22:05:19.834022+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:13.565707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.154118+0000","last_change":"2024-05-08T22:05:13.551818+0000","last_active":"2024-05-08T22:05:19.154118+0000","last_peered":"2024-05-08T22:05:19.154118+0000","last_clean":"2024-05-08T22:05:19.154118+0000","last_became_active":"2024-05-08T22:05:13.551080+0000","last_became_peered":"2024-05-08T22:05:13.551080+0000","last_unstale":"2024-05-08T22:05:19.154118+0000","last_undegraded":"2024-05-08T22:05:19.154118+0000","last_fullsized":"2024-05-08T22:05:19.154118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:31.636482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.517095+0000","last_change":"2024-05-08T22:05:14.066572+0000","last_active":"2024-05-08T22:05:18.517095+0000","last_peered":"2024-05-08T22:05:18.517095+0000","last_clean":"2024-05-08T22:05:18.517095+0000","last_became_active":"2024-05-08T22:05:14.066272+0000","last_became_peered":"2024-05-08T22:05:14.066272+0000","last_unstale":"2024-05-08T22:05:18.517095+0000","last_undegraded":"2024-05-08T22:05:18.517095+0000","last_fullsized":"2024-05-08T22:05:18.517095+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:20:27.423464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518519+0000","last_change":"2024-05-08T22:05:14.061435+0000","last_active":"2024-05-08T22:05:18.518519+0000","last_peered":"2024-05-08T22:05:18.518519+0000","last_clean":"2024-05-08T22:05:18.518519+0000","last_became_active":"2024-05-08T22:05:14.061079+0000","last_became_peered":"2024-05-08T22:05:14.061079+0000","last_unstale":"2024-05-08T22:05:18.518519+0000","last_undegraded":"2024-05-08T22:05:18.518519+0000","last_fullsized":"2024-05-08T22:05:18.518519+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:42.684743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"peering","last_fresh":"2024-05-08T22:05:40.817668+0000","last_change":"2024-05-08T22:05:40.817845+0000","last_active":"2024-05-08T22:05:40.817668+0000","last_peered":"2024-05-08T22:05:19.039319+0000","last_clean":"2024-05-08T22:05:19.039319+0000","last_became_active":"2024-05-08T22:05:14.071184+0000","last_became_peered":"2024-05-08T22:05:14.071184+0000","last_unstale":"2024-05-08T22:05:40.817668+0000","last_undegraded":"2024-05-08T22:05:40.817668+0000","last_fullsized":"2024-05-08T22:05:40.817668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:12:55.547522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5,8],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.039153+0000","last_change":"2024-05-08T22:05:14.066697+0000","last_active":"2024-05-08T22:05:19.039153+0000","last_peered":"2024-05-08T22:05:19.039153+0000","last_clean":"2024-05-08T22:05:19.039153+0000","last_became_active":"2024-05-08T22:05:14.065952+0000","last_became_peered":"2024-05-08T22:05:14.065952+0000","last_unstale":"2024-05-08T22:05:19.039153+0000","last_undegraded":"2024-05-08T22:05:19.039153+0000","last_fullsized":"2024-05-08T22:05:19.039153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:03:19.006644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.521494+0000","last_change":"2024-05-08T22:05:14.072295+0000","last_active":"2024-05-08T22:05:18.521494+0000","last_peered":"2024-05-08T22:05:18.521494+0000","last_clean":"2024-05-08T22:05:18.521494+0000","last_became_active":"2024-05-08T22:05:14.065034+0000","last_became_peered":"2024-05-08T22:05:14.065034+0000","last_unstale":"2024-05-08T22:05:18.521494+0000","last_undegraded":"2024-05-08T22:05:18.521494+0000","last_fullsized":"2024-05-08T22:05:18.521494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:08:41.380814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.154280+0000","last_change":"2024-05-08T22:05:14.067263+0000","last_active":"2024-05-08T22:05:19.154280+0000","last_peered":"2024-05-08T22:05:19.154280+0000","last_clean":"2024-05-08T22:05:19.154280+0000","last_became_active":"2024-05-08T22:05:14.066957+0000","last_became_peered":"2024-05-08T22:05:14.066957+0000","last_unstale":"2024-05-08T22:05:19.154280+0000","last_undegraded":"2024-05-08T22:05:19.154280+0000","last_fullsized":"2024-05-08T22:05:19.154280+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:29:35.638489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518101+0000","last_change":"2024-05-08T22:05:14.060561+0000","last_active":"2024-05-08T22:05:18.518101+0000","last_peered":"2024-05-08T22:05:18.518101+0000","last_clean":"2024-05-08T22:05:18.518101+0000","last_became_active":"2024-05-08T22:05:14.060150+0000","last_became_peered":"2024-05-08T22:05:14.060150+0000","last_unstale":"2024-05-08T22:05:18.518101+0000","last_undegraded":"2024-05-08T22:05:18.518101+0000","last_fullsized":"2024-05-08T22:05:18.518101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:22.710776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.039533+0000","last_change":"2024-05-08T22:05:14.056394+0000","last_active":"2024-05-08T22:05:19.039533+0000","last_peered":"2024-05-08T22:05:19.039533+0000","last_clean":"2024-05-08T22:05:19.039533+0000","last_became_active":"2024-05-08T22:05:14.055977+0000","last_became_peered":"2024-05-08T22:05:14.055977+0000","last_unstale":"2024-05-08T22:05:19.039533+0000","last_undegraded":"2024-05-08T22:05:19.039533+0000","last_fullsized":"2024-05-08T22:05:19.039533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:56:48.249379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714720+0000","last_change":"2024-05-08T22:05:14.061934+0000","last_active":"2024-05-08T22:05:19.714720+0000","last_peered":"2024-05-08T22:05:19.714720+0000","last_clean":"2024-05-08T22:05:19.714720+0000","last_became_active":"2024-05-08T22:05:14.061635+0000","last_became_peered":"2024-05-08T22:05:14.061635+0000","last_unstale":"2024-05-08T22:05:19.714720+0000","last_undegraded":"2024-05-08T22:05:19.714720+0000","last_fullsized":"2024-05-08T22:05:19.714720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:02.423338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.524933+0000","last_change":"2024-05-08T22:05:14.069982+0000","last_active":"2024-05-08T22:05:18.524933+0000","last_peered":"2024-05-08T22:05:18.524933+0000","last_clean":"2024-05-08T22:05:18.524933+0000","last_became_active":"2024-05-08T22:05:14.068293+0000","last_became_peered":"2024-05-08T22:05:14.068293+0000","last_unstale":"2024-05-08T22:05:18.524933+0000","last_undegraded":"2024-05-08T22:05:18.524933+0000","last_fullsized":"2024-05-08T22:05:18.524933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:36.396983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714278+0000","last_change":"2024-05-08T22:05:14.058393+0000","last_active":"2024-05-08T22:05:19.714278+0000","last_peered":"2024-05-08T22:05:19.714278+0000","last_clean":"2024-05-08T22:05:19.714278+0000","last_became_active":"2024-05-08T22:05:14.058083+0000","last_became_peered":"2024-05-08T22:05:14.058083+0000","last_unstale":"2024-05-08T22:05:19.714278+0000","last_undegraded":"2024-05-08T22:05:19.714278+0000","last_fullsized":"2024-05-08T22:05:19.714278+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:10:47.074957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518092+0000","last_change":"2024-05-08T22:05:14.056989+0000","last_active":"2024-05-08T22:05:18.518092+0000","last_peered":"2024-05-08T22:05:18.518092+0000","last_clean":"2024-05-08T22:05:18.518092+0000","last_became_active":"2024-05-08T22:05:14.056669+0000","last_became_peered":"2024-05-08T22:05:14.056669+0000","last_unstale":"2024-05-08T22:05:18.518092+0000","last_undegraded":"2024-05-08T22:05:18.518092+0000","last_fullsized":"2024-05-08T22:05:18.518092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:56:51.173885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.153888+0000","last_change":"2024-05-08T22:05:14.058635+0000","last_active":"2024-05-08T22:05:19.153888+0000","last_peered":"2024-05-08T22:05:19.153888+0000","last_clean":"2024-05-08T22:05:19.153888+0000","last_became_active":"2024-05-08T22:05:14.058243+0000","last_became_peered":"2024-05-08T22:05:14.058243+0000","last_unstale":"2024-05-08T22:05:19.153888+0000","last_undegraded":"2024-05-08T22:05:19.153888+0000","last_fullsized":"2024-05-08T22:05:19.153888+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:18.142563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518557+0000","last_change":"2024-05-08T22:05:14.060735+0000","last_active":"2024-05-08T22:05:18.518557+0000","last_peered":"2024-05-08T22:05:18.518557+0000","last_clean":"2024-05-08T22:05:18.518557+0000","last_became_active":"2024-05-08T22:05:14.060391+0000","last_became_peered":"2024-05-08T22:05:14.060391+0000","last_unstale":"2024-05-08T22:05:18.518557+0000","last_undegraded":"2024-05-08T22:05:18.518557+0000","last_fullsized":"2024-05-08T22:05:18.518557+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:30.528705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283045,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6728,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":23,"num_read_kb":19,"num_write":118,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":41050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579920,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36088,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709064,"statfs":{"total":95995035648,"available":95958081536,"internally_reserved":0,"allocated":901120,"data_stored":447161,"data_compressed":11671,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36043205},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.36899999999999999}]},{"osd":1,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.46000000000000002}]},{"osd":2,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.47299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":4,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.504}]},{"osd":5,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.622}]},{"osd":6,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.68200000000000005}]},{"osd":7,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.60199999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":9,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.73499999999999999}]},{"osd":10,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.748}]}]},{"osd":10,"up_from":67,"seq":287762808850,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":966656,"data_stored":1040606,"data_compressed":47667,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":1,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.442}]},{"osd":2,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.54100000000000004}]},{"osd":3,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.502},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.64100000000000001}]},{"osd":4,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.68300000000000005}]},{"osd":5,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.65700000000000003}]},{"osd":6,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.70099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":8,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.747},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.78500000000000003}]},{"osd":9,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.80400000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]}]},{"osd":9,"up_from":62,"seq":266287972375,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":447397,"data_compressed":11671,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.38600000000000001}]},{"osd":1,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.63500000000000001}]},{"osd":2,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.77600000000000002}]},{"osd":3,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.46000000000000002}]},{"osd":4,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.73199999999999998}]},{"osd":5,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.66300000000000003}]},{"osd":6,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.71199999999999997}]},{"osd":7,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.80200000000000005}]},{"osd":8,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.498},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.67900000000000005}]},{"osd":10,"last update":"Wed May 8 22:05:22 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.746},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.76400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]}]},{"osd":8,"up_from":55,"seq":236223201308,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":469876,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":0.40799999999999997}]},{"osd":1,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.47199999999999998}]},{"osd":2,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.42499999999999999}]},{"osd":3,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":4.8920000000000003,"5min":4.8920000000000003,"15min":4.8920000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.67100000000000004}]},{"osd":4,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.627}]},{"osd":5,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.52500000000000002}]},{"osd":6,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.9210000000000003,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.77900000000000003}]},{"osd":7,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.748},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.64700000000000002}]},{"osd":9,"last update":"Wed May 8 22:05:01 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.80600000000000005}]},{"osd":10,"last update":"Wed May 8 22:05:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.79100000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]}]},{"osd":7,"up_from":48,"seq":206158430237,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":1038103,"data_compressed":47667,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.41199999999999998}]},{"osd":1,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.39500000000000002,"15min":0.38700000000000001},"min":{"1min":0.30599999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.443},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40600000000000003,"15min":0.40300000000000002},"min":{"1min":0.29199999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.36699999999999999}]},{"osd":2,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54100000000000004,"15min":0.53600000000000003},"min":{"1min":0.34899999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.38400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70299999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.68000000000000005}]},{"osd":3,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.41599999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79800000000000004,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.501,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80200000000000005,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.52900000000000003}]},{"osd":4,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.33800000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71999999999999997,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.754,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.51000000000000001}]},{"osd":5,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64400000000000002,"15min":0.64500000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89900000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.625,"15min":0.626},"min":{"1min":0.44500000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.77700000000000002,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.63200000000000001}]},{"osd":6,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85099999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69699999999999995,"15min":0.69899999999999995},"min":{"1min":0.48199999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89000000000000001,"5min":1.002,"15min":1.002},"last":0.46800000000000003}]},{"osd":8,"last update":"Wed May 8 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.68700000000000006,"15min":0.69399999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.78000000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.72099999999999997,"15min":0.72699999999999998},"min":{"1min":0.55300000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.85899999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.74099999999999999}]},{"osd":9,"last update":"Wed May 8 22:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.61699999999999999}]},{"osd":10,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.70799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]}]},{"osd":6,"up_from":43,"seq":184683593762,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":455906,"data_compressed":14627,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.46200000000000002,"15min":0.45100000000000001},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.66300000000000003,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.51000000000000001,"15min":0.51300000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.64400000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.45900000000000002}]},{"osd":1,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57299999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.61899999999999999,"15min":0.60599999999999998},"min":{"1min":0.47399999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.77800000000000002}]},{"osd":2,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.66200000000000003,"15min":0.64800000000000002},"min":{"1min":0.53200000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65200000000000002,"15min":0.64700000000000002},"min":{"1min":0.51400000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89300000000000002,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.65300000000000002}]},{"osd":3,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60999999999999999,"15min":0.60699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.141,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.44},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.495,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.71499999999999997}]},{"osd":4,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69699999999999995,"15min":0.69299999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0569999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89700000000000002,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.68700000000000006}]},{"osd":5,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.754,"15min":0.74399999999999999},"min":{"1min":0.57999999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.995,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.75800000000000001,"15min":0.752},"min":{"1min":0.56399999999999995,"5min":0.438,"15min":0.438},"max":{"1min":1.052,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.64200000000000002}]},{"osd":7,"last update":"Wed May 8 22:05:23 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.99299999999999999,"5min":1.028,"15min":1.028},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76300000000000001,"15min":0.76000000000000001},"min":{"1min":0.56999999999999995,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.98499999999999999}]},{"osd":8,"last update":"Wed May 8 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.83199999999999996,"15min":0.83999999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0289999999999999,"5min":1.093,"15min":1.093},"last":0.871},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.79300000000000004,"15min":0.79600000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0149999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.67100000000000004}]},{"osd":9,"last update":"Wed May 8 22:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.73599999999999999}]},{"osd":10,"last update":"Wed May 8 22:05:19 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.88200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":905216,"data_stored":450102,"data_compressed":11671,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.52700000000000002,"15min":0.49299999999999999},"min":{"1min":0.38900000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.89900000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54000000000000004,"15min":0.51400000000000001},"min":{"1min":0.371,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81200000000000006,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.58299999999999996}]},{"osd":2,"last update":"Wed May 8 22:04:46 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.51300000000000001,"15min":0.48499999999999999},"min":{"1min":0.378,"5min":0.316,"15min":0.316},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.50600000000000001,"15min":0.48399999999999999},"min":{"1min":0.315,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.621}]},{"osd":3,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.53300000000000003,"15min":0.505},"min":{"1min":0.40000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.53300000000000003,"15min":0.51200000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.56899999999999995}]},{"osd":4,"last update":"Wed May 8 22:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.39900000000000002,"15min":0.376},"min":{"1min":0.307,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.64000000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.42499999999999999,"15min":0.39300000000000002},"min":{"1min":0.32400000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.76700000000000002,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.497}]},{"osd":5,"last update":"Wed May 8 22:04:43 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.56399999999999995,"15min":0.54800000000000004},"min":{"1min":0.39400000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58499999999999996,"15min":0.57299999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.45400000000000001}]},{"osd":6,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.63,"15min":0.61599999999999999},"min":{"1min":0.55100000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.46500000000000002}]},{"osd":7,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.36499999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86299999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.438}]},{"osd":8,"last update":"Wed May 8 22:04:37 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.60499999999999998}]},{"osd":9,"last update":"Wed May 8 22:05:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.70999999999999996}]},{"osd":10,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.65900000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]}]},{"osd":0,"up_from":8,"seq":34359738425,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":905216,"data_stored":447000,"data_compressed":11671,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57899999999999996,"15min":0.54900000000000004},"min":{"1min":0.42499999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.94099999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.55400000000000005,"15min":0.52400000000000002},"min":{"1min":0.40100000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83199999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.55500000000000005}]},{"osd":2,"last update":"Wed May 8 22:04:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.56399999999999995,"15min":0.54000000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.56100000000000005,"15min":0.54100000000000004},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.59099999999999997}]},{"osd":3,"last update":"Wed May 8 22:04:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.436,"15min":0.41799999999999998},"min":{"1min":0.34799999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.48799999999999999,"15min":0.48199999999999998},"min":{"1min":0.35499999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.877,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.82899999999999996}]},{"osd":4,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.57099999999999995,"15min":0.54900000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69399999999999995,"15min":0.69599999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96999999999999997,"5min":2.984,"15min":2.984},"last":0.78200000000000003}]},{"osd":5,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.65500000000000003,"15min":0.63900000000000001},"min":{"1min":0.48399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67100000000000004,"15min":0.66200000000000003},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.86599999999999999}]},{"osd":6,"last update":"Wed May 8 22:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.68600000000000005,"15min":0.66900000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.97599999999999998}]},{"osd":7,"last update":"Wed May 8 22:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.92000000000000004}]},{"osd":8,"last update":"Wed May 8 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.96099999999999997}]},{"osd":9,"last update":"Wed May 8 22:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.014}]},{"osd":10,"last update":"Wed May 8 22:05:22 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":1},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.73899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36132,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709020,"statfs":{"total":95995035648,"available":95958036480,"internally_reserved":0,"allocated":946176,"data_stored":470429,"data_compressed":11671,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.41999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.51200000000000001,"15min":0.504},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76100000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.55700000000000005}]},{"osd":1,"last update":"Wed May 8 22:04:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52600000000000002,"15min":0.52500000000000002},"min":{"1min":0.34100000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.74199999999999999,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.35499999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.77500000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.66500000000000004}]},{"osd":3,"last update":"Wed May 8 22:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.372,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65900000000000003,"5min":0.872,"15min":0.872},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56599999999999995,"15min":0.55900000000000005},"min":{"1min":0.40100000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72199999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.70499999999999996}]},{"osd":4,"last update":"Wed May 8 22:04:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.35699999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73399999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55700000000000005,"15min":0.55200000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.746}]},{"osd":5,"last update":"Wed May 8 22:04:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.436,"15min":0.433},"min":{"1min":0.34200000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.59999999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44400000000000001,"15min":0.441},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.72399999999999998}]},{"osd":6,"last update":"Wed May 8 22:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.76400000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66600000000000004,"15min":0.66700000000000004},"min":{"1min":0.50800000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.81499999999999995,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.89500000000000002}]},{"osd":7,"last update":"Wed May 8 22:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.94499999999999995}]},{"osd":8,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.97299999999999998}]},{"osd":9,"last update":"Wed May 8 22:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.83199999999999996}]},{"osd":10,"last update":"Wed May 8 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.877}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]}]},{"osd":3,"up_from":24,"seq":103079215149,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":456373,"data_compressed":14627,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.41699999999999998,"15min":0.39200000000000002},"min":{"1min":0.33500000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.504},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.42399999999999999,"15min":0.40200000000000002},"min":{"1min":0.35699999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.71299999999999997}]},{"osd":1,"last update":"Wed May 8 22:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.59899999999999998,"15min":0.55700000000000005},"min":{"1min":0.48599999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.56399999999999995,"15min":0.53000000000000003},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.77200000000000002}]},{"osd":2,"last update":"Wed May 8 22:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.61499999999999999,"15min":0.58899999999999997},"min":{"1min":0.51800000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.63900000000000001,"15min":0.60899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.55500000000000005}]},{"osd":4,"last update":"Wed May 8 22:05:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.62,"15min":0.59399999999999997},"min":{"1min":0.42399999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.66300000000000003,"15min":0.62},"min":{"1min":0.61299999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.98599999999999999}]},{"osd":5,"last update":"Wed May 8 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.67800000000000005,"15min":0.64500000000000002},"min":{"1min":0.46500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.64400000000000002,"15min":0.60899999999999999},"min":{"1min":0.46500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.76300000000000001}]},{"osd":6,"last update":"Wed May 8 22:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.63100000000000001,"15min":0.61499999999999999},"min":{"1min":0.46400000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.64600000000000002,"15min":0.624},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.874}]},{"osd":7,"last update":"Wed May 8 22:05:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.65700000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":1.008}]},{"osd":8,"last update":"Wed May 8 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.53100000000000003}]},{"osd":9,"last update":"Wed May 8 22:05:01 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.84999999999999998}]},{"osd":10,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.88400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986219,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":460781,"data_compressed":14627,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.44700000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.51900000000000002,"15min":0.50900000000000001},"min":{"1min":0.45000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.629}]},{"osd":1,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.374,"15min":0.35099999999999998},"min":{"1min":0.27200000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.66800000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.377,"15min":0.35499999999999998},"min":{"1min":0.28199999999999997,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.51000000000000001}]},{"osd":2,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.52600000000000002,"15min":0.50700000000000001},"min":{"1min":0.35499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71899999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.80900000000000005}]},{"osd":3,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.58799999999999997,"15min":0.56399999999999995},"min":{"1min":0.5,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84799999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56699999999999995,"15min":0.55300000000000005},"min":{"1min":0.39600000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.80000000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.67300000000000004}]},{"osd":5,"last update":"Wed May 8 22:05:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.36399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76200000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.57199999999999995,"15min":0.55000000000000004},"min":{"1min":0.42799999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.82199999999999995}]},{"osd":6,"last update":"Wed May 8 22:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.64800000000000002,"15min":0.63600000000000001},"min":{"1min":0.371,"5min":0.316,"15min":0.316},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.748},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.41799999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80500000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.60499999999999998}]},{"osd":7,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76800000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57299999999999995}]},{"osd":8,"last update":"Wed May 8 22:05:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68400000000000005,"15min":0.68600000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70799999999999996,"15min":0.70899999999999996},"min":{"1min":0.54100000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.79300000000000004}]},{"osd":9,"last update":"Wed May 8 22:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.624}]},{"osd":10,"last update":"Wed May 8 22:05:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.83499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":5,"up_from":36,"seq":154618822695,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":974848,"data_stored":1038375,"data_compressed":47667,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.378,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72899999999999998,"5min":0.749,"15min":0.749},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.48999999999999999}]},{"osd":1,"last update":"Wed May 8 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.41299999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.71699999999999997,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.622},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.37,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.58199999999999996}]},{"osd":2,"last update":"Wed May 8 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.254,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.52400000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.376,"15min":0.379},"min":{"1min":0.26600000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.46500000000000002,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.36199999999999999}]},{"osd":3,"last update":"Wed May 8 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57299999999999995,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.438},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.51800000000000002}]},{"osd":4,"last update":"Wed May 8 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.53300000000000003},"min":{"1min":0.38400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.39400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.751,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.64900000000000002}]},{"osd":6,"last update":"Wed May 8 22:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58599999999999997,"15min":0.58399999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.60699999999999998}]},{"osd":7,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.57399999999999995,"15min":0.56499999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.58799999999999997,"15min":0.57699999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.63700000000000001}]},{"osd":8,"last update":"Wed May 8 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.69399999999999995}]},{"osd":9,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.70599999999999996}]},{"osd":10,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.67400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":24792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:05:43.850 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:05:44.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.075+0000 7fd4e7a00700 1 -- 172.21.15.131:0/2488541237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4e0106b40 msgr2=0x7fd4e0108fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:44.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.075+0000 7fd4e7a00700 1 --2- 172.21.15.131:0/2488541237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4e0106b40 0x7fd4e0108fb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd4d800b0a0 tx=0x7fd4d800b3b0 comp rx=0 tx=0).stop 2024-05-08T22:05:44.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.075+0000 7fd4e7a00700 1 -- 172.21.15.131:0/2488541237 shutdown_connections 2024-05-08T22:05:44.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.075+0000 7fd4e7a00700 1 --2- 172.21.15.131:0/2488541237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4e0106b40 0x7fd4e0108fb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:44.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.075+0000 7fd4e7a00700 1 --2- 172.21.15.131:0/2488541237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e01018e0 0x7fd4e0103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:44.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.075+0000 7fd4e7a00700 1 --2- 172.21.15.131:0/2488541237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4e0104210 0x7fd4e0106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:44.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.075+0000 7fd4e7a00700 1 -- 172.21.15.131:0/2488541237 >> 172.21.15.131:0/2488541237 conn(0x7fd4e00fb2f0 msgr2=0x7fd4e00fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:44.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.075+0000 7fd4e7a00700 1 -- 172.21.15.131:0/2488541237 shutdown_connections 2024-05-08T22:05:44.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.075+0000 7fd4e7a00700 1 -- 172.21.15.131:0/2488541237 wait complete. 2024-05-08T22:05:44.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.076+0000 7fd4e7a00700 1 Processor -- start 2024-05-08T22:05:44.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.077+0000 7fd4e7a00700 1 -- start start 2024-05-08T22:05:44.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.077+0000 7fd4e7a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4e01018e0 0x7fd4e0101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:44.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.077+0000 7fd4e7a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e0104210 0x7fd4e00ff890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:44.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.077+0000 7fd4e7a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4e0106b40 0x7fd4e00ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:44.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.077+0000 7fd4e7a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e0100570 con 0x7fd4e01018e0 2024-05-08T22:05:44.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.077+0000 7fd4e7a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e01006e0 con 0x7fd4e0104210 2024-05-08T22:05:44.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.077+0000 7fd4e7a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e01a0f80 con 0x7fd4e0106b40 2024-05-08T22:05:44.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.077+0000 7fd4e5000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4e01018e0 0x7fd4e0101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:44.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.077+0000 7fd4e5000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4e01018e0 0x7fd4e0101200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38082/0 (socket says 172.21.15.131:38082) 2024-05-08T22:05:44.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.077+0000 7fd4e5000700 1 -- 172.21.15.131:0/3581575159 learned_addr learned my addr 172.21.15.131:0/3581575159 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:44.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.077+0000 7fd4e5a00700 1 --2- 172.21.15.131:0/3581575159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4e0106b40 0x7fd4e00ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:44.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.077+0000 7fd4dfe00700 1 --2- 172.21.15.131:0/3581575159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e0104210 0x7fd4e00ff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:44.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.078+0000 7fd4e5000700 1 -- 172.21.15.131:0/3581575159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4e0106b40 msgr2=0x7fd4e00ffdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:44.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.078+0000 7fd4e5000700 1 --2- 172.21.15.131:0/3581575159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4e0106b40 0x7fd4e00ffdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:44.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.078+0000 7fd4e5000700 1 -- 172.21.15.131:0/3581575159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e0104210 msgr2=0x7fd4e00ff890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:44.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.078+0000 7fd4e5000700 1 --2- 172.21.15.131:0/3581575159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e0104210 0x7fd4e00ff890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:44.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.078+0000 7fd4e5000700 1 -- 172.21.15.131:0/3581575159 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d8009d00 con 0x7fd4e01018e0 2024-05-08T22:05:44.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.078+0000 7fd4e5000700 1 --2- 172.21.15.131:0/3581575159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4e01018e0 0x7fd4e0101200 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400e9a0 tx=0x7fd4d400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:44.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.078+0000 7fd4dd600700 1 -- 172.21.15.131:0/3581575159 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4009790 con 0x7fd4e01018e0 2024-05-08T22:05:44.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.078+0000 7fd4dd600700 1 -- 172.21.15.131:0/3581575159 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4d400ce70 con 0x7fd4e01018e0 2024-05-08T22:05:44.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.078+0000 7fd4dd600700 1 -- 172.21.15.131:0/3581575159 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4018420 con 0x7fd4e01018e0 2024-05-08T22:05:44.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.078+0000 7fd4e7a00700 1 -- 172.21.15.131:0/3581575159 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e01a1210 con 0x7fd4e01018e0 2024-05-08T22:05:44.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.078+0000 7fd4e7a00700 1 -- 172.21.15.131:0/3581575159 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e01a1720 con 0x7fd4e01018e0 2024-05-08T22:05:44.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.079+0000 7fd4e7a00700 1 -- 172.21.15.131:0/3581575159 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e00fce70 con 0x7fd4e01018e0 2024-05-08T22:05:44.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.081+0000 7fd4dd600700 1 -- 172.21.15.131:0/3581575159 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fd4d40106f0 con 0x7fd4e01018e0 2024-05-08T22:05:44.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.083+0000 7fd4dd600700 1 --2- 172.21.15.131:0/3581575159 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd4c80a12b0 0x7fd4c80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:44.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.083+0000 7fd4dd600700 1 -- 172.21.15.131:0/3581575159 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd4d4014070 con 0x7fd4e01018e0 2024-05-08T22:05:44.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.084+0000 7fd4dfe00700 1 --2- 172.21.15.131:0/3581575159 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd4c80a12b0 0x7fd4c80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:44.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.084+0000 7fd4dfe00700 1 --2- 172.21.15.131:0/3581575159 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd4c80a12b0 0x7fd4c80a3770 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009c60 tx=0x7fd4cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:44.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.088+0000 7fd4dd600700 1 -- 172.21.15.131:0/3581575159 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd4d4052d60 con 0x7fd4e01018e0 2024-05-08T22:05:44.089 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:43 smithi132 ceph-mon[78180]: pgmap v256: 129 pgs: 3 peering, 126 active+clean; 583 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.5 KiB/s wr, 11 op/s; 13 B/s, 0 objects/s recovering 2024-05-08T22:05:44.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:43 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2299669838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:44.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:43 smithi131 ceph-mon[37260]: pgmap v256: 129 pgs: 3 peering, 126 active+clean; 583 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.5 KiB/s wr, 11 op/s; 13 B/s, 0 objects/s recovering 2024-05-08T22:05:44.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:43 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2299669838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:44.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:43 smithi157 ceph-mon[77746]: pgmap v256: 129 pgs: 3 peering, 126 active+clean; 583 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.5 KiB/s wr, 11 op/s; 13 B/s, 0 objects/s recovering 2024-05-08T22:05:44.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:43 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2299669838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:44.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.293+0000 7fd4e7a00700 1 -- 172.21.15.131:0/3581575159 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd4e002ce10 con 0x7fd4c80a12b0 2024-05-08T22:05:44.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.306+0000 7fd4dd600700 1 -- 172.21.15.131:0/3581575159 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+421790 (secure 0 0 0) 0x7fd4e002ce10 con 0x7fd4c80a12b0 2024-05-08T22:05:44.312 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:44.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.311+0000 7fd4e7a00700 1 -- 172.21.15.131:0/3581575159 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd4c80a12b0 msgr2=0x7fd4c80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:44.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.311+0000 7fd4e7a00700 1 --2- 172.21.15.131:0/3581575159 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd4c80a12b0 0x7fd4c80a3770 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009c60 tx=0x7fd4cc009260 comp rx=0 tx=0).stop 2024-05-08T22:05:44.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.311+0000 7fd4e7a00700 1 -- 172.21.15.131:0/3581575159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4e01018e0 msgr2=0x7fd4e0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:44.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.311+0000 7fd4e7a00700 1 --2- 172.21.15.131:0/3581575159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4e01018e0 0x7fd4e0101200 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400e9a0 tx=0x7fd4d400ecb0 comp rx=0 tx=0).stop 2024-05-08T22:05:44.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.311+0000 7fd4e7a00700 1 -- 172.21.15.131:0/3581575159 shutdown_connections 2024-05-08T22:05:44.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.311+0000 7fd4e7a00700 1 --2- 172.21.15.131:0/3581575159 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd4c80a12b0 0x7fd4c80a3770 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:44.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.311+0000 7fd4e7a00700 1 --2- 172.21.15.131:0/3581575159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4e0106b40 0x7fd4e00ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:44.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.311+0000 7fd4e7a00700 1 --2- 172.21.15.131:0/3581575159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4e01018e0 0x7fd4e0101200 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:44.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.311+0000 7fd4e7a00700 1 --2- 172.21.15.131:0/3581575159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e0104210 0x7fd4e00ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:44.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.311+0000 7fd4e7a00700 1 -- 172.21.15.131:0/3581575159 >> 172.21.15.131:0/3581575159 conn(0x7fd4e00fb2f0 msgr2=0x7fd4e0102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:44.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.311+0000 7fd4e7a00700 1 -- 172.21.15.131:0/3581575159 shutdown_connections 2024-05-08T22:05:44.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.311+0000 7fd4e7a00700 1 -- 172.21.15.131:0/3581575159 wait complete. 2024-05-08T22:05:44.317 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-08T22:05:44.332 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-05-08T22:05:42.717959+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597213,"num_objects":57,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":159,"num_read_kb":138,"num_write":300,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":242,"ondisk_log_size":242,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371992,"kb_used_data":10916,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124569832,"statfs":{"total":1151940427776,"available":1151559507968,"internally_reserved":0,"allocated":11177984,"data_stored":7222109,"data_compressed":262820,"data_compressed_allocated":352256,"data_compressed_original":2203648,"omap_allocated":19131,"internal_metadata":369603909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2392,"num_objects":33,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":23,"num_read_kb":19,"num_write":72,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002287"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518839+0000","last_change":"2024-05-08T22:05:14.061628+0000","last_active":"2024-05-08T22:05:18.518839+0000","last_peered":"2024-05-08T22:05:18.518839+0000","last_clean":"2024-05-08T22:05:18.518839+0000","last_became_active":"2024-05-08T22:05:14.060986+0000","last_became_peered":"2024-05-08T22:05:14.060986+0000","last_unstale":"2024-05-08T22:05:18.518839+0000","last_undegraded":"2024-05-08T22:05:18.518839+0000","last_fullsized":"2024-05-08T22:05:18.518839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:51.643118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.523337+0000","last_change":"2024-05-08T22:05:14.073300+0000","last_active":"2024-05-08T22:05:18.523337+0000","last_peered":"2024-05-08T22:05:18.523337+0000","last_clean":"2024-05-08T22:05:18.523337+0000","last_became_active":"2024-05-08T22:05:14.069058+0000","last_became_peered":"2024-05-08T22:05:14.069058+0000","last_unstale":"2024-05-08T22:05:18.523337+0000","last_undegraded":"2024-05-08T22:05:18.523337+0000","last_fullsized":"2024-05-08T22:05:18.523337+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:15.834956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.523429+0000","last_change":"2024-05-08T22:05:14.071676+0000","last_active":"2024-05-08T22:05:18.523429+0000","last_peered":"2024-05-08T22:05:18.523429+0000","last_clean":"2024-05-08T22:05:18.523429+0000","last_became_active":"2024-05-08T22:05:14.058345+0000","last_became_peered":"2024-05-08T22:05:14.058345+0000","last_unstale":"2024-05-08T22:05:18.523429+0000","last_undegraded":"2024-05-08T22:05:18.523429+0000","last_fullsized":"2024-05-08T22:05:18.523429+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:09.239058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.019162+0000","last_change":"2024-05-08T22:05:14.067170+0000","last_active":"2024-05-08T22:05:19.019162+0000","last_peered":"2024-05-08T22:05:19.019162+0000","last_clean":"2024-05-08T22:05:19.019162+0000","last_became_active":"2024-05-08T22:05:14.066850+0000","last_became_peered":"2024-05-08T22:05:14.066850+0000","last_unstale":"2024-05-08T22:05:19.019162+0000","last_undegraded":"2024-05-08T22:05:19.019162+0000","last_fullsized":"2024-05-08T22:05:19.019162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:28.439786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829281+0000","last_change":"2024-05-08T22:05:14.065177+0000","last_active":"2024-05-08T22:05:40.829281+0000","last_peered":"2024-05-08T22:05:40.829281+0000","last_clean":"2024-05-08T22:05:40.829281+0000","last_became_active":"2024-05-08T22:05:14.064833+0000","last_became_peered":"2024-05-08T22:05:14.064833+0000","last_unstale":"2024-05-08T22:05:40.829281+0000","last_undegraded":"2024-05-08T22:05:40.829281+0000","last_fullsized":"2024-05-08T22:05:40.829281+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:11.764154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.615458+0000","last_change":"2024-05-08T22:05:14.061610+0000","last_active":"2024-05-08T22:05:18.615458+0000","last_peered":"2024-05-08T22:05:18.615458+0000","last_clean":"2024-05-08T22:05:18.615458+0000","last_became_active":"2024-05-08T22:05:14.061277+0000","last_became_peered":"2024-05-08T22:05:14.061277+0000","last_unstale":"2024-05-08T22:05:18.615458+0000","last_undegraded":"2024-05-08T22:05:18.615458+0000","last_fullsized":"2024-05-08T22:05:18.615458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:19.154980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.516423+0000","last_change":"2024-05-08T22:05:14.063941+0000","last_active":"2024-05-08T22:05:18.516423+0000","last_peered":"2024-05-08T22:05:18.516423+0000","last_clean":"2024-05-08T22:05:18.516423+0000","last_became_active":"2024-05-08T22:05:14.063579+0000","last_became_peered":"2024-05-08T22:05:14.063579+0000","last_unstale":"2024-05-08T22:05:18.516423+0000","last_undegraded":"2024-05-08T22:05:18.516423+0000","last_fullsized":"2024-05-08T22:05:18.516423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:27.679067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714348+0000","last_change":"2024-05-08T22:05:14.060366+0000","last_active":"2024-05-08T22:05:19.714348+0000","last_peered":"2024-05-08T22:05:19.714348+0000","last_clean":"2024-05-08T22:05:19.714348+0000","last_became_active":"2024-05-08T22:05:14.059818+0000","last_became_peered":"2024-05-08T22:05:14.059818+0000","last_unstale":"2024-05-08T22:05:19.714348+0000","last_undegraded":"2024-05-08T22:05:19.714348+0000","last_fullsized":"2024-05-08T22:05:19.714348+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:53.118508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518750+0000","last_change":"2024-05-08T22:05:14.057185+0000","last_active":"2024-05-08T22:05:18.518750+0000","last_peered":"2024-05-08T22:05:18.518750+0000","last_clean":"2024-05-08T22:05:18.518750+0000","last_became_active":"2024-05-08T22:05:14.056889+0000","last_became_peered":"2024-05-08T22:05:14.056889+0000","last_unstale":"2024-05-08T22:05:18.518750+0000","last_undegraded":"2024-05-08T22:05:18.518750+0000","last_fullsized":"2024-05-08T22:05:18.518750+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:35.372705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.616206+0000","last_change":"2024-05-08T22:05:14.062282+0000","last_active":"2024-05-08T22:05:18.616206+0000","last_peered":"2024-05-08T22:05:18.616206+0000","last_clean":"2024-05-08T22:05:18.616206+0000","last_became_active":"2024-05-08T22:05:14.061574+0000","last_became_peered":"2024-05-08T22:05:14.061574+0000","last_unstale":"2024-05-08T22:05:18.616206+0000","last_undegraded":"2024-05-08T22:05:18.616206+0000","last_fullsized":"2024-05-08T22:05:18.616206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:16:32.873590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.516691+0000","last_change":"2024-05-08T22:05:14.061267+0000","last_active":"2024-05-08T22:05:18.516691+0000","last_peered":"2024-05-08T22:05:18.516691+0000","last_clean":"2024-05-08T22:05:18.516691+0000","last_became_active":"2024-05-08T22:05:14.060943+0000","last_became_peered":"2024-05-08T22:05:14.060943+0000","last_unstale":"2024-05-08T22:05:18.516691+0000","last_undegraded":"2024-05-08T22:05:18.516691+0000","last_fullsized":"2024-05-08T22:05:18.516691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:03.831138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.019220+0000","last_change":"2024-05-08T22:05:14.067136+0000","last_active":"2024-05-08T22:05:19.019220+0000","last_peered":"2024-05-08T22:05:19.019220+0000","last_clean":"2024-05-08T22:05:19.019220+0000","last_became_active":"2024-05-08T22:05:14.066810+0000","last_became_peered":"2024-05-08T22:05:14.066810+0000","last_unstale":"2024-05-08T22:05:19.019220+0000","last_undegraded":"2024-05-08T22:05:19.019220+0000","last_fullsized":"2024-05-08T22:05:19.019220+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:43.001024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.154187+0000","last_change":"2024-05-08T22:05:14.062119+0000","last_active":"2024-05-08T22:05:19.154187+0000","last_peered":"2024-05-08T22:05:19.154187+0000","last_clean":"2024-05-08T22:05:19.154187+0000","last_became_active":"2024-05-08T22:05:14.058977+0000","last_became_peered":"2024-05-08T22:05:14.058977+0000","last_unstale":"2024-05-08T22:05:19.154187+0000","last_undegraded":"2024-05-08T22:05:19.154187+0000","last_fullsized":"2024-05-08T22:05:19.154187+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:03.657985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.519879+0000","last_change":"2024-05-08T22:05:14.067363+0000","last_active":"2024-05-08T22:05:18.519879+0000","last_peered":"2024-05-08T22:05:18.519879+0000","last_clean":"2024-05-08T22:05:18.519879+0000","last_became_active":"2024-05-08T22:05:14.066119+0000","last_became_peered":"2024-05-08T22:05:14.066119+0000","last_unstale":"2024-05-08T22:05:18.519879+0000","last_undegraded":"2024-05-08T22:05:18.519879+0000","last_fullsized":"2024-05-08T22:05:18.519879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:43.426750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.517625+0000","last_change":"2024-05-08T22:05:14.073925+0000","last_active":"2024-05-08T22:05:18.517625+0000","last_peered":"2024-05-08T22:05:18.517625+0000","last_clean":"2024-05-08T22:05:18.517625+0000","last_became_active":"2024-05-08T22:05:14.072287+0000","last_became_peered":"2024-05-08T22:05:14.072287+0000","last_unstale":"2024-05-08T22:05:18.517625+0000","last_undegraded":"2024-05-08T22:05:18.517625+0000","last_fullsized":"2024-05-08T22:05:18.517625+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:13.537938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.517166+0000","last_change":"2024-05-08T22:05:14.057274+0000","last_active":"2024-05-08T22:05:18.517166+0000","last_peered":"2024-05-08T22:05:18.517166+0000","last_clean":"2024-05-08T22:05:18.517166+0000","last_became_active":"2024-05-08T22:05:14.056989+0000","last_became_peered":"2024-05-08T22:05:14.056989+0000","last_unstale":"2024-05-08T22:05:18.517166+0000","last_undegraded":"2024-05-08T22:05:18.517166+0000","last_fullsized":"2024-05-08T22:05:18.517166+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:32.316406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518749+0000","last_change":"2024-05-08T22:05:14.069263+0000","last_active":"2024-05-08T22:05:18.518749+0000","last_peered":"2024-05-08T22:05:18.518749+0000","last_clean":"2024-05-08T22:05:18.518749+0000","last_became_active":"2024-05-08T22:05:14.068960+0000","last_became_peered":"2024-05-08T22:05:14.068960+0000","last_unstale":"2024-05-08T22:05:18.518749+0000","last_undegraded":"2024-05-08T22:05:18.518749+0000","last_fullsized":"2024-05-08T22:05:18.518749+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:59:42.829110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.514007+0000","last_change":"2024-05-08T22:05:14.049393+0000","last_active":"2024-05-08T22:05:18.514007+0000","last_peered":"2024-05-08T22:05:18.514007+0000","last_clean":"2024-05-08T22:05:18.514007+0000","last_became_active":"2024-05-08T22:05:14.049094+0000","last_became_peered":"2024-05-08T22:05:14.049094+0000","last_unstale":"2024-05-08T22:05:18.514007+0000","last_undegraded":"2024-05-08T22:05:18.514007+0000","last_fullsized":"2024-05-08T22:05:18.514007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:42.106568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.039735+0000","last_change":"2024-05-08T22:05:14.055436+0000","last_active":"2024-05-08T22:05:19.039735+0000","last_peered":"2024-05-08T22:05:19.039735+0000","last_clean":"2024-05-08T22:05:19.039735+0000","last_became_active":"2024-05-08T22:05:14.055100+0000","last_became_peered":"2024-05-08T22:05:14.055100+0000","last_unstale":"2024-05-08T22:05:19.039735+0000","last_undegraded":"2024-05-08T22:05:19.039735+0000","last_fullsized":"2024-05-08T22:05:19.039735+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:52.833780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.022998+0000","last_change":"2024-05-08T22:05:18.532086+0000","last_active":"2024-05-08T22:05:21.022998+0000","last_peered":"2024-05-08T22:05:21.022998+0000","last_clean":"2024-05-08T22:05:21.022998+0000","last_became_active":"2024-05-08T22:05:18.531485+0000","last_became_peered":"2024-05-08T22:05:18.531485+0000","last_unstale":"2024-05-08T22:05:21.022998+0000","last_undegraded":"2024-05-08T22:05:21.022998+0000","last_fullsized":"2024-05-08T22:05:21.022998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:40:36.050367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:38.499417+0000","last_change":"2024-05-08T22:05:13.549611+0000","last_active":"2024-05-08T22:05:38.499417+0000","last_peered":"2024-05-08T22:05:38.499417+0000","last_clean":"2024-05-08T22:05:38.499417+0000","last_became_active":"2024-05-08T22:05:13.549140+0000","last_became_peered":"2024-05-08T22:05:13.549140+0000","last_unstale":"2024-05-08T22:05:38.499417+0000","last_undegraded":"2024-05-08T22:05:38.499417+0000","last_fullsized":"2024-05-08T22:05:38.499417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:54.785123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714719+0000","last_change":"2024-05-08T22:05:14.057570+0000","last_active":"2024-05-08T22:05:19.714719+0000","last_peered":"2024-05-08T22:05:19.714719+0000","last_clean":"2024-05-08T22:05:19.714719+0000","last_became_active":"2024-05-08T22:05:14.057291+0000","last_became_peered":"2024-05-08T22:05:14.057291+0000","last_unstale":"2024-05-08T22:05:19.714719+0000","last_undegraded":"2024-05-08T22:05:19.714719+0000","last_fullsized":"2024-05-08T22:05:19.714719+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:15:38.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"peering","last_fresh":"2024-05-08T22:05:40.819123+0000","last_change":"2024-05-08T22:05:40.819370+0000","last_active":"2024-05-08T22:05:40.819123+0000","last_peered":"2024-05-08T22:05:21.804836+0000","last_clean":"2024-05-08T22:05:21.804836+0000","last_became_active":"2024-05-08T22:05:18.529966+0000","last_became_peered":"2024-05-08T22:05:18.529966+0000","last_unstale":"2024-05-08T22:05:40.819123+0000","last_undegraded":"2024-05-08T22:05:40.819123+0000","last_fullsized":"2024-05-08T22:05:40.819123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:28:51.858677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6,8,9],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:38.500116+0000","last_change":"2024-05-08T22:05:13.551351+0000","last_active":"2024-05-08T22:05:38.500116+0000","last_peered":"2024-05-08T22:05:38.500116+0000","last_clean":"2024-05-08T22:05:38.500116+0000","last_became_active":"2024-05-08T22:05:13.551014+0000","last_became_peered":"2024-05-08T22:05:13.551014+0000","last_unstale":"2024-05-08T22:05:38.500116+0000","last_undegraded":"2024-05-08T22:05:38.500116+0000","last_fullsized":"2024-05-08T22:05:38.500116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:07:22.405251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.525219+0000","last_change":"2024-05-08T22:05:14.067270+0000","last_active":"2024-05-08T22:05:18.525219+0000","last_peered":"2024-05-08T22:05:18.525219+0000","last_clean":"2024-05-08T22:05:18.525219+0000","last_became_active":"2024-05-08T22:05:14.065496+0000","last_became_peered":"2024-05-08T22:05:14.065496+0000","last_unstale":"2024-05-08T22:05:18.525219+0000","last_undegraded":"2024-05-08T22:05:18.525219+0000","last_fullsized":"2024-05-08T22:05:18.525219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:07.087880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.804877+0000","last_change":"2024-05-08T22:05:18.526557+0000","last_active":"2024-05-08T22:05:21.804877+0000","last_peered":"2024-05-08T22:05:21.804877+0000","last_clean":"2024-05-08T22:05:21.804877+0000","last_became_active":"2024-05-08T22:05:18.526129+0000","last_became_peered":"2024-05-08T22:05:18.526129+0000","last_unstale":"2024-05-08T22:05:21.804877+0000","last_undegraded":"2024-05-08T22:05:21.804877+0000","last_fullsized":"2024-05-08T22:05:21.804877+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:36.376579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"78'2","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.510733+0000","last_change":"2024-05-08T22:05:13.553630+0000","last_active":"2024-05-08T22:05:18.510733+0000","last_peered":"2024-05-08T22:05:18.510733+0000","last_clean":"2024-05-08T22:05:18.510733+0000","last_became_active":"2024-05-08T22:05:13.552879+0000","last_became_peered":"2024-05-08T22:05:13.552879+0000","last_unstale":"2024-05-08T22:05:18.510733+0000","last_undegraded":"2024-05-08T22:05:18.510733+0000","last_fullsized":"2024-05-08T22:05:18.510733+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:39:58.253305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.517225+0000","last_change":"2024-05-08T22:05:14.060089+0000","last_active":"2024-05-08T22:05:18.517225+0000","last_peered":"2024-05-08T22:05:18.517225+0000","last_clean":"2024-05-08T22:05:18.517225+0000","last_became_active":"2024-05-08T22:05:14.057620+0000","last_became_peered":"2024-05-08T22:05:14.057620+0000","last_unstale":"2024-05-08T22:05:18.517225+0000","last_undegraded":"2024-05-08T22:05:18.517225+0000","last_fullsized":"2024-05-08T22:05:18.517225+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:29:33.127510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.804864+0000","last_change":"2024-05-08T22:05:18.529746+0000","last_active":"2024-05-08T22:05:21.804864+0000","last_peered":"2024-05-08T22:05:21.804864+0000","last_clean":"2024-05-08T22:05:21.804864+0000","last_became_active":"2024-05-08T22:05:18.529097+0000","last_became_peered":"2024-05-08T22:05:18.529097+0000","last_unstale":"2024-05-08T22:05:21.804864+0000","last_undegraded":"2024-05-08T22:05:21.804864+0000","last_fullsized":"2024-05-08T22:05:21.804864+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:59:36.249787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.510546+0000","last_change":"2024-05-08T22:05:13.547070+0000","last_active":"2024-05-08T22:05:18.510546+0000","last_peered":"2024-05-08T22:05:18.510546+0000","last_clean":"2024-05-08T22:05:18.510546+0000","last_became_active":"2024-05-08T22:05:13.546708+0000","last_became_peered":"2024-05-08T22:05:13.546708+0000","last_unstale":"2024-05-08T22:05:18.510546+0000","last_undegraded":"2024-05-08T22:05:18.510546+0000","last_fullsized":"2024-05-08T22:05:18.510546+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:10.577900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.526371+0000","last_change":"2024-05-08T22:05:14.064695+0000","last_active":"2024-05-08T22:05:18.526371+0000","last_peered":"2024-05-08T22:05:18.526371+0000","last_clean":"2024-05-08T22:05:18.526371+0000","last_became_active":"2024-05-08T22:05:14.064283+0000","last_became_peered":"2024-05-08T22:05:14.064283+0000","last_unstale":"2024-05-08T22:05:18.526371+0000","last_undegraded":"2024-05-08T22:05:18.526371+0000","last_fullsized":"2024-05-08T22:05:18.526371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:06.980594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.023095+0000","last_change":"2024-05-08T22:05:18.528475+0000","last_active":"2024-05-08T22:05:21.023095+0000","last_peered":"2024-05-08T22:05:21.023095+0000","last_clean":"2024-05-08T22:05:21.023095+0000","last_became_active":"2024-05-08T22:05:18.528089+0000","last_became_peered":"2024-05-08T22:05:18.528089+0000","last_unstale":"2024-05-08T22:05:21.023095+0000","last_undegraded":"2024-05-08T22:05:21.023095+0000","last_fullsized":"2024-05-08T22:05:21.023095+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:01.508984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:39.502402+0000","last_change":"2024-05-08T22:05:13.549191+0000","last_active":"2024-05-08T22:05:39.502402+0000","last_peered":"2024-05-08T22:05:39.502402+0000","last_clean":"2024-05-08T22:05:39.502402+0000","last_became_active":"2024-05-08T22:05:13.548737+0000","last_became_peered":"2024-05-08T22:05:13.548737+0000","last_unstale":"2024-05-08T22:05:39.502402+0000","last_undegraded":"2024-05-08T22:05:39.502402+0000","last_fullsized":"2024-05-08T22:05:39.502402+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:07.355656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830198+0000","last_change":"2024-05-08T22:05:14.065910+0000","last_active":"2024-05-08T22:05:40.830198+0000","last_peered":"2024-05-08T22:05:40.830198+0000","last_clean":"2024-05-08T22:05:40.830198+0000","last_became_active":"2024-05-08T22:05:14.065504+0000","last_became_peered":"2024-05-08T22:05:14.065504+0000","last_unstale":"2024-05-08T22:05:40.830198+0000","last_undegraded":"2024-05-08T22:05:40.830198+0000","last_fullsized":"2024-05-08T22:05:40.830198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:12.447292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.023073+0000","last_change":"2024-05-08T22:05:18.536861+0000","last_active":"2024-05-08T22:05:21.023073+0000","last_peered":"2024-05-08T22:05:21.023073+0000","last_clean":"2024-05-08T22:05:21.023073+0000","last_became_active":"2024-05-08T22:05:18.536472+0000","last_became_peered":"2024-05-08T22:05:18.536472+0000","last_unstale":"2024-05-08T22:05:21.023073+0000","last_undegraded":"2024-05-08T22:05:21.023073+0000","last_fullsized":"2024-05-08T22:05:21.023073+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:45:21.826812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"78'1","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.615381+0000","last_change":"2024-05-08T22:05:13.553326+0000","last_active":"2024-05-08T22:05:18.615381+0000","last_peered":"2024-05-08T22:05:18.615381+0000","last_clean":"2024-05-08T22:05:18.615381+0000","last_became_active":"2024-05-08T22:05:13.552983+0000","last_became_peered":"2024-05-08T22:05:13.552983+0000","last_unstale":"2024-05-08T22:05:18.615381+0000","last_undegraded":"2024-05-08T22:05:18.615381+0000","last_fullsized":"2024-05-08T22:05:18.615381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:17.195253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.526142+0000","last_change":"2024-05-08T22:05:14.068617+0000","last_active":"2024-05-08T22:05:18.526142+0000","last_peered":"2024-05-08T22:05:18.526142+0000","last_clean":"2024-05-08T22:05:18.526142+0000","last_became_active":"2024-05-08T22:05:14.067400+0000","last_became_peered":"2024-05-08T22:05:14.067400+0000","last_unstale":"2024-05-08T22:05:18.526142+0000","last_undegraded":"2024-05-08T22:05:18.526142+0000","last_fullsized":"2024-05-08T22:05:18.526142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:25.488212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.018557+0000","last_change":"2024-05-08T22:05:18.538294+0000","last_active":"2024-05-08T22:05:19.018557+0000","last_peered":"2024-05-08T22:05:19.018557+0000","last_clean":"2024-05-08T22:05:19.018557+0000","last_became_active":"2024-05-08T22:05:18.537984+0000","last_became_peered":"2024-05-08T22:05:18.537984+0000","last_unstale":"2024-05-08T22:05:19.018557+0000","last_undegraded":"2024-05-08T22:05:19.018557+0000","last_fullsized":"2024-05-08T22:05:19.018557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:32.117439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:38.500252+0000","last_change":"2024-05-08T22:05:13.557669+0000","last_active":"2024-05-08T22:05:38.500252+0000","last_peered":"2024-05-08T22:05:38.500252+0000","last_clean":"2024-05-08T22:05:38.500252+0000","last_became_active":"2024-05-08T22:05:13.555835+0000","last_became_peered":"2024-05-08T22:05:13.555835+0000","last_unstale":"2024-05-08T22:05:38.500252+0000","last_undegraded":"2024-05-08T22:05:38.500252+0000","last_fullsized":"2024-05-08T22:05:38.500252+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:02.377635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714321+0000","last_change":"2024-05-08T22:05:14.060144+0000","last_active":"2024-05-08T22:05:19.714321+0000","last_peered":"2024-05-08T22:05:19.714321+0000","last_clean":"2024-05-08T22:05:19.714321+0000","last_became_active":"2024-05-08T22:05:14.059387+0000","last_became_peered":"2024-05-08T22:05:14.059387+0000","last_unstale":"2024-05-08T22:05:19.714321+0000","last_undegraded":"2024-05-08T22:05:19.714321+0000","last_fullsized":"2024-05-08T22:05:19.714321+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:32:28.057831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.154473+0000","last_change":"2024-05-08T22:05:18.527293+0000","last_active":"2024-05-08T22:05:19.154473+0000","last_peered":"2024-05-08T22:05:19.154473+0000","last_clean":"2024-05-08T22:05:19.154473+0000","last_became_active":"2024-05-08T22:05:18.526690+0000","last_became_peered":"2024-05-08T22:05:18.526690+0000","last_unstale":"2024-05-08T22:05:19.154473+0000","last_undegraded":"2024-05-08T22:05:19.154473+0000","last_fullsized":"2024-05-08T22:05:19.154473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:50.343536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:20.445763+0000","last_change":"2024-05-08T22:05:13.549774+0000","last_active":"2024-05-08T22:05:20.445763+0000","last_peered":"2024-05-08T22:05:20.445763+0000","last_clean":"2024-05-08T22:05:20.445763+0000","last_became_active":"2024-05-08T22:05:13.549531+0000","last_became_peered":"2024-05-08T22:05:13.549531+0000","last_unstale":"2024-05-08T22:05:20.445763+0000","last_undegraded":"2024-05-08T22:05:20.445763+0000","last_fullsized":"2024-05-08T22:05:20.445763+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:59:23.798728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.513387+0000","last_change":"2024-05-08T22:05:14.054232+0000","last_active":"2024-05-08T22:05:18.513387+0000","last_peered":"2024-05-08T22:05:18.513387+0000","last_clean":"2024-05-08T22:05:18.513387+0000","last_became_active":"2024-05-08T22:05:14.053806+0000","last_became_peered":"2024-05-08T22:05:14.053806+0000","last_unstale":"2024-05-08T22:05:18.513387+0000","last_undegraded":"2024-05-08T22:05:18.513387+0000","last_fullsized":"2024-05-08T22:05:18.513387+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:32:21.678792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.504963+0000","last_change":"2024-05-08T22:05:18.535241+0000","last_active":"2024-05-08T22:05:21.504963+0000","last_peered":"2024-05-08T22:05:21.504963+0000","last_clean":"2024-05-08T22:05:21.504963+0000","last_became_active":"2024-05-08T22:05:18.534753+0000","last_became_peered":"2024-05-08T22:05:18.534753+0000","last_unstale":"2024-05-08T22:05:21.504963+0000","last_undegraded":"2024-05-08T22:05:21.504963+0000","last_fullsized":"2024-05-08T22:05:21.504963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:46:15.138437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.493918+0000","last_change":"2024-05-08T22:05:13.544899+0000","last_active":"2024-05-08T22:05:37.493918+0000","last_peered":"2024-05-08T22:05:37.493918+0000","last_clean":"2024-05-08T22:05:37.493918+0000","last_became_active":"2024-05-08T22:05:13.544498+0000","last_became_peered":"2024-05-08T22:05:13.544498+0000","last_unstale":"2024-05-08T22:05:37.493918+0000","last_undegraded":"2024-05-08T22:05:37.493918+0000","last_fullsized":"2024-05-08T22:05:37.493918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:58.169648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518687+0000","last_change":"2024-05-08T22:05:14.049377+0000","last_active":"2024-05-08T22:05:18.518687+0000","last_peered":"2024-05-08T22:05:18.518687+0000","last_clean":"2024-05-08T22:05:18.518687+0000","last_became_active":"2024-05-08T22:05:14.049039+0000","last_became_peered":"2024-05-08T22:05:14.049039+0000","last_unstale":"2024-05-08T22:05:18.518687+0000","last_undegraded":"2024-05-08T22:05:18.518687+0000","last_fullsized":"2024-05-08T22:05:18.518687+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:24.415392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.017866+0000","last_change":"2024-05-08T22:05:18.528912+0000","last_active":"2024-05-08T22:05:19.017866+0000","last_peered":"2024-05-08T22:05:19.017866+0000","last_clean":"2024-05-08T22:05:19.017866+0000","last_became_active":"2024-05-08T22:05:18.528433+0000","last_became_peered":"2024-05-08T22:05:18.528433+0000","last_unstale":"2024-05-08T22:05:19.017866+0000","last_undegraded":"2024-05-08T22:05:19.017866+0000","last_fullsized":"2024-05-08T22:05:19.017866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:05.574728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.492321+0000","last_change":"2024-05-08T22:05:13.550214+0000","last_active":"2024-05-08T22:05:37.492321+0000","last_peered":"2024-05-08T22:05:37.492321+0000","last_clean":"2024-05-08T22:05:37.492321+0000","last_became_active":"2024-05-08T22:05:13.549717+0000","last_became_peered":"2024-05-08T22:05:13.549717+0000","last_unstale":"2024-05-08T22:05:37.492321+0000","last_undegraded":"2024-05-08T22:05:37.492321+0000","last_fullsized":"2024-05-08T22:05:37.492321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:12:20.406333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.525310+0000","last_change":"2024-05-08T22:05:14.071214+0000","last_active":"2024-05-08T22:05:18.525310+0000","last_peered":"2024-05-08T22:05:18.525310+0000","last_clean":"2024-05-08T22:05:18.525310+0000","last_became_active":"2024-05-08T22:05:14.066777+0000","last_became_peered":"2024-05-08T22:05:14.066777+0000","last_unstale":"2024-05-08T22:05:18.525310+0000","last_undegraded":"2024-05-08T22:05:18.525310+0000","last_fullsized":"2024-05-08T22:05:18.525310+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:53.360911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.804896+0000","last_change":"2024-05-08T22:05:18.523688+0000","last_active":"2024-05-08T22:05:21.804896+0000","last_peered":"2024-05-08T22:05:21.804896+0000","last_clean":"2024-05-08T22:05:21.804896+0000","last_became_active":"2024-05-08T22:05:18.523236+0000","last_became_peered":"2024-05-08T22:05:18.523236+0000","last_unstale":"2024-05-08T22:05:21.804896+0000","last_undegraded":"2024-05-08T22:05:21.804896+0000","last_fullsized":"2024-05-08T22:05:21.804896+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:34.531699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.494031+0000","last_change":"2024-05-08T22:05:13.551027+0000","last_active":"2024-05-08T22:05:37.494031+0000","last_peered":"2024-05-08T22:05:37.494031+0000","last_clean":"2024-05-08T22:05:37.494031+0000","last_became_active":"2024-05-08T22:05:13.550471+0000","last_became_peered":"2024-05-08T22:05:13.550471+0000","last_unstale":"2024-05-08T22:05:37.494031+0000","last_undegraded":"2024-05-08T22:05:37.494031+0000","last_fullsized":"2024-05-08T22:05:37.494031+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:50.900419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.517424+0000","last_change":"2024-05-08T22:05:14.065191+0000","last_active":"2024-05-08T22:05:18.517424+0000","last_peered":"2024-05-08T22:05:18.517424+0000","last_clean":"2024-05-08T22:05:18.517424+0000","last_became_active":"2024-05-08T22:05:14.064893+0000","last_became_peered":"2024-05-08T22:05:14.064893+0000","last_unstale":"2024-05-08T22:05:18.517424+0000","last_undegraded":"2024-05-08T22:05:18.517424+0000","last_fullsized":"2024-05-08T22:05:18.517424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:11.344646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.017892+0000","last_change":"2024-05-08T22:05:18.530447+0000","last_active":"2024-05-08T22:05:19.017892+0000","last_peered":"2024-05-08T22:05:19.017892+0000","last_clean":"2024-05-08T22:05:19.017892+0000","last_became_active":"2024-05-08T22:05:18.528171+0000","last_became_peered":"2024-05-08T22:05:18.528171+0000","last_unstale":"2024-05-08T22:05:19.017892+0000","last_undegraded":"2024-05-08T22:05:19.017892+0000","last_fullsized":"2024-05-08T22:05:19.017892+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:09:43.347366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"83'12","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.493215+0000","last_change":"2024-05-08T22:05:13.547289+0000","last_active":"2024-05-08T22:05:37.493215+0000","last_peered":"2024-05-08T22:05:37.493215+0000","last_clean":"2024-05-08T22:05:37.493215+0000","last_became_active":"2024-05-08T22:05:13.546860+0000","last_became_peered":"2024-05-08T22:05:13.546860+0000","last_unstale":"2024-05-08T22:05:37.493215+0000","last_undegraded":"2024-05-08T22:05:37.493215+0000","last_fullsized":"2024-05-08T22:05:37.493215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:08:50.339361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831151+0000","last_change":"2024-05-08T22:05:14.057925+0000","last_active":"2024-05-08T22:05:40.831151+0000","last_peered":"2024-05-08T22:05:40.831151+0000","last_clean":"2024-05-08T22:05:40.831151+0000","last_became_active":"2024-05-08T22:05:14.057662+0000","last_became_peered":"2024-05-08T22:05:14.057662+0000","last_unstale":"2024-05-08T22:05:40.831151+0000","last_undegraded":"2024-05-08T22:05:40.831151+0000","last_fullsized":"2024-05-08T22:05:40.831151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:49.929610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.834134+0000","last_change":"2024-05-08T22:05:18.535509+0000","last_active":"2024-05-08T22:05:19.834134+0000","last_peered":"2024-05-08T22:05:19.834134+0000","last_clean":"2024-05-08T22:05:19.834134+0000","last_became_active":"2024-05-08T22:05:18.535024+0000","last_became_peered":"2024-05-08T22:05:18.535024+0000","last_unstale":"2024-05-08T22:05:19.834134+0000","last_undegraded":"2024-05-08T22:05:19.834134+0000","last_fullsized":"2024-05-08T22:05:19.834134+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:28.098737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:39.501569+0000","last_change":"2024-05-08T22:05:13.552096+0000","last_active":"2024-05-08T22:05:39.501569+0000","last_peered":"2024-05-08T22:05:39.501569+0000","last_clean":"2024-05-08T22:05:39.501569+0000","last_became_active":"2024-05-08T22:05:13.551655+0000","last_became_peered":"2024-05-08T22:05:13.551655+0000","last_unstale":"2024-05-08T22:05:39.501569+0000","last_undegraded":"2024-05-08T22:05:39.501569+0000","last_fullsized":"2024-05-08T22:05:39.501569+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:24.371008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.153911+0000","last_change":"2024-05-08T22:05:14.066075+0000","last_active":"2024-05-08T22:05:19.153911+0000","last_peered":"2024-05-08T22:05:19.153911+0000","last_clean":"2024-05-08T22:05:19.153911+0000","last_became_active":"2024-05-08T22:05:14.065581+0000","last_became_peered":"2024-05-08T22:05:14.065581+0000","last_unstale":"2024-05-08T22:05:19.153911+0000","last_undegraded":"2024-05-08T22:05:19.153911+0000","last_fullsized":"2024-05-08T22:05:19.153911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:15.403714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.018254+0000","last_change":"2024-05-08T22:05:18.534033+0000","last_active":"2024-05-08T22:05:19.018254+0000","last_peered":"2024-05-08T22:05:19.018254+0000","last_clean":"2024-05-08T22:05:19.018254+0000","last_became_active":"2024-05-08T22:05:18.533563+0000","last_became_peered":"2024-05-08T22:05:18.533563+0000","last_unstale":"2024-05-08T22:05:19.018254+0000","last_undegraded":"2024-05-08T22:05:19.018254+0000","last_fullsized":"2024-05-08T22:05:19.018254+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:37:52.657794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.493496+0000","last_change":"2024-05-08T22:05:13.549112+0000","last_active":"2024-05-08T22:05:37.493496+0000","last_peered":"2024-05-08T22:05:37.493496+0000","last_clean":"2024-05-08T22:05:37.493496+0000","last_became_active":"2024-05-08T22:05:13.548763+0000","last_became_peered":"2024-05-08T22:05:13.548763+0000","last_unstale":"2024-05-08T22:05:37.493496+0000","last_undegraded":"2024-05-08T22:05:37.493496+0000","last_fullsized":"2024-05-08T22:05:37.493496+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:26:56.448611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.516056+0000","last_change":"2024-05-08T22:05:14.067116+0000","last_active":"2024-05-08T22:05:18.516056+0000","last_peered":"2024-05-08T22:05:18.516056+0000","last_clean":"2024-05-08T22:05:18.516056+0000","last_became_active":"2024-05-08T22:05:14.064775+0000","last_became_peered":"2024-05-08T22:05:14.064775+0000","last_unstale":"2024-05-08T22:05:18.516056+0000","last_undegraded":"2024-05-08T22:05:18.516056+0000","last_fullsized":"2024-05-08T22:05:18.516056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:28:32.901405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.018404+0000","last_change":"2024-05-08T22:05:18.539546+0000","last_active":"2024-05-08T22:05:19.018404+0000","last_peered":"2024-05-08T22:05:19.018404+0000","last_clean":"2024-05-08T22:05:19.018404+0000","last_became_active":"2024-05-08T22:05:18.538998+0000","last_became_peered":"2024-05-08T22:05:18.538998+0000","last_unstale":"2024-05-08T22:05:19.018404+0000","last_undegraded":"2024-05-08T22:05:19.018404+0000","last_fullsized":"2024-05-08T22:05:19.018404+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:30.453881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"83'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819019+0000","last_change":"2024-05-08T22:05:13.549053+0000","last_active":"2024-05-08T22:05:40.819019+0000","last_peered":"2024-05-08T22:05:40.819019+0000","last_clean":"2024-05-08T22:05:40.819019+0000","last_became_active":"2024-05-08T22:05:13.548739+0000","last_became_peered":"2024-05-08T22:05:13.548739+0000","last_unstale":"2024-05-08T22:05:40.819019+0000","last_undegraded":"2024-05-08T22:05:40.819019+0000","last_fullsized":"2024-05-08T22:05:40.819019+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:07:10.298084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.516222+0000","last_change":"2024-05-08T22:05:14.061753+0000","last_active":"2024-05-08T22:05:18.516222+0000","last_peered":"2024-05-08T22:05:18.516222+0000","last_clean":"2024-05-08T22:05:18.516222+0000","last_became_active":"2024-05-08T22:05:14.061412+0000","last_became_peered":"2024-05-08T22:05:14.061412+0000","last_unstale":"2024-05-08T22:05:18.516222+0000","last_undegraded":"2024-05-08T22:05:18.516222+0000","last_fullsized":"2024-05-08T22:05:18.516222+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:54.380181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.718981+0000","last_change":"2024-05-08T22:05:18.532627+0000","last_active":"2024-05-08T22:05:21.718981+0000","last_peered":"2024-05-08T22:05:21.718981+0000","last_clean":"2024-05-08T22:05:21.718981+0000","last_became_active":"2024-05-08T22:05:18.532205+0000","last_became_peered":"2024-05-08T22:05:18.532205+0000","last_unstale":"2024-05-08T22:05:21.718981+0000","last_undegraded":"2024-05-08T22:05:21.718981+0000","last_fullsized":"2024-05-08T22:05:21.718981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:20:11.028645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":61,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.841832+0000","last_change":"2024-05-08T22:05:41.841832+0000","last_active":"2024-05-08T22:05:41.841832+0000","last_peered":"2024-05-08T22:05:41.841832+0000","last_clean":"2024-05-08T22:05:41.841832+0000","last_became_active":"2024-05-08T22:05:41.829584+0000","last_became_peered":"2024-05-08T22:05:41.829584+0000","last_unstale":"2024-05-08T22:05:41.841832+0000","last_undegraded":"2024-05-08T22:05:41.841832+0000","last_fullsized":"2024-05-08T22:05:41.841832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:52:19.876215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.525104+0000","last_change":"2024-05-08T22:05:14.063027+0000","last_active":"2024-05-08T22:05:18.525104+0000","last_peered":"2024-05-08T22:05:18.525104+0000","last_clean":"2024-05-08T22:05:18.525104+0000","last_became_active":"2024-05-08T22:05:14.062682+0000","last_became_peered":"2024-05-08T22:05:14.062682+0000","last_unstale":"2024-05-08T22:05:18.525104+0000","last_undegraded":"2024-05-08T22:05:18.525104+0000","last_fullsized":"2024-05-08T22:05:18.525104+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:47.248832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.834071+0000","last_change":"2024-05-08T22:05:18.528370+0000","last_active":"2024-05-08T22:05:19.834071+0000","last_peered":"2024-05-08T22:05:19.834071+0000","last_clean":"2024-05-08T22:05:19.834071+0000","last_became_active":"2024-05-08T22:05:18.527990+0000","last_became_peered":"2024-05-08T22:05:18.527990+0000","last_unstale":"2024-05-08T22:05:19.834071+0000","last_undegraded":"2024-05-08T22:05:19.834071+0000","last_fullsized":"2024-05-08T22:05:19.834071+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:07.524175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829860+0000","last_change":"2024-05-08T22:05:13.547543+0000","last_active":"2024-05-08T22:05:40.829860+0000","last_peered":"2024-05-08T22:05:40.829860+0000","last_clean":"2024-05-08T22:05:40.829860+0000","last_became_active":"2024-05-08T22:05:13.547191+0000","last_became_peered":"2024-05-08T22:05:13.547191+0000","last_unstale":"2024-05-08T22:05:40.829860+0000","last_undegraded":"2024-05-08T22:05:40.829860+0000","last_fullsized":"2024-05-08T22:05:40.829860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:53.213239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518149+0000","last_change":"2024-05-08T22:05:14.062677+0000","last_active":"2024-05-08T22:05:18.518149+0000","last_peered":"2024-05-08T22:05:18.518149+0000","last_clean":"2024-05-08T22:05:18.518149+0000","last_became_active":"2024-05-08T22:05:14.062306+0000","last_became_peered":"2024-05-08T22:05:14.062306+0000","last_unstale":"2024-05-08T22:05:18.518149+0000","last_undegraded":"2024-05-08T22:05:18.518149+0000","last_fullsized":"2024-05-08T22:05:18.518149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:00.356505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830129+0000","last_change":"2024-05-08T22:05:18.617075+0000","last_active":"2024-05-08T22:05:40.830129+0000","last_peered":"2024-05-08T22:05:40.830129+0000","last_clean":"2024-05-08T22:05:40.830129+0000","last_became_active":"2024-05-08T22:05:18.616653+0000","last_became_peered":"2024-05-08T22:05:18.616653+0000","last_unstale":"2024-05-08T22:05:40.830129+0000","last_undegraded":"2024-05-08T22:05:40.830129+0000","last_fullsized":"2024-05-08T22:05:40.830129+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:30.537064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.513484+0000","last_change":"2024-05-08T22:05:13.553457+0000","last_active":"2024-05-08T22:05:18.513484+0000","last_peered":"2024-05-08T22:05:18.513484+0000","last_clean":"2024-05-08T22:05:18.513484+0000","last_became_active":"2024-05-08T22:05:13.551583+0000","last_became_peered":"2024-05-08T22:05:13.551583+0000","last_unstale":"2024-05-08T22:05:18.513484+0000","last_undegraded":"2024-05-08T22:05:18.513484+0000","last_fullsized":"2024-05-08T22:05:18.513484+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:07:05.971413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.616101+0000","last_change":"2024-05-08T22:05:14.057476+0000","last_active":"2024-05-08T22:05:18.616101+0000","last_peered":"2024-05-08T22:05:18.616101+0000","last_clean":"2024-05-08T22:05:18.616101+0000","last_became_active":"2024-05-08T22:05:14.056550+0000","last_became_peered":"2024-05-08T22:05:14.056550+0000","last_unstale":"2024-05-08T22:05:18.616101+0000","last_undegraded":"2024-05-08T22:05:18.616101+0000","last_fullsized":"2024-05-08T22:05:18.616101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:41:29.264096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.718898+0000","last_change":"2024-05-08T22:05:18.530719+0000","last_active":"2024-05-08T22:05:21.718898+0000","last_peered":"2024-05-08T22:05:21.718898+0000","last_clean":"2024-05-08T22:05:21.718898+0000","last_became_active":"2024-05-08T22:05:18.529659+0000","last_became_peered":"2024-05-08T22:05:18.529659+0000","last_unstale":"2024-05-08T22:05:21.718898+0000","last_undegraded":"2024-05-08T22:05:21.718898+0000","last_fullsized":"2024-05-08T22:05:21.718898+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:13:06.514460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:20.446632+0000","last_change":"2024-05-08T22:05:13.556191+0000","last_active":"2024-05-08T22:05:20.446632+0000","last_peered":"2024-05-08T22:05:20.446632+0000","last_clean":"2024-05-08T22:05:20.446632+0000","last_became_active":"2024-05-08T22:05:13.555888+0000","last_became_peered":"2024-05-08T22:05:13.555888+0000","last_unstale":"2024-05-08T22:05:20.446632+0000","last_undegraded":"2024-05-08T22:05:20.446632+0000","last_fullsized":"2024-05-08T22:05:20.446632+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:05.637980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.615673+0000","last_change":"2024-05-08T22:05:14.069974+0000","last_active":"2024-05-08T22:05:18.615673+0000","last_peered":"2024-05-08T22:05:18.615673+0000","last_clean":"2024-05-08T22:05:18.615673+0000","last_became_active":"2024-05-08T22:05:14.069637+0000","last_became_peered":"2024-05-08T22:05:14.069637+0000","last_unstale":"2024-05-08T22:05:18.615673+0000","last_undegraded":"2024-05-08T22:05:18.615673+0000","last_fullsized":"2024-05-08T22:05:18.615673+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:15:03.030292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"peering","last_fresh":"2024-05-08T22:05:40.825443+0000","last_change":"2024-05-08T22:05:40.825855+0000","last_active":"2024-05-08T22:05:40.825443+0000","last_peered":"2024-05-08T22:05:21.719003+0000","last_clean":"2024-05-08T22:05:21.719003+0000","last_became_active":"2024-05-08T22:05:18.616536+0000","last_became_peered":"2024-05-08T22:05:18.616536+0000","last_unstale":"2024-05-08T22:05:40.825443+0000","last_undegraded":"2024-05-08T22:05:40.825443+0000","last_fullsized":"2024-05-08T22:05:40.825443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:30:44.106162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5,8,10],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.525034+0000","last_change":"2024-05-08T22:05:14.069096+0000","last_active":"2024-05-08T22:05:18.525034+0000","last_peered":"2024-05-08T22:05:18.525034+0000","last_clean":"2024-05-08T22:05:18.525034+0000","last_became_active":"2024-05-08T22:05:14.068796+0000","last_became_peered":"2024-05-08T22:05:14.068796+0000","last_unstale":"2024-05-08T22:05:18.525034+0000","last_undegraded":"2024-05-08T22:05:18.525034+0000","last_fullsized":"2024-05-08T22:05:18.525034+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:52:52.391196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.821076+0000","last_change":"2024-05-08T22:05:13.559286+0000","last_active":"2024-05-08T22:05:40.821076+0000","last_peered":"2024-05-08T22:05:40.821076+0000","last_clean":"2024-05-08T22:05:40.821076+0000","last_became_active":"2024-05-08T22:05:13.558922+0000","last_became_peered":"2024-05-08T22:05:13.558922+0000","last_unstale":"2024-05-08T22:05:40.821076+0000","last_undegraded":"2024-05-08T22:05:40.821076+0000","last_fullsized":"2024-05-08T22:05:40.821076+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:25:24.732422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.566174+0000","last_change":"2024-05-08T22:05:18.527031+0000","last_active":"2024-05-08T22:05:18.566174+0000","last_peered":"2024-05-08T22:05:18.566174+0000","last_clean":"2024-05-08T22:05:18.566174+0000","last_became_active":"2024-05-08T22:05:18.526573+0000","last_became_peered":"2024-05-08T22:05:18.526573+0000","last_unstale":"2024-05-08T22:05:18.566174+0000","last_undegraded":"2024-05-08T22:05:18.566174+0000","last_fullsized":"2024-05-08T22:05:18.566174+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:08.492359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714236+0000","last_change":"2024-05-08T22:05:14.053930+0000","last_active":"2024-05-08T22:05:19.714236+0000","last_peered":"2024-05-08T22:05:19.714236+0000","last_clean":"2024-05-08T22:05:19.714236+0000","last_became_active":"2024-05-08T22:05:14.052968+0000","last_became_peered":"2024-05-08T22:05:14.052968+0000","last_unstale":"2024-05-08T22:05:19.714236+0000","last_undegraded":"2024-05-08T22:05:19.714236+0000","last_fullsized":"2024-05-08T22:05:19.714236+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:48.748323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:39.501541+0000","last_change":"2024-05-08T22:05:13.545709+0000","last_active":"2024-05-08T22:05:39.501541+0000","last_peered":"2024-05-08T22:05:39.501541+0000","last_clean":"2024-05-08T22:05:39.501541+0000","last_became_active":"2024-05-08T22:05:13.545419+0000","last_became_peered":"2024-05-08T22:05:13.545419+0000","last_unstale":"2024-05-08T22:05:39.501541+0000","last_undegraded":"2024-05-08T22:05:39.501541+0000","last_fullsized":"2024-05-08T22:05:39.501541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:44.903903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":335,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:20.446030+0000","last_change":"2024-05-08T22:04:20.948328+0000","last_active":"2024-05-08T22:05:20.446030+0000","last_peered":"2024-05-08T22:05:20.446030+0000","last_clean":"2024-05-08T22:05:20.446030+0000","last_became_active":"2024-05-08T22:04:20.947887+0000","last_became_peered":"2024-05-08T22:04:20.947887+0000","last_unstale":"2024-05-08T22:05:20.446030+0000","last_undegraded":"2024-05-08T22:05:20.446030+0000","last_fullsized":"2024-05-08T22:05:20.446030+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:01:41.485277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:01:41.485277+0000","last_clean_scrub_stamp":"2024-05-08T22:01:41.485277+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:43:59.208120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.834146+0000","last_change":"2024-05-08T22:05:18.616989+0000","last_active":"2024-05-08T22:05:19.834146+0000","last_peered":"2024-05-08T22:05:19.834146+0000","last_clean":"2024-05-08T22:05:19.834146+0000","last_became_active":"2024-05-08T22:05:18.616560+0000","last_became_peered":"2024-05-08T22:05:18.616560+0000","last_unstale":"2024-05-08T22:05:19.834146+0000","last_undegraded":"2024-05-08T22:05:19.834146+0000","last_fullsized":"2024-05-08T22:05:19.834146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:45:31.141303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.153692+0000","last_change":"2024-05-08T22:05:14.066818+0000","last_active":"2024-05-08T22:05:19.153692+0000","last_peered":"2024-05-08T22:05:19.153692+0000","last_clean":"2024-05-08T22:05:19.153692+0000","last_became_active":"2024-05-08T22:05:14.065152+0000","last_became_peered":"2024-05-08T22:05:14.065152+0000","last_unstale":"2024-05-08T22:05:19.153692+0000","last_undegraded":"2024-05-08T22:05:19.153692+0000","last_fullsized":"2024-05-08T22:05:19.153692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:53:19.018852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:38.498880+0000","last_change":"2024-05-08T22:05:13.550365+0000","last_active":"2024-05-08T22:05:38.498880+0000","last_peered":"2024-05-08T22:05:38.498880+0000","last_clean":"2024-05-08T22:05:38.498880+0000","last_became_active":"2024-05-08T22:05:13.549677+0000","last_became_peered":"2024-05-08T22:05:13.549677+0000","last_unstale":"2024-05-08T22:05:38.498880+0000","last_undegraded":"2024-05-08T22:05:38.498880+0000","last_fullsized":"2024-05-08T22:05:38.498880+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:10.842838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.018502+0000","last_change":"2024-05-08T22:05:18.535777+0000","last_active":"2024-05-08T22:05:19.018502+0000","last_peered":"2024-05-08T22:05:19.018502+0000","last_clean":"2024-05-08T22:05:19.018502+0000","last_became_active":"2024-05-08T22:05:18.535327+0000","last_became_peered":"2024-05-08T22:05:18.535327+0000","last_unstale":"2024-05-08T22:05:19.018502+0000","last_undegraded":"2024-05-08T22:05:19.018502+0000","last_fullsized":"2024-05-08T22:05:19.018502+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:13.886989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831170+0000","last_change":"2024-05-08T22:05:13.554045+0000","last_active":"2024-05-08T22:05:40.831170+0000","last_peered":"2024-05-08T22:05:40.831170+0000","last_clean":"2024-05-08T22:05:40.831170+0000","last_became_active":"2024-05-08T22:05:13.553315+0000","last_became_peered":"2024-05-08T22:05:13.553315+0000","last_unstale":"2024-05-08T22:05:40.831170+0000","last_undegraded":"2024-05-08T22:05:40.831170+0000","last_fullsized":"2024-05-08T22:05:40.831170+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:13.131267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.154279+0000","last_change":"2024-05-08T22:05:14.065018+0000","last_active":"2024-05-08T22:05:19.154279+0000","last_peered":"2024-05-08T22:05:19.154279+0000","last_clean":"2024-05-08T22:05:19.154279+0000","last_became_active":"2024-05-08T22:05:14.064701+0000","last_became_peered":"2024-05-08T22:05:14.064701+0000","last_unstale":"2024-05-08T22:05:19.154279+0000","last_undegraded":"2024-05-08T22:05:19.154279+0000","last_fullsized":"2024-05-08T22:05:19.154279+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:48:43.468373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.504924+0000","last_change":"2024-05-08T22:05:18.617853+0000","last_active":"2024-05-08T22:05:21.504924+0000","last_peered":"2024-05-08T22:05:21.504924+0000","last_clean":"2024-05-08T22:05:21.504924+0000","last_became_active":"2024-05-08T22:05:18.616255+0000","last_became_peered":"2024-05-08T22:05:18.616255+0000","last_unstale":"2024-05-08T22:05:21.504924+0000","last_undegraded":"2024-05-08T22:05:21.504924+0000","last_fullsized":"2024-05-08T22:05:21.504924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:13.233425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:38.499114+0000","last_change":"2024-05-08T22:05:13.546274+0000","last_active":"2024-05-08T22:05:38.499114+0000","last_peered":"2024-05-08T22:05:38.499114+0000","last_clean":"2024-05-08T22:05:38.499114+0000","last_became_active":"2024-05-08T22:05:13.543651+0000","last_became_peered":"2024-05-08T22:05:13.543651+0000","last_unstale":"2024-05-08T22:05:38.499114+0000","last_undegraded":"2024-05-08T22:05:38.499114+0000","last_fullsized":"2024-05-08T22:05:38.499114+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:21:58.601261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714604+0000","last_change":"2024-05-08T22:05:14.056535+0000","last_active":"2024-05-08T22:05:19.714604+0000","last_peered":"2024-05-08T22:05:19.714604+0000","last_clean":"2024-05-08T22:05:19.714604+0000","last_became_active":"2024-05-08T22:05:14.056211+0000","last_became_peered":"2024-05-08T22:05:14.056211+0000","last_unstale":"2024-05-08T22:05:19.714604+0000","last_undegraded":"2024-05-08T22:05:19.714604+0000","last_fullsized":"2024-05-08T22:05:19.714604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:46:30.744490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.022982+0000","last_change":"2024-05-08T22:05:18.524939+0000","last_active":"2024-05-08T22:05:21.022982+0000","last_peered":"2024-05-08T22:05:21.022982+0000","last_clean":"2024-05-08T22:05:21.022982+0000","last_became_active":"2024-05-08T22:05:18.524518+0000","last_became_peered":"2024-05-08T22:05:18.524518+0000","last_unstale":"2024-05-08T22:05:21.022982+0000","last_undegraded":"2024-05-08T22:05:21.022982+0000","last_fullsized":"2024-05-08T22:05:21.022982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:50.192400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.493290+0000","last_change":"2024-05-08T22:05:13.554346+0000","last_active":"2024-05-08T22:05:37.493290+0000","last_peered":"2024-05-08T22:05:37.493290+0000","last_clean":"2024-05-08T22:05:37.493290+0000","last_became_active":"2024-05-08T22:05:13.554080+0000","last_became_peered":"2024-05-08T22:05:13.554080+0000","last_unstale":"2024-05-08T22:05:37.493290+0000","last_undegraded":"2024-05-08T22:05:37.493290+0000","last_fullsized":"2024-05-08T22:05:37.493290+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:29:49.695682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.523372+0000","last_change":"2024-05-08T22:05:14.071422+0000","last_active":"2024-05-08T22:05:18.523372+0000","last_peered":"2024-05-08T22:05:18.523372+0000","last_clean":"2024-05-08T22:05:18.523372+0000","last_became_active":"2024-05-08T22:05:14.070396+0000","last_became_peered":"2024-05-08T22:05:14.070396+0000","last_unstale":"2024-05-08T22:05:18.523372+0000","last_undegraded":"2024-05-08T22:05:18.523372+0000","last_fullsized":"2024-05-08T22:05:18.523372+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:37.957983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.804960+0000","last_change":"2024-05-08T22:05:18.617149+0000","last_active":"2024-05-08T22:05:21.804960+0000","last_peered":"2024-05-08T22:05:21.804960+0000","last_clean":"2024-05-08T22:05:21.804960+0000","last_became_active":"2024-05-08T22:05:18.616699+0000","last_became_peered":"2024-05-08T22:05:18.616699+0000","last_unstale":"2024-05-08T22:05:21.804960+0000","last_undegraded":"2024-05-08T22:05:21.804960+0000","last_fullsized":"2024-05-08T22:05:21.804960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:51.167142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.492588+0000","last_change":"2024-05-08T22:05:13.551212+0000","last_active":"2024-05-08T22:05:37.492588+0000","last_peered":"2024-05-08T22:05:37.492588+0000","last_clean":"2024-05-08T22:05:37.492588+0000","last_became_active":"2024-05-08T22:05:13.550909+0000","last_became_peered":"2024-05-08T22:05:13.550909+0000","last_unstale":"2024-05-08T22:05:37.492588+0000","last_undegraded":"2024-05-08T22:05:37.492588+0000","last_fullsized":"2024-05-08T22:05:37.492588+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:48.657928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829703+0000","last_change":"2024-05-08T22:05:14.053976+0000","last_active":"2024-05-08T22:05:40.829703+0000","last_peered":"2024-05-08T22:05:40.829703+0000","last_clean":"2024-05-08T22:05:40.829703+0000","last_became_active":"2024-05-08T22:05:14.053664+0000","last_became_peered":"2024-05-08T22:05:14.053664+0000","last_unstale":"2024-05-08T22:05:40.829703+0000","last_undegraded":"2024-05-08T22:05:40.829703+0000","last_fullsized":"2024-05-08T22:05:40.829703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:25:25.642768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.018493+0000","last_change":"2024-05-08T22:05:18.536771+0000","last_active":"2024-05-08T22:05:19.018493+0000","last_peered":"2024-05-08T22:05:19.018493+0000","last_clean":"2024-05-08T22:05:19.018493+0000","last_became_active":"2024-05-08T22:05:18.536260+0000","last_became_peered":"2024-05-08T22:05:18.536260+0000","last_unstale":"2024-05-08T22:05:19.018493+0000","last_undegraded":"2024-05-08T22:05:19.018493+0000","last_fullsized":"2024-05-08T22:05:19.018493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:34.414381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.494220+0000","last_change":"2024-05-08T22:05:13.540679+0000","last_active":"2024-05-08T22:05:37.494220+0000","last_peered":"2024-05-08T22:05:37.494220+0000","last_clean":"2024-05-08T22:05:37.494220+0000","last_became_active":"2024-05-08T22:05:13.539428+0000","last_became_peered":"2024-05-08T22:05:13.539428+0000","last_unstale":"2024-05-08T22:05:37.494220+0000","last_undegraded":"2024-05-08T22:05:37.494220+0000","last_fullsized":"2024-05-08T22:05:37.494220+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:59:34.455411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518423+0000","last_change":"2024-05-08T22:05:14.061849+0000","last_active":"2024-05-08T22:05:18.518423+0000","last_peered":"2024-05-08T22:05:18.518423+0000","last_clean":"2024-05-08T22:05:18.518423+0000","last_became_active":"2024-05-08T22:05:14.061028+0000","last_became_peered":"2024-05-08T22:05:14.061028+0000","last_unstale":"2024-05-08T22:05:18.518423+0000","last_undegraded":"2024-05-08T22:05:18.518423+0000","last_fullsized":"2024-05-08T22:05:18.518423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:11:59.489376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819278+0000","last_change":"2024-05-08T22:05:18.517994+0000","last_active":"2024-05-08T22:05:40.819278+0000","last_peered":"2024-05-08T22:05:40.819278+0000","last_clean":"2024-05-08T22:05:40.819278+0000","last_became_active":"2024-05-08T22:05:18.517655+0000","last_became_peered":"2024-05-08T22:05:18.517655+0000","last_unstale":"2024-05-08T22:05:40.819278+0000","last_undegraded":"2024-05-08T22:05:40.819278+0000","last_fullsized":"2024-05-08T22:05:40.819278+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:50:08.956277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.039182+0000","last_change":"2024-05-08T22:05:13.552851+0000","last_active":"2024-05-08T22:05:19.039182+0000","last_peered":"2024-05-08T22:05:19.039182+0000","last_clean":"2024-05-08T22:05:19.039182+0000","last_became_active":"2024-05-08T22:05:13.552507+0000","last_became_peered":"2024-05-08T22:05:13.552507+0000","last_unstale":"2024-05-08T22:05:19.039182+0000","last_undegraded":"2024-05-08T22:05:19.039182+0000","last_fullsized":"2024-05-08T22:05:19.039182+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:29.248623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714486+0000","last_change":"2024-05-08T22:05:14.062618+0000","last_active":"2024-05-08T22:05:19.714486+0000","last_peered":"2024-05-08T22:05:19.714486+0000","last_clean":"2024-05-08T22:05:19.714486+0000","last_became_active":"2024-05-08T22:05:14.062308+0000","last_became_peered":"2024-05-08T22:05:14.062308+0000","last_unstale":"2024-05-08T22:05:19.714486+0000","last_undegraded":"2024-05-08T22:05:19.714486+0000","last_fullsized":"2024-05-08T22:05:19.714486+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:34.195343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.804949+0000","last_change":"2024-05-08T22:05:18.616846+0000","last_active":"2024-05-08T22:05:21.804949+0000","last_peered":"2024-05-08T22:05:21.804949+0000","last_clean":"2024-05-08T22:05:21.804949+0000","last_became_active":"2024-05-08T22:05:18.616369+0000","last_became_peered":"2024-05-08T22:05:18.616369+0000","last_unstale":"2024-05-08T22:05:21.804949+0000","last_undegraded":"2024-05-08T22:05:21.804949+0000","last_fullsized":"2024-05-08T22:05:21.804949+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:26.128949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829370+0000","last_change":"2024-05-08T22:05:13.549712+0000","last_active":"2024-05-08T22:05:40.829370+0000","last_peered":"2024-05-08T22:05:40.829370+0000","last_clean":"2024-05-08T22:05:40.829370+0000","last_became_active":"2024-05-08T22:05:13.549508+0000","last_became_peered":"2024-05-08T22:05:13.549508+0000","last_unstale":"2024-05-08T22:05:40.829370+0000","last_undegraded":"2024-05-08T22:05:40.829370+0000","last_fullsized":"2024-05-08T22:05:40.829370+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:30:22.706389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.516692+0000","last_change":"2024-05-08T22:05:14.060920+0000","last_active":"2024-05-08T22:05:18.516692+0000","last_peered":"2024-05-08T22:05:18.516692+0000","last_clean":"2024-05-08T22:05:18.516692+0000","last_became_active":"2024-05-08T22:05:14.059914+0000","last_became_peered":"2024-05-08T22:05:14.059914+0000","last_unstale":"2024-05-08T22:05:18.516692+0000","last_undegraded":"2024-05-08T22:05:18.516692+0000","last_fullsized":"2024-05-08T22:05:18.516692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:15.513600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.534473+0000","last_change":"2024-05-08T22:05:18.534076+0000","last_active":"2024-05-08T22:05:18.534473+0000","last_peered":"2024-05-08T22:05:18.534473+0000","last_clean":"2024-05-08T22:05:18.534473+0000","last_became_active":"2024-05-08T22:05:18.533679+0000","last_became_peered":"2024-05-08T22:05:18.533679+0000","last_unstale":"2024-05-08T22:05:18.534473+0000","last_undegraded":"2024-05-08T22:05:18.534473+0000","last_fullsized":"2024-05-08T22:05:18.534473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:08.979193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.521141+0000","last_change":"2024-05-08T22:05:13.559282+0000","last_active":"2024-05-08T22:05:18.521141+0000","last_peered":"2024-05-08T22:05:18.521141+0000","last_clean":"2024-05-08T22:05:18.521141+0000","last_became_active":"2024-05-08T22:05:13.558876+0000","last_became_peered":"2024-05-08T22:05:13.558876+0000","last_unstale":"2024-05-08T22:05:18.521141+0000","last_undegraded":"2024-05-08T22:05:18.521141+0000","last_fullsized":"2024-05-08T22:05:18.521141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:56.062332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.516801+0000","last_change":"2024-05-08T22:05:14.065868+0000","last_active":"2024-05-08T22:05:18.516801+0000","last_peered":"2024-05-08T22:05:18.516801+0000","last_clean":"2024-05-08T22:05:18.516801+0000","last_became_active":"2024-05-08T22:05:14.065545+0000","last_became_peered":"2024-05-08T22:05:14.065545+0000","last_unstale":"2024-05-08T22:05:18.516801+0000","last_undegraded":"2024-05-08T22:05:18.516801+0000","last_fullsized":"2024-05-08T22:05:18.516801+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:56:34.309095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.154428+0000","last_change":"2024-05-08T22:05:18.534261+0000","last_active":"2024-05-08T22:05:19.154428+0000","last_peered":"2024-05-08T22:05:19.154428+0000","last_clean":"2024-05-08T22:05:19.154428+0000","last_became_active":"2024-05-08T22:05:18.531814+0000","last_became_peered":"2024-05-08T22:05:18.531814+0000","last_unstale":"2024-05-08T22:05:19.154428+0000","last_undegraded":"2024-05-08T22:05:19.154428+0000","last_fullsized":"2024-05-08T22:05:19.154428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:38.344546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:20.446341+0000","last_change":"2024-05-08T22:05:13.549344+0000","last_active":"2024-05-08T22:05:20.446341+0000","last_peered":"2024-05-08T22:05:20.446341+0000","last_clean":"2024-05-08T22:05:20.446341+0000","last_became_active":"2024-05-08T22:05:13.548993+0000","last_became_peered":"2024-05-08T22:05:13.548993+0000","last_unstale":"2024-05-08T22:05:20.446341+0000","last_undegraded":"2024-05-08T22:05:20.446341+0000","last_fullsized":"2024-05-08T22:05:20.446341+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:50.096602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.515948+0000","last_change":"2024-05-08T22:05:14.065758+0000","last_active":"2024-05-08T22:05:18.515948+0000","last_peered":"2024-05-08T22:05:18.515948+0000","last_clean":"2024-05-08T22:05:18.515948+0000","last_became_active":"2024-05-08T22:05:14.065442+0000","last_became_peered":"2024-05-08T22:05:14.065442+0000","last_unstale":"2024-05-08T22:05:18.515948+0000","last_undegraded":"2024-05-08T22:05:18.515948+0000","last_fullsized":"2024-05-08T22:05:18.515948+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:43.047796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.834022+0000","last_change":"2024-05-08T22:05:18.529804+0000","last_active":"2024-05-08T22:05:19.834022+0000","last_peered":"2024-05-08T22:05:19.834022+0000","last_clean":"2024-05-08T22:05:19.834022+0000","last_became_active":"2024-05-08T22:05:18.529346+0000","last_became_peered":"2024-05-08T22:05:18.529346+0000","last_unstale":"2024-05-08T22:05:19.834022+0000","last_undegraded":"2024-05-08T22:05:19.834022+0000","last_fullsized":"2024-05-08T22:05:19.834022+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:13.565707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.154118+0000","last_change":"2024-05-08T22:05:13.551818+0000","last_active":"2024-05-08T22:05:19.154118+0000","last_peered":"2024-05-08T22:05:19.154118+0000","last_clean":"2024-05-08T22:05:19.154118+0000","last_became_active":"2024-05-08T22:05:13.551080+0000","last_became_peered":"2024-05-08T22:05:13.551080+0000","last_unstale":"2024-05-08T22:05:19.154118+0000","last_undegraded":"2024-05-08T22:05:19.154118+0000","last_fullsized":"2024-05-08T22:05:19.154118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:31.636482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.517095+0000","last_change":"2024-05-08T22:05:14.066572+0000","last_active":"2024-05-08T22:05:18.517095+0000","last_peered":"2024-05-08T22:05:18.517095+0000","last_clean":"2024-05-08T22:05:18.517095+0000","last_became_active":"2024-05-08T22:05:14.066272+0000","last_became_peered":"2024-05-08T22:05:14.066272+0000","last_unstale":"2024-05-08T22:05:18.517095+0000","last_undegraded":"2024-05-08T22:05:18.517095+0000","last_fullsized":"2024-05-08T22:05:18.517095+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:20:27.423464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518519+0000","last_change":"2024-05-08T22:05:14.061435+0000","last_active":"2024-05-08T22:05:18.518519+0000","last_peered":"2024-05-08T22:05:18.518519+0000","last_clean":"2024-05-08T22:05:18.518519+0000","last_became_active":"2024-05-08T22:05:14.061079+0000","last_became_peered":"2024-05-08T22:05:14.061079+0000","last_unstale":"2024-05-08T22:05:18.518519+0000","last_undegraded":"2024-05-08T22:05:18.518519+0000","last_fullsized":"2024-05-08T22:05:18.518519+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:42.684743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"peering","last_fresh":"2024-05-08T22:05:40.817668+0000","last_change":"2024-05-08T22:05:40.817845+0000","last_active":"2024-05-08T22:05:40.817668+0000","last_peered":"2024-05-08T22:05:19.039319+0000","last_clean":"2024-05-08T22:05:19.039319+0000","last_became_active":"2024-05-08T22:05:14.071184+0000","last_became_peered":"2024-05-08T22:05:14.071184+0000","last_unstale":"2024-05-08T22:05:40.817668+0000","last_undegraded":"2024-05-08T22:05:40.817668+0000","last_fullsized":"2024-05-08T22:05:40.817668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:12:55.547522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5,8],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.039153+0000","last_change":"2024-05-08T22:05:14.066697+0000","last_active":"2024-05-08T22:05:19.039153+0000","last_peered":"2024-05-08T22:05:19.039153+0000","last_clean":"2024-05-08T22:05:19.039153+0000","last_became_active":"2024-05-08T22:05:14.065952+0000","last_became_peered":"2024-05-08T22:05:14.065952+0000","last_unstale":"2024-05-08T22:05:19.039153+0000","last_undegraded":"2024-05-08T22:05:19.039153+0000","last_fullsized":"2024-05-08T22:05:19.039153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:03:19.006644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.521494+0000","last_change":"2024-05-08T22:05:14.072295+0000","last_active":"2024-05-08T22:05:18.521494+0000","last_peered":"2024-05-08T22:05:18.521494+0000","last_clean":"2024-05-08T22:05:18.521494+0000","last_became_active":"2024-05-08T22:05:14.065034+0000","last_became_peered":"2024-05-08T22:05:14.065034+0000","last_unstale":"2024-05-08T22:05:18.521494+0000","last_undegraded":"2024-05-08T22:05:18.521494+0000","last_fullsized":"2024-05-08T22:05:18.521494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:08:41.380814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.154280+0000","last_change":"2024-05-08T22:05:14.067263+0000","last_active":"2024-05-08T22:05:19.154280+0000","last_peered":"2024-05-08T22:05:19.154280+0000","last_clean":"2024-05-08T22:05:19.154280+0000","last_became_active":"2024-05-08T22:05:14.066957+0000","last_became_peered":"2024-05-08T22:05:14.066957+0000","last_unstale":"2024-05-08T22:05:19.154280+0000","last_undegraded":"2024-05-08T22:05:19.154280+0000","last_fullsized":"2024-05-08T22:05:19.154280+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:29:35.638489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518101+0000","last_change":"2024-05-08T22:05:14.060561+0000","last_active":"2024-05-08T22:05:18.518101+0000","last_peered":"2024-05-08T22:05:18.518101+0000","last_clean":"2024-05-08T22:05:18.518101+0000","last_became_active":"2024-05-08T22:05:14.060150+0000","last_became_peered":"2024-05-08T22:05:14.060150+0000","last_unstale":"2024-05-08T22:05:18.518101+0000","last_undegraded":"2024-05-08T22:05:18.518101+0000","last_fullsized":"2024-05-08T22:05:18.518101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:22.710776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.039533+0000","last_change":"2024-05-08T22:05:14.056394+0000","last_active":"2024-05-08T22:05:19.039533+0000","last_peered":"2024-05-08T22:05:19.039533+0000","last_clean":"2024-05-08T22:05:19.039533+0000","last_became_active":"2024-05-08T22:05:14.055977+0000","last_became_peered":"2024-05-08T22:05:14.055977+0000","last_unstale":"2024-05-08T22:05:19.039533+0000","last_undegraded":"2024-05-08T22:05:19.039533+0000","last_fullsized":"2024-05-08T22:05:19.039533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:56:48.249379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714720+0000","last_change":"2024-05-08T22:05:14.061934+0000","last_active":"2024-05-08T22:05:19.714720+0000","last_peered":"2024-05-08T22:05:19.714720+0000","last_clean":"2024-05-08T22:05:19.714720+0000","last_became_active":"2024-05-08T22:05:14.061635+0000","last_became_peered":"2024-05-08T22:05:14.061635+0000","last_unstale":"2024-05-08T22:05:19.714720+0000","last_undegraded":"2024-05-08T22:05:19.714720+0000","last_fullsized":"2024-05-08T22:05:19.714720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:02.423338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.524933+0000","last_change":"2024-05-08T22:05:14.069982+0000","last_active":"2024-05-08T22:05:18.524933+0000","last_peered":"2024-05-08T22:05:18.524933+0000","last_clean":"2024-05-08T22:05:18.524933+0000","last_became_active":"2024-05-08T22:05:14.068293+0000","last_became_peered":"2024-05-08T22:05:14.068293+0000","last_unstale":"2024-05-08T22:05:18.524933+0000","last_undegraded":"2024-05-08T22:05:18.524933+0000","last_fullsized":"2024-05-08T22:05:18.524933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:36.396983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714278+0000","last_change":"2024-05-08T22:05:14.058393+0000","last_active":"2024-05-08T22:05:19.714278+0000","last_peered":"2024-05-08T22:05:19.714278+0000","last_clean":"2024-05-08T22:05:19.714278+0000","last_became_active":"2024-05-08T22:05:14.058083+0000","last_became_peered":"2024-05-08T22:05:14.058083+0000","last_unstale":"2024-05-08T22:05:19.714278+0000","last_undegraded":"2024-05-08T22:05:19.714278+0000","last_fullsized":"2024-05-08T22:05:19.714278+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:10:47.074957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518092+0000","last_change":"2024-05-08T22:05:14.056989+0000","last_active":"2024-05-08T22:05:18.518092+0000","last_peered":"2024-05-08T22:05:18.518092+0000","last_clean":"2024-05-08T22:05:18.518092+0000","last_became_active":"2024-05-08T22:05:14.056669+0000","last_became_peered":"2024-05-08T22:05:14.056669+0000","last_unstale":"2024-05-08T22:05:18.518092+0000","last_undegraded":"2024-05-08T22:05:18.518092+0000","last_fullsized":"2024-05-08T22:05:18.518092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:56:51.173885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.153888+0000","last_change":"2024-05-08T22:05:14.058635+0000","last_active":"2024-05-08T22:05:19.153888+0000","last_peered":"2024-05-08T22:05:19.153888+0000","last_clean":"2024-05-08T22:05:19.153888+0000","last_became_active":"2024-05-08T22:05:14.058243+0000","last_became_peered":"2024-05-08T22:05:14.058243+0000","last_unstale":"2024-05-08T22:05:19.153888+0000","last_undegraded":"2024-05-08T22:05:19.153888+0000","last_fullsized":"2024-05-08T22:05:19.153888+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:18.142563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518557+0000","last_change":"2024-05-08T22:05:14.060735+0000","last_active":"2024-05-08T22:05:18.518557+0000","last_peered":"2024-05-08T22:05:18.518557+0000","last_clean":"2024-05-08T22:05:18.518557+0000","last_became_active":"2024-05-08T22:05:14.060391+0000","last_became_peered":"2024-05-08T22:05:14.060391+0000","last_unstale":"2024-05-08T22:05:18.518557+0000","last_undegraded":"2024-05-08T22:05:18.518557+0000","last_fullsized":"2024-05-08T22:05:18.518557+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:30.528705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283045,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6728,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":23,"num_read_kb":19,"num_write":118,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":41050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579920,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36088,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709064,"statfs":{"total":95995035648,"available":95958081536,"internally_reserved":0,"allocated":901120,"data_stored":447161,"data_compressed":11671,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36043205},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.36899999999999999}]},{"osd":1,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.46000000000000002}]},{"osd":2,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.47299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":4,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.504}]},{"osd":5,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.622}]},{"osd":6,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.68200000000000005}]},{"osd":7,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.60199999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":9,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.73499999999999999}]},{"osd":10,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.748}]}]},{"osd":10,"up_from":67,"seq":287762808850,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":966656,"data_stored":1040606,"data_compressed":47667,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":1,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.442}]},{"osd":2,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.54100000000000004}]},{"osd":3,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.502},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.64100000000000001}]},{"osd":4,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.68300000000000005}]},{"osd":5,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.65700000000000003}]},{"osd":6,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.70099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":8,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.747},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.78500000000000003}]},{"osd":9,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.80400000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]}]},{"osd":9,"up_from":62,"seq":266287972375,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":447397,"data_compressed":11671,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.38600000000000001}]},{"osd":1,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.63500000000000001}]},{"osd":2,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.77600000000000002}]},{"osd":3,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.46000000000000002}]},{"osd":4,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.73199999999999998}]},{"osd":5,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.66300000000000003}]},{"osd":6,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.71199999999999997}]},{"osd":7,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.80200000000000005}]},{"osd":8,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.498},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.67900000000000005}]},{"osd":10,"last update":"Wed May 8 22:05:22 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.746},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.76400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]}]},{"osd":8,"up_from":55,"seq":236223201308,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":469876,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":0.40799999999999997}]},{"osd":1,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.47199999999999998}]},{"osd":2,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.42499999999999999}]},{"osd":3,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":4.8920000000000003,"5min":4.8920000000000003,"15min":4.8920000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.67100000000000004}]},{"osd":4,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.627}]},{"osd":5,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.52500000000000002}]},{"osd":6,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.9210000000000003,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.77900000000000003}]},{"osd":7,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.748},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.64700000000000002}]},{"osd":9,"last update":"Wed May 8 22:05:01 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.80600000000000005}]},{"osd":10,"last update":"Wed May 8 22:05:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.79100000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]}]},{"osd":7,"up_from":48,"seq":206158430237,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":1038103,"data_compressed":47667,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.41199999999999998}]},{"osd":1,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.39500000000000002,"15min":0.38700000000000001},"min":{"1min":0.30599999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.443},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40600000000000003,"15min":0.40300000000000002},"min":{"1min":0.29199999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.36699999999999999}]},{"osd":2,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54100000000000004,"15min":0.53600000000000003},"min":{"1min":0.34899999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.38400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70299999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.68000000000000005}]},{"osd":3,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.41599999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79800000000000004,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.501,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80200000000000005,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.52900000000000003}]},{"osd":4,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.33800000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71999999999999997,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.754,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.51000000000000001}]},{"osd":5,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64400000000000002,"15min":0.64500000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89900000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.625,"15min":0.626},"min":{"1min":0.44500000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.77700000000000002,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.63200000000000001}]},{"osd":6,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85099999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69699999999999995,"15min":0.69899999999999995},"min":{"1min":0.48199999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89000000000000001,"5min":1.002,"15min":1.002},"last":0.46800000000000003}]},{"osd":8,"last update":"Wed May 8 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.68700000000000006,"15min":0.69399999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.78000000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.72099999999999997,"15min":0.72699999999999998},"min":{"1min":0.55300000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.85899999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.74099999999999999}]},{"osd":9,"last update":"Wed May 8 22:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.61699999999999999}]},{"osd":10,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.70799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]}]},{"osd":6,"up_from":43,"seq":184683593762,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":455906,"data_compressed":14627,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.46200000000000002,"15min":0.45100000000000001},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.66300000000000003,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.51000000000000001,"15min":0.51300000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.64400000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.45900000000000002}]},{"osd":1,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57299999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.61899999999999999,"15min":0.60599999999999998},"min":{"1min":0.47399999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.77800000000000002}]},{"osd":2,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.66200000000000003,"15min":0.64800000000000002},"min":{"1min":0.53200000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65200000000000002,"15min":0.64700000000000002},"min":{"1min":0.51400000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89300000000000002,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.65300000000000002}]},{"osd":3,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60999999999999999,"15min":0.60699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.141,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.44},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.495,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.71499999999999997}]},{"osd":4,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69699999999999995,"15min":0.69299999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0569999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89700000000000002,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.68700000000000006}]},{"osd":5,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.754,"15min":0.74399999999999999},"min":{"1min":0.57999999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.995,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.75800000000000001,"15min":0.752},"min":{"1min":0.56399999999999995,"5min":0.438,"15min":0.438},"max":{"1min":1.052,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.64200000000000002}]},{"osd":7,"last update":"Wed May 8 22:05:23 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.99299999999999999,"5min":1.028,"15min":1.028},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76300000000000001,"15min":0.76000000000000001},"min":{"1min":0.56999999999999995,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.98499999999999999}]},{"osd":8,"last update":"Wed May 8 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.83199999999999996,"15min":0.83999999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0289999999999999,"5min":1.093,"15min":1.093},"last":0.871},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.79300000000000004,"15min":0.79600000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0149999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.67100000000000004}]},{"osd":9,"last update":"Wed May 8 22:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.73599999999999999}]},{"osd":10,"last update":"Wed May 8 22:05:19 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.88200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":905216,"data_stored":450102,"data_compressed":11671,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.52700000000000002,"15min":0.49299999999999999},"min":{"1min":0.38900000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.89900000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54000000000000004,"15min":0.51400000000000001},"min":{"1min":0.371,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81200000000000006,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.58299999999999996}]},{"osd":2,"last update":"Wed May 8 22:04:46 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.51300000000000001,"15min":0.48499999999999999},"min":{"1min":0.378,"5min":0.316,"15min":0.316},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.50600000000000001,"15min":0.48399999999999999},"min":{"1min":0.315,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.621}]},{"osd":3,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.53300000000000003,"15min":0.505},"min":{"1min":0.40000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.53300000000000003,"15min":0.51200000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.56899999999999995}]},{"osd":4,"last update":"Wed May 8 22:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.39900000000000002,"15min":0.376},"min":{"1min":0.307,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.64000000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.42499999999999999,"15min":0.39300000000000002},"min":{"1min":0.32400000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.76700000000000002,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.497}]},{"osd":5,"last update":"Wed May 8 22:04:43 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.56399999999999995,"15min":0.54800000000000004},"min":{"1min":0.39400000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58499999999999996,"15min":0.57299999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.45400000000000001}]},{"osd":6,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.63,"15min":0.61599999999999999},"min":{"1min":0.55100000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.46500000000000002}]},{"osd":7,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.36499999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86299999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.438}]},{"osd":8,"last update":"Wed May 8 22:04:37 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.60499999999999998}]},{"osd":9,"last update":"Wed May 8 22:05:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.70999999999999996}]},{"osd":10,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.65900000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]}]},{"osd":0,"up_from":8,"seq":34359738425,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":905216,"data_stored":447000,"data_compressed":11671,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57899999999999996,"15min":0.54900000000000004},"min":{"1min":0.42499999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.94099999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.55400000000000005,"15min":0.52400000000000002},"min":{"1min":0.40100000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83199999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.55500000000000005}]},{"osd":2,"last update":"Wed May 8 22:04:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.56399999999999995,"15min":0.54000000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.56100000000000005,"15min":0.54100000000000004},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.59099999999999997}]},{"osd":3,"last update":"Wed May 8 22:04:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.436,"15min":0.41799999999999998},"min":{"1min":0.34799999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.48799999999999999,"15min":0.48199999999999998},"min":{"1min":0.35499999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.877,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.82899999999999996}]},{"osd":4,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.57099999999999995,"15min":0.54900000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69399999999999995,"15min":0.69599999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96999999999999997,"5min":2.984,"15min":2.984},"last":0.78200000000000003}]},{"osd":5,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.65500000000000003,"15min":0.63900000000000001},"min":{"1min":0.48399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67100000000000004,"15min":0.66200000000000003},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.86599999999999999}]},{"osd":6,"last update":"Wed May 8 22:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.68600000000000005,"15min":0.66900000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.97599999999999998}]},{"osd":7,"last update":"Wed May 8 22:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.92000000000000004}]},{"osd":8,"last update":"Wed May 8 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.96099999999999997}]},{"osd":9,"last update":"Wed May 8 22:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.014}]},{"osd":10,"last update":"Wed May 8 22:05:22 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":1},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.73899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36132,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709020,"statfs":{"total":95995035648,"available":95958036480,"internally_reserved":0,"allocated":946176,"data_stored":470429,"data_compressed":11671,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.41999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.51200000000000001,"15min":0.504},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76100000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.55700000000000005}]},{"osd":1,"last update":"Wed May 8 22:04:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52600000000000002,"15min":0.52500000000000002},"min":{"1min":0.34100000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.74199999999999999,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.35499999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.77500000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.66500000000000004}]},{"osd":3,"last update":"Wed May 8 22:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.372,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65900000000000003,"5min":0.872,"15min":0.872},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56599999999999995,"15min":0.55900000000000005},"min":{"1min":0.40100000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72199999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.70499999999999996}]},{"osd":4,"last update":"Wed May 8 22:04:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.35699999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73399999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55700000000000005,"15min":0.55200000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.746}]},{"osd":5,"last update":"Wed May 8 22:04:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.436,"15min":0.433},"min":{"1min":0.34200000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.59999999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44400000000000001,"15min":0.441},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.72399999999999998}]},{"osd":6,"last update":"Wed May 8 22:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.76400000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66600000000000004,"15min":0.66700000000000004},"min":{"1min":0.50800000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.81499999999999995,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.89500000000000002}]},{"osd":7,"last update":"Wed May 8 22:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.94499999999999995}]},{"osd":8,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.97299999999999998}]},{"osd":9,"last update":"Wed May 8 22:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.83199999999999996}]},{"osd":10,"last update":"Wed May 8 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.877}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]}]},{"osd":3,"up_from":24,"seq":103079215149,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":456373,"data_compressed":14627,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.41699999999999998,"15min":0.39200000000000002},"min":{"1min":0.33500000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.504},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.42399999999999999,"15min":0.40200000000000002},"min":{"1min":0.35699999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.71299999999999997}]},{"osd":1,"last update":"Wed May 8 22:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.59899999999999998,"15min":0.55700000000000005},"min":{"1min":0.48599999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.56399999999999995,"15min":0.53000000000000003},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.77200000000000002}]},{"osd":2,"last update":"Wed May 8 22:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.61499999999999999,"15min":0.58899999999999997},"min":{"1min":0.51800000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.63900000000000001,"15min":0.60899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.55500000000000005}]},{"osd":4,"last update":"Wed May 8 22:05:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.62,"15min":0.59399999999999997},"min":{"1min":0.42399999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.66300000000000003,"15min":0.62},"min":{"1min":0.61299999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.98599999999999999}]},{"osd":5,"last update":"Wed May 8 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.67800000000000005,"15min":0.64500000000000002},"min":{"1min":0.46500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.64400000000000002,"15min":0.60899999999999999},"min":{"1min":0.46500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.76300000000000001}]},{"osd":6,"last update":"Wed May 8 22:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.63100000000000001,"15min":0.61499999999999999},"min":{"1min":0.46400000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.64600000000000002,"15min":0.624},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.874}]},{"osd":7,"last update":"Wed May 8 22:05:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.65700000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":1.008}]},{"osd":8,"last update":"Wed May 8 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.53100000000000003}]},{"osd":9,"last update":"Wed May 8 22:05:01 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.84999999999999998}]},{"osd":10,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.88400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986219,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":460781,"data_compressed":14627,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.44700000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.51900000000000002,"15min":0.50900000000000001},"min":{"1min":0.45000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.629}]},{"osd":1,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.374,"15min":0.35099999999999998},"min":{"1min":0.27200000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.66800000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.377,"15min":0.35499999999999998},"min":{"1min":0.28199999999999997,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.51000000000000001}]},{"osd":2,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.52600000000000002,"15min":0.50700000000000001},"min":{"1min":0.35499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71899999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.80900000000000005}]},{"osd":3,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.58799999999999997,"15min":0.56399999999999995},"min":{"1min":0.5,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84799999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56699999999999995,"15min":0.55300000000000005},"min":{"1min":0.39600000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.80000000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.67300000000000004}]},{"osd":5,"last update":"Wed May 8 22:05:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.36399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76200000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.57199999999999995,"15min":0.55000000000000004},"min":{"1min":0.42799999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.82199999999999995}]},{"osd":6,"last update":"Wed May 8 22:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.64800000000000002,"15min":0.63600000000000001},"min":{"1min":0.371,"5min":0.316,"15min":0.316},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.748},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.41799999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80500000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.60499999999999998}]},{"osd":7,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76800000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57299999999999995}]},{"osd":8,"last update":"Wed May 8 22:05:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68400000000000005,"15min":0.68600000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70799999999999996,"15min":0.70899999999999996},"min":{"1min":0.54100000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.79300000000000004}]},{"osd":9,"last update":"Wed May 8 22:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.624}]},{"osd":10,"last update":"Wed May 8 22:05:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.83499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":5,"up_from":36,"seq":154618822695,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":974848,"data_stored":1038375,"data_compressed":47667,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.378,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72899999999999998,"5min":0.749,"15min":0.749},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.48999999999999999}]},{"osd":1,"last update":"Wed May 8 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.41299999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.71699999999999997,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.622},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.37,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.58199999999999996}]},{"osd":2,"last update":"Wed May 8 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.254,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.52400000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.376,"15min":0.379},"min":{"1min":0.26600000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.46500000000000002,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.36199999999999999}]},{"osd":3,"last update":"Wed May 8 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57299999999999995,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.438},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.51800000000000002}]},{"osd":4,"last update":"Wed May 8 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.53300000000000003},"min":{"1min":0.38400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.39400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.751,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.64900000000000002}]},{"osd":6,"last update":"Wed May 8 22:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58599999999999997,"15min":0.58399999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.60699999999999998}]},{"osd":7,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.57399999999999995,"15min":0.56499999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.58799999999999997,"15min":0.57699999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.63700000000000001}]},{"osd":8,"last update":"Wed May 8 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.69399999999999995}]},{"osd":9,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.70599999999999996}]},{"osd":10,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.67400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":24792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:05:44.338 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.18 is not active+clean 2024-05-08T22:05:44.339 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.18', 'version': "0'0", 'reported_seq': 14, 'reported_epoch': 84, 'state': 'peering', 'last_fresh': '2024-05-08T22:05:40.819123+0000', 'last_change': '2024-05-08T22:05:40.819370+0000', 'last_active': '2024-05-08T22:05:40.819123+0000', 'last_peered': '2024-05-08T22:05:21.804836+0000', 'last_clean': '2024-05-08T22:05:21.804836+0000', 'last_became_active': '2024-05-08T22:05:18.529966+0000', 'last_became_peered': '2024-05-08T22:05:18.529966+0000', 'last_unstale': '2024-05-08T22:05:40.819123+0000', 'last_undegraded': '2024-05-08T22:05:40.819123+0000', 'last_fullsized': '2024-05-08T22:05:40.819123+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 82, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:05:17.499131+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:05:17.499131+0000', 'last_clean_scrub_stamp': '2024-05-08T22:05:17.499131+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T06:28:51.858677+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6, 8, 3], 'acting': [4, 6, 8, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3, 6, 8, 9], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T22:05:44.339 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.7 is not active+clean 2024-05-08T22:05:44.339 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.7', 'version': "0'0", 'reported_seq': 14, 'reported_epoch': 84, 'state': 'peering', 'last_fresh': '2024-05-08T22:05:40.825443+0000', 'last_change': '2024-05-08T22:05:40.825855+0000', 'last_active': '2024-05-08T22:05:40.825443+0000', 'last_peered': '2024-05-08T22:05:21.719003+0000', 'last_clean': '2024-05-08T22:05:21.719003+0000', 'last_became_active': '2024-05-08T22:05:18.616536+0000', 'last_became_peered': '2024-05-08T22:05:18.616536+0000', 'last_unstale': '2024-05-08T22:05:40.825443+0000', 'last_undegraded': '2024-05-08T22:05:40.825443+0000', 'last_fullsized': '2024-05-08T22:05:40.825443+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 82, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:05:17.499131+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:05:17.499131+0000', 'last_clean_scrub_stamp': '2024-05-08T22:05:17.499131+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T03:30:44.106162+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 8, 10, 0], 'acting': [6, 8, 10, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 5, 8, 10], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T22:05:44.339 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.21 is not active+clean 2024-05-08T22:05:44.339 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 17, 'reported_epoch': 84, 'state': 'peering', 'last_fresh': '2024-05-08T22:05:40.817668+0000', 'last_change': '2024-05-08T22:05:40.817845+0000', 'last_active': '2024-05-08T22:05:40.817668+0000', 'last_peered': '2024-05-08T22:05:19.039319+0000', 'last_clean': '2024-05-08T22:05:19.039319+0000', 'last_became_active': '2024-05-08T22:05:14.071184+0000', 'last_became_peered': '2024-05-08T22:05:14.071184+0000', 'last_unstale': '2024-05-08T22:05:40.817668+0000', 'last_undegraded': '2024-05-08T22:05:40.817668+0000', 'last_fullsized': '2024-05-08T22:05:40.817668+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 78, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:05:13.522431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:05:13.522431+0000', 'last_clean_scrub_stamp': '2024-05-08T22:05:13.522431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T00:12:55.547522+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5, 8], 'acting': [2, 5, 8], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4, 5, 8], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T22:05:44.340 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T22:05:44.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.562+0000 7fa5e8800700 1 -- 172.21.15.131:0/3905998061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5e0100e90 msgr2=0x7fa5e01012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:44.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.562+0000 7fa5e8800700 1 --2- 172.21.15.131:0/3905998061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5e0100e90 0x7fa5e01012b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009990 tx=0x7fa5d4009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:44.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.562+0000 7fa5e8800700 1 -- 172.21.15.131:0/3905998061 shutdown_connections 2024-05-08T22:05:44.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.562+0000 7fa5e8800700 1 --2- 172.21.15.131:0/3905998061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5e01029a0 0x7fa5e01092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:44.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.562+0000 7fa5e8800700 1 --2- 172.21.15.131:0/3905998061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5e0100e90 0x7fa5e01012b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:44.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.562+0000 7fa5e8800700 1 --2- 172.21.15.131:0/3905998061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e0101fe0 0x7fa5e0102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:44.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.562+0000 7fa5e8800700 1 -- 172.21.15.131:0/3905998061 >> 172.21.15.131:0/3905998061 conn(0x7fa5e00fc400 msgr2=0x7fa5e00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:44.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.562+0000 7fa5e8800700 1 -- 172.21.15.131:0/3905998061 shutdown_connections 2024-05-08T22:05:44.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.562+0000 7fa5e8800700 1 -- 172.21.15.131:0/3905998061 wait complete. 2024-05-08T22:05:44.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.563+0000 7fa5e8800700 1 Processor -- start 2024-05-08T22:05:44.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.564+0000 7fa5e8800700 1 -- start start 2024-05-08T22:05:44.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.564+0000 7fa5e8800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5e0100e90 0x7fa5e0074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:44.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.564+0000 7fa5e8800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5e0101fe0 0x7fa5e0073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:44.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.564+0000 7fa5e8800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e01029a0 0x7fa5e00738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:44.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.564+0000 7fa5e8800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e0074070 con 0x7fa5e0100e90 2024-05-08T22:05:44.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.564+0000 7fa5e8800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e00741e0 con 0x7fa5e0101fe0 2024-05-08T22:05:44.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.564+0000 7fa5e8800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e01a0ed0 con 0x7fa5e01029a0 2024-05-08T22:05:44.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.564+0000 7fa5e5e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5e0100e90 0x7fa5e0074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:44.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.564+0000 7fa5e5e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5e0100e90 0x7fa5e0074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38112/0 (socket says 172.21.15.131:38112) 2024-05-08T22:05:44.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.564+0000 7fa5e5e00700 1 -- 172.21.15.131:0/136196651 learned_addr learned my addr 172.21.15.131:0/136196651 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:44.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.564+0000 7fa5e5400700 1 --2- 172.21.15.131:0/136196651 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5e0101fe0 0x7fa5e0073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:44.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.564+0000 7fa5e6800700 1 --2- 172.21.15.131:0/136196651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e01029a0 0x7fa5e00738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:44.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.565+0000 7fa5e5e00700 1 -- 172.21.15.131:0/136196651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e01029a0 msgr2=0x7fa5e00738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:44.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.565+0000 7fa5e5e00700 1 --2- 172.21.15.131:0/136196651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e01029a0 0x7fa5e00738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:44.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.565+0000 7fa5e5e00700 1 -- 172.21.15.131:0/136196651 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5e0101fe0 msgr2=0x7fa5e0073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:44.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.565+0000 7fa5e5e00700 1 --2- 172.21.15.131:0/136196651 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5e0101fe0 0x7fa5e0073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:44.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.565+0000 7fa5e5e00700 1 -- 172.21.15.131:0/136196651 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d40095f0 con 0x7fa5e0100e90 2024-05-08T22:05:44.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.565+0000 7fa5e5400700 1 --2- 172.21.15.131:0/136196651 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5e0101fe0 0x7fa5e0073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:44.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.565+0000 7fa5e5e00700 1 --2- 172.21.15.131:0/136196651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5e0100e90 0x7fa5e0074d00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4000c00 tx=0x7fa5d4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:44.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.565+0000 7fa5daa00700 1 -- 172.21.15.131:0/136196651 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d401d070 con 0x7fa5e0100e90 2024-05-08T22:05:44.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.565+0000 7fa5e8800700 1 -- 172.21.15.131:0/136196651 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e01a1100 con 0x7fa5e0100e90 2024-05-08T22:05:44.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.565+0000 7fa5daa00700 1 -- 172.21.15.131:0/136196651 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5d4021410 con 0x7fa5e0100e90 2024-05-08T22:05:44.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.565+0000 7fa5e8800700 1 -- 172.21.15.131:0/136196651 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e01a14c0 con 0x7fa5e0100e90 2024-05-08T22:05:44.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.565+0000 7fa5daa00700 1 -- 172.21.15.131:0/136196651 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d40175f0 con 0x7fa5e0100e90 2024-05-08T22:05:44.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.566+0000 7fa5e8800700 1 -- 172.21.15.131:0/136196651 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e0108300 con 0x7fa5e0100e90 2024-05-08T22:05:44.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.568+0000 7fa5daa00700 1 -- 172.21.15.131:0/136196651 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fa5d4021580 con 0x7fa5e0100e90 2024-05-08T22:05:44.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.569+0000 7fa5daa00700 1 --2- 172.21.15.131:0/136196651 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa5c80a11b0 0x7fa5c80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:44.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.569+0000 7fa5daa00700 1 -- 172.21.15.131:0/136196651 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa5d40ce0b0 con 0x7fa5e0100e90 2024-05-08T22:05:44.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.570+0000 7fa5e5400700 1 --2- 172.21.15.131:0/136196651 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa5c80a11b0 0x7fa5c80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:44.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.570+0000 7fa5e5400700 1 --2- 172.21.15.131:0/136196651 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa5c80a11b0 0x7fa5c80a3670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0011490 tx=0x7fa5d0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:44.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.573+0000 7fa5daa00700 1 -- 172.21.15.131:0/136196651 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa5d4097400 con 0x7fa5e0100e90 2024-05-08T22:05:44.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.867+0000 7fa5e8800700 1 -- 172.21.15.131:0/136196651 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa5e0067c10 con 0x7fa5e0100e90 2024-05-08T22:05:44.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.868+0000 7fa5daa00700 1 -- 172.21.15.131:0/136196651 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3319 (secure 0 0 0) 0x7fa5d4026d10 con 0x7fa5e0100e90 2024-05-08T22:05:44.869 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:44.869 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":315,"monmap":{"epoch":3,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":85,"num_osds":12,"num_up_osds":12,"osd_up_since":1715205875,"num_in_osds":12,"osd_in_since":1715205858,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":126},{"state_name":"peering","count":3}],"num_pgs":129,"num_pools":4,"num_objects":61,"data_bytes":634543,"bytes_used":381202432,"bytes_avail":1151559225344,"bytes_total":1151940427776,"inactive_pgs_ratio":0.023255813866853714,"recovering_objects_per_sec":1,"recovering_bytes_per_sec":86,"recovering_keys_per_sec":0,"num_objects_recovered":10,"num_bytes_recovered":694,"num_keys_recovered":0,"read_bytes_sec":8701,"write_bytes_sec":8061,"read_op_per_sec":4,"write_op_per_sec":11},"fsmap":{"epoch":23,"id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"c","status":"up:standby-replay","gid":24520},{"filesystem_id":1,"rank":1,"name":"h","status":"up:standby-replay","gid":24524},{"filesystem_id":1,"rank":0,"name":"i","status":"up:standby-replay","gid":24538},{"filesystem_id":1,"rank":2,"name":"g","status":"up:active","gid":24550},{"filesystem_id":1,"rank":1,"name":"a","status":"up:active","gid":24562},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24566}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.131:8443/"}},"servicemap":{"epoch":18,"modified":"2024-05-08T22:05:36.716522+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"g":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"j":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"k":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"l":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T22:05:44.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.870+0000 7fa5e8800700 1 -- 172.21.15.131:0/136196651 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa5c80a11b0 msgr2=0x7fa5c80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:44.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.870+0000 7fa5e8800700 1 --2- 172.21.15.131:0/136196651 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa5c80a11b0 0x7fa5c80a3670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0011490 tx=0x7fa5d0009260 comp rx=0 tx=0).stop 2024-05-08T22:05:44.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.870+0000 7fa5e8800700 1 -- 172.21.15.131:0/136196651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5e0100e90 msgr2=0x7fa5e0074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:44.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.870+0000 7fa5e8800700 1 --2- 172.21.15.131:0/136196651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5e0100e90 0x7fa5e0074d00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4000c00 tx=0x7fa5d4005740 comp rx=0 tx=0).stop 2024-05-08T22:05:44.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.870+0000 7fa5e8800700 1 -- 172.21.15.131:0/136196651 shutdown_connections 2024-05-08T22:05:44.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.870+0000 7fa5e8800700 1 --2- 172.21.15.131:0/136196651 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa5c80a11b0 0x7fa5c80a3670 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:44.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.870+0000 7fa5e8800700 1 --2- 172.21.15.131:0/136196651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e01029a0 0x7fa5e00738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:44.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.870+0000 7fa5e8800700 1 --2- 172.21.15.131:0/136196651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5e0100e90 0x7fa5e0074d00 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:44.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.870+0000 7fa5e8800700 1 --2- 172.21.15.131:0/136196651 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5e0101fe0 0x7fa5e0073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:44.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.870+0000 7fa5e8800700 1 -- 172.21.15.131:0/136196651 >> 172.21.15.131:0/136196651 conn(0x7fa5e00fc400 msgr2=0x7fa5e0105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:44.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.870+0000 7fa5e8800700 1 -- 172.21.15.131:0/136196651 shutdown_connections 2024-05-08T22:05:44.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:44.870+0000 7fa5e8800700 1 -- 172.21.15.131:0/136196651 wait complete. 2024-05-08T22:05:44.891 INFO:tasks.cephfs.filesystem.ceph_manager:making progress, resetting timeout 2024-05-08T22:05:44.891 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:05:45.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.082+0000 7f5f76000700 1 -- 172.21.15.131:0/1524547169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f70104210 msgr2=0x7f5f70106600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:45.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.082+0000 7f5f76000700 1 --2- 172.21.15.131:0/1524547169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f70104210 0x7f5f70106600 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009990 tx=0x7f5f5c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:45.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.082+0000 7f5f76000700 1 -- 172.21.15.131:0/1524547169 shutdown_connections 2024-05-08T22:05:45.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.082+0000 7f5f76000700 1 --2- 172.21.15.131:0/1524547169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f70106b40 0x7f5f70108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:45.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.082+0000 7f5f76000700 1 --2- 172.21.15.131:0/1524547169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f701018e0 0x7f5f70103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:45.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.082+0000 7f5f76000700 1 --2- 172.21.15.131:0/1524547169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f70104210 0x7f5f70106600 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:45.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.082+0000 7f5f76000700 1 -- 172.21.15.131:0/1524547169 >> 172.21.15.131:0/1524547169 conn(0x7f5f700fb2f0 msgr2=0x7f5f700fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:45.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.083+0000 7f5f76000700 1 -- 172.21.15.131:0/1524547169 shutdown_connections 2024-05-08T22:05:45.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.083+0000 7f5f76000700 1 -- 172.21.15.131:0/1524547169 wait complete. 2024-05-08T22:05:45.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.083+0000 7f5f76000700 1 Processor -- start 2024-05-08T22:05:45.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.084+0000 7f5f76000700 1 -- start start 2024-05-08T22:05:45.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.084+0000 7f5f76000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f701018e0 0x7f5f70199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:45.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.084+0000 7f5f76000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f70104210 0x7f5f70199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:45.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.084+0000 7f5f76000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f70106b40 0x7f5f701a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:45.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.084+0000 7f5f76000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f701a0b50 con 0x7f5f701018e0 2024-05-08T22:05:45.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.084+0000 7f5f76000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f701a0c90 con 0x7f5f70104210 2024-05-08T22:05:45.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.084+0000 7f5f76000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f701a0e20 con 0x7f5f70106b40 2024-05-08T22:05:45.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.084+0000 7f5f6f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f701018e0 0x7f5f70199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:45.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.084+0000 7f5f6f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f701018e0 0x7f5f70199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38142/0 (socket says 172.21.15.131:38142) 2024-05-08T22:05:45.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.084+0000 7f5f6f400700 1 -- 172.21.15.131:0/1073078136 learned_addr learned my addr 172.21.15.131:0/1073078136 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:45.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.084+0000 7f5f6fe00700 1 --2- 172.21.15.131:0/1073078136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f70106b40 0x7f5f701a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:45.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.084+0000 7f5f6ea00700 1 --2- 172.21.15.131:0/1073078136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f70104210 0x7f5f70199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:45.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.085+0000 7f5f6f400700 1 -- 172.21.15.131:0/1073078136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f70106b40 msgr2=0x7f5f701a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:45.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.085+0000 7f5f6f400700 1 --2- 172.21.15.131:0/1073078136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f70106b40 0x7f5f701a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:45.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.085+0000 7f5f6f400700 1 -- 172.21.15.131:0/1073078136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f70104210 msgr2=0x7f5f70199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:45.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.085+0000 7f5f6f400700 1 --2- 172.21.15.131:0/1073078136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f70104210 0x7f5f70199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:45.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.085+0000 7f5f6f400700 1 -- 172.21.15.131:0/1073078136 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5c0095f0 con 0x7f5f701018e0 2024-05-08T22:05:45.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.085+0000 7f5f6f400700 1 --2- 172.21.15.131:0/1073078136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f701018e0 0x7f5f70199020 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000e9a0 tx=0x7f5f6000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:45.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.085+0000 7f5f67e00700 1 -- 172.21.15.131:0/1073078136 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60009790 con 0x7f5f701018e0 2024-05-08T22:05:45.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.085+0000 7f5f67e00700 1 -- 172.21.15.131:0/1073078136 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f60010430 con 0x7f5f701018e0 2024-05-08T22:05:45.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.085+0000 7f5f67e00700 1 -- 172.21.15.131:0/1073078136 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60018680 con 0x7f5f701018e0 2024-05-08T22:05:45.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.085+0000 7f5f76000700 1 -- 172.21.15.131:0/1073078136 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f701a1100 con 0x7f5f701018e0 2024-05-08T22:05:45.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.085+0000 7f5f76000700 1 -- 172.21.15.131:0/1073078136 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f701a15c0 con 0x7f5f701018e0 2024-05-08T22:05:45.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.086+0000 7f5f76000700 1 -- 172.21.15.131:0/1073078136 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f70108710 con 0x7f5f701018e0 2024-05-08T22:05:45.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:44 smithi132 ceph-mon[78180]: from='client.24854 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:05:45.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:44 smithi132 ceph-mon[78180]: from='client.24860 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:05:45.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.088+0000 7f5f67e00700 1 -- 172.21.15.131:0/1073078136 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f5f600105a0 con 0x7f5f701018e0 2024-05-08T22:05:45.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.089+0000 7f5f67e00700 1 --2- 172.21.15.131:0/1073078136 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f5f540a1230 0x7f5f540a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:45.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.089+0000 7f5f67e00700 1 -- 172.21.15.131:0/1073078136 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5f60014070 con 0x7f5f701018e0 2024-05-08T22:05:45.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.090+0000 7f5f6ea00700 1 --2- 172.21.15.131:0/1073078136 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f5f540a1230 0x7f5f540a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:45.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.090+0000 7f5f6ea00700 1 --2- 172.21.15.131:0/1073078136 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f5f540a1230 0x7f5f540a36f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c00b2e0 tx=0x7f5f5c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:45.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.093+0000 7f5f67e00700 1 -- 172.21.15.131:0/1073078136 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5f60052d60 con 0x7f5f701018e0 2024-05-08T22:05:45.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:44 smithi131 ceph-mon[37260]: from='client.24854 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:05:45.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:44 smithi131 ceph-mon[37260]: from='client.24860 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:05:45.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:44 smithi157 ceph-mon[77746]: from='client.24854 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:05:45.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:44 smithi157 ceph-mon[77746]: from='client.24860 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:05:45.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.296+0000 7f5f76000700 1 -- 172.21.15.131:0/1073078136 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5f7002ceb0 con 0x7f5f540a1230 2024-05-08T22:05:45.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.308+0000 7f5f67e00700 1 -- 172.21.15.131:0/1073078136 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422783 (secure 0 0 0) 0x7f5f7002ceb0 con 0x7f5f540a1230 2024-05-08T22:05:45.315 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:45.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.314+0000 7f5f76000700 1 -- 172.21.15.131:0/1073078136 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f5f540a1230 msgr2=0x7f5f540a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:45.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.314+0000 7f5f76000700 1 --2- 172.21.15.131:0/1073078136 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f5f540a1230 0x7f5f540a36f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c00b2e0 tx=0x7f5f5c005fb0 comp rx=0 tx=0).stop 2024-05-08T22:05:45.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.314+0000 7f5f76000700 1 -- 172.21.15.131:0/1073078136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f701018e0 msgr2=0x7f5f70199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:45.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.314+0000 7f5f76000700 1 --2- 172.21.15.131:0/1073078136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f701018e0 0x7f5f70199020 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000e9a0 tx=0x7f5f6000ecb0 comp rx=0 tx=0).stop 2024-05-08T22:05:45.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.314+0000 7f5f76000700 1 -- 172.21.15.131:0/1073078136 shutdown_connections 2024-05-08T22:05:45.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.314+0000 7f5f76000700 1 --2- 172.21.15.131:0/1073078136 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f5f540a1230 0x7f5f540a36f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:45.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.314+0000 7f5f76000700 1 --2- 172.21.15.131:0/1073078136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f70106b40 0x7f5f701a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:45.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.314+0000 7f5f76000700 1 --2- 172.21.15.131:0/1073078136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f701018e0 0x7f5f70199020 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:45.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.314+0000 7f5f76000700 1 --2- 172.21.15.131:0/1073078136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f70104210 0x7f5f70199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:45.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.314+0000 7f5f76000700 1 -- 172.21.15.131:0/1073078136 >> 172.21.15.131:0/1073078136 conn(0x7f5f700fb2f0 msgr2=0x7f5f70101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:45.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.314+0000 7f5f76000700 1 -- 172.21.15.131:0/1073078136 shutdown_connections 2024-05-08T22:05:45.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:45.314+0000 7f5f76000700 1 -- 172.21.15.131:0/1073078136 wait complete. 2024-05-08T22:05:45.320 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-08T22:05:45.335 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-05-08T22:05:44.718356+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":634543,"num_objects":61,"num_object_clones":0,"num_object_copies":185,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":175,"num_read_kb":187,"num_write":321,"num_write_kb":3018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1181430,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":260,"ondisk_log_size":260,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":420,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372268,"kb_used_data":11172,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124569556,"statfs":{"total":1151940427776,"available":1151559225344,"internally_reserved":0,"allocated":11440128,"data_stored":7414560,"data_compressed":292380,"data_compressed_allocated":393216,"data_compressed_original":2285568,"omap_allocated":19131,"internal_metadata":369603909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":39722,"num_objects":37,"num_object_clones":0,"num_object_copies":113,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":39,"num_read_kb":68,"num_write":93,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002130"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518839+0000","last_change":"2024-05-08T22:05:14.061628+0000","last_active":"2024-05-08T22:05:18.518839+0000","last_peered":"2024-05-08T22:05:18.518839+0000","last_clean":"2024-05-08T22:05:18.518839+0000","last_became_active":"2024-05-08T22:05:14.060986+0000","last_became_peered":"2024-05-08T22:05:14.060986+0000","last_unstale":"2024-05-08T22:05:18.518839+0000","last_undegraded":"2024-05-08T22:05:18.518839+0000","last_fullsized":"2024-05-08T22:05:18.518839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:51.643118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832603+0000","last_change":"2024-05-08T22:05:14.073300+0000","last_active":"2024-05-08T22:05:40.832603+0000","last_peered":"2024-05-08T22:05:40.832603+0000","last_clean":"2024-05-08T22:05:40.832603+0000","last_became_active":"2024-05-08T22:05:14.069058+0000","last_became_peered":"2024-05-08T22:05:14.069058+0000","last_unstale":"2024-05-08T22:05:40.832603+0000","last_undegraded":"2024-05-08T22:05:40.832603+0000","last_fullsized":"2024-05-08T22:05:40.832603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:15.834956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832503+0000","last_change":"2024-05-08T22:05:14.071676+0000","last_active":"2024-05-08T22:05:40.832503+0000","last_peered":"2024-05-08T22:05:40.832503+0000","last_clean":"2024-05-08T22:05:40.832503+0000","last_became_active":"2024-05-08T22:05:14.058345+0000","last_became_peered":"2024-05-08T22:05:14.058345+0000","last_unstale":"2024-05-08T22:05:40.832503+0000","last_undegraded":"2024-05-08T22:05:40.832503+0000","last_fullsized":"2024-05-08T22:05:40.832503+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:09.239058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.019162+0000","last_change":"2024-05-08T22:05:14.067170+0000","last_active":"2024-05-08T22:05:19.019162+0000","last_peered":"2024-05-08T22:05:19.019162+0000","last_clean":"2024-05-08T22:05:19.019162+0000","last_became_active":"2024-05-08T22:05:14.066850+0000","last_became_peered":"2024-05-08T22:05:14.066850+0000","last_unstale":"2024-05-08T22:05:19.019162+0000","last_undegraded":"2024-05-08T22:05:19.019162+0000","last_fullsized":"2024-05-08T22:05:19.019162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:28.439786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829281+0000","last_change":"2024-05-08T22:05:14.065177+0000","last_active":"2024-05-08T22:05:40.829281+0000","last_peered":"2024-05-08T22:05:40.829281+0000","last_clean":"2024-05-08T22:05:40.829281+0000","last_became_active":"2024-05-08T22:05:14.064833+0000","last_became_peered":"2024-05-08T22:05:14.064833+0000","last_unstale":"2024-05-08T22:05:40.829281+0000","last_undegraded":"2024-05-08T22:05:40.829281+0000","last_fullsized":"2024-05-08T22:05:40.829281+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:11.764154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.825297+0000","last_change":"2024-05-08T22:05:14.061610+0000","last_active":"2024-05-08T22:05:40.825297+0000","last_peered":"2024-05-08T22:05:40.825297+0000","last_clean":"2024-05-08T22:05:40.825297+0000","last_became_active":"2024-05-08T22:05:14.061277+0000","last_became_peered":"2024-05-08T22:05:14.061277+0000","last_unstale":"2024-05-08T22:05:40.825297+0000","last_undegraded":"2024-05-08T22:05:40.825297+0000","last_fullsized":"2024-05-08T22:05:40.825297+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:19.154980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830795+0000","last_change":"2024-05-08T22:05:14.063941+0000","last_active":"2024-05-08T22:05:40.830795+0000","last_peered":"2024-05-08T22:05:40.830795+0000","last_clean":"2024-05-08T22:05:40.830795+0000","last_became_active":"2024-05-08T22:05:14.063579+0000","last_became_peered":"2024-05-08T22:05:14.063579+0000","last_unstale":"2024-05-08T22:05:40.830795+0000","last_undegraded":"2024-05-08T22:05:40.830795+0000","last_fullsized":"2024-05-08T22:05:40.830795+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:27.679067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714348+0000","last_change":"2024-05-08T22:05:14.060366+0000","last_active":"2024-05-08T22:05:19.714348+0000","last_peered":"2024-05-08T22:05:19.714348+0000","last_clean":"2024-05-08T22:05:19.714348+0000","last_became_active":"2024-05-08T22:05:14.059818+0000","last_became_peered":"2024-05-08T22:05:14.059818+0000","last_unstale":"2024-05-08T22:05:19.714348+0000","last_undegraded":"2024-05-08T22:05:19.714348+0000","last_fullsized":"2024-05-08T22:05:19.714348+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:53.118508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.823157+0000","last_change":"2024-05-08T22:05:14.057185+0000","last_active":"2024-05-08T22:05:40.823157+0000","last_peered":"2024-05-08T22:05:40.823157+0000","last_clean":"2024-05-08T22:05:40.823157+0000","last_became_active":"2024-05-08T22:05:14.056889+0000","last_became_peered":"2024-05-08T22:05:14.056889+0000","last_unstale":"2024-05-08T22:05:40.823157+0000","last_undegraded":"2024-05-08T22:05:40.823157+0000","last_fullsized":"2024-05-08T22:05:40.823157+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:35.372705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824080+0000","last_change":"2024-05-08T22:05:14.062282+0000","last_active":"2024-05-08T22:05:40.824080+0000","last_peered":"2024-05-08T22:05:40.824080+0000","last_clean":"2024-05-08T22:05:40.824080+0000","last_became_active":"2024-05-08T22:05:14.061574+0000","last_became_peered":"2024-05-08T22:05:14.061574+0000","last_unstale":"2024-05-08T22:05:40.824080+0000","last_undegraded":"2024-05-08T22:05:40.824080+0000","last_fullsized":"2024-05-08T22:05:40.824080+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:16:32.873590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831100+0000","last_change":"2024-05-08T22:05:14.061267+0000","last_active":"2024-05-08T22:05:40.831100+0000","last_peered":"2024-05-08T22:05:40.831100+0000","last_clean":"2024-05-08T22:05:40.831100+0000","last_became_active":"2024-05-08T22:05:14.060943+0000","last_became_peered":"2024-05-08T22:05:14.060943+0000","last_unstale":"2024-05-08T22:05:40.831100+0000","last_undegraded":"2024-05-08T22:05:40.831100+0000","last_fullsized":"2024-05-08T22:05:40.831100+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:03.831138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.019220+0000","last_change":"2024-05-08T22:05:14.067136+0000","last_active":"2024-05-08T22:05:19.019220+0000","last_peered":"2024-05-08T22:05:19.019220+0000","last_clean":"2024-05-08T22:05:19.019220+0000","last_became_active":"2024-05-08T22:05:14.066810+0000","last_became_peered":"2024-05-08T22:05:14.066810+0000","last_unstale":"2024-05-08T22:05:19.019220+0000","last_undegraded":"2024-05-08T22:05:19.019220+0000","last_fullsized":"2024-05-08T22:05:19.019220+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:43.001024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.154187+0000","last_change":"2024-05-08T22:05:14.062119+0000","last_active":"2024-05-08T22:05:19.154187+0000","last_peered":"2024-05-08T22:05:19.154187+0000","last_clean":"2024-05-08T22:05:19.154187+0000","last_became_active":"2024-05-08T22:05:14.058977+0000","last_became_peered":"2024-05-08T22:05:14.058977+0000","last_unstale":"2024-05-08T22:05:19.154187+0000","last_undegraded":"2024-05-08T22:05:19.154187+0000","last_fullsized":"2024-05-08T22:05:19.154187+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:03.657985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832861+0000","last_change":"2024-05-08T22:05:14.067363+0000","last_active":"2024-05-08T22:05:40.832861+0000","last_peered":"2024-05-08T22:05:40.832861+0000","last_clean":"2024-05-08T22:05:40.832861+0000","last_became_active":"2024-05-08T22:05:14.066119+0000","last_became_peered":"2024-05-08T22:05:14.066119+0000","last_unstale":"2024-05-08T22:05:40.832861+0000","last_undegraded":"2024-05-08T22:05:40.832861+0000","last_fullsized":"2024-05-08T22:05:40.832861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:43.426750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832641+0000","last_change":"2024-05-08T22:05:14.073925+0000","last_active":"2024-05-08T22:05:40.832641+0000","last_peered":"2024-05-08T22:05:40.832641+0000","last_clean":"2024-05-08T22:05:40.832641+0000","last_became_active":"2024-05-08T22:05:14.072287+0000","last_became_peered":"2024-05-08T22:05:14.072287+0000","last_unstale":"2024-05-08T22:05:40.832641+0000","last_undegraded":"2024-05-08T22:05:40.832641+0000","last_fullsized":"2024-05-08T22:05:40.832641+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:13.537938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.823678+0000","last_change":"2024-05-08T22:05:14.057274+0000","last_active":"2024-05-08T22:05:40.823678+0000","last_peered":"2024-05-08T22:05:40.823678+0000","last_clean":"2024-05-08T22:05:40.823678+0000","last_became_active":"2024-05-08T22:05:14.056989+0000","last_became_peered":"2024-05-08T22:05:14.056989+0000","last_unstale":"2024-05-08T22:05:40.823678+0000","last_undegraded":"2024-05-08T22:05:40.823678+0000","last_fullsized":"2024-05-08T22:05:40.823678+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:32.316406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518749+0000","last_change":"2024-05-08T22:05:14.069263+0000","last_active":"2024-05-08T22:05:18.518749+0000","last_peered":"2024-05-08T22:05:18.518749+0000","last_clean":"2024-05-08T22:05:18.518749+0000","last_became_active":"2024-05-08T22:05:14.068960+0000","last_became_peered":"2024-05-08T22:05:14.068960+0000","last_unstale":"2024-05-08T22:05:18.518749+0000","last_undegraded":"2024-05-08T22:05:18.518749+0000","last_fullsized":"2024-05-08T22:05:18.518749+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:59:42.829110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817249+0000","last_change":"2024-05-08T22:05:14.049393+0000","last_active":"2024-05-08T22:05:40.817249+0000","last_peered":"2024-05-08T22:05:40.817249+0000","last_clean":"2024-05-08T22:05:40.817249+0000","last_became_active":"2024-05-08T22:05:14.049094+0000","last_became_peered":"2024-05-08T22:05:14.049094+0000","last_unstale":"2024-05-08T22:05:40.817249+0000","last_undegraded":"2024-05-08T22:05:40.817249+0000","last_fullsized":"2024-05-08T22:05:40.817249+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:42.106568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.039735+0000","last_change":"2024-05-08T22:05:14.055436+0000","last_active":"2024-05-08T22:05:19.039735+0000","last_peered":"2024-05-08T22:05:19.039735+0000","last_clean":"2024-05-08T22:05:19.039735+0000","last_became_active":"2024-05-08T22:05:14.055100+0000","last_became_peered":"2024-05-08T22:05:14.055100+0000","last_unstale":"2024-05-08T22:05:19.039735+0000","last_undegraded":"2024-05-08T22:05:19.039735+0000","last_fullsized":"2024-05-08T22:05:19.039735+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:52.833780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.022998+0000","last_change":"2024-05-08T22:05:18.532086+0000","last_active":"2024-05-08T22:05:21.022998+0000","last_peered":"2024-05-08T22:05:21.022998+0000","last_clean":"2024-05-08T22:05:21.022998+0000","last_became_active":"2024-05-08T22:05:18.531485+0000","last_became_peered":"2024-05-08T22:05:18.531485+0000","last_unstale":"2024-05-08T22:05:21.022998+0000","last_undegraded":"2024-05-08T22:05:21.022998+0000","last_fullsized":"2024-05-08T22:05:21.022998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:40:36.050367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817288+0000","last_change":"2024-05-08T22:05:13.549611+0000","last_active":"2024-05-08T22:05:40.817288+0000","last_peered":"2024-05-08T22:05:40.817288+0000","last_clean":"2024-05-08T22:05:40.817288+0000","last_became_active":"2024-05-08T22:05:13.549140+0000","last_became_peered":"2024-05-08T22:05:13.549140+0000","last_unstale":"2024-05-08T22:05:40.817288+0000","last_undegraded":"2024-05-08T22:05:40.817288+0000","last_fullsized":"2024-05-08T22:05:40.817288+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:54.785123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714719+0000","last_change":"2024-05-08T22:05:14.057570+0000","last_active":"2024-05-08T22:05:19.714719+0000","last_peered":"2024-05-08T22:05:19.714719+0000","last_clean":"2024-05-08T22:05:19.714719+0000","last_became_active":"2024-05-08T22:05:14.057291+0000","last_became_peered":"2024-05-08T22:05:14.057291+0000","last_unstale":"2024-05-08T22:05:19.714719+0000","last_undegraded":"2024-05-08T22:05:19.714719+0000","last_fullsized":"2024-05-08T22:05:19.714719+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:15:38.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"peering","last_fresh":"2024-05-08T22:05:40.819123+0000","last_change":"2024-05-08T22:05:40.819370+0000","last_active":"2024-05-08T22:05:40.819123+0000","last_peered":"2024-05-08T22:05:21.804836+0000","last_clean":"2024-05-08T22:05:21.804836+0000","last_became_active":"2024-05-08T22:05:18.529966+0000","last_became_peered":"2024-05-08T22:05:18.529966+0000","last_unstale":"2024-05-08T22:05:40.819123+0000","last_undegraded":"2024-05-08T22:05:40.819123+0000","last_fullsized":"2024-05-08T22:05:40.819123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:28:51.858677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6,8,9],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:38.500116+0000","last_change":"2024-05-08T22:05:13.551351+0000","last_active":"2024-05-08T22:05:38.500116+0000","last_peered":"2024-05-08T22:05:38.500116+0000","last_clean":"2024-05-08T22:05:38.500116+0000","last_became_active":"2024-05-08T22:05:13.551014+0000","last_became_peered":"2024-05-08T22:05:13.551014+0000","last_unstale":"2024-05-08T22:05:38.500116+0000","last_undegraded":"2024-05-08T22:05:38.500116+0000","last_fullsized":"2024-05-08T22:05:38.500116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:07:22.405251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.834013+0000","last_change":"2024-05-08T22:05:14.067270+0000","last_active":"2024-05-08T22:05:40.834013+0000","last_peered":"2024-05-08T22:05:40.834013+0000","last_clean":"2024-05-08T22:05:40.834013+0000","last_became_active":"2024-05-08T22:05:14.065496+0000","last_became_peered":"2024-05-08T22:05:14.065496+0000","last_unstale":"2024-05-08T22:05:40.834013+0000","last_undegraded":"2024-05-08T22:05:40.834013+0000","last_fullsized":"2024-05-08T22:05:40.834013+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:07.087880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.804877+0000","last_change":"2024-05-08T22:05:18.526557+0000","last_active":"2024-05-08T22:05:21.804877+0000","last_peered":"2024-05-08T22:05:21.804877+0000","last_clean":"2024-05-08T22:05:21.804877+0000","last_became_active":"2024-05-08T22:05:18.526129+0000","last_became_peered":"2024-05-08T22:05:18.526129+0000","last_unstale":"2024-05-08T22:05:21.804877+0000","last_undegraded":"2024-05-08T22:05:21.804877+0000","last_fullsized":"2024-05-08T22:05:21.804877+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:36.376579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"78'2","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.510733+0000","last_change":"2024-05-08T22:05:13.553630+0000","last_active":"2024-05-08T22:05:18.510733+0000","last_peered":"2024-05-08T22:05:18.510733+0000","last_clean":"2024-05-08T22:05:18.510733+0000","last_became_active":"2024-05-08T22:05:13.552879+0000","last_became_peered":"2024-05-08T22:05:13.552879+0000","last_unstale":"2024-05-08T22:05:18.510733+0000","last_undegraded":"2024-05-08T22:05:18.510733+0000","last_fullsized":"2024-05-08T22:05:18.510733+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:39:58.253305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826964+0000","last_change":"2024-05-08T22:05:14.060089+0000","last_active":"2024-05-08T22:05:40.826964+0000","last_peered":"2024-05-08T22:05:40.826964+0000","last_clean":"2024-05-08T22:05:40.826964+0000","last_became_active":"2024-05-08T22:05:14.057620+0000","last_became_peered":"2024-05-08T22:05:14.057620+0000","last_unstale":"2024-05-08T22:05:40.826964+0000","last_undegraded":"2024-05-08T22:05:40.826964+0000","last_fullsized":"2024-05-08T22:05:40.826964+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:29:33.127510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.804864+0000","last_change":"2024-05-08T22:05:18.529746+0000","last_active":"2024-05-08T22:05:21.804864+0000","last_peered":"2024-05-08T22:05:21.804864+0000","last_clean":"2024-05-08T22:05:21.804864+0000","last_became_active":"2024-05-08T22:05:18.529097+0000","last_became_peered":"2024-05-08T22:05:18.529097+0000","last_unstale":"2024-05-08T22:05:21.804864+0000","last_undegraded":"2024-05-08T22:05:21.804864+0000","last_fullsized":"2024-05-08T22:05:21.804864+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:59:36.249787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.510546+0000","last_change":"2024-05-08T22:05:13.547070+0000","last_active":"2024-05-08T22:05:18.510546+0000","last_peered":"2024-05-08T22:05:18.510546+0000","last_clean":"2024-05-08T22:05:18.510546+0000","last_became_active":"2024-05-08T22:05:13.546708+0000","last_became_peered":"2024-05-08T22:05:13.546708+0000","last_unstale":"2024-05-08T22:05:18.510546+0000","last_undegraded":"2024-05-08T22:05:18.510546+0000","last_fullsized":"2024-05-08T22:05:18.510546+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:10.577900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.526371+0000","last_change":"2024-05-08T22:05:14.064695+0000","last_active":"2024-05-08T22:05:18.526371+0000","last_peered":"2024-05-08T22:05:18.526371+0000","last_clean":"2024-05-08T22:05:18.526371+0000","last_became_active":"2024-05-08T22:05:14.064283+0000","last_became_peered":"2024-05-08T22:05:14.064283+0000","last_unstale":"2024-05-08T22:05:18.526371+0000","last_undegraded":"2024-05-08T22:05:18.526371+0000","last_fullsized":"2024-05-08T22:05:18.526371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:06.980594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.023095+0000","last_change":"2024-05-08T22:05:18.528475+0000","last_active":"2024-05-08T22:05:21.023095+0000","last_peered":"2024-05-08T22:05:21.023095+0000","last_clean":"2024-05-08T22:05:21.023095+0000","last_became_active":"2024-05-08T22:05:18.528089+0000","last_became_peered":"2024-05-08T22:05:18.528089+0000","last_unstale":"2024-05-08T22:05:21.023095+0000","last_undegraded":"2024-05-08T22:05:21.023095+0000","last_fullsized":"2024-05-08T22:05:21.023095+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:01.508984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:39.502402+0000","last_change":"2024-05-08T22:05:13.549191+0000","last_active":"2024-05-08T22:05:39.502402+0000","last_peered":"2024-05-08T22:05:39.502402+0000","last_clean":"2024-05-08T22:05:39.502402+0000","last_became_active":"2024-05-08T22:05:13.548737+0000","last_became_peered":"2024-05-08T22:05:13.548737+0000","last_unstale":"2024-05-08T22:05:39.502402+0000","last_undegraded":"2024-05-08T22:05:39.502402+0000","last_fullsized":"2024-05-08T22:05:39.502402+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:07.355656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830198+0000","last_change":"2024-05-08T22:05:14.065910+0000","last_active":"2024-05-08T22:05:40.830198+0000","last_peered":"2024-05-08T22:05:40.830198+0000","last_clean":"2024-05-08T22:05:40.830198+0000","last_became_active":"2024-05-08T22:05:14.065504+0000","last_became_peered":"2024-05-08T22:05:14.065504+0000","last_unstale":"2024-05-08T22:05:40.830198+0000","last_undegraded":"2024-05-08T22:05:40.830198+0000","last_fullsized":"2024-05-08T22:05:40.830198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:12.447292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.023073+0000","last_change":"2024-05-08T22:05:18.536861+0000","last_active":"2024-05-08T22:05:21.023073+0000","last_peered":"2024-05-08T22:05:21.023073+0000","last_clean":"2024-05-08T22:05:21.023073+0000","last_became_active":"2024-05-08T22:05:18.536472+0000","last_became_peered":"2024-05-08T22:05:18.536472+0000","last_unstale":"2024-05-08T22:05:21.023073+0000","last_undegraded":"2024-05-08T22:05:21.023073+0000","last_fullsized":"2024-05-08T22:05:21.023073+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:45:21.826812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.825252+0000","last_change":"2024-05-08T22:05:13.553326+0000","last_active":"2024-05-08T22:05:40.825252+0000","last_peered":"2024-05-08T22:05:40.825252+0000","last_clean":"2024-05-08T22:05:40.825252+0000","last_became_active":"2024-05-08T22:05:13.552983+0000","last_became_peered":"2024-05-08T22:05:13.552983+0000","last_unstale":"2024-05-08T22:05:40.825252+0000","last_undegraded":"2024-05-08T22:05:40.825252+0000","last_fullsized":"2024-05-08T22:05:40.825252+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:17.195253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.526142+0000","last_change":"2024-05-08T22:05:14.068617+0000","last_active":"2024-05-08T22:05:18.526142+0000","last_peered":"2024-05-08T22:05:18.526142+0000","last_clean":"2024-05-08T22:05:18.526142+0000","last_became_active":"2024-05-08T22:05:14.067400+0000","last_became_peered":"2024-05-08T22:05:14.067400+0000","last_unstale":"2024-05-08T22:05:18.526142+0000","last_undegraded":"2024-05-08T22:05:18.526142+0000","last_fullsized":"2024-05-08T22:05:18.526142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:25.488212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.018557+0000","last_change":"2024-05-08T22:05:18.538294+0000","last_active":"2024-05-08T22:05:19.018557+0000","last_peered":"2024-05-08T22:05:19.018557+0000","last_clean":"2024-05-08T22:05:19.018557+0000","last_became_active":"2024-05-08T22:05:18.537984+0000","last_became_peered":"2024-05-08T22:05:18.537984+0000","last_unstale":"2024-05-08T22:05:19.018557+0000","last_undegraded":"2024-05-08T22:05:19.018557+0000","last_fullsized":"2024-05-08T22:05:19.018557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:32.117439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:38.500252+0000","last_change":"2024-05-08T22:05:13.557669+0000","last_active":"2024-05-08T22:05:38.500252+0000","last_peered":"2024-05-08T22:05:38.500252+0000","last_clean":"2024-05-08T22:05:38.500252+0000","last_became_active":"2024-05-08T22:05:13.555835+0000","last_became_peered":"2024-05-08T22:05:13.555835+0000","last_unstale":"2024-05-08T22:05:38.500252+0000","last_undegraded":"2024-05-08T22:05:38.500252+0000","last_fullsized":"2024-05-08T22:05:38.500252+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:02.377635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714321+0000","last_change":"2024-05-08T22:05:14.060144+0000","last_active":"2024-05-08T22:05:19.714321+0000","last_peered":"2024-05-08T22:05:19.714321+0000","last_clean":"2024-05-08T22:05:19.714321+0000","last_became_active":"2024-05-08T22:05:14.059387+0000","last_became_peered":"2024-05-08T22:05:14.059387+0000","last_unstale":"2024-05-08T22:05:19.714321+0000","last_undegraded":"2024-05-08T22:05:19.714321+0000","last_fullsized":"2024-05-08T22:05:19.714321+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:32:28.057831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832449+0000","last_change":"2024-05-08T22:05:18.527293+0000","last_active":"2024-05-08T22:05:40.832449+0000","last_peered":"2024-05-08T22:05:40.832449+0000","last_clean":"2024-05-08T22:05:40.832449+0000","last_became_active":"2024-05-08T22:05:18.526690+0000","last_became_peered":"2024-05-08T22:05:18.526690+0000","last_unstale":"2024-05-08T22:05:40.832449+0000","last_undegraded":"2024-05-08T22:05:40.832449+0000","last_fullsized":"2024-05-08T22:05:40.832449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:50.343536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:20.445763+0000","last_change":"2024-05-08T22:05:13.549774+0000","last_active":"2024-05-08T22:05:20.445763+0000","last_peered":"2024-05-08T22:05:20.445763+0000","last_clean":"2024-05-08T22:05:20.445763+0000","last_became_active":"2024-05-08T22:05:13.549531+0000","last_became_peered":"2024-05-08T22:05:13.549531+0000","last_unstale":"2024-05-08T22:05:20.445763+0000","last_undegraded":"2024-05-08T22:05:20.445763+0000","last_fullsized":"2024-05-08T22:05:20.445763+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:59:23.798728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.815983+0000","last_change":"2024-05-08T22:05:14.054232+0000","last_active":"2024-05-08T22:05:40.815983+0000","last_peered":"2024-05-08T22:05:40.815983+0000","last_clean":"2024-05-08T22:05:40.815983+0000","last_became_active":"2024-05-08T22:05:14.053806+0000","last_became_peered":"2024-05-08T22:05:14.053806+0000","last_unstale":"2024-05-08T22:05:40.815983+0000","last_undegraded":"2024-05-08T22:05:40.815983+0000","last_fullsized":"2024-05-08T22:05:40.815983+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:32:21.678792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.504963+0000","last_change":"2024-05-08T22:05:18.535241+0000","last_active":"2024-05-08T22:05:21.504963+0000","last_peered":"2024-05-08T22:05:21.504963+0000","last_clean":"2024-05-08T22:05:21.504963+0000","last_became_active":"2024-05-08T22:05:18.534753+0000","last_became_peered":"2024-05-08T22:05:18.534753+0000","last_unstale":"2024-05-08T22:05:21.504963+0000","last_undegraded":"2024-05-08T22:05:21.504963+0000","last_fullsized":"2024-05-08T22:05:21.504963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:46:15.138437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.819136+0000","last_change":"2024-05-08T22:05:13.544899+0000","last_active":"2024-05-08T22:05:41.819136+0000","last_peered":"2024-05-08T22:05:41.819136+0000","last_clean":"2024-05-08T22:05:41.819136+0000","last_became_active":"2024-05-08T22:05:13.544498+0000","last_became_peered":"2024-05-08T22:05:13.544498+0000","last_unstale":"2024-05-08T22:05:41.819136+0000","last_undegraded":"2024-05-08T22:05:41.819136+0000","last_fullsized":"2024-05-08T22:05:41.819136+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:58.169648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.823071+0000","last_change":"2024-05-08T22:05:14.049377+0000","last_active":"2024-05-08T22:05:40.823071+0000","last_peered":"2024-05-08T22:05:40.823071+0000","last_clean":"2024-05-08T22:05:40.823071+0000","last_became_active":"2024-05-08T22:05:14.049039+0000","last_became_peered":"2024-05-08T22:05:14.049039+0000","last_unstale":"2024-05-08T22:05:40.823071+0000","last_undegraded":"2024-05-08T22:05:40.823071+0000","last_fullsized":"2024-05-08T22:05:40.823071+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:24.415392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.815955+0000","last_change":"2024-05-08T22:05:18.528912+0000","last_active":"2024-05-08T22:05:40.815955+0000","last_peered":"2024-05-08T22:05:40.815955+0000","last_clean":"2024-05-08T22:05:40.815955+0000","last_became_active":"2024-05-08T22:05:18.528433+0000","last_became_peered":"2024-05-08T22:05:18.528433+0000","last_unstale":"2024-05-08T22:05:40.815955+0000","last_undegraded":"2024-05-08T22:05:40.815955+0000","last_fullsized":"2024-05-08T22:05:40.815955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:05.574728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830944+0000","last_change":"2024-05-08T22:05:13.550214+0000","last_active":"2024-05-08T22:05:40.830944+0000","last_peered":"2024-05-08T22:05:40.830944+0000","last_clean":"2024-05-08T22:05:40.830944+0000","last_became_active":"2024-05-08T22:05:13.549717+0000","last_became_peered":"2024-05-08T22:05:13.549717+0000","last_unstale":"2024-05-08T22:05:40.830944+0000","last_undegraded":"2024-05-08T22:05:40.830944+0000","last_fullsized":"2024-05-08T22:05:40.830944+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:12:20.406333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.834200+0000","last_change":"2024-05-08T22:05:14.071214+0000","last_active":"2024-05-08T22:05:40.834200+0000","last_peered":"2024-05-08T22:05:40.834200+0000","last_clean":"2024-05-08T22:05:40.834200+0000","last_became_active":"2024-05-08T22:05:14.066777+0000","last_became_peered":"2024-05-08T22:05:14.066777+0000","last_unstale":"2024-05-08T22:05:40.834200+0000","last_undegraded":"2024-05-08T22:05:40.834200+0000","last_fullsized":"2024-05-08T22:05:40.834200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:53.360911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.804896+0000","last_change":"2024-05-08T22:05:18.523688+0000","last_active":"2024-05-08T22:05:21.804896+0000","last_peered":"2024-05-08T22:05:21.804896+0000","last_clean":"2024-05-08T22:05:21.804896+0000","last_became_active":"2024-05-08T22:05:18.523236+0000","last_became_peered":"2024-05-08T22:05:18.523236+0000","last_unstale":"2024-05-08T22:05:21.804896+0000","last_undegraded":"2024-05-08T22:05:21.804896+0000","last_fullsized":"2024-05-08T22:05:21.804896+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:34.531699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.494031+0000","last_change":"2024-05-08T22:05:13.551027+0000","last_active":"2024-05-08T22:05:37.494031+0000","last_peered":"2024-05-08T22:05:37.494031+0000","last_clean":"2024-05-08T22:05:37.494031+0000","last_became_active":"2024-05-08T22:05:13.550471+0000","last_became_peered":"2024-05-08T22:05:13.550471+0000","last_unstale":"2024-05-08T22:05:37.494031+0000","last_undegraded":"2024-05-08T22:05:37.494031+0000","last_fullsized":"2024-05-08T22:05:37.494031+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:50.900419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827682+0000","last_change":"2024-05-08T22:05:14.065191+0000","last_active":"2024-05-08T22:05:40.827682+0000","last_peered":"2024-05-08T22:05:40.827682+0000","last_clean":"2024-05-08T22:05:40.827682+0000","last_became_active":"2024-05-08T22:05:14.064893+0000","last_became_peered":"2024-05-08T22:05:14.064893+0000","last_unstale":"2024-05-08T22:05:40.827682+0000","last_undegraded":"2024-05-08T22:05:40.827682+0000","last_fullsized":"2024-05-08T22:05:40.827682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:11.344646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.816088+0000","last_change":"2024-05-08T22:05:18.530447+0000","last_active":"2024-05-08T22:05:40.816088+0000","last_peered":"2024-05-08T22:05:40.816088+0000","last_clean":"2024-05-08T22:05:40.816088+0000","last_became_active":"2024-05-08T22:05:18.528171+0000","last_became_peered":"2024-05-08T22:05:18.528171+0000","last_unstale":"2024-05-08T22:05:40.816088+0000","last_undegraded":"2024-05-08T22:05:40.816088+0000","last_fullsized":"2024-05-08T22:05:40.816088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:09:43.347366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"84'23","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:42.512780+0000","last_change":"2024-05-08T22:05:13.547289+0000","last_active":"2024-05-08T22:05:42.512780+0000","last_peered":"2024-05-08T22:05:42.512780+0000","last_clean":"2024-05-08T22:05:42.512780+0000","last_became_active":"2024-05-08T22:05:13.546860+0000","last_became_peered":"2024-05-08T22:05:13.546860+0000","last_unstale":"2024-05-08T22:05:42.512780+0000","last_undegraded":"2024-05-08T22:05:42.512780+0000","last_fullsized":"2024-05-08T22:05:42.512780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:08:50.339361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40427,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":12,"num_read_kb":45,"num_write":24,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831151+0000","last_change":"2024-05-08T22:05:14.057925+0000","last_active":"2024-05-08T22:05:40.831151+0000","last_peered":"2024-05-08T22:05:40.831151+0000","last_clean":"2024-05-08T22:05:40.831151+0000","last_became_active":"2024-05-08T22:05:14.057662+0000","last_became_peered":"2024-05-08T22:05:14.057662+0000","last_unstale":"2024-05-08T22:05:40.831151+0000","last_undegraded":"2024-05-08T22:05:40.831151+0000","last_fullsized":"2024-05-08T22:05:40.831151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:49.929610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.834134+0000","last_change":"2024-05-08T22:05:18.535509+0000","last_active":"2024-05-08T22:05:19.834134+0000","last_peered":"2024-05-08T22:05:19.834134+0000","last_clean":"2024-05-08T22:05:19.834134+0000","last_became_active":"2024-05-08T22:05:18.535024+0000","last_became_peered":"2024-05-08T22:05:18.535024+0000","last_unstale":"2024-05-08T22:05:19.834134+0000","last_undegraded":"2024-05-08T22:05:19.834134+0000","last_fullsized":"2024-05-08T22:05:19.834134+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:28.098737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:39.501569+0000","last_change":"2024-05-08T22:05:13.552096+0000","last_active":"2024-05-08T22:05:39.501569+0000","last_peered":"2024-05-08T22:05:39.501569+0000","last_clean":"2024-05-08T22:05:39.501569+0000","last_became_active":"2024-05-08T22:05:13.551655+0000","last_became_peered":"2024-05-08T22:05:13.551655+0000","last_unstale":"2024-05-08T22:05:39.501569+0000","last_undegraded":"2024-05-08T22:05:39.501569+0000","last_fullsized":"2024-05-08T22:05:39.501569+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:24.371008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.153911+0000","last_change":"2024-05-08T22:05:14.066075+0000","last_active":"2024-05-08T22:05:19.153911+0000","last_peered":"2024-05-08T22:05:19.153911+0000","last_clean":"2024-05-08T22:05:19.153911+0000","last_became_active":"2024-05-08T22:05:14.065581+0000","last_became_peered":"2024-05-08T22:05:14.065581+0000","last_unstale":"2024-05-08T22:05:19.153911+0000","last_undegraded":"2024-05-08T22:05:19.153911+0000","last_fullsized":"2024-05-08T22:05:19.153911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:15.403714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.018254+0000","last_change":"2024-05-08T22:05:18.534033+0000","last_active":"2024-05-08T22:05:19.018254+0000","last_peered":"2024-05-08T22:05:19.018254+0000","last_clean":"2024-05-08T22:05:19.018254+0000","last_became_active":"2024-05-08T22:05:18.533563+0000","last_became_peered":"2024-05-08T22:05:18.533563+0000","last_unstale":"2024-05-08T22:05:19.018254+0000","last_undegraded":"2024-05-08T22:05:19.018254+0000","last_fullsized":"2024-05-08T22:05:19.018254+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:37:52.657794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.493496+0000","last_change":"2024-05-08T22:05:13.549112+0000","last_active":"2024-05-08T22:05:37.493496+0000","last_peered":"2024-05-08T22:05:37.493496+0000","last_clean":"2024-05-08T22:05:37.493496+0000","last_became_active":"2024-05-08T22:05:13.548763+0000","last_became_peered":"2024-05-08T22:05:13.548763+0000","last_unstale":"2024-05-08T22:05:37.493496+0000","last_undegraded":"2024-05-08T22:05:37.493496+0000","last_fullsized":"2024-05-08T22:05:37.493496+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:26:56.448611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830785+0000","last_change":"2024-05-08T22:05:14.067116+0000","last_active":"2024-05-08T22:05:40.830785+0000","last_peered":"2024-05-08T22:05:40.830785+0000","last_clean":"2024-05-08T22:05:40.830785+0000","last_became_active":"2024-05-08T22:05:14.064775+0000","last_became_peered":"2024-05-08T22:05:14.064775+0000","last_unstale":"2024-05-08T22:05:40.830785+0000","last_undegraded":"2024-05-08T22:05:40.830785+0000","last_fullsized":"2024-05-08T22:05:40.830785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:28:32.901405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817060+0000","last_change":"2024-05-08T22:05:18.539546+0000","last_active":"2024-05-08T22:05:40.817060+0000","last_peered":"2024-05-08T22:05:40.817060+0000","last_clean":"2024-05-08T22:05:40.817060+0000","last_became_active":"2024-05-08T22:05:18.538998+0000","last_became_peered":"2024-05-08T22:05:18.538998+0000","last_unstale":"2024-05-08T22:05:40.817060+0000","last_undegraded":"2024-05-08T22:05:40.817060+0000","last_fullsized":"2024-05-08T22:05:40.817060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:30.453881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"83'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819019+0000","last_change":"2024-05-08T22:05:13.549053+0000","last_active":"2024-05-08T22:05:40.819019+0000","last_peered":"2024-05-08T22:05:40.819019+0000","last_clean":"2024-05-08T22:05:40.819019+0000","last_became_active":"2024-05-08T22:05:13.548739+0000","last_became_peered":"2024-05-08T22:05:13.548739+0000","last_unstale":"2024-05-08T22:05:40.819019+0000","last_undegraded":"2024-05-08T22:05:40.819019+0000","last_fullsized":"2024-05-08T22:05:40.819019+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:07:10.298084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830386+0000","last_change":"2024-05-08T22:05:14.061753+0000","last_active":"2024-05-08T22:05:40.830386+0000","last_peered":"2024-05-08T22:05:40.830386+0000","last_clean":"2024-05-08T22:05:40.830386+0000","last_became_active":"2024-05-08T22:05:14.061412+0000","last_became_peered":"2024-05-08T22:05:14.061412+0000","last_unstale":"2024-05-08T22:05:40.830386+0000","last_undegraded":"2024-05-08T22:05:40.830386+0000","last_fullsized":"2024-05-08T22:05:40.830386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:54.380181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.718981+0000","last_change":"2024-05-08T22:05:18.532627+0000","last_active":"2024-05-08T22:05:21.718981+0000","last_peered":"2024-05-08T22:05:21.718981+0000","last_clean":"2024-05-08T22:05:21.718981+0000","last_became_active":"2024-05-08T22:05:18.532205+0000","last_became_peered":"2024-05-08T22:05:18.532205+0000","last_unstale":"2024-05-08T22:05:21.718981+0000","last_undegraded":"2024-05-08T22:05:21.718981+0000","last_fullsized":"2024-05-08T22:05:21.718981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:20:11.028645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":61,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.841832+0000","last_change":"2024-05-08T22:05:41.841832+0000","last_active":"2024-05-08T22:05:41.841832+0000","last_peered":"2024-05-08T22:05:41.841832+0000","last_clean":"2024-05-08T22:05:41.841832+0000","last_became_active":"2024-05-08T22:05:41.829584+0000","last_became_peered":"2024-05-08T22:05:41.829584+0000","last_unstale":"2024-05-08T22:05:41.841832+0000","last_undegraded":"2024-05-08T22:05:41.841832+0000","last_fullsized":"2024-05-08T22:05:41.841832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:52:19.876215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.525104+0000","last_change":"2024-05-08T22:05:14.063027+0000","last_active":"2024-05-08T22:05:18.525104+0000","last_peered":"2024-05-08T22:05:18.525104+0000","last_clean":"2024-05-08T22:05:18.525104+0000","last_became_active":"2024-05-08T22:05:14.062682+0000","last_became_peered":"2024-05-08T22:05:14.062682+0000","last_unstale":"2024-05-08T22:05:18.525104+0000","last_undegraded":"2024-05-08T22:05:18.525104+0000","last_fullsized":"2024-05-08T22:05:18.525104+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:47.248832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.834071+0000","last_change":"2024-05-08T22:05:18.528370+0000","last_active":"2024-05-08T22:05:19.834071+0000","last_peered":"2024-05-08T22:05:19.834071+0000","last_clean":"2024-05-08T22:05:19.834071+0000","last_became_active":"2024-05-08T22:05:18.527990+0000","last_became_peered":"2024-05-08T22:05:18.527990+0000","last_unstale":"2024-05-08T22:05:19.834071+0000","last_undegraded":"2024-05-08T22:05:19.834071+0000","last_fullsized":"2024-05-08T22:05:19.834071+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:07.524175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829860+0000","last_change":"2024-05-08T22:05:13.547543+0000","last_active":"2024-05-08T22:05:40.829860+0000","last_peered":"2024-05-08T22:05:40.829860+0000","last_clean":"2024-05-08T22:05:40.829860+0000","last_became_active":"2024-05-08T22:05:13.547191+0000","last_became_peered":"2024-05-08T22:05:13.547191+0000","last_unstale":"2024-05-08T22:05:40.829860+0000","last_undegraded":"2024-05-08T22:05:40.829860+0000","last_fullsized":"2024-05-08T22:05:40.829860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:53.213239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518149+0000","last_change":"2024-05-08T22:05:14.062677+0000","last_active":"2024-05-08T22:05:18.518149+0000","last_peered":"2024-05-08T22:05:18.518149+0000","last_clean":"2024-05-08T22:05:18.518149+0000","last_became_active":"2024-05-08T22:05:14.062306+0000","last_became_peered":"2024-05-08T22:05:14.062306+0000","last_unstale":"2024-05-08T22:05:18.518149+0000","last_undegraded":"2024-05-08T22:05:18.518149+0000","last_fullsized":"2024-05-08T22:05:18.518149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:00.356505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830129+0000","last_change":"2024-05-08T22:05:18.617075+0000","last_active":"2024-05-08T22:05:40.830129+0000","last_peered":"2024-05-08T22:05:40.830129+0000","last_clean":"2024-05-08T22:05:40.830129+0000","last_became_active":"2024-05-08T22:05:18.616653+0000","last_became_peered":"2024-05-08T22:05:18.616653+0000","last_unstale":"2024-05-08T22:05:40.830129+0000","last_undegraded":"2024-05-08T22:05:40.830129+0000","last_fullsized":"2024-05-08T22:05:40.830129+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:30.537064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.816884+0000","last_change":"2024-05-08T22:05:13.553457+0000","last_active":"2024-05-08T22:05:40.816884+0000","last_peered":"2024-05-08T22:05:40.816884+0000","last_clean":"2024-05-08T22:05:40.816884+0000","last_became_active":"2024-05-08T22:05:13.551583+0000","last_became_peered":"2024-05-08T22:05:13.551583+0000","last_unstale":"2024-05-08T22:05:40.816884+0000","last_undegraded":"2024-05-08T22:05:40.816884+0000","last_fullsized":"2024-05-08T22:05:40.816884+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:07:05.971413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.822908+0000","last_change":"2024-05-08T22:05:14.057476+0000","last_active":"2024-05-08T22:05:40.822908+0000","last_peered":"2024-05-08T22:05:40.822908+0000","last_clean":"2024-05-08T22:05:40.822908+0000","last_became_active":"2024-05-08T22:05:14.056550+0000","last_became_peered":"2024-05-08T22:05:14.056550+0000","last_unstale":"2024-05-08T22:05:40.822908+0000","last_undegraded":"2024-05-08T22:05:40.822908+0000","last_fullsized":"2024-05-08T22:05:40.822908+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:41:29.264096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.718898+0000","last_change":"2024-05-08T22:05:18.530719+0000","last_active":"2024-05-08T22:05:21.718898+0000","last_peered":"2024-05-08T22:05:21.718898+0000","last_clean":"2024-05-08T22:05:21.718898+0000","last_became_active":"2024-05-08T22:05:18.529659+0000","last_became_peered":"2024-05-08T22:05:18.529659+0000","last_unstale":"2024-05-08T22:05:21.718898+0000","last_undegraded":"2024-05-08T22:05:21.718898+0000","last_fullsized":"2024-05-08T22:05:21.718898+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:13:06.514460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830542+0000","last_change":"2024-05-08T22:05:13.556191+0000","last_active":"2024-05-08T22:05:40.830542+0000","last_peered":"2024-05-08T22:05:40.830542+0000","last_clean":"2024-05-08T22:05:40.830542+0000","last_became_active":"2024-05-08T22:05:13.555888+0000","last_became_peered":"2024-05-08T22:05:13.555888+0000","last_unstale":"2024-05-08T22:05:40.830542+0000","last_undegraded":"2024-05-08T22:05:40.830542+0000","last_fullsized":"2024-05-08T22:05:40.830542+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:05.637980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.828023+0000","last_change":"2024-05-08T22:05:41.828023+0000","last_active":"2024-05-08T22:05:41.828023+0000","last_peered":"2024-05-08T22:05:41.828023+0000","last_clean":"2024-05-08T22:05:41.828023+0000","last_became_active":"2024-05-08T22:05:41.827550+0000","last_became_peered":"2024-05-08T22:05:41.827550+0000","last_unstale":"2024-05-08T22:05:41.828023+0000","last_undegraded":"2024-05-08T22:05:41.828023+0000","last_fullsized":"2024-05-08T22:05:41.828023+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:13.234423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"peering","last_fresh":"2024-05-08T22:05:40.825443+0000","last_change":"2024-05-08T22:05:40.825855+0000","last_active":"2024-05-08T22:05:40.825443+0000","last_peered":"2024-05-08T22:05:21.719003+0000","last_clean":"2024-05-08T22:05:21.719003+0000","last_became_active":"2024-05-08T22:05:18.616536+0000","last_became_peered":"2024-05-08T22:05:18.616536+0000","last_unstale":"2024-05-08T22:05:40.825443+0000","last_undegraded":"2024-05-08T22:05:40.825443+0000","last_fullsized":"2024-05-08T22:05:40.825443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:30:44.106162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5,8,10],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.525034+0000","last_change":"2024-05-08T22:05:14.069096+0000","last_active":"2024-05-08T22:05:18.525034+0000","last_peered":"2024-05-08T22:05:18.525034+0000","last_clean":"2024-05-08T22:05:18.525034+0000","last_became_active":"2024-05-08T22:05:14.068796+0000","last_became_peered":"2024-05-08T22:05:14.068796+0000","last_unstale":"2024-05-08T22:05:18.525034+0000","last_undegraded":"2024-05-08T22:05:18.525034+0000","last_fullsized":"2024-05-08T22:05:18.525034+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:52:52.391196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.821076+0000","last_change":"2024-05-08T22:05:13.559286+0000","last_active":"2024-05-08T22:05:40.821076+0000","last_peered":"2024-05-08T22:05:40.821076+0000","last_clean":"2024-05-08T22:05:40.821076+0000","last_became_active":"2024-05-08T22:05:13.558922+0000","last_became_peered":"2024-05-08T22:05:13.558922+0000","last_unstale":"2024-05-08T22:05:40.821076+0000","last_undegraded":"2024-05-08T22:05:40.821076+0000","last_fullsized":"2024-05-08T22:05:40.821076+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:25:24.732422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831007+0000","last_change":"2024-05-08T22:05:18.527031+0000","last_active":"2024-05-08T22:05:40.831007+0000","last_peered":"2024-05-08T22:05:40.831007+0000","last_clean":"2024-05-08T22:05:40.831007+0000","last_became_active":"2024-05-08T22:05:18.526573+0000","last_became_peered":"2024-05-08T22:05:18.526573+0000","last_unstale":"2024-05-08T22:05:40.831007+0000","last_undegraded":"2024-05-08T22:05:40.831007+0000","last_fullsized":"2024-05-08T22:05:40.831007+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:08.492359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714236+0000","last_change":"2024-05-08T22:05:14.053930+0000","last_active":"2024-05-08T22:05:19.714236+0000","last_peered":"2024-05-08T22:05:19.714236+0000","last_clean":"2024-05-08T22:05:19.714236+0000","last_became_active":"2024-05-08T22:05:14.052968+0000","last_became_peered":"2024-05-08T22:05:14.052968+0000","last_unstale":"2024-05-08T22:05:19.714236+0000","last_undegraded":"2024-05-08T22:05:19.714236+0000","last_fullsized":"2024-05-08T22:05:19.714236+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:48.748323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:39.501541+0000","last_change":"2024-05-08T22:05:13.545709+0000","last_active":"2024-05-08T22:05:39.501541+0000","last_peered":"2024-05-08T22:05:39.501541+0000","last_clean":"2024-05-08T22:05:39.501541+0000","last_became_active":"2024-05-08T22:05:13.545419+0000","last_became_peered":"2024-05-08T22:05:13.545419+0000","last_unstale":"2024-05-08T22:05:39.501541+0000","last_undegraded":"2024-05-08T22:05:39.501541+0000","last_fullsized":"2024-05-08T22:05:39.501541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:44.903903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":337,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830206+0000","last_change":"2024-05-08T22:04:20.948328+0000","last_active":"2024-05-08T22:05:40.830206+0000","last_peered":"2024-05-08T22:05:40.830206+0000","last_clean":"2024-05-08T22:05:40.830206+0000","last_became_active":"2024-05-08T22:04:20.947887+0000","last_became_peered":"2024-05-08T22:04:20.947887+0000","last_unstale":"2024-05-08T22:05:40.830206+0000","last_undegraded":"2024-05-08T22:05:40.830206+0000","last_fullsized":"2024-05-08T22:05:40.830206+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:01:41.485277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:01:41.485277+0000","last_clean_scrub_stamp":"2024-05-08T22:01:41.485277+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:43:59.208120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.834146+0000","last_change":"2024-05-08T22:05:18.616989+0000","last_active":"2024-05-08T22:05:19.834146+0000","last_peered":"2024-05-08T22:05:19.834146+0000","last_clean":"2024-05-08T22:05:19.834146+0000","last_became_active":"2024-05-08T22:05:18.616560+0000","last_became_peered":"2024-05-08T22:05:18.616560+0000","last_unstale":"2024-05-08T22:05:19.834146+0000","last_undegraded":"2024-05-08T22:05:19.834146+0000","last_fullsized":"2024-05-08T22:05:19.834146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:45:31.141303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.153692+0000","last_change":"2024-05-08T22:05:14.066818+0000","last_active":"2024-05-08T22:05:19.153692+0000","last_peered":"2024-05-08T22:05:19.153692+0000","last_clean":"2024-05-08T22:05:19.153692+0000","last_became_active":"2024-05-08T22:05:14.065152+0000","last_became_peered":"2024-05-08T22:05:14.065152+0000","last_unstale":"2024-05-08T22:05:19.153692+0000","last_undegraded":"2024-05-08T22:05:19.153692+0000","last_fullsized":"2024-05-08T22:05:19.153692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:53:19.018852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830323+0000","last_change":"2024-05-08T22:05:13.550365+0000","last_active":"2024-05-08T22:05:40.830323+0000","last_peered":"2024-05-08T22:05:40.830323+0000","last_clean":"2024-05-08T22:05:40.830323+0000","last_became_active":"2024-05-08T22:05:13.549677+0000","last_became_peered":"2024-05-08T22:05:13.549677+0000","last_unstale":"2024-05-08T22:05:40.830323+0000","last_undegraded":"2024-05-08T22:05:40.830323+0000","last_fullsized":"2024-05-08T22:05:40.830323+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:10.842838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.018502+0000","last_change":"2024-05-08T22:05:18.535777+0000","last_active":"2024-05-08T22:05:19.018502+0000","last_peered":"2024-05-08T22:05:19.018502+0000","last_clean":"2024-05-08T22:05:19.018502+0000","last_became_active":"2024-05-08T22:05:18.535327+0000","last_became_peered":"2024-05-08T22:05:18.535327+0000","last_unstale":"2024-05-08T22:05:19.018502+0000","last_undegraded":"2024-05-08T22:05:19.018502+0000","last_fullsized":"2024-05-08T22:05:19.018502+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:13.886989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831170+0000","last_change":"2024-05-08T22:05:13.554045+0000","last_active":"2024-05-08T22:05:40.831170+0000","last_peered":"2024-05-08T22:05:40.831170+0000","last_clean":"2024-05-08T22:05:40.831170+0000","last_became_active":"2024-05-08T22:05:13.553315+0000","last_became_peered":"2024-05-08T22:05:13.553315+0000","last_unstale":"2024-05-08T22:05:40.831170+0000","last_undegraded":"2024-05-08T22:05:40.831170+0000","last_fullsized":"2024-05-08T22:05:40.831170+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:13.131267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.154279+0000","last_change":"2024-05-08T22:05:14.065018+0000","last_active":"2024-05-08T22:05:19.154279+0000","last_peered":"2024-05-08T22:05:19.154279+0000","last_clean":"2024-05-08T22:05:19.154279+0000","last_became_active":"2024-05-08T22:05:14.064701+0000","last_became_peered":"2024-05-08T22:05:14.064701+0000","last_unstale":"2024-05-08T22:05:19.154279+0000","last_undegraded":"2024-05-08T22:05:19.154279+0000","last_fullsized":"2024-05-08T22:05:19.154279+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:48:43.468373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.504924+0000","last_change":"2024-05-08T22:05:18.617853+0000","last_active":"2024-05-08T22:05:21.504924+0000","last_peered":"2024-05-08T22:05:21.504924+0000","last_clean":"2024-05-08T22:05:21.504924+0000","last_became_active":"2024-05-08T22:05:18.616255+0000","last_became_peered":"2024-05-08T22:05:18.616255+0000","last_unstale":"2024-05-08T22:05:21.504924+0000","last_undegraded":"2024-05-08T22:05:21.504924+0000","last_fullsized":"2024-05-08T22:05:21.504924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:13.233425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.817925+0000","last_change":"2024-05-08T22:05:13.546274+0000","last_active":"2024-05-08T22:05:41.817925+0000","last_peered":"2024-05-08T22:05:41.817925+0000","last_clean":"2024-05-08T22:05:41.817925+0000","last_became_active":"2024-05-08T22:05:13.543651+0000","last_became_peered":"2024-05-08T22:05:13.543651+0000","last_unstale":"2024-05-08T22:05:41.817925+0000","last_undegraded":"2024-05-08T22:05:41.817925+0000","last_fullsized":"2024-05-08T22:05:41.817925+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:21:58.601261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714604+0000","last_change":"2024-05-08T22:05:14.056535+0000","last_active":"2024-05-08T22:05:19.714604+0000","last_peered":"2024-05-08T22:05:19.714604+0000","last_clean":"2024-05-08T22:05:19.714604+0000","last_became_active":"2024-05-08T22:05:14.056211+0000","last_became_peered":"2024-05-08T22:05:14.056211+0000","last_unstale":"2024-05-08T22:05:19.714604+0000","last_undegraded":"2024-05-08T22:05:19.714604+0000","last_fullsized":"2024-05-08T22:05:19.714604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:46:30.744490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.022982+0000","last_change":"2024-05-08T22:05:18.524939+0000","last_active":"2024-05-08T22:05:21.022982+0000","last_peered":"2024-05-08T22:05:21.022982+0000","last_clean":"2024-05-08T22:05:21.022982+0000","last_became_active":"2024-05-08T22:05:18.524518+0000","last_became_peered":"2024-05-08T22:05:18.524518+0000","last_unstale":"2024-05-08T22:05:21.022982+0000","last_undegraded":"2024-05-08T22:05:21.022982+0000","last_fullsized":"2024-05-08T22:05:21.022982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:50.192400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.493290+0000","last_change":"2024-05-08T22:05:13.554346+0000","last_active":"2024-05-08T22:05:37.493290+0000","last_peered":"2024-05-08T22:05:37.493290+0000","last_clean":"2024-05-08T22:05:37.493290+0000","last_became_active":"2024-05-08T22:05:13.554080+0000","last_became_peered":"2024-05-08T22:05:13.554080+0000","last_unstale":"2024-05-08T22:05:37.493290+0000","last_undegraded":"2024-05-08T22:05:37.493290+0000","last_fullsized":"2024-05-08T22:05:37.493290+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:29:49.695682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832687+0000","last_change":"2024-05-08T22:05:14.071422+0000","last_active":"2024-05-08T22:05:40.832687+0000","last_peered":"2024-05-08T22:05:40.832687+0000","last_clean":"2024-05-08T22:05:40.832687+0000","last_became_active":"2024-05-08T22:05:14.070396+0000","last_became_peered":"2024-05-08T22:05:14.070396+0000","last_unstale":"2024-05-08T22:05:40.832687+0000","last_undegraded":"2024-05-08T22:05:40.832687+0000","last_fullsized":"2024-05-08T22:05:40.832687+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:37.957983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.804960+0000","last_change":"2024-05-08T22:05:18.617149+0000","last_active":"2024-05-08T22:05:21.804960+0000","last_peered":"2024-05-08T22:05:21.804960+0000","last_clean":"2024-05-08T22:05:21.804960+0000","last_became_active":"2024-05-08T22:05:18.616699+0000","last_became_peered":"2024-05-08T22:05:18.616699+0000","last_unstale":"2024-05-08T22:05:21.804960+0000","last_undegraded":"2024-05-08T22:05:21.804960+0000","last_fullsized":"2024-05-08T22:05:21.804960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:51.167142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.823777+0000","last_change":"2024-05-08T22:05:13.551212+0000","last_active":"2024-05-08T22:05:40.823777+0000","last_peered":"2024-05-08T22:05:40.823777+0000","last_clean":"2024-05-08T22:05:40.823777+0000","last_became_active":"2024-05-08T22:05:13.550909+0000","last_became_peered":"2024-05-08T22:05:13.550909+0000","last_unstale":"2024-05-08T22:05:40.823777+0000","last_undegraded":"2024-05-08T22:05:40.823777+0000","last_fullsized":"2024-05-08T22:05:40.823777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:48.657928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829703+0000","last_change":"2024-05-08T22:05:14.053976+0000","last_active":"2024-05-08T22:05:40.829703+0000","last_peered":"2024-05-08T22:05:40.829703+0000","last_clean":"2024-05-08T22:05:40.829703+0000","last_became_active":"2024-05-08T22:05:14.053664+0000","last_became_peered":"2024-05-08T22:05:14.053664+0000","last_unstale":"2024-05-08T22:05:40.829703+0000","last_undegraded":"2024-05-08T22:05:40.829703+0000","last_fullsized":"2024-05-08T22:05:40.829703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:25:25.642768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817203+0000","last_change":"2024-05-08T22:05:18.536771+0000","last_active":"2024-05-08T22:05:40.817203+0000","last_peered":"2024-05-08T22:05:40.817203+0000","last_clean":"2024-05-08T22:05:40.817203+0000","last_became_active":"2024-05-08T22:05:18.536260+0000","last_became_peered":"2024-05-08T22:05:18.536260+0000","last_unstale":"2024-05-08T22:05:40.817203+0000","last_undegraded":"2024-05-08T22:05:40.817203+0000","last_fullsized":"2024-05-08T22:05:40.817203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:34.414381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:37.494220+0000","last_change":"2024-05-08T22:05:13.540679+0000","last_active":"2024-05-08T22:05:37.494220+0000","last_peered":"2024-05-08T22:05:37.494220+0000","last_clean":"2024-05-08T22:05:37.494220+0000","last_became_active":"2024-05-08T22:05:13.539428+0000","last_became_peered":"2024-05-08T22:05:13.539428+0000","last_unstale":"2024-05-08T22:05:37.494220+0000","last_undegraded":"2024-05-08T22:05:37.494220+0000","last_fullsized":"2024-05-08T22:05:37.494220+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:59:34.455411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518423+0000","last_change":"2024-05-08T22:05:14.061849+0000","last_active":"2024-05-08T22:05:18.518423+0000","last_peered":"2024-05-08T22:05:18.518423+0000","last_clean":"2024-05-08T22:05:18.518423+0000","last_became_active":"2024-05-08T22:05:14.061028+0000","last_became_peered":"2024-05-08T22:05:14.061028+0000","last_unstale":"2024-05-08T22:05:18.518423+0000","last_undegraded":"2024-05-08T22:05:18.518423+0000","last_fullsized":"2024-05-08T22:05:18.518423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:11:59.489376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819278+0000","last_change":"2024-05-08T22:05:18.517994+0000","last_active":"2024-05-08T22:05:40.819278+0000","last_peered":"2024-05-08T22:05:40.819278+0000","last_clean":"2024-05-08T22:05:40.819278+0000","last_became_active":"2024-05-08T22:05:18.517655+0000","last_became_peered":"2024-05-08T22:05:18.517655+0000","last_unstale":"2024-05-08T22:05:40.819278+0000","last_undegraded":"2024-05-08T22:05:40.819278+0000","last_fullsized":"2024-05-08T22:05:40.819278+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:50:08.956277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.039182+0000","last_change":"2024-05-08T22:05:13.552851+0000","last_active":"2024-05-08T22:05:19.039182+0000","last_peered":"2024-05-08T22:05:19.039182+0000","last_clean":"2024-05-08T22:05:19.039182+0000","last_became_active":"2024-05-08T22:05:13.552507+0000","last_became_peered":"2024-05-08T22:05:13.552507+0000","last_unstale":"2024-05-08T22:05:19.039182+0000","last_undegraded":"2024-05-08T22:05:19.039182+0000","last_fullsized":"2024-05-08T22:05:19.039182+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:29.248623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714486+0000","last_change":"2024-05-08T22:05:14.062618+0000","last_active":"2024-05-08T22:05:19.714486+0000","last_peered":"2024-05-08T22:05:19.714486+0000","last_clean":"2024-05-08T22:05:19.714486+0000","last_became_active":"2024-05-08T22:05:14.062308+0000","last_became_peered":"2024-05-08T22:05:14.062308+0000","last_unstale":"2024-05-08T22:05:19.714486+0000","last_undegraded":"2024-05-08T22:05:19.714486+0000","last_fullsized":"2024-05-08T22:05:19.714486+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:34.195343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:21.804949+0000","last_change":"2024-05-08T22:05:18.616846+0000","last_active":"2024-05-08T22:05:21.804949+0000","last_peered":"2024-05-08T22:05:21.804949+0000","last_clean":"2024-05-08T22:05:21.804949+0000","last_became_active":"2024-05-08T22:05:18.616369+0000","last_became_peered":"2024-05-08T22:05:18.616369+0000","last_unstale":"2024-05-08T22:05:21.804949+0000","last_undegraded":"2024-05-08T22:05:21.804949+0000","last_fullsized":"2024-05-08T22:05:21.804949+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:26.128949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829370+0000","last_change":"2024-05-08T22:05:13.549712+0000","last_active":"2024-05-08T22:05:40.829370+0000","last_peered":"2024-05-08T22:05:40.829370+0000","last_clean":"2024-05-08T22:05:40.829370+0000","last_became_active":"2024-05-08T22:05:13.549508+0000","last_became_peered":"2024-05-08T22:05:13.549508+0000","last_unstale":"2024-05-08T22:05:40.829370+0000","last_undegraded":"2024-05-08T22:05:40.829370+0000","last_fullsized":"2024-05-08T22:05:40.829370+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:30:22.706389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830535+0000","last_change":"2024-05-08T22:05:14.060920+0000","last_active":"2024-05-08T22:05:40.830535+0000","last_peered":"2024-05-08T22:05:40.830535+0000","last_clean":"2024-05-08T22:05:40.830535+0000","last_became_active":"2024-05-08T22:05:14.059914+0000","last_became_peered":"2024-05-08T22:05:14.059914+0000","last_unstale":"2024-05-08T22:05:40.830535+0000","last_undegraded":"2024-05-08T22:05:40.830535+0000","last_fullsized":"2024-05-08T22:05:40.830535+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:15.513600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.822527+0000","last_change":"2024-05-08T22:05:18.534076+0000","last_active":"2024-05-08T22:05:40.822527+0000","last_peered":"2024-05-08T22:05:40.822527+0000","last_clean":"2024-05-08T22:05:40.822527+0000","last_became_active":"2024-05-08T22:05:18.533679+0000","last_became_peered":"2024-05-08T22:05:18.533679+0000","last_unstale":"2024-05-08T22:05:40.822527+0000","last_undegraded":"2024-05-08T22:05:40.822527+0000","last_fullsized":"2024-05-08T22:05:40.822527+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:08.979193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.845713+0000","last_change":"2024-05-08T22:05:41.845354+0000","last_active":"2024-05-08T22:05:41.845713+0000","last_peered":"2024-05-08T22:05:41.845713+0000","last_clean":"2024-05-08T22:05:41.845713+0000","last_became_active":"2024-05-08T22:05:41.830506+0000","last_became_peered":"2024-05-08T22:05:41.830506+0000","last_unstale":"2024-05-08T22:05:41.845713+0000","last_undegraded":"2024-05-08T22:05:41.845713+0000","last_fullsized":"2024-05-08T22:05:41.845713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:53:23.081939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831111+0000","last_change":"2024-05-08T22:05:14.065868+0000","last_active":"2024-05-08T22:05:40.831111+0000","last_peered":"2024-05-08T22:05:40.831111+0000","last_clean":"2024-05-08T22:05:40.831111+0000","last_became_active":"2024-05-08T22:05:14.065545+0000","last_became_peered":"2024-05-08T22:05:14.065545+0000","last_unstale":"2024-05-08T22:05:40.831111+0000","last_undegraded":"2024-05-08T22:05:40.831111+0000","last_fullsized":"2024-05-08T22:05:40.831111+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:56:34.309095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832203+0000","last_change":"2024-05-08T22:05:18.534261+0000","last_active":"2024-05-08T22:05:40.832203+0000","last_peered":"2024-05-08T22:05:40.832203+0000","last_clean":"2024-05-08T22:05:40.832203+0000","last_became_active":"2024-05-08T22:05:18.531814+0000","last_became_peered":"2024-05-08T22:05:18.531814+0000","last_unstale":"2024-05-08T22:05:40.832203+0000","last_undegraded":"2024-05-08T22:05:40.832203+0000","last_fullsized":"2024-05-08T22:05:40.832203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:38.344546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:20.446341+0000","last_change":"2024-05-08T22:05:13.549344+0000","last_active":"2024-05-08T22:05:20.446341+0000","last_peered":"2024-05-08T22:05:20.446341+0000","last_clean":"2024-05-08T22:05:20.446341+0000","last_became_active":"2024-05-08T22:05:13.548993+0000","last_became_peered":"2024-05-08T22:05:13.548993+0000","last_unstale":"2024-05-08T22:05:20.446341+0000","last_undegraded":"2024-05-08T22:05:20.446341+0000","last_fullsized":"2024-05-08T22:05:20.446341+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:50.096602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830884+0000","last_change":"2024-05-08T22:05:14.065758+0000","last_active":"2024-05-08T22:05:40.830884+0000","last_peered":"2024-05-08T22:05:40.830884+0000","last_clean":"2024-05-08T22:05:40.830884+0000","last_became_active":"2024-05-08T22:05:14.065442+0000","last_became_peered":"2024-05-08T22:05:14.065442+0000","last_unstale":"2024-05-08T22:05:40.830884+0000","last_undegraded":"2024-05-08T22:05:40.830884+0000","last_fullsized":"2024-05-08T22:05:40.830884+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:43.047796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.834022+0000","last_change":"2024-05-08T22:05:18.529804+0000","last_active":"2024-05-08T22:05:19.834022+0000","last_peered":"2024-05-08T22:05:19.834022+0000","last_clean":"2024-05-08T22:05:19.834022+0000","last_became_active":"2024-05-08T22:05:18.529346+0000","last_became_peered":"2024-05-08T22:05:18.529346+0000","last_unstale":"2024-05-08T22:05:19.834022+0000","last_undegraded":"2024-05-08T22:05:19.834022+0000","last_fullsized":"2024-05-08T22:05:19.834022+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:13.565707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.154118+0000","last_change":"2024-05-08T22:05:13.551818+0000","last_active":"2024-05-08T22:05:19.154118+0000","last_peered":"2024-05-08T22:05:19.154118+0000","last_clean":"2024-05-08T22:05:19.154118+0000","last_became_active":"2024-05-08T22:05:13.551080+0000","last_became_peered":"2024-05-08T22:05:13.551080+0000","last_unstale":"2024-05-08T22:05:19.154118+0000","last_undegraded":"2024-05-08T22:05:19.154118+0000","last_fullsized":"2024-05-08T22:05:19.154118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:31.636482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831045+0000","last_change":"2024-05-08T22:05:14.066572+0000","last_active":"2024-05-08T22:05:40.831045+0000","last_peered":"2024-05-08T22:05:40.831045+0000","last_clean":"2024-05-08T22:05:40.831045+0000","last_became_active":"2024-05-08T22:05:14.066272+0000","last_became_peered":"2024-05-08T22:05:14.066272+0000","last_unstale":"2024-05-08T22:05:40.831045+0000","last_undegraded":"2024-05-08T22:05:40.831045+0000","last_fullsized":"2024-05-08T22:05:40.831045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:20:27.423464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518519+0000","last_change":"2024-05-08T22:05:14.061435+0000","last_active":"2024-05-08T22:05:18.518519+0000","last_peered":"2024-05-08T22:05:18.518519+0000","last_clean":"2024-05-08T22:05:18.518519+0000","last_became_active":"2024-05-08T22:05:14.061079+0000","last_became_peered":"2024-05-08T22:05:14.061079+0000","last_unstale":"2024-05-08T22:05:18.518519+0000","last_undegraded":"2024-05-08T22:05:18.518519+0000","last_fullsized":"2024-05-08T22:05:18.518519+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:42.684743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"peering","last_fresh":"2024-05-08T22:05:40.817668+0000","last_change":"2024-05-08T22:05:40.817845+0000","last_active":"2024-05-08T22:05:40.817668+0000","last_peered":"2024-05-08T22:05:19.039319+0000","last_clean":"2024-05-08T22:05:19.039319+0000","last_became_active":"2024-05-08T22:05:14.071184+0000","last_became_peered":"2024-05-08T22:05:14.071184+0000","last_unstale":"2024-05-08T22:05:40.817668+0000","last_undegraded":"2024-05-08T22:05:40.817668+0000","last_fullsized":"2024-05-08T22:05:40.817668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:12:55.547522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5,8],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.039153+0000","last_change":"2024-05-08T22:05:14.066697+0000","last_active":"2024-05-08T22:05:19.039153+0000","last_peered":"2024-05-08T22:05:19.039153+0000","last_clean":"2024-05-08T22:05:19.039153+0000","last_became_active":"2024-05-08T22:05:14.065952+0000","last_became_peered":"2024-05-08T22:05:14.065952+0000","last_unstale":"2024-05-08T22:05:19.039153+0000","last_undegraded":"2024-05-08T22:05:19.039153+0000","last_fullsized":"2024-05-08T22:05:19.039153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:03:19.006644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.833301+0000","last_change":"2024-05-08T22:05:14.072295+0000","last_active":"2024-05-08T22:05:40.833301+0000","last_peered":"2024-05-08T22:05:40.833301+0000","last_clean":"2024-05-08T22:05:40.833301+0000","last_became_active":"2024-05-08T22:05:14.065034+0000","last_became_peered":"2024-05-08T22:05:14.065034+0000","last_unstale":"2024-05-08T22:05:40.833301+0000","last_undegraded":"2024-05-08T22:05:40.833301+0000","last_fullsized":"2024-05-08T22:05:40.833301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:08:41.380814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.154280+0000","last_change":"2024-05-08T22:05:14.067263+0000","last_active":"2024-05-08T22:05:19.154280+0000","last_peered":"2024-05-08T22:05:19.154280+0000","last_clean":"2024-05-08T22:05:19.154280+0000","last_became_active":"2024-05-08T22:05:14.066957+0000","last_became_peered":"2024-05-08T22:05:14.066957+0000","last_unstale":"2024-05-08T22:05:19.154280+0000","last_undegraded":"2024-05-08T22:05:19.154280+0000","last_fullsized":"2024-05-08T22:05:19.154280+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:29:35.638489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518101+0000","last_change":"2024-05-08T22:05:14.060561+0000","last_active":"2024-05-08T22:05:18.518101+0000","last_peered":"2024-05-08T22:05:18.518101+0000","last_clean":"2024-05-08T22:05:18.518101+0000","last_became_active":"2024-05-08T22:05:14.060150+0000","last_became_peered":"2024-05-08T22:05:14.060150+0000","last_unstale":"2024-05-08T22:05:18.518101+0000","last_undegraded":"2024-05-08T22:05:18.518101+0000","last_fullsized":"2024-05-08T22:05:18.518101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:22.710776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.039533+0000","last_change":"2024-05-08T22:05:14.056394+0000","last_active":"2024-05-08T22:05:19.039533+0000","last_peered":"2024-05-08T22:05:19.039533+0000","last_clean":"2024-05-08T22:05:19.039533+0000","last_became_active":"2024-05-08T22:05:14.055977+0000","last_became_peered":"2024-05-08T22:05:14.055977+0000","last_unstale":"2024-05-08T22:05:19.039533+0000","last_undegraded":"2024-05-08T22:05:19.039533+0000","last_fullsized":"2024-05-08T22:05:19.039533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:56:48.249379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714720+0000","last_change":"2024-05-08T22:05:14.061934+0000","last_active":"2024-05-08T22:05:19.714720+0000","last_peered":"2024-05-08T22:05:19.714720+0000","last_clean":"2024-05-08T22:05:19.714720+0000","last_became_active":"2024-05-08T22:05:14.061635+0000","last_became_peered":"2024-05-08T22:05:14.061635+0000","last_unstale":"2024-05-08T22:05:19.714720+0000","last_undegraded":"2024-05-08T22:05:19.714720+0000","last_fullsized":"2024-05-08T22:05:19.714720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:02.423338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.524933+0000","last_change":"2024-05-08T22:05:14.069982+0000","last_active":"2024-05-08T22:05:18.524933+0000","last_peered":"2024-05-08T22:05:18.524933+0000","last_clean":"2024-05-08T22:05:18.524933+0000","last_became_active":"2024-05-08T22:05:14.068293+0000","last_became_peered":"2024-05-08T22:05:14.068293+0000","last_unstale":"2024-05-08T22:05:18.524933+0000","last_undegraded":"2024-05-08T22:05:18.524933+0000","last_fullsized":"2024-05-08T22:05:18.524933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:36.396983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.714278+0000","last_change":"2024-05-08T22:05:14.058393+0000","last_active":"2024-05-08T22:05:19.714278+0000","last_peered":"2024-05-08T22:05:19.714278+0000","last_clean":"2024-05-08T22:05:19.714278+0000","last_became_active":"2024-05-08T22:05:14.058083+0000","last_became_peered":"2024-05-08T22:05:14.058083+0000","last_unstale":"2024-05-08T22:05:19.714278+0000","last_undegraded":"2024-05-08T22:05:19.714278+0000","last_fullsized":"2024-05-08T22:05:19.714278+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:10:47.074957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.822859+0000","last_change":"2024-05-08T22:05:14.056989+0000","last_active":"2024-05-08T22:05:40.822859+0000","last_peered":"2024-05-08T22:05:40.822859+0000","last_clean":"2024-05-08T22:05:40.822859+0000","last_became_active":"2024-05-08T22:05:14.056669+0000","last_became_peered":"2024-05-08T22:05:14.056669+0000","last_unstale":"2024-05-08T22:05:40.822859+0000","last_undegraded":"2024-05-08T22:05:40.822859+0000","last_fullsized":"2024-05-08T22:05:40.822859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:56:51.173885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T22:05:19.153888+0000","last_change":"2024-05-08T22:05:14.058635+0000","last_active":"2024-05-08T22:05:19.153888+0000","last_peered":"2024-05-08T22:05:19.153888+0000","last_clean":"2024-05-08T22:05:19.153888+0000","last_became_active":"2024-05-08T22:05:14.058243+0000","last_became_peered":"2024-05-08T22:05:14.058243+0000","last_unstale":"2024-05-08T22:05:19.153888+0000","last_undegraded":"2024-05-08T22:05:19.153888+0000","last_fullsized":"2024-05-08T22:05:19.153888+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:18.142563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:05:18.518557+0000","last_change":"2024-05-08T22:05:14.060735+0000","last_active":"2024-05-08T22:05:18.518557+0000","last_peered":"2024-05-08T22:05:18.518557+0000","last_clean":"2024-05-08T22:05:18.518557+0000","last_became_active":"2024-05-08T22:05:14.060391+0000","last_became_peered":"2024-05-08T22:05:14.060391+0000","last_unstale":"2024-05-08T22:05:18.518557+0000","last_undegraded":"2024-05-08T22:05:18.518557+0000","last_fullsized":"2024-05-08T22:05:18.518557+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:30.528705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":28672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283045,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":43941,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":39,"num_read_kb":68,"num_write":138,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":348160,"data_stored":121061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579920,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36088,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709064,"statfs":{"total":95995035648,"available":95958081536,"internally_reserved":0,"allocated":901120,"data_stored":447161,"data_compressed":11671,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36043205},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.36899999999999999}]},{"osd":1,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.46000000000000002}]},{"osd":2,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.47299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":4,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.504}]},{"osd":5,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.622}]},{"osd":6,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.68200000000000005}]},{"osd":7,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.60199999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":9,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.73499999999999999}]},{"osd":10,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.748}]}]},{"osd":10,"up_from":67,"seq":287762808851,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":1048576,"data_stored":1097367,"data_compressed":53579,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":1,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.443}]},{"osd":2,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.60199999999999998}]},{"osd":3,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.623}]},{"osd":4,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.48599999999999999}]},{"osd":5,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.68899999999999995}]},{"osd":6,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.754},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":8,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.82899999999999996}]},{"osd":9,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.85499999999999998}]},{"osd":11,"last update":"Wed May 8 22:05:39 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.80700000000000005}]}]},{"osd":9,"up_from":62,"seq":266287972376,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":474383,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.57399999999999995}]},{"osd":1,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.68600000000000005}]},{"osd":2,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.749},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.66300000000000003}]},{"osd":3,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.60099999999999998}]},{"osd":4,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.85599999999999998}]},{"osd":5,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.81699999999999995}]},{"osd":6,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.71199999999999997}]},{"osd":7,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.80600000000000005}]},{"osd":8,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.84499999999999997}]},{"osd":10,"last update":"Wed May 8 22:05:22 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.877}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]},{"osd":8,"up_from":55,"seq":236223201308,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":469876,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":0.40799999999999997}]},{"osd":1,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.47199999999999998}]},{"osd":2,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.42499999999999999}]},{"osd":3,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":4.8920000000000003,"5min":4.8920000000000003,"15min":4.8920000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.67100000000000004}]},{"osd":4,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.627}]},{"osd":5,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.52500000000000002}]},{"osd":6,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.9210000000000003,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.77900000000000003}]},{"osd":7,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.748},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.64700000000000002}]},{"osd":9,"last update":"Wed May 8 22:05:01 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.80600000000000005}]},{"osd":10,"last update":"Wed May 8 22:05:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.79100000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]}]},{"osd":7,"up_from":48,"seq":206158430238,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":1066022,"data_compressed":53579,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.41199999999999998}]},{"osd":1,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.39500000000000002,"15min":0.38700000000000001},"min":{"1min":0.30599999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.443},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40600000000000003,"15min":0.40300000000000002},"min":{"1min":0.29199999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.36699999999999999}]},{"osd":2,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54100000000000004,"15min":0.53600000000000003},"min":{"1min":0.34899999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.38400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70299999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.68000000000000005}]},{"osd":3,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.41599999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79800000000000004,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.501,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80200000000000005,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.52900000000000003}]},{"osd":4,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.33800000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71999999999999997,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.754,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.51000000000000001}]},{"osd":5,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64400000000000002,"15min":0.64500000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89900000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.625,"15min":0.626},"min":{"1min":0.44500000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.77700000000000002,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.63200000000000001}]},{"osd":6,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85099999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69699999999999995,"15min":0.69899999999999995},"min":{"1min":0.48199999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89000000000000001,"5min":1.002,"15min":1.002},"last":0.46800000000000003}]},{"osd":8,"last update":"Wed May 8 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.68700000000000006,"15min":0.69399999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.78000000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.72099999999999997,"15min":0.72699999999999998},"min":{"1min":0.55300000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.85899999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.74099999999999999}]},{"osd":9,"last update":"Wed May 8 22:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.61699999999999999}]},{"osd":10,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.70799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]}]},{"osd":6,"up_from":43,"seq":184683593762,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":455906,"data_compressed":14627,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.46200000000000002,"15min":0.45100000000000001},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.66300000000000003,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.51000000000000001,"15min":0.51300000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.64400000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.45900000000000002}]},{"osd":1,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57299999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.61899999999999999,"15min":0.60599999999999998},"min":{"1min":0.47399999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.77800000000000002}]},{"osd":2,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.66200000000000003,"15min":0.64800000000000002},"min":{"1min":0.53200000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65200000000000002,"15min":0.64700000000000002},"min":{"1min":0.51400000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89300000000000002,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.65300000000000002}]},{"osd":3,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60999999999999999,"15min":0.60699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.141,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.44},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.495,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.71499999999999997}]},{"osd":4,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69699999999999995,"15min":0.69299999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0569999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89700000000000002,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.68700000000000006}]},{"osd":5,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.754,"15min":0.74399999999999999},"min":{"1min":0.57999999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.995,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.75800000000000001,"15min":0.752},"min":{"1min":0.56399999999999995,"5min":0.438,"15min":0.438},"max":{"1min":1.052,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.64200000000000002}]},{"osd":7,"last update":"Wed May 8 22:05:23 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.99299999999999999,"5min":1.028,"15min":1.028},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76300000000000001,"15min":0.76000000000000001},"min":{"1min":0.56999999999999995,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.98499999999999999}]},{"osd":8,"last update":"Wed May 8 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.83199999999999996,"15min":0.83999999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0289999999999999,"5min":1.093,"15min":1.093},"last":0.871},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.79300000000000004,"15min":0.79600000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0149999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.67100000000000004}]},{"osd":9,"last update":"Wed May 8 22:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.73599999999999999}]},{"osd":10,"last update":"Wed May 8 22:05:19 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.88200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":974848,"data_stored":511309,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.52700000000000002,"15min":0.49299999999999999},"min":{"1min":0.38900000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.89900000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54000000000000004,"15min":0.51400000000000001},"min":{"1min":0.371,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81200000000000006,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.56999999999999995}]},{"osd":2,"last update":"Wed May 8 22:04:46 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.51300000000000001,"15min":0.48499999999999999},"min":{"1min":0.378,"5min":0.316,"15min":0.316},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.50600000000000001,"15min":0.48399999999999999},"min":{"1min":0.315,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.69399999999999995}]},{"osd":3,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.53300000000000003,"15min":0.505},"min":{"1min":0.40000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.53300000000000003,"15min":0.51200000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.52100000000000002}]},{"osd":4,"last update":"Wed May 8 22:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.39900000000000002,"15min":0.376},"min":{"1min":0.307,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.64000000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.42499999999999999,"15min":0.39300000000000002},"min":{"1min":0.32400000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.76700000000000002,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.46100000000000002}]},{"osd":5,"last update":"Wed May 8 22:04:43 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.56399999999999995,"15min":0.54800000000000004},"min":{"1min":0.39400000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.754},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58499999999999996,"15min":0.57299999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.72799999999999998}]},{"osd":6,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.63,"15min":0.61599999999999999},"min":{"1min":0.55100000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.79900000000000004}]},{"osd":7,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.36499999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.62},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86299999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.74199999999999999}]},{"osd":8,"last update":"Wed May 8 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.65300000000000002,"15min":0.65900000000000003},"min":{"1min":0.45800000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.91200000000000003,"5min":1.115,"15min":1.115},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.53500000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96999999999999997,"5min":1.222,"15min":1.222},"last":0.92100000000000004}]},{"osd":9,"last update":"Wed May 8 22:05:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.876}]},{"osd":10,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.78500000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":946176,"data_stored":466578,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57899999999999996,"15min":0.54900000000000004},"min":{"1min":0.42499999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.94099999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.55400000000000005,"15min":0.52400000000000002},"min":{"1min":0.40100000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83199999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.59899999999999998}]},{"osd":2,"last update":"Wed May 8 22:04:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.56399999999999995,"15min":0.54000000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.56100000000000005,"15min":0.54100000000000004},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.70299999999999996}]},{"osd":3,"last update":"Wed May 8 22:04:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.436,"15min":0.41799999999999998},"min":{"1min":0.34799999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.624},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.48799999999999999,"15min":0.48199999999999998},"min":{"1min":0.35499999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.877,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.77300000000000002}]},{"osd":4,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.57099999999999995,"15min":0.54900000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69399999999999995,"15min":0.69599999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96999999999999997,"5min":2.984,"15min":2.984},"last":0.71599999999999997}]},{"osd":5,"last update":"Wed May 8 22:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.67800000000000005,"15min":0.64600000000000002},"min":{"1min":0.42399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0109999999999999,"5min":1.129,"15min":1.129},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67300000000000004,"15min":0.66300000000000003},"min":{"1min":0.49199999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.94699999999999995,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.79200000000000004}]},{"osd":6,"last update":"Wed May 8 22:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.68600000000000005,"15min":0.66900000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.81599999999999995}]},{"osd":7,"last update":"Wed May 8 22:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.755}]},{"osd":8,"last update":"Wed May 8 22:05:39 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.82199999999999995,"15min":0.82599999999999996},"min":{"1min":0.61399999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.099,"5min":1.427,"15min":1.427},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.79500000000000004,"15min":0.80300000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0369999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.82899999999999996}]},{"osd":9,"last update":"Wed May 8 22:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.008}]},{"osd":10,"last update":"Wed May 8 22:05:22 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.98499999999999999}]},{"osd":11,"last update":"Wed May 8 22:05:39 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":1.0209999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36132,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709020,"statfs":{"total":95995035648,"available":95958036480,"internally_reserved":0,"allocated":946176,"data_stored":470429,"data_compressed":11671,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.41999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.51200000000000001,"15min":0.504},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76100000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.55700000000000005}]},{"osd":1,"last update":"Wed May 8 22:04:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52600000000000002,"15min":0.52500000000000002},"min":{"1min":0.34100000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.74199999999999999,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.35499999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.77500000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.66500000000000004}]},{"osd":3,"last update":"Wed May 8 22:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.372,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65900000000000003,"5min":0.872,"15min":0.872},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56599999999999995,"15min":0.55900000000000005},"min":{"1min":0.40100000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72199999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.70499999999999996}]},{"osd":4,"last update":"Wed May 8 22:04:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.35699999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73399999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55700000000000005,"15min":0.55200000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.746}]},{"osd":5,"last update":"Wed May 8 22:04:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.436,"15min":0.433},"min":{"1min":0.34200000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.59999999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44400000000000001,"15min":0.441},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.72399999999999998}]},{"osd":6,"last update":"Wed May 8 22:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.76400000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66600000000000004,"15min":0.66700000000000004},"min":{"1min":0.50800000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.81499999999999995,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.89500000000000002}]},{"osd":7,"last update":"Wed May 8 22:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.94499999999999995}]},{"osd":8,"last update":"Wed May 8 22:04:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.97299999999999998}]},{"osd":9,"last update":"Wed May 8 22:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.83199999999999996}]},{"osd":10,"last update":"Wed May 8 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.877}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]}]},{"osd":3,"up_from":24,"seq":103079215149,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":456373,"data_compressed":14627,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.41699999999999998,"15min":0.39200000000000002},"min":{"1min":0.33500000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.504},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.42399999999999999,"15min":0.40200000000000002},"min":{"1min":0.35699999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.71299999999999997}]},{"osd":1,"last update":"Wed May 8 22:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.59899999999999998,"15min":0.55700000000000005},"min":{"1min":0.48599999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.56399999999999995,"15min":0.53000000000000003},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.77200000000000002}]},{"osd":2,"last update":"Wed May 8 22:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.61499999999999999,"15min":0.58899999999999997},"min":{"1min":0.51800000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.63900000000000001,"15min":0.60899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.55500000000000005}]},{"osd":4,"last update":"Wed May 8 22:05:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.62,"15min":0.59399999999999997},"min":{"1min":0.42399999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.66300000000000003,"15min":0.62},"min":{"1min":0.61299999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.98599999999999999}]},{"osd":5,"last update":"Wed May 8 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.67800000000000005,"15min":0.64500000000000002},"min":{"1min":0.46500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.64400000000000002,"15min":0.60899999999999999},"min":{"1min":0.46500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.76300000000000001}]},{"osd":6,"last update":"Wed May 8 22:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.63100000000000001,"15min":0.61499999999999999},"min":{"1min":0.46400000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.64600000000000002,"15min":0.624},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.874}]},{"osd":7,"last update":"Wed May 8 22:05:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.65700000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":1.008}]},{"osd":8,"last update":"Wed May 8 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.53100000000000003}]},{"osd":9,"last update":"Wed May 8 22:05:01 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.84999999999999998}]},{"osd":10,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.88400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986219,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":460781,"data_compressed":14627,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.44700000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.51900000000000002,"15min":0.50900000000000001},"min":{"1min":0.45000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.629}]},{"osd":1,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.374,"15min":0.35099999999999998},"min":{"1min":0.27200000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.66800000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.377,"15min":0.35499999999999998},"min":{"1min":0.28199999999999997,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.51000000000000001}]},{"osd":2,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.52600000000000002,"15min":0.50700000000000001},"min":{"1min":0.35499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71899999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.80900000000000005}]},{"osd":3,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.58799999999999997,"15min":0.56399999999999995},"min":{"1min":0.5,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84799999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56699999999999995,"15min":0.55300000000000005},"min":{"1min":0.39600000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.80000000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.67300000000000004}]},{"osd":5,"last update":"Wed May 8 22:05:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.36399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76200000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.57199999999999995,"15min":0.55000000000000004},"min":{"1min":0.42799999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.82199999999999995}]},{"osd":6,"last update":"Wed May 8 22:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.64800000000000002,"15min":0.63600000000000001},"min":{"1min":0.371,"5min":0.316,"15min":0.316},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.748},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.41799999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80500000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.60499999999999998}]},{"osd":7,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76800000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57299999999999995}]},{"osd":8,"last update":"Wed May 8 22:05:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68400000000000005,"15min":0.68600000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70799999999999996,"15min":0.70899999999999996},"min":{"1min":0.54100000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.79300000000000004}]},{"osd":9,"last update":"Wed May 8 22:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.624}]},{"osd":10,"last update":"Wed May 8 22:05:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.83499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":5,"up_from":36,"seq":154618822695,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":974848,"data_stored":1038375,"data_compressed":47667,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.378,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72899999999999998,"5min":0.749,"15min":0.749},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.48999999999999999}]},{"osd":1,"last update":"Wed May 8 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.41299999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.71699999999999997,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.622},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.37,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.58199999999999996}]},{"osd":2,"last update":"Wed May 8 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.254,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.52400000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.376,"15min":0.379},"min":{"1min":0.26600000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.46500000000000002,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.36199999999999999}]},{"osd":3,"last update":"Wed May 8 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57299999999999995,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.438},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.51800000000000002}]},{"osd":4,"last update":"Wed May 8 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.53300000000000003},"min":{"1min":0.38400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.39400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.751,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.64900000000000002}]},{"osd":6,"last update":"Wed May 8 22:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58599999999999997,"15min":0.58399999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.60699999999999998}]},{"osd":7,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.57399999999999995,"15min":0.56499999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.58799999999999997,"15min":0.57699999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.63700000000000001}]},{"osd":8,"last update":"Wed May 8 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.69399999999999995}]},{"osd":9,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.70599999999999996}]},{"osd":10,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.67400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":46209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":24792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":41957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:05:46.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:45 smithi132 ceph-mon[78180]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:05:46.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:45 smithi132 ceph-mon[78180]: pgmap v257: 129 pgs: 3 peering, 126 active+clean; 620 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.9 KiB/s wr, 16 op/s; 86 B/s, 1 objects/s recovering 2024-05-08T22:05:46.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:45 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/136196651' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-08T22:05:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:45 smithi131 ceph-mon[37260]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:05:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:45 smithi131 ceph-mon[37260]: pgmap v257: 129 pgs: 3 peering, 126 active+clean; 620 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.9 KiB/s wr, 16 op/s; 86 B/s, 1 objects/s recovering 2024-05-08T22:05:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:45 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/136196651' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-08T22:05:46.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:45 smithi157 ceph-mon[77746]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:05:46.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:45 smithi157 ceph-mon[77746]: pgmap v257: 129 pgs: 3 peering, 126 active+clean; 620 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.9 KiB/s wr, 16 op/s; 86 B/s, 1 objects/s recovering 2024-05-08T22:05:46.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:45 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/136196651' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-08T22:05:47.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:46 smithi131 ceph-mon[37260]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:05:47.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:46 smithi157 ceph-mon[77746]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:05:47.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:46 smithi132 ceph-mon[78180]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:05:48.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:47 smithi131 ceph-mon[37260]: pgmap v258: 129 pgs: 2 peering, 127 active+clean; 620 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.2 KiB/s wr, 14 op/s; 86 B/s, 1 keys/s, 1 objects/s recovering 2024-05-08T22:05:48.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:47 smithi157 ceph-mon[77746]: pgmap v258: 129 pgs: 2 peering, 127 active+clean; 620 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.2 KiB/s wr, 14 op/s; 86 B/s, 1 keys/s, 1 objects/s recovering 2024-05-08T22:05:48.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:47 smithi132 ceph-mon[78180]: pgmap v258: 129 pgs: 2 peering, 127 active+clean; 620 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.2 KiB/s wr, 14 op/s; 86 B/s, 1 keys/s, 1 objects/s recovering 2024-05-08T22:05:48.343 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:05:48.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.547+0000 7fed88200700 1 -- 172.21.15.131:0/1892253862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed801042d0 msgr2=0x7fed801066c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:48.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.547+0000 7fed88200700 1 --2- 172.21.15.131:0/1892253862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed801042d0 0x7fed801066c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fed6c0098f0 tx=0x7fed6c009c00 comp rx=0 tx=0).stop 2024-05-08T22:05:48.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.547+0000 7fed88200700 1 -- 172.21.15.131:0/1892253862 shutdown_connections 2024-05-08T22:05:48.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.547+0000 7fed88200700 1 --2- 172.21.15.131:0/1892253862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed80106c00 0x7fed80108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:48.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.547+0000 7fed88200700 1 --2- 172.21.15.131:0/1892253862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed801019a0 0x7fed80103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:48.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.547+0000 7fed88200700 1 --2- 172.21.15.131:0/1892253862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed801042d0 0x7fed801066c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:48.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.547+0000 7fed88200700 1 -- 172.21.15.131:0/1892253862 >> 172.21.15.131:0/1892253862 conn(0x7fed800fb2f0 msgr2=0x7fed800fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:48.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.547+0000 7fed88200700 1 -- 172.21.15.131:0/1892253862 shutdown_connections 2024-05-08T22:05:48.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.547+0000 7fed88200700 1 -- 172.21.15.131:0/1892253862 wait complete. 2024-05-08T22:05:48.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.548+0000 7fed88200700 1 Processor -- start 2024-05-08T22:05:48.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.548+0000 7fed88200700 1 -- start start 2024-05-08T22:05:48.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.549+0000 7fed88200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed801019a0 0x7fed801012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:48.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.549+0000 7fed88200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed801042d0 0x7fed800ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:48.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.549+0000 7fed88200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed80106c00 0x7fed800ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:48.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.549+0000 7fed88200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed80100630 con 0x7fed801042d0 2024-05-08T22:05:48.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.549+0000 7fed88200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed801007a0 con 0x7fed801019a0 2024-05-08T22:05:48.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.549+0000 7fed88200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed801a0eb0 con 0x7fed80106c00 2024-05-08T22:05:48.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.549+0000 7fed84e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed801042d0 0x7fed800ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:48.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.549+0000 7fed84e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed801042d0 0x7fed800ff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38166/0 (socket says 172.21.15.131:38166) 2024-05-08T22:05:48.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.549+0000 7fed84e00700 1 -- 172.21.15.131:0/1202795501 learned_addr learned my addr 172.21.15.131:0/1202795501 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:48.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.549+0000 7fed86200700 1 --2- 172.21.15.131:0/1202795501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed80106c00 0x7fed800ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:48.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.549+0000 7fed85800700 1 --2- 172.21.15.131:0/1202795501 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed801019a0 0x7fed801012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:48.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.549+0000 7fed84e00700 1 -- 172.21.15.131:0/1202795501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed80106c00 msgr2=0x7fed800ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:48.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.549+0000 7fed84e00700 1 --2- 172.21.15.131:0/1202795501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed80106c00 0x7fed800ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:48.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.549+0000 7fed84e00700 1 -- 172.21.15.131:0/1202795501 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed801019a0 msgr2=0x7fed801012c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:48.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.549+0000 7fed84e00700 1 --2- 172.21.15.131:0/1202795501 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed801019a0 0x7fed801012c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:48.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.549+0000 7fed84e00700 1 -- 172.21.15.131:0/1202795501 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6c009520 con 0x7fed801042d0 2024-05-08T22:05:48.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.550+0000 7fed84e00700 1 --2- 172.21.15.131:0/1202795501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed801042d0 0x7fed800ff950 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fed6c009810 tx=0x7fed6c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:48.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.550+0000 7fed7e000700 1 -- 172.21.15.131:0/1202795501 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c01c070 con 0x7fed801042d0 2024-05-08T22:05:48.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.550+0000 7fed7e000700 1 -- 172.21.15.131:0/1202795501 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed6c00f7d0 con 0x7fed801042d0 2024-05-08T22:05:48.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.550+0000 7fed88200700 1 -- 172.21.15.131:0/1202795501 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed801a10e0 con 0x7fed801042d0 2024-05-08T22:05:48.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.550+0000 7fed88200700 1 -- 172.21.15.131:0/1202795501 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed801a14a0 con 0x7fed801042d0 2024-05-08T22:05:48.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.550+0000 7fed7e000700 1 -- 172.21.15.131:0/1202795501 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c003970 con 0x7fed801042d0 2024-05-08T22:05:48.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.551+0000 7fed88200700 1 -- 172.21.15.131:0/1202795501 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed800fce70 con 0x7fed801042d0 2024-05-08T22:05:48.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.553+0000 7fed7e000700 1 -- 172.21.15.131:0/1202795501 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fed6c020030 con 0x7fed801042d0 2024-05-08T22:05:48.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.554+0000 7fed7e000700 1 --2- 172.21.15.131:0/1202795501 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fed680a11b0 0x7fed680a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:48.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.554+0000 7fed7e000700 1 -- 172.21.15.131:0/1202795501 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fed6c0ce210 con 0x7fed801042d0 2024-05-08T22:05:48.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.554+0000 7fed85800700 1 --2- 172.21.15.131:0/1202795501 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fed680a11b0 0x7fed680a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:48.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.555+0000 7fed85800700 1 --2- 172.21.15.131:0/1202795501 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fed680a11b0 0x7fed680a3670 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fed74005fd0 tx=0x7fed74009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:48.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.557+0000 7fed7e000700 1 -- 172.21.15.131:0/1202795501 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fed6c097560 con 0x7fed801042d0 2024-05-08T22:05:48.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.761+0000 7fed88200700 1 -- 172.21.15.131:0/1202795501 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fed800613e0 con 0x7fed680a11b0 2024-05-08T22:05:48.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.773+0000 7fed7e000700 1 -- 172.21.15.131:0/1202795501 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423186 (secure 0 0 0) 0x7fed800613e0 con 0x7fed680a11b0 2024-05-08T22:05:48.783 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:48.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.778+0000 7fed88200700 1 -- 172.21.15.131:0/1202795501 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fed680a11b0 msgr2=0x7fed680a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:48.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.778+0000 7fed88200700 1 --2- 172.21.15.131:0/1202795501 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fed680a11b0 0x7fed680a3670 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fed74005fd0 tx=0x7fed74009310 comp rx=0 tx=0).stop 2024-05-08T22:05:48.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.778+0000 7fed88200700 1 -- 172.21.15.131:0/1202795501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed801042d0 msgr2=0x7fed800ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:48.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.778+0000 7fed88200700 1 --2- 172.21.15.131:0/1202795501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed801042d0 0x7fed800ff950 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fed6c009810 tx=0x7fed6c00b790 comp rx=0 tx=0).stop 2024-05-08T22:05:48.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.778+0000 7fed88200700 1 -- 172.21.15.131:0/1202795501 shutdown_connections 2024-05-08T22:05:48.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.778+0000 7fed88200700 1 --2- 172.21.15.131:0/1202795501 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fed680a11b0 0x7fed680a3670 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:48.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.778+0000 7fed88200700 1 --2- 172.21.15.131:0/1202795501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed80106c00 0x7fed800ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:48.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.778+0000 7fed88200700 1 --2- 172.21.15.131:0/1202795501 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed801019a0 0x7fed801012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:48.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.778+0000 7fed88200700 1 --2- 172.21.15.131:0/1202795501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed801042d0 0x7fed800ff950 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:48.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.778+0000 7fed88200700 1 -- 172.21.15.131:0/1202795501 >> 172.21.15.131:0/1202795501 conn(0x7fed800fb2f0 msgr2=0x7fed80103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:48.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.779+0000 7fed88200700 1 -- 172.21.15.131:0/1202795501 shutdown_connections 2024-05-08T22:05:48.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:48.779+0000 7fed88200700 1 -- 172.21.15.131:0/1202795501 wait complete. 2024-05-08T22:05:48.789 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-08T22:05:48.799 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-05-08T22:05:48.719734+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":640136,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":207,"num_read_kb":209,"num_write":327,"num_write_kb":3031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181430,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":266,"ondisk_log_size":266,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":422,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372444,"kb_used_data":11324,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124569380,"statfs":{"total":1151940427776,"available":1151559045120,"internally_reserved":0,"allocated":11595776,"data_stored":7547213,"data_compressed":318984,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19131,"internal_metadata":369603909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43906,"num_objects":24,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":65,"num_read_kb":85,"num_write":59,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":694,"num_keys_recovered":10,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002313"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.821136+0000","last_change":"2024-05-08T22:05:14.061628+0000","last_active":"2024-05-08T22:05:40.821136+0000","last_peered":"2024-05-08T22:05:40.821136+0000","last_clean":"2024-05-08T22:05:40.821136+0000","last_became_active":"2024-05-08T22:05:14.060986+0000","last_became_peered":"2024-05-08T22:05:14.060986+0000","last_unstale":"2024-05-08T22:05:40.821136+0000","last_undegraded":"2024-05-08T22:05:40.821136+0000","last_fullsized":"2024-05-08T22:05:40.821136+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:51.643118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832603+0000","last_change":"2024-05-08T22:05:14.073300+0000","last_active":"2024-05-08T22:05:40.832603+0000","last_peered":"2024-05-08T22:05:40.832603+0000","last_clean":"2024-05-08T22:05:40.832603+0000","last_became_active":"2024-05-08T22:05:14.069058+0000","last_became_peered":"2024-05-08T22:05:14.069058+0000","last_unstale":"2024-05-08T22:05:40.832603+0000","last_undegraded":"2024-05-08T22:05:40.832603+0000","last_fullsized":"2024-05-08T22:05:40.832603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:15.834956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832503+0000","last_change":"2024-05-08T22:05:14.071676+0000","last_active":"2024-05-08T22:05:40.832503+0000","last_peered":"2024-05-08T22:05:40.832503+0000","last_clean":"2024-05-08T22:05:40.832503+0000","last_became_active":"2024-05-08T22:05:14.058345+0000","last_became_peered":"2024-05-08T22:05:14.058345+0000","last_unstale":"2024-05-08T22:05:40.832503+0000","last_undegraded":"2024-05-08T22:05:40.832503+0000","last_fullsized":"2024-05-08T22:05:40.832503+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:09.239058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819679+0000","last_change":"2024-05-08T22:05:14.067170+0000","last_active":"2024-05-08T22:05:40.819679+0000","last_peered":"2024-05-08T22:05:40.819679+0000","last_clean":"2024-05-08T22:05:40.819679+0000","last_became_active":"2024-05-08T22:05:14.066850+0000","last_became_peered":"2024-05-08T22:05:14.066850+0000","last_unstale":"2024-05-08T22:05:40.819679+0000","last_undegraded":"2024-05-08T22:05:40.819679+0000","last_fullsized":"2024-05-08T22:05:40.819679+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:28.439786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829281+0000","last_change":"2024-05-08T22:05:14.065177+0000","last_active":"2024-05-08T22:05:40.829281+0000","last_peered":"2024-05-08T22:05:40.829281+0000","last_clean":"2024-05-08T22:05:40.829281+0000","last_became_active":"2024-05-08T22:05:14.064833+0000","last_became_peered":"2024-05-08T22:05:14.064833+0000","last_unstale":"2024-05-08T22:05:40.829281+0000","last_undegraded":"2024-05-08T22:05:40.829281+0000","last_fullsized":"2024-05-08T22:05:40.829281+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:11.764154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.825297+0000","last_change":"2024-05-08T22:05:14.061610+0000","last_active":"2024-05-08T22:05:40.825297+0000","last_peered":"2024-05-08T22:05:40.825297+0000","last_clean":"2024-05-08T22:05:40.825297+0000","last_became_active":"2024-05-08T22:05:14.061277+0000","last_became_peered":"2024-05-08T22:05:14.061277+0000","last_unstale":"2024-05-08T22:05:40.825297+0000","last_undegraded":"2024-05-08T22:05:40.825297+0000","last_fullsized":"2024-05-08T22:05:40.825297+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:19.154980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830795+0000","last_change":"2024-05-08T22:05:14.063941+0000","last_active":"2024-05-08T22:05:40.830795+0000","last_peered":"2024-05-08T22:05:40.830795+0000","last_clean":"2024-05-08T22:05:40.830795+0000","last_became_active":"2024-05-08T22:05:14.063579+0000","last_became_peered":"2024-05-08T22:05:14.063579+0000","last_unstale":"2024-05-08T22:05:40.830795+0000","last_undegraded":"2024-05-08T22:05:40.830795+0000","last_fullsized":"2024-05-08T22:05:40.830795+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:27.679067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826760+0000","last_change":"2024-05-08T22:05:14.060366+0000","last_active":"2024-05-08T22:05:40.826760+0000","last_peered":"2024-05-08T22:05:40.826760+0000","last_clean":"2024-05-08T22:05:40.826760+0000","last_became_active":"2024-05-08T22:05:14.059818+0000","last_became_peered":"2024-05-08T22:05:14.059818+0000","last_unstale":"2024-05-08T22:05:40.826760+0000","last_undegraded":"2024-05-08T22:05:40.826760+0000","last_fullsized":"2024-05-08T22:05:40.826760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:53.118508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.823157+0000","last_change":"2024-05-08T22:05:14.057185+0000","last_active":"2024-05-08T22:05:40.823157+0000","last_peered":"2024-05-08T22:05:40.823157+0000","last_clean":"2024-05-08T22:05:40.823157+0000","last_became_active":"2024-05-08T22:05:14.056889+0000","last_became_peered":"2024-05-08T22:05:14.056889+0000","last_unstale":"2024-05-08T22:05:40.823157+0000","last_undegraded":"2024-05-08T22:05:40.823157+0000","last_fullsized":"2024-05-08T22:05:40.823157+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:35.372705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824080+0000","last_change":"2024-05-08T22:05:14.062282+0000","last_active":"2024-05-08T22:05:40.824080+0000","last_peered":"2024-05-08T22:05:40.824080+0000","last_clean":"2024-05-08T22:05:40.824080+0000","last_became_active":"2024-05-08T22:05:14.061574+0000","last_became_peered":"2024-05-08T22:05:14.061574+0000","last_unstale":"2024-05-08T22:05:40.824080+0000","last_undegraded":"2024-05-08T22:05:40.824080+0000","last_fullsized":"2024-05-08T22:05:40.824080+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:16:32.873590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831100+0000","last_change":"2024-05-08T22:05:14.061267+0000","last_active":"2024-05-08T22:05:40.831100+0000","last_peered":"2024-05-08T22:05:40.831100+0000","last_clean":"2024-05-08T22:05:40.831100+0000","last_became_active":"2024-05-08T22:05:14.060943+0000","last_became_peered":"2024-05-08T22:05:14.060943+0000","last_unstale":"2024-05-08T22:05:40.831100+0000","last_undegraded":"2024-05-08T22:05:40.831100+0000","last_fullsized":"2024-05-08T22:05:40.831100+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:03.831138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819303+0000","last_change":"2024-05-08T22:05:14.067136+0000","last_active":"2024-05-08T22:05:40.819303+0000","last_peered":"2024-05-08T22:05:40.819303+0000","last_clean":"2024-05-08T22:05:40.819303+0000","last_became_active":"2024-05-08T22:05:14.066810+0000","last_became_peered":"2024-05-08T22:05:14.066810+0000","last_unstale":"2024-05-08T22:05:40.819303+0000","last_undegraded":"2024-05-08T22:05:40.819303+0000","last_fullsized":"2024-05-08T22:05:40.819303+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:43.001024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824060+0000","last_change":"2024-05-08T22:05:14.062119+0000","last_active":"2024-05-08T22:05:40.824060+0000","last_peered":"2024-05-08T22:05:40.824060+0000","last_clean":"2024-05-08T22:05:40.824060+0000","last_became_active":"2024-05-08T22:05:14.058977+0000","last_became_peered":"2024-05-08T22:05:14.058977+0000","last_unstale":"2024-05-08T22:05:40.824060+0000","last_undegraded":"2024-05-08T22:05:40.824060+0000","last_fullsized":"2024-05-08T22:05:40.824060+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:03.657985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832861+0000","last_change":"2024-05-08T22:05:14.067363+0000","last_active":"2024-05-08T22:05:40.832861+0000","last_peered":"2024-05-08T22:05:40.832861+0000","last_clean":"2024-05-08T22:05:40.832861+0000","last_became_active":"2024-05-08T22:05:14.066119+0000","last_became_peered":"2024-05-08T22:05:14.066119+0000","last_unstale":"2024-05-08T22:05:40.832861+0000","last_undegraded":"2024-05-08T22:05:40.832861+0000","last_fullsized":"2024-05-08T22:05:40.832861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:43.426750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832641+0000","last_change":"2024-05-08T22:05:14.073925+0000","last_active":"2024-05-08T22:05:40.832641+0000","last_peered":"2024-05-08T22:05:40.832641+0000","last_clean":"2024-05-08T22:05:40.832641+0000","last_became_active":"2024-05-08T22:05:14.072287+0000","last_became_peered":"2024-05-08T22:05:14.072287+0000","last_unstale":"2024-05-08T22:05:40.832641+0000","last_undegraded":"2024-05-08T22:05:40.832641+0000","last_fullsized":"2024-05-08T22:05:40.832641+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:13.537938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.823678+0000","last_change":"2024-05-08T22:05:14.057274+0000","last_active":"2024-05-08T22:05:40.823678+0000","last_peered":"2024-05-08T22:05:40.823678+0000","last_clean":"2024-05-08T22:05:40.823678+0000","last_became_active":"2024-05-08T22:05:14.056989+0000","last_became_peered":"2024-05-08T22:05:14.056989+0000","last_unstale":"2024-05-08T22:05:40.823678+0000","last_undegraded":"2024-05-08T22:05:40.823678+0000","last_fullsized":"2024-05-08T22:05:40.823678+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:32.316406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.821053+0000","last_change":"2024-05-08T22:05:14.069263+0000","last_active":"2024-05-08T22:05:40.821053+0000","last_peered":"2024-05-08T22:05:40.821053+0000","last_clean":"2024-05-08T22:05:40.821053+0000","last_became_active":"2024-05-08T22:05:14.068960+0000","last_became_peered":"2024-05-08T22:05:14.068960+0000","last_unstale":"2024-05-08T22:05:40.821053+0000","last_undegraded":"2024-05-08T22:05:40.821053+0000","last_fullsized":"2024-05-08T22:05:40.821053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:59:42.829110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817249+0000","last_change":"2024-05-08T22:05:14.049393+0000","last_active":"2024-05-08T22:05:40.817249+0000","last_peered":"2024-05-08T22:05:40.817249+0000","last_clean":"2024-05-08T22:05:40.817249+0000","last_became_active":"2024-05-08T22:05:14.049094+0000","last_became_peered":"2024-05-08T22:05:14.049094+0000","last_unstale":"2024-05-08T22:05:40.817249+0000","last_undegraded":"2024-05-08T22:05:40.817249+0000","last_fullsized":"2024-05-08T22:05:40.817249+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:42.106568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.818062+0000","last_change":"2024-05-08T22:05:14.055436+0000","last_active":"2024-05-08T22:05:40.818062+0000","last_peered":"2024-05-08T22:05:40.818062+0000","last_clean":"2024-05-08T22:05:40.818062+0000","last_became_active":"2024-05-08T22:05:14.055100+0000","last_became_peered":"2024-05-08T22:05:14.055100+0000","last_unstale":"2024-05-08T22:05:40.818062+0000","last_undegraded":"2024-05-08T22:05:40.818062+0000","last_fullsized":"2024-05-08T22:05:40.818062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:52.833780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.816936+0000","last_change":"2024-05-08T22:05:18.532086+0000","last_active":"2024-05-08T22:05:40.816936+0000","last_peered":"2024-05-08T22:05:40.816936+0000","last_clean":"2024-05-08T22:05:40.816936+0000","last_became_active":"2024-05-08T22:05:18.531485+0000","last_became_peered":"2024-05-08T22:05:18.531485+0000","last_unstale":"2024-05-08T22:05:40.816936+0000","last_undegraded":"2024-05-08T22:05:40.816936+0000","last_fullsized":"2024-05-08T22:05:40.816936+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:40:36.050367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817288+0000","last_change":"2024-05-08T22:05:13.549611+0000","last_active":"2024-05-08T22:05:40.817288+0000","last_peered":"2024-05-08T22:05:40.817288+0000","last_clean":"2024-05-08T22:05:40.817288+0000","last_became_active":"2024-05-08T22:05:13.549140+0000","last_became_peered":"2024-05-08T22:05:13.549140+0000","last_unstale":"2024-05-08T22:05:40.817288+0000","last_undegraded":"2024-05-08T22:05:40.817288+0000","last_fullsized":"2024-05-08T22:05:40.817288+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:54.785123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.828016+0000","last_change":"2024-05-08T22:05:14.057570+0000","last_active":"2024-05-08T22:05:40.828016+0000","last_peered":"2024-05-08T22:05:40.828016+0000","last_clean":"2024-05-08T22:05:40.828016+0000","last_became_active":"2024-05-08T22:05:14.057291+0000","last_became_peered":"2024-05-08T22:05:14.057291+0000","last_unstale":"2024-05-08T22:05:40.828016+0000","last_undegraded":"2024-05-08T22:05:40.828016+0000","last_fullsized":"2024-05-08T22:05:40.828016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:15:38.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:42.147633+0000","last_change":"2024-05-08T22:05:42.147633+0000","last_active":"2024-05-08T22:05:42.147633+0000","last_peered":"2024-05-08T22:05:42.147633+0000","last_clean":"2024-05-08T22:05:42.147633+0000","last_became_active":"2024-05-08T22:05:42.147284+0000","last_became_peered":"2024-05-08T22:05:42.147284+0000","last_unstale":"2024-05-08T22:05:42.147633+0000","last_undegraded":"2024-05-08T22:05:42.147633+0000","last_fullsized":"2024-05-08T22:05:42.147633+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:32.864756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.828112+0000","last_change":"2024-05-08T22:05:13.551351+0000","last_active":"2024-05-08T22:05:40.828112+0000","last_peered":"2024-05-08T22:05:40.828112+0000","last_clean":"2024-05-08T22:05:40.828112+0000","last_became_active":"2024-05-08T22:05:13.551014+0000","last_became_peered":"2024-05-08T22:05:13.551014+0000","last_unstale":"2024-05-08T22:05:40.828112+0000","last_undegraded":"2024-05-08T22:05:40.828112+0000","last_fullsized":"2024-05-08T22:05:40.828112+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:07:22.405251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.834013+0000","last_change":"2024-05-08T22:05:14.067270+0000","last_active":"2024-05-08T22:05:40.834013+0000","last_peered":"2024-05-08T22:05:40.834013+0000","last_clean":"2024-05-08T22:05:40.834013+0000","last_became_active":"2024-05-08T22:05:14.065496+0000","last_became_peered":"2024-05-08T22:05:14.065496+0000","last_unstale":"2024-05-08T22:05:40.834013+0000","last_undegraded":"2024-05-08T22:05:40.834013+0000","last_fullsized":"2024-05-08T22:05:40.834013+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:07.087880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819016+0000","last_change":"2024-05-08T22:05:18.526557+0000","last_active":"2024-05-08T22:05:40.819016+0000","last_peered":"2024-05-08T22:05:40.819016+0000","last_clean":"2024-05-08T22:05:40.819016+0000","last_became_active":"2024-05-08T22:05:18.526129+0000","last_became_peered":"2024-05-08T22:05:18.526129+0000","last_unstale":"2024-05-08T22:05:40.819016+0000","last_undegraded":"2024-05-08T22:05:40.819016+0000","last_fullsized":"2024-05-08T22:05:40.819016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:36.376579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.835476+0000","last_change":"2024-05-08T22:05:41.835074+0000","last_active":"2024-05-08T22:05:41.835476+0000","last_peered":"2024-05-08T22:05:41.835476+0000","last_clean":"2024-05-08T22:05:41.835476+0000","last_became_active":"2024-05-08T22:05:41.829567+0000","last_became_peered":"2024-05-08T22:05:41.829567+0000","last_unstale":"2024-05-08T22:05:41.835476+0000","last_undegraded":"2024-05-08T22:05:41.835476+0000","last_fullsized":"2024-05-08T22:05:41.835476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:15.261352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826964+0000","last_change":"2024-05-08T22:05:14.060089+0000","last_active":"2024-05-08T22:05:40.826964+0000","last_peered":"2024-05-08T22:05:40.826964+0000","last_clean":"2024-05-08T22:05:40.826964+0000","last_became_active":"2024-05-08T22:05:14.057620+0000","last_became_peered":"2024-05-08T22:05:14.057620+0000","last_unstale":"2024-05-08T22:05:40.826964+0000","last_undegraded":"2024-05-08T22:05:40.826964+0000","last_fullsized":"2024-05-08T22:05:40.826964+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:29:33.127510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.818817+0000","last_change":"2024-05-08T22:05:18.529746+0000","last_active":"2024-05-08T22:05:40.818817+0000","last_peered":"2024-05-08T22:05:40.818817+0000","last_clean":"2024-05-08T22:05:40.818817+0000","last_became_active":"2024-05-08T22:05:18.529097+0000","last_became_peered":"2024-05-08T22:05:18.529097+0000","last_unstale":"2024-05-08T22:05:40.818817+0000","last_undegraded":"2024-05-08T22:05:40.818817+0000","last_fullsized":"2024-05-08T22:05:40.818817+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:59:36.249787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.837496+0000","last_change":"2024-05-08T22:05:41.837103+0000","last_active":"2024-05-08T22:05:41.837496+0000","last_peered":"2024-05-08T22:05:41.837496+0000","last_clean":"2024-05-08T22:05:41.837496+0000","last_became_active":"2024-05-08T22:05:41.829464+0000","last_became_peered":"2024-05-08T22:05:41.829464+0000","last_unstale":"2024-05-08T22:05:41.837496+0000","last_undegraded":"2024-05-08T22:05:41.837496+0000","last_fullsized":"2024-05-08T22:05:41.837496+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:55.747487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827543+0000","last_change":"2024-05-08T22:05:14.064695+0000","last_active":"2024-05-08T22:05:40.827543+0000","last_peered":"2024-05-08T22:05:40.827543+0000","last_clean":"2024-05-08T22:05:40.827543+0000","last_became_active":"2024-05-08T22:05:14.064283+0000","last_became_peered":"2024-05-08T22:05:14.064283+0000","last_unstale":"2024-05-08T22:05:40.827543+0000","last_undegraded":"2024-05-08T22:05:40.827543+0000","last_fullsized":"2024-05-08T22:05:40.827543+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:06.980594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817381+0000","last_change":"2024-05-08T22:05:18.528475+0000","last_active":"2024-05-08T22:05:40.817381+0000","last_peered":"2024-05-08T22:05:40.817381+0000","last_clean":"2024-05-08T22:05:40.817381+0000","last_became_active":"2024-05-08T22:05:18.528089+0000","last_became_peered":"2024-05-08T22:05:18.528089+0000","last_unstale":"2024-05-08T22:05:40.817381+0000","last_undegraded":"2024-05-08T22:05:40.817381+0000","last_fullsized":"2024-05-08T22:05:40.817381+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:01.508984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:43.509925+0000","last_change":"2024-05-08T22:05:13.549191+0000","last_active":"2024-05-08T22:05:43.509925+0000","last_peered":"2024-05-08T22:05:43.509925+0000","last_clean":"2024-05-08T22:05:43.509925+0000","last_became_active":"2024-05-08T22:05:13.548737+0000","last_became_peered":"2024-05-08T22:05:13.548737+0000","last_unstale":"2024-05-08T22:05:43.509925+0000","last_undegraded":"2024-05-08T22:05:43.509925+0000","last_fullsized":"2024-05-08T22:05:43.509925+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:07.355656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830198+0000","last_change":"2024-05-08T22:05:14.065910+0000","last_active":"2024-05-08T22:05:40.830198+0000","last_peered":"2024-05-08T22:05:40.830198+0000","last_clean":"2024-05-08T22:05:40.830198+0000","last_became_active":"2024-05-08T22:05:14.065504+0000","last_became_peered":"2024-05-08T22:05:14.065504+0000","last_unstale":"2024-05-08T22:05:40.830198+0000","last_undegraded":"2024-05-08T22:05:40.830198+0000","last_fullsized":"2024-05-08T22:05:40.830198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:12.447292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817194+0000","last_change":"2024-05-08T22:05:18.536861+0000","last_active":"2024-05-08T22:05:40.817194+0000","last_peered":"2024-05-08T22:05:40.817194+0000","last_clean":"2024-05-08T22:05:40.817194+0000","last_became_active":"2024-05-08T22:05:18.536472+0000","last_became_peered":"2024-05-08T22:05:18.536472+0000","last_unstale":"2024-05-08T22:05:40.817194+0000","last_undegraded":"2024-05-08T22:05:40.817194+0000","last_fullsized":"2024-05-08T22:05:40.817194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:45:21.826812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.825252+0000","last_change":"2024-05-08T22:05:13.553326+0000","last_active":"2024-05-08T22:05:40.825252+0000","last_peered":"2024-05-08T22:05:40.825252+0000","last_clean":"2024-05-08T22:05:40.825252+0000","last_became_active":"2024-05-08T22:05:13.552983+0000","last_became_peered":"2024-05-08T22:05:13.552983+0000","last_unstale":"2024-05-08T22:05:40.825252+0000","last_undegraded":"2024-05-08T22:05:40.825252+0000","last_fullsized":"2024-05-08T22:05:40.825252+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:17.195253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.828199+0000","last_change":"2024-05-08T22:05:14.068617+0000","last_active":"2024-05-08T22:05:40.828199+0000","last_peered":"2024-05-08T22:05:40.828199+0000","last_clean":"2024-05-08T22:05:40.828199+0000","last_became_active":"2024-05-08T22:05:14.067400+0000","last_became_peered":"2024-05-08T22:05:14.067400+0000","last_unstale":"2024-05-08T22:05:40.828199+0000","last_undegraded":"2024-05-08T22:05:40.828199+0000","last_fullsized":"2024-05-08T22:05:40.828199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:25.488212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.820967+0000","last_change":"2024-05-08T22:05:18.538294+0000","last_active":"2024-05-08T22:05:40.820967+0000","last_peered":"2024-05-08T22:05:40.820967+0000","last_clean":"2024-05-08T22:05:40.820967+0000","last_became_active":"2024-05-08T22:05:18.537984+0000","last_became_peered":"2024-05-08T22:05:18.537984+0000","last_unstale":"2024-05-08T22:05:40.820967+0000","last_undegraded":"2024-05-08T22:05:40.820967+0000","last_fullsized":"2024-05-08T22:05:40.820967+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:32.117439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.821162+0000","last_change":"2024-05-08T22:05:13.557669+0000","last_active":"2024-05-08T22:05:40.821162+0000","last_peered":"2024-05-08T22:05:40.821162+0000","last_clean":"2024-05-08T22:05:40.821162+0000","last_became_active":"2024-05-08T22:05:13.555835+0000","last_became_peered":"2024-05-08T22:05:13.555835+0000","last_unstale":"2024-05-08T22:05:40.821162+0000","last_undegraded":"2024-05-08T22:05:40.821162+0000","last_fullsized":"2024-05-08T22:05:40.821162+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:02.377635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826728+0000","last_change":"2024-05-08T22:05:14.060144+0000","last_active":"2024-05-08T22:05:40.826728+0000","last_peered":"2024-05-08T22:05:40.826728+0000","last_clean":"2024-05-08T22:05:40.826728+0000","last_became_active":"2024-05-08T22:05:14.059387+0000","last_became_peered":"2024-05-08T22:05:14.059387+0000","last_unstale":"2024-05-08T22:05:40.826728+0000","last_undegraded":"2024-05-08T22:05:40.826728+0000","last_fullsized":"2024-05-08T22:05:40.826728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:32:28.057831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832449+0000","last_change":"2024-05-08T22:05:18.527293+0000","last_active":"2024-05-08T22:05:40.832449+0000","last_peered":"2024-05-08T22:05:40.832449+0000","last_clean":"2024-05-08T22:05:40.832449+0000","last_became_active":"2024-05-08T22:05:18.526690+0000","last_became_peered":"2024-05-08T22:05:18.526690+0000","last_unstale":"2024-05-08T22:05:40.832449+0000","last_undegraded":"2024-05-08T22:05:40.832449+0000","last_fullsized":"2024-05-08T22:05:40.832449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:50.343536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826376+0000","last_change":"2024-05-08T22:05:13.549774+0000","last_active":"2024-05-08T22:05:40.826376+0000","last_peered":"2024-05-08T22:05:40.826376+0000","last_clean":"2024-05-08T22:05:40.826376+0000","last_became_active":"2024-05-08T22:05:13.549531+0000","last_became_peered":"2024-05-08T22:05:13.549531+0000","last_unstale":"2024-05-08T22:05:40.826376+0000","last_undegraded":"2024-05-08T22:05:40.826376+0000","last_fullsized":"2024-05-08T22:05:40.826376+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:59:23.798728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.815983+0000","last_change":"2024-05-08T22:05:14.054232+0000","last_active":"2024-05-08T22:05:40.815983+0000","last_peered":"2024-05-08T22:05:40.815983+0000","last_clean":"2024-05-08T22:05:40.815983+0000","last_became_active":"2024-05-08T22:05:14.053806+0000","last_became_peered":"2024-05-08T22:05:14.053806+0000","last_unstale":"2024-05-08T22:05:40.815983+0000","last_undegraded":"2024-05-08T22:05:40.815983+0000","last_fullsized":"2024-05-08T22:05:40.815983+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:32:21.678792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827239+0000","last_change":"2024-05-08T22:05:18.535241+0000","last_active":"2024-05-08T22:05:40.827239+0000","last_peered":"2024-05-08T22:05:40.827239+0000","last_clean":"2024-05-08T22:05:40.827239+0000","last_became_active":"2024-05-08T22:05:18.534753+0000","last_became_peered":"2024-05-08T22:05:18.534753+0000","last_unstale":"2024-05-08T22:05:40.827239+0000","last_undegraded":"2024-05-08T22:05:40.827239+0000","last_fullsized":"2024-05-08T22:05:40.827239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:46:15.138437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:46.513729+0000","last_change":"2024-05-08T22:05:13.544899+0000","last_active":"2024-05-08T22:05:46.513729+0000","last_peered":"2024-05-08T22:05:46.513729+0000","last_clean":"2024-05-08T22:05:46.513729+0000","last_became_active":"2024-05-08T22:05:13.544498+0000","last_became_peered":"2024-05-08T22:05:13.544498+0000","last_unstale":"2024-05-08T22:05:46.513729+0000","last_undegraded":"2024-05-08T22:05:46.513729+0000","last_fullsized":"2024-05-08T22:05:46.513729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:58.169648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.823071+0000","last_change":"2024-05-08T22:05:14.049377+0000","last_active":"2024-05-08T22:05:40.823071+0000","last_peered":"2024-05-08T22:05:40.823071+0000","last_clean":"2024-05-08T22:05:40.823071+0000","last_became_active":"2024-05-08T22:05:14.049039+0000","last_became_peered":"2024-05-08T22:05:14.049039+0000","last_unstale":"2024-05-08T22:05:40.823071+0000","last_undegraded":"2024-05-08T22:05:40.823071+0000","last_fullsized":"2024-05-08T22:05:40.823071+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:24.415392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.815955+0000","last_change":"2024-05-08T22:05:18.528912+0000","last_active":"2024-05-08T22:05:40.815955+0000","last_peered":"2024-05-08T22:05:40.815955+0000","last_clean":"2024-05-08T22:05:40.815955+0000","last_became_active":"2024-05-08T22:05:18.528433+0000","last_became_peered":"2024-05-08T22:05:18.528433+0000","last_unstale":"2024-05-08T22:05:40.815955+0000","last_undegraded":"2024-05-08T22:05:40.815955+0000","last_fullsized":"2024-05-08T22:05:40.815955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:05.574728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830944+0000","last_change":"2024-05-08T22:05:13.550214+0000","last_active":"2024-05-08T22:05:40.830944+0000","last_peered":"2024-05-08T22:05:40.830944+0000","last_clean":"2024-05-08T22:05:40.830944+0000","last_became_active":"2024-05-08T22:05:13.549717+0000","last_became_peered":"2024-05-08T22:05:13.549717+0000","last_unstale":"2024-05-08T22:05:40.830944+0000","last_undegraded":"2024-05-08T22:05:40.830944+0000","last_fullsized":"2024-05-08T22:05:40.830944+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:12:20.406333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.834200+0000","last_change":"2024-05-08T22:05:14.071214+0000","last_active":"2024-05-08T22:05:40.834200+0000","last_peered":"2024-05-08T22:05:40.834200+0000","last_clean":"2024-05-08T22:05:40.834200+0000","last_became_active":"2024-05-08T22:05:14.066777+0000","last_became_peered":"2024-05-08T22:05:14.066777+0000","last_unstale":"2024-05-08T22:05:40.834200+0000","last_undegraded":"2024-05-08T22:05:40.834200+0000","last_fullsized":"2024-05-08T22:05:40.834200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:53.360911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819973+0000","last_change":"2024-05-08T22:05:18.523688+0000","last_active":"2024-05-08T22:05:40.819973+0000","last_peered":"2024-05-08T22:05:40.819973+0000","last_clean":"2024-05-08T22:05:40.819973+0000","last_became_active":"2024-05-08T22:05:18.523236+0000","last_became_peered":"2024-05-08T22:05:18.523236+0000","last_unstale":"2024-05-08T22:05:40.819973+0000","last_undegraded":"2024-05-08T22:05:40.819973+0000","last_fullsized":"2024-05-08T22:05:40.819973+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:34.531699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827485+0000","last_change":"2024-05-08T22:05:13.551027+0000","last_active":"2024-05-08T22:05:40.827485+0000","last_peered":"2024-05-08T22:05:40.827485+0000","last_clean":"2024-05-08T22:05:40.827485+0000","last_became_active":"2024-05-08T22:05:13.550471+0000","last_became_peered":"2024-05-08T22:05:13.550471+0000","last_unstale":"2024-05-08T22:05:40.827485+0000","last_undegraded":"2024-05-08T22:05:40.827485+0000","last_fullsized":"2024-05-08T22:05:40.827485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:50.900419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827682+0000","last_change":"2024-05-08T22:05:14.065191+0000","last_active":"2024-05-08T22:05:40.827682+0000","last_peered":"2024-05-08T22:05:40.827682+0000","last_clean":"2024-05-08T22:05:40.827682+0000","last_became_active":"2024-05-08T22:05:14.064893+0000","last_became_peered":"2024-05-08T22:05:14.064893+0000","last_unstale":"2024-05-08T22:05:40.827682+0000","last_undegraded":"2024-05-08T22:05:40.827682+0000","last_fullsized":"2024-05-08T22:05:40.827682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:11.344646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.816088+0000","last_change":"2024-05-08T22:05:18.530447+0000","last_active":"2024-05-08T22:05:40.816088+0000","last_peered":"2024-05-08T22:05:40.816088+0000","last_clean":"2024-05-08T22:05:40.816088+0000","last_became_active":"2024-05-08T22:05:18.528171+0000","last_became_peered":"2024-05-08T22:05:18.528171+0000","last_unstale":"2024-05-08T22:05:40.816088+0000","last_undegraded":"2024-05-08T22:05:40.816088+0000","last_fullsized":"2024-05-08T22:05:40.816088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:09:43.347366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"85'24","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:47.521168+0000","last_change":"2024-05-08T22:05:13.547289+0000","last_active":"2024-05-08T22:05:47.521168+0000","last_peered":"2024-05-08T22:05:47.521168+0000","last_clean":"2024-05-08T22:05:47.521168+0000","last_became_active":"2024-05-08T22:05:13.546860+0000","last_became_peered":"2024-05-08T22:05:13.546860+0000","last_unstale":"2024-05-08T22:05:47.521168+0000","last_undegraded":"2024-05-08T22:05:47.521168+0000","last_fullsized":"2024-05-08T22:05:47.521168+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:08:50.339361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45627,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":23,"num_read_kb":56,"num_write":25,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831151+0000","last_change":"2024-05-08T22:05:14.057925+0000","last_active":"2024-05-08T22:05:40.831151+0000","last_peered":"2024-05-08T22:05:40.831151+0000","last_clean":"2024-05-08T22:05:40.831151+0000","last_became_active":"2024-05-08T22:05:14.057662+0000","last_became_peered":"2024-05-08T22:05:14.057662+0000","last_unstale":"2024-05-08T22:05:40.831151+0000","last_undegraded":"2024-05-08T22:05:40.831151+0000","last_fullsized":"2024-05-08T22:05:40.831151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:49.929610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827811+0000","last_change":"2024-05-08T22:05:18.535509+0000","last_active":"2024-05-08T22:05:40.827811+0000","last_peered":"2024-05-08T22:05:40.827811+0000","last_clean":"2024-05-08T22:05:40.827811+0000","last_became_active":"2024-05-08T22:05:18.535024+0000","last_became_peered":"2024-05-08T22:05:18.535024+0000","last_unstale":"2024-05-08T22:05:40.827811+0000","last_undegraded":"2024-05-08T22:05:40.827811+0000","last_fullsized":"2024-05-08T22:05:40.827811+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:28.098737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817350+0000","last_change":"2024-05-08T22:05:13.552096+0000","last_active":"2024-05-08T22:05:40.817350+0000","last_peered":"2024-05-08T22:05:40.817350+0000","last_clean":"2024-05-08T22:05:40.817350+0000","last_became_active":"2024-05-08T22:05:13.551655+0000","last_became_peered":"2024-05-08T22:05:13.551655+0000","last_unstale":"2024-05-08T22:05:40.817350+0000","last_undegraded":"2024-05-08T22:05:40.817350+0000","last_fullsized":"2024-05-08T22:05:40.817350+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:24.371008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.821551+0000","last_change":"2024-05-08T22:05:14.066075+0000","last_active":"2024-05-08T22:05:40.821551+0000","last_peered":"2024-05-08T22:05:40.821551+0000","last_clean":"2024-05-08T22:05:40.821551+0000","last_became_active":"2024-05-08T22:05:14.065581+0000","last_became_peered":"2024-05-08T22:05:14.065581+0000","last_unstale":"2024-05-08T22:05:40.821551+0000","last_undegraded":"2024-05-08T22:05:40.821551+0000","last_fullsized":"2024-05-08T22:05:40.821551+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:15.403714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.831782+0000","last_change":"2024-05-08T22:05:41.831782+0000","last_active":"2024-05-08T22:05:41.831782+0000","last_peered":"2024-05-08T22:05:41.831782+0000","last_clean":"2024-05-08T22:05:41.831782+0000","last_became_active":"2024-05-08T22:05:41.831335+0000","last_became_peered":"2024-05-08T22:05:41.831335+0000","last_unstale":"2024-05-08T22:05:41.831782+0000","last_undegraded":"2024-05-08T22:05:41.831782+0000","last_fullsized":"2024-05-08T22:05:41.831782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:39:34.031128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827308+0000","last_change":"2024-05-08T22:05:13.549112+0000","last_active":"2024-05-08T22:05:40.827308+0000","last_peered":"2024-05-08T22:05:40.827308+0000","last_clean":"2024-05-08T22:05:40.827308+0000","last_became_active":"2024-05-08T22:05:13.548763+0000","last_became_peered":"2024-05-08T22:05:13.548763+0000","last_unstale":"2024-05-08T22:05:40.827308+0000","last_undegraded":"2024-05-08T22:05:40.827308+0000","last_fullsized":"2024-05-08T22:05:40.827308+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:26:56.448611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830785+0000","last_change":"2024-05-08T22:05:14.067116+0000","last_active":"2024-05-08T22:05:40.830785+0000","last_peered":"2024-05-08T22:05:40.830785+0000","last_clean":"2024-05-08T22:05:40.830785+0000","last_became_active":"2024-05-08T22:05:14.064775+0000","last_became_peered":"2024-05-08T22:05:14.064775+0000","last_unstale":"2024-05-08T22:05:40.830785+0000","last_undegraded":"2024-05-08T22:05:40.830785+0000","last_fullsized":"2024-05-08T22:05:40.830785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:28:32.901405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817060+0000","last_change":"2024-05-08T22:05:18.539546+0000","last_active":"2024-05-08T22:05:40.817060+0000","last_peered":"2024-05-08T22:05:40.817060+0000","last_clean":"2024-05-08T22:05:40.817060+0000","last_became_active":"2024-05-08T22:05:18.538998+0000","last_became_peered":"2024-05-08T22:05:18.538998+0000","last_unstale":"2024-05-08T22:05:40.817060+0000","last_undegraded":"2024-05-08T22:05:40.817060+0000","last_fullsized":"2024-05-08T22:05:40.817060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:30.453881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"83'3","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:45.514173+0000","last_change":"2024-05-08T22:05:13.549053+0000","last_active":"2024-05-08T22:05:45.514173+0000","last_peered":"2024-05-08T22:05:45.514173+0000","last_clean":"2024-05-08T22:05:45.514173+0000","last_became_active":"2024-05-08T22:05:13.548739+0000","last_became_peered":"2024-05-08T22:05:13.548739+0000","last_unstale":"2024-05-08T22:05:45.514173+0000","last_undegraded":"2024-05-08T22:05:45.514173+0000","last_fullsized":"2024-05-08T22:05:45.514173+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:07:10.298084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830386+0000","last_change":"2024-05-08T22:05:14.061753+0000","last_active":"2024-05-08T22:05:40.830386+0000","last_peered":"2024-05-08T22:05:40.830386+0000","last_clean":"2024-05-08T22:05:40.830386+0000","last_became_active":"2024-05-08T22:05:14.061412+0000","last_became_peered":"2024-05-08T22:05:14.061412+0000","last_unstale":"2024-05-08T22:05:40.830386+0000","last_undegraded":"2024-05-08T22:05:40.830386+0000","last_fullsized":"2024-05-08T22:05:40.830386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:54.380181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824969+0000","last_change":"2024-05-08T22:05:18.532627+0000","last_active":"2024-05-08T22:05:40.824969+0000","last_peered":"2024-05-08T22:05:40.824969+0000","last_clean":"2024-05-08T22:05:40.824969+0000","last_became_active":"2024-05-08T22:05:18.532205+0000","last_became_peered":"2024-05-08T22:05:18.532205+0000","last_unstale":"2024-05-08T22:05:40.824969+0000","last_undegraded":"2024-05-08T22:05:40.824969+0000","last_fullsized":"2024-05-08T22:05:40.824969+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:20:11.028645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":61,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.841832+0000","last_change":"2024-05-08T22:05:41.841832+0000","last_active":"2024-05-08T22:05:41.841832+0000","last_peered":"2024-05-08T22:05:41.841832+0000","last_clean":"2024-05-08T22:05:41.841832+0000","last_became_active":"2024-05-08T22:05:41.829584+0000","last_became_peered":"2024-05-08T22:05:41.829584+0000","last_unstale":"2024-05-08T22:05:41.841832+0000","last_undegraded":"2024-05-08T22:05:41.841832+0000","last_fullsized":"2024-05-08T22:05:41.841832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:52:19.876215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824504+0000","last_change":"2024-05-08T22:05:14.063027+0000","last_active":"2024-05-08T22:05:40.824504+0000","last_peered":"2024-05-08T22:05:40.824504+0000","last_clean":"2024-05-08T22:05:40.824504+0000","last_became_active":"2024-05-08T22:05:14.062682+0000","last_became_peered":"2024-05-08T22:05:14.062682+0000","last_unstale":"2024-05-08T22:05:40.824504+0000","last_undegraded":"2024-05-08T22:05:40.824504+0000","last_fullsized":"2024-05-08T22:05:40.824504+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:47.248832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824420+0000","last_change":"2024-05-08T22:05:18.528370+0000","last_active":"2024-05-08T22:05:40.824420+0000","last_peered":"2024-05-08T22:05:40.824420+0000","last_clean":"2024-05-08T22:05:40.824420+0000","last_became_active":"2024-05-08T22:05:18.527990+0000","last_became_peered":"2024-05-08T22:05:18.527990+0000","last_unstale":"2024-05-08T22:05:40.824420+0000","last_undegraded":"2024-05-08T22:05:40.824420+0000","last_fullsized":"2024-05-08T22:05:40.824420+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:07.524175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.824561+0000","last_change":"2024-05-08T22:05:13.547543+0000","last_active":"2024-05-08T22:05:41.824561+0000","last_peered":"2024-05-08T22:05:41.824561+0000","last_clean":"2024-05-08T22:05:41.824561+0000","last_became_active":"2024-05-08T22:05:13.547191+0000","last_became_peered":"2024-05-08T22:05:13.547191+0000","last_unstale":"2024-05-08T22:05:41.824561+0000","last_undegraded":"2024-05-08T22:05:41.824561+0000","last_fullsized":"2024-05-08T22:05:41.824561+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:53.213239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.820871+0000","last_change":"2024-05-08T22:05:14.062677+0000","last_active":"2024-05-08T22:05:40.820871+0000","last_peered":"2024-05-08T22:05:40.820871+0000","last_clean":"2024-05-08T22:05:40.820871+0000","last_became_active":"2024-05-08T22:05:14.062306+0000","last_became_peered":"2024-05-08T22:05:14.062306+0000","last_unstale":"2024-05-08T22:05:40.820871+0000","last_undegraded":"2024-05-08T22:05:40.820871+0000","last_fullsized":"2024-05-08T22:05:40.820871+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:00.356505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830129+0000","last_change":"2024-05-08T22:05:18.617075+0000","last_active":"2024-05-08T22:05:40.830129+0000","last_peered":"2024-05-08T22:05:40.830129+0000","last_clean":"2024-05-08T22:05:40.830129+0000","last_became_active":"2024-05-08T22:05:18.616653+0000","last_became_peered":"2024-05-08T22:05:18.616653+0000","last_unstale":"2024-05-08T22:05:40.830129+0000","last_undegraded":"2024-05-08T22:05:40.830129+0000","last_fullsized":"2024-05-08T22:05:40.830129+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:30.537064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.816884+0000","last_change":"2024-05-08T22:05:13.553457+0000","last_active":"2024-05-08T22:05:40.816884+0000","last_peered":"2024-05-08T22:05:40.816884+0000","last_clean":"2024-05-08T22:05:40.816884+0000","last_became_active":"2024-05-08T22:05:13.551583+0000","last_became_peered":"2024-05-08T22:05:13.551583+0000","last_unstale":"2024-05-08T22:05:40.816884+0000","last_undegraded":"2024-05-08T22:05:40.816884+0000","last_fullsized":"2024-05-08T22:05:40.816884+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:07:05.971413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.822908+0000","last_change":"2024-05-08T22:05:14.057476+0000","last_active":"2024-05-08T22:05:40.822908+0000","last_peered":"2024-05-08T22:05:40.822908+0000","last_clean":"2024-05-08T22:05:40.822908+0000","last_became_active":"2024-05-08T22:05:14.056550+0000","last_became_peered":"2024-05-08T22:05:14.056550+0000","last_unstale":"2024-05-08T22:05:40.822908+0000","last_undegraded":"2024-05-08T22:05:40.822908+0000","last_fullsized":"2024-05-08T22:05:40.822908+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:41:29.264096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824298+0000","last_change":"2024-05-08T22:05:18.530719+0000","last_active":"2024-05-08T22:05:40.824298+0000","last_peered":"2024-05-08T22:05:40.824298+0000","last_clean":"2024-05-08T22:05:40.824298+0000","last_became_active":"2024-05-08T22:05:18.529659+0000","last_became_peered":"2024-05-08T22:05:18.529659+0000","last_unstale":"2024-05-08T22:05:40.824298+0000","last_undegraded":"2024-05-08T22:05:40.824298+0000","last_fullsized":"2024-05-08T22:05:40.824298+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:13:06.514460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830542+0000","last_change":"2024-05-08T22:05:13.556191+0000","last_active":"2024-05-08T22:05:40.830542+0000","last_peered":"2024-05-08T22:05:40.830542+0000","last_clean":"2024-05-08T22:05:40.830542+0000","last_became_active":"2024-05-08T22:05:13.555888+0000","last_became_peered":"2024-05-08T22:05:13.555888+0000","last_unstale":"2024-05-08T22:05:40.830542+0000","last_undegraded":"2024-05-08T22:05:40.830542+0000","last_fullsized":"2024-05-08T22:05:40.830542+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:05.637980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.828023+0000","last_change":"2024-05-08T22:05:41.828023+0000","last_active":"2024-05-08T22:05:41.828023+0000","last_peered":"2024-05-08T22:05:41.828023+0000","last_clean":"2024-05-08T22:05:41.828023+0000","last_became_active":"2024-05-08T22:05:41.827550+0000","last_became_peered":"2024-05-08T22:05:41.827550+0000","last_unstale":"2024-05-08T22:05:41.828023+0000","last_undegraded":"2024-05-08T22:05:41.828023+0000","last_fullsized":"2024-05-08T22:05:41.828023+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:13.234423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.828529+0000","last_change":"2024-05-08T22:05:41.828529+0000","last_active":"2024-05-08T22:05:41.828529+0000","last_peered":"2024-05-08T22:05:41.828529+0000","last_clean":"2024-05-08T22:05:41.828529+0000","last_became_active":"2024-05-08T22:05:41.828081+0000","last_became_peered":"2024-05-08T22:05:41.828081+0000","last_unstale":"2024-05-08T22:05:41.828529+0000","last_undegraded":"2024-05-08T22:05:41.828529+0000","last_fullsized":"2024-05-08T22:05:41.828529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:27.478823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826551+0000","last_change":"2024-05-08T22:05:14.069096+0000","last_active":"2024-05-08T22:05:40.826551+0000","last_peered":"2024-05-08T22:05:40.826551+0000","last_clean":"2024-05-08T22:05:40.826551+0000","last_became_active":"2024-05-08T22:05:14.068796+0000","last_became_peered":"2024-05-08T22:05:14.068796+0000","last_unstale":"2024-05-08T22:05:40.826551+0000","last_undegraded":"2024-05-08T22:05:40.826551+0000","last_fullsized":"2024-05-08T22:05:40.826551+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:52:52.391196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:45.513401+0000","last_change":"2024-05-08T22:05:13.559286+0000","last_active":"2024-05-08T22:05:45.513401+0000","last_peered":"2024-05-08T22:05:45.513401+0000","last_clean":"2024-05-08T22:05:45.513401+0000","last_became_active":"2024-05-08T22:05:13.558922+0000","last_became_peered":"2024-05-08T22:05:13.558922+0000","last_unstale":"2024-05-08T22:05:45.513401+0000","last_undegraded":"2024-05-08T22:05:45.513401+0000","last_fullsized":"2024-05-08T22:05:45.513401+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:25:24.732422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831007+0000","last_change":"2024-05-08T22:05:18.527031+0000","last_active":"2024-05-08T22:05:40.831007+0000","last_peered":"2024-05-08T22:05:40.831007+0000","last_clean":"2024-05-08T22:05:40.831007+0000","last_became_active":"2024-05-08T22:05:18.526573+0000","last_became_peered":"2024-05-08T22:05:18.526573+0000","last_unstale":"2024-05-08T22:05:40.831007+0000","last_undegraded":"2024-05-08T22:05:40.831007+0000","last_fullsized":"2024-05-08T22:05:40.831007+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:08.492359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827472+0000","last_change":"2024-05-08T22:05:14.053930+0000","last_active":"2024-05-08T22:05:40.827472+0000","last_peered":"2024-05-08T22:05:40.827472+0000","last_clean":"2024-05-08T22:05:40.827472+0000","last_became_active":"2024-05-08T22:05:14.052968+0000","last_became_peered":"2024-05-08T22:05:14.052968+0000","last_unstale":"2024-05-08T22:05:40.827472+0000","last_undegraded":"2024-05-08T22:05:40.827472+0000","last_fullsized":"2024-05-08T22:05:40.827472+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:48.748323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819345+0000","last_change":"2024-05-08T22:05:13.545709+0000","last_active":"2024-05-08T22:05:40.819345+0000","last_peered":"2024-05-08T22:05:40.819345+0000","last_clean":"2024-05-08T22:05:40.819345+0000","last_became_active":"2024-05-08T22:05:13.545419+0000","last_became_peered":"2024-05-08T22:05:13.545419+0000","last_unstale":"2024-05-08T22:05:40.819345+0000","last_undegraded":"2024-05-08T22:05:40.819345+0000","last_fullsized":"2024-05-08T22:05:40.819345+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:44.903903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":337,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830206+0000","last_change":"2024-05-08T22:04:20.948328+0000","last_active":"2024-05-08T22:05:40.830206+0000","last_peered":"2024-05-08T22:05:40.830206+0000","last_clean":"2024-05-08T22:05:40.830206+0000","last_became_active":"2024-05-08T22:04:20.947887+0000","last_became_peered":"2024-05-08T22:04:20.947887+0000","last_unstale":"2024-05-08T22:05:40.830206+0000","last_undegraded":"2024-05-08T22:05:40.830206+0000","last_fullsized":"2024-05-08T22:05:40.830206+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:01:41.485277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:01:41.485277+0000","last_clean_scrub_stamp":"2024-05-08T22:01:41.485277+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:43:59.208120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.824591+0000","last_change":"2024-05-08T22:05:41.824591+0000","last_active":"2024-05-08T22:05:41.824591+0000","last_peered":"2024-05-08T22:05:41.824591+0000","last_clean":"2024-05-08T22:05:41.824591+0000","last_became_active":"2024-05-08T22:05:41.824053+0000","last_became_peered":"2024-05-08T22:05:41.824053+0000","last_unstale":"2024-05-08T22:05:41.824591+0000","last_undegraded":"2024-05-08T22:05:41.824591+0000","last_fullsized":"2024-05-08T22:05:41.824591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:18:57.269459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.820969+0000","last_change":"2024-05-08T22:05:14.066818+0000","last_active":"2024-05-08T22:05:40.820969+0000","last_peered":"2024-05-08T22:05:40.820969+0000","last_clean":"2024-05-08T22:05:40.820969+0000","last_became_active":"2024-05-08T22:05:14.065152+0000","last_became_peered":"2024-05-08T22:05:14.065152+0000","last_unstale":"2024-05-08T22:05:40.820969+0000","last_undegraded":"2024-05-08T22:05:40.820969+0000","last_fullsized":"2024-05-08T22:05:40.820969+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:53:19.018852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830323+0000","last_change":"2024-05-08T22:05:13.550365+0000","last_active":"2024-05-08T22:05:40.830323+0000","last_peered":"2024-05-08T22:05:40.830323+0000","last_clean":"2024-05-08T22:05:40.830323+0000","last_became_active":"2024-05-08T22:05:13.549677+0000","last_became_peered":"2024-05-08T22:05:13.549677+0000","last_unstale":"2024-05-08T22:05:40.830323+0000","last_undegraded":"2024-05-08T22:05:40.830323+0000","last_fullsized":"2024-05-08T22:05:40.830323+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:10.842838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.820719+0000","last_change":"2024-05-08T22:05:18.535777+0000","last_active":"2024-05-08T22:05:40.820719+0000","last_peered":"2024-05-08T22:05:40.820719+0000","last_clean":"2024-05-08T22:05:40.820719+0000","last_became_active":"2024-05-08T22:05:18.535327+0000","last_became_peered":"2024-05-08T22:05:18.535327+0000","last_unstale":"2024-05-08T22:05:40.820719+0000","last_undegraded":"2024-05-08T22:05:40.820719+0000","last_fullsized":"2024-05-08T22:05:40.820719+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:13.886989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831170+0000","last_change":"2024-05-08T22:05:13.554045+0000","last_active":"2024-05-08T22:05:40.831170+0000","last_peered":"2024-05-08T22:05:40.831170+0000","last_clean":"2024-05-08T22:05:40.831170+0000","last_became_active":"2024-05-08T22:05:13.553315+0000","last_became_peered":"2024-05-08T22:05:13.553315+0000","last_unstale":"2024-05-08T22:05:40.831170+0000","last_undegraded":"2024-05-08T22:05:40.831170+0000","last_fullsized":"2024-05-08T22:05:40.831170+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:13.131267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824438+0000","last_change":"2024-05-08T22:05:14.065018+0000","last_active":"2024-05-08T22:05:40.824438+0000","last_peered":"2024-05-08T22:05:40.824438+0000","last_clean":"2024-05-08T22:05:40.824438+0000","last_became_active":"2024-05-08T22:05:14.064701+0000","last_became_peered":"2024-05-08T22:05:14.064701+0000","last_unstale":"2024-05-08T22:05:40.824438+0000","last_undegraded":"2024-05-08T22:05:40.824438+0000","last_fullsized":"2024-05-08T22:05:40.824438+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:48:43.468373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826618+0000","last_change":"2024-05-08T22:05:18.617853+0000","last_active":"2024-05-08T22:05:40.826618+0000","last_peered":"2024-05-08T22:05:40.826618+0000","last_clean":"2024-05-08T22:05:40.826618+0000","last_became_active":"2024-05-08T22:05:18.616255+0000","last_became_peered":"2024-05-08T22:05:18.616255+0000","last_unstale":"2024-05-08T22:05:40.826618+0000","last_undegraded":"2024-05-08T22:05:40.826618+0000","last_fullsized":"2024-05-08T22:05:40.826618+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:13.233425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.817925+0000","last_change":"2024-05-08T22:05:13.546274+0000","last_active":"2024-05-08T22:05:41.817925+0000","last_peered":"2024-05-08T22:05:41.817925+0000","last_clean":"2024-05-08T22:05:41.817925+0000","last_became_active":"2024-05-08T22:05:13.543651+0000","last_became_peered":"2024-05-08T22:05:13.543651+0000","last_unstale":"2024-05-08T22:05:41.817925+0000","last_undegraded":"2024-05-08T22:05:41.817925+0000","last_fullsized":"2024-05-08T22:05:41.817925+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:21:58.601261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826229+0000","last_change":"2024-05-08T22:05:14.056535+0000","last_active":"2024-05-08T22:05:40.826229+0000","last_peered":"2024-05-08T22:05:40.826229+0000","last_clean":"2024-05-08T22:05:40.826229+0000","last_became_active":"2024-05-08T22:05:14.056211+0000","last_became_peered":"2024-05-08T22:05:14.056211+0000","last_unstale":"2024-05-08T22:05:40.826229+0000","last_undegraded":"2024-05-08T22:05:40.826229+0000","last_fullsized":"2024-05-08T22:05:40.826229+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:46:30.744490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.816916+0000","last_change":"2024-05-08T22:05:18.524939+0000","last_active":"2024-05-08T22:05:40.816916+0000","last_peered":"2024-05-08T22:05:40.816916+0000","last_clean":"2024-05-08T22:05:40.816916+0000","last_became_active":"2024-05-08T22:05:18.524518+0000","last_became_peered":"2024-05-08T22:05:18.524518+0000","last_unstale":"2024-05-08T22:05:40.816916+0000","last_undegraded":"2024-05-08T22:05:40.816916+0000","last_fullsized":"2024-05-08T22:05:40.816916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:50.192400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819897+0000","last_change":"2024-05-08T22:05:13.554346+0000","last_active":"2024-05-08T22:05:40.819897+0000","last_peered":"2024-05-08T22:05:40.819897+0000","last_clean":"2024-05-08T22:05:40.819897+0000","last_became_active":"2024-05-08T22:05:13.554080+0000","last_became_peered":"2024-05-08T22:05:13.554080+0000","last_unstale":"2024-05-08T22:05:40.819897+0000","last_undegraded":"2024-05-08T22:05:40.819897+0000","last_fullsized":"2024-05-08T22:05:40.819897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:29:49.695682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832687+0000","last_change":"2024-05-08T22:05:14.071422+0000","last_active":"2024-05-08T22:05:40.832687+0000","last_peered":"2024-05-08T22:05:40.832687+0000","last_clean":"2024-05-08T22:05:40.832687+0000","last_became_active":"2024-05-08T22:05:14.070396+0000","last_became_peered":"2024-05-08T22:05:14.070396+0000","last_unstale":"2024-05-08T22:05:40.832687+0000","last_undegraded":"2024-05-08T22:05:40.832687+0000","last_fullsized":"2024-05-08T22:05:40.832687+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:37.957983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819540+0000","last_change":"2024-05-08T22:05:18.617149+0000","last_active":"2024-05-08T22:05:40.819540+0000","last_peered":"2024-05-08T22:05:40.819540+0000","last_clean":"2024-05-08T22:05:40.819540+0000","last_became_active":"2024-05-08T22:05:18.616699+0000","last_became_peered":"2024-05-08T22:05:18.616699+0000","last_unstale":"2024-05-08T22:05:40.819540+0000","last_undegraded":"2024-05-08T22:05:40.819540+0000","last_fullsized":"2024-05-08T22:05:40.819540+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:51.167142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.823777+0000","last_change":"2024-05-08T22:05:13.551212+0000","last_active":"2024-05-08T22:05:40.823777+0000","last_peered":"2024-05-08T22:05:40.823777+0000","last_clean":"2024-05-08T22:05:40.823777+0000","last_became_active":"2024-05-08T22:05:13.550909+0000","last_became_peered":"2024-05-08T22:05:13.550909+0000","last_unstale":"2024-05-08T22:05:40.823777+0000","last_undegraded":"2024-05-08T22:05:40.823777+0000","last_fullsized":"2024-05-08T22:05:40.823777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:48.657928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829703+0000","last_change":"2024-05-08T22:05:14.053976+0000","last_active":"2024-05-08T22:05:40.829703+0000","last_peered":"2024-05-08T22:05:40.829703+0000","last_clean":"2024-05-08T22:05:40.829703+0000","last_became_active":"2024-05-08T22:05:14.053664+0000","last_became_peered":"2024-05-08T22:05:14.053664+0000","last_unstale":"2024-05-08T22:05:40.829703+0000","last_undegraded":"2024-05-08T22:05:40.829703+0000","last_fullsized":"2024-05-08T22:05:40.829703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:25:25.642768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817203+0000","last_change":"2024-05-08T22:05:18.536771+0000","last_active":"2024-05-08T22:05:40.817203+0000","last_peered":"2024-05-08T22:05:40.817203+0000","last_clean":"2024-05-08T22:05:40.817203+0000","last_became_active":"2024-05-08T22:05:18.536260+0000","last_became_peered":"2024-05-08T22:05:18.536260+0000","last_unstale":"2024-05-08T22:05:40.817203+0000","last_undegraded":"2024-05-08T22:05:40.817203+0000","last_fullsized":"2024-05-08T22:05:40.817203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:34.414381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827085+0000","last_change":"2024-05-08T22:05:13.540679+0000","last_active":"2024-05-08T22:05:40.827085+0000","last_peered":"2024-05-08T22:05:40.827085+0000","last_clean":"2024-05-08T22:05:40.827085+0000","last_became_active":"2024-05-08T22:05:13.539428+0000","last_became_peered":"2024-05-08T22:05:13.539428+0000","last_unstale":"2024-05-08T22:05:40.827085+0000","last_undegraded":"2024-05-08T22:05:40.827085+0000","last_fullsized":"2024-05-08T22:05:40.827085+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:59:34.455411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.820331+0000","last_change":"2024-05-08T22:05:14.061849+0000","last_active":"2024-05-08T22:05:40.820331+0000","last_peered":"2024-05-08T22:05:40.820331+0000","last_clean":"2024-05-08T22:05:40.820331+0000","last_became_active":"2024-05-08T22:05:14.061028+0000","last_became_peered":"2024-05-08T22:05:14.061028+0000","last_unstale":"2024-05-08T22:05:40.820331+0000","last_undegraded":"2024-05-08T22:05:40.820331+0000","last_fullsized":"2024-05-08T22:05:40.820331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:11:59.489376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.477584+0000","last_change":"2024-05-08T22:05:18.517994+0000","last_active":"2024-05-08T22:05:41.477584+0000","last_peered":"2024-05-08T22:05:41.477584+0000","last_clean":"2024-05-08T22:05:41.477584+0000","last_became_active":"2024-05-08T22:05:18.517655+0000","last_became_peered":"2024-05-08T22:05:18.517655+0000","last_unstale":"2024-05-08T22:05:41.477584+0000","last_undegraded":"2024-05-08T22:05:41.477584+0000","last_fullsized":"2024-05-08T22:05:41.477584+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:50:08.956277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"84'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.025839+0000","last_change":"2024-05-08T22:05:13.552851+0000","last_active":"2024-05-08T22:05:41.025839+0000","last_peered":"2024-05-08T22:05:41.025839+0000","last_clean":"2024-05-08T22:05:41.025839+0000","last_became_active":"2024-05-08T22:05:13.552507+0000","last_became_peered":"2024-05-08T22:05:13.552507+0000","last_unstale":"2024-05-08T22:05:41.025839+0000","last_undegraded":"2024-05-08T22:05:41.025839+0000","last_fullsized":"2024-05-08T22:05:41.025839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:29.248623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.828040+0000","last_change":"2024-05-08T22:05:14.062618+0000","last_active":"2024-05-08T22:05:40.828040+0000","last_peered":"2024-05-08T22:05:40.828040+0000","last_clean":"2024-05-08T22:05:40.828040+0000","last_became_active":"2024-05-08T22:05:14.062308+0000","last_became_peered":"2024-05-08T22:05:14.062308+0000","last_unstale":"2024-05-08T22:05:40.828040+0000","last_undegraded":"2024-05-08T22:05:40.828040+0000","last_fullsized":"2024-05-08T22:05:40.828040+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:34.195343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819863+0000","last_change":"2024-05-08T22:05:18.616846+0000","last_active":"2024-05-08T22:05:40.819863+0000","last_peered":"2024-05-08T22:05:40.819863+0000","last_clean":"2024-05-08T22:05:40.819863+0000","last_became_active":"2024-05-08T22:05:18.616369+0000","last_became_peered":"2024-05-08T22:05:18.616369+0000","last_unstale":"2024-05-08T22:05:40.819863+0000","last_undegraded":"2024-05-08T22:05:40.819863+0000","last_fullsized":"2024-05-08T22:05:40.819863+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:26.128949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829370+0000","last_change":"2024-05-08T22:05:13.549712+0000","last_active":"2024-05-08T22:05:40.829370+0000","last_peered":"2024-05-08T22:05:40.829370+0000","last_clean":"2024-05-08T22:05:40.829370+0000","last_became_active":"2024-05-08T22:05:13.549508+0000","last_became_peered":"2024-05-08T22:05:13.549508+0000","last_unstale":"2024-05-08T22:05:40.829370+0000","last_undegraded":"2024-05-08T22:05:40.829370+0000","last_fullsized":"2024-05-08T22:05:40.829370+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:30:22.706389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830535+0000","last_change":"2024-05-08T22:05:14.060920+0000","last_active":"2024-05-08T22:05:40.830535+0000","last_peered":"2024-05-08T22:05:40.830535+0000","last_clean":"2024-05-08T22:05:40.830535+0000","last_became_active":"2024-05-08T22:05:14.059914+0000","last_became_peered":"2024-05-08T22:05:14.059914+0000","last_unstale":"2024-05-08T22:05:40.830535+0000","last_undegraded":"2024-05-08T22:05:40.830535+0000","last_fullsized":"2024-05-08T22:05:40.830535+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:15.513600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.822527+0000","last_change":"2024-05-08T22:05:18.534076+0000","last_active":"2024-05-08T22:05:40.822527+0000","last_peered":"2024-05-08T22:05:40.822527+0000","last_clean":"2024-05-08T22:05:40.822527+0000","last_became_active":"2024-05-08T22:05:18.533679+0000","last_became_peered":"2024-05-08T22:05:18.533679+0000","last_unstale":"2024-05-08T22:05:40.822527+0000","last_undegraded":"2024-05-08T22:05:40.822527+0000","last_fullsized":"2024-05-08T22:05:40.822527+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:08.979193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.845713+0000","last_change":"2024-05-08T22:05:41.845354+0000","last_active":"2024-05-08T22:05:41.845713+0000","last_peered":"2024-05-08T22:05:41.845713+0000","last_clean":"2024-05-08T22:05:41.845713+0000","last_became_active":"2024-05-08T22:05:41.830506+0000","last_became_peered":"2024-05-08T22:05:41.830506+0000","last_unstale":"2024-05-08T22:05:41.845713+0000","last_undegraded":"2024-05-08T22:05:41.845713+0000","last_fullsized":"2024-05-08T22:05:41.845713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:53:23.081939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831111+0000","last_change":"2024-05-08T22:05:14.065868+0000","last_active":"2024-05-08T22:05:40.831111+0000","last_peered":"2024-05-08T22:05:40.831111+0000","last_clean":"2024-05-08T22:05:40.831111+0000","last_became_active":"2024-05-08T22:05:14.065545+0000","last_became_peered":"2024-05-08T22:05:14.065545+0000","last_unstale":"2024-05-08T22:05:40.831111+0000","last_undegraded":"2024-05-08T22:05:40.831111+0000","last_fullsized":"2024-05-08T22:05:40.831111+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:56:34.309095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832203+0000","last_change":"2024-05-08T22:05:18.534261+0000","last_active":"2024-05-08T22:05:40.832203+0000","last_peered":"2024-05-08T22:05:40.832203+0000","last_clean":"2024-05-08T22:05:40.832203+0000","last_became_active":"2024-05-08T22:05:18.531814+0000","last_became_peered":"2024-05-08T22:05:18.531814+0000","last_unstale":"2024-05-08T22:05:40.832203+0000","last_undegraded":"2024-05-08T22:05:40.832203+0000","last_fullsized":"2024-05-08T22:05:40.832203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:38.344546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827613+0000","last_change":"2024-05-08T22:05:13.549344+0000","last_active":"2024-05-08T22:05:40.827613+0000","last_peered":"2024-05-08T22:05:40.827613+0000","last_clean":"2024-05-08T22:05:40.827613+0000","last_became_active":"2024-05-08T22:05:13.548993+0000","last_became_peered":"2024-05-08T22:05:13.548993+0000","last_unstale":"2024-05-08T22:05:40.827613+0000","last_undegraded":"2024-05-08T22:05:40.827613+0000","last_fullsized":"2024-05-08T22:05:40.827613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:50.096602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830884+0000","last_change":"2024-05-08T22:05:14.065758+0000","last_active":"2024-05-08T22:05:40.830884+0000","last_peered":"2024-05-08T22:05:40.830884+0000","last_clean":"2024-05-08T22:05:40.830884+0000","last_became_active":"2024-05-08T22:05:14.065442+0000","last_became_peered":"2024-05-08T22:05:14.065442+0000","last_unstale":"2024-05-08T22:05:40.830884+0000","last_undegraded":"2024-05-08T22:05:40.830884+0000","last_fullsized":"2024-05-08T22:05:40.830884+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:43.047796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826157+0000","last_change":"2024-05-08T22:05:18.529804+0000","last_active":"2024-05-08T22:05:40.826157+0000","last_peered":"2024-05-08T22:05:40.826157+0000","last_clean":"2024-05-08T22:05:40.826157+0000","last_became_active":"2024-05-08T22:05:18.529346+0000","last_became_peered":"2024-05-08T22:05:18.529346+0000","last_unstale":"2024-05-08T22:05:40.826157+0000","last_undegraded":"2024-05-08T22:05:40.826157+0000","last_fullsized":"2024-05-08T22:05:40.826157+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:13.565707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824885+0000","last_change":"2024-05-08T22:05:13.551818+0000","last_active":"2024-05-08T22:05:40.824885+0000","last_peered":"2024-05-08T22:05:40.824885+0000","last_clean":"2024-05-08T22:05:40.824885+0000","last_became_active":"2024-05-08T22:05:13.551080+0000","last_became_peered":"2024-05-08T22:05:13.551080+0000","last_unstale":"2024-05-08T22:05:40.824885+0000","last_undegraded":"2024-05-08T22:05:40.824885+0000","last_fullsized":"2024-05-08T22:05:40.824885+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:31.636482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831045+0000","last_change":"2024-05-08T22:05:14.066572+0000","last_active":"2024-05-08T22:05:40.831045+0000","last_peered":"2024-05-08T22:05:40.831045+0000","last_clean":"2024-05-08T22:05:40.831045+0000","last_became_active":"2024-05-08T22:05:14.066272+0000","last_became_peered":"2024-05-08T22:05:14.066272+0000","last_unstale":"2024-05-08T22:05:40.831045+0000","last_undegraded":"2024-05-08T22:05:40.831045+0000","last_fullsized":"2024-05-08T22:05:40.831045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:20:27.423464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.820443+0000","last_change":"2024-05-08T22:05:14.061435+0000","last_active":"2024-05-08T22:05:40.820443+0000","last_peered":"2024-05-08T22:05:40.820443+0000","last_clean":"2024-05-08T22:05:40.820443+0000","last_became_active":"2024-05-08T22:05:14.061079+0000","last_became_peered":"2024-05-08T22:05:14.061079+0000","last_unstale":"2024-05-08T22:05:40.820443+0000","last_undegraded":"2024-05-08T22:05:40.820443+0000","last_fullsized":"2024-05-08T22:05:40.820443+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:42.684743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.828352+0000","last_change":"2024-05-08T22:05:41.828352+0000","last_active":"2024-05-08T22:05:41.828352+0000","last_peered":"2024-05-08T22:05:41.828352+0000","last_clean":"2024-05-08T22:05:41.828352+0000","last_became_active":"2024-05-08T22:05:41.827782+0000","last_became_peered":"2024-05-08T22:05:41.827782+0000","last_unstale":"2024-05-08T22:05:41.828352+0000","last_undegraded":"2024-05-08T22:05:41.828352+0000","last_fullsized":"2024-05-08T22:05:41.828352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:08.883148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817061+0000","last_change":"2024-05-08T22:05:14.066697+0000","last_active":"2024-05-08T22:05:40.817061+0000","last_peered":"2024-05-08T22:05:40.817061+0000","last_clean":"2024-05-08T22:05:40.817061+0000","last_became_active":"2024-05-08T22:05:14.065952+0000","last_became_peered":"2024-05-08T22:05:14.065952+0000","last_unstale":"2024-05-08T22:05:40.817061+0000","last_undegraded":"2024-05-08T22:05:40.817061+0000","last_fullsized":"2024-05-08T22:05:40.817061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:03:19.006644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.833301+0000","last_change":"2024-05-08T22:05:14.072295+0000","last_active":"2024-05-08T22:05:40.833301+0000","last_peered":"2024-05-08T22:05:40.833301+0000","last_clean":"2024-05-08T22:05:40.833301+0000","last_became_active":"2024-05-08T22:05:14.065034+0000","last_became_peered":"2024-05-08T22:05:14.065034+0000","last_unstale":"2024-05-08T22:05:40.833301+0000","last_undegraded":"2024-05-08T22:05:40.833301+0000","last_fullsized":"2024-05-08T22:05:40.833301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:08:41.380814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824510+0000","last_change":"2024-05-08T22:05:14.067263+0000","last_active":"2024-05-08T22:05:40.824510+0000","last_peered":"2024-05-08T22:05:40.824510+0000","last_clean":"2024-05-08T22:05:40.824510+0000","last_became_active":"2024-05-08T22:05:14.066957+0000","last_became_peered":"2024-05-08T22:05:14.066957+0000","last_unstale":"2024-05-08T22:05:40.824510+0000","last_undegraded":"2024-05-08T22:05:40.824510+0000","last_fullsized":"2024-05-08T22:05:40.824510+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:29:35.638489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.822207+0000","last_change":"2024-05-08T22:05:14.060561+0000","last_active":"2024-05-08T22:05:40.822207+0000","last_peered":"2024-05-08T22:05:40.822207+0000","last_clean":"2024-05-08T22:05:40.822207+0000","last_became_active":"2024-05-08T22:05:14.060150+0000","last_became_peered":"2024-05-08T22:05:14.060150+0000","last_unstale":"2024-05-08T22:05:40.822207+0000","last_undegraded":"2024-05-08T22:05:40.822207+0000","last_fullsized":"2024-05-08T22:05:40.822207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:22.710776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819471+0000","last_change":"2024-05-08T22:05:14.056394+0000","last_active":"2024-05-08T22:05:40.819471+0000","last_peered":"2024-05-08T22:05:40.819471+0000","last_clean":"2024-05-08T22:05:40.819471+0000","last_became_active":"2024-05-08T22:05:14.055977+0000","last_became_peered":"2024-05-08T22:05:14.055977+0000","last_unstale":"2024-05-08T22:05:40.819471+0000","last_undegraded":"2024-05-08T22:05:40.819471+0000","last_fullsized":"2024-05-08T22:05:40.819471+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:56:48.249379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829006+0000","last_change":"2024-05-08T22:05:14.061934+0000","last_active":"2024-05-08T22:05:40.829006+0000","last_peered":"2024-05-08T22:05:40.829006+0000","last_clean":"2024-05-08T22:05:40.829006+0000","last_became_active":"2024-05-08T22:05:14.061635+0000","last_became_peered":"2024-05-08T22:05:14.061635+0000","last_unstale":"2024-05-08T22:05:40.829006+0000","last_undegraded":"2024-05-08T22:05:40.829006+0000","last_fullsized":"2024-05-08T22:05:40.829006+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:02.423338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826467+0000","last_change":"2024-05-08T22:05:14.069982+0000","last_active":"2024-05-08T22:05:40.826467+0000","last_peered":"2024-05-08T22:05:40.826467+0000","last_clean":"2024-05-08T22:05:40.826467+0000","last_became_active":"2024-05-08T22:05:14.068293+0000","last_became_peered":"2024-05-08T22:05:14.068293+0000","last_unstale":"2024-05-08T22:05:40.826467+0000","last_undegraded":"2024-05-08T22:05:40.826467+0000","last_fullsized":"2024-05-08T22:05:40.826467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:36.396983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826968+0000","last_change":"2024-05-08T22:05:14.058393+0000","last_active":"2024-05-08T22:05:40.826968+0000","last_peered":"2024-05-08T22:05:40.826968+0000","last_clean":"2024-05-08T22:05:40.826968+0000","last_became_active":"2024-05-08T22:05:14.058083+0000","last_became_peered":"2024-05-08T22:05:14.058083+0000","last_unstale":"2024-05-08T22:05:40.826968+0000","last_undegraded":"2024-05-08T22:05:40.826968+0000","last_fullsized":"2024-05-08T22:05:40.826968+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:10:47.074957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.822859+0000","last_change":"2024-05-08T22:05:14.056989+0000","last_active":"2024-05-08T22:05:40.822859+0000","last_peered":"2024-05-08T22:05:40.822859+0000","last_clean":"2024-05-08T22:05:40.822859+0000","last_became_active":"2024-05-08T22:05:14.056669+0000","last_became_peered":"2024-05-08T22:05:14.056669+0000","last_unstale":"2024-05-08T22:05:40.822859+0000","last_undegraded":"2024-05-08T22:05:40.822859+0000","last_fullsized":"2024-05-08T22:05:40.822859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:56:51.173885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.823896+0000","last_change":"2024-05-08T22:05:14.058635+0000","last_active":"2024-05-08T22:05:40.823896+0000","last_peered":"2024-05-08T22:05:40.823896+0000","last_clean":"2024-05-08T22:05:40.823896+0000","last_became_active":"2024-05-08T22:05:14.058243+0000","last_became_peered":"2024-05-08T22:05:14.058243+0000","last_unstale":"2024-05-08T22:05:40.823896+0000","last_undegraded":"2024-05-08T22:05:40.823896+0000","last_fullsized":"2024-05-08T22:05:40.823896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:18.142563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.822551+0000","last_change":"2024-05-08T22:05:14.060735+0000","last_active":"2024-05-08T22:05:40.822551+0000","last_peered":"2024-05-08T22:05:40.822551+0000","last_clean":"2024-05-08T22:05:40.822551+0000","last_became_active":"2024-05-08T22:05:14.060391+0000","last_became_peered":"2024-05-08T22:05:14.060391+0000","last_unstale":"2024-05-08T22:05:40.822551+0000","last_undegraded":"2024-05-08T22:05:40.822551+0000","last_fullsized":"2024-05-08T22:05:40.822551+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:30.528705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283045,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":49534,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":71,"num_read_kb":90,"num_write":144,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":694,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":385024,"data_stored":161288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579921,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36124,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709028,"statfs":{"total":95995035648,"available":95958044672,"internally_reserved":0,"allocated":929792,"data_stored":470051,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36043205},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.501}]},{"osd":1,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.70099999999999996}]},{"osd":2,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.40600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":4,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.64600000000000002}]},{"osd":5,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.436},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.58999999999999997}]},{"osd":6,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.67700000000000005}]},{"osd":7,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.748},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.73899999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":9,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.78200000000000003}]},{"osd":10,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.75800000000000001}]}]},{"osd":10,"up_from":67,"seq":287762808852,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":1052672,"data_stored":1102567,"data_compressed":53579,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":1,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.48799999999999999}]},{"osd":2,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.57499999999999996}]},{"osd":3,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.71699999999999997}]},{"osd":4,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.625},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.52800000000000002}]},{"osd":5,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.54400000000000004}]},{"osd":6,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.83599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":8,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.751}]},{"osd":9,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.84899999999999998}]},{"osd":11,"last update":"Wed May 8 22:05:39 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.77500000000000002}]}]},{"osd":9,"up_from":62,"seq":266287972376,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":474383,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.57399999999999995}]},{"osd":1,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.68600000000000005}]},{"osd":2,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.749},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.66300000000000003}]},{"osd":3,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.60099999999999998}]},{"osd":4,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.85599999999999998}]},{"osd":5,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.81699999999999995}]},{"osd":6,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.71199999999999997}]},{"osd":7,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.80600000000000005}]},{"osd":8,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.84499999999999997}]},{"osd":10,"last update":"Wed May 8 22:05:22 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.877}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]},{"osd":8,"up_from":55,"seq":236223201309,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":469876,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.36499999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.88200000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.622,"15min":0.63200000000000001},"min":{"1min":0.36799999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.79800000000000004,"5min":1.865,"15min":1.865},"last":0.79800000000000004}]},{"osd":1,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.60299999999999998,"15min":0.59399999999999997},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55700000000000005,"15min":0.55100000000000005},"min":{"1min":0.40799999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.68899999999999995}]},{"osd":2,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.42499999999999999,"15min":0.41899999999999998},"min":{"1min":0.32100000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63700000000000001,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.46800000000000003,"15min":0.46400000000000002},"min":{"1min":0.34200000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.41999999999999998}]},{"osd":3,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.77600000000000002,"15min":0.81100000000000005},"min":{"1min":0.45500000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82199999999999995,"5min":4.8920000000000003,"15min":4.8920000000000003},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63100000000000001,"15min":0.625},"min":{"1min":0.53700000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.85899999999999999}]},{"osd":4,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.62,"15min":0.60799999999999998},"min":{"1min":0.44400000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.83599999999999997}]},{"osd":5,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.50900000000000001,"15min":0.505},"min":{"1min":0.38600000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67000000000000004,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.53800000000000003}]},{"osd":6,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.879,"15min":0.90700000000000003},"min":{"1min":0.53300000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95299999999999996,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85999999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.73199999999999998}]},{"osd":7,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.66400000000000003,"15min":0.66200000000000003},"min":{"1min":0.42999999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.67200000000000004,"15min":0.66500000000000004},"min":{"1min":0.48699999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.76200000000000001}]},{"osd":9,"last update":"Wed May 8 22:05:01 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.78600000000000003}]},{"osd":10,"last update":"Wed May 8 22:05:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.81000000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]}]},{"osd":7,"up_from":48,"seq":206158430239,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":1066022,"data_compressed":53579,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.41199999999999998}]},{"osd":1,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.39500000000000002,"15min":0.38700000000000001},"min":{"1min":0.30599999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40600000000000003,"15min":0.40300000000000002},"min":{"1min":0.29199999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.373}]},{"osd":2,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54100000000000004,"15min":0.53600000000000003},"min":{"1min":0.34899999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.38400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70299999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.56999999999999995}]},{"osd":3,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.41599999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79800000000000004,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.501,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80200000000000005,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.58599999999999997}]},{"osd":4,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.33800000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71999999999999997,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.754,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.35299999999999998}]},{"osd":5,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64400000000000002,"15min":0.64500000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89900000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.625,"15min":0.626},"min":{"1min":0.44500000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.77700000000000002,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.52100000000000002}]},{"osd":6,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85099999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69699999999999995,"15min":0.69899999999999995},"min":{"1min":0.48199999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89000000000000001,"5min":1.002,"15min":1.002},"last":0.43099999999999999}]},{"osd":8,"last update":"Wed May 8 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.68700000000000006,"15min":0.69399999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.78000000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.72099999999999997,"15min":0.72699999999999998},"min":{"1min":0.55300000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.85899999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.69399999999999995}]},{"osd":9,"last update":"Wed May 8 22:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.64600000000000002}]},{"osd":10,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61499999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]}]},{"osd":6,"up_from":43,"seq":184683593763,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":465190,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.46200000000000002,"15min":0.45100000000000001},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.66300000000000003,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.51000000000000001,"15min":0.51300000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.64400000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.38100000000000001}]},{"osd":1,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57299999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.501},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.61899999999999999,"15min":0.60599999999999998},"min":{"1min":0.47399999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.53200000000000003}]},{"osd":2,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.66200000000000003,"15min":0.64800000000000002},"min":{"1min":0.53200000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65200000000000002,"15min":0.64700000000000002},"min":{"1min":0.51400000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89300000000000002,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.73099999999999998}]},{"osd":3,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60999999999999999,"15min":0.60699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.141,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.495,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.58199999999999996}]},{"osd":4,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69699999999999995,"15min":0.69299999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0569999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89700000000000002,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.66800000000000004}]},{"osd":5,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.754,"15min":0.74399999999999999},"min":{"1min":0.57999999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.995,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.75800000000000001,"15min":0.752},"min":{"1min":0.56399999999999995,"5min":0.438,"15min":0.438},"max":{"1min":1.052,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.61099999999999999}]},{"osd":7,"last update":"Wed May 8 22:05:23 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.99299999999999999,"5min":1.028,"15min":1.028},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76300000000000001,"15min":0.76000000000000001},"min":{"1min":0.56999999999999995,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.70299999999999996}]},{"osd":8,"last update":"Wed May 8 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.83199999999999996,"15min":0.83999999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0289999999999999,"5min":1.093,"15min":1.093},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.79300000000000004,"15min":0.79600000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0149999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.745}]},{"osd":9,"last update":"Wed May 8 22:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.65200000000000002}]},{"osd":10,"last update":"Wed May 8 22:05:19 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.627}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":991232,"data_stored":523995,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.52700000000000002,"15min":0.49299999999999999},"min":{"1min":0.38900000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.89900000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54000000000000004,"15min":0.51400000000000001},"min":{"1min":0.371,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81200000000000006,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.76000000000000001}]},{"osd":2,"last update":"Wed May 8 22:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.54800000000000004,"15min":0.497},"min":{"1min":0.40400000000000003,"5min":0.316,"15min":0.316},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.745},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.54100000000000004,"15min":0.496},"min":{"1min":0.42799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83599999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.53300000000000003}]},{"osd":3,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.53300000000000003,"15min":0.505},"min":{"1min":0.40000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.442},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.53300000000000003,"15min":0.51200000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.68300000000000005}]},{"osd":4,"last update":"Wed May 8 22:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.39900000000000002,"15min":0.376},"min":{"1min":0.307,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.64000000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.42499999999999999,"15min":0.39300000000000002},"min":{"1min":0.32400000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.76700000000000002,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.63}]},{"osd":5,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.59099999999999997,"15min":0.55700000000000005},"min":{"1min":0.47199999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.95499999999999996,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.60199999999999998,"15min":0.57899999999999996},"min":{"1min":0.45400000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89600000000000002,"5min":1.089,"15min":1.089},"last":0.67100000000000004}]},{"osd":6,"last update":"Wed May 8 22:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.63,"15min":0.61599999999999999},"min":{"1min":0.55100000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.65600000000000003}]},{"osd":7,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.36499999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86299999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.47899999999999998}]},{"osd":8,"last update":"Wed May 8 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.65300000000000002,"15min":0.65900000000000003},"min":{"1min":0.45800000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.91200000000000003,"5min":1.115,"15min":1.115},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.53500000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96999999999999997,"5min":1.222,"15min":1.222},"last":0.77200000000000002}]},{"osd":9,"last update":"Wed May 8 22:05:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.81699999999999995}]},{"osd":10,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.497},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.73499999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":946176,"data_stored":466578,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57899999999999996,"15min":0.54900000000000004},"min":{"1min":0.42499999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.94099999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.55400000000000005,"15min":0.52400000000000002},"min":{"1min":0.40100000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83199999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.59899999999999998}]},{"osd":2,"last update":"Wed May 8 22:04:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.56399999999999995,"15min":0.54000000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.56100000000000005,"15min":0.54100000000000004},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.70299999999999996}]},{"osd":3,"last update":"Wed May 8 22:04:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.436,"15min":0.41799999999999998},"min":{"1min":0.34799999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.624},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.48799999999999999,"15min":0.48199999999999998},"min":{"1min":0.35499999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.877,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.77300000000000002}]},{"osd":4,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.57099999999999995,"15min":0.54900000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69399999999999995,"15min":0.69599999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96999999999999997,"5min":2.984,"15min":2.984},"last":0.71599999999999997}]},{"osd":5,"last update":"Wed May 8 22:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.67800000000000005,"15min":0.64600000000000002},"min":{"1min":0.42399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0109999999999999,"5min":1.129,"15min":1.129},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67300000000000004,"15min":0.66300000000000003},"min":{"1min":0.49199999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.94699999999999995,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.79200000000000004}]},{"osd":6,"last update":"Wed May 8 22:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.68600000000000005,"15min":0.66900000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.81599999999999995}]},{"osd":7,"last update":"Wed May 8 22:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.755}]},{"osd":8,"last update":"Wed May 8 22:05:39 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.82199999999999995,"15min":0.82599999999999996},"min":{"1min":0.61399999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.099,"5min":1.427,"15min":1.427},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.79500000000000004,"15min":0.80300000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0369999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.82899999999999996}]},{"osd":9,"last update":"Wed May 8 22:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.008}]},{"osd":10,"last update":"Wed May 8 22:05:22 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.98499999999999999}]},{"osd":11,"last update":"Wed May 8 22:05:39 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":1.0209999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411379,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":978944,"data_stored":511059,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.41999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.51200000000000001,"15min":0.504},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76100000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.68100000000000005}]},{"osd":1,"last update":"Wed May 8 22:04:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52600000000000002,"15min":0.52500000000000002},"min":{"1min":0.34100000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.74199999999999999,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.35499999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.77500000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.91500000000000004}]},{"osd":3,"last update":"Wed May 8 22:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.372,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65900000000000003,"5min":0.872,"15min":0.872},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56599999999999995,"15min":0.55900000000000005},"min":{"1min":0.40100000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72199999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.56299999999999994}]},{"osd":4,"last update":"Wed May 8 22:04:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.35699999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73399999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55700000000000005,"15min":0.55200000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.81899999999999995}]},{"osd":5,"last update":"Wed May 8 22:04:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.436,"15min":0.433},"min":{"1min":0.34200000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.59999999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44400000000000001,"15min":0.441},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.53300000000000003}]},{"osd":6,"last update":"Wed May 8 22:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.76400000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66600000000000004,"15min":0.66700000000000004},"min":{"1min":0.50800000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.81499999999999995,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.78800000000000003}]},{"osd":7,"last update":"Wed May 8 22:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.94799999999999995}]},{"osd":8,"last update":"Wed May 8 22:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.36699999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.88},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.58799999999999997,"15min":0.57099999999999995},"min":{"1min":0.41299999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.80300000000000005}]},{"osd":9,"last update":"Wed May 8 22:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.93100000000000005}]},{"osd":10,"last update":"Wed May 8 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.90300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]}]},{"osd":3,"up_from":24,"seq":103079215150,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":465657,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.41699999999999998,"15min":0.39200000000000002},"min":{"1min":0.33500000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.42399999999999999,"15min":0.40200000000000002},"min":{"1min":0.35699999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.379}]},{"osd":1,"last update":"Wed May 8 22:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.59899999999999998,"15min":0.55700000000000005},"min":{"1min":0.48599999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.56399999999999995,"15min":0.53000000000000003},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.71699999999999997}]},{"osd":2,"last update":"Wed May 8 22:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.61499999999999999,"15min":0.58899999999999997},"min":{"1min":0.51800000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.63900000000000001,"15min":0.60899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.40699999999999997}]},{"osd":4,"last update":"Wed May 8 22:05:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.62,"15min":0.59399999999999997},"min":{"1min":0.42399999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.66300000000000003,"15min":0.62},"min":{"1min":0.61299999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.76200000000000001}]},{"osd":5,"last update":"Wed May 8 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.67800000000000005,"15min":0.64500000000000002},"min":{"1min":0.46500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.64400000000000002,"15min":0.60899999999999999},"min":{"1min":0.46500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.53400000000000003}]},{"osd":6,"last update":"Wed May 8 22:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.63100000000000001,"15min":0.61499999999999999},"min":{"1min":0.46400000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.64600000000000002,"15min":0.624},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.623}]},{"osd":7,"last update":"Wed May 8 22:05:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.65700000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.73399999999999999}]},{"osd":8,"last update":"Wed May 8 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.53100000000000003}]},{"osd":9,"last update":"Wed May 8 22:05:01 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.67800000000000005}]},{"osd":10,"last update":"Wed May 8 22:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.746},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.60899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]}]},{"osd":4,"up_from":31,"seq":133143986220,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":470177,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.44700000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.51900000000000002,"15min":0.50900000000000001},"min":{"1min":0.45000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.629}]},{"osd":1,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.374,"15min":0.35099999999999998},"min":{"1min":0.27200000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.66800000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.377,"15min":0.35499999999999998},"min":{"1min":0.28199999999999997,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.51000000000000001}]},{"osd":2,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.52600000000000002,"15min":0.50700000000000001},"min":{"1min":0.35499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71899999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.80900000000000005}]},{"osd":3,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.58799999999999997,"15min":0.56399999999999995},"min":{"1min":0.5,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84799999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56699999999999995,"15min":0.55300000000000005},"min":{"1min":0.39600000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.80000000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.67300000000000004}]},{"osd":5,"last update":"Wed May 8 22:05:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.36399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76200000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.57199999999999995,"15min":0.55000000000000004},"min":{"1min":0.42799999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.82199999999999995}]},{"osd":6,"last update":"Wed May 8 22:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.64800000000000002,"15min":0.63600000000000001},"min":{"1min":0.371,"5min":0.316,"15min":0.316},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.748},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.41799999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80500000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.60499999999999998}]},{"osd":7,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76800000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57299999999999995}]},{"osd":8,"last update":"Wed May 8 22:05:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68400000000000005,"15min":0.68600000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70799999999999996,"15min":0.70899999999999996},"min":{"1min":0.54100000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.79300000000000004}]},{"osd":9,"last update":"Wed May 8 22:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.624}]},{"osd":10,"last update":"Wed May 8 22:05:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.83499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":5,"up_from":36,"seq":154618822696,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":1003520,"data_stored":1061658,"data_compressed":53579,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.41199999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.71899999999999997,"5min":0.749,"15min":0.749},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.35199999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80100000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.53200000000000003}]},{"osd":1,"last update":"Wed May 8 22:05:43 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.32300000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52600000000000002,"15min":0.51600000000000001},"min":{"1min":0.34300000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71299999999999997,"5min":0.754,"15min":0.754},"last":0.59599999999999997}]},{"osd":2,"last update":"Wed May 8 22:05:43 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.377,"15min":0.36699999999999999},"min":{"1min":0.29799999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.39200000000000002,"15min":0.38400000000000001},"min":{"1min":0.309,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.40999999999999998}]},{"osd":3,"last update":"Wed May 8 22:05:43 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.57299999999999995},"min":{"1min":0.42099999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.76700000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.435},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56299999999999994,"15min":0.55100000000000005},"min":{"1min":0.47299999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.78700000000000003,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.68600000000000005}]},{"osd":4,"last update":"Wed May 8 22:05:43 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54700000000000004,"15min":0.53700000000000003},"min":{"1min":0.42999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82999999999999996,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.33600000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.82699999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.67200000000000004}]},{"osd":6,"last update":"Wed May 8 22:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58599999999999997,"15min":0.58399999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.65400000000000003}]},{"osd":7,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.57399999999999995,"15min":0.56499999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.58799999999999997,"15min":0.57699999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.51100000000000001}]},{"osd":8,"last update":"Wed May 8 22:05:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.50600000000000001,"15min":0.49399999999999999},"min":{"1min":0.372,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.62},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.502,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.73699999999999999}]},{"osd":9,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.70799999999999996}]},{"osd":10,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.76400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":58895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":46628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":47157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:05:48.809 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T22:05:48.810 INFO:tasks.cephadm:Setting up client nodes... 2024-05-08T22:05:48.810 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-08T22:05:49.203 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:05:50.051 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:49 smithi131 ceph-mon[37260]: pgmap v259: 129 pgs: 129 active+clean; 625 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 KiB/s wr, 15 op/s; 86 B/s, 1 keys/s, 1 objects/s recovering 2024-05-08T22:05:50.051 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:49 smithi131 ceph-mon[37260]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:05:50.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:49 smithi157 ceph-mon[77746]: pgmap v259: 129 pgs: 129 active+clean; 625 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 KiB/s wr, 15 op/s; 86 B/s, 1 keys/s, 1 objects/s recovering 2024-05-08T22:05:50.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:49 smithi157 ceph-mon[77746]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:05:50.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:49 smithi132 ceph-mon[78180]: pgmap v259: 129 pgs: 129 active+clean; 625 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 KiB/s wr, 15 op/s; 86 B/s, 1 keys/s, 1 objects/s recovering 2024-05-08T22:05:50.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:49 smithi132 ceph-mon[78180]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:05:50.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.603+0000 7fc07fe00700 1 -- 172.21.15.131:0/3493456667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0781021d0 msgr2=0x7fc078102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:50.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.603+0000 7fc07fe00700 1 --2- 172.21.15.131:0/3493456667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0781021d0 0x7fc078102630 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc0640099c0 tx=0x7fc064009cd0 comp rx=0 tx=0).stop 2024-05-08T22:05:50.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.603+0000 7fc07fe00700 1 -- 172.21.15.131:0/3493456667 shutdown_connections 2024-05-08T22:05:50.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.603+0000 7fc07fe00700 1 --2- 172.21.15.131:0/3493456667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc078102b70 0x7fc078109420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:50.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.603+0000 7fc07fe00700 1 --2- 172.21.15.131:0/3493456667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc078100fd0 0x7fc0781013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:50.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.603+0000 7fc07fe00700 1 --2- 172.21.15.131:0/3493456667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0781021d0 0x7fc078102630 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:50.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.603+0000 7fc07fe00700 1 -- 172.21.15.131:0/3493456667 >> 172.21.15.131:0/3493456667 conn(0x7fc0780fc570 msgr2=0x7fc0780fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:50.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.603+0000 7fc07fe00700 1 -- 172.21.15.131:0/3493456667 shutdown_connections 2024-05-08T22:05:50.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.603+0000 7fc07fe00700 1 -- 172.21.15.131:0/3493456667 wait complete. 2024-05-08T22:05:50.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.604+0000 7fc07fe00700 1 Processor -- start 2024-05-08T22:05:50.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.604+0000 7fc07fe00700 1 -- start start 2024-05-08T22:05:50.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.605+0000 7fc07fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc078100fd0 0x7fc078074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.605+0000 7fc07fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0781021d0 0x7fc078073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.605+0000 7fc07fe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc078102b70 0x7fc0780738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.605+0000 7fc07fe00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc078074070 con 0x7fc078100fd0 2024-05-08T22:05:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.605+0000 7fc07fe00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0780741e0 con 0x7fc078102b70 2024-05-08T22:05:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.605+0000 7fc07fe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0781a0fa0 con 0x7fc0781021d0 2024-05-08T22:05:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.605+0000 7fc07d400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc078100fd0 0x7fc078074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.605+0000 7fc07ca00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0781021d0 0x7fc078073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.605+0000 7fc07ca00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0781021d0 0x7fc078073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56820/0 (socket says 172.21.15.131:56820) 2024-05-08T22:05:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.605+0000 7fc07ca00700 1 -- 172.21.15.131:0/2429369564 learned_addr learned my addr 172.21.15.131:0/2429369564 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.605+0000 7fc07de00700 1 --2- 172.21.15.131:0/2429369564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc078102b70 0x7fc0780738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.605+0000 7fc07d400700 1 -- 172.21.15.131:0/2429369564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0781021d0 msgr2=0x7fc078073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.605+0000 7fc07d400700 1 --2- 172.21.15.131:0/2429369564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0781021d0 0x7fc078073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.605+0000 7fc07d400700 1 -- 172.21.15.131:0/2429369564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc078102b70 msgr2=0x7fc0780738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.606+0000 7fc07d400700 1 --2- 172.21.15.131:0/2429369564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc078102b70 0x7fc0780738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.606+0000 7fc07d400700 1 -- 172.21.15.131:0/2429369564 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0640095f0 con 0x7fc078100fd0 2024-05-08T22:05:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.606+0000 7fc07ca00700 1 --2- 172.21.15.131:0/2429369564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0781021d0 0x7fc078073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.606+0000 7fc07d400700 1 --2- 172.21.15.131:0/2429369564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc078100fd0 0x7fc078074cf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc078102030 tx=0x7fc06c00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.606+0000 7fc076000700 1 -- 172.21.15.131:0/2429369564 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c011830 con 0x7fc078100fd0 2024-05-08T22:05:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.606+0000 7fc07fe00700 1 -- 172.21.15.131:0/2429369564 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0781a1230 con 0x7fc078100fd0 2024-05-08T22:05:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.606+0000 7fc076000700 1 -- 172.21.15.131:0/2429369564 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc06c011e50 con 0x7fc078100fd0 2024-05-08T22:05:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.606+0000 7fc076000700 1 -- 172.21.15.131:0/2429369564 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c010d70 con 0x7fc078100fd0 2024-05-08T22:05:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.606+0000 7fc07fe00700 1 -- 172.21.15.131:0/2429369564 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0781a1780 con 0x7fc078100fd0 2024-05-08T22:05:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.607+0000 7fc07fe00700 1 -- 172.21.15.131:0/2429369564 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc078108400 con 0x7fc078100fd0 2024-05-08T22:05:50.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.609+0000 7fc076000700 1 -- 172.21.15.131:0/2429369564 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fc06c010410 con 0x7fc078100fd0 2024-05-08T22:05:50.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.610+0000 7fc076000700 1 --2- 172.21.15.131:0/2429369564 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc0600a11b0 0x7fc0600a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:50.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.611+0000 7fc076000700 1 -- 172.21.15.131:0/2429369564 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc06c0cc630 con 0x7fc078100fd0 2024-05-08T22:05:50.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.611+0000 7fc07ca00700 1 --2- 172.21.15.131:0/2429369564 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc0600a11b0 0x7fc0600a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:50.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.611+0000 7fc07ca00700 1 --2- 172.21.15.131:0/2429369564 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc0600a11b0 0x7fc0600a3670 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc064003820 tx=0x7fc064005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:50.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.615+0000 7fc076000700 1 -- 172.21.15.131:0/2429369564 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc06c095a00 con 0x7fc078100fd0 2024-05-08T22:05:50.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.901+0000 7fc07fe00700 1 -- 172.21.15.131:0/2429369564 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fc07810c8c0 con 0x7fc078100fd0 2024-05-08T22:05:50.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.906+0000 7fc076000700 1 -- 172.21.15.131:0/2429369564 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7fc06c010ed0 con 0x7fc078100fd0 2024-05-08T22:05:50.907 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2024-05-08T22:05:50.907 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQA+9ztmWVnSNRAAFU4w2R/VFppcx0tBq+PsZg== 2024-05-08T22:05:50.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.908+0000 7fc07fe00700 1 -- 172.21.15.131:0/2429369564 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc0600a11b0 msgr2=0x7fc0600a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:50.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.908+0000 7fc07fe00700 1 --2- 172.21.15.131:0/2429369564 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc0600a11b0 0x7fc0600a3670 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc064003820 tx=0x7fc064005fb0 comp rx=0 tx=0).stop 2024-05-08T22:05:50.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.908+0000 7fc07fe00700 1 -- 172.21.15.131:0/2429369564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc078100fd0 msgr2=0x7fc078074cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:50.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.908+0000 7fc07fe00700 1 --2- 172.21.15.131:0/2429369564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc078100fd0 0x7fc078074cf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc078102030 tx=0x7fc06c00b730 comp rx=0 tx=0).stop 2024-05-08T22:05:50.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.908+0000 7fc07fe00700 1 -- 172.21.15.131:0/2429369564 shutdown_connections 2024-05-08T22:05:50.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.908+0000 7fc07fe00700 1 --2- 172.21.15.131:0/2429369564 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc0600a11b0 0x7fc0600a3670 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:50.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.908+0000 7fc07fe00700 1 --2- 172.21.15.131:0/2429369564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc078102b70 0x7fc0780738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:50.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.908+0000 7fc07fe00700 1 --2- 172.21.15.131:0/2429369564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc078100fd0 0x7fc078074cf0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:50.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.908+0000 7fc07fe00700 1 --2- 172.21.15.131:0/2429369564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0781021d0 0x7fc078073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:50.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.908+0000 7fc07fe00700 1 -- 172.21.15.131:0/2429369564 >> 172.21.15.131:0/2429369564 conn(0x7fc0780fc570 msgr2=0x7fc078105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:50.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.908+0000 7fc07fe00700 1 -- 172.21.15.131:0/2429369564 shutdown_connections 2024-05-08T22:05:50.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:50.908+0000 7fc07fe00700 1 -- 172.21.15.131:0/2429369564 wait complete. 2024-05-08T22:05:51.378 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-08T22:05:51.378 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-08T22:05:51.378 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-08T22:05:51.475 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-08T22:05:51.475 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-08T22:05:51.475 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-08T22:05:51.877 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:05:52.116 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:51 smithi131 ceph-mon[37260]: pgmap v260: 129 pgs: 129 active+clean; 625 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.0 KiB/s wr, 13 op/s; 70 B/s, 1 keys/s, 1 objects/s recovering 2024-05-08T22:05:52.116 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:51 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2429369564' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T22:05:52.116 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:51 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2429369564' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T22:05:52.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:51 smithi157 ceph-mon[77746]: pgmap v260: 129 pgs: 129 active+clean; 625 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.0 KiB/s wr, 13 op/s; 70 B/s, 1 keys/s, 1 objects/s recovering 2024-05-08T22:05:52.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:51 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2429369564' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T22:05:52.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:51 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2429369564' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T22:05:52.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:51 smithi132 ceph-mon[78180]: pgmap v260: 129 pgs: 129 active+clean; 625 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.0 KiB/s wr, 13 op/s; 70 B/s, 1 keys/s, 1 objects/s recovering 2024-05-08T22:05:52.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:51 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2429369564' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T22:05:52.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:51 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2429369564' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T22:05:53.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.105+0000 7ff3eea00700 1 -- 172.21.15.131:0/4268310066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3e8107030 msgr2=0x7ff3e8107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:53.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.105+0000 7ff3eea00700 1 --2- 172.21.15.131:0/4268310066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3e8107030 0x7ff3e8107410 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8009990 tx=0x7ff3d8009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:53.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.106+0000 7ff3eea00700 1 -- 172.21.15.131:0/4268310066 shutdown_connections 2024-05-08T22:05:53.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.106+0000 7ff3eea00700 1 --2- 172.21.15.131:0/4268310066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3e8101990 0x7ff3e810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:53.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.106+0000 7ff3eea00700 1 --2- 172.21.15.131:0/4268310066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3e8107030 0x7ff3e8107410 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:53.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.106+0000 7ff3eea00700 1 --2- 172.21.15.131:0/4268310066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3e8100ff0 0x7ff3e8101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:53.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.106+0000 7ff3eea00700 1 -- 172.21.15.131:0/4268310066 >> 172.21.15.131:0/4268310066 conn(0x7ff3e8078190 msgr2=0x7ff3e80785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:53.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.106+0000 7ff3eea00700 1 -- 172.21.15.131:0/4268310066 shutdown_connections 2024-05-08T22:05:53.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.106+0000 7ff3eea00700 1 -- 172.21.15.131:0/4268310066 wait complete. 2024-05-08T22:05:53.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.107+0000 7ff3eea00700 1 Processor -- start 2024-05-08T22:05:53.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.107+0000 7ff3eea00700 1 -- start start 2024-05-08T22:05:53.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.107+0000 7ff3eea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3e8100ff0 0x7ff3e81995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:53.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.107+0000 7ff3eea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3e8101990 0x7ff3e8199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:53.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.107+0000 7ff3eea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3e8107030 0x7ff3e819deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:53.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.107+0000 7ff3eea00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e819e4c0 con 0x7ff3e8100ff0 2024-05-08T22:05:53.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.107+0000 7ff3eea00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e819e630 con 0x7ff3e8101990 2024-05-08T22:05:53.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.107+0000 7ff3eea00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e819e7f0 con 0x7ff3e8107030 2024-05-08T22:05:53.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.108+0000 7ff3e7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3e8100ff0 0x7ff3e81995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:53.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.108+0000 7ff3eca00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3e8107030 0x7ff3e819deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:53.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.108+0000 7ff3e7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3e8100ff0 0x7ff3e81995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38186/0 (socket says 172.21.15.131:38186) 2024-05-08T22:05:53.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.108+0000 7ff3e7e00700 1 -- 172.21.15.131:0/3491405231 learned_addr learned my addr 172.21.15.131:0/3491405231 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:53.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.108+0000 7ff3e7400700 1 --2- 172.21.15.131:0/3491405231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3e8101990 0x7ff3e8199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:53.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.108+0000 7ff3eca00700 1 -- 172.21.15.131:0/3491405231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3e8101990 msgr2=0x7ff3e8199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:53.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.108+0000 7ff3eca00700 1 --2- 172.21.15.131:0/3491405231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3e8101990 0x7ff3e8199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:53.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.108+0000 7ff3eca00700 1 -- 172.21.15.131:0/3491405231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3e8100ff0 msgr2=0x7ff3e81995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:53.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.108+0000 7ff3eca00700 1 --2- 172.21.15.131:0/3491405231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3e8100ff0 0x7ff3e81995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:53.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.108+0000 7ff3eca00700 1 -- 172.21.15.131:0/3491405231 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d4009520 con 0x7ff3e8107030 2024-05-08T22:05:53.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.108+0000 7ff3e7e00700 1 --2- 172.21.15.131:0/3491405231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3e8100ff0 0x7ff3e81995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:53.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.108+0000 7ff3eca00700 1 --2- 172.21.15.131:0/3491405231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3e8107030 0x7ff3e819deb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0007f10 tx=0x7ff3e000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:53.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.109+0000 7ff3e4c00700 1 -- 172.21.15.131:0/3491405231 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0004db0 con 0x7ff3e8107030 2024-05-08T22:05:53.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.109+0000 7ff3eea00700 1 -- 172.21.15.131:0/3491405231 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d80095f0 con 0x7ff3e8107030 2024-05-08T22:05:53.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.109+0000 7ff3e4c00700 1 -- 172.21.15.131:0/3491405231 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3e000f040 con 0x7ff3e8107030 2024-05-08T22:05:53.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.109+0000 7ff3e4c00700 1 -- 172.21.15.131:0/3491405231 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0005830 con 0x7ff3e8107030 2024-05-08T22:05:53.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.109+0000 7ff3eea00700 1 -- 172.21.15.131:0/3491405231 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e81a6160 con 0x7ff3e8107030 2024-05-08T22:05:53.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.110+0000 7ff3eea00700 1 -- 172.21.15.131:0/3491405231 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e8104370 con 0x7ff3e8107030 2024-05-08T22:05:53.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.112+0000 7ff3e4c00700 1 -- 172.21.15.131:0/3491405231 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7ff3e000db20 con 0x7ff3e8107030 2024-05-08T22:05:53.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.113+0000 7ff3e4c00700 1 --2- 172.21.15.131:0/3491405231 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff3cc0a11b0 0x7ff3cc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:53.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.113+0000 7ff3e4c00700 1 -- 172.21.15.131:0/3491405231 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff3e00cc4f0 con 0x7ff3e8107030 2024-05-08T22:05:53.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.113+0000 7ff3e7e00700 1 --2- 172.21.15.131:0/3491405231 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff3cc0a11b0 0x7ff3cc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:53.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.114+0000 7ff3e7e00700 1 --2- 172.21.15.131:0/3491405231 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff3cc0a11b0 0x7ff3cc0a3670 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8006010 tx=0x7ff3d8005bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:53.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.117+0000 7ff3e4c00700 1 -- 172.21.15.131:0/3491405231 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff3e00957b0 con 0x7ff3e8107030 2024-05-08T22:05:53.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.378+0000 7ff3eea00700 1 -- 172.21.15.131:0/3491405231 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7ff3e819a8d0 con 0x7ff3e8107030 2024-05-08T22:05:53.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.383+0000 7ff3e4c00700 1 -- 172.21.15.131:0/3491405231 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260292 (secure 0 0 0) 0x7ff3e0098dd0 con 0x7ff3e8107030 2024-05-08T22:05:53.388 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:53.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.387+0000 7ff3eea00700 1 -- 172.21.15.131:0/3491405231 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff3cc0a11b0 msgr2=0x7ff3cc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:53.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.387+0000 7ff3eea00700 1 --2- 172.21.15.131:0/3491405231 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff3cc0a11b0 0x7ff3cc0a3670 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8006010 tx=0x7ff3d8005bc0 comp rx=0 tx=0).stop 2024-05-08T22:05:53.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.387+0000 7ff3eea00700 1 -- 172.21.15.131:0/3491405231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3e8107030 msgr2=0x7ff3e819deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:53.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.387+0000 7ff3eea00700 1 --2- 172.21.15.131:0/3491405231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3e8107030 0x7ff3e819deb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0007f10 tx=0x7ff3e000d3b0 comp rx=0 tx=0).stop 2024-05-08T22:05:53.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.387+0000 7ff3eea00700 1 -- 172.21.15.131:0/3491405231 shutdown_connections 2024-05-08T22:05:53.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.387+0000 7ff3eea00700 1 --2- 172.21.15.131:0/3491405231 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff3cc0a11b0 0x7ff3cc0a3670 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:53.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.387+0000 7ff3eea00700 1 --2- 172.21.15.131:0/3491405231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3e8107030 0x7ff3e819deb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:53.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.387+0000 7ff3eea00700 1 --2- 172.21.15.131:0/3491405231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3e8100ff0 0x7ff3e81995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:53.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.387+0000 7ff3eea00700 1 --2- 172.21.15.131:0/3491405231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3e8101990 0x7ff3e8199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:53.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.387+0000 7ff3eea00700 1 -- 172.21.15.131:0/3491405231 >> 172.21.15.131:0/3491405231 conn(0x7ff3e8078190 msgr2=0x7ff3e810c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:53.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.388+0000 7ff3eea00700 1 -- 172.21.15.131:0/3491405231 shutdown_connections 2024-05-08T22:05:53.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:53.388+0000 7ff3eea00700 1 -- 172.21.15.131:0/3491405231 wait complete. 2024-05-08T22:05:53.885 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2485665042},{"type":"v1","addr":"172.21.15.131:6801","nonce":2485665042}]},"active_addr":"172.21.15.131:6801/2485665042","active_change":"2024-05-08T21:59:40.641514+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.131:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":1773942606}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":3120268590}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":126236497}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":2704707055}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":25489954}]}]} 2024-05-08T22:05:53.890 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-08T22:05:53.890 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-08T22:05:53.890 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T22:05:54.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:53 smithi131 ceph-mon[37260]: pgmap v261: 129 pgs: 129 active+clean; 625 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.3 KiB/s wr, 9 op/s; 63 B/s, 0 keys/s, 1 objects/s recovering 2024-05-08T22:05:54.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:53 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3491405231' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T22:05:54.258 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:05:54.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:53 smithi157 ceph-mon[77746]: pgmap v261: 129 pgs: 129 active+clean; 625 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.3 KiB/s wr, 9 op/s; 63 B/s, 0 keys/s, 1 objects/s recovering 2024-05-08T22:05:54.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:53 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3491405231' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T22:05:54.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:53 smithi132 ceph-mon[78180]: pgmap v261: 129 pgs: 129 active+clean; 625 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.3 KiB/s wr, 9 op/s; 63 B/s, 0 keys/s, 1 objects/s recovering 2024-05-08T22:05:54.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:53 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3491405231' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T22:05:55.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.623+0000 7f42d8400700 1 -- 172.21.15.131:0/202762638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42d0101990 msgr2=0x7f42d010de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:55.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.623+0000 7f42d8400700 1 --2- 172.21.15.131:0/202762638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42d0101990 0x7f42d010de70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f42c800b0a0 tx=0x7f42c800b3b0 comp rx=0 tx=0).stop 2024-05-08T22:05:55.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.623+0000 7f42d8400700 1 -- 172.21.15.131:0/202762638 shutdown_connections 2024-05-08T22:05:55.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.623+0000 7f42d8400700 1 --2- 172.21.15.131:0/202762638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42d0101990 0x7f42d010de70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:55.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.623+0000 7f42d8400700 1 --2- 172.21.15.131:0/202762638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42d0107030 0x7f42d0107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:55.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.623+0000 7f42d8400700 1 --2- 172.21.15.131:0/202762638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42d0100ff0 0x7f42d0101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:55.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.623+0000 7f42d8400700 1 -- 172.21.15.131:0/202762638 >> 172.21.15.131:0/202762638 conn(0x7f42d0078190 msgr2=0x7f42d00785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:55.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.624+0000 7f42d8400700 1 -- 172.21.15.131:0/202762638 shutdown_connections 2024-05-08T22:05:55.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.624+0000 7f42d8400700 1 -- 172.21.15.131:0/202762638 wait complete. 2024-05-08T22:05:55.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.625+0000 7f42d8400700 1 Processor -- start 2024-05-08T22:05:55.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.625+0000 7f42d8400700 1 -- start start 2024-05-08T22:05:55.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.625+0000 7f42d8400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42d0100ff0 0x7f42d01997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:55.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.625+0000 7f42d8400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42d0101990 0x7f42d0199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:55.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.625+0000 7f42d8400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42d0107030 0x7f42d019cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:55.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.625+0000 7f42d8400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d019d5e0 con 0x7f42d0100ff0 2024-05-08T22:05:55.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.625+0000 7f42d8400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d019d750 con 0x7f42d0101990 2024-05-08T22:05:55.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.625+0000 7f42d8400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d019d910 con 0x7f42d0107030 2024-05-08T22:05:55.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.625+0000 7f42d5a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42d0100ff0 0x7f42d01997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:55.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.625+0000 7f42d5000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42d0101990 0x7f42d0199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:55.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.626+0000 7f42d6400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42d0107030 0x7f42d019cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:55.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.626+0000 7f42d5a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42d0100ff0 0x7f42d01997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37840/0 (socket says 172.21.15.131:37840) 2024-05-08T22:05:55.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.626+0000 7f42d5a00700 1 -- 172.21.15.131:0/1319965844 learned_addr learned my addr 172.21.15.131:0/1319965844 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:55.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.626+0000 7f42d5000700 1 -- 172.21.15.131:0/1319965844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42d0107030 msgr2=0x7f42d019cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:55.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.626+0000 7f42d5000700 1 --2- 172.21.15.131:0/1319965844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42d0107030 0x7f42d019cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:55.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.626+0000 7f42d5000700 1 -- 172.21.15.131:0/1319965844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42d0100ff0 msgr2=0x7f42d01997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:55.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.626+0000 7f42d5000700 1 --2- 172.21.15.131:0/1319965844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42d0100ff0 0x7f42d01997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:55.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.626+0000 7f42d5000700 1 -- 172.21.15.131:0/1319965844 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c8009d00 con 0x7f42d0101990 2024-05-08T22:05:55.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.626+0000 7f42d5a00700 1 --2- 172.21.15.131:0/1319965844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42d0100ff0 0x7f42d01997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:55.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.626+0000 7f42d6400700 1 --2- 172.21.15.131:0/1319965844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42d0107030 0x7f42d019cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:55.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.626+0000 7f42d5000700 1 --2- 172.21.15.131:0/1319965844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42d0101990 0x7f42d0199ce0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00d750 tx=0x7f42bc00da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:55.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.626+0000 7f42ce000700 1 -- 172.21.15.131:0/1319965844 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc004020 con 0x7f42d0101990 2024-05-08T22:05:55.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.627+0000 7f42d8400700 1 -- 172.21.15.131:0/1319965844 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d019dbf0 con 0x7f42d0101990 2024-05-08T22:05:55.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.627+0000 7f42ce000700 1 -- 172.21.15.131:0/1319965844 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42bc004830 con 0x7f42d0101990 2024-05-08T22:05:55.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.627+0000 7f42ce000700 1 -- 172.21.15.131:0/1319965844 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc005550 con 0x7f42d0101990 2024-05-08T22:05:55.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.627+0000 7f42d8400700 1 -- 172.21.15.131:0/1319965844 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d01a6540 con 0x7f42d0101990 2024-05-08T22:05:55.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.628+0000 7f42d8400700 1 -- 172.21.15.131:0/1319965844 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d0104370 con 0x7f42d0101990 2024-05-08T22:05:55.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.630+0000 7f42ce000700 1 -- 172.21.15.131:0/1319965844 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f42bc004ca0 con 0x7f42d0101990 2024-05-08T22:05:55.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.631+0000 7f42ce000700 1 --2- 172.21.15.131:0/1319965844 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f42b80a11b0 0x7f42b80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:55.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.631+0000 7f42ce000700 1 -- 172.21.15.131:0/1319965844 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f42bc0ccab0 con 0x7f42d0101990 2024-05-08T22:05:55.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.631+0000 7f42d5a00700 1 --2- 172.21.15.131:0/1319965844 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f42b80a11b0 0x7f42b80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:55.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.632+0000 7f42d5a00700 1 --2- 172.21.15.131:0/1319965844 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f42b80a11b0 0x7f42b80a3670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f42c40103d0 tx=0x7f42c4007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:55.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.633+0000 7f42ce000700 1 -- 172.21.15.131:0/1319965844 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f42bc095d70 con 0x7f42d0101990 2024-05-08T22:05:55.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.837+0000 7f42d8400700 1 -- 172.21.15.131:0/1319965844 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42d01026e0 con 0x7f42d0101990 2024-05-08T22:05:55.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.839+0000 7f42ce000700 1 -- 172.21.15.131:0/1319965844 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+24534 (secure 0 0 0) 0x7f42bc014090 con 0x7f42d0101990 2024-05-08T22:05:55.841 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:55.841 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":85,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:05:41.813233+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:05:12.469858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:05:13.484446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:05:17.016186+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:05:55.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.842+0000 7f42d8400700 1 -- 172.21.15.131:0/1319965844 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f42b80a11b0 msgr2=0x7f42b80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:55.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.842+0000 7f42d8400700 1 --2- 172.21.15.131:0/1319965844 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f42b80a11b0 0x7f42b80a3670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f42c40103d0 tx=0x7f42c4007480 comp rx=0 tx=0).stop 2024-05-08T22:05:55.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.842+0000 7f42d8400700 1 -- 172.21.15.131:0/1319965844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42d0101990 msgr2=0x7f42d0199ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:55.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.842+0000 7f42d8400700 1 --2- 172.21.15.131:0/1319965844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42d0101990 0x7f42d0199ce0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00d750 tx=0x7f42bc00da60 comp rx=0 tx=0).stop 2024-05-08T22:05:55.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.842+0000 7f42d8400700 1 -- 172.21.15.131:0/1319965844 shutdown_connections 2024-05-08T22:05:55.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.842+0000 7f42d8400700 1 --2- 172.21.15.131:0/1319965844 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f42b80a11b0 0x7f42b80a3670 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:55.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.842+0000 7f42d8400700 1 --2- 172.21.15.131:0/1319965844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42d0107030 0x7f42d019cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:55.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.842+0000 7f42d8400700 1 --2- 172.21.15.131:0/1319965844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42d0100ff0 0x7f42d01997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:55.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.842+0000 7f42d8400700 1 --2- 172.21.15.131:0/1319965844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42d0101990 0x7f42d0199ce0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:55.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.842+0000 7f42d8400700 1 -- 172.21.15.131:0/1319965844 >> 172.21.15.131:0/1319965844 conn(0x7f42d0078190 msgr2=0x7f42d010c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:55.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.842+0000 7f42d8400700 1 -- 172.21.15.131:0/1319965844 shutdown_connections 2024-05-08T22:05:55.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:55.842+0000 7f42d8400700 1 -- 172.21.15.131:0/1319965844 wait complete. 2024-05-08T22:05:56.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:55 smithi131 ceph-mon[37260]: pgmap v262: 129 pgs: 129 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.0 KiB/s wr, 9 op/s; 48 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T22:05:56.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:55 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1319965844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:56.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:55 smithi157 ceph-mon[77746]: pgmap v262: 129 pgs: 129 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.0 KiB/s wr, 9 op/s; 48 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T22:05:56.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:55 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1319965844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:56.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:55 smithi132 ceph-mon[78180]: pgmap v262: 129 pgs: 129 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.0 KiB/s wr, 9 op/s; 48 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T22:05:56.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:55 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1319965844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:56.361 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-08T22:05:56.361 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T22:05:56.726 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:05:58.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.006+0000 7f4932c00700 1 -- 172.21.15.131:0/3258346478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f492c073710 msgr2=0x7f492c073b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:58.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.006+0000 7f4932c00700 1 --2- 172.21.15.131:0/3258346478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f492c073710 0x7f492c073b90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4918009990 tx=0x7f4918009ca0 comp rx=0 tx=0).stop 2024-05-08T22:05:58.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.006+0000 7f4932c00700 1 -- 172.21.15.131:0/3258346478 shutdown_connections 2024-05-08T22:05:58.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.006+0000 7f4932c00700 1 --2- 172.21.15.131:0/3258346478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f492c111c30 0x7f492c114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:58.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.006+0000 7f4932c00700 1 --2- 172.21.15.131:0/3258346478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f492c075000 0x7f492c0731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:58.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.006+0000 7f4932c00700 1 --2- 172.21.15.131:0/3258346478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f492c073710 0x7f492c073b90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:58.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.006+0000 7f4932c00700 1 -- 172.21.15.131:0/3258346478 >> 172.21.15.131:0/3258346478 conn(0x7f492c078190 msgr2=0x7f492c0785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:58.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.006+0000 7f4932c00700 1 -- 172.21.15.131:0/3258346478 shutdown_connections 2024-05-08T22:05:58.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.006+0000 7f4932c00700 1 -- 172.21.15.131:0/3258346478 wait complete. 2024-05-08T22:05:58.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.007+0000 7f4932c00700 1 Processor -- start 2024-05-08T22:05:58.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.007+0000 7f4932c00700 1 -- start start 2024-05-08T22:05:58.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.008+0000 7f4932c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f492c073710 0x7f492c1a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:58.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.008+0000 7f4932c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f492c075000 0x7f492c1a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:58.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.008+0000 7f4932c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f492c111c30 0x7f492c19d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:58.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.008+0000 7f4932c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c19df60 con 0x7f492c075000 2024-05-08T22:05:58.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.008+0000 7f4932c00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c19e0d0 con 0x7f492c111c30 2024-05-08T22:05:58.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.008+0000 7f4932c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c19e290 con 0x7f492c073710 2024-05-08T22:05:58.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.008+0000 7f492b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f492c075000 0x7f492c1a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:58.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.008+0000 7f492be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f492c073710 0x7f492c1a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:58.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.008+0000 7f492b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f492c075000 0x7f492c1a3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37854/0 (socket says 172.21.15.131:37854) 2024-05-08T22:05:58.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.008+0000 7f492b400700 1 -- 172.21.15.131:0/44202100 learned_addr learned my addr 172.21.15.131:0/44202100 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:05:58.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.008+0000 7f4930c00700 1 --2- 172.21.15.131:0/44202100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f492c111c30 0x7f492c19d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:58.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.009+0000 7f492b400700 1 -- 172.21.15.131:0/44202100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f492c073710 msgr2=0x7f492c1a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:58.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.009+0000 7f492b400700 1 --2- 172.21.15.131:0/44202100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f492c073710 0x7f492c1a37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:58.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.009+0000 7f492b400700 1 -- 172.21.15.131:0/44202100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f492c111c30 msgr2=0x7f492c19d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:58.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.009+0000 7f492b400700 1 --2- 172.21.15.131:0/44202100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f492c111c30 0x7f492c19d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:58.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.009+0000 7f492b400700 1 -- 172.21.15.131:0/44202100 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49180095f0 con 0x7f492c075000 2024-05-08T22:05:58.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.009+0000 7f4930c00700 1 --2- 172.21.15.131:0/44202100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f492c111c30 0x7f492c19d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:05:58.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.009+0000 7f492b400700 1 --2- 172.21.15.131:0/44202100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f492c075000 0x7f492c1a3d00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4918005910 tx=0x7f4918004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:58.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.009+0000 7f4928c00700 1 -- 172.21.15.131:0/44202100 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491801d070 con 0x7f492c075000 2024-05-08T22:05:58.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.009+0000 7f4928c00700 1 -- 172.21.15.131:0/44202100 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4918022470 con 0x7f492c075000 2024-05-08T22:05:58.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.009+0000 7f4932c00700 1 -- 172.21.15.131:0/44202100 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492c19e510 con 0x7f492c075000 2024-05-08T22:05:58.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.009+0000 7f4928c00700 1 -- 172.21.15.131:0/44202100 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4918017650 con 0x7f492c075000 2024-05-08T22:05:58.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.009+0000 7f4932c00700 1 -- 172.21.15.131:0/44202100 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492c1aa7b0 con 0x7f492c075000 2024-05-08T22:05:58.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.011+0000 7f4932c00700 1 -- 172.21.15.131:0/44202100 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492c0748b0 con 0x7f492c075000 2024-05-08T22:05:58.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.013+0000 7f4928c00700 1 -- 172.21.15.131:0/44202100 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f49180225e0 con 0x7f492c075000 2024-05-08T22:05:58.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.014+0000 7f4928c00700 1 --2- 172.21.15.131:0/44202100 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f49100a11b0 0x7f49100a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:05:58.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.014+0000 7f4928c00700 1 -- 172.21.15.131:0/44202100 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f49180cdf20 con 0x7f492c075000 2024-05-08T22:05:58.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.014+0000 7f492be00700 1 --2- 172.21.15.131:0/44202100 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f49100a11b0 0x7f49100a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:05:58.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.015+0000 7f492be00700 1 --2- 172.21.15.131:0/44202100 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f49100a11b0 0x7f49100a3670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f491c009c60 tx=0x7f491c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:05:58.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.018+0000 7f4928c00700 1 -- 172.21.15.131:0/44202100 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f491805dd60 con 0x7f492c075000 2024-05-08T22:05:58.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:57 smithi131 ceph-mon[37260]: pgmap v263: 129 pgs: 129 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.4 KiB/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T22:05:58.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.222+0000 7f4932c00700 1 -- 172.21.15.131:0/44202100 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f492c04ec60 con 0x7f492c075000 2024-05-08T22:05:58.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.224+0000 7f4928c00700 1 -- 172.21.15.131:0/44202100 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+24534 (secure 0 0 0) 0x7f4918097270 con 0x7f492c075000 2024-05-08T22:05:58.226 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:05:58.226 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":85,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:05:41.813233+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:05:12.469858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:05:13.484446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:05:17.016186+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:05:58.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.227+0000 7f4932c00700 1 -- 172.21.15.131:0/44202100 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f49100a11b0 msgr2=0x7f49100a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:58.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.227+0000 7f4932c00700 1 --2- 172.21.15.131:0/44202100 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f49100a11b0 0x7f49100a3670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f491c009c60 tx=0x7f491c009260 comp rx=0 tx=0).stop 2024-05-08T22:05:58.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.227+0000 7f4932c00700 1 -- 172.21.15.131:0/44202100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f492c075000 msgr2=0x7f492c1a3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:05:58.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.227+0000 7f4932c00700 1 --2- 172.21.15.131:0/44202100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f492c075000 0x7f492c1a3d00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4918005910 tx=0x7f4918004730 comp rx=0 tx=0).stop 2024-05-08T22:05:58.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.227+0000 7f4932c00700 1 -- 172.21.15.131:0/44202100 shutdown_connections 2024-05-08T22:05:58.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.227+0000 7f4932c00700 1 --2- 172.21.15.131:0/44202100 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f49100a11b0 0x7f49100a3670 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:58.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.227+0000 7f4932c00700 1 --2- 172.21.15.131:0/44202100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f492c111c30 0x7f492c19d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:58.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.227+0000 7f4932c00700 1 --2- 172.21.15.131:0/44202100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f492c073710 0x7f492c1a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:58.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.227+0000 7f4932c00700 1 --2- 172.21.15.131:0/44202100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f492c075000 0x7f492c1a3d00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:05:58.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.227+0000 7f4932c00700 1 -- 172.21.15.131:0/44202100 >> 172.21.15.131:0/44202100 conn(0x7f492c078190 msgr2=0x7f492c10fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:05:58.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.227+0000 7f4932c00700 1 -- 172.21.15.131:0/44202100 shutdown_connections 2024-05-08T22:05:58.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:05:58.227+0000 7f4932c00700 1 -- 172.21.15.131:0/44202100 wait complete. 2024-05-08T22:05:58.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:57 smithi157 ceph-mon[77746]: pgmap v263: 129 pgs: 129 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.4 KiB/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T22:05:58.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:57 smithi132 ceph-mon[78180]: pgmap v263: 129 pgs: 129 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.4 KiB/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T22:05:58.683 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-08T22:05:58.683 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-08T22:05:58.684 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-08T22:05:58.684 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-08T22:05:58.684 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-08T22:05:58.685 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-08T22:05:58.685 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-08T22:05:58.685 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-08T22:05:58.686 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-08T22:05:58.686 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-08T22:05:58.687 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-08T22:05:58.687 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-08T22:05:59.144 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:58 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/44202100' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:59.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:58 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/44202100' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:05:59.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:58 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/44202100' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:06:00.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:05:59 smithi131 ceph-mon[37260]: pgmap v264: 129 pgs: 129 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-05-08T22:06:00.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:05:59 smithi157 ceph-mon[77746]: pgmap v264: 129 pgs: 129 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-05-08T22:06:00.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:05:59 smithi132 ceph-mon[78180]: pgmap v264: 129 pgs: 129 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-05-08T22:06:00.381 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:00.738 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:01.191 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:01.747 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:01.852 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:02.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:01 smithi131 ceph-mon[37260]: pgmap v265: 129 pgs: 129 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-08T22:06:02.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:01 smithi157 ceph-mon[77746]: pgmap v265: 129 pgs: 129 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-08T22:06:02.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:01 smithi132 ceph-mon[78180]: pgmap v265: 129 pgs: 129 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-08T22:06:02.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.487+0000 7f54fcc00700 1 -- 172.21.15.131:0/2405603210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f8100ff0 msgr2=0x7f54f8101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:02.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.487+0000 7f54fcc00700 1 --2- 172.21.15.131:0/2405603210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f8100ff0 0x7f54f8101450 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009990 tx=0x7f54e4009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:02.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.487+0000 7f54fcc00700 1 -- 172.21.15.131:0/2405603210 shutdown_connections 2024-05-08T22:06:02.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.487+0000 7f54fcc00700 1 --2- 172.21.15.131:0/2405603210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54f8101990 0x7f54f810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:02.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.487+0000 7f54fcc00700 1 --2- 172.21.15.131:0/2405603210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54f8107030 0x7f54f8107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:02.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.487+0000 7f54fcc00700 1 --2- 172.21.15.131:0/2405603210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f8100ff0 0x7f54f8101450 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:02.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.487+0000 7f54fcc00700 1 -- 172.21.15.131:0/2405603210 >> 172.21.15.131:0/2405603210 conn(0x7f54f8078190 msgr2=0x7f54f80785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:02.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.487+0000 7f54fcc00700 1 -- 172.21.15.131:0/2405603210 shutdown_connections 2024-05-08T22:06:02.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.487+0000 7f54fcc00700 1 -- 172.21.15.131:0/2405603210 wait complete. 2024-05-08T22:06:02.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.489+0000 7f54fcc00700 1 Processor -- start 2024-05-08T22:06:02.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.489+0000 7f54fcc00700 1 -- start start 2024-05-08T22:06:02.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.489+0000 7f54fcc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54f8100ff0 0x7f54f8197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:02.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.489+0000 7f54fcc00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54f8101990 0x7f54f8197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:02.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.489+0000 7f54fcc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f8107030 0x7f54f819bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:02.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.490+0000 7f54fcc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f819c330 con 0x7f54f8100ff0 2024-05-08T22:06:02.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.490+0000 7f54fcc00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f819c4a0 con 0x7f54f8101990 2024-05-08T22:06:02.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.490+0000 7f54fcc00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f819c660 con 0x7f54f8107030 2024-05-08T22:06:02.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.490+0000 7f54f5e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54f8100ff0 0x7f54f8197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:02.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.490+0000 7f54f6800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f8107030 0x7f54f819bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:02.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.490+0000 7f54f5e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54f8100ff0 0x7f54f8197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37864/0 (socket says 172.21.15.131:37864) 2024-05-08T22:06:02.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.490+0000 7f54f5e00700 1 -- 172.21.15.131:0/2644360578 learned_addr learned my addr 172.21.15.131:0/2644360578 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:02.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.490+0000 7f54f6800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f8107030 0x7f54f819bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46978/0 (socket says 172.21.15.131:46978) 2024-05-08T22:06:02.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.490+0000 7f54f5400700 1 --2- 172.21.15.131:0/2644360578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54f8101990 0x7f54f8197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:02.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.490+0000 7f54f5e00700 1 -- 172.21.15.131:0/2644360578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f8107030 msgr2=0x7f54f819bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:02.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.490+0000 7f54f5e00700 1 --2- 172.21.15.131:0/2644360578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f8107030 0x7f54f819bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:02.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.490+0000 7f54f5e00700 1 -- 172.21.15.131:0/2644360578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54f8101990 msgr2=0x7f54f8197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:02.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.490+0000 7f54f5e00700 1 --2- 172.21.15.131:0/2644360578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54f8101990 0x7f54f8197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:02.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.490+0000 7f54f5e00700 1 -- 172.21.15.131:0/2644360578 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e40095f0 con 0x7f54f8100ff0 2024-05-08T22:06:02.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.490+0000 7f54f6800700 1 --2- 172.21.15.131:0/2644360578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f8107030 0x7f54f819bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:02.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.490+0000 7f54f5400700 1 --2- 172.21.15.131:0/2644360578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54f8101990 0x7f54f8197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:02.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.491+0000 7f54f5e00700 1 --2- 172.21.15.131:0/2644360578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54f8100ff0 0x7f54f8197430 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f54e800e9a0 tx=0x7f54e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:02.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.491+0000 7f54eea00700 1 -- 172.21.15.131:0/2644360578 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e8009790 con 0x7f54f8100ff0 2024-05-08T22:06:02.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.491+0000 7f54fcc00700 1 -- 172.21.15.131:0/2644360578 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f819c940 con 0x7f54f8100ff0 2024-05-08T22:06:02.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.491+0000 7f54eea00700 1 -- 172.21.15.131:0/2644360578 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54e8010430 con 0x7f54f8100ff0 2024-05-08T22:06:02.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.491+0000 7f54eea00700 1 -- 172.21.15.131:0/2644360578 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e8018620 con 0x7f54f8100ff0 2024-05-08T22:06:02.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.491+0000 7f54fcc00700 1 -- 172.21.15.131:0/2644360578 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f81a41a0 con 0x7f54f8100ff0 2024-05-08T22:06:02.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.492+0000 7f54fcc00700 1 -- 172.21.15.131:0/2644360578 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f54f81026e0 con 0x7f54f8100ff0 2024-05-08T22:06:02.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.494+0000 7f54eea00700 1 -- 172.21.15.131:0/2644360578 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f54e80105a0 con 0x7f54f8100ff0 2024-05-08T22:06:02.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.496+0000 7f54eea00700 1 --2- 172.21.15.131:0/2644360578 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f54dc0a11b0 0x7f54dc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:02.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.496+0000 7f54eea00700 1 -- 172.21.15.131:0/2644360578 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f54e8014070 con 0x7f54f8100ff0 2024-05-08T22:06:02.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.496+0000 7f54f5400700 1 --2- 172.21.15.131:0/2644360578 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f54dc0a11b0 0x7f54dc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:02.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.496+0000 7f54eea00700 1 --2- 172.21.15.131:0/2644360578 >> [v2:172.21.15.157:6800/3561264016,v1:172.21.15.157:6801/3561264016] conn(0x7f54dc0ac1d0 0x7f54dc0ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:02.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.496+0000 7f54eea00700 1 -- 172.21.15.131:0/2644360578 --> [v2:172.21.15.157:6800/3561264016,v1:172.21.15.157:6801/3561264016] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54dc0aed40 con 0x7f54dc0ac1d0 2024-05-08T22:06:02.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.496+0000 7f54eea00700 1 -- 172.21.15.131:0/2644360578 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54e80cd520 con 0x7f54f8100ff0 2024-05-08T22:06:02.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.497+0000 7f54f6800700 1 --2- 172.21.15.131:0/2644360578 >> [v2:172.21.15.157:6800/3561264016,v1:172.21.15.157:6801/3561264016] conn(0x7f54dc0ac1d0 0x7f54dc0ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:02.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.497+0000 7f54f5400700 1 --2- 172.21.15.131:0/2644360578 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f54dc0a11b0 0x7f54dc0a3670 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f54f8198990 tx=0x7f54e400b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:02.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.497+0000 7f54f6800700 1 --2- 172.21.15.131:0/2644360578 >> [v2:172.21.15.157:6800/3561264016,v1:172.21.15.157:6801/3561264016] conn(0x7f54dc0ac1d0 0x7f54dc0ae670 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:02.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.498+0000 7f54eea00700 1 -- 172.21.15.131:0/2644360578 <== osd.2 v2:172.21.15.157:6800/3561264016 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f54dc0aed40 con 0x7f54dc0ac1d0 2024-05-08T22:06:02.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.523+0000 7f54fcc00700 1 -- 172.21.15.131:0/2644360578 --> [v2:172.21.15.157:6800/3561264016,v1:172.21.15.157:6801/3561264016] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f54f8111180 con 0x7f54dc0ac1d0 2024-05-08T22:06:02.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.524+0000 7f54eea00700 1 -- 172.21.15.131:0/2644360578 <== osd.2 v2:172.21.15.157:6800/3561264016 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f54f8111180 con 0x7f54dc0ac1d0 2024-05-08T22:06:02.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.524+0000 7f54fcc00700 1 -- 172.21.15.131:0/2644360578 >> [v2:172.21.15.157:6800/3561264016,v1:172.21.15.157:6801/3561264016] conn(0x7f54dc0ac1d0 msgr2=0x7f54dc0ae670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:02.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.524+0000 7f54fcc00700 1 --2- 172.21.15.131:0/2644360578 >> [v2:172.21.15.157:6800/3561264016,v1:172.21.15.157:6801/3561264016] conn(0x7f54dc0ac1d0 0x7f54dc0ae670 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:02.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.524+0000 7f54fcc00700 1 -- 172.21.15.131:0/2644360578 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f54dc0a11b0 msgr2=0x7f54dc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:02.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.524+0000 7f54fcc00700 1 --2- 172.21.15.131:0/2644360578 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f54dc0a11b0 0x7f54dc0a3670 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f54f8198990 tx=0x7f54e400b260 comp rx=0 tx=0).stop 2024-05-08T22:06:02.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.524+0000 7f54fcc00700 1 -- 172.21.15.131:0/2644360578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54f8100ff0 msgr2=0x7f54f8197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:02.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.524+0000 7f54fcc00700 1 --2- 172.21.15.131:0/2644360578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54f8100ff0 0x7f54f8197430 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f54e800e9a0 tx=0x7f54e800ecb0 comp rx=0 tx=0).stop 2024-05-08T22:06:02.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.524+0000 7f54f6800700 1 -- 172.21.15.131:0/2644360578 reap_dead start 2024-05-08T22:06:02.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.524+0000 7f54fcc00700 1 -- 172.21.15.131:0/2644360578 shutdown_connections 2024-05-08T22:06:02.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.524+0000 7f54fcc00700 1 -- 172.21.15.131:0/2644360578 >> 172.21.15.131:0/2644360578 conn(0x7f54f8078190 msgr2=0x7f54f80ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:02.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.524+0000 7f54fcc00700 1 -- 172.21.15.131:0/2644360578 shutdown_connections 2024-05-08T22:06:02.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:02.525+0000 7f54fcc00700 1 -- 172.21.15.131:0/2644360578 wait complete. 2024-05-08T22:06:02.606 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:02.859 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:02.860 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:03.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.840+0000 7fb92ce00700 1 -- 172.21.15.131:0/2402635179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9281019c0 msgr2=0x7fb928105a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:03.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.840+0000 7fb92ce00700 1 --2- 172.21.15.131:0/2402635179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9281019c0 0x7fb928105a10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb910009990 tx=0x7fb910009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:03.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.840+0000 7fb92ce00700 1 -- 172.21.15.131:0/2402635179 shutdown_connections 2024-05-08T22:06:03.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.840+0000 7fb92ce00700 1 --2- 172.21.15.131:0/2402635179 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9281060f0 0x7fb92810dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:03.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.840+0000 7fb92ce00700 1 --2- 172.21.15.131:0/2402635179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb928101010 0x7fb9281013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:03.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.840+0000 7fb92ce00700 1 --2- 172.21.15.131:0/2402635179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9281019c0 0x7fb928105a10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:03.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.840+0000 7fb92ce00700 1 -- 172.21.15.131:0/2402635179 >> 172.21.15.131:0/2402635179 conn(0x7fb9280fc900 msgr2=0x7fb9280fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:03.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.840+0000 7fb92ce00700 1 -- 172.21.15.131:0/2402635179 shutdown_connections 2024-05-08T22:06:03.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.840+0000 7fb92ce00700 1 -- 172.21.15.131:0/2402635179 wait complete. 2024-05-08T22:06:03.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.841+0000 7fb92ce00700 1 Processor -- start 2024-05-08T22:06:03.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.841+0000 7fb92ce00700 1 -- start start 2024-05-08T22:06:03.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.842+0000 7fb92ce00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb928101010 0x7fb9281994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:03.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.842+0000 7fb92ce00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9281019c0 0x7fb9281999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:03.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.842+0000 7fb92ce00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9281060f0 0x7fb92819dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:03.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.842+0000 7fb92ce00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92819e3b0 con 0x7fb9281019c0 2024-05-08T22:06:03.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.842+0000 7fb92ce00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92819e520 con 0x7fb9281060f0 2024-05-08T22:06:03.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.842+0000 7fb92ce00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92819e6e0 con 0x7fb928101010 2024-05-08T22:06:03.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.842+0000 7fb91ea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9281019c0 0x7fb9281999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:03.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.842+0000 7fb927400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb928101010 0x7fb9281994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:03.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.842+0000 7fb91ea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9281019c0 0x7fb9281999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34652/0 (socket says 172.21.15.131:34652) 2024-05-08T22:06:03.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.842+0000 7fb91ea00700 1 -- 172.21.15.131:0/2649474677 learned_addr learned my addr 172.21.15.131:0/2649474677 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:03.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.842+0000 7fb927e00700 1 --2- 172.21.15.131:0/2649474677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9281060f0 0x7fb92819dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:03.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.842+0000 7fb927400700 1 -- 172.21.15.131:0/2649474677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9281060f0 msgr2=0x7fb92819dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:03.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.842+0000 7fb927400700 1 --2- 172.21.15.131:0/2649474677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9281060f0 0x7fb92819dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:03.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.842+0000 7fb927400700 1 -- 172.21.15.131:0/2649474677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9281019c0 msgr2=0x7fb9281999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:03.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.842+0000 7fb927400700 1 --2- 172.21.15.131:0/2649474677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9281019c0 0x7fb9281999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:03.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.842+0000 7fb927400700 1 -- 172.21.15.131:0/2649474677 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9100095f0 con 0x7fb928101010 2024-05-08T22:06:03.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.842+0000 7fb91ea00700 1 --2- 172.21.15.131:0/2649474677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9281019c0 0x7fb9281999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:03.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.843+0000 7fb927400700 1 --2- 172.21.15.131:0/2649474677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb928101010 0x7fb9281994b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb91800ef10 tx=0x7fb91800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:03.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.843+0000 7fb924c00700 1 -- 172.21.15.131:0/2649474677 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91800c9c0 con 0x7fb928101010 2024-05-08T22:06:03.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.843+0000 7fb92ce00700 1 -- 172.21.15.131:0/2649474677 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb92819e960 con 0x7fb928101010 2024-05-08T22:06:03.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.843+0000 7fb924c00700 1 -- 172.21.15.131:0/2649474677 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb91800f040 con 0x7fb928101010 2024-05-08T22:06:03.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.843+0000 7fb924c00700 1 -- 172.21.15.131:0/2649474677 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb918014480 con 0x7fb928101010 2024-05-08T22:06:03.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.843+0000 7fb92ce00700 1 -- 172.21.15.131:0/2649474677 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9281a6220 con 0x7fb928101010 2024-05-08T22:06:03.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.844+0000 7fb92ce00700 1 -- 172.21.15.131:0/2649474677 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb928110ff0 con 0x7fb928101010 2024-05-08T22:06:03.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.846+0000 7fb924c00700 1 -- 172.21.15.131:0/2649474677 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fb91800cb20 con 0x7fb928101010 2024-05-08T22:06:03.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.847+0000 7fb924c00700 1 --2- 172.21.15.131:0/2649474677 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb90c0a11b0 0x7fb90c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:03.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.847+0000 7fb924c00700 1 -- 172.21.15.131:0/2649474677 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb9180cdb30 con 0x7fb928101010 2024-05-08T22:06:03.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.847+0000 7fb91ea00700 1 --2- 172.21.15.131:0/2649474677 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb90c0a11b0 0x7fb90c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:03.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.848+0000 7fb924c00700 1 --2- 172.21.15.131:0/2649474677 >> [v2:172.21.15.131:6810/1075134307,v1:172.21.15.131:6811/1075134307] conn(0x7fb90c0ac1d0 0x7fb90c0ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:03.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.848+0000 7fb924c00700 1 -- 172.21.15.131:0/2649474677 --> [v2:172.21.15.131:6810/1075134307,v1:172.21.15.131:6811/1075134307] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb90c0aed40 con 0x7fb90c0ac1d0 2024-05-08T22:06:03.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.848+0000 7fb924c00700 1 -- 172.21.15.131:0/2649474677 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9180cdea0 con 0x7fb928101010 2024-05-08T22:06:03.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.848+0000 7fb927e00700 1 --2- 172.21.15.131:0/2649474677 >> [v2:172.21.15.131:6810/1075134307,v1:172.21.15.131:6811/1075134307] conn(0x7fb90c0ac1d0 0x7fb90c0ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:03.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.848+0000 7fb91ea00700 1 --2- 172.21.15.131:0/2649474677 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb90c0a11b0 0x7fb90c0a3670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb92819aa10 tx=0x7fb91000b280 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:03.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.849+0000 7fb927e00700 1 --2- 172.21.15.131:0/2649474677 >> [v2:172.21.15.131:6810/1075134307,v1:172.21.15.131:6811/1075134307] conn(0x7fb90c0ac1d0 0x7fb90c0ae670 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:03.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.849+0000 7fb924c00700 1 -- 172.21.15.131:0/2649474677 <== osd.3 v2:172.21.15.131:6810/1075134307 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fb90c0aed40 con 0x7fb90c0ac1d0 2024-05-08T22:06:03.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.866+0000 7fb92ce00700 1 -- 172.21.15.131:0/2649474677 --> [v2:172.21.15.131:6810/1075134307,v1:172.21.15.131:6811/1075134307] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb9281024f0 con 0x7fb90c0ac1d0 2024-05-08T22:06:03.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.867+0000 7fb924c00700 1 -- 172.21.15.131:0/2649474677 <== osd.3 v2:172.21.15.131:6810/1075134307 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb9281024f0 con 0x7fb90c0ac1d0 2024-05-08T22:06:03.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.867+0000 7fb92ce00700 1 -- 172.21.15.131:0/2649474677 >> [v2:172.21.15.131:6810/1075134307,v1:172.21.15.131:6811/1075134307] conn(0x7fb90c0ac1d0 msgr2=0x7fb90c0ae670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:03.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.867+0000 7fb92ce00700 1 --2- 172.21.15.131:0/2649474677 >> [v2:172.21.15.131:6810/1075134307,v1:172.21.15.131:6811/1075134307] conn(0x7fb90c0ac1d0 0x7fb90c0ae670 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.867+0000 7fb92ce00700 1 -- 172.21.15.131:0/2649474677 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb90c0a11b0 msgr2=0x7fb90c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.867+0000 7fb92ce00700 1 --2- 172.21.15.131:0/2649474677 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb90c0a11b0 0x7fb90c0a3670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb92819aa10 tx=0x7fb91000b280 comp rx=0 tx=0).stop 2024-05-08T22:06:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.867+0000 7fb92ce00700 1 -- 172.21.15.131:0/2649474677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb928101010 msgr2=0x7fb9281994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.867+0000 7fb92ce00700 1 --2- 172.21.15.131:0/2649474677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb928101010 0x7fb9281994b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb91800ef10 tx=0x7fb91800c3a0 comp rx=0 tx=0).stop 2024-05-08T22:06:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.867+0000 7fb927e00700 1 -- 172.21.15.131:0/2649474677 reap_dead start 2024-05-08T22:06:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.867+0000 7fb92ce00700 1 -- 172.21.15.131:0/2649474677 shutdown_connections 2024-05-08T22:06:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.867+0000 7fb92ce00700 1 -- 172.21.15.131:0/2649474677 >> 172.21.15.131:0/2649474677 conn(0x7fb9280fc900 msgr2=0x7fb9280fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.868+0000 7fb92ce00700 1 -- 172.21.15.131:0/2649474677 shutdown_connections 2024-05-08T22:06:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:03.868+0000 7fb92ce00700 1 -- 172.21.15.131:0/2649474677 wait complete. 2024-05-08T22:06:04.114 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:03 smithi131 ceph-mon[37260]: pgmap v266: 129 pgs: 129 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-08T22:06:04.137 INFO:teuthology.orchestra.run.smithi131.stdout:77309411383 2024-05-08T22:06:04.137 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-08T22:06:04.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:03 smithi157 ceph-mon[77746]: pgmap v266: 129 pgs: 129 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-08T22:06:04.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:03 smithi132 ceph-mon[78180]: pgmap v266: 129 pgs: 129 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-08T22:06:04.525 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:05.000 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:05.075 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:05.445 INFO:teuthology.orchestra.run.smithi131.stdout:103079215154 2024-05-08T22:06:05.446 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-08T22:06:06.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:05 smithi131 ceph-mon[37260]: pgmap v267: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-08T22:06:06.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:05 smithi157 ceph-mon[77746]: pgmap v267: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-08T22:06:06.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:05 smithi132 ceph-mon[78180]: pgmap v267: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-08T22:06:06.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.614+0000 7f6f2f200700 1 -- 172.21.15.131:0/3657167063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f28100ff0 msgr2=0x7f6f28101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:06.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.614+0000 7f6f2f200700 1 --2- 172.21.15.131:0/3657167063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f28100ff0 0x7f6f28101450 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009990 tx=0x7f6f14009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:06.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.614+0000 7f6f2f200700 1 -- 172.21.15.131:0/3657167063 shutdown_connections 2024-05-08T22:06:06.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.614+0000 7f6f2f200700 1 --2- 172.21.15.131:0/3657167063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f28101990 0x7f6f2810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:06.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.614+0000 7f6f2f200700 1 --2- 172.21.15.131:0/3657167063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f28107030 0x7f6f28107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:06.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.614+0000 7f6f2f200700 1 --2- 172.21.15.131:0/3657167063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f28100ff0 0x7f6f28101450 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:06.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.614+0000 7f6f2f200700 1 -- 172.21.15.131:0/3657167063 >> 172.21.15.131:0/3657167063 conn(0x7f6f28078190 msgr2=0x7f6f280785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:06.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.614+0000 7f6f2f200700 1 -- 172.21.15.131:0/3657167063 shutdown_connections 2024-05-08T22:06:06.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.614+0000 7f6f2f200700 1 -- 172.21.15.131:0/3657167063 wait complete. 2024-05-08T22:06:06.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.615+0000 7f6f2f200700 1 Processor -- start 2024-05-08T22:06:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.615+0000 7f6f2f200700 1 -- start start 2024-05-08T22:06:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.615+0000 7f6f2f200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f28100ff0 0x7f6f281104e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.615+0000 7f6f2f200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f28101990 0x7f6f28110a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.615+0000 7f6f2f200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f28107030 0x7f6f28114dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.615+0000 7f6f2f200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f281153d0 con 0x7f6f28107030 2024-05-08T22:06:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.615+0000 7f6f2f200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f28115540 con 0x7f6f28100ff0 2024-05-08T22:06:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.615+0000 7f6f2f200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f28115700 con 0x7f6f28101990 2024-05-08T22:06:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.616+0000 7f6f27e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f28100ff0 0x7f6f281104e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.616+0000 7f6f27e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f28100ff0 0x7f6f281104e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:39084/0 (socket says 172.21.15.131:39084) 2024-05-08T22:06:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.616+0000 7f6f27e00700 1 -- 172.21.15.131:0/1970913399 learned_addr learned my addr 172.21.15.131:0/1970913399 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.616+0000 7f6f27400700 1 --2- 172.21.15.131:0/1970913399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f28101990 0x7f6f28110a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.616+0000 7f6f27e00700 1 -- 172.21.15.131:0/1970913399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f28101990 msgr2=0x7f6f28110a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.616+0000 7f6f27e00700 1 --2- 172.21.15.131:0/1970913399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f28101990 0x7f6f28110a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.616+0000 7f6f27e00700 1 -- 172.21.15.131:0/1970913399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f28107030 msgr2=0x7f6f28114dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.616+0000 7f6f27e00700 1 --2- 172.21.15.131:0/1970913399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f28107030 0x7f6f28114dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.616+0000 7f6f27e00700 1 -- 172.21.15.131:0/1970913399 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f140095f0 con 0x7f6f28100ff0 2024-05-08T22:06:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.617+0000 7f6f27e00700 1 --2- 172.21.15.131:0/1970913399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f28100ff0 0x7f6f281104e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800e9a0 tx=0x7f6f1800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.617+0000 7f6f24c00700 1 -- 172.21.15.131:0/1970913399 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f18009790 con 0x7f6f28100ff0 2024-05-08T22:06:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.617+0000 7f6f24c00700 1 -- 172.21.15.131:0/1970913399 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f1800cb50 con 0x7f6f28100ff0 2024-05-08T22:06:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.617+0000 7f6f2f200700 1 -- 172.21.15.131:0/1970913399 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f281159e0 con 0x7f6f28100ff0 2024-05-08T22:06:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.617+0000 7f6f24c00700 1 -- 172.21.15.131:0/1970913399 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f18018680 con 0x7f6f28100ff0 2024-05-08T22:06:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.618+0000 7f6f2f200700 1 -- 172.21.15.131:0/1970913399 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f281b3090 con 0x7f6f28100ff0 2024-05-08T22:06:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.618+0000 7f6f2f200700 1 -- 172.21.15.131:0/1970913399 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6f281026e0 con 0x7f6f28100ff0 2024-05-08T22:06:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.621+0000 7f6f24c00700 1 -- 172.21.15.131:0/1970913399 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f6f1800ccc0 con 0x7f6f28100ff0 2024-05-08T22:06:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.622+0000 7f6f24c00700 1 --2- 172.21.15.131:0/1970913399 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f6f0c0a1210 0x7f6f0c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.622+0000 7f6f24c00700 1 -- 172.21.15.131:0/1970913399 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6f18014070 con 0x7f6f28100ff0 2024-05-08T22:06:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.622+0000 7f6f27400700 1 --2- 172.21.15.131:0/1970913399 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f6f0c0a1210 0x7f6f0c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.622+0000 7f6f24c00700 1 --2- 172.21.15.131:0/1970913399 >> [v2:172.21.15.131:6826/4217805693,v1:172.21.15.131:6827/4217805693] conn(0x7f6f0c0ac230 0x7f6f0c0ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.622+0000 7f6f24c00700 1 -- 172.21.15.131:0/1970913399 --> [v2:172.21.15.131:6826/4217805693,v1:172.21.15.131:6827/4217805693] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6f0c0aeda0 con 0x7f6f0c0ac230 2024-05-08T22:06:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.622+0000 7f6f24c00700 1 -- 172.21.15.131:0/1970913399 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f180cd530 con 0x7f6f28100ff0 2024-05-08T22:06:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.623+0000 7f6f2d200700 1 --2- 172.21.15.131:0/1970913399 >> [v2:172.21.15.131:6826/4217805693,v1:172.21.15.131:6827/4217805693] conn(0x7f6f0c0ac230 0x7f6f0c0ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.623+0000 7f6f27400700 1 --2- 172.21.15.131:0/1970913399 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f6f0c0a1210 0x7f6f0c0a36d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6f28111a40 tx=0x7f6f14005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.623+0000 7f6f2d200700 1 --2- 172.21.15.131:0/1970913399 >> [v2:172.21.15.131:6826/4217805693,v1:172.21.15.131:6827/4217805693] conn(0x7f6f0c0ac230 0x7f6f0c0ae6d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.624+0000 7f6f24c00700 1 -- 172.21.15.131:0/1970913399 <== osd.9 v2:172.21.15.131:6826/4217805693 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f6f0c0aeda0 con 0x7f6f0c0ac230 2024-05-08T22:06:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.641+0000 7f6f2f200700 1 -- 172.21.15.131:0/1970913399 --> [v2:172.21.15.131:6826/4217805693,v1:172.21.15.131:6827/4217805693] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6f2810ef80 con 0x7f6f0c0ac230 2024-05-08T22:06:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.642+0000 7f6f24c00700 1 -- 172.21.15.131:0/1970913399 <== osd.9 v2:172.21.15.131:6826/4217805693 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6f2810ef80 con 0x7f6f0c0ac230 2024-05-08T22:06:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.643+0000 7f6f2f200700 1 -- 172.21.15.131:0/1970913399 >> [v2:172.21.15.131:6826/4217805693,v1:172.21.15.131:6827/4217805693] conn(0x7f6f0c0ac230 msgr2=0x7f6f0c0ae6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.643+0000 7f6f2f200700 1 --2- 172.21.15.131:0/1970913399 >> [v2:172.21.15.131:6826/4217805693,v1:172.21.15.131:6827/4217805693] conn(0x7f6f0c0ac230 0x7f6f0c0ae6d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.643+0000 7f6f2f200700 1 -- 172.21.15.131:0/1970913399 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f6f0c0a1210 msgr2=0x7f6f0c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.643+0000 7f6f2f200700 1 --2- 172.21.15.131:0/1970913399 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f6f0c0a1210 0x7f6f0c0a36d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6f28111a40 tx=0x7f6f14005fd0 comp rx=0 tx=0).stop 2024-05-08T22:06:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.643+0000 7f6f2f200700 1 -- 172.21.15.131:0/1970913399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f28100ff0 msgr2=0x7f6f281104e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.643+0000 7f6f2f200700 1 --2- 172.21.15.131:0/1970913399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f28100ff0 0x7f6f281104e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800e9a0 tx=0x7f6f1800ecb0 comp rx=0 tx=0).stop 2024-05-08T22:06:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.643+0000 7f6f2d200700 1 -- 172.21.15.131:0/1970913399 reap_dead start 2024-05-08T22:06:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.643+0000 7f6f2f200700 1 -- 172.21.15.131:0/1970913399 shutdown_connections 2024-05-08T22:06:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.643+0000 7f6f2f200700 1 -- 172.21.15.131:0/1970913399 >> 172.21.15.131:0/1970913399 conn(0x7f6f28078190 msgr2=0x7f6f280ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.643+0000 7f6f2f200700 1 -- 172.21.15.131:0/1970913399 shutdown_connections 2024-05-08T22:06:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:06.643+0000 7f6f2f200700 1 -- 172.21.15.131:0/1970913399 wait complete. 2024-05-08T22:06:07.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.131+0000 7f7f9de00700 1 -- 172.21.15.131:0/753492589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f98101990 msgr2=0x7f7f9810de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:07.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.131+0000 7f7f9de00700 1 --2- 172.21.15.131:0/753492589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f98101990 0x7f7f9810de70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000b0a0 tx=0x7f7f9000b3b0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.131+0000 7f7f9de00700 1 -- 172.21.15.131:0/753492589 shutdown_connections 2024-05-08T22:06:07.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.132+0000 7f7f9de00700 1 --2- 172.21.15.131:0/753492589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f98101990 0x7f7f9810de70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.132+0000 7f7f9de00700 1 --2- 172.21.15.131:0/753492589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f98107030 0x7f7f98107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.132+0000 7f7f9de00700 1 --2- 172.21.15.131:0/753492589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f98100ff0 0x7f7f98101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.132+0000 7f7f9de00700 1 -- 172.21.15.131:0/753492589 >> 172.21.15.131:0/753492589 conn(0x7f7f98078190 msgr2=0x7f7f980785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:07.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.132+0000 7f7f9de00700 1 -- 172.21.15.131:0/753492589 shutdown_connections 2024-05-08T22:06:07.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.132+0000 7f7f9de00700 1 -- 172.21.15.131:0/753492589 wait complete. 2024-05-08T22:06:07.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.133+0000 7f7f9de00700 1 Processor -- start 2024-05-08T22:06:07.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.133+0000 7f7f9de00700 1 -- start start 2024-05-08T22:06:07.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.133+0000 7f7f9de00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f98100ff0 0x7f7f98199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:07.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.133+0000 7f7f9de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f98101990 0x7f7f98199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:07.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.133+0000 7f7f9de00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f98107030 0x7f7f9819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:07.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.133+0000 7f7f9de00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9819d590 con 0x7f7f98101990 2024-05-08T22:06:07.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.133+0000 7f7f9de00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9819d700 con 0x7f7f98107030 2024-05-08T22:06:07.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.133+0000 7f7f9de00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9819d8c0 con 0x7f7f98100ff0 2024-05-08T22:06:07.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.133+0000 7f7f96000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f98101990 0x7f7f98199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:07.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.133+0000 7f7f96a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f98100ff0 0x7f7f98199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:07.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.134+0000 7f7f96000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f98101990 0x7f7f98199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34698/0 (socket says 172.21.15.131:34698) 2024-05-08T22:06:07.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.134+0000 7f7f97400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f98107030 0x7f7f9819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:07.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.134+0000 7f7f96000700 1 -- 172.21.15.131:0/2038569335 learned_addr learned my addr 172.21.15.131:0/2038569335 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:07.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.134+0000 7f7f96a00700 1 -- 172.21.15.131:0/2038569335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f98107030 msgr2=0x7f7f9819cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:07.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.134+0000 7f7f96a00700 1 --2- 172.21.15.131:0/2038569335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f98107030 0x7f7f9819cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.134+0000 7f7f96a00700 1 -- 172.21.15.131:0/2038569335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f98101990 msgr2=0x7f7f98199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:07.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.134+0000 7f7f96a00700 1 --2- 172.21.15.131:0/2038569335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f98101990 0x7f7f98199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.134+0000 7f7f96a00700 1 -- 172.21.15.131:0/2038569335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f90009d00 con 0x7f7f98100ff0 2024-05-08T22:06:07.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.134+0000 7f7f96000700 1 --2- 172.21.15.131:0/2038569335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f98101990 0x7f7f98199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:07.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.134+0000 7f7f96a00700 1 --2- 172.21.15.131:0/2038569335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f98100ff0 0x7f7f98199750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800e9f0 tx=0x7f7f8800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:07.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.135+0000 7f7f9ca00700 1 -- 172.21.15.131:0/2038569335 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8800c9c0 con 0x7f7f98100ff0 2024-05-08T22:06:07.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.135+0000 7f7f9ca00700 1 -- 172.21.15.131:0/2038569335 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f88010430 con 0x7f7f98100ff0 2024-05-08T22:06:07.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.135+0000 7f7f9de00700 1 -- 172.21.15.131:0/2038569335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f9819dba0 con 0x7f7f98100ff0 2024-05-08T22:06:07.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.135+0000 7f7f9ca00700 1 -- 172.21.15.131:0/2038569335 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88018680 con 0x7f7f98100ff0 2024-05-08T22:06:07.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.135+0000 7f7f9de00700 1 -- 172.21.15.131:0/2038569335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f981a64f0 con 0x7f7f98100ff0 2024-05-08T22:06:07.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.135+0000 7f7f9de00700 1 -- 172.21.15.131:0/2038569335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7f981111b0 con 0x7f7f98100ff0 2024-05-08T22:06:07.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.137+0000 7f7f9ca00700 1 -- 172.21.15.131:0/2038569335 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f7f88010860 con 0x7f7f98100ff0 2024-05-08T22:06:07.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.139+0000 7f7f9ca00700 1 --2- 172.21.15.131:0/2038569335 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7f7c0a1230 0x7f7f7c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:07.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.139+0000 7f7f9ca00700 1 -- 172.21.15.131:0/2038569335 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7f88014070 con 0x7f7f98100ff0 2024-05-08T22:06:07.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.139+0000 7f7f96000700 1 --2- 172.21.15.131:0/2038569335 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7f7c0a1230 0x7f7f7c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:07.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.139+0000 7f7f9ca00700 1 --2- 172.21.15.131:0/2038569335 >> [v2:172.21.15.132:6800/2358880332,v1:172.21.15.132:6801/2358880332] conn(0x7f7f7c0ac240 0x7f7f7c0ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:07.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.139+0000 7f7f9ca00700 1 -- 172.21.15.131:0/2038569335 --> [v2:172.21.15.132:6800/2358880332,v1:172.21.15.132:6801/2358880332] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f7c0aedb0 con 0x7f7f7c0ac240 2024-05-08T22:06:07.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.139+0000 7f7f9ca00700 1 -- 172.21.15.131:0/2038569335 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f880d12a0 con 0x7f7f98100ff0 2024-05-08T22:06:07.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.139+0000 7f7f97400700 1 --2- 172.21.15.131:0/2038569335 >> [v2:172.21.15.132:6800/2358880332,v1:172.21.15.132:6801/2358880332] conn(0x7f7f7c0ac240 0x7f7f7c0ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:07.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.139+0000 7f7f96000700 1 --2- 172.21.15.131:0/2038569335 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7f7c0a1230 0x7f7f7c0a36f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7f9819acb0 tx=0x7f7f84009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:07.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.140+0000 7f7f97400700 1 --2- 172.21.15.131:0/2038569335 >> [v2:172.21.15.132:6800/2358880332,v1:172.21.15.132:6801/2358880332] conn(0x7f7f7c0ac240 0x7f7f7c0ae6e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:07.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.141+0000 7f7f9ca00700 1 -- 172.21.15.131:0/2038569335 <== osd.1 v2:172.21.15.132:6800/2358880332 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f7f7c0aedb0 con 0x7f7f7c0ac240 2024-05-08T22:06:07.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.157+0000 7f7f9de00700 1 -- 172.21.15.131:0/2038569335 --> [v2:172.21.15.132:6800/2358880332,v1:172.21.15.132:6801/2358880332] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7f981026e0 con 0x7f7f7c0ac240 2024-05-08T22:06:07.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.158+0000 7f7f9ca00700 1 -- 172.21.15.131:0/2038569335 <== osd.1 v2:172.21.15.132:6800/2358880332 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7f981026e0 con 0x7f7f7c0ac240 2024-05-08T22:06:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.158+0000 7f7f9de00700 1 -- 172.21.15.131:0/2038569335 >> [v2:172.21.15.132:6800/2358880332,v1:172.21.15.132:6801/2358880332] conn(0x7f7f7c0ac240 msgr2=0x7f7f7c0ae6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.158+0000 7f7f9de00700 1 --2- 172.21.15.131:0/2038569335 >> [v2:172.21.15.132:6800/2358880332,v1:172.21.15.132:6801/2358880332] conn(0x7f7f7c0ac240 0x7f7f7c0ae6e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.159+0000 7f7f9de00700 1 -- 172.21.15.131:0/2038569335 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7f7c0a1230 msgr2=0x7f7f7c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.159+0000 7f7f9de00700 1 --2- 172.21.15.131:0/2038569335 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7f7c0a1230 0x7f7f7c0a36f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7f9819acb0 tx=0x7f7f84009260 comp rx=0 tx=0).stop 2024-05-08T22:06:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.159+0000 7f7f9de00700 1 -- 172.21.15.131:0/2038569335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f98100ff0 msgr2=0x7f7f98199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.159+0000 7f7f9de00700 1 --2- 172.21.15.131:0/2038569335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f98100ff0 0x7f7f98199750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800e9f0 tx=0x7f7f8800ed00 comp rx=0 tx=0).stop 2024-05-08T22:06:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.159+0000 7f7f97400700 1 -- 172.21.15.131:0/2038569335 reap_dead start 2024-05-08T22:06:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.159+0000 7f7f9de00700 1 -- 172.21.15.131:0/2038569335 shutdown_connections 2024-05-08T22:06:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.159+0000 7f7f9de00700 1 -- 172.21.15.131:0/2038569335 >> 172.21.15.131:0/2038569335 conn(0x7f7f98078190 msgr2=0x7f7f9810c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.159+0000 7f7f9de00700 1 -- 172.21.15.131:0/2038569335 shutdown_connections 2024-05-08T22:06:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:07.159+0000 7f7f9de00700 1 -- 172.21.15.131:0/2038569335 wait complete. 2024-05-08T22:06:08.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.133+0000 7f3d21c00700 1 -- 172.21.15.131:0/2997963372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d1c107030 msgr2=0x7f3d1c107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:08.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.133+0000 7f3d21c00700 1 --2- 172.21.15.131:0/2997963372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d1c107030 0x7f3d1c107410 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c009990 tx=0x7f3d0c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:08.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.133+0000 7f3d21c00700 1 -- 172.21.15.131:0/2997963372 shutdown_connections 2024-05-08T22:06:08.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.133+0000 7f3d21c00700 1 --2- 172.21.15.131:0/2997963372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d1c101990 0x7f3d1c10de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:08.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.133+0000 7f3d21c00700 1 --2- 172.21.15.131:0/2997963372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d1c107030 0x7f3d1c107410 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:08.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.133+0000 7f3d21c00700 1 --2- 172.21.15.131:0/2997963372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d1c100ff0 0x7f3d1c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:08.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.133+0000 7f3d21c00700 1 -- 172.21.15.131:0/2997963372 >> 172.21.15.131:0/2997963372 conn(0x7f3d1c078190 msgr2=0x7f3d1c0785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:08.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.133+0000 7f3d21c00700 1 -- 172.21.15.131:0/2997963372 shutdown_connections 2024-05-08T22:06:08.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.133+0000 7f3d21c00700 1 -- 172.21.15.131:0/2997963372 wait complete. 2024-05-08T22:06:08.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.134+0000 7f3d21c00700 1 Processor -- start 2024-05-08T22:06:08.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.134+0000 7f3d21c00700 1 -- start start 2024-05-08T22:06:08.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.135+0000 7f3d21c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d1c100ff0 0x7f3d1c1995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:08.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.135+0000 7f3d21c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d1c101990 0x7f3d1c199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:08.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.135+0000 7f3d21c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d1c107030 0x7f3d1c19dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:08.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.135+0000 7f3d21c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c19e4f0 con 0x7f3d1c101990 2024-05-08T22:06:08.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.135+0000 7f3d21c00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c19e660 con 0x7f3d1c107030 2024-05-08T22:06:08.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.135+0000 7f3d21c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c19e820 con 0x7f3d1c100ff0 2024-05-08T22:06:08.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.135+0000 7f3d1a000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d1c101990 0x7f3d1c199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:08.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.135+0000 7f3d1aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d1c100ff0 0x7f3d1c1995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:08.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.135+0000 7f3d1b400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d1c107030 0x7f3d1c19dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:08.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.135+0000 7f3d1aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d1c100ff0 0x7f3d1c1995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:42408/0 (socket says 172.21.15.131:42408) 2024-05-08T22:06:08.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.135+0000 7f3d1aa00700 1 -- 172.21.15.131:0/3401884003 learned_addr learned my addr 172.21.15.131:0/3401884003 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:08.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.135+0000 7f3d1aa00700 1 -- 172.21.15.131:0/3401884003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d1c107030 msgr2=0x7f3d1c19dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.135+0000 7f3d1aa00700 1 --2- 172.21.15.131:0/3401884003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d1c107030 0x7f3d1c19dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.135+0000 7f3d1aa00700 1 -- 172.21.15.131:0/3401884003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d1c101990 msgr2=0x7f3d1c199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.135+0000 7f3d1aa00700 1 --2- 172.21.15.131:0/3401884003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d1c101990 0x7f3d1c199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.135+0000 7f3d1aa00700 1 -- 172.21.15.131:0/3401884003 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0c0095f0 con 0x7f3d1c100ff0 2024-05-08T22:06:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.135+0000 7f3d1b400700 1 --2- 172.21.15.131:0/3401884003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d1c107030 0x7f3d1c19dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.136+0000 7f3d1aa00700 1 --2- 172.21.15.131:0/3401884003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d1c100ff0 0x7f3d1c1995f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c005910 tx=0x7f3d0c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.136+0000 7f3d13400700 1 -- 172.21.15.131:0/3401884003 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c01d070 con 0x7f3d1c100ff0 2024-05-08T22:06:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.136+0000 7f3d21c00700 1 -- 172.21.15.131:0/3401884003 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d1c19eaa0 con 0x7f3d1c100ff0 2024-05-08T22:06:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.136+0000 7f3d13400700 1 -- 172.21.15.131:0/3401884003 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d0c022470 con 0x7f3d1c100ff0 2024-05-08T22:06:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.136+0000 7f3d13400700 1 -- 172.21.15.131:0/3401884003 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c017650 con 0x7f3d1c100ff0 2024-05-08T22:06:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.136+0000 7f3d21c00700 1 -- 172.21.15.131:0/3401884003 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d1c1a6360 con 0x7f3d1c100ff0 2024-05-08T22:06:08.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.137+0000 7f3d21c00700 1 -- 172.21.15.131:0/3401884003 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3d1c1111c0 con 0x7f3d1c100ff0 2024-05-08T22:06:08.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.139+0000 7f3d13400700 1 -- 172.21.15.131:0/3401884003 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f3d0c0225e0 con 0x7f3d1c100ff0 2024-05-08T22:06:08.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:07 smithi131 ceph-mon[37260]: pgmap v268: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T22:06:08.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.140+0000 7f3d13400700 1 --2- 172.21.15.131:0/3401884003 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f3d000a11b0 0x7f3d000a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:08.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.140+0000 7f3d13400700 1 -- 172.21.15.131:0/3401884003 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3d0c0ce740 con 0x7f3d1c100ff0 2024-05-08T22:06:08.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.140+0000 7f3d1a000700 1 --2- 172.21.15.131:0/3401884003 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f3d000a11b0 0x7f3d000a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:08.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.141+0000 7f3d13400700 1 --2- 172.21.15.131:0/3401884003 >> [v2:172.21.15.157:6808/1488005925,v1:172.21.15.157:6809/1488005925] conn(0x7f3d000ac1d0 0x7f3d000ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:08.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.141+0000 7f3d13400700 1 -- 172.21.15.131:0/3401884003 --> [v2:172.21.15.157:6808/1488005925,v1:172.21.15.157:6809/1488005925] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3d000aed40 con 0x7f3d000ac1d0 2024-05-08T22:06:08.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.141+0000 7f3d13400700 1 -- 172.21.15.131:0/3401884003 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d0c0ceab0 con 0x7f3d1c100ff0 2024-05-08T22:06:08.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.141+0000 7f3d1b400700 1 --2- 172.21.15.131:0/3401884003 >> [v2:172.21.15.157:6808/1488005925,v1:172.21.15.157:6809/1488005925] conn(0x7f3d000ac1d0 0x7f3d000ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:08.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.141+0000 7f3d1a000700 1 --2- 172.21.15.131:0/3401884003 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f3d000a11b0 0x7f3d000a3670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c19ab50 tx=0x7f3d08009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:08.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.141+0000 7f3d1b400700 1 --2- 172.21.15.131:0/3401884003 >> [v2:172.21.15.157:6808/1488005925,v1:172.21.15.157:6809/1488005925] conn(0x7f3d000ac1d0 0x7f3d000ae670 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:08.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.142+0000 7f3d13400700 1 -- 172.21.15.131:0/3401884003 <== osd.5 v2:172.21.15.157:6808/1488005925 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f3d000aed40 con 0x7f3d000ac1d0 2024-05-08T22:06:08.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.159+0000 7f3d21c00700 1 -- 172.21.15.131:0/3401884003 --> [v2:172.21.15.157:6808/1488005925,v1:172.21.15.157:6809/1488005925] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3d1c19ed70 con 0x7f3d000ac1d0 2024-05-08T22:06:08.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.160+0000 7f3d13400700 1 -- 172.21.15.131:0/3401884003 <== osd.5 v2:172.21.15.157:6808/1488005925 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3d1400f040 con 0x7f3d000ac1d0 2024-05-08T22:06:08.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.160+0000 7f3d21c00700 1 -- 172.21.15.131:0/3401884003 >> [v2:172.21.15.157:6808/1488005925,v1:172.21.15.157:6809/1488005925] conn(0x7f3d000ac1d0 msgr2=0x7f3d000ae670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:08.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.161+0000 7f3d21c00700 1 --2- 172.21.15.131:0/3401884003 >> [v2:172.21.15.157:6808/1488005925,v1:172.21.15.157:6809/1488005925] conn(0x7f3d000ac1d0 0x7f3d000ae670 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:08.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.161+0000 7f3d21c00700 1 -- 172.21.15.131:0/3401884003 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f3d000a11b0 msgr2=0x7f3d000a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:08.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.161+0000 7f3d21c00700 1 --2- 172.21.15.131:0/3401884003 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f3d000a11b0 0x7f3d000a3670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c19ab50 tx=0x7f3d08009260 comp rx=0 tx=0).stop 2024-05-08T22:06:08.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.161+0000 7f3d21c00700 1 -- 172.21.15.131:0/3401884003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d1c100ff0 msgr2=0x7f3d1c1995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:08.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.161+0000 7f3d21c00700 1 --2- 172.21.15.131:0/3401884003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d1c100ff0 0x7f3d1c1995f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c005910 tx=0x7f3d0c004730 comp rx=0 tx=0).stop 2024-05-08T22:06:08.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.161+0000 7f3d1b400700 1 -- 172.21.15.131:0/3401884003 reap_dead start 2024-05-08T22:06:08.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.161+0000 7f3d21c00700 1 -- 172.21.15.131:0/3401884003 shutdown_connections 2024-05-08T22:06:08.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.161+0000 7f3d21c00700 1 -- 172.21.15.131:0/3401884003 >> 172.21.15.131:0/3401884003 conn(0x7f3d1c078190 msgr2=0x7f3d1c10c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:08.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.161+0000 7f3d21c00700 1 -- 172.21.15.131:0/3401884003 shutdown_connections 2024-05-08T22:06:08.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:08.161+0000 7f3d21c00700 1 -- 172.21.15.131:0/3401884003 wait complete. 2024-05-08T22:06:08.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:07 smithi157 ceph-mon[77746]: pgmap v268: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T22:06:08.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:07 smithi132 ceph-mon[78180]: pgmap v268: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T22:06:09.540 INFO:teuthology.orchestra.run.smithi131.stdout:55834574908 2024-05-08T22:06:09.540 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-08T22:06:10.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.764+0000 7f0491c00700 1 -- 172.21.15.131:0/4112182041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f048c068950 msgr2=0x7f048c068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:10.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.764+0000 7f0491c00700 1 --2- 172.21.15.131:0/4112182041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f048c068950 0x7f048c068db0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0478009990 tx=0x7f0478009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:10.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.764+0000 7f0491c00700 1 -- 172.21.15.131:0/4112182041 shutdown_connections 2024-05-08T22:06:10.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.764+0000 7f0491c00700 1 --2- 172.21.15.131:0/4112182041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f048c0692f0 0x7f048c10de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:10.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.764+0000 7f0491c00700 1 --2- 172.21.15.131:0/4112182041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f048c107020 0x7f048c107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:10.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.764+0000 7f0491c00700 1 --2- 172.21.15.131:0/4112182041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f048c068950 0x7f048c068db0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:10.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.764+0000 7f0491c00700 1 -- 172.21.15.131:0/4112182041 >> 172.21.15.131:0/4112182041 conn(0x7f048c075f80 msgr2=0x7f048c076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:10.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.764+0000 7f0491c00700 1 -- 172.21.15.131:0/4112182041 shutdown_connections 2024-05-08T22:06:10.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.764+0000 7f0491c00700 1 -- 172.21.15.131:0/4112182041 wait complete. 2024-05-08T22:06:10.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.765+0000 7f0491c00700 1 Processor -- start 2024-05-08T22:06:10.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.765+0000 7f0491c00700 1 -- start start 2024-05-08T22:06:10.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.765+0000 7f0491c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f048c068950 0x7f048c0fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:10.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.765+0000 7f0491c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f048c0692f0 0x7f048c1004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:10.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.765+0000 7f0491c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f048c107020 0x7f048c100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:10.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.765+0000 7f0491c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c1a5750 con 0x7f048c0692f0 2024-05-08T22:06:10.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.765+0000 7f0491c00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c1a58c0 con 0x7f048c107020 2024-05-08T22:06:10.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.765+0000 7f0491c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c1a5a80 con 0x7f048c068950 2024-05-08T22:06:10.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.766+0000 7f048aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f048c068950 0x7f048c0fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:10.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.766+0000 7f048a000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f048c0692f0 0x7f048c1004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:10.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.766+0000 7f048aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f048c068950 0x7f048c0fffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:42436/0 (socket says 172.21.15.131:42436) 2024-05-08T22:06:10.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.766+0000 7f048aa00700 1 -- 172.21.15.131:0/3710845355 learned_addr learned my addr 172.21.15.131:0/3710845355 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:10.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.766+0000 7f048b400700 1 --2- 172.21.15.131:0/3710845355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f048c107020 0x7f048c100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:10.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.766+0000 7f048aa00700 1 -- 172.21.15.131:0/3710845355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f048c107020 msgr2=0x7f048c100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:10.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.766+0000 7f048aa00700 1 --2- 172.21.15.131:0/3710845355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f048c107020 0x7f048c100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:10.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.766+0000 7f048aa00700 1 -- 172.21.15.131:0/3710845355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f048c0692f0 msgr2=0x7f048c1004e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:10.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.766+0000 7f048aa00700 1 --2- 172.21.15.131:0/3710845355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f048c0692f0 0x7f048c1004e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:10.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.766+0000 7f048aa00700 1 -- 172.21.15.131:0/3710845355 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c009520 con 0x7f048c068950 2024-05-08T22:06:10.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.767+0000 7f048aa00700 1 --2- 172.21.15.131:0/3710845355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f048c068950 0x7f048c0fffa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f047c00eb30 tx=0x7f047c00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:10.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.767+0000 7f0483400700 1 -- 172.21.15.131:0/3710845355 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c009880 con 0x7f048c068950 2024-05-08T22:06:10.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.767+0000 7f0483400700 1 -- 172.21.15.131:0/3710845355 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f047c0044e0 con 0x7f048c068950 2024-05-08T22:06:10.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.767+0000 7f0483400700 1 -- 172.21.15.131:0/3710845355 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c003d80 con 0x7f048c068950 2024-05-08T22:06:10.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.767+0000 7f0491c00700 1 -- 172.21.15.131:0/3710845355 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04780095f0 con 0x7f048c068950 2024-05-08T22:06:10.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.767+0000 7f0491c00700 1 -- 172.21.15.131:0/3710845355 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048c1a6050 con 0x7f048c068950 2024-05-08T22:06:10.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.768+0000 7f0491c00700 1 -- 172.21.15.131:0/3710845355 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f048c1026d0 con 0x7f048c068950 2024-05-08T22:06:10.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.770+0000 7f0483400700 1 -- 172.21.15.131:0/3710845355 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f047c003ee0 con 0x7f048c068950 2024-05-08T22:06:10.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.771+0000 7f0483400700 1 --2- 172.21.15.131:0/3710845355 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f04700a11b0 0x7f04700a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:10.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.772+0000 7f0483400700 1 -- 172.21.15.131:0/3710845355 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f047c014070 con 0x7f048c068950 2024-05-08T22:06:10.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.772+0000 7f048a000700 1 --2- 172.21.15.131:0/3710845355 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f04700a11b0 0x7f04700a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.772+0000 7f0483400700 1 --2- 172.21.15.131:0/3710845355 >> [v2:172.21.15.132:6824/306651644,v1:172.21.15.132:6825/306651644] conn(0x7f04700ac1d0 0x7f04700ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.772+0000 7f0483400700 1 -- 172.21.15.131:0/3710845355 --> [v2:172.21.15.132:6824/306651644,v1:172.21.15.132:6825/306651644] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f04700aed40 con 0x7f04700ac1d0 2024-05-08T22:06:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.772+0000 7f0483400700 1 -- 172.21.15.131:0/3710845355 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f047c0cd760 con 0x7f048c068950 2024-05-08T22:06:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.772+0000 7f048b400700 1 --2- 172.21.15.131:0/3710845355 >> [v2:172.21.15.132:6824/306651644,v1:172.21.15.132:6825/306651644] conn(0x7f04700ac1d0 0x7f04700ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.773+0000 7f048a000700 1 --2- 172.21.15.131:0/3710845355 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f04700a11b0 0x7f04700a3670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f04780098e0 tx=0x7f0478005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.773+0000 7f048b400700 1 --2- 172.21.15.131:0/3710845355 >> [v2:172.21.15.132:6824/306651644,v1:172.21.15.132:6825/306651644] conn(0x7f04700ac1d0 0x7f04700ae670 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.774+0000 7f0483400700 1 -- 172.21.15.131:0/3710845355 <== osd.10 v2:172.21.15.132:6824/306651644 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f04700aed40 con 0x7f04700ac1d0 2024-05-08T22:06:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.791+0000 7f0491c00700 1 -- 172.21.15.131:0/3710845355 --> [v2:172.21.15.132:6824/306651644,v1:172.21.15.132:6825/306651644] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f048c111120 con 0x7f04700ac1d0 2024-05-08T22:06:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.792+0000 7f0483400700 1 -- 172.21.15.131:0/3710845355 <== osd.10 v2:172.21.15.132:6824/306651644 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f048c111120 con 0x7f04700ac1d0 2024-05-08T22:06:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.793+0000 7f0491c00700 1 -- 172.21.15.131:0/3710845355 >> [v2:172.21.15.132:6824/306651644,v1:172.21.15.132:6825/306651644] conn(0x7f04700ac1d0 msgr2=0x7f04700ae670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:10.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.793+0000 7f0491c00700 1 --2- 172.21.15.131:0/3710845355 >> [v2:172.21.15.132:6824/306651644,v1:172.21.15.132:6825/306651644] conn(0x7f04700ac1d0 0x7f04700ae670 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:10.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.793+0000 7f0491c00700 1 -- 172.21.15.131:0/3710845355 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f04700a11b0 msgr2=0x7f04700a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:10.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.793+0000 7f0491c00700 1 --2- 172.21.15.131:0/3710845355 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f04700a11b0 0x7f04700a3670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f04780098e0 tx=0x7f0478005960 comp rx=0 tx=0).stop 2024-05-08T22:06:10.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.793+0000 7f0491c00700 1 -- 172.21.15.131:0/3710845355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f048c068950 msgr2=0x7f048c0fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:10.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.793+0000 7f0491c00700 1 --2- 172.21.15.131:0/3710845355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f048c068950 0x7f048c0fffa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f047c00eb30 tx=0x7f047c00ee40 comp rx=0 tx=0).stop 2024-05-08T22:06:10.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.793+0000 7f048b400700 1 -- 172.21.15.131:0/3710845355 reap_dead start 2024-05-08T22:06:10.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.793+0000 7f0491c00700 1 -- 172.21.15.131:0/3710845355 shutdown_connections 2024-05-08T22:06:10.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.793+0000 7f0491c00700 1 -- 172.21.15.131:0/3710845355 >> 172.21.15.131:0/3710845355 conn(0x7f048c075f80 msgr2=0x7f048c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:10.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.793+0000 7f0491c00700 1 -- 172.21.15.131:0/3710845355 shutdown_connections 2024-05-08T22:06:10.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:09.793+0000 7f0491c00700 1 -- 172.21.15.131:0/3710845355 wait complete. 2024-05-08T22:06:10.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:09 smithi131 ceph-mon[37260]: pgmap v269: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-08T22:06:10.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.122+0000 7f8784800700 1 -- 172.21.15.131:0/3729614364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f877c107030 msgr2=0x7f877c107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:10.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.123+0000 7f8784800700 1 --2- 172.21.15.131:0/3729614364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f877c107030 0x7f877c107410 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8770009990 tx=0x7f8770009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:10.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.123+0000 7f8784800700 1 -- 172.21.15.131:0/3729614364 shutdown_connections 2024-05-08T22:06:10.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.123+0000 7f8784800700 1 --2- 172.21.15.131:0/3729614364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f877c101990 0x7f877c10de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:10.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.123+0000 7f8784800700 1 --2- 172.21.15.131:0/3729614364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f877c107030 0x7f877c107410 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:10.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.123+0000 7f8784800700 1 --2- 172.21.15.131:0/3729614364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f877c100ff0 0x7f877c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:10.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.123+0000 7f8784800700 1 -- 172.21.15.131:0/3729614364 >> 172.21.15.131:0/3729614364 conn(0x7f877c078190 msgr2=0x7f877c0785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:10.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.123+0000 7f8784800700 1 -- 172.21.15.131:0/3729614364 shutdown_connections 2024-05-08T22:06:10.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.123+0000 7f8784800700 1 -- 172.21.15.131:0/3729614364 wait complete. 2024-05-08T22:06:10.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.124+0000 7f8784800700 1 Processor -- start 2024-05-08T22:06:10.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.124+0000 7f8784800700 1 -- start start 2024-05-08T22:06:10.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.124+0000 7f8784800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f877c100ff0 0x7f877c199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:10.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.124+0000 7f8784800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f877c101990 0x7f877c199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:10.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.124+0000 7f8784800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f877c107030 0x7f877c19df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:10.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.124+0000 7f8784800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c19e510 con 0x7f877c100ff0 2024-05-08T22:06:10.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.124+0000 7f8784800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c19e680 con 0x7f877c107030 2024-05-08T22:06:10.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.124+0000 7f8784800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c19e840 con 0x7f877c101990 2024-05-08T22:06:10.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.125+0000 7f8781e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f877c100ff0 0x7f877c199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:10.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.125+0000 7f8781e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f877c100ff0 0x7f877c199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34746/0 (socket says 172.21.15.131:34746) 2024-05-08T22:06:10.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.125+0000 7f8781e00700 1 -- 172.21.15.131:0/1850670044 learned_addr learned my addr 172.21.15.131:0/1850670044 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:10.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.125+0000 7f8782800700 1 --2- 172.21.15.131:0/1850670044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f877c107030 0x7f877c19df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:10.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.125+0000 7f8781400700 1 --2- 172.21.15.131:0/1850670044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f877c101990 0x7f877c199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:10.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.125+0000 7f8781e00700 1 -- 172.21.15.131:0/1850670044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f877c101990 msgr2=0x7f877c199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:10.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.125+0000 7f8781e00700 1 --2- 172.21.15.131:0/1850670044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f877c101990 0x7f877c199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:10.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.125+0000 7f8781e00700 1 -- 172.21.15.131:0/1850670044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f877c107030 msgr2=0x7f877c19df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:10.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.125+0000 7f8781e00700 1 --2- 172.21.15.131:0/1850670044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f877c107030 0x7f877c19df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:10.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.125+0000 7f8781e00700 1 -- 172.21.15.131:0/1850670044 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87700095f0 con 0x7f877c100ff0 2024-05-08T22:06:10.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.125+0000 7f8781e00700 1 --2- 172.21.15.131:0/1850670044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f877c100ff0 0x7f877c199610 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8770000c00 tx=0x7f8770005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:10.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.125+0000 7f8776a00700 1 -- 172.21.15.131:0/1850670044 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877001d070 con 0x7f877c100ff0 2024-05-08T22:06:10.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.126+0000 7f8776a00700 1 -- 172.21.15.131:0/1850670044 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8770022470 con 0x7f877c100ff0 2024-05-08T22:06:10.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.126+0000 7f8784800700 1 -- 172.21.15.131:0/1850670044 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f877c19eac0 con 0x7f877c100ff0 2024-05-08T22:06:10.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.126+0000 7f8776a00700 1 -- 172.21.15.131:0/1850670044 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8770017650 con 0x7f877c100ff0 2024-05-08T22:06:10.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.126+0000 7f8784800700 1 -- 172.21.15.131:0/1850670044 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f877c1a6380 con 0x7f877c100ff0 2024-05-08T22:06:10.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.127+0000 7f8784800700 1 -- 172.21.15.131:0/1850670044 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f877c1111c0 con 0x7f877c100ff0 2024-05-08T22:06:10.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.129+0000 7f8776a00700 1 -- 172.21.15.131:0/1850670044 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f87700225e0 con 0x7f877c100ff0 2024-05-08T22:06:10.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.130+0000 7f8776a00700 1 --2- 172.21.15.131:0/1850670044 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f87640a11b0 0x7f87640a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:10.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.130+0000 7f8776a00700 1 -- 172.21.15.131:0/1850670044 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f87700cea80 con 0x7f877c100ff0 2024-05-08T22:06:10.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.130+0000 7f8781400700 1 --2- 172.21.15.131:0/1850670044 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f87640a11b0 0x7f87640a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:10.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.130+0000 7f8776a00700 1 --2- 172.21.15.131:0/1850670044 >> [v2:172.21.15.157:6816/3594097089,v1:172.21.15.157:6817/3594097089] conn(0x7f87640ac1d0 0x7f87640ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:10.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.130+0000 7f8776a00700 1 -- 172.21.15.131:0/1850670044 --> [v2:172.21.15.157:6816/3594097089,v1:172.21.15.157:6817/3594097089] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f87640aed40 con 0x7f87640ac1d0 2024-05-08T22:06:10.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.130+0000 7f8776a00700 1 -- 172.21.15.131:0/1850670044 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87700cedf0 con 0x7f877c100ff0 2024-05-08T22:06:10.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.130+0000 7f8782800700 1 --2- 172.21.15.131:0/1850670044 >> [v2:172.21.15.157:6816/3594097089,v1:172.21.15.157:6817/3594097089] conn(0x7f87640ac1d0 0x7f87640ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:10.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.131+0000 7f8781400700 1 --2- 172.21.15.131:0/1850670044 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f87640a11b0 0x7f87640a3670 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f877c19ab70 tx=0x7f876c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:10.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.131+0000 7f8782800700 1 --2- 172.21.15.131:0/1850670044 >> [v2:172.21.15.157:6816/3594097089,v1:172.21.15.157:6817/3594097089] conn(0x7f87640ac1d0 0x7f87640ae670 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:10.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.132+0000 7f8776a00700 1 -- 172.21.15.131:0/1850670044 <== osd.8 v2:172.21.15.157:6816/3594097089 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f87640aed40 con 0x7f87640ac1d0 2024-05-08T22:06:10.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.149+0000 7f8784800700 1 -- 172.21.15.131:0/1850670044 --> [v2:172.21.15.157:6816/3594097089,v1:172.21.15.157:6817/3594097089] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f877c19ed90 con 0x7f87640ac1d0 2024-05-08T22:06:10.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.150+0000 7f8776a00700 1 -- 172.21.15.131:0/1850670044 <== osd.8 v2:172.21.15.157:6816/3594097089 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f877800f040 con 0x7f87640ac1d0 2024-05-08T22:06:10.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.150+0000 7f8784800700 1 -- 172.21.15.131:0/1850670044 >> [v2:172.21.15.157:6816/3594097089,v1:172.21.15.157:6817/3594097089] conn(0x7f87640ac1d0 msgr2=0x7f87640ae670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:10.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.150+0000 7f8784800700 1 --2- 172.21.15.131:0/1850670044 >> [v2:172.21.15.157:6816/3594097089,v1:172.21.15.157:6817/3594097089] conn(0x7f87640ac1d0 0x7f87640ae670 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:10.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.150+0000 7f8784800700 1 -- 172.21.15.131:0/1850670044 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f87640a11b0 msgr2=0x7f87640a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:10.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.150+0000 7f8784800700 1 --2- 172.21.15.131:0/1850670044 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f87640a11b0 0x7f87640a3670 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f877c19ab70 tx=0x7f876c009260 comp rx=0 tx=0).stop 2024-05-08T22:06:10.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.150+0000 7f8784800700 1 -- 172.21.15.131:0/1850670044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f877c100ff0 msgr2=0x7f877c199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:10.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.150+0000 7f8784800700 1 --2- 172.21.15.131:0/1850670044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f877c100ff0 0x7f877c199610 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8770000c00 tx=0x7f8770005740 comp rx=0 tx=0).stop 2024-05-08T22:06:10.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.150+0000 7f8782800700 1 -- 172.21.15.131:0/1850670044 reap_dead start 2024-05-08T22:06:10.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.150+0000 7f8784800700 1 -- 172.21.15.131:0/1850670044 shutdown_connections 2024-05-08T22:06:10.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.150+0000 7f8784800700 1 -- 172.21.15.131:0/1850670044 >> 172.21.15.131:0/1850670044 conn(0x7f877c078190 msgr2=0x7f877c0ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:10.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.151+0000 7f8784800700 1 -- 172.21.15.131:0/1850670044 shutdown_connections 2024-05-08T22:06:10.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:10.151+0000 7f8784800700 1 -- 172.21.15.131:0/1850670044 wait complete. 2024-05-08T22:06:10.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:09 smithi157 ceph-mon[77746]: pgmap v269: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-08T22:06:10.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:09 smithi132 ceph-mon[78180]: pgmap v269: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-08T22:06:10.467 INFO:teuthology.orchestra.run.smithi131.stdout:154618822701 2024-05-08T22:06:10.468 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-08T22:06:10.641 INFO:teuthology.orchestra.run.smithi131.stdout:266287972381 2024-05-08T22:06:10.641 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-08T22:06:10.892 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:11.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.300+0000 7fb583200700 1 -- 172.21.15.131:0/3157076671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb57c111c30 msgr2=0x7fb57c114040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:11.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.300+0000 7fb583200700 1 --2- 172.21.15.131:0/3157076671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb57c111c30 0x7fb57c114040 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb57400b0a0 tx=0x7fb57400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:06:11.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.300+0000 7fb583200700 1 -- 172.21.15.131:0/3157076671 shutdown_connections 2024-05-08T22:06:11.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.300+0000 7fb583200700 1 --2- 172.21.15.131:0/3157076671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb57c111c30 0x7fb57c114040 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:11.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.300+0000 7fb583200700 1 --2- 172.21.15.131:0/3157076671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb57c075000 0x7fb57c0731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:11.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.300+0000 7fb583200700 1 --2- 172.21.15.131:0/3157076671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb57c073710 0x7fb57c073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:11.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.300+0000 7fb583200700 1 -- 172.21.15.131:0/3157076671 >> 172.21.15.131:0/3157076671 conn(0x7fb57c078190 msgr2=0x7fb57c0785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:11.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.300+0000 7fb583200700 1 -- 172.21.15.131:0/3157076671 shutdown_connections 2024-05-08T22:06:11.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.300+0000 7fb583200700 1 -- 172.21.15.131:0/3157076671 wait complete. 2024-05-08T22:06:11.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.301+0000 7fb583200700 1 Processor -- start 2024-05-08T22:06:11.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.301+0000 7fb583200700 1 -- start start 2024-05-08T22:06:11.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.301+0000 7fb583200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb57c073710 0x7fb57c1a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:11.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.301+0000 7fb583200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb57c075000 0x7fb57c1a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:11.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.301+0000 7fb583200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb57c111c30 0x7fb57c19d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:11.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.301+0000 7fb583200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c19dff0 con 0x7fb57c075000 2024-05-08T22:06:11.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.301+0000 7fb583200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c19e160 con 0x7fb57c073710 2024-05-08T22:06:11.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.301+0000 7fb583200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c19e320 con 0x7fb57c111c30 2024-05-08T22:06:11.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.302+0000 7fb57b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb57c075000 0x7fb57c1a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:11.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.302+0000 7fb57b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb57c075000 0x7fb57c1a3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34782/0 (socket says 172.21.15.131:34782) 2024-05-08T22:06:11.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.302+0000 7fb57b400700 1 -- 172.21.15.131:0/3975632361 learned_addr learned my addr 172.21.15.131:0/3975632361 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:11.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.302+0000 7fb581200700 1 --2- 172.21.15.131:0/3975632361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb57c111c30 0x7fb57c19d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:11.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.302+0000 7fb57be00700 1 --2- 172.21.15.131:0/3975632361 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb57c073710 0x7fb57c1a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:11.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.302+0000 7fb57b400700 1 -- 172.21.15.131:0/3975632361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb57c111c30 msgr2=0x7fb57c19d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:11.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.302+0000 7fb57b400700 1 --2- 172.21.15.131:0/3975632361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb57c111c30 0x7fb57c19d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:11.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.302+0000 7fb57b400700 1 -- 172.21.15.131:0/3975632361 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb57c073710 msgr2=0x7fb57c1a3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:11.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.302+0000 7fb57b400700 1 --2- 172.21.15.131:0/3975632361 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb57c073710 0x7fb57c1a3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:11.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.302+0000 7fb57b400700 1 -- 172.21.15.131:0/3975632361 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb574009d00 con 0x7fb57c075000 2024-05-08T22:06:11.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.303+0000 7fb57b400700 1 --2- 172.21.15.131:0/3975632361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb57c075000 0x7fb57c1a3d90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb56800e9a0 tx=0x7fb56800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:11.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.303+0000 7fb578c00700 1 -- 172.21.15.131:0/3975632361 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb568009790 con 0x7fb57c075000 2024-05-08T22:06:11.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.303+0000 7fb578c00700 1 -- 172.21.15.131:0/3975632361 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb568010430 con 0x7fb57c075000 2024-05-08T22:06:11.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.303+0000 7fb583200700 1 -- 172.21.15.131:0/3975632361 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb57c19e600 con 0x7fb57c075000 2024-05-08T22:06:11.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.303+0000 7fb583200700 1 -- 172.21.15.131:0/3975632361 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb57c1aa840 con 0x7fb57c075000 2024-05-08T22:06:11.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.304+0000 7fb578c00700 1 -- 172.21.15.131:0/3975632361 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb568018680 con 0x7fb57c075000 2024-05-08T22:06:11.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.304+0000 7fb583200700 1 -- 172.21.15.131:0/3975632361 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb57c1177f0 con 0x7fb57c075000 2024-05-08T22:06:11.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.306+0000 7fb578c00700 1 -- 172.21.15.131:0/3975632361 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fb5680105a0 con 0x7fb57c075000 2024-05-08T22:06:11.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.307+0000 7fb578c00700 1 --2- 172.21.15.131:0/3975632361 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb5600a1230 0x7fb5600a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:11.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.307+0000 7fb578c00700 1 -- 172.21.15.131:0/3975632361 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb568014070 con 0x7fb57c075000 2024-05-08T22:06:11.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.307+0000 7fb57be00700 1 --2- 172.21.15.131:0/3975632361 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb5600a1230 0x7fb5600a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:11.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.308+0000 7fb578c00700 1 --2- 172.21.15.131:0/3975632361 >> [v2:172.21.15.131:6802/206657594,v1:172.21.15.131:6803/206657594] conn(0x7fb5600ac250 0x7fb5600ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:11.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.308+0000 7fb578c00700 1 -- 172.21.15.131:0/3975632361 --> [v2:172.21.15.131:6802/206657594,v1:172.21.15.131:6803/206657594] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb5600aedc0 con 0x7fb5600ac250 2024-05-08T22:06:11.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.308+0000 7fb578c00700 1 -- 172.21.15.131:0/3975632361 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5680cf340 con 0x7fb57c075000 2024-05-08T22:06:11.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.308+0000 7fb581200700 1 --2- 172.21.15.131:0/3975632361 >> [v2:172.21.15.131:6802/206657594,v1:172.21.15.131:6803/206657594] conn(0x7fb5600ac250 0x7fb5600ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:11.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.308+0000 7fb57be00700 1 --2- 172.21.15.131:0/3975632361 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb5600a1230 0x7fb5600a36f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009be0 tx=0x7fb56c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:11.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.308+0000 7fb581200700 1 --2- 172.21.15.131:0/3975632361 >> [v2:172.21.15.131:6802/206657594,v1:172.21.15.131:6803/206657594] conn(0x7fb5600ac250 0x7fb5600ae6f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:11.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.309+0000 7fb578c00700 1 -- 172.21.15.131:0/3975632361 <== osd.0 v2:172.21.15.131:6802/206657594 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fb5600aedc0 con 0x7fb5600ac250 2024-05-08T22:06:11.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.325+0000 7fb583200700 1 -- 172.21.15.131:0/3975632361 --> [v2:172.21.15.131:6802/206657594,v1:172.21.15.131:6803/206657594] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb57c111340 con 0x7fb5600ac250 2024-05-08T22:06:11.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.326+0000 7fb578c00700 1 -- 172.21.15.131:0/3975632361 <== osd.0 v2:172.21.15.131:6802/206657594 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb57c111340 con 0x7fb5600ac250 2024-05-08T22:06:11.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.326+0000 7fb583200700 1 -- 172.21.15.131:0/3975632361 >> [v2:172.21.15.131:6802/206657594,v1:172.21.15.131:6803/206657594] conn(0x7fb5600ac250 msgr2=0x7fb5600ae6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:11.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.326+0000 7fb583200700 1 --2- 172.21.15.131:0/3975632361 >> [v2:172.21.15.131:6802/206657594,v1:172.21.15.131:6803/206657594] conn(0x7fb5600ac250 0x7fb5600ae6f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:11.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.326+0000 7fb583200700 1 -- 172.21.15.131:0/3975632361 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb5600a1230 msgr2=0x7fb5600a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:11.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.326+0000 7fb583200700 1 --2- 172.21.15.131:0/3975632361 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb5600a1230 0x7fb5600a36f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009be0 tx=0x7fb56c009260 comp rx=0 tx=0).stop 2024-05-08T22:06:11.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.326+0000 7fb583200700 1 -- 172.21.15.131:0/3975632361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb57c075000 msgr2=0x7fb57c1a3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:11.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.326+0000 7fb583200700 1 --2- 172.21.15.131:0/3975632361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb57c075000 0x7fb57c1a3d90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb56800e9a0 tx=0x7fb56800ecb0 comp rx=0 tx=0).stop 2024-05-08T22:06:11.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.326+0000 7fb581200700 1 -- 172.21.15.131:0/3975632361 reap_dead start 2024-05-08T22:06:11.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.326+0000 7fb583200700 1 -- 172.21.15.131:0/3975632361 shutdown_connections 2024-05-08T22:06:11.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.326+0000 7fb583200700 1 -- 172.21.15.131:0/3975632361 >> 172.21.15.131:0/3975632361 conn(0x7fb57c078190 msgr2=0x7fb57c1109f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:11.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.326+0000 7fb583200700 1 -- 172.21.15.131:0/3975632361 shutdown_connections 2024-05-08T22:06:11.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.327+0000 7fb583200700 1 -- 172.21.15.131:0/3975632361 wait complete. 2024-05-08T22:06:11.451 INFO:teuthology.orchestra.run.smithi131.stdout:287762808857 2024-05-08T22:06:11.451 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-08T22:06:11.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.624+0000 7f44ae600700 1 -- 172.21.15.131:0/563105993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44a8107590 msgr2=0x7f44a8107a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:11.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.624+0000 7f44ae600700 1 --2- 172.21.15.131:0/563105993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44a8107590 0x7f44a8107a10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4494009990 tx=0x7f4494009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:11.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.624+0000 7f44ae600700 1 -- 172.21.15.131:0/563105993 shutdown_connections 2024-05-08T22:06:11.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.624+0000 7f44ae600700 1 --2- 172.21.15.131:0/563105993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44a8107f50 0x7f44a8114410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:11.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.624+0000 7f44ae600700 1 --2- 172.21.15.131:0/563105993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44a810d640 0x7f44a810da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:11.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.624+0000 7f44ae600700 1 --2- 172.21.15.131:0/563105993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44a8107590 0x7f44a8107a10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:11.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.624+0000 7f44ae600700 1 -- 172.21.15.131:0/563105993 >> 172.21.15.131:0/563105993 conn(0x7f44a80746f0 msgr2=0x7f44a8076b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:11.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.625+0000 7f44ae600700 1 -- 172.21.15.131:0/563105993 shutdown_connections 2024-05-08T22:06:11.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.625+0000 7f44ae600700 1 -- 172.21.15.131:0/563105993 wait complete. 2024-05-08T22:06:11.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.626+0000 7f44ae600700 1 Processor -- start 2024-05-08T22:06:11.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.626+0000 7f44ae600700 1 -- start start 2024-05-08T22:06:11.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.626+0000 7f44ae600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44a8107590 0x7f44a81a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:11.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.626+0000 7f44ae600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44a8107f50 0x7f44a81a3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:11.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.626+0000 7f44ae600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44a810d640 0x7f44a819d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:11.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.626+0000 7f44ae600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a819de10 con 0x7f44a8107590 2024-05-08T22:06:11.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.626+0000 7f44ae600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a819df80 con 0x7f44a810d640 2024-05-08T22:06:11.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.626+0000 7f44ae600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a819e140 con 0x7f44a8107f50 2024-05-08T22:06:11.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.627+0000 7f44a7e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44a810d640 0x7f44a819d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:11.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.627+0000 7f44a6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44a8107f50 0x7f44a81a3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:11.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.627+0000 7f44a7e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44a810d640 0x7f44a819d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:39192/0 (socket says 172.21.15.131:39192) 2024-05-08T22:06:11.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.627+0000 7f44a7e00700 1 -- 172.21.15.131:0/8381975 learned_addr learned my addr 172.21.15.131:0/8381975 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:11.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.627+0000 7f44a7e00700 1 -- 172.21.15.131:0/8381975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44a8107f50 msgr2=0x7f44a81a3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:11.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.627+0000 7f44a7e00700 1 --2- 172.21.15.131:0/8381975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44a8107f50 0x7f44a81a3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:11.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.627+0000 7f44a7e00700 1 -- 172.21.15.131:0/8381975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44a8107590 msgr2=0x7f44a81a3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:11.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.627+0000 7f44a7e00700 1 --2- 172.21.15.131:0/8381975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44a8107590 0x7f44a81a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:11.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.627+0000 7f44a7e00700 1 -- 172.21.15.131:0/8381975 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44940095f0 con 0x7f44a810d640 2024-05-08T22:06:11.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.628+0000 7f44a7e00700 1 --2- 172.21.15.131:0/8381975 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44a810d640 0x7f44a819d7a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f44a0007ee0 tx=0x7f44a000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:11.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.628+0000 7f449fe00700 1 -- 172.21.15.131:0/8381975 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a0017070 con 0x7f44a810d640 2024-05-08T22:06:11.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.628+0000 7f449fe00700 1 -- 172.21.15.131:0/8381975 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44a000f040 con 0x7f44a810d640 2024-05-08T22:06:11.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.628+0000 7f449fe00700 1 -- 172.21.15.131:0/8381975 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a0005740 con 0x7f44a810d640 2024-05-08T22:06:11.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.628+0000 7f44ae600700 1 -- 172.21.15.131:0/8381975 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a819e420 con 0x7f44a810d640 2024-05-08T22:06:11.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.628+0000 7f44ae600700 1 -- 172.21.15.131:0/8381975 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a81aa650 con 0x7f44a810d640 2024-05-08T22:06:11.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.629+0000 7f44ae600700 1 -- 172.21.15.131:0/8381975 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f44a81177c0 con 0x7f44a810d640 2024-05-08T22:06:11.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.631+0000 7f449fe00700 1 -- 172.21.15.131:0/8381975 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f44a000dc60 con 0x7f44a810d640 2024-05-08T22:06:11.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.632+0000 7f449fe00700 1 --2- 172.21.15.131:0/8381975 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f448c0a1230 0x7f448c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:11.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.632+0000 7f449fe00700 1 -- 172.21.15.131:0/8381975 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f44a00cc600 con 0x7f44a810d640 2024-05-08T22:06:11.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.633+0000 7f44a7400700 1 --2- 172.21.15.131:0/8381975 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f448c0a1230 0x7f448c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:11.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.633+0000 7f449fe00700 1 --2- 172.21.15.131:0/8381975 >> [v2:172.21.15.132:6816/464003238,v1:172.21.15.132:6817/464003238] conn(0x7f448c0ac250 0x7f448c0ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:11.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.633+0000 7f449fe00700 1 -- 172.21.15.131:0/8381975 --> [v2:172.21.15.132:6816/464003238,v1:172.21.15.132:6817/464003238] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f448c0aedc0 con 0x7f448c0ac250 2024-05-08T22:06:11.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.633+0000 7f449fe00700 1 -- 172.21.15.131:0/8381975 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44a00cc970 con 0x7f44a810d640 2024-05-08T22:06:11.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.633+0000 7f44a6a00700 1 --2- 172.21.15.131:0/8381975 >> [v2:172.21.15.132:6816/464003238,v1:172.21.15.132:6817/464003238] conn(0x7f448c0ac250 0x7f448c0ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:11.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.633+0000 7f44a7400700 1 --2- 172.21.15.131:0/8381975 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f448c0a1230 0x7f448c0a36f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4498005fd0 tx=0x7f4498005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:11.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.634+0000 7f44a6a00700 1 --2- 172.21.15.131:0/8381975 >> [v2:172.21.15.132:6816/464003238,v1:172.21.15.132:6817/464003238] conn(0x7f448c0ac250 0x7f448c0ae6f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:11.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.635+0000 7f449fe00700 1 -- 172.21.15.131:0/8381975 <== osd.7 v2:172.21.15.132:6816/464003238 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f448c0aedc0 con 0x7f448c0ac250 2024-05-08T22:06:11.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.652+0000 7f44ae600700 1 -- 172.21.15.131:0/8381975 --> [v2:172.21.15.132:6816/464003238,v1:172.21.15.132:6817/464003238] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f44a80616a0 con 0x7f448c0ac250 2024-05-08T22:06:11.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.652+0000 7f449fe00700 1 -- 172.21.15.131:0/8381975 <== osd.7 v2:172.21.15.132:6816/464003238 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f44a80616a0 con 0x7f448c0ac250 2024-05-08T22:06:11.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.653+0000 7f44ae600700 1 -- 172.21.15.131:0/8381975 >> [v2:172.21.15.132:6816/464003238,v1:172.21.15.132:6817/464003238] conn(0x7f448c0ac250 msgr2=0x7f448c0ae6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:11.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.653+0000 7f44ae600700 1 --2- 172.21.15.131:0/8381975 >> [v2:172.21.15.132:6816/464003238,v1:172.21.15.132:6817/464003238] conn(0x7f448c0ac250 0x7f448c0ae6f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:11.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.653+0000 7f44ae600700 1 -- 172.21.15.131:0/8381975 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f448c0a1230 msgr2=0x7f448c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:11.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.653+0000 7f44ae600700 1 --2- 172.21.15.131:0/8381975 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f448c0a1230 0x7f448c0a36f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4498005fd0 tx=0x7f4498005f60 comp rx=0 tx=0).stop 2024-05-08T22:06:11.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.653+0000 7f44ae600700 1 -- 172.21.15.131:0/8381975 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44a810d640 msgr2=0x7f44a819d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:11.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.653+0000 7f44ae600700 1 --2- 172.21.15.131:0/8381975 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44a810d640 0x7f44a819d7a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f44a0007ee0 tx=0x7f44a000d3b0 comp rx=0 tx=0).stop 2024-05-08T22:06:11.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.653+0000 7f44a7e00700 1 -- 172.21.15.131:0/8381975 reap_dead start 2024-05-08T22:06:11.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.653+0000 7f44ae600700 1 -- 172.21.15.131:0/8381975 shutdown_connections 2024-05-08T22:06:11.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.653+0000 7f44ae600700 1 -- 172.21.15.131:0/8381975 >> 172.21.15.131:0/8381975 conn(0x7f44a80746f0 msgr2=0x7f44a8075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:11.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.653+0000 7f44ae600700 1 -- 172.21.15.131:0/8381975 shutdown_connections 2024-05-08T22:06:11.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.654+0000 7f44ae600700 1 -- 172.21.15.131:0/8381975 wait complete. 2024-05-08T22:06:11.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.018+0000 7fe853000700 1 -- 172.21.15.131:0/4253113385 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe84c101020 msgr2=0x7fe84c1014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:11.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.018+0000 7fe853000700 1 --2- 172.21.15.131:0/4253113385 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe84c101020 0x7fe84c1014a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe8380098f0 tx=0x7fe838009c00 comp rx=0 tx=0).stop 2024-05-08T22:06:11.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.018+0000 7fe853000700 1 -- 172.21.15.131:0/4253113385 shutdown_connections 2024-05-08T22:06:11.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.018+0000 7fe853000700 1 --2- 172.21.15.131:0/4253113385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe84c1019e0 0x7fe84c10ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:11.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.018+0000 7fe853000700 1 --2- 172.21.15.131:0/4253113385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe84c107040 0x7fe84c107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:11.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.018+0000 7fe853000700 1 --2- 172.21.15.131:0/4253113385 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe84c101020 0x7fe84c1014a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:11.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.018+0000 7fe853000700 1 -- 172.21.15.131:0/4253113385 >> 172.21.15.131:0/4253113385 conn(0x7fe84c078100 msgr2=0x7fe84c078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:11.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.018+0000 7fe853000700 1 -- 172.21.15.131:0/4253113385 shutdown_connections 2024-05-08T22:06:11.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.018+0000 7fe853000700 1 -- 172.21.15.131:0/4253113385 wait complete. 2024-05-08T22:06:11.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.020+0000 7fe853000700 1 Processor -- start 2024-05-08T22:06:11.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.020+0000 7fe853000700 1 -- start start 2024-05-08T22:06:11.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.020+0000 7fe853000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe84c101020 0x7fe84c1995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:11.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.020+0000 7fe853000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe84c1019e0 0x7fe84c199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:11.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.020+0000 7fe853000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe84c107040 0x7fe84c19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:11.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.020+0000 7fe853000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c19e480 con 0x7fe84c101020 2024-05-08T22:06:11.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.020+0000 7fe853000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c19e5f0 con 0x7fe84c1019e0 2024-05-08T22:06:11.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.020+0000 7fe853000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c19e7b0 con 0x7fe84c107040 2024-05-08T22:06:11.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.021+0000 7fe84b400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe84c1019e0 0x7fe84c199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:11.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.021+0000 7fe84b400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe84c1019e0 0x7fe84c199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:39156/0 (socket says 172.21.15.131:39156) 2024-05-08T22:06:11.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.021+0000 7fe84b400700 1 -- 172.21.15.131:0/6214560 learned_addr learned my addr 172.21.15.131:0/6214560 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:11.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.021+0000 7fe84b400700 1 -- 172.21.15.131:0/6214560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe84c107040 msgr2=0x7fe84c19de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:11.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.021+0000 7fe84b400700 1 --2- 172.21.15.131:0/6214560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe84c107040 0x7fe84c19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:11.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.021+0000 7fe84b400700 1 -- 172.21.15.131:0/6214560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe84c101020 msgr2=0x7fe84c1995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:11.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.021+0000 7fe84b400700 1 --2- 172.21.15.131:0/6214560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe84c101020 0x7fe84c1995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:11.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.021+0000 7fe84b400700 1 -- 172.21.15.131:0/6214560 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe838009520 con 0x7fe84c1019e0 2024-05-08T22:06:11.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.021+0000 7fe84b400700 1 --2- 172.21.15.131:0/6214560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe84c1019e0 0x7fe84c199b10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe838005950 tx=0x7fe838010ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:11.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.022+0000 7fe848c00700 1 -- 172.21.15.131:0/6214560 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83801c070 con 0x7fe84c1019e0 2024-05-08T22:06:11.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.022+0000 7fe848c00700 1 -- 172.21.15.131:0/6214560 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe838005040 con 0x7fe84c1019e0 2024-05-08T22:06:11.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.022+0000 7fe848c00700 1 -- 172.21.15.131:0/6214560 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838003970 con 0x7fe84c1019e0 2024-05-08T22:06:11.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.022+0000 7fe853000700 1 -- 172.21.15.131:0/6214560 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84c19ea30 con 0x7fe84c1019e0 2024-05-08T22:06:11.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.022+0000 7fe853000700 1 -- 172.21.15.131:0/6214560 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe84c1a6120 con 0x7fe84c1019e0 2024-05-08T22:06:11.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.023+0000 7fe853000700 1 -- 172.21.15.131:0/6214560 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe84c1111a0 con 0x7fe84c1019e0 2024-05-08T22:06:11.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.028+0000 7fe848c00700 1 -- 172.21.15.131:0/6214560 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fe838020030 con 0x7fe84c1019e0 2024-05-08T22:06:11.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.029+0000 7fe848c00700 1 --2- 172.21.15.131:0/6214560 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe8300a1300 0x7fe8300a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:11.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.029+0000 7fe848c00700 1 -- 172.21.15.131:0/6214560 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe8380ce090 con 0x7fe84c1019e0 2024-05-08T22:06:11.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.029+0000 7fe848c00700 1 --2- 172.21.15.131:0/6214560 >> [v2:172.21.15.132:6808/1317083238,v1:172.21.15.132:6809/1317083238] conn(0x7fe8300ac320 0x7fe8300ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:11.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.030+0000 7fe848c00700 1 -- 172.21.15.131:0/6214560 --> [v2:172.21.15.132:6808/1317083238,v1:172.21.15.132:6809/1317083238] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe8300aee90 con 0x7fe8300ac320 2024-05-08T22:06:11.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.030+0000 7fe848c00700 1 -- 172.21.15.131:0/6214560 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8380ce4a0 con 0x7fe84c1019e0 2024-05-08T22:06:11.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.030+0000 7fe851000700 1 --2- 172.21.15.131:0/6214560 >> [v2:172.21.15.132:6808/1317083238,v1:172.21.15.132:6809/1317083238] conn(0x7fe8300ac320 0x7fe8300ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:11.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.031+0000 7fe84be00700 1 --2- 172.21.15.131:0/6214560 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe8300a1300 0x7fe8300a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:11.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.031+0000 7fe851000700 1 --2- 172.21.15.131:0/6214560 >> [v2:172.21.15.132:6808/1317083238,v1:172.21.15.132:6809/1317083238] conn(0x7fe8300ac320 0x7fe8300ae7c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:11.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.031+0000 7fe84be00700 1 --2- 172.21.15.131:0/6214560 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe8300a1300 0x7fe8300a37c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe83c005950 tx=0x7fe83c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:11.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.032+0000 7fe848c00700 1 -- 172.21.15.131:0/6214560 <== osd.4 v2:172.21.15.132:6808/1317083238 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fe8300aee90 con 0x7fe8300ac320 2024-05-08T22:06:11.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.057+0000 7fe853000700 1 -- 172.21.15.131:0/6214560 --> [v2:172.21.15.132:6808/1317083238,v1:172.21.15.132:6809/1317083238] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe84c0744a0 con 0x7fe8300ac320 2024-05-08T22:06:11.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.058+0000 7fe848c00700 1 -- 172.21.15.131:0/6214560 <== osd.4 v2:172.21.15.132:6808/1317083238 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe84c0744a0 con 0x7fe8300ac320 2024-05-08T22:06:11.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.058+0000 7fe853000700 1 -- 172.21.15.131:0/6214560 >> [v2:172.21.15.132:6808/1317083238,v1:172.21.15.132:6809/1317083238] conn(0x7fe8300ac320 msgr2=0x7fe8300ae7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:11.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.058+0000 7fe853000700 1 --2- 172.21.15.131:0/6214560 >> [v2:172.21.15.132:6808/1317083238,v1:172.21.15.132:6809/1317083238] conn(0x7fe8300ac320 0x7fe8300ae7c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:11.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.058+0000 7fe853000700 1 -- 172.21.15.131:0/6214560 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe8300a1300 msgr2=0x7fe8300a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:11.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.058+0000 7fe853000700 1 --2- 172.21.15.131:0/6214560 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe8300a1300 0x7fe8300a37c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe83c005950 tx=0x7fe83c0058e0 comp rx=0 tx=0).stop 2024-05-08T22:06:11.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.058+0000 7fe853000700 1 -- 172.21.15.131:0/6214560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe84c1019e0 msgr2=0x7fe84c199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:11.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.058+0000 7fe853000700 1 --2- 172.21.15.131:0/6214560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe84c1019e0 0x7fe84c199b10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe838005950 tx=0x7fe838010ff0 comp rx=0 tx=0).stop 2024-05-08T22:06:11.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.059+0000 7fe851000700 1 -- 172.21.15.131:0/6214560 reap_dead start 2024-05-08T22:06:11.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.059+0000 7fe853000700 1 -- 172.21.15.131:0/6214560 shutdown_connections 2024-05-08T22:06:11.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.059+0000 7fe853000700 1 -- 172.21.15.131:0/6214560 >> 172.21.15.131:0/6214560 conn(0x7fe84c078100 msgr2=0x7fe84c0ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:11.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.059+0000 7fe853000700 1 -- 172.21.15.131:0/6214560 shutdown_connections 2024-05-08T22:06:11.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:11.059+0000 7fe853000700 1 -- 172.21.15.131:0/6214560 wait complete. 2024-05-08T22:06:11.985 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:11 smithi131 ceph-mon[37260]: pgmap v270: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T22:06:12.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:11 smithi157 ceph-mon[77746]: pgmap v270: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T22:06:12.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:11 smithi132 ceph-mon[78180]: pgmap v270: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T22:06:12.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.674+0000 7fc1df000700 1 -- 172.21.15.131:0/3663288179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1d8073710 msgr2=0x7fc1d8073b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:12.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.674+0000 7fc1df000700 1 --2- 172.21.15.131:0/3663288179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1d8073710 0x7fc1d8073b90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009990 tx=0x7fc1c4009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:12.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.674+0000 7fc1df000700 1 -- 172.21.15.131:0/3663288179 shutdown_connections 2024-05-08T22:06:12.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.674+0000 7fc1df000700 1 --2- 172.21.15.131:0/3663288179 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1d8111c30 0x7fc1d8114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:12.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.674+0000 7fc1df000700 1 --2- 172.21.15.131:0/3663288179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1d8075000 0x7fc1d80731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:12.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.674+0000 7fc1df000700 1 --2- 172.21.15.131:0/3663288179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1d8073710 0x7fc1d8073b90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:12.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.674+0000 7fc1df000700 1 -- 172.21.15.131:0/3663288179 >> 172.21.15.131:0/3663288179 conn(0x7fc1d8078190 msgr2=0x7fc1d80785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:12.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.674+0000 7fc1df000700 1 -- 172.21.15.131:0/3663288179 shutdown_connections 2024-05-08T22:06:12.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.674+0000 7fc1df000700 1 -- 172.21.15.131:0/3663288179 wait complete. 2024-05-08T22:06:12.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.675+0000 7fc1df000700 1 Processor -- start 2024-05-08T22:06:12.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.676+0000 7fc1df000700 1 -- start start 2024-05-08T22:06:12.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.676+0000 7fc1df000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1d8073710 0x7fc1d819da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:12.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.676+0000 7fc1df000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1d8075000 0x7fc1d819df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:12.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.676+0000 7fc1df000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1d8111c30 0x7fc1d81a2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:12.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.676+0000 7fc1df000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d81a2910 con 0x7fc1d8075000 2024-05-08T22:06:12.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.676+0000 7fc1df000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d81a2a80 con 0x7fc1d8073710 2024-05-08T22:06:12.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.676+0000 7fc1df000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d81a2c40 con 0x7fc1d8111c30 2024-05-08T22:06:12.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.676+0000 7fc1d7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1d8075000 0x7fc1d819df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:12.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.676+0000 7fc1d7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1d8075000 0x7fc1d819df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34822/0 (socket says 172.21.15.131:34822) 2024-05-08T22:06:12.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.676+0000 7fc1d7400700 1 -- 172.21.15.131:0/3692142205 learned_addr learned my addr 172.21.15.131:0/3692142205 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:12.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.676+0000 7fc1d7e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1d8073710 0x7fc1d819da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:12.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.676+0000 7fc1dd000700 1 --2- 172.21.15.131:0/3692142205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1d8111c30 0x7fc1d81a2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:12.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.677+0000 7fc1d7400700 1 -- 172.21.15.131:0/3692142205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1d8111c30 msgr2=0x7fc1d81a2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:12.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.677+0000 7fc1d7400700 1 --2- 172.21.15.131:0/3692142205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1d8111c30 0x7fc1d81a2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:12.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.677+0000 7fc1d7400700 1 -- 172.21.15.131:0/3692142205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1d8073710 msgr2=0x7fc1d819da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:12.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.677+0000 7fc1d7400700 1 --2- 172.21.15.131:0/3692142205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1d8073710 0x7fc1d819da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:12.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.677+0000 7fc1d7400700 1 -- 172.21.15.131:0/3692142205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c40095f0 con 0x7fc1d8075000 2024-05-08T22:06:12.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.677+0000 7fc1dd000700 1 --2- 172.21.15.131:0/3692142205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1d8111c30 0x7fc1d81a2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:12.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.677+0000 7fc1d7400700 1 --2- 172.21.15.131:0/3692142205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1d8075000 0x7fc1d819df60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4000c00 tx=0x7fc1c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:12.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.677+0000 7fc1d4c00700 1 -- 172.21.15.131:0/3692142205 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c401d070 con 0x7fc1d8075000 2024-05-08T22:06:12.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.677+0000 7fc1d4c00700 1 -- 172.21.15.131:0/3692142205 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1c4022470 con 0x7fc1d8075000 2024-05-08T22:06:12.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.677+0000 7fc1df000700 1 -- 172.21.15.131:0/3692142205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d81a2ec0 con 0x7fc1d8075000 2024-05-08T22:06:12.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.678+0000 7fc1d4c00700 1 -- 172.21.15.131:0/3692142205 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4017650 con 0x7fc1d8075000 2024-05-08T22:06:12.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.678+0000 7fc1df000700 1 -- 172.21.15.131:0/3692142205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1d81a3190 con 0x7fc1d8075000 2024-05-08T22:06:12.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.679+0000 7fc1df000700 1 -- 172.21.15.131:0/3692142205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc1d81177f0 con 0x7fc1d8075000 2024-05-08T22:06:12.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.681+0000 7fc1d4c00700 1 -- 172.21.15.131:0/3692142205 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fc1c40225e0 con 0x7fc1d8075000 2024-05-08T22:06:12.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.682+0000 7fc1d4c00700 1 --2- 172.21.15.131:0/3692142205 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc1bc0a11b0 0x7fc1bc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:12.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.683+0000 7fc1d4c00700 1 -- 172.21.15.131:0/3692142205 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc1c40ce7e0 con 0x7fc1d8075000 2024-05-08T22:06:12.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.683+0000 7fc1d7e00700 1 --2- 172.21.15.131:0/3692142205 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc1bc0a11b0 0x7fc1bc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:12.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.683+0000 7fc1d4c00700 1 --2- 172.21.15.131:0/3692142205 >> [v2:172.21.15.157:6824/2884744696,v1:172.21.15.157:6825/2884744696] conn(0x7fc1bc0ac1d0 0x7fc1bc0ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:12.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.683+0000 7fc1d4c00700 1 -- 172.21.15.131:0/3692142205 --> [v2:172.21.15.157:6824/2884744696,v1:172.21.15.157:6825/2884744696] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc1bc0aed40 con 0x7fc1bc0ac1d0 2024-05-08T22:06:12.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.683+0000 7fc1d4c00700 1 -- 172.21.15.131:0/3692142205 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1c40ceb50 con 0x7fc1d8075000 2024-05-08T22:06:12.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.683+0000 7fc1d7e00700 1 --2- 172.21.15.131:0/3692142205 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc1bc0a11b0 0x7fc1bc0a3670 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8009c60 tx=0x7fc1c8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:12.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.683+0000 7fc1dd000700 1 --2- 172.21.15.131:0/3692142205 >> [v2:172.21.15.157:6824/2884744696,v1:172.21.15.157:6825/2884744696] conn(0x7fc1bc0ac1d0 0x7fc1bc0ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:12.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.684+0000 7fc1dd000700 1 --2- 172.21.15.131:0/3692142205 >> [v2:172.21.15.157:6824/2884744696,v1:172.21.15.157:6825/2884744696] conn(0x7fc1bc0ac1d0 0x7fc1bc0ae670 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:12.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.685+0000 7fc1d4c00700 1 -- 172.21.15.131:0/3692142205 <== osd.11 v2:172.21.15.157:6824/2884744696 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fc1bc0aed40 con 0x7fc1bc0ac1d0 2024-05-08T22:06:12.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.702+0000 7fc1df000700 1 -- 172.21.15.131:0/3692142205 --> [v2:172.21.15.157:6824/2884744696,v1:172.21.15.157:6825/2884744696] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc1d81105f0 con 0x7fc1bc0ac1d0 2024-05-08T22:06:12.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.702+0000 7fc1d4c00700 1 -- 172.21.15.131:0/3692142205 <== osd.11 v2:172.21.15.157:6824/2884744696 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc1d81105f0 con 0x7fc1bc0ac1d0 2024-05-08T22:06:12.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.703+0000 7fc1df000700 1 -- 172.21.15.131:0/3692142205 >> [v2:172.21.15.157:6824/2884744696,v1:172.21.15.157:6825/2884744696] conn(0x7fc1bc0ac1d0 msgr2=0x7fc1bc0ae670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:12.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.703+0000 7fc1df000700 1 --2- 172.21.15.131:0/3692142205 >> [v2:172.21.15.157:6824/2884744696,v1:172.21.15.157:6825/2884744696] conn(0x7fc1bc0ac1d0 0x7fc1bc0ae670 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:12.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.703+0000 7fc1df000700 1 -- 172.21.15.131:0/3692142205 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc1bc0a11b0 msgr2=0x7fc1bc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:12.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.703+0000 7fc1df000700 1 --2- 172.21.15.131:0/3692142205 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc1bc0a11b0 0x7fc1bc0a3670 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8009c60 tx=0x7fc1c8009260 comp rx=0 tx=0).stop 2024-05-08T22:06:12.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.703+0000 7fc1df000700 1 -- 172.21.15.131:0/3692142205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1d8075000 msgr2=0x7fc1d819df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:12.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.703+0000 7fc1df000700 1 --2- 172.21.15.131:0/3692142205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1d8075000 0x7fc1d819df60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4000c00 tx=0x7fc1c4004730 comp rx=0 tx=0).stop 2024-05-08T22:06:12.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.703+0000 7fc1dd000700 1 -- 172.21.15.131:0/3692142205 reap_dead start 2024-05-08T22:06:12.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.703+0000 7fc1df000700 1 -- 172.21.15.131:0/3692142205 shutdown_connections 2024-05-08T22:06:12.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.703+0000 7fc1df000700 1 -- 172.21.15.131:0/3692142205 >> 172.21.15.131:0/3692142205 conn(0x7fc1d8078190 msgr2=0x7fc1d810fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:12.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.703+0000 7fc1df000700 1 -- 172.21.15.131:0/3692142205 shutdown_connections 2024-05-08T22:06:12.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.703+0000 7fc1df000700 1 -- 172.21.15.131:0/3692142205 wait complete. 2024-05-08T22:06:12.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.786+0000 7f2185c00700 1 -- 172.21.15.131:0/1788170774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2180075000 msgr2=0x7f21800731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:12.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.786+0000 7f2185c00700 1 --2- 172.21.15.131:0/1788170774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2180075000 0x7f21800731d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2170009990 tx=0x7f2170009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:12.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.786+0000 7f2185c00700 1 -- 172.21.15.131:0/1788170774 shutdown_connections 2024-05-08T22:06:12.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.786+0000 7f2185c00700 1 --2- 172.21.15.131:0/1788170774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2180111c30 0x7f2180114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:12.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.786+0000 7f2185c00700 1 --2- 172.21.15.131:0/1788170774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2180075000 0x7f21800731d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:12.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.786+0000 7f2185c00700 1 --2- 172.21.15.131:0/1788170774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2180073710 0x7f2180073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:12.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.786+0000 7f2185c00700 1 -- 172.21.15.131:0/1788170774 >> 172.21.15.131:0/1788170774 conn(0x7f2180078190 msgr2=0x7f21800785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:12.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.786+0000 7f2185c00700 1 -- 172.21.15.131:0/1788170774 shutdown_connections 2024-05-08T22:06:12.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.787+0000 7f2185c00700 1 -- 172.21.15.131:0/1788170774 wait complete. 2024-05-08T22:06:12.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.788+0000 7f2185c00700 1 Processor -- start 2024-05-08T22:06:12.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.788+0000 7f2185c00700 1 -- start start 2024-05-08T22:06:12.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.788+0000 7f2185c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2180073710 0x7f21801a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:12.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.788+0000 7f2185c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2180075000 0x7f21801a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.788+0000 7f2185c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2180111c30 0x7f218019d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.788+0000 7f2185c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218019df60 con 0x7f2180075000 2024-05-08T22:06:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.788+0000 7f2185c00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218019e0d0 con 0x7f2180073710 2024-05-08T22:06:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.788+0000 7f2185c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218019e290 con 0x7f2180111c30 2024-05-08T22:06:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.789+0000 7f217ea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2180073710 0x7f21801a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:12.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.789+0000 7f217ea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2180073710 0x7f21801a37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:39218/0 (socket says 172.21.15.131:39218) 2024-05-08T22:06:12.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.789+0000 7f217ea00700 1 -- 172.21.15.131:0/1108860497 learned_addr learned my addr 172.21.15.131:0/1108860497 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:12.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.789+0000 7f217e000700 1 --2- 172.21.15.131:0/1108860497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2180075000 0x7f21801a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:12.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.789+0000 7f217f400700 1 --2- 172.21.15.131:0/1108860497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2180111c30 0x7f218019d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:12.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.789+0000 7f217e000700 1 -- 172.21.15.131:0/1108860497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2180111c30 msgr2=0x7f218019d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:12.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.789+0000 7f217e000700 1 --2- 172.21.15.131:0/1108860497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2180111c30 0x7f218019d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:12.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.789+0000 7f217e000700 1 -- 172.21.15.131:0/1108860497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2180073710 msgr2=0x7f21801a37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:12.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.789+0000 7f217e000700 1 --2- 172.21.15.131:0/1108860497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2180073710 0x7f21801a37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:12.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.789+0000 7f217e000700 1 -- 172.21.15.131:0/1108860497 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216c009520 con 0x7f2180075000 2024-05-08T22:06:12.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.790+0000 7f217ea00700 1 --2- 172.21.15.131:0/1108860497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2180073710 0x7f21801a37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:06:12.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.790+0000 7f217e000700 1 --2- 172.21.15.131:0/1108860497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2180075000 0x7f21801a3cf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f216c00e9f0 tx=0x7f216c00edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:12.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.790+0000 7f2177400700 1 -- 172.21.15.131:0/1108860497 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c00cb70 con 0x7f2180075000 2024-05-08T22:06:12.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.790+0000 7f2185c00700 1 -- 172.21.15.131:0/1108860497 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21700095f0 con 0x7f2180075000 2024-05-08T22:06:12.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.790+0000 7f2185c00700 1 -- 172.21.15.131:0/1108860497 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f218019e760 con 0x7f2180075000 2024-05-08T22:06:12.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.790+0000 7f2177400700 1 -- 172.21.15.131:0/1108860497 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f216c0044e0 con 0x7f2180075000 2024-05-08T22:06:12.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.790+0000 7f2177400700 1 -- 172.21.15.131:0/1108860497 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c010430 con 0x7f2180075000 2024-05-08T22:06:12.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.791+0000 7f2185c00700 1 -- 172.21.15.131:0/1108860497 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2160000fc0 con 0x7f2180075000 2024-05-08T22:06:12.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.794+0000 7f2177400700 1 -- 172.21.15.131:0/1108860497 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f216c00ccd0 con 0x7f2180075000 2024-05-08T22:06:12.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.795+0000 7f2177400700 1 --2- 172.21.15.131:0/1108860497 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f21640a1230 0x7f21640a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:12.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.795+0000 7f2177400700 1 -- 172.21.15.131:0/1108860497 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f216c014070 con 0x7f2180075000 2024-05-08T22:06:12.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.796+0000 7f2177400700 1 --2- 172.21.15.131:0/1108860497 >> [v2:172.21.15.131:6818/260285513,v1:172.21.15.131:6819/260285513] conn(0x7f21640ac250 0x7f21640ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:12.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.796+0000 7f2177400700 1 -- 172.21.15.131:0/1108860497 --> [v2:172.21.15.131:6818/260285513,v1:172.21.15.131:6819/260285513] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f21640aedc0 con 0x7f21640ac250 2024-05-08T22:06:12.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.796+0000 7f2177400700 1 -- 172.21.15.131:0/1108860497 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f216c0cd5a0 con 0x7f2180075000 2024-05-08T22:06:12.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.796+0000 7f217ea00700 1 --2- 172.21.15.131:0/1108860497 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f21640a1230 0x7f21640a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:12.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.796+0000 7f217f400700 1 --2- 172.21.15.131:0/1108860497 >> [v2:172.21.15.131:6818/260285513,v1:172.21.15.131:6819/260285513] conn(0x7f21640ac250 0x7f21640ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:12.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.797+0000 7f217f400700 1 --2- 172.21.15.131:0/1108860497 >> [v2:172.21.15.131:6818/260285513,v1:172.21.15.131:6819/260285513] conn(0x7f21640ac250 0x7f21640ae6f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:12.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.797+0000 7f217ea00700 1 --2- 172.21.15.131:0/1108860497 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f21640a1230 0x7f21640a36f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2170006010 tx=0x7f217000b2a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:12.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.798+0000 7f2177400700 1 -- 172.21.15.131:0/1108860497 <== osd.6 v2:172.21.15.131:6818/260285513 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f21640aedc0 con 0x7f21640ac250 2024-05-08T22:06:12.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.815+0000 7f2185c00700 1 -- 172.21.15.131:0/1108860497 --> [v2:172.21.15.131:6818/260285513,v1:172.21.15.131:6819/260285513] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2160002d40 con 0x7f21640ac250 2024-05-08T22:06:12.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.815+0000 7f2177400700 1 -- 172.21.15.131:0/1108860497 <== osd.6 v2:172.21.15.131:6818/260285513 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2160002d40 con 0x7f21640ac250 2024-05-08T22:06:12.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.816+0000 7f2185c00700 1 -- 172.21.15.131:0/1108860497 >> [v2:172.21.15.131:6818/260285513,v1:172.21.15.131:6819/260285513] conn(0x7f21640ac250 msgr2=0x7f21640ae6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:12.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.816+0000 7f2185c00700 1 --2- 172.21.15.131:0/1108860497 >> [v2:172.21.15.131:6818/260285513,v1:172.21.15.131:6819/260285513] conn(0x7f21640ac250 0x7f21640ae6f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:12.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.816+0000 7f2185c00700 1 -- 172.21.15.131:0/1108860497 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f21640a1230 msgr2=0x7f21640a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:12.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.816+0000 7f2185c00700 1 --2- 172.21.15.131:0/1108860497 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f21640a1230 0x7f21640a36f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2170006010 tx=0x7f217000b2a0 comp rx=0 tx=0).stop 2024-05-08T22:06:12.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.816+0000 7f2185c00700 1 -- 172.21.15.131:0/1108860497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2180075000 msgr2=0x7f21801a3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:12.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.816+0000 7f2185c00700 1 --2- 172.21.15.131:0/1108860497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2180075000 0x7f21801a3cf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f216c00e9f0 tx=0x7f216c00edb0 comp rx=0 tx=0).stop 2024-05-08T22:06:12.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.816+0000 7f217f400700 1 -- 172.21.15.131:0/1108860497 reap_dead start 2024-05-08T22:06:12.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.816+0000 7f2185c00700 1 -- 172.21.15.131:0/1108860497 shutdown_connections 2024-05-08T22:06:12.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.816+0000 7f2185c00700 1 -- 172.21.15.131:0/1108860497 >> 172.21.15.131:0/1108860497 conn(0x7f2180078190 msgr2=0x7f21801109f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:12.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.816+0000 7f2185c00700 1 -- 172.21.15.131:0/1108860497 shutdown_connections 2024-05-08T22:06:12.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:12.817+0000 7f2185c00700 1 -- 172.21.15.131:0/1108860497 wait complete. 2024-05-08T22:06:13.226 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:13.230 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:12 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:06:13.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:12 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:06:13.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:12 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:06:13.906 INFO:teuthology.orchestra.run.smithi131.stdout:236223201314 2024-05-08T22:06:13.907 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-08T22:06:13.960 INFO:teuthology.orchestra.run.smithi131.stdout:206158430244 2024-05-08T22:06:13.961 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-08T22:06:14.021 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:13 smithi131 ceph-mon[37260]: pgmap v271: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T22:06:14.021 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:13 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:06:14.021 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:13 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:06:14.021 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:13 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:06:14.021 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:13 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:06:14.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:13 smithi157 ceph-mon[77746]: pgmap v271: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T22:06:14.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:13 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:06:14.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:13 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:06:14.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:13 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:06:14.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:13 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:06:14.278 INFO:teuthology.orchestra.run.smithi131.stdout:34359738432 2024-05-08T22:06:14.279 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-08T22:06:14.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:13 smithi132 ceph-mon[78180]: pgmap v271: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T22:06:14.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:13 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:06:14.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:13 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:06:14.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:13 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:06:14.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:13 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:06:14.398 INFO:teuthology.orchestra.run.smithi131.stdout:317827579927 2024-05-08T22:06:14.399 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-08T22:06:14.427 INFO:teuthology.orchestra.run.smithi131.stdout:184683593769 2024-05-08T22:06:14.427 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-08T22:06:14.813 INFO:teuthology.orchestra.run.smithi131.stdout:133143986225 2024-05-08T22:06:14.813 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-08T22:06:15.193 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:15.277 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:15.287 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:15.289 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:15.291 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:15.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.503+0000 7f4e30800700 1 -- 172.21.15.131:0/871495774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e281084f0 msgr2=0x7f4e281100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:15.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.503+0000 7f4e30800700 1 --2- 172.21.15.131:0/871495774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e281084f0 0x7f4e281100c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4e2400b0a0 tx=0x7f4e2400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:06:15.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.503+0000 7f4e30800700 1 -- 172.21.15.131:0/871495774 shutdown_connections 2024-05-08T22:06:15.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.503+0000 7f4e30800700 1 --2- 172.21.15.131:0/871495774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e281084f0 0x7f4e281100c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:15.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.503+0000 7f4e30800700 1 --2- 172.21.15.131:0/871495774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e28103410 0x7f4e281037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:15.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.503+0000 7f4e30800700 1 --2- 172.21.15.131:0/871495774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e28103dc0 0x7f4e28107e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:15.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.503+0000 7f4e30800700 1 -- 172.21.15.131:0/871495774 >> 172.21.15.131:0/871495774 conn(0x7f4e280fec80 msgr2=0x7f4e281010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:15.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.503+0000 7f4e30800700 1 -- 172.21.15.131:0/871495774 shutdown_connections 2024-05-08T22:06:15.853 INFO:teuthology.orchestra.run.smithi131.stdout:103079215156 2024-05-08T22:06:15.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.503+0000 7f4e30800700 1 -- 172.21.15.131:0/871495774 wait complete. 2024-05-08T22:06:15.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.504+0000 7f4e30800700 1 Processor -- start 2024-05-08T22:06:15.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.504+0000 7f4e30800700 1 -- start start 2024-05-08T22:06:15.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.504+0000 7f4e30800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e28103410 0x7f4e28199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:15.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.504+0000 7f4e30800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e28103dc0 0x7f4e28199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:15.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.504+0000 7f4e30800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e281084f0 0x7f4e2819e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:15.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.504+0000 7f4e30800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2819e620 con 0x7f4e28103410 2024-05-08T22:06:15.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.504+0000 7f4e30800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2819e790 con 0x7f4e281084f0 2024-05-08T22:06:15.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.504+0000 7f4e30800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2819e950 con 0x7f4e28103dc0 2024-05-08T22:06:15.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.505+0000 7f4e2de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e28103410 0x7f4e28199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:15.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.505+0000 7f4e2de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e28103410 0x7f4e28199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44970/0 (socket says 172.21.15.131:44970) 2024-05-08T22:06:15.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.505+0000 7f4e2de00700 1 -- 172.21.15.131:0/3277930536 learned_addr learned my addr 172.21.15.131:0/3277930536 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:15.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.505+0000 7f4e2d400700 1 --2- 172.21.15.131:0/3277930536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e28103dc0 0x7f4e28199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:15.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.505+0000 7f4e2e800700 1 --2- 172.21.15.131:0/3277930536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e281084f0 0x7f4e2819e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:15.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.505+0000 7f4e2de00700 1 -- 172.21.15.131:0/3277930536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e28103dc0 msgr2=0x7f4e28199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:15.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.505+0000 7f4e2de00700 1 --2- 172.21.15.131:0/3277930536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e28103dc0 0x7f4e28199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:15.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.505+0000 7f4e2de00700 1 -- 172.21.15.131:0/3277930536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e281084f0 msgr2=0x7f4e2819e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:15.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.505+0000 7f4e2de00700 1 --2- 172.21.15.131:0/3277930536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e281084f0 0x7f4e2819e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:15.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.505+0000 7f4e2de00700 1 -- 172.21.15.131:0/3277930536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e1c009520 con 0x7f4e28103410 2024-05-08T22:06:15.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.505+0000 7f4e2d400700 1 --2- 172.21.15.131:0/3277930536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e28103dc0 0x7f4e28199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:06:15.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.505+0000 7f4e2e800700 1 --2- 172.21.15.131:0/3277930536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e281084f0 0x7f4e2819e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:06:15.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.505+0000 7f4e2de00700 1 --2- 172.21.15.131:0/3277930536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e28103410 0x7f4e28199770 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00eb30 tx=0x7f4e1c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:15.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.505+0000 7f4e22a00700 1 -- 172.21.15.131:0/3277930536 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c009880 con 0x7f4e28103410 2024-05-08T22:06:15.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.505+0000 7f4e22a00700 1 -- 172.21.15.131:0/3277930536 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e1c0044e0 con 0x7f4e28103410 2024-05-08T22:06:15.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.505+0000 7f4e22a00700 1 -- 172.21.15.131:0/3277930536 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c010430 con 0x7f4e28103410 2024-05-08T22:06:15.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.505+0000 7f4e30800700 1 -- 172.21.15.131:0/3277930536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e24009d00 con 0x7f4e28103410 2024-05-08T22:06:15.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.506+0000 7f4e30800700 1 -- 172.21.15.131:0/3277930536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e281a62c0 con 0x7f4e28103410 2024-05-08T22:06:15.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.507+0000 7f4e30800700 1 -- 172.21.15.131:0/3277930536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e2810b7b0 con 0x7f4e28103410 2024-05-08T22:06:15.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.508+0000 7f4e22a00700 1 -- 172.21.15.131:0/3277930536 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f4e1c003ee0 con 0x7f4e28103410 2024-05-08T22:06:15.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.511+0000 7f4e22a00700 1 --2- 172.21.15.131:0/3277930536 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f4e100a11b0 0x7f4e100a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:15.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.511+0000 7f4e22a00700 1 -- 172.21.15.131:0/3277930536 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4e1c014070 con 0x7f4e28103410 2024-05-08T22:06:15.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.511+0000 7f4e2d400700 1 --2- 172.21.15.131:0/3277930536 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f4e100a11b0 0x7f4e100a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:15.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.511+0000 7f4e2d400700 1 --2- 172.21.15.131:0/3277930536 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f4e100a11b0 0x7f4e100a3670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4e2819ac80 tx=0x7f4e180091e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:15.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.516+0000 7f4e22a00700 1 -- 172.21.15.131:0/3277930536 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4e1c0965b0 con 0x7f4e28103410 2024-05-08T22:06:15.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.741+0000 7f4e30800700 1 -- 172.21.15.131:0/3277930536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4e281048f0 con 0x7f4e28103410 2024-05-08T22:06:15.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.741+0000 7f4e22a00700 1 -- 172.21.15.131:0/3277930536 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4e1c099bd0 con 0x7f4e28103410 2024-05-08T22:06:15.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.744+0000 7f4e30800700 1 -- 172.21.15.131:0/3277930536 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f4e100a11b0 msgr2=0x7f4e100a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:15.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.744+0000 7f4e30800700 1 --2- 172.21.15.131:0/3277930536 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f4e100a11b0 0x7f4e100a3670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4e2819ac80 tx=0x7f4e180091e0 comp rx=0 tx=0).stop 2024-05-08T22:06:15.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.744+0000 7f4e30800700 1 -- 172.21.15.131:0/3277930536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e28103410 msgr2=0x7f4e28199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:15.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.744+0000 7f4e30800700 1 --2- 172.21.15.131:0/3277930536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e28103410 0x7f4e28199770 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00eb30 tx=0x7f4e1c00eef0 comp rx=0 tx=0).stop 2024-05-08T22:06:15.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.744+0000 7f4e30800700 1 -- 172.21.15.131:0/3277930536 shutdown_connections 2024-05-08T22:06:15.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.744+0000 7f4e30800700 1 --2- 172.21.15.131:0/3277930536 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f4e100a11b0 0x7f4e100a3670 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:15.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.744+0000 7f4e30800700 1 --2- 172.21.15.131:0/3277930536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e281084f0 0x7f4e2819e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:15.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.744+0000 7f4e30800700 1 --2- 172.21.15.131:0/3277930536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e28103410 0x7f4e28199770 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:15.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.744+0000 7f4e30800700 1 --2- 172.21.15.131:0/3277930536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e28103dc0 0x7f4e28199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:15.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.744+0000 7f4e30800700 1 -- 172.21.15.131:0/3277930536 >> 172.21.15.131:0/3277930536 conn(0x7f4e280fec80 msgr2=0x7f4e28101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:15.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.744+0000 7f4e30800700 1 -- 172.21.15.131:0/3277930536 shutdown_connections 2024-05-08T22:06:15.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:15.744+0000 7f4e30800700 1 -- 172.21.15.131:0/3277930536 wait complete. 2024-05-08T22:06:16.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:15 smithi157 ceph-mon[77746]: pgmap v272: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T22:06:16.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:15 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3277930536' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T22:06:16.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:15 smithi132 ceph-mon[78180]: pgmap v272: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T22:06:16.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:15 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3277930536' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T22:06:16.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:15 smithi131 ceph-mon[37260]: pgmap v272: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T22:06:16.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:15 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3277930536' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T22:06:18.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:17 smithi157 ceph-mon[77746]: pgmap v273: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T22:06:18.308 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:17 smithi131 ceph-mon[37260]: pgmap v273: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T22:06:18.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:17 smithi132 ceph-mon[78180]: pgmap v273: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T22:06:19.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.873+0000 7f1f87200700 1 -- 172.21.15.131:0/2720525528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f80107030 msgr2=0x7f1f80107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:19.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.873+0000 7f1f87200700 1 --2- 172.21.15.131:0/2720525528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f80107030 0x7f1f80107410 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009990 tx=0x7f1f70009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:19.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.873+0000 7f1f87200700 1 -- 172.21.15.131:0/2720525528 shutdown_connections 2024-05-08T22:06:19.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.873+0000 7f1f87200700 1 --2- 172.21.15.131:0/2720525528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f80101990 0x7f1f8010de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:19.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.873+0000 7f1f87200700 1 --2- 172.21.15.131:0/2720525528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f80107030 0x7f1f80107410 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:19.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.873+0000 7f1f87200700 1 --2- 172.21.15.131:0/2720525528 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f80100ff0 0x7f1f80101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:19.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.873+0000 7f1f87200700 1 -- 172.21.15.131:0/2720525528 >> 172.21.15.131:0/2720525528 conn(0x7f1f80078190 msgr2=0x7f1f800785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:19.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.873+0000 7f1f87200700 1 -- 172.21.15.131:0/2720525528 shutdown_connections 2024-05-08T22:06:19.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.873+0000 7f1f87200700 1 -- 172.21.15.131:0/2720525528 wait complete. 2024-05-08T22:06:19.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.874+0000 7f1f87200700 1 Processor -- start 2024-05-08T22:06:19.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.875+0000 7f1f87200700 1 -- start start 2024-05-08T22:06:19.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.875+0000 7f1f87200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f80100ff0 0x7f1f800732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:19.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.875+0000 7f1f87200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f80101990 0x7f1f800737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:19.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.875+0000 7f1f87200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f80107030 0x7f1f80073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:19.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.875+0000 7f1f87200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f801a57b0 con 0x7f1f80100ff0 2024-05-08T22:06:19.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.875+0000 7f1f87200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f801a5920 con 0x7f1f80107030 2024-05-08T22:06:19.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.875+0000 7f1f87200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f801a5ae0 con 0x7f1f80101990 2024-05-08T22:06:19.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.875+0000 7f1f7fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f80100ff0 0x7f1f800732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:19.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.875+0000 7f1f7fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f80100ff0 0x7f1f800732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44988/0 (socket says 172.21.15.131:44988) 2024-05-08T22:06:19.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.875+0000 7f1f7fe00700 1 -- 172.21.15.131:0/565950665 learned_addr learned my addr 172.21.15.131:0/565950665 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:19.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.876+0000 7f1f7f400700 1 --2- 172.21.15.131:0/565950665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f80101990 0x7f1f800737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:19.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.876+0000 7f1f85200700 1 --2- 172.21.15.131:0/565950665 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f80107030 0x7f1f80073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:19.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.876+0000 7f1f7fe00700 1 -- 172.21.15.131:0/565950665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f80101990 msgr2=0x7f1f800737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:19.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.876+0000 7f1f7fe00700 1 --2- 172.21.15.131:0/565950665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f80101990 0x7f1f800737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:19.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.876+0000 7f1f7fe00700 1 -- 172.21.15.131:0/565950665 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f80107030 msgr2=0x7f1f80073d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:19.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.876+0000 7f1f7fe00700 1 --2- 172.21.15.131:0/565950665 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f80107030 0x7f1f80073d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:19.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.876+0000 7f1f7fe00700 1 -- 172.21.15.131:0/565950665 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f700095f0 con 0x7f1f80100ff0 2024-05-08T22:06:19.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.876+0000 7f1f85200700 1 --2- 172.21.15.131:0/565950665 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f80107030 0x7f1f80073d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:06:19.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.876+0000 7f1f7fe00700 1 --2- 172.21.15.131:0/565950665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f80100ff0 0x7f1f800732a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1f70000c00 tx=0x7f1f70005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:19.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.876+0000 7f1f7cc00700 1 -- 172.21.15.131:0/565950665 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7001d070 con 0x7f1f80100ff0 2024-05-08T22:06:19.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.876+0000 7f1f7cc00700 1 -- 172.21.15.131:0/565950665 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f70022470 con 0x7f1f80100ff0 2024-05-08T22:06:19.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.876+0000 7f1f87200700 1 -- 172.21.15.131:0/565950665 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f801a5d60 con 0x7f1f80100ff0 2024-05-08T22:06:19.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.876+0000 7f1f7cc00700 1 -- 172.21.15.131:0/565950665 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70017650 con 0x7f1f80100ff0 2024-05-08T22:06:19.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.877+0000 7f1f87200700 1 -- 172.21.15.131:0/565950665 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f801a6170 con 0x7f1f80100ff0 2024-05-08T22:06:19.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.878+0000 7f1f87200700 1 -- 172.21.15.131:0/565950665 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f80109550 con 0x7f1f80100ff0 2024-05-08T22:06:19.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.880+0000 7f1f7cc00700 1 -- 172.21.15.131:0/565950665 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f1f700225e0 con 0x7f1f80100ff0 2024-05-08T22:06:19.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.881+0000 7f1f7cc00700 1 --2- 172.21.15.131:0/565950665 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1f640a11b0 0x7f1f640a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:19.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.881+0000 7f1f7cc00700 1 -- 172.21.15.131:0/565950665 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1f700ce090 con 0x7f1f80100ff0 2024-05-08T22:06:19.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.881+0000 7f1f7f400700 1 --2- 172.21.15.131:0/565950665 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1f640a11b0 0x7f1f640a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:19.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.882+0000 7f1f7f400700 1 --2- 172.21.15.131:0/565950665 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1f640a11b0 0x7f1f640a3670 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1f800749d0 tx=0x7f1f6c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:19.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:17.885+0000 7f1f7cc00700 1 -- 172.21.15.131:0/565950665 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1f70097500 con 0x7f1f80100ff0 2024-05-08T22:06:19.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:18.116+0000 7f1f87200700 1 -- 172.21.15.131:0/565950665 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1f80074840 con 0x7f1f80100ff0 2024-05-08T22:06:19.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:18.116+0000 7f1f7cc00700 1 -- 172.21.15.131:0/565950665 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1f70027090 con 0x7f1f80100ff0 2024-05-08T22:06:19.125 INFO:teuthology.orchestra.run.smithi131.stdout:77309411386 2024-05-08T22:06:19.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:18.119+0000 7f1f87200700 1 -- 172.21.15.131:0/565950665 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1f640a11b0 msgr2=0x7f1f640a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:19.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:18.119+0000 7f1f87200700 1 --2- 172.21.15.131:0/565950665 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1f640a11b0 0x7f1f640a3670 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1f800749d0 tx=0x7f1f6c009260 comp rx=0 tx=0).stop 2024-05-08T22:06:19.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:18.119+0000 7f1f87200700 1 -- 172.21.15.131:0/565950665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f80100ff0 msgr2=0x7f1f800732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:19.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:18.119+0000 7f1f87200700 1 --2- 172.21.15.131:0/565950665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f80100ff0 0x7f1f800732a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1f70000c00 tx=0x7f1f70005740 comp rx=0 tx=0).stop 2024-05-08T22:06:19.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:18.119+0000 7f1f87200700 1 -- 172.21.15.131:0/565950665 shutdown_connections 2024-05-08T22:06:19.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:18.119+0000 7f1f87200700 1 --2- 172.21.15.131:0/565950665 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1f640a11b0 0x7f1f640a3670 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:19.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:18.119+0000 7f1f87200700 1 --2- 172.21.15.131:0/565950665 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f80107030 0x7f1f80073d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:19.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:18.119+0000 7f1f87200700 1 --2- 172.21.15.131:0/565950665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f80100ff0 0x7f1f800732a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:19.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:18.119+0000 7f1f87200700 1 --2- 172.21.15.131:0/565950665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f80101990 0x7f1f800737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:19.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:18.119+0000 7f1f87200700 1 -- 172.21.15.131:0/565950665 >> 172.21.15.131:0/565950665 conn(0x7f1f80078190 msgr2=0x7f1f800ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:19.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:18.119+0000 7f1f87200700 1 -- 172.21.15.131:0/565950665 shutdown_connections 2024-05-08T22:06:19.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:18.119+0000 7f1f87200700 1 -- 172.21.15.131:0/565950665 wait complete. 2024-05-08T22:06:19.155 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215154 got 103079215156 for osd.3 2024-05-08T22:06:19.156 DEBUG:teuthology.parallel:result is None 2024-05-08T22:06:19.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:18 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/565950665' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T22:06:19.284 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:18 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/565950665' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T22:06:19.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:18 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/565950665' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T22:06:20.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.066+0000 7f6697e00700 1 -- 172.21.15.131:0/702072976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6698101f70 msgr2=0x7f6698102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:20.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.066+0000 7f6697e00700 1 --2- 172.21.15.131:0/702072976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6698101f70 0x7f6698102350 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6688009920 tx=0x7f6688009c30 comp rx=0 tx=0).stop 2024-05-08T22:06:20.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.067+0000 7f6697e00700 1 -- 172.21.15.131:0/702072976 shutdown_connections 2024-05-08T22:06:20.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.067+0000 7f6697e00700 1 --2- 172.21.15.131:0/702072976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6698107500 0x7f669810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.067+0000 7f6697e00700 1 --2- 172.21.15.131:0/702072976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6698101f70 0x7f6698102350 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.067+0000 7f6697e00700 1 --2- 172.21.15.131:0/702072976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6698102920 0x7f6698106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.067+0000 7f6697e00700 1 -- 172.21.15.131:0/702072976 >> 172.21.15.131:0/702072976 conn(0x7f66980fb7e0 msgr2=0x7f66980fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:20.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.067+0000 7f6697e00700 1 -- 172.21.15.131:0/702072976 shutdown_connections 2024-05-08T22:06:20.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.067+0000 7f6697e00700 1 -- 172.21.15.131:0/702072976 wait complete. 2024-05-08T22:06:20.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.068+0000 7f6697e00700 1 Processor -- start 2024-05-08T22:06:20.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.068+0000 7f6697e00700 1 -- start start 2024-05-08T22:06:20.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.068+0000 7f6697e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6698101f70 0x7f66981995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.068+0000 7f6697e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6698102920 0x7f6698199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.068+0000 7f6697e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6698107500 0x7f669819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.068+0000 7f6697e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669819e4e0 con 0x7f6698101f70 2024-05-08T22:06:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.068+0000 7f6697e00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669819e650 con 0x7f6698107500 2024-05-08T22:06:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.068+0000 7f6697e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669819e810 con 0x7f6698102920 2024-05-08T22:06:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.068+0000 7f6696a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6698101f70 0x7f66981995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.069+0000 7f6696a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6698101f70 0x7f66981995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44996/0 (socket says 172.21.15.131:44996) 2024-05-08T22:06:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.069+0000 7f6696a00700 1 -- 172.21.15.131:0/1505974229 learned_addr learned my addr 172.21.15.131:0/1505974229 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.069+0000 7f6697400700 1 --2- 172.21.15.131:0/1505974229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6698107500 0x7f669819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.069+0000 7f6696000700 1 --2- 172.21.15.131:0/1505974229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6698102920 0x7f6698199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:20.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.069+0000 7f6696a00700 1 -- 172.21.15.131:0/1505974229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6698102920 msgr2=0x7f6698199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:20.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.069+0000 7f6696a00700 1 --2- 172.21.15.131:0/1505974229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6698102920 0x7f6698199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.069+0000 7f6696a00700 1 -- 172.21.15.131:0/1505974229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6698107500 msgr2=0x7f669819ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:20.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.069+0000 7f6696a00700 1 --2- 172.21.15.131:0/1505974229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6698107500 0x7f669819ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.069+0000 7f6696a00700 1 -- 172.21.15.131:0/1505974229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6690009d00 con 0x7f6698101f70 2024-05-08T22:06:20.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.069+0000 7f6696a00700 1 --2- 172.21.15.131:0/1505974229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6698101f70 0x7f66981995e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6688000c00 tx=0x7f668800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:20.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.069+0000 7f669ca00700 1 -- 172.21.15.131:0/1505974229 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668801c070 con 0x7f6698101f70 2024-05-08T22:06:20.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.069+0000 7f669ca00700 1 -- 172.21.15.131:0/1505974229 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f668800baa0 con 0x7f6698101f70 2024-05-08T22:06:20.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.070+0000 7f6697e00700 1 -- 172.21.15.131:0/1505974229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6688009520 con 0x7f6698101f70 2024-05-08T22:06:20.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.070+0000 7f669ca00700 1 -- 172.21.15.131:0/1505974229 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66880038e0 con 0x7f6698101f70 2024-05-08T22:06:20.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.070+0000 7f6697e00700 1 -- 172.21.15.131:0/1505974229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66981a6180 con 0x7f6698101f70 2024-05-08T22:06:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.071+0000 7f6697e00700 1 -- 172.21.15.131:0/1505974229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f669810b590 con 0x7f6698101f70 2024-05-08T22:06:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.073+0000 7f669ca00700 1 -- 172.21.15.131:0/1505974229 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f6688020030 con 0x7f6698101f70 2024-05-08T22:06:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.074+0000 7f669ca00700 1 --2- 172.21.15.131:0/1505974229 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f667c0a11b0 0x7f667c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.074+0000 7f669ca00700 1 -- 172.21.15.131:0/1505974229 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f66880cdcf0 con 0x7f6698101f70 2024-05-08T22:06:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.074+0000 7f6696000700 1 --2- 172.21.15.131:0/1505974229 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f667c0a11b0 0x7f667c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:20.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.075+0000 7f6696000700 1 --2- 172.21.15.131:0/1505974229 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f667c0a11b0 0x7f667c0a3670 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f669819ab40 tx=0x7f6684009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:20.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.076+0000 7f669ca00700 1 -- 172.21.15.131:0/1505974229 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6688097040 con 0x7f6698101f70 2024-05-08T22:06:20.239 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:20.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:19 smithi157 ceph-mon[77746]: pgmap v274: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:06:20.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.292+0000 7f7af2a00700 1 -- 172.21.15.131:0/2129968880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7aec073710 msgr2=0x7f7aec073b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:20.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.292+0000 7f7af2a00700 1 --2- 172.21.15.131:0/2129968880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7aec073710 0x7f7aec073b90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009990 tx=0x7f7ad4009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.293+0000 7f7af2a00700 1 -- 172.21.15.131:0/2129968880 shutdown_connections 2024-05-08T22:06:20.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.293+0000 7f7af2a00700 1 --2- 172.21.15.131:0/2129968880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7aec111c30 0x7f7aec114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.293+0000 7f7af2a00700 1 --2- 172.21.15.131:0/2129968880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aec075000 0x7f7aec0731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.293+0000 7f7af2a00700 1 --2- 172.21.15.131:0/2129968880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7aec073710 0x7f7aec073b90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.293+0000 7f7af2a00700 1 -- 172.21.15.131:0/2129968880 >> 172.21.15.131:0/2129968880 conn(0x7f7aec078190 msgr2=0x7f7aec0785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:20.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.293+0000 7f7af2a00700 1 -- 172.21.15.131:0/2129968880 shutdown_connections 2024-05-08T22:06:20.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.293+0000 7f7af2a00700 1 -- 172.21.15.131:0/2129968880 wait complete. 2024-05-08T22:06:20.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.294+0000 7f7af2a00700 1 Processor -- start 2024-05-08T22:06:20.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.294+0000 7f7af2a00700 1 -- start start 2024-05-08T22:06:20.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.294+0000 7f7af2a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7aec073710 0x7f7aec1a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:20.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.294+0000 7f7af2a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7aec075000 0x7f7aec1a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:20.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.294+0000 7f7af2a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aec111c30 0x7f7aec19d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:20.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.294+0000 7f7af2a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec19e0f0 con 0x7f7aec073710 2024-05-08T22:06:20.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.294+0000 7f7af2a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec19e260 con 0x7f7aec075000 2024-05-08T22:06:20.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.294+0000 7f7af2a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec19e420 con 0x7f7aec111c30 2024-05-08T22:06:20.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.295+0000 7f7aebe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7aec073710 0x7f7aec1a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:20.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.295+0000 7f7aebe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7aec073710 0x7f7aec1a3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45008/0 (socket says 172.21.15.131:45008) 2024-05-08T22:06:20.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.295+0000 7f7aebe00700 1 -- 172.21.15.131:0/556884511 learned_addr learned my addr 172.21.15.131:0/556884511 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:20.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.295+0000 7f7af0a00700 1 --2- 172.21.15.131:0/556884511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aec111c30 0x7f7aec19d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:20.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.295+0000 7f7ae3e00700 1 --2- 172.21.15.131:0/556884511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7aec075000 0x7f7aec1a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:20.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.295+0000 7f7af0a00700 1 -- 172.21.15.131:0/556884511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7aec075000 msgr2=0x7f7aec1a3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:20.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.295+0000 7f7af0a00700 1 --2- 172.21.15.131:0/556884511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7aec075000 0x7f7aec1a3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.295+0000 7f7af0a00700 1 -- 172.21.15.131:0/556884511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7aec073710 msgr2=0x7f7aec1a3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:20.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.295+0000 7f7af0a00700 1 --2- 172.21.15.131:0/556884511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7aec073710 0x7f7aec1a3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.295+0000 7f7af0a00700 1 -- 172.21.15.131:0/556884511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad40095f0 con 0x7f7aec111c30 2024-05-08T22:06:20.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.295+0000 7f7ae3e00700 1 --2- 172.21.15.131:0/556884511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7aec075000 0x7f7aec1a3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:06:20.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.296+0000 7f7af0a00700 1 --2- 172.21.15.131:0/556884511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aec111c30 0x7f7aec19d940 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7aec1053d0 tx=0x7f7ae400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:20.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.296+0000 7f7ae9600700 1 -- 172.21.15.131:0/556884511 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4004db0 con 0x7f7aec111c30 2024-05-08T22:06:20.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.296+0000 7f7ae9600700 1 -- 172.21.15.131:0/556884511 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ae400f040 con 0x7f7aec111c30 2024-05-08T22:06:20.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.296+0000 7f7af2a00700 1 -- 172.21.15.131:0/556884511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aec19e700 con 0x7f7aec111c30 2024-05-08T22:06:20.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.296+0000 7f7af2a00700 1 -- 172.21.15.131:0/556884511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aec1aa860 con 0x7f7aec111c30 2024-05-08T22:06:20.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.296+0000 7f7ae9600700 1 -- 172.21.15.131:0/556884511 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4005920 con 0x7f7aec111c30 2024-05-08T22:06:20.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.297+0000 7f7af2a00700 1 -- 172.21.15.131:0/556884511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac80052f0 con 0x7f7aec111c30 2024-05-08T22:06:20.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.299+0000 7f6697e00700 1 -- 172.21.15.131:0/1505974229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f6698069230 con 0x7f6698101f70 2024-05-08T22:06:20.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.299+0000 7f669ca00700 1 -- 172.21.15.131:0/1505974229 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f668809a660 con 0x7f6698101f70 2024-05-08T22:06:20.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.299+0000 7f7ae9600700 1 -- 172.21.15.131:0/556884511 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f7ae4005020 con 0x7f7aec111c30 2024-05-08T22:06:20.301 INFO:teuthology.orchestra.run.smithi131.stdout:266287972383 2024-05-08T22:06:20.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.301+0000 7f7ae9600700 1 --2- 172.21.15.131:0/556884511 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7ad00a1230 0x7f7ad00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:20.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.301+0000 7f7ae9600700 1 -- 172.21.15.131:0/556884511 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7ae40cc570 con 0x7f7aec111c30 2024-05-08T22:06:20.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.301+0000 7f7aebe00700 1 --2- 172.21.15.131:0/556884511 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7ad00a1230 0x7f7ad00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:20.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.302+0000 7f6697e00700 1 -- 172.21.15.131:0/1505974229 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f667c0a11b0 msgr2=0x7f667c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:20.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.302+0000 7f6697e00700 1 --2- 172.21.15.131:0/1505974229 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f667c0a11b0 0x7f667c0a3670 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f669819ab40 tx=0x7f6684009260 comp rx=0 tx=0).stop 2024-05-08T22:06:20.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.302+0000 7f6697e00700 1 -- 172.21.15.131:0/1505974229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6698101f70 msgr2=0x7f66981995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:20.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.302+0000 7f6697e00700 1 --2- 172.21.15.131:0/1505974229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6698101f70 0x7f66981995e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6688000c00 tx=0x7f668800fa00 comp rx=0 tx=0).stop 2024-05-08T22:06:20.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.302+0000 7f6697e00700 1 -- 172.21.15.131:0/1505974229 shutdown_connections 2024-05-08T22:06:20.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.302+0000 7f6697e00700 1 --2- 172.21.15.131:0/1505974229 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f667c0a11b0 0x7f667c0a3670 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.302+0000 7f6697e00700 1 --2- 172.21.15.131:0/1505974229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6698107500 0x7f669819ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.302+0000 7f6697e00700 1 --2- 172.21.15.131:0/1505974229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6698101f70 0x7f66981995e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.302+0000 7f6697e00700 1 --2- 172.21.15.131:0/1505974229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6698102920 0x7f6698199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.302+0000 7f6697e00700 1 -- 172.21.15.131:0/1505974229 >> 172.21.15.131:0/1505974229 conn(0x7f66980fb7e0 msgr2=0x7f6698100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:20.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.302+0000 7f7aebe00700 1 --2- 172.21.15.131:0/556884511 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7ad00a1230 0x7f7ad00a36f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009d60 tx=0x7f7adc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:20.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.302+0000 7f6697e00700 1 -- 172.21.15.131:0/1505974229 shutdown_connections 2024-05-08T22:06:20.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.302+0000 7f6697e00700 1 -- 172.21.15.131:0/1505974229 wait complete. 2024-05-08T22:06:20.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.303+0000 7f7ae9600700 1 -- 172.21.15.131:0/556884511 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7ae4095830 con 0x7f7aec111c30 2024-05-08T22:06:20.326 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:19 smithi131 ceph-mon[37260]: pgmap v274: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:06:20.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:19 smithi132 ceph-mon[78180]: pgmap v274: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:06:20.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.413+0000 7f9448a00700 1 -- 172.21.15.131:0/2732066889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94441005a0 msgr2=0x7f944410d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:20.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.413+0000 7f9448a00700 1 --2- 172.21.15.131:0/2732066889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94441005a0 0x7f944410d1d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9430009990 tx=0x7f9430009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.413+0000 7f9448a00700 1 -- 172.21.15.131:0/2732066889 shutdown_connections 2024-05-08T22:06:20.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.413+0000 7f9448a00700 1 --2- 172.21.15.131:0/2732066889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f944410d710 0x7f944410fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.413+0000 7f9448a00700 1 --2- 172.21.15.131:0/2732066889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94440ffbf0 0x7f94440fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.413+0000 7f9448a00700 1 --2- 172.21.15.131:0/2732066889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94441005a0 0x7f944410d1d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.413+0000 7f9448a00700 1 -- 172.21.15.131:0/2732066889 >> 172.21.15.131:0/2732066889 conn(0x7f94440fb7e0 msgr2=0x7f94440fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:20.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.413+0000 7f9448a00700 1 -- 172.21.15.131:0/2732066889 shutdown_connections 2024-05-08T22:06:20.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.413+0000 7f9448a00700 1 -- 172.21.15.131:0/2732066889 wait complete. 2024-05-08T22:06:20.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.414+0000 7f9448a00700 1 Processor -- start 2024-05-08T22:06:20.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.415+0000 7f9448a00700 1 -- start start 2024-05-08T22:06:20.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.415+0000 7f9448a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94440ffbf0 0x7f9444199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:20.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.415+0000 7f9448a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94441005a0 0x7f9444199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:20.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.415+0000 7f9448a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f944410d710 0x7f944419ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:20.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.415+0000 7f9448a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944419e400 con 0x7f94441005a0 2024-05-08T22:06:20.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.415+0000 7f9448a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944419e570 con 0x7f94440ffbf0 2024-05-08T22:06:20.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.415+0000 7f9448a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944419e730 con 0x7f944410d710 2024-05-08T22:06:20.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.415+0000 7f9441e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94441005a0 0x7f9444199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:20.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.415+0000 7f9442800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94440ffbf0 0x7f9444199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:20.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.416+0000 7f9442800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94440ffbf0 0x7f9444199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:45920/0 (socket says 172.21.15.131:45920) 2024-05-08T22:06:20.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.416+0000 7f9441e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94441005a0 0x7f9444199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45012/0 (socket says 172.21.15.131:45012) 2024-05-08T22:06:20.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.416+0000 7f9442800700 1 -- 172.21.15.131:0/1405337172 learned_addr learned my addr 172.21.15.131:0/1405337172 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:20.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.416+0000 7f9443200700 1 --2- 172.21.15.131:0/1405337172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f944410d710 0x7f944419ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:20.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.416+0000 7f9442800700 1 -- 172.21.15.131:0/1405337172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f944410d710 msgr2=0x7f944419ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:20.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.416+0000 7f9442800700 1 --2- 172.21.15.131:0/1405337172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f944410d710 0x7f944419ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.416+0000 7f9442800700 1 -- 172.21.15.131:0/1405337172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94441005a0 msgr2=0x7f9444199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:20.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.416+0000 7f9442800700 1 --2- 172.21.15.131:0/1405337172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94441005a0 0x7f9444199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.416+0000 7f9442800700 1 -- 172.21.15.131:0/1405337172 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94300095f0 con 0x7f94440ffbf0 2024-05-08T22:06:20.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.416+0000 7f9443200700 1 --2- 172.21.15.131:0/1405337172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f944410d710 0x7f944419ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:20.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.416+0000 7f9441e00700 1 --2- 172.21.15.131:0/1405337172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94441005a0 0x7f9444199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:20.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.416+0000 7f9442800700 1 --2- 172.21.15.131:0/1405337172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94440ffbf0 0x7f9444199500 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f943400d950 tx=0x7f943400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:20.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.417+0000 7f943b400700 1 -- 172.21.15.131:0/1405337172 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943400f890 con 0x7f94440ffbf0 2024-05-08T22:06:20.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.417+0000 7f943b400700 1 -- 172.21.15.131:0/1405337172 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9434010460 con 0x7f94440ffbf0 2024-05-08T22:06:20.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.417+0000 7f943b400700 1 -- 172.21.15.131:0/1405337172 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943400e610 con 0x7f94440ffbf0 2024-05-08T22:06:20.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.418+0000 7f9448a00700 1 -- 172.21.15.131:0/1405337172 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f944419ea10 con 0x7f94440ffbf0 2024-05-08T22:06:20.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.418+0000 7f9448a00700 1 -- 172.21.15.131:0/1405337172 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94441a6270 con 0x7f94440ffbf0 2024-05-08T22:06:20.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.419+0000 7f9448a00700 1 -- 172.21.15.131:0/1405337172 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9444108930 con 0x7f94440ffbf0 2024-05-08T22:06:20.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.421+0000 7f943b400700 1 -- 172.21.15.131:0/1405337172 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f94340108d0 con 0x7f94440ffbf0 2024-05-08T22:06:20.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.424+0000 7f943b400700 1 --2- 172.21.15.131:0/1405337172 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f94280a11b0 0x7f94280a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:20.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.424+0000 7f943b400700 1 -- 172.21.15.131:0/1405337172 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f94340ccc80 con 0x7f94440ffbf0 2024-05-08T22:06:20.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.424+0000 7f9441e00700 1 --2- 172.21.15.131:0/1405337172 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f94280a11b0 0x7f94280a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:20.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.425+0000 7f9441e00700 1 --2- 172.21.15.131:0/1405337172 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f94280a11b0 0x7f94280a3670 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f94300098e0 tx=0x7f9430005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:20.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.427+0000 7f943b400700 1 -- 172.21.15.131:0/1405337172 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9434095fa0 con 0x7f94440ffbf0 2024-05-08T22:06:20.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.562+0000 7f7af2a00700 1 -- 172.21.15.131:0/556884511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f7ac8005870 con 0x7f7aec111c30 2024-05-08T22:06:20.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.563+0000 7f7ae9600700 1 -- 172.21.15.131:0/556884511 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f7ae4098e50 con 0x7f7aec111c30 2024-05-08T22:06:20.564 INFO:teuthology.orchestra.run.smithi131.stdout:287762808859 2024-05-08T22:06:20.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.565+0000 7f7af2a00700 1 -- 172.21.15.131:0/556884511 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7ad00a1230 msgr2=0x7f7ad00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:20.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.565+0000 7f7af2a00700 1 --2- 172.21.15.131:0/556884511 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7ad00a1230 0x7f7ad00a36f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009d60 tx=0x7f7adc009260 comp rx=0 tx=0).stop 2024-05-08T22:06:20.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.565+0000 7f7af2a00700 1 -- 172.21.15.131:0/556884511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aec111c30 msgr2=0x7f7aec19d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:20.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.565+0000 7f7af2a00700 1 --2- 172.21.15.131:0/556884511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aec111c30 0x7f7aec19d940 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7aec1053d0 tx=0x7f7ae400d3b0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.565+0000 7f7af2a00700 1 -- 172.21.15.131:0/556884511 shutdown_connections 2024-05-08T22:06:20.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.565+0000 7f7af2a00700 1 --2- 172.21.15.131:0/556884511 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7ad00a1230 0x7f7ad00a36f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.565+0000 7f7af2a00700 1 --2- 172.21.15.131:0/556884511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aec111c30 0x7f7aec19d940 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.565+0000 7f7af2a00700 1 --2- 172.21.15.131:0/556884511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7aec073710 0x7f7aec1a3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.565+0000 7f7af2a00700 1 --2- 172.21.15.131:0/556884511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7aec075000 0x7f7aec1a3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.565+0000 7f7af2a00700 1 -- 172.21.15.131:0/556884511 >> 172.21.15.131:0/556884511 conn(0x7f7aec078190 msgr2=0x7f7aec10fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:20.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.566+0000 7f7af2a00700 1 -- 172.21.15.131:0/556884511 shutdown_connections 2024-05-08T22:06:20.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.566+0000 7f7af2a00700 1 -- 172.21.15.131:0/556884511 wait complete. 2024-05-08T22:06:20.595 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:20.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.689+0000 7f9448a00700 1 -- 172.21.15.131:0/1405337172 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f94440691d0 con 0x7f94440ffbf0 2024-05-08T22:06:20.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.689+0000 7f943b400700 1 -- 172.21.15.131:0/1405337172 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f94340995c0 con 0x7f94440ffbf0 2024-05-08T22:06:20.691 INFO:teuthology.orchestra.run.smithi131.stdout:236223201316 2024-05-08T22:06:20.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.692+0000 7f9448a00700 1 -- 172.21.15.131:0/1405337172 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f94280a11b0 msgr2=0x7f94280a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:20.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.692+0000 7f9448a00700 1 --2- 172.21.15.131:0/1405337172 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f94280a11b0 0x7f94280a3670 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f94300098e0 tx=0x7f9430005960 comp rx=0 tx=0).stop 2024-05-08T22:06:20.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.692+0000 7f9448a00700 1 -- 172.21.15.131:0/1405337172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94440ffbf0 msgr2=0x7f9444199500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:20.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.692+0000 7f9448a00700 1 --2- 172.21.15.131:0/1405337172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94440ffbf0 0x7f9444199500 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f943400d950 tx=0x7f943400dc60 comp rx=0 tx=0).stop 2024-05-08T22:06:20.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.692+0000 7f9448a00700 1 -- 172.21.15.131:0/1405337172 shutdown_connections 2024-05-08T22:06:20.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.692+0000 7f9448a00700 1 --2- 172.21.15.131:0/1405337172 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f94280a11b0 0x7f94280a3670 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.692+0000 7f9448a00700 1 --2- 172.21.15.131:0/1405337172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f944410d710 0x7f944419ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.692+0000 7f9448a00700 1 --2- 172.21.15.131:0/1405337172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94440ffbf0 0x7f9444199500 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.692+0000 7f9448a00700 1 --2- 172.21.15.131:0/1405337172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94441005a0 0x7f9444199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:20.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.692+0000 7f9448a00700 1 -- 172.21.15.131:0/1405337172 >> 172.21.15.131:0/1405337172 conn(0x7f94440fb7e0 msgr2=0x7f94440fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:20.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.692+0000 7f9448a00700 1 -- 172.21.15.131:0/1405337172 shutdown_connections 2024-05-08T22:06:20.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:20.693+0000 7f9448a00700 1 -- 172.21.15.131:0/1405337172 wait complete. 2024-05-08T22:06:21.228 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:20 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1505974229' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T22:06:21.228 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:20 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/556884511' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T22:06:21.228 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:20 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1405337172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T22:06:21.252 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:21.263 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411383 got 77309411386 for osd.2 2024-05-08T22:06:21.263 DEBUG:teuthology.parallel:result is None 2024-05-08T22:06:21.273 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972381 got 266287972383 for osd.9 2024-05-08T22:06:21.274 DEBUG:teuthology.parallel:result is None 2024-05-08T22:06:21.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:20 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1505974229' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T22:06:21.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:20 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/556884511' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T22:06:21.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:20 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1405337172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T22:06:21.301 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:21.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:20 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1505974229' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T22:06:21.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:20 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/556884511' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T22:06:21.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:20 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1405337172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T22:06:21.403 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:21.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.452+0000 7f359e200700 1 -- 172.21.15.131:0/3089758781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3598107030 msgr2=0x7f3598107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:21.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.452+0000 7f359e200700 1 --2- 172.21.15.131:0/3089758781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3598107030 0x7f3598107410 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3588009990 tx=0x7f3588009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.452+0000 7f359e200700 1 -- 172.21.15.131:0/3089758781 shutdown_connections 2024-05-08T22:06:21.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.452+0000 7f359e200700 1 --2- 172.21.15.131:0/3089758781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3598101990 0x7f359810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.452+0000 7f359e200700 1 --2- 172.21.15.131:0/3089758781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3598107030 0x7f3598107410 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.452+0000 7f359e200700 1 --2- 172.21.15.131:0/3089758781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3598100ff0 0x7f3598101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.452+0000 7f359e200700 1 -- 172.21.15.131:0/3089758781 >> 172.21.15.131:0/3089758781 conn(0x7f3598078190 msgr2=0x7f35980785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:21.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.453+0000 7f359e200700 1 -- 172.21.15.131:0/3089758781 shutdown_connections 2024-05-08T22:06:21.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.453+0000 7f359e200700 1 -- 172.21.15.131:0/3089758781 wait complete. 2024-05-08T22:06:21.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.454+0000 7f359e200700 1 Processor -- start 2024-05-08T22:06:21.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.454+0000 7f359e200700 1 -- start start 2024-05-08T22:06:21.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.454+0000 7f359e200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3598100ff0 0x7f35980732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.454+0000 7f359e200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3598101990 0x7f35980737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.454+0000 7f359e200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3598107030 0x7f3598073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.454+0000 7f359e200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35981a57b0 con 0x7f3598107030 2024-05-08T22:06:21.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.454+0000 7f359e200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35981a5920 con 0x7f3598101990 2024-05-08T22:06:21.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.454+0000 7f359e200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35981a5ae0 con 0x7f3598100ff0 2024-05-08T22:06:21.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.454+0000 7f3597e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3598107030 0x7f3598073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:21.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.454+0000 7f3596a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3598101990 0x7f35980737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:21.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.455+0000 7f3597e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3598107030 0x7f3598073d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45050/0 (socket says 172.21.15.131:45050) 2024-05-08T22:06:21.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.455+0000 7f3597e00700 1 -- 172.21.15.131:0/1527349359 learned_addr learned my addr 172.21.15.131:0/1527349359 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:21.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.455+0000 7f3597400700 1 --2- 172.21.15.131:0/1527349359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3598100ff0 0x7f35980732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:21.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.455+0000 7f3597e00700 1 -- 172.21.15.131:0/1527349359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3598100ff0 msgr2=0x7f35980732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:21.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.455+0000 7f3597e00700 1 --2- 172.21.15.131:0/1527349359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3598100ff0 0x7f35980732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.455+0000 7f3597e00700 1 -- 172.21.15.131:0/1527349359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3598101990 msgr2=0x7f35980737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:21.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.455+0000 7f3597e00700 1 --2- 172.21.15.131:0/1527349359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3598101990 0x7f35980737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.455+0000 7f3597e00700 1 -- 172.21.15.131:0/1527349359 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35880095f0 con 0x7f3598107030 2024-05-08T22:06:21.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.455+0000 7f3596a00700 1 --2- 172.21.15.131:0/1527349359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3598101990 0x7f35980737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:21.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.455+0000 7f3597e00700 1 --2- 172.21.15.131:0/1527349359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3598107030 0x7f3598073d20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f359000efc0 tx=0x7f359000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:21.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.455+0000 7f358fe00700 1 -- 172.21.15.131:0/1527349359 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3590018070 con 0x7f3598107030 2024-05-08T22:06:21.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.455+0000 7f358fe00700 1 -- 172.21.15.131:0/1527349359 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f359000f040 con 0x7f3598107030 2024-05-08T22:06:21.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.455+0000 7f359e200700 1 -- 172.21.15.131:0/1527349359 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35981a5d60 con 0x7f3598107030 2024-05-08T22:06:21.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.455+0000 7f358fe00700 1 -- 172.21.15.131:0/1527349359 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359000cc10 con 0x7f3598107030 2024-05-08T22:06:21.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.456+0000 7f359e200700 1 -- 172.21.15.131:0/1527349359 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35981a6210 con 0x7f3598107030 2024-05-08T22:06:21.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.457+0000 7f359e200700 1 -- 172.21.15.131:0/1527349359 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3598109550 con 0x7f3598107030 2024-05-08T22:06:21.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.002+0000 7fe74c200700 1 -- 172.21.15.131:0/3671347103 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe744075000 msgr2=0x7fe7440731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:21.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.002+0000 7fe74c200700 1 --2- 172.21.15.131:0/3671347103 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe744075000 0x7fe7440731d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe7380098f0 tx=0x7fe738009c00 comp rx=0 tx=0).stop 2024-05-08T22:06:21.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.002+0000 7fe74c200700 1 -- 172.21.15.131:0/3671347103 shutdown_connections 2024-05-08T22:06:21.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.002+0000 7fe74c200700 1 --2- 172.21.15.131:0/3671347103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe744111c30 0x7fe744114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.002+0000 7fe74c200700 1 --2- 172.21.15.131:0/3671347103 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe744075000 0x7fe7440731d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.002+0000 7fe74c200700 1 --2- 172.21.15.131:0/3671347103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe744073710 0x7fe744073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.002+0000 7fe74c200700 1 -- 172.21.15.131:0/3671347103 >> 172.21.15.131:0/3671347103 conn(0x7fe744078190 msgr2=0x7fe7440785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:21.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.002+0000 7fe74c200700 1 -- 172.21.15.131:0/3671347103 shutdown_connections 2024-05-08T22:06:21.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.003+0000 7fe74c200700 1 -- 172.21.15.131:0/3671347103 wait complete. 2024-05-08T22:06:21.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.003+0000 7fe74c200700 1 Processor -- start 2024-05-08T22:06:21.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.004+0000 7fe74c200700 1 -- start start 2024-05-08T22:06:21.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.004+0000 7fe74c200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe744073710 0x7fe7441a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.004+0000 7fe74c200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe744075000 0x7fe7441a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.004+0000 7fe74c200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe744111c30 0x7fe74419d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.004+0000 7fe74c200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74419df90 con 0x7fe744111c30 2024-05-08T22:06:21.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.004+0000 7fe74c200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74419e100 con 0x7fe744075000 2024-05-08T22:06:21.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.004+0000 7fe74c200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74419e2c0 con 0x7fe744073710 2024-05-08T22:06:21.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.004+0000 7fe74a200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe744111c30 0x7fe74419d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:21.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.004+0000 7fe74a200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe744111c30 0x7fe74419d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45036/0 (socket says 172.21.15.131:45036) 2024-05-08T22:06:21.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.004+0000 7fe748e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe744075000 0x7fe7441a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:21.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.004+0000 7fe74a200700 1 -- 172.21.15.131:0/2830462934 learned_addr learned my addr 172.21.15.131:0/2830462934 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:21.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.004+0000 7fe749800700 1 --2- 172.21.15.131:0/2830462934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe744073710 0x7fe7441a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:21.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.005+0000 7fe74a200700 1 -- 172.21.15.131:0/2830462934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe744073710 msgr2=0x7fe7441a37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:21.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.005+0000 7fe74a200700 1 --2- 172.21.15.131:0/2830462934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe744073710 0x7fe7441a37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.005+0000 7fe74a200700 1 -- 172.21.15.131:0/2830462934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe744075000 msgr2=0x7fe7441a3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:21.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.005+0000 7fe74a200700 1 --2- 172.21.15.131:0/2830462934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe744075000 0x7fe7441a3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.005+0000 7fe74a200700 1 -- 172.21.15.131:0/2830462934 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe730009520 con 0x7fe744111c30 2024-05-08T22:06:21.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.005+0000 7fe749800700 1 --2- 172.21.15.131:0/2830462934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe744073710 0x7fe7441a37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:06:21.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.005+0000 7fe74a200700 1 --2- 172.21.15.131:0/2830462934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe744111c30 0x7fe74419d930 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe73c00ef90 tx=0x7fe73c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:21.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.005+0000 7fe742000700 1 -- 172.21.15.131:0/2830462934 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73c018070 con 0x7fe744111c30 2024-05-08T22:06:21.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.005+0000 7fe742000700 1 -- 172.21.15.131:0/2830462934 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe73c00f040 con 0x7fe744111c30 2024-05-08T22:06:21.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.459+0000 7f358fe00700 1 -- 172.21.15.131:0/1527349359 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f3590003a00 con 0x7f3598107030 2024-05-08T22:06:21.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.005+0000 7fe74c200700 1 -- 172.21.15.131:0/2830462934 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe738009520 con 0x7fe744111c30 2024-05-08T22:06:21.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.005+0000 7fe742000700 1 -- 172.21.15.131:0/2830462934 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73c00cc50 con 0x7fe744111c30 2024-05-08T22:06:21.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.005+0000 7fe74c200700 1 -- 172.21.15.131:0/2830462934 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7441aa610 con 0x7fe744111c30 2024-05-08T22:06:21.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.006+0000 7fe74c200700 1 -- 172.21.15.131:0/2830462934 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7440748b0 con 0x7fe744111c30 2024-05-08T22:06:21.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.008+0000 7fe742000700 1 -- 172.21.15.131:0/2830462934 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fe73c0092e0 con 0x7fe744111c30 2024-05-08T22:06:21.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.009+0000 7fe742000700 1 --2- 172.21.15.131:0/2830462934 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe72c0a11b0 0x7fe72c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.010+0000 7fe742000700 1 -- 172.21.15.131:0/2830462934 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe73c0cd220 con 0x7fe744111c30 2024-05-08T22:06:21.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.010+0000 7fe749800700 1 --2- 172.21.15.131:0/2830462934 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe72c0a11b0 0x7fe72c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:21.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.011+0000 7fe749800700 1 --2- 172.21.15.131:0/2830462934 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe72c0a11b0 0x7fe72c0a3670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe738009fd0 tx=0x7fe738019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:21.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.014+0000 7fe742000700 1 -- 172.21.15.131:0/2830462934 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe73c096570 con 0x7fe744111c30 2024-05-08T22:06:21.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.235+0000 7fe74c200700 1 -- 172.21.15.131:0/2830462934 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe744117570 con 0x7fe744111c30 2024-05-08T22:06:21.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.235+0000 7fe742000700 1 -- 172.21.15.131:0/2830462934 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe73c099b90 con 0x7fe744111c30 2024-05-08T22:06:21.471 INFO:teuthology.orchestra.run.smithi131.stdout:55834574911 2024-05-08T22:06:21.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.238+0000 7fe74c200700 1 -- 172.21.15.131:0/2830462934 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe72c0a11b0 msgr2=0x7fe72c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:21.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.238+0000 7fe74c200700 1 --2- 172.21.15.131:0/2830462934 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe72c0a11b0 0x7fe72c0a3670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe738009fd0 tx=0x7fe738019040 comp rx=0 tx=0).stop 2024-05-08T22:06:21.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.238+0000 7fe74c200700 1 -- 172.21.15.131:0/2830462934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe744111c30 msgr2=0x7fe74419d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:21.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.238+0000 7fe74c200700 1 --2- 172.21.15.131:0/2830462934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe744111c30 0x7fe74419d930 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe73c00ef90 tx=0x7fe73c00c3a0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.238+0000 7fe74c200700 1 -- 172.21.15.131:0/2830462934 shutdown_connections 2024-05-08T22:06:21.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.238+0000 7fe74c200700 1 --2- 172.21.15.131:0/2830462934 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe72c0a11b0 0x7fe72c0a3670 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.238+0000 7fe74c200700 1 --2- 172.21.15.131:0/2830462934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe744111c30 0x7fe74419d930 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.238+0000 7fe74c200700 1 --2- 172.21.15.131:0/2830462934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe744073710 0x7fe7441a37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.238+0000 7fe74c200700 1 --2- 172.21.15.131:0/2830462934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe744075000 0x7fe7441a3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.238+0000 7fe74c200700 1 -- 172.21.15.131:0/2830462934 >> 172.21.15.131:0/2830462934 conn(0x7fe744078190 msgr2=0x7fe74410fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:21.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.238+0000 7fe74c200700 1 -- 172.21.15.131:0/2830462934 shutdown_connections 2024-05-08T22:06:21.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.238+0000 7fe74c200700 1 -- 172.21.15.131:0/2830462934 wait complete. 2024-05-08T22:06:21.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.461+0000 7f358fe00700 1 --2- 172.21.15.131:0/1527349359 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f357c0a1230 0x7f357c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:21.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.461+0000 7f358fe00700 1 -- 172.21.15.131:0/1527349359 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f35900cc760 con 0x7f3598107030 2024-05-08T22:06:21.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.462+0000 7f3597400700 1 --2- 172.21.15.131:0/1527349359 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f357c0a1230 0x7f357c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:21.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.463+0000 7f3597400700 1 --2- 172.21.15.131:0/1527349359 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f357c0a1230 0x7f357c0a36f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3588006010 tx=0x7f358800b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:21.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.467+0000 7f358fe00700 1 -- 172.21.15.131:0/1527349359 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3590095ab0 con 0x7f3598107030 2024-05-08T22:06:21.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.705+0000 7f359e200700 1 -- 172.21.15.131:0/1527349359 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f35981026e0 con 0x7f3598107030 2024-05-08T22:06:21.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.705+0000 7f358fe00700 1 -- 172.21.15.131:0/1527349359 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f35900990d0 con 0x7f3598107030 2024-05-08T22:06:21.707 INFO:teuthology.orchestra.run.smithi131.stdout:154618822704 2024-05-08T22:06:21.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.708+0000 7f359e200700 1 -- 172.21.15.131:0/1527349359 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f357c0a1230 msgr2=0x7f357c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:21.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.708+0000 7f359e200700 1 --2- 172.21.15.131:0/1527349359 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f357c0a1230 0x7f357c0a36f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3588006010 tx=0x7f358800b260 comp rx=0 tx=0).stop 2024-05-08T22:06:21.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.708+0000 7f359e200700 1 -- 172.21.15.131:0/1527349359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3598107030 msgr2=0x7f3598073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:21.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.708+0000 7f359e200700 1 --2- 172.21.15.131:0/1527349359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3598107030 0x7f3598073d20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f359000efc0 tx=0x7f359000c3a0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.708+0000 7f359e200700 1 -- 172.21.15.131:0/1527349359 shutdown_connections 2024-05-08T22:06:21.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.708+0000 7f359e200700 1 --2- 172.21.15.131:0/1527349359 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f357c0a1230 0x7f357c0a36f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.708+0000 7f359e200700 1 --2- 172.21.15.131:0/1527349359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3598107030 0x7f3598073d20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.708+0000 7f359e200700 1 --2- 172.21.15.131:0/1527349359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3598100ff0 0x7f35980732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.708+0000 7f359e200700 1 --2- 172.21.15.131:0/1527349359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3598101990 0x7f35980737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:21.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.708+0000 7f359e200700 1 -- 172.21.15.131:0/1527349359 >> 172.21.15.131:0/1527349359 conn(0x7f3598078190 msgr2=0x7f35980ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:21.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.708+0000 7f359e200700 1 -- 172.21.15.131:0/1527349359 shutdown_connections 2024-05-08T22:06:21.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:21.708+0000 7f359e200700 1 -- 172.21.15.131:0/1527349359 wait complete. 2024-05-08T22:06:22.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:21 smithi157 ceph-mon[77746]: pgmap v275: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T22:06:22.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:21 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2830462934' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T22:06:22.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:21 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1527349359' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T22:06:22.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:21 smithi132 ceph-mon[78180]: pgmap v275: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T22:06:22.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:21 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2830462934' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T22:06:22.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:21 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1527349359' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T22:06:22.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:21 smithi131 ceph-mon[37260]: pgmap v275: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T22:06:22.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:21 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2830462934' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T22:06:22.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:21 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1527349359' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T22:06:22.465 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808857 got 287762808859 for osd.10 2024-05-08T22:06:22.465 DEBUG:teuthology.parallel:result is None 2024-05-08T22:06:22.665 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201314 got 236223201316 for osd.8 2024-05-08T22:06:22.665 DEBUG:teuthology.parallel:result is None 2024-05-08T22:06:23.065 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822701 got 154618822704 for osd.5 2024-05-08T22:06:23.065 DEBUG:teuthology.parallel:result is None 2024-05-08T22:06:24.160 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574908 got 55834574911 for osd.1 2024-05-08T22:06:24.160 DEBUG:teuthology.parallel:result is None 2024-05-08T22:06:24.189 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:23 smithi131 ceph-mon[37260]: pgmap v276: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-08T22:06:24.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:23 smithi157 ceph-mon[77746]: pgmap v276: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-08T22:06:24.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:23 smithi132 ceph-mon[78180]: pgmap v276: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-08T22:06:25.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.917+0000 7fab23600700 1 -- 172.21.15.131:0/208446952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab1c0731d0 msgr2=0x7fab1c0735b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:25.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.917+0000 7fab23600700 1 --2- 172.21.15.131:0/208446952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab1c0731d0 0x7fab1c0735b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fab10009990 tx=0x7fab10009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:25.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.918+0000 7fab23600700 1 -- 172.21.15.131:0/208446952 shutdown_connections 2024-05-08T22:06:25.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.918+0000 7fab23600700 1 --2- 172.21.15.131:0/208446952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab1c10d9a0 0x7fab1c10fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:25.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.918+0000 7fab23600700 1 --2- 172.21.15.131:0/208446952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab1c0731d0 0x7fab1c0735b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:25.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.918+0000 7fab23600700 1 --2- 172.21.15.131:0/208446952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab1c073af0 0x7fab1c10d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:25.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.918+0000 7fab23600700 1 -- 172.21.15.131:0/208446952 >> 172.21.15.131:0/208446952 conn(0x7fab1c078190 msgr2=0x7fab1c0785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:25.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.918+0000 7fab23600700 1 -- 172.21.15.131:0/208446952 shutdown_connections 2024-05-08T22:06:25.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.918+0000 7fab23600700 1 -- 172.21.15.131:0/208446952 wait complete. 2024-05-08T22:06:25.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.919+0000 7fab23600700 1 Processor -- start 2024-05-08T22:06:25.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.919+0000 7fab23600700 1 -- start start 2024-05-08T22:06:25.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.920+0000 7fab23600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab1c0731d0 0x7fab1c1994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:25.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.920+0000 7fab23600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab1c073af0 0x7fab1c199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:25.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.920+0000 7fab23600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab1c10d9a0 0x7fab1c19dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:25.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.920+0000 7fab23600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c19e370 con 0x7fab1c10d9a0 2024-05-08T22:06:25.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.920+0000 7fab23600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c19e4e0 con 0x7fab1c073af0 2024-05-08T22:06:25.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.920+0000 7fab23600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c19e6a0 con 0x7fab1c0731d0 2024-05-08T22:06:25.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.920+0000 7fab20c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab1c0731d0 0x7fab1c1994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:25.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.920+0000 7fab20c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab1c0731d0 0x7fab1c1994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46436/0 (socket says 172.21.15.131:46436) 2024-05-08T22:06:25.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.920+0000 7fab20c00700 1 -- 172.21.15.131:0/3898374273 learned_addr learned my addr 172.21.15.131:0/3898374273 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:25.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.920+0000 7fab1be00700 1 --2- 172.21.15.131:0/3898374273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab1c073af0 0x7fab1c199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:25.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.920+0000 7fab21600700 1 --2- 172.21.15.131:0/3898374273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab1c10d9a0 0x7fab1c19dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:25.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.921+0000 7fab21600700 1 -- 172.21.15.131:0/3898374273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab1c0731d0 msgr2=0x7fab1c1994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:25.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.921+0000 7fab21600700 1 --2- 172.21.15.131:0/3898374273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab1c0731d0 0x7fab1c1994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:25.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.921+0000 7fab21600700 1 -- 172.21.15.131:0/3898374273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab1c073af0 msgr2=0x7fab1c199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:25.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.921+0000 7fab21600700 1 --2- 172.21.15.131:0/3898374273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab1c073af0 0x7fab1c199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:25.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.921+0000 7fab21600700 1 -- 172.21.15.131:0/3898374273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab100095f0 con 0x7fab1c10d9a0 2024-05-08T22:06:25.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.921+0000 7fab1be00700 1 --2- 172.21.15.131:0/3898374273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab1c073af0 0x7fab1c199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:25.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.921+0000 7fab21600700 1 --2- 172.21.15.131:0/3898374273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab1c10d9a0 0x7fab1c19dd60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fab1400ef90 tx=0x7fab1400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:25.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.921+0000 7fab19600700 1 -- 172.21.15.131:0/3898374273 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab14018070 con 0x7fab1c10d9a0 2024-05-08T22:06:25.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.921+0000 7fab19600700 1 -- 172.21.15.131:0/3898374273 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab1400f040 con 0x7fab1c10d9a0 2024-05-08T22:06:25.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.921+0000 7fab19600700 1 -- 172.21.15.131:0/3898374273 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1400cb80 con 0x7fab1c10d9a0 2024-05-08T22:06:25.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.921+0000 7fab23600700 1 -- 172.21.15.131:0/3898374273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1c19e920 con 0x7fab1c10d9a0 2024-05-08T22:06:25.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.922+0000 7fab23600700 1 -- 172.21.15.131:0/3898374273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1c1a61e0 con 0x7fab1c10d9a0 2024-05-08T22:06:25.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.923+0000 7fab23600700 1 -- 172.21.15.131:0/3898374273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab1c0748b0 con 0x7fab1c10d9a0 2024-05-08T22:06:25.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.925+0000 7fab19600700 1 -- 172.21.15.131:0/3898374273 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fab1400cd40 con 0x7fab1c10d9a0 2024-05-08T22:06:25.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.926+0000 7fab19600700 1 --2- 172.21.15.131:0/3898374273 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fab040a1230 0x7fab040a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:25.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.926+0000 7fab19600700 1 -- 172.21.15.131:0/3898374273 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fab140cd1c0 con 0x7fab1c10d9a0 2024-05-08T22:06:25.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.926+0000 7fab20c00700 1 --2- 172.21.15.131:0/3898374273 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fab040a1230 0x7fab040a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:25.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.927+0000 7fab20c00700 1 --2- 172.21.15.131:0/3898374273 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fab040a1230 0x7fab040a36f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fab10009350 tx=0x7fab10005ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:25.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:25.933+0000 7fab19600700 1 -- 172.21.15.131:0/3898374273 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fab14096510 con 0x7fab1c10d9a0 2024-05-08T22:06:26.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.047+0000 7ff643e00700 1 -- 172.21.15.131:0/1879826221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff644101f70 msgr2=0x7ff644102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:26.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.047+0000 7ff643e00700 1 --2- 172.21.15.131:0/1879826221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff644101f70 0x7ff644102350 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff634009990 tx=0x7ff634009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.048+0000 7ff643e00700 1 -- 172.21.15.131:0/1879826221 shutdown_connections 2024-05-08T22:06:26.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.048+0000 7ff643e00700 1 --2- 172.21.15.131:0/1879826221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff644107500 0x7ff64410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.048+0000 7ff643e00700 1 --2- 172.21.15.131:0/1879826221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff644101f70 0x7ff644102350 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff634009990 tx=0x7ff634009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.048+0000 7ff643e00700 1 --2- 172.21.15.131:0/1879826221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff644102920 0x7ff644106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.048+0000 7ff643e00700 1 -- 172.21.15.131:0/1879826221 >> 172.21.15.131:0/1879826221 conn(0x7ff6440fb7e0 msgr2=0x7ff6440fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:26.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.048+0000 7ff643e00700 1 -- 172.21.15.131:0/1879826221 shutdown_connections 2024-05-08T22:06:26.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.048+0000 7ff643e00700 1 -- 172.21.15.131:0/1879826221 wait complete. 2024-05-08T22:06:26.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.050+0000 7ff643e00700 1 Processor -- start 2024-05-08T22:06:26.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.050+0000 7ff643e00700 1 -- start start 2024-05-08T22:06:26.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.050+0000 7ff643e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff644102920 0x7ff644072a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:26.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.050+0000 7ff643e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff644107500 0x7ff64406aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:26.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.050+0000 7ff643e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff64406afd0 0x7ff64406b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:26.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.050+0000 7ff643e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6441ac420 con 0x7ff644102920 2024-05-08T22:06:26.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.051+0000 7ff643e00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6440ffb30 con 0x7ff644107500 2024-05-08T22:06:26.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.051+0000 7ff643e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6440ffcf0 con 0x7ff64406afd0 2024-05-08T22:06:26.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.051+0000 7ff642a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff644102920 0x7ff644072a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:26.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.051+0000 7ff642a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff644102920 0x7ff644072a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49388/0 (socket says 172.21.15.131:49388) 2024-05-08T22:06:26.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.051+0000 7ff642a00700 1 -- 172.21.15.131:0/2314382200 learned_addr learned my addr 172.21.15.131:0/2314382200 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:26.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.051+0000 7ff642000700 1 --2- 172.21.15.131:0/2314382200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff644107500 0x7ff64406aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:26.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.052+0000 7ff643400700 1 --2- 172.21.15.131:0/2314382200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff64406afd0 0x7ff64406b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:26.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.052+0000 7ff642000700 1 -- 172.21.15.131:0/2314382200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff64406afd0 msgr2=0x7ff64406b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:26.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.052+0000 7ff642000700 1 --2- 172.21.15.131:0/2314382200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff64406afd0 0x7ff64406b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.052+0000 7ff642000700 1 -- 172.21.15.131:0/2314382200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff644102920 msgr2=0x7ff644072a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:26.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.052+0000 7ff642000700 1 --2- 172.21.15.131:0/2314382200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff644102920 0x7ff644072a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.052+0000 7ff642000700 1 -- 172.21.15.131:0/2314382200 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6340095f0 con 0x7ff644107500 2024-05-08T22:06:26.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.052+0000 7ff642000700 1 --2- 172.21.15.131:0/2314382200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff644107500 0x7ff64406aa90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff63000e9c0 tx=0x7ff63000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:26.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.053+0000 7ff648a00700 1 -- 172.21.15.131:0/2314382200 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63000cbd0 con 0x7ff644107500 2024-05-08T22:06:26.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.053+0000 7ff643e00700 1 -- 172.21.15.131:0/2314382200 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6440fffd0 con 0x7ff644107500 2024-05-08T22:06:26.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.053+0000 7ff648a00700 1 -- 172.21.15.131:0/2314382200 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff63000eeb0 con 0x7ff644107500 2024-05-08T22:06:26.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.053+0000 7ff648a00700 1 -- 172.21.15.131:0/2314382200 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff630018680 con 0x7ff644107500 2024-05-08T22:06:26.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.053+0000 7ff643e00700 1 -- 172.21.15.131:0/2314382200 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff644100520 con 0x7ff644107500 2024-05-08T22:06:26.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.055+0000 7ff643e00700 1 -- 172.21.15.131:0/2314382200 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff644069230 con 0x7ff644107500 2024-05-08T22:06:26.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.058+0000 7ff648a00700 1 -- 172.21.15.131:0/2314382200 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7ff630010430 con 0x7ff644107500 2024-05-08T22:06:26.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.060+0000 7ff648a00700 1 --2- 172.21.15.131:0/2314382200 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff6280a12b0 0x7ff6280a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:26.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.060+0000 7ff648a00700 1 -- 172.21.15.131:0/2314382200 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff630014070 con 0x7ff644107500 2024-05-08T22:06:26.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.061+0000 7ff642a00700 1 --2- 172.21.15.131:0/2314382200 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff6280a12b0 0x7ff6280a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:26.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.063+0000 7ff642a00700 1 --2- 172.21.15.131:0/2314382200 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff6280a12b0 0x7ff6280a3770 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff63400b2e0 tx=0x7ff634005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:26.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.064+0000 7ff648a00700 1 -- 172.21.15.131:0/2314382200 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff630052d60 con 0x7ff644107500 2024-05-08T22:06:26.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.206+0000 7fab23600700 1 -- 172.21.15.131:0/3898374273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fab1c04ec60 con 0x7fab1c10d9a0 2024-05-08T22:06:26.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.206+0000 7fab19600700 1 -- 172.21.15.131:0/3898374273 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fab14099b30 con 0x7fab1c10d9a0 2024-05-08T22:06:26.208 INFO:teuthology.orchestra.run.smithi131.stdout:133143986228 2024-05-08T22:06:26.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.209+0000 7fab23600700 1 -- 172.21.15.131:0/3898374273 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fab040a1230 msgr2=0x7fab040a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:26.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.209+0000 7fab23600700 1 --2- 172.21.15.131:0/3898374273 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fab040a1230 0x7fab040a36f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fab10009350 tx=0x7fab10005ab0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.209+0000 7fab23600700 1 -- 172.21.15.131:0/3898374273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab1c10d9a0 msgr2=0x7fab1c19dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:26.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.209+0000 7fab23600700 1 --2- 172.21.15.131:0/3898374273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab1c10d9a0 0x7fab1c19dd60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fab1400ef90 tx=0x7fab1400c3a0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.209+0000 7fab23600700 1 -- 172.21.15.131:0/3898374273 shutdown_connections 2024-05-08T22:06:26.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.209+0000 7fab23600700 1 --2- 172.21.15.131:0/3898374273 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fab040a1230 0x7fab040a36f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.209+0000 7fab23600700 1 --2- 172.21.15.131:0/3898374273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab1c10d9a0 0x7fab1c19dd60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.209+0000 7fab23600700 1 --2- 172.21.15.131:0/3898374273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab1c0731d0 0x7fab1c1994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.209+0000 7fab23600700 1 --2- 172.21.15.131:0/3898374273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab1c073af0 0x7fab1c199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.209+0000 7fab23600700 1 -- 172.21.15.131:0/3898374273 >> 172.21.15.131:0/3898374273 conn(0x7fab1c078190 msgr2=0x7fab1c0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:26.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.210+0000 7fab23600700 1 -- 172.21.15.131:0/3898374273 shutdown_connections 2024-05-08T22:06:26.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.210+0000 7fab23600700 1 -- 172.21.15.131:0/3898374273 wait complete. 2024-05-08T22:06:26.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:25 smithi131 ceph-mon[37260]: pgmap v277: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-08T22:06:26.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:25 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:06:26.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:25 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:06:26.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:25 smithi157 ceph-mon[77746]: pgmap v277: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-08T22:06:26.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:25 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:06:26.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:25 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:06:26.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.316+0000 7ff643e00700 1 -- 172.21.15.131:0/2314382200 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff64406bf70 con 0x7ff644107500 2024-05-08T22:06:26.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.316+0000 7ff648a00700 1 -- 172.21.15.131:0/2314382200 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff6300962a0 con 0x7ff644107500 2024-05-08T22:06:26.318 INFO:teuthology.orchestra.run.smithi131.stdout:184683593771 2024-05-08T22:06:26.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.319+0000 7ff643e00700 1 -- 172.21.15.131:0/2314382200 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff6280a12b0 msgr2=0x7ff6280a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:26.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.319+0000 7ff643e00700 1 --2- 172.21.15.131:0/2314382200 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff6280a12b0 0x7ff6280a3770 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff63400b2e0 tx=0x7ff634005fd0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.319+0000 7ff643e00700 1 -- 172.21.15.131:0/2314382200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff644107500 msgr2=0x7ff64406aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:26.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.319+0000 7ff643e00700 1 --2- 172.21.15.131:0/2314382200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff644107500 0x7ff64406aa90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff63000e9c0 tx=0x7ff63000ecd0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.319+0000 7ff643e00700 1 -- 172.21.15.131:0/2314382200 shutdown_connections 2024-05-08T22:06:26.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.319+0000 7ff643e00700 1 --2- 172.21.15.131:0/2314382200 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff6280a12b0 0x7ff6280a3770 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.319+0000 7ff643e00700 1 --2- 172.21.15.131:0/2314382200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff64406afd0 0x7ff64406b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.319+0000 7ff643e00700 1 --2- 172.21.15.131:0/2314382200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff644102920 0x7ff644072a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.319+0000 7ff643e00700 1 --2- 172.21.15.131:0/2314382200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff644107500 0x7ff64406aa90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.319+0000 7ff643e00700 1 -- 172.21.15.131:0/2314382200 >> 172.21.15.131:0/2314382200 conn(0x7ff6440fb7e0 msgr2=0x7ff644105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:26.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.320+0000 7ff643e00700 1 -- 172.21.15.131:0/2314382200 shutdown_connections 2024-05-08T22:06:26.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.320+0000 7ff643e00700 1 -- 172.21.15.131:0/2314382200 wait complete. 2024-05-08T22:06:26.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:25 smithi132 ceph-mon[78180]: pgmap v277: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-08T22:06:26.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:25 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:06:26.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:25 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:06:26.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.504+0000 7f6907c00700 1 -- 172.21.15.131:0/3969175948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6900101990 msgr2=0x7f690010de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:26.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.504+0000 7f6907c00700 1 --2- 172.21.15.131:0/3969175948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6900101990 0x7f690010de70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f68f800b0a0 tx=0x7f68f800b3b0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.504+0000 7f6907c00700 1 -- 172.21.15.131:0/3969175948 shutdown_connections 2024-05-08T22:06:26.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.504+0000 7f6907c00700 1 --2- 172.21.15.131:0/3969175948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6900101990 0x7f690010de70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.504+0000 7f6907c00700 1 --2- 172.21.15.131:0/3969175948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6900107030 0x7f6900107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.504+0000 7f6907c00700 1 --2- 172.21.15.131:0/3969175948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6900100ff0 0x7f6900101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.504+0000 7f6907c00700 1 -- 172.21.15.131:0/3969175948 >> 172.21.15.131:0/3969175948 conn(0x7f6900078190 msgr2=0x7f69000785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:26.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.504+0000 7f6907c00700 1 -- 172.21.15.131:0/3969175948 shutdown_connections 2024-05-08T22:06:26.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.505+0000 7f6907c00700 1 -- 172.21.15.131:0/3969175948 wait complete. 2024-05-08T22:06:26.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.506+0000 7f6907c00700 1 Processor -- start 2024-05-08T22:06:26.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.506+0000 7f6907c00700 1 -- start start 2024-05-08T22:06:26.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.506+0000 7f6907c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6900100ff0 0x7f6900199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:26.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.506+0000 7f6907c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6900101990 0x7f6900199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:26.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.506+0000 7f6907c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6900107030 0x7f690019cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:26.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.506+0000 7f6907c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690019d590 con 0x7f6900100ff0 2024-05-08T22:06:26.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.506+0000 7f6907c00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690019d700 con 0x7f6900107030 2024-05-08T22:06:26.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.506+0000 7f6907c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690019d8c0 con 0x7f6900101990 2024-05-08T22:06:26.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.507+0000 7f68ffe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6900101990 0x7f6900199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:26.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.507+0000 7f68ffe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6900101990 0x7f6900199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46456/0 (socket says 172.21.15.131:46456) 2024-05-08T22:06:26.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.507+0000 7f68ffe00700 1 -- 172.21.15.131:0/4182506360 learned_addr learned my addr 172.21.15.131:0/4182506360 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:26.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.507+0000 7f6905200700 1 --2- 172.21.15.131:0/4182506360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6900100ff0 0x7f6900199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:26.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.507+0000 7f68ffe00700 1 -- 172.21.15.131:0/4182506360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6900107030 msgr2=0x7f690019cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:26.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.507+0000 7f6905c00700 1 --2- 172.21.15.131:0/4182506360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6900107030 0x7f690019cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:26.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.507+0000 7f68ffe00700 1 --2- 172.21.15.131:0/4182506360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6900107030 0x7f690019cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.507+0000 7f68ffe00700 1 -- 172.21.15.131:0/4182506360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6900100ff0 msgr2=0x7f6900199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:26.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.507+0000 7f68ffe00700 1 --2- 172.21.15.131:0/4182506360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6900100ff0 0x7f6900199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.507+0000 7f68ffe00700 1 -- 172.21.15.131:0/4182506360 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f8009d00 con 0x7f6900101990 2024-05-08T22:06:26.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.507+0000 7f6905200700 1 --2- 172.21.15.131:0/4182506360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6900100ff0 0x7f6900199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:06:26.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.507+0000 7f6905c00700 1 --2- 172.21.15.131:0/4182506360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6900107030 0x7f690019cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:06:26.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.508+0000 7f68ffe00700 1 --2- 172.21.15.131:0/4182506360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6900101990 0x7f6900199c90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f68f000d980 tx=0x7f68f000dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:26.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.509+0000 7f68fd600700 1 -- 172.21.15.131:0/4182506360 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f000bb60 con 0x7f6900101990 2024-05-08T22:06:26.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.509+0000 7f68fd600700 1 -- 172.21.15.131:0/4182506360 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68f0007640 con 0x7f6900101990 2024-05-08T22:06:26.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.509+0000 7f68fd600700 1 -- 172.21.15.131:0/4182506360 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f0004e90 con 0x7f6900101990 2024-05-08T22:06:26.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.509+0000 7f6907c00700 1 -- 172.21.15.131:0/4182506360 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690019dba0 con 0x7f6900101990 2024-05-08T22:06:26.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.509+0000 7f6907c00700 1 -- 172.21.15.131:0/4182506360 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69001a64f0 con 0x7f6900101990 2024-05-08T22:06:26.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.511+0000 7f6907c00700 1 -- 172.21.15.131:0/4182506360 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6900104370 con 0x7f6900101990 2024-05-08T22:06:26.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.513+0000 7f68fd600700 1 -- 172.21.15.131:0/4182506360 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f68f000bcc0 con 0x7f6900101990 2024-05-08T22:06:26.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.514+0000 7f68fd600700 1 --2- 172.21.15.131:0/4182506360 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f68e80a12b0 0x7f68e80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:26.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.514+0000 7f68fd600700 1 -- 172.21.15.131:0/4182506360 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f68f00ccc00 con 0x7f6900101990 2024-05-08T22:06:26.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.515+0000 7f6905200700 1 --2- 172.21.15.131:0/4182506360 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f68e80a12b0 0x7f68e80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:26.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.516+0000 7f6905200700 1 --2- 172.21.15.131:0/4182506360 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f68e80a12b0 0x7f68e80a3770 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f68f4005e10 tx=0x7f68f4005d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:26.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.518+0000 7f68fd600700 1 -- 172.21.15.131:0/4182506360 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f68f0095ec0 con 0x7f6900101990 2024-05-08T22:06:26.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.643+0000 7f1f44600700 1 -- 172.21.15.131:0/4118433626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f3c101990 msgr2=0x7f1f3c10de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:26.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.643+0000 7f1f44600700 1 --2- 172.21.15.131:0/4118433626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f3c101990 0x7f1f3c10de70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400b0a0 tx=0x7f1f3400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.643+0000 7f1f44600700 1 -- 172.21.15.131:0/4118433626 shutdown_connections 2024-05-08T22:06:26.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.643+0000 7f1f44600700 1 --2- 172.21.15.131:0/4118433626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f3c101990 0x7f1f3c10de70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.643+0000 7f1f44600700 1 --2- 172.21.15.131:0/4118433626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f3c107030 0x7f1f3c107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.643+0000 7f1f44600700 1 --2- 172.21.15.131:0/4118433626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f3c100ff0 0x7f1f3c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.643+0000 7f1f44600700 1 -- 172.21.15.131:0/4118433626 >> 172.21.15.131:0/4118433626 conn(0x7f1f3c078190 msgr2=0x7f1f3c0785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:26.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.643+0000 7f1f44600700 1 -- 172.21.15.131:0/4118433626 shutdown_connections 2024-05-08T22:06:26.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.643+0000 7f1f44600700 1 -- 172.21.15.131:0/4118433626 wait complete. 2024-05-08T22:06:26.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.645+0000 7f1f44600700 1 Processor -- start 2024-05-08T22:06:26.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.645+0000 7f1f44600700 1 -- start start 2024-05-08T22:06:26.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.646+0000 7f1f44600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f3c100ff0 0x7f1f3c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:26.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.646+0000 7f1f44600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f3c101990 0x7f1f3c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:26.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.646+0000 7f1f44600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f3c107030 0x7f1f3c073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:26.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.646+0000 7f1f44600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c1a5900 con 0x7f1f3c101990 2024-05-08T22:06:26.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.646+0000 7f1f44600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c1a5a70 con 0x7f1f3c107030 2024-05-08T22:06:26.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.646+0000 7f1f44600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c1a5c30 con 0x7f1f3c100ff0 2024-05-08T22:06:26.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.646+0000 7f1f41c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f3c100ff0 0x7f1f3c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:26.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.646+0000 7f1f41200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f3c101990 0x7f1f3c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:26.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.646+0000 7f1f41c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f3c100ff0 0x7f1f3c0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46478/0 (socket says 172.21.15.131:46478) 2024-05-08T22:06:26.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.646+0000 7f1f41c00700 1 -- 172.21.15.131:0/3389822572 learned_addr learned my addr 172.21.15.131:0/3389822572 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:26.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.646+0000 7f1f42600700 1 --2- 172.21.15.131:0/3389822572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f3c107030 0x7f1f3c073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:26.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.647+0000 7f1f42600700 1 -- 172.21.15.131:0/3389822572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f3c100ff0 msgr2=0x7f1f3c0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:26.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.647+0000 7f1f42600700 1 --2- 172.21.15.131:0/3389822572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f3c100ff0 0x7f1f3c0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.647+0000 7f1f42600700 1 -- 172.21.15.131:0/3389822572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f3c101990 msgr2=0x7f1f3c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:26.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.647+0000 7f1f42600700 1 --2- 172.21.15.131:0/3389822572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f3c101990 0x7f1f3c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.647+0000 7f1f42600700 1 -- 172.21.15.131:0/3389822572 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f34009d00 con 0x7f1f3c107030 2024-05-08T22:06:26.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.647+0000 7f1f41c00700 1 --2- 172.21.15.131:0/3389822572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f3c100ff0 0x7f1f3c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:26.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.647+0000 7f1f41200700 1 --2- 172.21.15.131:0/3389822572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f3c101990 0x7f1f3c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:26.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.647+0000 7f1f42600700 1 --2- 172.21.15.131:0/3389822572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f3c107030 0x7f1f3c073d20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400b070 tx=0x7f1f340077d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:26.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.647+0000 7f1f3a000700 1 -- 172.21.15.131:0/3389822572 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3400e040 con 0x7f1f3c107030 2024-05-08T22:06:26.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.647+0000 7f1f3a000700 1 -- 172.21.15.131:0/3389822572 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f3401d470 con 0x7f1f3c107030 2024-05-08T22:06:26.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.647+0000 7f1f44600700 1 -- 172.21.15.131:0/3389822572 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3c1a5eb0 con 0x7f1f3c107030 2024-05-08T22:06:26.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.647+0000 7f1f3a000700 1 -- 172.21.15.131:0/3389822572 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f340052a0 con 0x7f1f3c107030 2024-05-08T22:06:26.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.648+0000 7f1f44600700 1 -- 172.21.15.131:0/3389822572 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3c1a6320 con 0x7f1f3c107030 2024-05-08T22:06:26.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.649+0000 7f1f44600700 1 -- 172.21.15.131:0/3389822572 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f3c109550 con 0x7f1f3c107030 2024-05-08T22:06:26.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.652+0000 7f1f3a000700 1 -- 172.21.15.131:0/3389822572 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f1f3401d5e0 con 0x7f1f3c107030 2024-05-08T22:06:26.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.655+0000 7f1f3a000700 1 --2- 172.21.15.131:0/3389822572 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1f240a1230 0x7f1f240a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:26.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.655+0000 7f1f3a000700 1 -- 172.21.15.131:0/3389822572 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1f340ce350 con 0x7f1f3c107030 2024-05-08T22:06:26.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.655+0000 7f1f41c00700 1 --2- 172.21.15.131:0/3389822572 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1f240a1230 0x7f1f240a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:26.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.655+0000 7f1f3a000700 1 -- 172.21.15.131:0/3389822572 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1f3409ac60 con 0x7f1f3c107030 2024-05-08T22:06:26.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.656+0000 7f1f41c00700 1 --2- 172.21.15.131:0/3389822572 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1f240a1230 0x7f1f240a36f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1f30011550 tx=0x7f1f30009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:26.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.559+0000 7efbf4a00700 1 -- 172.21.15.131:0/2691995124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efbf0073340 msgr2=0x7efbf0073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:26.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.559+0000 7efbf4a00700 1 --2- 172.21.15.131:0/2691995124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efbf0073340 0x7efbf0073720 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efbe0009990 tx=0x7efbe0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.560+0000 7efbf4a00700 1 -- 172.21.15.131:0/2691995124 shutdown_connections 2024-05-08T22:06:26.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.560+0000 7efbf4a00700 1 --2- 172.21.15.131:0/2691995124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efbf0111dc0 0x7efbf01141b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.560+0000 7efbf4a00700 1 --2- 172.21.15.131:0/2691995124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efbf0073340 0x7efbf0073720 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.560+0000 7efbf4a00700 1 --2- 172.21.15.131:0/2691995124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efbf0073c60 0x7efbf0111880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.560+0000 7efbf4a00700 1 -- 172.21.15.131:0/2691995124 >> 172.21.15.131:0/2691995124 conn(0x7efbf00fc900 msgr2=0x7efbf00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:26.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.560+0000 7efbf4a00700 1 -- 172.21.15.131:0/2691995124 shutdown_connections 2024-05-08T22:06:26.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.560+0000 7efbf4a00700 1 -- 172.21.15.131:0/2691995124 wait complete. 2024-05-08T22:06:26.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.561+0000 7efbf4a00700 1 Processor -- start 2024-05-08T22:06:26.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.561+0000 7efbf4a00700 1 -- start start 2024-05-08T22:06:26.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.561+0000 7efbf4a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efbf0073340 0x7efbf01a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:26.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.561+0000 7efbf4a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efbf0073c60 0x7efbf01a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:26.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.561+0000 7efbf4a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efbf0111dc0 0x7efbf019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:26.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.561+0000 7efbf4a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf019dea0 con 0x7efbf0073c60 2024-05-08T22:06:26.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.561+0000 7efbf4a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf019e010 con 0x7efbf0111dc0 2024-05-08T22:06:26.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.561+0000 7efbf4a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf019e1d0 con 0x7efbf0073340 2024-05-08T22:06:26.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.562+0000 7efbede00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efbf0073c60 0x7efbf01a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:26.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.562+0000 7efbee800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efbf0073340 0x7efbf01a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:26.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.562+0000 7efbee800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efbf0073340 0x7efbf01a35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46466/0 (socket says 172.21.15.131:46466) 2024-05-08T22:06:26.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.562+0000 7efbee800700 1 -- 172.21.15.131:0/3711369660 learned_addr learned my addr 172.21.15.131:0/3711369660 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:26.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.562+0000 7efbef200700 1 --2- 172.21.15.131:0/3711369660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efbf0111dc0 0x7efbf019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:26.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.562+0000 7efbee800700 1 -- 172.21.15.131:0/3711369660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efbf0111dc0 msgr2=0x7efbf019d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:26.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.562+0000 7efbee800700 1 --2- 172.21.15.131:0/3711369660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efbf0111dc0 0x7efbf019d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.562+0000 7efbee800700 1 -- 172.21.15.131:0/3711369660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efbf0073c60 msgr2=0x7efbf01a3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:26.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.562+0000 7efbee800700 1 --2- 172.21.15.131:0/3711369660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efbf0073c60 0x7efbf01a3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.562+0000 7efbee800700 1 -- 172.21.15.131:0/3711369660 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbe00095f0 con 0x7efbf0073340 2024-05-08T22:06:26.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.563+0000 7efbee800700 1 --2- 172.21.15.131:0/3711369660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efbf0073340 0x7efbf01a35f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efbe0000c00 tx=0x7efbe0005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:26.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.563+0000 7efbe7400700 1 -- 172.21.15.131:0/3711369660 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe001d070 con 0x7efbf0073340 2024-05-08T22:06:26.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.563+0000 7efbf4a00700 1 -- 172.21.15.131:0/3711369660 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf019e450 con 0x7efbf0073340 2024-05-08T22:06:26.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.563+0000 7efbe7400700 1 -- 172.21.15.131:0/3711369660 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efbe0022470 con 0x7efbf0073340 2024-05-08T22:06:26.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.563+0000 7efbe7400700 1 -- 172.21.15.131:0/3711369660 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe0017650 con 0x7efbf0073340 2024-05-08T22:06:26.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.564+0000 7efbf4a00700 1 -- 172.21.15.131:0/3711369660 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf01aa570 con 0x7efbf0073340 2024-05-08T22:06:26.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.565+0000 7efbf4a00700 1 -- 172.21.15.131:0/3711369660 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbf01042e0 con 0x7efbf0073340 2024-05-08T22:06:26.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.566+0000 7efbe7400700 1 -- 172.21.15.131:0/3711369660 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7efbe00225e0 con 0x7efbf0073340 2024-05-08T22:06:26.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.567+0000 7efbe7400700 1 --2- 172.21.15.131:0/3711369660 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7efbd40a11b0 0x7efbd40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:26.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.567+0000 7efbe7400700 1 -- 172.21.15.131:0/3711369660 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efbe00cdf50 con 0x7efbf0073340 2024-05-08T22:06:26.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.568+0000 7efbede00700 1 --2- 172.21.15.131:0/3711369660 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7efbd40a11b0 0x7efbd40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:26.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.569+0000 7efbede00700 1 --2- 172.21.15.131:0/3711369660 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7efbd40a11b0 0x7efbd40a3670 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efbf019ee80 tx=0x7efbdc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:26.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.571+0000 7efbe7400700 1 -- 172.21.15.131:0/3711369660 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efbe0097300 con 0x7efbf0073340 2024-05-08T22:06:26.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.752+0000 7f6907c00700 1 -- 172.21.15.131:0/4182506360 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f690019aa60 con 0x7f6900101990 2024-05-08T22:06:26.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.753+0000 7f68fd600700 1 -- 172.21.15.131:0/4182506360 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f68f0016020 con 0x7f6900101990 2024-05-08T22:06:26.754 INFO:teuthology.orchestra.run.smithi131.stdout:206158430247 2024-05-08T22:06:26.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.756+0000 7f6907c00700 1 -- 172.21.15.131:0/4182506360 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f68e80a12b0 msgr2=0x7f68e80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:26.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.756+0000 7f6907c00700 1 --2- 172.21.15.131:0/4182506360 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f68e80a12b0 0x7f68e80a3770 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f68f4005e10 tx=0x7f68f4005d60 comp rx=0 tx=0).stop 2024-05-08T22:06:26.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.756+0000 7f6907c00700 1 -- 172.21.15.131:0/4182506360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6900101990 msgr2=0x7f6900199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:26.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.756+0000 7f6907c00700 1 --2- 172.21.15.131:0/4182506360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6900101990 0x7f6900199c90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f68f000d980 tx=0x7f68f000dd40 comp rx=0 tx=0).stop 2024-05-08T22:06:26.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.756+0000 7f6907c00700 1 -- 172.21.15.131:0/4182506360 shutdown_connections 2024-05-08T22:06:26.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.756+0000 7f6907c00700 1 --2- 172.21.15.131:0/4182506360 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f68e80a12b0 0x7f68e80a3770 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.756+0000 7f6907c00700 1 --2- 172.21.15.131:0/4182506360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6900107030 0x7f690019cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.756+0000 7f6907c00700 1 --2- 172.21.15.131:0/4182506360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6900100ff0 0x7f6900199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.756+0000 7f6907c00700 1 --2- 172.21.15.131:0/4182506360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6900101990 0x7f6900199c90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.756+0000 7f6907c00700 1 -- 172.21.15.131:0/4182506360 >> 172.21.15.131:0/4182506360 conn(0x7f6900078190 msgr2=0x7f690010c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:26.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.756+0000 7f6907c00700 1 -- 172.21.15.131:0/4182506360 shutdown_connections 2024-05-08T22:06:26.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.756+0000 7f6907c00700 1 -- 172.21.15.131:0/4182506360 wait complete. 2024-05-08T22:06:26.809 INFO:teuthology.orchestra.run.smithi131.stdout:317827579929 2024-05-08T22:06:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.793+0000 7efbf4a00700 1 -- 172.21.15.131:0/3711369660 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7efbf019e5e0 con 0x7efbf0073340 2024-05-08T22:06:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.793+0000 7efbe7400700 1 -- 172.21.15.131:0/3711369660 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7efbe009a920 con 0x7efbf0073340 2024-05-08T22:06:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.795+0000 7efbf4a00700 1 -- 172.21.15.131:0/3711369660 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7efbd40a11b0 msgr2=0x7efbd40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.795+0000 7efbf4a00700 1 --2- 172.21.15.131:0/3711369660 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7efbd40a11b0 0x7efbd40a3670 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efbf019ee80 tx=0x7efbdc009260 comp rx=0 tx=0).stop 2024-05-08T22:06:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.795+0000 7efbf4a00700 1 -- 172.21.15.131:0/3711369660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efbf0073340 msgr2=0x7efbf01a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.795+0000 7efbf4a00700 1 --2- 172.21.15.131:0/3711369660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efbf0073340 0x7efbf01a35f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efbe0000c00 tx=0x7efbe0005740 comp rx=0 tx=0).stop 2024-05-08T22:06:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.795+0000 7efbf4a00700 1 -- 172.21.15.131:0/3711369660 shutdown_connections 2024-05-08T22:06:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.795+0000 7efbf4a00700 1 --2- 172.21.15.131:0/3711369660 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7efbd40a11b0 0x7efbd40a3670 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.795+0000 7efbf4a00700 1 --2- 172.21.15.131:0/3711369660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efbf0111dc0 0x7efbf019d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.795+0000 7efbf4a00700 1 --2- 172.21.15.131:0/3711369660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efbf0073340 0x7efbf01a35f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.795+0000 7efbf4a00700 1 --2- 172.21.15.131:0/3711369660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efbf0073c60 0x7efbf01a3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.795+0000 7efbf4a00700 1 -- 172.21.15.131:0/3711369660 >> 172.21.15.131:0/3711369660 conn(0x7efbf00fc900 msgr2=0x7efbf00fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:26.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.796+0000 7efbf4a00700 1 -- 172.21.15.131:0/3711369660 shutdown_connections 2024-05-08T22:06:26.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.796+0000 7efbf4a00700 1 -- 172.21.15.131:0/3711369660 wait complete. 2024-05-08T22:06:26.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.876+0000 7f1f44600700 1 -- 172.21.15.131:0/3389822572 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1f3c1026e0 con 0x7f1f3c107030 2024-05-08T22:06:26.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.877+0000 7f1f3a000700 1 -- 172.21.15.131:0/3389822572 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1f3409ae40 con 0x7f1f3c107030 2024-05-08T22:06:26.878 INFO:teuthology.orchestra.run.smithi131.stdout:34359738435 2024-05-08T22:06:26.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.879+0000 7f1f44600700 1 -- 172.21.15.131:0/3389822572 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1f240a1230 msgr2=0x7f1f240a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:26.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.879+0000 7f1f44600700 1 --2- 172.21.15.131:0/3389822572 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1f240a1230 0x7f1f240a36f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1f30011550 tx=0x7f1f30009260 comp rx=0 tx=0).stop 2024-05-08T22:06:26.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.879+0000 7f1f44600700 1 -- 172.21.15.131:0/3389822572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f3c107030 msgr2=0x7f1f3c073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:26.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.879+0000 7f1f44600700 1 --2- 172.21.15.131:0/3389822572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f3c107030 0x7f1f3c073d20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400b070 tx=0x7f1f340077d0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.879+0000 7f1f44600700 1 -- 172.21.15.131:0/3389822572 shutdown_connections 2024-05-08T22:06:26.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.879+0000 7f1f44600700 1 --2- 172.21.15.131:0/3389822572 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f1f240a1230 0x7f1f240a36f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.879+0000 7f1f44600700 1 --2- 172.21.15.131:0/3389822572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f3c107030 0x7f1f3c073d20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.879+0000 7f1f44600700 1 --2- 172.21.15.131:0/3389822572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f3c100ff0 0x7f1f3c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.879+0000 7f1f44600700 1 --2- 172.21.15.131:0/3389822572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f3c101990 0x7f1f3c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:26.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.879+0000 7f1f44600700 1 -- 172.21.15.131:0/3389822572 >> 172.21.15.131:0/3389822572 conn(0x7f1f3c078190 msgr2=0x7f1f3c0ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:26.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.879+0000 7f1f44600700 1 -- 172.21.15.131:0/3389822572 shutdown_connections 2024-05-08T22:06:26.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:26.879+0000 7f1f44600700 1 -- 172.21.15.131:0/3389822572 wait complete. 2024-05-08T22:06:27.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:26 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3898374273' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T22:06:27.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:26 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2314382200' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T22:06:27.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:26 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:06:27.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:26 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:06:27.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:26 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:06:27.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:26 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/4182506360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T22:06:27.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:26 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3711369660' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T22:06:27.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:26 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3389822572' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T22:06:27.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:26 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3898374273' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T22:06:27.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:26 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2314382200' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T22:06:27.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:26 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:06:27.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:26 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:06:27.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:26 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:06:27.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:26 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/4182506360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T22:06:27.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:26 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3711369660' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T22:06:27.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:26 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3389822572' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T22:06:27.340 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593769 got 184683593771 for osd.6 2024-05-08T22:06:27.340 DEBUG:teuthology.parallel:result is None 2024-05-08T22:06:27.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:26 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3898374273' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T22:06:27.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:26 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2314382200' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T22:06:27.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:26 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:06:27.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:26 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:06:27.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:26 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' 2024-05-08T22:06:27.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:26 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/4182506360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T22:06:27.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:26 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3711369660' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T22:06:27.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:26 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3389822572' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T22:06:27.601 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986225 got 133143986228 for osd.4 2024-05-08T22:06:27.601 DEBUG:teuthology.parallel:result is None 2024-05-08T22:06:28.085 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579927 got 317827579929 for osd.11 2024-05-08T22:06:28.085 DEBUG:teuthology.parallel:result is None 2024-05-08T22:06:28.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:27 smithi131 ceph-mon[37260]: pgmap v278: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T22:06:28.270 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738432 got 34359738435 for osd.0 2024-05-08T22:06:28.271 DEBUG:teuthology.parallel:result is None 2024-05-08T22:06:28.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:27 smithi157 ceph-mon[77746]: pgmap v278: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T22:06:28.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:27 smithi132 ceph-mon[78180]: pgmap v278: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T22:06:28.420 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430244 got 206158430247 for osd.7 2024-05-08T22:06:28.420 DEBUG:teuthology.parallel:result is None 2024-05-08T22:06:28.420 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-08T22:06:28.420 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T22:06:28.785 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:30.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.062+0000 7fb9c0a00700 1 -- 172.21.15.131:0/4256324581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9bc0692f0 msgr2=0x7fb9bc10de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:30.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.062+0000 7fb9c0a00700 1 --2- 172.21.15.131:0/4256324581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9bc0692f0 0x7fb9bc10de60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400b0a0 tx=0x7fb9b400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:06:30.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.062+0000 7fb9c0a00700 1 -- 172.21.15.131:0/4256324581 shutdown_connections 2024-05-08T22:06:30.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.062+0000 7fb9c0a00700 1 --2- 172.21.15.131:0/4256324581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9bc0692f0 0x7fb9bc10de60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:30.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.062+0000 7fb9c0a00700 1 --2- 172.21.15.131:0/4256324581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9bc107020 0x7fb9bc107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:30.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.062+0000 7fb9c0a00700 1 --2- 172.21.15.131:0/4256324581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9bc068950 0x7fb9bc068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:30.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.062+0000 7fb9c0a00700 1 -- 172.21.15.131:0/4256324581 >> 172.21.15.131:0/4256324581 conn(0x7fb9bc075f80 msgr2=0x7fb9bc076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:30.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.063+0000 7fb9c0a00700 1 -- 172.21.15.131:0/4256324581 shutdown_connections 2024-05-08T22:06:30.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.063+0000 7fb9c0a00700 1 -- 172.21.15.131:0/4256324581 wait complete. 2024-05-08T22:06:30.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.064+0000 7fb9c0a00700 1 Processor -- start 2024-05-08T22:06:30.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.064+0000 7fb9c0a00700 1 -- start start 2024-05-08T22:06:30.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.065+0000 7fb9c0a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9bc068950 0x7fb9bc197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:30.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.065+0000 7fb9c0a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9bc0692f0 0x7fb9bc197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:30.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.065+0000 7fb9c0a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9bc107020 0x7fb9bc19bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:30.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.065+0000 7fb9c0a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc19c3d0 con 0x7fb9bc107020 2024-05-08T22:06:30.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.065+0000 7fb9c0a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc19c540 con 0x7fb9bc0692f0 2024-05-08T22:06:30.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.065+0000 7fb9c0a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc19c700 con 0x7fb9bc068950 2024-05-08T22:06:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.065+0000 7fb9ba800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9bc107020 0x7fb9bc19bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.065+0000 7fb9b9e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9bc068950 0x7fb9bc197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.065+0000 7fb9b9e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9bc068950 0x7fb9bc197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46486/0 (socket says 172.21.15.131:46486) 2024-05-08T22:06:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.065+0000 7fb9ba800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9bc107020 0x7fb9bc19bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49470/0 (socket says 172.21.15.131:49470) 2024-05-08T22:06:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.065+0000 7fb9b9e00700 1 -- 172.21.15.131:0/2943606333 learned_addr learned my addr 172.21.15.131:0/2943606333 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.065+0000 7fb9b9400700 1 --2- 172.21.15.131:0/2943606333 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9bc0692f0 0x7fb9bc197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.065+0000 7fb9ba800700 1 -- 172.21.15.131:0/2943606333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9bc068950 msgr2=0x7fb9bc197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.065+0000 7fb9ba800700 1 --2- 172.21.15.131:0/2943606333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9bc068950 0x7fb9bc197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.065+0000 7fb9ba800700 1 -- 172.21.15.131:0/2943606333 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9bc0692f0 msgr2=0x7fb9bc197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.065+0000 7fb9ba800700 1 --2- 172.21.15.131:0/2943606333 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9bc0692f0 0x7fb9bc197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.065+0000 7fb9ba800700 1 -- 172.21.15.131:0/2943606333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a8009520 con 0x7fb9bc107020 2024-05-08T22:06:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.066+0000 7fb9ba800700 1 --2- 172.21.15.131:0/2943606333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9bc107020 0x7fb9bc19bdc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400b070 tx=0x7fb9b4007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.066+0000 7fb9b2a00700 1 -- 172.21.15.131:0/2943606333 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b400e040 con 0x7fb9bc107020 2024-05-08T22:06:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.066+0000 7fb9b2a00700 1 -- 172.21.15.131:0/2943606333 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9b401d470 con 0x7fb9bc107020 2024-05-08T22:06:30.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.066+0000 7fb9c0a00700 1 -- 172.21.15.131:0/2943606333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b4009d00 con 0x7fb9bc107020 2024-05-08T22:06:30.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.066+0000 7fb9b2a00700 1 -- 172.21.15.131:0/2943606333 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4005280 con 0x7fb9bc107020 2024-05-08T22:06:30.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.066+0000 7fb9c0a00700 1 -- 172.21.15.131:0/2943606333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9bc19cc10 con 0x7fb9bc107020 2024-05-08T22:06:30.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.067+0000 7fb9c0a00700 1 -- 172.21.15.131:0/2943606333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9bc109540 con 0x7fb9bc107020 2024-05-08T22:06:30.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.069+0000 7fb9b2a00700 1 -- 172.21.15.131:0/2943606333 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fb9b401d5e0 con 0x7fb9bc107020 2024-05-08T22:06:30.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.070+0000 7fb9b2a00700 1 --2- 172.21.15.131:0/2943606333 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb9a00a11b0 0x7fb9a00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:30.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.070+0000 7fb9b2a00700 1 -- 172.21.15.131:0/2943606333 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb9b40ce0c0 con 0x7fb9bc107020 2024-05-08T22:06:30.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.071+0000 7fb9b9e00700 1 --2- 172.21.15.131:0/2943606333 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb9a00a11b0 0x7fb9a00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:30.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.071+0000 7fb9b9e00700 1 --2- 172.21.15.131:0/2943606333 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb9a00a11b0 0x7fb9a00a3670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc076a10 tx=0x7fb9ac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:30.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.075+0000 7fb9b2a00700 1 -- 172.21.15.131:0/2943606333 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb9b4097410 con 0x7fb9bc107020 2024-05-08T22:06:30.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:29 smithi131 ceph-mon[37260]: pgmap v279: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:06:30.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:29 smithi157 ceph-mon[77746]: pgmap v279: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:06:30.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.279+0000 7fb9c0a00700 1 -- 172.21.15.131:0/2943606333 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb9bc1987e0 con 0x7fb9a00a11b0 2024-05-08T22:06:30.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.292+0000 7fb9b2a00700 1 -- 172.21.15.131:0/2943606333 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427547 (secure 0 0 0) 0x7fb9bc1987e0 con 0x7fb9a00a11b0 2024-05-08T22:06:30.302 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:06:30.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.298+0000 7fb9c0a00700 1 -- 172.21.15.131:0/2943606333 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb9a00a11b0 msgr2=0x7fb9a00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:30.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.298+0000 7fb9c0a00700 1 --2- 172.21.15.131:0/2943606333 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb9a00a11b0 0x7fb9a00a3670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc076a10 tx=0x7fb9ac009260 comp rx=0 tx=0).stop 2024-05-08T22:06:30.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.298+0000 7fb9c0a00700 1 -- 172.21.15.131:0/2943606333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9bc107020 msgr2=0x7fb9bc19bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:30.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.298+0000 7fb9c0a00700 1 --2- 172.21.15.131:0/2943606333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9bc107020 0x7fb9bc19bdc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400b070 tx=0x7fb9b4007860 comp rx=0 tx=0).stop 2024-05-08T22:06:30.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.298+0000 7fb9c0a00700 1 -- 172.21.15.131:0/2943606333 shutdown_connections 2024-05-08T22:06:30.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.298+0000 7fb9c0a00700 1 --2- 172.21.15.131:0/2943606333 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb9a00a11b0 0x7fb9a00a3670 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:30.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.298+0000 7fb9c0a00700 1 --2- 172.21.15.131:0/2943606333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9bc107020 0x7fb9bc19bdc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:30.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.298+0000 7fb9c0a00700 1 --2- 172.21.15.131:0/2943606333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9bc068950 0x7fb9bc197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:30.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.298+0000 7fb9c0a00700 1 --2- 172.21.15.131:0/2943606333 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9bc0692f0 0x7fb9bc197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:30.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.298+0000 7fb9c0a00700 1 -- 172.21.15.131:0/2943606333 >> 172.21.15.131:0/2943606333 conn(0x7fb9bc075f80 msgr2=0x7fb9bc0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:30.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.298+0000 7fb9c0a00700 1 -- 172.21.15.131:0/2943606333 shutdown_connections 2024-05-08T22:06:30.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:30.298+0000 7fb9c0a00700 1 -- 172.21.15.131:0/2943606333 wait complete. 2024-05-08T22:06:30.307 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-08T22:06:30.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:29 smithi132 ceph-mon[78180]: pgmap v279: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:06:30.778 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2024-05-08T22:06:28.732216+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653960,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":449,"num_read_kb":344,"num_write":336,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181430,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372456,"kb_used_data":11356,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124569368,"statfs":{"total":1151940427776,"available":1151559032832,"internally_reserved":0,"allocated":11628544,"data_stored":7575999,"data_compressed":318984,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19131,"internal_metadata":369603909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003613"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.821136+0000","last_change":"2024-05-08T22:05:14.061628+0000","last_active":"2024-05-08T22:05:40.821136+0000","last_peered":"2024-05-08T22:05:40.821136+0000","last_clean":"2024-05-08T22:05:40.821136+0000","last_became_active":"2024-05-08T22:05:14.060986+0000","last_became_peered":"2024-05-08T22:05:14.060986+0000","last_unstale":"2024-05-08T22:05:40.821136+0000","last_undegraded":"2024-05-08T22:05:40.821136+0000","last_fullsized":"2024-05-08T22:05:40.821136+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:51.643118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832603+0000","last_change":"2024-05-08T22:05:14.073300+0000","last_active":"2024-05-08T22:05:40.832603+0000","last_peered":"2024-05-08T22:05:40.832603+0000","last_clean":"2024-05-08T22:05:40.832603+0000","last_became_active":"2024-05-08T22:05:14.069058+0000","last_became_peered":"2024-05-08T22:05:14.069058+0000","last_unstale":"2024-05-08T22:05:40.832603+0000","last_undegraded":"2024-05-08T22:05:40.832603+0000","last_fullsized":"2024-05-08T22:05:40.832603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:15.834956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832503+0000","last_change":"2024-05-08T22:05:14.071676+0000","last_active":"2024-05-08T22:05:40.832503+0000","last_peered":"2024-05-08T22:05:40.832503+0000","last_clean":"2024-05-08T22:05:40.832503+0000","last_became_active":"2024-05-08T22:05:14.058345+0000","last_became_peered":"2024-05-08T22:05:14.058345+0000","last_unstale":"2024-05-08T22:05:40.832503+0000","last_undegraded":"2024-05-08T22:05:40.832503+0000","last_fullsized":"2024-05-08T22:05:40.832503+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:09.239058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819679+0000","last_change":"2024-05-08T22:05:14.067170+0000","last_active":"2024-05-08T22:05:40.819679+0000","last_peered":"2024-05-08T22:05:40.819679+0000","last_clean":"2024-05-08T22:05:40.819679+0000","last_became_active":"2024-05-08T22:05:14.066850+0000","last_became_peered":"2024-05-08T22:05:14.066850+0000","last_unstale":"2024-05-08T22:05:40.819679+0000","last_undegraded":"2024-05-08T22:05:40.819679+0000","last_fullsized":"2024-05-08T22:05:40.819679+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:28.439786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829281+0000","last_change":"2024-05-08T22:05:14.065177+0000","last_active":"2024-05-08T22:05:40.829281+0000","last_peered":"2024-05-08T22:05:40.829281+0000","last_clean":"2024-05-08T22:05:40.829281+0000","last_became_active":"2024-05-08T22:05:14.064833+0000","last_became_peered":"2024-05-08T22:05:14.064833+0000","last_unstale":"2024-05-08T22:05:40.829281+0000","last_undegraded":"2024-05-08T22:05:40.829281+0000","last_fullsized":"2024-05-08T22:05:40.829281+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:11.764154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.825297+0000","last_change":"2024-05-08T22:05:14.061610+0000","last_active":"2024-05-08T22:05:40.825297+0000","last_peered":"2024-05-08T22:05:40.825297+0000","last_clean":"2024-05-08T22:05:40.825297+0000","last_became_active":"2024-05-08T22:05:14.061277+0000","last_became_peered":"2024-05-08T22:05:14.061277+0000","last_unstale":"2024-05-08T22:05:40.825297+0000","last_undegraded":"2024-05-08T22:05:40.825297+0000","last_fullsized":"2024-05-08T22:05:40.825297+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:19.154980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830795+0000","last_change":"2024-05-08T22:05:14.063941+0000","last_active":"2024-05-08T22:05:40.830795+0000","last_peered":"2024-05-08T22:05:40.830795+0000","last_clean":"2024-05-08T22:05:40.830795+0000","last_became_active":"2024-05-08T22:05:14.063579+0000","last_became_peered":"2024-05-08T22:05:14.063579+0000","last_unstale":"2024-05-08T22:05:40.830795+0000","last_undegraded":"2024-05-08T22:05:40.830795+0000","last_fullsized":"2024-05-08T22:05:40.830795+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:27.679067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826760+0000","last_change":"2024-05-08T22:05:14.060366+0000","last_active":"2024-05-08T22:05:40.826760+0000","last_peered":"2024-05-08T22:05:40.826760+0000","last_clean":"2024-05-08T22:05:40.826760+0000","last_became_active":"2024-05-08T22:05:14.059818+0000","last_became_peered":"2024-05-08T22:05:14.059818+0000","last_unstale":"2024-05-08T22:05:40.826760+0000","last_undegraded":"2024-05-08T22:05:40.826760+0000","last_fullsized":"2024-05-08T22:05:40.826760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:53.118508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.823157+0000","last_change":"2024-05-08T22:05:14.057185+0000","last_active":"2024-05-08T22:05:40.823157+0000","last_peered":"2024-05-08T22:05:40.823157+0000","last_clean":"2024-05-08T22:05:40.823157+0000","last_became_active":"2024-05-08T22:05:14.056889+0000","last_became_peered":"2024-05-08T22:05:14.056889+0000","last_unstale":"2024-05-08T22:05:40.823157+0000","last_undegraded":"2024-05-08T22:05:40.823157+0000","last_fullsized":"2024-05-08T22:05:40.823157+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:35.372705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824080+0000","last_change":"2024-05-08T22:05:14.062282+0000","last_active":"2024-05-08T22:05:40.824080+0000","last_peered":"2024-05-08T22:05:40.824080+0000","last_clean":"2024-05-08T22:05:40.824080+0000","last_became_active":"2024-05-08T22:05:14.061574+0000","last_became_peered":"2024-05-08T22:05:14.061574+0000","last_unstale":"2024-05-08T22:05:40.824080+0000","last_undegraded":"2024-05-08T22:05:40.824080+0000","last_fullsized":"2024-05-08T22:05:40.824080+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:16:32.873590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831100+0000","last_change":"2024-05-08T22:05:14.061267+0000","last_active":"2024-05-08T22:05:40.831100+0000","last_peered":"2024-05-08T22:05:40.831100+0000","last_clean":"2024-05-08T22:05:40.831100+0000","last_became_active":"2024-05-08T22:05:14.060943+0000","last_became_peered":"2024-05-08T22:05:14.060943+0000","last_unstale":"2024-05-08T22:05:40.831100+0000","last_undegraded":"2024-05-08T22:05:40.831100+0000","last_fullsized":"2024-05-08T22:05:40.831100+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:03.831138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819303+0000","last_change":"2024-05-08T22:05:14.067136+0000","last_active":"2024-05-08T22:05:40.819303+0000","last_peered":"2024-05-08T22:05:40.819303+0000","last_clean":"2024-05-08T22:05:40.819303+0000","last_became_active":"2024-05-08T22:05:14.066810+0000","last_became_peered":"2024-05-08T22:05:14.066810+0000","last_unstale":"2024-05-08T22:05:40.819303+0000","last_undegraded":"2024-05-08T22:05:40.819303+0000","last_fullsized":"2024-05-08T22:05:40.819303+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:43.001024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824060+0000","last_change":"2024-05-08T22:05:14.062119+0000","last_active":"2024-05-08T22:05:40.824060+0000","last_peered":"2024-05-08T22:05:40.824060+0000","last_clean":"2024-05-08T22:05:40.824060+0000","last_became_active":"2024-05-08T22:05:14.058977+0000","last_became_peered":"2024-05-08T22:05:14.058977+0000","last_unstale":"2024-05-08T22:05:40.824060+0000","last_undegraded":"2024-05-08T22:05:40.824060+0000","last_fullsized":"2024-05-08T22:05:40.824060+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:03.657985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832861+0000","last_change":"2024-05-08T22:05:14.067363+0000","last_active":"2024-05-08T22:05:40.832861+0000","last_peered":"2024-05-08T22:05:40.832861+0000","last_clean":"2024-05-08T22:05:40.832861+0000","last_became_active":"2024-05-08T22:05:14.066119+0000","last_became_peered":"2024-05-08T22:05:14.066119+0000","last_unstale":"2024-05-08T22:05:40.832861+0000","last_undegraded":"2024-05-08T22:05:40.832861+0000","last_fullsized":"2024-05-08T22:05:40.832861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:43.426750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832641+0000","last_change":"2024-05-08T22:05:14.073925+0000","last_active":"2024-05-08T22:05:40.832641+0000","last_peered":"2024-05-08T22:05:40.832641+0000","last_clean":"2024-05-08T22:05:40.832641+0000","last_became_active":"2024-05-08T22:05:14.072287+0000","last_became_peered":"2024-05-08T22:05:14.072287+0000","last_unstale":"2024-05-08T22:05:40.832641+0000","last_undegraded":"2024-05-08T22:05:40.832641+0000","last_fullsized":"2024-05-08T22:05:40.832641+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:13.537938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.823678+0000","last_change":"2024-05-08T22:05:14.057274+0000","last_active":"2024-05-08T22:05:40.823678+0000","last_peered":"2024-05-08T22:05:40.823678+0000","last_clean":"2024-05-08T22:05:40.823678+0000","last_became_active":"2024-05-08T22:05:14.056989+0000","last_became_peered":"2024-05-08T22:05:14.056989+0000","last_unstale":"2024-05-08T22:05:40.823678+0000","last_undegraded":"2024-05-08T22:05:40.823678+0000","last_fullsized":"2024-05-08T22:05:40.823678+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:32.316406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.821053+0000","last_change":"2024-05-08T22:05:14.069263+0000","last_active":"2024-05-08T22:05:40.821053+0000","last_peered":"2024-05-08T22:05:40.821053+0000","last_clean":"2024-05-08T22:05:40.821053+0000","last_became_active":"2024-05-08T22:05:14.068960+0000","last_became_peered":"2024-05-08T22:05:14.068960+0000","last_unstale":"2024-05-08T22:05:40.821053+0000","last_undegraded":"2024-05-08T22:05:40.821053+0000","last_fullsized":"2024-05-08T22:05:40.821053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:59:42.829110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817249+0000","last_change":"2024-05-08T22:05:14.049393+0000","last_active":"2024-05-08T22:05:40.817249+0000","last_peered":"2024-05-08T22:05:40.817249+0000","last_clean":"2024-05-08T22:05:40.817249+0000","last_became_active":"2024-05-08T22:05:14.049094+0000","last_became_peered":"2024-05-08T22:05:14.049094+0000","last_unstale":"2024-05-08T22:05:40.817249+0000","last_undegraded":"2024-05-08T22:05:40.817249+0000","last_fullsized":"2024-05-08T22:05:40.817249+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:42.106568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.818062+0000","last_change":"2024-05-08T22:05:14.055436+0000","last_active":"2024-05-08T22:05:40.818062+0000","last_peered":"2024-05-08T22:05:40.818062+0000","last_clean":"2024-05-08T22:05:40.818062+0000","last_became_active":"2024-05-08T22:05:14.055100+0000","last_became_peered":"2024-05-08T22:05:14.055100+0000","last_unstale":"2024-05-08T22:05:40.818062+0000","last_undegraded":"2024-05-08T22:05:40.818062+0000","last_fullsized":"2024-05-08T22:05:40.818062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:52.833780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.816936+0000","last_change":"2024-05-08T22:05:18.532086+0000","last_active":"2024-05-08T22:05:40.816936+0000","last_peered":"2024-05-08T22:05:40.816936+0000","last_clean":"2024-05-08T22:05:40.816936+0000","last_became_active":"2024-05-08T22:05:18.531485+0000","last_became_peered":"2024-05-08T22:05:18.531485+0000","last_unstale":"2024-05-08T22:05:40.816936+0000","last_undegraded":"2024-05-08T22:05:40.816936+0000","last_fullsized":"2024-05-08T22:05:40.816936+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:40:36.050367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817288+0000","last_change":"2024-05-08T22:05:13.549611+0000","last_active":"2024-05-08T22:05:40.817288+0000","last_peered":"2024-05-08T22:05:40.817288+0000","last_clean":"2024-05-08T22:05:40.817288+0000","last_became_active":"2024-05-08T22:05:13.549140+0000","last_became_peered":"2024-05-08T22:05:13.549140+0000","last_unstale":"2024-05-08T22:05:40.817288+0000","last_undegraded":"2024-05-08T22:05:40.817288+0000","last_fullsized":"2024-05-08T22:05:40.817288+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:54.785123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.828016+0000","last_change":"2024-05-08T22:05:14.057570+0000","last_active":"2024-05-08T22:05:40.828016+0000","last_peered":"2024-05-08T22:05:40.828016+0000","last_clean":"2024-05-08T22:05:40.828016+0000","last_became_active":"2024-05-08T22:05:14.057291+0000","last_became_peered":"2024-05-08T22:05:14.057291+0000","last_unstale":"2024-05-08T22:05:40.828016+0000","last_undegraded":"2024-05-08T22:05:40.828016+0000","last_fullsized":"2024-05-08T22:05:40.828016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:15:38.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:42.147633+0000","last_change":"2024-05-08T22:05:42.147633+0000","last_active":"2024-05-08T22:05:42.147633+0000","last_peered":"2024-05-08T22:05:42.147633+0000","last_clean":"2024-05-08T22:05:42.147633+0000","last_became_active":"2024-05-08T22:05:42.147284+0000","last_became_peered":"2024-05-08T22:05:42.147284+0000","last_unstale":"2024-05-08T22:05:42.147633+0000","last_undegraded":"2024-05-08T22:05:42.147633+0000","last_fullsized":"2024-05-08T22:05:42.147633+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:32.864756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.828112+0000","last_change":"2024-05-08T22:05:13.551351+0000","last_active":"2024-05-08T22:05:40.828112+0000","last_peered":"2024-05-08T22:05:40.828112+0000","last_clean":"2024-05-08T22:05:40.828112+0000","last_became_active":"2024-05-08T22:05:13.551014+0000","last_became_peered":"2024-05-08T22:05:13.551014+0000","last_unstale":"2024-05-08T22:05:40.828112+0000","last_undegraded":"2024-05-08T22:05:40.828112+0000","last_fullsized":"2024-05-08T22:05:40.828112+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:07:22.405251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.834013+0000","last_change":"2024-05-08T22:05:14.067270+0000","last_active":"2024-05-08T22:05:40.834013+0000","last_peered":"2024-05-08T22:05:40.834013+0000","last_clean":"2024-05-08T22:05:40.834013+0000","last_became_active":"2024-05-08T22:05:14.065496+0000","last_became_peered":"2024-05-08T22:05:14.065496+0000","last_unstale":"2024-05-08T22:05:40.834013+0000","last_undegraded":"2024-05-08T22:05:40.834013+0000","last_fullsized":"2024-05-08T22:05:40.834013+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:07.087880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819016+0000","last_change":"2024-05-08T22:05:18.526557+0000","last_active":"2024-05-08T22:05:40.819016+0000","last_peered":"2024-05-08T22:05:40.819016+0000","last_clean":"2024-05-08T22:05:40.819016+0000","last_became_active":"2024-05-08T22:05:18.526129+0000","last_became_peered":"2024-05-08T22:05:18.526129+0000","last_unstale":"2024-05-08T22:05:40.819016+0000","last_undegraded":"2024-05-08T22:05:40.819016+0000","last_fullsized":"2024-05-08T22:05:40.819016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:36.376579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.835476+0000","last_change":"2024-05-08T22:05:41.835074+0000","last_active":"2024-05-08T22:05:41.835476+0000","last_peered":"2024-05-08T22:05:41.835476+0000","last_clean":"2024-05-08T22:05:41.835476+0000","last_became_active":"2024-05-08T22:05:41.829567+0000","last_became_peered":"2024-05-08T22:05:41.829567+0000","last_unstale":"2024-05-08T22:05:41.835476+0000","last_undegraded":"2024-05-08T22:05:41.835476+0000","last_fullsized":"2024-05-08T22:05:41.835476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:15.261352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826964+0000","last_change":"2024-05-08T22:05:14.060089+0000","last_active":"2024-05-08T22:05:40.826964+0000","last_peered":"2024-05-08T22:05:40.826964+0000","last_clean":"2024-05-08T22:05:40.826964+0000","last_became_active":"2024-05-08T22:05:14.057620+0000","last_became_peered":"2024-05-08T22:05:14.057620+0000","last_unstale":"2024-05-08T22:05:40.826964+0000","last_undegraded":"2024-05-08T22:05:40.826964+0000","last_fullsized":"2024-05-08T22:05:40.826964+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:29:33.127510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.818817+0000","last_change":"2024-05-08T22:05:18.529746+0000","last_active":"2024-05-08T22:05:40.818817+0000","last_peered":"2024-05-08T22:05:40.818817+0000","last_clean":"2024-05-08T22:05:40.818817+0000","last_became_active":"2024-05-08T22:05:18.529097+0000","last_became_peered":"2024-05-08T22:05:18.529097+0000","last_unstale":"2024-05-08T22:05:40.818817+0000","last_undegraded":"2024-05-08T22:05:40.818817+0000","last_fullsized":"2024-05-08T22:05:40.818817+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:59:36.249787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.837496+0000","last_change":"2024-05-08T22:05:41.837103+0000","last_active":"2024-05-08T22:05:41.837496+0000","last_peered":"2024-05-08T22:05:41.837496+0000","last_clean":"2024-05-08T22:05:41.837496+0000","last_became_active":"2024-05-08T22:05:41.829464+0000","last_became_peered":"2024-05-08T22:05:41.829464+0000","last_unstale":"2024-05-08T22:05:41.837496+0000","last_undegraded":"2024-05-08T22:05:41.837496+0000","last_fullsized":"2024-05-08T22:05:41.837496+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:55.747487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827543+0000","last_change":"2024-05-08T22:05:14.064695+0000","last_active":"2024-05-08T22:05:40.827543+0000","last_peered":"2024-05-08T22:05:40.827543+0000","last_clean":"2024-05-08T22:05:40.827543+0000","last_became_active":"2024-05-08T22:05:14.064283+0000","last_became_peered":"2024-05-08T22:05:14.064283+0000","last_unstale":"2024-05-08T22:05:40.827543+0000","last_undegraded":"2024-05-08T22:05:40.827543+0000","last_fullsized":"2024-05-08T22:05:40.827543+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:06.980594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817381+0000","last_change":"2024-05-08T22:05:18.528475+0000","last_active":"2024-05-08T22:05:40.817381+0000","last_peered":"2024-05-08T22:05:40.817381+0000","last_clean":"2024-05-08T22:05:40.817381+0000","last_became_active":"2024-05-08T22:05:18.528089+0000","last_became_peered":"2024-05-08T22:05:18.528089+0000","last_unstale":"2024-05-08T22:05:40.817381+0000","last_undegraded":"2024-05-08T22:05:40.817381+0000","last_fullsized":"2024-05-08T22:05:40.817381+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:01.508984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":73,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:06:23.566363+0000","last_change":"2024-05-08T22:05:13.549191+0000","last_active":"2024-05-08T22:06:23.566363+0000","last_peered":"2024-05-08T22:06:23.566363+0000","last_clean":"2024-05-08T22:06:23.566363+0000","last_became_active":"2024-05-08T22:05:13.548737+0000","last_became_peered":"2024-05-08T22:05:13.548737+0000","last_unstale":"2024-05-08T22:06:23.566363+0000","last_undegraded":"2024-05-08T22:06:23.566363+0000","last_fullsized":"2024-05-08T22:06:23.566363+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:07.355656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830198+0000","last_change":"2024-05-08T22:05:14.065910+0000","last_active":"2024-05-08T22:05:40.830198+0000","last_peered":"2024-05-08T22:05:40.830198+0000","last_clean":"2024-05-08T22:05:40.830198+0000","last_became_active":"2024-05-08T22:05:14.065504+0000","last_became_peered":"2024-05-08T22:05:14.065504+0000","last_unstale":"2024-05-08T22:05:40.830198+0000","last_undegraded":"2024-05-08T22:05:40.830198+0000","last_fullsized":"2024-05-08T22:05:40.830198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:12.447292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817194+0000","last_change":"2024-05-08T22:05:18.536861+0000","last_active":"2024-05-08T22:05:40.817194+0000","last_peered":"2024-05-08T22:05:40.817194+0000","last_clean":"2024-05-08T22:05:40.817194+0000","last_became_active":"2024-05-08T22:05:18.536472+0000","last_became_peered":"2024-05-08T22:05:18.536472+0000","last_unstale":"2024-05-08T22:05:40.817194+0000","last_undegraded":"2024-05-08T22:05:40.817194+0000","last_fullsized":"2024-05-08T22:05:40.817194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:45:21.826812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.825252+0000","last_change":"2024-05-08T22:05:13.553326+0000","last_active":"2024-05-08T22:05:40.825252+0000","last_peered":"2024-05-08T22:05:40.825252+0000","last_clean":"2024-05-08T22:05:40.825252+0000","last_became_active":"2024-05-08T22:05:13.552983+0000","last_became_peered":"2024-05-08T22:05:13.552983+0000","last_unstale":"2024-05-08T22:05:40.825252+0000","last_undegraded":"2024-05-08T22:05:40.825252+0000","last_fullsized":"2024-05-08T22:05:40.825252+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:17.195253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.828199+0000","last_change":"2024-05-08T22:05:14.068617+0000","last_active":"2024-05-08T22:05:40.828199+0000","last_peered":"2024-05-08T22:05:40.828199+0000","last_clean":"2024-05-08T22:05:40.828199+0000","last_became_active":"2024-05-08T22:05:14.067400+0000","last_became_peered":"2024-05-08T22:05:14.067400+0000","last_unstale":"2024-05-08T22:05:40.828199+0000","last_undegraded":"2024-05-08T22:05:40.828199+0000","last_fullsized":"2024-05-08T22:05:40.828199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:25.488212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.820967+0000","last_change":"2024-05-08T22:05:18.538294+0000","last_active":"2024-05-08T22:05:40.820967+0000","last_peered":"2024-05-08T22:05:40.820967+0000","last_clean":"2024-05-08T22:05:40.820967+0000","last_became_active":"2024-05-08T22:05:18.537984+0000","last_became_peered":"2024-05-08T22:05:18.537984+0000","last_unstale":"2024-05-08T22:05:40.820967+0000","last_undegraded":"2024-05-08T22:05:40.820967+0000","last_fullsized":"2024-05-08T22:05:40.820967+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:32.117439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.821162+0000","last_change":"2024-05-08T22:05:13.557669+0000","last_active":"2024-05-08T22:05:40.821162+0000","last_peered":"2024-05-08T22:05:40.821162+0000","last_clean":"2024-05-08T22:05:40.821162+0000","last_became_active":"2024-05-08T22:05:13.555835+0000","last_became_peered":"2024-05-08T22:05:13.555835+0000","last_unstale":"2024-05-08T22:05:40.821162+0000","last_undegraded":"2024-05-08T22:05:40.821162+0000","last_fullsized":"2024-05-08T22:05:40.821162+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:02.377635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826728+0000","last_change":"2024-05-08T22:05:14.060144+0000","last_active":"2024-05-08T22:05:40.826728+0000","last_peered":"2024-05-08T22:05:40.826728+0000","last_clean":"2024-05-08T22:05:40.826728+0000","last_became_active":"2024-05-08T22:05:14.059387+0000","last_became_peered":"2024-05-08T22:05:14.059387+0000","last_unstale":"2024-05-08T22:05:40.826728+0000","last_undegraded":"2024-05-08T22:05:40.826728+0000","last_fullsized":"2024-05-08T22:05:40.826728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:32:28.057831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832449+0000","last_change":"2024-05-08T22:05:18.527293+0000","last_active":"2024-05-08T22:05:40.832449+0000","last_peered":"2024-05-08T22:05:40.832449+0000","last_clean":"2024-05-08T22:05:40.832449+0000","last_became_active":"2024-05-08T22:05:18.526690+0000","last_became_peered":"2024-05-08T22:05:18.526690+0000","last_unstale":"2024-05-08T22:05:40.832449+0000","last_undegraded":"2024-05-08T22:05:40.832449+0000","last_fullsized":"2024-05-08T22:05:40.832449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:50.343536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826376+0000","last_change":"2024-05-08T22:05:13.549774+0000","last_active":"2024-05-08T22:05:40.826376+0000","last_peered":"2024-05-08T22:05:40.826376+0000","last_clean":"2024-05-08T22:05:40.826376+0000","last_became_active":"2024-05-08T22:05:13.549531+0000","last_became_peered":"2024-05-08T22:05:13.549531+0000","last_unstale":"2024-05-08T22:05:40.826376+0000","last_undegraded":"2024-05-08T22:05:40.826376+0000","last_fullsized":"2024-05-08T22:05:40.826376+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:59:23.798728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.815983+0000","last_change":"2024-05-08T22:05:14.054232+0000","last_active":"2024-05-08T22:05:40.815983+0000","last_peered":"2024-05-08T22:05:40.815983+0000","last_clean":"2024-05-08T22:05:40.815983+0000","last_became_active":"2024-05-08T22:05:14.053806+0000","last_became_peered":"2024-05-08T22:05:14.053806+0000","last_unstale":"2024-05-08T22:05:40.815983+0000","last_undegraded":"2024-05-08T22:05:40.815983+0000","last_fullsized":"2024-05-08T22:05:40.815983+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:32:21.678792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827239+0000","last_change":"2024-05-08T22:05:18.535241+0000","last_active":"2024-05-08T22:05:40.827239+0000","last_peered":"2024-05-08T22:05:40.827239+0000","last_clean":"2024-05-08T22:05:40.827239+0000","last_became_active":"2024-05-08T22:05:18.534753+0000","last_became_peered":"2024-05-08T22:05:18.534753+0000","last_unstale":"2024-05-08T22:05:40.827239+0000","last_undegraded":"2024-05-08T22:05:40.827239+0000","last_fullsized":"2024-05-08T22:05:40.827239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:46:15.138437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":70,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:06:26.569953+0000","last_change":"2024-05-08T22:05:13.544899+0000","last_active":"2024-05-08T22:06:26.569953+0000","last_peered":"2024-05-08T22:06:26.569953+0000","last_clean":"2024-05-08T22:06:26.569953+0000","last_became_active":"2024-05-08T22:05:13.544498+0000","last_became_peered":"2024-05-08T22:05:13.544498+0000","last_unstale":"2024-05-08T22:06:26.569953+0000","last_undegraded":"2024-05-08T22:06:26.569953+0000","last_fullsized":"2024-05-08T22:06:26.569953+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:58.169648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.823071+0000","last_change":"2024-05-08T22:05:14.049377+0000","last_active":"2024-05-08T22:05:40.823071+0000","last_peered":"2024-05-08T22:05:40.823071+0000","last_clean":"2024-05-08T22:05:40.823071+0000","last_became_active":"2024-05-08T22:05:14.049039+0000","last_became_peered":"2024-05-08T22:05:14.049039+0000","last_unstale":"2024-05-08T22:05:40.823071+0000","last_undegraded":"2024-05-08T22:05:40.823071+0000","last_fullsized":"2024-05-08T22:05:40.823071+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:24.415392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.815955+0000","last_change":"2024-05-08T22:05:18.528912+0000","last_active":"2024-05-08T22:05:40.815955+0000","last_peered":"2024-05-08T22:05:40.815955+0000","last_clean":"2024-05-08T22:05:40.815955+0000","last_became_active":"2024-05-08T22:05:18.528433+0000","last_became_peered":"2024-05-08T22:05:18.528433+0000","last_unstale":"2024-05-08T22:05:40.815955+0000","last_undegraded":"2024-05-08T22:05:40.815955+0000","last_fullsized":"2024-05-08T22:05:40.815955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:05.574728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830944+0000","last_change":"2024-05-08T22:05:13.550214+0000","last_active":"2024-05-08T22:05:40.830944+0000","last_peered":"2024-05-08T22:05:40.830944+0000","last_clean":"2024-05-08T22:05:40.830944+0000","last_became_active":"2024-05-08T22:05:13.549717+0000","last_became_peered":"2024-05-08T22:05:13.549717+0000","last_unstale":"2024-05-08T22:05:40.830944+0000","last_undegraded":"2024-05-08T22:05:40.830944+0000","last_fullsized":"2024-05-08T22:05:40.830944+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:12:20.406333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.834200+0000","last_change":"2024-05-08T22:05:14.071214+0000","last_active":"2024-05-08T22:05:40.834200+0000","last_peered":"2024-05-08T22:05:40.834200+0000","last_clean":"2024-05-08T22:05:40.834200+0000","last_became_active":"2024-05-08T22:05:14.066777+0000","last_became_peered":"2024-05-08T22:05:14.066777+0000","last_unstale":"2024-05-08T22:05:40.834200+0000","last_undegraded":"2024-05-08T22:05:40.834200+0000","last_fullsized":"2024-05-08T22:05:40.834200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:53.360911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819973+0000","last_change":"2024-05-08T22:05:18.523688+0000","last_active":"2024-05-08T22:05:40.819973+0000","last_peered":"2024-05-08T22:05:40.819973+0000","last_clean":"2024-05-08T22:05:40.819973+0000","last_became_active":"2024-05-08T22:05:18.523236+0000","last_became_peered":"2024-05-08T22:05:18.523236+0000","last_unstale":"2024-05-08T22:05:40.819973+0000","last_undegraded":"2024-05-08T22:05:40.819973+0000","last_fullsized":"2024-05-08T22:05:40.819973+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:34.531699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827485+0000","last_change":"2024-05-08T22:05:13.551027+0000","last_active":"2024-05-08T22:05:40.827485+0000","last_peered":"2024-05-08T22:05:40.827485+0000","last_clean":"2024-05-08T22:05:40.827485+0000","last_became_active":"2024-05-08T22:05:13.550471+0000","last_became_peered":"2024-05-08T22:05:13.550471+0000","last_unstale":"2024-05-08T22:05:40.827485+0000","last_undegraded":"2024-05-08T22:05:40.827485+0000","last_fullsized":"2024-05-08T22:05:40.827485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:50.900419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827682+0000","last_change":"2024-05-08T22:05:14.065191+0000","last_active":"2024-05-08T22:05:40.827682+0000","last_peered":"2024-05-08T22:05:40.827682+0000","last_clean":"2024-05-08T22:05:40.827682+0000","last_became_active":"2024-05-08T22:05:14.064893+0000","last_became_peered":"2024-05-08T22:05:14.064893+0000","last_unstale":"2024-05-08T22:05:40.827682+0000","last_undegraded":"2024-05-08T22:05:40.827682+0000","last_fullsized":"2024-05-08T22:05:40.827682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:11.344646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.816088+0000","last_change":"2024-05-08T22:05:18.530447+0000","last_active":"2024-05-08T22:05:40.816088+0000","last_peered":"2024-05-08T22:05:40.816088+0000","last_clean":"2024-05-08T22:05:40.816088+0000","last_became_active":"2024-05-08T22:05:18.528171+0000","last_became_peered":"2024-05-08T22:05:18.528171+0000","last_unstale":"2024-05-08T22:05:40.816088+0000","last_undegraded":"2024-05-08T22:05:40.816088+0000","last_fullsized":"2024-05-08T22:05:40.816088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:09:43.347366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"85'31","reported_seq":154,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:06:27.578597+0000","last_change":"2024-05-08T22:05:13.547289+0000","last_active":"2024-05-08T22:06:27.578597+0000","last_peered":"2024-05-08T22:06:27.578597+0000","last_clean":"2024-05-08T22:06:27.578597+0000","last_became_active":"2024-05-08T22:05:13.546860+0000","last_became_peered":"2024-05-08T22:05:13.546860+0000","last_unstale":"2024-05-08T22:06:27.578597+0000","last_undegraded":"2024-05-08T22:06:27.578597+0000","last_fullsized":"2024-05-08T22:06:27.578597+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:08:50.339361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59451,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":105,"num_read_kb":111,"num_write":32,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831151+0000","last_change":"2024-05-08T22:05:14.057925+0000","last_active":"2024-05-08T22:05:40.831151+0000","last_peered":"2024-05-08T22:05:40.831151+0000","last_clean":"2024-05-08T22:05:40.831151+0000","last_became_active":"2024-05-08T22:05:14.057662+0000","last_became_peered":"2024-05-08T22:05:14.057662+0000","last_unstale":"2024-05-08T22:05:40.831151+0000","last_undegraded":"2024-05-08T22:05:40.831151+0000","last_fullsized":"2024-05-08T22:05:40.831151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:49.929610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827811+0000","last_change":"2024-05-08T22:05:18.535509+0000","last_active":"2024-05-08T22:05:40.827811+0000","last_peered":"2024-05-08T22:05:40.827811+0000","last_clean":"2024-05-08T22:05:40.827811+0000","last_became_active":"2024-05-08T22:05:18.535024+0000","last_became_peered":"2024-05-08T22:05:18.535024+0000","last_unstale":"2024-05-08T22:05:40.827811+0000","last_undegraded":"2024-05-08T22:05:40.827811+0000","last_fullsized":"2024-05-08T22:05:40.827811+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:28.098737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817350+0000","last_change":"2024-05-08T22:05:13.552096+0000","last_active":"2024-05-08T22:05:40.817350+0000","last_peered":"2024-05-08T22:05:40.817350+0000","last_clean":"2024-05-08T22:05:40.817350+0000","last_became_active":"2024-05-08T22:05:13.551655+0000","last_became_peered":"2024-05-08T22:05:13.551655+0000","last_unstale":"2024-05-08T22:05:40.817350+0000","last_undegraded":"2024-05-08T22:05:40.817350+0000","last_fullsized":"2024-05-08T22:05:40.817350+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:24.371008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.821551+0000","last_change":"2024-05-08T22:05:14.066075+0000","last_active":"2024-05-08T22:05:40.821551+0000","last_peered":"2024-05-08T22:05:40.821551+0000","last_clean":"2024-05-08T22:05:40.821551+0000","last_became_active":"2024-05-08T22:05:14.065581+0000","last_became_peered":"2024-05-08T22:05:14.065581+0000","last_unstale":"2024-05-08T22:05:40.821551+0000","last_undegraded":"2024-05-08T22:05:40.821551+0000","last_fullsized":"2024-05-08T22:05:40.821551+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:15.403714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.831782+0000","last_change":"2024-05-08T22:05:41.831782+0000","last_active":"2024-05-08T22:05:41.831782+0000","last_peered":"2024-05-08T22:05:41.831782+0000","last_clean":"2024-05-08T22:05:41.831782+0000","last_became_active":"2024-05-08T22:05:41.831335+0000","last_became_peered":"2024-05-08T22:05:41.831335+0000","last_unstale":"2024-05-08T22:05:41.831782+0000","last_undegraded":"2024-05-08T22:05:41.831782+0000","last_fullsized":"2024-05-08T22:05:41.831782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:39:34.031128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827308+0000","last_change":"2024-05-08T22:05:13.549112+0000","last_active":"2024-05-08T22:05:40.827308+0000","last_peered":"2024-05-08T22:05:40.827308+0000","last_clean":"2024-05-08T22:05:40.827308+0000","last_became_active":"2024-05-08T22:05:13.548763+0000","last_became_peered":"2024-05-08T22:05:13.548763+0000","last_unstale":"2024-05-08T22:05:40.827308+0000","last_undegraded":"2024-05-08T22:05:40.827308+0000","last_fullsized":"2024-05-08T22:05:40.827308+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:26:56.448611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830785+0000","last_change":"2024-05-08T22:05:14.067116+0000","last_active":"2024-05-08T22:05:40.830785+0000","last_peered":"2024-05-08T22:05:40.830785+0000","last_clean":"2024-05-08T22:05:40.830785+0000","last_became_active":"2024-05-08T22:05:14.064775+0000","last_became_peered":"2024-05-08T22:05:14.064775+0000","last_unstale":"2024-05-08T22:05:40.830785+0000","last_undegraded":"2024-05-08T22:05:40.830785+0000","last_fullsized":"2024-05-08T22:05:40.830785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:28:32.901405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817060+0000","last_change":"2024-05-08T22:05:18.539546+0000","last_active":"2024-05-08T22:05:40.817060+0000","last_peered":"2024-05-08T22:05:40.817060+0000","last_clean":"2024-05-08T22:05:40.817060+0000","last_became_active":"2024-05-08T22:05:18.538998+0000","last_became_peered":"2024-05-08T22:05:18.538998+0000","last_unstale":"2024-05-08T22:05:40.817060+0000","last_undegraded":"2024-05-08T22:05:40.817060+0000","last_fullsized":"2024-05-08T22:05:40.817060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:30.453881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"83'3","reported_seq":71,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:06:25.576405+0000","last_change":"2024-05-08T22:05:13.549053+0000","last_active":"2024-05-08T22:06:25.576405+0000","last_peered":"2024-05-08T22:06:25.576405+0000","last_clean":"2024-05-08T22:06:25.576405+0000","last_became_active":"2024-05-08T22:05:13.548739+0000","last_became_peered":"2024-05-08T22:05:13.548739+0000","last_unstale":"2024-05-08T22:06:25.576405+0000","last_undegraded":"2024-05-08T22:06:25.576405+0000","last_fullsized":"2024-05-08T22:06:25.576405+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:07:10.298084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830386+0000","last_change":"2024-05-08T22:05:14.061753+0000","last_active":"2024-05-08T22:05:40.830386+0000","last_peered":"2024-05-08T22:05:40.830386+0000","last_clean":"2024-05-08T22:05:40.830386+0000","last_became_active":"2024-05-08T22:05:14.061412+0000","last_became_peered":"2024-05-08T22:05:14.061412+0000","last_unstale":"2024-05-08T22:05:40.830386+0000","last_undegraded":"2024-05-08T22:05:40.830386+0000","last_fullsized":"2024-05-08T22:05:40.830386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:54.380181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824969+0000","last_change":"2024-05-08T22:05:18.532627+0000","last_active":"2024-05-08T22:05:40.824969+0000","last_peered":"2024-05-08T22:05:40.824969+0000","last_clean":"2024-05-08T22:05:40.824969+0000","last_became_active":"2024-05-08T22:05:18.532205+0000","last_became_peered":"2024-05-08T22:05:18.532205+0000","last_unstale":"2024-05-08T22:05:40.824969+0000","last_undegraded":"2024-05-08T22:05:40.824969+0000","last_fullsized":"2024-05-08T22:05:40.824969+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:20:11.028645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":61,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.841832+0000","last_change":"2024-05-08T22:05:41.841832+0000","last_active":"2024-05-08T22:05:41.841832+0000","last_peered":"2024-05-08T22:05:41.841832+0000","last_clean":"2024-05-08T22:05:41.841832+0000","last_became_active":"2024-05-08T22:05:41.829584+0000","last_became_peered":"2024-05-08T22:05:41.829584+0000","last_unstale":"2024-05-08T22:05:41.841832+0000","last_undegraded":"2024-05-08T22:05:41.841832+0000","last_fullsized":"2024-05-08T22:05:41.841832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:52:19.876215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824504+0000","last_change":"2024-05-08T22:05:14.063027+0000","last_active":"2024-05-08T22:05:40.824504+0000","last_peered":"2024-05-08T22:05:40.824504+0000","last_clean":"2024-05-08T22:05:40.824504+0000","last_became_active":"2024-05-08T22:05:14.062682+0000","last_became_peered":"2024-05-08T22:05:14.062682+0000","last_unstale":"2024-05-08T22:05:40.824504+0000","last_undegraded":"2024-05-08T22:05:40.824504+0000","last_fullsized":"2024-05-08T22:05:40.824504+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:47.248832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824420+0000","last_change":"2024-05-08T22:05:18.528370+0000","last_active":"2024-05-08T22:05:40.824420+0000","last_peered":"2024-05-08T22:05:40.824420+0000","last_clean":"2024-05-08T22:05:40.824420+0000","last_became_active":"2024-05-08T22:05:18.527990+0000","last_became_peered":"2024-05-08T22:05:18.527990+0000","last_unstale":"2024-05-08T22:05:40.824420+0000","last_undegraded":"2024-05-08T22:05:40.824420+0000","last_fullsized":"2024-05-08T22:05:40.824420+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:07.524175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.824561+0000","last_change":"2024-05-08T22:05:13.547543+0000","last_active":"2024-05-08T22:05:41.824561+0000","last_peered":"2024-05-08T22:05:41.824561+0000","last_clean":"2024-05-08T22:05:41.824561+0000","last_became_active":"2024-05-08T22:05:13.547191+0000","last_became_peered":"2024-05-08T22:05:13.547191+0000","last_unstale":"2024-05-08T22:05:41.824561+0000","last_undegraded":"2024-05-08T22:05:41.824561+0000","last_fullsized":"2024-05-08T22:05:41.824561+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:53.213239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.820871+0000","last_change":"2024-05-08T22:05:14.062677+0000","last_active":"2024-05-08T22:05:40.820871+0000","last_peered":"2024-05-08T22:05:40.820871+0000","last_clean":"2024-05-08T22:05:40.820871+0000","last_became_active":"2024-05-08T22:05:14.062306+0000","last_became_peered":"2024-05-08T22:05:14.062306+0000","last_unstale":"2024-05-08T22:05:40.820871+0000","last_undegraded":"2024-05-08T22:05:40.820871+0000","last_fullsized":"2024-05-08T22:05:40.820871+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:00.356505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830129+0000","last_change":"2024-05-08T22:05:18.617075+0000","last_active":"2024-05-08T22:05:40.830129+0000","last_peered":"2024-05-08T22:05:40.830129+0000","last_clean":"2024-05-08T22:05:40.830129+0000","last_became_active":"2024-05-08T22:05:18.616653+0000","last_became_peered":"2024-05-08T22:05:18.616653+0000","last_unstale":"2024-05-08T22:05:40.830129+0000","last_undegraded":"2024-05-08T22:05:40.830129+0000","last_fullsized":"2024-05-08T22:05:40.830129+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:30.537064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.816884+0000","last_change":"2024-05-08T22:05:13.553457+0000","last_active":"2024-05-08T22:05:40.816884+0000","last_peered":"2024-05-08T22:05:40.816884+0000","last_clean":"2024-05-08T22:05:40.816884+0000","last_became_active":"2024-05-08T22:05:13.551583+0000","last_became_peered":"2024-05-08T22:05:13.551583+0000","last_unstale":"2024-05-08T22:05:40.816884+0000","last_undegraded":"2024-05-08T22:05:40.816884+0000","last_fullsized":"2024-05-08T22:05:40.816884+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:07:05.971413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.822908+0000","last_change":"2024-05-08T22:05:14.057476+0000","last_active":"2024-05-08T22:05:40.822908+0000","last_peered":"2024-05-08T22:05:40.822908+0000","last_clean":"2024-05-08T22:05:40.822908+0000","last_became_active":"2024-05-08T22:05:14.056550+0000","last_became_peered":"2024-05-08T22:05:14.056550+0000","last_unstale":"2024-05-08T22:05:40.822908+0000","last_undegraded":"2024-05-08T22:05:40.822908+0000","last_fullsized":"2024-05-08T22:05:40.822908+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:41:29.264096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824298+0000","last_change":"2024-05-08T22:05:18.530719+0000","last_active":"2024-05-08T22:05:40.824298+0000","last_peered":"2024-05-08T22:05:40.824298+0000","last_clean":"2024-05-08T22:05:40.824298+0000","last_became_active":"2024-05-08T22:05:18.529659+0000","last_became_peered":"2024-05-08T22:05:18.529659+0000","last_unstale":"2024-05-08T22:05:40.824298+0000","last_undegraded":"2024-05-08T22:05:40.824298+0000","last_fullsized":"2024-05-08T22:05:40.824298+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:13:06.514460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830542+0000","last_change":"2024-05-08T22:05:13.556191+0000","last_active":"2024-05-08T22:05:40.830542+0000","last_peered":"2024-05-08T22:05:40.830542+0000","last_clean":"2024-05-08T22:05:40.830542+0000","last_became_active":"2024-05-08T22:05:13.555888+0000","last_became_peered":"2024-05-08T22:05:13.555888+0000","last_unstale":"2024-05-08T22:05:40.830542+0000","last_undegraded":"2024-05-08T22:05:40.830542+0000","last_fullsized":"2024-05-08T22:05:40.830542+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:05.637980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.828023+0000","last_change":"2024-05-08T22:05:41.828023+0000","last_active":"2024-05-08T22:05:41.828023+0000","last_peered":"2024-05-08T22:05:41.828023+0000","last_clean":"2024-05-08T22:05:41.828023+0000","last_became_active":"2024-05-08T22:05:41.827550+0000","last_became_peered":"2024-05-08T22:05:41.827550+0000","last_unstale":"2024-05-08T22:05:41.828023+0000","last_undegraded":"2024-05-08T22:05:41.828023+0000","last_fullsized":"2024-05-08T22:05:41.828023+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:13.234423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.828529+0000","last_change":"2024-05-08T22:05:41.828529+0000","last_active":"2024-05-08T22:05:41.828529+0000","last_peered":"2024-05-08T22:05:41.828529+0000","last_clean":"2024-05-08T22:05:41.828529+0000","last_became_active":"2024-05-08T22:05:41.828081+0000","last_became_peered":"2024-05-08T22:05:41.828081+0000","last_unstale":"2024-05-08T22:05:41.828529+0000","last_undegraded":"2024-05-08T22:05:41.828529+0000","last_fullsized":"2024-05-08T22:05:41.828529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:27.478823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826551+0000","last_change":"2024-05-08T22:05:14.069096+0000","last_active":"2024-05-08T22:05:40.826551+0000","last_peered":"2024-05-08T22:05:40.826551+0000","last_clean":"2024-05-08T22:05:40.826551+0000","last_became_active":"2024-05-08T22:05:14.068796+0000","last_became_peered":"2024-05-08T22:05:14.068796+0000","last_unstale":"2024-05-08T22:05:40.826551+0000","last_undegraded":"2024-05-08T22:05:40.826551+0000","last_fullsized":"2024-05-08T22:05:40.826551+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:52:52.391196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":70,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:06:25.575875+0000","last_change":"2024-05-08T22:05:13.559286+0000","last_active":"2024-05-08T22:06:25.575875+0000","last_peered":"2024-05-08T22:06:25.575875+0000","last_clean":"2024-05-08T22:06:25.575875+0000","last_became_active":"2024-05-08T22:05:13.558922+0000","last_became_peered":"2024-05-08T22:05:13.558922+0000","last_unstale":"2024-05-08T22:06:25.575875+0000","last_undegraded":"2024-05-08T22:06:25.575875+0000","last_fullsized":"2024-05-08T22:06:25.575875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:25:24.732422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831007+0000","last_change":"2024-05-08T22:05:18.527031+0000","last_active":"2024-05-08T22:05:40.831007+0000","last_peered":"2024-05-08T22:05:40.831007+0000","last_clean":"2024-05-08T22:05:40.831007+0000","last_became_active":"2024-05-08T22:05:18.526573+0000","last_became_peered":"2024-05-08T22:05:18.526573+0000","last_unstale":"2024-05-08T22:05:40.831007+0000","last_undegraded":"2024-05-08T22:05:40.831007+0000","last_fullsized":"2024-05-08T22:05:40.831007+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:08.492359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827472+0000","last_change":"2024-05-08T22:05:14.053930+0000","last_active":"2024-05-08T22:05:40.827472+0000","last_peered":"2024-05-08T22:05:40.827472+0000","last_clean":"2024-05-08T22:05:40.827472+0000","last_became_active":"2024-05-08T22:05:14.052968+0000","last_became_peered":"2024-05-08T22:05:14.052968+0000","last_unstale":"2024-05-08T22:05:40.827472+0000","last_undegraded":"2024-05-08T22:05:40.827472+0000","last_fullsized":"2024-05-08T22:05:40.827472+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:48.748323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819345+0000","last_change":"2024-05-08T22:05:13.545709+0000","last_active":"2024-05-08T22:05:40.819345+0000","last_peered":"2024-05-08T22:05:40.819345+0000","last_clean":"2024-05-08T22:05:40.819345+0000","last_became_active":"2024-05-08T22:05:13.545419+0000","last_became_peered":"2024-05-08T22:05:13.545419+0000","last_unstale":"2024-05-08T22:05:40.819345+0000","last_undegraded":"2024-05-08T22:05:40.819345+0000","last_fullsized":"2024-05-08T22:05:40.819345+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:44.903903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":337,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830206+0000","last_change":"2024-05-08T22:04:20.948328+0000","last_active":"2024-05-08T22:05:40.830206+0000","last_peered":"2024-05-08T22:05:40.830206+0000","last_clean":"2024-05-08T22:05:40.830206+0000","last_became_active":"2024-05-08T22:04:20.947887+0000","last_became_peered":"2024-05-08T22:04:20.947887+0000","last_unstale":"2024-05-08T22:05:40.830206+0000","last_undegraded":"2024-05-08T22:05:40.830206+0000","last_fullsized":"2024-05-08T22:05:40.830206+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:01:41.485277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:01:41.485277+0000","last_clean_scrub_stamp":"2024-05-08T22:01:41.485277+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:43:59.208120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.824591+0000","last_change":"2024-05-08T22:05:41.824591+0000","last_active":"2024-05-08T22:05:41.824591+0000","last_peered":"2024-05-08T22:05:41.824591+0000","last_clean":"2024-05-08T22:05:41.824591+0000","last_became_active":"2024-05-08T22:05:41.824053+0000","last_became_peered":"2024-05-08T22:05:41.824053+0000","last_unstale":"2024-05-08T22:05:41.824591+0000","last_undegraded":"2024-05-08T22:05:41.824591+0000","last_fullsized":"2024-05-08T22:05:41.824591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:18:57.269459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.820969+0000","last_change":"2024-05-08T22:05:14.066818+0000","last_active":"2024-05-08T22:05:40.820969+0000","last_peered":"2024-05-08T22:05:40.820969+0000","last_clean":"2024-05-08T22:05:40.820969+0000","last_became_active":"2024-05-08T22:05:14.065152+0000","last_became_peered":"2024-05-08T22:05:14.065152+0000","last_unstale":"2024-05-08T22:05:40.820969+0000","last_undegraded":"2024-05-08T22:05:40.820969+0000","last_fullsized":"2024-05-08T22:05:40.820969+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:53:19.018852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830323+0000","last_change":"2024-05-08T22:05:13.550365+0000","last_active":"2024-05-08T22:05:40.830323+0000","last_peered":"2024-05-08T22:05:40.830323+0000","last_clean":"2024-05-08T22:05:40.830323+0000","last_became_active":"2024-05-08T22:05:13.549677+0000","last_became_peered":"2024-05-08T22:05:13.549677+0000","last_unstale":"2024-05-08T22:05:40.830323+0000","last_undegraded":"2024-05-08T22:05:40.830323+0000","last_fullsized":"2024-05-08T22:05:40.830323+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:10.842838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.820719+0000","last_change":"2024-05-08T22:05:18.535777+0000","last_active":"2024-05-08T22:05:40.820719+0000","last_peered":"2024-05-08T22:05:40.820719+0000","last_clean":"2024-05-08T22:05:40.820719+0000","last_became_active":"2024-05-08T22:05:18.535327+0000","last_became_peered":"2024-05-08T22:05:18.535327+0000","last_unstale":"2024-05-08T22:05:40.820719+0000","last_undegraded":"2024-05-08T22:05:40.820719+0000","last_fullsized":"2024-05-08T22:05:40.820719+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:13.886989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831170+0000","last_change":"2024-05-08T22:05:13.554045+0000","last_active":"2024-05-08T22:05:40.831170+0000","last_peered":"2024-05-08T22:05:40.831170+0000","last_clean":"2024-05-08T22:05:40.831170+0000","last_became_active":"2024-05-08T22:05:13.553315+0000","last_became_peered":"2024-05-08T22:05:13.553315+0000","last_unstale":"2024-05-08T22:05:40.831170+0000","last_undegraded":"2024-05-08T22:05:40.831170+0000","last_fullsized":"2024-05-08T22:05:40.831170+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:13.131267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824438+0000","last_change":"2024-05-08T22:05:14.065018+0000","last_active":"2024-05-08T22:05:40.824438+0000","last_peered":"2024-05-08T22:05:40.824438+0000","last_clean":"2024-05-08T22:05:40.824438+0000","last_became_active":"2024-05-08T22:05:14.064701+0000","last_became_peered":"2024-05-08T22:05:14.064701+0000","last_unstale":"2024-05-08T22:05:40.824438+0000","last_undegraded":"2024-05-08T22:05:40.824438+0000","last_fullsized":"2024-05-08T22:05:40.824438+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:48:43.468373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826618+0000","last_change":"2024-05-08T22:05:18.617853+0000","last_active":"2024-05-08T22:05:40.826618+0000","last_peered":"2024-05-08T22:05:40.826618+0000","last_clean":"2024-05-08T22:05:40.826618+0000","last_became_active":"2024-05-08T22:05:18.616255+0000","last_became_peered":"2024-05-08T22:05:18.616255+0000","last_unstale":"2024-05-08T22:05:40.826618+0000","last_undegraded":"2024-05-08T22:05:40.826618+0000","last_fullsized":"2024-05-08T22:05:40.826618+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:13.233425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'5","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:42.786006+0000","last_change":"2024-05-08T22:05:13.546274+0000","last_active":"2024-05-08T22:05:42.786006+0000","last_peered":"2024-05-08T22:05:42.786006+0000","last_clean":"2024-05-08T22:05:42.786006+0000","last_became_active":"2024-05-08T22:05:13.543651+0000","last_became_peered":"2024-05-08T22:05:13.543651+0000","last_unstale":"2024-05-08T22:05:42.786006+0000","last_undegraded":"2024-05-08T22:05:42.786006+0000","last_fullsized":"2024-05-08T22:05:42.786006+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:21:58.601261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826229+0000","last_change":"2024-05-08T22:05:14.056535+0000","last_active":"2024-05-08T22:05:40.826229+0000","last_peered":"2024-05-08T22:05:40.826229+0000","last_clean":"2024-05-08T22:05:40.826229+0000","last_became_active":"2024-05-08T22:05:14.056211+0000","last_became_peered":"2024-05-08T22:05:14.056211+0000","last_unstale":"2024-05-08T22:05:40.826229+0000","last_undegraded":"2024-05-08T22:05:40.826229+0000","last_fullsized":"2024-05-08T22:05:40.826229+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:46:30.744490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.816916+0000","last_change":"2024-05-08T22:05:18.524939+0000","last_active":"2024-05-08T22:05:40.816916+0000","last_peered":"2024-05-08T22:05:40.816916+0000","last_clean":"2024-05-08T22:05:40.816916+0000","last_became_active":"2024-05-08T22:05:18.524518+0000","last_became_peered":"2024-05-08T22:05:18.524518+0000","last_unstale":"2024-05-08T22:05:40.816916+0000","last_undegraded":"2024-05-08T22:05:40.816916+0000","last_fullsized":"2024-05-08T22:05:40.816916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:50.192400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819897+0000","last_change":"2024-05-08T22:05:13.554346+0000","last_active":"2024-05-08T22:05:40.819897+0000","last_peered":"2024-05-08T22:05:40.819897+0000","last_clean":"2024-05-08T22:05:40.819897+0000","last_became_active":"2024-05-08T22:05:13.554080+0000","last_became_peered":"2024-05-08T22:05:13.554080+0000","last_unstale":"2024-05-08T22:05:40.819897+0000","last_undegraded":"2024-05-08T22:05:40.819897+0000","last_fullsized":"2024-05-08T22:05:40.819897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:29:49.695682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832687+0000","last_change":"2024-05-08T22:05:14.071422+0000","last_active":"2024-05-08T22:05:40.832687+0000","last_peered":"2024-05-08T22:05:40.832687+0000","last_clean":"2024-05-08T22:05:40.832687+0000","last_became_active":"2024-05-08T22:05:14.070396+0000","last_became_peered":"2024-05-08T22:05:14.070396+0000","last_unstale":"2024-05-08T22:05:40.832687+0000","last_undegraded":"2024-05-08T22:05:40.832687+0000","last_fullsized":"2024-05-08T22:05:40.832687+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:37.957983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819540+0000","last_change":"2024-05-08T22:05:18.617149+0000","last_active":"2024-05-08T22:05:40.819540+0000","last_peered":"2024-05-08T22:05:40.819540+0000","last_clean":"2024-05-08T22:05:40.819540+0000","last_became_active":"2024-05-08T22:05:18.616699+0000","last_became_peered":"2024-05-08T22:05:18.616699+0000","last_unstale":"2024-05-08T22:05:40.819540+0000","last_undegraded":"2024-05-08T22:05:40.819540+0000","last_fullsized":"2024-05-08T22:05:40.819540+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:51.167142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.823777+0000","last_change":"2024-05-08T22:05:13.551212+0000","last_active":"2024-05-08T22:05:40.823777+0000","last_peered":"2024-05-08T22:05:40.823777+0000","last_clean":"2024-05-08T22:05:40.823777+0000","last_became_active":"2024-05-08T22:05:13.550909+0000","last_became_peered":"2024-05-08T22:05:13.550909+0000","last_unstale":"2024-05-08T22:05:40.823777+0000","last_undegraded":"2024-05-08T22:05:40.823777+0000","last_fullsized":"2024-05-08T22:05:40.823777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:48.657928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829703+0000","last_change":"2024-05-08T22:05:14.053976+0000","last_active":"2024-05-08T22:05:40.829703+0000","last_peered":"2024-05-08T22:05:40.829703+0000","last_clean":"2024-05-08T22:05:40.829703+0000","last_became_active":"2024-05-08T22:05:14.053664+0000","last_became_peered":"2024-05-08T22:05:14.053664+0000","last_unstale":"2024-05-08T22:05:40.829703+0000","last_undegraded":"2024-05-08T22:05:40.829703+0000","last_fullsized":"2024-05-08T22:05:40.829703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:25:25.642768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817203+0000","last_change":"2024-05-08T22:05:18.536771+0000","last_active":"2024-05-08T22:05:40.817203+0000","last_peered":"2024-05-08T22:05:40.817203+0000","last_clean":"2024-05-08T22:05:40.817203+0000","last_became_active":"2024-05-08T22:05:18.536260+0000","last_became_peered":"2024-05-08T22:05:18.536260+0000","last_unstale":"2024-05-08T22:05:40.817203+0000","last_undegraded":"2024-05-08T22:05:40.817203+0000","last_fullsized":"2024-05-08T22:05:40.817203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:34.414381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827085+0000","last_change":"2024-05-08T22:05:13.540679+0000","last_active":"2024-05-08T22:05:40.827085+0000","last_peered":"2024-05-08T22:05:40.827085+0000","last_clean":"2024-05-08T22:05:40.827085+0000","last_became_active":"2024-05-08T22:05:13.539428+0000","last_became_peered":"2024-05-08T22:05:13.539428+0000","last_unstale":"2024-05-08T22:05:40.827085+0000","last_undegraded":"2024-05-08T22:05:40.827085+0000","last_fullsized":"2024-05-08T22:05:40.827085+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:59:34.455411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.820331+0000","last_change":"2024-05-08T22:05:14.061849+0000","last_active":"2024-05-08T22:05:40.820331+0000","last_peered":"2024-05-08T22:05:40.820331+0000","last_clean":"2024-05-08T22:05:40.820331+0000","last_became_active":"2024-05-08T22:05:14.061028+0000","last_became_peered":"2024-05-08T22:05:14.061028+0000","last_unstale":"2024-05-08T22:05:40.820331+0000","last_undegraded":"2024-05-08T22:05:40.820331+0000","last_fullsized":"2024-05-08T22:05:40.820331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:11:59.489376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.477584+0000","last_change":"2024-05-08T22:05:18.517994+0000","last_active":"2024-05-08T22:05:41.477584+0000","last_peered":"2024-05-08T22:05:41.477584+0000","last_clean":"2024-05-08T22:05:41.477584+0000","last_became_active":"2024-05-08T22:05:18.517655+0000","last_became_peered":"2024-05-08T22:05:18.517655+0000","last_unstale":"2024-05-08T22:05:41.477584+0000","last_undegraded":"2024-05-08T22:05:41.477584+0000","last_fullsized":"2024-05-08T22:05:41.477584+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:50:08.956277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"84'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.025839+0000","last_change":"2024-05-08T22:05:13.552851+0000","last_active":"2024-05-08T22:05:41.025839+0000","last_peered":"2024-05-08T22:05:41.025839+0000","last_clean":"2024-05-08T22:05:41.025839+0000","last_became_active":"2024-05-08T22:05:13.552507+0000","last_became_peered":"2024-05-08T22:05:13.552507+0000","last_unstale":"2024-05-08T22:05:41.025839+0000","last_undegraded":"2024-05-08T22:05:41.025839+0000","last_fullsized":"2024-05-08T22:05:41.025839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:29.248623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.828040+0000","last_change":"2024-05-08T22:05:14.062618+0000","last_active":"2024-05-08T22:05:40.828040+0000","last_peered":"2024-05-08T22:05:40.828040+0000","last_clean":"2024-05-08T22:05:40.828040+0000","last_became_active":"2024-05-08T22:05:14.062308+0000","last_became_peered":"2024-05-08T22:05:14.062308+0000","last_unstale":"2024-05-08T22:05:40.828040+0000","last_undegraded":"2024-05-08T22:05:40.828040+0000","last_fullsized":"2024-05-08T22:05:40.828040+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:34.195343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819863+0000","last_change":"2024-05-08T22:05:18.616846+0000","last_active":"2024-05-08T22:05:40.819863+0000","last_peered":"2024-05-08T22:05:40.819863+0000","last_clean":"2024-05-08T22:05:40.819863+0000","last_became_active":"2024-05-08T22:05:18.616369+0000","last_became_peered":"2024-05-08T22:05:18.616369+0000","last_unstale":"2024-05-08T22:05:40.819863+0000","last_undegraded":"2024-05-08T22:05:40.819863+0000","last_fullsized":"2024-05-08T22:05:40.819863+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:26.128949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829370+0000","last_change":"2024-05-08T22:05:13.549712+0000","last_active":"2024-05-08T22:05:40.829370+0000","last_peered":"2024-05-08T22:05:40.829370+0000","last_clean":"2024-05-08T22:05:40.829370+0000","last_became_active":"2024-05-08T22:05:13.549508+0000","last_became_peered":"2024-05-08T22:05:13.549508+0000","last_unstale":"2024-05-08T22:05:40.829370+0000","last_undegraded":"2024-05-08T22:05:40.829370+0000","last_fullsized":"2024-05-08T22:05:40.829370+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:30:22.706389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830535+0000","last_change":"2024-05-08T22:05:14.060920+0000","last_active":"2024-05-08T22:05:40.830535+0000","last_peered":"2024-05-08T22:05:40.830535+0000","last_clean":"2024-05-08T22:05:40.830535+0000","last_became_active":"2024-05-08T22:05:14.059914+0000","last_became_peered":"2024-05-08T22:05:14.059914+0000","last_unstale":"2024-05-08T22:05:40.830535+0000","last_undegraded":"2024-05-08T22:05:40.830535+0000","last_fullsized":"2024-05-08T22:05:40.830535+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:15.513600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.822527+0000","last_change":"2024-05-08T22:05:18.534076+0000","last_active":"2024-05-08T22:05:40.822527+0000","last_peered":"2024-05-08T22:05:40.822527+0000","last_clean":"2024-05-08T22:05:40.822527+0000","last_became_active":"2024-05-08T22:05:18.533679+0000","last_became_peered":"2024-05-08T22:05:18.533679+0000","last_unstale":"2024-05-08T22:05:40.822527+0000","last_undegraded":"2024-05-08T22:05:40.822527+0000","last_fullsized":"2024-05-08T22:05:40.822527+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:08.979193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.845713+0000","last_change":"2024-05-08T22:05:41.845354+0000","last_active":"2024-05-08T22:05:41.845713+0000","last_peered":"2024-05-08T22:05:41.845713+0000","last_clean":"2024-05-08T22:05:41.845713+0000","last_became_active":"2024-05-08T22:05:41.830506+0000","last_became_peered":"2024-05-08T22:05:41.830506+0000","last_unstale":"2024-05-08T22:05:41.845713+0000","last_undegraded":"2024-05-08T22:05:41.845713+0000","last_fullsized":"2024-05-08T22:05:41.845713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:53:23.081939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831111+0000","last_change":"2024-05-08T22:05:14.065868+0000","last_active":"2024-05-08T22:05:40.831111+0000","last_peered":"2024-05-08T22:05:40.831111+0000","last_clean":"2024-05-08T22:05:40.831111+0000","last_became_active":"2024-05-08T22:05:14.065545+0000","last_became_peered":"2024-05-08T22:05:14.065545+0000","last_unstale":"2024-05-08T22:05:40.831111+0000","last_undegraded":"2024-05-08T22:05:40.831111+0000","last_fullsized":"2024-05-08T22:05:40.831111+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:56:34.309095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832203+0000","last_change":"2024-05-08T22:05:18.534261+0000","last_active":"2024-05-08T22:05:40.832203+0000","last_peered":"2024-05-08T22:05:40.832203+0000","last_clean":"2024-05-08T22:05:40.832203+0000","last_became_active":"2024-05-08T22:05:18.531814+0000","last_became_peered":"2024-05-08T22:05:18.531814+0000","last_unstale":"2024-05-08T22:05:40.832203+0000","last_undegraded":"2024-05-08T22:05:40.832203+0000","last_fullsized":"2024-05-08T22:05:40.832203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:38.344546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827613+0000","last_change":"2024-05-08T22:05:13.549344+0000","last_active":"2024-05-08T22:05:40.827613+0000","last_peered":"2024-05-08T22:05:40.827613+0000","last_clean":"2024-05-08T22:05:40.827613+0000","last_became_active":"2024-05-08T22:05:13.548993+0000","last_became_peered":"2024-05-08T22:05:13.548993+0000","last_unstale":"2024-05-08T22:05:40.827613+0000","last_undegraded":"2024-05-08T22:05:40.827613+0000","last_fullsized":"2024-05-08T22:05:40.827613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:50.096602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830884+0000","last_change":"2024-05-08T22:05:14.065758+0000","last_active":"2024-05-08T22:05:40.830884+0000","last_peered":"2024-05-08T22:05:40.830884+0000","last_clean":"2024-05-08T22:05:40.830884+0000","last_became_active":"2024-05-08T22:05:14.065442+0000","last_became_peered":"2024-05-08T22:05:14.065442+0000","last_unstale":"2024-05-08T22:05:40.830884+0000","last_undegraded":"2024-05-08T22:05:40.830884+0000","last_fullsized":"2024-05-08T22:05:40.830884+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:43.047796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826157+0000","last_change":"2024-05-08T22:05:18.529804+0000","last_active":"2024-05-08T22:05:40.826157+0000","last_peered":"2024-05-08T22:05:40.826157+0000","last_clean":"2024-05-08T22:05:40.826157+0000","last_became_active":"2024-05-08T22:05:18.529346+0000","last_became_peered":"2024-05-08T22:05:18.529346+0000","last_unstale":"2024-05-08T22:05:40.826157+0000","last_undegraded":"2024-05-08T22:05:40.826157+0000","last_fullsized":"2024-05-08T22:05:40.826157+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:13.565707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824885+0000","last_change":"2024-05-08T22:05:13.551818+0000","last_active":"2024-05-08T22:05:40.824885+0000","last_peered":"2024-05-08T22:05:40.824885+0000","last_clean":"2024-05-08T22:05:40.824885+0000","last_became_active":"2024-05-08T22:05:13.551080+0000","last_became_peered":"2024-05-08T22:05:13.551080+0000","last_unstale":"2024-05-08T22:05:40.824885+0000","last_undegraded":"2024-05-08T22:05:40.824885+0000","last_fullsized":"2024-05-08T22:05:40.824885+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:31.636482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831045+0000","last_change":"2024-05-08T22:05:14.066572+0000","last_active":"2024-05-08T22:05:40.831045+0000","last_peered":"2024-05-08T22:05:40.831045+0000","last_clean":"2024-05-08T22:05:40.831045+0000","last_became_active":"2024-05-08T22:05:14.066272+0000","last_became_peered":"2024-05-08T22:05:14.066272+0000","last_unstale":"2024-05-08T22:05:40.831045+0000","last_undegraded":"2024-05-08T22:05:40.831045+0000","last_fullsized":"2024-05-08T22:05:40.831045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:20:27.423464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.820443+0000","last_change":"2024-05-08T22:05:14.061435+0000","last_active":"2024-05-08T22:05:40.820443+0000","last_peered":"2024-05-08T22:05:40.820443+0000","last_clean":"2024-05-08T22:05:40.820443+0000","last_became_active":"2024-05-08T22:05:14.061079+0000","last_became_peered":"2024-05-08T22:05:14.061079+0000","last_unstale":"2024-05-08T22:05:40.820443+0000","last_undegraded":"2024-05-08T22:05:40.820443+0000","last_fullsized":"2024-05-08T22:05:40.820443+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:42.684743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.828352+0000","last_change":"2024-05-08T22:05:41.828352+0000","last_active":"2024-05-08T22:05:41.828352+0000","last_peered":"2024-05-08T22:05:41.828352+0000","last_clean":"2024-05-08T22:05:41.828352+0000","last_became_active":"2024-05-08T22:05:41.827782+0000","last_became_peered":"2024-05-08T22:05:41.827782+0000","last_unstale":"2024-05-08T22:05:41.828352+0000","last_undegraded":"2024-05-08T22:05:41.828352+0000","last_fullsized":"2024-05-08T22:05:41.828352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:08.883148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817061+0000","last_change":"2024-05-08T22:05:14.066697+0000","last_active":"2024-05-08T22:05:40.817061+0000","last_peered":"2024-05-08T22:05:40.817061+0000","last_clean":"2024-05-08T22:05:40.817061+0000","last_became_active":"2024-05-08T22:05:14.065952+0000","last_became_peered":"2024-05-08T22:05:14.065952+0000","last_unstale":"2024-05-08T22:05:40.817061+0000","last_undegraded":"2024-05-08T22:05:40.817061+0000","last_fullsized":"2024-05-08T22:05:40.817061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:03:19.006644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.833301+0000","last_change":"2024-05-08T22:05:14.072295+0000","last_active":"2024-05-08T22:05:40.833301+0000","last_peered":"2024-05-08T22:05:40.833301+0000","last_clean":"2024-05-08T22:05:40.833301+0000","last_became_active":"2024-05-08T22:05:14.065034+0000","last_became_peered":"2024-05-08T22:05:14.065034+0000","last_unstale":"2024-05-08T22:05:40.833301+0000","last_undegraded":"2024-05-08T22:05:40.833301+0000","last_fullsized":"2024-05-08T22:05:40.833301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:08:41.380814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824510+0000","last_change":"2024-05-08T22:05:14.067263+0000","last_active":"2024-05-08T22:05:40.824510+0000","last_peered":"2024-05-08T22:05:40.824510+0000","last_clean":"2024-05-08T22:05:40.824510+0000","last_became_active":"2024-05-08T22:05:14.066957+0000","last_became_peered":"2024-05-08T22:05:14.066957+0000","last_unstale":"2024-05-08T22:05:40.824510+0000","last_undegraded":"2024-05-08T22:05:40.824510+0000","last_fullsized":"2024-05-08T22:05:40.824510+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:29:35.638489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.822207+0000","last_change":"2024-05-08T22:05:14.060561+0000","last_active":"2024-05-08T22:05:40.822207+0000","last_peered":"2024-05-08T22:05:40.822207+0000","last_clean":"2024-05-08T22:05:40.822207+0000","last_became_active":"2024-05-08T22:05:14.060150+0000","last_became_peered":"2024-05-08T22:05:14.060150+0000","last_unstale":"2024-05-08T22:05:40.822207+0000","last_undegraded":"2024-05-08T22:05:40.822207+0000","last_fullsized":"2024-05-08T22:05:40.822207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:22.710776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819471+0000","last_change":"2024-05-08T22:05:14.056394+0000","last_active":"2024-05-08T22:05:40.819471+0000","last_peered":"2024-05-08T22:05:40.819471+0000","last_clean":"2024-05-08T22:05:40.819471+0000","last_became_active":"2024-05-08T22:05:14.055977+0000","last_became_peered":"2024-05-08T22:05:14.055977+0000","last_unstale":"2024-05-08T22:05:40.819471+0000","last_undegraded":"2024-05-08T22:05:40.819471+0000","last_fullsized":"2024-05-08T22:05:40.819471+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:56:48.249379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829006+0000","last_change":"2024-05-08T22:05:14.061934+0000","last_active":"2024-05-08T22:05:40.829006+0000","last_peered":"2024-05-08T22:05:40.829006+0000","last_clean":"2024-05-08T22:05:40.829006+0000","last_became_active":"2024-05-08T22:05:14.061635+0000","last_became_peered":"2024-05-08T22:05:14.061635+0000","last_unstale":"2024-05-08T22:05:40.829006+0000","last_undegraded":"2024-05-08T22:05:40.829006+0000","last_fullsized":"2024-05-08T22:05:40.829006+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:02.423338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826467+0000","last_change":"2024-05-08T22:05:14.069982+0000","last_active":"2024-05-08T22:05:40.826467+0000","last_peered":"2024-05-08T22:05:40.826467+0000","last_clean":"2024-05-08T22:05:40.826467+0000","last_became_active":"2024-05-08T22:05:14.068293+0000","last_became_peered":"2024-05-08T22:05:14.068293+0000","last_unstale":"2024-05-08T22:05:40.826467+0000","last_undegraded":"2024-05-08T22:05:40.826467+0000","last_fullsized":"2024-05-08T22:05:40.826467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:36.396983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826968+0000","last_change":"2024-05-08T22:05:14.058393+0000","last_active":"2024-05-08T22:05:40.826968+0000","last_peered":"2024-05-08T22:05:40.826968+0000","last_clean":"2024-05-08T22:05:40.826968+0000","last_became_active":"2024-05-08T22:05:14.058083+0000","last_became_peered":"2024-05-08T22:05:14.058083+0000","last_unstale":"2024-05-08T22:05:40.826968+0000","last_undegraded":"2024-05-08T22:05:40.826968+0000","last_fullsized":"2024-05-08T22:05:40.826968+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:10:47.074957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.822859+0000","last_change":"2024-05-08T22:05:14.056989+0000","last_active":"2024-05-08T22:05:40.822859+0000","last_peered":"2024-05-08T22:05:40.822859+0000","last_clean":"2024-05-08T22:05:40.822859+0000","last_became_active":"2024-05-08T22:05:14.056669+0000","last_became_peered":"2024-05-08T22:05:14.056669+0000","last_unstale":"2024-05-08T22:05:40.822859+0000","last_undegraded":"2024-05-08T22:05:40.822859+0000","last_fullsized":"2024-05-08T22:05:40.822859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:56:51.173885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.823896+0000","last_change":"2024-05-08T22:05:14.058635+0000","last_active":"2024-05-08T22:05:40.823896+0000","last_peered":"2024-05-08T22:05:40.823896+0000","last_clean":"2024-05-08T22:05:40.823896+0000","last_became_active":"2024-05-08T22:05:14.058243+0000","last_became_peered":"2024-05-08T22:05:14.058243+0000","last_unstale":"2024-05-08T22:05:40.823896+0000","last_undegraded":"2024-05-08T22:05:40.823896+0000","last_fullsized":"2024-05-08T22:05:40.823896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:18.142563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.822551+0000","last_change":"2024-05-08T22:05:14.060735+0000","last_active":"2024-05-08T22:05:40.822551+0000","last_peered":"2024-05-08T22:05:40.822551+0000","last_clean":"2024-05-08T22:05:40.822551+0000","last_became_active":"2024-05-08T22:05:14.060391+0000","last_became_peered":"2024-05-08T22:05:14.060391+0000","last_unstale":"2024-05-08T22:05:40.822551+0000","last_undegraded":"2024-05-08T22:05:40.822551+0000","last_fullsized":"2024-05-08T22:05:40.822551+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:30.528705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283045,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63358,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":313,"num_read_kb":225,"num_write":153,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":694,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":190074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579930,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36124,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709028,"statfs":{"total":95995035648,"available":95958044672,"internally_reserved":0,"allocated":929792,"data_stored":470051,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36043205},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.441}]},{"osd":1,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.59499999999999997}]},{"osd":2,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.47699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":4,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.67900000000000005}]},{"osd":5,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.38},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.57899999999999996}]},{"osd":6,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.54900000000000004}]},{"osd":7,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.63400000000000001}]},{"osd":8,"last update":"Wed May 8 22:06:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.623},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.70899999999999996}]},{"osd":9,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.76800000000000002}]},{"osd":10,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.69799999999999995}]}]},{"osd":10,"up_from":67,"seq":287762808861,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1069056,"data_stored":1116391,"data_compressed":53579,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:06:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.258,"5min":7.258,"15min":7.258},"last":0.56899999999999995}]},{"osd":1,"last update":"Wed May 8 22:06:19 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.439,"15min":0.439},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.47399999999999998,"15min":0.47199999999999998},"min":{"1min":0.32600000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.53800000000000003}]},{"osd":2,"last update":"Wed May 8 22:06:19 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.624,"15min":0.61899999999999999},"min":{"1min":0.51700000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.628,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.48799999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77000000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.65600000000000003}]},{"osd":3,"last update":"Wed May 8 22:06:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.46400000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.68000000000000005,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.46000000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.72799999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.67800000000000005}]},{"osd":4,"last update":"Wed May 8 22:06:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.38,"5min":0.379,"15min":0.379},"max":{"1min":0.68100000000000005,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52300000000000002,"15min":0.52000000000000002},"min":{"1min":0.34000000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.753,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.76800000000000002}]},{"osd":5,"last update":"Wed May 8 22:06:19 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.46400000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66100000000000003,"15min":0.66200000000000003},"min":{"1min":0.54400000000000004,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.81999999999999995,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.91300000000000003}]},{"osd":6,"last update":"Wed May 8 22:06:19 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.72499999999999998,"15min":0.67900000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":7.2380000000000004,"5min":7.2380000000000004,"15min":7.2380000000000004},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.69899999999999995,"15min":0.69299999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":8,"last update":"Wed May 8 22:06:19 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72199999999999998,"15min":0.72799999999999998},"min":{"1min":0.499,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.86099999999999999,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.54600000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.88400000000000001,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.93799999999999994}]},{"osd":9,"last update":"Wed May 8 22:06:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.755,"15min":0.75800000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.88600000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.117,"5min":0.83699999999999997,"15min":0.79000000000000004},"min":{"1min":0.58799999999999997,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":7.2939999999999996,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.89500000000000002}]},{"osd":11,"last update":"Wed May 8 22:05:39 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.92300000000000004}]}]},{"osd":9,"up_from":62,"seq":266287972385,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":474383,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:06:01 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.52100000000000002,"15min":0.52800000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.58299999999999996,"15min":0.59399999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.95499999999999996,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.36099999999999999}]},{"osd":1,"last update":"Wed May 8 22:06:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.76500000000000001,"15min":0.78000000000000003},"min":{"1min":0.45700000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.92700000000000005,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85699999999999998,"5min":0.997,"15min":0.997},"last":0.54500000000000004}]},{"osd":2,"last update":"Wed May 8 22:06:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.78100000000000003,"15min":0.79300000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98899999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.73899999999999999,"15min":0.746},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.97599999999999998,"5min":1.167,"15min":1.167},"last":0.67600000000000005}]},{"osd":3,"last update":"Wed May 8 22:06:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.78100000000000003,"15min":0.80500000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.99299999999999999,"5min":1.024,"15min":1.024},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.60199999999999998,"15min":0.60999999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.878,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.41499999999999998}]},{"osd":4,"last update":"Wed May 8 22:06:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.77900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0249999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.82599999999999996,"15min":0.84099999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.90400000000000003,"5min":1.081,"15min":1.081},"last":0.50800000000000001}]},{"osd":5,"last update":"Wed May 8 22:06:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.84299999999999997,"15min":0.86499999999999999},"min":{"1min":0.51800000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0629999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.878,"15min":0.89600000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.075,"5min":1.175,"15min":1.175},"last":0.53700000000000003}]},{"osd":6,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.71199999999999997}]},{"osd":7,"last update":"Wed May 8 22:06:01 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.77500000000000002,"15min":0.78800000000000003},"min":{"1min":0.55000000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.93500000000000005,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.91000000000000003,"15min":0.93200000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.095,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.59199999999999997}]},{"osd":8,"last update":"Wed May 8 22:06:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.81599999999999995,"15min":0.83299999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.006,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.92900000000000005,"15min":0.95299999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.157,"5min":1.478,"15min":1.478},"last":0.48799999999999999}]},{"osd":10,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.88300000000000001,"15min":0.89500000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1879999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.874,"15min":0.88400000000000001},"min":{"1min":0.65600000000000003,"5min":0.628,"15min":0.628},"max":{"1min":1.042,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.65600000000000003}]},{"osd":11,"last update":"Wed May 8 22:06:17 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.61499999999999999}]}]},{"osd":8,"up_from":55,"seq":236223201318,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":469876,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.36499999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.88200000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.746},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.622,"15min":0.63200000000000001},"min":{"1min":0.36799999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.79800000000000004,"5min":1.865,"15min":1.865},"last":0.626}]},{"osd":1,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.60299999999999998,"15min":0.59399999999999997},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55700000000000005,"15min":0.55100000000000005},"min":{"1min":0.40799999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.78600000000000003}]},{"osd":2,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.42499999999999999,"15min":0.41899999999999998},"min":{"1min":0.32100000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63700000000000001,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.46800000000000003,"15min":0.46400000000000002},"min":{"1min":0.34200000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.59199999999999997}]},{"osd":3,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.77600000000000002,"15min":0.81100000000000005},"min":{"1min":0.45500000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82199999999999995,"5min":4.8920000000000003,"15min":4.8920000000000003},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63100000000000001,"15min":0.625},"min":{"1min":0.53700000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.69699999999999995}]},{"osd":4,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.62,"15min":0.60799999999999998},"min":{"1min":0.44400000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.85599999999999998}]},{"osd":5,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.50900000000000001,"15min":0.505},"min":{"1min":0.38600000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67000000000000004,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.81799999999999995}]},{"osd":6,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.879,"15min":0.90700000000000003},"min":{"1min":0.53300000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95299999999999996,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85999999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.83999999999999997}]},{"osd":7,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.66400000000000003,"15min":0.66200000000000003},"min":{"1min":0.42999999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.879},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.67200000000000004,"15min":0.66500000000000004},"min":{"1min":0.48699999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.96999999999999997}]},{"osd":9,"last update":"Wed May 8 22:06:03 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.78000000000000003,"15min":0.77500000000000002},"min":{"1min":0.59599999999999997,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":1.014},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.73899999999999999,"15min":0.73299999999999998},"min":{"1min":0.52600000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94799999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.92300000000000004}]},{"osd":10,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.71799999999999997,"15min":0.70499999999999996},"min":{"1min":0.55000000000000004,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94699999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.76400000000000001,"15min":0.752},"min":{"1min":0.68400000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.98199999999999998}]},{"osd":11,"last update":"Wed May 8 22:06:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.90800000000000003}]}]},{"osd":7,"up_from":48,"seq":206158430248,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":1066022,"data_compressed":53579,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.41199999999999998}]},{"osd":1,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.38800000000000001},"min":{"1min":0.29699999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.57499999999999996,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40999999999999998,"15min":0.40400000000000003},"min":{"1min":0.27900000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.55400000000000005,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.53500000000000003}]},{"osd":2,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.41599999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71499999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.57599999999999996,"15min":0.56999999999999995},"min":{"1min":0.46200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80400000000000005,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.86799999999999999}]},{"osd":3,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.41299999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76800000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.748},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68400000000000005,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.60899999999999999}]},{"osd":4,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.49399999999999999,"15min":0.49299999999999999},"min":{"1min":0.35899999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.505,"15min":0.51400000000000001},"min":{"1min":0.34599999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69499999999999995,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.499}]},{"osd":5,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.46800000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77500000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63,"15min":0.628},"min":{"1min":0.495,"5min":0.436,"15min":0.436},"max":{"1min":0.83999999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.81799999999999995}]},{"osd":6,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65800000000000003,"15min":0.66800000000000004},"min":{"1min":0.504,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.80900000000000005,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.67700000000000005,"15min":0.69299999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84099999999999997,"5min":1.002,"15min":1.002},"last":0.59299999999999997}]},{"osd":8,"last update":"Wed May 8 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.68700000000000006,"15min":0.69399999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.78000000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.72099999999999997,"15min":0.72699999999999998},"min":{"1min":0.55300000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.85899999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.79500000000000004}]},{"osd":9,"last update":"Wed May 8 22:06:03 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.72699999999999998,"15min":0.73099999999999998},"min":{"1min":0.55400000000000005,"5min":0.501,"15min":0.501},"max":{"1min":0.89900000000000002,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.73199999999999998,"15min":0.74099999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88900000000000001,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.71799999999999997}]},{"osd":10,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.69999999999999996,"15min":0.71099999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.82099999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.68500000000000005},"min":{"1min":0.499,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.86799999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.64400000000000002}]},{"osd":11,"last update":"Wed May 8 22:06:14 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.80600000000000005}]}]},{"osd":6,"up_from":43,"seq":184683593772,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":465190,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46000000000000002,"15min":0.45000000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.55300000000000005,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.50700000000000001,"15min":0.51200000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.40500000000000003}]},{"osd":1,"last update":"Wed May 8 22:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.45700000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.86199999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.629,"15min":0.60899999999999999},"min":{"1min":0.47599999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92900000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.65200000000000002}]},{"osd":2,"last update":"Wed May 8 22:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67100000000000004,"15min":0.65100000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82499999999999996,"5min":1.089,"15min":1.089},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.65800000000000003,"15min":0.64900000000000002},"min":{"1min":0.42499999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92400000000000004,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.66500000000000004}]},{"osd":3,"last update":"Wed May 8 22:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.38600000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.873,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.504},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.47199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77000000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.61099999999999999}]},{"osd":4,"last update":"Wed May 8 22:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.438,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89900000000000002,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.71399999999999997,"15min":0.71999999999999997},"min":{"1min":0.54400000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.97799999999999998,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.53300000000000003}]},{"osd":5,"last update":"Wed May 8 22:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75600000000000001,"15min":0.74399999999999999},"min":{"1min":0.47899999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.86899999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.748,"15min":0.748},"min":{"1min":0.44400000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.91700000000000004,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.59399999999999997}]},{"osd":7,"last update":"Wed May 8 22:05:23 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.99299999999999999,"5min":1.028,"15min":1.028},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76300000000000001,"15min":0.76000000000000001},"min":{"1min":0.56999999999999995,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.76900000000000002}]},{"osd":8,"last update":"Wed May 8 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.83199999999999996,"15min":0.83999999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0289999999999999,"5min":1.093,"15min":1.093},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.79300000000000004,"15min":0.79600000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0149999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.80800000000000005}]},{"osd":9,"last update":"Wed May 8 22:06:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.84899999999999998,"15min":0.85699999999999998},"min":{"1min":0.59899999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.81999999999999995,"15min":0.83599999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.98899999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.78900000000000003}]},{"osd":10,"last update":"Wed May 8 22:06:20 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.68500000000000005,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.83099999999999996,"15min":0.82099999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":0.629}]},{"osd":11,"last update":"Wed May 8 22:06:20 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":0.747}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":991232,"data_stored":525133,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.52700000000000002,"15min":0.49299999999999999},"min":{"1min":0.38900000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.89900000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54000000000000004,"15min":0.51400000000000001},"min":{"1min":0.371,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81200000000000006,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.51200000000000001}]},{"osd":2,"last update":"Wed May 8 22:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.54800000000000004,"15min":0.497},"min":{"1min":0.40400000000000003,"5min":0.316,"15min":0.316},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.54100000000000004,"15min":0.496},"min":{"1min":0.42799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83599999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.67600000000000005}]},{"osd":3,"last update":"Wed May 8 22:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.55500000000000005,"15min":0.51200000000000001},"min":{"1min":0.37,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76600000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.51500000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72599999999999998,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.59899999999999998}]},{"osd":4,"last update":"Wed May 8 22:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.39900000000000002,"15min":0.376},"min":{"1min":0.307,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.64000000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.42499999999999999,"15min":0.39300000000000002},"min":{"1min":0.32400000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.76700000000000002,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.72499999999999998}]},{"osd":5,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.59099999999999997,"15min":0.55700000000000005},"min":{"1min":0.47199999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.95499999999999996,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.60199999999999998,"15min":0.57899999999999996},"min":{"1min":0.45400000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89600000000000002,"5min":1.089,"15min":1.089},"last":0.75800000000000001}]},{"osd":6,"last update":"Wed May 8 22:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.78100000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.64900000000000002,"15min":0.623},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.85299999999999998,"5min":1.05,"15min":1.05},"last":0.77000000000000002}]},{"osd":7,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58999999999999997,"15min":0.57199999999999995},"min":{"1min":0.44400000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.871,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.39100000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74199999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.70999999999999996}]},{"osd":8,"last update":"Wed May 8 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.65300000000000002,"15min":0.65900000000000003},"min":{"1min":0.45800000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.91200000000000003,"5min":1.115,"15min":1.115},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.53500000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96999999999999997,"5min":1.222,"15min":1.222},"last":0.85699999999999998}]},{"osd":9,"last update":"Wed May 8 22:06:04 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71699999999999997,"15min":0.72099999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.83599999999999997,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.747},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.79900000000000004,"15min":0.79500000000000004},"min":{"1min":0.63600000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.95099999999999996,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.878}]},{"osd":10,"last update":"Wed May 8 22:06:19 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76400000000000001,"15min":0.76500000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.94699999999999995,"5min":1.179,"15min":1.179},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.52400000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.93700000000000006,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.79500000000000004}]},{"osd":11,"last update":"Wed May 8 22:06:14 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.81699999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":946176,"data_stored":466578,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57899999999999996,"15min":0.54900000000000004},"min":{"1min":0.42499999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.94099999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.55400000000000005,"15min":0.52400000000000002},"min":{"1min":0.40100000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83199999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.76700000000000002}]},{"osd":2,"last update":"Wed May 8 22:05:47 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.60199999999999998,"15min":0.55300000000000005},"min":{"1min":0.50900000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.91700000000000004,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.59099999999999997,"15min":0.55100000000000005},"min":{"1min":0.502,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91200000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.57499999999999996}]},{"osd":3,"last update":"Wed May 8 22:06:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.55600000000000005}]},{"osd":4,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.57099999999999995,"15min":0.54900000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69399999999999995,"15min":0.69599999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96999999999999997,"5min":2.984,"15min":2.984},"last":0.629}]},{"osd":5,"last update":"Wed May 8 22:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.67800000000000005,"15min":0.64600000000000002},"min":{"1min":0.42399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0109999999999999,"5min":1.129,"15min":1.129},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67300000000000004,"15min":0.66300000000000003},"min":{"1min":0.49199999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.94699999999999995,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.68500000000000005}]},{"osd":6,"last update":"Wed May 8 22:06:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.60599999999999998,"15min":0.58199999999999996},"min":{"1min":0.45000000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88,"5min":1.254,"15min":1.254},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.70099999999999996,"15min":0.67400000000000004},"min":{"1min":0.49399999999999999,"5min":0.375,"15min":0.375},"max":{"1min":1.1859999999999999,"5min":1.208,"15min":1.208},"last":0.501}]},{"osd":7,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.81299999999999994}]},{"osd":8,"last update":"Wed May 8 22:05:39 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.82199999999999995,"15min":0.82599999999999996},"min":{"1min":0.61399999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.099,"5min":1.427,"15min":1.427},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.79500000000000004,"15min":0.80300000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0369999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.83999999999999997}]},{"osd":9,"last update":"Wed May 8 22:06:03 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.78500000000000003,"15min":0.79000000000000004},"min":{"1min":0.50800000000000001,"5min":0.502,"15min":0.502},"max":{"1min":1.155,"5min":1.284,"15min":1.284},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.89300000000000002,"15min":0.89800000000000002},"min":{"1min":0.59099999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.2170000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.78100000000000003}]},{"osd":10,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.61599999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1339999999999999,"5min":1.2,"15min":1.2},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.876,"15min":0.879},"min":{"1min":0.69599999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.84599999999999997}]},{"osd":11,"last update":"Wed May 8 22:05:39 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.90100000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":995328,"data_stored":524883,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55000000000000004,"15min":0.52400000000000002},"min":{"1min":0.377,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52200000000000002,"15min":0.50800000000000001},"min":{"1min":0.39200000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.47099999999999997}]},{"osd":1,"last update":"Wed May 8 22:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.36599999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.57099999999999995,"15min":0.55300000000000005},"min":{"1min":0.40400000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.72199999999999998}]},{"osd":3,"last update":"Wed May 8 22:06:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.57599999999999996,"15min":0.56000000000000005},"min":{"1min":0.41999999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.58199999999999996,"15min":0.56399999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77600000000000002,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.55500000000000005}]},{"osd":4,"last update":"Wed May 8 22:06:15 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.63},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.74199999999999999}]},{"osd":5,"last update":"Wed May 8 22:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.46800000000000003,"15min":0.44400000000000001},"min":{"1min":0.38100000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.46600000000000003,"15min":0.44800000000000001},"min":{"1min":0.34799999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.435}]},{"osd":6,"last update":"Wed May 8 22:06:09 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.45700000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.84299999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.69399999999999995,"15min":0.67600000000000005},"min":{"1min":0.57699999999999996,"5min":0.442,"15min":0.442},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.57899999999999996}]},{"osd":7,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.755}]},{"osd":8,"last update":"Wed May 8 22:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.36699999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.58799999999999997,"15min":0.57099999999999995},"min":{"1min":0.41299999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.54100000000000004}]},{"osd":9,"last update":"Wed May 8 22:06:03 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.73999999999999999,"15min":0.71599999999999997},"min":{"1min":0.629,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.72999999999999998,"15min":0.70899999999999996},"min":{"1min":0.54600000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.82699999999999996}]},{"osd":10,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.73899999999999999,"15min":0.71299999999999997},"min":{"1min":0.65700000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.72199999999999998,"15min":0.70399999999999996},"min":{"1min":0.66800000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.73399999999999999}]},{"osd":11,"last update":"Wed May 8 22:06:15 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.70899999999999996}]}]},{"osd":3,"up_from":24,"seq":103079215159,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":465657,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:06:11 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.46899999999999997,"15min":0.40899999999999997},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":3.2530000000000001,"5min":3.2530000000000001,"15min":3.2530000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.50700000000000001,"15min":0.42899999999999999},"min":{"1min":0.36499999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":3.4460000000000002,"5min":3.4460000000000002,"15min":3.4460000000000002},"last":1.1619999999999999}]},{"osd":1,"last update":"Wed May 8 22:06:11 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.66500000000000004,"15min":0.57899999999999996},"min":{"1min":0.432,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.7090000000000001,"5min":3.7090000000000001,"15min":3.7090000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.626,"15min":0.55000000000000004},"min":{"1min":0.59299999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":3.3639999999999999,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":0.60599999999999998}]},{"osd":2,"last update":"Wed May 8 22:06:11 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.68899999999999995,"15min":0.61299999999999999},"min":{"1min":0.52300000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":3.577,"5min":3.577,"15min":3.577},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.70199999999999996,"15min":0.63},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.7770000000000001,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.69199999999999995}]},{"osd":4,"last update":"Wed May 8 22:06:25 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.67700000000000005,"15min":0.61299999999999999},"min":{"1min":0.46600000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":3.8109999999999999,"5min":3.8109999999999999,"15min":3.8109999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.72199999999999998,"15min":0.64000000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.847,"5min":3.847,"15min":3.847},"last":0.80600000000000005}]},{"osd":5,"last update":"Wed May 8 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.67800000000000005,"15min":0.64500000000000002},"min":{"1min":0.46500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.64400000000000002,"15min":0.60899999999999999},"min":{"1min":0.46500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":1.018}]},{"osd":6,"last update":"Wed May 8 22:06:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.68200000000000005,"15min":0.63200000000000001},"min":{"1min":0.42299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.653,"5min":3.653,"15min":3.653},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.70099999999999996,"15min":0.64200000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":3.9649999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.96299999999999997}]},{"osd":7,"last update":"Wed May 8 22:06:25 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.73999999999999999,"15min":0.72299999999999998},"min":{"1min":0.55500000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.86399999999999999,"15min":0.84699999999999998},"min":{"1min":0.57599999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":3.887,"5min":3.887,"15min":3.887},"last":0.87}]},{"osd":8,"last update":"Wed May 8 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.53100000000000003}]},{"osd":9,"last update":"Wed May 8 22:06:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":0.90600000000000003,"15min":0.877},"min":{"1min":0.54900000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":4.0289999999999999,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.81699999999999995,"15min":0.80000000000000004},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.3940000000000001,"5min":3.3940000000000001,"15min":3.3940000000000001},"last":0.747}]},{"osd":10,"last update":"Wed May 8 22:06:20 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.85599999999999998,"15min":0.83999999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":3.298,"5min":3.298,"15min":3.298},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.04,"5min":0.89200000000000002,"15min":0.86699999999999999},"min":{"1min":0.60899999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":3.5049999999999999,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":1.1240000000000001}]},{"osd":11,"last update":"Wed May 8 22:06:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":3.6150000000000002,"5min":3.6150000000000002,"15min":3.6150000000000002},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":3.9950000000000001,"5min":3.9950000000000001,"15min":3.9950000000000001},"last":0.89800000000000002}]}]},{"osd":4,"up_from":31,"seq":133143986229,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":470177,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.44700000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.51900000000000002,"15min":0.50900000000000001},"min":{"1min":0.45000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.629}]},{"osd":1,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.374,"15min":0.35099999999999998},"min":{"1min":0.27200000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.66800000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.377,"15min":0.35499999999999998},"min":{"1min":0.28199999999999997,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.40300000000000002}]},{"osd":2,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.52600000000000002,"15min":0.50700000000000001},"min":{"1min":0.35499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71899999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.58199999999999996}]},{"osd":3,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.58799999999999997,"15min":0.56399999999999995},"min":{"1min":0.5,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84799999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56699999999999995,"15min":0.55300000000000005},"min":{"1min":0.39600000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.80000000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.439}]},{"osd":5,"last update":"Wed May 8 22:05:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.36399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76200000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.57199999999999995,"15min":0.55000000000000004},"min":{"1min":0.42799999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.69799999999999995}]},{"osd":6,"last update":"Wed May 8 22:06:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65700000000000003,"15min":0.63900000000000001},"min":{"1min":0.53300000000000003,"5min":0.316,"15min":0.316},"max":{"1min":0.81499999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.621,"15min":0.61399999999999999},"min":{"1min":0.40400000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.56499999999999995}]},{"osd":7,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76800000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.59499999999999997}]},{"osd":8,"last update":"Wed May 8 22:05:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68400000000000005,"15min":0.68600000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70799999999999996,"15min":0.70899999999999996},"min":{"1min":0.54100000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.755}]},{"osd":9,"last update":"Wed May 8 22:06:03 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.56200000000000006,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.80400000000000005,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.755,"15min":0.75700000000000001},"min":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.85699999999999998,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.71699999999999997}]},{"osd":10,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.73599999999999999,"15min":0.73999999999999999},"min":{"1min":0.55800000000000005,"5min":0.434,"15min":0.434},"max":{"1min":0.84799999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.627,"15min":0.625},"min":{"1min":0.50600000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.68799999999999994}]},{"osd":11,"last update":"Wed May 8 22:06:14 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.70899999999999996}]}]},{"osd":5,"up_from":36,"seq":154618822705,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":1003520,"data_stored":1061658,"data_compressed":53579,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.41199999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.71899999999999997,"5min":0.749,"15min":0.749},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.35199999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80100000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.69499999999999995}]},{"osd":1,"last update":"Wed May 8 22:05:43 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.32300000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52600000000000002,"15min":0.51600000000000001},"min":{"1min":0.34300000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71299999999999997,"5min":0.754,"15min":0.754},"last":0.51000000000000001}]},{"osd":2,"last update":"Wed May 8 22:05:43 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.377,"15min":0.36699999999999999},"min":{"1min":0.29799999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.39200000000000002,"15min":0.38400000000000001},"min":{"1min":0.309,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.628}]},{"osd":3,"last update":"Wed May 8 22:05:43 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.57299999999999995},"min":{"1min":0.42099999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.76700000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56299999999999994,"15min":0.55100000000000005},"min":{"1min":0.47299999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.78700000000000003,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.72099999999999997}]},{"osd":4,"last update":"Wed May 8 22:05:43 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54700000000000004,"15min":0.53700000000000003},"min":{"1min":0.42999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82999999999999996,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.33600000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.82699999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.752}]},{"osd":6,"last update":"Wed May 8 22:06:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.59599999999999997,"15min":0.58799999999999997},"min":{"1min":0.41199999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81599999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.58799999999999997,"15min":0.56200000000000006},"min":{"1min":0.51700000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.73999999999999999}]},{"osd":7,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.57399999999999995,"15min":0.56499999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.58799999999999997,"15min":0.57699999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.82099999999999995}]},{"osd":8,"last update":"Wed May 8 22:05:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.50600000000000001,"15min":0.49399999999999999},"min":{"1min":0.372,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.502,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.90600000000000003}]},{"osd":9,"last update":"Wed May 8 22:05:59 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.70999999999999996,"15min":0.70099999999999996},"min":{"1min":0.57499999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.69799999999999995,"15min":0.68999999999999995},"min":{"1min":0.45200000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.78900000000000003}]},{"osd":10,"last update":"Wed May 8 22:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.86599999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":60033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:06:30.821 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T22:06:31.187 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:32.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:31 smithi131 ceph-mon[37260]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:06:32.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:31 smithi131 ceph-mon[37260]: pgmap v280: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T22:06:32.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:31 smithi157 ceph-mon[77746]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:06:32.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:31 smithi157 ceph-mon[77746]: pgmap v280: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T22:06:32.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:31 smithi132 ceph-mon[78180]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:06:32.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:31 smithi132 ceph-mon[78180]: pgmap v280: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T22:06:32.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.584+0000 7fe00aa00700 1 -- 172.21.15.131:0/3693413733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe004101990 msgr2=0x7fe00410de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:32.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.584+0000 7fe00aa00700 1 --2- 172.21.15.131:0/3693413733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe004101990 0x7fe00410de70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdffc00b0a0 tx=0x7fdffc00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:06:32.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.584+0000 7fe00aa00700 1 -- 172.21.15.131:0/3693413733 shutdown_connections 2024-05-08T22:06:32.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.584+0000 7fe00aa00700 1 --2- 172.21.15.131:0/3693413733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe004101990 0x7fe00410de70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:32.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.584+0000 7fe00aa00700 1 --2- 172.21.15.131:0/3693413733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe004107030 0x7fe004107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:32.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.584+0000 7fe00aa00700 1 --2- 172.21.15.131:0/3693413733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe004100ff0 0x7fe004101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:32.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.584+0000 7fe00aa00700 1 -- 172.21.15.131:0/3693413733 >> 172.21.15.131:0/3693413733 conn(0x7fe004078190 msgr2=0x7fe0040785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:32.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.584+0000 7fe00aa00700 1 -- 172.21.15.131:0/3693413733 shutdown_connections 2024-05-08T22:06:32.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.584+0000 7fe00aa00700 1 -- 172.21.15.131:0/3693413733 wait complete. 2024-05-08T22:06:32.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.585+0000 7fe00aa00700 1 Processor -- start 2024-05-08T22:06:32.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.586+0000 7fe00aa00700 1 -- start start 2024-05-08T22:06:32.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.586+0000 7fe00aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe004100ff0 0x7fe004197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:32.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.586+0000 7fe00aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe004101990 0x7fe004197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:32.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.586+0000 7fe00aa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe004107030 0x7fe00419be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:32.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.586+0000 7fe00aa00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00419c420 con 0x7fe004100ff0 2024-05-08T22:06:32.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.586+0000 7fe00aa00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00419c590 con 0x7fe004107030 2024-05-08T22:06:32.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.586+0000 7fe00aa00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00419c750 con 0x7fe004101990 2024-05-08T22:06:32.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.586+0000 7fe003e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe004100ff0 0x7fe004197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:32.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.586+0000 7fe003e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe004100ff0 0x7fe004197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49480/0 (socket says 172.21.15.131:49480) 2024-05-08T22:06:32.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.586+0000 7fe003e00700 1 -- 172.21.15.131:0/1504535930 learned_addr learned my addr 172.21.15.131:0/1504535930 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:32.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.586+0000 7fe008a00700 1 --2- 172.21.15.131:0/1504535930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe004107030 0x7fe00419be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:32.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.586+0000 7fe003400700 1 --2- 172.21.15.131:0/1504535930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe004101990 0x7fe004197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:32.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.587+0000 7fe003e00700 1 -- 172.21.15.131:0/1504535930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe004101990 msgr2=0x7fe004197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:32.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.587+0000 7fe003e00700 1 --2- 172.21.15.131:0/1504535930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe004101990 0x7fe004197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:32.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.587+0000 7fe003e00700 1 -- 172.21.15.131:0/1504535930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe004107030 msgr2=0x7fe00419be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:32.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.587+0000 7fe003e00700 1 --2- 172.21.15.131:0/1504535930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe004107030 0x7fe00419be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:32.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.587+0000 7fe003e00700 1 -- 172.21.15.131:0/1504535930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdffc009d00 con 0x7fe004100ff0 2024-05-08T22:06:32.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.587+0000 7fe008a00700 1 --2- 172.21.15.131:0/1504535930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe004107030 0x7fe00419be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:32.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.587+0000 7fe003e00700 1 --2- 172.21.15.131:0/1504535930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe004100ff0 0x7fe004197520 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdff400e9a0 tx=0x7fdff400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:32.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.587+0000 7fe000c00700 1 -- 172.21.15.131:0/1504535930 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff4009790 con 0x7fe004100ff0 2024-05-08T22:06:32.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.587+0000 7fe000c00700 1 -- 172.21.15.131:0/1504535930 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdff4010430 con 0x7fe004100ff0 2024-05-08T22:06:32.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.587+0000 7fe00aa00700 1 -- 172.21.15.131:0/1504535930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00419ca30 con 0x7fe004100ff0 2024-05-08T22:06:32.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.587+0000 7fe000c00700 1 -- 172.21.15.131:0/1504535930 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff4018680 con 0x7fe004100ff0 2024-05-08T22:06:32.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.587+0000 7fe00aa00700 1 -- 172.21.15.131:0/1504535930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0041a4290 con 0x7fe004100ff0 2024-05-08T22:06:32.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.588+0000 7fe00aa00700 1 -- 172.21.15.131:0/1504535930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe004109550 con 0x7fe004100ff0 2024-05-08T22:06:32.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.590+0000 7fe000c00700 1 -- 172.21.15.131:0/1504535930 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fdff40105a0 con 0x7fe004100ff0 2024-05-08T22:06:32.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.591+0000 7fe000c00700 1 --2- 172.21.15.131:0/1504535930 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdfe80a11b0 0x7fdfe80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:32.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.592+0000 7fe000c00700 1 -- 172.21.15.131:0/1504535930 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdff4014070 con 0x7fe004100ff0 2024-05-08T22:06:32.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.592+0000 7fe003400700 1 --2- 172.21.15.131:0/1504535930 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdfe80a11b0 0x7fdfe80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:32.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.592+0000 7fe003400700 1 --2- 172.21.15.131:0/1504535930 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdfe80a11b0 0x7fdfe80a3670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe004198a80 tx=0x7fdff0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:32.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.596+0000 7fe000c00700 1 -- 172.21.15.131:0/1504535930 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdff4052d60 con 0x7fe004100ff0 2024-05-08T22:06:32.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.795+0000 7fe00aa00700 1 -- 172.21.15.131:0/1504535930 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe004198830 con 0x7fdfe80a11b0 2024-05-08T22:06:32.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.807+0000 7fe000c00700 1 -- 172.21.15.131:0/1504535930 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427622 (secure 0 0 0) 0x7fe004198830 con 0x7fdfe80a11b0 2024-05-08T22:06:32.815 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:06:32.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.813+0000 7fe00aa00700 1 -- 172.21.15.131:0/1504535930 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdfe80a11b0 msgr2=0x7fdfe80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:32.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.813+0000 7fe00aa00700 1 --2- 172.21.15.131:0/1504535930 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdfe80a11b0 0x7fdfe80a3670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe004198a80 tx=0x7fdff0009260 comp rx=0 tx=0).stop 2024-05-08T22:06:32.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.813+0000 7fe00aa00700 1 -- 172.21.15.131:0/1504535930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe004100ff0 msgr2=0x7fe004197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:32.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.813+0000 7fe00aa00700 1 --2- 172.21.15.131:0/1504535930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe004100ff0 0x7fe004197520 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdff400e9a0 tx=0x7fdff400ecb0 comp rx=0 tx=0).stop 2024-05-08T22:06:32.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.813+0000 7fe00aa00700 1 -- 172.21.15.131:0/1504535930 shutdown_connections 2024-05-08T22:06:32.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.813+0000 7fe00aa00700 1 --2- 172.21.15.131:0/1504535930 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdfe80a11b0 0x7fdfe80a3670 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:32.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.813+0000 7fe00aa00700 1 --2- 172.21.15.131:0/1504535930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe004107030 0x7fe00419be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:32.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.813+0000 7fe00aa00700 1 --2- 172.21.15.131:0/1504535930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe004100ff0 0x7fe004197520 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:32.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.813+0000 7fe00aa00700 1 --2- 172.21.15.131:0/1504535930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe004101990 0x7fe004197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:32.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.813+0000 7fe00aa00700 1 -- 172.21.15.131:0/1504535930 >> 172.21.15.131:0/1504535930 conn(0x7fe004078190 msgr2=0x7fe0040ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:32.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.813+0000 7fe00aa00700 1 -- 172.21.15.131:0/1504535930 shutdown_connections 2024-05-08T22:06:32.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:32.813+0000 7fe00aa00700 1 -- 172.21.15.131:0/1504535930 wait complete. 2024-05-08T22:06:32.820 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-08T22:06:33.302 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2024-05-08T22:06:32.733384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653960,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":464,"num_read_kb":349,"num_write":336,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181430,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372456,"kb_used_data":11356,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124569368,"statfs":{"total":1151940427776,"available":1151559032832,"internally_reserved":0,"allocated":11628544,"data_stored":7575999,"data_compressed":318984,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19131,"internal_metadata":369603909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003405"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.821136+0000","last_change":"2024-05-08T22:05:14.061628+0000","last_active":"2024-05-08T22:05:40.821136+0000","last_peered":"2024-05-08T22:05:40.821136+0000","last_clean":"2024-05-08T22:05:40.821136+0000","last_became_active":"2024-05-08T22:05:14.060986+0000","last_became_peered":"2024-05-08T22:05:14.060986+0000","last_unstale":"2024-05-08T22:05:40.821136+0000","last_undegraded":"2024-05-08T22:05:40.821136+0000","last_fullsized":"2024-05-08T22:05:40.821136+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:51.643118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832603+0000","last_change":"2024-05-08T22:05:14.073300+0000","last_active":"2024-05-08T22:05:40.832603+0000","last_peered":"2024-05-08T22:05:40.832603+0000","last_clean":"2024-05-08T22:05:40.832603+0000","last_became_active":"2024-05-08T22:05:14.069058+0000","last_became_peered":"2024-05-08T22:05:14.069058+0000","last_unstale":"2024-05-08T22:05:40.832603+0000","last_undegraded":"2024-05-08T22:05:40.832603+0000","last_fullsized":"2024-05-08T22:05:40.832603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:15.834956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832503+0000","last_change":"2024-05-08T22:05:14.071676+0000","last_active":"2024-05-08T22:05:40.832503+0000","last_peered":"2024-05-08T22:05:40.832503+0000","last_clean":"2024-05-08T22:05:40.832503+0000","last_became_active":"2024-05-08T22:05:14.058345+0000","last_became_peered":"2024-05-08T22:05:14.058345+0000","last_unstale":"2024-05-08T22:05:40.832503+0000","last_undegraded":"2024-05-08T22:05:40.832503+0000","last_fullsized":"2024-05-08T22:05:40.832503+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:09.239058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819679+0000","last_change":"2024-05-08T22:05:14.067170+0000","last_active":"2024-05-08T22:05:40.819679+0000","last_peered":"2024-05-08T22:05:40.819679+0000","last_clean":"2024-05-08T22:05:40.819679+0000","last_became_active":"2024-05-08T22:05:14.066850+0000","last_became_peered":"2024-05-08T22:05:14.066850+0000","last_unstale":"2024-05-08T22:05:40.819679+0000","last_undegraded":"2024-05-08T22:05:40.819679+0000","last_fullsized":"2024-05-08T22:05:40.819679+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:28.439786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829281+0000","last_change":"2024-05-08T22:05:14.065177+0000","last_active":"2024-05-08T22:05:40.829281+0000","last_peered":"2024-05-08T22:05:40.829281+0000","last_clean":"2024-05-08T22:05:40.829281+0000","last_became_active":"2024-05-08T22:05:14.064833+0000","last_became_peered":"2024-05-08T22:05:14.064833+0000","last_unstale":"2024-05-08T22:05:40.829281+0000","last_undegraded":"2024-05-08T22:05:40.829281+0000","last_fullsized":"2024-05-08T22:05:40.829281+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:11.764154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.825297+0000","last_change":"2024-05-08T22:05:14.061610+0000","last_active":"2024-05-08T22:05:40.825297+0000","last_peered":"2024-05-08T22:05:40.825297+0000","last_clean":"2024-05-08T22:05:40.825297+0000","last_became_active":"2024-05-08T22:05:14.061277+0000","last_became_peered":"2024-05-08T22:05:14.061277+0000","last_unstale":"2024-05-08T22:05:40.825297+0000","last_undegraded":"2024-05-08T22:05:40.825297+0000","last_fullsized":"2024-05-08T22:05:40.825297+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:19.154980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830795+0000","last_change":"2024-05-08T22:05:14.063941+0000","last_active":"2024-05-08T22:05:40.830795+0000","last_peered":"2024-05-08T22:05:40.830795+0000","last_clean":"2024-05-08T22:05:40.830795+0000","last_became_active":"2024-05-08T22:05:14.063579+0000","last_became_peered":"2024-05-08T22:05:14.063579+0000","last_unstale":"2024-05-08T22:05:40.830795+0000","last_undegraded":"2024-05-08T22:05:40.830795+0000","last_fullsized":"2024-05-08T22:05:40.830795+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:27.679067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826760+0000","last_change":"2024-05-08T22:05:14.060366+0000","last_active":"2024-05-08T22:05:40.826760+0000","last_peered":"2024-05-08T22:05:40.826760+0000","last_clean":"2024-05-08T22:05:40.826760+0000","last_became_active":"2024-05-08T22:05:14.059818+0000","last_became_peered":"2024-05-08T22:05:14.059818+0000","last_unstale":"2024-05-08T22:05:40.826760+0000","last_undegraded":"2024-05-08T22:05:40.826760+0000","last_fullsized":"2024-05-08T22:05:40.826760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:53.118508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.823157+0000","last_change":"2024-05-08T22:05:14.057185+0000","last_active":"2024-05-08T22:05:40.823157+0000","last_peered":"2024-05-08T22:05:40.823157+0000","last_clean":"2024-05-08T22:05:40.823157+0000","last_became_active":"2024-05-08T22:05:14.056889+0000","last_became_peered":"2024-05-08T22:05:14.056889+0000","last_unstale":"2024-05-08T22:05:40.823157+0000","last_undegraded":"2024-05-08T22:05:40.823157+0000","last_fullsized":"2024-05-08T22:05:40.823157+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:35.372705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824080+0000","last_change":"2024-05-08T22:05:14.062282+0000","last_active":"2024-05-08T22:05:40.824080+0000","last_peered":"2024-05-08T22:05:40.824080+0000","last_clean":"2024-05-08T22:05:40.824080+0000","last_became_active":"2024-05-08T22:05:14.061574+0000","last_became_peered":"2024-05-08T22:05:14.061574+0000","last_unstale":"2024-05-08T22:05:40.824080+0000","last_undegraded":"2024-05-08T22:05:40.824080+0000","last_fullsized":"2024-05-08T22:05:40.824080+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:16:32.873590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831100+0000","last_change":"2024-05-08T22:05:14.061267+0000","last_active":"2024-05-08T22:05:40.831100+0000","last_peered":"2024-05-08T22:05:40.831100+0000","last_clean":"2024-05-08T22:05:40.831100+0000","last_became_active":"2024-05-08T22:05:14.060943+0000","last_became_peered":"2024-05-08T22:05:14.060943+0000","last_unstale":"2024-05-08T22:05:40.831100+0000","last_undegraded":"2024-05-08T22:05:40.831100+0000","last_fullsized":"2024-05-08T22:05:40.831100+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:03.831138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819303+0000","last_change":"2024-05-08T22:05:14.067136+0000","last_active":"2024-05-08T22:05:40.819303+0000","last_peered":"2024-05-08T22:05:40.819303+0000","last_clean":"2024-05-08T22:05:40.819303+0000","last_became_active":"2024-05-08T22:05:14.066810+0000","last_became_peered":"2024-05-08T22:05:14.066810+0000","last_unstale":"2024-05-08T22:05:40.819303+0000","last_undegraded":"2024-05-08T22:05:40.819303+0000","last_fullsized":"2024-05-08T22:05:40.819303+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:43.001024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824060+0000","last_change":"2024-05-08T22:05:14.062119+0000","last_active":"2024-05-08T22:05:40.824060+0000","last_peered":"2024-05-08T22:05:40.824060+0000","last_clean":"2024-05-08T22:05:40.824060+0000","last_became_active":"2024-05-08T22:05:14.058977+0000","last_became_peered":"2024-05-08T22:05:14.058977+0000","last_unstale":"2024-05-08T22:05:40.824060+0000","last_undegraded":"2024-05-08T22:05:40.824060+0000","last_fullsized":"2024-05-08T22:05:40.824060+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:03.657985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832861+0000","last_change":"2024-05-08T22:05:14.067363+0000","last_active":"2024-05-08T22:05:40.832861+0000","last_peered":"2024-05-08T22:05:40.832861+0000","last_clean":"2024-05-08T22:05:40.832861+0000","last_became_active":"2024-05-08T22:05:14.066119+0000","last_became_peered":"2024-05-08T22:05:14.066119+0000","last_unstale":"2024-05-08T22:05:40.832861+0000","last_undegraded":"2024-05-08T22:05:40.832861+0000","last_fullsized":"2024-05-08T22:05:40.832861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:43.426750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832641+0000","last_change":"2024-05-08T22:05:14.073925+0000","last_active":"2024-05-08T22:05:40.832641+0000","last_peered":"2024-05-08T22:05:40.832641+0000","last_clean":"2024-05-08T22:05:40.832641+0000","last_became_active":"2024-05-08T22:05:14.072287+0000","last_became_peered":"2024-05-08T22:05:14.072287+0000","last_unstale":"2024-05-08T22:05:40.832641+0000","last_undegraded":"2024-05-08T22:05:40.832641+0000","last_fullsized":"2024-05-08T22:05:40.832641+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:13.537938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.823678+0000","last_change":"2024-05-08T22:05:14.057274+0000","last_active":"2024-05-08T22:05:40.823678+0000","last_peered":"2024-05-08T22:05:40.823678+0000","last_clean":"2024-05-08T22:05:40.823678+0000","last_became_active":"2024-05-08T22:05:14.056989+0000","last_became_peered":"2024-05-08T22:05:14.056989+0000","last_unstale":"2024-05-08T22:05:40.823678+0000","last_undegraded":"2024-05-08T22:05:40.823678+0000","last_fullsized":"2024-05-08T22:05:40.823678+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:32.316406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.821053+0000","last_change":"2024-05-08T22:05:14.069263+0000","last_active":"2024-05-08T22:05:40.821053+0000","last_peered":"2024-05-08T22:05:40.821053+0000","last_clean":"2024-05-08T22:05:40.821053+0000","last_became_active":"2024-05-08T22:05:14.068960+0000","last_became_peered":"2024-05-08T22:05:14.068960+0000","last_unstale":"2024-05-08T22:05:40.821053+0000","last_undegraded":"2024-05-08T22:05:40.821053+0000","last_fullsized":"2024-05-08T22:05:40.821053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:59:42.829110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817249+0000","last_change":"2024-05-08T22:05:14.049393+0000","last_active":"2024-05-08T22:05:40.817249+0000","last_peered":"2024-05-08T22:05:40.817249+0000","last_clean":"2024-05-08T22:05:40.817249+0000","last_became_active":"2024-05-08T22:05:14.049094+0000","last_became_peered":"2024-05-08T22:05:14.049094+0000","last_unstale":"2024-05-08T22:05:40.817249+0000","last_undegraded":"2024-05-08T22:05:40.817249+0000","last_fullsized":"2024-05-08T22:05:40.817249+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:42.106568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.818062+0000","last_change":"2024-05-08T22:05:14.055436+0000","last_active":"2024-05-08T22:05:40.818062+0000","last_peered":"2024-05-08T22:05:40.818062+0000","last_clean":"2024-05-08T22:05:40.818062+0000","last_became_active":"2024-05-08T22:05:14.055100+0000","last_became_peered":"2024-05-08T22:05:14.055100+0000","last_unstale":"2024-05-08T22:05:40.818062+0000","last_undegraded":"2024-05-08T22:05:40.818062+0000","last_fullsized":"2024-05-08T22:05:40.818062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:52.833780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.816936+0000","last_change":"2024-05-08T22:05:18.532086+0000","last_active":"2024-05-08T22:05:40.816936+0000","last_peered":"2024-05-08T22:05:40.816936+0000","last_clean":"2024-05-08T22:05:40.816936+0000","last_became_active":"2024-05-08T22:05:18.531485+0000","last_became_peered":"2024-05-08T22:05:18.531485+0000","last_unstale":"2024-05-08T22:05:40.816936+0000","last_undegraded":"2024-05-08T22:05:40.816936+0000","last_fullsized":"2024-05-08T22:05:40.816936+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:40:36.050367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817288+0000","last_change":"2024-05-08T22:05:13.549611+0000","last_active":"2024-05-08T22:05:40.817288+0000","last_peered":"2024-05-08T22:05:40.817288+0000","last_clean":"2024-05-08T22:05:40.817288+0000","last_became_active":"2024-05-08T22:05:13.549140+0000","last_became_peered":"2024-05-08T22:05:13.549140+0000","last_unstale":"2024-05-08T22:05:40.817288+0000","last_undegraded":"2024-05-08T22:05:40.817288+0000","last_fullsized":"2024-05-08T22:05:40.817288+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:54.785123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.828016+0000","last_change":"2024-05-08T22:05:14.057570+0000","last_active":"2024-05-08T22:05:40.828016+0000","last_peered":"2024-05-08T22:05:40.828016+0000","last_clean":"2024-05-08T22:05:40.828016+0000","last_became_active":"2024-05-08T22:05:14.057291+0000","last_became_peered":"2024-05-08T22:05:14.057291+0000","last_unstale":"2024-05-08T22:05:40.828016+0000","last_undegraded":"2024-05-08T22:05:40.828016+0000","last_fullsized":"2024-05-08T22:05:40.828016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:15:38.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:42.147633+0000","last_change":"2024-05-08T22:05:42.147633+0000","last_active":"2024-05-08T22:05:42.147633+0000","last_peered":"2024-05-08T22:05:42.147633+0000","last_clean":"2024-05-08T22:05:42.147633+0000","last_became_active":"2024-05-08T22:05:42.147284+0000","last_became_peered":"2024-05-08T22:05:42.147284+0000","last_unstale":"2024-05-08T22:05:42.147633+0000","last_undegraded":"2024-05-08T22:05:42.147633+0000","last_fullsized":"2024-05-08T22:05:42.147633+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:32.864756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.828112+0000","last_change":"2024-05-08T22:05:13.551351+0000","last_active":"2024-05-08T22:05:40.828112+0000","last_peered":"2024-05-08T22:05:40.828112+0000","last_clean":"2024-05-08T22:05:40.828112+0000","last_became_active":"2024-05-08T22:05:13.551014+0000","last_became_peered":"2024-05-08T22:05:13.551014+0000","last_unstale":"2024-05-08T22:05:40.828112+0000","last_undegraded":"2024-05-08T22:05:40.828112+0000","last_fullsized":"2024-05-08T22:05:40.828112+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:07:22.405251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.834013+0000","last_change":"2024-05-08T22:05:14.067270+0000","last_active":"2024-05-08T22:05:40.834013+0000","last_peered":"2024-05-08T22:05:40.834013+0000","last_clean":"2024-05-08T22:05:40.834013+0000","last_became_active":"2024-05-08T22:05:14.065496+0000","last_became_peered":"2024-05-08T22:05:14.065496+0000","last_unstale":"2024-05-08T22:05:40.834013+0000","last_undegraded":"2024-05-08T22:05:40.834013+0000","last_fullsized":"2024-05-08T22:05:40.834013+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:07.087880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819016+0000","last_change":"2024-05-08T22:05:18.526557+0000","last_active":"2024-05-08T22:05:40.819016+0000","last_peered":"2024-05-08T22:05:40.819016+0000","last_clean":"2024-05-08T22:05:40.819016+0000","last_became_active":"2024-05-08T22:05:18.526129+0000","last_became_peered":"2024-05-08T22:05:18.526129+0000","last_unstale":"2024-05-08T22:05:40.819016+0000","last_undegraded":"2024-05-08T22:05:40.819016+0000","last_fullsized":"2024-05-08T22:05:40.819016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:36.376579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.835476+0000","last_change":"2024-05-08T22:05:41.835074+0000","last_active":"2024-05-08T22:05:41.835476+0000","last_peered":"2024-05-08T22:05:41.835476+0000","last_clean":"2024-05-08T22:05:41.835476+0000","last_became_active":"2024-05-08T22:05:41.829567+0000","last_became_peered":"2024-05-08T22:05:41.829567+0000","last_unstale":"2024-05-08T22:05:41.835476+0000","last_undegraded":"2024-05-08T22:05:41.835476+0000","last_fullsized":"2024-05-08T22:05:41.835476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:15.261352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826964+0000","last_change":"2024-05-08T22:05:14.060089+0000","last_active":"2024-05-08T22:05:40.826964+0000","last_peered":"2024-05-08T22:05:40.826964+0000","last_clean":"2024-05-08T22:05:40.826964+0000","last_became_active":"2024-05-08T22:05:14.057620+0000","last_became_peered":"2024-05-08T22:05:14.057620+0000","last_unstale":"2024-05-08T22:05:40.826964+0000","last_undegraded":"2024-05-08T22:05:40.826964+0000","last_fullsized":"2024-05-08T22:05:40.826964+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:29:33.127510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.818817+0000","last_change":"2024-05-08T22:05:18.529746+0000","last_active":"2024-05-08T22:05:40.818817+0000","last_peered":"2024-05-08T22:05:40.818817+0000","last_clean":"2024-05-08T22:05:40.818817+0000","last_became_active":"2024-05-08T22:05:18.529097+0000","last_became_peered":"2024-05-08T22:05:18.529097+0000","last_unstale":"2024-05-08T22:05:40.818817+0000","last_undegraded":"2024-05-08T22:05:40.818817+0000","last_fullsized":"2024-05-08T22:05:40.818817+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:59:36.249787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.837496+0000","last_change":"2024-05-08T22:05:41.837103+0000","last_active":"2024-05-08T22:05:41.837496+0000","last_peered":"2024-05-08T22:05:41.837496+0000","last_clean":"2024-05-08T22:05:41.837496+0000","last_became_active":"2024-05-08T22:05:41.829464+0000","last_became_peered":"2024-05-08T22:05:41.829464+0000","last_unstale":"2024-05-08T22:05:41.837496+0000","last_undegraded":"2024-05-08T22:05:41.837496+0000","last_fullsized":"2024-05-08T22:05:41.837496+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:55.747487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827543+0000","last_change":"2024-05-08T22:05:14.064695+0000","last_active":"2024-05-08T22:05:40.827543+0000","last_peered":"2024-05-08T22:05:40.827543+0000","last_clean":"2024-05-08T22:05:40.827543+0000","last_became_active":"2024-05-08T22:05:14.064283+0000","last_became_peered":"2024-05-08T22:05:14.064283+0000","last_unstale":"2024-05-08T22:05:40.827543+0000","last_undegraded":"2024-05-08T22:05:40.827543+0000","last_fullsized":"2024-05-08T22:05:40.827543+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:06.980594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817381+0000","last_change":"2024-05-08T22:05:18.528475+0000","last_active":"2024-05-08T22:05:40.817381+0000","last_peered":"2024-05-08T22:05:40.817381+0000","last_clean":"2024-05-08T22:05:40.817381+0000","last_became_active":"2024-05-08T22:05:18.528089+0000","last_became_peered":"2024-05-08T22:05:18.528089+0000","last_unstale":"2024-05-08T22:05:40.817381+0000","last_undegraded":"2024-05-08T22:05:40.817381+0000","last_fullsized":"2024-05-08T22:05:40.817381+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:01.508984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":78,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:06:28.573923+0000","last_change":"2024-05-08T22:05:13.549191+0000","last_active":"2024-05-08T22:06:28.573923+0000","last_peered":"2024-05-08T22:06:28.573923+0000","last_clean":"2024-05-08T22:06:28.573923+0000","last_became_active":"2024-05-08T22:05:13.548737+0000","last_became_peered":"2024-05-08T22:05:13.548737+0000","last_unstale":"2024-05-08T22:06:28.573923+0000","last_undegraded":"2024-05-08T22:06:28.573923+0000","last_fullsized":"2024-05-08T22:06:28.573923+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:07.355656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830198+0000","last_change":"2024-05-08T22:05:14.065910+0000","last_active":"2024-05-08T22:05:40.830198+0000","last_peered":"2024-05-08T22:05:40.830198+0000","last_clean":"2024-05-08T22:05:40.830198+0000","last_became_active":"2024-05-08T22:05:14.065504+0000","last_became_peered":"2024-05-08T22:05:14.065504+0000","last_unstale":"2024-05-08T22:05:40.830198+0000","last_undegraded":"2024-05-08T22:05:40.830198+0000","last_fullsized":"2024-05-08T22:05:40.830198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:12.447292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817194+0000","last_change":"2024-05-08T22:05:18.536861+0000","last_active":"2024-05-08T22:05:40.817194+0000","last_peered":"2024-05-08T22:05:40.817194+0000","last_clean":"2024-05-08T22:05:40.817194+0000","last_became_active":"2024-05-08T22:05:18.536472+0000","last_became_peered":"2024-05-08T22:05:18.536472+0000","last_unstale":"2024-05-08T22:05:40.817194+0000","last_undegraded":"2024-05-08T22:05:40.817194+0000","last_fullsized":"2024-05-08T22:05:40.817194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:45:21.826812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.825252+0000","last_change":"2024-05-08T22:05:13.553326+0000","last_active":"2024-05-08T22:05:40.825252+0000","last_peered":"2024-05-08T22:05:40.825252+0000","last_clean":"2024-05-08T22:05:40.825252+0000","last_became_active":"2024-05-08T22:05:13.552983+0000","last_became_peered":"2024-05-08T22:05:13.552983+0000","last_unstale":"2024-05-08T22:05:40.825252+0000","last_undegraded":"2024-05-08T22:05:40.825252+0000","last_fullsized":"2024-05-08T22:05:40.825252+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:17.195253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.828199+0000","last_change":"2024-05-08T22:05:14.068617+0000","last_active":"2024-05-08T22:05:40.828199+0000","last_peered":"2024-05-08T22:05:40.828199+0000","last_clean":"2024-05-08T22:05:40.828199+0000","last_became_active":"2024-05-08T22:05:14.067400+0000","last_became_peered":"2024-05-08T22:05:14.067400+0000","last_unstale":"2024-05-08T22:05:40.828199+0000","last_undegraded":"2024-05-08T22:05:40.828199+0000","last_fullsized":"2024-05-08T22:05:40.828199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:25.488212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.820967+0000","last_change":"2024-05-08T22:05:18.538294+0000","last_active":"2024-05-08T22:05:40.820967+0000","last_peered":"2024-05-08T22:05:40.820967+0000","last_clean":"2024-05-08T22:05:40.820967+0000","last_became_active":"2024-05-08T22:05:18.537984+0000","last_became_peered":"2024-05-08T22:05:18.537984+0000","last_unstale":"2024-05-08T22:05:40.820967+0000","last_undegraded":"2024-05-08T22:05:40.820967+0000","last_fullsized":"2024-05-08T22:05:40.820967+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:32.117439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.821162+0000","last_change":"2024-05-08T22:05:13.557669+0000","last_active":"2024-05-08T22:05:40.821162+0000","last_peered":"2024-05-08T22:05:40.821162+0000","last_clean":"2024-05-08T22:05:40.821162+0000","last_became_active":"2024-05-08T22:05:13.555835+0000","last_became_peered":"2024-05-08T22:05:13.555835+0000","last_unstale":"2024-05-08T22:05:40.821162+0000","last_undegraded":"2024-05-08T22:05:40.821162+0000","last_fullsized":"2024-05-08T22:05:40.821162+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:02.377635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826728+0000","last_change":"2024-05-08T22:05:14.060144+0000","last_active":"2024-05-08T22:05:40.826728+0000","last_peered":"2024-05-08T22:05:40.826728+0000","last_clean":"2024-05-08T22:05:40.826728+0000","last_became_active":"2024-05-08T22:05:14.059387+0000","last_became_peered":"2024-05-08T22:05:14.059387+0000","last_unstale":"2024-05-08T22:05:40.826728+0000","last_undegraded":"2024-05-08T22:05:40.826728+0000","last_fullsized":"2024-05-08T22:05:40.826728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:32:28.057831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832449+0000","last_change":"2024-05-08T22:05:18.527293+0000","last_active":"2024-05-08T22:05:40.832449+0000","last_peered":"2024-05-08T22:05:40.832449+0000","last_clean":"2024-05-08T22:05:40.832449+0000","last_became_active":"2024-05-08T22:05:18.526690+0000","last_became_peered":"2024-05-08T22:05:18.526690+0000","last_unstale":"2024-05-08T22:05:40.832449+0000","last_undegraded":"2024-05-08T22:05:40.832449+0000","last_fullsized":"2024-05-08T22:05:40.832449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:50.343536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826376+0000","last_change":"2024-05-08T22:05:13.549774+0000","last_active":"2024-05-08T22:05:40.826376+0000","last_peered":"2024-05-08T22:05:40.826376+0000","last_clean":"2024-05-08T22:05:40.826376+0000","last_became_active":"2024-05-08T22:05:13.549531+0000","last_became_peered":"2024-05-08T22:05:13.549531+0000","last_unstale":"2024-05-08T22:05:40.826376+0000","last_undegraded":"2024-05-08T22:05:40.826376+0000","last_fullsized":"2024-05-08T22:05:40.826376+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:59:23.798728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.815983+0000","last_change":"2024-05-08T22:05:14.054232+0000","last_active":"2024-05-08T22:05:40.815983+0000","last_peered":"2024-05-08T22:05:40.815983+0000","last_clean":"2024-05-08T22:05:40.815983+0000","last_became_active":"2024-05-08T22:05:14.053806+0000","last_became_peered":"2024-05-08T22:05:14.053806+0000","last_unstale":"2024-05-08T22:05:40.815983+0000","last_undegraded":"2024-05-08T22:05:40.815983+0000","last_fullsized":"2024-05-08T22:05:40.815983+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:32:21.678792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827239+0000","last_change":"2024-05-08T22:05:18.535241+0000","last_active":"2024-05-08T22:05:40.827239+0000","last_peered":"2024-05-08T22:05:40.827239+0000","last_clean":"2024-05-08T22:05:40.827239+0000","last_became_active":"2024-05-08T22:05:18.534753+0000","last_became_peered":"2024-05-08T22:05:18.534753+0000","last_unstale":"2024-05-08T22:05:40.827239+0000","last_undegraded":"2024-05-08T22:05:40.827239+0000","last_fullsized":"2024-05-08T22:05:40.827239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:46:15.138437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":70,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:06:26.569953+0000","last_change":"2024-05-08T22:05:13.544899+0000","last_active":"2024-05-08T22:06:26.569953+0000","last_peered":"2024-05-08T22:06:26.569953+0000","last_clean":"2024-05-08T22:06:26.569953+0000","last_became_active":"2024-05-08T22:05:13.544498+0000","last_became_peered":"2024-05-08T22:05:13.544498+0000","last_unstale":"2024-05-08T22:06:26.569953+0000","last_undegraded":"2024-05-08T22:06:26.569953+0000","last_fullsized":"2024-05-08T22:06:26.569953+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:58.169648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.823071+0000","last_change":"2024-05-08T22:05:14.049377+0000","last_active":"2024-05-08T22:05:40.823071+0000","last_peered":"2024-05-08T22:05:40.823071+0000","last_clean":"2024-05-08T22:05:40.823071+0000","last_became_active":"2024-05-08T22:05:14.049039+0000","last_became_peered":"2024-05-08T22:05:14.049039+0000","last_unstale":"2024-05-08T22:05:40.823071+0000","last_undegraded":"2024-05-08T22:05:40.823071+0000","last_fullsized":"2024-05-08T22:05:40.823071+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:24.415392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.815955+0000","last_change":"2024-05-08T22:05:18.528912+0000","last_active":"2024-05-08T22:05:40.815955+0000","last_peered":"2024-05-08T22:05:40.815955+0000","last_clean":"2024-05-08T22:05:40.815955+0000","last_became_active":"2024-05-08T22:05:18.528433+0000","last_became_peered":"2024-05-08T22:05:18.528433+0000","last_unstale":"2024-05-08T22:05:40.815955+0000","last_undegraded":"2024-05-08T22:05:40.815955+0000","last_fullsized":"2024-05-08T22:05:40.815955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:05.574728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830944+0000","last_change":"2024-05-08T22:05:13.550214+0000","last_active":"2024-05-08T22:05:40.830944+0000","last_peered":"2024-05-08T22:05:40.830944+0000","last_clean":"2024-05-08T22:05:40.830944+0000","last_became_active":"2024-05-08T22:05:13.549717+0000","last_became_peered":"2024-05-08T22:05:13.549717+0000","last_unstale":"2024-05-08T22:05:40.830944+0000","last_undegraded":"2024-05-08T22:05:40.830944+0000","last_fullsized":"2024-05-08T22:05:40.830944+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:12:20.406333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.834200+0000","last_change":"2024-05-08T22:05:14.071214+0000","last_active":"2024-05-08T22:05:40.834200+0000","last_peered":"2024-05-08T22:05:40.834200+0000","last_clean":"2024-05-08T22:05:40.834200+0000","last_became_active":"2024-05-08T22:05:14.066777+0000","last_became_peered":"2024-05-08T22:05:14.066777+0000","last_unstale":"2024-05-08T22:05:40.834200+0000","last_undegraded":"2024-05-08T22:05:40.834200+0000","last_fullsized":"2024-05-08T22:05:40.834200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:53.360911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819973+0000","last_change":"2024-05-08T22:05:18.523688+0000","last_active":"2024-05-08T22:05:40.819973+0000","last_peered":"2024-05-08T22:05:40.819973+0000","last_clean":"2024-05-08T22:05:40.819973+0000","last_became_active":"2024-05-08T22:05:18.523236+0000","last_became_peered":"2024-05-08T22:05:18.523236+0000","last_unstale":"2024-05-08T22:05:40.819973+0000","last_undegraded":"2024-05-08T22:05:40.819973+0000","last_fullsized":"2024-05-08T22:05:40.819973+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:34.531699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827485+0000","last_change":"2024-05-08T22:05:13.551027+0000","last_active":"2024-05-08T22:05:40.827485+0000","last_peered":"2024-05-08T22:05:40.827485+0000","last_clean":"2024-05-08T22:05:40.827485+0000","last_became_active":"2024-05-08T22:05:13.550471+0000","last_became_peered":"2024-05-08T22:05:13.550471+0000","last_unstale":"2024-05-08T22:05:40.827485+0000","last_undegraded":"2024-05-08T22:05:40.827485+0000","last_fullsized":"2024-05-08T22:05:40.827485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:50.900419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827682+0000","last_change":"2024-05-08T22:05:14.065191+0000","last_active":"2024-05-08T22:05:40.827682+0000","last_peered":"2024-05-08T22:05:40.827682+0000","last_clean":"2024-05-08T22:05:40.827682+0000","last_became_active":"2024-05-08T22:05:14.064893+0000","last_became_peered":"2024-05-08T22:05:14.064893+0000","last_unstale":"2024-05-08T22:05:40.827682+0000","last_undegraded":"2024-05-08T22:05:40.827682+0000","last_fullsized":"2024-05-08T22:05:40.827682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:11.344646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.816088+0000","last_change":"2024-05-08T22:05:18.530447+0000","last_active":"2024-05-08T22:05:40.816088+0000","last_peered":"2024-05-08T22:05:40.816088+0000","last_clean":"2024-05-08T22:05:40.816088+0000","last_became_active":"2024-05-08T22:05:18.528171+0000","last_became_peered":"2024-05-08T22:05:18.528171+0000","last_unstale":"2024-05-08T22:05:40.816088+0000","last_undegraded":"2024-05-08T22:05:40.816088+0000","last_fullsized":"2024-05-08T22:05:40.816088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:09:43.347366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"85'31","reported_seq":154,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:06:27.578597+0000","last_change":"2024-05-08T22:05:13.547289+0000","last_active":"2024-05-08T22:06:27.578597+0000","last_peered":"2024-05-08T22:06:27.578597+0000","last_clean":"2024-05-08T22:06:27.578597+0000","last_became_active":"2024-05-08T22:05:13.546860+0000","last_became_peered":"2024-05-08T22:05:13.546860+0000","last_unstale":"2024-05-08T22:06:27.578597+0000","last_undegraded":"2024-05-08T22:06:27.578597+0000","last_fullsized":"2024-05-08T22:06:27.578597+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:08:50.339361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59451,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":105,"num_read_kb":111,"num_write":32,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831151+0000","last_change":"2024-05-08T22:05:14.057925+0000","last_active":"2024-05-08T22:05:40.831151+0000","last_peered":"2024-05-08T22:05:40.831151+0000","last_clean":"2024-05-08T22:05:40.831151+0000","last_became_active":"2024-05-08T22:05:14.057662+0000","last_became_peered":"2024-05-08T22:05:14.057662+0000","last_unstale":"2024-05-08T22:05:40.831151+0000","last_undegraded":"2024-05-08T22:05:40.831151+0000","last_fullsized":"2024-05-08T22:05:40.831151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:49.929610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827811+0000","last_change":"2024-05-08T22:05:18.535509+0000","last_active":"2024-05-08T22:05:40.827811+0000","last_peered":"2024-05-08T22:05:40.827811+0000","last_clean":"2024-05-08T22:05:40.827811+0000","last_became_active":"2024-05-08T22:05:18.535024+0000","last_became_peered":"2024-05-08T22:05:18.535024+0000","last_unstale":"2024-05-08T22:05:40.827811+0000","last_undegraded":"2024-05-08T22:05:40.827811+0000","last_fullsized":"2024-05-08T22:05:40.827811+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:28.098737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817350+0000","last_change":"2024-05-08T22:05:13.552096+0000","last_active":"2024-05-08T22:05:40.817350+0000","last_peered":"2024-05-08T22:05:40.817350+0000","last_clean":"2024-05-08T22:05:40.817350+0000","last_became_active":"2024-05-08T22:05:13.551655+0000","last_became_peered":"2024-05-08T22:05:13.551655+0000","last_unstale":"2024-05-08T22:05:40.817350+0000","last_undegraded":"2024-05-08T22:05:40.817350+0000","last_fullsized":"2024-05-08T22:05:40.817350+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:24.371008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.821551+0000","last_change":"2024-05-08T22:05:14.066075+0000","last_active":"2024-05-08T22:05:40.821551+0000","last_peered":"2024-05-08T22:05:40.821551+0000","last_clean":"2024-05-08T22:05:40.821551+0000","last_became_active":"2024-05-08T22:05:14.065581+0000","last_became_peered":"2024-05-08T22:05:14.065581+0000","last_unstale":"2024-05-08T22:05:40.821551+0000","last_undegraded":"2024-05-08T22:05:40.821551+0000","last_fullsized":"2024-05-08T22:05:40.821551+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:15.403714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.831782+0000","last_change":"2024-05-08T22:05:41.831782+0000","last_active":"2024-05-08T22:05:41.831782+0000","last_peered":"2024-05-08T22:05:41.831782+0000","last_clean":"2024-05-08T22:05:41.831782+0000","last_became_active":"2024-05-08T22:05:41.831335+0000","last_became_peered":"2024-05-08T22:05:41.831335+0000","last_unstale":"2024-05-08T22:05:41.831782+0000","last_undegraded":"2024-05-08T22:05:41.831782+0000","last_fullsized":"2024-05-08T22:05:41.831782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:39:34.031128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827308+0000","last_change":"2024-05-08T22:05:13.549112+0000","last_active":"2024-05-08T22:05:40.827308+0000","last_peered":"2024-05-08T22:05:40.827308+0000","last_clean":"2024-05-08T22:05:40.827308+0000","last_became_active":"2024-05-08T22:05:13.548763+0000","last_became_peered":"2024-05-08T22:05:13.548763+0000","last_unstale":"2024-05-08T22:05:40.827308+0000","last_undegraded":"2024-05-08T22:05:40.827308+0000","last_fullsized":"2024-05-08T22:05:40.827308+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:26:56.448611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830785+0000","last_change":"2024-05-08T22:05:14.067116+0000","last_active":"2024-05-08T22:05:40.830785+0000","last_peered":"2024-05-08T22:05:40.830785+0000","last_clean":"2024-05-08T22:05:40.830785+0000","last_became_active":"2024-05-08T22:05:14.064775+0000","last_became_peered":"2024-05-08T22:05:14.064775+0000","last_unstale":"2024-05-08T22:05:40.830785+0000","last_undegraded":"2024-05-08T22:05:40.830785+0000","last_fullsized":"2024-05-08T22:05:40.830785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:28:32.901405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817060+0000","last_change":"2024-05-08T22:05:18.539546+0000","last_active":"2024-05-08T22:05:40.817060+0000","last_peered":"2024-05-08T22:05:40.817060+0000","last_clean":"2024-05-08T22:05:40.817060+0000","last_became_active":"2024-05-08T22:05:18.538998+0000","last_became_peered":"2024-05-08T22:05:18.538998+0000","last_unstale":"2024-05-08T22:05:40.817060+0000","last_undegraded":"2024-05-08T22:05:40.817060+0000","last_fullsized":"2024-05-08T22:05:40.817060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:30.453881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"83'3","reported_seq":76,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:06:30.583413+0000","last_change":"2024-05-08T22:05:13.549053+0000","last_active":"2024-05-08T22:06:30.583413+0000","last_peered":"2024-05-08T22:06:30.583413+0000","last_clean":"2024-05-08T22:06:30.583413+0000","last_became_active":"2024-05-08T22:05:13.548739+0000","last_became_peered":"2024-05-08T22:05:13.548739+0000","last_unstale":"2024-05-08T22:06:30.583413+0000","last_undegraded":"2024-05-08T22:06:30.583413+0000","last_fullsized":"2024-05-08T22:06:30.583413+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:07:10.298084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830386+0000","last_change":"2024-05-08T22:05:14.061753+0000","last_active":"2024-05-08T22:05:40.830386+0000","last_peered":"2024-05-08T22:05:40.830386+0000","last_clean":"2024-05-08T22:05:40.830386+0000","last_became_active":"2024-05-08T22:05:14.061412+0000","last_became_peered":"2024-05-08T22:05:14.061412+0000","last_unstale":"2024-05-08T22:05:40.830386+0000","last_undegraded":"2024-05-08T22:05:40.830386+0000","last_fullsized":"2024-05-08T22:05:40.830386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:54.380181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824969+0000","last_change":"2024-05-08T22:05:18.532627+0000","last_active":"2024-05-08T22:05:40.824969+0000","last_peered":"2024-05-08T22:05:40.824969+0000","last_clean":"2024-05-08T22:05:40.824969+0000","last_became_active":"2024-05-08T22:05:18.532205+0000","last_became_peered":"2024-05-08T22:05:18.532205+0000","last_unstale":"2024-05-08T22:05:40.824969+0000","last_undegraded":"2024-05-08T22:05:40.824969+0000","last_fullsized":"2024-05-08T22:05:40.824969+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:20:11.028645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":61,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.841832+0000","last_change":"2024-05-08T22:05:41.841832+0000","last_active":"2024-05-08T22:05:41.841832+0000","last_peered":"2024-05-08T22:05:41.841832+0000","last_clean":"2024-05-08T22:05:41.841832+0000","last_became_active":"2024-05-08T22:05:41.829584+0000","last_became_peered":"2024-05-08T22:05:41.829584+0000","last_unstale":"2024-05-08T22:05:41.841832+0000","last_undegraded":"2024-05-08T22:05:41.841832+0000","last_fullsized":"2024-05-08T22:05:41.841832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:52:19.876215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824504+0000","last_change":"2024-05-08T22:05:14.063027+0000","last_active":"2024-05-08T22:05:40.824504+0000","last_peered":"2024-05-08T22:05:40.824504+0000","last_clean":"2024-05-08T22:05:40.824504+0000","last_became_active":"2024-05-08T22:05:14.062682+0000","last_became_peered":"2024-05-08T22:05:14.062682+0000","last_unstale":"2024-05-08T22:05:40.824504+0000","last_undegraded":"2024-05-08T22:05:40.824504+0000","last_fullsized":"2024-05-08T22:05:40.824504+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:47.248832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824420+0000","last_change":"2024-05-08T22:05:18.528370+0000","last_active":"2024-05-08T22:05:40.824420+0000","last_peered":"2024-05-08T22:05:40.824420+0000","last_clean":"2024-05-08T22:05:40.824420+0000","last_became_active":"2024-05-08T22:05:18.527990+0000","last_became_peered":"2024-05-08T22:05:18.527990+0000","last_unstale":"2024-05-08T22:05:40.824420+0000","last_undegraded":"2024-05-08T22:05:40.824420+0000","last_fullsized":"2024-05-08T22:05:40.824420+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:07.524175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.824561+0000","last_change":"2024-05-08T22:05:13.547543+0000","last_active":"2024-05-08T22:05:41.824561+0000","last_peered":"2024-05-08T22:05:41.824561+0000","last_clean":"2024-05-08T22:05:41.824561+0000","last_became_active":"2024-05-08T22:05:13.547191+0000","last_became_peered":"2024-05-08T22:05:13.547191+0000","last_unstale":"2024-05-08T22:05:41.824561+0000","last_undegraded":"2024-05-08T22:05:41.824561+0000","last_fullsized":"2024-05-08T22:05:41.824561+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:53.213239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.820871+0000","last_change":"2024-05-08T22:05:14.062677+0000","last_active":"2024-05-08T22:05:40.820871+0000","last_peered":"2024-05-08T22:05:40.820871+0000","last_clean":"2024-05-08T22:05:40.820871+0000","last_became_active":"2024-05-08T22:05:14.062306+0000","last_became_peered":"2024-05-08T22:05:14.062306+0000","last_unstale":"2024-05-08T22:05:40.820871+0000","last_undegraded":"2024-05-08T22:05:40.820871+0000","last_fullsized":"2024-05-08T22:05:40.820871+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:00.356505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830129+0000","last_change":"2024-05-08T22:05:18.617075+0000","last_active":"2024-05-08T22:05:40.830129+0000","last_peered":"2024-05-08T22:05:40.830129+0000","last_clean":"2024-05-08T22:05:40.830129+0000","last_became_active":"2024-05-08T22:05:18.616653+0000","last_became_peered":"2024-05-08T22:05:18.616653+0000","last_unstale":"2024-05-08T22:05:40.830129+0000","last_undegraded":"2024-05-08T22:05:40.830129+0000","last_fullsized":"2024-05-08T22:05:40.830129+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:30.537064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.816884+0000","last_change":"2024-05-08T22:05:13.553457+0000","last_active":"2024-05-08T22:05:40.816884+0000","last_peered":"2024-05-08T22:05:40.816884+0000","last_clean":"2024-05-08T22:05:40.816884+0000","last_became_active":"2024-05-08T22:05:13.551583+0000","last_became_peered":"2024-05-08T22:05:13.551583+0000","last_unstale":"2024-05-08T22:05:40.816884+0000","last_undegraded":"2024-05-08T22:05:40.816884+0000","last_fullsized":"2024-05-08T22:05:40.816884+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:07:05.971413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.822908+0000","last_change":"2024-05-08T22:05:14.057476+0000","last_active":"2024-05-08T22:05:40.822908+0000","last_peered":"2024-05-08T22:05:40.822908+0000","last_clean":"2024-05-08T22:05:40.822908+0000","last_became_active":"2024-05-08T22:05:14.056550+0000","last_became_peered":"2024-05-08T22:05:14.056550+0000","last_unstale":"2024-05-08T22:05:40.822908+0000","last_undegraded":"2024-05-08T22:05:40.822908+0000","last_fullsized":"2024-05-08T22:05:40.822908+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:41:29.264096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824298+0000","last_change":"2024-05-08T22:05:18.530719+0000","last_active":"2024-05-08T22:05:40.824298+0000","last_peered":"2024-05-08T22:05:40.824298+0000","last_clean":"2024-05-08T22:05:40.824298+0000","last_became_active":"2024-05-08T22:05:18.529659+0000","last_became_peered":"2024-05-08T22:05:18.529659+0000","last_unstale":"2024-05-08T22:05:40.824298+0000","last_undegraded":"2024-05-08T22:05:40.824298+0000","last_fullsized":"2024-05-08T22:05:40.824298+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:13:06.514460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830542+0000","last_change":"2024-05-08T22:05:13.556191+0000","last_active":"2024-05-08T22:05:40.830542+0000","last_peered":"2024-05-08T22:05:40.830542+0000","last_clean":"2024-05-08T22:05:40.830542+0000","last_became_active":"2024-05-08T22:05:13.555888+0000","last_became_peered":"2024-05-08T22:05:13.555888+0000","last_unstale":"2024-05-08T22:05:40.830542+0000","last_undegraded":"2024-05-08T22:05:40.830542+0000","last_fullsized":"2024-05-08T22:05:40.830542+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:05.637980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.828023+0000","last_change":"2024-05-08T22:05:41.828023+0000","last_active":"2024-05-08T22:05:41.828023+0000","last_peered":"2024-05-08T22:05:41.828023+0000","last_clean":"2024-05-08T22:05:41.828023+0000","last_became_active":"2024-05-08T22:05:41.827550+0000","last_became_peered":"2024-05-08T22:05:41.827550+0000","last_unstale":"2024-05-08T22:05:41.828023+0000","last_undegraded":"2024-05-08T22:05:41.828023+0000","last_fullsized":"2024-05-08T22:05:41.828023+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:13.234423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.828529+0000","last_change":"2024-05-08T22:05:41.828529+0000","last_active":"2024-05-08T22:05:41.828529+0000","last_peered":"2024-05-08T22:05:41.828529+0000","last_clean":"2024-05-08T22:05:41.828529+0000","last_became_active":"2024-05-08T22:05:41.828081+0000","last_became_peered":"2024-05-08T22:05:41.828081+0000","last_unstale":"2024-05-08T22:05:41.828529+0000","last_undegraded":"2024-05-08T22:05:41.828529+0000","last_fullsized":"2024-05-08T22:05:41.828529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:27.478823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826551+0000","last_change":"2024-05-08T22:05:14.069096+0000","last_active":"2024-05-08T22:05:40.826551+0000","last_peered":"2024-05-08T22:05:40.826551+0000","last_clean":"2024-05-08T22:05:40.826551+0000","last_became_active":"2024-05-08T22:05:14.068796+0000","last_became_peered":"2024-05-08T22:05:14.068796+0000","last_unstale":"2024-05-08T22:05:40.826551+0000","last_undegraded":"2024-05-08T22:05:40.826551+0000","last_fullsized":"2024-05-08T22:05:40.826551+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:52:52.391196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":75,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:06:30.582780+0000","last_change":"2024-05-08T22:05:13.559286+0000","last_active":"2024-05-08T22:06:30.582780+0000","last_peered":"2024-05-08T22:06:30.582780+0000","last_clean":"2024-05-08T22:06:30.582780+0000","last_became_active":"2024-05-08T22:05:13.558922+0000","last_became_peered":"2024-05-08T22:05:13.558922+0000","last_unstale":"2024-05-08T22:06:30.582780+0000","last_undegraded":"2024-05-08T22:06:30.582780+0000","last_fullsized":"2024-05-08T22:06:30.582780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:25:24.732422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831007+0000","last_change":"2024-05-08T22:05:18.527031+0000","last_active":"2024-05-08T22:05:40.831007+0000","last_peered":"2024-05-08T22:05:40.831007+0000","last_clean":"2024-05-08T22:05:40.831007+0000","last_became_active":"2024-05-08T22:05:18.526573+0000","last_became_peered":"2024-05-08T22:05:18.526573+0000","last_unstale":"2024-05-08T22:05:40.831007+0000","last_undegraded":"2024-05-08T22:05:40.831007+0000","last_fullsized":"2024-05-08T22:05:40.831007+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:08.492359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827472+0000","last_change":"2024-05-08T22:05:14.053930+0000","last_active":"2024-05-08T22:05:40.827472+0000","last_peered":"2024-05-08T22:05:40.827472+0000","last_clean":"2024-05-08T22:05:40.827472+0000","last_became_active":"2024-05-08T22:05:14.052968+0000","last_became_peered":"2024-05-08T22:05:14.052968+0000","last_unstale":"2024-05-08T22:05:40.827472+0000","last_undegraded":"2024-05-08T22:05:40.827472+0000","last_fullsized":"2024-05-08T22:05:40.827472+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:48.748323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819345+0000","last_change":"2024-05-08T22:05:13.545709+0000","last_active":"2024-05-08T22:05:40.819345+0000","last_peered":"2024-05-08T22:05:40.819345+0000","last_clean":"2024-05-08T22:05:40.819345+0000","last_became_active":"2024-05-08T22:05:13.545419+0000","last_became_peered":"2024-05-08T22:05:13.545419+0000","last_unstale":"2024-05-08T22:05:40.819345+0000","last_undegraded":"2024-05-08T22:05:40.819345+0000","last_fullsized":"2024-05-08T22:05:40.819345+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:44.903903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":337,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830206+0000","last_change":"2024-05-08T22:04:20.948328+0000","last_active":"2024-05-08T22:05:40.830206+0000","last_peered":"2024-05-08T22:05:40.830206+0000","last_clean":"2024-05-08T22:05:40.830206+0000","last_became_active":"2024-05-08T22:04:20.947887+0000","last_became_peered":"2024-05-08T22:04:20.947887+0000","last_unstale":"2024-05-08T22:05:40.830206+0000","last_undegraded":"2024-05-08T22:05:40.830206+0000","last_fullsized":"2024-05-08T22:05:40.830206+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:01:41.485277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:01:41.485277+0000","last_clean_scrub_stamp":"2024-05-08T22:01:41.485277+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:43:59.208120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.824591+0000","last_change":"2024-05-08T22:05:41.824591+0000","last_active":"2024-05-08T22:05:41.824591+0000","last_peered":"2024-05-08T22:05:41.824591+0000","last_clean":"2024-05-08T22:05:41.824591+0000","last_became_active":"2024-05-08T22:05:41.824053+0000","last_became_peered":"2024-05-08T22:05:41.824053+0000","last_unstale":"2024-05-08T22:05:41.824591+0000","last_undegraded":"2024-05-08T22:05:41.824591+0000","last_fullsized":"2024-05-08T22:05:41.824591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:18:57.269459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.820969+0000","last_change":"2024-05-08T22:05:14.066818+0000","last_active":"2024-05-08T22:05:40.820969+0000","last_peered":"2024-05-08T22:05:40.820969+0000","last_clean":"2024-05-08T22:05:40.820969+0000","last_became_active":"2024-05-08T22:05:14.065152+0000","last_became_peered":"2024-05-08T22:05:14.065152+0000","last_unstale":"2024-05-08T22:05:40.820969+0000","last_undegraded":"2024-05-08T22:05:40.820969+0000","last_fullsized":"2024-05-08T22:05:40.820969+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:53:19.018852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830323+0000","last_change":"2024-05-08T22:05:13.550365+0000","last_active":"2024-05-08T22:05:40.830323+0000","last_peered":"2024-05-08T22:05:40.830323+0000","last_clean":"2024-05-08T22:05:40.830323+0000","last_became_active":"2024-05-08T22:05:13.549677+0000","last_became_peered":"2024-05-08T22:05:13.549677+0000","last_unstale":"2024-05-08T22:05:40.830323+0000","last_undegraded":"2024-05-08T22:05:40.830323+0000","last_fullsized":"2024-05-08T22:05:40.830323+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:10.842838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.820719+0000","last_change":"2024-05-08T22:05:18.535777+0000","last_active":"2024-05-08T22:05:40.820719+0000","last_peered":"2024-05-08T22:05:40.820719+0000","last_clean":"2024-05-08T22:05:40.820719+0000","last_became_active":"2024-05-08T22:05:18.535327+0000","last_became_peered":"2024-05-08T22:05:18.535327+0000","last_unstale":"2024-05-08T22:05:40.820719+0000","last_undegraded":"2024-05-08T22:05:40.820719+0000","last_fullsized":"2024-05-08T22:05:40.820719+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:13.886989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831170+0000","last_change":"2024-05-08T22:05:13.554045+0000","last_active":"2024-05-08T22:05:40.831170+0000","last_peered":"2024-05-08T22:05:40.831170+0000","last_clean":"2024-05-08T22:05:40.831170+0000","last_became_active":"2024-05-08T22:05:13.553315+0000","last_became_peered":"2024-05-08T22:05:13.553315+0000","last_unstale":"2024-05-08T22:05:40.831170+0000","last_undegraded":"2024-05-08T22:05:40.831170+0000","last_fullsized":"2024-05-08T22:05:40.831170+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:13.131267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824438+0000","last_change":"2024-05-08T22:05:14.065018+0000","last_active":"2024-05-08T22:05:40.824438+0000","last_peered":"2024-05-08T22:05:40.824438+0000","last_clean":"2024-05-08T22:05:40.824438+0000","last_became_active":"2024-05-08T22:05:14.064701+0000","last_became_peered":"2024-05-08T22:05:14.064701+0000","last_unstale":"2024-05-08T22:05:40.824438+0000","last_undegraded":"2024-05-08T22:05:40.824438+0000","last_fullsized":"2024-05-08T22:05:40.824438+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:48:43.468373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826618+0000","last_change":"2024-05-08T22:05:18.617853+0000","last_active":"2024-05-08T22:05:40.826618+0000","last_peered":"2024-05-08T22:05:40.826618+0000","last_clean":"2024-05-08T22:05:40.826618+0000","last_became_active":"2024-05-08T22:05:18.616255+0000","last_became_peered":"2024-05-08T22:05:18.616255+0000","last_unstale":"2024-05-08T22:05:40.826618+0000","last_undegraded":"2024-05-08T22:05:40.826618+0000","last_fullsized":"2024-05-08T22:05:40.826618+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:13.233425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'5","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:42.786006+0000","last_change":"2024-05-08T22:05:13.546274+0000","last_active":"2024-05-08T22:05:42.786006+0000","last_peered":"2024-05-08T22:05:42.786006+0000","last_clean":"2024-05-08T22:05:42.786006+0000","last_became_active":"2024-05-08T22:05:13.543651+0000","last_became_peered":"2024-05-08T22:05:13.543651+0000","last_unstale":"2024-05-08T22:05:42.786006+0000","last_undegraded":"2024-05-08T22:05:42.786006+0000","last_fullsized":"2024-05-08T22:05:42.786006+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:21:58.601261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826229+0000","last_change":"2024-05-08T22:05:14.056535+0000","last_active":"2024-05-08T22:05:40.826229+0000","last_peered":"2024-05-08T22:05:40.826229+0000","last_clean":"2024-05-08T22:05:40.826229+0000","last_became_active":"2024-05-08T22:05:14.056211+0000","last_became_peered":"2024-05-08T22:05:14.056211+0000","last_unstale":"2024-05-08T22:05:40.826229+0000","last_undegraded":"2024-05-08T22:05:40.826229+0000","last_fullsized":"2024-05-08T22:05:40.826229+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:46:30.744490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.816916+0000","last_change":"2024-05-08T22:05:18.524939+0000","last_active":"2024-05-08T22:05:40.816916+0000","last_peered":"2024-05-08T22:05:40.816916+0000","last_clean":"2024-05-08T22:05:40.816916+0000","last_became_active":"2024-05-08T22:05:18.524518+0000","last_became_peered":"2024-05-08T22:05:18.524518+0000","last_unstale":"2024-05-08T22:05:40.816916+0000","last_undegraded":"2024-05-08T22:05:40.816916+0000","last_fullsized":"2024-05-08T22:05:40.816916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:50.192400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819897+0000","last_change":"2024-05-08T22:05:13.554346+0000","last_active":"2024-05-08T22:05:40.819897+0000","last_peered":"2024-05-08T22:05:40.819897+0000","last_clean":"2024-05-08T22:05:40.819897+0000","last_became_active":"2024-05-08T22:05:13.554080+0000","last_became_peered":"2024-05-08T22:05:13.554080+0000","last_unstale":"2024-05-08T22:05:40.819897+0000","last_undegraded":"2024-05-08T22:05:40.819897+0000","last_fullsized":"2024-05-08T22:05:40.819897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:29:49.695682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832687+0000","last_change":"2024-05-08T22:05:14.071422+0000","last_active":"2024-05-08T22:05:40.832687+0000","last_peered":"2024-05-08T22:05:40.832687+0000","last_clean":"2024-05-08T22:05:40.832687+0000","last_became_active":"2024-05-08T22:05:14.070396+0000","last_became_peered":"2024-05-08T22:05:14.070396+0000","last_unstale":"2024-05-08T22:05:40.832687+0000","last_undegraded":"2024-05-08T22:05:40.832687+0000","last_fullsized":"2024-05-08T22:05:40.832687+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:37.957983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819540+0000","last_change":"2024-05-08T22:05:18.617149+0000","last_active":"2024-05-08T22:05:40.819540+0000","last_peered":"2024-05-08T22:05:40.819540+0000","last_clean":"2024-05-08T22:05:40.819540+0000","last_became_active":"2024-05-08T22:05:18.616699+0000","last_became_peered":"2024-05-08T22:05:18.616699+0000","last_unstale":"2024-05-08T22:05:40.819540+0000","last_undegraded":"2024-05-08T22:05:40.819540+0000","last_fullsized":"2024-05-08T22:05:40.819540+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:51.167142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.823777+0000","last_change":"2024-05-08T22:05:13.551212+0000","last_active":"2024-05-08T22:05:40.823777+0000","last_peered":"2024-05-08T22:05:40.823777+0000","last_clean":"2024-05-08T22:05:40.823777+0000","last_became_active":"2024-05-08T22:05:13.550909+0000","last_became_peered":"2024-05-08T22:05:13.550909+0000","last_unstale":"2024-05-08T22:05:40.823777+0000","last_undegraded":"2024-05-08T22:05:40.823777+0000","last_fullsized":"2024-05-08T22:05:40.823777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:48.657928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829703+0000","last_change":"2024-05-08T22:05:14.053976+0000","last_active":"2024-05-08T22:05:40.829703+0000","last_peered":"2024-05-08T22:05:40.829703+0000","last_clean":"2024-05-08T22:05:40.829703+0000","last_became_active":"2024-05-08T22:05:14.053664+0000","last_became_peered":"2024-05-08T22:05:14.053664+0000","last_unstale":"2024-05-08T22:05:40.829703+0000","last_undegraded":"2024-05-08T22:05:40.829703+0000","last_fullsized":"2024-05-08T22:05:40.829703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:25:25.642768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817203+0000","last_change":"2024-05-08T22:05:18.536771+0000","last_active":"2024-05-08T22:05:40.817203+0000","last_peered":"2024-05-08T22:05:40.817203+0000","last_clean":"2024-05-08T22:05:40.817203+0000","last_became_active":"2024-05-08T22:05:18.536260+0000","last_became_peered":"2024-05-08T22:05:18.536260+0000","last_unstale":"2024-05-08T22:05:40.817203+0000","last_undegraded":"2024-05-08T22:05:40.817203+0000","last_fullsized":"2024-05-08T22:05:40.817203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:34.414381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827085+0000","last_change":"2024-05-08T22:05:13.540679+0000","last_active":"2024-05-08T22:05:40.827085+0000","last_peered":"2024-05-08T22:05:40.827085+0000","last_clean":"2024-05-08T22:05:40.827085+0000","last_became_active":"2024-05-08T22:05:13.539428+0000","last_became_peered":"2024-05-08T22:05:13.539428+0000","last_unstale":"2024-05-08T22:05:40.827085+0000","last_undegraded":"2024-05-08T22:05:40.827085+0000","last_fullsized":"2024-05-08T22:05:40.827085+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:59:34.455411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.820331+0000","last_change":"2024-05-08T22:05:14.061849+0000","last_active":"2024-05-08T22:05:40.820331+0000","last_peered":"2024-05-08T22:05:40.820331+0000","last_clean":"2024-05-08T22:05:40.820331+0000","last_became_active":"2024-05-08T22:05:14.061028+0000","last_became_peered":"2024-05-08T22:05:14.061028+0000","last_unstale":"2024-05-08T22:05:40.820331+0000","last_undegraded":"2024-05-08T22:05:40.820331+0000","last_fullsized":"2024-05-08T22:05:40.820331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:11:59.489376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.477584+0000","last_change":"2024-05-08T22:05:18.517994+0000","last_active":"2024-05-08T22:05:41.477584+0000","last_peered":"2024-05-08T22:05:41.477584+0000","last_clean":"2024-05-08T22:05:41.477584+0000","last_became_active":"2024-05-08T22:05:18.517655+0000","last_became_peered":"2024-05-08T22:05:18.517655+0000","last_unstale":"2024-05-08T22:05:41.477584+0000","last_undegraded":"2024-05-08T22:05:41.477584+0000","last_fullsized":"2024-05-08T22:05:41.477584+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:50:08.956277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"84'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.025839+0000","last_change":"2024-05-08T22:05:13.552851+0000","last_active":"2024-05-08T22:05:41.025839+0000","last_peered":"2024-05-08T22:05:41.025839+0000","last_clean":"2024-05-08T22:05:41.025839+0000","last_became_active":"2024-05-08T22:05:13.552507+0000","last_became_peered":"2024-05-08T22:05:13.552507+0000","last_unstale":"2024-05-08T22:05:41.025839+0000","last_undegraded":"2024-05-08T22:05:41.025839+0000","last_fullsized":"2024-05-08T22:05:41.025839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:29.248623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.828040+0000","last_change":"2024-05-08T22:05:14.062618+0000","last_active":"2024-05-08T22:05:40.828040+0000","last_peered":"2024-05-08T22:05:40.828040+0000","last_clean":"2024-05-08T22:05:40.828040+0000","last_became_active":"2024-05-08T22:05:14.062308+0000","last_became_peered":"2024-05-08T22:05:14.062308+0000","last_unstale":"2024-05-08T22:05:40.828040+0000","last_undegraded":"2024-05-08T22:05:40.828040+0000","last_fullsized":"2024-05-08T22:05:40.828040+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:34.195343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819863+0000","last_change":"2024-05-08T22:05:18.616846+0000","last_active":"2024-05-08T22:05:40.819863+0000","last_peered":"2024-05-08T22:05:40.819863+0000","last_clean":"2024-05-08T22:05:40.819863+0000","last_became_active":"2024-05-08T22:05:18.616369+0000","last_became_peered":"2024-05-08T22:05:18.616369+0000","last_unstale":"2024-05-08T22:05:40.819863+0000","last_undegraded":"2024-05-08T22:05:40.819863+0000","last_fullsized":"2024-05-08T22:05:40.819863+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:26.128949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829370+0000","last_change":"2024-05-08T22:05:13.549712+0000","last_active":"2024-05-08T22:05:40.829370+0000","last_peered":"2024-05-08T22:05:40.829370+0000","last_clean":"2024-05-08T22:05:40.829370+0000","last_became_active":"2024-05-08T22:05:13.549508+0000","last_became_peered":"2024-05-08T22:05:13.549508+0000","last_unstale":"2024-05-08T22:05:40.829370+0000","last_undegraded":"2024-05-08T22:05:40.829370+0000","last_fullsized":"2024-05-08T22:05:40.829370+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:30:22.706389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830535+0000","last_change":"2024-05-08T22:05:14.060920+0000","last_active":"2024-05-08T22:05:40.830535+0000","last_peered":"2024-05-08T22:05:40.830535+0000","last_clean":"2024-05-08T22:05:40.830535+0000","last_became_active":"2024-05-08T22:05:14.059914+0000","last_became_peered":"2024-05-08T22:05:14.059914+0000","last_unstale":"2024-05-08T22:05:40.830535+0000","last_undegraded":"2024-05-08T22:05:40.830535+0000","last_fullsized":"2024-05-08T22:05:40.830535+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:15.513600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.822527+0000","last_change":"2024-05-08T22:05:18.534076+0000","last_active":"2024-05-08T22:05:40.822527+0000","last_peered":"2024-05-08T22:05:40.822527+0000","last_clean":"2024-05-08T22:05:40.822527+0000","last_became_active":"2024-05-08T22:05:18.533679+0000","last_became_peered":"2024-05-08T22:05:18.533679+0000","last_unstale":"2024-05-08T22:05:40.822527+0000","last_undegraded":"2024-05-08T22:05:40.822527+0000","last_fullsized":"2024-05-08T22:05:40.822527+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:08.979193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.845713+0000","last_change":"2024-05-08T22:05:41.845354+0000","last_active":"2024-05-08T22:05:41.845713+0000","last_peered":"2024-05-08T22:05:41.845713+0000","last_clean":"2024-05-08T22:05:41.845713+0000","last_became_active":"2024-05-08T22:05:41.830506+0000","last_became_peered":"2024-05-08T22:05:41.830506+0000","last_unstale":"2024-05-08T22:05:41.845713+0000","last_undegraded":"2024-05-08T22:05:41.845713+0000","last_fullsized":"2024-05-08T22:05:41.845713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:53:23.081939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831111+0000","last_change":"2024-05-08T22:05:14.065868+0000","last_active":"2024-05-08T22:05:40.831111+0000","last_peered":"2024-05-08T22:05:40.831111+0000","last_clean":"2024-05-08T22:05:40.831111+0000","last_became_active":"2024-05-08T22:05:14.065545+0000","last_became_peered":"2024-05-08T22:05:14.065545+0000","last_unstale":"2024-05-08T22:05:40.831111+0000","last_undegraded":"2024-05-08T22:05:40.831111+0000","last_fullsized":"2024-05-08T22:05:40.831111+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:56:34.309095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.832203+0000","last_change":"2024-05-08T22:05:18.534261+0000","last_active":"2024-05-08T22:05:40.832203+0000","last_peered":"2024-05-08T22:05:40.832203+0000","last_clean":"2024-05-08T22:05:40.832203+0000","last_became_active":"2024-05-08T22:05:18.531814+0000","last_became_peered":"2024-05-08T22:05:18.531814+0000","last_unstale":"2024-05-08T22:05:40.832203+0000","last_undegraded":"2024-05-08T22:05:40.832203+0000","last_fullsized":"2024-05-08T22:05:40.832203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:38.344546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.827613+0000","last_change":"2024-05-08T22:05:13.549344+0000","last_active":"2024-05-08T22:05:40.827613+0000","last_peered":"2024-05-08T22:05:40.827613+0000","last_clean":"2024-05-08T22:05:40.827613+0000","last_became_active":"2024-05-08T22:05:13.548993+0000","last_became_peered":"2024-05-08T22:05:13.548993+0000","last_unstale":"2024-05-08T22:05:40.827613+0000","last_undegraded":"2024-05-08T22:05:40.827613+0000","last_fullsized":"2024-05-08T22:05:40.827613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:50.096602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.830884+0000","last_change":"2024-05-08T22:05:14.065758+0000","last_active":"2024-05-08T22:05:40.830884+0000","last_peered":"2024-05-08T22:05:40.830884+0000","last_clean":"2024-05-08T22:05:40.830884+0000","last_became_active":"2024-05-08T22:05:14.065442+0000","last_became_peered":"2024-05-08T22:05:14.065442+0000","last_unstale":"2024-05-08T22:05:40.830884+0000","last_undegraded":"2024-05-08T22:05:40.830884+0000","last_fullsized":"2024-05-08T22:05:40.830884+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:43.047796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826157+0000","last_change":"2024-05-08T22:05:18.529804+0000","last_active":"2024-05-08T22:05:40.826157+0000","last_peered":"2024-05-08T22:05:40.826157+0000","last_clean":"2024-05-08T22:05:40.826157+0000","last_became_active":"2024-05-08T22:05:18.529346+0000","last_became_peered":"2024-05-08T22:05:18.529346+0000","last_unstale":"2024-05-08T22:05:40.826157+0000","last_undegraded":"2024-05-08T22:05:40.826157+0000","last_fullsized":"2024-05-08T22:05:40.826157+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:13.565707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824885+0000","last_change":"2024-05-08T22:05:13.551818+0000","last_active":"2024-05-08T22:05:40.824885+0000","last_peered":"2024-05-08T22:05:40.824885+0000","last_clean":"2024-05-08T22:05:40.824885+0000","last_became_active":"2024-05-08T22:05:13.551080+0000","last_became_peered":"2024-05-08T22:05:13.551080+0000","last_unstale":"2024-05-08T22:05:40.824885+0000","last_undegraded":"2024-05-08T22:05:40.824885+0000","last_fullsized":"2024-05-08T22:05:40.824885+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:31.636482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.831045+0000","last_change":"2024-05-08T22:05:14.066572+0000","last_active":"2024-05-08T22:05:40.831045+0000","last_peered":"2024-05-08T22:05:40.831045+0000","last_clean":"2024-05-08T22:05:40.831045+0000","last_became_active":"2024-05-08T22:05:14.066272+0000","last_became_peered":"2024-05-08T22:05:14.066272+0000","last_unstale":"2024-05-08T22:05:40.831045+0000","last_undegraded":"2024-05-08T22:05:40.831045+0000","last_fullsized":"2024-05-08T22:05:40.831045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:20:27.423464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.820443+0000","last_change":"2024-05-08T22:05:14.061435+0000","last_active":"2024-05-08T22:05:40.820443+0000","last_peered":"2024-05-08T22:05:40.820443+0000","last_clean":"2024-05-08T22:05:40.820443+0000","last_became_active":"2024-05-08T22:05:14.061079+0000","last_became_peered":"2024-05-08T22:05:14.061079+0000","last_unstale":"2024-05-08T22:05:40.820443+0000","last_undegraded":"2024-05-08T22:05:40.820443+0000","last_fullsized":"2024-05-08T22:05:40.820443+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:42.684743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T22:05:41.828352+0000","last_change":"2024-05-08T22:05:41.828352+0000","last_active":"2024-05-08T22:05:41.828352+0000","last_peered":"2024-05-08T22:05:41.828352+0000","last_clean":"2024-05-08T22:05:41.828352+0000","last_became_active":"2024-05-08T22:05:41.827782+0000","last_became_peered":"2024-05-08T22:05:41.827782+0000","last_unstale":"2024-05-08T22:05:41.828352+0000","last_undegraded":"2024-05-08T22:05:41.828352+0000","last_fullsized":"2024-05-08T22:05:41.828352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:08.883148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.817061+0000","last_change":"2024-05-08T22:05:14.066697+0000","last_active":"2024-05-08T22:05:40.817061+0000","last_peered":"2024-05-08T22:05:40.817061+0000","last_clean":"2024-05-08T22:05:40.817061+0000","last_became_active":"2024-05-08T22:05:14.065952+0000","last_became_peered":"2024-05-08T22:05:14.065952+0000","last_unstale":"2024-05-08T22:05:40.817061+0000","last_undegraded":"2024-05-08T22:05:40.817061+0000","last_fullsized":"2024-05-08T22:05:40.817061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:03:19.006644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.833301+0000","last_change":"2024-05-08T22:05:14.072295+0000","last_active":"2024-05-08T22:05:40.833301+0000","last_peered":"2024-05-08T22:05:40.833301+0000","last_clean":"2024-05-08T22:05:40.833301+0000","last_became_active":"2024-05-08T22:05:14.065034+0000","last_became_peered":"2024-05-08T22:05:14.065034+0000","last_unstale":"2024-05-08T22:05:40.833301+0000","last_undegraded":"2024-05-08T22:05:40.833301+0000","last_fullsized":"2024-05-08T22:05:40.833301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:08:41.380814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.824510+0000","last_change":"2024-05-08T22:05:14.067263+0000","last_active":"2024-05-08T22:05:40.824510+0000","last_peered":"2024-05-08T22:05:40.824510+0000","last_clean":"2024-05-08T22:05:40.824510+0000","last_became_active":"2024-05-08T22:05:14.066957+0000","last_became_peered":"2024-05-08T22:05:14.066957+0000","last_unstale":"2024-05-08T22:05:40.824510+0000","last_undegraded":"2024-05-08T22:05:40.824510+0000","last_fullsized":"2024-05-08T22:05:40.824510+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:29:35.638489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.822207+0000","last_change":"2024-05-08T22:05:14.060561+0000","last_active":"2024-05-08T22:05:40.822207+0000","last_peered":"2024-05-08T22:05:40.822207+0000","last_clean":"2024-05-08T22:05:40.822207+0000","last_became_active":"2024-05-08T22:05:14.060150+0000","last_became_peered":"2024-05-08T22:05:14.060150+0000","last_unstale":"2024-05-08T22:05:40.822207+0000","last_undegraded":"2024-05-08T22:05:40.822207+0000","last_fullsized":"2024-05-08T22:05:40.822207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:22.710776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.819471+0000","last_change":"2024-05-08T22:05:14.056394+0000","last_active":"2024-05-08T22:05:40.819471+0000","last_peered":"2024-05-08T22:05:40.819471+0000","last_clean":"2024-05-08T22:05:40.819471+0000","last_became_active":"2024-05-08T22:05:14.055977+0000","last_became_peered":"2024-05-08T22:05:14.055977+0000","last_unstale":"2024-05-08T22:05:40.819471+0000","last_undegraded":"2024-05-08T22:05:40.819471+0000","last_fullsized":"2024-05-08T22:05:40.819471+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:56:48.249379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.829006+0000","last_change":"2024-05-08T22:05:14.061934+0000","last_active":"2024-05-08T22:05:40.829006+0000","last_peered":"2024-05-08T22:05:40.829006+0000","last_clean":"2024-05-08T22:05:40.829006+0000","last_became_active":"2024-05-08T22:05:14.061635+0000","last_became_peered":"2024-05-08T22:05:14.061635+0000","last_unstale":"2024-05-08T22:05:40.829006+0000","last_undegraded":"2024-05-08T22:05:40.829006+0000","last_fullsized":"2024-05-08T22:05:40.829006+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:02.423338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826467+0000","last_change":"2024-05-08T22:05:14.069982+0000","last_active":"2024-05-08T22:05:40.826467+0000","last_peered":"2024-05-08T22:05:40.826467+0000","last_clean":"2024-05-08T22:05:40.826467+0000","last_became_active":"2024-05-08T22:05:14.068293+0000","last_became_peered":"2024-05-08T22:05:14.068293+0000","last_unstale":"2024-05-08T22:05:40.826467+0000","last_undegraded":"2024-05-08T22:05:40.826467+0000","last_fullsized":"2024-05-08T22:05:40.826467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:36.396983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.826968+0000","last_change":"2024-05-08T22:05:14.058393+0000","last_active":"2024-05-08T22:05:40.826968+0000","last_peered":"2024-05-08T22:05:40.826968+0000","last_clean":"2024-05-08T22:05:40.826968+0000","last_became_active":"2024-05-08T22:05:14.058083+0000","last_became_peered":"2024-05-08T22:05:14.058083+0000","last_unstale":"2024-05-08T22:05:40.826968+0000","last_undegraded":"2024-05-08T22:05:40.826968+0000","last_fullsized":"2024-05-08T22:05:40.826968+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:10:47.074957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.822859+0000","last_change":"2024-05-08T22:05:14.056989+0000","last_active":"2024-05-08T22:05:40.822859+0000","last_peered":"2024-05-08T22:05:40.822859+0000","last_clean":"2024-05-08T22:05:40.822859+0000","last_became_active":"2024-05-08T22:05:14.056669+0000","last_became_peered":"2024-05-08T22:05:14.056669+0000","last_unstale":"2024-05-08T22:05:40.822859+0000","last_undegraded":"2024-05-08T22:05:40.822859+0000","last_fullsized":"2024-05-08T22:05:40.822859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:56:51.173885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.823896+0000","last_change":"2024-05-08T22:05:14.058635+0000","last_active":"2024-05-08T22:05:40.823896+0000","last_peered":"2024-05-08T22:05:40.823896+0000","last_clean":"2024-05-08T22:05:40.823896+0000","last_became_active":"2024-05-08T22:05:14.058243+0000","last_became_peered":"2024-05-08T22:05:14.058243+0000","last_unstale":"2024-05-08T22:05:40.823896+0000","last_undegraded":"2024-05-08T22:05:40.823896+0000","last_fullsized":"2024-05-08T22:05:40.823896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:18.142563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T22:05:40.822551+0000","last_change":"2024-05-08T22:05:14.060735+0000","last_active":"2024-05-08T22:05:40.822551+0000","last_peered":"2024-05-08T22:05:40.822551+0000","last_clean":"2024-05-08T22:05:40.822551+0000","last_became_active":"2024-05-08T22:05:14.060391+0000","last_became_peered":"2024-05-08T22:05:14.060391+0000","last_unstale":"2024-05-08T22:05:40.822551+0000","last_undegraded":"2024-05-08T22:05:40.822551+0000","last_fullsized":"2024-05-08T22:05:40.822551+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:30.528705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283045,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63358,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":328,"num_read_kb":230,"num_write":153,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":694,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":190074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579931,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36124,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709028,"statfs":{"total":95995035648,"available":95958044672,"internally_reserved":0,"allocated":929792,"data_stored":470051,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36043205},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.61699999999999999}]},{"osd":1,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.41899999999999998}]},{"osd":2,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.371},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.39200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":4,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.51100000000000001}]},{"osd":5,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.63300000000000001}]},{"osd":6,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.58899999999999997}]},{"osd":7,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.64300000000000002}]},{"osd":8,"last update":"Wed May 8 22:06:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.72199999999999998}]},{"osd":9,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.74399999999999999}]},{"osd":10,"last update":"Wed May 8 22:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.626}]}]},{"osd":10,"up_from":67,"seq":287762808861,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1069056,"data_stored":1116391,"data_compressed":53579,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:06:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.258,"5min":7.258,"15min":7.258},"last":0.56899999999999995}]},{"osd":1,"last update":"Wed May 8 22:06:19 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.439,"15min":0.439},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.47399999999999998,"15min":0.47199999999999998},"min":{"1min":0.32600000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.53800000000000003}]},{"osd":2,"last update":"Wed May 8 22:06:19 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.624,"15min":0.61899999999999999},"min":{"1min":0.51700000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.628,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.48799999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77000000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.65600000000000003}]},{"osd":3,"last update":"Wed May 8 22:06:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.46400000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.68000000000000005,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.46000000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.72799999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.67800000000000005}]},{"osd":4,"last update":"Wed May 8 22:06:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.38,"5min":0.379,"15min":0.379},"max":{"1min":0.68100000000000005,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52300000000000002,"15min":0.52000000000000002},"min":{"1min":0.34000000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.753,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.76800000000000002}]},{"osd":5,"last update":"Wed May 8 22:06:19 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.46400000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66100000000000003,"15min":0.66200000000000003},"min":{"1min":0.54400000000000004,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.81999999999999995,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.91300000000000003}]},{"osd":6,"last update":"Wed May 8 22:06:19 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.72499999999999998,"15min":0.67900000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":7.2380000000000004,"5min":7.2380000000000004,"15min":7.2380000000000004},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.69899999999999995,"15min":0.69299999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":8,"last update":"Wed May 8 22:06:19 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72199999999999998,"15min":0.72799999999999998},"min":{"1min":0.499,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.86099999999999999,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.54600000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.88400000000000001,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.93799999999999994}]},{"osd":9,"last update":"Wed May 8 22:06:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.755,"15min":0.75800000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.88600000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.117,"5min":0.83699999999999997,"15min":0.79000000000000004},"min":{"1min":0.58799999999999997,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":7.2939999999999996,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.89500000000000002}]},{"osd":11,"last update":"Wed May 8 22:05:39 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.92300000000000004}]}]},{"osd":9,"up_from":62,"seq":266287972386,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":474383,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:06:01 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.52100000000000002,"15min":0.52800000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.58299999999999996,"15min":0.59399999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.95499999999999996,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":1.0409999999999999}]},{"osd":1,"last update":"Wed May 8 22:06:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.76500000000000001,"15min":0.78000000000000003},"min":{"1min":0.45700000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.92700000000000005,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.754},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85699999999999998,"5min":0.997,"15min":0.997},"last":0.79900000000000004}]},{"osd":2,"last update":"Wed May 8 22:06:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.78100000000000003,"15min":0.79300000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98899999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.73899999999999999,"15min":0.746},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.97599999999999998,"5min":1.167,"15min":1.167},"last":0.84999999999999998}]},{"osd":3,"last update":"Wed May 8 22:06:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.78100000000000003,"15min":0.80500000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.99299999999999999,"5min":1.024,"15min":1.024},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.60199999999999998,"15min":0.60999999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.878,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.74199999999999999}]},{"osd":4,"last update":"Wed May 8 22:06:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.77900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0249999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.82599999999999996,"15min":0.84099999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.90400000000000003,"5min":1.081,"15min":1.081},"last":0.82299999999999995}]},{"osd":5,"last update":"Wed May 8 22:06:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.84299999999999997,"15min":0.86499999999999999},"min":{"1min":0.51800000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0629999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.878,"15min":0.89600000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.075,"5min":1.175,"15min":1.175},"last":0.90400000000000003}]},{"osd":6,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.71199999999999997}]},{"osd":7,"last update":"Wed May 8 22:06:01 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.77500000000000002,"15min":0.78800000000000003},"min":{"1min":0.55000000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.93500000000000005,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.91000000000000003,"15min":0.93200000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.095,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":1.0109999999999999}]},{"osd":8,"last update":"Wed May 8 22:06:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.81599999999999995,"15min":0.83299999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.006,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.92900000000000005,"15min":0.95299999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.157,"5min":1.478,"15min":1.478},"last":1.099}]},{"osd":10,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.88300000000000001,"15min":0.89500000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1879999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":1.2170000000000001},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.874,"15min":0.88400000000000001},"min":{"1min":0.65600000000000003,"5min":0.628,"15min":0.628},"max":{"1min":1.042,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.95099999999999996}]},{"osd":11,"last update":"Wed May 8 22:06:17 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.071},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.98199999999999998}]}]},{"osd":8,"up_from":55,"seq":236223201319,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":469876,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.36499999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.88200000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.622,"15min":0.63200000000000001},"min":{"1min":0.36799999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.79800000000000004,"5min":1.865,"15min":1.865},"last":0.48699999999999999}]},{"osd":1,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.60299999999999998,"15min":0.59399999999999997},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55700000000000005,"15min":0.55100000000000005},"min":{"1min":0.40799999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.52800000000000002}]},{"osd":2,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.42499999999999999,"15min":0.41899999999999998},"min":{"1min":0.32100000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63700000000000001,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.46800000000000003,"15min":0.46400000000000002},"min":{"1min":0.34200000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.377}]},{"osd":3,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.77600000000000002,"15min":0.81100000000000005},"min":{"1min":0.45500000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82199999999999995,"5min":4.8920000000000003,"15min":4.8920000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63100000000000001,"15min":0.625},"min":{"1min":0.53700000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.57099999999999995}]},{"osd":4,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.62,"15min":0.60799999999999998},"min":{"1min":0.44400000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.63400000000000001}]},{"osd":5,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.50900000000000001,"15min":0.505},"min":{"1min":0.38600000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67000000000000004,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.65400000000000003}]},{"osd":6,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.879,"15min":0.90700000000000003},"min":{"1min":0.53300000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95299999999999996,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85999999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.58899999999999997}]},{"osd":7,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.66400000000000003,"15min":0.66200000000000003},"min":{"1min":0.42999999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.67200000000000004,"15min":0.66500000000000004},"min":{"1min":0.48699999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.68600000000000005}]},{"osd":9,"last update":"Wed May 8 22:06:03 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.78000000000000003,"15min":0.77500000000000002},"min":{"1min":0.59599999999999997,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.73899999999999999,"15min":0.73299999999999998},"min":{"1min":0.52600000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94799999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.61899999999999999}]},{"osd":10,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.71799999999999997,"15min":0.70499999999999996},"min":{"1min":0.55000000000000004,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94699999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.76400000000000001,"15min":0.752},"min":{"1min":0.68400000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.78000000000000003}]},{"osd":11,"last update":"Wed May 8 22:06:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.75700000000000001}]}]},{"osd":7,"up_from":48,"seq":206158430248,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":1066022,"data_compressed":53579,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.41199999999999998}]},{"osd":1,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.38800000000000001},"min":{"1min":0.29699999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.57499999999999996,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40999999999999998,"15min":0.40400000000000003},"min":{"1min":0.27900000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.55400000000000005,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.53500000000000003}]},{"osd":2,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.41599999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71499999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.57599999999999996,"15min":0.56999999999999995},"min":{"1min":0.46200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80400000000000005,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.86799999999999999}]},{"osd":3,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.41299999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76800000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.748},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68400000000000005,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.60899999999999999}]},{"osd":4,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.49399999999999999,"15min":0.49299999999999999},"min":{"1min":0.35899999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.505,"15min":0.51400000000000001},"min":{"1min":0.34599999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69499999999999995,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.499}]},{"osd":5,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.46800000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77500000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63,"15min":0.628},"min":{"1min":0.495,"5min":0.436,"15min":0.436},"max":{"1min":0.83999999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.81799999999999995}]},{"osd":6,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65800000000000003,"15min":0.66800000000000004},"min":{"1min":0.504,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.80900000000000005,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.67700000000000005,"15min":0.69299999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84099999999999997,"5min":1.002,"15min":1.002},"last":0.59299999999999997}]},{"osd":8,"last update":"Wed May 8 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.68700000000000006,"15min":0.69399999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.78000000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.72099999999999997,"15min":0.72699999999999998},"min":{"1min":0.55300000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.85899999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.79500000000000004}]},{"osd":9,"last update":"Wed May 8 22:06:03 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.72699999999999998,"15min":0.73099999999999998},"min":{"1min":0.55400000000000005,"5min":0.501,"15min":0.501},"max":{"1min":0.89900000000000002,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.73199999999999998,"15min":0.74099999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88900000000000001,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.71799999999999997}]},{"osd":10,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.69999999999999996,"15min":0.71099999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.82099999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.68500000000000005},"min":{"1min":0.499,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.86799999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.64400000000000002}]},{"osd":11,"last update":"Wed May 8 22:06:14 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.80600000000000005}]}]},{"osd":6,"up_from":43,"seq":184683593773,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":465190,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46000000000000002,"15min":0.45000000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.55300000000000005,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.50700000000000001,"15min":0.51200000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.51200000000000001}]},{"osd":1,"last update":"Wed May 8 22:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.45700000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.86199999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.629,"15min":0.60899999999999999},"min":{"1min":0.47599999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92900000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.54100000000000004}]},{"osd":2,"last update":"Wed May 8 22:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67100000000000004,"15min":0.65100000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82499999999999996,"5min":1.089,"15min":1.089},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.65800000000000003,"15min":0.64900000000000002},"min":{"1min":0.42499999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92400000000000004,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.70999999999999996}]},{"osd":3,"last update":"Wed May 8 22:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.38600000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.873,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.47199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77000000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.46400000000000002}]},{"osd":4,"last update":"Wed May 8 22:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.438,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89900000000000002,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.63},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.71399999999999997,"15min":0.71999999999999997},"min":{"1min":0.54400000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.97799999999999998,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.65600000000000003}]},{"osd":5,"last update":"Wed May 8 22:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75600000000000001,"15min":0.74399999999999999},"min":{"1min":0.47899999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.86899999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.748,"15min":0.748},"min":{"1min":0.44400000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.91700000000000004,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.67800000000000005}]},{"osd":7,"last update":"Wed May 8 22:06:28 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.747,"15min":0.73999999999999999},"min":{"1min":0.54900000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.9690000000000001,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":0.755},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.77700000000000002,"15min":0.76500000000000001},"min":{"1min":0.63200000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.77600000000000002}]},{"osd":8,"last update":"Wed May 8 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.83199999999999996,"15min":0.83999999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0289999999999999,"5min":1.093,"15min":1.093},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.79300000000000004,"15min":0.79600000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0149999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.79600000000000004}]},{"osd":9,"last update":"Wed May 8 22:06:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.84899999999999998,"15min":0.85699999999999998},"min":{"1min":0.59899999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.81999999999999995,"15min":0.83599999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.98899999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.84299999999999997}]},{"osd":10,"last update":"Wed May 8 22:06:20 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.68500000000000005,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.877},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.83099999999999996,"15min":0.82099999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":0.84899999999999998}]},{"osd":11,"last update":"Wed May 8 22:06:20 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":0.81399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":991232,"data_stored":525133,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.52700000000000002,"15min":0.49299999999999999},"min":{"1min":0.38900000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.89900000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54000000000000004,"15min":0.51400000000000001},"min":{"1min":0.371,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81200000000000006,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.51200000000000001}]},{"osd":2,"last update":"Wed May 8 22:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.54800000000000004,"15min":0.497},"min":{"1min":0.40400000000000003,"5min":0.316,"15min":0.316},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.54100000000000004,"15min":0.496},"min":{"1min":0.42799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83599999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.67600000000000005}]},{"osd":3,"last update":"Wed May 8 22:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.55500000000000005,"15min":0.51200000000000001},"min":{"1min":0.37,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76600000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.51500000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72599999999999998,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.59899999999999998}]},{"osd":4,"last update":"Wed May 8 22:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.39900000000000002,"15min":0.376},"min":{"1min":0.307,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.64000000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.42499999999999999,"15min":0.39300000000000002},"min":{"1min":0.32400000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.76700000000000002,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.72499999999999998}]},{"osd":5,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.59099999999999997,"15min":0.55700000000000005},"min":{"1min":0.47199999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.95499999999999996,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.60199999999999998,"15min":0.57899999999999996},"min":{"1min":0.45400000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89600000000000002,"5min":1.089,"15min":1.089},"last":0.75800000000000001}]},{"osd":6,"last update":"Wed May 8 22:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.78100000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.64900000000000002,"15min":0.623},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.85299999999999998,"5min":1.05,"15min":1.05},"last":0.77000000000000002}]},{"osd":7,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58999999999999997,"15min":0.57199999999999995},"min":{"1min":0.44400000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.871,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.39100000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74199999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.70999999999999996}]},{"osd":8,"last update":"Wed May 8 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.65300000000000002,"15min":0.65900000000000003},"min":{"1min":0.45800000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.91200000000000003,"5min":1.115,"15min":1.115},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.53500000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96999999999999997,"5min":1.222,"15min":1.222},"last":0.85699999999999998}]},{"osd":9,"last update":"Wed May 8 22:06:04 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71699999999999997,"15min":0.72099999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.83599999999999997,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.747},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.79900000000000004,"15min":0.79500000000000004},"min":{"1min":0.63600000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.95099999999999996,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.878}]},{"osd":10,"last update":"Wed May 8 22:06:19 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76400000000000001,"15min":0.76500000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.94699999999999995,"5min":1.179,"15min":1.179},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.52400000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.93700000000000006,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.79500000000000004}]},{"osd":11,"last update":"Wed May 8 22:06:14 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.81699999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":946176,"data_stored":466578,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57899999999999996,"15min":0.54900000000000004},"min":{"1min":0.42499999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.94099999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.55400000000000005,"15min":0.52400000000000002},"min":{"1min":0.40100000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83199999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.41899999999999998}]},{"osd":2,"last update":"Wed May 8 22:05:47 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.60199999999999998,"15min":0.55300000000000005},"min":{"1min":0.50900000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.91700000000000004,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.59099999999999997,"15min":0.55100000000000005},"min":{"1min":0.502,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91200000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.53900000000000003}]},{"osd":3,"last update":"Wed May 8 22:06:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.624}]},{"osd":4,"last update":"Wed May 8 22:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.57099999999999995,"15min":0.54900000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69399999999999995,"15min":0.69599999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96999999999999997,"5min":2.984,"15min":2.984},"last":0.56799999999999995}]},{"osd":5,"last update":"Wed May 8 22:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.67800000000000005,"15min":0.64600000000000002},"min":{"1min":0.42399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0109999999999999,"5min":1.129,"15min":1.129},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67300000000000004,"15min":0.66300000000000003},"min":{"1min":0.49199999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.94699999999999995,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.498}]},{"osd":6,"last update":"Wed May 8 22:06:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.60599999999999998,"15min":0.58199999999999996},"min":{"1min":0.45000000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88,"5min":1.254,"15min":1.254},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.70099999999999996,"15min":0.67400000000000004},"min":{"1min":0.49399999999999999,"5min":0.375,"15min":0.375},"max":{"1min":1.1859999999999999,"5min":1.208,"15min":1.208},"last":0.67900000000000005}]},{"osd":7,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.69699999999999995}]},{"osd":8,"last update":"Wed May 8 22:05:39 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.82199999999999995,"15min":0.82599999999999996},"min":{"1min":0.61399999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.099,"5min":1.427,"15min":1.427},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.79500000000000004,"15min":0.80300000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0369999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.66900000000000004}]},{"osd":9,"last update":"Wed May 8 22:06:03 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.78500000000000003,"15min":0.79000000000000004},"min":{"1min":0.50800000000000001,"5min":0.502,"15min":0.502},"max":{"1min":1.155,"5min":1.284,"15min":1.284},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.89300000000000002,"15min":0.89800000000000002},"min":{"1min":0.59099999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.2170000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.97799999999999998}]},{"osd":10,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.61599999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1339999999999999,"5min":1.2,"15min":1.2},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.876,"15min":0.879},"min":{"1min":0.69599999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.746}]},{"osd":11,"last update":"Wed May 8 22:05:39 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.70499999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":995328,"data_stored":524883,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55000000000000004,"15min":0.52400000000000002},"min":{"1min":0.377,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.498},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52200000000000002,"15min":0.50800000000000001},"min":{"1min":0.39200000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.51800000000000002}]},{"osd":1,"last update":"Wed May 8 22:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.36599999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.57099999999999995,"15min":0.55300000000000005},"min":{"1min":0.40400000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.63100000000000001}]},{"osd":3,"last update":"Wed May 8 22:06:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.57599999999999996,"15min":0.56000000000000005},"min":{"1min":0.41999999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.58199999999999996,"15min":0.56399999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77600000000000002,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.71799999999999997}]},{"osd":4,"last update":"Wed May 8 22:06:15 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.877}]},{"osd":5,"last update":"Wed May 8 22:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.46800000000000003,"15min":0.44400000000000001},"min":{"1min":0.38100000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.46600000000000003,"15min":0.44800000000000001},"min":{"1min":0.34799999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.55200000000000005}]},{"osd":6,"last update":"Wed May 8 22:06:09 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.45700000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.84299999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.69399999999999995,"15min":0.67600000000000005},"min":{"1min":0.57699999999999996,"5min":0.442,"15min":0.442},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.69799999999999995}]},{"osd":7,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.76500000000000001}]},{"osd":8,"last update":"Wed May 8 22:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.36699999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.58799999999999997,"15min":0.57099999999999995},"min":{"1min":0.41299999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.79100000000000004}]},{"osd":9,"last update":"Wed May 8 22:06:03 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.73999999999999999,"15min":0.71599999999999997},"min":{"1min":0.629,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.72999999999999998,"15min":0.70899999999999996},"min":{"1min":0.54600000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.996}]},{"osd":10,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.73899999999999999,"15min":0.71299999999999997},"min":{"1min":0.65700000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.72199999999999998,"15min":0.70399999999999996},"min":{"1min":0.66800000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.83799999999999997}]},{"osd":11,"last update":"Wed May 8 22:06:15 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.89600000000000002}]}]},{"osd":3,"up_from":24,"seq":103079215160,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":465657,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:06:11 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.46899999999999997,"15min":0.40899999999999997},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":3.2530000000000001,"5min":3.2530000000000001,"15min":3.2530000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.50700000000000001,"15min":0.42899999999999999},"min":{"1min":0.36499999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":3.4460000000000002,"5min":3.4460000000000002,"15min":3.4460000000000002},"last":0.54200000000000004}]},{"osd":1,"last update":"Wed May 8 22:06:11 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.66500000000000004,"15min":0.57899999999999996},"min":{"1min":0.432,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.7090000000000001,"5min":3.7090000000000001,"15min":3.7090000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.626,"15min":0.55000000000000004},"min":{"1min":0.59299999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":3.3639999999999999,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":0.36199999999999999}]},{"osd":2,"last update":"Wed May 8 22:06:11 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.68899999999999995,"15min":0.61299999999999999},"min":{"1min":0.52300000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":3.577,"5min":3.577,"15min":3.577},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.70199999999999996,"15min":0.63},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.7770000000000001,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.65900000000000003}]},{"osd":4,"last update":"Wed May 8 22:06:25 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.67700000000000005,"15min":0.61299999999999999},"min":{"1min":0.46600000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":3.8109999999999999,"5min":3.8109999999999999,"15min":3.8109999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.72199999999999998,"15min":0.64000000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.847,"5min":3.847,"15min":3.847},"last":0.59299999999999997}]},{"osd":5,"last update":"Wed May 8 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.67800000000000005,"15min":0.64500000000000002},"min":{"1min":0.46500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.748},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.64400000000000002,"15min":0.60899999999999999},"min":{"1min":0.46500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.67700000000000005}]},{"osd":6,"last update":"Wed May 8 22:06:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.68200000000000005,"15min":0.63200000000000001},"min":{"1min":0.42299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.653,"5min":3.653,"15min":3.653},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.70099999999999996,"15min":0.64200000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":3.9649999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.68799999999999994}]},{"osd":7,"last update":"Wed May 8 22:06:25 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.73999999999999999,"15min":0.72299999999999998},"min":{"1min":0.55500000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.86399999999999999,"15min":0.84699999999999998},"min":{"1min":0.57599999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":3.887,"5min":3.887,"15min":3.887},"last":0.627}]},{"osd":8,"last update":"Wed May 8 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.53100000000000003}]},{"osd":9,"last update":"Wed May 8 22:06:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":0.90600000000000003,"15min":0.877},"min":{"1min":0.54900000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":4.0289999999999999,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.81699999999999995,"15min":0.80000000000000004},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.3940000000000001,"5min":3.3940000000000001,"15min":3.3940000000000001},"last":0.92100000000000004}]},{"osd":10,"last update":"Wed May 8 22:06:20 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.85599999999999998,"15min":0.83999999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":3.298,"5min":3.298,"15min":3.298},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.04,"5min":0.89200000000000002,"15min":0.86699999999999999},"min":{"1min":0.60899999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":3.5049999999999999,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":0.70299999999999996}]},{"osd":11,"last update":"Wed May 8 22:06:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":3.6150000000000002,"5min":3.6150000000000002,"15min":3.6150000000000002},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":3.9950000000000001,"5min":3.9950000000000001,"15min":3.9950000000000001},"last":0.71499999999999997}]}]},{"osd":4,"up_from":31,"seq":133143986230,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":470177,"data_compressed":17583,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.44700000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.51900000000000002,"15min":0.50900000000000001},"min":{"1min":0.45000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.629}]},{"osd":1,"last update":"Wed May 8 22:06:28 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.39100000000000001,"15min":0.35699999999999998},"min":{"1min":0.33500000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.40000000000000002,"15min":0.36199999999999999},"min":{"1min":0.36299999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.51900000000000002}]},{"osd":2,"last update":"Wed May 8 22:06:28 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.53600000000000003,"15min":0.51100000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82099999999999995,"5min":0.88,"15min":0.88},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54800000000000004,"15min":0.51400000000000001},"min":{"1min":0.436,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.64300000000000002}]},{"osd":3,"last update":"Wed May 8 22:06:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.56699999999999995},"min":{"1min":0.46000000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.70499999999999996,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57299999999999995,"15min":0.55500000000000005},"min":{"1min":0.439,"5min":0.377,"15min":0.377},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.59699999999999998}]},{"osd":5,"last update":"Wed May 8 22:05:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.36399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76200000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.57199999999999995,"15min":0.55000000000000004},"min":{"1min":0.42799999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.79700000000000004}]},{"osd":6,"last update":"Wed May 8 22:06:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65700000000000003,"15min":0.63900000000000001},"min":{"1min":0.53300000000000003,"5min":0.316,"15min":0.316},"max":{"1min":0.81499999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.621,"15min":0.61399999999999999},"min":{"1min":0.40400000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.71399999999999997}]},{"osd":7,"last update":"Wed May 8 22:06:28 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57999999999999996,"15min":0.58599999999999997},"min":{"1min":0.45900000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.46200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.83599999999999997}]},{"osd":8,"last update":"Wed May 8 22:05:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68400000000000005,"15min":0.68600000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70799999999999996,"15min":0.70899999999999996},"min":{"1min":0.54100000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.874}]},{"osd":9,"last update":"Wed May 8 22:06:03 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.56200000000000006,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.80400000000000005,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.755,"15min":0.75700000000000001},"min":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.85699999999999998,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.85899999999999999}]},{"osd":10,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.73599999999999999,"15min":0.73999999999999999},"min":{"1min":0.55800000000000005,"5min":0.434,"15min":0.434},"max":{"1min":0.84799999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.627,"15min":0.625},"min":{"1min":0.50600000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.78100000000000003}]},{"osd":11,"last update":"Wed May 8 22:06:14 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.88400000000000001}]}]},{"osd":5,"up_from":36,"seq":154618822706,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":1003520,"data_stored":1061658,"data_compressed":53579,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:05:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.41199999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.71899999999999997,"5min":0.749,"15min":0.749},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.35199999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80100000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.40000000000000002}]},{"osd":1,"last update":"Wed May 8 22:05:43 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.32300000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52600000000000002,"15min":0.51600000000000001},"min":{"1min":0.34300000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71299999999999997,"5min":0.754,"15min":0.754},"last":0.34399999999999997}]},{"osd":2,"last update":"Wed May 8 22:05:43 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.377,"15min":0.36699999999999999},"min":{"1min":0.29799999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.39200000000000002,"15min":0.38400000000000001},"min":{"1min":0.309,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.504}]},{"osd":3,"last update":"Wed May 8 22:05:43 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.57299999999999995},"min":{"1min":0.42099999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.76700000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56299999999999994,"15min":0.55100000000000005},"min":{"1min":0.47299999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.78700000000000003,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.54300000000000004}]},{"osd":4,"last update":"Wed May 8 22:05:43 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54700000000000004,"15min":0.53700000000000003},"min":{"1min":0.42999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82999999999999996,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.33600000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.82699999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.56999999999999995}]},{"osd":6,"last update":"Wed May 8 22:06:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.59599999999999997,"15min":0.58799999999999997},"min":{"1min":0.41199999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81599999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.58799999999999997,"15min":0.56200000000000006},"min":{"1min":0.51700000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.63500000000000001}]},{"osd":7,"last update":"Wed May 8 22:06:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.60899999999999999,"15min":0.57699999999999996},"min":{"1min":0.54500000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.61799999999999999,"15min":0.58699999999999997},"min":{"1min":0.51100000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66200000000000003}]},{"osd":8,"last update":"Wed May 8 22:05:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.50600000000000001,"15min":0.49399999999999999},"min":{"1min":0.372,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.502,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.67100000000000004}]},{"osd":9,"last update":"Wed May 8 22:05:59 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.70999999999999996,"15min":0.70099999999999996},"min":{"1min":0.57499999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.69799999999999995,"15min":0.68999999999999995},"min":{"1min":0.45200000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.68999999999999995}]},{"osd":10,"last update":"Wed May 8 22:06:25 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.752,"15min":0.74299999999999999},"min":{"1min":0.65900000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.74199999999999999,"15min":0.73299999999999998},"min":{"1min":0.66100000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.98599999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.71499999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":60033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:06:33.310 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-08T22:06:33.310 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-08T22:06:33.310 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-08T22:06:33.310 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-08T22:06:33.676 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:34.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:33 smithi131 ceph-mon[37260]: pgmap v281: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T22:06:34.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:33 smithi131 ceph-mon[37260]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:06:34.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:33 smithi157 ceph-mon[77746]: pgmap v281: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T22:06:34.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:33 smithi157 ceph-mon[77746]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:06:34.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:33 smithi132 ceph-mon[78180]: pgmap v281: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T22:06:34.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:33 smithi132 ceph-mon[78180]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:06:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.914+0000 7fdb1ec00700 1 -- 172.21.15.131:0/2663286858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb18107500 msgr2=0x7fdb1810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.914+0000 7fdb1ec00700 1 --2- 172.21.15.131:0/2663286858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb18107500 0x7fdb1810de80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdb10009990 tx=0x7fdb10009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.914+0000 7fdb1ec00700 1 -- 172.21.15.131:0/2663286858 shutdown_connections 2024-05-08T22:06:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.914+0000 7fdb1ec00700 1 --2- 172.21.15.131:0/2663286858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb18107500 0x7fdb1810de80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.914+0000 7fdb1ec00700 1 --2- 172.21.15.131:0/2663286858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb18101f70 0x7fdb18102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.914+0000 7fdb1ec00700 1 --2- 172.21.15.131:0/2663286858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb18102920 0x7fdb18106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.914+0000 7fdb1ec00700 1 -- 172.21.15.131:0/2663286858 >> 172.21.15.131:0/2663286858 conn(0x7fdb180fb7e0 msgr2=0x7fdb180fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.914+0000 7fdb1ec00700 1 -- 172.21.15.131:0/2663286858 shutdown_connections 2024-05-08T22:06:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.914+0000 7fdb1ec00700 1 -- 172.21.15.131:0/2663286858 wait complete. 2024-05-08T22:06:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.915+0000 7fdb1ec00700 1 Processor -- start 2024-05-08T22:06:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.916+0000 7fdb1ec00700 1 -- start start 2024-05-08T22:06:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.916+0000 7fdb1ec00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb18101f70 0x7fdb18199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.916+0000 7fdb1ec00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb18102920 0x7fdb18199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.916+0000 7fdb1ec00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb18107500 0x7fdb1819de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.916+0000 7fdb1ec00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1819e410 con 0x7fdb18107500 2024-05-08T22:06:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.916+0000 7fdb1ec00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1819e580 con 0x7fdb18101f70 2024-05-08T22:06:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.916+0000 7fdb1ec00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1819e740 con 0x7fdb18102920 2024-05-08T22:06:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.916+0000 7fdb1d800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb18101f70 0x7fdb18199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.916+0000 7fdb1e200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb18107500 0x7fdb1819de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.916+0000 7fdb1d800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb18101f70 0x7fdb18199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:53292/0 (socket says 172.21.15.131:53292) 2024-05-08T22:06:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.916+0000 7fdb1d800700 1 -- 172.21.15.131:0/2416411228 learned_addr learned my addr 172.21.15.131:0/2416411228 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.917+0000 7fdb1ce00700 1 --2- 172.21.15.131:0/2416411228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb18102920 0x7fdb18199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.917+0000 7fdb1d800700 1 -- 172.21.15.131:0/2416411228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb18102920 msgr2=0x7fdb18199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.917+0000 7fdb1d800700 1 --2- 172.21.15.131:0/2416411228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb18102920 0x7fdb18199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.917+0000 7fdb1d800700 1 -- 172.21.15.131:0/2416411228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb18107500 msgr2=0x7fdb1819de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.917+0000 7fdb1d800700 1 --2- 172.21.15.131:0/2416411228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb18107500 0x7fdb1819de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.917+0000 7fdb1d800700 1 -- 172.21.15.131:0/2416411228 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb100095f0 con 0x7fdb18101f70 2024-05-08T22:06:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.917+0000 7fdb1d800700 1 --2- 172.21.15.131:0/2416411228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb18101f70 0x7fdb18199510 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00efc0 tx=0x7fdb0c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.917+0000 7fdb16000700 1 -- 172.21.15.131:0/2416411228 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c018070 con 0x7fdb18101f70 2024-05-08T22:06:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.918+0000 7fdb1ec00700 1 -- 172.21.15.131:0/2416411228 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb1819e9c0 con 0x7fdb18101f70 2024-05-08T22:06:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.918+0000 7fdb16000700 1 -- 172.21.15.131:0/2416411228 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb0c00f040 con 0x7fdb18101f70 2024-05-08T22:06:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.918+0000 7fdb16000700 1 -- 172.21.15.131:0/2416411228 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c004280 con 0x7fdb18101f70 2024-05-08T22:06:34.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.918+0000 7fdb1ec00700 1 -- 172.21.15.131:0/2416411228 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb181a6280 con 0x7fdb18101f70 2024-05-08T22:06:34.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.919+0000 7fdb1ec00700 1 -- 172.21.15.131:0/2416411228 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb1810b590 con 0x7fdb18101f70 2024-05-08T22:06:34.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.921+0000 7fdb16000700 1 -- 172.21.15.131:0/2416411228 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fdb0c003a00 con 0x7fdb18101f70 2024-05-08T22:06:34.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.922+0000 7fdb16000700 1 --2- 172.21.15.131:0/2416411228 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdb000a11b0 0x7fdb000a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:34.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.922+0000 7fdb16000700 1 -- 172.21.15.131:0/2416411228 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdb0c0cd5e0 con 0x7fdb18101f70 2024-05-08T22:06:34.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.922+0000 7fdb1ce00700 1 --2- 172.21.15.131:0/2416411228 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdb000a11b0 0x7fdb000a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:34.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.923+0000 7fdb1ce00700 1 --2- 172.21.15.131:0/2416411228 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdb000a11b0 0x7fdb000a3670 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdb1819aa70 tx=0x7fdb04009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:34.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:34.925+0000 7fdb16000700 1 -- 172.21.15.131:0/2416411228 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdb0c096990 con 0x7fdb18101f70 2024-05-08T22:06:35.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.175+0000 7fdb1ec00700 1 -- 172.21.15.131:0/2416411228 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fdb18069230 con 0x7fdb18101f70 2024-05-08T22:06:35.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.176+0000 7fdb16000700 1 -- 172.21.15.131:0/2416411228 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fdb0c099fb0 con 0x7fdb18101f70 2024-05-08T22:06:35.177 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:06:35.177 INFO:teuthology.orchestra.run.smithi131.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T22:06:35.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.179+0000 7fdb1ec00700 1 -- 172.21.15.131:0/2416411228 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdb000a11b0 msgr2=0x7fdb000a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:35.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.179+0000 7fdb1ec00700 1 --2- 172.21.15.131:0/2416411228 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdb000a11b0 0x7fdb000a3670 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdb1819aa70 tx=0x7fdb04009260 comp rx=0 tx=0).stop 2024-05-08T22:06:35.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.179+0000 7fdb1ec00700 1 -- 172.21.15.131:0/2416411228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb18101f70 msgr2=0x7fdb18199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:35.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.179+0000 7fdb1ec00700 1 --2- 172.21.15.131:0/2416411228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb18101f70 0x7fdb18199510 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00efc0 tx=0x7fdb0c00c3a0 comp rx=0 tx=0).stop 2024-05-08T22:06:35.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.179+0000 7fdb1ec00700 1 -- 172.21.15.131:0/2416411228 shutdown_connections 2024-05-08T22:06:35.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.179+0000 7fdb1ec00700 1 --2- 172.21.15.131:0/2416411228 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fdb000a11b0 0x7fdb000a3670 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:35.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.179+0000 7fdb1ec00700 1 --2- 172.21.15.131:0/2416411228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb18107500 0x7fdb1819de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:35.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.179+0000 7fdb1ec00700 1 --2- 172.21.15.131:0/2416411228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb18101f70 0x7fdb18199510 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:35.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.179+0000 7fdb1ec00700 1 --2- 172.21.15.131:0/2416411228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb18102920 0x7fdb18199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:35.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.179+0000 7fdb1ec00700 1 -- 172.21.15.131:0/2416411228 >> 172.21.15.131:0/2416411228 conn(0x7fdb180fb7e0 msgr2=0x7fdb180fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:35.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.179+0000 7fdb1ec00700 1 -- 172.21.15.131:0/2416411228 shutdown_connections 2024-05-08T22:06:35.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.179+0000 7fdb1ec00700 1 -- 172.21.15.131:0/2416411228 wait complete. 2024-05-08T22:06:35.625 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-08T22:06:35.626 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:06:35.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.822+0000 7f5580c00700 1 -- 172.21.15.131:0/1917414632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f557c101fc0 msgr2=0x7f557c102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:35.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.822+0000 7f5580c00700 1 --2- 172.21.15.131:0/1917414632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f557c101fc0 0x7f557c102440 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5568009990 tx=0x7f5568009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:35.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.822+0000 7f5580c00700 1 -- 172.21.15.131:0/1917414632 shutdown_connections 2024-05-08T22:06:35.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.822+0000 7f5580c00700 1 --2- 172.21.15.131:0/1917414632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f557c102980 0x7f557c1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:35.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.822+0000 7f5580c00700 1 --2- 172.21.15.131:0/1917414632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c100e70 0x7f557c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:35.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.822+0000 7f5580c00700 1 --2- 172.21.15.131:0/1917414632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f557c101fc0 0x7f557c102440 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:35.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.822+0000 7f5580c00700 1 -- 172.21.15.131:0/1917414632 >> 172.21.15.131:0/1917414632 conn(0x7f557c0fc3e0 msgr2=0x7f557c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:35.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.822+0000 7f5580c00700 1 -- 172.21.15.131:0/1917414632 shutdown_connections 2024-05-08T22:06:35.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.822+0000 7f5580c00700 1 -- 172.21.15.131:0/1917414632 wait complete. 2024-05-08T22:06:35.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.823+0000 7f5580c00700 1 Processor -- start 2024-05-08T22:06:35.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.824+0000 7f5580c00700 1 -- start start 2024-05-08T22:06:35.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.824+0000 7f5580c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c100e70 0x7f557c199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:35.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.824+0000 7f5580c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f557c101fc0 0x7f557c199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:35.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.824+0000 7f5580c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f557c102980 0x7f557c1a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:35.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.824+0000 7f5580c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557c1a0b60 con 0x7f557c102980 2024-05-08T22:06:35.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.824+0000 7f5580c00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557c1a0ca0 con 0x7f557c101fc0 2024-05-08T22:06:35.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.824+0000 7f5580c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557c1a0e30 con 0x7f557c100e70 2024-05-08T22:06:35.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.824+0000 7f557a800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f557c102980 0x7f557c1a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:35.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.824+0000 7f5579400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f557c101fc0 0x7f557c199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:35.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.824+0000 7f5579e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c100e70 0x7f557c199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:35.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.824+0000 7f557a800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f557c102980 0x7f557c1a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40232/0 (socket says 172.21.15.131:40232) 2024-05-08T22:06:35.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.824+0000 7f557a800700 1 -- 172.21.15.131:0/2754406387 learned_addr learned my addr 172.21.15.131:0/2754406387 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:35.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.825+0000 7f557a800700 1 -- 172.21.15.131:0/2754406387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c100e70 msgr2=0x7f557c199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:35.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.825+0000 7f557a800700 1 --2- 172.21.15.131:0/2754406387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c100e70 0x7f557c199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:35.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.825+0000 7f557a800700 1 -- 172.21.15.131:0/2754406387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f557c101fc0 msgr2=0x7f557c199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:35.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.825+0000 7f557a800700 1 --2- 172.21.15.131:0/2754406387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f557c101fc0 0x7f557c199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:35.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.825+0000 7f557a800700 1 -- 172.21.15.131:0/2754406387 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55680095f0 con 0x7f557c102980 2024-05-08T22:06:35.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.825+0000 7f557a800700 1 --2- 172.21.15.131:0/2754406387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f557c102980 0x7f557c1a0590 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f557400ef30 tx=0x7f557400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:35.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.825+0000 7f5572a00700 1 -- 172.21.15.131:0/2754406387 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5574018070 con 0x7f557c102980 2024-05-08T22:06:35.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.825+0000 7f5572a00700 1 -- 172.21.15.131:0/2754406387 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f557400f040 con 0x7f557c102980 2024-05-08T22:06:35.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.825+0000 7f5572a00700 1 -- 172.21.15.131:0/2754406387 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557400cce0 con 0x7f557c102980 2024-05-08T22:06:35.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.825+0000 7f5580c00700 1 -- 172.21.15.131:0/2754406387 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f557c1a10b0 con 0x7f557c102980 2024-05-08T22:06:35.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.825+0000 7f5580c00700 1 -- 172.21.15.131:0/2754406387 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f557c1a1570 con 0x7f557c102980 2024-05-08T22:06:35.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.827+0000 7f5580c00700 1 -- 172.21.15.131:0/2754406387 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f557c067c10 con 0x7f557c102980 2024-05-08T22:06:35.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.829+0000 7f5572a00700 1 -- 172.21.15.131:0/2754406387 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f5574004ad0 con 0x7f557c102980 2024-05-08T22:06:35.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.831+0000 7f5572a00700 1 --2- 172.21.15.131:0/2754406387 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f55600a11b0 0x7f55600a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:35.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.831+0000 7f5572a00700 1 -- 172.21.15.131:0/2754406387 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f55740cd1c0 con 0x7f557c102980 2024-05-08T22:06:35.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.831+0000 7f5579e00700 1 --2- 172.21.15.131:0/2754406387 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f55600a11b0 0x7f55600a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:35.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.832+0000 7f5579e00700 1 --2- 172.21.15.131:0/2754406387 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f55600a11b0 0x7f55600a3670 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f556c007c60 tx=0x7f556c0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:35.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:35.834+0000 7f5572a00700 1 -- 172.21.15.131:0/2754406387 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5574099fa0 con 0x7f557c102980 2024-05-08T22:06:36.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.047+0000 7f5580c00700 1 -- 172.21.15.131:0/2754406387 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f557c10c6c0 con 0x7f557c102980 2024-05-08T22:06:36.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.049+0000 7f5572a00700 1 -- 172.21.15.131:0/2754406387 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+24534 (secure 0 0 0) 0x7f5574099b30 con 0x7f557c102980 2024-05-08T22:06:36.050 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:06:36.050 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":85,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:05:41.813233+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:05:12.469858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:05:13.484446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:05:17.016186+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:06:36.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.052+0000 7f5580c00700 1 -- 172.21.15.131:0/2754406387 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f55600a11b0 msgr2=0x7f55600a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:36.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.052+0000 7f5580c00700 1 --2- 172.21.15.131:0/2754406387 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f55600a11b0 0x7f55600a3670 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f556c007c60 tx=0x7f556c0073d0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.052+0000 7f5580c00700 1 -- 172.21.15.131:0/2754406387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f557c102980 msgr2=0x7f557c1a0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:36.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.052+0000 7f5580c00700 1 --2- 172.21.15.131:0/2754406387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f557c102980 0x7f557c1a0590 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f557400ef30 tx=0x7f557400c3a0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.052+0000 7f5580c00700 1 -- 172.21.15.131:0/2754406387 shutdown_connections 2024-05-08T22:06:36.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.052+0000 7f5580c00700 1 --2- 172.21.15.131:0/2754406387 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f55600a11b0 0x7f55600a3670 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.052+0000 7f5580c00700 1 --2- 172.21.15.131:0/2754406387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f557c102980 0x7f557c1a0590 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.052+0000 7f5580c00700 1 --2- 172.21.15.131:0/2754406387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c100e70 0x7f557c199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.052+0000 7f5580c00700 1 --2- 172.21.15.131:0/2754406387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f557c101fc0 0x7f557c199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.052+0000 7f5580c00700 1 -- 172.21.15.131:0/2754406387 >> 172.21.15.131:0/2754406387 conn(0x7f557c0fc3e0 msgr2=0x7f557c0731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:36.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.052+0000 7f5580c00700 1 -- 172.21.15.131:0/2754406387 shutdown_connections 2024-05-08T22:06:36.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.052+0000 7f5580c00700 1 -- 172.21.15.131:0/2754406387 wait complete. 2024-05-08T22:06:36.073 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:01:41.271078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:05:12.469858+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:05:13.484446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:05:17.016186+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:06:36.074 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:06:36.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.270+0000 7f195ce00700 1 -- 172.21.15.131:0/400849257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1958100f80 msgr2=0x7f19581013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:36.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.270+0000 7f195ce00700 1 --2- 172.21.15.131:0/400849257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1958100f80 0x7f19581013a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f19480099c0 tx=0x7f1948009cd0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.270+0000 7f195ce00700 1 -- 172.21.15.131:0/400849257 shutdown_connections 2024-05-08T22:06:36.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.270+0000 7f195ce00700 1 --2- 172.21.15.131:0/400849257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1958102b10 0x7f19581093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.270+0000 7f195ce00700 1 --2- 172.21.15.131:0/400849257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1958100f80 0x7f19581013a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.270+0000 7f195ce00700 1 --2- 172.21.15.131:0/400849257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1958102170 0x7f19581025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.270+0000 7f195ce00700 1 -- 172.21.15.131:0/400849257 >> 172.21.15.131:0/400849257 conn(0x7f19580fc4f0 msgr2=0x7f19580fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:36.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.270+0000 7f195ce00700 1 -- 172.21.15.131:0/400849257 shutdown_connections 2024-05-08T22:06:36.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.271+0000 7f195ce00700 1 -- 172.21.15.131:0/400849257 wait complete. 2024-05-08T22:06:36.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.271+0000 7f195ce00700 1 Processor -- start 2024-05-08T22:06:36.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.272+0000 7f195ce00700 1 -- start start 2024-05-08T22:06:36.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.272+0000 7f195ce00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1958100f80 0x7f1958074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:36.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.272+0000 7f195ce00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1958102170 0x7f1958073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:36.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.272+0000 7f195ce00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1958102b10 0x7f19580738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:36.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.272+0000 7f195ce00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1958074070 con 0x7f1958100f80 2024-05-08T22:06:36.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.272+0000 7f195ce00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19580741e0 con 0x7f1958102b10 2024-05-08T22:06:36.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.272+0000 7f195ce00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19581a0fe0 con 0x7f1958102170 2024-05-08T22:06:36.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.272+0000 7f1955e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1958100f80 0x7f1958074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:36.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.272+0000 7f1955e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1958100f80 0x7f1958074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40244/0 (socket says 172.21.15.131:40244) 2024-05-08T22:06:36.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.272+0000 7f1955e00700 1 -- 172.21.15.131:0/1450552484 learned_addr learned my addr 172.21.15.131:0/1450552484 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:36.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.272+0000 7f1955400700 1 --2- 172.21.15.131:0/1450552484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1958102170 0x7f1958073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:36.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.272+0000 7f1956800700 1 --2- 172.21.15.131:0/1450552484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1958102b10 0x7f19580738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:36.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.273+0000 7f1955e00700 1 -- 172.21.15.131:0/1450552484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1958102170 msgr2=0x7f1958073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:36.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.273+0000 7f1955e00700 1 --2- 172.21.15.131:0/1450552484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1958102170 0x7f1958073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.273+0000 7f1955e00700 1 -- 172.21.15.131:0/1450552484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1958102b10 msgr2=0x7f19580738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:36.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.273+0000 7f1955e00700 1 --2- 172.21.15.131:0/1450552484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1958102b10 0x7f19580738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.273+0000 7f1955e00700 1 -- 172.21.15.131:0/1450552484 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19480095f0 con 0x7f1958100f80 2024-05-08T22:06:36.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.273+0000 7f1955e00700 1 --2- 172.21.15.131:0/1450552484 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1958100f80 0x7f1958074d00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1948003730 tx=0x7f194800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:36.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.273+0000 7f194ea00700 1 -- 172.21.15.131:0/1450552484 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194801d070 con 0x7f1958100f80 2024-05-08T22:06:36.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.273+0000 7f195ce00700 1 -- 172.21.15.131:0/1450552484 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19581a1210 con 0x7f1958100f80 2024-05-08T22:06:36.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.273+0000 7f194ea00700 1 -- 172.21.15.131:0/1450552484 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1948005050 con 0x7f1958100f80 2024-05-08T22:06:36.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.273+0000 7f194ea00700 1 -- 172.21.15.131:0/1450552484 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1948004a50 con 0x7f1958100f80 2024-05-08T22:06:36.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.273+0000 7f195ce00700 1 -- 172.21.15.131:0/1450552484 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19581a15d0 con 0x7f1958100f80 2024-05-08T22:06:36.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:35 smithi157 ceph-mon[77746]: pgmap v282: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:06:36.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:35 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2416411228' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T22:06:36.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.274+0000 7f195ce00700 1 -- 172.21.15.131:0/1450552484 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1958108450 con 0x7f1958100f80 2024-05-08T22:06:36.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.276+0000 7f194ea00700 1 -- 172.21.15.131:0/1450552484 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f19480051c0 con 0x7f1958100f80 2024-05-08T22:06:36.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.277+0000 7f194ea00700 1 --2- 172.21.15.131:0/1450552484 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f193c0a11b0 0x7f193c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:36.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.277+0000 7f194ea00700 1 -- 172.21.15.131:0/1450552484 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f19480cdf50 con 0x7f1958100f80 2024-05-08T22:06:36.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.278+0000 7f1955400700 1 --2- 172.21.15.131:0/1450552484 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f193c0a11b0 0x7f193c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:36.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.278+0000 7f1955400700 1 --2- 172.21.15.131:0/1450552484 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f193c0a11b0 0x7f193c0a3670 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1944009c60 tx=0x7f1944009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:36.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.280+0000 7f194ea00700 1 -- 172.21.15.131:0/1450552484 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f19480972a0 con 0x7f1958100f80 2024-05-08T22:06:36.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:35 smithi132 ceph-mon[78180]: pgmap v282: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:06:36.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:35 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2416411228' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T22:06:36.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:35 smithi131 ceph-mon[37260]: pgmap v282: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:06:36.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:35 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2416411228' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T22:06:36.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.486+0000 7f195ce00700 1 -- 172.21.15.131:0/1450552484 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1958107e30 con 0x7f1958100f80 2024-05-08T22:06:36.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.487+0000 7f194ea00700 1 -- 172.21.15.131:0/1450552484 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f194809a8c0 con 0x7f1958100f80 2024-05-08T22:06:36.488 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-08T22:06:36.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.489+0000 7f195ce00700 1 -- 172.21.15.131:0/1450552484 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f193c0a11b0 msgr2=0x7f193c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:36.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.489+0000 7f195ce00700 1 --2- 172.21.15.131:0/1450552484 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f193c0a11b0 0x7f193c0a3670 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1944009c60 tx=0x7f1944009260 comp rx=0 tx=0).stop 2024-05-08T22:06:36.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.489+0000 7f195ce00700 1 -- 172.21.15.131:0/1450552484 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1958100f80 msgr2=0x7f1958074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:36.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.489+0000 7f195ce00700 1 --2- 172.21.15.131:0/1450552484 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1958100f80 0x7f1958074d00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1948003730 tx=0x7f194800f690 comp rx=0 tx=0).stop 2024-05-08T22:06:36.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.489+0000 7f195ce00700 1 -- 172.21.15.131:0/1450552484 shutdown_connections 2024-05-08T22:06:36.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.489+0000 7f195ce00700 1 --2- 172.21.15.131:0/1450552484 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f193c0a11b0 0x7f193c0a3670 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.489+0000 7f195ce00700 1 --2- 172.21.15.131:0/1450552484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1958102b10 0x7f19580738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.489+0000 7f195ce00700 1 --2- 172.21.15.131:0/1450552484 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1958100f80 0x7f1958074d00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.489+0000 7f195ce00700 1 --2- 172.21.15.131:0/1450552484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1958102170 0x7f1958073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.489+0000 7f195ce00700 1 -- 172.21.15.131:0/1450552484 >> 172.21.15.131:0/1450552484 conn(0x7f19580fc4f0 msgr2=0x7f1958105590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:36.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.489+0000 7f195ce00700 1 -- 172.21.15.131:0/1450552484 shutdown_connections 2024-05-08T22:06:36.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.489+0000 7f195ce00700 1 -- 172.21.15.131:0/1450552484 wait complete. 2024-05-08T22:06:36.509 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:06:36.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.704+0000 7fb1aca00700 1 -- 172.21.15.131:0/3569912490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1a8074fe0 msgr2=0x7fb1a8073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:36.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.704+0000 7fb1aca00700 1 --2- 172.21.15.131:0/3569912490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1a8074fe0 0x7fb1a8073430 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb198009990 tx=0x7fb198009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.705+0000 7fb1aca00700 1 -- 172.21.15.131:0/3569912490 shutdown_connections 2024-05-08T22:06:36.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.705+0000 7fb1aca00700 1 --2- 172.21.15.131:0/3569912490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1a8108e30 0x7fb1a810b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.705+0000 7fb1aca00700 1 --2- 172.21.15.131:0/3569912490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1a8074fe0 0x7fb1a8073430 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.705+0000 7fb1aca00700 1 --2- 172.21.15.131:0/3569912490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1a8073a00 0x7fb1a8073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.705+0000 7fb1aca00700 1 -- 172.21.15.131:0/3569912490 >> 172.21.15.131:0/3569912490 conn(0x7fb1a80fc500 msgr2=0x7fb1a80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:36.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.705+0000 7fb1aca00700 1 -- 172.21.15.131:0/3569912490 shutdown_connections 2024-05-08T22:06:36.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.705+0000 7fb1aca00700 1 -- 172.21.15.131:0/3569912490 wait complete. 2024-05-08T22:06:36.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.706+0000 7fb1aca00700 1 Processor -- start 2024-05-08T22:06:36.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.706+0000 7fb1aca00700 1 -- start start 2024-05-08T22:06:36.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.706+0000 7fb1aca00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1a8073a00 0x7fb1a8199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:36.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.706+0000 7fb1aca00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1a8074fe0 0x7fb1a81996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:36.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.706+0000 7fb1aca00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1a8108e30 0x7fb1a81a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:36.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.706+0000 7fb1aca00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a81a0c60 con 0x7fb1a8074fe0 2024-05-08T22:06:36.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.706+0000 7fb1aca00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a81a0da0 con 0x7fb1a8073a00 2024-05-08T22:06:36.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.706+0000 7fb1aca00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a81a0f30 con 0x7fb1a8108e30 2024-05-08T22:06:36.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.706+0000 7fb1a5400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1a8074fe0 0x7fb1a81996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:36.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.706+0000 7fb1a5400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1a8074fe0 0x7fb1a81996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40250/0 (socket says 172.21.15.131:40250) 2024-05-08T22:06:36.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.706+0000 7fb1a5400700 1 -- 172.21.15.131:0/1890738026 learned_addr learned my addr 172.21.15.131:0/1890738026 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:36.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.707+0000 7fb1a5e00700 1 --2- 172.21.15.131:0/1890738026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1a8073a00 0x7fb1a8199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:36.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.707+0000 7fb1a6800700 1 --2- 172.21.15.131:0/1890738026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1a8108e30 0x7fb1a81a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:36.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.707+0000 7fb1a5400700 1 -- 172.21.15.131:0/1890738026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1a8108e30 msgr2=0x7fb1a81a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:36.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.707+0000 7fb1a5400700 1 --2- 172.21.15.131:0/1890738026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1a8108e30 0x7fb1a81a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.707+0000 7fb1a5400700 1 -- 172.21.15.131:0/1890738026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1a8073a00 msgr2=0x7fb1a8199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:36.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.707+0000 7fb1a5400700 1 --2- 172.21.15.131:0/1890738026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1a8073a00 0x7fb1a8199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.707+0000 7fb1a5400700 1 -- 172.21.15.131:0/1890738026 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1980095f0 con 0x7fb1a8074fe0 2024-05-08T22:06:36.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.707+0000 7fb1a5e00700 1 --2- 172.21.15.131:0/1890738026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1a8073a00 0x7fb1a8199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:36.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.707+0000 7fb1a5400700 1 --2- 172.21.15.131:0/1890738026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1a8074fe0 0x7fb1a81996c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb19400d980 tx=0x7fb19400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:36.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.707+0000 7fb19ea00700 1 -- 172.21.15.131:0/1890738026 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19400f890 con 0x7fb1a8074fe0 2024-05-08T22:06:36.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.707+0000 7fb1aca00700 1 -- 172.21.15.131:0/1890738026 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a81a1210 con 0x7fb1a8074fe0 2024-05-08T22:06:36.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.708+0000 7fb19ea00700 1 -- 172.21.15.131:0/1890738026 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb194010460 con 0x7fb1a8074fe0 2024-05-08T22:06:36.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.708+0000 7fb19ea00700 1 -- 172.21.15.131:0/1890738026 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1940195c0 con 0x7fb1a8074fe0 2024-05-08T22:06:36.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.708+0000 7fb1aca00700 1 -- 172.21.15.131:0/1890738026 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a81a1770 con 0x7fb1a8074fe0 2024-05-08T22:06:36.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.709+0000 7fb1aca00700 1 -- 172.21.15.131:0/1890738026 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a8107bf0 con 0x7fb1a8074fe0 2024-05-08T22:06:36.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.711+0000 7fb19ea00700 1 -- 172.21.15.131:0/1890738026 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fb1940105d0 con 0x7fb1a8074fe0 2024-05-08T22:06:36.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.712+0000 7fb19ea00700 1 --2- 172.21.15.131:0/1890738026 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb18c0a11b0 0x7fb18c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:36.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.712+0000 7fb19ea00700 1 -- 172.21.15.131:0/1890738026 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb1940cd030 con 0x7fb1a8074fe0 2024-05-08T22:06:36.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.712+0000 7fb1a5e00700 1 --2- 172.21.15.131:0/1890738026 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb18c0a11b0 0x7fb18c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:36.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.713+0000 7fb1a5e00700 1 --2- 172.21.15.131:0/1890738026 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb18c0a11b0 0x7fb18c0a3670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb19800b2e0 tx=0x7fb198005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:36.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.714+0000 7fb19ea00700 1 -- 172.21.15.131:0/1890738026 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb194096350 con 0x7fb1a8074fe0 2024-05-08T22:06:36.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.926+0000 7fb1aca00700 1 -- 172.21.15.131:0/1890738026 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb1a810eae0 con 0x7fb1a8074fe0 2024-05-08T22:06:36.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.926+0000 7fb19ea00700 1 -- 172.21.15.131:0/1890738026 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb194099970 con 0x7fb1a8074fe0 2024-05-08T22:06:36.928 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-08T22:06:36.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.929+0000 7fb1aca00700 1 -- 172.21.15.131:0/1890738026 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb18c0a11b0 msgr2=0x7fb18c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:36.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.929+0000 7fb1aca00700 1 --2- 172.21.15.131:0/1890738026 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb18c0a11b0 0x7fb18c0a3670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb19800b2e0 tx=0x7fb198005fb0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.929+0000 7fb1aca00700 1 -- 172.21.15.131:0/1890738026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1a8074fe0 msgr2=0x7fb1a81996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:36.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.929+0000 7fb1aca00700 1 --2- 172.21.15.131:0/1890738026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1a8074fe0 0x7fb1a81996c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb19400d980 tx=0x7fb19400dc90 comp rx=0 tx=0).stop 2024-05-08T22:06:36.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.929+0000 7fb1aca00700 1 -- 172.21.15.131:0/1890738026 shutdown_connections 2024-05-08T22:06:36.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.929+0000 7fb1aca00700 1 --2- 172.21.15.131:0/1890738026 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb18c0a11b0 0x7fb18c0a3670 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.929+0000 7fb1aca00700 1 --2- 172.21.15.131:0/1890738026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1a8108e30 0x7fb1a81a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.929+0000 7fb1aca00700 1 --2- 172.21.15.131:0/1890738026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1a8073a00 0x7fb1a8199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.929+0000 7fb1aca00700 1 --2- 172.21.15.131:0/1890738026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1a8074fe0 0x7fb1a81996c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:36.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.929+0000 7fb1aca00700 1 -- 172.21.15.131:0/1890738026 >> 172.21.15.131:0/1890738026 conn(0x7fb1a80fc500 msgr2=0x7fb1a8104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:36.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.929+0000 7fb1aca00700 1 -- 172.21.15.131:0/1890738026 shutdown_connections 2024-05-08T22:06:36.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:36.929+0000 7fb1aca00700 1 -- 172.21.15.131:0/1890738026 wait complete. 2024-05-08T22:06:36.949 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:06:37.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.146+0000 7fd23c000700 1 -- 172.21.15.131:0/2993467105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd234100f20 msgr2=0x7fd234101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:37.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.146+0000 7fd23c000700 1 --2- 172.21.15.131:0/2993467105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd234100f20 0x7fd234101360 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd228009990 tx=0x7fd228009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:37.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.146+0000 7fd23c000700 1 -- 172.21.15.131:0/2993467105 shutdown_connections 2024-05-08T22:06:37.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.146+0000 7fd23c000700 1 --2- 172.21.15.131:0/2993467105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd234102ab0 0x7fd2341093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:37.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.146+0000 7fd23c000700 1 --2- 172.21.15.131:0/2993467105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd234100f20 0x7fd234101360 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:37.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.146+0000 7fd23c000700 1 --2- 172.21.15.131:0/2993467105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2341020f0 0x7fd234102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:37.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.146+0000 7fd23c000700 1 -- 172.21.15.131:0/2993467105 >> 172.21.15.131:0/2993467105 conn(0x7fd2340fc500 msgr2=0x7fd2340fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:37.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.146+0000 7fd23c000700 1 -- 172.21.15.131:0/2993467105 shutdown_connections 2024-05-08T22:06:37.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.146+0000 7fd23c000700 1 -- 172.21.15.131:0/2993467105 wait complete. 2024-05-08T22:06:37.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.147+0000 7fd23c000700 1 Processor -- start 2024-05-08T22:06:37.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.147+0000 7fd23c000700 1 -- start start 2024-05-08T22:06:37.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.148+0000 7fd23c000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd234100f20 0x7fd234199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:37.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.148+0000 7fd23c000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2341020f0 0x7fd234199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:37.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.148+0000 7fd23c000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd234102ab0 0x7fd2341a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:37.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.148+0000 7fd23c000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2341a0c40 con 0x7fd234100f20 2024-05-08T22:06:37.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.148+0000 7fd23c000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2341a0d80 con 0x7fd2341020f0 2024-05-08T22:06:37.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.148+0000 7fd23c000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2341a0f10 con 0x7fd234102ab0 2024-05-08T22:06:37.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.148+0000 7fd239600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd234100f20 0x7fd234199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:37.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.148+0000 7fd239600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd234100f20 0x7fd234199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40274/0 (socket says 172.21.15.131:40274) 2024-05-08T22:06:37.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.148+0000 7fd239600700 1 -- 172.21.15.131:0/3908691749 learned_addr learned my addr 172.21.15.131:0/3908691749 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:37.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.148+0000 7fd23a000700 1 --2- 172.21.15.131:0/3908691749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd234102ab0 0x7fd2341a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:37.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.148+0000 7fd238c00700 1 --2- 172.21.15.131:0/3908691749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2341020f0 0x7fd234199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:37.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.148+0000 7fd239600700 1 -- 172.21.15.131:0/3908691749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd234102ab0 msgr2=0x7fd2341a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:37.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.148+0000 7fd239600700 1 --2- 172.21.15.131:0/3908691749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd234102ab0 0x7fd2341a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:37.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.148+0000 7fd239600700 1 -- 172.21.15.131:0/3908691749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2341020f0 msgr2=0x7fd234199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:37.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.148+0000 7fd239600700 1 --2- 172.21.15.131:0/3908691749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2341020f0 0x7fd234199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:37.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.148+0000 7fd239600700 1 -- 172.21.15.131:0/3908691749 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2280095f0 con 0x7fd234100f20 2024-05-08T22:06:37.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.148+0000 7fd238c00700 1 --2- 172.21.15.131:0/3908691749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2341020f0 0x7fd234199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:06:37.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.149+0000 7fd239600700 1 --2- 172.21.15.131:0/3908691749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd234100f20 0x7fd234199110 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd22800b750 tx=0x7fd228005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:37.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.149+0000 7fd232000700 1 -- 172.21.15.131:0/3908691749 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22801d070 con 0x7fd234100f20 2024-05-08T22:06:37.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.149+0000 7fd232000700 1 -- 172.21.15.131:0/3908691749 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd228021410 con 0x7fd234100f20 2024-05-08T22:06:37.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.149+0000 7fd232000700 1 -- 172.21.15.131:0/3908691749 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd228017650 con 0x7fd234100f20 2024-05-08T22:06:37.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.149+0000 7fd23c000700 1 -- 172.21.15.131:0/3908691749 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2341a1190 con 0x7fd234100f20 2024-05-08T22:06:37.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.149+0000 7fd23c000700 1 -- 172.21.15.131:0/3908691749 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2341a1560 con 0x7fd234100f20 2024-05-08T22:06:37.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.150+0000 7fd23c000700 1 -- 172.21.15.131:0/3908691749 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd234067cf0 con 0x7fd234100f20 2024-05-08T22:06:37.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.152+0000 7fd232000700 1 -- 172.21.15.131:0/3908691749 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fd228021580 con 0x7fd234100f20 2024-05-08T22:06:37.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.153+0000 7fd232000700 1 --2- 172.21.15.131:0/3908691749 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd21c0a11b0 0x7fd21c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:37.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.153+0000 7fd232000700 1 -- 172.21.15.131:0/3908691749 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd2280ce2b0 con 0x7fd234100f20 2024-05-08T22:06:37.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.153+0000 7fd238c00700 1 --2- 172.21.15.131:0/3908691749 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd21c0a11b0 0x7fd21c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:37.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.154+0000 7fd238c00700 1 --2- 172.21.15.131:0/3908691749 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd21c0a11b0 0x7fd21c0a3670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd220005950 tx=0x7fd2200058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:37.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.157+0000 7fd232000700 1 -- 172.21.15.131:0/3908691749 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd228097600 con 0x7fd234100f20 2024-05-08T22:06:37.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:36 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2754406387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:06:37.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:36 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1450552484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:06:37.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:36 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1890738026' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:06:37.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:36 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2754406387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:06:37.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:36 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1450552484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:06:37.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:36 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1890738026' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:06:37.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.366+0000 7fd23c000700 1 -- 172.21.15.131:0/3908691749 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd2341a1780 con 0x7fd234100f20 2024-05-08T22:06:37.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.366+0000 7fd232000700 1 -- 172.21.15.131:0/3908691749 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd22809ac20 con 0x7fd234100f20 2024-05-08T22:06:37.367 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-08T22:06:37.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.368+0000 7fd23c000700 1 -- 172.21.15.131:0/3908691749 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd21c0a11b0 msgr2=0x7fd21c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:37.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.368+0000 7fd23c000700 1 --2- 172.21.15.131:0/3908691749 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd21c0a11b0 0x7fd21c0a3670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd220005950 tx=0x7fd2200058e0 comp rx=0 tx=0).stop 2024-05-08T22:06:37.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.368+0000 7fd23c000700 1 -- 172.21.15.131:0/3908691749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd234100f20 msgr2=0x7fd234199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:37.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.368+0000 7fd23c000700 1 --2- 172.21.15.131:0/3908691749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd234100f20 0x7fd234199110 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd22800b750 tx=0x7fd228005740 comp rx=0 tx=0).stop 2024-05-08T22:06:37.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.369+0000 7fd23c000700 1 -- 172.21.15.131:0/3908691749 shutdown_connections 2024-05-08T22:06:37.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.369+0000 7fd23c000700 1 --2- 172.21.15.131:0/3908691749 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd21c0a11b0 0x7fd21c0a3670 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:37.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.369+0000 7fd23c000700 1 --2- 172.21.15.131:0/3908691749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd234102ab0 0x7fd2341a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:37.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.369+0000 7fd23c000700 1 --2- 172.21.15.131:0/3908691749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd234100f20 0x7fd234199110 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:37.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.369+0000 7fd23c000700 1 --2- 172.21.15.131:0/3908691749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2341020f0 0x7fd234199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:37.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.369+0000 7fd23c000700 1 -- 172.21.15.131:0/3908691749 >> 172.21.15.131:0/3908691749 conn(0x7fd2340fc500 msgr2=0x7fd2340731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:37.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.369+0000 7fd23c000700 1 -- 172.21.15.131:0/3908691749 shutdown_connections 2024-05-08T22:06:37.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.369+0000 7fd23c000700 1 -- 172.21.15.131:0/3908691749 wait complete. 2024-05-08T22:06:37.386 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:36 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2754406387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:06:37.386 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:36 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1450552484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:06:37.386 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:36 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1890738026' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:06:37.389 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:06:37.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.585+0000 7fadd0200700 1 -- 172.21.15.131:0/1555179023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadc8102170 msgr2=0x7fadc81025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:37.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.585+0000 7fadd0200700 1 --2- 172.21.15.131:0/1555179023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadc8102170 0x7fadc81025d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fadb40099c0 tx=0x7fadb4009cd0 comp rx=0 tx=0).stop 2024-05-08T22:06:37.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.585+0000 7fadd0200700 1 -- 172.21.15.131:0/1555179023 shutdown_connections 2024-05-08T22:06:37.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.585+0000 7fadd0200700 1 --2- 172.21.15.131:0/1555179023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadc8102b10 0x7fadc81093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:37.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.585+0000 7fadd0200700 1 --2- 172.21.15.131:0/1555179023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadc8100f80 0x7fadc81013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:37.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.585+0000 7fadd0200700 1 --2- 172.21.15.131:0/1555179023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadc8102170 0x7fadc81025d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:37.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.585+0000 7fadd0200700 1 -- 172.21.15.131:0/1555179023 >> 172.21.15.131:0/1555179023 conn(0x7fadc80fc4f0 msgr2=0x7fadc80fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:37.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.585+0000 7fadd0200700 1 -- 172.21.15.131:0/1555179023 shutdown_connections 2024-05-08T22:06:37.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.585+0000 7fadd0200700 1 -- 172.21.15.131:0/1555179023 wait complete. 2024-05-08T22:06:37.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.586+0000 7fadd0200700 1 Processor -- start 2024-05-08T22:06:37.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.586+0000 7fadd0200700 1 -- start start 2024-05-08T22:06:37.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.587+0000 7fadd0200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadc8100f80 0x7fadc8074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:37.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.587+0000 7fadd0200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadc8102170 0x7fadc8073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:37.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.587+0000 7fadd0200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadc8102b10 0x7fadc80738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:37.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.587+0000 7fadd0200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc8074070 con 0x7fadc8100f80 2024-05-08T22:06:37.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.587+0000 7fadd0200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc80741e0 con 0x7fadc8102170 2024-05-08T22:06:37.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.587+0000 7fadd0200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc81a0f90 con 0x7fadc8102b10 2024-05-08T22:06:37.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.587+0000 7fadcce00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadc8102170 0x7fadc8073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:37.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.587+0000 7fadcd800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadc8100f80 0x7fadc8074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:37.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.587+0000 7fadcce00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadc8102170 0x7fadc8073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:53380/0 (socket says 172.21.15.131:53380) 2024-05-08T22:06:37.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.587+0000 7fadcce00700 1 -- 172.21.15.131:0/501783931 learned_addr learned my addr 172.21.15.131:0/501783931 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:37.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.587+0000 7fadce200700 1 --2- 172.21.15.131:0/501783931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadc8102b10 0x7fadc80738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:37.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.587+0000 7fadcd800700 1 -- 172.21.15.131:0/501783931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadc8102b10 msgr2=0x7fadc80738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:37.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.587+0000 7fadcd800700 1 --2- 172.21.15.131:0/501783931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadc8102b10 0x7fadc80738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:37.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.587+0000 7fadcd800700 1 -- 172.21.15.131:0/501783931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadc8102170 msgr2=0x7fadc8073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:37.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.587+0000 7fadcd800700 1 --2- 172.21.15.131:0/501783931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadc8102170 0x7fadc8073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:37.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.587+0000 7fadcd800700 1 -- 172.21.15.131:0/501783931 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb40095f0 con 0x7fadc8100f80 2024-05-08T22:06:37.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.587+0000 7fadcce00700 1 --2- 172.21.15.131:0/501783931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadc8102170 0x7fadc8073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:37.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.588+0000 7fadcd800700 1 --2- 172.21.15.131:0/501783931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadc8100f80 0x7fadc8074d00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00e9c0 tx=0x7fadbc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:37.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.588+0000 7fadc6000700 1 -- 172.21.15.131:0/501783931 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc004230 con 0x7fadc8100f80 2024-05-08T22:06:37.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.588+0000 7fadc6000700 1 -- 172.21.15.131:0/501783931 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadbc004390 con 0x7fadc8100f80 2024-05-08T22:06:37.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.588+0000 7fadd0200700 1 -- 172.21.15.131:0/501783931 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc81a1220 con 0x7fadc8100f80 2024-05-08T22:06:37.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.588+0000 7fadc6000700 1 -- 172.21.15.131:0/501783931 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc003d30 con 0x7fadc8100f80 2024-05-08T22:06:37.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.588+0000 7fadd0200700 1 -- 172.21.15.131:0/501783931 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc81a16e0 con 0x7fadc8100f80 2024-05-08T22:06:37.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.589+0000 7fadd0200700 1 -- 172.21.15.131:0/501783931 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc8108450 con 0x7fadc8100f80 2024-05-08T22:06:37.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.591+0000 7fadc6000700 1 -- 172.21.15.131:0/501783931 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fadbc00c760 con 0x7fadc8100f80 2024-05-08T22:06:37.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.592+0000 7fadc6000700 1 --2- 172.21.15.131:0/501783931 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fadb00a11b0 0x7fadb00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:37.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.593+0000 7fadc6000700 1 -- 172.21.15.131:0/501783931 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fadbc014070 con 0x7fadc8100f80 2024-05-08T22:06:37.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.593+0000 7fadcce00700 1 --2- 172.21.15.131:0/501783931 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fadb00a11b0 0x7fadb00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:37.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.593+0000 7fadcce00700 1 --2- 172.21.15.131:0/501783931 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fadb00a11b0 0x7fadb00a3670 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fadb40095c0 tx=0x7fadb4009530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:37.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.597+0000 7fadc6000700 1 -- 172.21.15.131:0/501783931 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fadbc096370 con 0x7fadc8100f80 2024-05-08T22:06:37.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.803+0000 7fadd0200700 1 -- 172.21.15.131:0/501783931 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fadc810c9b0 con 0x7fadc8100f80 2024-05-08T22:06:37.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.803+0000 7fadc6000700 1 -- 172.21.15.131:0/501783931 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v85) v1 ==== 103+0+11 (secure 0 0 0) 0x7fadbc003e90 con 0x7fadc8100f80 2024-05-08T22:06:37.804 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-08T22:06:37.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.806+0000 7fadd0200700 1 -- 172.21.15.131:0/501783931 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fadb00a11b0 msgr2=0x7fadb00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:37.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.806+0000 7fadd0200700 1 --2- 172.21.15.131:0/501783931 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fadb00a11b0 0x7fadb00a3670 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fadb40095c0 tx=0x7fadb4009530 comp rx=0 tx=0).stop 2024-05-08T22:06:37.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.806+0000 7fadd0200700 1 -- 172.21.15.131:0/501783931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadc8100f80 msgr2=0x7fadc8074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:37.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.806+0000 7fadd0200700 1 --2- 172.21.15.131:0/501783931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadc8100f80 0x7fadc8074d00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00e9c0 tx=0x7fadbc00ecd0 comp rx=0 tx=0).stop 2024-05-08T22:06:37.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.806+0000 7fadd0200700 1 -- 172.21.15.131:0/501783931 shutdown_connections 2024-05-08T22:06:37.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.806+0000 7fadd0200700 1 --2- 172.21.15.131:0/501783931 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fadb00a11b0 0x7fadb00a3670 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:37.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.806+0000 7fadd0200700 1 --2- 172.21.15.131:0/501783931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadc8102b10 0x7fadc80738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:37.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.806+0000 7fadd0200700 1 --2- 172.21.15.131:0/501783931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadc8100f80 0x7fadc8074d00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:37.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.806+0000 7fadd0200700 1 --2- 172.21.15.131:0/501783931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadc8102170 0x7fadc8073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:37.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.806+0000 7fadd0200700 1 -- 172.21.15.131:0/501783931 >> 172.21.15.131:0/501783931 conn(0x7fadc80fc4f0 msgr2=0x7fadc8105590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:37.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.806+0000 7fadd0200700 1 -- 172.21.15.131:0/501783931 shutdown_connections 2024-05-08T22:06:37.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:37.806+0000 7fadd0200700 1 -- 172.21.15.131:0/501783931 wait complete. 2024-05-08T22:06:37.826 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:06:38.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.020+0000 7fd6a9c00700 1 -- 172.21.15.131:0/2487252678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6a41029a0 msgr2=0x7fd6a41092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:38.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.020+0000 7fd6a9c00700 1 --2- 172.21.15.131:0/2487252678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6a41029a0 0x7fd6a41092c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00b0a0 tx=0x7fd69c00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.020+0000 7fd6a9c00700 1 -- 172.21.15.131:0/2487252678 shutdown_connections 2024-05-08T22:06:38.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.020+0000 7fd6a9c00700 1 --2- 172.21.15.131:0/2487252678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6a41029a0 0x7fd6a41092c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.020+0000 7fd6a9c00700 1 --2- 172.21.15.131:0/2487252678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6a4100e90 0x7fd6a41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.020+0000 7fd6a9c00700 1 --2- 172.21.15.131:0/2487252678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6a4101fe0 0x7fd6a4102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.020+0000 7fd6a9c00700 1 -- 172.21.15.131:0/2487252678 >> 172.21.15.131:0/2487252678 conn(0x7fd6a40fc400 msgr2=0x7fd6a40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:38.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.020+0000 7fd6a9c00700 1 -- 172.21.15.131:0/2487252678 shutdown_connections 2024-05-08T22:06:38.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.020+0000 7fd6a9c00700 1 -- 172.21.15.131:0/2487252678 wait complete. 2024-05-08T22:06:38.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.021+0000 7fd6a9c00700 1 Processor -- start 2024-05-08T22:06:38.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.021+0000 7fd6a9c00700 1 -- start start 2024-05-08T22:06:38.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.021+0000 7fd6a9c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6a4100e90 0x7fd6a4074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:38.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.021+0000 7fd6a9c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6a4101fe0 0x7fd6a4073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:38.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.022+0000 7fd6a9c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6a41029a0 0x7fd6a40738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:38.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.022+0000 7fd6a9c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a4074070 con 0x7fd6a41029a0 2024-05-08T22:06:38.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.022+0000 7fd6a9c00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a40741e0 con 0x7fd6a4100e90 2024-05-08T22:06:38.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.022+0000 7fd6a9c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a41a0f90 con 0x7fd6a4101fe0 2024-05-08T22:06:38.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.022+0000 7fd6a3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6a41029a0 0x7fd6a40738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:38.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.022+0000 7fd6a2000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6a4101fe0 0x7fd6a4073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:38.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.022+0000 7fd6a2a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6a4100e90 0x7fd6a4074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:38.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.022+0000 7fd6a3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6a41029a0 0x7fd6a40738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40302/0 (socket says 172.21.15.131:40302) 2024-05-08T22:06:38.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.022+0000 7fd6a3400700 1 -- 172.21.15.131:0/3879578247 learned_addr learned my addr 172.21.15.131:0/3879578247 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:38.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.022+0000 7fd6a2000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6a4101fe0 0x7fd6a4073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59672/0 (socket says 172.21.15.131:59672) 2024-05-08T22:06:38.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.022+0000 7fd6a2a00700 1 -- 172.21.15.131:0/3879578247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6a4101fe0 msgr2=0x7fd6a4073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:38.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.022+0000 7fd6a2a00700 1 --2- 172.21.15.131:0/3879578247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6a4101fe0 0x7fd6a4073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.022+0000 7fd6a2a00700 1 -- 172.21.15.131:0/3879578247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6a41029a0 msgr2=0x7fd6a40738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:38.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.022+0000 7fd6a2a00700 1 --2- 172.21.15.131:0/3879578247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6a41029a0 0x7fd6a40738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.022+0000 7fd6a2a00700 1 -- 172.21.15.131:0/3879578247 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c009d00 con 0x7fd6a4100e90 2024-05-08T22:06:38.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.022+0000 7fd6a3400700 1 --2- 172.21.15.131:0/3879578247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6a41029a0 0x7fd6a40738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:38.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.022+0000 7fd6a2a00700 1 --2- 172.21.15.131:0/3879578247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6a4100e90 0x7fd6a4074d00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd69400e9f0 tx=0x7fd69400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:38.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.023+0000 7fd69b400700 1 -- 172.21.15.131:0/3879578247 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd694004220 con 0x7fd6a4100e90 2024-05-08T22:06:38.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.023+0000 7fd69b400700 1 -- 172.21.15.131:0/3879578247 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd694004cf0 con 0x7fd6a4100e90 2024-05-08T22:06:38.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.023+0000 7fd6a9c00700 1 -- 172.21.15.131:0/3879578247 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6a41a1220 con 0x7fd6a4100e90 2024-05-08T22:06:38.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.023+0000 7fd69b400700 1 -- 172.21.15.131:0/3879578247 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6940056d0 con 0x7fd6a4100e90 2024-05-08T22:06:38.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.023+0000 7fd6a9c00700 1 -- 172.21.15.131:0/3879578247 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a41a16e0 con 0x7fd6a4100e90 2024-05-08T22:06:38.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.024+0000 7fd6a9c00700 1 -- 172.21.15.131:0/3879578247 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6a4108230 con 0x7fd6a4100e90 2024-05-08T22:06:38.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.026+0000 7fd69b400700 1 -- 172.21.15.131:0/3879578247 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fd694004750 con 0x7fd6a4100e90 2024-05-08T22:06:38.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.027+0000 7fd69b400700 1 --2- 172.21.15.131:0/3879578247 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd6880a11b0 0x7fd6880a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:38.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.027+0000 7fd69b400700 1 -- 172.21.15.131:0/3879578247 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd694014070 con 0x7fd6a4100e90 2024-05-08T22:06:38.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.027+0000 7fd6a2000700 1 --2- 172.21.15.131:0/3879578247 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd6880a11b0 0x7fd6880a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:38.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.028+0000 7fd6a2000700 1 --2- 172.21.15.131:0/3879578247 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd6880a11b0 0x7fd6880a3670 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd690009c60 tx=0x7fd690009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:38.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.030+0000 7fd69b400700 1 -- 172.21.15.131:0/3879578247 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd6940966a0 con 0x7fd6a4100e90 2024-05-08T22:06:38.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:37 smithi131 ceph-mon[37260]: pgmap v283: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T22:06:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:37 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3908691749' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:06:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:37 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/501783931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:06:38.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:37 smithi157 ceph-mon[77746]: pgmap v283: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T22:06:38.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:37 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3908691749' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:06:38.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:37 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/501783931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:06:38.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.283+0000 7fd6a9c00700 1 -- 172.21.15.131:0/3879578247 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6a410c9b0 con 0x7fd6a4100e90 2024-05-08T22:06:38.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.285+0000 7fd69b400700 1 -- 172.21.15.131:0/3879578247 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10889 (secure 0 0 0) 0x7fd694004e60 con 0x7fd6a4100e90 2024-05-08T22:06:38.285 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:06:38.286 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:05:14.034908+0000","modified":"2024-05-08T22:05:39.494566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:06:38.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.287+0000 7fd6a9c00700 1 -- 172.21.15.131:0/3879578247 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd6880a11b0 msgr2=0x7fd6880a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:38.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.287+0000 7fd6a9c00700 1 --2- 172.21.15.131:0/3879578247 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd6880a11b0 0x7fd6880a3670 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd690009c60 tx=0x7fd690009260 comp rx=0 tx=0).stop 2024-05-08T22:06:38.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.287+0000 7fd6a9c00700 1 -- 172.21.15.131:0/3879578247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6a4100e90 msgr2=0x7fd6a4074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:38.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.287+0000 7fd6a9c00700 1 --2- 172.21.15.131:0/3879578247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6a4100e90 0x7fd6a4074d00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd69400e9f0 tx=0x7fd69400ed00 comp rx=0 tx=0).stop 2024-05-08T22:06:38.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.287+0000 7fd6a9c00700 1 -- 172.21.15.131:0/3879578247 shutdown_connections 2024-05-08T22:06:38.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.287+0000 7fd6a9c00700 1 --2- 172.21.15.131:0/3879578247 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd6880a11b0 0x7fd6880a3670 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.287+0000 7fd6a9c00700 1 --2- 172.21.15.131:0/3879578247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6a41029a0 0x7fd6a40738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.287+0000 7fd6a9c00700 1 --2- 172.21.15.131:0/3879578247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6a4100e90 0x7fd6a4074d00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.287+0000 7fd6a9c00700 1 --2- 172.21.15.131:0/3879578247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6a4101fe0 0x7fd6a4073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.287+0000 7fd6a9c00700 1 -- 172.21.15.131:0/3879578247 >> 172.21.15.131:0/3879578247 conn(0x7fd6a40fc400 msgr2=0x7fd6a4105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:38.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.287+0000 7fd6a9c00700 1 -- 172.21.15.131:0/3879578247 shutdown_connections 2024-05-08T22:06:38.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.287+0000 7fd6a9c00700 1 -- 172.21.15.131:0/3879578247 wait complete. 2024-05-08T22:06:38.289 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 23 2024-05-08T22:06:38.308 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:06:38.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:37 smithi132 ceph-mon[78180]: pgmap v283: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T22:06:38.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:37 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3908691749' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:06:38.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:37 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/501783931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:06:38.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.500+0000 7fb665800700 1 -- 172.21.15.131:0/3655432053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb660100e70 msgr2=0x7fb660101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:38.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.500+0000 7fb665800700 1 --2- 172.21.15.131:0/3655432053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb660100e70 0x7fb660101290 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb650009990 tx=0x7fb650009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.500+0000 7fb665800700 1 -- 172.21.15.131:0/3655432053 shutdown_connections 2024-05-08T22:06:38.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.500+0000 7fb665800700 1 --2- 172.21.15.131:0/3655432053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb660102980 0x7fb6601092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.500+0000 7fb665800700 1 --2- 172.21.15.131:0/3655432053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb660100e70 0x7fb660101290 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.500+0000 7fb665800700 1 --2- 172.21.15.131:0/3655432053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb660101fc0 0x7fb660102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.500+0000 7fb665800700 1 -- 172.21.15.131:0/3655432053 >> 172.21.15.131:0/3655432053 conn(0x7fb6600fc3e0 msgr2=0x7fb6600fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:38.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.500+0000 7fb665800700 1 -- 172.21.15.131:0/3655432053 shutdown_connections 2024-05-08T22:06:38.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.500+0000 7fb665800700 1 -- 172.21.15.131:0/3655432053 wait complete. 2024-05-08T22:06:38.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.501+0000 7fb665800700 1 Processor -- start 2024-05-08T22:06:38.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.502+0000 7fb665800700 1 -- start start 2024-05-08T22:06:38.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.502+0000 7fb665800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb660100e70 0x7fb660199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:38.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.502+0000 7fb665800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb660101fc0 0x7fb6601995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:38.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.502+0000 7fb665800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb660102980 0x7fb6601a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:38.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.502+0000 7fb665800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6601a0ba0 con 0x7fb660102980 2024-05-08T22:06:38.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.502+0000 7fb665800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6601a0ce0 con 0x7fb660100e70 2024-05-08T22:06:38.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.502+0000 7fb665800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6601a0e70 con 0x7fb660101fc0 2024-05-08T22:06:38.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.502+0000 7fb65e000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb660101fc0 0x7fb6601995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:38.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.502+0000 7fb65f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb660102980 0x7fb6601a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:38.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.502+0000 7fb65e000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb660101fc0 0x7fb6601995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59684/0 (socket says 172.21.15.131:59684) 2024-05-08T22:06:38.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.502+0000 7fb65ea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb660100e70 0x7fb660199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:38.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.502+0000 7fb65e000700 1 -- 172.21.15.131:0/930280426 learned_addr learned my addr 172.21.15.131:0/930280426 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:38.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.503+0000 7fb65e000700 1 -- 172.21.15.131:0/930280426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb660100e70 msgr2=0x7fb660199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:38.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.503+0000 7fb65e000700 1 --2- 172.21.15.131:0/930280426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb660100e70 0x7fb660199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.503+0000 7fb65e000700 1 -- 172.21.15.131:0/930280426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb660102980 msgr2=0x7fb6601a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:38.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.503+0000 7fb65e000700 1 --2- 172.21.15.131:0/930280426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb660102980 0x7fb6601a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.503+0000 7fb65e000700 1 -- 172.21.15.131:0/930280426 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6500095f0 con 0x7fb660101fc0 2024-05-08T22:06:38.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.503+0000 7fb65ea00700 1 --2- 172.21.15.131:0/930280426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb660100e70 0x7fb660199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:38.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.503+0000 7fb65e000700 1 --2- 172.21.15.131:0/930280426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb660101fc0 0x7fb6601995b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00e9a0 tx=0x7fb64c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:38.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.503+0000 7fb657400700 1 -- 172.21.15.131:0/930280426 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c009790 con 0x7fb660101fc0 2024-05-08T22:06:38.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.503+0000 7fb665800700 1 -- 172.21.15.131:0/930280426 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6601a1150 con 0x7fb660101fc0 2024-05-08T22:06:38.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.503+0000 7fb657400700 1 -- 172.21.15.131:0/930280426 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb64c010430 con 0x7fb660101fc0 2024-05-08T22:06:38.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.503+0000 7fb657400700 1 -- 172.21.15.131:0/930280426 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c018680 con 0x7fb660101fc0 2024-05-08T22:06:38.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.503+0000 7fb665800700 1 -- 172.21.15.131:0/930280426 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6601a1660 con 0x7fb660101fc0 2024-05-08T22:06:38.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.504+0000 7fb665800700 1 -- 172.21.15.131:0/930280426 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb660067c10 con 0x7fb660101fc0 2024-05-08T22:06:38.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.506+0000 7fb657400700 1 -- 172.21.15.131:0/930280426 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fb64c0105a0 con 0x7fb660101fc0 2024-05-08T22:06:38.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.507+0000 7fb657400700 1 --2- 172.21.15.131:0/930280426 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb6440a11b0 0x7fb6440a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:38.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.508+0000 7fb657400700 1 -- 172.21.15.131:0/930280426 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb64c014070 con 0x7fb660101fc0 2024-05-08T22:06:38.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.508+0000 7fb65ea00700 1 --2- 172.21.15.131:0/930280426 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb6440a11b0 0x7fb6440a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:38.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.508+0000 7fb65ea00700 1 --2- 172.21.15.131:0/930280426 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb6440a11b0 0x7fb6440a3670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb65000b2e0 tx=0x7fb650005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:38.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.513+0000 7fb657400700 1 -- 172.21.15.131:0/930280426 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb64c052d60 con 0x7fb660101fc0 2024-05-08T22:06:38.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.730+0000 7fb665800700 1 -- 172.21.15.131:0/930280426 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb66010c890 con 0x7fb660101fc0 2024-05-08T22:06:38.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.733+0000 7fb657400700 1 -- 172.21.15.131:0/930280426 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+24534 (secure 0 0 0) 0x7fb64c096260 con 0x7fb660101fc0 2024-05-08T22:06:38.733 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:06:38.733 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":85,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:05:41.813233+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:05:12.469858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:05:13.484446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:05:17.016186+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:06:38.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.735+0000 7fb665800700 1 -- 172.21.15.131:0/930280426 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb6440a11b0 msgr2=0x7fb6440a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:38.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.735+0000 7fb665800700 1 --2- 172.21.15.131:0/930280426 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb6440a11b0 0x7fb6440a3670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb65000b2e0 tx=0x7fb650005fb0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.735+0000 7fb665800700 1 -- 172.21.15.131:0/930280426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb660101fc0 msgr2=0x7fb6601995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:38.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.735+0000 7fb665800700 1 --2- 172.21.15.131:0/930280426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb660101fc0 0x7fb6601995b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00e9a0 tx=0x7fb64c00ecb0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.735+0000 7fb665800700 1 -- 172.21.15.131:0/930280426 shutdown_connections 2024-05-08T22:06:38.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.735+0000 7fb665800700 1 --2- 172.21.15.131:0/930280426 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fb6440a11b0 0x7fb6440a3670 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.735+0000 7fb665800700 1 --2- 172.21.15.131:0/930280426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb660102980 0x7fb6601a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.735+0000 7fb665800700 1 --2- 172.21.15.131:0/930280426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb660100e70 0x7fb660199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.735+0000 7fb665800700 1 --2- 172.21.15.131:0/930280426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb660101fc0 0x7fb6601995b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.735+0000 7fb665800700 1 -- 172.21.15.131:0/930280426 >> 172.21.15.131:0/930280426 conn(0x7fb6600fc3e0 msgr2=0x7fb6600731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:38.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.735+0000 7fb665800700 1 -- 172.21.15.131:0/930280426 shutdown_connections 2024-05-08T22:06:38.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.735+0000 7fb665800700 1 -- 172.21.15.131:0/930280426 wait complete. 2024-05-08T22:06:38.756 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:01:41.271078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:05:12.469858+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:05:13.484446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:05:17.016186+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:06:38.757 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:06:38.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.951+0000 7f075ca00700 1 -- 172.21.15.131:0/4229228963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0758074fe0 msgr2=0x7f0758073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:38.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.951+0000 7f075ca00700 1 --2- 172.21.15.131:0/4229228963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0758074fe0 0x7f0758073430 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0748009990 tx=0x7f0748009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.951+0000 7f075ca00700 1 -- 172.21.15.131:0/4229228963 shutdown_connections 2024-05-08T22:06:38.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.951+0000 7f075ca00700 1 --2- 172.21.15.131:0/4229228963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0758108e30 0x7f075810b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.951+0000 7f075ca00700 1 --2- 172.21.15.131:0/4229228963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0758074fe0 0x7f0758073430 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.951+0000 7f075ca00700 1 --2- 172.21.15.131:0/4229228963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0758073a00 0x7f0758073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.951+0000 7f075ca00700 1 -- 172.21.15.131:0/4229228963 >> 172.21.15.131:0/4229228963 conn(0x7f07580fc500 msgr2=0x7f07580fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:38.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.951+0000 7f075ca00700 1 -- 172.21.15.131:0/4229228963 shutdown_connections 2024-05-08T22:06:38.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.951+0000 7f075ca00700 1 -- 172.21.15.131:0/4229228963 wait complete. 2024-05-08T22:06:38.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.952+0000 7f075ca00700 1 Processor -- start 2024-05-08T22:06:38.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.952+0000 7f075ca00700 1 -- start start 2024-05-08T22:06:38.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.952+0000 7f075ca00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0758073a00 0x7f0758199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:38.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.952+0000 7f075ca00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0758074fe0 0x7f0758199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:38.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.953+0000 7f075ca00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0758108e30 0x7f07581a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:38.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.953+0000 7f075ca00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07581a0c40 con 0x7f0758073a00 2024-05-08T22:06:38.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.953+0000 7f075ca00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07581a0d80 con 0x7f0758074fe0 2024-05-08T22:06:38.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.953+0000 7f075ca00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07581a0f10 con 0x7f0758108e30 2024-05-08T22:06:38.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.953+0000 7f0755e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0758073a00 0x7f0758199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:38.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.953+0000 7f0755e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0758073a00 0x7f0758199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40344/0 (socket says 172.21.15.131:40344) 2024-05-08T22:06:38.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.953+0000 7f0755e00700 1 -- 172.21.15.131:0/2007776974 learned_addr learned my addr 172.21.15.131:0/2007776974 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:38.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.953+0000 7f0755400700 1 --2- 172.21.15.131:0/2007776974 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0758074fe0 0x7f0758199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:38.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.953+0000 7f0756800700 1 --2- 172.21.15.131:0/2007776974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0758108e30 0x7f07581a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:38.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.953+0000 7f0755e00700 1 -- 172.21.15.131:0/2007776974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0758108e30 msgr2=0x7f07581a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:38.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.953+0000 7f0755e00700 1 --2- 172.21.15.131:0/2007776974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0758108e30 0x7f07581a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.953+0000 7f0755e00700 1 -- 172.21.15.131:0/2007776974 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0758074fe0 msgr2=0x7f0758199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:38.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.953+0000 7f0755e00700 1 --2- 172.21.15.131:0/2007776974 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0758074fe0 0x7f0758199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:38.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.953+0000 7f0755e00700 1 -- 172.21.15.131:0/2007776974 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0750009c40 con 0x7f0758073a00 2024-05-08T22:06:38.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.953+0000 7f0755400700 1 --2- 172.21.15.131:0/2007776974 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0758074fe0 0x7f0758199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:38.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.953+0000 7f0755e00700 1 --2- 172.21.15.131:0/2007776974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0758073a00 0x7f0758199110 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0748005910 tx=0x7f0748009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:38.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.954+0000 7f074ea00700 1 -- 172.21.15.131:0/2007776974 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074801d070 con 0x7f0758073a00 2024-05-08T22:06:38.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.954+0000 7f075ca00700 1 -- 172.21.15.131:0/2007776974 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07480095f0 con 0x7f0758073a00 2024-05-08T22:06:38.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.954+0000 7f074ea00700 1 -- 172.21.15.131:0/2007776974 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0748021410 con 0x7f0758073a00 2024-05-08T22:06:38.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.954+0000 7f074ea00700 1 -- 172.21.15.131:0/2007776974 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0748017650 con 0x7f0758073a00 2024-05-08T22:06:38.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.954+0000 7f075ca00700 1 -- 172.21.15.131:0/2007776974 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07581a1420 con 0x7f0758073a00 2024-05-08T22:06:38.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.955+0000 7f075ca00700 1 -- 172.21.15.131:0/2007776974 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0758107bf0 con 0x7f0758073a00 2024-05-08T22:06:38.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.957+0000 7f074ea00700 1 -- 172.21.15.131:0/2007776974 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f0748021580 con 0x7f0758073a00 2024-05-08T22:06:38.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.958+0000 7f074ea00700 1 --2- 172.21.15.131:0/2007776974 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f073c0a11b0 0x7f073c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:38.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.958+0000 7f074ea00700 1 -- 172.21.15.131:0/2007776974 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f07480ce0a0 con 0x7f0758073a00 2024-05-08T22:06:38.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.958+0000 7f0755400700 1 --2- 172.21.15.131:0/2007776974 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f073c0a11b0 0x7f073c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:38.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.959+0000 7f0755400700 1 --2- 172.21.15.131:0/2007776974 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f073c0a11b0 0x7f073c0a3670 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0758074d00 tx=0x7f0744009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:38.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:38.962+0000 7f074ea00700 1 -- 172.21.15.131:0/2007776974 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f07480973f0 con 0x7f0758073a00 2024-05-08T22:06:39.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:38 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3879578247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:06:39.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:38 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/930280426' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:06:39.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.171+0000 7f075ca00700 1 -- 172.21.15.131:0/2007776974 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f075810ebe0 con 0x7f0758073a00 2024-05-08T22:06:39.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.171+0000 7f074ea00700 1 -- 172.21.15.131:0/2007776974 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f074809aa10 con 0x7f0758073a00 2024-05-08T22:06:39.173 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-08T22:06:39.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.174+0000 7f075ca00700 1 -- 172.21.15.131:0/2007776974 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f073c0a11b0 msgr2=0x7f073c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:39.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.174+0000 7f075ca00700 1 --2- 172.21.15.131:0/2007776974 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f073c0a11b0 0x7f073c0a3670 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0758074d00 tx=0x7f0744009260 comp rx=0 tx=0).stop 2024-05-08T22:06:39.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.174+0000 7f075ca00700 1 -- 172.21.15.131:0/2007776974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0758073a00 msgr2=0x7f0758199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:39.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.174+0000 7f075ca00700 1 --2- 172.21.15.131:0/2007776974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0758073a00 0x7f0758199110 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0748005910 tx=0x7f0748009580 comp rx=0 tx=0).stop 2024-05-08T22:06:39.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.174+0000 7f075ca00700 1 -- 172.21.15.131:0/2007776974 shutdown_connections 2024-05-08T22:06:39.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.174+0000 7f075ca00700 1 --2- 172.21.15.131:0/2007776974 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f073c0a11b0 0x7f073c0a3670 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:39.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.174+0000 7f075ca00700 1 --2- 172.21.15.131:0/2007776974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0758108e30 0x7f07581a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:39.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.174+0000 7f075ca00700 1 --2- 172.21.15.131:0/2007776974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0758073a00 0x7f0758199110 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:39.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.174+0000 7f075ca00700 1 --2- 172.21.15.131:0/2007776974 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0758074fe0 0x7f0758199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:39.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.174+0000 7f075ca00700 1 -- 172.21.15.131:0/2007776974 >> 172.21.15.131:0/2007776974 conn(0x7f07580fc500 msgr2=0x7f0758104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:39.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.175+0000 7f075ca00700 1 -- 172.21.15.131:0/2007776974 shutdown_connections 2024-05-08T22:06:39.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.175+0000 7f075ca00700 1 -- 172.21.15.131:0/2007776974 wait complete. 2024-05-08T22:06:39.194 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:06:39.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:38 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3879578247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:06:39.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:38 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/930280426' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:06:39.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:38 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3879578247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:06:39.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:38 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/930280426' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:06:39.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.389+0000 7f49cac00700 1 -- 172.21.15.131:0/395003630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c41020f0 msgr2=0x7f49c4102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:39.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.389+0000 7f49cac00700 1 --2- 172.21.15.131:0/395003630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c41020f0 0x7f49c4102570 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009990 tx=0x7f49b0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:39.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.389+0000 7f49cac00700 1 -- 172.21.15.131:0/395003630 shutdown_connections 2024-05-08T22:06:39.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.389+0000 7f49cac00700 1 --2- 172.21.15.131:0/395003630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49c4102ab0 0x7f49c41093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:39.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.389+0000 7f49cac00700 1 --2- 172.21.15.131:0/395003630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c4100f20 0x7f49c4101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:39.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.389+0000 7f49cac00700 1 --2- 172.21.15.131:0/395003630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c41020f0 0x7f49c4102570 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:39.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.389+0000 7f49cac00700 1 -- 172.21.15.131:0/395003630 >> 172.21.15.131:0/395003630 conn(0x7f49c40fc500 msgr2=0x7f49c40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:39.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.389+0000 7f49cac00700 1 -- 172.21.15.131:0/395003630 shutdown_connections 2024-05-08T22:06:39.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.389+0000 7f49cac00700 1 -- 172.21.15.131:0/395003630 wait complete. 2024-05-08T22:06:39.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.390+0000 7f49cac00700 1 Processor -- start 2024-05-08T22:06:39.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.391+0000 7f49cac00700 1 -- start start 2024-05-08T22:06:39.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.391+0000 7f49cac00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c4100f20 0x7f49c4074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:39.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.391+0000 7f49cac00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49c41020f0 0x7f49c4073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:39.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.391+0000 7f49cac00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c4102ab0 0x7f49c40738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:39.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.391+0000 7f49cac00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c4074070 con 0x7f49c4102ab0 2024-05-08T22:06:39.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.391+0000 7f49cac00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c40741e0 con 0x7f49c41020f0 2024-05-08T22:06:39.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.391+0000 7f49cac00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c41a0fb0 con 0x7f49c4100f20 2024-05-08T22:06:39.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.391+0000 7f49c3400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49c41020f0 0x7f49c4073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:39.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.391+0000 7f49c8c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c4102ab0 0x7f49c40738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:39.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.391+0000 7f49c3400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49c41020f0 0x7f49c4073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:53464/0 (socket says 172.21.15.131:53464) 2024-05-08T22:06:39.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.391+0000 7f49c3400700 1 -- 172.21.15.131:0/126792725 learned_addr learned my addr 172.21.15.131:0/126792725 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:39.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.391+0000 7f49c3e00700 1 --2- 172.21.15.131:0/126792725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c4100f20 0x7f49c4074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:39.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.393+0000 7f49c3400700 1 -- 172.21.15.131:0/126792725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c4100f20 msgr2=0x7f49c4074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:39.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.393+0000 7f49c3400700 1 --2- 172.21.15.131:0/126792725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c4100f20 0x7f49c4074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:39.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.393+0000 7f49c3400700 1 -- 172.21.15.131:0/126792725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c4102ab0 msgr2=0x7f49c40738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:39.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.393+0000 7f49c3400700 1 --2- 172.21.15.131:0/126792725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c4102ab0 0x7f49c40738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:39.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.393+0000 7f49c3400700 1 -- 172.21.15.131:0/126792725 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b00095f0 con 0x7f49c41020f0 2024-05-08T22:06:39.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.393+0000 7f49c8c00700 1 --2- 172.21.15.131:0/126792725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c4102ab0 0x7f49c40738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:39.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.393+0000 7f49c3e00700 1 --2- 172.21.15.131:0/126792725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c4100f20 0x7f49c4074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:39.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.393+0000 7f49c3400700 1 --2- 172.21.15.131:0/126792725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49c41020f0 0x7f49c4073390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f49b0000c00 tx=0x7f49b0005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:39.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.393+0000 7f49c0c00700 1 -- 172.21.15.131:0/126792725 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b001d070 con 0x7f49c41020f0 2024-05-08T22:06:39.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.393+0000 7f49c0c00700 1 -- 172.21.15.131:0/126792725 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49b0021410 con 0x7f49c41020f0 2024-05-08T22:06:39.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.393+0000 7f49cac00700 1 -- 172.21.15.131:0/126792725 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c41a11e0 con 0x7f49c41020f0 2024-05-08T22:06:39.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.393+0000 7f49c0c00700 1 -- 172.21.15.131:0/126792725 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b0017650 con 0x7f49c41020f0 2024-05-08T22:06:39.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.393+0000 7f49cac00700 1 -- 172.21.15.131:0/126792725 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c41a1600 con 0x7f49c41020f0 2024-05-08T22:06:39.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.393+0000 7f49cac00700 1 -- 172.21.15.131:0/126792725 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c41084b0 con 0x7f49c41020f0 2024-05-08T22:06:39.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.396+0000 7f49c0c00700 1 -- 172.21.15.131:0/126792725 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f49b0021580 con 0x7f49c41020f0 2024-05-08T22:06:39.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.398+0000 7f49c0c00700 1 --2- 172.21.15.131:0/126792725 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f49a80a1230 0x7f49a80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:39.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.398+0000 7f49c0c00700 1 -- 172.21.15.131:0/126792725 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f49b00ce780 con 0x7f49c41020f0 2024-05-08T22:06:39.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.398+0000 7f49c3e00700 1 --2- 172.21.15.131:0/126792725 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f49a80a1230 0x7f49a80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:39.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.399+0000 7f49c3e00700 1 --2- 172.21.15.131:0/126792725 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f49a80a1230 0x7f49a80a36f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f49c4100c40 tx=0x7f49b4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:39.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.400+0000 7f49c0c00700 1 -- 172.21.15.131:0/126792725 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f49b0097ad0 con 0x7f49c41020f0 2024-05-08T22:06:39.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.608+0000 7f49cac00700 1 -- 172.21.15.131:0/126792725 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f49c410c910 con 0x7f49c41020f0 2024-05-08T22:06:39.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.610+0000 7f49c0c00700 1 -- 172.21.15.131:0/126792725 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f49b009b0f0 con 0x7f49c41020f0 2024-05-08T22:06:39.611 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-08T22:06:39.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.613+0000 7f49cac00700 1 -- 172.21.15.131:0/126792725 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f49a80a1230 msgr2=0x7f49a80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:39.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.613+0000 7f49cac00700 1 --2- 172.21.15.131:0/126792725 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f49a80a1230 0x7f49a80a36f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f49c4100c40 tx=0x7f49b4009260 comp rx=0 tx=0).stop 2024-05-08T22:06:39.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.613+0000 7f49cac00700 1 -- 172.21.15.131:0/126792725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49c41020f0 msgr2=0x7f49c4073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:39.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.613+0000 7f49cac00700 1 --2- 172.21.15.131:0/126792725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49c41020f0 0x7f49c4073390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f49b0000c00 tx=0x7f49b0005740 comp rx=0 tx=0).stop 2024-05-08T22:06:39.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.613+0000 7f49cac00700 1 -- 172.21.15.131:0/126792725 shutdown_connections 2024-05-08T22:06:39.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.613+0000 7f49cac00700 1 --2- 172.21.15.131:0/126792725 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f49a80a1230 0x7f49a80a36f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:39.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.613+0000 7f49cac00700 1 --2- 172.21.15.131:0/126792725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c4102ab0 0x7f49c40738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:39.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.613+0000 7f49cac00700 1 --2- 172.21.15.131:0/126792725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c4100f20 0x7f49c4074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:39.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.613+0000 7f49cac00700 1 --2- 172.21.15.131:0/126792725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49c41020f0 0x7f49c4073390 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:39.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.613+0000 7f49cac00700 1 -- 172.21.15.131:0/126792725 >> 172.21.15.131:0/126792725 conn(0x7f49c40fc500 msgr2=0x7f49c41055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:39.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.613+0000 7f49cac00700 1 -- 172.21.15.131:0/126792725 shutdown_connections 2024-05-08T22:06:39.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.613+0000 7f49cac00700 1 -- 172.21.15.131:0/126792725 wait complete. 2024-05-08T22:06:39.633 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:06:39.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.830+0000 7f74ab000700 1 -- 172.21.15.131:0/250857831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74a4100f20 msgr2=0x7f74a4101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:39.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.830+0000 7f74ab000700 1 --2- 172.21.15.131:0/250857831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74a4100f20 0x7f74a4101360 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7494009990 tx=0x7f7494009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:39.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.830+0000 7f74ab000700 1 -- 172.21.15.131:0/250857831 shutdown_connections 2024-05-08T22:06:39.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.830+0000 7f74ab000700 1 --2- 172.21.15.131:0/250857831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74a4102ab0 0x7f74a41093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:39.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.830+0000 7f74ab000700 1 --2- 172.21.15.131:0/250857831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74a4100f20 0x7f74a4101360 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:39.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.830+0000 7f74ab000700 1 --2- 172.21.15.131:0/250857831 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74a41020f0 0x7f74a4102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:39.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.830+0000 7f74ab000700 1 -- 172.21.15.131:0/250857831 >> 172.21.15.131:0/250857831 conn(0x7f74a40fc500 msgr2=0x7f74a40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:39.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.830+0000 7f74ab000700 1 -- 172.21.15.131:0/250857831 shutdown_connections 2024-05-08T22:06:39.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.830+0000 7f74ab000700 1 -- 172.21.15.131:0/250857831 wait complete. 2024-05-08T22:06:39.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.831+0000 7f74ab000700 1 Processor -- start 2024-05-08T22:06:39.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.831+0000 7f74ab000700 1 -- start start 2024-05-08T22:06:39.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.831+0000 7f74ab000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74a4100f20 0x7f74a4074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:39.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.831+0000 7f74ab000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74a41020f0 0x7f74a4073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:39.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.831+0000 7f74ab000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74a4102ab0 0x7f74a40738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:39.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.831+0000 7f74ab000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a4074070 con 0x7f74a41020f0 2024-05-08T22:06:39.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.831+0000 7f74ab000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a40741e0 con 0x7f74a4100f20 2024-05-08T22:06:39.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.831+0000 7f74ab000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a41a0fd0 con 0x7f74a4102ab0 2024-05-08T22:06:39.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.832+0000 7f74a3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74a41020f0 0x7f74a4073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:39.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.832+0000 7f74a3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74a41020f0 0x7f74a4073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40374/0 (socket says 172.21.15.131:40374) 2024-05-08T22:06:39.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.832+0000 7f74a3e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74a4100f20 0x7f74a4074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:39.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.832+0000 7f74a3400700 1 -- 172.21.15.131:0/3616942847 learned_addr learned my addr 172.21.15.131:0/3616942847 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:39.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.832+0000 7f74a9000700 1 --2- 172.21.15.131:0/3616942847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74a4102ab0 0x7f74a40738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:39.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.832+0000 7f74a3400700 1 -- 172.21.15.131:0/3616942847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74a4102ab0 msgr2=0x7f74a40738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:39.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.832+0000 7f74a3400700 1 --2- 172.21.15.131:0/3616942847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74a4102ab0 0x7f74a40738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:39.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.832+0000 7f74a3400700 1 -- 172.21.15.131:0/3616942847 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74a4100f20 msgr2=0x7f74a4074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:39.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.832+0000 7f74a3400700 1 --2- 172.21.15.131:0/3616942847 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74a4100f20 0x7f74a4074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:39.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.832+0000 7f74a3400700 1 -- 172.21.15.131:0/3616942847 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7490009520 con 0x7f74a41020f0 2024-05-08T22:06:39.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.832+0000 7f74a3e00700 1 --2- 172.21.15.131:0/3616942847 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74a4100f20 0x7f74a4074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:39.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.832+0000 7f74a9000700 1 --2- 172.21.15.131:0/3616942847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74a4102ab0 0x7f74a40738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:06:39.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.832+0000 7f74a3400700 1 --2- 172.21.15.131:0/3616942847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74a41020f0 0x7f74a4073390 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f749000eb30 tx=0x7f749000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:39.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.832+0000 7f74a0c00700 1 -- 172.21.15.131:0/3616942847 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7490009880 con 0x7f74a41020f0 2024-05-08T22:06:39.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.833+0000 7f74a0c00700 1 -- 172.21.15.131:0/3616942847 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74900044e0 con 0x7f74a41020f0 2024-05-08T22:06:39.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.833+0000 7f74ab000700 1 -- 172.21.15.131:0/3616942847 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74940095f0 con 0x7f74a41020f0 2024-05-08T22:06:39.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.833+0000 7f74a0c00700 1 -- 172.21.15.131:0/3616942847 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7490003df0 con 0x7f74a41020f0 2024-05-08T22:06:39.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.833+0000 7f74ab000700 1 -- 172.21.15.131:0/3616942847 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a41a15e0 con 0x7f74a41020f0 2024-05-08T22:06:39.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.834+0000 7f74ab000700 1 -- 172.21.15.131:0/3616942847 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74a4108400 con 0x7f74a41020f0 2024-05-08T22:06:39.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.835+0000 7f74a0c00700 1 -- 172.21.15.131:0/3616942847 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f7490004000 con 0x7f74a41020f0 2024-05-08T22:06:39.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.837+0000 7f74a0c00700 1 --2- 172.21.15.131:0/3616942847 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f74880a11b0 0x7f74880a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:39.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.837+0000 7f74a0c00700 1 -- 172.21.15.131:0/3616942847 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7490014070 con 0x7f74a41020f0 2024-05-08T22:06:39.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.837+0000 7f74a3e00700 1 --2- 172.21.15.131:0/3616942847 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f74880a11b0 0x7f74880a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:39.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.838+0000 7f74a3e00700 1 --2- 172.21.15.131:0/3616942847 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f74880a11b0 0x7f74880a3670 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f749400b620 tx=0x7f7494005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:39.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:39.840+0000 7f74a0c00700 1 -- 172.21.15.131:0/3616942847 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7490096500 con 0x7f74a41020f0 2024-05-08T22:06:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.051+0000 7f74ab000700 1 -- 172.21.15.131:0/3616942847 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f74a410c7f0 con 0x7f74a41020f0 2024-05-08T22:06:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.051+0000 7f74a0c00700 1 -- 172.21.15.131:0/3616942847 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7490099b20 con 0x7f74a41020f0 2024-05-08T22:06:40.052 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-08T22:06:40.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.054+0000 7f74ab000700 1 -- 172.21.15.131:0/3616942847 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f74880a11b0 msgr2=0x7f74880a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:40.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.054+0000 7f74ab000700 1 --2- 172.21.15.131:0/3616942847 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f74880a11b0 0x7f74880a3670 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f749400b620 tx=0x7f7494005fd0 comp rx=0 tx=0).stop 2024-05-08T22:06:40.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.054+0000 7f74ab000700 1 -- 172.21.15.131:0/3616942847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74a41020f0 msgr2=0x7f74a4073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:40.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.054+0000 7f74ab000700 1 --2- 172.21.15.131:0/3616942847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74a41020f0 0x7f74a4073390 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f749000eb30 tx=0x7f749000ee40 comp rx=0 tx=0).stop 2024-05-08T22:06:40.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.054+0000 7f74ab000700 1 -- 172.21.15.131:0/3616942847 shutdown_connections 2024-05-08T22:06:40.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.054+0000 7f74ab000700 1 --2- 172.21.15.131:0/3616942847 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f74880a11b0 0x7f74880a3670 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:40.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.054+0000 7f74ab000700 1 --2- 172.21.15.131:0/3616942847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74a4102ab0 0x7f74a40738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:40.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.054+0000 7f74ab000700 1 --2- 172.21.15.131:0/3616942847 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74a4100f20 0x7f74a4074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:40.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.054+0000 7f74ab000700 1 --2- 172.21.15.131:0/3616942847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74a41020f0 0x7f74a4073390 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:40.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.054+0000 7f74ab000700 1 -- 172.21.15.131:0/3616942847 >> 172.21.15.131:0/3616942847 conn(0x7f74a40fc500 msgr2=0x7f74a4105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:40.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.054+0000 7f74ab000700 1 -- 172.21.15.131:0/3616942847 shutdown_connections 2024-05-08T22:06:40.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.054+0000 7f74ab000700 1 -- 172.21.15.131:0/3616942847 wait complete. 2024-05-08T22:06:40.074 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:06:40.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.269+0000 7fd1dee00700 1 -- 172.21.15.131:0/2160628070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1d8073a00 msgr2=0x7fd1d8073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.269+0000 7fd1dee00700 1 --2- 172.21.15.131:0/2160628070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1d8073a00 0x7fd1d8073e80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4009990 tx=0x7fd1c4009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.269+0000 7fd1dee00700 1 -- 172.21.15.131:0/2160628070 shutdown_connections 2024-05-08T22:06:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.269+0000 7fd1dee00700 1 --2- 172.21.15.131:0/2160628070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1d8108c70 0x7fd1d810b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.269+0000 7fd1dee00700 1 --2- 172.21.15.131:0/2160628070 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1d8074fe0 0x7fd1d8073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.269+0000 7fd1dee00700 1 --2- 172.21.15.131:0/2160628070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1d8073a00 0x7fd1d8073e80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.269+0000 7fd1dee00700 1 -- 172.21.15.131:0/2160628070 >> 172.21.15.131:0/2160628070 conn(0x7fd1d80fc2e0 msgr2=0x7fd1d80fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.269+0000 7fd1dee00700 1 -- 172.21.15.131:0/2160628070 shutdown_connections 2024-05-08T22:06:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.269+0000 7fd1dee00700 1 -- 172.21.15.131:0/2160628070 wait complete. 2024-05-08T22:06:40.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.270+0000 7fd1dee00700 1 Processor -- start 2024-05-08T22:06:40.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.270+0000 7fd1dee00700 1 -- start start 2024-05-08T22:06:40.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.271+0000 7fd1dee00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1d8073a00 0x7fd1d8198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:40.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.271+0000 7fd1dee00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1d8074fe0 0x7fd1d81994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:40.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.271+0000 7fd1dee00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1d8108c70 0x7fd1d81a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:40.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.271+0000 7fd1dee00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d81a0ae0 con 0x7fd1d8074fe0 2024-05-08T22:06:40.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.271+0000 7fd1dee00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d81a0c20 con 0x7fd1d8108c70 2024-05-08T22:06:40.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.271+0000 7fd1dee00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d81a0db0 con 0x7fd1d8073a00 2024-05-08T22:06:40.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.271+0000 7fd1dd000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1d8074fe0 0x7fd1d81994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:40.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.271+0000 7fd1dda00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1d8073a00 0x7fd1d8198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:40.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.271+0000 7fd1dd000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1d8074fe0 0x7fd1d81994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40380/0 (socket says 172.21.15.131:40380) 2024-05-08T22:06:40.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.271+0000 7fd1dd000700 1 -- 172.21.15.131:0/1426625498 learned_addr learned my addr 172.21.15.131:0/1426625498 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:40.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.271+0000 7fd1de400700 1 --2- 172.21.15.131:0/1426625498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1d8108c70 0x7fd1d81a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:40.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.271+0000 7fd1dd000700 1 -- 172.21.15.131:0/1426625498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1d8073a00 msgr2=0x7fd1d8198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:40.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.271+0000 7fd1dd000700 1 --2- 172.21.15.131:0/1426625498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1d8073a00 0x7fd1d8198fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:40.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.271+0000 7fd1dd000700 1 -- 172.21.15.131:0/1426625498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1d8108c70 msgr2=0x7fd1d81a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:40.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.271+0000 7fd1dd000700 1 --2- 172.21.15.131:0/1426625498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1d8108c70 0x7fd1d81a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:40.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.271+0000 7fd1dd000700 1 -- 172.21.15.131:0/1426625498 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c40095f0 con 0x7fd1d8074fe0 2024-05-08T22:06:40.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.271+0000 7fd1de400700 1 --2- 172.21.15.131:0/1426625498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1d8108c70 0x7fd1d81a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:06:40.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.272+0000 7fd1dd000700 1 --2- 172.21.15.131:0/1426625498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1d8074fe0 0x7fd1d81994f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4000c00 tx=0x7fd1c4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:40.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.272+0000 7fd1d6000700 1 -- 172.21.15.131:0/1426625498 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c401d070 con 0x7fd1d8074fe0 2024-05-08T22:06:40.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.272+0000 7fd1d6000700 1 -- 172.21.15.131:0/1426625498 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1c4021410 con 0x7fd1d8074fe0 2024-05-08T22:06:40.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.272+0000 7fd1d6000700 1 -- 172.21.15.131:0/1426625498 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4017650 con 0x7fd1d8074fe0 2024-05-08T22:06:40.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.272+0000 7fd1dee00700 1 -- 172.21.15.131:0/1426625498 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d81a1030 con 0x7fd1d8074fe0 2024-05-08T22:06:40.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.272+0000 7fd1dee00700 1 -- 172.21.15.131:0/1426625498 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d81a1450 con 0x7fd1d8074fe0 2024-05-08T22:06:40.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.273+0000 7fd1dee00700 1 -- 172.21.15.131:0/1426625498 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1d8107a30 con 0x7fd1d8074fe0 2024-05-08T22:06:40.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.275+0000 7fd1d6000700 1 -- 172.21.15.131:0/1426625498 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fd1c4021580 con 0x7fd1d8074fe0 2024-05-08T22:06:40.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.276+0000 7fd1d6000700 1 --2- 172.21.15.131:0/1426625498 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd1c00a1230 0x7fd1c00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:40.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.276+0000 7fd1d6000700 1 -- 172.21.15.131:0/1426625498 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd1c40ce260 con 0x7fd1d8074fe0 2024-05-08T22:06:40.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.276+0000 7fd1dda00700 1 --2- 172.21.15.131:0/1426625498 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd1c00a1230 0x7fd1c00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:40.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:39 smithi157 ceph-mon[77746]: pgmap v284: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:06:40.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:39 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2007776974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:06:40.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:39 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/126792725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:06:40.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.277+0000 7fd1dda00700 1 --2- 172.21.15.131:0/1426625498 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd1c00a1230 0x7fd1c00a36f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc009c60 tx=0x7fd1cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:40.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.280+0000 7fd1d6000700 1 -- 172.21.15.131:0/1426625498 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd1c40975b0 con 0x7fd1d8074fe0 2024-05-08T22:06:40.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:39 smithi132 ceph-mon[78180]: pgmap v284: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:06:40.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:39 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2007776974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:06:40.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:39 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/126792725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:06:40.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:39 smithi131 ceph-mon[37260]: pgmap v284: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:06:40.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:39 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2007776974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:06:40.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:39 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/126792725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:06:40.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.485+0000 7fd1dee00700 1 -- 172.21.15.131:0/1426625498 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd1d810e7f0 con 0x7fd1d8074fe0 2024-05-08T22:06:40.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.485+0000 7fd1d6000700 1 -- 172.21.15.131:0/1426625498 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v85) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd1c4021860 con 0x7fd1d8074fe0 2024-05-08T22:06:40.485 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-08T22:06:40.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.488+0000 7fd1dee00700 1 -- 172.21.15.131:0/1426625498 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd1c00a1230 msgr2=0x7fd1c00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:40.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.488+0000 7fd1dee00700 1 --2- 172.21.15.131:0/1426625498 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd1c00a1230 0x7fd1c00a36f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc009c60 tx=0x7fd1cc009260 comp rx=0 tx=0).stop 2024-05-08T22:06:40.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.488+0000 7fd1dee00700 1 -- 172.21.15.131:0/1426625498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1d8074fe0 msgr2=0x7fd1d81994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:40.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.488+0000 7fd1dee00700 1 --2- 172.21.15.131:0/1426625498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1d8074fe0 0x7fd1d81994f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4000c00 tx=0x7fd1c4005740 comp rx=0 tx=0).stop 2024-05-08T22:06:40.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.488+0000 7fd1dee00700 1 -- 172.21.15.131:0/1426625498 shutdown_connections 2024-05-08T22:06:40.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.488+0000 7fd1dee00700 1 --2- 172.21.15.131:0/1426625498 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd1c00a1230 0x7fd1c00a36f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:40.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.488+0000 7fd1dee00700 1 --2- 172.21.15.131:0/1426625498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1d8108c70 0x7fd1d81a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:40.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.488+0000 7fd1dee00700 1 --2- 172.21.15.131:0/1426625498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1d8073a00 0x7fd1d8198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:40.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.488+0000 7fd1dee00700 1 --2- 172.21.15.131:0/1426625498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1d8074fe0 0x7fd1d81994f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:40.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.488+0000 7fd1dee00700 1 -- 172.21.15.131:0/1426625498 >> 172.21.15.131:0/1426625498 conn(0x7fd1d80fc2e0 msgr2=0x7fd1d8104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:40.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.488+0000 7fd1dee00700 1 -- 172.21.15.131:0/1426625498 shutdown_connections 2024-05-08T22:06:40.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.488+0000 7fd1dee00700 1 -- 172.21.15.131:0/1426625498 wait complete. 2024-05-08T22:06:40.508 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:06:40.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.701+0000 7fad0b400700 1 -- 172.21.15.131:0/2358979362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad04101fd0 msgr2=0x7fad04102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:40.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.701+0000 7fad0b400700 1 --2- 172.21.15.131:0/2358979362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad04101fd0 0x7fad04102450 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7facf0009990 tx=0x7facf0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:40.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.701+0000 7fad0b400700 1 -- 172.21.15.131:0/2358979362 shutdown_connections 2024-05-08T22:06:40.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.701+0000 7fad0b400700 1 --2- 172.21.15.131:0/2358979362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad04102990 0x7fad041092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:40.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.702+0000 7fad0b400700 1 --2- 172.21.15.131:0/2358979362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad04100e80 0x7fad041012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:40.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.702+0000 7fad0b400700 1 --2- 172.21.15.131:0/2358979362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad04101fd0 0x7fad04102450 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:40.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.702+0000 7fad0b400700 1 -- 172.21.15.131:0/2358979362 >> 172.21.15.131:0/2358979362 conn(0x7fad040fc410 msgr2=0x7fad040fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:40.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.702+0000 7fad0b400700 1 -- 172.21.15.131:0/2358979362 shutdown_connections 2024-05-08T22:06:40.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.702+0000 7fad0b400700 1 -- 172.21.15.131:0/2358979362 wait complete. 2024-05-08T22:06:40.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.703+0000 7fad0b400700 1 Processor -- start 2024-05-08T22:06:40.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.703+0000 7fad0b400700 1 -- start start 2024-05-08T22:06:40.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.703+0000 7fad0b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad04100e80 0x7fad04074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:40.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.703+0000 7fad0b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad04101fd0 0x7fad04073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:40.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.703+0000 7fad0b400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad04102990 0x7fad040738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:40.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.703+0000 7fad0b400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad04074070 con 0x7fad04100e80 2024-05-08T22:06:40.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.703+0000 7fad0b400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad040741e0 con 0x7fad04102990 2024-05-08T22:06:40.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.703+0000 7fad0b400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad041a0ec0 con 0x7fad04101fd0 2024-05-08T22:06:40.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.703+0000 7fad08a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad04100e80 0x7fad04074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:40.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.704+0000 7fad03e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad04101fd0 0x7fad04073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:40.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.704+0000 7fad09400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad04102990 0x7fad040738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:40.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.703+0000 7fad08a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad04100e80 0x7fad04074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40404/0 (socket says 172.21.15.131:40404) 2024-05-08T22:06:40.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.704+0000 7fad08a00700 1 -- 172.21.15.131:0/3894575238 learned_addr learned my addr 172.21.15.131:0/3894575238 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:40.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.704+0000 7fad03e00700 1 -- 172.21.15.131:0/3894575238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad04102990 msgr2=0x7fad040738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:40.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.704+0000 7fad03e00700 1 --2- 172.21.15.131:0/3894575238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad04102990 0x7fad040738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:40.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.704+0000 7fad03e00700 1 -- 172.21.15.131:0/3894575238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad04100e80 msgr2=0x7fad04074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:40.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.704+0000 7fad03e00700 1 --2- 172.21.15.131:0/3894575238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad04100e80 0x7fad04074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:40.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.704+0000 7fad03e00700 1 -- 172.21.15.131:0/3894575238 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facfc009c40 con 0x7fad04101fd0 2024-05-08T22:06:40.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.704+0000 7fad08a00700 1 --2- 172.21.15.131:0/3894575238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad04100e80 0x7fad04074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:40.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.704+0000 7fad03e00700 1 --2- 172.21.15.131:0/3894575238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad04101fd0 0x7fad04073390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7facf000b620 tx=0x7facf0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:40.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.704+0000 7fad01600700 1 -- 172.21.15.131:0/3894575238 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf001d070 con 0x7fad04101fd0 2024-05-08T22:06:40.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.705+0000 7fad0b400700 1 -- 172.21.15.131:0/3894575238 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf00095f0 con 0x7fad04101fd0 2024-05-08T22:06:40.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.705+0000 7fad01600700 1 -- 172.21.15.131:0/3894575238 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facf0021410 con 0x7fad04101fd0 2024-05-08T22:06:40.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.705+0000 7fad01600700 1 -- 172.21.15.131:0/3894575238 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf0017650 con 0x7fad04101fd0 2024-05-08T22:06:40.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.705+0000 7fad0b400700 1 -- 172.21.15.131:0/3894575238 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad041a1330 con 0x7fad04101fd0 2024-05-08T22:06:40.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.705+0000 7fad0b400700 1 -- 172.21.15.131:0/3894575238 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad041082f0 con 0x7fad04101fd0 2024-05-08T22:06:40.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.708+0000 7fad01600700 1 -- 172.21.15.131:0/3894575238 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7facf0021580 con 0x7fad04101fd0 2024-05-08T22:06:40.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.709+0000 7fad01600700 1 --2- 172.21.15.131:0/3894575238 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7facec0a11b0 0x7facec0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:40.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.709+0000 7fad01600700 1 -- 172.21.15.131:0/3894575238 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7facf00ce7f0 con 0x7fad04101fd0 2024-05-08T22:06:40.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.709+0000 7fad08a00700 1 --2- 172.21.15.131:0/3894575238 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7facec0a11b0 0x7facec0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:40.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.710+0000 7fad08a00700 1 --2- 172.21.15.131:0/3894575238 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7facec0a11b0 0x7facec0a3670 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7facf8011590 tx=0x7facf8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:40.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.711+0000 7fad01600700 1 -- 172.21.15.131:0/3894575238 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7facf0097b40 con 0x7fad04101fd0 2024-05-08T22:06:40.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.968+0000 7fad0b400700 1 -- 172.21.15.131:0/3894575238 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad04067c10 con 0x7fad04101fd0 2024-05-08T22:06:40.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.969+0000 7fad01600700 1 -- 172.21.15.131:0/3894575238 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10889 (secure 0 0 0) 0x7facf009b160 con 0x7fad04101fd0 2024-05-08T22:06:40.969 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:06:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:05:14.034908+0000","modified":"2024-05-08T22:05:39.494566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:06:40.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.972+0000 7fad0b400700 1 -- 172.21.15.131:0/3894575238 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7facec0a11b0 msgr2=0x7facec0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:40.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.972+0000 7fad0b400700 1 --2- 172.21.15.131:0/3894575238 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7facec0a11b0 0x7facec0a3670 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7facf8011590 tx=0x7facf8009260 comp rx=0 tx=0).stop 2024-05-08T22:06:40.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.972+0000 7fad0b400700 1 -- 172.21.15.131:0/3894575238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad04101fd0 msgr2=0x7fad04073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:40.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.972+0000 7fad0b400700 1 --2- 172.21.15.131:0/3894575238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad04101fd0 0x7fad04073390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7facf000b620 tx=0x7facf0004730 comp rx=0 tx=0).stop 2024-05-08T22:06:40.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.972+0000 7fad0b400700 1 -- 172.21.15.131:0/3894575238 shutdown_connections 2024-05-08T22:06:40.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.972+0000 7fad0b400700 1 --2- 172.21.15.131:0/3894575238 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7facec0a11b0 0x7facec0a3670 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:40.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.972+0000 7fad0b400700 1 --2- 172.21.15.131:0/3894575238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad04102990 0x7fad040738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:40.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.972+0000 7fad0b400700 1 --2- 172.21.15.131:0/3894575238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad04100e80 0x7fad04074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:40.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.972+0000 7fad0b400700 1 --2- 172.21.15.131:0/3894575238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad04101fd0 0x7fad04073390 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:40.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.972+0000 7fad0b400700 1 -- 172.21.15.131:0/3894575238 >> 172.21.15.131:0/3894575238 conn(0x7fad040fc410 msgr2=0x7fad04105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:40.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.972+0000 7fad0b400700 1 -- 172.21.15.131:0/3894575238 shutdown_connections 2024-05-08T22:06:40.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:40.972+0000 7fad0b400700 1 -- 172.21.15.131:0/3894575238 wait complete. 2024-05-08T22:06:40.973 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 23 2024-05-08T22:06:40.997 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:06:41.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.191+0000 7f8103600700 1 -- 172.21.15.131:0/1641933557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80fc104210 msgr2=0x7f80fc106600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:41.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.191+0000 7f8103600700 1 --2- 172.21.15.131:0/1641933557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80fc104210 0x7f80fc106600 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f80e80098f0 tx=0x7f80e8009c00 comp rx=0 tx=0).stop 2024-05-08T22:06:41.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.191+0000 7f8103600700 1 -- 172.21.15.131:0/1641933557 shutdown_connections 2024-05-08T22:06:41.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.191+0000 7f8103600700 1 --2- 172.21.15.131:0/1641933557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80fc106b40 0x7f80fc108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:41.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.191+0000 7f8103600700 1 --2- 172.21.15.131:0/1641933557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80fc1018e0 0x7f80fc103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:41.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.191+0000 7f8103600700 1 --2- 172.21.15.131:0/1641933557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80fc104210 0x7f80fc106600 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:41.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.191+0000 7f8103600700 1 -- 172.21.15.131:0/1641933557 >> 172.21.15.131:0/1641933557 conn(0x7f80fc0fb2f0 msgr2=0x7f80fc0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:41.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.191+0000 7f8103600700 1 -- 172.21.15.131:0/1641933557 shutdown_connections 2024-05-08T22:06:41.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.191+0000 7f8103600700 1 -- 172.21.15.131:0/1641933557 wait complete. 2024-05-08T22:06:41.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.192+0000 7f8103600700 1 Processor -- start 2024-05-08T22:06:41.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.192+0000 7f8103600700 1 -- start start 2024-05-08T22:06:41.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.193+0000 7f8103600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80fc1018e0 0x7f80fc199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:41.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.193+0000 7f8103600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80fc104210 0x7f80fc199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:41.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.193+0000 7f8103600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80fc106b40 0x7f80fc1a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:41.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.193+0000 7f8103600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc1a0b70 con 0x7f80fc106b40 2024-05-08T22:06:41.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.193+0000 7f8103600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc1a0cb0 con 0x7f80fc1018e0 2024-05-08T22:06:41.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.193+0000 7f8103600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc1a0e40 con 0x7f80fc104210 2024-05-08T22:06:41.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.193+0000 7f8101600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80fc106b40 0x7f80fc1a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:41.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.193+0000 7f8100c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80fc1018e0 0x7f80fc199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:41.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.193+0000 7f8100c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80fc1018e0 0x7f80fc199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:53568/0 (socket says 172.21.15.131:53568) 2024-05-08T22:06:41.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.193+0000 7f8101600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80fc106b40 0x7f80fc1a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40418/0 (socket says 172.21.15.131:40418) 2024-05-08T22:06:41.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.193+0000 7f8100c00700 1 -- 172.21.15.131:0/2827778284 learned_addr learned my addr 172.21.15.131:0/2827778284 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:41.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.193+0000 7f80fbe00700 1 --2- 172.21.15.131:0/2827778284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80fc104210 0x7f80fc199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:41.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.193+0000 7f8100c00700 1 -- 172.21.15.131:0/2827778284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80fc104210 msgr2=0x7f80fc199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:41.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.193+0000 7f8100c00700 1 --2- 172.21.15.131:0/2827778284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80fc104210 0x7f80fc199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:41.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.193+0000 7f8100c00700 1 -- 172.21.15.131:0/2827778284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80fc106b40 msgr2=0x7f80fc1a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:41.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.193+0000 7f8100c00700 1 --2- 172.21.15.131:0/2827778284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80fc106b40 0x7f80fc1a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:41.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.193+0000 7f8100c00700 1 -- 172.21.15.131:0/2827778284 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f00095f0 con 0x7f80fc1018e0 2024-05-08T22:06:41.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.194+0000 7f8100c00700 1 --2- 172.21.15.131:0/2827778284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80fc1018e0 0x7f80fc199040 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f80f000ec10 tx=0x7f80f000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:41.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.194+0000 7f80f9600700 1 -- 172.21.15.131:0/2827778284 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f000cc90 con 0x7f80fc1018e0 2024-05-08T22:06:41.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.194+0000 7f80f9600700 1 -- 172.21.15.131:0/2827778284 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80f00044e0 con 0x7f80fc1018e0 2024-05-08T22:06:41.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.194+0000 7f8103600700 1 -- 172.21.15.131:0/2827778284 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e8009520 con 0x7f80fc1018e0 2024-05-08T22:06:41.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.194+0000 7f80f9600700 1 -- 172.21.15.131:0/2827778284 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f0010430 con 0x7f80fc1018e0 2024-05-08T22:06:41.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.194+0000 7f8103600700 1 -- 172.21.15.131:0/2827778284 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80fc1a14a0 con 0x7f80fc1018e0 2024-05-08T22:06:41.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.195+0000 7f8103600700 1 -- 172.21.15.131:0/2827778284 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80fc108710 con 0x7f80fc1018e0 2024-05-08T22:06:41.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.197+0000 7f80f9600700 1 -- 172.21.15.131:0/2827778284 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f80f0003f90 con 0x7f80fc1018e0 2024-05-08T22:06:41.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.198+0000 7f80f9600700 1 --2- 172.21.15.131:0/2827778284 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f80e40a11b0 0x7f80e40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:41.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.198+0000 7f80f9600700 1 -- 172.21.15.131:0/2827778284 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f80f0014070 con 0x7f80fc1018e0 2024-05-08T22:06:41.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.198+0000 7f80fbe00700 1 --2- 172.21.15.131:0/2827778284 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f80e40a11b0 0x7f80e40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:41.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.199+0000 7f80fbe00700 1 --2- 172.21.15.131:0/2827778284 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f80e40a11b0 0x7f80e40a3670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f80e800fa40 tx=0x7f80e8019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:41.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.201+0000 7f80f9600700 1 -- 172.21.15.131:0/2827778284 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f80f0096760 con 0x7f80fc1018e0 2024-05-08T22:06:41.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:40 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3616942847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:06:41.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:40 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1426625498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:06:41.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-05-08T22:06:41.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-05-08T22:06:41.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-05-08T22:06:41.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:40 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 7]}]: dispatch 2024-05-08T22:06:41.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:40 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3894575238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:06:41.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:40 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3616942847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:06:41.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:40 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1426625498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:06:41.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-05-08T22:06:41.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-05-08T22:06:41.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-05-08T22:06:41.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:40 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 7]}]: dispatch 2024-05-08T22:06:41.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:40 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3894575238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:06:41.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:40 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3616942847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:06:41.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:40 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1426625498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:06:41.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-05-08T22:06:41.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-05-08T22:06:41.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-05-08T22:06:41.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:40 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 7]}]: dispatch 2024-05-08T22:06:41.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:40 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3894575238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:06:41.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.412+0000 7f8103600700 1 -- 172.21.15.131:0/2827778284 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80fc067c10 con 0x7f80fc1018e0 2024-05-08T22:06:41.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.414+0000 7f80f9600700 1 -- 172.21.15.131:0/2827778284 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24721 (secure 0 0 0) 0x7f80f00a0070 con 0x7f80fc1018e0 2024-05-08T22:06:41.415 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:06:41.415 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":86,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:06:40.970205+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:05:12.469858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:05:13.484446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:05:17.016186+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:06:41.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.417+0000 7f8103600700 1 -- 172.21.15.131:0/2827778284 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f80e40a11b0 msgr2=0x7f80e40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:41.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.417+0000 7f8103600700 1 --2- 172.21.15.131:0/2827778284 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f80e40a11b0 0x7f80e40a3670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f80e800fa40 tx=0x7f80e8019040 comp rx=0 tx=0).stop 2024-05-08T22:06:41.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.417+0000 7f8103600700 1 -- 172.21.15.131:0/2827778284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80fc1018e0 msgr2=0x7f80fc199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:41.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.417+0000 7f8103600700 1 --2- 172.21.15.131:0/2827778284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80fc1018e0 0x7f80fc199040 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f80f000ec10 tx=0x7f80f000ef20 comp rx=0 tx=0).stop 2024-05-08T22:06:41.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.417+0000 7f8103600700 1 -- 172.21.15.131:0/2827778284 shutdown_connections 2024-05-08T22:06:41.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.417+0000 7f8103600700 1 --2- 172.21.15.131:0/2827778284 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f80e40a11b0 0x7f80e40a3670 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:41.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.417+0000 7f8103600700 1 --2- 172.21.15.131:0/2827778284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80fc106b40 0x7f80fc1a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:41.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.417+0000 7f8103600700 1 --2- 172.21.15.131:0/2827778284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80fc1018e0 0x7f80fc199040 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:41.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.417+0000 7f8103600700 1 --2- 172.21.15.131:0/2827778284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80fc104210 0x7f80fc199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:41.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.417+0000 7f8103600700 1 -- 172.21.15.131:0/2827778284 >> 172.21.15.131:0/2827778284 conn(0x7f80fc0fb2f0 msgr2=0x7f80fc101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:41.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.417+0000 7f8103600700 1 -- 172.21.15.131:0/2827778284 shutdown_connections 2024-05-08T22:06:41.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.417+0000 7f8103600700 1 -- 172.21.15.131:0/2827778284 wait complete. 2024-05-08T22:06:41.436 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:06:41.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.633+0000 7f6ce3000700 1 -- 172.21.15.131:0/2968624560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cdc1019a0 msgr2=0x7f6cdc103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:41.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.633+0000 7f6ce3000700 1 --2- 172.21.15.131:0/2968624560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cdc1019a0 0x7f6cdc103d90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0098f0 tx=0x7f6ccc009c00 comp rx=0 tx=0).stop 2024-05-08T22:06:41.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.633+0000 7f6ce3000700 1 -- 172.21.15.131:0/2968624560 shutdown_connections 2024-05-08T22:06:41.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.633+0000 7f6ce3000700 1 --2- 172.21.15.131:0/2968624560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cdc106c00 0x7f6cdc108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:41.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.633+0000 7f6ce3000700 1 --2- 172.21.15.131:0/2968624560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cdc1019a0 0x7f6cdc103d90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:41.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.633+0000 7f6ce3000700 1 --2- 172.21.15.131:0/2968624560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6cdc1042d0 0x7f6cdc1066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:41.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.633+0000 7f6ce3000700 1 -- 172.21.15.131:0/2968624560 >> 172.21.15.131:0/2968624560 conn(0x7f6cdc0fb2f0 msgr2=0x7f6cdc0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:41.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.633+0000 7f6ce3000700 1 -- 172.21.15.131:0/2968624560 shutdown_connections 2024-05-08T22:06:41.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.633+0000 7f6ce3000700 1 -- 172.21.15.131:0/2968624560 wait complete. 2024-05-08T22:06:41.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.634+0000 7f6ce3000700 1 Processor -- start 2024-05-08T22:06:41.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.634+0000 7f6ce3000700 1 -- start start 2024-05-08T22:06:41.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.635+0000 7f6ce3000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6cdc1019a0 0x7f6cdc199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:41.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.635+0000 7f6ce3000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cdc1042d0 0x7f6cdc1995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:41.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.635+0000 7f6ce3000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cdc106c00 0x7f6cdc1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:41.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.635+0000 7f6ce3000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc1a0bb0 con 0x7f6cdc1042d0 2024-05-08T22:06:41.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.635+0000 7f6ce3000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc1a0cf0 con 0x7f6cdc1019a0 2024-05-08T22:06:41.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.635+0000 7f6ce3000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc1a0e80 con 0x7f6cdc106c00 2024-05-08T22:06:41.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.635+0000 7f6cdb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cdc1042d0 0x7f6cdc1995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:41.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.635+0000 7f6cdb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cdc1042d0 0x7f6cdc1995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40444/0 (socket says 172.21.15.131:40444) 2024-05-08T22:06:41.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.635+0000 7f6cdb400700 1 -- 172.21.15.131:0/1719724057 learned_addr learned my addr 172.21.15.131:0/1719724057 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:41.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.635+0000 7f6ce1000700 1 --2- 172.21.15.131:0/1719724057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cdc106c00 0x7f6cdc1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:41.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.635+0000 7f6cdbe00700 1 --2- 172.21.15.131:0/1719724057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6cdc1019a0 0x7f6cdc199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:41.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.635+0000 7f6cdb400700 1 -- 172.21.15.131:0/1719724057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cdc106c00 msgr2=0x7f6cdc1a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:41.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.635+0000 7f6cdb400700 1 --2- 172.21.15.131:0/1719724057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cdc106c00 0x7f6cdc1a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:41.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.635+0000 7f6cdb400700 1 -- 172.21.15.131:0/1719724057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6cdc1019a0 msgr2=0x7f6cdc199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:41.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.635+0000 7f6cdb400700 1 --2- 172.21.15.131:0/1719724057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6cdc1019a0 0x7f6cdc199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:41.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.635+0000 7f6cdb400700 1 -- 172.21.15.131:0/1719724057 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd4009d00 con 0x7f6cdc1042d0 2024-05-08T22:06:41.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.636+0000 7f6cdb400700 1 --2- 172.21.15.131:0/1719724057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cdc1042d0 0x7f6cdc1995c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800e9c0 tx=0x7f6cc800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:41.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.636+0000 7f6cd8c00700 1 -- 172.21.15.131:0/1719724057 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc80097b0 con 0x7f6cdc1042d0 2024-05-08T22:06:41.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.636+0000 7f6cd8c00700 1 -- 172.21.15.131:0/1719724057 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cc800ee60 con 0x7f6cdc1042d0 2024-05-08T22:06:41.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.636+0000 7f6ce3000700 1 -- 172.21.15.131:0/1719724057 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ccc009520 con 0x7f6cdc1042d0 2024-05-08T22:06:41.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.636+0000 7f6cd8c00700 1 -- 172.21.15.131:0/1719724057 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc80054f0 con 0x7f6cdc1042d0 2024-05-08T22:06:41.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.636+0000 7f6ce3000700 1 -- 172.21.15.131:0/1719724057 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cdc1a1490 con 0x7f6cdc1042d0 2024-05-08T22:06:41.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.637+0000 7f6ce3000700 1 -- 172.21.15.131:0/1719724057 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cdc108780 con 0x7f6cdc1042d0 2024-05-08T22:06:41.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.639+0000 7f6cd8c00700 1 -- 172.21.15.131:0/1719724057 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f6cc8004750 con 0x7f6cdc1042d0 2024-05-08T22:06:41.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.640+0000 7f6cd8c00700 1 --2- 172.21.15.131:0/1719724057 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f6cc00a11b0 0x7f6cc00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:41.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.640+0000 7f6cd8c00700 1 -- 172.21.15.131:0/1719724057 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f6cc8014070 con 0x7f6cdc1042d0 2024-05-08T22:06:41.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.640+0000 7f6cdbe00700 1 --2- 172.21.15.131:0/1719724057 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f6cc00a11b0 0x7f6cc00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:41.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.641+0000 7f6cdbe00700 1 --2- 172.21.15.131:0/1719724057 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f6cc00a11b0 0x7f6cc00a3670 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc00fa40 tx=0x7f6ccc019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:41.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.644+0000 7f6cd8c00700 1 -- 172.21.15.131:0/1719724057 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6cc8096300 con 0x7f6cdc1042d0 2024-05-08T22:06:41.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.895+0000 7f6ce3000700 1 -- 172.21.15.131:0/1719724057 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6cdc067c10 con 0x7f6cdc1042d0 2024-05-08T22:06:41.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.896+0000 7f6cd8c00700 1 -- 172.21.15.131:0/1719724057 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10889 (secure 0 0 0) 0x7f6cc8099920 con 0x7f6cdc1042d0 2024-05-08T22:06:41.896 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:06:41.896 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:05:14.034908+0000","modified":"2024-05-08T22:05:39.494566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:06:41.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.898+0000 7f6ce3000700 1 -- 172.21.15.131:0/1719724057 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f6cc00a11b0 msgr2=0x7f6cc00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:41.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.898+0000 7f6ce3000700 1 --2- 172.21.15.131:0/1719724057 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f6cc00a11b0 0x7f6cc00a3670 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc00fa40 tx=0x7f6ccc019040 comp rx=0 tx=0).stop 2024-05-08T22:06:41.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.898+0000 7f6ce3000700 1 -- 172.21.15.131:0/1719724057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cdc1042d0 msgr2=0x7f6cdc1995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:41.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.899+0000 7f6ce3000700 1 --2- 172.21.15.131:0/1719724057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cdc1042d0 0x7f6cdc1995c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800e9c0 tx=0x7f6cc800ecd0 comp rx=0 tx=0).stop 2024-05-08T22:06:41.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.899+0000 7f6ce3000700 1 -- 172.21.15.131:0/1719724057 shutdown_connections 2024-05-08T22:06:41.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.899+0000 7f6ce3000700 1 --2- 172.21.15.131:0/1719724057 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f6cc00a11b0 0x7f6cc00a3670 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:41.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.899+0000 7f6ce3000700 1 --2- 172.21.15.131:0/1719724057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cdc106c00 0x7f6cdc1a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:41.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.899+0000 7f6ce3000700 1 --2- 172.21.15.131:0/1719724057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6cdc1019a0 0x7f6cdc199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:41.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.899+0000 7f6ce3000700 1 --2- 172.21.15.131:0/1719724057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cdc1042d0 0x7f6cdc1995c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:41.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.899+0000 7f6ce3000700 1 -- 172.21.15.131:0/1719724057 >> 172.21.15.131:0/1719724057 conn(0x7f6cdc0fb2f0 msgr2=0x7f6cdc101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:41.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.899+0000 7f6ce3000700 1 -- 172.21.15.131:0/1719724057 shutdown_connections 2024-05-08T22:06:41.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:41.899+0000 7f6ce3000700 1 -- 172.21.15.131:0/1719724057 wait complete. 2024-05-08T22:06:41.900 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 23 2024-05-08T22:06:41.919 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-08T22:05:14.034908+0000', 'modified': '2024-05-08T22:05:39.494566+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24566, 'mds_1': 24562, 'mds_2': 24550}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24520': {'gid': 24520, 'name': 'c', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6837/2693198274', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6836', 'nonce': 2693198274}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 2693198274}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24524': {'gid': 24524, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.132:6837/2685833178', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6836', 'nonce': 2685833178}, {'type': 'v1', 'addr': '172.21.15.132:6837', 'nonce': 2685833178}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24538': {'gid': 24538, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6839/3040193644', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6838', 'nonce': 3040193644}, {'type': 'v1', 'addr': '172.21.15.157:6839', 'nonce': 3040193644}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24550': {'gid': 24550, 'name': 'g', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.131:6839/1622880274', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6838', 'nonce': 1622880274}, {'type': 'v1', 'addr': '172.21.15.131:6839', 'nonce': 1622880274}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24562': {'gid': 24562, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.131:6841/1686013987', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6840', 'nonce': 1686013987}, {'type': 'v1', 'addr': '172.21.15.131:6841', 'nonce': 1686013987}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24566': {'gid': 24566, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.132:6839/4205858754', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6838', 'nonce': 4205858754}, {'type': 'v1', 'addr': '172.21.15.132:6839', 'nonce': 4205858754}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:06:41.919 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-08T22:06:41.919 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-05-08T22:06:42.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.123+0000 7f8436c00700 1 -- 172.21.15.131:0/4044082130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8430100f20 msgr2=0x7f8430101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.123+0000 7f8436c00700 1 --2- 172.21.15.131:0/4044082130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8430100f20 0x7f8430101360 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8420009990 tx=0x7f8420009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.123+0000 7f8436c00700 1 -- 172.21.15.131:0/4044082130 shutdown_connections 2024-05-08T22:06:42.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.123+0000 7f8436c00700 1 --2- 172.21.15.131:0/4044082130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8430102ab0 0x7f84301093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.123+0000 7f8436c00700 1 --2- 172.21.15.131:0/4044082130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8430100f20 0x7f8430101360 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.123+0000 7f8436c00700 1 --2- 172.21.15.131:0/4044082130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84301020f0 0x7f8430102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.123+0000 7f8436c00700 1 -- 172.21.15.131:0/4044082130 >> 172.21.15.131:0/4044082130 conn(0x7f84300fc500 msgr2=0x7f84300fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:42.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.123+0000 7f8436c00700 1 -- 172.21.15.131:0/4044082130 shutdown_connections 2024-05-08T22:06:42.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.123+0000 7f8436c00700 1 -- 172.21.15.131:0/4044082130 wait complete. 2024-05-08T22:06:42.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.124+0000 7f8436c00700 1 Processor -- start 2024-05-08T22:06:42.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.124+0000 7f8436c00700 1 -- start start 2024-05-08T22:06:42.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.125+0000 7f8436c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8430100f20 0x7f8430074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.125+0000 7f8436c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84301020f0 0x7f8430073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.125+0000 7f8436c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8430102ab0 0x7f84300738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.125+0000 7f8436c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8430074070 con 0x7f8430100f20 2024-05-08T22:06:42.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.125+0000 7f8436c00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84300741e0 con 0x7f8430102ab0 2024-05-08T22:06:42.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.125+0000 7f8436c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84301a0fb0 con 0x7f84301020f0 2024-05-08T22:06:42.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.125+0000 7f842fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8430100f20 0x7f8430074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.125+0000 7f842f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84301020f0 0x7f8430073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.125+0000 7f842f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84301020f0 0x7f8430073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59824/0 (socket says 172.21.15.131:59824) 2024-05-08T22:06:42.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.125+0000 7f842f400700 1 -- 172.21.15.131:0/3339359155 learned_addr learned my addr 172.21.15.131:0/3339359155 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:42.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.125+0000 7f842fe00700 1 --2- 172.21.15.131:0/3339359155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8430100f20 0x7f8430074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40466/0 (socket says 172.21.15.131:40466) 2024-05-08T22:06:42.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.125+0000 7f8434c00700 1 --2- 172.21.15.131:0/3339359155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8430102ab0 0x7f84300738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.126+0000 7f842f400700 1 -- 172.21.15.131:0/3339359155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8430102ab0 msgr2=0x7f84300738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.126+0000 7f842f400700 1 --2- 172.21.15.131:0/3339359155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8430102ab0 0x7f84300738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.126+0000 7f842f400700 1 -- 172.21.15.131:0/3339359155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8430100f20 msgr2=0x7f8430074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.126+0000 7f842f400700 1 --2- 172.21.15.131:0/3339359155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8430100f20 0x7f8430074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.126+0000 7f842f400700 1 -- 172.21.15.131:0/3339359155 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84200095f0 con 0x7f84301020f0 2024-05-08T22:06:42.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.126+0000 7f842fe00700 1 --2- 172.21.15.131:0/3339359155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8430100f20 0x7f8430074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:42.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.126+0000 7f8434c00700 1 --2- 172.21.15.131:0/3339359155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8430102ab0 0x7f84300738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:42.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.126+0000 7f842f400700 1 --2- 172.21.15.131:0/3339359155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84301020f0 0x7f8430073390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f841c00d950 tx=0x7f841c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:42.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.126+0000 7f842cc00700 1 -- 172.21.15.131:0/3339359155 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c00f890 con 0x7f84301020f0 2024-05-08T22:06:42.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.126+0000 7f8436c00700 1 -- 172.21.15.131:0/3339359155 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84301a11e0 con 0x7f84301020f0 2024-05-08T22:06:42.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.126+0000 7f842cc00700 1 -- 172.21.15.131:0/3339359155 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f841c010460 con 0x7f84301020f0 2024-05-08T22:06:42.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.126+0000 7f842cc00700 1 -- 172.21.15.131:0/3339359155 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c00e610 con 0x7f84301020f0 2024-05-08T22:06:42.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.126+0000 7f8436c00700 1 -- 172.21.15.131:0/3339359155 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84301a16e0 con 0x7f84301020f0 2024-05-08T22:06:42.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.131+0000 7f842cc00700 1 -- 172.21.15.131:0/3339359155 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f841c0108d0 con 0x7f84301020f0 2024-05-08T22:06:42.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.132+0000 7f844a61fb80 1 Processor -- start 2024-05-08T22:06:42.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.132+0000 7f844a61fb80 1 -- start start 2024-05-08T22:06:42.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.132+0000 7f844a61fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97e0684f0 0x55d97e068910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.132+0000 7f844a61fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e068e50 0x55d97e0692d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.132+0000 7f844a61fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97e069810 0x55d97e069c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.132+0000 7f844a61fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97dfbf100 con 0x55d97e068e50 2024-05-08T22:06:42.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.132+0000 7f844a61fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97dfbcac0 con 0x55d97e0684f0 2024-05-08T22:06:42.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.132+0000 7f844a61fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97dfb83b0 con 0x55d97e069810 2024-05-08T22:06:42.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.132+0000 7f842cc00700 1 --2- 172.21.15.131:0/3339359155 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f84140a11b0 0x7f84140a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.132+0000 7f842fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e068e50 0x55d97e0692d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.132+0000 7f842cc00700 1 -- 172.21.15.131:0/3339359155 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f841c0ccc90 con 0x7f84301020f0 2024-05-08T22:06:42.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.132+0000 7f842fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e068e50 0x55d97e0692d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40474/0 (socket says 172.21.15.131:40474) 2024-05-08T22:06:42.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.132+0000 7f842fe00700 1 -- 172.21.15.131:0/3020411882 learned_addr learned my addr 172.21.15.131:0/3020411882 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:42.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.132+0000 7f8434c00700 1 --2- 172.21.15.131:0/3020411882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97e0684f0 0x55d97e068910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.132+0000 7f842f400700 1 --2- 172.21.15.131:0/3020411882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97e069810 0x55d97e069c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.133+0000 7f8434c00700 1 --2- 172.21.15.131:0/3339359155 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f84140a11b0 0x7f84140a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.133+0000 7f842fe00700 1 -- 172.21.15.131:0/3020411882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97e069810 msgr2=0x55d97e069c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:42.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.133+0000 7f842fe00700 1 --2- 172.21.15.131:0/3020411882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97e069810 0x55d97e069c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.133+0000 7f842fe00700 1 -- 172.21.15.131:0/3020411882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97e0684f0 msgr2=0x55d97e068910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:42.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.133+0000 7f842fe00700 1 --2- 172.21.15.131:0/3020411882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97e0684f0 0x55d97e068910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.133+0000 7f842fe00700 1 -- 172.21.15.131:0/3020411882 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d97de39360 con 0x55d97e068e50 2024-05-08T22:06:42.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.133+0000 7f842fe00700 1 --2- 172.21.15.131:0/3020411882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e068e50 0x55d97e0692d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8420009990 tx=0x7f8420005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c18e4d1bc4bb44e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:42.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.133+0000 7f8434c00700 1 --2- 172.21.15.131:0/3339359155 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f84140a11b0 0x7f84140a3670 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8428009fd0 tx=0x7f842800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:42.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.133+0000 7f8425200700 1 -- 172.21.15.131:0/3020411882 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842001d030 con 0x55d97e068e50 2024-05-08T22:06:42.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.133+0000 7f8425200700 1 -- 172.21.15.131:0/3020411882 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f842001e040 con 0x55d97e068e50 2024-05-08T22:06:42.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.133+0000 7f8425200700 1 -- 172.21.15.131:0/3020411882 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8420004d50 con 0x55d97e068e50 2024-05-08T22:06:42.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.134+0000 7f844a61fb80 1 -- 172.21.15.131:0/3020411882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e068e50 msgr2=0x55d97e0692d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.134+0000 7f844a61fb80 1 --2- 172.21.15.131:0/3020411882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e068e50 0x55d97e0692d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8420009990 tx=0x7f8420005bc0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.134+0000 7f844a61fb80 1 -- 172.21.15.131:0/3020411882 shutdown_connections 2024-05-08T22:06:42.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.134+0000 7f844a61fb80 1 --2- 172.21.15.131:0/3020411882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97e069810 0x55d97e069c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.134+0000 7f844a61fb80 1 --2- 172.21.15.131:0/3020411882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97e0684f0 0x55d97e068910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.134+0000 7f844a61fb80 1 --2- 172.21.15.131:0/3020411882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e068e50 0x55d97e0692d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.134+0000 7f844a61fb80 1 -- 172.21.15.131:0/3020411882 >> 172.21.15.131:0/3020411882 conn(0x55d97dfc61c0 msgr2=0x55d97e062d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:42.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.134+0000 7f844a61fb80 1 -- 172.21.15.131:0/3020411882 shutdown_connections 2024-05-08T22:06:42.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.134+0000 7f844a61fb80 1 -- 172.21.15.131:0/3020411882 wait complete. 2024-05-08T22:06:42.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.135+0000 7f844a61fb80 1 -- start start 2024-05-08T22:06:42.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.135+0000 7f844a61fb80 1 Processor -- start 2024-05-08T22:06:42.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.135+0000 7f844a61fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97e0684f0 0x55d97e071d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.135+0000 7f844a61fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e068e50 0x55d97e072270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.135+0000 7f844a61fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97e069810 0x55d97e0792b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.135+0000 7f844a61fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97df54510 con 0x55d97e068e50 2024-05-08T22:06:42.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.135+0000 7f844a61fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97df541b0 con 0x55d97e0684f0 2024-05-08T22:06:42.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.135+0000 7f844a61fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97de3b650 con 0x55d97e069810 2024-05-08T22:06:42.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.135+0000 7f844a61fb80 10 client.0 _ll_register_callbacks cb 0x55d97df5dae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:06:42.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.136+0000 7f842f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e068e50 0x55d97e072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.136+0000 7f842f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e068e50 0x55d97e072270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40480/0 (socket says 172.21.15.131:40480) 2024-05-08T22:06:42.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.136+0000 7f842f400700 1 -- 172.21.15.131:0/2133526548 learned_addr learned my addr 172.21.15.131:0/2133526548 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:42.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.136+0000 7f8434c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97e069810 0x55d97e0792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.136+0000 7f842fe00700 1 --2- 172.21.15.131:0/2133526548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97e0684f0 0x55d97e071d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.136+0000 7f842f400700 1 -- 172.21.15.131:0/2133526548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97e069810 msgr2=0x55d97e0792b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:42.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.136+0000 7f842f400700 1 --2- 172.21.15.131:0/2133526548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97e069810 0x55d97e0792b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.136+0000 7f842f400700 1 -- 172.21.15.131:0/2133526548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97e0684f0 msgr2=0x55d97e071d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:42.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.136+0000 7f842f400700 1 --2- 172.21.15.131:0/2133526548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97e0684f0 0x55d97e071d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.136+0000 7f842f400700 1 -- 172.21.15.131:0/2133526548 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8430109d00 con 0x55d97e068e50 2024-05-08T22:06:42.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.136+0000 7f842f400700 1 --2- 172.21.15.131:0/2133526548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e068e50 0x55d97e072270 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f841c00dfb0 tx=0x7f841c099520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2434dd4004805cf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:42.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.136+0000 7f8419600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:06:42.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.136+0000 7f8419600700 1 -- 172.21.15.131:0/2133526548 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c00a450 con 0x55d97e068e50 2024-05-08T22:06:42.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.136+0000 7f8419600700 1 -- 172.21.15.131:0/2133526548 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f841c099bc0 con 0x55d97e068e50 2024-05-08T22:06:42.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.136+0000 7f844a61fb80 1 -- 172.21.15.131:0/2133526548 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d97df11220 con 0x55d97e068e50 2024-05-08T22:06:42.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.137+0000 7f8419600700 1 -- 172.21.15.131:0/2133526548 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f841c09d9f0 con 0x55d97e068e50 2024-05-08T22:06:42.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.137+0000 7f844a61fb80 10 client.15126 fetch_fsmap learned FSMap version 23 2024-05-08T22:06:42.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.137+0000 7f844a61fb80 1 -- 172.21.15.131:0/2133526548 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d97dde5b10 con 0x55d97e068e50 2024-05-08T22:06:42.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.137+0000 7f8419600700 1 -- 172.21.15.131:0/2133526548 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d97df11220 con 0x55d97e068e50 2024-05-08T22:06:42.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.137+0000 7f8419600700 1 -- 172.21.15.131:0/2133526548 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f841c0990b0 con 0x55d97e068e50 2024-05-08T22:06:42.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.137+0000 7f844a61fb80 10 client.15126 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T22:06:42.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.137+0000 7f844a61fb80 10 client.15126 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-08T22:06:42.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.137+0000 7f844a61fb80 20 client.15126 populate_metadata read hostname 'smithi131' 2024-05-08T22:06:42.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.137+0000 7f844a61fb80 1 --2- 172.21.15.131:0/2133526548 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d97e07c8e0 0x55d97e0821d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.137+0000 7f844a61fb80 4 client.15126 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:06:42.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.137+0000 7f844a61fb80 1 -- 172.21.15.131:0/2133526548 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d97e0377c0 con 0x55d97e07c8e0 2024-05-08T22:06:42.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.137+0000 7f842fe00700 1 --2- 172.21.15.131:0/2133526548 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d97e07c8e0 0x55d97e0821d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.138+0000 7f842fe00700 1 --2- 172.21.15.131:0/2133526548 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d97e07c8e0 0x55d97e0821d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a576d1652181ed1c server_cookie=81d9a3286768f24c in_seq=0 out_seq=0 2024-05-08T22:06:42.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.138+0000 7f8419600700 10 client.15126 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:06:42.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.139+0000 7f8419600700 1 -- 172.21.15.131:0/2133526548 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d97e0377c0 con 0x55d97e07c8e0 2024-05-08T22:06:42.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.139+0000 7f8419600700 10 client.15126 handle_command_reply: tid=0 2024-05-08T22:06:42.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.139+0000 7f844a61fb80 1 client.15126 shutdown 2024-05-08T22:06:42.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.139+0000 7f844a61fb80 1 -- 172.21.15.131:0/2133526548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e068e50 msgr2=0x55d97e072270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.139+0000 7f844a61fb80 1 --2- 172.21.15.131:0/2133526548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e068e50 0x55d97e072270 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f841c00dfb0 tx=0x7f841c099520 comp rx=0 tx=0).stop 2024-05-08T22:06:42.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.139+0000 7f844a61fb80 1 -- 172.21.15.131:0/2133526548 shutdown_connections 2024-05-08T22:06:42.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.139+0000 7f844a61fb80 1 --2- 172.21.15.131:0/2133526548 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d97e07c8e0 0x55d97e0821d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.139+0000 7f844a61fb80 1 --2- 172.21.15.131:0/2133526548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97e069810 0x55d97e0792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.139+0000 7f8419600700 0 client.15126 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:06:42.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.139+0000 7f844a61fb80 1 --2- 172.21.15.131:0/2133526548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97e0684f0 0x55d97e071d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.139+0000 7f844a61fb80 1 --2- 172.21.15.131:0/2133526548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e068e50 0x55d97e072270 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.139+0000 7f844a61fb80 1 -- 172.21.15.131:0/2133526548 >> 172.21.15.131:0/2133526548 conn(0x55d97dfc61c0 msgr2=0x55d97e062550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:42.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.139+0000 7f844a61fb80 1 -- 172.21.15.131:0/2133526548 shutdown_connections 2024-05-08T22:06:42.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.139+0000 7f844a61fb80 1 -- 172.21.15.131:0/2133526548 wait complete. 2024-05-08T22:06:42.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.139+0000 7f844a61fb80 20 client.15126 trim_cache size 0 max 16384 2024-05-08T22:06:42.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.157+0000 7f844a61fb80 1 Processor -- start 2024-05-08T22:06:42.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.158+0000 7f844a61fb80 1 -- start start 2024-05-08T22:06:42.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.158+0000 7f844a61fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e0684f0 0x55d97e0688d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.158+0000 7f844a61fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97e068e50 0x55d97e071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.158+0000 7f844a61fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97e069810 0x55d97e079270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.158+0000 7f844a61fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97df11220 con 0x55d97e0684f0 2024-05-08T22:06:42.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.158+0000 7f844a61fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97df762b0 con 0x55d97e068e50 2024-05-08T22:06:42.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.158+0000 7f844a61fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97dfb5010 con 0x55d97e069810 2024-05-08T22:06:42.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.158+0000 7f842fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e0684f0 0x55d97e0688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.158+0000 7f842fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e0684f0 0x55d97e0688d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40492/0 (socket says 172.21.15.131:40492) 2024-05-08T22:06:42.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.158+0000 7f842f400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97e068e50 0x55d97e071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.158+0000 7f842fe00700 1 -- 172.21.15.131:0/2880062833 learned_addr learned my addr 172.21.15.131:0/2880062833 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:42.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.158+0000 7f8434c00700 1 --2- 172.21.15.131:0/2880062833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97e069810 0x55d97e079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.159+0000 7f842fe00700 1 -- 172.21.15.131:0/2880062833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97e069810 msgr2=0x55d97e079270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:42.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.159+0000 7f842fe00700 1 --2- 172.21.15.131:0/2880062833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97e069810 0x55d97e079270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.159+0000 7f842fe00700 1 -- 172.21.15.131:0/2880062833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97e068e50 msgr2=0x55d97e071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:42.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.159+0000 7f842fe00700 1 --2- 172.21.15.131:0/2880062833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97e068e50 0x55d97e071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.159+0000 7f842fe00700 1 -- 172.21.15.131:0/2880062833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d97df7e510 con 0x55d97e0684f0 2024-05-08T22:06:42.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.159+0000 7f842f400700 1 --2- 172.21.15.131:0/2880062833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97e068e50 0x55d97e071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:42.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.159+0000 7f842fe00700 1 --2- 172.21.15.131:0/2880062833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e0684f0 0x55d97e0688d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f84200046c0 tx=0x7f84200094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd7de38431de684 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:42.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.159+0000 7f8425c00700 1 -- 172.21.15.131:0/2880062833 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8420032ea0 con 0x55d97e0684f0 2024-05-08T22:06:42.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.159+0000 7f8425c00700 1 -- 172.21.15.131:0/2880062833 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f842000be80 con 0x55d97e0684f0 2024-05-08T22:06:42.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.159+0000 7f8425c00700 1 -- 172.21.15.131:0/2880062833 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842001d040 con 0x55d97e0684f0 2024-05-08T22:06:42.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.160+0000 7f844a61fb80 1 -- 172.21.15.131:0/2880062833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e0684f0 msgr2=0x55d97e0688d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.160+0000 7f844a61fb80 1 --2- 172.21.15.131:0/2880062833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e0684f0 0x55d97e0688d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f84200046c0 tx=0x7f84200094b0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.160+0000 7f844a61fb80 1 -- 172.21.15.131:0/2880062833 shutdown_connections 2024-05-08T22:06:42.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.160+0000 7f844a61fb80 1 --2- 172.21.15.131:0/2880062833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97e069810 0x55d97e079270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.160+0000 7f844a61fb80 1 --2- 172.21.15.131:0/2880062833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e0684f0 0x55d97e0688d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.160+0000 7f844a61fb80 1 --2- 172.21.15.131:0/2880062833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97e068e50 0x55d97e071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.160+0000 7f844a61fb80 1 -- 172.21.15.131:0/2880062833 >> 172.21.15.131:0/2880062833 conn(0x55d97dfc61c0 msgr2=0x55d97e069bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:42.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.160+0000 7f844a61fb80 1 -- 172.21.15.131:0/2880062833 shutdown_connections 2024-05-08T22:06:42.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.160+0000 7f844a61fb80 1 -- 172.21.15.131:0/2880062833 wait complete. 2024-05-08T22:06:42.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.160+0000 7f844a61fb80 1 -- start start 2024-05-08T22:06:42.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.161+0000 7f844a61fb80 1 Processor -- start 2024-05-08T22:06:42.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.161+0000 7f844a61fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e0684f0 0x55d97e0733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.161+0000 7f844a61fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97e068e50 0x55d97e073930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.161+0000 7f844a61fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97e069810 0x55d97e078690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.161+0000 7f844a61fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97e078bd0 con 0x55d97e0684f0 2024-05-08T22:06:42.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.161+0000 7f844a61fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97e078d40 con 0x55d97e069810 2024-05-08T22:06:42.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.161+0000 7f844a61fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97e078f00 con 0x55d97e068e50 2024-05-08T22:06:42.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.161+0000 7f844a61fb80 10 client.0 _ll_register_callbacks cb 0x55d97df5dae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:06:42.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.161+0000 7f842f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97e068e50 0x55d97e073930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.161+0000 7f842fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e0684f0 0x55d97e0733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.161+0000 7f842f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97e068e50 0x55d97e073930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59876/0 (socket says 172.21.15.131:59876) 2024-05-08T22:06:42.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.161+0000 7f842f400700 1 -- 172.21.15.131:0/289051028 learned_addr learned my addr 172.21.15.131:0/289051028 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:42.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.161+0000 7f8434c00700 1 --2- 172.21.15.131:0/289051028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97e069810 0x55d97e078690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.162+0000 7f842f400700 1 -- 172.21.15.131:0/289051028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97e069810 msgr2=0x55d97e078690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:42.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.162+0000 7f842f400700 1 --2- 172.21.15.131:0/289051028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97e069810 0x55d97e078690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.162+0000 7f842f400700 1 -- 172.21.15.131:0/289051028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e0684f0 msgr2=0x55d97e0733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:42.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.162+0000 7f842f400700 1 --2- 172.21.15.131:0/289051028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e0684f0 0x55d97e0733f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.162+0000 7f842f400700 1 -- 172.21.15.131:0/289051028 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d97dde5b10 con 0x55d97e068e50 2024-05-08T22:06:42.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.162+0000 7f842fe00700 1 --2- 172.21.15.131:0/289051028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e0684f0 0x55d97e0733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:42.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.162+0000 7f842f400700 1 --2- 172.21.15.131:0/289051028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97e068e50 0x55d97e073930 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f841c0998c0 tx=0x7f841c005910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1c0256014ada0db server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:42.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.162+0000 7f8419600700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:06:42.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.162+0000 7f8419600700 1 -- 172.21.15.131:0/289051028 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c063070 con 0x55d97e068e50 2024-05-08T22:06:42.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.162+0000 7f844a61fb80 1 -- 172.21.15.131:0/289051028 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d97dfb0e70 con 0x55d97e068e50 2024-05-08T22:06:42.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.162+0000 7f8419600700 1 -- 172.21.15.131:0/289051028 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f841c099bc0 con 0x55d97e068e50 2024-05-08T22:06:42.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.162+0000 7f8419600700 1 -- 172.21.15.131:0/289051028 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f841c014070 con 0x55d97e068e50 2024-05-08T22:06:42.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.162+0000 7f844a61fb80 10 client.25174 fetch_fsmap learned FSMap version 23 2024-05-08T22:06:42.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.162+0000 7f844a61fb80 1 -- 172.21.15.131:0/289051028 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d97de39360 con 0x55d97e068e50 2024-05-08T22:06:42.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.163+0000 7f8419600700 1 -- 172.21.15.131:0/289051028 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d97dfb0e70 con 0x55d97e068e50 2024-05-08T22:06:42.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.163+0000 7f8419600700 1 -- 172.21.15.131:0/289051028 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f841c09abd0 con 0x55d97e068e50 2024-05-08T22:06:42.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.163+0000 7f844a61fb80 10 client.25174 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T22:06:42.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.163+0000 7f844a61fb80 10 client.25174 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-08T22:06:42.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.163+0000 7f844a61fb80 20 client.25174 populate_metadata read hostname 'smithi131' 2024-05-08T22:06:42.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.163+0000 7f844a61fb80 1 --2- 172.21.15.131:0/289051028 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d97e0d7e00 0x55d97e0d4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.163+0000 7f844a61fb80 4 client.25174 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:06:42.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.163+0000 7f844a61fb80 1 -- 172.21.15.131:0/289051028 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d97de06ca0 con 0x55d97e0d7e00 2024-05-08T22:06:42.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.163+0000 7f842fe00700 1 --2- 172.21.15.131:0/289051028 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d97e0d7e00 0x55d97e0d4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.164+0000 7f842fe00700 1 --2- 172.21.15.131:0/289051028 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d97e0d7e00 0x55d97e0d4830 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a05600eb5b573695 server_cookie=c86a1d9bbbd67f83 in_seq=0 out_seq=0 2024-05-08T22:06:42.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.164+0000 7f8419600700 10 client.25174 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:06:42.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.164+0000 7f8419600700 1 -- 172.21.15.131:0/289051028 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55d97de06ca0 con 0x55d97e0d7e00 2024-05-08T22:06:42.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.164+0000 7f8419600700 10 client.25174 handle_command_reply: tid=0 2024-05-08T22:06:42.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.164+0000 7f844a61fb80 1 client.25174 shutdown 2024-05-08T22:06:42.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.164+0000 7f844a61fb80 1 -- 172.21.15.131:0/289051028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97e068e50 msgr2=0x55d97e073930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.164+0000 7f844a61fb80 1 --2- 172.21.15.131:0/289051028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97e068e50 0x55d97e073930 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f841c0998c0 tx=0x7f841c005910 comp rx=0 tx=0).stop 2024-05-08T22:06:42.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.164+0000 7f844a61fb80 1 -- 172.21.15.131:0/289051028 shutdown_connections 2024-05-08T22:06:42.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.164+0000 7f844a61fb80 1 --2- 172.21.15.131:0/289051028 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d97e0d7e00 0x55d97e0d4830 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.164+0000 7f844a61fb80 1 --2- 172.21.15.131:0/289051028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97e069810 0x55d97e078690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.164+0000 7f8419600700 0 client.25174 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:06:42.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.164+0000 7f844a61fb80 1 --2- 172.21.15.131:0/289051028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97e0684f0 0x55d97e0733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.164+0000 7f844a61fb80 1 --2- 172.21.15.131:0/289051028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97e068e50 0x55d97e073930 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.164+0000 7f844a61fb80 1 -- 172.21.15.131:0/289051028 >> 172.21.15.131:0/289051028 conn(0x55d97dfc61c0 msgr2=0x55d97e071ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:42.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.164+0000 7f844a61fb80 1 -- 172.21.15.131:0/289051028 shutdown_connections 2024-05-08T22:06:42.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.164+0000 7f844a61fb80 1 -- 172.21.15.131:0/289051028 wait complete. 2024-05-08T22:06:42.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.164+0000 7f844a61fb80 20 client.25174 trim_cache size 0 max 16384 2024-05-08T22:06:42.171 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:06:42.171 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "06f1a696-0d86-11ef-bc97-c7b262605968", 2024-05-08T22:06:42.171 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 2, 2024-05-08T22:06:42.171 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24550, 2024-05-08T22:06:42.171 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-08T22:06:42.171 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-08T22:06:42.171 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-08T22:06:42.171 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 63.677015257000001, 2024-05-08T22:06:42.171 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 23, 2024-05-08T22:06:42.171 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 87, 2024-05-08T22:06:42.171 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 83, 2024-05-08T22:06:42.171 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 98.551507049999998 2024-05-08T22:06:42.172 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:06:42.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.165+0000 7f8436c00700 1 -- 172.21.15.131:0/3339359155 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f84140a11b0 msgr2=0x7f84140a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.165+0000 7f8436c00700 1 --2- 172.21.15.131:0/3339359155 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f84140a11b0 0x7f84140a3670 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8428009fd0 tx=0x7f842800b040 comp rx=0 tx=0).stop 2024-05-08T22:06:42.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.165+0000 7f8436c00700 1 -- 172.21.15.131:0/3339359155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84301020f0 msgr2=0x7f8430073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.165+0000 7f8436c00700 1 --2- 172.21.15.131:0/3339359155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84301020f0 0x7f8430073390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f841c00d950 tx=0x7f841c00dc60 comp rx=0 tx=0).stop 2024-05-08T22:06:42.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.165+0000 7f8436c00700 1 -- 172.21.15.131:0/3339359155 shutdown_connections 2024-05-08T22:06:42.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.165+0000 7f8436c00700 1 --2- 172.21.15.131:0/3339359155 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f84140a11b0 0x7f84140a3670 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.165+0000 7f8436c00700 1 --2- 172.21.15.131:0/3339359155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8430102ab0 0x7f84300738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.165+0000 7f8436c00700 1 --2- 172.21.15.131:0/3339359155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8430100f20 0x7f8430074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.165+0000 7f8436c00700 1 --2- 172.21.15.131:0/3339359155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84301020f0 0x7f8430073390 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.165+0000 7f8436c00700 1 -- 172.21.15.131:0/3339359155 >> 172.21.15.131:0/3339359155 conn(0x7f84300fc500 msgr2=0x7f84301055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:42.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.165+0000 7f8436c00700 1 -- 172.21.15.131:0/3339359155 shutdown_connections 2024-05-08T22:06:42.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.165+0000 7f8436c00700 1 -- 172.21.15.131:0/3339359155 wait complete. 2024-05-08T22:06:42.181 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:41 smithi131 ceph-mon[37260]: pgmap v285: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T22:06:42.181 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-05-08T22:06:42.182 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-05-08T22:06:42.182 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-05-08T22:06:42.182 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:41 smithi131 ceph-mon[37260]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 7]}]': finished 2024-05-08T22:06:42.182 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:41 smithi131 ceph-mon[37260]: osdmap e86: 12 total, 12 up, 12 in 2024-05-08T22:06:42.182 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:41 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2827778284' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:06:42.182 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:41 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1719724057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:06:42.182 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:41 smithi131 ceph-mon[37260]: osdmap e87: 12 total, 12 up, 12 in 2024-05-08T22:06:42.184 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-08T22:06:42.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:41 smithi157 ceph-mon[77746]: pgmap v285: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T22:06:42.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-05-08T22:06:42.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-05-08T22:06:42.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-05-08T22:06:42.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:41 smithi157 ceph-mon[77746]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 7]}]': finished 2024-05-08T22:06:42.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:41 smithi157 ceph-mon[77746]: osdmap e86: 12 total, 12 up, 12 in 2024-05-08T22:06:42.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:41 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2827778284' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:06:42.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:41 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1719724057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:06:42.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:41 smithi157 ceph-mon[77746]: osdmap e87: 12 total, 12 up, 12 in 2024-05-08T22:06:42.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:41 smithi132 ceph-mon[78180]: pgmap v285: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T22:06:42.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-05-08T22:06:42.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-05-08T22:06:42.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-05-08T22:06:42.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:41 smithi132 ceph-mon[78180]: from='mgr.14150 172.21.15.131:0/2241377251' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 7]}]': finished 2024-05-08T22:06:42.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:41 smithi132 ceph-mon[78180]: osdmap e86: 12 total, 12 up, 12 in 2024-05-08T22:06:42.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:41 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2827778284' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:06:42.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:41 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1719724057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:06:42.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:41 smithi132 ceph-mon[78180]: osdmap e87: 12 total, 12 up, 12 in 2024-05-08T22:06:42.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.377+0000 7fce18400700 1 -- 172.21.15.131:0/279941566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce101019e0 msgr2=0x7fce10103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.377+0000 7fce18400700 1 --2- 172.21.15.131:0/279941566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce101019e0 0x7fce10103dd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fce04009990 tx=0x7fce04009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.377+0000 7fce18400700 1 -- 172.21.15.131:0/279941566 shutdown_connections 2024-05-08T22:06:42.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.377+0000 7fce18400700 1 --2- 172.21.15.131:0/279941566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce10106c40 0x7fce101090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.377+0000 7fce18400700 1 --2- 172.21.15.131:0/279941566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce101019e0 0x7fce10103dd0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.377+0000 7fce18400700 1 --2- 172.21.15.131:0/279941566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce10104310 0x7fce10106700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.377+0000 7fce18400700 1 -- 172.21.15.131:0/279941566 >> 172.21.15.131:0/279941566 conn(0x7fce100fb3f0 msgr2=0x7fce100fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:42.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.377+0000 7fce18400700 1 -- 172.21.15.131:0/279941566 shutdown_connections 2024-05-08T22:06:42.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.377+0000 7fce18400700 1 -- 172.21.15.131:0/279941566 wait complete. 2024-05-08T22:06:42.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.378+0000 7fce18400700 1 Processor -- start 2024-05-08T22:06:42.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.379+0000 7fce18400700 1 -- start start 2024-05-08T22:06:42.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.379+0000 7fce18400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce101019e0 0x7fce10196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.379+0000 7fce18400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce10104310 0x7fce10197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.379+0000 7fce18400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce10106c40 0x7fce1019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.379+0000 7fce18400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1019e9f0 con 0x7fce101019e0 2024-05-08T22:06:42.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.379+0000 7fce18400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1019eb30 con 0x7fce10106c40 2024-05-08T22:06:42.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.379+0000 7fce18400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1019ecc0 con 0x7fce10104310 2024-05-08T22:06:42.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.379+0000 7fce15a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce101019e0 0x7fce10196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.380+0000 7fce15a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce101019e0 0x7fce10196ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40516/0 (socket says 172.21.15.131:40516) 2024-05-08T22:06:42.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.380+0000 7fce15a00700 1 -- 172.21.15.131:0/800505153 learned_addr learned my addr 172.21.15.131:0/800505153 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:42.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.380+0000 7fce15000700 1 --2- 172.21.15.131:0/800505153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce10104310 0x7fce10197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.380+0000 7fce16400700 1 --2- 172.21.15.131:0/800505153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce10106c40 0x7fce1019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.380+0000 7fce15a00700 1 -- 172.21.15.131:0/800505153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce10104310 msgr2=0x7fce10197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.380+0000 7fce15a00700 1 --2- 172.21.15.131:0/800505153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce10104310 0x7fce10197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.380+0000 7fce15a00700 1 -- 172.21.15.131:0/800505153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce10106c40 msgr2=0x7fce1019e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.380+0000 7fce15a00700 1 --2- 172.21.15.131:0/800505153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce10106c40 0x7fce1019e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.380+0000 7fce15a00700 1 -- 172.21.15.131:0/800505153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce040095f0 con 0x7fce101019e0 2024-05-08T22:06:42.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.380+0000 7fce15a00700 1 --2- 172.21.15.131:0/800505153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce101019e0 0x7fce10196ec0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fce04005910 tx=0x7fce04009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:42.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.380+0000 7fce0e000700 1 -- 172.21.15.131:0/800505153 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0401d070 con 0x7fce101019e0 2024-05-08T22:06:42.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.380+0000 7fce0e000700 1 -- 172.21.15.131:0/800505153 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce04021410 con 0x7fce101019e0 2024-05-08T22:06:42.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.380+0000 7fce18400700 1 -- 172.21.15.131:0/800505153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1019ef40 con 0x7fce101019e0 2024-05-08T22:06:42.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.380+0000 7fce0e000700 1 -- 172.21.15.131:0/800505153 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce04005720 con 0x7fce101019e0 2024-05-08T22:06:42.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.381+0000 7fce18400700 1 -- 172.21.15.131:0/800505153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce1019f2b0 con 0x7fce101019e0 2024-05-08T22:06:42.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.384+0000 7fce0e000700 1 -- 172.21.15.131:0/800505153 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fce04021580 con 0x7fce101019e0 2024-05-08T22:06:42.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.385+0000 7fce0e000700 1 --2- 172.21.15.131:0/800505153 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fcdf80a11b0 0x7fcdf80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.385+0000 7fce0e000700 1 -- 172.21.15.131:0/800505153 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fce040ce1b0 con 0x7fce101019e0 2024-05-08T22:06:42.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.385+0000 7fce2beedb80 1 Processor -- start 2024-05-08T22:06:42.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.385+0000 7fce15000700 1 --2- 172.21.15.131:0/800505153 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fcdf80a11b0 0x7fcdf80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.385+0000 7fce2beedb80 1 -- start start 2024-05-08T22:06:42.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.386+0000 7fce2beedb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562448bac4f0 0x562448bac910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.386+0000 7fce2beedb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bace50 0x562448bad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.386+0000 7fce2beedb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562448bad810 0x562448badc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.386+0000 7fce2beedb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562448b03100 con 0x562448bace50 2024-05-08T22:06:42.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.386+0000 7fce2beedb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562448b00ac0 con 0x562448bad810 2024-05-08T22:06:42.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.386+0000 7fce2beedb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562448afc3b0 con 0x562448bac4f0 2024-05-08T22:06:42.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.386+0000 7fce15000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bace50 0x562448bad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.386+0000 7fce15000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bace50 0x562448bad2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40528/0 (socket says 172.21.15.131:40528) 2024-05-08T22:06:42.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.386+0000 7fce15000700 1 -- 172.21.15.131:0/1463167290 learned_addr learned my addr 172.21.15.131:0/1463167290 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:42.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.386+0000 7fce15a00700 1 --2- 172.21.15.131:0/1463167290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562448bac4f0 0x562448bac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.386+0000 7fce16400700 1 --2- 172.21.15.131:0/1463167290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562448bad810 0x562448badc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.386+0000 7fce15000700 1 --2- 172.21.15.131:0/800505153 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fcdf80a11b0 0x7fcdf80a3670 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009cb0 tx=0x7fcdfc011720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:42.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.386+0000 7fce15000700 1 -- 172.21.15.131:0/1463167290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562448bac4f0 msgr2=0x562448bac910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:42.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.386+0000 7fce15000700 1 --2- 172.21.15.131:0/1463167290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562448bac4f0 0x562448bac910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.386+0000 7fce15000700 1 -- 172.21.15.131:0/1463167290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562448bad810 msgr2=0x562448badc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:42.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.386+0000 7fce15000700 1 --2- 172.21.15.131:0/1463167290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562448bad810 0x562448badc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.386+0000 7fce15000700 1 -- 172.21.15.131:0/1463167290 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56244897d360 con 0x562448bace50 2024-05-08T22:06:42.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.386+0000 7fce16400700 1 --2- 172.21.15.131:0/1463167290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562448bad810 0x562448badc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:06:42.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.387+0000 7fce15000700 1 --2- 172.21.15.131:0/1463167290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bace50 0x562448bad2d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc005c50 tx=0x7fcdfc01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de0144c6d29b5f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:42.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.387+0000 7fce02a00700 1 -- 172.21.15.131:0/1463167290 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc01ddb0 con 0x562448bace50 2024-05-08T22:06:42.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.387+0000 7fce02a00700 1 -- 172.21.15.131:0/1463167290 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdfc004cf0 con 0x562448bace50 2024-05-08T22:06:42.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.387+0000 7fce02a00700 1 -- 172.21.15.131:0/1463167290 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc00f5c0 con 0x562448bace50 2024-05-08T22:06:42.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.387+0000 7fce2beedb80 1 -- 172.21.15.131:0/1463167290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bace50 msgr2=0x562448bad2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.387+0000 7fce2beedb80 1 --2- 172.21.15.131:0/1463167290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bace50 0x562448bad2d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc005c50 tx=0x7fcdfc01c000 comp rx=0 tx=0).stop 2024-05-08T22:06:42.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.388+0000 7fce2beedb80 1 -- 172.21.15.131:0/1463167290 shutdown_connections 2024-05-08T22:06:42.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.388+0000 7fce2beedb80 1 --2- 172.21.15.131:0/1463167290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562448bad810 0x562448badc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.388+0000 7fce2beedb80 1 --2- 172.21.15.131:0/1463167290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562448bac4f0 0x562448bac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.388+0000 7fce2beedb80 1 --2- 172.21.15.131:0/1463167290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bace50 0x562448bad2d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.388+0000 7fce2beedb80 1 -- 172.21.15.131:0/1463167290 >> 172.21.15.131:0/1463167290 conn(0x562448b0a1c0 msgr2=0x562448ba6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:42.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.388+0000 7fce2beedb80 1 -- 172.21.15.131:0/1463167290 shutdown_connections 2024-05-08T22:06:42.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.388+0000 7fce2beedb80 1 -- 172.21.15.131:0/1463167290 wait complete. 2024-05-08T22:06:42.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.388+0000 7fce2beedb80 1 -- start start 2024-05-08T22:06:42.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.388+0000 7fce2beedb80 1 Processor -- start 2024-05-08T22:06:42.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.389+0000 7fce2beedb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bac4f0 0x562448bba170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.389+0000 7fce2beedb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562448bace50 0x562448bba6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.389+0000 7fce2beedb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562448bad810 0x562448bc16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.389+0000 7fce2beedb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562448a98510 con 0x562448bac4f0 2024-05-08T22:06:42.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.389+0000 7fce2beedb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562448a981b0 con 0x562448bace50 2024-05-08T22:06:42.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.389+0000 7fce2beedb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244897f650 con 0x562448bad810 2024-05-08T22:06:42.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.389+0000 7fce2beedb80 10 client.0 _ll_register_callbacks cb 0x562448aa1ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:06:42.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.389+0000 7fce15a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bac4f0 0x562448bba170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.389+0000 7fce15a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bac4f0 0x562448bba170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40532/0 (socket says 172.21.15.131:40532) 2024-05-08T22:06:42.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.389+0000 7fce15a00700 1 -- 172.21.15.131:0/3516456966 learned_addr learned my addr 172.21.15.131:0/3516456966 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:42.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.389+0000 7fce16400700 1 --2- 172.21.15.131:0/3516456966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562448bad810 0x562448bc16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.389+0000 7fce15000700 1 --2- 172.21.15.131:0/3516456966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562448bace50 0x562448bba6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.390+0000 7fce15a00700 1 -- 172.21.15.131:0/3516456966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562448bad810 msgr2=0x562448bc16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:42.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.390+0000 7fce15a00700 1 --2- 172.21.15.131:0/3516456966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562448bad810 0x562448bc16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.390+0000 7fce15a00700 1 -- 172.21.15.131:0/3516456966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562448bace50 msgr2=0x562448bba6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:42.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.390+0000 7fce15a00700 1 --2- 172.21.15.131:0/3516456966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562448bace50 0x562448bba6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.390+0000 7fce15a00700 1 -- 172.21.15.131:0/3516456966 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdfc011cd0 con 0x562448bac4f0 2024-05-08T22:06:42.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.390+0000 7fce15a00700 1 --2- 172.21.15.131:0/3516456966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bac4f0 0x562448bba170 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fce04006010 tx=0x7fce04021b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4927a713eff6d96b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:42.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.390+0000 7fcdf7400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:06:42.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.390+0000 7fcdf7400700 1 -- 172.21.15.131:0/3516456966 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0409b5d0 con 0x562448bac4f0 2024-05-08T22:06:42.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.390+0000 7fcdf7400700 1 -- 172.21.15.131:0/3516456966 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fce0409b170 con 0x562448bac4f0 2024-05-08T22:06:42.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.390+0000 7fce2beedb80 1 -- 172.21.15.131:0/3516456966 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562448a55220 con 0x562448bac4f0 2024-05-08T22:06:42.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.390+0000 7fcdf7400700 1 -- 172.21.15.131:0/3516456966 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce0409efa0 con 0x562448bac4f0 2024-05-08T22:06:42.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.390+0000 7fce2beedb80 10 client.15144 fetch_fsmap learned FSMap version 23 2024-05-08T22:06:42.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.390+0000 7fce2beedb80 1 -- 172.21.15.131:0/3516456966 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562448929b10 con 0x562448bac4f0 2024-05-08T22:06:42.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.390+0000 7fcdf7400700 1 -- 172.21.15.131:0/3516456966 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562448a55220 con 0x562448bac4f0 2024-05-08T22:06:42.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.391+0000 7fcdf7400700 1 -- 172.21.15.131:0/3516456966 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fce040a31f0 con 0x562448bac4f0 2024-05-08T22:06:42.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.391+0000 7fce2beedb80 10 client.15144 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T22:06:42.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.391+0000 7fce2beedb80 10 client.15144 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T22:06:42.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.391+0000 7fce2beedb80 20 client.15144 populate_metadata read hostname 'smithi131' 2024-05-08T22:06:42.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.391+0000 7fce2beedb80 1 --2- 172.21.15.131:0/3516456966 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x562448bc4f40 0x562448bca200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.391+0000 7fce2beedb80 4 client.15144 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:06:42.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.391+0000 7fce2beedb80 1 -- 172.21.15.131:0/3516456966 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562448b7b7c0 con 0x562448bc4f40 2024-05-08T22:06:42.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.391+0000 7fce15000700 1 --2- 172.21.15.131:0/3516456966 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x562448bc4f40 0x562448bca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.392+0000 7fcdf7400700 10 client.15144 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:06:42.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.391+0000 7fce15000700 1 --2- 172.21.15.131:0/3516456966 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x562448bc4f40 0x562448bca200 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=263654e65926e6fa server_cookie=485b958c48012f49 in_seq=0 out_seq=0 2024-05-08T22:06:42.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.392+0000 7fcdf7400700 1 -- 172.21.15.131:0/3516456966 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562448b7b7c0 con 0x562448bc4f40 2024-05-08T22:06:42.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.392+0000 7fcdf7400700 10 client.15144 handle_command_reply: tid=0 2024-05-08T22:06:42.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.392+0000 7fce2beedb80 1 client.15144 shutdown 2024-05-08T22:06:42.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.392+0000 7fce2beedb80 1 -- 172.21.15.131:0/3516456966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bac4f0 msgr2=0x562448bba170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.392+0000 7fce2beedb80 1 --2- 172.21.15.131:0/3516456966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bac4f0 0x562448bba170 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fce04006010 tx=0x7fce04021b40 comp rx=0 tx=0).stop 2024-05-08T22:06:42.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.393+0000 7fce2beedb80 1 -- 172.21.15.131:0/3516456966 shutdown_connections 2024-05-08T22:06:42.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.393+0000 7fce2beedb80 1 --2- 172.21.15.131:0/3516456966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562448bad810 0x562448bc16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.393+0000 7fce2beedb80 1 --2- 172.21.15.131:0/3516456966 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x562448bc4f40 0x562448bca200 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.393+0000 7fce2beedb80 1 --2- 172.21.15.131:0/3516456966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bac4f0 0x562448bba170 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.393+0000 7fce2beedb80 1 --2- 172.21.15.131:0/3516456966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562448bace50 0x562448bba6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.393+0000 7fce2beedb80 1 -- 172.21.15.131:0/3516456966 >> 172.21.15.131:0/3516456966 conn(0x562448b0a1c0 msgr2=0x562448ba6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:42.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.393+0000 7fcdf7400700 0 client.15144 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:06:42.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.393+0000 7fce2beedb80 1 -- 172.21.15.131:0/3516456966 shutdown_connections 2024-05-08T22:06:42.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.393+0000 7fce2beedb80 1 -- 172.21.15.131:0/3516456966 wait complete. 2024-05-08T22:06:42.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.393+0000 7fce2beedb80 20 client.15144 trim_cache size 0 max 16384 2024-05-08T22:06:42.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.411+0000 7fce2beedb80 1 Processor -- start 2024-05-08T22:06:42.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.411+0000 7fce2beedb80 1 -- start start 2024-05-08T22:06:42.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.411+0000 7fce2beedb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bac4f0 0x562448bac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.411+0000 7fce2beedb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562448bace50 0x562448bc1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.411+0000 7fce2beedb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562448bad810 0x562448ba75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.411+0000 7fce2beedb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562448aba2b0 con 0x562448bac4f0 2024-05-08T22:06:42.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.411+0000 7fce2beedb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562448af5150 con 0x562448bad810 2024-05-08T22:06:42.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.411+0000 7fce2beedb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562448a9b5a0 con 0x562448bace50 2024-05-08T22:06:42.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.412+0000 7fce15a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bac4f0 0x562448bac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.412+0000 7fce15a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bac4f0 0x562448bac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40536/0 (socket says 172.21.15.131:40536) 2024-05-08T22:06:42.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.412+0000 7fce15a00700 1 -- 172.21.15.131:0/575930114 learned_addr learned my addr 172.21.15.131:0/575930114 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:42.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.412+0000 7fce16400700 1 --2- 172.21.15.131:0/575930114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562448bad810 0x562448ba75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.412+0000 7fce15000700 1 --2- 172.21.15.131:0/575930114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562448bace50 0x562448bc1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.412+0000 7fce15a00700 1 -- 172.21.15.131:0/575930114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562448bace50 msgr2=0x562448bc1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:42.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.412+0000 7fce15a00700 1 --2- 172.21.15.131:0/575930114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562448bace50 0x562448bc1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.412+0000 7fce15a00700 1 -- 172.21.15.131:0/575930114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562448bad810 msgr2=0x562448ba75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:42.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.412+0000 7fce15a00700 1 --2- 172.21.15.131:0/575930114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562448bad810 0x562448ba75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.412+0000 7fce15a00700 1 -- 172.21.15.131:0/575930114 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562448ba87f0 con 0x562448bac4f0 2024-05-08T22:06:42.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.412+0000 7fce16400700 1 --2- 172.21.15.131:0/575930114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562448bad810 0x562448ba75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:06:42.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.412+0000 7fce15a00700 1 --2- 172.21.15.131:0/575930114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bac4f0 0x562448bac8d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fce04000c00 tx=0x7fce040975e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f18bd9f0667a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:42.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.413+0000 7fce03400700 1 -- 172.21.15.131:0/575930114 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0409b5d0 con 0x562448bac4f0 2024-05-08T22:06:42.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.413+0000 7fce03400700 1 -- 172.21.15.131:0/575930114 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce040b93d0 con 0x562448bac4f0 2024-05-08T22:06:42.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.413+0000 7fce03400700 1 -- 172.21.15.131:0/575930114 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0409a8c0 con 0x562448bac4f0 2024-05-08T22:06:42.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.413+0000 7fce2beedb80 1 -- 172.21.15.131:0/575930114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bac4f0 msgr2=0x562448bac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.413+0000 7fce2beedb80 1 --2- 172.21.15.131:0/575930114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bac4f0 0x562448bac8d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fce04000c00 tx=0x7fce040975e0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.413+0000 7fce2beedb80 1 -- 172.21.15.131:0/575930114 shutdown_connections 2024-05-08T22:06:42.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.413+0000 7fce2beedb80 1 --2- 172.21.15.131:0/575930114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562448bad810 0x562448ba75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.413+0000 7fce2beedb80 1 --2- 172.21.15.131:0/575930114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bac4f0 0x562448bac8d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.413+0000 7fce2beedb80 1 --2- 172.21.15.131:0/575930114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562448bace50 0x562448bc1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.413+0000 7fce2beedb80 1 -- 172.21.15.131:0/575930114 >> 172.21.15.131:0/575930114 conn(0x562448b0a1c0 msgr2=0x562448badbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:42.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.413+0000 7fce2beedb80 1 -- 172.21.15.131:0/575930114 shutdown_connections 2024-05-08T22:06:42.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.413+0000 7fce2beedb80 1 -- 172.21.15.131:0/575930114 wait complete. 2024-05-08T22:06:42.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.414+0000 7fce2beedb80 1 -- start start 2024-05-08T22:06:42.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.414+0000 7fce2beedb80 1 Processor -- start 2024-05-08T22:06:42.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.414+0000 7fce2beedb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562448bac4f0 0x562448bbb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.414+0000 7fce2beedb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bace50 0x562448bbbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.414+0000 7fce2beedb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562448bad810 0x562448bc0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.414+0000 7fce2beedb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562448bc1050 con 0x562448bace50 2024-05-08T22:06:42.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.414+0000 7fce2beedb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562448bc11c0 con 0x562448bad810 2024-05-08T22:06:42.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.414+0000 7fce2beedb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562448bc1380 con 0x562448bac4f0 2024-05-08T22:06:42.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.414+0000 7fce2beedb80 10 client.0 _ll_register_callbacks cb 0x562448aa1ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:06:42.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.415+0000 7fce16400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562448bad810 0x562448bc0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.415+0000 7fce15000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bace50 0x562448bbbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.415+0000 7fce16400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562448bad810 0x562448bc0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:53668/0 (socket says 172.21.15.131:53668) 2024-05-08T22:06:42.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.415+0000 7fce16400700 1 -- 172.21.15.131:0/527182434 learned_addr learned my addr 172.21.15.131:0/527182434 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:42.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.415+0000 7fce15a00700 1 --2- 172.21.15.131:0/527182434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562448bac4f0 0x562448bbb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.415+0000 7fce16400700 1 -- 172.21.15.131:0/527182434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562448bac4f0 msgr2=0x562448bbb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:42.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.415+0000 7fce16400700 1 --2- 172.21.15.131:0/527182434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562448bac4f0 0x562448bbb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.415+0000 7fce16400700 1 -- 172.21.15.131:0/527182434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bace50 msgr2=0x562448bbbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:42.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.415+0000 7fce16400700 1 --2- 172.21.15.131:0/527182434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bace50 0x562448bbbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.415+0000 7fce16400700 1 -- 172.21.15.131:0/527182434 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562448929b10 con 0x562448bad810 2024-05-08T22:06:42.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.415+0000 7fce15000700 1 --2- 172.21.15.131:0/527182434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bace50 0x562448bbbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.415+0000 7fce15a00700 1 --2- 172.21.15.131:0/527182434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562448bac4f0 0x562448bbb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.415+0000 7fce16400700 1 --2- 172.21.15.131:0/527182434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562448bad810 0x562448bc0b10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fce080093c0 tx=0x7fce080093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6a7ea24795dc8b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.415+0000 7fcdf7400700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:06:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.416+0000 7fcdf7400700 1 -- 172.21.15.131:0/527182434 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0800e040 con 0x562448bad810 2024-05-08T22:06:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.416+0000 7fcdf7400700 1 -- 172.21.15.131:0/527182434 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fce08003e10 con 0x562448bad810 2024-05-08T22:06:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.416+0000 7fce2beedb80 1 -- 172.21.15.131:0/527182434 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcdfc004cf0 con 0x562448bad810 2024-05-08T22:06:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.416+0000 7fcdf7400700 1 -- 172.21.15.131:0/527182434 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce08004bb0 con 0x562448bad810 2024-05-08T22:06:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.416+0000 7fce2beedb80 10 client.25181 fetch_fsmap learned FSMap version 23 2024-05-08T22:06:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.416+0000 7fce2beedb80 1 -- 172.21.15.131:0/527182434 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562448ac2510 con 0x562448bad810 2024-05-08T22:06:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.416+0000 7fcdf7400700 1 -- 172.21.15.131:0/527182434 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdfc004cf0 con 0x562448bad810 2024-05-08T22:06:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.416+0000 7fcdf7400700 1 -- 172.21.15.131:0/527182434 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fce08004450 con 0x562448bad810 2024-05-08T22:06:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.416+0000 7fce2beedb80 10 client.25181 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T22:06:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.416+0000 7fce2beedb80 10 client.25181 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T22:06:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.416+0000 7fce2beedb80 20 client.25181 populate_metadata read hostname 'smithi131' 2024-05-08T22:06:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.416+0000 7fce2beedb80 1 --2- 172.21.15.131:0/527182434 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x562448c203a0 0x562448c1ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.416+0000 7fce2beedb80 4 client.25181 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:06:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.416+0000 7fce2beedb80 1 -- 172.21.15.131:0/527182434 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562448af4e70 con 0x562448c203a0 2024-05-08T22:06:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.417+0000 7fce15a00700 1 --2- 172.21.15.131:0/527182434 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x562448c203a0 0x562448c1ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.417+0000 7fcdf7400700 10 client.25181 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:06:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.417+0000 7fce15a00700 1 --2- 172.21.15.131:0/527182434 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x562448c203a0 0x562448c1ccf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=58c0c7c1cdf365b7 server_cookie=3d4bb22cf40f4a2e in_seq=0 out_seq=0 2024-05-08T22:06:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.417+0000 7fcdf7400700 1 -- 172.21.15.131:0/527182434 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x562448af4e70 con 0x562448c203a0 2024-05-08T22:06:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.417+0000 7fcdf7400700 10 client.25181 handle_command_reply: tid=0 2024-05-08T22:06:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.417+0000 7fce2beedb80 1 client.25181 shutdown 2024-05-08T22:06:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.418+0000 7fce2beedb80 1 -- 172.21.15.131:0/527182434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562448bad810 msgr2=0x562448bc0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.418+0000 7fce2beedb80 1 --2- 172.21.15.131:0/527182434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562448bad810 0x562448bc0b10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fce080093c0 tx=0x7fce080093f0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.418+0000 7fce2beedb80 1 -- 172.21.15.131:0/527182434 shutdown_connections 2024-05-08T22:06:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.418+0000 7fce2beedb80 1 --2- 172.21.15.131:0/527182434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562448bad810 0x562448bc0b10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.418+0000 7fce2beedb80 1 --2- 172.21.15.131:0/527182434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562448bac4f0 0x562448bbb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.418+0000 7fce2beedb80 1 --2- 172.21.15.131:0/527182434 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x562448c203a0 0x562448c1ccf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.418+0000 7fce2beedb80 1 --2- 172.21.15.131:0/527182434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562448bace50 0x562448bbbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.418+0000 7fce2beedb80 1 -- 172.21.15.131:0/527182434 >> 172.21.15.131:0/527182434 conn(0x562448b0a1c0 msgr2=0x562448bb9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.418+0000 7fcdf7400700 0 client.25181 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:06:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.418+0000 7fce2beedb80 1 -- 172.21.15.131:0/527182434 shutdown_connections 2024-05-08T22:06:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.418+0000 7fce2beedb80 1 -- 172.21.15.131:0/527182434 wait complete. 2024-05-08T22:06:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.418+0000 7fce2beedb80 20 client.25181 trim_cache size 0 max 16384 2024-05-08T22:06:42.423 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:06:42.423 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "06f1a696-0d86-11ef-bc97-c7b262605968", 2024-05-08T22:06:42.423 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 1, 2024-05-08T22:06:42.423 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24562, 2024-05-08T22:06:42.424 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-08T22:06:42.424 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-08T22:06:42.424 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-08T22:06:42.424 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 64.936292222000006, 2024-05-08T22:06:42.424 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 23, 2024-05-08T22:06:42.424 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 87, 2024-05-08T22:06:42.424 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 83, 2024-05-08T22:06:42.424 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 96.396869770999999 2024-05-08T22:06:42.424 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:06:42.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.419+0000 7fce18400700 1 -- 172.21.15.131:0/800505153 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fcdf80a11b0 msgr2=0x7fcdf80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.419+0000 7fce18400700 1 --2- 172.21.15.131:0/800505153 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fcdf80a11b0 0x7fcdf80a3670 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009cb0 tx=0x7fcdfc011720 comp rx=0 tx=0).stop 2024-05-08T22:06:42.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.419+0000 7fce18400700 1 -- 172.21.15.131:0/800505153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce101019e0 msgr2=0x7fce10196ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.419+0000 7fce18400700 1 --2- 172.21.15.131:0/800505153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce101019e0 0x7fce10196ec0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fce04005910 tx=0x7fce04009580 comp rx=0 tx=0).stop 2024-05-08T22:06:42.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.419+0000 7fce18400700 1 -- 172.21.15.131:0/800505153 shutdown_connections 2024-05-08T22:06:42.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.419+0000 7fce18400700 1 --2- 172.21.15.131:0/800505153 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fcdf80a11b0 0x7fcdf80a3670 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.419+0000 7fce18400700 1 --2- 172.21.15.131:0/800505153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce10106c40 0x7fce1019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.419+0000 7fce18400700 1 --2- 172.21.15.131:0/800505153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce101019e0 0x7fce10196ec0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.419+0000 7fce18400700 1 --2- 172.21.15.131:0/800505153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce10104310 0x7fce10197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.419+0000 7fce18400700 1 -- 172.21.15.131:0/800505153 >> 172.21.15.131:0/800505153 conn(0x7fce100fb3f0 msgr2=0x7fce10103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:42.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.419+0000 7fce18400700 1 -- 172.21.15.131:0/800505153 shutdown_connections 2024-05-08T22:06:42.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.419+0000 7fce18400700 1 -- 172.21.15.131:0/800505153 wait complete. 2024-05-08T22:06:42.438 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T22:06:42.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.635+0000 7f553e200700 1 -- 172.21.15.131:0/1315034682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5538100f20 msgr2=0x7f5538101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.635+0000 7f553e200700 1 --2- 172.21.15.131:0/1315034682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5538100f20 0x7f5538101360 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5528009990 tx=0x7f5528009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.635+0000 7f553e200700 1 -- 172.21.15.131:0/1315034682 shutdown_connections 2024-05-08T22:06:42.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.635+0000 7f553e200700 1 --2- 172.21.15.131:0/1315034682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5538102ab0 0x7f55381093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.635+0000 7f553e200700 1 --2- 172.21.15.131:0/1315034682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5538100f20 0x7f5538101360 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.635+0000 7f553e200700 1 --2- 172.21.15.131:0/1315034682 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55381020f0 0x7f5538102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.635+0000 7f553e200700 1 -- 172.21.15.131:0/1315034682 >> 172.21.15.131:0/1315034682 conn(0x7f55380fc500 msgr2=0x7f55380fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:42.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.635+0000 7f553e200700 1 -- 172.21.15.131:0/1315034682 shutdown_connections 2024-05-08T22:06:42.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.635+0000 7f553e200700 1 -- 172.21.15.131:0/1315034682 wait complete. 2024-05-08T22:06:42.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.637+0000 7f553e200700 1 Processor -- start 2024-05-08T22:06:42.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.637+0000 7f553e200700 1 -- start start 2024-05-08T22:06:42.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.637+0000 7f553e200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5538100f20 0x7f5538199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.637+0000 7f553e200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55381020f0 0x7f5538199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.638+0000 7f553e200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5538102ab0 0x7f55381a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.638+0000 7f553e200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55381a0c80 con 0x7f5538102ab0 2024-05-08T22:06:42.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.638+0000 7f553e200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55381a0dc0 con 0x7f55381020f0 2024-05-08T22:06:42.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.638+0000 7f553e200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55381a0f50 con 0x7f5538100f20 2024-05-08T22:06:42.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.638+0000 7f5537400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5538100f20 0x7f5538199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.638+0000 7f5537e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5538102ab0 0x7f55381a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.638+0000 7f5537400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5538100f20 0x7f5538199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59952/0 (socket says 172.21.15.131:59952) 2024-05-08T22:06:42.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.638+0000 7f5537400700 1 -- 172.21.15.131:0/931421716 learned_addr learned my addr 172.21.15.131:0/931421716 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:42.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.638+0000 7f5536a00700 1 --2- 172.21.15.131:0/931421716 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55381020f0 0x7f5538199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.638+0000 7f5537e00700 1 -- 172.21.15.131:0/931421716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5538100f20 msgr2=0x7f5538199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.638+0000 7f5537e00700 1 --2- 172.21.15.131:0/931421716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5538100f20 0x7f5538199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.638+0000 7f5537e00700 1 -- 172.21.15.131:0/931421716 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55381020f0 msgr2=0x7f5538199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.638+0000 7f5537e00700 1 --2- 172.21.15.131:0/931421716 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55381020f0 0x7f5538199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.638+0000 7f5537e00700 1 -- 172.21.15.131:0/931421716 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55280095f0 con 0x7f5538102ab0 2024-05-08T22:06:42.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.638+0000 7f5537400700 1 --2- 172.21.15.131:0/931421716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5538100f20 0x7f5538199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:42.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.639+0000 7f5537e00700 1 --2- 172.21.15.131:0/931421716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5538102ab0 0x7f55381a06b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f553000efc0 tx=0x7f553000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:42.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.639+0000 7f552fe00700 1 -- 172.21.15.131:0/931421716 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5530018070 con 0x7f5538102ab0 2024-05-08T22:06:42.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.639+0000 7f553e200700 1 -- 172.21.15.131:0/931421716 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55381a11d0 con 0x7f5538102ab0 2024-05-08T22:06:42.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.639+0000 7f552fe00700 1 -- 172.21.15.131:0/931421716 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f553000f040 con 0x7f5538102ab0 2024-05-08T22:06:42.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.639+0000 7f552fe00700 1 -- 172.21.15.131:0/931421716 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553000cd40 con 0x7f5538102ab0 2024-05-08T22:06:42.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.639+0000 7f553e200700 1 -- 172.21.15.131:0/931421716 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55381a1730 con 0x7f5538102ab0 2024-05-08T22:06:42.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.642+0000 7f552fe00700 1 -- 172.21.15.131:0/931421716 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f5530003a00 con 0x7f5538102ab0 2024-05-08T22:06:42.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.643+0000 7f552fe00700 1 --2- 172.21.15.131:0/931421716 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f551c0a11b0 0x7f551c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.643+0000 7f5551cc8b80 1 Processor -- start 2024-05-08T22:06:42.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.643+0000 7f552fe00700 1 -- 172.21.15.131:0/931421716 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f55300cc980 con 0x7f5538102ab0 2024-05-08T22:06:42.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.643+0000 7f5551cc8b80 1 -- start start 2024-05-08T22:06:42.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.643+0000 7f5537400700 1 --2- 172.21.15.131:0/931421716 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f551c0a11b0 0x7f551c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.644+0000 7f5551cc8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7a07664f0 0x55c7a0766910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.644+0000 7f5551cc8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a0766e50 0x55c7a07672d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.644+0000 7f5551cc8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7a0767810 0x55c7a0767c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.644+0000 7f5551cc8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a06bd100 con 0x55c7a0766e50 2024-05-08T22:06:42.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.644+0000 7f5551cc8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a06baac0 con 0x55c7a0767810 2024-05-08T22:06:42.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.644+0000 7f5551cc8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a06b63b0 con 0x55c7a07664f0 2024-05-08T22:06:42.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.644+0000 7f5536a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a0766e50 0x55c7a07672d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.644+0000 7f5536a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a0766e50 0x55c7a07672d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40566/0 (socket says 172.21.15.131:40566) 2024-05-08T22:06:42.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.644+0000 7f5537400700 1 --2- 172.21.15.131:0/931421716 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f551c0a11b0 0x7f551c0a3670 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f552800b2e0 tx=0x7f5528005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:42.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.644+0000 7f5536a00700 1 -- 172.21.15.131:0/829464409 learned_addr learned my addr 172.21.15.131:0/829464409 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:42.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.644+0000 7f5537e00700 1 --2- 172.21.15.131:0/829464409 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7a0767810 0x55c7a0767c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.644+0000 7f5537400700 1 --2- 172.21.15.131:0/829464409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7a07664f0 0x55c7a0766910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.644+0000 7f5536a00700 1 -- 172.21.15.131:0/829464409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7a07664f0 msgr2=0x55c7a0766910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:42.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.645+0000 7f5536a00700 1 --2- 172.21.15.131:0/829464409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7a07664f0 0x55c7a0766910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.645+0000 7f5536a00700 1 -- 172.21.15.131:0/829464409 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7a0767810 msgr2=0x55c7a0767c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:42.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.645+0000 7f5536a00700 1 --2- 172.21.15.131:0/829464409 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7a0767810 0x55c7a0767c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.645+0000 7f5536a00700 1 -- 172.21.15.131:0/829464409 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7a0537360 con 0x55c7a0766e50 2024-05-08T22:06:42.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.645+0000 7f5537400700 1 --2- 172.21.15.131:0/829464409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7a07664f0 0x55c7a0766910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:06:42.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.645+0000 7f5536a00700 1 --2- 172.21.15.131:0/829464409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a0766e50 0x55c7a07672d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5524009d80 tx=0x7f5524011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1548c22e628dc25 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:42.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.645+0000 7f5523e00700 1 -- 172.21.15.131:0/829464409 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5524004230 con 0x55c7a0766e50 2024-05-08T22:06:42.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.645+0000 7f5523e00700 1 -- 172.21.15.131:0/829464409 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5524004390 con 0x55c7a0766e50 2024-05-08T22:06:42.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.645+0000 7f5523e00700 1 -- 172.21.15.131:0/829464409 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5524019860 con 0x55c7a0766e50 2024-05-08T22:06:42.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.646+0000 7f5551cc8b80 1 -- 172.21.15.131:0/829464409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a0766e50 msgr2=0x55c7a07672d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.646+0000 7f5551cc8b80 1 --2- 172.21.15.131:0/829464409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a0766e50 0x55c7a07672d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5524009d80 tx=0x7f5524011720 comp rx=0 tx=0).stop 2024-05-08T22:06:42.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.646+0000 7f5551cc8b80 1 -- 172.21.15.131:0/829464409 shutdown_connections 2024-05-08T22:06:42.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.646+0000 7f5551cc8b80 1 --2- 172.21.15.131:0/829464409 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7a0767810 0x55c7a0767c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.646+0000 7f5551cc8b80 1 --2- 172.21.15.131:0/829464409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7a07664f0 0x55c7a0766910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.646+0000 7f5551cc8b80 1 --2- 172.21.15.131:0/829464409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a0766e50 0x55c7a07672d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.646+0000 7f5551cc8b80 1 -- 172.21.15.131:0/829464409 >> 172.21.15.131:0/829464409 conn(0x55c7a06c41c0 msgr2=0x55c7a0760d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:42.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.646+0000 7f5551cc8b80 1 -- 172.21.15.131:0/829464409 shutdown_connections 2024-05-08T22:06:42.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.646+0000 7f5551cc8b80 1 -- 172.21.15.131:0/829464409 wait complete. 2024-05-08T22:06:42.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.647+0000 7f5551cc8b80 1 -- start start 2024-05-08T22:06:42.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.647+0000 7f5551cc8b80 1 Processor -- start 2024-05-08T22:06:42.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.647+0000 7f5551cc8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7a07664f0 0x55c7a076fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.647+0000 7f5551cc8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a0766e50 0x55c7a07702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.647+0000 7f5551cc8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7a0767810 0x55c7a0777330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.647+0000 7f5551cc8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a0652510 con 0x55c7a0766e50 2024-05-08T22:06:42.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.647+0000 7f5551cc8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a06521b0 con 0x55c7a0767810 2024-05-08T22:06:42.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.647+0000 7f5551cc8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a0539650 con 0x55c7a07664f0 2024-05-08T22:06:42.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.647+0000 7f5551cc8b80 10 client.0 _ll_register_callbacks cb 0x55c7a065bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:06:42.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.647+0000 7f5536a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a0766e50 0x55c7a07702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.647+0000 7f5536a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a0766e50 0x55c7a07702f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40568/0 (socket says 172.21.15.131:40568) 2024-05-08T22:06:42.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.648+0000 7f5536a00700 1 -- 172.21.15.131:0/1112491470 learned_addr learned my addr 172.21.15.131:0/1112491470 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:42.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.648+0000 7f5537400700 1 --2- 172.21.15.131:0/1112491470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7a07664f0 0x55c7a076fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.648+0000 7f5537e00700 1 --2- 172.21.15.131:0/1112491470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7a0767810 0x55c7a0777330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.648+0000 7f5536a00700 1 -- 172.21.15.131:0/1112491470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7a07664f0 msgr2=0x55c7a076fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:42.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.648+0000 7f5536a00700 1 --2- 172.21.15.131:0/1112491470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7a07664f0 0x55c7a076fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.648+0000 7f5536a00700 1 -- 172.21.15.131:0/1112491470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7a0767810 msgr2=0x55c7a0777330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:42.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.648+0000 7f5536a00700 1 --2- 172.21.15.131:0/1112491470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7a0767810 0x55c7a0777330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.648+0000 7f5536a00700 1 -- 172.21.15.131:0/1112491470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55240099e0 con 0x55c7a0766e50 2024-05-08T22:06:42.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.648+0000 7f5537400700 1 --2- 172.21.15.131:0/1112491470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7a07664f0 0x55c7a076fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:42.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.648+0000 7f5536a00700 1 --2- 172.21.15.131:0/1112491470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a0766e50 0x55c7a07702f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5524000c00 tx=0x7f55240038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba0697232548100 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:42.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.648+0000 7f5520c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:06:42.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.648+0000 7f5520c00700 1 -- 172.21.15.131:0/1112491470 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5524003dd0 con 0x55c7a0766e50 2024-05-08T22:06:42.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.648+0000 7f5551cc8b80 1 -- 172.21.15.131:0/1112491470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7a060f220 con 0x55c7a0766e50 2024-05-08T22:06:42.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.649+0000 7f5520c00700 1 -- 172.21.15.131:0/1112491470 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f5524005ce0 con 0x55c7a0766e50 2024-05-08T22:06:42.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.649+0000 7f5520c00700 1 -- 172.21.15.131:0/1112491470 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5524003ae0 con 0x55c7a0766e50 2024-05-08T22:06:42.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.649+0000 7f5551cc8b80 10 client.15162 fetch_fsmap learned FSMap version 23 2024-05-08T22:06:42.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.649+0000 7f5551cc8b80 1 -- 172.21.15.131:0/1112491470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c7a04e3b10 con 0x55c7a0766e50 2024-05-08T22:06:42.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.649+0000 7f5520c00700 1 -- 172.21.15.131:0/1112491470 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c7a060f220 con 0x55c7a0766e50 2024-05-08T22:06:42.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.649+0000 7f5520c00700 1 -- 172.21.15.131:0/1112491470 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5524021040 con 0x55c7a0766e50 2024-05-08T22:06:42.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.649+0000 7f5551cc8b80 10 client.15162 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T22:06:42.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.649+0000 7f5551cc8b80 10 client.15162 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T22:06:42.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.649+0000 7f5551cc8b80 20 client.15162 populate_metadata read hostname 'smithi131' 2024-05-08T22:06:42.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.649+0000 7f5551cc8b80 1 --2- 172.21.15.131:0/1112491470 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55c7a077aea0 0x55c7a07801a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.649+0000 7f5551cc8b80 4 client.15162 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:06:42.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.649+0000 7f5551cc8b80 1 -- 172.21.15.131:0/1112491470 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7a07357c0 con 0x55c7a077aea0 2024-05-08T22:06:42.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.650+0000 7f5537400700 1 --2- 172.21.15.131:0/1112491470 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55c7a077aea0 0x55c7a07801a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.650+0000 7f5520c00700 10 client.15162 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:06:42.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.650+0000 7f5537400700 1 --2- 172.21.15.131:0/1112491470 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55c7a077aea0 0x55c7a07801a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66f4db5d0f09f37 server_cookie=59eb94660b0a7a51 in_seq=0 out_seq=0 2024-05-08T22:06:42.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.651+0000 7f5520c00700 1 -- 172.21.15.131:0/1112491470 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c7a07357c0 con 0x55c7a077aea0 2024-05-08T22:06:42.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.651+0000 7f5520c00700 10 client.15162 handle_command_reply: tid=0 2024-05-08T22:06:42.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.651+0000 7f5551cc8b80 1 client.15162 shutdown 2024-05-08T22:06:42.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.651+0000 7f5551cc8b80 1 -- 172.21.15.131:0/1112491470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a0766e50 msgr2=0x55c7a07702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.651+0000 7f5551cc8b80 1 --2- 172.21.15.131:0/1112491470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a0766e50 0x55c7a07702f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5524000c00 tx=0x7f55240038b0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.651+0000 7f5551cc8b80 1 -- 172.21.15.131:0/1112491470 shutdown_connections 2024-05-08T22:06:42.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.651+0000 7f5551cc8b80 1 --2- 172.21.15.131:0/1112491470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7a0767810 0x55c7a0777330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.651+0000 7f5551cc8b80 1 --2- 172.21.15.131:0/1112491470 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55c7a077aea0 0x55c7a07801a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.651+0000 7f5551cc8b80 1 --2- 172.21.15.131:0/1112491470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7a07664f0 0x55c7a076fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.651+0000 7f5551cc8b80 1 --2- 172.21.15.131:0/1112491470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a0766e50 0x55c7a07702f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.651+0000 7f5520c00700 0 client.15162 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:06:42.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.651+0000 7f5551cc8b80 1 -- 172.21.15.131:0/1112491470 >> 172.21.15.131:0/1112491470 conn(0x55c7a06c41c0 msgr2=0x55c7a0760550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:42.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.652+0000 7f5551cc8b80 1 -- 172.21.15.131:0/1112491470 shutdown_connections 2024-05-08T22:06:42.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.652+0000 7f5551cc8b80 1 -- 172.21.15.131:0/1112491470 wait complete. 2024-05-08T22:06:42.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.652+0000 7f5551cc8b80 20 client.15162 trim_cache size 0 max 16384 2024-05-08T22:06:42.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.670+0000 7f5551cc8b80 1 Processor -- start 2024-05-08T22:06:42.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.670+0000 7f5551cc8b80 1 -- start start 2024-05-08T22:06:42.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.670+0000 7f5551cc8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7a07664f0 0x55c7a07668d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.670+0000 7f5551cc8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a0766e50 0x55c7a07779e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.670+0000 7f5551cc8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7a0767810 0x55c7a07772f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.670+0000 7f5551cc8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a06b3010 con 0x55c7a0766e50 2024-05-08T22:06:42.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.670+0000 7f5551cc8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a06555a0 con 0x55c7a0767810 2024-05-08T22:06:42.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.670+0000 7f5551cc8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a07784b0 con 0x55c7a07664f0 2024-05-08T22:06:42.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.671+0000 7f5536a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a0766e50 0x55c7a07779e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.671+0000 7f5537400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7a07664f0 0x55c7a07668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.671+0000 7f5536a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a0766e50 0x55c7a07779e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40572/0 (socket says 172.21.15.131:40572) 2024-05-08T22:06:42.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.671+0000 7f5536a00700 1 -- 172.21.15.131:0/2073979168 learned_addr learned my addr 172.21.15.131:0/2073979168 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:42.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.671+0000 7f5537e00700 1 --2- 172.21.15.131:0/2073979168 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7a0767810 0x55c7a07772f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.671+0000 7f5537400700 1 -- 172.21.15.131:0/2073979168 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7a0767810 msgr2=0x55c7a07772f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:42.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.671+0000 7f5537400700 1 --2- 172.21.15.131:0/2073979168 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7a0767810 0x55c7a07772f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.671+0000 7f5537400700 1 -- 172.21.15.131:0/2073979168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a0766e50 msgr2=0x55c7a07779e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:42.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.671+0000 7f5537400700 1 --2- 172.21.15.131:0/2073979168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a0766e50 0x55c7a07779e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.671+0000 7f5537400700 1 -- 172.21.15.131:0/2073979168 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7a067c510 con 0x55c7a07664f0 2024-05-08T22:06:42.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.671+0000 7f5536a00700 1 --2- 172.21.15.131:0/2073979168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a0766e50 0x55c7a07779e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:42.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.671+0000 7f5537400700 1 --2- 172.21.15.131:0/2073979168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7a07664f0 0x55c7a07668d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f55280092e0 tx=0x7f5528023000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ace6364bc867be1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:42.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.671+0000 7f552d200700 1 -- 172.21.15.131:0/2073979168 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004d40 con 0x55c7a07664f0 2024-05-08T22:06:42.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.672+0000 7f552d200700 1 -- 172.21.15.131:0/2073979168 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f552802a040 con 0x55c7a07664f0 2024-05-08T22:06:42.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.672+0000 7f552d200700 1 -- 172.21.15.131:0/2073979168 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552800f750 con 0x55c7a07664f0 2024-05-08T22:06:42.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.672+0000 7f5551cc8b80 1 -- 172.21.15.131:0/2073979168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7a07664f0 msgr2=0x55c7a07668d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.672+0000 7f5551cc8b80 1 --2- 172.21.15.131:0/2073979168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7a07664f0 0x55c7a07668d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f55280092e0 tx=0x7f5528023000 comp rx=0 tx=0).stop 2024-05-08T22:06:42.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.672+0000 7f5551cc8b80 1 -- 172.21.15.131:0/2073979168 shutdown_connections 2024-05-08T22:06:42.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.672+0000 7f5551cc8b80 1 --2- 172.21.15.131:0/2073979168 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7a0767810 0x55c7a07772f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.672+0000 7f5551cc8b80 1 --2- 172.21.15.131:0/2073979168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7a07664f0 0x55c7a07668d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.672+0000 7f5551cc8b80 1 --2- 172.21.15.131:0/2073979168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a0766e50 0x55c7a07779e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.672+0000 7f5551cc8b80 1 -- 172.21.15.131:0/2073979168 >> 172.21.15.131:0/2073979168 conn(0x55c7a06c41c0 msgr2=0x55c7a0767bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:42.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.672+0000 7f5551cc8b80 1 -- 172.21.15.131:0/2073979168 shutdown_connections 2024-05-08T22:06:42.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.672+0000 7f5551cc8b80 1 -- 172.21.15.131:0/2073979168 wait complete. 2024-05-08T22:06:42.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.673+0000 7f5551cc8b80 1 -- start start 2024-05-08T22:06:42.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.673+0000 7f5551cc8b80 1 Processor -- start 2024-05-08T22:06:42.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.673+0000 7f5551cc8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a07664f0 0x55c7a07841c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.673+0000 7f5551cc8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7a0766e50 0x55c7a07772f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.673+0000 7f5551cc8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7a0767810 0x55c7a0784720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.673+0000 7f5551cc8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a0784e30 con 0x55c7a07664f0 2024-05-08T22:06:42.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.673+0000 7f5551cc8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a0784fa0 con 0x55c7a0767810 2024-05-08T22:06:42.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.673+0000 7f5551cc8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a075fcc0 con 0x55c7a0766e50 2024-05-08T22:06:42.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.673+0000 7f5551cc8b80 10 client.0 _ll_register_callbacks cb 0x55c7a065bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:06:42.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.673+0000 7f5537e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7a0767810 0x55c7a0784720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.673+0000 7f5537400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a07664f0 0x55c7a07841c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.673+0000 7f5537e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7a0767810 0x55c7a0784720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:53746/0 (socket says 172.21.15.131:53746) 2024-05-08T22:06:42.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.673+0000 7f5537e00700 1 -- 172.21.15.131:0/420071381 learned_addr learned my addr 172.21.15.131:0/420071381 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:42.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.674+0000 7f5536a00700 1 --2- 172.21.15.131:0/420071381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7a0766e50 0x55c7a07772f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.674+0000 7f5537e00700 1 -- 172.21.15.131:0/420071381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7a0766e50 msgr2=0x55c7a07772f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:42.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.674+0000 7f5537e00700 1 --2- 172.21.15.131:0/420071381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7a0766e50 0x55c7a07772f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.674+0000 7f5537e00700 1 -- 172.21.15.131:0/420071381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a07664f0 msgr2=0x55c7a07841c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:06:42.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.674+0000 7f5537e00700 1 --2- 172.21.15.131:0/420071381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a07664f0 0x55c7a07841c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.674+0000 7f5537e00700 1 -- 172.21.15.131:0/420071381 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7a04e3b10 con 0x55c7a0767810 2024-05-08T22:06:42.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.674+0000 7f5537400700 1 --2- 172.21.15.131:0/420071381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a07664f0 0x55c7a07841c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:42.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.674+0000 7f5537e00700 1 --2- 172.21.15.131:0/420071381 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7a0767810 0x55c7a0784720 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5530099910 tx=0x7f55300994c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a8ba0239f4f134a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:42.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.674+0000 7f5520c00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:06:42.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.674+0000 7f5520c00700 1 -- 172.21.15.131:0/420071381 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553009a080 con 0x55c7a0767810 2024-05-08T22:06:42.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.674+0000 7f5520c00700 1 -- 172.21.15.131:0/420071381 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f5530099980 con 0x55c7a0767810 2024-05-08T22:06:42.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.674+0000 7f5551cc8b80 1 -- 172.21.15.131:0/420071381 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5524004390 con 0x55c7a0767810 2024-05-08T22:06:42.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.674+0000 7f5520c00700 1 -- 172.21.15.131:0/420071381 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f553009d7b0 con 0x55c7a0767810 2024-05-08T22:06:42.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.674+0000 7f5551cc8b80 10 client.25199 fetch_fsmap learned FSMap version 23 2024-05-08T22:06:42.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.674+0000 7f5551cc8b80 1 -- 172.21.15.131:0/420071381 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f5538109d00 con 0x55c7a0767810 2024-05-08T22:06:42.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.675+0000 7f5520c00700 1 -- 172.21.15.131:0/420071381 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5530063030 con 0x55c7a0767810 2024-05-08T22:06:42.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.675+0000 7f5520c00700 1 -- 172.21.15.131:0/420071381 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5530099be0 con 0x55c7a0767810 2024-05-08T22:06:42.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.675+0000 7f5551cc8b80 10 client.25199 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T22:06:42.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.675+0000 7f5551cc8b80 10 client.25199 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T22:06:42.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.675+0000 7f5551cc8b80 20 client.25199 populate_metadata read hostname 'smithi131' 2024-05-08T22:06:42.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.675+0000 7f5551cc8b80 1 --2- 172.21.15.131:0/420071381 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55c7a07d5d00 0x55c7a07d2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:42.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.675+0000 7f5551cc8b80 4 client.25199 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:06:42.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.675+0000 7f5551cc8b80 1 -- 172.21.15.131:0/420071381 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c7a06aee70 con 0x55c7a07d5d00 2024-05-08T22:06:42.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.675+0000 7f5537400700 1 --2- 172.21.15.131:0/420071381 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55c7a07d5d00 0x55c7a07d2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:42.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.676+0000 7f5537400700 1 --2- 172.21.15.131:0/420071381 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55c7a07d5d00 0x55c7a07d2690 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aacd0dc9cd56d522 server_cookie=9ab063acfa00c139 in_seq=0 out_seq=0 2024-05-08T22:06:42.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.676+0000 7f5520c00700 10 client.25199 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:06:42.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.676+0000 7f5520c00700 1 -- 172.21.15.131:0/420071381 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55c7a06aee70 con 0x55c7a07d5d00 2024-05-08T22:06:42.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.676+0000 7f5520c00700 10 client.25199 handle_command_reply: tid=0 2024-05-08T22:06:42.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.676+0000 7f5551cc8b80 1 client.25199 shutdown 2024-05-08T22:06:42.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.676+0000 7f5551cc8b80 1 -- 172.21.15.131:0/420071381 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7a0767810 msgr2=0x55c7a0784720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.676+0000 7f5551cc8b80 1 --2- 172.21.15.131:0/420071381 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7a0767810 0x55c7a0784720 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5530099910 tx=0x7f55300994c0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.676+0000 7f5551cc8b80 1 -- 172.21.15.131:0/420071381 shutdown_connections 2024-05-08T22:06:42.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.676+0000 7f5551cc8b80 1 --2- 172.21.15.131:0/420071381 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7a0767810 0x55c7a0784720 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.676+0000 7f5551cc8b80 1 --2- 172.21.15.131:0/420071381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7a07664f0 0x55c7a07841c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.676+0000 7f5551cc8b80 1 --2- 172.21.15.131:0/420071381 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55c7a07d5d00 0x55c7a07d2690 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.676+0000 7f5551cc8b80 1 --2- 172.21.15.131:0/420071381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7a0766e50 0x55c7a07772f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.676+0000 7f5520c00700 0 client.25199 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:06:42.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.676+0000 7f5551cc8b80 1 -- 172.21.15.131:0/420071381 >> 172.21.15.131:0/420071381 conn(0x55c7a06c41c0 msgr2=0x55c7a0760d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:42.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.676+0000 7f5551cc8b80 1 -- 172.21.15.131:0/420071381 shutdown_connections 2024-05-08T22:06:42.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.676+0000 7f5551cc8b80 1 -- 172.21.15.131:0/420071381 wait complete. 2024-05-08T22:06:42.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.677+0000 7f5551cc8b80 20 client.25199 trim_cache size 0 max 16384 2024-05-08T22:06:42.692 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:06:42.692 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "06f1a696-0d86-11ef-bc97-c7b262605968", 2024-05-08T22:06:42.692 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-05-08T22:06:42.692 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24566, 2024-05-08T22:06:42.692 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-08T22:06:42.693 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-08T22:06:42.693 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-08T22:06:42.693 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 88.638058624999999, 2024-05-08T22:06:42.693 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 23, 2024-05-08T22:06:42.693 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 87, 2024-05-08T22:06:42.693 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 83, 2024-05-08T22:06:42.693 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 94.894567241999994 2024-05-08T22:06:42.693 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:06:42.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.677+0000 7f553e200700 1 -- 172.21.15.131:0/931421716 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f551c0a11b0 msgr2=0x7f551c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.677+0000 7f553e200700 1 --2- 172.21.15.131:0/931421716 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f551c0a11b0 0x7f551c0a3670 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f552800b2e0 tx=0x7f5528005fd0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.677+0000 7f553e200700 1 -- 172.21.15.131:0/931421716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5538102ab0 msgr2=0x7f55381a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:42.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.677+0000 7f553e200700 1 --2- 172.21.15.131:0/931421716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5538102ab0 0x7f55381a06b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f553000efc0 tx=0x7f553000c3a0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.677+0000 7f553e200700 1 -- 172.21.15.131:0/931421716 shutdown_connections 2024-05-08T22:06:42.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.677+0000 7f553e200700 1 --2- 172.21.15.131:0/931421716 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f551c0a11b0 0x7f551c0a3670 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.677+0000 7f553e200700 1 --2- 172.21.15.131:0/931421716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5538102ab0 0x7f55381a06b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.677+0000 7f553e200700 1 --2- 172.21.15.131:0/931421716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5538100f20 0x7f5538199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.677+0000 7f553e200700 1 --2- 172.21.15.131:0/931421716 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55381020f0 0x7f5538199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:42.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.677+0000 7f553e200700 1 -- 172.21.15.131:0/931421716 >> 172.21.15.131:0/931421716 conn(0x7f55380fc500 msgr2=0x7f5538107920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:42.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.677+0000 7f553e200700 1 -- 172.21.15.131:0/931421716 shutdown_connections 2024-05-08T22:06:42.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:42.677+0000 7f553e200700 1 -- 172.21.15.131:0/931421716 wait complete. 2024-05-08T22:06:42.699 INFO:tasks.cephadm:Setup complete, yielding 2024-05-08T22:06:42.699 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-08T22:06:42.710 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi131.front.sepia.ceph.com 2024-05-08T22:06:42.711 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-08T22:06:43.074 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:43.285 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:43 smithi131 ceph-mon[37260]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-05-08T22:06:43.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:43 smithi132 ceph-mon[78180]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-05-08T22:06:43.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:43 smithi157 ceph-mon[77746]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-05-08T22:06:44.172 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:44 smithi131 ceph-mon[37260]: pgmap v288: 129 pgs: 1 peering, 128 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T22:06:44.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:44 smithi132 ceph-mon[78180]: pgmap v288: 129 pgs: 1 peering, 128 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T22:06:44.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.435+0000 7f28bea00700 1 -- 172.21.15.131:0/3885575705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28b8101f70 msgr2=0x7f28b81023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:44.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.435+0000 7f28bea00700 1 --2- 172.21.15.131:0/3885575705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28b8101f70 0x7f28b81023f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009990 tx=0x7f28a4009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:44.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.436+0000 7f28bea00700 1 -- 172.21.15.131:0/3885575705 shutdown_connections 2024-05-08T22:06:44.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.436+0000 7f28bea00700 1 --2- 172.21.15.131:0/3885575705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28b8102930 0x7f28b8109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:44.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.436+0000 7f28bea00700 1 --2- 172.21.15.131:0/3885575705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28b8100e10 0x7f28b8101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:44.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.436+0000 7f28bea00700 1 --2- 172.21.15.131:0/3885575705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28b8101f70 0x7f28b81023f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:44.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.436+0000 7f28bea00700 1 -- 172.21.15.131:0/3885575705 >> 172.21.15.131:0/3885575705 conn(0x7f28b80fc3b0 msgr2=0x7f28b80fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:44.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.436+0000 7f28bea00700 1 -- 172.21.15.131:0/3885575705 shutdown_connections 2024-05-08T22:06:44.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.436+0000 7f28bea00700 1 -- 172.21.15.131:0/3885575705 wait complete. 2024-05-08T22:06:44.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.437+0000 7f28bea00700 1 Processor -- start 2024-05-08T22:06:44.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.437+0000 7f28bea00700 1 -- start start 2024-05-08T22:06:44.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.437+0000 7f28bea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28b8100e10 0x7f28b8198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:44.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.437+0000 7f28bea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28b8101f70 0x7f28b81994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:44.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.437+0000 7f28bea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28b8102930 0x7f28b81a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:44.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.437+0000 7f28bea00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b81a0aa0 con 0x7f28b8102930 2024-05-08T22:06:44.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.437+0000 7f28bea00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b81a0be0 con 0x7f28b8100e10 2024-05-08T22:06:44.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.437+0000 7f28bea00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b81a0d70 con 0x7f28b8101f70 2024-05-08T22:06:44.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.438+0000 7f28bd600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28b8100e10 0x7f28b8198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:44.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.438+0000 7f28be000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28b8102930 0x7f28b81a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:44.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.438+0000 7f28bd600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28b8100e10 0x7f28b8198f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:53476/0 (socket says 172.21.15.131:53476) 2024-05-08T22:06:44.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.438+0000 7f28bd600700 1 -- 172.21.15.131:0/955363515 learned_addr learned my addr 172.21.15.131:0/955363515 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:44.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.438+0000 7f28bcc00700 1 --2- 172.21.15.131:0/955363515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28b8101f70 0x7f28b81994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:44.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.438+0000 7f28bd600700 1 -- 172.21.15.131:0/955363515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28b8101f70 msgr2=0x7f28b81994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:44.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.438+0000 7f28bd600700 1 --2- 172.21.15.131:0/955363515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28b8101f70 0x7f28b81994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:44.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.438+0000 7f28bd600700 1 -- 172.21.15.131:0/955363515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28b8102930 msgr2=0x7f28b81a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:44.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.438+0000 7f28bd600700 1 --2- 172.21.15.131:0/955363515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28b8102930 0x7f28b81a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:44.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.438+0000 7f28bd600700 1 -- 172.21.15.131:0/955363515 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ac009520 con 0x7f28b8100e10 2024-05-08T22:06:44.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.438+0000 7f28bd600700 1 --2- 172.21.15.131:0/955363515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28b8100e10 0x7f28b8198f60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00eb30 tx=0x7f28ac00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:44.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.438+0000 7f28b6000700 1 -- 172.21.15.131:0/955363515 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac009880 con 0x7f28b8100e10 2024-05-08T22:06:44.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.438+0000 7f28b6000700 1 -- 172.21.15.131:0/955363515 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28ac0044e0 con 0x7f28b8100e10 2024-05-08T22:06:44.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.438+0000 7f28bea00700 1 -- 172.21.15.131:0/955363515 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a40095f0 con 0x7f28b8100e10 2024-05-08T22:06:44.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.439+0000 7f28b6000700 1 -- 172.21.15.131:0/955363515 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac003df0 con 0x7f28b8100e10 2024-05-08T22:06:44.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.439+0000 7f28bea00700 1 -- 172.21.15.131:0/955363515 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b81a12d0 con 0x7f28b8100e10 2024-05-08T22:06:44.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.440+0000 7f28bea00700 1 -- 172.21.15.131:0/955363515 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28b804ec60 con 0x7f28b8100e10 2024-05-08T22:06:44.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.442+0000 7f28b6000700 1 -- 172.21.15.131:0/955363515 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f28ac004000 con 0x7f28b8100e10 2024-05-08T22:06:44.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.443+0000 7f28b6000700 1 --2- 172.21.15.131:0/955363515 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f28a00a11b0 0x7f28a00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:44.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.443+0000 7f28b6000700 1 -- 172.21.15.131:0/955363515 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f28ac014070 con 0x7f28b8100e10 2024-05-08T22:06:44.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.443+0000 7f28bcc00700 1 --2- 172.21.15.131:0/955363515 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f28a00a11b0 0x7f28a00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:44.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.444+0000 7f28bcc00700 1 --2- 172.21.15.131:0/955363515 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f28a00a11b0 0x7f28a00a3670 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f28a400b2e0 tx=0x7f28a4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:44.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.445+0000 7f28b6000700 1 -- 172.21.15.131:0/955363515 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f28ac096880 con 0x7f28b8100e10 2024-05-08T22:06:44.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:44 smithi157 ceph-mon[77746]: pgmap v288: 129 pgs: 1 peering, 128 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T22:06:44.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.668+0000 7f28bea00700 1 -- 172.21.15.131:0/955363515 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f28b80748f0 con 0x7f28a00a11b0 2024-05-08T22:06:44.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.669+0000 7f28b6000700 1 -- 172.21.15.131:0/955363515 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f28b80748f0 con 0x7f28a00a11b0 2024-05-08T22:06:44.669 INFO:teuthology.orchestra.run.smithi131.stdout:Backend: cephadm 2024-05-08T22:06:44.669 INFO:teuthology.orchestra.run.smithi131.stdout:Available: Yes 2024-05-08T22:06:44.669 INFO:teuthology.orchestra.run.smithi131.stdout:Paused: No 2024-05-08T22:06:44.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.671+0000 7f28bea00700 1 -- 172.21.15.131:0/955363515 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f28a00a11b0 msgr2=0x7f28a00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:44.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.671+0000 7f28bea00700 1 --2- 172.21.15.131:0/955363515 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f28a00a11b0 0x7f28a00a3670 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f28a400b2e0 tx=0x7f28a4005fb0 comp rx=0 tx=0).stop 2024-05-08T22:06:44.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.671+0000 7f28bea00700 1 -- 172.21.15.131:0/955363515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28b8100e10 msgr2=0x7f28b8198f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:44.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.671+0000 7f28bea00700 1 --2- 172.21.15.131:0/955363515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28b8100e10 0x7f28b8198f60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00eb30 tx=0x7f28ac00ee40 comp rx=0 tx=0).stop 2024-05-08T22:06:44.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.671+0000 7f28bea00700 1 -- 172.21.15.131:0/955363515 shutdown_connections 2024-05-08T22:06:44.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.672+0000 7f28bea00700 1 --2- 172.21.15.131:0/955363515 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f28a00a11b0 0x7f28a00a3670 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:44.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.672+0000 7f28bea00700 1 --2- 172.21.15.131:0/955363515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28b8102930 0x7f28b81a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:44.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.672+0000 7f28bea00700 1 --2- 172.21.15.131:0/955363515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28b8100e10 0x7f28b8198f60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:44.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.672+0000 7f28bea00700 1 --2- 172.21.15.131:0/955363515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28b8101f70 0x7f28b81994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:44.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.672+0000 7f28bea00700 1 -- 172.21.15.131:0/955363515 >> 172.21.15.131:0/955363515 conn(0x7f28b80fc3b0 msgr2=0x7f28b80731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:44.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.672+0000 7f28bea00700 1 -- 172.21.15.131:0/955363515 shutdown_connections 2024-05-08T22:06:44.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:44.672+0000 7f28bea00700 1 -- 172.21.15.131:0/955363515 wait complete. 2024-05-08T22:06:45.144 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-08T22:06:45.511 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:46.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:46 smithi132 ceph-mon[78180]: from='client.25205 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:06:46.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:46 smithi132 ceph-mon[78180]: pgmap v289: 129 pgs: 1 peering, 128 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-08T22:06:46.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:46 smithi131 ceph-mon[37260]: from='client.25205 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:06:46.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:46 smithi131 ceph-mon[37260]: pgmap v289: 129 pgs: 1 peering, 128 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-08T22:06:46.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:46 smithi157 ceph-mon[77746]: from='client.25205 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:06:46.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:46 smithi157 ceph-mon[77746]: pgmap v289: 129 pgs: 1 peering, 128 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-08T22:06:46.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.799+0000 7f21bee00700 1 -- 172.21.15.131:0/3382752893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21b8100e20 msgr2=0x7f21b8101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:46.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.799+0000 7f21bee00700 1 --2- 172.21.15.131:0/3382752893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21b8100e20 0x7f21b8101240 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009990 tx=0x7f21ac009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:46.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.799+0000 7f21bee00700 1 -- 172.21.15.131:0/3382752893 shutdown_connections 2024-05-08T22:06:46.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.799+0000 7f21bee00700 1 --2- 172.21.15.131:0/3382752893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21b8102940 0x7f21b8109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:46.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.799+0000 7f21bee00700 1 --2- 172.21.15.131:0/3382752893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21b8100e20 0x7f21b8101240 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:46.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.799+0000 7f21bee00700 1 --2- 172.21.15.131:0/3382752893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21b8101f80 0x7f21b8102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:46.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.799+0000 7f21bee00700 1 -- 172.21.15.131:0/3382752893 >> 172.21.15.131:0/3382752893 conn(0x7f21b80fc3e0 msgr2=0x7f21b80fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:46.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.799+0000 7f21bee00700 1 -- 172.21.15.131:0/3382752893 shutdown_connections 2024-05-08T22:06:46.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.799+0000 7f21bee00700 1 -- 172.21.15.131:0/3382752893 wait complete. 2024-05-08T22:06:46.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.800+0000 7f21bee00700 1 Processor -- start 2024-05-08T22:06:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.801+0000 7f21bee00700 1 -- start start 2024-05-08T22:06:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.801+0000 7f21bee00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21b8100e20 0x7f21b8198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.801+0000 7f21bee00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21b8101f80 0x7f21b81994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.801+0000 7f21bee00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21b8102940 0x7f21b81a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.801+0000 7f21bee00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b81a0ae0 con 0x7f21b8100e20 2024-05-08T22:06:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.801+0000 7f21bee00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b81a0c20 con 0x7f21b8101f80 2024-05-08T22:06:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.801+0000 7f21bee00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b81a0db0 con 0x7f21b8102940 2024-05-08T22:06:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.801+0000 7f21bda00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21b8100e20 0x7f21b8198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.801+0000 7f21bda00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21b8100e20 0x7f21b8198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56264/0 (socket says 172.21.15.131:56264) 2024-05-08T22:06:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.801+0000 7f21bda00700 1 -- 172.21.15.131:0/3289430857 learned_addr learned my addr 172.21.15.131:0/3289430857 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.801+0000 7f21bd000700 1 --2- 172.21.15.131:0/3289430857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21b8101f80 0x7f21b81994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.801+0000 7f21be400700 1 --2- 172.21.15.131:0/3289430857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21b8102940 0x7f21b81a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.802+0000 7f21bda00700 1 -- 172.21.15.131:0/3289430857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21b8102940 msgr2=0x7f21b81a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.802+0000 7f21bda00700 1 --2- 172.21.15.131:0/3289430857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21b8102940 0x7f21b81a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.802+0000 7f21bda00700 1 -- 172.21.15.131:0/3289430857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21b8101f80 msgr2=0x7f21b81994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.802+0000 7f21bda00700 1 --2- 172.21.15.131:0/3289430857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21b8101f80 0x7f21b81994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.802+0000 7f21bda00700 1 -- 172.21.15.131:0/3289430857 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac0095f0 con 0x7f21b8100e20 2024-05-08T22:06:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.802+0000 7f21bda00700 1 --2- 172.21.15.131:0/3289430857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21b8100e20 0x7f21b8198fa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f21ac005910 tx=0x7f21ac005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.802+0000 7f21b6000700 1 -- 172.21.15.131:0/3289430857 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac01d070 con 0x7f21b8100e20 2024-05-08T22:06:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.802+0000 7f21b6000700 1 -- 172.21.15.131:0/3289430857 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21ac021410 con 0x7f21b8100e20 2024-05-08T22:06:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.802+0000 7f21bee00700 1 -- 172.21.15.131:0/3289430857 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b81a1030 con 0x7f21b8100e20 2024-05-08T22:06:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.802+0000 7f21b6000700 1 -- 172.21.15.131:0/3289430857 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac017650 con 0x7f21b8100e20 2024-05-08T22:06:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.802+0000 7f21bee00700 1 -- 172.21.15.131:0/3289430857 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b81a1440 con 0x7f21b8100e20 2024-05-08T22:06:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.803+0000 7f21bee00700 1 -- 172.21.15.131:0/3289430857 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b804ec60 con 0x7f21b8100e20 2024-05-08T22:06:46.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.805+0000 7f21b6000700 1 -- 172.21.15.131:0/3289430857 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f21ac021810 con 0x7f21b8100e20 2024-05-08T22:06:46.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.807+0000 7f21b6000700 1 --2- 172.21.15.131:0/3289430857 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f21a00a11b0 0x7f21a00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:46.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.808+0000 7f21b6000700 1 -- 172.21.15.131:0/3289430857 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f21ac0ce350 con 0x7f21b8100e20 2024-05-08T22:06:46.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.808+0000 7f21bd000700 1 --2- 172.21.15.131:0/3289430857 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f21a00a11b0 0x7f21a00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.808+0000 7f21bd000700 1 --2- 172.21.15.131:0/3289430857 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f21a00a11b0 0x7f21a00a3670 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f21a4009c60 tx=0x7f21a4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:46.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:46.812+0000 7f21b6000700 1 -- 172.21.15.131:0/3289430857 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f21ac097650 con 0x7f21b8100e20 2024-05-08T22:06:47.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:47.025+0000 7f21bee00700 1 -- 172.21.15.131:0/3289430857 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f21b80748f0 con 0x7f21a00a11b0 2024-05-08T22:06:47.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:47.038+0000 7f21b6000700 1 -- 172.21.15.131:0/3289430857 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f21b80748f0 con 0x7f21a00a11b0 2024-05-08T22:06:47.038 INFO:teuthology.orchestra.run.smithi131.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-08T22:06:47.039 INFO:teuthology.orchestra.run.smithi131.stdout:mds.a smithi131 running (101s) 21s ago 101s 17.2M - 18.2.2-1322-gd21e4f96 af23193ff691 0271925cd6a7 2024-05-08T22:06:47.039 INFO:teuthology.orchestra.run.smithi131.stdout:mds.b smithi132 running (99s) 33s ago 99s 32.6M - 18.2.2-1322-gd21e4f96 af23193ff691 a9f281e5efa2 2024-05-08T22:06:47.039 INFO:teuthology.orchestra.run.smithi131.stdout:mds.c smithi157 running (109s) 33s ago 109s 17.8M - 18.2.2-1322-gd21e4f96 af23193ff691 a6cd41bd7be5 2024-05-08T22:06:47.039 INFO:teuthology.orchestra.run.smithi131.stdout:mds.d smithi131 running (115s) 21s ago 114s 13.8M - 18.2.2-1322-gd21e4f96 af23193ff691 959420d43727 2024-05-08T22:06:47.039 INFO:teuthology.orchestra.run.smithi131.stdout:mds.e smithi132 running (113s) 33s ago 113s 15.1M - 18.2.2-1322-gd21e4f96 af23193ff691 7b23598426c8 2024-05-08T22:06:47.039 INFO:teuthology.orchestra.run.smithi131.stdout:mds.f smithi157 running (111s) 33s ago 111s 16.5M - 18.2.2-1322-gd21e4f96 af23193ff691 cf1ee9d65125 2024-05-08T22:06:47.039 INFO:teuthology.orchestra.run.smithi131.stdout:mds.g smithi131 running (103s) 21s ago 103s 17.1M - 18.2.2-1322-gd21e4f96 af23193ff691 9bb6f1647f58 2024-05-08T22:06:47.039 INFO:teuthology.orchestra.run.smithi131.stdout:mds.h smithi132 running (108s) 33s ago 107s 19.8M - 18.2.2-1322-gd21e4f96 af23193ff691 6dfed167cf71 2024-05-08T22:06:47.039 INFO:teuthology.orchestra.run.smithi131.stdout:mds.i smithi157 running (106s) 33s ago 106s 21.3M - 18.2.2-1322-gd21e4f96 af23193ff691 ab16faefb86a 2024-05-08T22:06:47.039 INFO:teuthology.orchestra.run.smithi131.stdout:mds.j smithi131 running (117s) 21s ago 117s 13.8M - 18.2.2-1322-gd21e4f96 af23193ff691 1e7269324a2d 2024-05-08T22:06:47.039 INFO:teuthology.orchestra.run.smithi131.stdout:mds.k smithi132 running (2m) 33s ago 2m 14.3M - 18.2.2-1322-gd21e4f96 af23193ff691 fdc5db421f7b 2024-05-08T22:06:47.039 INFO:teuthology.orchestra.run.smithi131.stdout:mds.l smithi157 running (2m) 33s ago 2m 16.5M - 18.2.2-1322-gd21e4f96 af23193ff691 b1be67ca5ab4 2024-05-08T22:06:47.039 INFO:teuthology.orchestra.run.smithi131.stdout:mgr.x smithi131 *:9283,8765 running (7m) 21s ago 7m 537M - 18.2.2-1322-gd21e4f96 af23193ff691 756987c4fd85 2024-05-08T22:06:47.040 INFO:teuthology.orchestra.run.smithi131.stdout:mgr.y smithi132 *:8443,8765 running (6m) 33s ago 6m 460M - 18.2.2-1322-gd21e4f96 af23193ff691 d6c7d8e08dec 2024-05-08T22:06:47.040 INFO:teuthology.orchestra.run.smithi131.stdout:mgr.z smithi157 *:8443,8765 running (6m) 33s ago 6m 458M - 18.2.2-1322-gd21e4f96 af23193ff691 107228a62c87 2024-05-08T22:06:47.040 INFO:teuthology.orchestra.run.smithi131.stdout:mon.a smithi131 running (8m) 21s ago 8m 91.4M 2048M 18.2.2-1322-gd21e4f96 af23193ff691 e4dc3ef057c3 2024-05-08T22:06:47.040 INFO:teuthology.orchestra.run.smithi131.stdout:mon.b smithi132 running (6m) 33s ago 6m 64.0M 2048M 18.2.2-1322-gd21e4f96 af23193ff691 3a5e4aecd9be 2024-05-08T22:06:47.040 INFO:teuthology.orchestra.run.smithi131.stdout:mon.c smithi157 running (6m) 33s ago 6m 66.9M 2048M 18.2.2-1322-gd21e4f96 af23193ff691 b00de7be5f3e 2024-05-08T22:06:47.040 INFO:teuthology.orchestra.run.smithi131.stdout:osd.0 smithi131 running (5m) 21s ago 5m 140M 4317M 18.2.2-1322-gd21e4f96 af23193ff691 769393a35546 2024-05-08T22:06:47.040 INFO:teuthology.orchestra.run.smithi131.stdout:osd.1 smithi132 running (5m) 33s ago 5m 148M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 5de60751577d 2024-05-08T22:06:47.040 INFO:teuthology.orchestra.run.smithi131.stdout:osd.2 smithi157 running (5m) 33s ago 5m 149M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 27e2cdd55f39 2024-05-08T22:06:47.040 INFO:teuthology.orchestra.run.smithi131.stdout:osd.3 smithi131 running (4m) 21s ago 4m 63.9M 4317M 18.2.2-1322-gd21e4f96 af23193ff691 d6b31d3be4a3 2024-05-08T22:06:47.040 INFO:teuthology.orchestra.run.smithi131.stdout:osd.4 smithi132 running (4m) 33s ago 4m 65.7M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 db5ce46c7db7 2024-05-08T22:06:47.040 INFO:teuthology.orchestra.run.smithi131.stdout:osd.5 smithi157 running (4m) 33s ago 4m 138M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 567c002c4f38 2024-05-08T22:06:47.040 INFO:teuthology.orchestra.run.smithi131.stdout:osd.6 smithi131 running (3m) 21s ago 3m 60.7M 4317M 18.2.2-1322-gd21e4f96 af23193ff691 59556a379dce 2024-05-08T22:06:47.040 INFO:teuthology.orchestra.run.smithi131.stdout:osd.7 smithi132 running (3m) 33s ago 3m 66.8M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 451d3c6e3601 2024-05-08T22:06:47.040 INFO:teuthology.orchestra.run.smithi131.stdout:osd.8 smithi157 running (3m) 33s ago 3m 60.6M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 acd9eb881bc0 2024-05-08T22:06:47.041 INFO:teuthology.orchestra.run.smithi131.stdout:osd.9 smithi131 running (2m) 21s ago 2m 58.5M 4317M 18.2.2-1322-gd21e4f96 af23193ff691 43b692654a79 2024-05-08T22:06:47.041 INFO:teuthology.orchestra.run.smithi131.stdout:osd.10 smithi132 running (2m) 33s ago 2m 131M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 a4a319d1584e 2024-05-08T22:06:47.041 INFO:teuthology.orchestra.run.smithi131.stdout:osd.11 smithi157 running (2m) 33s ago 2m 130M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 8c1ac8dbab3d 2024-05-08T22:06:47.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:47.041+0000 7f21bee00700 1 -- 172.21.15.131:0/3289430857 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f21a00a11b0 msgr2=0x7f21a00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:47.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:47.041+0000 7f21bee00700 1 --2- 172.21.15.131:0/3289430857 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f21a00a11b0 0x7f21a00a3670 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f21a4009c60 tx=0x7f21a4009260 comp rx=0 tx=0).stop 2024-05-08T22:06:47.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:47.041+0000 7f21bee00700 1 -- 172.21.15.131:0/3289430857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21b8100e20 msgr2=0x7f21b8198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:47.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:47.041+0000 7f21bee00700 1 --2- 172.21.15.131:0/3289430857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21b8100e20 0x7f21b8198fa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f21ac005910 tx=0x7f21ac005740 comp rx=0 tx=0).stop 2024-05-08T22:06:47.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:47.041+0000 7f21bee00700 1 -- 172.21.15.131:0/3289430857 shutdown_connections 2024-05-08T22:06:47.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:47.041+0000 7f21bee00700 1 --2- 172.21.15.131:0/3289430857 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f21a00a11b0 0x7f21a00a3670 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:47.041+0000 7f21bee00700 1 --2- 172.21.15.131:0/3289430857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21b8102940 0x7f21b81a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:47.041+0000 7f21bee00700 1 --2- 172.21.15.131:0/3289430857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21b8100e20 0x7f21b8198fa0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:47.041+0000 7f21bee00700 1 --2- 172.21.15.131:0/3289430857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21b8101f80 0x7f21b81994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:47.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:47.041+0000 7f21bee00700 1 -- 172.21.15.131:0/3289430857 >> 172.21.15.131:0/3289430857 conn(0x7f21b80fc3e0 msgr2=0x7f21b80731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:47.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:47.041+0000 7f21bee00700 1 -- 172.21.15.131:0/3289430857 shutdown_connections 2024-05-08T22:06:47.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:47.041+0000 7f21bee00700 1 -- 172.21.15.131:0/3289430857 wait complete. 2024-05-08T22:06:47.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:47 smithi131 ceph-mon[37260]: pgmap v290: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:06:47.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:47 smithi131 ceph-mon[37260]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:06:47.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:47 smithi131 ceph-mon[37260]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-05-08T22:06:47.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:47 smithi131 ceph-mon[37260]: Cluster is now healthy 2024-05-08T22:06:47.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:47 smithi157 ceph-mon[77746]: pgmap v290: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:06:47.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:47 smithi157 ceph-mon[77746]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:06:47.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:47 smithi157 ceph-mon[77746]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-05-08T22:06:47.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:47 smithi157 ceph-mon[77746]: Cluster is now healthy 2024-05-08T22:06:47.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:47 smithi132 ceph-mon[78180]: pgmap v290: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:06:47.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:47 smithi132 ceph-mon[78180]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:06:47.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:47 smithi132 ceph-mon[78180]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-05-08T22:06:47.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:47 smithi132 ceph-mon[78180]: Cluster is now healthy 2024-05-08T22:06:47.693 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-08T22:06:48.058 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:49.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.500+0000 7f022b000700 1 -- 172.21.15.131:0/240896386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0224102ba0 msgr2=0x7f0224109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:49.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.500+0000 7f022b000700 1 --2- 172.21.15.131:0/240896386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0224102ba0 0x7f0224109450 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f021c00b0d0 tx=0x7f021c00b3e0 comp rx=0 tx=0).stop 2024-05-08T22:06:49.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.500+0000 7f022b000700 1 -- 172.21.15.131:0/240896386 shutdown_connections 2024-05-08T22:06:49.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.500+0000 7f022b000700 1 --2- 172.21.15.131:0/240896386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0224102ba0 0x7f0224109450 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:49.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.500+0000 7f022b000700 1 --2- 172.21.15.131:0/240896386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0224101000 0x7f0224101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:49.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.500+0000 7f022b000700 1 --2- 172.21.15.131:0/240896386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0224102200 0x7f0224102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:49.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.500+0000 7f022b000700 1 -- 172.21.15.131:0/240896386 >> 172.21.15.131:0/240896386 conn(0x7f02240fc5a0 msgr2=0x7f02240fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:49.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.500+0000 7f022b000700 1 -- 172.21.15.131:0/240896386 shutdown_connections 2024-05-08T22:06:49.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.500+0000 7f022b000700 1 -- 172.21.15.131:0/240896386 wait complete. 2024-05-08T22:06:49.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.501+0000 7f022b000700 1 Processor -- start 2024-05-08T22:06:49.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.501+0000 7f022b000700 1 -- start start 2024-05-08T22:06:49.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.502+0000 7f022b000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0224101000 0x7f0224197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:49.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.502+0000 7f022b000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0224102200 0x7f0224197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:49.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.502+0000 7f022b000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0224102ba0 0x7f022419e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:49.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.502+0000 7f022b000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022419eb40 con 0x7f0224102200 2024-05-08T22:06:49.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.502+0000 7f022b000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022419ec80 con 0x7f0224101000 2024-05-08T22:06:49.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.502+0000 7f022b000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022419ee10 con 0x7f0224102ba0 2024-05-08T22:06:49.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.502+0000 7f0223400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0224102200 0x7f0224197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:49.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.502+0000 7f0223400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0224102200 0x7f0224197540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56288/0 (socket says 172.21.15.131:56288) 2024-05-08T22:06:49.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.502+0000 7f0223400700 1 -- 172.21.15.131:0/3395434679 learned_addr learned my addr 172.21.15.131:0/3395434679 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:49.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.502+0000 7f0223e00700 1 --2- 172.21.15.131:0/3395434679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0224101000 0x7f0224197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:49.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.502+0000 7f0229000700 1 --2- 172.21.15.131:0/3395434679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0224102ba0 0x7f022419e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:49.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.503+0000 7f0223e00700 1 -- 172.21.15.131:0/3395434679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0224102ba0 msgr2=0x7f022419e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:49.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.503+0000 7f0223e00700 1 --2- 172.21.15.131:0/3395434679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0224102ba0 0x7f022419e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:49.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.503+0000 7f0223e00700 1 -- 172.21.15.131:0/3395434679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0224102200 msgr2=0x7f0224197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:49.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.503+0000 7f0223e00700 1 --2- 172.21.15.131:0/3395434679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0224102200 0x7f0224197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:49.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.503+0000 7f0223e00700 1 -- 172.21.15.131:0/3395434679 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021c009d00 con 0x7f0224101000 2024-05-08T22:06:49.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.503+0000 7f0223e00700 1 --2- 172.21.15.131:0/3395434679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0224101000 0x7f0224197000 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f021400e9c0 tx=0x7f021400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:49.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.503+0000 7f0220c00700 1 -- 172.21.15.131:0/3395434679 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214004220 con 0x7f0224101000 2024-05-08T22:06:49.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.503+0000 7f022b000700 1 -- 172.21.15.131:0/3395434679 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f022419f0f0 con 0x7f0224101000 2024-05-08T22:06:49.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.503+0000 7f0220c00700 1 -- 172.21.15.131:0/3395434679 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0214004380 con 0x7f0224101000 2024-05-08T22:06:49.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.503+0000 7f0220c00700 1 -- 172.21.15.131:0/3395434679 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214003c50 con 0x7f0224101000 2024-05-08T22:06:49.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.503+0000 7f022b000700 1 -- 172.21.15.131:0/3395434679 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f022419f640 con 0x7f0224101000 2024-05-08T22:06:49.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.504+0000 7f022b000700 1 -- 172.21.15.131:0/3395434679 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0224108430 con 0x7f0224101000 2024-05-08T22:06:49.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.506+0000 7f0220c00700 1 -- 172.21.15.131:0/3395434679 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f021400c7c0 con 0x7f0224101000 2024-05-08T22:06:49.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.508+0000 7f0220c00700 1 --2- 172.21.15.131:0/3395434679 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f02080a11b0 0x7f02080a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:49.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.508+0000 7f0220c00700 1 -- 172.21.15.131:0/3395434679 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f0214014070 con 0x7f0224101000 2024-05-08T22:06:49.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.508+0000 7f0223400700 1 --2- 172.21.15.131:0/3395434679 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f02080a11b0 0x7f02080a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:49.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.508+0000 7f0223400700 1 --2- 172.21.15.131:0/3395434679 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f02080a11b0 0x7f02080a3670 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0210009d20 tx=0x7f0210009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:49.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.510+0000 7f0220c00700 1 -- 172.21.15.131:0/3395434679 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0214096a00 con 0x7f0224101000 2024-05-08T22:06:49.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.730+0000 7f022b000700 1 -- 172.21.15.131:0/3395434679 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f02240613e0 con 0x7f02080a11b0 2024-05-08T22:06:49.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.735+0000 7f0220c00700 1 -- 172.21.15.131:0/3395434679 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f02240613e0 con 0x7f02080a11b0 2024-05-08T22:06:49.736 INFO:teuthology.orchestra.run.smithi131.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-08T22:06:49.737 INFO:teuthology.orchestra.run.smithi131.stdout:mds.all 12/12 35s ago 2m smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-05-08T22:06:49.737 INFO:teuthology.orchestra.run.smithi131.stdout:mgr 3/3 35s ago 6m smithi131=x;smithi132=y;smithi157=z;count:3 2024-05-08T22:06:49.737 INFO:teuthology.orchestra.run.smithi131.stdout:mon 3/3 35s ago 6m smithi131:172.21.15.131=a;smithi132:172.21.15.132=b;smithi157:172.21.15.157=c;count:3 2024-05-08T22:06:49.737 INFO:teuthology.orchestra.run.smithi131.stdout:osd 12 35s ago - 2024-05-08T22:06:49.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.738+0000 7f022b000700 1 -- 172.21.15.131:0/3395434679 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f02080a11b0 msgr2=0x7f02080a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:49.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.738+0000 7f022b000700 1 --2- 172.21.15.131:0/3395434679 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f02080a11b0 0x7f02080a3670 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0210009d20 tx=0x7f0210009260 comp rx=0 tx=0).stop 2024-05-08T22:06:49.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.738+0000 7f022b000700 1 -- 172.21.15.131:0/3395434679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0224101000 msgr2=0x7f0224197000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:49.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.738+0000 7f022b000700 1 --2- 172.21.15.131:0/3395434679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0224101000 0x7f0224197000 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f021400e9c0 tx=0x7f021400ecd0 comp rx=0 tx=0).stop 2024-05-08T22:06:49.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.738+0000 7f022b000700 1 -- 172.21.15.131:0/3395434679 shutdown_connections 2024-05-08T22:06:49.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.738+0000 7f022b000700 1 --2- 172.21.15.131:0/3395434679 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f02080a11b0 0x7f02080a3670 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:49.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.738+0000 7f022b000700 1 --2- 172.21.15.131:0/3395434679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0224102ba0 0x7f022419e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:49.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.738+0000 7f022b000700 1 --2- 172.21.15.131:0/3395434679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0224101000 0x7f0224197000 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:49.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.738+0000 7f022b000700 1 --2- 172.21.15.131:0/3395434679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0224102200 0x7f0224197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:49.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.738+0000 7f022b000700 1 -- 172.21.15.131:0/3395434679 >> 172.21.15.131:0/3395434679 conn(0x7f02240fc5a0 msgr2=0x7f0224105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:49.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.738+0000 7f022b000700 1 -- 172.21.15.131:0/3395434679 shutdown_connections 2024-05-08T22:06:49.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:49.738+0000 7f022b000700 1 -- 172.21.15.131:0/3395434679 wait complete. 2024-05-08T22:06:50.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:49 smithi132 ceph-mon[78180]: pgmap v291: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-08T22:06:50.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:49 smithi131 ceph-mon[37260]: pgmap v291: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-08T22:06:50.205 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-08T22:06:50.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:49 smithi157 ceph-mon[77746]: pgmap v291: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-08T22:06:50.570 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:51.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:50 smithi132 ceph-mon[78180]: from='client.25217 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:06:51.136 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:50 smithi131 ceph-mon[37260]: from='client.25217 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:06:51.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:50 smithi157 ceph-mon[77746]: from='client.25217 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:06:51.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:51 smithi131 ceph-mon[37260]: pgmap v292: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-08T22:06:51.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.918+0000 7f3c13a00700 1 -- 172.21.15.131:0/3566843419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c0c101000 msgr2=0x7f3c0c101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:51.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.918+0000 7f3c13a00700 1 --2- 172.21.15.131:0/3566843419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c0c101000 0x7f3c0c101420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3c000099c0 tx=0x7f3c00009cd0 comp rx=0 tx=0).stop 2024-05-08T22:06:51.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.918+0000 7f3c13a00700 1 -- 172.21.15.131:0/3566843419 shutdown_connections 2024-05-08T22:06:51.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.918+0000 7f3c13a00700 1 --2- 172.21.15.131:0/3566843419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c0c102ba0 0x7f3c0c109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:51.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.918+0000 7f3c13a00700 1 --2- 172.21.15.131:0/3566843419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c0c101000 0x7f3c0c101420 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:51.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.918+0000 7f3c13a00700 1 --2- 172.21.15.131:0/3566843419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c0c102200 0x7f3c0c102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:51.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.918+0000 7f3c13a00700 1 -- 172.21.15.131:0/3566843419 >> 172.21.15.131:0/3566843419 conn(0x7f3c0c0fc5a0 msgr2=0x7f3c0c0fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:51.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.918+0000 7f3c13a00700 1 -- 172.21.15.131:0/3566843419 shutdown_connections 2024-05-08T22:06:51.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.919+0000 7f3c13a00700 1 -- 172.21.15.131:0/3566843419 wait complete. 2024-05-08T22:06:51.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.920+0000 7f3c13a00700 1 Processor -- start 2024-05-08T22:06:51.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.920+0000 7f3c13a00700 1 -- start start 2024-05-08T22:06:51.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.920+0000 7f3c13a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c0c101000 0x7f3c0c199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:51.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.920+0000 7f3c13a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c0c102200 0x7f3c0c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:51.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.920+0000 7f3c13a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c0c102ba0 0x7f3c0c1a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:51.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.920+0000 7f3c13a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c1a0ca0 con 0x7f3c0c102ba0 2024-05-08T22:06:51.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.920+0000 7f3c13a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c1a0de0 con 0x7f3c0c101000 2024-05-08T22:06:51.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.920+0000 7f3c13a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c1a0f70 con 0x7f3c0c102200 2024-05-08T22:06:51.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.920+0000 7f3c11a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c0c102ba0 0x7f3c0c1a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:51.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.920+0000 7f3c11a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c0c102ba0 0x7f3c0c1a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56304/0 (socket says 172.21.15.131:56304) 2024-05-08T22:06:51.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.920+0000 7f3c11a00700 1 -- 172.21.15.131:0/3791307307 learned_addr learned my addr 172.21.15.131:0/3791307307 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:51.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.920+0000 7f3c11000700 1 --2- 172.21.15.131:0/3791307307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c0c101000 0x7f3c0c199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:51.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.920+0000 7f3c0be00700 1 --2- 172.21.15.131:0/3791307307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c0c102200 0x7f3c0c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:51.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.921+0000 7f3c11000700 1 -- 172.21.15.131:0/3791307307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c0c102200 msgr2=0x7f3c0c199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:51.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.921+0000 7f3c11000700 1 --2- 172.21.15.131:0/3791307307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c0c102200 0x7f3c0c199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:51.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.921+0000 7f3c11000700 1 -- 172.21.15.131:0/3791307307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c0c102ba0 msgr2=0x7f3c0c1a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:51.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.921+0000 7f3c11000700 1 --2- 172.21.15.131:0/3791307307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c0c102ba0 0x7f3c0c1a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:51.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.921+0000 7f3c11000700 1 -- 172.21.15.131:0/3791307307 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c000095f0 con 0x7f3c0c101000 2024-05-08T22:06:51.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.921+0000 7f3c11000700 1 --2- 172.21.15.131:0/3791307307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c0c101000 0x7f3c0c199200 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3c000058e0 tx=0x7f3c0000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:51.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.921+0000 7f3c09600700 1 -- 172.21.15.131:0/3791307307 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0001d070 con 0x7f3c0c101000 2024-05-08T22:06:51.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.921+0000 7f3c13a00700 1 -- 172.21.15.131:0/3791307307 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c0c1a11f0 con 0x7f3c0c101000 2024-05-08T22:06:51.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.921+0000 7f3c13a00700 1 -- 172.21.15.131:0/3791307307 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c0c1a1660 con 0x7f3c0c101000 2024-05-08T22:06:51.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.922+0000 7f3c09600700 1 -- 172.21.15.131:0/3791307307 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c00005030 con 0x7f3c0c101000 2024-05-08T22:06:51.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.922+0000 7f3c09600700 1 -- 172.21.15.131:0/3791307307 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c00004970 con 0x7f3c0c101000 2024-05-08T22:06:51.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.923+0000 7f3c13a00700 1 -- 172.21.15.131:0/3791307307 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bec005320 con 0x7f3c0c101000 2024-05-08T22:06:51.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.924+0000 7f3c09600700 1 -- 172.21.15.131:0/3791307307 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f3c000051a0 con 0x7f3c0c101000 2024-05-08T22:06:51.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.926+0000 7f3c09600700 1 --2- 172.21.15.131:0/3791307307 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f3bf40a11b0 0x7f3bf40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:51.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.926+0000 7f3c09600700 1 -- 172.21.15.131:0/3791307307 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f3c000cda70 con 0x7f3c0c101000 2024-05-08T22:06:51.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.926+0000 7f3c0be00700 1 --2- 172.21.15.131:0/3791307307 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f3bf40a11b0 0x7f3bf40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:51.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.926+0000 7f3c0be00700 1 --2- 172.21.15.131:0/3791307307 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f3bf40a11b0 0x7f3bf40a3670 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8009c60 tx=0x7f3bf8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:51.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:51.928+0000 7f3c09600700 1 -- 172.21.15.131:0/3791307307 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3c00096e90 con 0x7f3c0c101000 2024-05-08T22:06:52.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:51 smithi132 ceph-mon[78180]: pgmap v292: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-08T22:06:52.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:52.136+0000 7f3c13a00700 1 -- 172.21.15.131:0/3791307307 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3bec000bf0 con 0x7f3bf40a11b0 2024-05-08T22:06:52.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:52.138+0000 7f3c09600700 1 -- 172.21.15.131:0/3791307307 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f3bec000bf0 con 0x7f3bf40a11b0 2024-05-08T22:06:52.139 INFO:teuthology.orchestra.run.smithi131.stdout:HOST ADDR LABELS STATUS 2024-05-08T22:06:52.139 INFO:teuthology.orchestra.run.smithi131.stdout:smithi131 172.21.15.131 2024-05-08T22:06:52.139 INFO:teuthology.orchestra.run.smithi131.stdout:smithi132 172.21.15.132 2024-05-08T22:06:52.139 INFO:teuthology.orchestra.run.smithi131.stdout:smithi157 172.21.15.157 2024-05-08T22:06:52.139 INFO:teuthology.orchestra.run.smithi131.stdout:3 hosts in cluster 2024-05-08T22:06:52.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:52.140+0000 7f3c13a00700 1 -- 172.21.15.131:0/3791307307 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f3bf40a11b0 msgr2=0x7f3bf40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:52.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:52.140+0000 7f3c13a00700 1 --2- 172.21.15.131:0/3791307307 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f3bf40a11b0 0x7f3bf40a3670 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8009c60 tx=0x7f3bf8009260 comp rx=0 tx=0).stop 2024-05-08T22:06:52.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:52.140+0000 7f3c13a00700 1 -- 172.21.15.131:0/3791307307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c0c101000 msgr2=0x7f3c0c199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:52.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:52.140+0000 7f3c13a00700 1 --2- 172.21.15.131:0/3791307307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c0c101000 0x7f3c0c199200 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3c000058e0 tx=0x7f3c0000f690 comp rx=0 tx=0).stop 2024-05-08T22:06:52.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:52.140+0000 7f3c13a00700 1 -- 172.21.15.131:0/3791307307 shutdown_connections 2024-05-08T22:06:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:52.140+0000 7f3c13a00700 1 --2- 172.21.15.131:0/3791307307 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f3bf40a11b0 0x7f3bf40a3670 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:52.140+0000 7f3c13a00700 1 --2- 172.21.15.131:0/3791307307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c0c102ba0 0x7f3c0c1a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:52.140+0000 7f3c13a00700 1 --2- 172.21.15.131:0/3791307307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c0c101000 0x7f3c0c199200 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:52.140+0000 7f3c13a00700 1 --2- 172.21.15.131:0/3791307307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c0c102200 0x7f3c0c199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:52.140+0000 7f3c13a00700 1 -- 172.21.15.131:0/3791307307 >> 172.21.15.131:0/3791307307 conn(0x7f3c0c0fc5a0 msgr2=0x7f3c0c107a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:52.140+0000 7f3c13a00700 1 -- 172.21.15.131:0/3791307307 shutdown_connections 2024-05-08T22:06:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:52.140+0000 7f3c13a00700 1 -- 172.21.15.131:0/3791307307 wait complete. 2024-05-08T22:06:52.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:51 smithi157 ceph-mon[77746]: pgmap v292: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-08T22:06:52.718 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-08T22:06:53.119 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:53.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:53 smithi131 ceph-mon[37260]: from='client.25223 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:06:53.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:53 smithi131 ceph-mon[37260]: pgmap v293: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T22:06:54.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:53 smithi132 ceph-mon[78180]: from='client.25223 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:06:54.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:53 smithi132 ceph-mon[78180]: pgmap v293: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T22:06:54.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:53 smithi157 ceph-mon[77746]: from='client.25223 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:06:54.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:53 smithi157 ceph-mon[77746]: pgmap v293: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T22:06:54.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.366+0000 7f2e1b800700 1 -- 172.21.15.131:0/2709683875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e14106cf0 msgr2=0x7f2e14109160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:54.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.366+0000 7f2e1b800700 1 --2- 172.21.15.131:0/2709683875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e14106cf0 0x7f2e14109160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c00b0a0 tx=0x7f2e0c00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:06:54.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.366+0000 7f2e1b800700 1 -- 172.21.15.131:0/2709683875 shutdown_connections 2024-05-08T22:06:54.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.366+0000 7f2e1b800700 1 --2- 172.21.15.131:0/2709683875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e14106cf0 0x7f2e14109160 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:54.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.366+0000 7f2e1b800700 1 --2- 172.21.15.131:0/2709683875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e14101a90 0x7f2e14103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:54.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.366+0000 7f2e1b800700 1 --2- 172.21.15.131:0/2709683875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e141043c0 0x7f2e141067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:54.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.366+0000 7f2e1b800700 1 -- 172.21.15.131:0/2709683875 >> 172.21.15.131:0/2709683875 conn(0x7f2e140fb4a0 msgr2=0x7f2e140fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:54.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.366+0000 7f2e1b800700 1 -- 172.21.15.131:0/2709683875 shutdown_connections 2024-05-08T22:06:54.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.366+0000 7f2e1b800700 1 -- 172.21.15.131:0/2709683875 wait complete. 2024-05-08T22:06:54.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.368+0000 7f2e1b800700 1 Processor -- start 2024-05-08T22:06:54.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.368+0000 7f2e1b800700 1 -- start start 2024-05-08T22:06:54.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.368+0000 7f2e1b800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e14101a90 0x7f2e141013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:54.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.368+0000 7f2e1b800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e141043c0 0x7f2e140ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:54.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.368+0000 7f2e1b800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e14106cf0 0x7f2e140fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:54.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.368+0000 7f2e1b800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e14100720 con 0x7f2e141043c0 2024-05-08T22:06:54.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.368+0000 7f2e1b800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e14100890 con 0x7f2e14101a90 2024-05-08T22:06:54.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.368+0000 7f2e1b800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e141a1070 con 0x7f2e14106cf0 2024-05-08T22:06:54.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.368+0000 7f2e13e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e141043c0 0x7f2e140ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:54.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.369+0000 7f2e13e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e141043c0 0x7f2e140ffa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56076/0 (socket says 172.21.15.131:56076) 2024-05-08T22:06:54.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.369+0000 7f2e13e00700 1 -- 172.21.15.131:0/757080555 learned_addr learned my addr 172.21.15.131:0/757080555 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:54.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.369+0000 7f2e18e00700 1 --2- 172.21.15.131:0/757080555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e14101a90 0x7f2e141013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:54.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.369+0000 7f2e19800700 1 --2- 172.21.15.131:0/757080555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e14106cf0 0x7f2e140fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:54.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.369+0000 7f2e13e00700 1 -- 172.21.15.131:0/757080555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e14106cf0 msgr2=0x7f2e140fff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:54.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.369+0000 7f2e13e00700 1 --2- 172.21.15.131:0/757080555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e14106cf0 0x7f2e140fff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:54.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.369+0000 7f2e13e00700 1 -- 172.21.15.131:0/757080555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e14101a90 msgr2=0x7f2e141013a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:54.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.369+0000 7f2e13e00700 1 --2- 172.21.15.131:0/757080555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e14101a90 0x7f2e141013a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:54.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.369+0000 7f2e13e00700 1 -- 172.21.15.131:0/757080555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0c009d00 con 0x7f2e141043c0 2024-05-08T22:06:54.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.369+0000 7f2e18e00700 1 --2- 172.21.15.131:0/757080555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e14101a90 0x7f2e141013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:54.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.369+0000 7f2e13e00700 1 --2- 172.21.15.131:0/757080555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e141043c0 0x7f2e140ffa40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000e9a0 tx=0x7f2e0000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:54.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.369+0000 7f2e11600700 1 -- 172.21.15.131:0/757080555 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e00009790 con 0x7f2e141043c0 2024-05-08T22:06:54.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.370+0000 7f2e11600700 1 -- 172.21.15.131:0/757080555 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e00010430 con 0x7f2e141043c0 2024-05-08T22:06:54.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.370+0000 7f2e1b800700 1 -- 172.21.15.131:0/757080555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e141a1300 con 0x7f2e141043c0 2024-05-08T22:06:54.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.370+0000 7f2e11600700 1 -- 172.21.15.131:0/757080555 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e00018680 con 0x7f2e141043c0 2024-05-08T22:06:54.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.370+0000 7f2e1b800700 1 -- 172.21.15.131:0/757080555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e141a1800 con 0x7f2e141043c0 2024-05-08T22:06:54.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.371+0000 7f2e1b800700 1 -- 172.21.15.131:0/757080555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e140fd020 con 0x7f2e141043c0 2024-05-08T22:06:54.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.373+0000 7f2e11600700 1 -- 172.21.15.131:0/757080555 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f2e000105a0 con 0x7f2e141043c0 2024-05-08T22:06:54.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.374+0000 7f2e11600700 1 --2- 172.21.15.131:0/757080555 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f2dfc0a11b0 0x7f2dfc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:54.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.374+0000 7f2e11600700 1 -- 172.21.15.131:0/757080555 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f2e00014070 con 0x7f2e141043c0 2024-05-08T22:06:54.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.374+0000 7f2e18e00700 1 --2- 172.21.15.131:0/757080555 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f2dfc0a11b0 0x7f2dfc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:54.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.375+0000 7f2e18e00700 1 --2- 172.21.15.131:0/757080555 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f2dfc0a11b0 0x7f2dfc0a3670 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2e08011490 tx=0x7f2e08009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:54.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.379+0000 7f2e11600700 1 -- 172.21.15.131:0/757080555 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2e00052d60 con 0x7f2e141043c0 2024-05-08T22:06:54.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.596+0000 7f2e1b800700 1 -- 172.21.15.131:0/757080555 --> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e140613e0 con 0x7f2dfc0a11b0 2024-05-08T22:06:54.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.598+0000 7f2e11600700 1 -- 172.21.15.131:0/757080555 <== mgr.14150 v2:172.21.15.131:6800/2485665042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f2e140613e0 con 0x7f2dfc0a11b0 2024-05-08T22:06:54.600 INFO:teuthology.orchestra.run.smithi131.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-08T22:06:54.600 INFO:teuthology.orchestra.run.smithi131.stdout:smithi131 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300N4400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-08T22:06:54.600 INFO:teuthology.orchestra.run.smithi131.stdout:smithi132 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6232000B400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-08T22:06:54.600 INFO:teuthology.orchestra.run.smithi131.stdout:smithi157 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002V480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-05-08T22:06:54.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.601+0000 7f2e1b800700 1 -- 172.21.15.131:0/757080555 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f2dfc0a11b0 msgr2=0x7f2dfc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:54.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.601+0000 7f2e1b800700 1 --2- 172.21.15.131:0/757080555 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f2dfc0a11b0 0x7f2dfc0a3670 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2e08011490 tx=0x7f2e08009260 comp rx=0 tx=0).stop 2024-05-08T22:06:54.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.601+0000 7f2e1b800700 1 -- 172.21.15.131:0/757080555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e141043c0 msgr2=0x7f2e140ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:54.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.601+0000 7f2e1b800700 1 --2- 172.21.15.131:0/757080555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e141043c0 0x7f2e140ffa40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000e9a0 tx=0x7f2e0000ecb0 comp rx=0 tx=0).stop 2024-05-08T22:06:54.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.601+0000 7f2e1b800700 1 -- 172.21.15.131:0/757080555 shutdown_connections 2024-05-08T22:06:54.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.601+0000 7f2e1b800700 1 --2- 172.21.15.131:0/757080555 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f2dfc0a11b0 0x7f2dfc0a3670 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:54.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.601+0000 7f2e1b800700 1 --2- 172.21.15.131:0/757080555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e14106cf0 0x7f2e140fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:54.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.601+0000 7f2e1b800700 1 --2- 172.21.15.131:0/757080555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e14101a90 0x7f2e141013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:54.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.601+0000 7f2e1b800700 1 --2- 172.21.15.131:0/757080555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e141043c0 0x7f2e140ffa40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:54.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.601+0000 7f2e1b800700 1 -- 172.21.15.131:0/757080555 >> 172.21.15.131:0/757080555 conn(0x7f2e140fb4a0 msgr2=0x7f2e141023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:54.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.601+0000 7f2e1b800700 1 -- 172.21.15.131:0/757080555 shutdown_connections 2024-05-08T22:06:54.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:54.601+0000 7f2e1b800700 1 -- 172.21.15.131:0/757080555 wait complete. 2024-05-08T22:06:55.049 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-08T22:06:55.059 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi131.front.sepia.ceph.com 2024-05-08T22:06:55.059 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-08T22:06:55.427 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:56.048 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:55 smithi131 ceph-mon[37260]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:06:56.048 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:55 smithi131 ceph-mon[37260]: pgmap v294: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:06:56.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:55 smithi132 ceph-mon[78180]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:06:56.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:55 smithi132 ceph-mon[78180]: pgmap v294: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:06:56.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:55 smithi157 ceph-mon[77746]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:06:56.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:55 smithi157 ceph-mon[77746]: pgmap v294: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:06:56.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.772+0000 7f0745200700 1 -- 172.21.15.131:0/1840809072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0740100e10 msgr2=0x7f0740101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:56.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.772+0000 7f0745200700 1 --2- 172.21.15.131:0/1840809072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0740100e10 0x7f0740101230 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0730009990 tx=0x7f0730009ca0 comp rx=0 tx=0).stop 2024-05-08T22:06:56.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.772+0000 7f0745200700 1 -- 172.21.15.131:0/1840809072 shutdown_connections 2024-05-08T22:06:56.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.772+0000 7f0745200700 1 --2- 172.21.15.131:0/1840809072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0740102930 0x7f0740109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:56.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.772+0000 7f0745200700 1 --2- 172.21.15.131:0/1840809072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0740100e10 0x7f0740101230 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:56.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.772+0000 7f0745200700 1 --2- 172.21.15.131:0/1840809072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0740101f70 0x7f07401023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:56.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.772+0000 7f0745200700 1 -- 172.21.15.131:0/1840809072 >> 172.21.15.131:0/1840809072 conn(0x7f07400fc3b0 msgr2=0x7f07400fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:56.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.773+0000 7f0745200700 1 -- 172.21.15.131:0/1840809072 shutdown_connections 2024-05-08T22:06:56.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.773+0000 7f0745200700 1 -- 172.21.15.131:0/1840809072 wait complete. 2024-05-08T22:06:56.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.774+0000 7f0745200700 1 Processor -- start 2024-05-08T22:06:56.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.774+0000 7f0745200700 1 -- start start 2024-05-08T22:06:56.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.774+0000 7f0745200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0740100e10 0x7f0740198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:56.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.774+0000 7f0745200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0740101f70 0x7f0740199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:56.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.774+0000 7f0745200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0740102930 0x7f07401a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:56.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.774+0000 7f0745200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07401a0a20 con 0x7f0740102930 2024-05-08T22:06:56.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.774+0000 7f0745200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07401a0b60 con 0x7f0740100e10 2024-05-08T22:06:56.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.774+0000 7f0745200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07401a0cf0 con 0x7f0740101f70 2024-05-08T22:06:56.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.774+0000 7f073fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0740102930 0x7f07401a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:56.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.774+0000 7f073fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0740102930 0x7f07401a0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56100/0 (socket says 172.21.15.131:56100) 2024-05-08T22:06:56.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.774+0000 7f073f400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0740100e10 0x7f0740198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:56.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.774+0000 7f073fe00700 1 -- 172.21.15.131:0/3514231522 learned_addr learned my addr 172.21.15.131:0/3514231522 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:56.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.774+0000 7f073ea00700 1 --2- 172.21.15.131:0/3514231522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0740101f70 0x7f0740199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:56.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.775+0000 7f073fe00700 1 -- 172.21.15.131:0/3514231522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0740101f70 msgr2=0x7f0740199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:56.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.775+0000 7f073fe00700 1 --2- 172.21.15.131:0/3514231522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0740101f70 0x7f0740199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:56.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.775+0000 7f073fe00700 1 -- 172.21.15.131:0/3514231522 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0740100e10 msgr2=0x7f0740198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:56.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.775+0000 7f073fe00700 1 --2- 172.21.15.131:0/3514231522 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0740100e10 0x7f0740198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:56.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.775+0000 7f073fe00700 1 -- 172.21.15.131:0/3514231522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07300095f0 con 0x7f0740102930 2024-05-08T22:06:56.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.775+0000 7f073ea00700 1 --2- 172.21.15.131:0/3514231522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0740101f70 0x7f0740199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:06:56.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.775+0000 7f073fe00700 1 --2- 172.21.15.131:0/3514231522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0740102930 0x7f07401a0450 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f073800ef60 tx=0x7f073800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:56.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.775+0000 7f0737e00700 1 -- 172.21.15.131:0/3514231522 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0738018070 con 0x7f0740102930 2024-05-08T22:06:56.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.775+0000 7f0737e00700 1 -- 172.21.15.131:0/3514231522 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f073800f040 con 0x7f0740102930 2024-05-08T22:06:56.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.775+0000 7f0737e00700 1 -- 172.21.15.131:0/3514231522 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073800ccc0 con 0x7f0740102930 2024-05-08T22:06:56.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.775+0000 7f0745200700 1 -- 172.21.15.131:0/3514231522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07401a0f70 con 0x7f0740102930 2024-05-08T22:06:56.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.776+0000 7f0745200700 1 -- 172.21.15.131:0/3514231522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07401a1390 con 0x7f0740102930 2024-05-08T22:06:56.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.777+0000 7f0745200700 1 -- 172.21.15.131:0/3514231522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f074004ec60 con 0x7f0740102930 2024-05-08T22:06:56.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.778+0000 7f0737e00700 1 -- 172.21.15.131:0/3514231522 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f0738003a00 con 0x7f0740102930 2024-05-08T22:06:56.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.780+0000 7f0737e00700 1 --2- 172.21.15.131:0/3514231522 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f07240a1230 0x7f07240a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:56.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.780+0000 7f0737e00700 1 -- 172.21.15.131:0/3514231522 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f07380cc820 con 0x7f0740102930 2024-05-08T22:06:56.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.780+0000 7f073f400700 1 --2- 172.21.15.131:0/3514231522 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f07240a1230 0x7f07240a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:56.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.781+0000 7f073f400700 1 --2- 172.21.15.131:0/3514231522 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f07240a1230 0x7f07240a36f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f073000b2e0 tx=0x7f073000b6f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:56.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:56.784+0000 7f0737e00700 1 -- 172.21.15.131:0/3514231522 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0738095b20 con 0x7f0740102930 2024-05-08T22:06:57.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:57.036+0000 7f0745200700 1 -- 172.21.15.131:0/3514231522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f074010c620 con 0x7f0740102930 2024-05-08T22:06:57.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:57.036+0000 7f0737e00700 1 -- 172.21.15.131:0/3514231522 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+2914 (secure 0 0 0) 0x7f0738099140 con 0x7f0740102930 2024-05-08T22:06:57.038 INFO:teuthology.orchestra.run.smithi131.stdout:e23 2024-05-08T22:06:57.038 INFO:teuthology.orchestra.run.smithi131.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T22:06:57.038 INFO:teuthology.orchestra.run.smithi131.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T22:06:57.038 INFO:teuthology.orchestra.run.smithi131.stdout:legacy client fscid: 1 2024-05-08T22:06:57.038 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:06:57.038 INFO:teuthology.orchestra.run.smithi131.stdout:Filesystem 'cephfs' (1) 2024-05-08T22:06:57.038 INFO:teuthology.orchestra.run.smithi131.stdout:fs_name cephfs 2024-05-08T22:06:57.038 INFO:teuthology.orchestra.run.smithi131.stdout:epoch 23 2024-05-08T22:06:57.038 INFO:teuthology.orchestra.run.smithi131.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-08T22:06:57.039 INFO:teuthology.orchestra.run.smithi131.stdout:created 2024-05-08T22:05:14.034908+0000 2024-05-08T22:06:57.039 INFO:teuthology.orchestra.run.smithi131.stdout:modified 2024-05-08T22:05:39.494566+0000 2024-05-08T22:06:57.039 INFO:teuthology.orchestra.run.smithi131.stdout:tableserver 0 2024-05-08T22:06:57.039 INFO:teuthology.orchestra.run.smithi131.stdout:root 0 2024-05-08T22:06:57.039 INFO:teuthology.orchestra.run.smithi131.stdout:session_timeout 300 2024-05-08T22:06:57.039 INFO:teuthology.orchestra.run.smithi131.stdout:session_autoclose 300 2024-05-08T22:06:57.039 INFO:teuthology.orchestra.run.smithi131.stdout:max_file_size 1099511627776 2024-05-08T22:06:57.039 INFO:teuthology.orchestra.run.smithi131.stdout:max_xattr_size 65536 2024-05-08T22:06:57.039 INFO:teuthology.orchestra.run.smithi131.stdout:required_client_features {} 2024-05-08T22:06:57.039 INFO:teuthology.orchestra.run.smithi131.stdout:last_failure 0 2024-05-08T22:06:57.039 INFO:teuthology.orchestra.run.smithi131.stdout:last_failure_osd_epoch 0 2024-05-08T22:06:57.039 INFO:teuthology.orchestra.run.smithi131.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T22:06:57.039 INFO:teuthology.orchestra.run.smithi131.stdout:max_mds 3 2024-05-08T22:06:57.039 INFO:teuthology.orchestra.run.smithi131.stdout:in 0,1,2 2024-05-08T22:06:57.040 INFO:teuthology.orchestra.run.smithi131.stdout:up {0=24566,1=24562,2=24550} 2024-05-08T22:06:57.040 INFO:teuthology.orchestra.run.smithi131.stdout:failed 2024-05-08T22:06:57.040 INFO:teuthology.orchestra.run.smithi131.stdout:damaged 2024-05-08T22:06:57.040 INFO:teuthology.orchestra.run.smithi131.stdout:stopped 2024-05-08T22:06:57.040 INFO:teuthology.orchestra.run.smithi131.stdout:data_pools [3,4] 2024-05-08T22:06:57.040 INFO:teuthology.orchestra.run.smithi131.stdout:metadata_pool 2 2024-05-08T22:06:57.040 INFO:teuthology.orchestra.run.smithi131.stdout:inline_data disabled 2024-05-08T22:06:57.040 INFO:teuthology.orchestra.run.smithi131.stdout:balancer 2024-05-08T22:06:57.040 INFO:teuthology.orchestra.run.smithi131.stdout:bal_rank_mask -1 2024-05-08T22:06:57.040 INFO:teuthology.orchestra.run.smithi131.stdout:standby_count_wanted 0 2024-05-08T22:06:57.040 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.b{0:24566} state up:active seq 3 addr [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T22:06:57.040 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.i{0:24538} state up:standby-replay seq 1 addr [v2:172.21.15.157:6838/3040193644,v1:172.21.15.157:6839/3040193644] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T22:06:57.040 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.a{1:24562} state up:active seq 9 addr [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T22:06:57.041 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.h{1:24524} state up:standby-replay seq 1 addr [v2:172.21.15.132:6836/2685833178,v1:172.21.15.132:6837/2685833178] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T22:06:57.041 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.g{2:24550} state up:active seq 10 addr [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T22:06:57.041 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.c{2:24520} state up:standby-replay seq 1 addr [v2:172.21.15.157:6836/2693198274,v1:172.21.15.157:6837/2693198274] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T22:06:57.041 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:06:57.041 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:06:57.041 INFO:teuthology.orchestra.run.smithi131.stdout:Standby daemons: 2024-05-08T22:06:57.041 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:06:57.041 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.d{-1:14478} state up:standby seq 1 addr [v2:172.21.15.131:6836/1062828950,v1:172.21.15.131:6837/1062828950] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T22:06:57.041 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.l{-1:24461} state up:standby seq 1 addr [v2:172.21.15.157:6832/1034338270,v1:172.21.15.157:6833/1034338270] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T22:06:57.041 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.k{-1:24467} state up:standby seq 1 addr [v2:172.21.15.132:6832/505027564,v1:172.21.15.132:6833/505027564] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T22:06:57.041 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.j{-1:24472} state up:standby seq 1 addr [v2:172.21.15.131:6834/2147999088,v1:172.21.15.131:6835/2147999088] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T22:06:57.041 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.e{-1:24500} state up:standby seq 1 addr [v2:172.21.15.132:6834/2610471069,v1:172.21.15.132:6835/2610471069] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T22:06:57.041 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.f{-1:24508} state up:standby seq 1 addr [v2:172.21.15.157:6834/1436814390,v1:172.21.15.157:6835/1436814390] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T22:06:57.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:57.039+0000 7f0745200700 1 -- 172.21.15.131:0/3514231522 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f07240a1230 msgr2=0x7f07240a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:57.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:57.039+0000 7f0745200700 1 --2- 172.21.15.131:0/3514231522 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f07240a1230 0x7f07240a36f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f073000b2e0 tx=0x7f073000b6f0 comp rx=0 tx=0).stop 2024-05-08T22:06:57.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:57.039+0000 7f0745200700 1 -- 172.21.15.131:0/3514231522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0740102930 msgr2=0x7f07401a0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:57.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:57.039+0000 7f0745200700 1 --2- 172.21.15.131:0/3514231522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0740102930 0x7f07401a0450 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f073800ef60 tx=0x7f073800ef90 comp rx=0 tx=0).stop 2024-05-08T22:06:57.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:57.039+0000 7f0745200700 1 -- 172.21.15.131:0/3514231522 shutdown_connections 2024-05-08T22:06:57.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:57.039+0000 7f0745200700 1 --2- 172.21.15.131:0/3514231522 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f07240a1230 0x7f07240a36f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:57.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:57.039+0000 7f0745200700 1 --2- 172.21.15.131:0/3514231522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0740102930 0x7f07401a0450 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:57.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:57.039+0000 7f0745200700 1 --2- 172.21.15.131:0/3514231522 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0740100e10 0x7f0740198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:57.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:57.039+0000 7f0745200700 1 --2- 172.21.15.131:0/3514231522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0740101f70 0x7f0740199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:57.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:57.039+0000 7f0745200700 1 -- 172.21.15.131:0/3514231522 >> 172.21.15.131:0/3514231522 conn(0x7f07400fc3b0 msgr2=0x7f07400731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:57.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:57.039+0000 7f0745200700 1 -- 172.21.15.131:0/3514231522 shutdown_connections 2024-05-08T22:06:57.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:57.039+0000 7f0745200700 1 -- 172.21.15.131:0/3514231522 wait complete. 2024-05-08T22:06:57.043 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 23 2024-05-08T22:06:57.476 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-08T22:06:57.844 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:06:58.086 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:57 smithi131 ceph-mon[37260]: pgmap v295: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T22:06:58.086 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:57 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3514231522' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T22:06:58.089 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:57 smithi132 ceph-mon[78180]: pgmap v295: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T22:06:58.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:57 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3514231522' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T22:06:58.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:57 smithi157 ceph-mon[77746]: pgmap v295: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T22:06:58.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:57 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3514231522' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T22:06:59.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.188+0000 7f4068200700 1 -- 172.21.15.131:0/2758444056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4060107600 msgr2=0x7f4060107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:59.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.188+0000 7f4068200700 1 --2- 172.21.15.131:0/2758444056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4060107600 0x7f4060107a20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f40540099c0 tx=0x7f4054009cd0 comp rx=0 tx=0).stop 2024-05-08T22:06:59.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.188+0000 7f4068200700 1 -- 172.21.15.131:0/2758444056 shutdown_connections 2024-05-08T22:06:59.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.188+0000 7f4068200700 1 --2- 172.21.15.131:0/2758444056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40601091a0 0x7f406010fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:59.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.188+0000 7f4068200700 1 --2- 172.21.15.131:0/2758444056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4060107600 0x7f4060107a20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:59.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.188+0000 7f4068200700 1 --2- 172.21.15.131:0/2758444056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4060108800 0x7f4060108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:59.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.188+0000 7f4068200700 1 -- 172.21.15.131:0/2758444056 >> 172.21.15.131:0/2758444056 conn(0x7f40600765d0 msgr2=0x7f4060078a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:59.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.188+0000 7f4068200700 1 -- 172.21.15.131:0/2758444056 shutdown_connections 2024-05-08T22:06:59.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.188+0000 7f4068200700 1 -- 172.21.15.131:0/2758444056 wait complete. 2024-05-08T22:06:59.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.189+0000 7f4068200700 1 Processor -- start 2024-05-08T22:06:59.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.189+0000 7f4068200700 1 -- start start 2024-05-08T22:06:59.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.190+0000 7f4068200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4060107600 0x7f406019d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:59.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.190+0000 7f4068200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4060108800 0x7f406019daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:59.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.190+0000 7f4068200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40601091a0 0x7f40601a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:59.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.190+0000 7f4068200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40601a50a0 con 0x7f40601091a0 2024-05-08T22:06:59.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.190+0000 7f4068200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40601a51e0 con 0x7f4060108800 2024-05-08T22:06:59.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.190+0000 7f4068200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40601a5370 con 0x7f4060107600 2024-05-08T22:06:59.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.190+0000 7f4065800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4060107600 0x7f406019d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:59.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.190+0000 7f4065800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4060107600 0x7f406019d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:33884/0 (socket says 172.21.15.131:33884) 2024-05-08T22:06:59.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.190+0000 7f4065800700 1 -- 172.21.15.131:0/766167288 learned_addr learned my addr 172.21.15.131:0/766167288 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:06:59.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.190+0000 7f4064e00700 1 --2- 172.21.15.131:0/766167288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4060108800 0x7f406019daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:59.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.190+0000 7f4066200700 1 --2- 172.21.15.131:0/766167288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40601091a0 0x7f40601a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:59.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.191+0000 7f4065800700 1 -- 172.21.15.131:0/766167288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4060108800 msgr2=0x7f406019daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:59.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.191+0000 7f4065800700 1 --2- 172.21.15.131:0/766167288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4060108800 0x7f406019daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:59.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.191+0000 7f4065800700 1 -- 172.21.15.131:0/766167288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40601091a0 msgr2=0x7f40601a4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:59.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.191+0000 7f4065800700 1 --2- 172.21.15.131:0/766167288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40601091a0 0x7f40601a4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:59.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.191+0000 7f4065800700 1 -- 172.21.15.131:0/766167288 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40540095f0 con 0x7f4060107600 2024-05-08T22:06:59.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.191+0000 7f4066200700 1 --2- 172.21.15.131:0/766167288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40601091a0 0x7f40601a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:06:59.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.191+0000 7f4065800700 1 --2- 172.21.15.131:0/766167288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4060107600 0x7f406019d5b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4054009fd0 tx=0x7f405400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:59.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.191+0000 7f405e000700 1 -- 172.21.15.131:0/766167288 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405401d070 con 0x7f4060107600 2024-05-08T22:06:59.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.191+0000 7f405e000700 1 -- 172.21.15.131:0/766167288 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f405400bda0 con 0x7f4060107600 2024-05-08T22:06:59.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.191+0000 7f4068200700 1 -- 172.21.15.131:0/766167288 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40601a55f0 con 0x7f4060107600 2024-05-08T22:06:59.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.191+0000 7f405e000700 1 -- 172.21.15.131:0/766167288 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054004c70 con 0x7f4060107600 2024-05-08T22:06:59.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.191+0000 7f4068200700 1 -- 172.21.15.131:0/766167288 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40601a5a00 con 0x7f4060107600 2024-05-08T22:06:59.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.192+0000 7f4068200700 1 -- 172.21.15.131:0/766167288 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f406010ea80 con 0x7f4060107600 2024-05-08T22:06:59.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.194+0000 7f405e000700 1 -- 172.21.15.131:0/766167288 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f4054005050 con 0x7f4060107600 2024-05-08T22:06:59.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.195+0000 7f405e000700 1 --2- 172.21.15.131:0/766167288 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f40480a11b0 0x7f40480a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:06:59.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.196+0000 7f405e000700 1 -- 172.21.15.131:0/766167288 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f40540cdc50 con 0x7f4060107600 2024-05-08T22:06:59.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.196+0000 7f4064e00700 1 --2- 172.21.15.131:0/766167288 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f40480a11b0 0x7f40480a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:06:59.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.196+0000 7f4064e00700 1 --2- 172.21.15.131:0/766167288 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f40480a11b0 0x7f40480a3670 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f404c007c60 tx=0x7f404c0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:06:59.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.200+0000 7f405e000700 1 -- 172.21.15.131:0/766167288 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4054096f50 con 0x7f4060107600 2024-05-08T22:06:59.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.404+0000 7f4068200700 1 -- 172.21.15.131:0/766167288 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f406004ec60 con 0x7f4060107600 2024-05-08T22:06:59.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.405+0000 7f405e000700 1 -- 172.21.15.131:0/766167288 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) v1 ==== 56+0+5727 (secure 0 0 0) 0x7f40540ba020 con 0x7f4060107600 2024-05-08T22:06:59.407 INFO:teuthology.orchestra.run.smithi131.stdout:epoch 87 2024-05-08T22:06:59.407 INFO:teuthology.orchestra.run.smithi131.stdout:fsid 06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T22:06:59.407 INFO:teuthology.orchestra.run.smithi131.stdout:created 2024-05-08T21:58:43.079038+0000 2024-05-08T22:06:59.407 INFO:teuthology.orchestra.run.smithi131.stdout:modified 2024-05-08T22:06:41.971918+0000 2024-05-08T22:06:59.407 INFO:teuthology.orchestra.run.smithi131.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-08T22:06:59.407 INFO:teuthology.orchestra.run.smithi131.stdout:crush_version 27 2024-05-08T22:06:59.407 INFO:teuthology.orchestra.run.smithi131.stdout:full_ratio 0.95 2024-05-08T22:06:59.408 INFO:teuthology.orchestra.run.smithi131.stdout:backfillfull_ratio 0.9 2024-05-08T22:06:59.408 INFO:teuthology.orchestra.run.smithi131.stdout:nearfull_ratio 0.85 2024-05-08T22:06:59.408 INFO:teuthology.orchestra.run.smithi131.stdout:require_min_compat_client luminous 2024-05-08T22:06:59.408 INFO:teuthology.orchestra.run.smithi131.stdout:min_compat_client luminous 2024-05-08T22:06:59.408 INFO:teuthology.orchestra.run.smithi131.stdout:require_osd_release reef 2024-05-08T22:06:59.408 INFO:teuthology.orchestra.run.smithi131.stdout:stretch_mode_enabled false 2024-05-08T22:06:59.408 INFO:teuthology.orchestra.run.smithi131.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-08T22:06:59.409 INFO:teuthology.orchestra.run.smithi131.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-08T22:06:59.409 INFO:teuthology.orchestra.run.smithi131.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-08T22:06:59.409 INFO:teuthology.orchestra.run.smithi131.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 83 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-08T22:06:59.409 INFO:teuthology.orchestra.run.smithi131.stdout:max_osd 12 2024-05-08T22:06:59.409 INFO:teuthology.orchestra.run.smithi131.stdout:osd.0 up in weight 1 up_from 8 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6802/206657594,v1:172.21.15.131:6803/206657594] [v2:172.21.15.131:6804/206657594,v1:172.21.15.131:6805/206657594] exists,up 1fba15ea-c8d0-49f9-a7cc-80efdeff2934 2024-05-08T22:06:59.409 INFO:teuthology.orchestra.run.smithi131.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.132:6800/2358880332,v1:172.21.15.132:6801/2358880332] [v2:172.21.15.132:6802/2358880332,v1:172.21.15.132:6803/2358880332] exists,up f6d2536e-d9a6-4073-8297-987d9e885657 2024-05-08T22:06:59.410 INFO:teuthology.orchestra.run.smithi131.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6800/3561264016,v1:172.21.15.157:6801/3561264016] [v2:172.21.15.157:6802/3561264016,v1:172.21.15.157:6803/3561264016] exists,up 992291bf-2d07-42f7-8722-efdbe3dafb17 2024-05-08T22:06:59.410 INFO:teuthology.orchestra.run.smithi131.stdout:osd.3 up in weight 1 up_from 24 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6810/1075134307,v1:172.21.15.131:6811/1075134307] [v2:172.21.15.131:6812/1075134307,v1:172.21.15.131:6813/1075134307] exists,up b8281ef9-2390-4b3e-9ba8-0f7e71ca3616 2024-05-08T22:06:59.410 INFO:teuthology.orchestra.run.smithi131.stdout:osd.4 up in weight 1 up_from 31 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.132:6808/1317083238,v1:172.21.15.132:6809/1317083238] [v2:172.21.15.132:6810/1317083238,v1:172.21.15.132:6811/1317083238] exists,up f1592d54-01e0-4201-905f-89664e2da391 2024-05-08T22:06:59.410 INFO:teuthology.orchestra.run.smithi131.stdout:osd.5 up in weight 1 up_from 36 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6808/1488005925,v1:172.21.15.157:6809/1488005925] [v2:172.21.15.157:6810/1488005925,v1:172.21.15.157:6811/1488005925] exists,up 2d54acce-6950-42eb-b65b-97931044a3f4 2024-05-08T22:06:59.410 INFO:teuthology.orchestra.run.smithi131.stdout:osd.6 up in weight 1 up_from 43 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6818/260285513,v1:172.21.15.131:6819/260285513] [v2:172.21.15.131:6820/260285513,v1:172.21.15.131:6821/260285513] exists,up fecf0f79-034b-4ad3-a194-eb7c3ed23776 2024-05-08T22:06:59.410 INFO:teuthology.orchestra.run.smithi131.stdout:osd.7 up in weight 1 up_from 48 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.132:6816/464003238,v1:172.21.15.132:6817/464003238] [v2:172.21.15.132:6818/464003238,v1:172.21.15.132:6819/464003238] exists,up 0fbdd862-cf84-4929-a74c-5c4be1ce7d5e 2024-05-08T22:06:59.410 INFO:teuthology.orchestra.run.smithi131.stdout:osd.8 up in weight 1 up_from 55 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6816/3594097089,v1:172.21.15.157:6817/3594097089] [v2:172.21.15.157:6818/3594097089,v1:172.21.15.157:6819/3594097089] exists,up 47077c11-3248-442f-b15c-692931137f8a 2024-05-08T22:06:59.410 INFO:teuthology.orchestra.run.smithi131.stdout:osd.9 up in weight 1 up_from 62 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6826/4217805693,v1:172.21.15.131:6827/4217805693] [v2:172.21.15.131:6828/4217805693,v1:172.21.15.131:6829/4217805693] exists,up b027ba0a-8aec-45af-8137-b261c1588499 2024-05-08T22:06:59.410 INFO:teuthology.orchestra.run.smithi131.stdout:osd.10 up in weight 1 up_from 67 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.132:6824/306651644,v1:172.21.15.132:6825/306651644] [v2:172.21.15.132:6826/306651644,v1:172.21.15.132:6827/306651644] exists,up 430013a4-d468-49b2-9162-165ac5a4a7f8 2024-05-08T22:06:59.410 INFO:teuthology.orchestra.run.smithi131.stdout:osd.11 up in weight 1 up_from 74 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6824/2884744696,v1:172.21.15.157:6825/2884744696] [v2:172.21.15.157:6826/2884744696,v1:172.21.15.157:6827/2884744696] exists,up 76622ed0-d3b9-4f55-9206-fc96c56e18ed 2024-05-08T22:06:59.410 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 2.c [2,1] 2024-05-08T22:06:59.410 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 2.10 [2,4] 2024-05-08T22:06:59.410 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 2.1c [8,11] 2024-05-08T22:06:59.411 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 2.1d [8,11] 2024-05-08T22:06:59.411 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 3.2 [3,9] 2024-05-08T22:06:59.411 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 3.3 [4,9] 2024-05-08T22:06:59.411 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 3.1a [5,6] 2024-05-08T22:06:59.411 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 3.1e [3,9] 2024-05-08T22:06:59.411 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 3.21 [4,8] 2024-05-08T22:06:59.411 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 3.27 [4,7] 2024-05-08T22:06:59.411 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 4.6 [9,11] 2024-05-08T22:06:59.411 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 4.7 [5,8] 2024-05-08T22:06:59.411 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 4.14 [9,8] 2024-05-08T22:06:59.411 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 4.18 [9,8] 2024-05-08T22:06:59.411 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:6801/1418340798 expires 2024-05-09T21:59:40.641373+0000 2024-05-08T22:06:59.411 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:6800/1418340798 expires 2024-05-09T21:59:40.641373+0000 2024-05-08T22:06:59.412 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:0/1241655400 expires 2024-05-09T21:59:40.641373+0000 2024-05-08T22:06:59.412 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:0/1716895483 expires 2024-05-09T21:59:40.641373+0000 2024-05-08T22:06:59.412 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:0/1981506111 expires 2024-05-09T21:59:15.548383+0000 2024-05-08T22:06:59.412 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:0/976983862 expires 2024-05-09T21:59:15.548383+0000 2024-05-08T22:06:59.412 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:6801/1564023164 expires 2024-05-09T21:59:15.548383+0000 2024-05-08T22:06:59.412 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:0/3309822030 expires 2024-05-09T21:59:15.548383+0000 2024-05-08T22:06:59.412 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:0/3082367829 expires 2024-05-09T21:59:40.641373+0000 2024-05-08T22:06:59.412 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:6800/1564023164 expires 2024-05-09T21:59:15.548383+0000 2024-05-08T22:06:59.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.408+0000 7f4068200700 1 -- 172.21.15.131:0/766167288 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f40480a11b0 msgr2=0x7f40480a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:59.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.408+0000 7f4068200700 1 --2- 172.21.15.131:0/766167288 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f40480a11b0 0x7f40480a3670 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f404c007c60 tx=0x7f404c0073d0 comp rx=0 tx=0).stop 2024-05-08T22:06:59.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.408+0000 7f4068200700 1 -- 172.21.15.131:0/766167288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4060107600 msgr2=0x7f406019d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:06:59.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.408+0000 7f4068200700 1 --2- 172.21.15.131:0/766167288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4060107600 0x7f406019d5b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4054009fd0 tx=0x7f405400fa00 comp rx=0 tx=0).stop 2024-05-08T22:06:59.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.408+0000 7f4068200700 1 -- 172.21.15.131:0/766167288 shutdown_connections 2024-05-08T22:06:59.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.408+0000 7f4068200700 1 --2- 172.21.15.131:0/766167288 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f40480a11b0 0x7f40480a3670 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:59.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.408+0000 7f4068200700 1 --2- 172.21.15.131:0/766167288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40601091a0 0x7f40601a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:59.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.408+0000 7f4068200700 1 --2- 172.21.15.131:0/766167288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4060107600 0x7f406019d5b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:59.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.408+0000 7f4068200700 1 --2- 172.21.15.131:0/766167288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4060108800 0x7f406019daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:06:59.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.408+0000 7f4068200700 1 -- 172.21.15.131:0/766167288 >> 172.21.15.131:0/766167288 conn(0x7f40600765d0 msgr2=0x7f406010bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:06:59.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.408+0000 7f4068200700 1 -- 172.21.15.131:0/766167288 shutdown_connections 2024-05-08T22:06:59.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:06:59.408+0000 7f4068200700 1 -- 172.21.15.131:0/766167288 wait complete. 2024-05-08T22:06:59.883 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-08T22:06:59.895 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:07:00.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:59 smithi132 ceph-mon[78180]: pgmap v296: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:07:00.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:06:59 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/766167288' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T22:07:00.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.093+0000 7f5c34e00700 1 -- 172.21.15.131:0/683667295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c30102950 msgr2=0x7f5c30109270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:00.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.093+0000 7f5c34e00700 1 --2- 172.21.15.131:0/683667295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c30102950 0x7f5c30109270 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5c2800b0a0 tx=0x7f5c2800b3b0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.094+0000 7f5c34e00700 1 -- 172.21.15.131:0/683667295 shutdown_connections 2024-05-08T22:07:00.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.094+0000 7f5c34e00700 1 --2- 172.21.15.131:0/683667295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c30102950 0x7f5c30109270 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.094+0000 7f5c34e00700 1 --2- 172.21.15.131:0/683667295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c30100e40 0x7f5c30101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.094+0000 7f5c34e00700 1 --2- 172.21.15.131:0/683667295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30101f90 0x7f5c30102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.094+0000 7f5c34e00700 1 -- 172.21.15.131:0/683667295 >> 172.21.15.131:0/683667295 conn(0x7f5c300fc3b0 msgr2=0x7f5c300fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:00.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.094+0000 7f5c34e00700 1 -- 172.21.15.131:0/683667295 shutdown_connections 2024-05-08T22:07:00.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.094+0000 7f5c34e00700 1 -- 172.21.15.131:0/683667295 wait complete. 2024-05-08T22:07:00.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.095+0000 7f5c34e00700 1 Processor -- start 2024-05-08T22:07:00.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.095+0000 7f5c34e00700 1 -- start start 2024-05-08T22:07:00.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.096+0000 7f5c34e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c30100e40 0x7f5c30074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:00.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.096+0000 7f5c34e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30101f90 0x7f5c30073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:00.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.096+0000 7f5c34e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c30102950 0x7f5c300738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:00.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.096+0000 7f5c34e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c30074070 con 0x7f5c30100e40 2024-05-08T22:07:00.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.096+0000 7f5c34e00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c300741e0 con 0x7f5c30102950 2024-05-08T22:07:00.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.096+0000 7f5c34e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c301a0f50 con 0x7f5c30101f90 2024-05-08T22:07:00.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.096+0000 7f5c2f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c30100e40 0x7f5c30074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:00.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.096+0000 7f5c2f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c30100e40 0x7f5c30074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56148/0 (socket says 172.21.15.131:56148) 2024-05-08T22:07:00.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.096+0000 7f5c2f400700 1 -- 172.21.15.131:0/432656083 learned_addr learned my addr 172.21.15.131:0/432656083 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:00.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.096+0000 7f5c2fe00700 1 --2- 172.21.15.131:0/432656083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c30102950 0x7f5c300738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:00.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.096+0000 7f5c2ea00700 1 --2- 172.21.15.131:0/432656083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30101f90 0x7f5c30073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:00.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.097+0000 7f5c2f400700 1 -- 172.21.15.131:0/432656083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30101f90 msgr2=0x7f5c30073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:00.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.097+0000 7f5c2f400700 1 --2- 172.21.15.131:0/432656083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30101f90 0x7f5c30073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.097+0000 7f5c2f400700 1 -- 172.21.15.131:0/432656083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c30102950 msgr2=0x7f5c300738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:00.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.097+0000 7f5c2f400700 1 --2- 172.21.15.131:0/432656083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c30102950 0x7f5c300738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.097+0000 7f5c2f400700 1 -- 172.21.15.131:0/432656083 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c28009d00 con 0x7f5c30100e40 2024-05-08T22:07:00.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.097+0000 7f5c2fe00700 1 --2- 172.21.15.131:0/432656083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c30102950 0x7f5c300738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:00.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.097+0000 7f5c2f400700 1 --2- 172.21.15.131:0/432656083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c30100e40 0x7f5c30074d00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000e9a0 tx=0x7f5c2000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:00.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.097+0000 7f5c27e00700 1 -- 172.21.15.131:0/432656083 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c20009790 con 0x7f5c30100e40 2024-05-08T22:07:00.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.097+0000 7f5c27e00700 1 -- 172.21.15.131:0/432656083 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c2000ce70 con 0x7f5c30100e40 2024-05-08T22:07:00.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.097+0000 7f5c34e00700 1 -- 172.21.15.131:0/432656083 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c301a11e0 con 0x7f5c30100e40 2024-05-08T22:07:00.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.097+0000 7f5c27e00700 1 -- 172.21.15.131:0/432656083 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c20018420 con 0x7f5c30100e40 2024-05-08T22:07:00.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.097+0000 7f5c34e00700 1 -- 172.21.15.131:0/432656083 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c301a16f0 con 0x7f5c30100e40 2024-05-08T22:07:00.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.098+0000 7f5c34e00700 1 -- 172.21.15.131:0/432656083 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c301082b0 con 0x7f5c30100e40 2024-05-08T22:07:00.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.100+0000 7f5c27e00700 1 -- 172.21.15.131:0/432656083 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f5c200106f0 con 0x7f5c30100e40 2024-05-08T22:07:00.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.102+0000 7f5c27e00700 1 --2- 172.21.15.131:0/432656083 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f5c140a11b0 0x7f5c140a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:00.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.102+0000 7f5c27e00700 1 -- 172.21.15.131:0/432656083 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f5c20014070 con 0x7f5c30100e40 2024-05-08T22:07:00.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.102+0000 7f5c2ea00700 1 --2- 172.21.15.131:0/432656083 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f5c140a11b0 0x7f5c140a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:00.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.103+0000 7f5c2ea00700 1 --2- 172.21.15.131:0/432656083 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f5c140a11b0 0x7f5c140a3670 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c004630 tx=0x7f5c1c00b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:00.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.107+0000 7f5c27e00700 1 -- 172.21.15.131:0/432656083 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5c20052d60 con 0x7f5c30100e40 2024-05-08T22:07:00.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:59 smithi131 ceph-mon[37260]: pgmap v296: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:07:00.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:06:59 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/766167288' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T22:07:00.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:59 smithi157 ceph-mon[77746]: pgmap v296: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:07:00.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:06:59 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/766167288' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T22:07:00.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.315+0000 7f5c34e00700 1 -- 172.21.15.131:0/432656083 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c30067ba0 con 0x7f5c30100e40 2024-05-08T22:07:00.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.317+0000 7f5c27e00700 1 -- 172.21.15.131:0/432656083 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24721 (secure 0 0 0) 0x7f5c20099ca0 con 0x7f5c30100e40 2024-05-08T22:07:00.318 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:07:00.319 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":87,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:06:41.971918+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:05:12.469858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:05:13.484446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:05:17.016186+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:07:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.319+0000 7f5c34e00700 1 -- 172.21.15.131:0/432656083 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f5c140a11b0 msgr2=0x7f5c140a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.320+0000 7f5c34e00700 1 --2- 172.21.15.131:0/432656083 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f5c140a11b0 0x7f5c140a3670 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c004630 tx=0x7f5c1c00b3c0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.320+0000 7f5c34e00700 1 -- 172.21.15.131:0/432656083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c30100e40 msgr2=0x7f5c30074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.320+0000 7f5c34e00700 1 --2- 172.21.15.131:0/432656083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c30100e40 0x7f5c30074d00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000e9a0 tx=0x7f5c2000ecb0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.320+0000 7f5c34e00700 1 -- 172.21.15.131:0/432656083 shutdown_connections 2024-05-08T22:07:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.320+0000 7f5c34e00700 1 --2- 172.21.15.131:0/432656083 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f5c140a11b0 0x7f5c140a3670 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.320+0000 7f5c34e00700 1 --2- 172.21.15.131:0/432656083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c30102950 0x7f5c300738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.320+0000 7f5c34e00700 1 --2- 172.21.15.131:0/432656083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c30100e40 0x7f5c30074d00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.320+0000 7f5c34e00700 1 --2- 172.21.15.131:0/432656083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30101f90 0x7f5c30073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.320+0000 7f5c34e00700 1 -- 172.21.15.131:0/432656083 >> 172.21.15.131:0/432656083 conn(0x7f5c300fc3b0 msgr2=0x7f5c301053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.320+0000 7f5c34e00700 1 -- 172.21.15.131:0/432656083 shutdown_connections 2024-05-08T22:07:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.320+0000 7f5c34e00700 1 -- 172.21.15.131:0/432656083 wait complete. 2024-05-08T22:07:00.340 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:01:41.271078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:05:12.469858+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:05:13.484446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:05:17.016186+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:07:00.341 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:07:00.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.536+0000 7fd7fc800700 1 -- 172.21.15.131:0/2641485504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7f4100f80 msgr2=0x7fd7f41013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:00.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.536+0000 7fd7fc800700 1 --2- 172.21.15.131:0/2641485504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7f4100f80 0x7fd7f41013a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80099c0 tx=0x7fd7e8009cd0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.536+0000 7fd7fc800700 1 -- 172.21.15.131:0/2641485504 shutdown_connections 2024-05-08T22:07:00.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.536+0000 7fd7fc800700 1 --2- 172.21.15.131:0/2641485504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7f4102b10 0x7fd7f41093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.536+0000 7fd7fc800700 1 --2- 172.21.15.131:0/2641485504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7f4100f80 0x7fd7f41013a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.536+0000 7fd7fc800700 1 --2- 172.21.15.131:0/2641485504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7f4102170 0x7fd7f41025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.536+0000 7fd7fc800700 1 -- 172.21.15.131:0/2641485504 >> 172.21.15.131:0/2641485504 conn(0x7fd7f40fc4f0 msgr2=0x7fd7f40fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:00.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.536+0000 7fd7fc800700 1 -- 172.21.15.131:0/2641485504 shutdown_connections 2024-05-08T22:07:00.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.536+0000 7fd7fc800700 1 -- 172.21.15.131:0/2641485504 wait complete. 2024-05-08T22:07:00.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.537+0000 7fd7fc800700 1 Processor -- start 2024-05-08T22:07:00.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.537+0000 7fd7fc800700 1 -- start start 2024-05-08T22:07:00.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.538+0000 7fd7fc800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7f4100f80 0x7fd7f4199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:00.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.538+0000 7fd7fc800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7f4102170 0x7fd7f4199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:00.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.538+0000 7fd7fc800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7f4102b10 0x7fd7f41a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:00.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.538+0000 7fd7fc800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f41a0c60 con 0x7fd7f4100f80 2024-05-08T22:07:00.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.538+0000 7fd7fc800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f41a0da0 con 0x7fd7f4102b10 2024-05-08T22:07:00.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.538+0000 7fd7fc800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f41a0f30 con 0x7fd7f4102170 2024-05-08T22:07:00.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.538+0000 7fd7f9e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7f4100f80 0x7fd7f4199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:00.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.538+0000 7fd7f9e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7f4100f80 0x7fd7f4199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56156/0 (socket says 172.21.15.131:56156) 2024-05-08T22:07:00.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.538+0000 7fd7f9e00700 1 -- 172.21.15.131:0/1744227220 learned_addr learned my addr 172.21.15.131:0/1744227220 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:00.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.538+0000 7fd7f9400700 1 --2- 172.21.15.131:0/1744227220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7f4102170 0x7fd7f4199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:00.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.538+0000 7fd7fa800700 1 --2- 172.21.15.131:0/1744227220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7f4102b10 0x7fd7f41a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:00.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.538+0000 7fd7f9e00700 1 -- 172.21.15.131:0/1744227220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7f4102170 msgr2=0x7fd7f4199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:00.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.538+0000 7fd7f9e00700 1 --2- 172.21.15.131:0/1744227220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7f4102170 0x7fd7f4199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.538+0000 7fd7f9e00700 1 -- 172.21.15.131:0/1744227220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7f4102b10 msgr2=0x7fd7f41a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:00.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.538+0000 7fd7f9e00700 1 --2- 172.21.15.131:0/1744227220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7f4102b10 0x7fd7f41a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.538+0000 7fd7f9e00700 1 -- 172.21.15.131:0/1744227220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e80095f0 con 0x7fd7f4100f80 2024-05-08T22:07:00.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.538+0000 7fd7f9400700 1 --2- 172.21.15.131:0/1744227220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7f4102170 0x7fd7f4199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:00.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.539+0000 7fd7f9e00700 1 --2- 172.21.15.131:0/1744227220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7f4100f80 0x7fd7f4199130 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8003730 tx=0x7fd7e800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:00.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.539+0000 7fd7eea00700 1 -- 172.21.15.131:0/1744227220 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e801d070 con 0x7fd7f4100f80 2024-05-08T22:07:00.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.539+0000 7fd7eea00700 1 -- 172.21.15.131:0/1744227220 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7e800bda0 con 0x7fd7f4100f80 2024-05-08T22:07:00.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.539+0000 7fd7fc800700 1 -- 172.21.15.131:0/1744227220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f41a11b0 con 0x7fd7f4100f80 2024-05-08T22:07:00.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.539+0000 7fd7eea00700 1 -- 172.21.15.131:0/1744227220 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e80173f0 con 0x7fd7f4100f80 2024-05-08T22:07:00.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.539+0000 7fd7fc800700 1 -- 172.21.15.131:0/1744227220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f41a15d0 con 0x7fd7f4100f80 2024-05-08T22:07:00.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.540+0000 7fd7fc800700 1 -- 172.21.15.131:0/1744227220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f4067cf0 con 0x7fd7f4100f80 2024-05-08T22:07:00.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.542+0000 7fd7eea00700 1 -- 172.21.15.131:0/1744227220 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fd7e8022470 con 0x7fd7f4100f80 2024-05-08T22:07:00.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.543+0000 7fd7eea00700 1 --2- 172.21.15.131:0/1744227220 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd7dc0a11b0 0x7fd7dc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:00.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.543+0000 7fd7eea00700 1 -- 172.21.15.131:0/1744227220 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fd7e80ce3d0 con 0x7fd7f4100f80 2024-05-08T22:07:00.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.544+0000 7fd7f9400700 1 --2- 172.21.15.131:0/1744227220 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd7dc0a11b0 0x7fd7dc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:00.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.544+0000 7fd7f9400700 1 --2- 172.21.15.131:0/1744227220 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd7dc0a11b0 0x7fd7dc0a3670 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4010420 tx=0x7fd7e40073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:00.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.545+0000 7fd7eea00700 1 -- 172.21.15.131:0/1744227220 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd7e80976d0 con 0x7fd7f4100f80 2024-05-08T22:07:00.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.755+0000 7fd7fc800700 1 -- 172.21.15.131:0/1744227220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd7f410c970 con 0x7fd7f4100f80 2024-05-08T22:07:00.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.755+0000 7fd7eea00700 1 -- 172.21.15.131:0/1744227220 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd7e809acf0 con 0x7fd7f4100f80 2024-05-08T22:07:00.756 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-08T22:07:00.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.758+0000 7fd7fc800700 1 -- 172.21.15.131:0/1744227220 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd7dc0a11b0 msgr2=0x7fd7dc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:00.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.758+0000 7fd7fc800700 1 --2- 172.21.15.131:0/1744227220 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd7dc0a11b0 0x7fd7dc0a3670 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4010420 tx=0x7fd7e40073d0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.758+0000 7fd7fc800700 1 -- 172.21.15.131:0/1744227220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7f4100f80 msgr2=0x7fd7f4199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:00.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.758+0000 7fd7fc800700 1 --2- 172.21.15.131:0/1744227220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7f4100f80 0x7fd7f4199130 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8003730 tx=0x7fd7e800fa00 comp rx=0 tx=0).stop 2024-05-08T22:07:00.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.758+0000 7fd7fc800700 1 -- 172.21.15.131:0/1744227220 shutdown_connections 2024-05-08T22:07:00.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.758+0000 7fd7fc800700 1 --2- 172.21.15.131:0/1744227220 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd7dc0a11b0 0x7fd7dc0a3670 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.758+0000 7fd7fc800700 1 --2- 172.21.15.131:0/1744227220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7f4102b10 0x7fd7f41a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.758+0000 7fd7fc800700 1 --2- 172.21.15.131:0/1744227220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7f4100f80 0x7fd7f4199130 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.758+0000 7fd7fc800700 1 --2- 172.21.15.131:0/1744227220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7f4102170 0x7fd7f4199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.758+0000 7fd7fc800700 1 -- 172.21.15.131:0/1744227220 >> 172.21.15.131:0/1744227220 conn(0x7fd7f40fc4f0 msgr2=0x7fd7f40731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:00.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.758+0000 7fd7fc800700 1 -- 172.21.15.131:0/1744227220 shutdown_connections 2024-05-08T22:07:00.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.758+0000 7fd7fc800700 1 -- 172.21.15.131:0/1744227220 wait complete. 2024-05-08T22:07:00.778 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:07:00.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.978+0000 7fa548000700 1 -- 172.21.15.131:0/3834668738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5401020f0 msgr2=0x7fa540102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:00.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.978+0000 7fa548000700 1 --2- 172.21.15.131:0/3834668738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5401020f0 0x7fa540102570 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009990 tx=0x7fa52c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.978+0000 7fa548000700 1 -- 172.21.15.131:0/3834668738 shutdown_connections 2024-05-08T22:07:00.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.978+0000 7fa548000700 1 --2- 172.21.15.131:0/3834668738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa540102ab0 0x7fa5401093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.978+0000 7fa548000700 1 --2- 172.21.15.131:0/3834668738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa540100f20 0x7fa540101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.978+0000 7fa548000700 1 --2- 172.21.15.131:0/3834668738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5401020f0 0x7fa540102570 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.978+0000 7fa548000700 1 -- 172.21.15.131:0/3834668738 >> 172.21.15.131:0/3834668738 conn(0x7fa5400fc500 msgr2=0x7fa5400fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:00.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.978+0000 7fa548000700 1 -- 172.21.15.131:0/3834668738 shutdown_connections 2024-05-08T22:07:00.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.978+0000 7fa548000700 1 -- 172.21.15.131:0/3834668738 wait complete. 2024-05-08T22:07:00.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.979+0000 7fa548000700 1 Processor -- start 2024-05-08T22:07:00.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.979+0000 7fa548000700 1 -- start start 2024-05-08T22:07:00.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.980+0000 7fa548000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa540100f20 0x7fa540074cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:00.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.980+0000 7fa548000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5401020f0 0x7fa540073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:00.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.980+0000 7fa548000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa540102ab0 0x7fa5400738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:00.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.980+0000 7fa548000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa540073fd0 con 0x7fa540102ab0 2024-05-08T22:07:00.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.980+0000 7fa548000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa540074140 con 0x7fa5401020f0 2024-05-08T22:07:00.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.980+0000 7fa548000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5401a1000 con 0x7fa540100f20 2024-05-08T22:07:00.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.980+0000 7fa546000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa540102ab0 0x7fa5400738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:00.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.980+0000 7fa546000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa540102ab0 0x7fa5400738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56168/0 (socket says 172.21.15.131:56168) 2024-05-08T22:07:00.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.980+0000 7fa546000700 1 -- 172.21.15.131:0/2620891254 learned_addr learned my addr 172.21.15.131:0/2620891254 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:00.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.980+0000 7fa544c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5401020f0 0x7fa540073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:00.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.980+0000 7fa545600700 1 --2- 172.21.15.131:0/2620891254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa540100f20 0x7fa540074cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:00.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.980+0000 7fa546000700 1 -- 172.21.15.131:0/2620891254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa540100f20 msgr2=0x7fa540074cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:00.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.980+0000 7fa546000700 1 --2- 172.21.15.131:0/2620891254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa540100f20 0x7fa540074cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.980+0000 7fa546000700 1 -- 172.21.15.131:0/2620891254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5401020f0 msgr2=0x7fa540073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:00.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.980+0000 7fa546000700 1 --2- 172.21.15.131:0/2620891254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5401020f0 0x7fa540073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:00.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.980+0000 7fa546000700 1 -- 172.21.15.131:0/2620891254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c0095f0 con 0x7fa540102ab0 2024-05-08T22:07:00.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.981+0000 7fa546000700 1 --2- 172.21.15.131:0/2620891254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa540102ab0 0x7fa5400738d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa53800ef30 tx=0x7fa53800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:00.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.981+0000 7fa53e000700 1 -- 172.21.15.131:0/2620891254 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538018070 con 0x7fa540102ab0 2024-05-08T22:07:00.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.981+0000 7fa53e000700 1 -- 172.21.15.131:0/2620891254 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa53800f040 con 0x7fa540102ab0 2024-05-08T22:07:00.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.981+0000 7fa548000700 1 -- 172.21.15.131:0/2620891254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5401a1200 con 0x7fa540102ab0 2024-05-08T22:07:00.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.981+0000 7fa53e000700 1 -- 172.21.15.131:0/2620891254 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53800cd00 con 0x7fa540102ab0 2024-05-08T22:07:00.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.981+0000 7fa548000700 1 -- 172.21.15.131:0/2620891254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5401a1710 con 0x7fa540102ab0 2024-05-08T22:07:00.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.982+0000 7fa548000700 1 -- 172.21.15.131:0/2620891254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa540108480 con 0x7fa540102ab0 2024-05-08T22:07:00.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.984+0000 7fa53e000700 1 -- 172.21.15.131:0/2620891254 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fa538003a00 con 0x7fa540102ab0 2024-05-08T22:07:00.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.985+0000 7fa53e000700 1 --2- 172.21.15.131:0/2620891254 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa5280a1230 0x7fa5280a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:00.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.985+0000 7fa53e000700 1 -- 172.21.15.131:0/2620891254 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa5380cc900 con 0x7fa540102ab0 2024-05-08T22:07:00.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.986+0000 7fa545600700 1 --2- 172.21.15.131:0/2620891254 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa5280a1230 0x7fa5280a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:00.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.986+0000 7fa545600700 1 --2- 172.21.15.131:0/2620891254 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa5280a1230 0x7fa5280a36f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa540100c40 tx=0x7fa534009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:00.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:00.990+0000 7fa53e000700 1 -- 172.21.15.131:0/2620891254 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa538095c00 con 0x7fa540102ab0 2024-05-08T22:07:01.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:00 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/432656083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:07:01.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:00 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1744227220' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:07:01.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:00 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/432656083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:07:01.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:00 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1744227220' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:07:01.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.196+0000 7fa548000700 1 -- 172.21.15.131:0/2620891254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa54010c800 con 0x7fa540102ab0 2024-05-08T22:07:01.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.196+0000 7fa53e000700 1 -- 172.21.15.131:0/2620891254 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa538099220 con 0x7fa540102ab0 2024-05-08T22:07:01.197 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-08T22:07:01.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.198+0000 7fa548000700 1 -- 172.21.15.131:0/2620891254 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa5280a1230 msgr2=0x7fa5280a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:01.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.198+0000 7fa548000700 1 --2- 172.21.15.131:0/2620891254 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa5280a1230 0x7fa5280a36f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa540100c40 tx=0x7fa534009260 comp rx=0 tx=0).stop 2024-05-08T22:07:01.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.198+0000 7fa548000700 1 -- 172.21.15.131:0/2620891254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa540102ab0 msgr2=0x7fa5400738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:01.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.198+0000 7fa548000700 1 --2- 172.21.15.131:0/2620891254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa540102ab0 0x7fa5400738d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa53800ef30 tx=0x7fa53800c3a0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.199+0000 7fa548000700 1 -- 172.21.15.131:0/2620891254 shutdown_connections 2024-05-08T22:07:01.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.199+0000 7fa548000700 1 --2- 172.21.15.131:0/2620891254 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa5280a1230 0x7fa5280a36f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.199+0000 7fa548000700 1 --2- 172.21.15.131:0/2620891254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa540102ab0 0x7fa5400738d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.199+0000 7fa548000700 1 --2- 172.21.15.131:0/2620891254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa540100f20 0x7fa540074cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.199+0000 7fa548000700 1 --2- 172.21.15.131:0/2620891254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5401020f0 0x7fa540073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.199+0000 7fa548000700 1 -- 172.21.15.131:0/2620891254 >> 172.21.15.131:0/2620891254 conn(0x7fa5400fc500 msgr2=0x7fa5401055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:01.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.199+0000 7fa548000700 1 -- 172.21.15.131:0/2620891254 shutdown_connections 2024-05-08T22:07:01.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.199+0000 7fa548000700 1 -- 172.21.15.131:0/2620891254 wait complete. 2024-05-08T22:07:01.220 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:07:01.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:00 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/432656083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:07:01.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:00 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1744227220' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:07:01.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.418+0000 7f52cd600700 1 -- 172.21.15.131:0/3145503385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c8102170 msgr2=0x7f52c81025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:01.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.418+0000 7f52cd600700 1 --2- 172.21.15.131:0/3145503385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c8102170 0x7f52c81025d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f52b40098f0 tx=0x7f52b4009c00 comp rx=0 tx=0).stop 2024-05-08T22:07:01.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.418+0000 7f52cd600700 1 -- 172.21.15.131:0/3145503385 shutdown_connections 2024-05-08T22:07:01.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.418+0000 7f52cd600700 1 --2- 172.21.15.131:0/3145503385 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c8102b10 0x7f52c81093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.418+0000 7f52cd600700 1 --2- 172.21.15.131:0/3145503385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52c8100f80 0x7f52c81013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.418+0000 7f52cd600700 1 --2- 172.21.15.131:0/3145503385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c8102170 0x7f52c81025d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.418+0000 7f52cd600700 1 -- 172.21.15.131:0/3145503385 >> 172.21.15.131:0/3145503385 conn(0x7f52c80fc4f0 msgr2=0x7f52c80fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:01.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.418+0000 7f52cd600700 1 -- 172.21.15.131:0/3145503385 shutdown_connections 2024-05-08T22:07:01.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.418+0000 7f52cd600700 1 -- 172.21.15.131:0/3145503385 wait complete. 2024-05-08T22:07:01.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.419+0000 7f52cd600700 1 Processor -- start 2024-05-08T22:07:01.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.419+0000 7f52cd600700 1 -- start start 2024-05-08T22:07:01.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.419+0000 7f52cd600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c8100f80 0x7f52c8196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.419+0000 7f52cd600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c8102170 0x7f52c81974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.419+0000 7f52cd600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52c8102b10 0x7f52c819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.419+0000 7f52cd600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c819ea60 con 0x7f52c8102b10 2024-05-08T22:07:01.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.419+0000 7f52cd600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c819eba0 con 0x7f52c8102170 2024-05-08T22:07:01.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.419+0000 7f52cd600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c819ed30 con 0x7f52c8100f80 2024-05-08T22:07:01.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.420+0000 7f52c7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52c8102b10 0x7f52c819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:01.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.420+0000 7f52c6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c8100f80 0x7f52c8196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:01.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.420+0000 7f52c7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52c8102b10 0x7f52c819e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56186/0 (socket says 172.21.15.131:56186) 2024-05-08T22:07:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.420+0000 7f52c7400700 1 -- 172.21.15.131:0/3577861003 learned_addr learned my addr 172.21.15.131:0/3577861003 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.420+0000 7f52c6000700 1 --2- 172.21.15.131:0/3577861003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c8102170 0x7f52c81974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.420+0000 7f52c7400700 1 -- 172.21.15.131:0/3577861003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c8100f80 msgr2=0x7f52c8196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.420+0000 7f52c7400700 1 --2- 172.21.15.131:0/3577861003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c8100f80 0x7f52c8196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.420+0000 7f52c7400700 1 -- 172.21.15.131:0/3577861003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c8102170 msgr2=0x7f52c81974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.420+0000 7f52c7400700 1 --2- 172.21.15.131:0/3577861003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c8102170 0x7f52c81974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.420+0000 7f52c7400700 1 -- 172.21.15.131:0/3577861003 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b4009520 con 0x7f52c8102b10 2024-05-08T22:07:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.420+0000 7f52c6a00700 1 --2- 172.21.15.131:0/3577861003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c8100f80 0x7f52c8196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.420+0000 7f52c7400700 1 --2- 172.21.15.131:0/3577861003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52c8102b10 0x7f52c819e490 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f52c000ef30 tx=0x7f52c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.421+0000 7f52bf400700 1 -- 172.21.15.131:0/3577861003 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c0018070 con 0x7f52c8102b10 2024-05-08T22:07:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.421+0000 7f52bf400700 1 -- 172.21.15.131:0/3577861003 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52c000f040 con 0x7f52c8102b10 2024-05-08T22:07:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.421+0000 7f52cd600700 1 -- 172.21.15.131:0/3577861003 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c819efb0 con 0x7f52c8102b10 2024-05-08T22:07:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.421+0000 7f52bf400700 1 -- 172.21.15.131:0/3577861003 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c000ccf0 con 0x7f52c8102b10 2024-05-08T22:07:01.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.421+0000 7f52cd600700 1 -- 172.21.15.131:0/3577861003 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c819f3e0 con 0x7f52c8102b10 2024-05-08T22:07:01.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.422+0000 7f52cd600700 1 -- 172.21.15.131:0/3577861003 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c8108450 con 0x7f52c8102b10 2024-05-08T22:07:01.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.424+0000 7f52bf400700 1 -- 172.21.15.131:0/3577861003 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f52c0003a00 con 0x7f52c8102b10 2024-05-08T22:07:01.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.425+0000 7f52bf400700 1 --2- 172.21.15.131:0/3577861003 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f52ac0a11b0 0x7f52ac0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.425+0000 7f52bf400700 1 -- 172.21.15.131:0/3577861003 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f52c00cd2b0 con 0x7f52c8102b10 2024-05-08T22:07:01.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.425+0000 7f52c6a00700 1 --2- 172.21.15.131:0/3577861003 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f52ac0a11b0 0x7f52ac0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:01.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.426+0000 7f52c6a00700 1 --2- 172.21.15.131:0/3577861003 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f52ac0a11b0 0x7f52ac0a3670 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f52c8101fd0 tx=0x7f52b8009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:01.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.427+0000 7f52bf400700 1 -- 172.21.15.131:0/3577861003 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f52c00965b0 con 0x7f52c8102b10 2024-05-08T22:07:01.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.636+0000 7f52cd600700 1 -- 172.21.15.131:0/3577861003 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f52c810c7c0 con 0x7f52c8102b10 2024-05-08T22:07:01.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.636+0000 7f52bf400700 1 -- 172.21.15.131:0/3577861003 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f52c0099bd0 con 0x7f52c8102b10 2024-05-08T22:07:01.638 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-08T22:07:01.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.639+0000 7f52cd600700 1 -- 172.21.15.131:0/3577861003 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f52ac0a11b0 msgr2=0x7f52ac0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:01.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.639+0000 7f52cd600700 1 --2- 172.21.15.131:0/3577861003 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f52ac0a11b0 0x7f52ac0a3670 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f52c8101fd0 tx=0x7f52b8009310 comp rx=0 tx=0).stop 2024-05-08T22:07:01.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.639+0000 7f52cd600700 1 -- 172.21.15.131:0/3577861003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52c8102b10 msgr2=0x7f52c819e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:01.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.639+0000 7f52cd600700 1 --2- 172.21.15.131:0/3577861003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52c8102b10 0x7f52c819e490 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f52c000ef30 tx=0x7f52c000c3a0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.640+0000 7f52cd600700 1 -- 172.21.15.131:0/3577861003 shutdown_connections 2024-05-08T22:07:01.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.640+0000 7f52cd600700 1 --2- 172.21.15.131:0/3577861003 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f52ac0a11b0 0x7f52ac0a3670 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.640+0000 7f52cd600700 1 --2- 172.21.15.131:0/3577861003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52c8102b10 0x7f52c819e490 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.640+0000 7f52cd600700 1 --2- 172.21.15.131:0/3577861003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52c8100f80 0x7f52c8196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.640+0000 7f52cd600700 1 --2- 172.21.15.131:0/3577861003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c8102170 0x7f52c81974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.640+0000 7f52cd600700 1 -- 172.21.15.131:0/3577861003 >> 172.21.15.131:0/3577861003 conn(0x7f52c80fc4f0 msgr2=0x7f52c8105590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:01.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.640+0000 7f52cd600700 1 -- 172.21.15.131:0/3577861003 shutdown_connections 2024-05-08T22:07:01.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.640+0000 7f52cd600700 1 -- 172.21.15.131:0/3577861003 wait complete. 2024-05-08T22:07:01.660 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:07:01.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.858+0000 7fc46b200700 1 -- 172.21.15.131:0/1191250247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc46c100db0 msgr2=0x7fc46c1011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:01.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.858+0000 7fc46b200700 1 --2- 172.21.15.131:0/1191250247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc46c100db0 0x7fc46c1011d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0099c0 tx=0x7fc45c009cd0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.858+0000 7fc46b200700 1 -- 172.21.15.131:0/1191250247 shutdown_connections 2024-05-08T22:07:01.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.858+0000 7fc46b200700 1 --2- 172.21.15.131:0/1191250247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc46c102940 0x7fc46c1091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.858+0000 7fc46b200700 1 --2- 172.21.15.131:0/1191250247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc46c100db0 0x7fc46c1011d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.858+0000 7fc46b200700 1 --2- 172.21.15.131:0/1191250247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc46c101fa0 0x7fc46c102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.858+0000 7fc46b200700 1 -- 172.21.15.131:0/1191250247 >> 172.21.15.131:0/1191250247 conn(0x7fc46c0fc2e0 msgr2=0x7fc46c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:01.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.858+0000 7fc46b200700 1 -- 172.21.15.131:0/1191250247 shutdown_connections 2024-05-08T22:07:01.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.858+0000 7fc46b200700 1 -- 172.21.15.131:0/1191250247 wait complete. 2024-05-08T22:07:01.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.859+0000 7fc46b200700 1 Processor -- start 2024-05-08T22:07:01.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.859+0000 7fc46b200700 1 -- start start 2024-05-08T22:07:01.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.860+0000 7fc46b200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc46c100db0 0x7fc46c198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.860+0000 7fc46b200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc46c101fa0 0x7fc46c1994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.860+0000 7fc46b200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc46c102940 0x7fc46c1a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.860+0000 7fc46b200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c1a0a90 con 0x7fc46c101fa0 2024-05-08T22:07:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.860+0000 7fc46b200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c1a0bd0 con 0x7fc46c100db0 2024-05-08T22:07:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.860+0000 7fc46b200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c1a0d60 con 0x7fc46c102940 2024-05-08T22:07:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.860+0000 7fc469400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc46c101fa0 0x7fc46c1994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.860+0000 7fc469400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc46c101fa0 0x7fc46c1994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56208/0 (socket says 172.21.15.131:56208) 2024-05-08T22:07:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.860+0000 7fc469400700 1 -- 172.21.15.131:0/2980040232 learned_addr learned my addr 172.21.15.131:0/2980040232 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.860+0000 7fc469e00700 1 --2- 172.21.15.131:0/2980040232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc46c100db0 0x7fc46c198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.860+0000 7fc46a800700 1 --2- 172.21.15.131:0/2980040232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc46c102940 0x7fc46c1a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.860+0000 7fc469400700 1 -- 172.21.15.131:0/2980040232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc46c102940 msgr2=0x7fc46c1a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.860+0000 7fc469400700 1 --2- 172.21.15.131:0/2980040232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc46c102940 0x7fc46c1a04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.860+0000 7fc469400700 1 -- 172.21.15.131:0/2980040232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc46c100db0 msgr2=0x7fc46c198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.860+0000 7fc469400700 1 --2- 172.21.15.131:0/2980040232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc46c100db0 0x7fc46c198f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.860+0000 7fc469400700 1 -- 172.21.15.131:0/2980040232 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc464009c40 con 0x7fc46c101fa0 2024-05-08T22:07:01.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.861+0000 7fc469400700 1 --2- 172.21.15.131:0/2980040232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc46c101fa0 0x7fc46c1994a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc45800e9c0 tx=0x7fc45800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:01.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.861+0000 7fc462a00700 1 -- 172.21.15.131:0/2980040232 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4580097b0 con 0x7fc46c101fa0 2024-05-08T22:07:01.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.861+0000 7fc462a00700 1 -- 172.21.15.131:0/2980040232 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc458004cf0 con 0x7fc46c101fa0 2024-05-08T22:07:01.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.861+0000 7fc462a00700 1 -- 172.21.15.131:0/2980040232 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc458005600 con 0x7fc46c101fa0 2024-05-08T22:07:01.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.861+0000 7fc46b200700 1 -- 172.21.15.131:0/2980040232 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc45c0095f0 con 0x7fc46c101fa0 2024-05-08T22:07:01.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.861+0000 7fc46b200700 1 -- 172.21.15.131:0/2980040232 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc46c1a12e0 con 0x7fc46c101fa0 2024-05-08T22:07:01.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.862+0000 7fc46b200700 1 -- 172.21.15.131:0/2980040232 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc46c067ba0 con 0x7fc46c101fa0 2024-05-08T22:07:01.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.864+0000 7fc462a00700 1 -- 172.21.15.131:0/2980040232 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fc45800cc60 con 0x7fc46c101fa0 2024-05-08T22:07:01.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.865+0000 7fc462a00700 1 --2- 172.21.15.131:0/2980040232 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc4500a1230 0x7fc4500a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:01.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.865+0000 7fc462a00700 1 -- 172.21.15.131:0/2980040232 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fc458014070 con 0x7fc46c101fa0 2024-05-08T22:07:01.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.865+0000 7fc469e00700 1 --2- 172.21.15.131:0/2980040232 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc4500a1230 0x7fc4500a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:01.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.866+0000 7fc469e00700 1 --2- 172.21.15.131:0/2980040232 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc4500a1230 0x7fc4500a36f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009990 tx=0x7fc45c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:01.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:01.869+0000 7fc462a00700 1 -- 172.21.15.131:0/2980040232 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc4580962d0 con 0x7fc46c101fa0 2024-05-08T22:07:02.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.078+0000 7fc46b200700 1 -- 172.21.15.131:0/2980040232 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc46c10c880 con 0x7fc46c101fa0 2024-05-08T22:07:02.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.079+0000 7fc462a00700 1 -- 172.21.15.131:0/2980040232 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc4580998f0 con 0x7fc46c101fa0 2024-05-08T22:07:02.080 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-08T22:07:02.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.081+0000 7fc46b200700 1 -- 172.21.15.131:0/2980040232 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc4500a1230 msgr2=0x7fc4500a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:02.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.081+0000 7fc46b200700 1 --2- 172.21.15.131:0/2980040232 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc4500a1230 0x7fc4500a36f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009990 tx=0x7fc45c000bc0 comp rx=0 tx=0).stop 2024-05-08T22:07:02.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.081+0000 7fc46b200700 1 -- 172.21.15.131:0/2980040232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc46c101fa0 msgr2=0x7fc46c1994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:02.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.081+0000 7fc46b200700 1 --2- 172.21.15.131:0/2980040232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc46c101fa0 0x7fc46c1994a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc45800e9c0 tx=0x7fc45800ecd0 comp rx=0 tx=0).stop 2024-05-08T22:07:02.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.082+0000 7fc46b200700 1 -- 172.21.15.131:0/2980040232 shutdown_connections 2024-05-08T22:07:02.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.082+0000 7fc46b200700 1 --2- 172.21.15.131:0/2980040232 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc4500a1230 0x7fc4500a36f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:02.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.082+0000 7fc46b200700 1 --2- 172.21.15.131:0/2980040232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc46c102940 0x7fc46c1a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:02.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.082+0000 7fc46b200700 1 --2- 172.21.15.131:0/2980040232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc46c100db0 0x7fc46c198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:02.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.082+0000 7fc46b200700 1 --2- 172.21.15.131:0/2980040232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc46c101fa0 0x7fc46c1994a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:02.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.082+0000 7fc46b200700 1 -- 172.21.15.131:0/2980040232 >> 172.21.15.131:0/2980040232 conn(0x7fc46c0fc2e0 msgr2=0x7fc46c0731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:02.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.082+0000 7fc46b200700 1 -- 172.21.15.131:0/2980040232 shutdown_connections 2024-05-08T22:07:02.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.082+0000 7fc46b200700 1 -- 172.21.15.131:0/2980040232 wait complete. 2024-05-08T22:07:02.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:01 smithi132 ceph-mon[78180]: pgmap v297: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T22:07:02.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:01 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2620891254' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:07:02.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:01 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3577861003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:07:02.099 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:01 smithi131 ceph-mon[37260]: pgmap v297: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T22:07:02.099 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:01 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2620891254' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:07:02.100 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:01 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3577861003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:07:02.102 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:07:02.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:01 smithi157 ceph-mon[77746]: pgmap v297: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T22:07:02.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:01 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2620891254' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:07:02.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:01 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3577861003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:07:02.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.293+0000 7f682f200700 1 -- 172.21.15.131:0/2169154282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68301041e0 msgr2=0x7f68301065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:02.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.293+0000 7f682f200700 1 --2- 172.21.15.131:0/2169154282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68301041e0 0x7f68301065d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f681c009990 tx=0x7f681c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:02.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.293+0000 7f682f200700 1 -- 172.21.15.131:0/2169154282 shutdown_connections 2024-05-08T22:07:02.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.293+0000 7f682f200700 1 --2- 172.21.15.131:0/2169154282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6830106b10 0x7f6830108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:02.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.293+0000 7f682f200700 1 --2- 172.21.15.131:0/2169154282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68301018b0 0x7f6830103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:02.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.293+0000 7f682f200700 1 --2- 172.21.15.131:0/2169154282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68301041e0 0x7f68301065d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:02.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.293+0000 7f682f200700 1 -- 172.21.15.131:0/2169154282 >> 172.21.15.131:0/2169154282 conn(0x7f68300fb2c0 msgr2=0x7f68300fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:02.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.293+0000 7f682f200700 1 -- 172.21.15.131:0/2169154282 shutdown_connections 2024-05-08T22:07:02.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.293+0000 7f682f200700 1 -- 172.21.15.131:0/2169154282 wait complete. 2024-05-08T22:07:02.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.294+0000 7f682f200700 1 Processor -- start 2024-05-08T22:07:02.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.294+0000 7f682f200700 1 -- start start 2024-05-08T22:07:02.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.295+0000 7f682f200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68301018b0 0x7f68301011d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:02.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.295+0000 7f682f200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68301041e0 0x7f68300ff860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:02.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.295+0000 7f682f200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6830106b10 0x7f68300ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:02.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.295+0000 7f682f200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6830100540 con 0x7f6830106b10 2024-05-08T22:07:02.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.295+0000 7f682f200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68301006b0 con 0x7f68301018b0 2024-05-08T22:07:02.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.295+0000 7f682f200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68301a0de0 con 0x7f68301041e0 2024-05-08T22:07:02.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.295+0000 7f682d400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68301041e0 0x7f68300ff860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:02.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.295+0000 7f682e800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6830106b10 0x7f68300ffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:02.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.295+0000 7f682de00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68301018b0 0x7f68301011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:02.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.295+0000 7f682d400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68301041e0 0x7f68300ff860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:33986/0 (socket says 172.21.15.131:33986) 2024-05-08T22:07:02.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.295+0000 7f682d400700 1 -- 172.21.15.131:0/1733590822 learned_addr learned my addr 172.21.15.131:0/1733590822 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:02.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.295+0000 7f682de00700 1 -- 172.21.15.131:0/1733590822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68301041e0 msgr2=0x7f68300ff860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:02.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.295+0000 7f682de00700 1 --2- 172.21.15.131:0/1733590822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68301041e0 0x7f68300ff860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:02.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.295+0000 7f682de00700 1 -- 172.21.15.131:0/1733590822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6830106b10 msgr2=0x7f68300ffda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:02.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.295+0000 7f682de00700 1 --2- 172.21.15.131:0/1733590822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6830106b10 0x7f68300ffda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:02.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.295+0000 7f682de00700 1 -- 172.21.15.131:0/1733590822 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681c0095f0 con 0x7f68301018b0 2024-05-08T22:07:02.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.295+0000 7f682e800700 1 --2- 172.21.15.131:0/1733590822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6830106b10 0x7f68300ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:02.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.296+0000 7f682de00700 1 --2- 172.21.15.131:0/1733590822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68301018b0 0x7f68301011d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f682000e9f0 tx=0x7f682000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:02.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.296+0000 7f6826a00700 1 -- 172.21.15.131:0/1733590822 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682000c960 con 0x7f68301018b0 2024-05-08T22:07:02.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.296+0000 7f682f200700 1 -- 172.21.15.131:0/1733590822 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68301a1070 con 0x7f68301018b0 2024-05-08T22:07:02.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.296+0000 7f682f200700 1 -- 172.21.15.131:0/1733590822 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68301a15d0 con 0x7f68301018b0 2024-05-08T22:07:02.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.296+0000 7f6826a00700 1 -- 172.21.15.131:0/1733590822 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6820010430 con 0x7f68301018b0 2024-05-08T22:07:02.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.296+0000 7f6826a00700 1 -- 172.21.15.131:0/1733590822 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6820018680 con 0x7f68301018b0 2024-05-08T22:07:02.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.297+0000 7f682f200700 1 -- 172.21.15.131:0/1733590822 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6810005320 con 0x7f68301018b0 2024-05-08T22:07:02.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.299+0000 7f6826a00700 1 -- 172.21.15.131:0/1733590822 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f6820010830 con 0x7f68301018b0 2024-05-08T22:07:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.300+0000 7f6826a00700 1 --2- 172.21.15.131:0/1733590822 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f68140a1230 0x7f68140a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.301+0000 7f6826a00700 1 -- 172.21.15.131:0/1733590822 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f6820014070 con 0x7f68301018b0 2024-05-08T22:07:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.301+0000 7f682d400700 1 --2- 172.21.15.131:0/1733590822 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f68140a1230 0x7f68140a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:02.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.301+0000 7f682d400700 1 --2- 172.21.15.131:0/1733590822 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f68140a1230 0x7f68140a36f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f681c00b2e0 tx=0x7f681c00b750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:02.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.305+0000 7f6826a00700 1 -- 172.21.15.131:0/1733590822 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6820052d60 con 0x7f68301018b0 2024-05-08T22:07:02.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.560+0000 7f682f200700 1 -- 172.21.15.131:0/1733590822 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6810005190 con 0x7f68301018b0 2024-05-08T22:07:02.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.561+0000 7f6826a00700 1 -- 172.21.15.131:0/1733590822 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10889 (secure 0 0 0) 0x7f6820096070 con 0x7f68301018b0 2024-05-08T22:07:02.563 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:07:02.563 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:05:14.034908+0000","modified":"2024-05-08T22:05:39.494566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:07:02.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.564+0000 7f682f200700 1 -- 172.21.15.131:0/1733590822 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f68140a1230 msgr2=0x7f68140a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:02.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.564+0000 7f682f200700 1 --2- 172.21.15.131:0/1733590822 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f68140a1230 0x7f68140a36f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f681c00b2e0 tx=0x7f681c00b750 comp rx=0 tx=0).stop 2024-05-08T22:07:02.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.564+0000 7f682f200700 1 -- 172.21.15.131:0/1733590822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68301018b0 msgr2=0x7f68301011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:02.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.564+0000 7f682f200700 1 --2- 172.21.15.131:0/1733590822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68301018b0 0x7f68301011d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f682000e9f0 tx=0x7f682000ed00 comp rx=0 tx=0).stop 2024-05-08T22:07:02.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.564+0000 7f682f200700 1 -- 172.21.15.131:0/1733590822 shutdown_connections 2024-05-08T22:07:02.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.564+0000 7f682f200700 1 --2- 172.21.15.131:0/1733590822 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f68140a1230 0x7f68140a36f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:02.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.564+0000 7f682f200700 1 --2- 172.21.15.131:0/1733590822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6830106b10 0x7f68300ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:02.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.564+0000 7f682f200700 1 --2- 172.21.15.131:0/1733590822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68301018b0 0x7f68301011d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:02.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.564+0000 7f682f200700 1 --2- 172.21.15.131:0/1733590822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68301041e0 0x7f68300ff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:02.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.564+0000 7f682f200700 1 -- 172.21.15.131:0/1733590822 >> 172.21.15.131:0/1733590822 conn(0x7f68300fb2c0 msgr2=0x7f68301022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:02.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.564+0000 7f682f200700 1 -- 172.21.15.131:0/1733590822 shutdown_connections 2024-05-08T22:07:02.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.564+0000 7f682f200700 1 -- 172.21.15.131:0/1733590822 wait complete. 2024-05-08T22:07:02.567 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 23 2024-05-08T22:07:02.585 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:07:02.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.778+0000 7f4b59000700 1 -- 172.21.15.131:0/186182708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b54101fd0 msgr2=0x7f4b54102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:02.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.778+0000 7f4b59000700 1 --2- 172.21.15.131:0/186182708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b54101fd0 0x7f4b54102450 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009990 tx=0x7f4b40009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:02.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.778+0000 7f4b59000700 1 -- 172.21.15.131:0/186182708 shutdown_connections 2024-05-08T22:07:02.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.778+0000 7f4b59000700 1 --2- 172.21.15.131:0/186182708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b54102990 0x7f4b541092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:02.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.778+0000 7f4b59000700 1 --2- 172.21.15.131:0/186182708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b54100e80 0x7f4b541012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:02.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.778+0000 7f4b59000700 1 --2- 172.21.15.131:0/186182708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b54101fd0 0x7f4b54102450 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:02.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.778+0000 7f4b59000700 1 -- 172.21.15.131:0/186182708 >> 172.21.15.131:0/186182708 conn(0x7f4b540fc410 msgr2=0x7f4b540fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:02.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.778+0000 7f4b59000700 1 -- 172.21.15.131:0/186182708 shutdown_connections 2024-05-08T22:07:02.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.778+0000 7f4b59000700 1 -- 172.21.15.131:0/186182708 wait complete. 2024-05-08T22:07:02.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.779+0000 7f4b59000700 1 Processor -- start 2024-05-08T22:07:02.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.779+0000 7f4b59000700 1 -- start start 2024-05-08T22:07:02.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.779+0000 7f4b59000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b54100e80 0x7f4b54074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:02.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.779+0000 7f4b59000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b54101fd0 0x7f4b54073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:02.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.779+0000 7f4b59000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b54102990 0x7f4b540738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:02.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.779+0000 7f4b59000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b54074070 con 0x7f4b54101fd0 2024-05-08T22:07:02.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.779+0000 7f4b59000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b540741e0 con 0x7f4b54102990 2024-05-08T22:07:02.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.779+0000 7f4b59000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b541a0e30 con 0x7f4b54100e80 2024-05-08T22:07:02.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.780+0000 7f4b51400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b54101fd0 0x7f4b54073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:02.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.780+0000 7f4b52800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b54102990 0x7f4b540738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:02.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.780+0000 7f4b51400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b54101fd0 0x7f4b54073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56238/0 (socket says 172.21.15.131:56238) 2024-05-08T22:07:02.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.780+0000 7f4b51400700 1 -- 172.21.15.131:0/1093993824 learned_addr learned my addr 172.21.15.131:0/1093993824 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:02.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.780+0000 7f4b51e00700 1 --2- 172.21.15.131:0/1093993824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b54100e80 0x7f4b54074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:02.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.780+0000 7f4b52800700 1 -- 172.21.15.131:0/1093993824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b54100e80 msgr2=0x7f4b54074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:02.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.780+0000 7f4b52800700 1 --2- 172.21.15.131:0/1093993824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b54100e80 0x7f4b54074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:02.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.780+0000 7f4b52800700 1 -- 172.21.15.131:0/1093993824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b54101fd0 msgr2=0x7f4b54073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:02.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.780+0000 7f4b52800700 1 --2- 172.21.15.131:0/1093993824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b54101fd0 0x7f4b54073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:02.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.780+0000 7f4b52800700 1 -- 172.21.15.131:0/1093993824 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4c009c40 con 0x7f4b54102990 2024-05-08T22:07:02.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.780+0000 7f4b51400700 1 --2- 172.21.15.131:0/1093993824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b54101fd0 0x7f4b54073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:02.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.780+0000 7f4b52800700 1 --2- 172.21.15.131:0/1093993824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b54102990 0x7f4b540738d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00efb0 tx=0x7f4b4c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:02.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.781+0000 7f4b4aa00700 1 -- 172.21.15.131:0/1093993824 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c003a00 con 0x7f4b54102990 2024-05-08T22:07:02.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.781+0000 7f4b4aa00700 1 -- 172.21.15.131:0/1093993824 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b4c00f040 con 0x7f4b54102990 2024-05-08T22:07:02.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.781+0000 7f4b59000700 1 -- 172.21.15.131:0/1093993824 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b400095f0 con 0x7f4b54102990 2024-05-08T22:07:02.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.781+0000 7f4b4aa00700 1 -- 172.21.15.131:0/1093993824 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c004540 con 0x7f4b54102990 2024-05-08T22:07:02.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.781+0000 7f4b59000700 1 -- 172.21.15.131:0/1093993824 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b541a1440 con 0x7f4b54102990 2024-05-08T22:07:02.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.782+0000 7f4b59000700 1 -- 172.21.15.131:0/1093993824 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b541082c0 con 0x7f4b54102990 2024-05-08T22:07:02.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.784+0000 7f4b4aa00700 1 -- 172.21.15.131:0/1093993824 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f4b4c0046a0 con 0x7f4b54102990 2024-05-08T22:07:02.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.785+0000 7f4b4aa00700 1 --2- 172.21.15.131:0/1093993824 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f4b380a11b0 0x7f4b380a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:02.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.785+0000 7f4b4aa00700 1 -- 172.21.15.131:0/1093993824 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f4b4c0ccbc0 con 0x7f4b54102990 2024-05-08T22:07:02.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.785+0000 7f4b51e00700 1 --2- 172.21.15.131:0/1093993824 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f4b380a11b0 0x7f4b380a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:02.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.786+0000 7f4b51e00700 1 --2- 172.21.15.131:0/1093993824 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f4b380a11b0 0x7f4b380a3670 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009c60 tx=0x7f4b44009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:02.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:02.788+0000 7f4b4aa00700 1 -- 172.21.15.131:0/1093993824 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4b4c095f00 con 0x7f4b54102990 2024-05-08T22:07:02.833 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:02 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2980040232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:07:03.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.000+0000 7f4b59000700 1 -- 172.21.15.131:0/1093993824 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b54067c10 con 0x7f4b54102990 2024-05-08T22:07:03.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.002+0000 7f4b4aa00700 1 -- 172.21.15.131:0/1093993824 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24721 (secure 0 0 0) 0x7f4b4c099520 con 0x7f4b54102990 2024-05-08T22:07:03.003 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:07:03.003 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":87,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:06:41.971918+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:05:12.469858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:05:13.484446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:05:17.016186+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:07:03.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.004+0000 7f4b59000700 1 -- 172.21.15.131:0/1093993824 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f4b380a11b0 msgr2=0x7f4b380a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:03.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.004+0000 7f4b59000700 1 --2- 172.21.15.131:0/1093993824 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f4b380a11b0 0x7f4b380a3670 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009c60 tx=0x7f4b44009260 comp rx=0 tx=0).stop 2024-05-08T22:07:03.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.004+0000 7f4b59000700 1 -- 172.21.15.131:0/1093993824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b54102990 msgr2=0x7f4b540738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:03.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.004+0000 7f4b59000700 1 --2- 172.21.15.131:0/1093993824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b54102990 0x7f4b540738d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00efb0 tx=0x7f4b4c00c3a0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.004+0000 7f4b59000700 1 -- 172.21.15.131:0/1093993824 shutdown_connections 2024-05-08T22:07:03.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.004+0000 7f4b59000700 1 --2- 172.21.15.131:0/1093993824 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f4b380a11b0 0x7f4b380a3670 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.004+0000 7f4b59000700 1 --2- 172.21.15.131:0/1093993824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b54102990 0x7f4b540738d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.004+0000 7f4b59000700 1 --2- 172.21.15.131:0/1093993824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b54100e80 0x7f4b54074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.004+0000 7f4b59000700 1 --2- 172.21.15.131:0/1093993824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b54101fd0 0x7f4b54073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.004+0000 7f4b59000700 1 -- 172.21.15.131:0/1093993824 >> 172.21.15.131:0/1093993824 conn(0x7f4b540fc410 msgr2=0x7f4b54105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:03.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.005+0000 7f4b59000700 1 -- 172.21.15.131:0/1093993824 shutdown_connections 2024-05-08T22:07:03.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.005+0000 7f4b59000700 1 -- 172.21.15.131:0/1093993824 wait complete. 2024-05-08T22:07:03.024 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:01:41.271078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:05:12.469858+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:05:13.484446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:05:17.016186+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:07:03.025 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:07:03.084 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:02 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1733590822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:07:03.089 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:02 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2980040232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:07:03.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:02 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1733590822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:07:03.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.221+0000 7fe6c9000700 1 -- 172.21.15.131:0/302003819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6c4102b10 msgr2=0x7fe6c41093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:03.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.221+0000 7fe6c9000700 1 --2- 172.21.15.131:0/302003819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6c4102b10 0x7fe6c41093c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00b0d0 tx=0x7fe6bc00b3e0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.221+0000 7fe6c9000700 1 -- 172.21.15.131:0/302003819 shutdown_connections 2024-05-08T22:07:03.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.221+0000 7fe6c9000700 1 --2- 172.21.15.131:0/302003819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6c4102b10 0x7fe6c41093c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.221+0000 7fe6c9000700 1 --2- 172.21.15.131:0/302003819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6c4100f80 0x7fe6c41013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.221+0000 7fe6c9000700 1 --2- 172.21.15.131:0/302003819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6c4102170 0x7fe6c41025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.221+0000 7fe6c9000700 1 -- 172.21.15.131:0/302003819 >> 172.21.15.131:0/302003819 conn(0x7fe6c40fc4f0 msgr2=0x7fe6c40fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:03.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.221+0000 7fe6c9000700 1 -- 172.21.15.131:0/302003819 shutdown_connections 2024-05-08T22:07:03.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.221+0000 7fe6c9000700 1 -- 172.21.15.131:0/302003819 wait complete. 2024-05-08T22:07:03.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.222+0000 7fe6c9000700 1 Processor -- start 2024-05-08T22:07:03.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.222+0000 7fe6c9000700 1 -- start start 2024-05-08T22:07:03.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.223+0000 7fe6c9000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6c4100f80 0x7fe6c4199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:03.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.223+0000 7fe6c9000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6c4102170 0x7fe6c4199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:03.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.223+0000 7fe6c9000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6c4102b10 0x7fe6c41a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:03.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.223+0000 7fe6c9000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c41a0d70 con 0x7fe6c4102b10 2024-05-08T22:07:03.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.223+0000 7fe6c9000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c41a0eb0 con 0x7fe6c4100f80 2024-05-08T22:07:03.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.223+0000 7fe6c9000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c41a1040 con 0x7fe6c4102170 2024-05-08T22:07:03.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.223+0000 7fe6c1400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6c4102170 0x7fe6c4199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:03.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.223+0000 7fe6c1400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6c4102170 0x7fe6c4199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34020/0 (socket says 172.21.15.131:34020) 2024-05-08T22:07:03.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.223+0000 7fe6c2800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6c4102b10 0x7fe6c41a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:03.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.223+0000 7fe6c1400700 1 -- 172.21.15.131:0/3110447218 learned_addr learned my addr 172.21.15.131:0/3110447218 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:03.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.223+0000 7fe6c1e00700 1 --2- 172.21.15.131:0/3110447218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6c4100f80 0x7fe6c4199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:03.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.223+0000 7fe6c1400700 1 -- 172.21.15.131:0/3110447218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6c4100f80 msgr2=0x7fe6c4199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:03.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.223+0000 7fe6c1400700 1 --2- 172.21.15.131:0/3110447218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6c4100f80 0x7fe6c4199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.223+0000 7fe6c1400700 1 -- 172.21.15.131:0/3110447218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6c4102b10 msgr2=0x7fe6c41a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:03.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.223+0000 7fe6c1400700 1 --2- 172.21.15.131:0/3110447218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6c4102b10 0x7fe6c41a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.223+0000 7fe6c1400700 1 -- 172.21.15.131:0/3110447218 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc009d00 con 0x7fe6c4102170 2024-05-08T22:07:03.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.223+0000 7fe6c2800700 1 --2- 172.21.15.131:0/3110447218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6c4102b10 0x7fe6c41a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:03.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.224+0000 7fe6c1400700 1 --2- 172.21.15.131:0/3110447218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6c4102170 0x7fe6c4199780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000e9c0 tx=0x7fe6b000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:03.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.224+0000 7fe6baa00700 1 -- 172.21.15.131:0/3110447218 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b00097b0 con 0x7fe6c4102170 2024-05-08T22:07:03.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.224+0000 7fe6c9000700 1 -- 172.21.15.131:0/3110447218 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c41a1320 con 0x7fe6c4102170 2024-05-08T22:07:03.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.224+0000 7fe6baa00700 1 -- 172.21.15.131:0/3110447218 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6b000ee60 con 0x7fe6c4102170 2024-05-08T22:07:03.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.224+0000 7fe6baa00700 1 -- 172.21.15.131:0/3110447218 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b00054f0 con 0x7fe6c4102170 2024-05-08T22:07:03.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.224+0000 7fe6c9000700 1 -- 172.21.15.131:0/3110447218 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c41a1830 con 0x7fe6c4102170 2024-05-08T22:07:03.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.225+0000 7fe6c9000700 1 -- 172.21.15.131:0/3110447218 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c4067cf0 con 0x7fe6c4102170 2024-05-08T22:07:03.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.227+0000 7fe6baa00700 1 -- 172.21.15.131:0/3110447218 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fe6b0004750 con 0x7fe6c4102170 2024-05-08T22:07:03.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.228+0000 7fe6baa00700 1 --2- 172.21.15.131:0/3110447218 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe6a80a11b0 0x7fe6a80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:03.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.228+0000 7fe6baa00700 1 -- 172.21.15.131:0/3110447218 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fe6b0014070 con 0x7fe6c4102170 2024-05-08T22:07:03.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.228+0000 7fe6c1e00700 1 --2- 172.21.15.131:0/3110447218 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe6a80a11b0 0x7fe6a80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:03.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.229+0000 7fe6c1e00700 1 --2- 172.21.15.131:0/3110447218 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe6a80a11b0 0x7fe6a80a3670 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4101fd0 tx=0x7fe6b4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:03.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.230+0000 7fe6baa00700 1 -- 172.21.15.131:0/3110447218 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe6b00964a0 con 0x7fe6c4102170 2024-05-08T22:07:03.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:02 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2980040232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:07:03.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:02 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1733590822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:07:03.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.435+0000 7fe6c9000700 1 -- 172.21.15.131:0/3110447218 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe6c410c9a0 con 0x7fe6c4102170 2024-05-08T22:07:03.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.436+0000 7fe6baa00700 1 -- 172.21.15.131:0/3110447218 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe6b0099ac0 con 0x7fe6c4102170 2024-05-08T22:07:03.436 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-08T22:07:03.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.438+0000 7fe6c9000700 1 -- 172.21.15.131:0/3110447218 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe6a80a11b0 msgr2=0x7fe6a80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:03.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.438+0000 7fe6c9000700 1 --2- 172.21.15.131:0/3110447218 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe6a80a11b0 0x7fe6a80a3670 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4101fd0 tx=0x7fe6b4009260 comp rx=0 tx=0).stop 2024-05-08T22:07:03.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.438+0000 7fe6c9000700 1 -- 172.21.15.131:0/3110447218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6c4102170 msgr2=0x7fe6c4199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:03.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.438+0000 7fe6c9000700 1 --2- 172.21.15.131:0/3110447218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6c4102170 0x7fe6c4199780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000e9c0 tx=0x7fe6b000ecd0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.438+0000 7fe6c9000700 1 -- 172.21.15.131:0/3110447218 shutdown_connections 2024-05-08T22:07:03.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.438+0000 7fe6c9000700 1 --2- 172.21.15.131:0/3110447218 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fe6a80a11b0 0x7fe6a80a3670 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.438+0000 7fe6c9000700 1 --2- 172.21.15.131:0/3110447218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6c4102b10 0x7fe6c41a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.438+0000 7fe6c9000700 1 --2- 172.21.15.131:0/3110447218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6c4100f80 0x7fe6c4199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.438+0000 7fe6c9000700 1 --2- 172.21.15.131:0/3110447218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6c4102170 0x7fe6c4199780 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.438+0000 7fe6c9000700 1 -- 172.21.15.131:0/3110447218 >> 172.21.15.131:0/3110447218 conn(0x7fe6c40fc4f0 msgr2=0x7fe6c40731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:03.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.438+0000 7fe6c9000700 1 -- 172.21.15.131:0/3110447218 shutdown_connections 2024-05-08T22:07:03.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.438+0000 7fe6c9000700 1 -- 172.21.15.131:0/3110447218 wait complete. 2024-05-08T22:07:03.459 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:07:03.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.655+0000 7f7a26a00700 1 -- 172.21.15.131:0/159497851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a20102940 msgr2=0x7f7a201091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:03.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.655+0000 7f7a26a00700 1 --2- 172.21.15.131:0/159497851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a20102940 0x7f7a201091f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800b0d0 tx=0x7f7a1800b3e0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.655+0000 7f7a26a00700 1 -- 172.21.15.131:0/159497851 shutdown_connections 2024-05-08T22:07:03.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.655+0000 7f7a26a00700 1 --2- 172.21.15.131:0/159497851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a20102940 0x7f7a201091f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.655+0000 7f7a26a00700 1 --2- 172.21.15.131:0/159497851 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a20100db0 0x7f7a201011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.655+0000 7f7a26a00700 1 --2- 172.21.15.131:0/159497851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a20101fa0 0x7f7a20102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.655+0000 7f7a26a00700 1 -- 172.21.15.131:0/159497851 >> 172.21.15.131:0/159497851 conn(0x7f7a200fc2e0 msgr2=0x7f7a200fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:03.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.655+0000 7f7a26a00700 1 -- 172.21.15.131:0/159497851 shutdown_connections 2024-05-08T22:07:03.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.655+0000 7f7a26a00700 1 -- 172.21.15.131:0/159497851 wait complete. 2024-05-08T22:07:03.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.656+0000 7f7a26a00700 1 Processor -- start 2024-05-08T22:07:03.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.656+0000 7f7a26a00700 1 -- start start 2024-05-08T22:07:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.657+0000 7f7a26a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a20100db0 0x7f7a20074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.657+0000 7f7a26a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a20101fa0 0x7f7a20073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.657+0000 7f7a26a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a20102940 0x7f7a200738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.657+0000 7f7a26a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a20074070 con 0x7f7a20101fa0 2024-05-08T22:07:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.657+0000 7f7a26a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a200741e0 con 0x7f7a20100db0 2024-05-08T22:07:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.657+0000 7f7a26a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a201a0ed0 con 0x7f7a20102940 2024-05-08T22:07:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.657+0000 7f7a24c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a20101fa0 0x7f7a20073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.657+0000 7f7a24c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a20101fa0 0x7f7a20073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56284/0 (socket says 172.21.15.131:56284) 2024-05-08T22:07:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.657+0000 7f7a25600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a20100db0 0x7f7a20074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.657+0000 7f7a24c00700 1 -- 172.21.15.131:0/1203961946 learned_addr learned my addr 172.21.15.131:0/1203961946 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.657+0000 7f7a26000700 1 --2- 172.21.15.131:0/1203961946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a20102940 0x7f7a200738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:03.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.657+0000 7f7a25600700 1 -- 172.21.15.131:0/1203961946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a20102940 msgr2=0x7f7a200738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:03.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.657+0000 7f7a25600700 1 --2- 172.21.15.131:0/1203961946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a20102940 0x7f7a200738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.657+0000 7f7a25600700 1 -- 172.21.15.131:0/1203961946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a20101fa0 msgr2=0x7f7a20073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:03.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.657+0000 7f7a25600700 1 --2- 172.21.15.131:0/1203961946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a20101fa0 0x7f7a20073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.657+0000 7f7a25600700 1 -- 172.21.15.131:0/1203961946 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a18009d00 con 0x7f7a20100db0 2024-05-08T22:07:03.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.658+0000 7f7a25600700 1 --2- 172.21.15.131:0/1203961946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a20100db0 0x7f7a20074d00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7a1400e9c0 tx=0x7f7a1400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:03.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.658+0000 7f7a1e000700 1 -- 172.21.15.131:0/1203961946 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a14004220 con 0x7f7a20100db0 2024-05-08T22:07:03.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.658+0000 7f7a1e000700 1 -- 172.21.15.131:0/1203961946 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a14004380 con 0x7f7a20100db0 2024-05-08T22:07:03.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.658+0000 7f7a26a00700 1 -- 172.21.15.131:0/1203961946 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a201a1160 con 0x7f7a20100db0 2024-05-08T22:07:03.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.658+0000 7f7a1e000700 1 -- 172.21.15.131:0/1203961946 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a14003b70 con 0x7f7a20100db0 2024-05-08T22:07:03.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.658+0000 7f7a26a00700 1 -- 172.21.15.131:0/1203961946 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a201a16c0 con 0x7f7a20100db0 2024-05-08T22:07:03.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.659+0000 7f7a26a00700 1 -- 172.21.15.131:0/1203961946 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a20108160 con 0x7f7a20100db0 2024-05-08T22:07:03.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.661+0000 7f7a1e000700 1 -- 172.21.15.131:0/1203961946 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f7a14003cd0 con 0x7f7a20100db0 2024-05-08T22:07:03.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.662+0000 7f7a1e000700 1 --2- 172.21.15.131:0/1203961946 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7a080a1230 0x7f7a080a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:03.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.662+0000 7f7a1e000700 1 -- 172.21.15.131:0/1203961946 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f7a14014070 con 0x7f7a20100db0 2024-05-08T22:07:03.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.663+0000 7f7a24c00700 1 --2- 172.21.15.131:0/1203961946 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7a080a1230 0x7f7a080a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:03.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.663+0000 7f7a24c00700 1 --2- 172.21.15.131:0/1203961946 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7a080a1230 0x7f7a080a36f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009d60 tx=0x7f7a0c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:03.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.665+0000 7f7a1e000700 1 -- 172.21.15.131:0/1203961946 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7a14096380 con 0x7f7a20100db0 2024-05-08T22:07:03.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.874+0000 7f7a26a00700 1 -- 172.21.15.131:0/1203961946 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7a2010c640 con 0x7f7a20100db0 2024-05-08T22:07:03.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.875+0000 7f7a1e000700 1 -- 172.21.15.131:0/1203961946 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7a140999a0 con 0x7f7a20100db0 2024-05-08T22:07:03.876 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-08T22:07:03.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.877+0000 7f7a26a00700 1 -- 172.21.15.131:0/1203961946 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7a080a1230 msgr2=0x7f7a080a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:03.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.877+0000 7f7a26a00700 1 --2- 172.21.15.131:0/1203961946 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7a080a1230 0x7f7a080a36f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009d60 tx=0x7f7a0c009260 comp rx=0 tx=0).stop 2024-05-08T22:07:03.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.877+0000 7f7a26a00700 1 -- 172.21.15.131:0/1203961946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a20100db0 msgr2=0x7f7a20074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:03.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.877+0000 7f7a26a00700 1 --2- 172.21.15.131:0/1203961946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a20100db0 0x7f7a20074d00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7a1400e9c0 tx=0x7f7a1400ecd0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.877+0000 7f7a26a00700 1 -- 172.21.15.131:0/1203961946 shutdown_connections 2024-05-08T22:07:03.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.877+0000 7f7a26a00700 1 --2- 172.21.15.131:0/1203961946 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7a080a1230 0x7f7a080a36f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.877+0000 7f7a26a00700 1 --2- 172.21.15.131:0/1203961946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a20102940 0x7f7a200738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.877+0000 7f7a26a00700 1 --2- 172.21.15.131:0/1203961946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a20100db0 0x7f7a20074d00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.877+0000 7f7a26a00700 1 --2- 172.21.15.131:0/1203961946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a20101fa0 0x7f7a20073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:03.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.877+0000 7f7a26a00700 1 -- 172.21.15.131:0/1203961946 >> 172.21.15.131:0/1203961946 conn(0x7f7a200fc2e0 msgr2=0x7f7a201052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:03.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.878+0000 7f7a26a00700 1 -- 172.21.15.131:0/1203961946 shutdown_connections 2024-05-08T22:07:03.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:03.878+0000 7f7a26a00700 1 -- 172.21.15.131:0/1203961946 wait complete. 2024-05-08T22:07:03.898 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:07:04.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:03 smithi132 ceph-mon[78180]: pgmap v298: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T22:07:04.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:03 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1093993824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:07:04.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:03 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3110447218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:07:04.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.093+0000 7ff92d800700 1 -- 172.21.15.131:0/4173507802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff928100f20 msgr2=0x7ff928101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:04.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.093+0000 7ff92d800700 1 --2- 172.21.15.131:0/4173507802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff928100f20 0x7ff928101360 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff918009990 tx=0x7ff918009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.093+0000 7ff92d800700 1 -- 172.21.15.131:0/4173507802 shutdown_connections 2024-05-08T22:07:04.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.093+0000 7ff92d800700 1 --2- 172.21.15.131:0/4173507802 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff928102ab0 0x7ff9281093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.093+0000 7ff92d800700 1 --2- 172.21.15.131:0/4173507802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff928100f20 0x7ff928101360 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.093+0000 7ff92d800700 1 --2- 172.21.15.131:0/4173507802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9281020f0 0x7ff928102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.093+0000 7ff92d800700 1 -- 172.21.15.131:0/4173507802 >> 172.21.15.131:0/4173507802 conn(0x7ff9280fc500 msgr2=0x7ff9280fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:04.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.094+0000 7ff92d800700 1 -- 172.21.15.131:0/4173507802 shutdown_connections 2024-05-08T22:07:04.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.094+0000 7ff92d800700 1 -- 172.21.15.131:0/4173507802 wait complete. 2024-05-08T22:07:04.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.094+0000 7ff92d800700 1 Processor -- start 2024-05-08T22:07:04.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.095+0000 7ff92d800700 1 -- start start 2024-05-08T22:07:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.095+0000 7ff92d800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff928100f20 0x7ff928074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.095+0000 7ff92d800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9281020f0 0x7ff928073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.095+0000 7ff92d800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff928102ab0 0x7ff9280738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.095+0000 7ff92d800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff928074070 con 0x7ff928100f20 2024-05-08T22:07:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.095+0000 7ff92d800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9280741e0 con 0x7ff928102ab0 2024-05-08T22:07:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.095+0000 7ff92d800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9281a0ef0 con 0x7ff9281020f0 2024-05-08T22:07:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.095+0000 7ff926a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff928100f20 0x7ff928074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.095+0000 7ff926a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff928100f20 0x7ff928074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38846/0 (socket says 172.21.15.131:38846) 2024-05-08T22:07:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.095+0000 7ff926a00700 1 -- 172.21.15.131:0/219121495 learned_addr learned my addr 172.21.15.131:0/219121495 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.095+0000 7ff927400700 1 --2- 172.21.15.131:0/219121495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff928102ab0 0x7ff9280738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.095+0000 7ff926000700 1 --2- 172.21.15.131:0/219121495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9281020f0 0x7ff928073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.096+0000 7ff926a00700 1 -- 172.21.15.131:0/219121495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9281020f0 msgr2=0x7ff928073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.096+0000 7ff926a00700 1 --2- 172.21.15.131:0/219121495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9281020f0 0x7ff928073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.096+0000 7ff926a00700 1 -- 172.21.15.131:0/219121495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff928102ab0 msgr2=0x7ff9280738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:04.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.096+0000 7ff926a00700 1 --2- 172.21.15.131:0/219121495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff928102ab0 0x7ff9280738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.096+0000 7ff926a00700 1 -- 172.21.15.131:0/219121495 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff914009520 con 0x7ff928100f20 2024-05-08T22:07:04.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.096+0000 7ff926a00700 1 --2- 172.21.15.131:0/219121495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff928100f20 0x7ff928074d00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff918004710 tx=0x7ff918009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:04.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.096+0000 7ff91f400700 1 -- 172.21.15.131:0/219121495 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91801d070 con 0x7ff928100f20 2024-05-08T22:07:04.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.096+0000 7ff91f400700 1 -- 172.21.15.131:0/219121495 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff918021410 con 0x7ff928100f20 2024-05-08T22:07:04.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.096+0000 7ff92d800700 1 -- 172.21.15.131:0/219121495 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9180095f0 con 0x7ff928100f20 2024-05-08T22:07:04.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.096+0000 7ff91f400700 1 -- 172.21.15.131:0/219121495 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff918017650 con 0x7ff928100f20 2024-05-08T22:07:04.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.096+0000 7ff92d800700 1 -- 172.21.15.131:0/219121495 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9281a1410 con 0x7ff928100f20 2024-05-08T22:07:04.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.098+0000 7ff92d800700 1 -- 172.21.15.131:0/219121495 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9281083b0 con 0x7ff928100f20 2024-05-08T22:07:04.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.099+0000 7ff91f400700 1 -- 172.21.15.131:0/219121495 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7ff918021580 con 0x7ff928100f20 2024-05-08T22:07:04.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.102+0000 7ff91f400700 1 --2- 172.21.15.131:0/219121495 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff90c0a1230 0x7ff90c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:04.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.102+0000 7ff91f400700 1 -- 172.21.15.131:0/219121495 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7ff9180ce330 con 0x7ff928100f20 2024-05-08T22:07:04.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.102+0000 7ff926000700 1 --2- 172.21.15.131:0/219121495 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff90c0a1230 0x7ff90c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:04.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.102+0000 7ff926000700 1 --2- 172.21.15.131:0/219121495 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff90c0a1230 0x7ff90c0a36f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff914009df0 tx=0x7ff914009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:04.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.105+0000 7ff91f400700 1 -- 172.21.15.131:0/219121495 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff918097860 con 0x7ff928100f20 2024-05-08T22:07:04.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:03 smithi131 ceph-mon[37260]: pgmap v298: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T22:07:04.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:03 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1093993824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:07:04.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:03 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3110447218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:07:04.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:03 smithi157 ceph-mon[77746]: pgmap v298: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T22:07:04.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:03 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1093993824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:07:04.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:03 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3110447218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:07:04.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.311+0000 7ff92d800700 1 -- 172.21.15.131:0/219121495 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff928107d90 con 0x7ff928100f20 2024-05-08T22:07:04.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.312+0000 7ff91f400700 1 -- 172.21.15.131:0/219121495 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff91809ae80 con 0x7ff928100f20 2024-05-08T22:07:04.313 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-08T22:07:04.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.314+0000 7ff92d800700 1 -- 172.21.15.131:0/219121495 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff90c0a1230 msgr2=0x7ff90c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:04.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.314+0000 7ff92d800700 1 --2- 172.21.15.131:0/219121495 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff90c0a1230 0x7ff90c0a36f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff914009df0 tx=0x7ff914009260 comp rx=0 tx=0).stop 2024-05-08T22:07:04.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.314+0000 7ff92d800700 1 -- 172.21.15.131:0/219121495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff928100f20 msgr2=0x7ff928074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:04.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.314+0000 7ff92d800700 1 --2- 172.21.15.131:0/219121495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff928100f20 0x7ff928074d00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff918004710 tx=0x7ff918009580 comp rx=0 tx=0).stop 2024-05-08T22:07:04.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.314+0000 7ff92d800700 1 -- 172.21.15.131:0/219121495 shutdown_connections 2024-05-08T22:07:04.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.314+0000 7ff92d800700 1 --2- 172.21.15.131:0/219121495 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff90c0a1230 0x7ff90c0a36f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.314+0000 7ff92d800700 1 --2- 172.21.15.131:0/219121495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff928102ab0 0x7ff9280738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.314+0000 7ff92d800700 1 --2- 172.21.15.131:0/219121495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff928100f20 0x7ff928074d00 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.314+0000 7ff92d800700 1 --2- 172.21.15.131:0/219121495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9281020f0 0x7ff928073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.314+0000 7ff92d800700 1 -- 172.21.15.131:0/219121495 >> 172.21.15.131:0/219121495 conn(0x7ff9280fc500 msgr2=0x7ff9281054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:04.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.314+0000 7ff92d800700 1 -- 172.21.15.131:0/219121495 shutdown_connections 2024-05-08T22:07:04.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.314+0000 7ff92d800700 1 -- 172.21.15.131:0/219121495 wait complete. 2024-05-08T22:07:04.335 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:07:04.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.530+0000 7fa959a00700 1 -- 172.21.15.131:0/2912076502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa954100f20 msgr2=0x7fa954101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:04.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.530+0000 7fa959a00700 1 --2- 172.21.15.131:0/2912076502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa954100f20 0x7fa954101360 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa9440098f0 tx=0x7fa944009c00 comp rx=0 tx=0).stop 2024-05-08T22:07:04.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.530+0000 7fa959a00700 1 -- 172.21.15.131:0/2912076502 shutdown_connections 2024-05-08T22:07:04.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.530+0000 7fa959a00700 1 --2- 172.21.15.131:0/2912076502 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa954102ab0 0x7fa9541093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.530+0000 7fa959a00700 1 --2- 172.21.15.131:0/2912076502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa954100f20 0x7fa954101360 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.530+0000 7fa959a00700 1 --2- 172.21.15.131:0/2912076502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9541020f0 0x7fa954102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.530+0000 7fa959a00700 1 -- 172.21.15.131:0/2912076502 >> 172.21.15.131:0/2912076502 conn(0x7fa9540fc500 msgr2=0x7fa9540fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:04.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.531+0000 7fa959a00700 1 -- 172.21.15.131:0/2912076502 shutdown_connections 2024-05-08T22:07:04.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.531+0000 7fa959a00700 1 -- 172.21.15.131:0/2912076502 wait complete. 2024-05-08T22:07:04.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.532+0000 7fa959a00700 1 Processor -- start 2024-05-08T22:07:04.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.532+0000 7fa959a00700 1 -- start start 2024-05-08T22:07:04.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.532+0000 7fa959a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa954100f20 0x7fa954196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:04.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.532+0000 7fa959a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9541020f0 0x7fa9541974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:04.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.532+0000 7fa959a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa954102ab0 0x7fa95419e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:04.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.532+0000 7fa959a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95419eae0 con 0x7fa954100f20 2024-05-08T22:07:04.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.532+0000 7fa959a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95419ec20 con 0x7fa9541020f0 2024-05-08T22:07:04.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.532+0000 7fa959a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95419edb0 con 0x7fa954102ab0 2024-05-08T22:07:04.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.532+0000 7fa952000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9541020f0 0x7fa9541974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:04.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.532+0000 7fa952a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa954100f20 0x7fa954196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:04.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.532+0000 7fa952000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9541020f0 0x7fa9541974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:36908/0 (socket says 172.21.15.131:36908) 2024-05-08T22:07:04.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.532+0000 7fa952000700 1 -- 172.21.15.131:0/2756250938 learned_addr learned my addr 172.21.15.131:0/2756250938 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:04.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.532+0000 7fa953400700 1 --2- 172.21.15.131:0/2756250938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa954102ab0 0x7fa95419e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:04.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.533+0000 7fa952000700 1 -- 172.21.15.131:0/2756250938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa954102ab0 msgr2=0x7fa95419e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:04.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.533+0000 7fa952000700 1 --2- 172.21.15.131:0/2756250938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa954102ab0 0x7fa95419e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.533+0000 7fa952000700 1 -- 172.21.15.131:0/2756250938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa954100f20 msgr2=0x7fa954196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:04.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.533+0000 7fa952000700 1 --2- 172.21.15.131:0/2756250938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa954100f20 0x7fa954196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.533+0000 7fa952000700 1 -- 172.21.15.131:0/2756250938 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c009d00 con 0x7fa9541020f0 2024-05-08T22:07:04.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.533+0000 7fa952a00700 1 --2- 172.21.15.131:0/2756250938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa954100f20 0x7fa954196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:04.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.533+0000 7fa952000700 1 --2- 172.21.15.131:0/2756250938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9541020f0 0x7fa9541974f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa94000e9a0 tx=0x7fa94000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:04.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.533+0000 7fa94b400700 1 -- 172.21.15.131:0/2756250938 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa940009790 con 0x7fa9541020f0 2024-05-08T22:07:04.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.533+0000 7fa959a00700 1 -- 172.21.15.131:0/2756250938 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa944009520 con 0x7fa9541020f0 2024-05-08T22:07:04.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.533+0000 7fa94b400700 1 -- 172.21.15.131:0/2756250938 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa940010430 con 0x7fa9541020f0 2024-05-08T22:07:04.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.533+0000 7fa94b400700 1 -- 172.21.15.131:0/2756250938 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa940018680 con 0x7fa9541020f0 2024-05-08T22:07:04.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.533+0000 7fa959a00700 1 -- 172.21.15.131:0/2756250938 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95419f410 con 0x7fa9541020f0 2024-05-08T22:07:04.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.534+0000 7fa959a00700 1 -- 172.21.15.131:0/2756250938 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9541084b0 con 0x7fa9541020f0 2024-05-08T22:07:04.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.536+0000 7fa94b400700 1 -- 172.21.15.131:0/2756250938 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fa9400105a0 con 0x7fa9541020f0 2024-05-08T22:07:04.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.538+0000 7fa94b400700 1 --2- 172.21.15.131:0/2756250938 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa9380a11b0 0x7fa9380a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:04.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.538+0000 7fa94b400700 1 -- 172.21.15.131:0/2756250938 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa940014070 con 0x7fa9541020f0 2024-05-08T22:07:04.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.538+0000 7fa952a00700 1 --2- 172.21.15.131:0/2756250938 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa9380a11b0 0x7fa9380a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:04.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.538+0000 7fa952a00700 1 --2- 172.21.15.131:0/2756250938 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa9380a11b0 0x7fa9380a3670 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa9440098c0 tx=0x7fa944019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:04.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.540+0000 7fa94b400700 1 -- 172.21.15.131:0/2756250938 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa940096430 con 0x7fa9541020f0 2024-05-08T22:07:04.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.744+0000 7fa959a00700 1 -- 172.21.15.131:0/2756250938 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa95410c7d0 con 0x7fa9541020f0 2024-05-08T22:07:04.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.745+0000 7fa94b400700 1 -- 172.21.15.131:0/2756250938 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa9400260a0 con 0x7fa9541020f0 2024-05-08T22:07:04.746 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-08T22:07:04.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.748+0000 7fa959a00700 1 -- 172.21.15.131:0/2756250938 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa9380a11b0 msgr2=0x7fa9380a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:04.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.748+0000 7fa959a00700 1 --2- 172.21.15.131:0/2756250938 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa9380a11b0 0x7fa9380a3670 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa9440098c0 tx=0x7fa944019040 comp rx=0 tx=0).stop 2024-05-08T22:07:04.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.748+0000 7fa959a00700 1 -- 172.21.15.131:0/2756250938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9541020f0 msgr2=0x7fa9541974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:04.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.748+0000 7fa959a00700 1 --2- 172.21.15.131:0/2756250938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9541020f0 0x7fa9541974f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa94000e9a0 tx=0x7fa94000ecb0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.748+0000 7fa959a00700 1 -- 172.21.15.131:0/2756250938 shutdown_connections 2024-05-08T22:07:04.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.748+0000 7fa959a00700 1 --2- 172.21.15.131:0/2756250938 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa9380a11b0 0x7fa9380a3670 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.748+0000 7fa959a00700 1 --2- 172.21.15.131:0/2756250938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa954102ab0 0x7fa95419e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.748+0000 7fa959a00700 1 --2- 172.21.15.131:0/2756250938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa954100f20 0x7fa954196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.748+0000 7fa959a00700 1 --2- 172.21.15.131:0/2756250938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9541020f0 0x7fa9541974f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.748+0000 7fa959a00700 1 -- 172.21.15.131:0/2756250938 >> 172.21.15.131:0/2756250938 conn(0x7fa9540fc500 msgr2=0x7fa9541055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:04.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.748+0000 7fa959a00700 1 -- 172.21.15.131:0/2756250938 shutdown_connections 2024-05-08T22:07:04.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.748+0000 7fa959a00700 1 -- 172.21.15.131:0/2756250938 wait complete. 2024-05-08T22:07:04.769 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:07:04.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.962+0000 7f7f1b200700 1 -- 172.21.15.131:0/2618843307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f1c101f90 msgr2=0x7f7f1c102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:04.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.962+0000 7f7f1b200700 1 --2- 172.21.15.131:0/2618843307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f1c101f90 0x7f7f1c102410 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009990 tx=0x7f7f08009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.962+0000 7f7f1b200700 1 -- 172.21.15.131:0/2618843307 shutdown_connections 2024-05-08T22:07:04.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.962+0000 7f7f1b200700 1 --2- 172.21.15.131:0/2618843307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f1c102950 0x7f7f1c109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.962+0000 7f7f1b200700 1 --2- 172.21.15.131:0/2618843307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f1c100e40 0x7f7f1c101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.962+0000 7f7f1b200700 1 --2- 172.21.15.131:0/2618843307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f1c101f90 0x7f7f1c102410 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.962+0000 7f7f1b200700 1 -- 172.21.15.131:0/2618843307 >> 172.21.15.131:0/2618843307 conn(0x7f7f1c0fc3b0 msgr2=0x7f7f1c0fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:04.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.962+0000 7f7f1b200700 1 -- 172.21.15.131:0/2618843307 shutdown_connections 2024-05-08T22:07:04.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.962+0000 7f7f1b200700 1 -- 172.21.15.131:0/2618843307 wait complete. 2024-05-08T22:07:04.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.963+0000 7f7f1b200700 1 Processor -- start 2024-05-08T22:07:04.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.964+0000 7f7f1b200700 1 -- start start 2024-05-08T22:07:04.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.964+0000 7f7f1b200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f1c100e40 0x7f7f1c074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:04.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.964+0000 7f7f1b200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f1c101f90 0x7f7f1c073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:04.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.964+0000 7f7f1b200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f1c102950 0x7f7f1c0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:04.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.964+0000 7f7f1b200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c074070 con 0x7f7f1c102950 2024-05-08T22:07:04.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.964+0000 7f7f1b200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c0741e0 con 0x7f7f1c101f90 2024-05-08T22:07:04.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.964+0000 7f7f1b200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c1a0e30 con 0x7f7f1c100e40 2024-05-08T22:07:04.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.964+0000 7f7f1a800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f1c102950 0x7f7f1c0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:04.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.964+0000 7f7f1a800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f1c102950 0x7f7f1c0738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38866/0 (socket says 172.21.15.131:38866) 2024-05-08T22:07:04.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.964+0000 7f7f1a800700 1 -- 172.21.15.131:0/3118968265 learned_addr learned my addr 172.21.15.131:0/3118968265 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:04.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.964+0000 7f7f19e00700 1 --2- 172.21.15.131:0/3118968265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f1c100e40 0x7f7f1c074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:04.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.964+0000 7f7f19400700 1 --2- 172.21.15.131:0/3118968265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f1c101f90 0x7f7f1c073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:04.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.965+0000 7f7f19e00700 1 -- 172.21.15.131:0/3118968265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f1c101f90 msgr2=0x7f7f1c073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:04.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.965+0000 7f7f19e00700 1 --2- 172.21.15.131:0/3118968265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f1c101f90 0x7f7f1c073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.965+0000 7f7f19e00700 1 -- 172.21.15.131:0/3118968265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f1c102950 msgr2=0x7f7f1c0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:04.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.965+0000 7f7f19e00700 1 --2- 172.21.15.131:0/3118968265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f1c102950 0x7f7f1c0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:04.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.965+0000 7f7f19e00700 1 -- 172.21.15.131:0/3118968265 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f080095f0 con 0x7f7f1c100e40 2024-05-08T22:07:04.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.965+0000 7f7f1a800700 1 --2- 172.21.15.131:0/3118968265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f1c102950 0x7f7f1c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:04.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.965+0000 7f7f19e00700 1 --2- 172.21.15.131:0/3118968265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f1c100e40 0x7f7f1c074d00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00e9f0 tx=0x7f7f0c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:04.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.965+0000 7f7f12a00700 1 -- 172.21.15.131:0/3118968265 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c00c9c0 con 0x7f7f1c100e40 2024-05-08T22:07:04.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.965+0000 7f7f1b200700 1 -- 172.21.15.131:0/3118968265 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f1c1a1060 con 0x7f7f1c100e40 2024-05-08T22:07:04.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.965+0000 7f7f12a00700 1 -- 172.21.15.131:0/3118968265 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f0c010430 con 0x7f7f1c100e40 2024-05-08T22:07:04.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.965+0000 7f7f12a00700 1 -- 172.21.15.131:0/3118968265 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c018680 con 0x7f7f1c100e40 2024-05-08T22:07:04.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.965+0000 7f7f1b200700 1 -- 172.21.15.131:0/3118968265 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1c1a1510 con 0x7f7f1c100e40 2024-05-08T22:07:04.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.966+0000 7f7f1b200700 1 -- 172.21.15.131:0/3118968265 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f1c108280 con 0x7f7f1c100e40 2024-05-08T22:07:04.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.968+0000 7f7f12a00700 1 -- 172.21.15.131:0/3118968265 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f7f0c010860 con 0x7f7f1c100e40 2024-05-08T22:07:04.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.969+0000 7f7f12a00700 1 --2- 172.21.15.131:0/3118968265 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7f000a11b0 0x7f7f000a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:04.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.969+0000 7f7f12a00700 1 -- 172.21.15.131:0/3118968265 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f7f0c014070 con 0x7f7f1c100e40 2024-05-08T22:07:04.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.969+0000 7f7f19400700 1 --2- 172.21.15.131:0/3118968265 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7f000a11b0 0x7f7f000a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:04.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.970+0000 7f7f19400700 1 --2- 172.21.15.131:0/3118968265 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7f000a11b0 0x7f7f000a3670 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7f0800b2e0 tx=0x7f7f08005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:04.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:04.974+0000 7f7f12a00700 1 -- 172.21.15.131:0/3118968265 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7f0c052d60 con 0x7f7f1c100e40 2024-05-08T22:07:05.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:04 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1203961946' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:07:05.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:04 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/219121495' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:07:05.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:04 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2756250938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:07:05.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.226+0000 7f7f1b200700 1 -- 172.21.15.131:0/3118968265 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f1c10c690 con 0x7f7f1c100e40 2024-05-08T22:07:05.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.227+0000 7f7f12a00700 1 -- 172.21.15.131:0/3118968265 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10889 (secure 0 0 0) 0x7f7f0c0d0030 con 0x7f7f1c100e40 2024-05-08T22:07:05.228 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:07:05.229 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:05:14.034908+0000","modified":"2024-05-08T22:05:39.494566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:07:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.230+0000 7f7f1b200700 1 -- 172.21.15.131:0/3118968265 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7f000a11b0 msgr2=0x7f7f000a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.230+0000 7f7f1b200700 1 --2- 172.21.15.131:0/3118968265 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7f000a11b0 0x7f7f000a3670 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7f0800b2e0 tx=0x7f7f08005fb0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.230+0000 7f7f1b200700 1 -- 172.21.15.131:0/3118968265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f1c100e40 msgr2=0x7f7f1c074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.230+0000 7f7f1b200700 1 --2- 172.21.15.131:0/3118968265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f1c100e40 0x7f7f1c074d00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00e9f0 tx=0x7f7f0c00ed00 comp rx=0 tx=0).stop 2024-05-08T22:07:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.230+0000 7f7f1b200700 1 -- 172.21.15.131:0/3118968265 shutdown_connections 2024-05-08T22:07:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.230+0000 7f7f1b200700 1 --2- 172.21.15.131:0/3118968265 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f7f000a11b0 0x7f7f000a3670 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.230+0000 7f7f1b200700 1 --2- 172.21.15.131:0/3118968265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f1c102950 0x7f7f1c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.230+0000 7f7f1b200700 1 --2- 172.21.15.131:0/3118968265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f1c100e40 0x7f7f1c074d00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.230+0000 7f7f1b200700 1 --2- 172.21.15.131:0/3118968265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f1c101f90 0x7f7f1c073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.230+0000 7f7f1b200700 1 -- 172.21.15.131:0/3118968265 >> 172.21.15.131:0/3118968265 conn(0x7f7f1c0fc3b0 msgr2=0x7f7f1c1053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.230+0000 7f7f1b200700 1 -- 172.21.15.131:0/3118968265 shutdown_connections 2024-05-08T22:07:05.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.230+0000 7f7f1b200700 1 -- 172.21.15.131:0/3118968265 wait complete. 2024-05-08T22:07:05.232 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 23 2024-05-08T22:07:05.251 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:07:05.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:04 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1203961946' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:07:05.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:04 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/219121495' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:07:05.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:04 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2756250938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:07:05.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:04 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1203961946' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:07:05.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:04 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/219121495' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:07:05.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:04 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2756250938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:07:05.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.444+0000 7fa3f5c00700 1 -- 172.21.15.131:0/2296336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3f0100e70 msgr2=0x7fa3f0101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:05.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.444+0000 7fa3f5c00700 1 --2- 172.21.15.131:0/2296336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3f0100e70 0x7fa3f0101290 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009990 tx=0x7fa3e0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.444+0000 7fa3f5c00700 1 -- 172.21.15.131:0/2296336 shutdown_connections 2024-05-08T22:07:05.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.444+0000 7fa3f5c00700 1 --2- 172.21.15.131:0/2296336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3f0102980 0x7fa3f01092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.444+0000 7fa3f5c00700 1 --2- 172.21.15.131:0/2296336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3f0100e70 0x7fa3f0101290 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.444+0000 7fa3f5c00700 1 --2- 172.21.15.131:0/2296336 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3f0101fc0 0x7fa3f0102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.444+0000 7fa3f5c00700 1 -- 172.21.15.131:0/2296336 >> 172.21.15.131:0/2296336 conn(0x7fa3f00fc3e0 msgr2=0x7fa3f00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:05.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.444+0000 7fa3f5c00700 1 -- 172.21.15.131:0/2296336 shutdown_connections 2024-05-08T22:07:05.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.444+0000 7fa3f5c00700 1 -- 172.21.15.131:0/2296336 wait complete. 2024-05-08T22:07:05.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.446+0000 7fa3f5c00700 1 Processor -- start 2024-05-08T22:07:05.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.446+0000 7fa3f5c00700 1 -- start start 2024-05-08T22:07:05.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.446+0000 7fa3f5c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3f0100e70 0x7fa3f0199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:05.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.446+0000 7fa3f5c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3f0101fc0 0x7fa3f0199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:05.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.447+0000 7fa3f5c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3f0102980 0x7fa3f01a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:05.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.447+0000 7fa3f5c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f01a0b30 con 0x7fa3f0102980 2024-05-08T22:07:05.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.447+0000 7fa3f5c00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f01a0c70 con 0x7fa3f0100e70 2024-05-08T22:07:05.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.447+0000 7fa3f5c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f01a0e00 con 0x7fa3f0101fc0 2024-05-08T22:07:05.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.447+0000 7fa3ef400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3f0102980 0x7fa3f01a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:05.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.447+0000 7fa3ee000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3f0101fc0 0x7fa3f0199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:05.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.447+0000 7fa3ee000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3f0101fc0 0x7fa3f0199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55532/0 (socket says 172.21.15.131:55532) 2024-05-08T22:07:05.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.447+0000 7fa3eea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3f0100e70 0x7fa3f0199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:05.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.447+0000 7fa3ee000700 1 -- 172.21.15.131:0/4134166820 learned_addr learned my addr 172.21.15.131:0/4134166820 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:05.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.447+0000 7fa3ef400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3f0102980 0x7fa3f01a0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38888/0 (socket says 172.21.15.131:38888) 2024-05-08T22:07:05.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.447+0000 7fa3ee000700 1 -- 172.21.15.131:0/4134166820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3f0100e70 msgr2=0x7fa3f0199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:05.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.447+0000 7fa3ee000700 1 --2- 172.21.15.131:0/4134166820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3f0100e70 0x7fa3f0199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.447+0000 7fa3ee000700 1 -- 172.21.15.131:0/4134166820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3f0102980 msgr2=0x7fa3f01a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:05.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.447+0000 7fa3ee000700 1 --2- 172.21.15.131:0/4134166820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3f0102980 0x7fa3f01a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.447+0000 7fa3ee000700 1 -- 172.21.15.131:0/4134166820 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e00095f0 con 0x7fa3f0101fc0 2024-05-08T22:07:05.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.447+0000 7fa3ef400700 1 --2- 172.21.15.131:0/4134166820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3f0102980 0x7fa3f01a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:05.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.447+0000 7fa3eea00700 1 --2- 172.21.15.131:0/4134166820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3f0100e70 0x7fa3f0199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:05.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.448+0000 7fa3ee000700 1 --2- 172.21.15.131:0/4134166820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3f0101fc0 0x7fa3f0199540 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00e9a0 tx=0x7fa3dc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:05.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.448+0000 7fa3e7400700 1 -- 172.21.15.131:0/4134166820 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc009790 con 0x7fa3f0101fc0 2024-05-08T22:07:05.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.448+0000 7fa3e7400700 1 -- 172.21.15.131:0/4134166820 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3dc010430 con 0x7fa3f0101fc0 2024-05-08T22:07:05.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.448+0000 7fa3f5c00700 1 -- 172.21.15.131:0/4134166820 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f01a10e0 con 0x7fa3f0101fc0 2024-05-08T22:07:05.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.448+0000 7fa3e7400700 1 -- 172.21.15.131:0/4134166820 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc018680 con 0x7fa3f0101fc0 2024-05-08T22:07:05.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.448+0000 7fa3f5c00700 1 -- 172.21.15.131:0/4134166820 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f01a1640 con 0x7fa3f0101fc0 2024-05-08T22:07:05.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.449+0000 7fa3f5c00700 1 -- 172.21.15.131:0/4134166820 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f0067c10 con 0x7fa3f0101fc0 2024-05-08T22:07:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.452+0000 7fa3e7400700 1 -- 172.21.15.131:0/4134166820 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fa3dc0105a0 con 0x7fa3f0101fc0 2024-05-08T22:07:05.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.453+0000 7fa3e7400700 1 --2- 172.21.15.131:0/4134166820 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa3d40a1230 0x7fa3d40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:05.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.453+0000 7fa3e7400700 1 -- 172.21.15.131:0/4134166820 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa3dc014070 con 0x7fa3f0101fc0 2024-05-08T22:07:05.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.453+0000 7fa3eea00700 1 --2- 172.21.15.131:0/4134166820 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa3d40a1230 0x7fa3d40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:05.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.454+0000 7fa3eea00700 1 --2- 172.21.15.131:0/4134166820 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa3d40a1230 0x7fa3d40a36f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000b2e0 tx=0x7fa3e0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:05.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.455+0000 7fa3e7400700 1 -- 172.21.15.131:0/4134166820 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa3dc096460 con 0x7fa3f0101fc0 2024-05-08T22:07:05.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.670+0000 7fa3f5c00700 1 -- 172.21.15.131:0/4134166820 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3f010c860 con 0x7fa3f0101fc0 2024-05-08T22:07:05.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.673+0000 7fa3e7400700 1 -- 172.21.15.131:0/4134166820 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24721 (secure 0 0 0) 0x7fa3dc026020 con 0x7fa3f0101fc0 2024-05-08T22:07:05.674 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:07:05.674 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":87,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:06:41.971918+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:05:12.469858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:05:13.484446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:05:17.016186+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:07:05.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.675+0000 7fa3f5c00700 1 -- 172.21.15.131:0/4134166820 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa3d40a1230 msgr2=0x7fa3d40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:05.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.675+0000 7fa3f5c00700 1 --2- 172.21.15.131:0/4134166820 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa3d40a1230 0x7fa3d40a36f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000b2e0 tx=0x7fa3e0005fd0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.675+0000 7fa3f5c00700 1 -- 172.21.15.131:0/4134166820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3f0101fc0 msgr2=0x7fa3f0199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:05.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.675+0000 7fa3f5c00700 1 --2- 172.21.15.131:0/4134166820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3f0101fc0 0x7fa3f0199540 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00e9a0 tx=0x7fa3dc00ecb0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.675+0000 7fa3f5c00700 1 -- 172.21.15.131:0/4134166820 shutdown_connections 2024-05-08T22:07:05.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.675+0000 7fa3f5c00700 1 --2- 172.21.15.131:0/4134166820 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fa3d40a1230 0x7fa3d40a36f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.675+0000 7fa3f5c00700 1 --2- 172.21.15.131:0/4134166820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3f0102980 0x7fa3f01a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.675+0000 7fa3f5c00700 1 --2- 172.21.15.131:0/4134166820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3f0100e70 0x7fa3f0199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.675+0000 7fa3f5c00700 1 --2- 172.21.15.131:0/4134166820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3f0101fc0 0x7fa3f0199540 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.675+0000 7fa3f5c00700 1 -- 172.21.15.131:0/4134166820 >> 172.21.15.131:0/4134166820 conn(0x7fa3f00fc3e0 msgr2=0x7fa3f00731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:05.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.676+0000 7fa3f5c00700 1 -- 172.21.15.131:0/4134166820 shutdown_connections 2024-05-08T22:07:05.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.676+0000 7fa3f5c00700 1 -- 172.21.15.131:0/4134166820 wait complete. 2024-05-08T22:07:05.696 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-08T22:05:14.034908+0000', 'modified': '2024-05-08T22:05:39.494566+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24566, 'mds_1': 24562, 'mds_2': 24550}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24520': {'gid': 24520, 'name': 'c', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6837/2693198274', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6836', 'nonce': 2693198274}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 2693198274}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24524': {'gid': 24524, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.132:6837/2685833178', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6836', 'nonce': 2685833178}, {'type': 'v1', 'addr': '172.21.15.132:6837', 'nonce': 2685833178}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24538': {'gid': 24538, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6839/3040193644', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6838', 'nonce': 3040193644}, {'type': 'v1', 'addr': '172.21.15.157:6839', 'nonce': 3040193644}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24550': {'gid': 24550, 'name': 'g', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.131:6839/1622880274', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6838', 'nonce': 1622880274}, {'type': 'v1', 'addr': '172.21.15.131:6839', 'nonce': 1622880274}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24562': {'gid': 24562, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.131:6841/1686013987', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6840', 'nonce': 1686013987}, {'type': 'v1', 'addr': '172.21.15.131:6841', 'nonce': 1686013987}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24566': {'gid': 24566, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.132:6839/4205858754', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6838', 'nonce': 4205858754}, {'type': 'v1', 'addr': '172.21.15.132:6839', 'nonce': 4205858754}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:07:05.696 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-08T22:07:05.696 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-05-08T22:07:05.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.892+0000 7efeae800700 1 -- 172.21.15.131:0/2590907486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efea81019e0 msgr2=0x7efea8103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:05.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.892+0000 7efeae800700 1 --2- 172.21.15.131:0/2590907486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efea81019e0 0x7efea8103dd0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efe98009990 tx=0x7efe98009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.892+0000 7efeae800700 1 -- 172.21.15.131:0/2590907486 shutdown_connections 2024-05-08T22:07:05.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.892+0000 7efeae800700 1 --2- 172.21.15.131:0/2590907486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efea8106c40 0x7efea81090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.892+0000 7efeae800700 1 --2- 172.21.15.131:0/2590907486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efea81019e0 0x7efea8103dd0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.892+0000 7efeae800700 1 --2- 172.21.15.131:0/2590907486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efea8104310 0x7efea8106700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.892+0000 7efeae800700 1 -- 172.21.15.131:0/2590907486 >> 172.21.15.131:0/2590907486 conn(0x7efea80fb3f0 msgr2=0x7efea80fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:05.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.892+0000 7efeae800700 1 -- 172.21.15.131:0/2590907486 shutdown_connections 2024-05-08T22:07:05.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.892+0000 7efeae800700 1 -- 172.21.15.131:0/2590907486 wait complete. 2024-05-08T22:07:05.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.893+0000 7efeae800700 1 Processor -- start 2024-05-08T22:07:05.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.894+0000 7efeae800700 1 -- start start 2024-05-08T22:07:05.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.894+0000 7efeae800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efea81019e0 0x7efea81990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:05.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.894+0000 7efeae800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efea8104310 0x7efea8199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:05.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.894+0000 7efeae800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efea8106c40 0x7efea81a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:05.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.894+0000 7efeae800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea81a0c20 con 0x7efea8106c40 2024-05-08T22:07:05.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.894+0000 7efeae800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea81a0d60 con 0x7efea81019e0 2024-05-08T22:07:05.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.894+0000 7efeae800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea81a0ef0 con 0x7efea8104310 2024-05-08T22:07:05.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.894+0000 7efea7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efea8106c40 0x7efea81a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:05.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.894+0000 7efea6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efea8104310 0x7efea8199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:05.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.894+0000 7efea7400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efea81019e0 0x7efea81990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:05.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.894+0000 7efea7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efea8106c40 0x7efea81a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38914/0 (socket says 172.21.15.131:38914) 2024-05-08T22:07:05.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.894+0000 7efea7e00700 1 -- 172.21.15.131:0/2961500487 learned_addr learned my addr 172.21.15.131:0/2961500487 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:05.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.895+0000 7efea7e00700 1 -- 172.21.15.131:0/2961500487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efea8104310 msgr2=0x7efea8199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:05.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.895+0000 7efea7e00700 1 --2- 172.21.15.131:0/2961500487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efea8104310 0x7efea8199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.895+0000 7efea7e00700 1 -- 172.21.15.131:0/2961500487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efea81019e0 msgr2=0x7efea81990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:05.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.895+0000 7efea7e00700 1 --2- 172.21.15.131:0/2961500487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efea81019e0 0x7efea81990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.895+0000 7efea7e00700 1 -- 172.21.15.131:0/2961500487 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe980095f0 con 0x7efea8106c40 2024-05-08T22:07:05.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.895+0000 7efea7e00700 1 --2- 172.21.15.131:0/2961500487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efea8106c40 0x7efea81a0650 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efea000ef30 tx=0x7efea000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:05.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.895+0000 7efe9fe00700 1 -- 172.21.15.131:0/2961500487 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea0018070 con 0x7efea8106c40 2024-05-08T22:07:05.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.895+0000 7efe9fe00700 1 -- 172.21.15.131:0/2961500487 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efea000f040 con 0x7efea8106c40 2024-05-08T22:07:05.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.895+0000 7efeae800700 1 -- 172.21.15.131:0/2961500487 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea81a1170 con 0x7efea8106c40 2024-05-08T22:07:05.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.895+0000 7efe9fe00700 1 -- 172.21.15.131:0/2961500487 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea000cd00 con 0x7efea8106c40 2024-05-08T22:07:05.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.895+0000 7efeae800700 1 -- 172.21.15.131:0/2961500487 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea81a1630 con 0x7efea8106c40 2024-05-08T22:07:05.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.898+0000 7efe9fe00700 1 -- 172.21.15.131:0/2961500487 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7efea0003a00 con 0x7efea8106c40 2024-05-08T22:07:05.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.899+0000 7efe9fe00700 1 --2- 172.21.15.131:0/2961500487 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7efe8c0a11b0 0x7efe8c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:05.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.900+0000 7efe9fe00700 1 -- 172.21.15.131:0/2961500487 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7efea00cc930 con 0x7efea8106c40 2024-05-08T22:07:05.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.900+0000 7efec23dab80 1 Processor -- start 2024-05-08T22:07:05.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.900+0000 7efea7400700 1 --2- 172.21.15.131:0/2961500487 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7efe8c0a11b0 0x7efe8c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:05.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.900+0000 7efec23dab80 1 -- start start 2024-05-08T22:07:05.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.900+0000 7efec23dab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b24f0 0x556acc5b2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:05.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.900+0000 7efec23dab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556acc5b2e50 0x556acc5b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:05.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.900+0000 7efec23dab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acc5b3810 0x556acc5b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:05.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.900+0000 7efec23dab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acc509100 con 0x556acc5b24f0 2024-05-08T22:07:05.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.900+0000 7efec23dab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acc506ac0 con 0x556acc5b3810 2024-05-08T22:07:05.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.900+0000 7efec23dab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acc5023b0 con 0x556acc5b2e50 2024-05-08T22:07:05.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.901+0000 7efea7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b24f0 0x556acc5b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:05.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.901+0000 7efea7e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acc5b3810 0x556acc5b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:05.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.901+0000 7efea7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b24f0 0x556acc5b2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38916/0 (socket says 172.21.15.131:38916) 2024-05-08T22:07:05.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.901+0000 7efea7400700 1 -- 172.21.15.131:0/2577314286 learned_addr learned my addr 172.21.15.131:0/2577314286 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:05.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.901+0000 7efea6a00700 1 --2- 172.21.15.131:0/2577314286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556acc5b2e50 0x556acc5b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:05.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.901+0000 7efea7400700 1 --2- 172.21.15.131:0/2961500487 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7efe8c0a11b0 0x7efe8c0a3670 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7efe9800b2e0 tx=0x7efe9800b910 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:05.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.901+0000 7efea7e00700 1 -- 172.21.15.131:0/2577314286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556acc5b2e50 msgr2=0x556acc5b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:05.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.901+0000 7efea7e00700 1 --2- 172.21.15.131:0/2577314286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556acc5b2e50 0x556acc5b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.901+0000 7efea7e00700 1 -- 172.21.15.131:0/2577314286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b24f0 msgr2=0x556acc5b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:05.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.901+0000 7efea7e00700 1 --2- 172.21.15.131:0/2577314286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b24f0 0x556acc5b2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.901+0000 7efea7e00700 1 -- 172.21.15.131:0/2577314286 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556acc383360 con 0x556acc5b3810 2024-05-08T22:07:05.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.901+0000 7efea7400700 1 --2- 172.21.15.131:0/2577314286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b24f0 0x556acc5b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:05.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.901+0000 7efea7e00700 1 --2- 172.21.15.131:0/2577314286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acc5b3810 0x556acc5b3c90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efea00c9f30 tx=0x7efea00994f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1b8b0aba1ce3baf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:05.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.901+0000 7efe9cc00700 1 -- 172.21.15.131:0/2577314286 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea0004ad0 con 0x556acc5b3810 2024-05-08T22:07:05.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.901+0000 7efe9cc00700 1 -- 172.21.15.131:0/2577314286 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efea00b7e90 con 0x556acc5b3810 2024-05-08T22:07:05.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.902+0000 7efe9cc00700 1 -- 172.21.15.131:0/2577314286 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea00a1d20 con 0x556acc5b3810 2024-05-08T22:07:05.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.902+0000 7efec23dab80 1 -- 172.21.15.131:0/2577314286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acc5b3810 msgr2=0x556acc5b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:05.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.902+0000 7efec23dab80 1 --2- 172.21.15.131:0/2577314286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acc5b3810 0x556acc5b3c90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efea00c9f30 tx=0x7efea00994f0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.902+0000 7efec23dab80 1 -- 172.21.15.131:0/2577314286 shutdown_connections 2024-05-08T22:07:05.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.902+0000 7efec23dab80 1 --2- 172.21.15.131:0/2577314286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acc5b3810 0x556acc5b3c90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.902+0000 7efec23dab80 1 --2- 172.21.15.131:0/2577314286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b24f0 0x556acc5b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.902+0000 7efec23dab80 1 --2- 172.21.15.131:0/2577314286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556acc5b2e50 0x556acc5b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.902+0000 7efec23dab80 1 -- 172.21.15.131:0/2577314286 >> 172.21.15.131:0/2577314286 conn(0x556acc5101c0 msgr2=0x556acc5acd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:05.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.902+0000 7efec23dab80 1 -- 172.21.15.131:0/2577314286 shutdown_connections 2024-05-08T22:07:05.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.902+0000 7efec23dab80 1 -- 172.21.15.131:0/2577314286 wait complete. 2024-05-08T22:07:05.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.903+0000 7efec23dab80 1 -- start start 2024-05-08T22:07:05.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.903+0000 7efec23dab80 1 Processor -- start 2024-05-08T22:07:05.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.903+0000 7efec23dab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556acc5b24f0 0x556acc5bbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:05.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.903+0000 7efec23dab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acc5b2e50 0x556acc5bc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:05.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.904+0000 7efec23dab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b3810 0x556acc5c33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:05.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.904+0000 7efec23dab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acc49e510 con 0x556acc5b3810 2024-05-08T22:07:05.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.904+0000 7efec23dab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acc49e1b0 con 0x556acc5b2e50 2024-05-08T22:07:05.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.904+0000 7efec23dab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acc385650 con 0x556acc5b24f0 2024-05-08T22:07:05.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.904+0000 7efec23dab80 10 client.0 _ll_register_callbacks cb 0x556acc4a7ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:07:05.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.904+0000 7efea7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556acc5b24f0 0x556acc5bbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:05.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.904+0000 7efea7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b3810 0x556acc5c33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:05.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.904+0000 7efea7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556acc5b24f0 0x556acc5bbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55566/0 (socket says 172.21.15.131:55566) 2024-05-08T22:07:05.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.904+0000 7efea7400700 1 -- 172.21.15.131:0/2441285573 learned_addr learned my addr 172.21.15.131:0/2441285573 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:05.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.904+0000 7efea6a00700 1 --2- 172.21.15.131:0/2441285573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acc5b2e50 0x556acc5bc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:05.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.904+0000 7efea7400700 1 -- 172.21.15.131:0/2441285573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acc5b2e50 msgr2=0x556acc5bc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:05.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.904+0000 7efea7400700 1 --2- 172.21.15.131:0/2441285573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acc5b2e50 0x556acc5bc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.904+0000 7efea7400700 1 -- 172.21.15.131:0/2441285573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b3810 msgr2=0x556acc5c33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:05.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.904+0000 7efea7400700 1 --2- 172.21.15.131:0/2441285573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b3810 0x556acc5c33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.904+0000 7efea7400700 1 -- 172.21.15.131:0/2441285573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea8109ca0 con 0x556acc5b24f0 2024-05-08T22:07:05.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.904+0000 7efea6a00700 1 --2- 172.21.15.131:0/2441285573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acc5b2e50 0x556acc5bc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:07:05.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.904+0000 7efea7e00700 1 --2- 172.21.15.131:0/2441285573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b3810 0x556acc5c33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:05.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.904+0000 7efea7400700 1 --2- 172.21.15.131:0/2441285573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556acc5b24f0 0x556acc5bbe40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efe98000c00 tx=0x7efe98009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=809d9101cec1fe67 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:05.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.904+0000 7efe91600700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:07:05.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.905+0000 7efe91600700 1 -- 172.21.15.131:0/2441285573 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe98004b70 con 0x556acc5b24f0 2024-05-08T22:07:05.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.905+0000 7efec23dab80 1 -- 172.21.15.131:0/2441285573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556acc45b220 con 0x556acc5b24f0 2024-05-08T22:07:05.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.905+0000 7efe91600700 1 -- 172.21.15.131:0/2441285573 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7efe9802c040 con 0x556acc5b24f0 2024-05-08T22:07:05.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.905+0000 7efe91600700 1 -- 172.21.15.131:0/2441285573 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe98004cd0 con 0x556acc5b24f0 2024-05-08T22:07:05.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.905+0000 7efec23dab80 10 client.25342 fetch_fsmap learned FSMap version 23 2024-05-08T22:07:05.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.905+0000 7efec23dab80 1 -- 172.21.15.131:0/2441285573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7efea81a1630 con 0x556acc5b24f0 2024-05-08T22:07:05.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.905+0000 7efe91600700 1 -- 172.21.15.131:0/2441285573 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556acc45b220 con 0x556acc5b24f0 2024-05-08T22:07:05.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.905+0000 7efe91600700 1 -- 172.21.15.131:0/2441285573 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efe98026410 con 0x556acc5b24f0 2024-05-08T22:07:05.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.906+0000 7efec23dab80 10 client.25342 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T22:07:05.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.906+0000 7efec23dab80 10 client.25342 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-08T22:07:05.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.906+0000 7efec23dab80 20 client.25342 populate_metadata read hostname 'smithi131' 2024-05-08T22:07:05.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.906+0000 7efec23dab80 1 --2- 172.21.15.131:0/2441285573 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x556acc5c6ec0 0x556acc5cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:05.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.906+0000 7efec23dab80 4 client.25342 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:07:05.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.906+0000 7efec23dab80 1 -- 172.21.15.131:0/2441285573 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556acc5817c0 con 0x556acc5c6ec0 2024-05-08T22:07:05.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.906+0000 7efea6a00700 1 --2- 172.21.15.131:0/2441285573 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x556acc5c6ec0 0x556acc5cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:05.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.906+0000 7efea6a00700 1 --2- 172.21.15.131:0/2441285573 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x556acc5c6ec0 0x556acc5cc1a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea256fc0d7059174 server_cookie=a3059f99804e6348 in_seq=0 out_seq=0 2024-05-08T22:07:05.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.906+0000 7efe91600700 10 client.25342 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:07:05.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.907+0000 7efe91600700 1 -- 172.21.15.131:0/2441285573 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556acc5817c0 con 0x556acc5c6ec0 2024-05-08T22:07:05.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.907+0000 7efe91600700 10 client.25342 handle_command_reply: tid=0 2024-05-08T22:07:05.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.907+0000 7efec23dab80 1 client.25342 shutdown 2024-05-08T22:07:05.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.907+0000 7efec23dab80 1 -- 172.21.15.131:0/2441285573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556acc5b24f0 msgr2=0x556acc5bbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:05.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.907+0000 7efec23dab80 1 --2- 172.21.15.131:0/2441285573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556acc5b24f0 0x556acc5bbe40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efe98000c00 tx=0x7efe98009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.907+0000 7efec23dab80 1 -- 172.21.15.131:0/2441285573 shutdown_connections 2024-05-08T22:07:05.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.907+0000 7efec23dab80 1 --2- 172.21.15.131:0/2441285573 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x556acc5c6ec0 0x556acc5cc1a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.907+0000 7efec23dab80 1 --2- 172.21.15.131:0/2441285573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b3810 0x556acc5c33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.907+0000 7efec23dab80 1 --2- 172.21.15.131:0/2441285573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556acc5b24f0 0x556acc5bbe40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.907+0000 7efec23dab80 1 --2- 172.21.15.131:0/2441285573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acc5b2e50 0x556acc5bc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.907+0000 7efe91600700 0 client.25342 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:07:05.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.907+0000 7efec23dab80 1 -- 172.21.15.131:0/2441285573 >> 172.21.15.131:0/2441285573 conn(0x556acc5101c0 msgr2=0x556acc5ac550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:05.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.908+0000 7efec23dab80 1 -- 172.21.15.131:0/2441285573 shutdown_connections 2024-05-08T22:07:05.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.908+0000 7efec23dab80 1 -- 172.21.15.131:0/2441285573 wait complete. 2024-05-08T22:07:05.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.908+0000 7efec23dab80 20 client.25342 trim_cache size 0 max 16384 2024-05-08T22:07:05.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.926+0000 7efec23dab80 1 Processor -- start 2024-05-08T22:07:05.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.926+0000 7efec23dab80 1 -- start start 2024-05-08T22:07:05.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.927+0000 7efec23dab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b24f0 0x556acc5bc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:05.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.927+0000 7efec23dab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acc5b2e50 0x556acc5b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:05.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.927+0000 7efec23dab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556acc5b3810 0x556acc5c3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:05.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.927+0000 7efec23dab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acc45b220 con 0x556acc5b24f0 2024-05-08T22:07:05.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.927+0000 7efec23dab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acc4c02b0 con 0x556acc5b2e50 2024-05-08T22:07:05.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.927+0000 7efec23dab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acc4fb150 con 0x556acc5b3810 2024-05-08T22:07:05.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.927+0000 7efea7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b24f0 0x556acc5bc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:05.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.927+0000 7efea7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b24f0 0x556acc5bc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38926/0 (socket says 172.21.15.131:38926) 2024-05-08T22:07:05.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.927+0000 7efea7400700 1 -- 172.21.15.131:0/3629108516 learned_addr learned my addr 172.21.15.131:0/3629108516 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:05.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.927+0000 7efea7e00700 1 --2- 172.21.15.131:0/3629108516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556acc5b3810 0x556acc5c3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:05.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.927+0000 7efea6a00700 1 --2- 172.21.15.131:0/3629108516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acc5b2e50 0x556acc5b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:05.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.927+0000 7efea7400700 1 -- 172.21.15.131:0/3629108516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556acc5b3810 msgr2=0x556acc5c3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:05.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.927+0000 7efea7400700 1 --2- 172.21.15.131:0/3629108516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556acc5b3810 0x556acc5c3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.927+0000 7efea7400700 1 -- 172.21.15.131:0/3629108516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acc5b2e50 msgr2=0x556acc5b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:05.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.927+0000 7efea7400700 1 --2- 172.21.15.131:0/3629108516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acc5b2e50 0x556acc5b28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.927+0000 7efea7400700 1 -- 172.21.15.131:0/3629108516 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556acc32fb10 con 0x556acc5b24f0 2024-05-08T22:07:05.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.927+0000 7efea7400700 1 --2- 172.21.15.131:0/3629108516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b24f0 0x556acc5bc480 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7efe9800c010 tx=0x7efe9800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76dee8d5ed969209 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:05.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.928+0000 7efe9d600700 1 -- 172.21.15.131:0/3629108516 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe980058e0 con 0x556acc5b24f0 2024-05-08T22:07:05.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.928+0000 7efe9d600700 1 -- 172.21.15.131:0/3629108516 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe9801e040 con 0x556acc5b24f0 2024-05-08T22:07:05.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.928+0000 7efe9d600700 1 -- 172.21.15.131:0/3629108516 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9800f800 con 0x556acc5b24f0 2024-05-08T22:07:05.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.928+0000 7efec23dab80 1 -- 172.21.15.131:0/3629108516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b24f0 msgr2=0x556acc5bc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:05.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.928+0000 7efec23dab80 1 --2- 172.21.15.131:0/3629108516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b24f0 0x556acc5bc480 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7efe9800c010 tx=0x7efe9800b410 comp rx=0 tx=0).stop 2024-05-08T22:07:05.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.928+0000 7efec23dab80 1 -- 172.21.15.131:0/3629108516 shutdown_connections 2024-05-08T22:07:05.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.928+0000 7efec23dab80 1 --2- 172.21.15.131:0/3629108516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556acc5b3810 0x556acc5c3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.928+0000 7efec23dab80 1 --2- 172.21.15.131:0/3629108516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b24f0 0x556acc5bc480 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.928+0000 7efec23dab80 1 --2- 172.21.15.131:0/3629108516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acc5b2e50 0x556acc5b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.928+0000 7efec23dab80 1 -- 172.21.15.131:0/3629108516 >> 172.21.15.131:0/3629108516 conn(0x556acc5101c0 msgr2=0x556acc5b9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:05.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.928+0000 7efec23dab80 1 -- 172.21.15.131:0/3629108516 shutdown_connections 2024-05-08T22:07:05.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.928+0000 7efec23dab80 1 -- 172.21.15.131:0/3629108516 wait complete. 2024-05-08T22:07:05.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.929+0000 7efec23dab80 1 -- start start 2024-05-08T22:07:05.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.929+0000 7efec23dab80 1 Processor -- start 2024-05-08T22:07:05.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.929+0000 7efec23dab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b24f0 0x556acc5bf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:05.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.929+0000 7efec23dab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556acc5b2e50 0x556acc5bfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:05.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.929+0000 7efec23dab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acc5b3810 0x556acc5c2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:05.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.929+0000 7efec23dab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acc5c3000 con 0x556acc5b24f0 2024-05-08T22:07:05.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.929+0000 7efec23dab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acc5c3170 con 0x556acc5b3810 2024-05-08T22:07:05.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.929+0000 7efec23dab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acc5c3330 con 0x556acc5b2e50 2024-05-08T22:07:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.929+0000 7efec23dab80 10 client.0 _ll_register_callbacks cb 0x556acc4a7ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:07:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.930+0000 7efea7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b24f0 0x556acc5bf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.930+0000 7efea7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b24f0 0x556acc5bf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38934/0 (socket says 172.21.15.131:38934) 2024-05-08T22:07:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.930+0000 7efea7400700 1 -- 172.21.15.131:0/2151152107 learned_addr learned my addr 172.21.15.131:0/2151152107 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.930+0000 7efea6a00700 1 --2- 172.21.15.131:0/2151152107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556acc5b2e50 0x556acc5bfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.930+0000 7efea7e00700 1 --2- 172.21.15.131:0/2151152107 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acc5b3810 0x556acc5c2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.930+0000 7efea7400700 1 -- 172.21.15.131:0/2151152107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556acc5b2e50 msgr2=0x556acc5bfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.930+0000 7efea7400700 1 --2- 172.21.15.131:0/2151152107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556acc5b2e50 0x556acc5bfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.930+0000 7efea7400700 1 -- 172.21.15.131:0/2151152107 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acc5b3810 msgr2=0x556acc5c2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.930+0000 7efea7400700 1 --2- 172.21.15.131:0/2151152107 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acc5b3810 0x556acc5c2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.930+0000 7efea7400700 1 -- 172.21.15.131:0/2151152107 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea81a1630 con 0x556acc5b24f0 2024-05-08T22:07:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.930+0000 7efea7e00700 1 --2- 172.21.15.131:0/2151152107 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acc5b3810 0x556acc5c2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:07:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.930+0000 7efea7400700 1 --2- 172.21.15.131:0/2151152107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b24f0 0x556acc5bf560 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7efe98043040 tx=0x7efe98005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8866b275fbb91a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.930+0000 7efe91600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:07:05.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.930+0000 7efe91600700 1 -- 172.21.15.131:0/2151152107 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe98004d70 con 0x556acc5b24f0 2024-05-08T22:07:05.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.931+0000 7efec23dab80 1 -- 172.21.15.131:0/2151152107 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efea00b7e90 con 0x556acc5b24f0 2024-05-08T22:07:05.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.931+0000 7efe91600700 1 -- 172.21.15.131:0/2151152107 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7efe9800f540 con 0x556acc5b24f0 2024-05-08T22:07:05.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.931+0000 7efe91600700 1 -- 172.21.15.131:0/2151152107 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe98004ed0 con 0x556acc5b24f0 2024-05-08T22:07:05.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.931+0000 7efec23dab80 10 client.15306 fetch_fsmap learned FSMap version 23 2024-05-08T22:07:05.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.931+0000 7efec23dab80 1 -- 172.21.15.131:0/2151152107 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556acc4c8510 con 0x556acc5b24f0 2024-05-08T22:07:05.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.931+0000 7efe91600700 1 -- 172.21.15.131:0/2151152107 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efea00b7e90 con 0x556acc5b24f0 2024-05-08T22:07:05.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.931+0000 7efe91600700 1 -- 172.21.15.131:0/2151152107 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efe980053d0 con 0x556acc5b24f0 2024-05-08T22:07:05.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.931+0000 7efec23dab80 10 client.15306 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T22:07:05.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.931+0000 7efec23dab80 10 client.15306 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-08T22:07:05.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.931+0000 7efec23dab80 20 client.15306 populate_metadata read hostname 'smithi131' 2024-05-08T22:07:05.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.931+0000 7efec23dab80 1 --2- 172.21.15.131:0/2151152107 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x556acc6220e0 0x556acc61eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:05.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.931+0000 7efec23dab80 4 client.15306 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:07:05.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.931+0000 7efec23dab80 1 -- 172.21.15.131:0/2151152107 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556acc4fae70 con 0x556acc6220e0 2024-05-08T22:07:05.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.932+0000 7efea6a00700 1 --2- 172.21.15.131:0/2151152107 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x556acc6220e0 0x556acc61eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:05.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.932+0000 7efea6a00700 1 --2- 172.21.15.131:0/2151152107 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x556acc6220e0 0x556acc61eaa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8856a916419579c0 server_cookie=5bc6c463bdf555a5 in_seq=0 out_seq=0 2024-05-08T22:07:05.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.932+0000 7efe91600700 10 client.15306 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:07:05.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.932+0000 7efe91600700 1 -- 172.21.15.131:0/2151152107 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x556acc4fae70 con 0x556acc6220e0 2024-05-08T22:07:05.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.932+0000 7efe91600700 10 client.15306 handle_command_reply: tid=0 2024-05-08T22:07:05.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.932+0000 7efec23dab80 1 client.15306 shutdown 2024-05-08T22:07:05.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.932+0000 7efec23dab80 1 -- 172.21.15.131:0/2151152107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b24f0 msgr2=0x556acc5bf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:05.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.932+0000 7efec23dab80 1 --2- 172.21.15.131:0/2151152107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b24f0 0x556acc5bf560 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7efe98043040 tx=0x7efe98005a40 comp rx=0 tx=0).stop 2024-05-08T22:07:05.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.932+0000 7efec23dab80 1 -- 172.21.15.131:0/2151152107 shutdown_connections 2024-05-08T22:07:05.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.932+0000 7efec23dab80 1 --2- 172.21.15.131:0/2151152107 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x556acc6220e0 0x556acc61eaa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.932+0000 7efec23dab80 1 --2- 172.21.15.131:0/2151152107 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acc5b3810 0x556acc5c2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.933+0000 7efe91600700 0 client.15306 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:07:05.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.933+0000 7efec23dab80 1 --2- 172.21.15.131:0/2151152107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556acc5b24f0 0x556acc5bf560 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.933+0000 7efec23dab80 1 --2- 172.21.15.131:0/2151152107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556acc5b2e50 0x556acc5bfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.933+0000 7efec23dab80 1 -- 172.21.15.131:0/2151152107 >> 172.21.15.131:0/2151152107 conn(0x556acc5101c0 msgr2=0x556acc5d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:05.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.933+0000 7efec23dab80 1 -- 172.21.15.131:0/2151152107 shutdown_connections 2024-05-08T22:07:05.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.933+0000 7efec23dab80 1 -- 172.21.15.131:0/2151152107 wait complete. 2024-05-08T22:07:05.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.933+0000 7efec23dab80 20 client.15306 trim_cache size 0 max 16384 2024-05-08T22:07:05.939 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:07:05.939 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "06f1a696-0d86-11ef-bc97-c7b262605968", 2024-05-08T22:07:05.939 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 2, 2024-05-08T22:07:05.939 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24550, 2024-05-08T22:07:05.939 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-08T22:07:05.940 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-08T22:07:05.940 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-08T22:07:05.940 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 87.44647046, 2024-05-08T22:07:05.940 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 23, 2024-05-08T22:07:05.940 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 87, 2024-05-08T22:07:05.940 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 83, 2024-05-08T22:07:05.940 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 122.32096146400001 2024-05-08T22:07:05.940 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:07:05.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.933+0000 7efeae800700 1 -- 172.21.15.131:0/2961500487 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7efe8c0a11b0 msgr2=0x7efe8c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:05.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.933+0000 7efeae800700 1 --2- 172.21.15.131:0/2961500487 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7efe8c0a11b0 0x7efe8c0a3670 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7efe9800b2e0 tx=0x7efe9800b910 comp rx=0 tx=0).stop 2024-05-08T22:07:05.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.933+0000 7efeae800700 1 -- 172.21.15.131:0/2961500487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efea8106c40 msgr2=0x7efea81a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:05.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.933+0000 7efeae800700 1 --2- 172.21.15.131:0/2961500487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efea8106c40 0x7efea81a0650 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efea000ef30 tx=0x7efea000c3a0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.933+0000 7efeae800700 1 -- 172.21.15.131:0/2961500487 shutdown_connections 2024-05-08T22:07:05.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.933+0000 7efeae800700 1 --2- 172.21.15.131:0/2961500487 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7efe8c0a11b0 0x7efe8c0a3670 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.933+0000 7efeae800700 1 --2- 172.21.15.131:0/2961500487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efea8106c40 0x7efea81a0650 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.933+0000 7efeae800700 1 --2- 172.21.15.131:0/2961500487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efea81019e0 0x7efea81990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.933+0000 7efeae800700 1 --2- 172.21.15.131:0/2961500487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efea8104310 0x7efea8199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:05.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.933+0000 7efeae800700 1 -- 172.21.15.131:0/2961500487 >> 172.21.15.131:0/2961500487 conn(0x7efea80fb3f0 msgr2=0x7efea8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:05.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.934+0000 7efeae800700 1 -- 172.21.15.131:0/2961500487 shutdown_connections 2024-05-08T22:07:05.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:05.934+0000 7efeae800700 1 -- 172.21.15.131:0/2961500487 wait complete. 2024-05-08T22:07:05.951 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:05 smithi131 ceph-mon[37260]: pgmap v299: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:07:05.951 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:05 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3118968265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:07:05.951 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:05 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/4134166820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:07:05.953 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-08T22:07:06.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.151+0000 7ff03ea00700 1 -- 172.21.15.131:0/2999096603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff038074fe0 msgr2=0x7ff038073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:06.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.151+0000 7ff03ea00700 1 --2- 172.21.15.131:0/2999096603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff038074fe0 0x7ff038073430 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff0280098f0 tx=0x7ff028009c00 comp rx=0 tx=0).stop 2024-05-08T22:07:06.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.151+0000 7ff03ea00700 1 -- 172.21.15.131:0/2999096603 shutdown_connections 2024-05-08T22:07:06.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.151+0000 7ff03ea00700 1 --2- 172.21.15.131:0/2999096603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff038108e40 0x7ff03810b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.151+0000 7ff03ea00700 1 --2- 172.21.15.131:0/2999096603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff038074fe0 0x7ff038073430 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.151+0000 7ff03ea00700 1 --2- 172.21.15.131:0/2999096603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff038073a00 0x7ff038073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.151+0000 7ff03ea00700 1 -- 172.21.15.131:0/2999096603 >> 172.21.15.131:0/2999096603 conn(0x7ff0380fc4f0 msgr2=0x7ff0380fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:06.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.151+0000 7ff03ea00700 1 -- 172.21.15.131:0/2999096603 shutdown_connections 2024-05-08T22:07:06.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.152+0000 7ff03ea00700 1 -- 172.21.15.131:0/2999096603 wait complete. 2024-05-08T22:07:06.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.153+0000 7ff03ea00700 1 Processor -- start 2024-05-08T22:07:06.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.153+0000 7ff03ea00700 1 -- start start 2024-05-08T22:07:06.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.153+0000 7ff03ea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff038073a00 0x7ff038199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.153+0000 7ff03ea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff038074fe0 0x7ff038199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.154+0000 7ff03ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff038108e40 0x7ff0381a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.154+0000 7ff03ea00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0381a0c80 con 0x7ff038074fe0 2024-05-08T22:07:06.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.154+0000 7ff03ea00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0381a0dc0 con 0x7ff038073a00 2024-05-08T22:07:06.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.154+0000 7ff03ea00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0381a0f50 con 0x7ff038108e40 2024-05-08T22:07:06.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.154+0000 7ff037400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff038074fe0 0x7ff038199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.154+0000 7ff037e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff038073a00 0x7ff038199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.154+0000 7ff037e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff038073a00 0x7ff038199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:37024/0 (socket says 172.21.15.131:37024) 2024-05-08T22:07:06.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.154+0000 7ff037e00700 1 -- 172.21.15.131:0/3682091356 learned_addr learned my addr 172.21.15.131:0/3682091356 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:06.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.154+0000 7ff037400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff038074fe0 0x7ff038199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38946/0 (socket says 172.21.15.131:38946) 2024-05-08T22:07:06.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.154+0000 7ff03ca00700 1 --2- 172.21.15.131:0/3682091356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff038108e40 0x7ff0381a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.154+0000 7ff037400700 1 -- 172.21.15.131:0/3682091356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff038108e40 msgr2=0x7ff0381a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:06.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.154+0000 7ff037400700 1 --2- 172.21.15.131:0/3682091356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff038108e40 0x7ff0381a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.154+0000 7ff037400700 1 -- 172.21.15.131:0/3682091356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff038073a00 msgr2=0x7ff038199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:06.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.154+0000 7ff037400700 1 --2- 172.21.15.131:0/3682091356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff038073a00 0x7ff038199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.154+0000 7ff037400700 1 -- 172.21.15.131:0/3682091356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff028009520 con 0x7ff038074fe0 2024-05-08T22:07:06.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.154+0000 7ff037400700 1 --2- 172.21.15.131:0/3682091356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff038074fe0 0x7ff038199690 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff02400e9c0 tx=0x7ff02400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:06.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.155+0000 7ff034c00700 1 -- 172.21.15.131:0/3682091356 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff024004220 con 0x7ff038074fe0 2024-05-08T22:07:06.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.155+0000 7ff03ea00700 1 -- 172.21.15.131:0/3682091356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0381a1230 con 0x7ff038074fe0 2024-05-08T22:07:06.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.155+0000 7ff034c00700 1 -- 172.21.15.131:0/3682091356 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff024004380 con 0x7ff038074fe0 2024-05-08T22:07:06.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.155+0000 7ff034c00700 1 -- 172.21.15.131:0/3682091356 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff024003bc0 con 0x7ff038074fe0 2024-05-08T22:07:06.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.155+0000 7ff03ea00700 1 -- 172.21.15.131:0/3682091356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0381a1790 con 0x7ff038074fe0 2024-05-08T22:07:06.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.158+0000 7ff034c00700 1 -- 172.21.15.131:0/3682091356 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7ff024003d20 con 0x7ff038074fe0 2024-05-08T22:07:06.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.159+0000 7ff0524c7b80 1 Processor -- start 2024-05-08T22:07:06.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.159+0000 7ff0524c7b80 1 -- start start 2024-05-08T22:07:06.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.159+0000 7ff034c00700 1 --2- 172.21.15.131:0/3682091356 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff01c0a11b0 0x7ff01c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.159+0000 7ff0524c7b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565038f0a4f0 0x565038f0a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.159+0000 7ff0524c7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565038f0ae50 0x565038f0b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.159+0000 7ff0524c7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565038f0b810 0x565038f0bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.159+0000 7ff0524c7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x565038e61100 con 0x565038f0b810 2024-05-08T22:07:06.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.159+0000 7ff0524c7b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x565038e5eac0 con 0x565038f0a4f0 2024-05-08T22:07:06.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.159+0000 7ff0524c7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x565038e5a3b0 con 0x565038f0ae50 2024-05-08T22:07:06.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.159+0000 7ff034c00700 1 -- 172.21.15.131:0/3682091356 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7ff024014070 con 0x7ff038074fe0 2024-05-08T22:07:06.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.160+0000 7ff03ca00700 1 --2- 172.21.15.131:0/3682091356 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff01c0a11b0 0x7ff01c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.160+0000 7ff037400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565038f0ae50 0x565038f0b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.160+0000 7ff037e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565038f0a4f0 0x565038f0a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.160+0000 7ff03ca00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565038f0b810 0x565038f0bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.160+0000 7ff037e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565038f0a4f0 0x565038f0a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:37028/0 (socket says 172.21.15.131:37028) 2024-05-08T22:07:06.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.160+0000 7ff037400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565038f0ae50 0x565038f0b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55614/0 (socket says 172.21.15.131:55614) 2024-05-08T22:07:06.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.160+0000 7ff037e00700 1 -- 172.21.15.131:0/2484664753 learned_addr learned my addr 172.21.15.131:0/2484664753 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:06.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.160+0000 7ff037400700 1 -- 172.21.15.131:0/2484664753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565038f0a4f0 msgr2=0x565038f0a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:06.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.160+0000 7ff037400700 1 --2- 172.21.15.131:0/2484664753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565038f0a4f0 0x565038f0a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.160+0000 7ff037400700 1 -- 172.21.15.131:0/2484664753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565038f0b810 msgr2=0x565038f0bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:06.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.160+0000 7ff037400700 1 --2- 172.21.15.131:0/2484664753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565038f0b810 0x565038f0bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.160+0000 7ff037400700 1 -- 172.21.15.131:0/2484664753 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565038cdb360 con 0x565038f0ae50 2024-05-08T22:07:06.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.160+0000 7ff037e00700 1 --2- 172.21.15.131:0/2484664753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565038f0a4f0 0x565038f0a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:06.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.160+0000 7ff037400700 1 --2- 172.21.15.131:0/2484664753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565038f0ae50 0x565038f0b2d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff0240ca620 tx=0x7ff024099e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86890afb043df319 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:06.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.160+0000 7ff03ca00700 1 --2- 172.21.15.131:0/3682091356 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff01c0a11b0 0x7ff01c0a3670 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff030008be0 tx=0x7ff0300092d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:06.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.161+0000 7ff02d200700 1 -- 172.21.15.131:0/2484664753 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02409a690 con 0x565038f0ae50 2024-05-08T22:07:06.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.161+0000 7ff02d200700 1 -- 172.21.15.131:0/2484664753 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff024099f90 con 0x565038f0ae50 2024-05-08T22:07:06.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.161+0000 7ff02d200700 1 -- 172.21.15.131:0/2484664753 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff024098330 con 0x565038f0ae50 2024-05-08T22:07:06.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.161+0000 7ff0524c7b80 1 -- 172.21.15.131:0/2484664753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565038f0ae50 msgr2=0x565038f0b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:06.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.161+0000 7ff0524c7b80 1 --2- 172.21.15.131:0/2484664753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565038f0ae50 0x565038f0b2d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff0240ca620 tx=0x7ff024099e90 comp rx=0 tx=0).stop 2024-05-08T22:07:06.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.161+0000 7ff0524c7b80 1 -- 172.21.15.131:0/2484664753 shutdown_connections 2024-05-08T22:07:06.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.161+0000 7ff0524c7b80 1 --2- 172.21.15.131:0/2484664753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565038f0b810 0x565038f0bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.161+0000 7ff0524c7b80 1 --2- 172.21.15.131:0/2484664753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565038f0a4f0 0x565038f0a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.161+0000 7ff0524c7b80 1 --2- 172.21.15.131:0/2484664753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565038f0ae50 0x565038f0b2d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.161+0000 7ff0524c7b80 1 -- 172.21.15.131:0/2484664753 >> 172.21.15.131:0/2484664753 conn(0x565038e681c0 msgr2=0x565038f04d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:06.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.161+0000 7ff0524c7b80 1 -- 172.21.15.131:0/2484664753 shutdown_connections 2024-05-08T22:07:06.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.162+0000 7ff0524c7b80 1 -- 172.21.15.131:0/2484664753 wait complete. 2024-05-08T22:07:06.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.162+0000 7ff0524c7b80 1 -- start start 2024-05-08T22:07:06.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.162+0000 7ff0524c7b80 1 Processor -- start 2024-05-08T22:07:06.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.163+0000 7ff0524c7b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565038f0a4f0 0x565038f13db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.163+0000 7ff0524c7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565038f0ae50 0x565038f142f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.163+0000 7ff0524c7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565038f0b810 0x565038f1b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.163+0000 7ff0524c7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x565038df6510 con 0x565038f0ae50 2024-05-08T22:07:06.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.163+0000 7ff0524c7b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x565038df61b0 con 0x565038f0a4f0 2024-05-08T22:07:06.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.163+0000 7ff0524c7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x565038cdd650 con 0x565038f0b810 2024-05-08T22:07:06.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.163+0000 7ff0524c7b80 10 client.0 _ll_register_callbacks cb 0x565038dffae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:07:06.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.163+0000 7ff037400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565038f0ae50 0x565038f142f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.163+0000 7ff03ca00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565038f0b810 0x565038f1b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.163+0000 7ff037e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565038f0a4f0 0x565038f13db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.163+0000 7ff037e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565038f0a4f0 0x565038f13db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:37042/0 (socket says 172.21.15.131:37042) 2024-05-08T22:07:06.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.163+0000 7ff037e00700 1 -- 172.21.15.131:0/331993192 learned_addr learned my addr 172.21.15.131:0/331993192 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:06.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.163+0000 7ff03ca00700 1 -- 172.21.15.131:0/331993192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565038f0a4f0 msgr2=0x565038f13db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:06.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.163+0000 7ff03ca00700 1 --2- 172.21.15.131:0/331993192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565038f0a4f0 0x565038f13db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.163+0000 7ff03ca00700 1 -- 172.21.15.131:0/331993192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565038f0ae50 msgr2=0x565038f142f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:06.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.163+0000 7ff03ca00700 1 --2- 172.21.15.131:0/331993192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565038f0ae50 0x565038f142f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.163+0000 7ff03ca00700 1 -- 172.21.15.131:0/331993192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03810bef0 con 0x565038f0b810 2024-05-08T22:07:06.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.163+0000 7ff037400700 1 --2- 172.21.15.131:0/331993192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565038f0ae50 0x565038f142f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:06.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.164+0000 7ff03ca00700 1 --2- 172.21.15.131:0/331993192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565038f0b810 0x565038f1b330 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x565038f05cf0 tx=0x7ff03000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13f9a8a487774331 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:06.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.164+0000 7ff021600700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:07:06.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.164+0000 7ff021600700 1 -- 172.21.15.131:0/331993192 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03000e070 con 0x565038f0b810 2024-05-08T22:07:06.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.164+0000 7ff0524c7b80 1 -- 172.21.15.131:0/331993192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565038db3220 con 0x565038f0b810 2024-05-08T22:07:06.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.164+0000 7ff021600700 1 -- 172.21.15.131:0/331993192 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7ff03001aac0 con 0x565038f0b810 2024-05-08T22:07:06.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.164+0000 7ff021600700 1 -- 172.21.15.131:0/331993192 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff03001ac80 con 0x565038f0b810 2024-05-08T22:07:06.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.164+0000 7ff0524c7b80 10 client.25360 fetch_fsmap learned FSMap version 23 2024-05-08T22:07:06.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.164+0000 7ff0524c7b80 1 -- 172.21.15.131:0/331993192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7ff0381a1790 con 0x565038f0b810 2024-05-08T22:07:06.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.164+0000 7ff021600700 1 -- 172.21.15.131:0/331993192 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565038db3220 con 0x565038f0b810 2024-05-08T22:07:06.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.165+0000 7ff021600700 1 -- 172.21.15.131:0/331993192 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff0300074a0 con 0x565038f0b810 2024-05-08T22:07:06.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.165+0000 7ff0524c7b80 10 client.25360 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T22:07:06.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.165+0000 7ff0524c7b80 10 client.25360 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T22:07:06.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.165+0000 7ff0524c7b80 20 client.25360 populate_metadata read hostname 'smithi131' 2024-05-08T22:07:06.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.165+0000 7ff0524c7b80 1 --2- 172.21.15.131:0/331993192 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x565038f1efc0 0x565038f242a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.165+0000 7ff0524c7b80 4 client.25360 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:07:06.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.165+0000 7ff0524c7b80 1 -- 172.21.15.131:0/331993192 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565038ed97c0 con 0x565038f1efc0 2024-05-08T22:07:06.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.165+0000 7ff037e00700 1 --2- 172.21.15.131:0/331993192 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x565038f1efc0 0x565038f242a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.166+0000 7ff037e00700 1 --2- 172.21.15.131:0/331993192 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x565038f1efc0 0x565038f242a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=716b28489e987b0a server_cookie=b04562d9f5a6e304 in_seq=0 out_seq=0 2024-05-08T22:07:06.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.166+0000 7ff021600700 10 client.25360 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:07:06.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.166+0000 7ff021600700 1 -- 172.21.15.131:0/331993192 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565038ed97c0 con 0x565038f1efc0 2024-05-08T22:07:06.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.166+0000 7ff021600700 10 client.25360 handle_command_reply: tid=0 2024-05-08T22:07:06.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.166+0000 7ff0524c7b80 1 client.25360 shutdown 2024-05-08T22:07:06.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.166+0000 7ff0524c7b80 1 -- 172.21.15.131:0/331993192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565038f0b810 msgr2=0x565038f1b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:06.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.166+0000 7ff0524c7b80 1 --2- 172.21.15.131:0/331993192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565038f0b810 0x565038f1b330 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x565038f05cf0 tx=0x7ff03000b040 comp rx=0 tx=0).stop 2024-05-08T22:07:06.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.167+0000 7ff0524c7b80 1 -- 172.21.15.131:0/331993192 shutdown_connections 2024-05-08T22:07:06.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.167+0000 7ff0524c7b80 1 --2- 172.21.15.131:0/331993192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565038f0b810 0x565038f1b330 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.167+0000 7ff0524c7b80 1 --2- 172.21.15.131:0/331993192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565038f0a4f0 0x565038f13db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.167+0000 7ff0524c7b80 1 --2- 172.21.15.131:0/331993192 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x565038f1efc0 0x565038f242a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.167+0000 7ff0524c7b80 1 --2- 172.21.15.131:0/331993192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565038f0ae50 0x565038f142f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.167+0000 7ff0524c7b80 1 -- 172.21.15.131:0/331993192 >> 172.21.15.131:0/331993192 conn(0x565038e681c0 msgr2=0x565038f04550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:06.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.167+0000 7ff021600700 0 client.25360 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:07:06.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.167+0000 7ff0524c7b80 1 -- 172.21.15.131:0/331993192 shutdown_connections 2024-05-08T22:07:06.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.167+0000 7ff0524c7b80 1 -- 172.21.15.131:0/331993192 wait complete. 2024-05-08T22:07:06.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.167+0000 7ff0524c7b80 20 client.25360 trim_cache size 0 max 16384 2024-05-08T22:07:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.185+0000 7ff0524c7b80 1 Processor -- start 2024-05-08T22:07:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.186+0000 7ff0524c7b80 1 -- start start 2024-05-08T22:07:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.186+0000 7ff0524c7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565038f0a4f0 0x565038f0a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.186+0000 7ff0524c7b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565038f0ae50 0x565038f1b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.186+0000 7ff0524c7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565038f0b810 0x565038f11da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.186+0000 7ff0524c7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x565038db3220 con 0x565038f0a4f0 2024-05-08T22:07:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.186+0000 7ff0524c7b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x565038f1e720 con 0x565038f0ae50 2024-05-08T22:07:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.186+0000 7ff0524c7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x565038e53150 con 0x565038f0b810 2024-05-08T22:07:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.186+0000 7ff037e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565038f0a4f0 0x565038f0a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.186+0000 7ff037e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565038f0a4f0 0x565038f0a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38970/0 (socket says 172.21.15.131:38970) 2024-05-08T22:07:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.186+0000 7ff037e00700 1 -- 172.21.15.131:0/4262429366 learned_addr learned my addr 172.21.15.131:0/4262429366 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:06.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.186+0000 7ff03ca00700 1 --2- 172.21.15.131:0/4262429366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565038f0b810 0x565038f11da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.186+0000 7ff037400700 1 --2- 172.21.15.131:0/4262429366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565038f0ae50 0x565038f1b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.186+0000 7ff037e00700 1 -- 172.21.15.131:0/4262429366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565038f0b810 msgr2=0x565038f11da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:06.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.186+0000 7ff037e00700 1 --2- 172.21.15.131:0/4262429366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565038f0b810 0x565038f11da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.186+0000 7ff037e00700 1 -- 172.21.15.131:0/4262429366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565038f0ae50 msgr2=0x565038f1b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:06.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.186+0000 7ff037e00700 1 --2- 172.21.15.131:0/4262429366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565038f0ae50 0x565038f1b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.186+0000 7ff037e00700 1 -- 172.21.15.131:0/4262429366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565038c87b10 con 0x565038f0a4f0 2024-05-08T22:07:06.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.187+0000 7ff037e00700 1 --2- 172.21.15.131:0/4262429366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565038f0a4f0 0x565038f0a8d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff02800a010 tx=0x7ff028005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0eb3f1be54e02b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:06.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.187+0000 7ff02dc00700 1 -- 172.21.15.131:0/4262429366 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02800b560 con 0x565038f0a4f0 2024-05-08T22:07:06.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.187+0000 7ff02dc00700 1 -- 172.21.15.131:0/4262429366 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff02801d040 con 0x565038f0a4f0 2024-05-08T22:07:06.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.187+0000 7ff02dc00700 1 -- 172.21.15.131:0/4262429366 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff028003680 con 0x565038f0a4f0 2024-05-08T22:07:06.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.188+0000 7ff0524c7b80 1 -- 172.21.15.131:0/4262429366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565038f0a4f0 msgr2=0x565038f0a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:06.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.188+0000 7ff0524c7b80 1 --2- 172.21.15.131:0/4262429366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565038f0a4f0 0x565038f0a8d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff02800a010 tx=0x7ff028005e20 comp rx=0 tx=0).stop 2024-05-08T22:07:06.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.188+0000 7ff0524c7b80 1 -- 172.21.15.131:0/4262429366 shutdown_connections 2024-05-08T22:07:06.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.188+0000 7ff0524c7b80 1 --2- 172.21.15.131:0/4262429366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565038f0b810 0x565038f11da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.188+0000 7ff0524c7b80 1 --2- 172.21.15.131:0/4262429366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565038f0a4f0 0x565038f0a8d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.188+0000 7ff0524c7b80 1 --2- 172.21.15.131:0/4262429366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565038f0ae50 0x565038f1b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.188+0000 7ff0524c7b80 1 -- 172.21.15.131:0/4262429366 >> 172.21.15.131:0/4262429366 conn(0x565038e681c0 msgr2=0x565038f0bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:06.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.188+0000 7ff0524c7b80 1 -- 172.21.15.131:0/4262429366 shutdown_connections 2024-05-08T22:07:06.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.188+0000 7ff0524c7b80 1 -- 172.21.15.131:0/4262429366 wait complete. 2024-05-08T22:07:06.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.188+0000 7ff0524c7b80 1 -- start start 2024-05-08T22:07:06.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.188+0000 7ff0524c7b80 1 Processor -- start 2024-05-08T22:07:06.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.189+0000 7ff0524c7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565038f0a4f0 0x565038f18280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.189+0000 7ff0524c7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565038f0ae50 0x565038f1a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.189+0000 7ff0524c7b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565038f0b810 0x565038f1ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.189+0000 7ff0524c7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x565038f1b250 con 0x565038f0ae50 2024-05-08T22:07:06.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.189+0000 7ff0524c7b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x565038f1b3c0 con 0x565038f0b810 2024-05-08T22:07:06.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.189+0000 7ff0524c7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x565038f1b580 con 0x565038f0a4f0 2024-05-08T22:07:06.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.189+0000 7ff0524c7b80 10 client.0 _ll_register_callbacks cb 0x565038dffae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:07:06.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.189+0000 7ff037400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565038f0ae50 0x565038f1a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.189+0000 7ff037e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565038f0a4f0 0x565038f18280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.189+0000 7ff037400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565038f0ae50 0x565038f1a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38986/0 (socket says 172.21.15.131:38986) 2024-05-08T22:07:06.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.189+0000 7ff037400700 1 -- 172.21.15.131:0/2878306813 learned_addr learned my addr 172.21.15.131:0/2878306813 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:06.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.189+0000 7ff03ca00700 1 --2- 172.21.15.131:0/2878306813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565038f0b810 0x565038f1ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.189+0000 7ff037e00700 1 -- 172.21.15.131:0/2878306813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565038f0b810 msgr2=0x565038f1ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:06.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.189+0000 7ff037e00700 1 --2- 172.21.15.131:0/2878306813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565038f0b810 0x565038f1ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.189+0000 7ff037e00700 1 -- 172.21.15.131:0/2878306813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565038f0ae50 msgr2=0x565038f1a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:06.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.189+0000 7ff037e00700 1 --2- 172.21.15.131:0/2878306813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565038f0ae50 0x565038f1a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.189+0000 7ff037e00700 1 -- 172.21.15.131:0/2878306813 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff02800f840 con 0x565038f0a4f0 2024-05-08T22:07:06.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.189+0000 7ff037400700 1 --2- 172.21.15.131:0/2878306813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565038f0ae50 0x565038f1a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:06.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.190+0000 7ff037e00700 1 --2- 172.21.15.131:0/2878306813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565038f0a4f0 0x565038f18280 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff028009c40 tx=0x7ff028005730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db8337771953a0b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:06.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.190+0000 7ff021600700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:07:06.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.190+0000 7ff021600700 1 -- 172.21.15.131:0/2878306813 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02800b760 con 0x565038f0a4f0 2024-05-08T22:07:06.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.190+0000 7ff021600700 1 -- 172.21.15.131:0/2878306813 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7ff02800b8c0 con 0x565038f0a4f0 2024-05-08T22:07:06.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.190+0000 7ff0524c7b80 1 -- 172.21.15.131:0/2878306813 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff024099f90 con 0x565038f0a4f0 2024-05-08T22:07:06.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.190+0000 7ff021600700 1 -- 172.21.15.131:0/2878306813 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff02800ba80 con 0x565038f0a4f0 2024-05-08T22:07:06.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.190+0000 7ff0524c7b80 10 client.25366 fetch_fsmap learned FSMap version 23 2024-05-08T22:07:06.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.190+0000 7ff0524c7b80 1 -- 172.21.15.131:0/2878306813 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x565038e20510 con 0x565038f0a4f0 2024-05-08T22:07:06.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.191+0000 7ff021600700 1 -- 172.21.15.131:0/2878306813 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff024099f90 con 0x565038f0a4f0 2024-05-08T22:07:06.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.191+0000 7ff021600700 1 -- 172.21.15.131:0/2878306813 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff028009920 con 0x565038f0a4f0 2024-05-08T22:07:06.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.191+0000 7ff0524c7b80 10 client.25366 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T22:07:06.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.191+0000 7ff0524c7b80 10 client.25366 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T22:07:06.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.191+0000 7ff0524c7b80 20 client.25366 populate_metadata read hostname 'smithi131' 2024-05-08T22:07:06.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.191+0000 7ff0524c7b80 1 --2- 172.21.15.131:0/2878306813 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x565038f7a460 0x565038f76df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.191+0000 7ff0524c7b80 4 client.25366 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:07:06.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.191+0000 7ff0524c7b80 1 -- 172.21.15.131:0/2878306813 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "status"}) v1 -- 0x565038e52e70 con 0x565038f7a460 2024-05-08T22:07:06.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.191+0000 7ff037400700 1 --2- 172.21.15.131:0/2878306813 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x565038f7a460 0x565038f76df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.192+0000 7ff021600700 10 client.25366 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:07:06.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.192+0000 7ff037400700 1 --2- 172.21.15.131:0/2878306813 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x565038f7a460 0x565038f76df0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dbdcd93b1103afb7 server_cookie=3b20f18f732301ee in_seq=0 out_seq=0 2024-05-08T22:07:06.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.192+0000 7ff021600700 1 -- 172.21.15.131:0/2878306813 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x565038e52e70 con 0x565038f7a460 2024-05-08T22:07:06.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.192+0000 7ff021600700 10 client.25366 handle_command_reply: tid=0 2024-05-08T22:07:06.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.192+0000 7ff0524c7b80 1 client.25366 shutdown 2024-05-08T22:07:06.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.192+0000 7ff0524c7b80 1 -- 172.21.15.131:0/2878306813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565038f0a4f0 msgr2=0x565038f18280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:06.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.192+0000 7ff0524c7b80 1 --2- 172.21.15.131:0/2878306813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565038f0a4f0 0x565038f18280 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff028009c40 tx=0x7ff028005730 comp rx=0 tx=0).stop 2024-05-08T22:07:06.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.192+0000 7ff0524c7b80 1 -- 172.21.15.131:0/2878306813 shutdown_connections 2024-05-08T22:07:06.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.192+0000 7ff0524c7b80 1 --2- 172.21.15.131:0/2878306813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565038f0b810 0x565038f1ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.192+0000 7ff0524c7b80 1 --2- 172.21.15.131:0/2878306813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565038f0a4f0 0x565038f18280 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.192+0000 7ff0524c7b80 1 --2- 172.21.15.131:0/2878306813 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x565038f7a460 0x565038f76df0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.192+0000 7ff0524c7b80 1 --2- 172.21.15.131:0/2878306813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565038f0ae50 0x565038f1a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.192+0000 7ff0524c7b80 1 -- 172.21.15.131:0/2878306813 >> 172.21.15.131:0/2878306813 conn(0x565038e681c0 msgr2=0x565038f132c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:06.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.192+0000 7ff021600700 0 client.25366 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:07:06.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.192+0000 7ff0524c7b80 1 -- 172.21.15.131:0/2878306813 shutdown_connections 2024-05-08T22:07:06.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.192+0000 7ff0524c7b80 1 -- 172.21.15.131:0/2878306813 wait complete. 2024-05-08T22:07:06.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.193+0000 7ff0524c7b80 20 client.25366 trim_cache size 0 max 16384 2024-05-08T22:07:06.199 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:07:06.199 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "06f1a696-0d86-11ef-bc97-c7b262605968", 2024-05-08T22:07:06.199 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 1, 2024-05-08T22:07:06.199 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24562, 2024-05-08T22:07:06.199 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-08T22:07:06.200 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-08T22:07:06.200 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-08T22:07:06.200 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 88.711871138000006, 2024-05-08T22:07:06.200 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 23, 2024-05-08T22:07:06.200 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 87, 2024-05-08T22:07:06.200 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 83, 2024-05-08T22:07:06.200 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 120.172452607 2024-05-08T22:07:06.200 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:07:06.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.193+0000 7ff03ea00700 1 -- 172.21.15.131:0/3682091356 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff01c0a11b0 msgr2=0x7ff01c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:06.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.193+0000 7ff03ea00700 1 --2- 172.21.15.131:0/3682091356 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff01c0a11b0 0x7ff01c0a3670 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff030008be0 tx=0x7ff0300092d0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.193+0000 7ff03ea00700 1 -- 172.21.15.131:0/3682091356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff038074fe0 msgr2=0x7ff038199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:06.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.193+0000 7ff03ea00700 1 --2- 172.21.15.131:0/3682091356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff038074fe0 0x7ff038199690 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff02400e9c0 tx=0x7ff02400ecd0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.193+0000 7ff03ea00700 1 -- 172.21.15.131:0/3682091356 shutdown_connections 2024-05-08T22:07:06.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.193+0000 7ff03ea00700 1 --2- 172.21.15.131:0/3682091356 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7ff01c0a11b0 0x7ff01c0a3670 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.193+0000 7ff03ea00700 1 --2- 172.21.15.131:0/3682091356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff038108e40 0x7ff0381a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.193+0000 7ff03ea00700 1 --2- 172.21.15.131:0/3682091356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff038073a00 0x7ff038199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.193+0000 7ff03ea00700 1 --2- 172.21.15.131:0/3682091356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff038074fe0 0x7ff038199690 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.193+0000 7ff03ea00700 1 -- 172.21.15.131:0/3682091356 >> 172.21.15.131:0/3682091356 conn(0x7ff0380fc4f0 msgr2=0x7ff038104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:06.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.193+0000 7ff03ea00700 1 -- 172.21.15.131:0/3682091356 shutdown_connections 2024-05-08T22:07:06.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.194+0000 7ff03ea00700 1 -- 172.21.15.131:0/3682091356 wait complete. 2024-05-08T22:07:06.213 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T22:07:06.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:05 smithi157 ceph-mon[77746]: pgmap v299: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:07:06.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:05 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3118968265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:07:06.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:05 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/4134166820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:07:06.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:05 smithi132 ceph-mon[78180]: pgmap v299: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:07:06.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:05 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3118968265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:07:06.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:05 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/4134166820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:07:06.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.405+0000 7f166aa00700 1 -- 172.21.15.131:0/240425714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16641020f0 msgr2=0x7f1664102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:06.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.405+0000 7f166aa00700 1 --2- 172.21.15.131:0/240425714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16641020f0 0x7f1664102570 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f164c009990 tx=0x7f164c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.405+0000 7f166aa00700 1 -- 172.21.15.131:0/240425714 shutdown_connections 2024-05-08T22:07:06.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.405+0000 7f166aa00700 1 --2- 172.21.15.131:0/240425714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1664102ab0 0x7f16641093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.405+0000 7f166aa00700 1 --2- 172.21.15.131:0/240425714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1664100f20 0x7f1664101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.405+0000 7f166aa00700 1 --2- 172.21.15.131:0/240425714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16641020f0 0x7f1664102570 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.405+0000 7f166aa00700 1 -- 172.21.15.131:0/240425714 >> 172.21.15.131:0/240425714 conn(0x7f16640fc500 msgr2=0x7f16640fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:06.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.405+0000 7f166aa00700 1 -- 172.21.15.131:0/240425714 shutdown_connections 2024-05-08T22:07:06.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.405+0000 7f166aa00700 1 -- 172.21.15.131:0/240425714 wait complete. 2024-05-08T22:07:06.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.406+0000 7f166aa00700 1 Processor -- start 2024-05-08T22:07:06.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.406+0000 7f166aa00700 1 -- start start 2024-05-08T22:07:06.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.406+0000 7f166aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1664100f20 0x7f1664199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.406+0000 7f166aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16641020f0 0x7f16641996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.406+0000 7f166aa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1664102ab0 0x7f16641a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.406+0000 7f166aa00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16641a0ca0 con 0x7f16641020f0 2024-05-08T22:07:06.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.406+0000 7f166aa00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16641a0de0 con 0x7f1664102ab0 2024-05-08T22:07:06.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.406+0000 7f166aa00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16641a0f70 con 0x7f1664100f20 2024-05-08T22:07:06.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.407+0000 7f165be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16641020f0 0x7f16641996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.407+0000 7f165be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16641020f0 0x7f16641996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39004/0 (socket says 172.21.15.131:39004) 2024-05-08T22:07:06.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.407+0000 7f165be00700 1 -- 172.21.15.131:0/1244961269 learned_addr learned my addr 172.21.15.131:0/1244961269 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:06.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.407+0000 7f1668a00700 1 --2- 172.21.15.131:0/1244961269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1664102ab0 0x7f16641a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.407+0000 7f1663e00700 1 --2- 172.21.15.131:0/1244961269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1664100f20 0x7f1664199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.407+0000 7f165be00700 1 -- 172.21.15.131:0/1244961269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1664100f20 msgr2=0x7f1664199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:06.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.407+0000 7f165be00700 1 --2- 172.21.15.131:0/1244961269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1664100f20 0x7f1664199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.407+0000 7f165be00700 1 -- 172.21.15.131:0/1244961269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1664102ab0 msgr2=0x7f16641a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:06.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.407+0000 7f165be00700 1 --2- 172.21.15.131:0/1244961269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1664102ab0 0x7f16641a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.407+0000 7f165be00700 1 -- 172.21.15.131:0/1244961269 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164c0095f0 con 0x7f16641020f0 2024-05-08T22:07:06.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.407+0000 7f165be00700 1 --2- 172.21.15.131:0/1244961269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16641020f0 0x7f16641996b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f164c005910 tx=0x7f164c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:06.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.407+0000 7f1661600700 1 -- 172.21.15.131:0/1244961269 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c01d070 con 0x7f16641020f0 2024-05-08T22:07:06.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.408+0000 7f1661600700 1 -- 172.21.15.131:0/1244961269 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f164c021410 con 0x7f16641020f0 2024-05-08T22:07:06.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.408+0000 7f166aa00700 1 -- 172.21.15.131:0/1244961269 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16641a11f0 con 0x7f16641020f0 2024-05-08T22:07:06.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.408+0000 7f1661600700 1 -- 172.21.15.131:0/1244961269 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c017650 con 0x7f16641020f0 2024-05-08T22:07:06.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.408+0000 7f166aa00700 1 -- 172.21.15.131:0/1244961269 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16641a1670 con 0x7f16641020f0 2024-05-08T22:07:06.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.411+0000 7f1661600700 1 -- 172.21.15.131:0/1244961269 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f164c021580 con 0x7f16641020f0 2024-05-08T22:07:06.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.412+0000 7f1661600700 1 --2- 172.21.15.131:0/1244961269 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f16480a1230 0x7f16480a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.412+0000 7f167e5bfb80 1 Processor -- start 2024-05-08T22:07:06.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.412+0000 7f167e5bfb80 1 -- start start 2024-05-08T22:07:06.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.412+0000 7f1661600700 1 -- 172.21.15.131:0/1244961269 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f164c0ce140 con 0x7f16641020f0 2024-05-08T22:07:06.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.412+0000 7f1663e00700 1 --2- 172.21.15.131:0/1244961269 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f16480a1230 0x7f16480a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.412+0000 7f167e5bfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573dbcb84f0 0x5573dbcb8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.413+0000 7f167e5bfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573dbcb8e50 0x5573dbcb92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.413+0000 7f167e5bfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb9810 0x5573dbcb9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.413+0000 7f167e5bfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573dbc0f100 con 0x5573dbcb9810 2024-05-08T22:07:06.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.413+0000 7f167e5bfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573dbc0cac0 con 0x5573dbcb8e50 2024-05-08T22:07:06.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.413+0000 7f167e5bfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573dbc083b0 con 0x5573dbcb84f0 2024-05-08T22:07:06.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.413+0000 7f1668a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb9810 0x5573dbcb9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.413+0000 7f1668a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb9810 0x5573dbcb9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39008/0 (socket says 172.21.15.131:39008) 2024-05-08T22:07:06.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.413+0000 7f1668a00700 1 -- 172.21.15.131:0/3819968540 learned_addr learned my addr 172.21.15.131:0/3819968540 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:06.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.413+0000 7f165be00700 1 --2- 172.21.15.131:0/3819968540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573dbcb8e50 0x5573dbcb92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.413+0000 7f1663e00700 1 --2- 172.21.15.131:0/3819968540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573dbcb84f0 0x5573dbcb8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.413+0000 7f1663e00700 1 --2- 172.21.15.131:0/1244961269 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f16480a1230 0x7f16480a36f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1664100c40 tx=0x7f1654011440 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:06.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.413+0000 7f1668a00700 1 -- 172.21.15.131:0/3819968540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573dbcb84f0 msgr2=0x5573dbcb8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:06.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.413+0000 7f1668a00700 1 --2- 172.21.15.131:0/3819968540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573dbcb84f0 0x5573dbcb8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.413+0000 7f1668a00700 1 -- 172.21.15.131:0/3819968540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573dbcb8e50 msgr2=0x5573dbcb92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:06.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.413+0000 7f1668a00700 1 --2- 172.21.15.131:0/3819968540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573dbcb8e50 0x5573dbcb92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.413+0000 7f1668a00700 1 -- 172.21.15.131:0/3819968540 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573dba89360 con 0x5573dbcb9810 2024-05-08T22:07:06.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.413+0000 7f1663e00700 1 --2- 172.21.15.131:0/3819968540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573dbcb84f0 0x5573dbcb8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:06.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.414+0000 7f1668a00700 1 --2- 172.21.15.131:0/3819968540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb9810 0x5573dbcb9c90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f165c011680 tx=0x7f165c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4580ab5af2cc0e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:06.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.414+0000 7f1659200700 1 -- 172.21.15.131:0/3819968540 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c00e040 con 0x5573dbcb9810 2024-05-08T22:07:06.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.414+0000 7f1659200700 1 -- 172.21.15.131:0/3819968540 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f165c00f040 con 0x5573dbcb9810 2024-05-08T22:07:06.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.414+0000 7f1659200700 1 -- 172.21.15.131:0/3819968540 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c004850 con 0x5573dbcb9810 2024-05-08T22:07:06.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.415+0000 7f167e5bfb80 1 -- 172.21.15.131:0/3819968540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb9810 msgr2=0x5573dbcb9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:06.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.415+0000 7f167e5bfb80 1 --2- 172.21.15.131:0/3819968540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb9810 0x5573dbcb9c90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f165c011680 tx=0x7f165c009920 comp rx=0 tx=0).stop 2024-05-08T22:07:06.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.415+0000 7f167e5bfb80 1 -- 172.21.15.131:0/3819968540 shutdown_connections 2024-05-08T22:07:06.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.415+0000 7f167e5bfb80 1 --2- 172.21.15.131:0/3819968540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb9810 0x5573dbcb9c90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.415+0000 7f167e5bfb80 1 --2- 172.21.15.131:0/3819968540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573dbcb84f0 0x5573dbcb8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.415+0000 7f167e5bfb80 1 --2- 172.21.15.131:0/3819968540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573dbcb8e50 0x5573dbcb92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.415+0000 7f167e5bfb80 1 -- 172.21.15.131:0/3819968540 >> 172.21.15.131:0/3819968540 conn(0x5573dbc161c0 msgr2=0x5573dbcb2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:06.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.415+0000 7f167e5bfb80 1 -- 172.21.15.131:0/3819968540 shutdown_connections 2024-05-08T22:07:06.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.415+0000 7f167e5bfb80 1 -- 172.21.15.131:0/3819968540 wait complete. 2024-05-08T22:07:06.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.415+0000 7f167e5bfb80 1 -- start start 2024-05-08T22:07:06.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.416+0000 7f167e5bfb80 1 Processor -- start 2024-05-08T22:07:06.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.416+0000 7f167e5bfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573dbcb84f0 0x5573dbcc1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.416+0000 7f167e5bfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573dbcb8e50 0x5573dbcc2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.416+0000 7f167e5bfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb9810 0x5573dbcc9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.416+0000 7f167e5bfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573dbba4510 con 0x5573dbcb9810 2024-05-08T22:07:06.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.416+0000 7f167e5bfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573dbba41b0 con 0x5573dbcb84f0 2024-05-08T22:07:06.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.416+0000 7f167e5bfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573dba8b650 con 0x5573dbcb8e50 2024-05-08T22:07:06.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.416+0000 7f167e5bfb80 10 client.0 _ll_register_callbacks cb 0x5573dbbadae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:07:06.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.416+0000 7f1668a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb9810 0x5573dbcc9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.416+0000 7f1663e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573dbcb84f0 0x5573dbcc1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.416+0000 7f1668a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb9810 0x5573dbcc9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39012/0 (socket says 172.21.15.131:39012) 2024-05-08T22:07:06.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.416+0000 7f1668a00700 1 -- 172.21.15.131:0/2951344348 learned_addr learned my addr 172.21.15.131:0/2951344348 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:06.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.416+0000 7f165be00700 1 --2- 172.21.15.131:0/2951344348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573dbcb8e50 0x5573dbcc2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.417+0000 7f1668a00700 1 -- 172.21.15.131:0/2951344348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573dbcb8e50 msgr2=0x5573dbcc2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:06.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.417+0000 7f1668a00700 1 --2- 172.21.15.131:0/2951344348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573dbcb8e50 0x5573dbcc2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.417+0000 7f1668a00700 1 -- 172.21.15.131:0/2951344348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573dbcb84f0 msgr2=0x5573dbcc1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:06.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.417+0000 7f1668a00700 1 --2- 172.21.15.131:0/2951344348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573dbcb84f0 0x5573dbcc1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.417+0000 7f1668a00700 1 -- 172.21.15.131:0/2951344348 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1654011b60 con 0x5573dbcb9810 2024-05-08T22:07:06.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.417+0000 7f1663e00700 1 --2- 172.21.15.131:0/2951344348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573dbcb84f0 0x5573dbcc1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:06.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.417+0000 7f1668a00700 1 --2- 172.21.15.131:0/2951344348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb9810 0x5573dbcc9340 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f165c006080 tx=0x7f165c009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1c615652063299 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:06.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.417+0000 7f1651600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:07:06.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.417+0000 7f1651600700 1 -- 172.21.15.131:0/2951344348 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c00e070 con 0x5573dbcb9810 2024-05-08T22:07:06.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.417+0000 7f1651600700 1 -- 172.21.15.131:0/2951344348 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f165c003c10 con 0x5573dbcb9810 2024-05-08T22:07:06.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.417+0000 7f167e5bfb80 1 -- 172.21.15.131:0/2951344348 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573dbb61220 con 0x5573dbcb9810 2024-05-08T22:07:06.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.417+0000 7f1651600700 1 -- 172.21.15.131:0/2951344348 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f165c003dd0 con 0x5573dbcb9810 2024-05-08T22:07:06.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.417+0000 7f167e5bfb80 10 client.15336 fetch_fsmap learned FSMap version 23 2024-05-08T22:07:06.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.417+0000 7f167e5bfb80 1 -- 172.21.15.131:0/2951344348 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f165c011330 con 0x5573dbcb9810 2024-05-08T22:07:06.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.418+0000 7f1651600700 1 -- 172.21.15.131:0/2951344348 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5573dbb61220 con 0x5573dbcb9810 2024-05-08T22:07:06.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.418+0000 7f1651600700 1 -- 172.21.15.131:0/2951344348 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f165c027460 con 0x5573dbcb9810 2024-05-08T22:07:06.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.418+0000 7f167e5bfb80 10 client.15336 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T22:07:06.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.418+0000 7f167e5bfb80 10 client.15336 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T22:07:06.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.418+0000 7f167e5bfb80 20 client.15336 populate_metadata read hostname 'smithi131' 2024-05-08T22:07:06.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.418+0000 7f167e5bfb80 1 --2- 172.21.15.131:0/2951344348 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5573dbcccef0 0x5573dbcd21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.418+0000 7f167e5bfb80 4 client.15336 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:07:06.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.418+0000 7f167e5bfb80 1 -- 172.21.15.131:0/2951344348 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573dbc877c0 con 0x5573dbcccef0 2024-05-08T22:07:06.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.418+0000 7f1663e00700 1 --2- 172.21.15.131:0/2951344348 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5573dbcccef0 0x5573dbcd21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.419+0000 7f1651600700 10 client.15336 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:07:06.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.419+0000 7f1663e00700 1 --2- 172.21.15.131:0/2951344348 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5573dbcccef0 0x5573dbcd21f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=448838370425b1cb server_cookie=a8cda7320804ab7 in_seq=0 out_seq=0 2024-05-08T22:07:06.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.420+0000 7f1651600700 1 -- 172.21.15.131:0/2951344348 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5573dbc877c0 con 0x5573dbcccef0 2024-05-08T22:07:06.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.420+0000 7f1651600700 10 client.15336 handle_command_reply: tid=0 2024-05-08T22:07:06.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.420+0000 7f167e5bfb80 1 client.15336 shutdown 2024-05-08T22:07:06.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.420+0000 7f167e5bfb80 1 -- 172.21.15.131:0/2951344348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb9810 msgr2=0x5573dbcc9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:06.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.420+0000 7f167e5bfb80 1 --2- 172.21.15.131:0/2951344348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb9810 0x5573dbcc9340 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f165c006080 tx=0x7f165c009510 comp rx=0 tx=0).stop 2024-05-08T22:07:06.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.420+0000 7f167e5bfb80 1 -- 172.21.15.131:0/2951344348 shutdown_connections 2024-05-08T22:07:06.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.420+0000 7f167e5bfb80 1 --2- 172.21.15.131:0/2951344348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb9810 0x5573dbcc9340 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.420+0000 7f167e5bfb80 1 --2- 172.21.15.131:0/2951344348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573dbcb84f0 0x5573dbcc1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.420+0000 7f167e5bfb80 1 --2- 172.21.15.131:0/2951344348 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5573dbcccef0 0x5573dbcd21f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.420+0000 7f167e5bfb80 1 --2- 172.21.15.131:0/2951344348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573dbcb8e50 0x5573dbcc2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.420+0000 7f167e5bfb80 1 -- 172.21.15.131:0/2951344348 >> 172.21.15.131:0/2951344348 conn(0x5573dbc161c0 msgr2=0x5573dbcb2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:06.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.420+0000 7f1651600700 0 client.15336 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:07:06.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.420+0000 7f167e5bfb80 1 -- 172.21.15.131:0/2951344348 shutdown_connections 2024-05-08T22:07:06.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.420+0000 7f167e5bfb80 1 -- 172.21.15.131:0/2951344348 wait complete. 2024-05-08T22:07:06.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.420+0000 7f167e5bfb80 20 client.15336 trim_cache size 0 max 16384 2024-05-08T22:07:06.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.439+0000 7f167e5bfb80 1 Processor -- start 2024-05-08T22:07:06.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.439+0000 7f167e5bfb80 1 -- start start 2024-05-08T22:07:06.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.439+0000 7f167e5bfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb84f0 0x5573dbcb88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.439+0000 7f167e5bfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573dbcb8e50 0x5573dbcc12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.439+0000 7f167e5bfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573dbcb9810 0x5573dbcc9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.439+0000 7f167e5bfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573dbc877c0 con 0x5573dbcb84f0 2024-05-08T22:07:06.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.439+0000 7f167e5bfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573dbb61220 con 0x5573dbcb8e50 2024-05-08T22:07:06.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.439+0000 7f167e5bfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573dbcb3810 con 0x5573dbcb9810 2024-05-08T22:07:06.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.439+0000 7f1663e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb84f0 0x5573dbcb88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.439+0000 7f1663e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb84f0 0x5573dbcb88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39024/0 (socket says 172.21.15.131:39024) 2024-05-08T22:07:06.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.439+0000 7f1663e00700 1 -- 172.21.15.131:0/3628802879 learned_addr learned my addr 172.21.15.131:0/3628802879 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:06.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.440+0000 7f1668a00700 1 --2- 172.21.15.131:0/3628802879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573dbcb9810 0x5573dbcc9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.440+0000 7f165be00700 1 --2- 172.21.15.131:0/3628802879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573dbcb8e50 0x5573dbcc12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.440+0000 7f1663e00700 1 -- 172.21.15.131:0/3628802879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573dbcb9810 msgr2=0x5573dbcc9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:06.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.440+0000 7f1663e00700 1 --2- 172.21.15.131:0/3628802879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573dbcb9810 0x5573dbcc9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.440+0000 7f1663e00700 1 -- 172.21.15.131:0/3628802879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573dbcb8e50 msgr2=0x5573dbcc12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:06.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.440+0000 7f1663e00700 1 --2- 172.21.15.131:0/3628802879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573dbcb8e50 0x5573dbcc12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.440+0000 7f1663e00700 1 -- 172.21.15.131:0/3628802879 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573dba35b10 con 0x5573dbcb84f0 2024-05-08T22:07:06.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.440+0000 7f165be00700 1 --2- 172.21.15.131:0/3628802879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573dbcb8e50 0x5573dbcc12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:07:06.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.440+0000 7f1663e00700 1 --2- 172.21.15.131:0/3628802879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb84f0 0x5573dbcb88d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x5573dbcc1d90 tx=0x7f165401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a4f05fa6b00db0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:06.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.440+0000 7f1659c00700 1 -- 172.21.15.131:0/3628802879 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1654023040 con 0x5573dbcb84f0 2024-05-08T22:07:06.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.440+0000 7f1659c00700 1 -- 172.21.15.131:0/3628802879 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1654004020 con 0x5573dbcb84f0 2024-05-08T22:07:06.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.440+0000 7f1659c00700 1 -- 172.21.15.131:0/3628802879 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1654004ca0 con 0x5573dbcb84f0 2024-05-08T22:07:06.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.441+0000 7f167e5bfb80 1 -- 172.21.15.131:0/3628802879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb84f0 msgr2=0x5573dbcb88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:06.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.441+0000 7f167e5bfb80 1 --2- 172.21.15.131:0/3628802879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb84f0 0x5573dbcb88d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x5573dbcc1d90 tx=0x7f165401c000 comp rx=0 tx=0).stop 2024-05-08T22:07:06.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.441+0000 7f167e5bfb80 1 -- 172.21.15.131:0/3628802879 shutdown_connections 2024-05-08T22:07:06.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.441+0000 7f167e5bfb80 1 --2- 172.21.15.131:0/3628802879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573dbcb9810 0x5573dbcc9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.441+0000 7f167e5bfb80 1 --2- 172.21.15.131:0/3628802879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb84f0 0x5573dbcb88d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.441+0000 7f167e5bfb80 1 --2- 172.21.15.131:0/3628802879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573dbcb8e50 0x5573dbcc12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.441+0000 7f167e5bfb80 1 -- 172.21.15.131:0/3628802879 >> 172.21.15.131:0/3628802879 conn(0x5573dbc161c0 msgr2=0x5573dbcaf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:06.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.441+0000 7f167e5bfb80 1 -- 172.21.15.131:0/3628802879 shutdown_connections 2024-05-08T22:07:06.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.441+0000 7f167e5bfb80 1 -- 172.21.15.131:0/3628802879 wait complete. 2024-05-08T22:07:06.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.441+0000 7f167e5bfb80 1 -- start start 2024-05-08T22:07:06.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.442+0000 7f167e5bfb80 1 Processor -- start 2024-05-08T22:07:06.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.442+0000 7f167e5bfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573dbcb84f0 0x5573dbcb1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.442+0000 7f167e5bfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb8e50 0x5573dbcb2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.442+0000 7f167e5bfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573dbcb9810 0x5573dbcc8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.442+0000 7f167e5bfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573dbcc8e40 con 0x5573dbcb8e50 2024-05-08T22:07:06.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.442+0000 7f167e5bfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573dbcc8fb0 con 0x5573dbcb84f0 2024-05-08T22:07:06.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.442+0000 7f167e5bfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573dbcc9170 con 0x5573dbcb9810 2024-05-08T22:07:06.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.442+0000 7f167e5bfb80 10 client.0 _ll_register_callbacks cb 0x5573dbbadae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:07:06.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.442+0000 7f165be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb8e50 0x5573dbcb2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.442+0000 7f165be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb8e50 0x5573dbcb2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39038/0 (socket says 172.21.15.131:39038) 2024-05-08T22:07:06.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.442+0000 7f165be00700 1 -- 172.21.15.131:0/3176931524 learned_addr learned my addr 172.21.15.131:0/3176931524 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:06.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.442+0000 7f1668a00700 1 --2- 172.21.15.131:0/3176931524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573dbcb9810 0x5573dbcc8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.442+0000 7f1663e00700 1 --2- 172.21.15.131:0/3176931524 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573dbcb84f0 0x5573dbcb1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.442+0000 7f165be00700 1 -- 172.21.15.131:0/3176931524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573dbcb9810 msgr2=0x5573dbcc8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:06.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.442+0000 7f165be00700 1 --2- 172.21.15.131:0/3176931524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573dbcb9810 0x5573dbcc8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.442+0000 7f165be00700 1 -- 172.21.15.131:0/3176931524 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573dbcb84f0 msgr2=0x5573dbcb1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:06.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.443+0000 7f165be00700 1 --2- 172.21.15.131:0/3176931524 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573dbcb84f0 0x5573dbcb1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.443+0000 7f165be00700 1 -- 172.21.15.131:0/3176931524 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f165400f990 con 0x5573dbcb8e50 2024-05-08T22:07:06.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.443+0000 7f1668a00700 1 --2- 172.21.15.131:0/3176931524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573dbcb9810 0x5573dbcc8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:07:06.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.443+0000 7f165be00700 1 --2- 172.21.15.131:0/3176931524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb8e50 0x5573dbcb2450 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f164c00b620 tx=0x7f164c09ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a672cb6e5b4acfb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:06.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.443+0000 7f1651600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:07:06.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.443+0000 7f1651600700 1 -- 172.21.15.131:0/3176931524 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c09b560 con 0x5573dbcb8e50 2024-05-08T22:07:06.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.443+0000 7f1651600700 1 -- 172.21.15.131:0/3176931524 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f164c09b100 con 0x5573dbcb8e50 2024-05-08T22:07:06.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.443+0000 7f167e5bfb80 1 -- 172.21.15.131:0/3176931524 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f165c00f040 con 0x5573dbcb8e50 2024-05-08T22:07:06.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.443+0000 7f1651600700 1 -- 172.21.15.131:0/3176931524 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f164c021a70 con 0x5573dbcb8e50 2024-05-08T22:07:06.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.443+0000 7f167e5bfb80 10 client.15342 fetch_fsmap learned FSMap version 23 2024-05-08T22:07:06.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.443+0000 7f167e5bfb80 1 -- 172.21.15.131:0/3176931524 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5573dbbce510 con 0x5573dbcb8e50 2024-05-08T22:07:06.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.443+0000 7f1651600700 1 -- 172.21.15.131:0/3176931524 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f165c00f040 con 0x5573dbcb8e50 2024-05-08T22:07:06.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.444+0000 7f1651600700 1 -- 172.21.15.131:0/3176931524 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f164c0a3060 con 0x5573dbcb8e50 2024-05-08T22:07:06.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.444+0000 7f167e5bfb80 10 client.15342 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T22:07:06.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.444+0000 7f167e5bfb80 10 client.15342 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T22:07:06.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.444+0000 7f167e5bfb80 20 client.15342 populate_metadata read hostname 'smithi131' 2024-05-08T22:07:06.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.444+0000 7f167e5bfb80 1 --2- 172.21.15.131:0/3176931524 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5573dbd28210 0x5573dbd24b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.444+0000 7f167e5bfb80 4 client.15342 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:07:06.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.444+0000 7f167e5bfb80 1 -- 172.21.15.131:0/3176931524 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5573dbc00e70 con 0x5573dbd28210 2024-05-08T22:07:06.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.444+0000 7f1663e00700 1 --2- 172.21.15.131:0/3176931524 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5573dbd28210 0x5573dbd24b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.445+0000 7f1651600700 10 client.15342 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:07:06.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.445+0000 7f1663e00700 1 --2- 172.21.15.131:0/3176931524 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5573dbd28210 0x5573dbd24b60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a717879d573ae74 server_cookie=d4b40d571ee2cad1 in_seq=0 out_seq=0 2024-05-08T22:07:06.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.445+0000 7f1651600700 1 -- 172.21.15.131:0/3176931524 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5573dbc00e70 con 0x5573dbd28210 2024-05-08T22:07:06.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.445+0000 7f1651600700 10 client.15342 handle_command_reply: tid=0 2024-05-08T22:07:06.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.445+0000 7f167e5bfb80 1 client.15342 shutdown 2024-05-08T22:07:06.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.445+0000 7f167e5bfb80 1 -- 172.21.15.131:0/3176931524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb8e50 msgr2=0x5573dbcb2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:06.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.445+0000 7f167e5bfb80 1 --2- 172.21.15.131:0/3176931524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb8e50 0x5573dbcb2450 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f164c00b620 tx=0x7f164c09ab10 comp rx=0 tx=0).stop 2024-05-08T22:07:06.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.445+0000 7f167e5bfb80 1 -- 172.21.15.131:0/3176931524 shutdown_connections 2024-05-08T22:07:06.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.445+0000 7f167e5bfb80 1 --2- 172.21.15.131:0/3176931524 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5573dbd28210 0x5573dbd24b60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.445+0000 7f167e5bfb80 1 --2- 172.21.15.131:0/3176931524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573dbcb9810 0x5573dbcc8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.445+0000 7f167e5bfb80 1 --2- 172.21.15.131:0/3176931524 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573dbcb84f0 0x5573dbcb1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.445+0000 7f1651600700 0 client.15342 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:07:06.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.445+0000 7f167e5bfb80 1 --2- 172.21.15.131:0/3176931524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573dbcb8e50 0x5573dbcb2450 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.445+0000 7f167e5bfb80 1 -- 172.21.15.131:0/3176931524 >> 172.21.15.131:0/3176931524 conn(0x5573dbc161c0 msgr2=0x5573dbcbc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:06.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.445+0000 7f167e5bfb80 1 -- 172.21.15.131:0/3176931524 shutdown_connections 2024-05-08T22:07:06.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.445+0000 7f167e5bfb80 1 -- 172.21.15.131:0/3176931524 wait complete. 2024-05-08T22:07:06.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.446+0000 7f167e5bfb80 20 client.15342 trim_cache size 0 max 16384 2024-05-08T22:07:06.452 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:07:06.452 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "06f1a696-0d86-11ef-bc97-c7b262605968", 2024-05-08T22:07:06.452 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-05-08T22:07:06.452 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24566, 2024-05-08T22:07:06.452 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-08T22:07:06.452 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-08T22:07:06.452 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-08T22:07:06.452 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 112.40807257199999, 2024-05-08T22:07:06.452 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 23, 2024-05-08T22:07:06.452 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 87, 2024-05-08T22:07:06.452 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 83, 2024-05-08T22:07:06.452 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 118.664581466 2024-05-08T22:07:06.453 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:07:06.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.446+0000 7f166aa00700 1 -- 172.21.15.131:0/1244961269 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f16480a1230 msgr2=0x7f16480a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:06.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.446+0000 7f166aa00700 1 --2- 172.21.15.131:0/1244961269 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f16480a1230 0x7f16480a36f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1664100c40 tx=0x7f1654011440 comp rx=0 tx=0).stop 2024-05-08T22:07:06.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.446+0000 7f166aa00700 1 -- 172.21.15.131:0/1244961269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16641020f0 msgr2=0x7f16641996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:06.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.446+0000 7f166aa00700 1 --2- 172.21.15.131:0/1244961269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16641020f0 0x7f16641996b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f164c005910 tx=0x7f164c005740 comp rx=0 tx=0).stop 2024-05-08T22:07:06.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.446+0000 7f166aa00700 1 -- 172.21.15.131:0/1244961269 shutdown_connections 2024-05-08T22:07:06.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.446+0000 7f166aa00700 1 --2- 172.21.15.131:0/1244961269 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f16480a1230 0x7f16480a36f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.446+0000 7f166aa00700 1 --2- 172.21.15.131:0/1244961269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1664102ab0 0x7f16641a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.446+0000 7f166aa00700 1 --2- 172.21.15.131:0/1244961269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1664100f20 0x7f1664199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.446+0000 7f166aa00700 1 --2- 172.21.15.131:0/1244961269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16641020f0 0x7f16641996b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.446+0000 7f166aa00700 1 -- 172.21.15.131:0/1244961269 >> 172.21.15.131:0/1244961269 conn(0x7f16640fc500 msgr2=0x7f16640731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:06.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.446+0000 7f166aa00700 1 -- 172.21.15.131:0/1244961269 shutdown_connections 2024-05-08T22:07:06.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.446+0000 7f166aa00700 1 -- 172.21.15.131:0/1244961269 wait complete. 2024-05-08T22:07:06.466 INFO:teuthology.run_tasks:Running task sequential... 2024-05-08T22:07:06.476 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-08T22:07:06.477 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-08T22:07:06.477 INFO:teuthology.task.print:Enabling mgr modules 2024-05-08T22:07:06.477 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-08T22:07:06.478 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T22:07:06.478 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi131.front.sepia.ceph.com 2024-05-08T22:07:06.479 DEBUG:teuthology.orchestra.run.smithi131:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-08T22:07:06.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.668+0000 7fc4b6a00700 1 -- 172.21.15.131:0/2052612264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4b0102060 msgr2=0x7fc4b01024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:06.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.668+0000 7fc4b6a00700 1 --2- 172.21.15.131:0/2052612264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4b0102060 0x7fc4b01024e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009990 tx=0x7fc49c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.668+0000 7fc4b6a00700 1 -- 172.21.15.131:0/2052612264 shutdown_connections 2024-05-08T22:07:06.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.668+0000 7fc4b6a00700 1 --2- 172.21.15.131:0/2052612264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4b0102a20 0x7fc4b0109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.668+0000 7fc4b6a00700 1 --2- 172.21.15.131:0/2052612264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4b0100f10 0x7fc4b0101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.668+0000 7fc4b6a00700 1 --2- 172.21.15.131:0/2052612264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4b0102060 0x7fc4b01024e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.668+0000 7fc4b6a00700 1 -- 172.21.15.131:0/2052612264 >> 172.21.15.131:0/2052612264 conn(0x7fc4b00fc4a0 msgr2=0x7fc4b00fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:06.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.668+0000 7fc4b6a00700 1 -- 172.21.15.131:0/2052612264 shutdown_connections 2024-05-08T22:07:06.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.668+0000 7fc4b6a00700 1 -- 172.21.15.131:0/2052612264 wait complete. 2024-05-08T22:07:06.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.669+0000 7fc4b6a00700 1 Processor -- start 2024-05-08T22:07:06.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.669+0000 7fc4b6a00700 1 -- start start 2024-05-08T22:07:06.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.670+0000 7fc4b6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4b0100f10 0x7fc4b0199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.670+0000 7fc4b6a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4b0102060 0x7fc4b0199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.670+0000 7fc4b6a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4b0102a20 0x7fc4b01a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.670+0000 7fc4b6a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b01a0c10 con 0x7fc4b0102060 2024-05-08T22:07:06.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.670+0000 7fc4b6a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b01a0d50 con 0x7fc4b0102a20 2024-05-08T22:07:06.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.670+0000 7fc4b6a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b01a0ee0 con 0x7fc4b0100f10 2024-05-08T22:07:06.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.670+0000 7fc4afe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4b0100f10 0x7fc4b0199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.670+0000 7fc4afe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4b0100f10 0x7fc4b0199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55718/0 (socket says 172.21.15.131:55718) 2024-05-08T22:07:06.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.670+0000 7fc4afe00700 1 -- 172.21.15.131:0/3896352746 learned_addr learned my addr 172.21.15.131:0/3896352746 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:06.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.670+0000 7fc4af400700 1 --2- 172.21.15.131:0/3896352746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4b0102060 0x7fc4b0199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.670+0000 7fc4b4a00700 1 --2- 172.21.15.131:0/3896352746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4b0102a20 0x7fc4b01a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.670+0000 7fc4afe00700 1 -- 172.21.15.131:0/3896352746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4b0102a20 msgr2=0x7fc4b01a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:06.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.670+0000 7fc4afe00700 1 --2- 172.21.15.131:0/3896352746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4b0102a20 0x7fc4b01a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.670+0000 7fc4afe00700 1 -- 172.21.15.131:0/3896352746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4b0102060 msgr2=0x7fc4b0199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:06.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.670+0000 7fc4afe00700 1 --2- 172.21.15.131:0/3896352746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4b0102060 0x7fc4b0199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:06.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.670+0000 7fc4afe00700 1 -- 172.21.15.131:0/3896352746 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a0009520 con 0x7fc4b0100f10 2024-05-08T22:07:06.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.670+0000 7fc4b4a00700 1 --2- 172.21.15.131:0/3896352746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4b0102a20 0x7fc4b01a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:07:06.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.671+0000 7fc4afe00700 1 --2- 172.21.15.131:0/3896352746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4b0100f10 0x7fc4b0199130 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000eb30 tx=0x7fc4a000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:06.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.671+0000 7fc4acc00700 1 -- 172.21.15.131:0/3896352746 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0009880 con 0x7fc4b0100f10 2024-05-08T22:07:06.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.671+0000 7fc4b6a00700 1 -- 172.21.15.131:0/3896352746 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49c0095f0 con 0x7fc4b0100f10 2024-05-08T22:07:06.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.671+0000 7fc4acc00700 1 -- 172.21.15.131:0/3896352746 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4a00044e0 con 0x7fc4b0100f10 2024-05-08T22:07:06.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.671+0000 7fc4acc00700 1 -- 172.21.15.131:0/3896352746 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0003e90 con 0x7fc4b0100f10 2024-05-08T22:07:06.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.671+0000 7fc4b6a00700 1 -- 172.21.15.131:0/3896352746 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b01a1540 con 0x7fc4b0100f10 2024-05-08T22:07:06.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.672+0000 7fc4b6a00700 1 -- 172.21.15.131:0/3896352746 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b0067e30 con 0x7fc4b0100f10 2024-05-08T22:07:06.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.674+0000 7fc4acc00700 1 -- 172.21.15.131:0/3896352746 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fc4a0004000 con 0x7fc4b0100f10 2024-05-08T22:07:06.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.675+0000 7fc4acc00700 1 --2- 172.21.15.131:0/3896352746 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc4940a11b0 0x7fc4940a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:06.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.675+0000 7fc4acc00700 1 -- 172.21.15.131:0/3896352746 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fc4a0014070 con 0x7fc4b0100f10 2024-05-08T22:07:06.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.675+0000 7fc4af400700 1 --2- 172.21.15.131:0/3896352746 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc4940a11b0 0x7fc4940a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:06.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.676+0000 7fc4af400700 1 --2- 172.21.15.131:0/3896352746 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc4940a11b0 0x7fc4940a3670 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00b2e0 tx=0x7fc49c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:06.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.677+0000 7fc4acc00700 1 -- 172.21.15.131:0/3896352746 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc4a0096650 con 0x7fc4b0100f10 2024-05-08T22:07:06.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:06.947+0000 7fc4b6a00700 1 -- 172.21.15.131:0/3896352746 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fc4b010c9c0 con 0x7fc4b0100f10 2024-05-08T22:07:07.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:07.863+0000 7fc4acc00700 1 -- 172.21.15.131:0/3896352746 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mgrmap(e 18) v1 ==== 135942+0+0 (secure 0 0 0) 0x7fc4a00cab90 con 0x7fc4b0100f10 2024-05-08T22:07:07.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:07.864+0000 7fc4acc00700 1 -- 172.21.15.131:0/3896352746 <== mon.1 v2:172.21.15.157:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v18) v1 ==== 92+0+0 (secure 0 0 0) 0x7fc4a00cdff0 con 0x7fc4b0100f10 2024-05-08T22:07:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:07.867+0000 7fc4b6a00700 1 -- 172.21.15.131:0/3896352746 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc4940a11b0 msgr2=0x7fc4940a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:07.867+0000 7fc4b6a00700 1 --2- 172.21.15.131:0/3896352746 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc4940a11b0 0x7fc4940a3670 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00b2e0 tx=0x7fc49c005fd0 comp rx=0 tx=0).stop 2024-05-08T22:07:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:07.867+0000 7fc4b6a00700 1 -- 172.21.15.131:0/3896352746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4b0100f10 msgr2=0x7fc4b0199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:07.867+0000 7fc4b6a00700 1 --2- 172.21.15.131:0/3896352746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4b0100f10 0x7fc4b0199130 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000eb30 tx=0x7fc4a000ee40 comp rx=0 tx=0).stop 2024-05-08T22:07:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:07.867+0000 7fc4b6a00700 1 -- 172.21.15.131:0/3896352746 shutdown_connections 2024-05-08T22:07:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:07.867+0000 7fc4b6a00700 1 --2- 172.21.15.131:0/3896352746 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fc4940a11b0 0x7fc4940a3670 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:07.867+0000 7fc4b6a00700 1 --2- 172.21.15.131:0/3896352746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4b0102a20 0x7fc4b01a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:07.867+0000 7fc4b6a00700 1 --2- 172.21.15.131:0/3896352746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4b0100f10 0x7fc4b0199130 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:07.867+0000 7fc4b6a00700 1 --2- 172.21.15.131:0/3896352746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4b0102060 0x7fc4b0199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:07.867+0000 7fc4b6a00700 1 -- 172.21.15.131:0/3896352746 >> 172.21.15.131:0/3896352746 conn(0x7fc4b00fc4a0 msgr2=0x7fc4b00731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:07.867+0000 7fc4b6a00700 1 -- 172.21.15.131:0/3896352746 shutdown_connections 2024-05-08T22:07:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:07.867+0000 7fc4b6a00700 1 -- 172.21.15.131:0/3896352746 wait complete. 2024-05-08T22:07:07.889 DEBUG:teuthology.orchestra.run.smithi131:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-08T22:07:08.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.097+0000 7f75f6600700 1 -- 172.21.15.131:0/1260313681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f0073a00 msgr2=0x7f75f0073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:08.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.097+0000 7f75f6600700 1 --2- 172.21.15.131:0/1260313681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f0073a00 0x7f75f0073e80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f75dc009990 tx=0x7f75dc009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:08.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.097+0000 7f75f6600700 1 -- 172.21.15.131:0/1260313681 shutdown_connections 2024-05-08T22:07:08.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.097+0000 7f75f6600700 1 --2- 172.21.15.131:0/1260313681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75f0108dd0 0x7f75f010b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:08.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.097+0000 7f75f6600700 1 --2- 172.21.15.131:0/1260313681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75f0074fe0 0x7f75f0073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:08.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.097+0000 7f75f6600700 1 --2- 172.21.15.131:0/1260313681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f0073a00 0x7f75f0073e80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:08.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.097+0000 7f75f6600700 1 -- 172.21.15.131:0/1260313681 >> 172.21.15.131:0/1260313681 conn(0x7f75f00fc4a0 msgr2=0x7f75f00fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:08.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.097+0000 7f75f6600700 1 -- 172.21.15.131:0/1260313681 shutdown_connections 2024-05-08T22:07:08.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.097+0000 7f75f6600700 1 -- 172.21.15.131:0/1260313681 wait complete. 2024-05-08T22:07:08.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.098+0000 7f75f6600700 1 Processor -- start 2024-05-08T22:07:08.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.099+0000 7f75f6600700 1 -- start start 2024-05-08T22:07:08.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.099+0000 7f75f6600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75f0073a00 0x7f75f01990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:08.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.099+0000 7f75f6600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75f0074fe0 0x7f75f0199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:08.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.099+0000 7f75f6600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f0108dd0 0x7f75f01a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:08.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.099+0000 7f75f6600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f01a0c20 con 0x7f75f0074fe0 2024-05-08T22:07:08.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.099+0000 7f75f6600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f01a0d60 con 0x7f75f0073a00 2024-05-08T22:07:08.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.099+0000 7f75f6600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f01a0ef0 con 0x7f75f0108dd0 2024-05-08T22:07:08.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.099+0000 7f75ef400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75f0073a00 0x7f75f01990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:08.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.099+0000 7f75ef400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75f0073a00 0x7f75f01990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:37168/0 (socket says 172.21.15.131:37168) 2024-05-08T22:07:08.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.099+0000 7f75ef400700 1 -- 172.21.15.131:0/1427504309 learned_addr learned my addr 172.21.15.131:0/1427504309 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:08.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.099+0000 7f75efe00700 1 --2- 172.21.15.131:0/1427504309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f0108dd0 0x7f75f01a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:08.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.099+0000 7f75eea00700 1 --2- 172.21.15.131:0/1427504309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75f0074fe0 0x7f75f0199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:08.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.100+0000 7f75ef400700 1 -- 172.21.15.131:0/1427504309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f0108dd0 msgr2=0x7f75f01a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:08.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.100+0000 7f75ef400700 1 --2- 172.21.15.131:0/1427504309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f0108dd0 0x7f75f01a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:08.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.100+0000 7f75ef400700 1 -- 172.21.15.131:0/1427504309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75f0074fe0 msgr2=0x7f75f0199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:08.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.100+0000 7f75ef400700 1 --2- 172.21.15.131:0/1427504309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75f0074fe0 0x7f75f0199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:08.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.100+0000 7f75ef400700 1 -- 172.21.15.131:0/1427504309 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e0009520 con 0x7f75f0073a00 2024-05-08T22:07:08.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.100+0000 7f75efe00700 1 --2- 172.21.15.131:0/1427504309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f0108dd0 0x7f75f01a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:08.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.100+0000 7f75ef400700 1 --2- 172.21.15.131:0/1427504309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75f0073a00 0x7f75f01990f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f75e000eb30 tx=0x7f75e000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:08.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.100+0000 7f75e7e00700 1 -- 172.21.15.131:0/1427504309 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e0009880 con 0x7f75f0073a00 2024-05-08T22:07:08.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.100+0000 7f75e7e00700 1 -- 172.21.15.131:0/1427504309 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75e00044e0 con 0x7f75f0073a00 2024-05-08T22:07:08.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.100+0000 7f75f6600700 1 -- 172.21.15.131:0/1427504309 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75dc0095f0 con 0x7f75f0073a00 2024-05-08T22:07:08.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.100+0000 7f75e7e00700 1 -- 172.21.15.131:0/1427504309 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e0003d80 con 0x7f75f0073a00 2024-05-08T22:07:08.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.100+0000 7f75f6600700 1 -- 172.21.15.131:0/1427504309 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f01a1550 con 0x7f75f0073a00 2024-05-08T22:07:08.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.101+0000 7f75f6600700 1 -- 172.21.15.131:0/1427504309 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f0107b90 con 0x7f75f0073a00 2024-05-08T22:07:08.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.103+0000 7f75e7e00700 1 -- 172.21.15.131:0/1427504309 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 135942+0+0 (secure 0 0 0) 0x7f75e0003f20 con 0x7f75f0073a00 2024-05-08T22:07:08.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.104+0000 7f75e7e00700 1 --2- 172.21.15.131:0/1427504309 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f75d40a1200 0x7f75d40a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:08.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.104+0000 7f75eea00700 1 -- 172.21.15.131:0/1427504309 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f75d40a1200 msgr2=0x7f75d40a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2485665042 2024-05-08T22:07:08.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.104+0000 7f75e7e00700 1 -- 172.21.15.131:0/1427504309 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f75e0014070 con 0x7f75f0073a00 2024-05-08T22:07:08.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.105+0000 7f75eea00700 1 --2- 172.21.15.131:0/1427504309 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f75d40a1200 0x7f75d40a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:07:08.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.108+0000 7f75e7e00700 1 -- 172.21.15.131:0/1427504309 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f75e0096680 con 0x7f75f0073a00 2024-05-08T22:07:08.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:07 smithi131 ceph-mon[37260]: pgmap v300: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T22:07:08.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:07 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3896352746' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-08T22:07:08.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:07 smithi131 ceph-mon[37260]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-08T22:07:08.141 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:07 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: ignoring --setuser ceph since I am not root 2024-05-08T22:07:08.141 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:07 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: ignoring --setgroup ceph since I am not root 2024-05-08T22:07:08.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:07 smithi157 ceph-mon[77746]: pgmap v300: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T22:07:08.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:07 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3896352746' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-08T22:07:08.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:07 smithi157 ceph-mon[77746]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-08T22:07:08.276 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:07 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: ignoring --setuser ceph since I am not root 2024-05-08T22:07:08.276 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:07 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: ignoring --setgroup ceph since I am not root 2024-05-08T22:07:08.277 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:07 smithi157 ceph-mgr[79623]: -- 172.21.15.157:0/474218529 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a3caba21a0 con 0x55a3cab78800 2024-05-08T22:07:08.299 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:07 smithi132 ceph-mon[78180]: pgmap v300: 129 pgs: 129 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T22:07:08.299 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:07 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3896352746' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-08T22:07:08.299 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:07 smithi132 ceph-mon[78180]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-08T22:07:08.299 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:07 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: ignoring --setuser ceph since I am not root 2024-05-08T22:07:08.299 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:07 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: ignoring --setgroup ceph since I am not root 2024-05-08T22:07:08.299 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:07 smithi132 ceph-mgr[79689]: -- 172.21.15.132:0/3839980091 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ecd18f21a0 con 0x55ecd18c9000 2024-05-08T22:07:08.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.306+0000 7f75eea00700 1 -- 172.21.15.131:0/1427504309 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f75d40a1200 msgr2=0x7f75d40a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2485665042 2024-05-08T22:07:08.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.306+0000 7f75eea00700 1 --2- 172.21.15.131:0/1427504309 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f75d40a1200 0x7f75d40a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T22:07:08.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.324+0000 7f75f6600700 1 -- 172.21.15.131:0/1427504309 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f75f010ebd0 con 0x7f75f0073a00 2024-05-08T22:07:08.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.330+0000 7f75e7e00700 1 -- 172.21.15.131:0/1427504309 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7f75e0099ca0 con 0x7f75f0073a00 2024-05-08T22:07:08.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.332+0000 7f75f6600700 1 -- 172.21.15.131:0/1427504309 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f75d40a1200 msgr2=0x7f75d40a36c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:07:08.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.332+0000 7f75f6600700 1 --2- 172.21.15.131:0/1427504309 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f75d40a1200 0x7f75d40a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:08.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.332+0000 7f75f6600700 1 -- 172.21.15.131:0/1427504309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75f0073a00 msgr2=0x7f75f01990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:08.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.332+0000 7f75f6600700 1 --2- 172.21.15.131:0/1427504309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75f0073a00 0x7f75f01990f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f75e000eb30 tx=0x7f75e000ee40 comp rx=0 tx=0).stop 2024-05-08T22:07:08.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.332+0000 7f75f6600700 1 -- 172.21.15.131:0/1427504309 shutdown_connections 2024-05-08T22:07:08.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.332+0000 7f75f6600700 1 --2- 172.21.15.131:0/1427504309 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f75d40a1200 0x7f75d40a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:08.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.332+0000 7f75f6600700 1 --2- 172.21.15.131:0/1427504309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f0108dd0 0x7f75f01a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:08.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.332+0000 7f75f6600700 1 --2- 172.21.15.131:0/1427504309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75f0073a00 0x7f75f01990f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:08.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.332+0000 7f75f6600700 1 --2- 172.21.15.131:0/1427504309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75f0074fe0 0x7f75f0199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:08.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.332+0000 7f75f6600700 1 -- 172.21.15.131:0/1427504309 >> 172.21.15.131:0/1427504309 conn(0x7f75f00fc4a0 msgr2=0x7f75f0104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:08.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.332+0000 7f75f6600700 1 -- 172.21.15.131:0/1427504309 shutdown_connections 2024-05-08T22:07:08.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.332+0000 7f75f6600700 1 -- 172.21.15.131:0/1427504309 wait complete. 2024-05-08T22:07:08.353 DEBUG:teuthology.orchestra.run.smithi131:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-08T22:07:08.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.549+0000 7f1070c00700 1 -- 172.21.15.131:0/3337567048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f106c100f30 msgr2=0x7f106c101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:08.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.549+0000 7f1070c00700 1 --2- 172.21.15.131:0/3337567048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f106c100f30 0x7f106c101350 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f105c009990 tx=0x7f105c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:08.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.549+0000 7f1070c00700 1 -- 172.21.15.131:0/3337567048 shutdown_connections 2024-05-08T22:07:08.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.549+0000 7f1070c00700 1 --2- 172.21.15.131:0/3337567048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f106c102a40 0x7f106c109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:08.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.549+0000 7f1070c00700 1 --2- 172.21.15.131:0/3337567048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f106c100f30 0x7f106c101350 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:08.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.549+0000 7f1070c00700 1 --2- 172.21.15.131:0/3337567048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f106c102080 0x7f106c102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:08.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.549+0000 7f1070c00700 1 -- 172.21.15.131:0/3337567048 >> 172.21.15.131:0/3337567048 conn(0x7f106c0fc4a0 msgr2=0x7f106c0fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:08.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.549+0000 7f1070c00700 1 -- 172.21.15.131:0/3337567048 shutdown_connections 2024-05-08T22:07:08.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.549+0000 7f1070c00700 1 -- 172.21.15.131:0/3337567048 wait complete. 2024-05-08T22:07:08.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.550+0000 7f1070c00700 1 Processor -- start 2024-05-08T22:07:08.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.550+0000 7f1070c00700 1 -- start start 2024-05-08T22:07:08.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.550+0000 7f1070c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f106c100f30 0x7f106c074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:08.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.550+0000 7f1070c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f106c102080 0x7f106c073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:08.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.550+0000 7f1070c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f106c102a40 0x7f106c0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:08.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.550+0000 7f1070c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c074070 con 0x7f106c102a40 2024-05-08T22:07:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.550+0000 7f1070c00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c0741e0 con 0x7f106c102080 2024-05-08T22:07:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.550+0000 7f1070c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c1a0ec0 con 0x7f106c100f30 2024-05-08T22:07:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.551+0000 7f1069e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f106c100f30 0x7f106c074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.551+0000 7f1069400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f106c102080 0x7f106c073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.551+0000 7f1069400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f106c102080 0x7f106c073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:37186/0 (socket says 172.21.15.131:37186) 2024-05-08T22:07:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.551+0000 7f106a800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f106c102a40 0x7f106c0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.551+0000 7f1069e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f106c100f30 0x7f106c074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55742/0 (socket says 172.21.15.131:55742) 2024-05-08T22:07:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.551+0000 7f1069400700 1 -- 172.21.15.131:0/2877429210 learned_addr learned my addr 172.21.15.131:0/2877429210 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.551+0000 7f1069400700 1 -- 172.21.15.131:0/2877429210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f106c100f30 msgr2=0x7f106c074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.551+0000 7f1069400700 1 --2- 172.21.15.131:0/2877429210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f106c100f30 0x7f106c074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.551+0000 7f1069400700 1 -- 172.21.15.131:0/2877429210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f106c102a40 msgr2=0x7f106c0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.551+0000 7f1069400700 1 --2- 172.21.15.131:0/2877429210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f106c102a40 0x7f106c0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.551+0000 7f1069400700 1 -- 172.21.15.131:0/2877429210 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105c0095f0 con 0x7f106c102080 2024-05-08T22:07:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.551+0000 7f1069e00700 1 --2- 172.21.15.131:0/2877429210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f106c100f30 0x7f106c074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.551+0000 7f106a800700 1 --2- 172.21.15.131:0/2877429210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f106c102a40 0x7f106c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.552+0000 7f1069400700 1 --2- 172.21.15.131:0/2877429210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f106c102080 0x7f106c073390 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f105800e9a0 tx=0x7f105800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.552+0000 7f1062a00700 1 -- 172.21.15.131:0/2877429210 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058009790 con 0x7f106c102080 2024-05-08T22:07:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.552+0000 7f1062a00700 1 -- 172.21.15.131:0/2877429210 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f105800ce70 con 0x7f106c102080 2024-05-08T22:07:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.552+0000 7f1062a00700 1 -- 172.21.15.131:0/2877429210 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058018420 con 0x7f106c102080 2024-05-08T22:07:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.552+0000 7f1070c00700 1 -- 172.21.15.131:0/2877429210 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106c1a1150 con 0x7f106c102080 2024-05-08T22:07:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.552+0000 7f1070c00700 1 -- 172.21.15.131:0/2877429210 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106c1a1610 con 0x7f106c102080 2024-05-08T22:07:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.553+0000 7f1070c00700 1 -- 172.21.15.131:0/2877429210 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f106c1083a0 con 0x7f106c102080 2024-05-08T22:07:08.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.555+0000 7f1062a00700 1 -- 172.21.15.131:0/2877429210 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 135942+0+0 (secure 0 0 0) 0x7f10580106f0 con 0x7f106c102080 2024-05-08T22:07:08.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.556+0000 7f1062a00700 1 --2- 172.21.15.131:0/2877429210 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f10500a1280 0x7f10500a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:08.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.556+0000 7f1069e00700 1 -- 172.21.15.131:0/2877429210 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f10500a1280 msgr2=0x7f10500a3740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2485665042 2024-05-08T22:07:08.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.556+0000 7f1062a00700 1 -- 172.21.15.131:0/2877429210 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f1058014070 con 0x7f106c102080 2024-05-08T22:07:08.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.556+0000 7f1069e00700 1 --2- 172.21.15.131:0/2877429210 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f10500a1280 0x7f10500a3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:07:08.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.560+0000 7f1062a00700 1 -- 172.21.15.131:0/2877429210 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1058052d60 con 0x7f106c102080 2024-05-08T22:07:08.563 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:08 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:07:08.297+0000 7f73e3650200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T22:07:08.564 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:08 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:07:08.391+0000 7f73e3650200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T22:07:08.564 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:08 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:07:08.478+0000 7f73e3650200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T22:07:08.573 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:08 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:07:08.304+0000 7f1c761dd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T22:07:08.573 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:08 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:07:08.398+0000 7f1c761dd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T22:07:08.573 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:08 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:07:08.486+0000 7f1c761dd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T22:07:08.633 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:08 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:08.489+0000 7fed4830b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T22:07:08.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.757+0000 7f1069e00700 1 -- 172.21.15.131:0/2877429210 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f10500a1280 msgr2=0x7f10500a3740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2485665042 2024-05-08T22:07:08.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.757+0000 7f1069e00700 1 --2- 172.21.15.131:0/2877429210 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f10500a1280 0x7f10500a3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T22:07:08.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.792+0000 7f1070c00700 1 -- 172.21.15.131:0/2877429210 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f106c10c720 con 0x7f106c102080 2024-05-08T22:07:08.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.797+0000 7f1062a00700 1 -- 172.21.15.131:0/2877429210 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7f1058099c80 con 0x7f106c102080 2024-05-08T22:07:08.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.800+0000 7f1070c00700 1 -- 172.21.15.131:0/2877429210 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f10500a1280 msgr2=0x7f10500a3740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:07:08.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.800+0000 7f1070c00700 1 --2- 172.21.15.131:0/2877429210 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f10500a1280 0x7f10500a3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:08.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.800+0000 7f1070c00700 1 -- 172.21.15.131:0/2877429210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f106c102080 msgr2=0x7f106c073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:08.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.800+0000 7f1070c00700 1 --2- 172.21.15.131:0/2877429210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f106c102080 0x7f106c073390 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f105800e9a0 tx=0x7f105800ecb0 comp rx=0 tx=0).stop 2024-05-08T22:07:08.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.800+0000 7f1070c00700 1 -- 172.21.15.131:0/2877429210 shutdown_connections 2024-05-08T22:07:08.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.800+0000 7f1070c00700 1 --2- 172.21.15.131:0/2877429210 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7f10500a1280 0x7f10500a3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:08.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.800+0000 7f1070c00700 1 --2- 172.21.15.131:0/2877429210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f106c102a40 0x7f106c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:08.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.800+0000 7f1070c00700 1 --2- 172.21.15.131:0/2877429210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f106c100f30 0x7f106c074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:08.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.800+0000 7f1070c00700 1 --2- 172.21.15.131:0/2877429210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f106c102080 0x7f106c073390 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:08.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.800+0000 7f1070c00700 1 -- 172.21.15.131:0/2877429210 >> 172.21.15.131:0/2877429210 conn(0x7f106c0fc4a0 msgr2=0x7f106c1054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:08.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.800+0000 7f1070c00700 1 -- 172.21.15.131:0/2877429210 shutdown_connections 2024-05-08T22:07:08.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:08.801+0000 7f1070c00700 1 -- 172.21.15.131:0/2877429210 wait complete. 2024-05-08T22:07:08.822 INFO:teuthology.run_tasks:Running task kclient... 2024-05-08T22:07:08.836 INFO:tasks.kclient:Mounting kernel clients... 2024-05-08T22:07:08.836 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-08T22:07:08.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:07:08.837 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-05-08T22:07:08.839 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:08 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:07:08.563+0000 7f73e3650200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T22:07:08.885 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-08T22:07:08.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:07:08.887 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T22:07:08.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:08 smithi131 ceph-mon[37260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-08T22:07:08.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:08 smithi131 ceph-mon[37260]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-05-08T22:07:08.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:08 smithi131 ceph-mon[37260]: from='client.? ' entity='client.admin' 2024-05-08T22:07:08.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:08 smithi131 ceph-mon[37260]: from='client.? ' entity='client.admin' 2024-05-08T22:07:08.893 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:08 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:08.632+0000 7fed4830b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T22:07:08.893 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:08 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:08.750+0000 7fed4830b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T22:07:08.893 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:08 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:08.872+0000 7fed4830b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T22:07:08.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:07:08.963 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link delete ceph-brx 2024-05-08T22:07:09.060 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-05-08T22:07:09.060 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T22:07:09.061 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T22:07:09.061 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-08T22:07:09.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:08 smithi132 ceph-mon[78180]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-08T22:07:09.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:08 smithi132 ceph-mon[78180]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-05-08T22:07:09.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:08 smithi132 ceph-mon[78180]: from='client.? ' entity='client.admin' 2024-05-08T22:07:09.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:08 smithi132 ceph-mon[78180]: from='client.? ' entity='client.admin' 2024-05-08T22:07:09.207 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:08 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:07:08.572+0000 7f1c761dd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T22:07:09.207 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:08 smithi157 ceph-mon[77746]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-08T22:07:09.207 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:08 smithi157 ceph-mon[77746]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-05-08T22:07:09.207 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:08 smithi157 ceph-mon[77746]: from='client.? ' entity='client.admin' 2024-05-08T22:07:09.208 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:08 smithi157 ceph-mon[77746]: from='client.? ' entity='client.admin' 2024-05-08T22:07:09.441 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:07:09.445 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:09 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:07:09.192+0000 7f73e3650200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:07:09.462 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:09 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:07:09.206+0000 7f1c761dd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:07:09.776 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:09 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:07:09.461+0000 7f1c761dd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T22:07:09.776 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:09 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:07:09.550+0000 7f1c761dd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T22:07:09.776 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:09 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:07:09.660+0000 7f1c761dd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T22:07:09.800 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:09 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:07:09.445+0000 7f73e3650200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T22:07:09.800 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:09 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:07:09.531+0000 7f73e3650200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T22:07:09.800 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:09 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:07:09.640+0000 7f73e3650200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T22:07:10.055 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:09 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:09.793+0000 7fed4830b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:07:10.090 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:09 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:07:09.799+0000 7f73e3650200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T22:07:10.090 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:09 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:07:09.928+0000 7f73e3650200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T22:07:10.090 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:10 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:07:10.014+0000 7f73e3650200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T22:07:10.276 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:09 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:07:09.820+0000 7f1c761dd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T22:07:10.276 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:09 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:07:09.949+0000 7f1c761dd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T22:07:10.276 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:10 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:07:10.036+0000 7f1c761dd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T22:07:10.390 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:10 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:10.129+0000 7fed4830b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T22:07:10.391 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:10 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:10.244+0000 7fed4830b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T22:07:10.648 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:10 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:10.395+0000 7fed4830b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T22:07:10.648 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:10 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:10.616+0000 7fed4830b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T22:07:10.776 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:10 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:07:10.409+0000 7f1c761dd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T22:07:10.777 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:10 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:07:10.385+0000 7f73e3650200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T22:07:10.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.830+0000 7fd788600700 1 -- 172.21.15.131:0/2722071004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7800731d0 msgr2=0x7fd7800735b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:10.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.830+0000 7fd788600700 1 --2- 172.21.15.131:0/2722071004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7800731d0 0x7fd7800735b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd774009990 tx=0x7fd774009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:10.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.830+0000 7fd788600700 1 -- 172.21.15.131:0/2722071004 shutdown_connections 2024-05-08T22:07:10.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.830+0000 7fd788600700 1 --2- 172.21.15.131:0/2722071004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd78010d9d0 0x7fd78010fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:10.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.830+0000 7fd788600700 1 --2- 172.21.15.131:0/2722071004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7800731d0 0x7fd7800735b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:10.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.830+0000 7fd788600700 1 --2- 172.21.15.131:0/2722071004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd780073af0 0x7fd78010d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:10.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.830+0000 7fd788600700 1 -- 172.21.15.131:0/2722071004 >> 172.21.15.131:0/2722071004 conn(0x7fd780078100 msgr2=0x7fd780078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:10.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.830+0000 7fd788600700 1 -- 172.21.15.131:0/2722071004 shutdown_connections 2024-05-08T22:07:10.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.830+0000 7fd788600700 1 -- 172.21.15.131:0/2722071004 wait complete. 2024-05-08T22:07:10.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.831+0000 7fd788600700 1 Processor -- start 2024-05-08T22:07:10.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.832+0000 7fd788600700 1 -- start start 2024-05-08T22:07:10.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.832+0000 7fd788600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7800731d0 0x7fd7801995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:10.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.832+0000 7fd788600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd780073af0 0x7fd780199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:10.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.832+0000 7fd788600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd78010d9d0 0x7fd78019de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:10.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.832+0000 7fd788600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78019e4a0 con 0x7fd780073af0 2024-05-08T22:07:10.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.832+0000 7fd788600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78019e610 con 0x7fd7800731d0 2024-05-08T22:07:10.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.832+0000 7fd788600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78019e7d0 con 0x7fd78010d9d0 2024-05-08T22:07:10.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.832+0000 7fd785c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7800731d0 0x7fd7801995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:10.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.832+0000 7fd785c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7800731d0 0x7fd7801995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:37192/0 (socket says 172.21.15.131:37192) 2024-05-08T22:07:10.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.832+0000 7fd785c00700 1 -- 172.21.15.131:0/3754577987 learned_addr learned my addr 172.21.15.131:0/3754577987 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:10.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.832+0000 7fd785200700 1 --2- 172.21.15.131:0/3754577987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd780073af0 0x7fd780199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:10.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.832+0000 7fd786600700 1 --2- 172.21.15.131:0/3754577987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd78010d9d0 0x7fd78019de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:10.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.833+0000 7fd785c00700 1 -- 172.21.15.131:0/3754577987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd78010d9d0 msgr2=0x7fd78019de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:10.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.833+0000 7fd785c00700 1 --2- 172.21.15.131:0/3754577987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd78010d9d0 0x7fd78019de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:10.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.833+0000 7fd785c00700 1 -- 172.21.15.131:0/3754577987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd780073af0 msgr2=0x7fd780199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:10.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.833+0000 7fd785c00700 1 --2- 172.21.15.131:0/3754577987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd780073af0 0x7fd780199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:10.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.833+0000 7fd785c00700 1 -- 172.21.15.131:0/3754577987 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7740095f0 con 0x7fd7800731d0 2024-05-08T22:07:10.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.833+0000 7fd785c00700 1 --2- 172.21.15.131:0/3754577987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7800731d0 0x7fd7801995f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd774000c00 tx=0x7fd774004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:10.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.833+0000 7fd77e000700 1 -- 172.21.15.131:0/3754577987 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77401d070 con 0x7fd7800731d0 2024-05-08T22:07:10.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.833+0000 7fd788600700 1 -- 172.21.15.131:0/3754577987 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd78019ea50 con 0x7fd7800731d0 2024-05-08T22:07:10.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.833+0000 7fd77e000700 1 -- 172.21.15.131:0/3754577987 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd774022470 con 0x7fd7800731d0 2024-05-08T22:07:10.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.833+0000 7fd77e000700 1 -- 172.21.15.131:0/3754577987 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd774017650 con 0x7fd7800731d0 2024-05-08T22:07:10.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.833+0000 7fd788600700 1 -- 172.21.15.131:0/3754577987 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7801a6310 con 0x7fd7800731d0 2024-05-08T22:07:10.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.835+0000 7fd788600700 1 -- 172.21.15.131:0/3754577987 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd780108be0 con 0x7fd7800731d0 2024-05-08T22:07:10.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.836+0000 7fd77e000700 1 -- 172.21.15.131:0/3754577987 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 135942+0+0 (secure 0 0 0) 0x7fd7740225e0 con 0x7fd7800731d0 2024-05-08T22:07:10.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.838+0000 7fd77e000700 1 --2- 172.21.15.131:0/3754577987 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd7680a1280 0x7fd7680a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:10.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.838+0000 7fd785200700 1 -- 172.21.15.131:0/3754577987 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd7680a1280 msgr2=0x7fd7680a3740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2485665042 2024-05-08T22:07:10.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.838+0000 7fd77e000700 1 -- 172.21.15.131:0/3754577987 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fd7740cdf90 con 0x7fd7800731d0 2024-05-08T22:07:10.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.838+0000 7fd785200700 1 --2- 172.21.15.131:0/3754577987 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd7680a1280 0x7fd7680a3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:07:10.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:10.842+0000 7fd77e000700 1 -- 172.21.15.131:0/3754577987 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd774097290 con 0x7fd7800731d0 2024-05-08T22:07:11.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:11.039+0000 7fd785200700 1 -- 172.21.15.131:0/3754577987 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd7680a1280 msgr2=0x7fd7680a3740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2485665042 2024-05-08T22:07:11.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:11.039+0000 7fd785200700 1 --2- 172.21.15.131:0/3754577987 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd7680a1280 0x7fd7680a3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T22:07:11.070 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:10 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:07:10.776+0000 7f73e3650200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T22:07:11.070 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:10 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:07:10.975+0000 7f73e3650200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T22:07:11.097 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:10 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:07:10.803+0000 7f1c761dd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T22:07:11.097 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:11 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:07:11.002+0000 7f1c761dd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T22:07:11.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:11.096+0000 7fd788600700 1 -- 172.21.15.131:0/3754577987 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd78004ec90 con 0x7fd7800731d0 2024-05-08T22:07:11.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:11.097+0000 7fd77e000700 1 -- 172.21.15.131:0/3754577987 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+88 (secure 0 0 0) 0x7fd77409a8b0 con 0x7fd7800731d0 2024-05-08T22:07:11.098 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-08T22:07:11.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:11.100+0000 7fd788600700 1 -- 172.21.15.131:0/3754577987 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd7680a1280 msgr2=0x7fd7680a3740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:07:11.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:11.100+0000 7fd788600700 1 --2- 172.21.15.131:0/3754577987 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd7680a1280 0x7fd7680a3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:11.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:11.100+0000 7fd788600700 1 -- 172.21.15.131:0/3754577987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7800731d0 msgr2=0x7fd7801995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:11.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:11.100+0000 7fd788600700 1 --2- 172.21.15.131:0/3754577987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7800731d0 0x7fd7801995f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd774000c00 tx=0x7fd774004730 comp rx=0 tx=0).stop 2024-05-08T22:07:11.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:11.100+0000 7fd788600700 1 -- 172.21.15.131:0/3754577987 shutdown_connections 2024-05-08T22:07:11.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:11.100+0000 7fd788600700 1 --2- 172.21.15.131:0/3754577987 >> [v2:172.21.15.131:6800/2485665042,v1:172.21.15.131:6801/2485665042] conn(0x7fd7680a1280 0x7fd7680a3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:11.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:11.100+0000 7fd788600700 1 --2- 172.21.15.131:0/3754577987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd78010d9d0 0x7fd78019de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:11.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:11.100+0000 7fd788600700 1 --2- 172.21.15.131:0/3754577987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7800731d0 0x7fd7801995f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:11.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:11.100+0000 7fd788600700 1 --2- 172.21.15.131:0/3754577987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd780073af0 0x7fd780199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:11.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:11.100+0000 7fd788600700 1 -- 172.21.15.131:0/3754577987 >> 172.21.15.131:0/3754577987 conn(0x7fd780078100 msgr2=0x7fd7800fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:11.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:11.100+0000 7fd788600700 1 -- 172.21.15.131:0/3754577987 shutdown_connections 2024-05-08T22:07:11.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:11.100+0000 7fd788600700 1 -- 172.21.15.131:0/3754577987 wait complete. 2024-05-08T22:07:11.124 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:10 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:10.799+0000 7fed4830b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T22:07:11.124 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:10 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:10.918+0000 7fed4830b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T22:07:11.340 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:11 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:07:11.069+0000 7f73e3650200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T22:07:11.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:11 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3754577987' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T22:07:11.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:11 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3754577987' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T22:07:11.526 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:11 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:07:11.096+0000 7f1c761dd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T22:07:11.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:11 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3754577987' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T22:07:11.539 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T22:07:11.539 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T22:07:11.539 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-05-08T22:07:11.539 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T22:07:11.539 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T22:07:11.539 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T22:07:11.539 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T22:07:11.539 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T22:07:11.539 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:07:11.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:07:11.540 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-05-08T22:07:11.583 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:07:11.584 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:07:11.584 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:07:11.584 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:07:11.584 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-05-08T22:07:11.584 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:07:11.584 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T22:07:11.584 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:07:11.584 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-05-08T22:07:11.584 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T22:07:11.584 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:07:11.584 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-05-08T22:07:11.584 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T22:07:11.584 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:07:11.585 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T22:07:11.585 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:07:11.585 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T22:07:11.585 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 41653sec preferred_lft 41653sec 2024-05-08T22:07:11.585 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-05-08T22:07:11.585 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:07:11.586 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T22:07:11.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:07:11.586 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2024-05-08T22:07:11.586 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-08T22:07:11.586 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add name ceph-brx type bridge 2024-05-08T22:07:11.586 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip addr flush dev ceph-brx 2024-05-08T22:07:11.586 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set ceph-brx up 2024-05-08T22:07:11.587 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T22:07:11.587 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-08T22:07:11.726 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:11 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:11.426+0000 7fed4830b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T22:07:11.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:07:11.832 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T22:07:11.902 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-05-08T22:07:11.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:07:11.906 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-05-08T22:07:11.950 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-05-08T22:07:11.950 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T22:07:11.951 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T22:07:11.951 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T22:07:11.951 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T22:07:11.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:07:11.952 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2024-05-08T22:07:11.952 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-08T22:07:11.952 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T22:07:11.952 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T22:07:11.952 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T22:07:11.952 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-08T22:07:12.090 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:11 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:07:11.832+0000 7f73e3650200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T22:07:12.090 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:11 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:07:11.928+0000 7f73e3650200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T22:07:12.090 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:12 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:07:12.077+0000 7f73e3650200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T22:07:12.090 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:11 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:11.974+0000 7fed4830b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T22:07:12.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:07:12.162 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-05-08T22:07:12.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:07:12.205 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list-id 2024-05-08T22:07:12.276 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:11 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:07:11.865+0000 7f1c761dd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T22:07:12.276 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:11 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:07:11.963+0000 7f1c761dd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T22:07:12.276 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:12 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:07:12.113+0000 7f1c761dd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T22:07:12.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:07:12.288 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2024-05-08T22:07:12.288 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-08T22:07:12.288 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T22:07:12.288 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-08T22:07:12.288 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-08T22:07:12.385 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:12 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:12.250+0000 7fed4830b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T22:07:12.385 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:12 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:12.384+0000 7fed4830b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T22:07:12.471 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-08T22:07:12.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:07:12.471 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2024-05-08T22:07:12.471 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-08T22:07:12.471 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-08T22:07:12.471 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T22:07:12.471 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-08T22:07:12.472 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-08T22:07:12.472 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-08T22:07:12.472 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-08T22:07:12.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:07:12.729 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2024-05-08T22:07:12.729 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-08T22:07:12.729 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-05-08T22:07:12.729 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T22:07:12.729 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-08T22:07:12.899 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:07:12.899 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T22:07:12.941 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:07:12.942 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:07:12.942 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:07:13.026 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-08T22:07:13.026 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-08T22:07:13.027 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/8d0ca3e4-ef2e-4394-b208-510ba6b717e9 2024-05-08T22:07:13.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:07:13.027 DEBUG:teuthology.orchestra.run.smithi131:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/8d0ca3e4-ef2e-4394-b208-510ba6b717e9 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-05-08T22:07:13.027 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:12 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:07:12.750+0000 7f1c761dd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T22:07:13.027 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:12 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:07:12.934+0000 7f1c761dd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T22:07:13.086 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:12 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:07:12.707+0000 7f73e3650200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T22:07:13.086 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:12 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:07:12.889+0000 7f73e3650200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T22:07:13.169 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-05-08T22:07:13.170 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-05-08T22:07:13.170 INFO:teuthology.orchestra.run.smithi131.stdout:invalid new device string format 2024-05-08T22:07:13.339 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:13 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:07:13.085+0000 7f73e3650200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T22:07:13.526 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:13 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:07:13.130+0000 7f1c761dd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T22:07:13.795 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:13 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:13.453+0000 7fed4830b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T22:07:13.795 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:13 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:13.589+0000 7fed4830b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T22:07:14.140 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:13 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:13.794+0000 7fed4830b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T22:07:14.315 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:14 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:07:14.022+0000 7f73e3650200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T22:07:14.362 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:14 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:07:14.073+0000 7f1c761dd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T22:07:14.590 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:14 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:07:14.314+0000 7f73e3650200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:07:14.590 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:14 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:07:14.403+0000 7f73e3650200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T22:07:14.646 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:14 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:07:14.361+0000 7f1c761dd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:07:14.646 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:14 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:07:14.451+0000 7f1c761dd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T22:07:14.647 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:07:14 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z[79601]: 2024-05-08T22:07:14.645+0000 7f1c761dd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T22:07:14.894 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:14 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:14.640+0000 7fed4830b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T22:07:14.895 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:14 smithi131 ceph-mon[37260]: Standby manager daemon y restarted 2024-05-08T22:07:14.895 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:14 smithi131 ceph-mon[37260]: Standby manager daemon y started 2024-05-08T22:07:14.895 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:14 smithi131 ceph-mon[37260]: from='mgr.? 172.21.15.132:0/1274168110' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T22:07:14.895 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:14 smithi131 ceph-mon[37260]: from='mgr.? 172.21.15.132:0/1274168110' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T22:07:14.895 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:14 smithi131 ceph-mon[37260]: from='mgr.? 172.21.15.132:0/1274168110' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T22:07:14.895 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:14 smithi131 ceph-mon[37260]: from='mgr.? 172.21.15.132:0/1274168110' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T22:07:14.895 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:14 smithi131 ceph-mon[37260]: Standby manager daemon z restarted 2024-05-08T22:07:14.895 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:14 smithi131 ceph-mon[37260]: Standby manager daemon z started 2024-05-08T22:07:15.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:14 smithi157 ceph-mon[77746]: Standby manager daemon y restarted 2024-05-08T22:07:15.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:14 smithi157 ceph-mon[77746]: Standby manager daemon y started 2024-05-08T22:07:15.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:14 smithi157 ceph-mon[77746]: from='mgr.? 172.21.15.132:0/1274168110' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T22:07:15.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:14 smithi157 ceph-mon[77746]: from='mgr.? 172.21.15.132:0/1274168110' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T22:07:15.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:14 smithi157 ceph-mon[77746]: from='mgr.? 172.21.15.132:0/1274168110' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T22:07:15.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:14 smithi157 ceph-mon[77746]: from='mgr.? 172.21.15.132:0/1274168110' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T22:07:15.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:14 smithi157 ceph-mon[77746]: Standby manager daemon z restarted 2024-05-08T22:07:15.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:14 smithi157 ceph-mon[77746]: Standby manager daemon z started 2024-05-08T22:07:15.090 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:07:14 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y[79666]: 2024-05-08T22:07:14.596+0000 7f73e3650200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T22:07:15.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:14 smithi132 ceph-mon[78180]: Standby manager daemon y restarted 2024-05-08T22:07:15.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:14 smithi132 ceph-mon[78180]: Standby manager daemon y started 2024-05-08T22:07:15.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:14 smithi132 ceph-mon[78180]: from='mgr.? 172.21.15.132:0/1274168110' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T22:07:15.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:14 smithi132 ceph-mon[78180]: from='mgr.? 172.21.15.132:0/1274168110' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T22:07:15.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:14 smithi132 ceph-mon[78180]: from='mgr.? 172.21.15.132:0/1274168110' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T22:07:15.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:14 smithi132 ceph-mon[78180]: from='mgr.? 172.21.15.132:0/1274168110' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T22:07:15.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:14 smithi132 ceph-mon[78180]: Standby manager daemon z restarted 2024-05-08T22:07:15.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:14 smithi132 ceph-mon[78180]: Standby manager daemon z started 2024-05-08T22:07:15.157 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:14 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:14.893+0000 7fed4830b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T22:07:15.640 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:15 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:15.156+0000 7fed4830b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T22:07:16.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:15 smithi157 ceph-mon[77746]: from='mgr.? 172.21.15.157:0/2280240829' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T22:07:16.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:15 smithi157 ceph-mon[77746]: from='mgr.? 172.21.15.157:0/2280240829' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T22:07:16.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:15 smithi157 ceph-mon[77746]: from='mgr.? 172.21.15.157:0/2280240829' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T22:07:16.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:15 smithi157 ceph-mon[77746]: from='mgr.? 172.21.15.157:0/2280240829' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T22:07:16.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:15 smithi157 ceph-mon[77746]: mgrmap e19: x(active, since 7m), standbys: y, z 2024-05-08T22:07:16.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:15 smithi132 ceph-mon[78180]: from='mgr.? 172.21.15.157:0/2280240829' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T22:07:16.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:15 smithi132 ceph-mon[78180]: from='mgr.? 172.21.15.157:0/2280240829' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T22:07:16.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:15 smithi132 ceph-mon[78180]: from='mgr.? 172.21.15.157:0/2280240829' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T22:07:16.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:15 smithi132 ceph-mon[78180]: from='mgr.? 172.21.15.157:0/2280240829' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T22:07:16.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:15 smithi132 ceph-mon[78180]: mgrmap e19: x(active, since 7m), standbys: y, z 2024-05-08T22:07:16.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:15 smithi131 ceph-mon[37260]: from='mgr.? 172.21.15.157:0/2280240829' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T22:07:16.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:15 smithi131 ceph-mon[37260]: from='mgr.? 172.21.15.157:0/2280240829' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T22:07:16.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:15 smithi131 ceph-mon[37260]: from='mgr.? 172.21.15.157:0/2280240829' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T22:07:16.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:15 smithi131 ceph-mon[37260]: from='mgr.? 172.21.15.157:0/2280240829' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T22:07:16.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:15 smithi131 ceph-mon[37260]: mgrmap e19: x(active, since 7m), standbys: y, z 2024-05-08T22:07:16.640 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:16 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:16.325+0000 7fed4830b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T22:07:17.094 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:16 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:16.710+0000 7fed4830b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:07:17.094 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:16 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:16.831+0000 7fed4830b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T22:07:17.130 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-05-08T22:07:17.130 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-05-08T22:07:17.130 INFO:teuthology.orchestra.run.smithi131.stdout:invalid new device string format 2024-05-08T22:07:17.130 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: resolved to: "172.21.15.131:6789,172.21.15.132:6789,172.21.15.157:6789" 2024-05-08T22:07:17.130 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.131:6789,172.21.15.132:6789,172.21.15.157:6789:/volumes/qa/sv_1/8d0ca3e4-ef2e-4394-b208-510ba6b717e9 2024-05-08T22:07:17.130 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=06f1a696-0d86-11ef-bc97-c7b262605968" will pass to kernel 2024-05-08T22:07:17.143 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T22:07:17.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:07:17.143 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:07:17.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:07:17.239 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T22:07:17.345 INFO:teuthology.orchestra.run.smithi131.stdout:client25426 2024-05-08T22:07:17.345 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-08T22:07:17.346 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:07:17.391 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:17 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:17.093+0000 7fed4830b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T22:07:17.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: Active manager daemon x restarted 2024-05-08T22:07:17.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: Activating manager daemon x 2024-05-08T22:07:17.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: osdmap e88: 12 total, 12 up, 12 in 2024-05-08T22:07:17.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: mgrmap e20: x(active, starting, since 0.0102559s), standbys: y, z 2024-05-08T22:07:17.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:07:17.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:07:17.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:07:17.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T22:07:17.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T22:07:17.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T22:07:17.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T22:07:17.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T22:07:17.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T22:07:17.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T22:07:17.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T22:07:17.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T22:07:17.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T22:07:17.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T22:07:17.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T22:07:17.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T22:07:17.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T22:07:17.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T22:07:17.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:07:17.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:07:17.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:07:17.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:07:17.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:07:17.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:07:17.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:07:17.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:07:17.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:07:17.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:07:17.395 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:07:17.395 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:07:17.395 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T22:07:17.395 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T22:07:17.395 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:17 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T22:07:17.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:07:17.402 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T22:07:17.409 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: Active manager daemon x restarted 2024-05-08T22:07:17.410 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: Activating manager daemon x 2024-05-08T22:07:17.410 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: osdmap e88: 12 total, 12 up, 12 in 2024-05-08T22:07:17.410 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: mgrmap e20: x(active, starting, since 0.0102559s), standbys: y, z 2024-05-08T22:07:17.410 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:07:17.410 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:07:17.410 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:07:17.410 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T22:07:17.410 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T22:07:17.410 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T22:07:17.410 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T22:07:17.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T22:07:17.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T22:07:17.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T22:07:17.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T22:07:17.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T22:07:17.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T22:07:17.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T22:07:17.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T22:07:17.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T22:07:17.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T22:07:17.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T22:07:17.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:07:17.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:07:17.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:07:17.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:07:17.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:07:17.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:07:17.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:07:17.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:07:17.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:07:17.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:07:17.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:07:17.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:07:17.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T22:07:17.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T22:07:17.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:17 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T22:07:17.421 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: Active manager daemon x restarted 2024-05-08T22:07:17.421 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: Activating manager daemon x 2024-05-08T22:07:17.421 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: osdmap e88: 12 total, 12 up, 12 in 2024-05-08T22:07:17.421 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: mgrmap e20: x(active, starting, since 0.0102559s), standbys: y, z 2024-05-08T22:07:17.421 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:07:17.422 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:07:17.422 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:07:17.422 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T22:07:17.422 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T22:07:17.422 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T22:07:17.422 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T22:07:17.422 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T22:07:17.422 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T22:07:17.422 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T22:07:17.422 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T22:07:17.422 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T22:07:17.422 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T22:07:17.422 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T22:07:17.422 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T22:07:17.422 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T22:07:17.423 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T22:07:17.423 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T22:07:17.423 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:07:17.423 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:07:17.423 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:07:17.423 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:07:17.423 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:07:17.423 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:07:17.423 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:07:17.423 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:07:17.423 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:07:17.423 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:07:17.423 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:07:17.423 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:07:17.423 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T22:07:17.424 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T22:07:17.424 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:17 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T22:07:17.499 INFO:teuthology.orchestra.run.smithi131.stdout:client25426 2024-05-08T22:07:17.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:07:17.499 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T22:07:17.584 INFO:teuthology.orchestra.run.smithi131.stdout:client25426 2024-05-08T22:07:17.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:07:17.585 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/06f1a696-0d86-11ef-bc97-c7b262605968.client25426/status') 2024-05-08T22:07:17.688 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:17 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:17.490+0000 7fec9d600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:07:17.688 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:17 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:17.490+0000 7fec9d600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:07:17.689 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:17 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:17.490+0000 7fec9d600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:07:17.689 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:17 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:17.490+0000 7fec99000700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:07:17.689 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:17 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:17.490+0000 7fec99000700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:07:17.689 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:17 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:17.490+0000 7fec99000700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:07:17.689 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:17 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:17.490+0000 7fec99000700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:07:17.689 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:17 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:17.490+0000 7fec99000700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:07:17.689 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:17 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:17.491+0000 7fec9d600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:07:17.689 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:17 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:17.491+0000 7fec9d600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:07:17.692 INFO:teuthology.orchestra.run.smithi131.stdout:instance: client.25426 (3)172.21.15.131:0/3960393696 2024-05-08T22:07:17.692 INFO:teuthology.orchestra.run.smithi131.stdout:blocklisted: false 2024-05-08T22:07:17.692 INFO:teuthology.orchestra.run.smithi131.stderr:0+1 records in 2024-05-08T22:07:17.692 INFO:teuthology.orchestra.run.smithi131.stderr:0+1 records out 2024-05-08T22:07:17.692 INFO:teuthology.orchestra.run.smithi131.stderr:72 bytes copied, 7.0096e-05 s, 1.0 MB/s 2024-05-08T22:07:17.696 INFO:teuthology.run_tasks:Running task exec... 2024-05-08T22:07:17.706 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T22:07:17.706 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi131.front.sepia.ceph.com 2024-05-08T22:07:17.706 DEBUG:teuthology.orchestra.run.smithi131:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-05-08T22:07:17.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.908+0000 7ffadf200700 1 -- 172.21.15.131:0/3871618353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffae0073a00 msgr2=0x7ffae0073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:17.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.908+0000 7ffadf200700 1 --2- 172.21.15.131:0/3871618353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffae0073a00 0x7ffae0073e80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009990 tx=0x7ffacc009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:17.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.908+0000 7ffadf200700 1 -- 172.21.15.131:0/3871618353 shutdown_connections 2024-05-08T22:07:17.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.908+0000 7ffadf200700 1 --2- 172.21.15.131:0/3871618353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae0108c20 0x7ffae010b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:17.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.908+0000 7ffadf200700 1 --2- 172.21.15.131:0/3871618353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffae0074fe0 0x7ffae0073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:17.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.908+0000 7ffadf200700 1 --2- 172.21.15.131:0/3871618353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffae0073a00 0x7ffae0073e80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:17.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.908+0000 7ffadf200700 1 -- 172.21.15.131:0/3871618353 >> 172.21.15.131:0/3871618353 conn(0x7ffae00fc2d0 msgr2=0x7ffae00fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:17.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.908+0000 7ffadf200700 1 -- 172.21.15.131:0/3871618353 shutdown_connections 2024-05-08T22:07:17.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.908+0000 7ffadf200700 1 -- 172.21.15.131:0/3871618353 wait complete. 2024-05-08T22:07:17.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.909+0000 7ffadf200700 1 Processor -- start 2024-05-08T22:07:17.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.909+0000 7ffadf200700 1 -- start start 2024-05-08T22:07:17.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.909+0000 7ffadf200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffae0073a00 0x7ffae0198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:17.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.909+0000 7ffadf200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffae0074fe0 0x7ffae0199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:17.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.909+0000 7ffadf200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae0108c20 0x7ffae01a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:17.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.910+0000 7ffadf200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae01a0a30 con 0x7ffae0074fe0 2024-05-08T22:07:17.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.910+0000 7ffadf200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae01a0b70 con 0x7ffae0073a00 2024-05-08T22:07:17.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.910+0000 7ffadf200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae01a0d00 con 0x7ffae0108c20 2024-05-08T22:07:17.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.910+0000 7ffadde00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffae0073a00 0x7ffae0198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:17.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.910+0000 7ffadd400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffae0074fe0 0x7ffae0199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:17.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.910+0000 7ffadde00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffae0073a00 0x7ffae0198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:37602/0 (socket says 172.21.15.131:37602) 2024-05-08T22:07:17.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.910+0000 7ffadde00700 1 -- 172.21.15.131:0/26516268 learned_addr learned my addr 172.21.15.131:0/26516268 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:17.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.910+0000 7ffade800700 1 --2- 172.21.15.131:0/26516268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae0108c20 0x7ffae01a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:17.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.910+0000 7ffadde00700 1 -- 172.21.15.131:0/26516268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae0108c20 msgr2=0x7ffae01a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:17.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.910+0000 7ffadde00700 1 --2- 172.21.15.131:0/26516268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae0108c20 0x7ffae01a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:17.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.910+0000 7ffadde00700 1 -- 172.21.15.131:0/26516268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffae0074fe0 msgr2=0x7ffae0199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:17.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.910+0000 7ffadde00700 1 --2- 172.21.15.131:0/26516268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffae0074fe0 0x7ffae0199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:17.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.910+0000 7ffadde00700 1 -- 172.21.15.131:0/26516268 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffacc0095f0 con 0x7ffae0073a00 2024-05-08T22:07:17.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.910+0000 7ffade800700 1 --2- 172.21.15.131:0/26516268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae0108c20 0x7ffae01a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:17.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.911+0000 7ffadde00700 1 --2- 172.21.15.131:0/26516268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffae0073a00 0x7ffae0198f00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffad000e9a0 tx=0x7ffad000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:17.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.911+0000 7ffad6a00700 1 -- 172.21.15.131:0/26516268 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad0009790 con 0x7ffae0073a00 2024-05-08T22:07:17.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.911+0000 7ffad6a00700 1 -- 172.21.15.131:0/26516268 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffad0010430 con 0x7ffae0073a00 2024-05-08T22:07:17.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.911+0000 7ffadf200700 1 -- 172.21.15.131:0/26516268 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae01a0fe0 con 0x7ffae0073a00 2024-05-08T22:07:17.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.911+0000 7ffad6a00700 1 -- 172.21.15.131:0/26516268 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad0018680 con 0x7ffae0073a00 2024-05-08T22:07:17.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.911+0000 7ffadf200700 1 -- 172.21.15.131:0/26516268 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae01a1410 con 0x7ffae0073a00 2024-05-08T22:07:17.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.912+0000 7ffadf200700 1 -- 172.21.15.131:0/26516268 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae01079e0 con 0x7ffae0073a00 2024-05-08T22:07:17.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.915+0000 7ffad6a00700 1 -- 172.21.15.131:0/26516268 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 20) v1 ==== 135558+0+0 (secure 0 0 0) 0x7ffad00105a0 con 0x7ffae0073a00 2024-05-08T22:07:17.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.917+0000 7ffad6a00700 1 -- 172.21.15.131:0/26516268 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7ffad0014070 con 0x7ffae0073a00 2024-05-08T22:07:17.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:17.925+0000 7ffad6a00700 1 -- 172.21.15.131:0/26516268 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ffad0052d60 con 0x7ffae0073a00 2024-05-08T22:07:18.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:18.176+0000 7ffad6a00700 1 -- 172.21.15.131:0/26516268 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mgrmap(e 21) v1 ==== 135685+0+0 (secure 0 0 0) 0x7ffad0010880 con 0x7ffae0073a00 2024-05-08T22:07:18.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:18.177+0000 7ffad6a00700 1 --2- 172.21.15.131:0/26516268 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ffac40a9600 0x7ffac40ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:18.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:18.180+0000 7ffadd400700 1 --2- 172.21.15.131:0/26516268 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ffac40a9600 0x7ffac40ab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:18.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:18.191+0000 7ffadd400700 1 --2- 172.21.15.131:0/26516268 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ffac40a9600 0x7ffac40ab9f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffacc000c00 tx=0x7ffacc005fb0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:18.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:18.230+0000 7ffadf200700 1 -- 172.21.15.131:0/26516268 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7ffae004ec60 con 0x7ffae0073a00 2024-05-08T22:07:18.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:18 smithi157 ceph-mon[77746]: Manager daemon x is now available 2024-05-08T22:07:18.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:18 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:07:18.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:18 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T22:07:18.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:18 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T22:07:18.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:18 smithi132 ceph-mon[78180]: Manager daemon x is now available 2024-05-08T22:07:18.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:18 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:07:18.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:18 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T22:07:18.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:18 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T22:07:18.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:18 smithi131 ceph-mon[37260]: Manager daemon x is now available 2024-05-08T22:07:18.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:18 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:07:18.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:18 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T22:07:18.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:18 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T22:07:19.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.175+0000 7ffad6a00700 1 -- 172.21.15.131:0/26516268 <== mon.2 v2:172.21.15.132:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7ffad00909d0 con 0x7ffae0073a00 2024-05-08T22:07:19.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.178+0000 7ffadf200700 1 -- 172.21.15.131:0/26516268 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ffac40a9600 msgr2=0x7ffac40ab9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:19.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.178+0000 7ffadf200700 1 --2- 172.21.15.131:0/26516268 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ffac40a9600 0x7ffac40ab9f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffacc000c00 tx=0x7ffacc005fb0 comp rx=0 tx=0).stop 2024-05-08T22:07:19.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.178+0000 7ffadf200700 1 -- 172.21.15.131:0/26516268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffae0073a00 msgr2=0x7ffae0198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:19.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.178+0000 7ffadf200700 1 --2- 172.21.15.131:0/26516268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffae0073a00 0x7ffae0198f00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffad000e9a0 tx=0x7ffad000ecb0 comp rx=0 tx=0).stop 2024-05-08T22:07:19.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.178+0000 7ffadf200700 1 -- 172.21.15.131:0/26516268 shutdown_connections 2024-05-08T22:07:19.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.178+0000 7ffadf200700 1 --2- 172.21.15.131:0/26516268 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ffac40a9600 0x7ffac40ab9f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:19.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.178+0000 7ffadf200700 1 --2- 172.21.15.131:0/26516268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae0108c20 0x7ffae01a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:19.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.178+0000 7ffadf200700 1 --2- 172.21.15.131:0/26516268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffae0073a00 0x7ffae0198f00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:19.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.178+0000 7ffadf200700 1 --2- 172.21.15.131:0/26516268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffae0074fe0 0x7ffae0199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:19.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.178+0000 7ffadf200700 1 -- 172.21.15.131:0/26516268 >> 172.21.15.131:0/26516268 conn(0x7ffae00fc2d0 msgr2=0x7ffae0104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:19.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.179+0000 7ffadf200700 1 -- 172.21.15.131:0/26516268 shutdown_connections 2024-05-08T22:07:19.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.179+0000 7ffadf200700 1 -- 172.21.15.131:0/26516268 wait complete. 2024-05-08T22:07:19.199 DEBUG:teuthology.orchestra.run.smithi131:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-05-08T22:07:19.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.413+0000 7f99c5800700 1 -- 172.21.15.131:0/3952084826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c0073a00 msgr2=0x7f99c0073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:19.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.413+0000 7f99c5800700 1 --2- 172.21.15.131:0/3952084826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c0073a00 0x7f99c0073e80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f99ac009990 tx=0x7f99ac009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:19.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.413+0000 7f99c5800700 1 -- 172.21.15.131:0/3952084826 shutdown_connections 2024-05-08T22:07:19.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.413+0000 7f99c5800700 1 --2- 172.21.15.131:0/3952084826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99c010d4f0 0x7f99c010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:19.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.413+0000 7f99c5800700 1 --2- 172.21.15.131:0/3952084826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99c0074fe0 0x7f99c0073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:19.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.413+0000 7f99c5800700 1 --2- 172.21.15.131:0/3952084826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c0073a00 0x7f99c0073e80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:19.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.413+0000 7f99c5800700 1 -- 172.21.15.131:0/3952084826 >> 172.21.15.131:0/3952084826 conn(0x7f99c00fc490 msgr2=0x7f99c00fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:19.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.413+0000 7f99c5800700 1 -- 172.21.15.131:0/3952084826 shutdown_connections 2024-05-08T22:07:19.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.413+0000 7f99c5800700 1 -- 172.21.15.131:0/3952084826 wait complete. 2024-05-08T22:07:19.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.414+0000 7f99c5800700 1 Processor -- start 2024-05-08T22:07:19.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.414+0000 7f99c5800700 1 -- start start 2024-05-08T22:07:19.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.415+0000 7f99c5800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c0073a00 0x7f99c019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:19.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.415+0000 7f99c5800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99c0074fe0 0x7f99c019d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:19.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.415+0000 7f99c5800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99c010d4f0 0x7f99c01a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:19.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.415+0000 7f99c5800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c01a4f70 con 0x7f99c0074fe0 2024-05-08T22:07:19.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.415+0000 7f99c5800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c01a50b0 con 0x7f99c010d4f0 2024-05-08T22:07:19.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.415+0000 7f99c5800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c01a5240 con 0x7f99c0073a00 2024-05-08T22:07:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.415+0000 7f99bea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c0073a00 0x7f99c019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.415+0000 7f99bea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c0073a00 0x7f99c019d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:38174/0 (socket says 172.21.15.131:38174) 2024-05-08T22:07:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.415+0000 7f99be000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99c0074fe0 0x7f99c019d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.415+0000 7f99bea00700 1 -- 172.21.15.131:0/1580072535 learned_addr learned my addr 172.21.15.131:0/1580072535 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.415+0000 7f99bf400700 1 --2- 172.21.15.131:0/1580072535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99c010d4f0 0x7f99c01a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.415+0000 7f99bea00700 1 -- 172.21.15.131:0/1580072535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99c010d4f0 msgr2=0x7f99c01a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.415+0000 7f99bea00700 1 --2- 172.21.15.131:0/1580072535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99c010d4f0 0x7f99c01a49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.415+0000 7f99bea00700 1 -- 172.21.15.131:0/1580072535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99c0074fe0 msgr2=0x7f99c019d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.415+0000 7f99bea00700 1 --2- 172.21.15.131:0/1580072535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99c0074fe0 0x7f99c019d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:19.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.415+0000 7f99bea00700 1 -- 172.21.15.131:0/1580072535 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b0009520 con 0x7f99c0073a00 2024-05-08T22:07:19.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.416+0000 7f99bea00700 1 --2- 172.21.15.131:0/1580072535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c0073a00 0x7f99c019d440 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f99b000eb30 tx=0x7f99b000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:19.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.416+0000 7f99b7400700 1 -- 172.21.15.131:0/1580072535 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b0009880 con 0x7f99c0073a00 2024-05-08T22:07:19.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.416+0000 7f99b7400700 1 -- 172.21.15.131:0/1580072535 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99b00044e0 con 0x7f99c0073a00 2024-05-08T22:07:19.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.416+0000 7f99c5800700 1 -- 172.21.15.131:0/1580072535 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b8009c40 con 0x7f99c0073a00 2024-05-08T22:07:19.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.416+0000 7f99b7400700 1 -- 172.21.15.131:0/1580072535 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b0003d80 con 0x7f99c0073a00 2024-05-08T22:07:19.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.416+0000 7f99c5800700 1 -- 172.21.15.131:0/1580072535 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99ac0095f0 con 0x7f99c0073a00 2024-05-08T22:07:19.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.417+0000 7f99c5800700 1 -- 172.21.15.131:0/1580072535 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c0103750 con 0x7f99c0073a00 2024-05-08T22:07:19.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.419+0000 7f99b7400700 1 -- 172.21.15.131:0/1580072535 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 21) v1 ==== 135685+0+0 (secure 0 0 0) 0x7f99b0004020 con 0x7f99c0073a00 2024-05-08T22:07:19.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.420+0000 7f99b7400700 1 --2- 172.21.15.131:0/1580072535 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f99a40a0fa0 0x7f99a40a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:19.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.420+0000 7f99b7400700 1 -- 172.21.15.131:0/1580072535 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f99b0014070 con 0x7f99c0073a00 2024-05-08T22:07:19.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.420+0000 7f99be000700 1 --2- 172.21.15.131:0/1580072535 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f99a40a0fa0 0x7f99a40a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:19.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.421+0000 7f99be000700 1 --2- 172.21.15.131:0/1580072535 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f99a40a0fa0 0x7f99a40a3460 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00b680 tx=0x7f99ac005fd0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:19.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.424+0000 7f99b7400700 1 -- 172.21.15.131:0/1580072535 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f99b0096320 con 0x7f99c0073a00 2024-05-08T22:07:19.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:19 smithi157 ceph-mon[77746]: mgrmap e21: x(active, since 1.07699s), standbys: y, z 2024-05-08T22:07:19.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:19 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/26516268' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-08T22:07:19.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:19 smithi157 ceph-mon[77746]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-08T22:07:19.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:19 smithi157 ceph-mon[77746]: [08/May/2024:22:07:18] ENGINE Bus STARTING 2024-05-08T22:07:19.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:19 smithi157 ceph-mon[77746]: [08/May/2024:22:07:18] ENGINE Serving on http://172.21.15.131:8765 2024-05-08T22:07:19.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:19 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:19.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:19 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:19.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:19 smithi157 ceph-mon[77746]: [08/May/2024:22:07:18] ENGINE Serving on https://172.21.15.131:7150 2024-05-08T22:07:19.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:19 smithi157 ceph-mon[77746]: [08/May/2024:22:07:18] ENGINE Bus STARTED 2024-05-08T22:07:19.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:19 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:19.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:19 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:19.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:19 smithi132 ceph-mon[78180]: mgrmap e21: x(active, since 1.07699s), standbys: y, z 2024-05-08T22:07:19.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:19 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/26516268' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-08T22:07:19.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:19 smithi132 ceph-mon[78180]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-08T22:07:19.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:19 smithi132 ceph-mon[78180]: [08/May/2024:22:07:18] ENGINE Bus STARTING 2024-05-08T22:07:19.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:19 smithi132 ceph-mon[78180]: [08/May/2024:22:07:18] ENGINE Serving on http://172.21.15.131:8765 2024-05-08T22:07:19.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:19 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:19.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:19 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:19.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:19 smithi132 ceph-mon[78180]: [08/May/2024:22:07:18] ENGINE Serving on https://172.21.15.131:7150 2024-05-08T22:07:19.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:19 smithi132 ceph-mon[78180]: [08/May/2024:22:07:18] ENGINE Bus STARTED 2024-05-08T22:07:19.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:19 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:19.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:19 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:19.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:19 smithi131 ceph-mon[37260]: mgrmap e21: x(active, since 1.07699s), standbys: y, z 2024-05-08T22:07:19.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:19 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/26516268' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-08T22:07:19.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:19 smithi131 ceph-mon[37260]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-08T22:07:19.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:19 smithi131 ceph-mon[37260]: [08/May/2024:22:07:18] ENGINE Bus STARTING 2024-05-08T22:07:19.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:19 smithi131 ceph-mon[37260]: [08/May/2024:22:07:18] ENGINE Serving on http://172.21.15.131:8765 2024-05-08T22:07:19.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:19 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:19.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:19 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:19.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:19 smithi131 ceph-mon[37260]: [08/May/2024:22:07:18] ENGINE Serving on https://172.21.15.131:7150 2024-05-08T22:07:19.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:19 smithi131 ceph-mon[37260]: [08/May/2024:22:07:18] ENGINE Bus STARTED 2024-05-08T22:07:19.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:19 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:19.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:19 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:19.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.686+0000 7f99c5800700 1 -- 172.21.15.131:0/1580072535 --> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7f99c00613e0 con 0x7f99a40a0fa0 2024-05-08T22:07:19.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.706+0000 7f99b7400700 1 -- 172.21.15.131:0/1580072535 <== mgr.15366 v2:172.21.15.131:6800/1157588501 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7f99c00613e0 con 0x7f99a40a0fa0 2024-05-08T22:07:19.707 INFO:teuthology.orchestra.run.smithi131.stdout:{} 2024-05-08T22:07:19.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.708+0000 7f99c5800700 1 -- 172.21.15.131:0/1580072535 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f99a40a0fa0 msgr2=0x7f99a40a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:19.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.708+0000 7f99c5800700 1 --2- 172.21.15.131:0/1580072535 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f99a40a0fa0 0x7f99a40a3460 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00b680 tx=0x7f99ac005fd0 comp rx=0 tx=0).stop 2024-05-08T22:07:19.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.708+0000 7f99c5800700 1 -- 172.21.15.131:0/1580072535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c0073a00 msgr2=0x7f99c019d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:19.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.708+0000 7f99c5800700 1 --2- 172.21.15.131:0/1580072535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c0073a00 0x7f99c019d440 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f99b000eb30 tx=0x7f99b000ee40 comp rx=0 tx=0).stop 2024-05-08T22:07:19.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.708+0000 7f99c5800700 1 -- 172.21.15.131:0/1580072535 shutdown_connections 2024-05-08T22:07:19.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.708+0000 7f99c5800700 1 --2- 172.21.15.131:0/1580072535 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f99a40a0fa0 0x7f99a40a3460 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:19.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.708+0000 7f99c5800700 1 --2- 172.21.15.131:0/1580072535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99c010d4f0 0x7f99c01a49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:19.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.708+0000 7f99c5800700 1 --2- 172.21.15.131:0/1580072535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c0073a00 0x7f99c019d440 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:19.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.708+0000 7f99c5800700 1 --2- 172.21.15.131:0/1580072535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99c0074fe0 0x7f99c019d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:19.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.708+0000 7f99c5800700 1 -- 172.21.15.131:0/1580072535 >> 172.21.15.131:0/1580072535 conn(0x7f99c00fc490 msgr2=0x7f99c0100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:19.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.708+0000 7f99c5800700 1 -- 172.21.15.131:0/1580072535 shutdown_connections 2024-05-08T22:07:19.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.708+0000 7f99c5800700 1 -- 172.21.15.131:0/1580072535 wait complete. 2024-05-08T22:07:19.728 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-08T22:07:19.741 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-08T22:07:19.765 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:07:19.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.974+0000 7f8923e00700 1 -- 172.21.15.131:0/890374350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f891c074fe0 msgr2=0x7f891c073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:19.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.974+0000 7f8923e00700 1 --2- 172.21.15.131:0/890374350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f891c074fe0 0x7f891c073430 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8910009990 tx=0x7f8910009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:19.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.974+0000 7f8923e00700 1 -- 172.21.15.131:0/890374350 shutdown_connections 2024-05-08T22:07:19.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.974+0000 7f8923e00700 1 --2- 172.21.15.131:0/890374350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f891c10d440 0x7f891c10f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:19.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.974+0000 7f8923e00700 1 --2- 172.21.15.131:0/890374350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f891c074fe0 0x7f891c073430 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:19.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.974+0000 7f8923e00700 1 --2- 172.21.15.131:0/890374350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f891c073a00 0x7f891c073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:19.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.974+0000 7f8923e00700 1 -- 172.21.15.131:0/890374350 >> 172.21.15.131:0/890374350 conn(0x7f891c0fc3e0 msgr2=0x7f891c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:19.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.974+0000 7f8923e00700 1 -- 172.21.15.131:0/890374350 shutdown_connections 2024-05-08T22:07:19.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.974+0000 7f8923e00700 1 -- 172.21.15.131:0/890374350 wait complete. 2024-05-08T22:07:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.975+0000 7f8923e00700 1 Processor -- start 2024-05-08T22:07:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.976+0000 7f8923e00700 1 -- start start 2024-05-08T22:07:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.976+0000 7f8923e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f891c073a00 0x7f891c19d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.976+0000 7f8923e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f891c074fe0 0x7f891c19d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.976+0000 7f8923e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f891c10d440 0x7f891c1a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.976+0000 7f8923e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c1a4f00 con 0x7f891c10d440 2024-05-08T22:07:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.976+0000 7f8923e00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c1a5040 con 0x7f891c074fe0 2024-05-08T22:07:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.976+0000 7f8923e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c1a51d0 con 0x7f891c073a00 2024-05-08T22:07:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.976+0000 7f8921400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f891c073a00 0x7f891c19d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.977+0000 7f8920a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f891c074fe0 0x7f891c19d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.977+0000 7f8921400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f891c073a00 0x7f891c19d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:38216/0 (socket says 172.21.15.131:38216) 2024-05-08T22:07:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.977+0000 7f8921400700 1 -- 172.21.15.131:0/1826086872 learned_addr learned my addr 172.21.15.131:0/1826086872 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:19.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.977+0000 7f8921e00700 1 --2- 172.21.15.131:0/1826086872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f891c10d440 0x7f891c1a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:19.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.977+0000 7f8921400700 1 -- 172.21.15.131:0/1826086872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f891c074fe0 msgr2=0x7f891c19d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:19.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.977+0000 7f8921400700 1 --2- 172.21.15.131:0/1826086872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f891c074fe0 0x7f891c19d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:19.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.977+0000 7f8921400700 1 -- 172.21.15.131:0/1826086872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f891c10d440 msgr2=0x7f891c1a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:19.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.977+0000 7f8921400700 1 --2- 172.21.15.131:0/1826086872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f891c10d440 0x7f891c1a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:19.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.977+0000 7f8921400700 1 -- 172.21.15.131:0/1826086872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89100095f0 con 0x7f891c073a00 2024-05-08T22:07:19.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.977+0000 7f8921400700 1 --2- 172.21.15.131:0/1826086872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f891c073a00 0x7f891c19d3d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8910000c00 tx=0x7f8910004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:19.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.977+0000 7f891a000700 1 -- 172.21.15.131:0/1826086872 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891001d070 con 0x7f891c073a00 2024-05-08T22:07:19.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.977+0000 7f891a000700 1 -- 172.21.15.131:0/1826086872 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89100054f0 con 0x7f891c073a00 2024-05-08T22:07:19.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.977+0000 7f8923e00700 1 -- 172.21.15.131:0/1826086872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f891c1a5450 con 0x7f891c073a00 2024-05-08T22:07:19.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.977+0000 7f891a000700 1 -- 172.21.15.131:0/1826086872 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8910004bd0 con 0x7f891c073a00 2024-05-08T22:07:19.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.978+0000 7f8923e00700 1 -- 172.21.15.131:0/1826086872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891c1a5810 con 0x7f891c073a00 2024-05-08T22:07:19.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.978+0000 7f8923e00700 1 -- 172.21.15.131:0/1826086872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f891c1036a0 con 0x7f891c073a00 2024-05-08T22:07:19.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.980+0000 7f891a000700 1 -- 172.21.15.131:0/1826086872 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 22) v1 ==== 135994+0+0 (secure 0 0 0) 0x7f891000fb90 con 0x7f891c073a00 2024-05-08T22:07:19.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.981+0000 7f891a000700 1 --2- 172.21.15.131:0/1826086872 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f89040a1290 0x7f89040a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:19.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.982+0000 7f891a000700 1 -- 172.21.15.131:0/1826086872 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f89100ce260 con 0x7f891c073a00 2024-05-08T22:07:19.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.982+0000 7f8920a00700 1 --2- 172.21.15.131:0/1826086872 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f89040a1290 0x7f89040a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:19.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.982+0000 7f8920a00700 1 --2- 172.21.15.131:0/1826086872 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f89040a1290 0x7f89040a3750 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f891c074d00 tx=0x7f8908009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:19.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:19.985+0000 7f891a000700 1 -- 172.21.15.131:0/1826086872 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8910097350 con 0x7f891c073a00 2024-05-08T22:07:20.140 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:19.693+0000 7fec97200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:07:20.141 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:19.693+0000 7fec97200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:07:20.141 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:19.693+0000 7fec97200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:07:20.141 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:19.693+0000 7fec97200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:07:20.141 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:07:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:07:19.693+0000 7fec97200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:07:20.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.207+0000 7f8923e00700 1 -- 172.21.15.131:0/1826086872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f891c112c80 con 0x7f891c073a00 2024-05-08T22:07:20.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.209+0000 7f891a000700 1 -- 172.21.15.131:0/1826086872 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+25165 (secure 0 0 0) 0x7f891009a970 con 0x7f891c073a00 2024-05-08T22:07:20.210 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:07:20.210 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":88,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:07:17.096317+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:05:12.469858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:05:13.484446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:05:17.016186+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/2704707055":"2024-05-09T22:07:17.096274+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6801/2485665042":"2024-05-09T22:07:17.096274+0000","172.21.15.131:0/126236497":"2024-05-09T22:07:17.096274+0000","172.21.15.131:0/25489954":"2024-05-09T22:07:17.096274+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/1773942606":"2024-05-09T22:07:17.096274+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/2485665042":"2024-05-09T22:07:17.096274+0000","172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3120268590":"2024-05-09T22:07:17.096274+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:07:20.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.211+0000 7f8923e00700 1 -- 172.21.15.131:0/1826086872 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f89040a1290 msgr2=0x7f89040a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:20.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.211+0000 7f8923e00700 1 --2- 172.21.15.131:0/1826086872 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f89040a1290 0x7f89040a3750 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f891c074d00 tx=0x7f8908009260 comp rx=0 tx=0).stop 2024-05-08T22:07:20.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.211+0000 7f8923e00700 1 -- 172.21.15.131:0/1826086872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f891c073a00 msgr2=0x7f891c19d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:20.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.211+0000 7f8923e00700 1 --2- 172.21.15.131:0/1826086872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f891c073a00 0x7f891c19d3d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8910000c00 tx=0x7f8910004730 comp rx=0 tx=0).stop 2024-05-08T22:07:20.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.211+0000 7f8923e00700 1 -- 172.21.15.131:0/1826086872 shutdown_connections 2024-05-08T22:07:20.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.211+0000 7f8923e00700 1 --2- 172.21.15.131:0/1826086872 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f89040a1290 0x7f89040a3750 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:20.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.211+0000 7f8923e00700 1 --2- 172.21.15.131:0/1826086872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f891c10d440 0x7f891c1a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:20.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.211+0000 7f8923e00700 1 --2- 172.21.15.131:0/1826086872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f891c073a00 0x7f891c19d3d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:20.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.211+0000 7f8923e00700 1 --2- 172.21.15.131:0/1826086872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f891c074fe0 0x7f891c19d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:20.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.211+0000 7f8923e00700 1 -- 172.21.15.131:0/1826086872 >> 172.21.15.131:0/1826086872 conn(0x7f891c0fc3e0 msgr2=0x7f891c1007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:20.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.211+0000 7f8923e00700 1 -- 172.21.15.131:0/1826086872 shutdown_connections 2024-05-08T22:07:20.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.211+0000 7f8923e00700 1 -- 172.21.15.131:0/1826086872 wait complete. 2024-05-08T22:07:20.232 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:01:41.271078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:05:12.469858+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:05:13.484446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:05:17.016186+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:07:20.233 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:07:20.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.434+0000 7f2867a00700 1 -- 172.21.15.131:0/3377463854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2860102b10 msgr2=0x7f28601093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:20.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.434+0000 7f2867a00700 1 --2- 172.21.15.131:0/3377463854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2860102b10 0x7f28601093c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2858009f90 tx=0x7f285800b330 comp rx=0 tx=0).stop 2024-05-08T22:07:20.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.435+0000 7f2867a00700 1 -- 172.21.15.131:0/3377463854 shutdown_connections 2024-05-08T22:07:20.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.435+0000 7f2867a00700 1 --2- 172.21.15.131:0/3377463854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2860102b10 0x7f28601093c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:20.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.435+0000 7f2867a00700 1 --2- 172.21.15.131:0/3377463854 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2860100f80 0x7f28601013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:20.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.435+0000 7f2867a00700 1 --2- 172.21.15.131:0/3377463854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2860102170 0x7f28601025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:20.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.435+0000 7f2867a00700 1 -- 172.21.15.131:0/3377463854 >> 172.21.15.131:0/3377463854 conn(0x7f28600fc4f0 msgr2=0x7f28600fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:20.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.435+0000 7f2867a00700 1 -- 172.21.15.131:0/3377463854 shutdown_connections 2024-05-08T22:07:20.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.435+0000 7f2867a00700 1 -- 172.21.15.131:0/3377463854 wait complete. 2024-05-08T22:07:20.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.436+0000 7f2867a00700 1 Processor -- start 2024-05-08T22:07:20.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.436+0000 7f2867a00700 1 -- start start 2024-05-08T22:07:20.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.436+0000 7f2867a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2860100f80 0x7f28601992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:20.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.436+0000 7f2867a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2860102170 0x7f2860199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:20.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.436+0000 7f2867a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2860102b10 0x7f28601a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:20.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.436+0000 7f2867a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28601a0df0 con 0x7f2860102b10 2024-05-08T22:07:20.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.436+0000 7f2867a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28601a0f30 con 0x7f2860102170 2024-05-08T22:07:20.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.436+0000 7f2867a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28601a10c0 con 0x7f2860100f80 2024-05-08T22:07:20.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.436+0000 7f285fe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2860102170 0x7f2860199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:20.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.436+0000 7f2865000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2860100f80 0x7f28601992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:20.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.436+0000 7f285fe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2860102170 0x7f2860199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:37710/0 (socket says 172.21.15.131:37710) 2024-05-08T22:07:20.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.436+0000 7f285fe00700 1 -- 172.21.15.131:0/150574540 learned_addr learned my addr 172.21.15.131:0/150574540 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:20.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.437+0000 7f2865a00700 1 --2- 172.21.15.131:0/150574540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2860102b10 0x7f28601a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:20.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.437+0000 7f285fe00700 1 -- 172.21.15.131:0/150574540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2860100f80 msgr2=0x7f28601992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:20.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.437+0000 7f285fe00700 1 --2- 172.21.15.131:0/150574540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2860100f80 0x7f28601992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:20.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.437+0000 7f285fe00700 1 -- 172.21.15.131:0/150574540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2860102b10 msgr2=0x7f28601a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:20.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.437+0000 7f285fe00700 1 --2- 172.21.15.131:0/150574540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2860102b10 0x7f28601a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:20.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.437+0000 7f285fe00700 1 -- 172.21.15.131:0/150574540 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2858009c40 con 0x7f2860102170 2024-05-08T22:07:20.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.437+0000 7f2865a00700 1 --2- 172.21.15.131:0/150574540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2860102b10 0x7f28601a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:07:20.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.437+0000 7f285fe00700 1 --2- 172.21.15.131:0/150574540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2860102170 0x7f2860199800 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f284c00e9c0 tx=0x7f284c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:20.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.437+0000 7f285d600700 1 -- 172.21.15.131:0/150574540 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c00cbd0 con 0x7f2860102170 2024-05-08T22:07:20.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.437+0000 7f285d600700 1 -- 172.21.15.131:0/150574540 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f284c00eeb0 con 0x7f2860102170 2024-05-08T22:07:20.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.437+0000 7f2867a00700 1 -- 172.21.15.131:0/150574540 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28601a13a0 con 0x7f2860102170 2024-05-08T22:07:20.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.437+0000 7f285d600700 1 -- 172.21.15.131:0/150574540 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c005580 con 0x7f2860102170 2024-05-08T22:07:20.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.438+0000 7f2867a00700 1 -- 172.21.15.131:0/150574540 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28601a18b0 con 0x7f2860102170 2024-05-08T22:07:20.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.439+0000 7f2867a00700 1 -- 172.21.15.131:0/150574540 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2860067cf0 con 0x7f2860102170 2024-05-08T22:07:20.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.440+0000 7f285d600700 1 -- 172.21.15.131:0/150574540 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 22) v1 ==== 135994+0+0 (secure 0 0 0) 0x7f284c004750 con 0x7f2860102170 2024-05-08T22:07:20.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.442+0000 7f285d600700 1 --2- 172.21.15.131:0/150574540 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f28480a1290 0x7f28480a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:20.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.442+0000 7f285d600700 1 -- 172.21.15.131:0/150574540 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f284c014070 con 0x7f2860102170 2024-05-08T22:07:20.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.442+0000 7f2865000700 1 --2- 172.21.15.131:0/150574540 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f28480a1290 0x7f28480a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:20.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.442+0000 7f2865000700 1 --2- 172.21.15.131:0/150574540 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f28480a1290 0x7f28480a3750 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2860101fd0 tx=0x7f2854009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:20.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.444+0000 7f285d600700 1 -- 172.21.15.131:0/150574540 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f284c096520 con 0x7f2860102170 2024-05-08T22:07:20.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:20 smithi157 ceph-mon[77746]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:07:20.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:20 smithi157 ceph-mon[77746]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-08T22:07:20.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:20 smithi157 ceph-mon[77746]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T22:07:20.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:20 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:20.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:20 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:20.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:20 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:20.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:20 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:20.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:20 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:20.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:20 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:20.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:20 smithi157 ceph-mon[77746]: Adjusting osd_memory_target on smithi157 to 182.5M 2024-05-08T22:07:20.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:20 smithi157 ceph-mon[77746]: Unable to set osd_memory_target on smithi157 to 191374848: error parsing value: Value '191374848' is below minimum 939524096 2024-05-08T22:07:20.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:20 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:20.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:20 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:20.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:20 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:20.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:20 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:20.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:20 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:20.528 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:20 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:20.528 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:20 smithi157 ceph-mon[77746]: Adjusting osd_memory_target on smithi132 to 182.5M 2024-05-08T22:07:20.528 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:20 smithi157 ceph-mon[77746]: Unable to set osd_memory_target on smithi132 to 191378432: error parsing value: Value '191378432' is below minimum 939524096 2024-05-08T22:07:20.528 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:20 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:20.528 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:20 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:20.528 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:20 smithi157 ceph-mon[77746]: from='client.25477 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:07:20.528 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:20 smithi157 ceph-mon[77746]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-05-08T22:07:20.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:20 smithi132 ceph-mon[78180]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:07:20.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:20 smithi132 ceph-mon[78180]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-08T22:07:20.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:20 smithi132 ceph-mon[78180]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T22:07:20.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:20 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:20.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:20 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:20.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:20 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:20.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:20 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:20.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:20 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:20.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:20 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:20.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:20 smithi132 ceph-mon[78180]: Adjusting osd_memory_target on smithi157 to 182.5M 2024-05-08T22:07:20.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:20 smithi132 ceph-mon[78180]: Unable to set osd_memory_target on smithi157 to 191374848: error parsing value: Value '191374848' is below minimum 939524096 2024-05-08T22:07:20.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:20 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:20.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:20 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:20.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:20 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:20.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:20 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:20.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:20 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:20.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:20 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:20.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:20 smithi132 ceph-mon[78180]: Adjusting osd_memory_target on smithi132 to 182.5M 2024-05-08T22:07:20.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:20 smithi132 ceph-mon[78180]: Unable to set osd_memory_target on smithi132 to 191378432: error parsing value: Value '191378432' is below minimum 939524096 2024-05-08T22:07:20.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:20 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:20.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:20 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:20.592 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:20 smithi132 ceph-mon[78180]: from='client.25477 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:07:20.592 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:20 smithi132 ceph-mon[78180]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-05-08T22:07:20.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:20 smithi131 ceph-mon[37260]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:07:20.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:20 smithi131 ceph-mon[37260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-08T22:07:20.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:20 smithi131 ceph-mon[37260]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T22:07:20.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:20 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:20.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:20 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:20.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:20 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:20.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:20 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:20.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:20 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:20.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:20 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:20.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:20 smithi131 ceph-mon[37260]: Adjusting osd_memory_target on smithi157 to 182.5M 2024-05-08T22:07:20.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:20 smithi131 ceph-mon[37260]: Unable to set osd_memory_target on smithi157 to 191374848: error parsing value: Value '191374848' is below minimum 939524096 2024-05-08T22:07:20.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:20 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:20.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:20 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:20.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:20 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:20.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:20 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:20.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:20 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:20.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:20 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:20.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:20 smithi131 ceph-mon[37260]: Adjusting osd_memory_target on smithi132 to 182.5M 2024-05-08T22:07:20.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:20 smithi131 ceph-mon[37260]: Unable to set osd_memory_target on smithi132 to 191378432: error parsing value: Value '191378432' is below minimum 939524096 2024-05-08T22:07:20.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:20 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:20.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:20 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:20.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:20 smithi131 ceph-mon[37260]: from='client.25477 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:07:20.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:20 smithi131 ceph-mon[37260]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-05-08T22:07:20.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.660+0000 7f2867a00700 1 -- 172.21.15.131:0/150574540 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f286010c810 con 0x7f2860102170 2024-05-08T22:07:20.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.661+0000 7f285d600700 1 -- 172.21.15.131:0/150574540 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f284c099b40 con 0x7f2860102170 2024-05-08T22:07:20.662 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-08T22:07:20.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.663+0000 7f2867a00700 1 -- 172.21.15.131:0/150574540 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f28480a1290 msgr2=0x7f28480a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:20.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.663+0000 7f2867a00700 1 --2- 172.21.15.131:0/150574540 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f28480a1290 0x7f28480a3750 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2860101fd0 tx=0x7f2854009260 comp rx=0 tx=0).stop 2024-05-08T22:07:20.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.664+0000 7f2867a00700 1 -- 172.21.15.131:0/150574540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2860102170 msgr2=0x7f2860199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:20.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.664+0000 7f2867a00700 1 --2- 172.21.15.131:0/150574540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2860102170 0x7f2860199800 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f284c00e9c0 tx=0x7f284c00ecd0 comp rx=0 tx=0).stop 2024-05-08T22:07:20.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.664+0000 7f2867a00700 1 -- 172.21.15.131:0/150574540 shutdown_connections 2024-05-08T22:07:20.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.664+0000 7f2867a00700 1 --2- 172.21.15.131:0/150574540 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f28480a1290 0x7f28480a3750 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:20.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.664+0000 7f2867a00700 1 --2- 172.21.15.131:0/150574540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2860102b10 0x7f28601a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:20.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.664+0000 7f2867a00700 1 --2- 172.21.15.131:0/150574540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2860100f80 0x7f28601992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:20.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.664+0000 7f2867a00700 1 --2- 172.21.15.131:0/150574540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2860102170 0x7f2860199800 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:20.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.664+0000 7f2867a00700 1 -- 172.21.15.131:0/150574540 >> 172.21.15.131:0/150574540 conn(0x7f28600fc4f0 msgr2=0x7f28600731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:20.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.664+0000 7f2867a00700 1 -- 172.21.15.131:0/150574540 shutdown_connections 2024-05-08T22:07:20.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.664+0000 7f2867a00700 1 -- 172.21.15.131:0/150574540 wait complete. 2024-05-08T22:07:20.686 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:07:20.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.898+0000 7ff0f3e00700 1 -- 172.21.15.131:0/180648623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0f40739a0 msgr2=0x7ff0f4073e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:20.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.898+0000 7ff0f3e00700 1 --2- 172.21.15.131:0/180648623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0f40739a0 0x7ff0f4073e20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009990 tx=0x7ff0e0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:20.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.898+0000 7ff0f3e00700 1 -- 172.21.15.131:0/180648623 shutdown_connections 2024-05-08T22:07:20.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.898+0000 7ff0f3e00700 1 --2- 172.21.15.131:0/180648623 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0f4108cc0 0x7ff0f410b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:20.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.898+0000 7ff0f3e00700 1 --2- 172.21.15.131:0/180648623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0f4074f80 0x7ff0f40733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:20.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.898+0000 7ff0f3e00700 1 --2- 172.21.15.131:0/180648623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0f40739a0 0x7ff0f4073e20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:20.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.898+0000 7ff0f3e00700 1 -- 172.21.15.131:0/180648623 >> 172.21.15.131:0/180648623 conn(0x7ff0f40fc380 msgr2=0x7ff0f40fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:20.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.898+0000 7ff0f3e00700 1 -- 172.21.15.131:0/180648623 shutdown_connections 2024-05-08T22:07:20.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.898+0000 7ff0f3e00700 1 -- 172.21.15.131:0/180648623 wait complete. 2024-05-08T22:07:20.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.899+0000 7ff0f3e00700 1 Processor -- start 2024-05-08T22:07:20.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.899+0000 7ff0f3e00700 1 -- start start 2024-05-08T22:07:20.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.900+0000 7ff0f3e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0f40739a0 0x7ff0f4198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:20.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.900+0000 7ff0f3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0f4074f80 0x7ff0f4199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:20.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.900+0000 7ff0f3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0f4108cc0 0x7ff0f41a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:20.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.900+0000 7ff0f3e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f41a0a40 con 0x7ff0f4108cc0 2024-05-08T22:07:20.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.900+0000 7ff0f3e00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f41a0b80 con 0x7ff0f40739a0 2024-05-08T22:07:20.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.900+0000 7ff0f3e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f41a0d10 con 0x7ff0f4074f80 2024-05-08T22:07:20.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.900+0000 7ff0f2000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0f4074f80 0x7ff0f4199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:20.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.900+0000 7ff0f2a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0f40739a0 0x7ff0f4198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:20.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.900+0000 7ff0f2a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0f40739a0 0x7ff0f4198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:37738/0 (socket says 172.21.15.131:37738) 2024-05-08T22:07:20.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.900+0000 7ff0f2000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0f4074f80 0x7ff0f4199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:38246/0 (socket says 172.21.15.131:38246) 2024-05-08T22:07:20.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.900+0000 7ff0f2a00700 1 -- 172.21.15.131:0/971461814 learned_addr learned my addr 172.21.15.131:0/971461814 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:20.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.900+0000 7ff0f3400700 1 --2- 172.21.15.131:0/971461814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0f4108cc0 0x7ff0f41a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:20.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.900+0000 7ff0f2a00700 1 -- 172.21.15.131:0/971461814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0f4074f80 msgr2=0x7ff0f4199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:20.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.900+0000 7ff0f2a00700 1 --2- 172.21.15.131:0/971461814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0f4074f80 0x7ff0f4199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:20.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.900+0000 7ff0f2a00700 1 -- 172.21.15.131:0/971461814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0f4108cc0 msgr2=0x7ff0f41a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:20.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.900+0000 7ff0f2a00700 1 --2- 172.21.15.131:0/971461814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0f4108cc0 0x7ff0f41a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:20.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.900+0000 7ff0f2a00700 1 -- 172.21.15.131:0/971461814 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e4009520 con 0x7ff0f40739a0 2024-05-08T22:07:20.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.900+0000 7ff0f3400700 1 --2- 172.21.15.131:0/971461814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0f4108cc0 0x7ff0f41a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:20.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.901+0000 7ff0f2a00700 1 --2- 172.21.15.131:0/971461814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0f40739a0 0x7ff0f4198f10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400eb30 tx=0x7ff0e400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:20.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.901+0000 7ff0eb400700 1 -- 172.21.15.131:0/971461814 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4009880 con 0x7ff0f40739a0 2024-05-08T22:07:20.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.901+0000 7ff0eb400700 1 -- 172.21.15.131:0/971461814 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0e40044e0 con 0x7ff0f40739a0 2024-05-08T22:07:20.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.901+0000 7ff0f3e00700 1 -- 172.21.15.131:0/971461814 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e00095f0 con 0x7ff0f40739a0 2024-05-08T22:07:20.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.901+0000 7ff0eb400700 1 -- 172.21.15.131:0/971461814 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4003d80 con 0x7ff0f40739a0 2024-05-08T22:07:20.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.901+0000 7ff0f3e00700 1 -- 172.21.15.131:0/971461814 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f41a1280 con 0x7ff0f40739a0 2024-05-08T22:07:20.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.902+0000 7ff0f3e00700 1 -- 172.21.15.131:0/971461814 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f4107ad0 con 0x7ff0f40739a0 2024-05-08T22:07:20.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.904+0000 7ff0eb400700 1 -- 172.21.15.131:0/971461814 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 22) v1 ==== 135994+0+0 (secure 0 0 0) 0x7ff0e4004000 con 0x7ff0f40739a0 2024-05-08T22:07:20.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.905+0000 7ff0eb400700 1 --2- 172.21.15.131:0/971461814 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff0d80a1290 0x7ff0d80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:20.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.906+0000 7ff0eb400700 1 -- 172.21.15.131:0/971461814 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7ff0e4014070 con 0x7ff0f40739a0 2024-05-08T22:07:20.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.906+0000 7ff0f2000700 1 --2- 172.21.15.131:0/971461814 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff0d80a1290 0x7ff0d80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:20.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.906+0000 7ff0f2000700 1 --2- 172.21.15.131:0/971461814 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff0d80a1290 0x7ff0d80a3750 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000b2e0 tx=0x7ff0e0005fd0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:20.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:20.908+0000 7ff0eb400700 1 -- 172.21.15.131:0/971461814 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff0e4096a30 con 0x7ff0f40739a0 2024-05-08T22:07:21.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.140+0000 7ff0f3e00700 1 -- 172.21.15.131:0/971461814 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff0f410e820 con 0x7ff0f40739a0 2024-05-08T22:07:21.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.141+0000 7ff0eb400700 1 -- 172.21.15.131:0/971461814 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff0e409a050 con 0x7ff0f40739a0 2024-05-08T22:07:21.142 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-08T22:07:21.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.143+0000 7ff0f3e00700 1 -- 172.21.15.131:0/971461814 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff0d80a1290 msgr2=0x7ff0d80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:21.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.143+0000 7ff0f3e00700 1 --2- 172.21.15.131:0/971461814 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff0d80a1290 0x7ff0d80a3750 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000b2e0 tx=0x7ff0e0005fd0 comp rx=0 tx=0).stop 2024-05-08T22:07:21.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.143+0000 7ff0f3e00700 1 -- 172.21.15.131:0/971461814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0f40739a0 msgr2=0x7ff0f4198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:21.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.143+0000 7ff0f3e00700 1 --2- 172.21.15.131:0/971461814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0f40739a0 0x7ff0f4198f10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400eb30 tx=0x7ff0e400ee40 comp rx=0 tx=0).stop 2024-05-08T22:07:21.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.143+0000 7ff0f3e00700 1 -- 172.21.15.131:0/971461814 shutdown_connections 2024-05-08T22:07:21.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.143+0000 7ff0f3e00700 1 --2- 172.21.15.131:0/971461814 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff0d80a1290 0x7ff0d80a3750 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:21.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.143+0000 7ff0f3e00700 1 --2- 172.21.15.131:0/971461814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0f4108cc0 0x7ff0f41a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:21.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.143+0000 7ff0f3e00700 1 --2- 172.21.15.131:0/971461814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0f40739a0 0x7ff0f4198f10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:21.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.143+0000 7ff0f3e00700 1 --2- 172.21.15.131:0/971461814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0f4074f80 0x7ff0f4199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:21.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.143+0000 7ff0f3e00700 1 -- 172.21.15.131:0/971461814 >> 172.21.15.131:0/971461814 conn(0x7ff0f40fc380 msgr2=0x7ff0f4104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:21.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.143+0000 7ff0f3e00700 1 -- 172.21.15.131:0/971461814 shutdown_connections 2024-05-08T22:07:21.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.144+0000 7ff0f3e00700 1 -- 172.21.15.131:0/971461814 wait complete. 2024-05-08T22:07:21.164 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:07:21.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.372+0000 7ff832400700 1 -- 172.21.15.131:0/420861993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff82c106c40 msgr2=0x7ff82c1090b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:21.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.372+0000 7ff832400700 1 --2- 172.21.15.131:0/420861993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff82c106c40 0x7ff82c1090b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff824009f90 tx=0x7ff82400b330 comp rx=0 tx=0).stop 2024-05-08T22:07:21.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.372+0000 7ff832400700 1 -- 172.21.15.131:0/420861993 shutdown_connections 2024-05-08T22:07:21.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.372+0000 7ff832400700 1 --2- 172.21.15.131:0/420861993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff82c106c40 0x7ff82c1090b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:21.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.372+0000 7ff832400700 1 --2- 172.21.15.131:0/420861993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff82c1019e0 0x7ff82c103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:21.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.372+0000 7ff832400700 1 --2- 172.21.15.131:0/420861993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff82c104310 0x7ff82c106700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:21.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.372+0000 7ff832400700 1 -- 172.21.15.131:0/420861993 >> 172.21.15.131:0/420861993 conn(0x7ff82c0fb3f0 msgr2=0x7ff82c0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:21.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.372+0000 7ff832400700 1 -- 172.21.15.131:0/420861993 shutdown_connections 2024-05-08T22:07:21.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.372+0000 7ff832400700 1 -- 172.21.15.131:0/420861993 wait complete. 2024-05-08T22:07:21.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.373+0000 7ff832400700 1 Processor -- start 2024-05-08T22:07:21.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.374+0000 7ff832400700 1 -- start start 2024-05-08T22:07:21.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.374+0000 7ff832400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff82c1019e0 0x7ff82c197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:21.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.374+0000 7ff832400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff82c104310 0x7ff82c197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:21.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.374+0000 7ff832400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff82c106c40 0x7ff82c19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:21.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.374+0000 7ff832400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c19eb50 con 0x7ff82c1019e0 2024-05-08T22:07:21.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.374+0000 7ff832400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c19ec90 con 0x7ff82c106c40 2024-05-08T22:07:21.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.374+0000 7ff832400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c19ee20 con 0x7ff82c104310 2024-05-08T22:07:21.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.374+0000 7ff82b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff82c1019e0 0x7ff82c197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:21.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.374+0000 7ff82aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff82c104310 0x7ff82c197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:21.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.374+0000 7ff82b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff82c1019e0 0x7ff82c197020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41782/0 (socket says 172.21.15.131:41782) 2024-05-08T22:07:21.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.374+0000 7ff82b400700 1 -- 172.21.15.131:0/1897620454 learned_addr learned my addr 172.21.15.131:0/1897620454 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:21.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.374+0000 7ff82be00700 1 --2- 172.21.15.131:0/1897620454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff82c106c40 0x7ff82c19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:21.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.375+0000 7ff82aa00700 1 -- 172.21.15.131:0/1897620454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff82c106c40 msgr2=0x7ff82c19e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:21.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.375+0000 7ff82aa00700 1 --2- 172.21.15.131:0/1897620454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff82c106c40 0x7ff82c19e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:21.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.375+0000 7ff82aa00700 1 -- 172.21.15.131:0/1897620454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff82c1019e0 msgr2=0x7ff82c197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:21.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.375+0000 7ff82aa00700 1 --2- 172.21.15.131:0/1897620454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff82c1019e0 0x7ff82c197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:21.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.375+0000 7ff82aa00700 1 -- 172.21.15.131:0/1897620454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff824009c40 con 0x7ff82c104310 2024-05-08T22:07:21.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.375+0000 7ff82be00700 1 --2- 172.21.15.131:0/1897620454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff82c106c40 0x7ff82c19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:21.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.375+0000 7ff82aa00700 1 --2- 172.21.15.131:0/1897620454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff82c104310 0x7ff82c197560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff81800e9c0 tx=0x7ff81800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:21.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.375+0000 7ff823e00700 1 -- 172.21.15.131:0/1897620454 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff818004220 con 0x7ff82c104310 2024-05-08T22:07:21.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.375+0000 7ff823e00700 1 -- 172.21.15.131:0/1897620454 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff818004cf0 con 0x7ff82c104310 2024-05-08T22:07:21.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.375+0000 7ff832400700 1 -- 172.21.15.131:0/1897620454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff82c0ff830 con 0x7ff82c104310 2024-05-08T22:07:21.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.375+0000 7ff823e00700 1 -- 172.21.15.131:0/1897620454 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81800cd30 con 0x7ff82c104310 2024-05-08T22:07:21.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.375+0000 7ff832400700 1 -- 172.21.15.131:0/1897620454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82c0ffcf0 con 0x7ff82c104310 2024-05-08T22:07:21.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.376+0000 7ff832400700 1 -- 172.21.15.131:0/1897620454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff82c0fcf70 con 0x7ff82c104310 2024-05-08T22:07:21.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.378+0000 7ff823e00700 1 -- 172.21.15.131:0/1897620454 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 22) v1 ==== 135994+0+0 (secure 0 0 0) 0x7ff818004750 con 0x7ff82c104310 2024-05-08T22:07:21.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.380+0000 7ff823e00700 1 --2- 172.21.15.131:0/1897620454 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff8100a1290 0x7ff8100a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:21.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.381+0000 7ff823e00700 1 -- 172.21.15.131:0/1897620454 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7ff818014070 con 0x7ff82c104310 2024-05-08T22:07:21.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.381+0000 7ff82b400700 1 --2- 172.21.15.131:0/1897620454 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff8100a1290 0x7ff8100a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:21.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.381+0000 7ff82b400700 1 --2- 172.21.15.131:0/1897620454 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff8100a1290 0x7ff8100a3750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff81c010e40 tx=0x7ff81c012040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:21.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.382+0000 7ff823e00700 1 -- 172.21.15.131:0/1897620454 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff818096030 con 0x7ff82c104310 2024-05-08T22:07:21.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:21 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1826086872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:07:21.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:21 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/150574540' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:07:21.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:21 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:21.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:21 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:21.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:21 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:21.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:21 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:21.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:21 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:21.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:21 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:21.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:21 smithi157 ceph-mon[77746]: Adjusting osd_memory_target on smithi131 to 221.2M 2024-05-08T22:07:21.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:21 smithi157 ceph-mon[77746]: Unable to set osd_memory_target on smithi131 to 232010956: error parsing value: Value '232010956' is below minimum 939524096 2024-05-08T22:07:21.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:21 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:07:21.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:21 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:07:21.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:21 smithi157 ceph-mon[77746]: Updating smithi131:/etc/ceph/ceph.conf 2024-05-08T22:07:21.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:21 smithi157 ceph-mon[77746]: Updating smithi132:/etc/ceph/ceph.conf 2024-05-08T22:07:21.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:21 smithi157 ceph-mon[77746]: Updating smithi157:/etc/ceph/ceph.conf 2024-05-08T22:07:21.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:21 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/971461814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:07:21.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:21 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1826086872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:07:21.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:21 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/150574540' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:07:21.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:21 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:21.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:21 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:21.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:21 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:21.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:21 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:21.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:21 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:21.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:21 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:21.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:21 smithi132 ceph-mon[78180]: Adjusting osd_memory_target on smithi131 to 221.2M 2024-05-08T22:07:21.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:21 smithi132 ceph-mon[78180]: Unable to set osd_memory_target on smithi131 to 232010956: error parsing value: Value '232010956' is below minimum 939524096 2024-05-08T22:07:21.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:21 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:07:21.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:21 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:07:21.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:21 smithi132 ceph-mon[78180]: Updating smithi131:/etc/ceph/ceph.conf 2024-05-08T22:07:21.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:21 smithi132 ceph-mon[78180]: Updating smithi132:/etc/ceph/ceph.conf 2024-05-08T22:07:21.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:21 smithi132 ceph-mon[78180]: Updating smithi157:/etc/ceph/ceph.conf 2024-05-08T22:07:21.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:21 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/971461814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:07:21.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.622+0000 7ff832400700 1 -- 172.21.15.131:0/1897620454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff82c10c7c0 con 0x7ff82c104310 2024-05-08T22:07:21.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.622+0000 7ff823e00700 1 -- 172.21.15.131:0/1897620454 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff818004e60 con 0x7ff82c104310 2024-05-08T22:07:21.623 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-08T22:07:21.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.625+0000 7ff832400700 1 -- 172.21.15.131:0/1897620454 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff8100a1290 msgr2=0x7ff8100a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:21.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.625+0000 7ff832400700 1 --2- 172.21.15.131:0/1897620454 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff8100a1290 0x7ff8100a3750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff81c010e40 tx=0x7ff81c012040 comp rx=0 tx=0).stop 2024-05-08T22:07:21.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.625+0000 7ff832400700 1 -- 172.21.15.131:0/1897620454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff82c104310 msgr2=0x7ff82c197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:21.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.625+0000 7ff832400700 1 --2- 172.21.15.131:0/1897620454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff82c104310 0x7ff82c197560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff81800e9c0 tx=0x7ff81800ecd0 comp rx=0 tx=0).stop 2024-05-08T22:07:21.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.625+0000 7ff832400700 1 -- 172.21.15.131:0/1897620454 shutdown_connections 2024-05-08T22:07:21.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.625+0000 7ff832400700 1 --2- 172.21.15.131:0/1897620454 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff8100a1290 0x7ff8100a3750 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:21.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.625+0000 7ff832400700 1 --2- 172.21.15.131:0/1897620454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff82c106c40 0x7ff82c19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:21.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.625+0000 7ff832400700 1 --2- 172.21.15.131:0/1897620454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff82c1019e0 0x7ff82c197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:21.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.625+0000 7ff832400700 1 --2- 172.21.15.131:0/1897620454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff82c104310 0x7ff82c197560 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:21.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.625+0000 7ff832400700 1 -- 172.21.15.131:0/1897620454 >> 172.21.15.131:0/1897620454 conn(0x7ff82c0fb3f0 msgr2=0x7ff82c1023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:21.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.625+0000 7ff832400700 1 -- 172.21.15.131:0/1897620454 shutdown_connections 2024-05-08T22:07:21.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.625+0000 7ff832400700 1 -- 172.21.15.131:0/1897620454 wait complete. 2024-05-08T22:07:21.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:21 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1826086872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:07:21.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:21 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/150574540' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:07:21.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:21 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:21.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:21 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:21.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:21 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:21.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:21 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:21.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:21 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:21.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:21 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:07:21.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:21 smithi131 ceph-mon[37260]: Adjusting osd_memory_target on smithi131 to 221.2M 2024-05-08T22:07:21.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:21 smithi131 ceph-mon[37260]: Unable to set osd_memory_target on smithi131 to 232010956: error parsing value: Value '232010956' is below minimum 939524096 2024-05-08T22:07:21.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:21 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:07:21.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:21 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:07:21.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:21 smithi131 ceph-mon[37260]: Updating smithi131:/etc/ceph/ceph.conf 2024-05-08T22:07:21.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:21 smithi131 ceph-mon[37260]: Updating smithi132:/etc/ceph/ceph.conf 2024-05-08T22:07:21.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:21 smithi131 ceph-mon[37260]: Updating smithi157:/etc/ceph/ceph.conf 2024-05-08T22:07:21.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:21 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/971461814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:07:21.646 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:07:21.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.860+0000 7ff399800700 1 -- 172.21.15.131:0/922933904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3941020f0 msgr2=0x7ff394102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:21.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.860+0000 7ff399800700 1 --2- 172.21.15.131:0/922933904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3941020f0 0x7ff394102570 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0098f0 tx=0x7ff37c009c00 comp rx=0 tx=0).stop 2024-05-08T22:07:21.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.860+0000 7ff399800700 1 -- 172.21.15.131:0/922933904 shutdown_connections 2024-05-08T22:07:21.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.860+0000 7ff399800700 1 --2- 172.21.15.131:0/922933904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff394102ab0 0x7ff3941093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:21.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.860+0000 7ff399800700 1 --2- 172.21.15.131:0/922933904 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff394100f20 0x7ff394101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:21.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.860+0000 7ff399800700 1 --2- 172.21.15.131:0/922933904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3941020f0 0x7ff394102570 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:21.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.860+0000 7ff399800700 1 -- 172.21.15.131:0/922933904 >> 172.21.15.131:0/922933904 conn(0x7ff3940fc500 msgr2=0x7ff3940fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:21.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.860+0000 7ff399800700 1 -- 172.21.15.131:0/922933904 shutdown_connections 2024-05-08T22:07:21.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.860+0000 7ff399800700 1 -- 172.21.15.131:0/922933904 wait complete. 2024-05-08T22:07:21.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.861+0000 7ff399800700 1 Processor -- start 2024-05-08T22:07:21.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.862+0000 7ff399800700 1 -- start start 2024-05-08T22:07:21.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.862+0000 7ff399800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff394100f20 0x7ff394199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:21.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.862+0000 7ff399800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3941020f0 0x7ff3941996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:21.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.862+0000 7ff399800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff394102ab0 0x7ff3941a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:21.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.862+0000 7ff399800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3941a0ca0 con 0x7ff394100f20 2024-05-08T22:07:21.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.862+0000 7ff399800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3941a0de0 con 0x7ff3941020f0 2024-05-08T22:07:21.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.862+0000 7ff399800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3941a0f70 con 0x7ff394102ab0 2024-05-08T22:07:21.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.862+0000 7ff392a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff394100f20 0x7ff394199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:21.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.862+0000 7ff38be00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3941020f0 0x7ff3941996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:21.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.862+0000 7ff393400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff394102ab0 0x7ff3941a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:21.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.862+0000 7ff38be00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3941020f0 0x7ff3941996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:37772/0 (socket says 172.21.15.131:37772) 2024-05-08T22:07:21.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.862+0000 7ff38be00700 1 -- 172.21.15.131:0/827702956 learned_addr learned my addr 172.21.15.131:0/827702956 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:21.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.862+0000 7ff392a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff394100f20 0x7ff394199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41800/0 (socket says 172.21.15.131:41800) 2024-05-08T22:07:21.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.863+0000 7ff38be00700 1 -- 172.21.15.131:0/827702956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff394102ab0 msgr2=0x7ff3941a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:21.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.863+0000 7ff38be00700 1 --2- 172.21.15.131:0/827702956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff394102ab0 0x7ff3941a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:21.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.863+0000 7ff38be00700 1 -- 172.21.15.131:0/827702956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff394100f20 msgr2=0x7ff394199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:21.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.863+0000 7ff38be00700 1 --2- 172.21.15.131:0/827702956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff394100f20 0x7ff394199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:21.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.863+0000 7ff38be00700 1 -- 172.21.15.131:0/827702956 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3840095f0 con 0x7ff3941020f0 2024-05-08T22:07:21.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.863+0000 7ff392a00700 1 --2- 172.21.15.131:0/827702956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff394100f20 0x7ff394199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:21.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.863+0000 7ff38be00700 1 --2- 172.21.15.131:0/827702956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3941020f0 0x7ff3941996b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff37c009810 tx=0x7ff37c00b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:21.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.863+0000 7ff38b400700 1 -- 172.21.15.131:0/827702956 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c01c070 con 0x7ff3941020f0 2024-05-08T22:07:21.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.863+0000 7ff399800700 1 -- 172.21.15.131:0/827702956 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38c009c40 con 0x7ff3941020f0 2024-05-08T22:07:21.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.863+0000 7ff38b400700 1 -- 172.21.15.131:0/827702956 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff37c00f7d0 con 0x7ff3941020f0 2024-05-08T22:07:21.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.863+0000 7ff38b400700 1 -- 172.21.15.131:0/827702956 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c003970 con 0x7ff3941020f0 2024-05-08T22:07:21.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.864+0000 7ff399800700 1 -- 172.21.15.131:0/827702956 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff37c009520 con 0x7ff3941020f0 2024-05-08T22:07:21.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.864+0000 7ff399800700 1 -- 172.21.15.131:0/827702956 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff39410d0c0 con 0x7ff3941020f0 2024-05-08T22:07:21.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.866+0000 7ff38b400700 1 -- 172.21.15.131:0/827702956 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7ff37c020030 con 0x7ff3941020f0 2024-05-08T22:07:21.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.867+0000 7ff38b400700 1 --2- 172.21.15.131:0/827702956 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff3780a13b0 0x7ff3780a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:21.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.868+0000 7ff38b400700 1 -- 172.21.15.131:0/827702956 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7ff37c0ce510 con 0x7ff3941020f0 2024-05-08T22:07:21.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.868+0000 7ff392a00700 1 --2- 172.21.15.131:0/827702956 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff3780a13b0 0x7ff3780a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:21.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.868+0000 7ff392a00700 1 --2- 172.21.15.131:0/827702956 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff3780a13b0 0x7ff3780a3870 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff394100c40 tx=0x7ff384009310 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:21.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:21.870+0000 7ff38b400700 1 -- 172.21.15.131:0/827702956 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff37c097530 con 0x7ff3941020f0 2024-05-08T22:07:22.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.101+0000 7ff399800700 1 -- 172.21.15.131:0/827702956 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff39410c9b0 con 0x7ff3941020f0 2024-05-08T22:07:22.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.102+0000 7ff38b400700 1 -- 172.21.15.131:0/827702956 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff37c09ab50 con 0x7ff3941020f0 2024-05-08T22:07:22.103 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-08T22:07:22.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.106+0000 7ff399800700 1 -- 172.21.15.131:0/827702956 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff3780a13b0 msgr2=0x7ff3780a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:22.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.106+0000 7ff399800700 1 --2- 172.21.15.131:0/827702956 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff3780a13b0 0x7ff3780a3870 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff394100c40 tx=0x7ff384009310 comp rx=0 tx=0).stop 2024-05-08T22:07:22.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.106+0000 7ff399800700 1 -- 172.21.15.131:0/827702956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3941020f0 msgr2=0x7ff3941996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:22.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.106+0000 7ff399800700 1 --2- 172.21.15.131:0/827702956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3941020f0 0x7ff3941996b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff37c009810 tx=0x7ff37c00b790 comp rx=0 tx=0).stop 2024-05-08T22:07:22.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.106+0000 7ff399800700 1 -- 172.21.15.131:0/827702956 shutdown_connections 2024-05-08T22:07:22.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.106+0000 7ff399800700 1 --2- 172.21.15.131:0/827702956 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff3780a13b0 0x7ff3780a3870 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:22.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.106+0000 7ff399800700 1 --2- 172.21.15.131:0/827702956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff394102ab0 0x7ff3941a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:22.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.106+0000 7ff399800700 1 --2- 172.21.15.131:0/827702956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff394100f20 0x7ff394199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:22.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.106+0000 7ff399800700 1 --2- 172.21.15.131:0/827702956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3941020f0 0x7ff3941996b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:22.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.106+0000 7ff399800700 1 -- 172.21.15.131:0/827702956 >> 172.21.15.131:0/827702956 conn(0x7ff3940fc500 msgr2=0x7ff3941079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:22.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.106+0000 7ff399800700 1 -- 172.21.15.131:0/827702956 shutdown_connections 2024-05-08T22:07:22.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.106+0000 7ff399800700 1 -- 172.21.15.131:0/827702956 wait complete. 2024-05-08T22:07:22.128 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:07:22.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.339+0000 7f1d42e00700 1 -- 172.21.15.131:0/3114034353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d3c0ff6d0 msgr2=0x7f1d3c101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:22.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.339+0000 7f1d42e00700 1 --2- 172.21.15.131:0/3114034353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d3c0ff6d0 0x7f1d3c101b00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009990 tx=0x7f1d2c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:22.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.339+0000 7f1d42e00700 1 -- 172.21.15.131:0/3114034353 shutdown_connections 2024-05-08T22:07:22.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.339+0000 7f1d42e00700 1 --2- 172.21.15.131:0/3114034353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d3c104a10 0x7f1d3c104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:22.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.339+0000 7f1d42e00700 1 --2- 172.21.15.131:0/3114034353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d3c0ff6d0 0x7f1d3c101b00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:22.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.339+0000 7f1d42e00700 1 --2- 172.21.15.131:0/3114034353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d3c102040 0x7f1d3c1044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:22.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.339+0000 7f1d42e00700 1 -- 172.21.15.131:0/3114034353 >> 172.21.15.131:0/3114034353 conn(0x7f1d3c0fb2f0 msgr2=0x7f1d3c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:22.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.339+0000 7f1d42e00700 1 -- 172.21.15.131:0/3114034353 shutdown_connections 2024-05-08T22:07:22.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.339+0000 7f1d42e00700 1 -- 172.21.15.131:0/3114034353 wait complete. 2024-05-08T22:07:22.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.340+0000 7f1d42e00700 1 Processor -- start 2024-05-08T22:07:22.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.340+0000 7f1d42e00700 1 -- start start 2024-05-08T22:07:22.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.340+0000 7f1d42e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d3c0ff6d0 0x7f1d3c19d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:22.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.340+0000 7f1d42e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d3c102040 0x7f1d3c19d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:22.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.340+0000 7f1d42e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d3c104a10 0x7f1d3c1a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:22.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.340+0000 7f1d42e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c1a4f20 con 0x7f1d3c102040 2024-05-08T22:07:22.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.340+0000 7f1d42e00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c1a5060 con 0x7f1d3c0ff6d0 2024-05-08T22:07:22.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.340+0000 7f1d42e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c1a51f0 con 0x7f1d3c104a10 2024-05-08T22:07:22.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.341+0000 7f1d3be00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d3c0ff6d0 0x7f1d3c19d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:22.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.341+0000 7f1d3b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d3c102040 0x7f1d3c19d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:22.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.341+0000 7f1d3be00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d3c0ff6d0 0x7f1d3c19d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:37780/0 (socket says 172.21.15.131:37780) 2024-05-08T22:07:22.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.341+0000 7f1d3be00700 1 -- 172.21.15.131:0/2916606785 learned_addr learned my addr 172.21.15.131:0/2916606785 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:22.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.341+0000 7f1d40e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d3c104a10 0x7f1d3c1a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:22.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.341+0000 7f1d3be00700 1 -- 172.21.15.131:0/2916606785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d3c104a10 msgr2=0x7f1d3c1a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:22.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.341+0000 7f1d3be00700 1 --2- 172.21.15.131:0/2916606785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d3c104a10 0x7f1d3c1a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:22.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.341+0000 7f1d3be00700 1 -- 172.21.15.131:0/2916606785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d3c102040 msgr2=0x7f1d3c19d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:22.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.341+0000 7f1d3be00700 1 --2- 172.21.15.131:0/2916606785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d3c102040 0x7f1d3c19d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:22.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.341+0000 7f1d3be00700 1 -- 172.21.15.131:0/2916606785 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2c0095f0 con 0x7f1d3c0ff6d0 2024-05-08T22:07:22.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.341+0000 7f1d3be00700 1 --2- 172.21.15.131:0/2916606785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d3c0ff6d0 0x7f1d3c19d3f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c005910 tx=0x7f1d2c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:22.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.342+0000 7f1d38c00700 1 -- 172.21.15.131:0/2916606785 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c01d070 con 0x7f1d3c0ff6d0 2024-05-08T22:07:22.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.342+0000 7f1d38c00700 1 -- 172.21.15.131:0/2916606785 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d2c021410 con 0x7f1d3c0ff6d0 2024-05-08T22:07:22.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.342+0000 7f1d38c00700 1 -- 172.21.15.131:0/2916606785 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c005720 con 0x7f1d3c0ff6d0 2024-05-08T22:07:22.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.342+0000 7f1d42e00700 1 -- 172.21.15.131:0/2916606785 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d3c1a5470 con 0x7f1d3c0ff6d0 2024-05-08T22:07:22.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.342+0000 7f1d42e00700 1 -- 172.21.15.131:0/2916606785 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d3c1a5840 con 0x7f1d3c0ff6d0 2024-05-08T22:07:22.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.343+0000 7f1d42e00700 1 -- 172.21.15.131:0/2916606785 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d3c0fce70 con 0x7f1d3c0ff6d0 2024-05-08T22:07:22.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.345+0000 7f1d38c00700 1 -- 172.21.15.131:0/2916606785 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7f1d2c021580 con 0x7f1d3c0ff6d0 2024-05-08T22:07:22.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.346+0000 7f1d38c00700 1 --2- 172.21.15.131:0/2916606785 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1d200a1430 0x7f1d200a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:22.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.346+0000 7f1d38c00700 1 -- 172.21.15.131:0/2916606785 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f1d2c0ce650 con 0x7f1d3c0ff6d0 2024-05-08T22:07:22.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.346+0000 7f1d3b400700 1 --2- 172.21.15.131:0/2916606785 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1d200a1430 0x7f1d200a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:22.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.347+0000 7f1d3b400700 1 --2- 172.21.15.131:0/2916606785 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1d200a1430 0x7f1d200a38f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009c60 tx=0x7f1d28009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:22.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.350+0000 7f1d38c00700 1 -- 172.21.15.131:0/2916606785 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1d2c0974f0 con 0x7f1d3c0ff6d0 2024-05-08T22:07:22.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:22 smithi131 ceph-mon[37260]: Updating smithi157:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:07:22.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:22 smithi131 ceph-mon[37260]: Updating smithi132:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:07:22.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:22 smithi131 ceph-mon[37260]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:07:22.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:22 smithi131 ceph-mon[37260]: Updating smithi131:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:07:22.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:22 smithi131 ceph-mon[37260]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:07:22.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:22 smithi131 ceph-mon[37260]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:07:22.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:22 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1897620454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:07:22.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:22 smithi131 ceph-mon[37260]: Updating smithi132:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T22:07:22.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:22 smithi131 ceph-mon[37260]: Updating smithi157:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T22:07:22.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:22 smithi131 ceph-mon[37260]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-05-08T22:07:22.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:22 smithi131 ceph-mon[37260]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:07:22.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:22 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:22.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:22 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:22.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:22 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:22.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:22 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:22.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:22 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/827702956' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:07:22.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:22 smithi157 ceph-mon[77746]: Updating smithi157:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:07:22.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:22 smithi157 ceph-mon[77746]: Updating smithi132:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:07:22.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:22 smithi157 ceph-mon[77746]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:07:22.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:22 smithi157 ceph-mon[77746]: Updating smithi131:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:07:22.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:22 smithi157 ceph-mon[77746]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:07:22.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:22 smithi157 ceph-mon[77746]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:07:22.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:22 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1897620454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:07:22.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:22 smithi157 ceph-mon[77746]: Updating smithi132:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T22:07:22.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:22 smithi157 ceph-mon[77746]: Updating smithi157:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T22:07:22.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:22 smithi157 ceph-mon[77746]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-05-08T22:07:22.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:22 smithi157 ceph-mon[77746]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:07:22.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:22 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:22.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:22 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:22.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:22 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:22.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:22 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:22.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:22 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/827702956' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:07:22.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:22 smithi132 ceph-mon[78180]: Updating smithi157:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:07:22.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:22 smithi132 ceph-mon[78180]: Updating smithi132:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:07:22.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:22 smithi132 ceph-mon[78180]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:07:22.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:22 smithi132 ceph-mon[78180]: Updating smithi131:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:07:22.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:22 smithi132 ceph-mon[78180]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:07:22.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:22 smithi132 ceph-mon[78180]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:07:22.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:22 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1897620454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:07:22.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:22 smithi132 ceph-mon[78180]: Updating smithi132:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T22:07:22.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:22 smithi132 ceph-mon[78180]: Updating smithi157:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T22:07:22.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:22 smithi132 ceph-mon[78180]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-05-08T22:07:22.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:22 smithi132 ceph-mon[78180]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:07:22.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:22 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:22.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:22 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:22.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:22 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:22.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:22 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:22.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:22 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/827702956' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:07:22.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.597+0000 7f1d42e00700 1 -- 172.21.15.131:0/2916606785 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d3c112cb0 con 0x7f1d3c0ff6d0 2024-05-08T22:07:22.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.599+0000 7f1d38c00700 1 -- 172.21.15.131:0/2916606785 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+25165 (secure 0 0 0) 0x7f1d2c09ab10 con 0x7f1d3c0ff6d0 2024-05-08T22:07:22.600 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:07:22.601 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":88,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:07:17.096317+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:05:12.469858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:05:13.484446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:05:17.016186+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/2704707055":"2024-05-09T22:07:17.096274+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6801/2485665042":"2024-05-09T22:07:17.096274+0000","172.21.15.131:0/126236497":"2024-05-09T22:07:17.096274+0000","172.21.15.131:0/25489954":"2024-05-09T22:07:17.096274+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/1773942606":"2024-05-09T22:07:17.096274+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/2485665042":"2024-05-09T22:07:17.096274+0000","172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3120268590":"2024-05-09T22:07:17.096274+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:07:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.603+0000 7f1d42e00700 1 -- 172.21.15.131:0/2916606785 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1d200a1430 msgr2=0x7f1d200a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.603+0000 7f1d42e00700 1 --2- 172.21.15.131:0/2916606785 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1d200a1430 0x7f1d200a38f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009c60 tx=0x7f1d28009260 comp rx=0 tx=0).stop 2024-05-08T22:07:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.603+0000 7f1d42e00700 1 -- 172.21.15.131:0/2916606785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d3c0ff6d0 msgr2=0x7f1d3c19d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.603+0000 7f1d42e00700 1 --2- 172.21.15.131:0/2916606785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d3c0ff6d0 0x7f1d3c19d3f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c005910 tx=0x7f1d2c009580 comp rx=0 tx=0).stop 2024-05-08T22:07:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.603+0000 7f1d42e00700 1 -- 172.21.15.131:0/2916606785 shutdown_connections 2024-05-08T22:07:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.603+0000 7f1d42e00700 1 --2- 172.21.15.131:0/2916606785 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1d200a1430 0x7f1d200a38f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.603+0000 7f1d42e00700 1 --2- 172.21.15.131:0/2916606785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d3c104a10 0x7f1d3c1a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.603+0000 7f1d42e00700 1 --2- 172.21.15.131:0/2916606785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d3c0ff6d0 0x7f1d3c19d3f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.603+0000 7f1d42e00700 1 --2- 172.21.15.131:0/2916606785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d3c102040 0x7f1d3c19d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.603+0000 7f1d42e00700 1 -- 172.21.15.131:0/2916606785 >> 172.21.15.131:0/2916606785 conn(0x7f1d3c0fb2f0 msgr2=0x7f1d3c1029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.603+0000 7f1d42e00700 1 -- 172.21.15.131:0/2916606785 shutdown_connections 2024-05-08T22:07:22.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.603+0000 7f1d42e00700 1 -- 172.21.15.131:0/2916606785 wait complete. 2024-05-08T22:07:22.626 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:01:41.271078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:05:12.469858+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:05:13.484446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:05:17.016186+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:07:22.627 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:07:22.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.832+0000 7f665e600700 1 -- 172.21.15.131:0/2827504740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6658073c60 msgr2=0x7f66581119e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:22.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.832+0000 7f665e600700 1 --2- 172.21.15.131:0/2827504740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6658073c60 0x7f66581119e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6644009990 tx=0x7f6644009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:22.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.832+0000 7f665e600700 1 -- 172.21.15.131:0/2827504740 shutdown_connections 2024-05-08T22:07:22.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.832+0000 7f665e600700 1 --2- 172.21.15.131:0/2827504740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6658111f20 0x7f6658114310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:22.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.832+0000 7f665e600700 1 --2- 172.21.15.131:0/2827504740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6658073340 0x7f6658073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:22.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.832+0000 7f665e600700 1 --2- 172.21.15.131:0/2827504740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6658073c60 0x7f66581119e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:22.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.832+0000 7f665e600700 1 -- 172.21.15.131:0/2827504740 >> 172.21.15.131:0/2827504740 conn(0x7f66580fc9e0 msgr2=0x7f66580fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:22.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.832+0000 7f665e600700 1 -- 172.21.15.131:0/2827504740 shutdown_connections 2024-05-08T22:07:22.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.832+0000 7f665e600700 1 -- 172.21.15.131:0/2827504740 wait complete. 2024-05-08T22:07:22.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.833+0000 7f665e600700 1 Processor -- start 2024-05-08T22:07:22.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.833+0000 7f665e600700 1 -- start start 2024-05-08T22:07:22.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.834+0000 7f665e600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6658073340 0x7f665819da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:22.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.834+0000 7f665e600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6658073c60 0x7f665819dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:22.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.834+0000 7f665e600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6658111f20 0x7f66581a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:22.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.834+0000 7f665e600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66581a2940 con 0x7f6658073340 2024-05-08T22:07:22.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.834+0000 7f665e600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66581a2ab0 con 0x7f6658111f20 2024-05-08T22:07:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.834+0000 7f665e600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66581a2c70 con 0x7f6658073c60 2024-05-08T22:07:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.834+0000 7f6657400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6658073340 0x7f665819da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.834+0000 7f6657400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6658073340 0x7f665819da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41822/0 (socket says 172.21.15.131:41822) 2024-05-08T22:07:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.834+0000 7f6657400700 1 -- 172.21.15.131:0/1688880791 learned_addr learned my addr 172.21.15.131:0/1688880791 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.834+0000 7f6656a00700 1 --2- 172.21.15.131:0/1688880791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6658073c60 0x7f665819dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.834+0000 7f6657e00700 1 --2- 172.21.15.131:0/1688880791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6658111f20 0x7f66581a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.834+0000 7f6657400700 1 -- 172.21.15.131:0/1688880791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6658073c60 msgr2=0x7f665819dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.834+0000 7f6657400700 1 --2- 172.21.15.131:0/1688880791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6658073c60 0x7f665819dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.834+0000 7f6657400700 1 -- 172.21.15.131:0/1688880791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6658111f20 msgr2=0x7f66581a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.834+0000 7f6657400700 1 --2- 172.21.15.131:0/1688880791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6658111f20 0x7f66581a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.835+0000 7f6657400700 1 -- 172.21.15.131:0/1688880791 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66440095f0 con 0x7f6658073340 2024-05-08T22:07:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.835+0000 7f6657400700 1 --2- 172.21.15.131:0/1688880791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6658073340 0x7f665819da60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f664800e9c0 tx=0x7f664800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.835+0000 7f664fe00700 1 -- 172.21.15.131:0/1688880791 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66480097b0 con 0x7f6658073340 2024-05-08T22:07:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.835+0000 7f664fe00700 1 -- 172.21.15.131:0/1688880791 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f664800ee60 con 0x7f6658073340 2024-05-08T22:07:22.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.835+0000 7f665e600700 1 -- 172.21.15.131:0/1688880791 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66581a2f50 con 0x7f6658073340 2024-05-08T22:07:22.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.835+0000 7f664fe00700 1 -- 172.21.15.131:0/1688880791 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66480054f0 con 0x7f6658073340 2024-05-08T22:07:22.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.835+0000 7f665e600700 1 -- 172.21.15.131:0/1688880791 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66581aa800 con 0x7f6658073340 2024-05-08T22:07:22.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.836+0000 7f665e600700 1 -- 172.21.15.131:0/1688880791 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6658104400 con 0x7f6658073340 2024-05-08T22:07:22.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.839+0000 7f664fe00700 1 -- 172.21.15.131:0/1688880791 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7f6648004750 con 0x7f6658073340 2024-05-08T22:07:22.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.840+0000 7f664fe00700 1 --2- 172.21.15.131:0/1688880791 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f663c0a1430 0x7f663c0a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:22.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.840+0000 7f664fe00700 1 -- 172.21.15.131:0/1688880791 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f6648014070 con 0x7f6658073340 2024-05-08T22:07:22.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.841+0000 7f6656a00700 1 --2- 172.21.15.131:0/1688880791 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f663c0a1430 0x7f663c0a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:22.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.841+0000 7f6656a00700 1 --2- 172.21.15.131:0/1688880791 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f663c0a1430 0x7f663c0a38f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f665819efb0 tx=0x7f664400b260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:22.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:22.843+0000 7f664fe00700 1 -- 172.21.15.131:0/1688880791 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6648096520 con 0x7f6658073340 2024-05-08T22:07:23.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.067+0000 7f665e600700 1 -- 172.21.15.131:0/1688880791 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f665819ed60 con 0x7f6658073340 2024-05-08T22:07:23.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.068+0000 7f664fe00700 1 -- 172.21.15.131:0/1688880791 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6648005690 con 0x7f6658073340 2024-05-08T22:07:23.069 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-08T22:07:23.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.070+0000 7f665e600700 1 -- 172.21.15.131:0/1688880791 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f663c0a1430 msgr2=0x7f663c0a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:23.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.070+0000 7f665e600700 1 --2- 172.21.15.131:0/1688880791 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f663c0a1430 0x7f663c0a38f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f665819efb0 tx=0x7f664400b260 comp rx=0 tx=0).stop 2024-05-08T22:07:23.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.070+0000 7f665e600700 1 -- 172.21.15.131:0/1688880791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6658073340 msgr2=0x7f665819da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:23.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.070+0000 7f665e600700 1 --2- 172.21.15.131:0/1688880791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6658073340 0x7f665819da60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f664800e9c0 tx=0x7f664800ecd0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.070+0000 7f665e600700 1 -- 172.21.15.131:0/1688880791 shutdown_connections 2024-05-08T22:07:23.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.070+0000 7f665e600700 1 --2- 172.21.15.131:0/1688880791 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f663c0a1430 0x7f663c0a38f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.070+0000 7f665e600700 1 --2- 172.21.15.131:0/1688880791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6658111f20 0x7f66581a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.070+0000 7f665e600700 1 --2- 172.21.15.131:0/1688880791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6658073340 0x7f665819da60 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.070+0000 7f665e600700 1 --2- 172.21.15.131:0/1688880791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6658073c60 0x7f665819dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.070+0000 7f665e600700 1 -- 172.21.15.131:0/1688880791 >> 172.21.15.131:0/1688880791 conn(0x7f66580fc9e0 msgr2=0x7f66580fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:23.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.070+0000 7f665e600700 1 -- 172.21.15.131:0/1688880791 shutdown_connections 2024-05-08T22:07:23.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.071+0000 7f665e600700 1 -- 172.21.15.131:0/1688880791 wait complete. 2024-05-08T22:07:23.092 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:07:23.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.297+0000 7f7e23000700 1 -- 172.21.15.131:0/1347947344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e1c100680 msgr2=0x7f7e1c104b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:23.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.297+0000 7f7e23000700 1 --2- 172.21.15.131:0/1347947344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e1c100680 0x7f7e1c104b40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7e080098f0 tx=0x7f7e08009c00 comp rx=0 tx=0).stop 2024-05-08T22:07:23.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.297+0000 7f7e23000700 1 -- 172.21.15.131:0/1347947344 shutdown_connections 2024-05-08T22:07:23.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.297+0000 7f7e23000700 1 --2- 172.21.15.131:0/1347947344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1c105160 0x7f7e1c105610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.297+0000 7f7e23000700 1 --2- 172.21.15.131:0/1347947344 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e1c0ffd60 0x7f7e1c100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.297+0000 7f7e23000700 1 --2- 172.21.15.131:0/1347947344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e1c100680 0x7f7e1c104b40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.297+0000 7f7e23000700 1 -- 172.21.15.131:0/1347947344 >> 172.21.15.131:0/1347947344 conn(0x7f7e1c0fb770 msgr2=0x7f7e1c0fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:23.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.297+0000 7f7e23000700 1 -- 172.21.15.131:0/1347947344 shutdown_connections 2024-05-08T22:07:23.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.297+0000 7f7e23000700 1 -- 172.21.15.131:0/1347947344 wait complete. 2024-05-08T22:07:23.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.299+0000 7f7e23000700 1 Processor -- start 2024-05-08T22:07:23.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.299+0000 7f7e23000700 1 -- start start 2024-05-08T22:07:23.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.299+0000 7f7e23000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e1c0ffd60 0x7f7e1c1a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:23.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.299+0000 7f7e23000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e1c100680 0x7f7e1c1a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:23.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.299+0000 7f7e23000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1c105160 0x7f7e1c19d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:23.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.299+0000 7f7e23000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c19ddd0 con 0x7f7e1c0ffd60 2024-05-08T22:07:23.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.299+0000 7f7e23000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c19df40 con 0x7f7e1c100680 2024-05-08T22:07:23.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.299+0000 7f7e23000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c19e100 con 0x7f7e1c105160 2024-05-08T22:07:23.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.300+0000 7f7e21c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e1c0ffd60 0x7f7e1c1a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:23.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.300+0000 7f7e21200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e1c100680 0x7f7e1c1a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:23.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.300+0000 7f7e21c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e1c0ffd60 0x7f7e1c1a3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41834/0 (socket says 172.21.15.131:41834) 2024-05-08T22:07:23.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.300+0000 7f7e21c00700 1 -- 172.21.15.131:0/3407164876 learned_addr learned my addr 172.21.15.131:0/3407164876 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:23.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.300+0000 7f7e22600700 1 --2- 172.21.15.131:0/3407164876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1c105160 0x7f7e1c19d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:23.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.300+0000 7f7e21c00700 1 -- 172.21.15.131:0/3407164876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1c105160 msgr2=0x7f7e1c19d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:23.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.300+0000 7f7e21c00700 1 --2- 172.21.15.131:0/3407164876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1c105160 0x7f7e1c19d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.300+0000 7f7e21c00700 1 -- 172.21.15.131:0/3407164876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e1c100680 msgr2=0x7f7e1c1a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:23.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.300+0000 7f7e21c00700 1 --2- 172.21.15.131:0/3407164876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e1c100680 0x7f7e1c1a3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.300+0000 7f7e21c00700 1 -- 172.21.15.131:0/3407164876 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e10009d00 con 0x7f7e1c0ffd60 2024-05-08T22:07:23.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.300+0000 7f7e21200700 1 --2- 172.21.15.131:0/3407164876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e1c100680 0x7f7e1c1a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:23.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.300+0000 7f7e21c00700 1 --2- 172.21.15.131:0/3407164876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e1c0ffd60 0x7f7e1c1a3640 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000e3c0 tx=0x7f7e1000e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:23.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.301+0000 7f7e1a000700 1 -- 172.21.15.131:0/3407164876 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e10018070 con 0x7f7e1c0ffd60 2024-05-08T22:07:23.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.301+0000 7f7e1a000700 1 -- 172.21.15.131:0/3407164876 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e1000f040 con 0x7f7e1c0ffd60 2024-05-08T22:07:23.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.301+0000 7f7e23000700 1 -- 172.21.15.131:0/3407164876 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e08009520 con 0x7f7e1c0ffd60 2024-05-08T22:07:23.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.301+0000 7f7e1a000700 1 -- 172.21.15.131:0/3407164876 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e100042f0 con 0x7f7e1c0ffd60 2024-05-08T22:07:23.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.301+0000 7f7e23000700 1 -- 172.21.15.131:0/3407164876 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e1c1aa450 con 0x7f7e1c0ffd60 2024-05-08T22:07:23.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.302+0000 7f7e23000700 1 -- 172.21.15.131:0/3407164876 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e1c0688d0 con 0x7f7e1c0ffd60 2024-05-08T22:07:23.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.304+0000 7f7e1a000700 1 -- 172.21.15.131:0/3407164876 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7f7e100090d0 con 0x7f7e1c0ffd60 2024-05-08T22:07:23.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.306+0000 7f7e1a000700 1 --2- 172.21.15.131:0/3407164876 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7e040a13b0 0x7f7e040a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:23.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.306+0000 7f7e1a000700 1 -- 172.21.15.131:0/3407164876 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f7e100ccfe0 con 0x7f7e1c0ffd60 2024-05-08T22:07:23.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.306+0000 7f7e21200700 1 --2- 172.21.15.131:0/3407164876 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7e040a13b0 0x7f7e040a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:23.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.306+0000 7f7e21200700 1 --2- 172.21.15.131:0/3407164876 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7e040a13b0 0x7f7e040a3870 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7e080098c0 tx=0x7f7e08019040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:23.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.311+0000 7f7e1a000700 1 -- 172.21.15.131:0/3407164876 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7e10095ec0 con 0x7f7e1c0ffd60 2024-05-08T22:07:23.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:23 smithi131 ceph-mon[37260]: Updating smithi131:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T22:07:23.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:23 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2916606785' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:07:23.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:23 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:23.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:23 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:23.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:23 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:23.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:23 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1688880791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:07:23.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:23 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:07:23.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:23 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:07:23.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:23 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:07:23.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:23 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:23.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.519+0000 7f7e23000700 1 -- 172.21.15.131:0/3407164876 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7e1c19ecf0 con 0x7f7e1c0ffd60 2024-05-08T22:07:23.570 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:23 smithi157 ceph-mon[77746]: Updating smithi131:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T22:07:23.570 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:23 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2916606785' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:07:23.570 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:23 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:23.570 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:23 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:23.571 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:23 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:23.571 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:23 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1688880791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:07:23.571 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:23 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:07:23.571 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:23 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:07:23.571 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:23 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:07:23.571 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:23 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:23.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.520+0000 7f7e1a000700 1 -- 172.21.15.131:0/3407164876 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7e100994e0 con 0x7f7e1c0ffd60 2024-05-08T22:07:23.571 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-08T22:07:23.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.522+0000 7f7e23000700 1 -- 172.21.15.131:0/3407164876 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7e040a13b0 msgr2=0x7f7e040a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:23.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.522+0000 7f7e23000700 1 --2- 172.21.15.131:0/3407164876 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7e040a13b0 0x7f7e040a3870 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7e080098c0 tx=0x7f7e08019040 comp rx=0 tx=0).stop 2024-05-08T22:07:23.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.522+0000 7f7e23000700 1 -- 172.21.15.131:0/3407164876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e1c0ffd60 msgr2=0x7f7e1c1a3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:23.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.522+0000 7f7e23000700 1 --2- 172.21.15.131:0/3407164876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e1c0ffd60 0x7f7e1c1a3640 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000e3c0 tx=0x7f7e1000e6d0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.522+0000 7f7e23000700 1 -- 172.21.15.131:0/3407164876 shutdown_connections 2024-05-08T22:07:23.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.522+0000 7f7e23000700 1 --2- 172.21.15.131:0/3407164876 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7e040a13b0 0x7f7e040a3870 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.522+0000 7f7e23000700 1 --2- 172.21.15.131:0/3407164876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1c105160 0x7f7e1c19d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.522+0000 7f7e23000700 1 --2- 172.21.15.131:0/3407164876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e1c0ffd60 0x7f7e1c1a3640 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.522+0000 7f7e23000700 1 --2- 172.21.15.131:0/3407164876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e1c100680 0x7f7e1c1a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.522+0000 7f7e23000700 1 -- 172.21.15.131:0/3407164876 >> 172.21.15.131:0/3407164876 conn(0x7f7e1c0fb770 msgr2=0x7f7e1c0fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:23.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.523+0000 7f7e23000700 1 -- 172.21.15.131:0/3407164876 shutdown_connections 2024-05-08T22:07:23.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.523+0000 7f7e23000700 1 -- 172.21.15.131:0/3407164876 wait complete. 2024-05-08T22:07:23.574 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:07:23.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:23 smithi132 ceph-mon[78180]: Updating smithi131:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T22:07:23.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:23 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2916606785' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:07:23.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:23 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:23.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:23 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:23.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:23 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:23.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:23 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1688880791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:07:23.592 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:23 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:07:23.592 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:23 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:07:23.592 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:23 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:07:23.592 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:23 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:07:23.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.770+0000 7f45d9600700 1 -- 172.21.15.131:0/2782473205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45d410b390 msgr2=0x7f45d410d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:23.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.770+0000 7f45d9600700 1 --2- 172.21.15.131:0/2782473205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45d410b390 0x7f45d410d850 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00b0a0 tx=0x7f45cc00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.770+0000 7f45d9600700 1 -- 172.21.15.131:0/2782473205 shutdown_connections 2024-05-08T22:07:23.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.770+0000 7f45d9600700 1 --2- 172.21.15.131:0/2782473205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45d410b390 0x7f45d410d850 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.770+0000 7f45d9600700 1 --2- 172.21.15.131:0/2782473205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d4101f70 0x7f45d4102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.770+0000 7f45d9600700 1 --2- 172.21.15.131:0/2782473205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d4102890 0x7f45d410ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.770+0000 7f45d9600700 1 -- 172.21.15.131:0/2782473205 >> 172.21.15.131:0/2782473205 conn(0x7f45d40fb770 msgr2=0x7f45d40fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:23.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.770+0000 7f45d9600700 1 -- 172.21.15.131:0/2782473205 shutdown_connections 2024-05-08T22:07:23.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.770+0000 7f45d9600700 1 -- 172.21.15.131:0/2782473205 wait complete. 2024-05-08T22:07:23.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.771+0000 7f45d9600700 1 Processor -- start 2024-05-08T22:07:23.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.772+0000 7f45d9600700 1 -- start start 2024-05-08T22:07:23.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.772+0000 7f45d9600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45d4101f70 0x7f45d40ffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:23.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.772+0000 7f45d9600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d4102890 0x7f45d4100100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:23.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.772+0000 7f45d9600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d410b390 0x7f45d4100640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:23.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.772+0000 7f45d9600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d41a57d0 con 0x7f45d410b390 2024-05-08T22:07:23.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.772+0000 7f45d9600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d41a5940 con 0x7f45d4101f70 2024-05-08T22:07:23.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.772+0000 7f45d9600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d41a5b00 con 0x7f45d4102890 2024-05-08T22:07:23.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.772+0000 7f45d3e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45d4101f70 0x7f45d40ffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:23.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.772+0000 7f45d3e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45d4101f70 0x7f45d40ffbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:37818/0 (socket says 172.21.15.131:37818) 2024-05-08T22:07:23.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.772+0000 7f45d3e00700 1 -- 172.21.15.131:0/2210705158 learned_addr learned my addr 172.21.15.131:0/2210705158 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:23.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.772+0000 7f45d8c00700 1 --2- 172.21.15.131:0/2210705158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d410b390 0x7f45d4100640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:23.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.772+0000 7f45d3400700 1 --2- 172.21.15.131:0/2210705158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d4102890 0x7f45d4100100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:23.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.773+0000 7f45d3e00700 1 -- 172.21.15.131:0/2210705158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d4102890 msgr2=0x7f45d4100100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:23.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.773+0000 7f45d3e00700 1 --2- 172.21.15.131:0/2210705158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d4102890 0x7f45d4100100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.773+0000 7f45d3e00700 1 -- 172.21.15.131:0/2210705158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d410b390 msgr2=0x7f45d4100640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:23.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.773+0000 7f45d3e00700 1 --2- 172.21.15.131:0/2210705158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d410b390 0x7f45d4100640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.773+0000 7f45d3e00700 1 -- 172.21.15.131:0/2210705158 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c0009520 con 0x7f45d4101f70 2024-05-08T22:07:23.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.773+0000 7f45d3e00700 1 --2- 172.21.15.131:0/2210705158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45d4101f70 0x7f45d40ffbc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f45c400e9c0 tx=0x7f45c400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:23.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.773+0000 7f45d0c00700 1 -- 172.21.15.131:0/2210705158 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c40097b0 con 0x7f45d4101f70 2024-05-08T22:07:23.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.773+0000 7f45d0c00700 1 -- 172.21.15.131:0/2210705158 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45c400ee60 con 0x7f45d4101f70 2024-05-08T22:07:23.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.773+0000 7f45d9600700 1 -- 172.21.15.131:0/2210705158 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45cc009d00 con 0x7f45d4101f70 2024-05-08T22:07:23.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.773+0000 7f45d0c00700 1 -- 172.21.15.131:0/2210705158 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c40054f0 con 0x7f45d4101f70 2024-05-08T22:07:23.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.773+0000 7f45d9600700 1 -- 172.21.15.131:0/2210705158 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d41a6110 con 0x7f45d4101f70 2024-05-08T22:07:23.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.774+0000 7f45d9600700 1 -- 172.21.15.131:0/2210705158 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d4106420 con 0x7f45d4101f70 2024-05-08T22:07:23.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.776+0000 7f45d0c00700 1 -- 172.21.15.131:0/2210705158 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7f45c4004750 con 0x7f45d4101f70 2024-05-08T22:07:23.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.777+0000 7f45d0c00700 1 --2- 172.21.15.131:0/2210705158 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f45b80a13b0 0x7f45b80a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:23.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.778+0000 7f45d0c00700 1 -- 172.21.15.131:0/2210705158 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f45c4014070 con 0x7f45d4101f70 2024-05-08T22:07:23.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.778+0000 7f45d3400700 1 --2- 172.21.15.131:0/2210705158 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f45b80a13b0 0x7f45b80a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:23.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.778+0000 7f45d3400700 1 --2- 172.21.15.131:0/2210705158 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f45b80a13b0 0x7f45b80a3870 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f45d41012f0 tx=0x7f45c0009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:23.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.780+0000 7f45d0c00700 1 -- 172.21.15.131:0/2210705158 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f45c4096570 con 0x7f45d4101f70 2024-05-08T22:07:23.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.986+0000 7f45d9600700 1 -- 172.21.15.131:0/2210705158 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f45d4067ba0 con 0x7f45d4101f70 2024-05-08T22:07:23.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.987+0000 7f45d0c00700 1 -- 172.21.15.131:0/2210705158 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f45c4099b90 con 0x7f45d4101f70 2024-05-08T22:07:23.988 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-08T22:07:23.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.989+0000 7f45d9600700 1 -- 172.21.15.131:0/2210705158 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f45b80a13b0 msgr2=0x7f45b80a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:23.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.989+0000 7f45d9600700 1 --2- 172.21.15.131:0/2210705158 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f45b80a13b0 0x7f45b80a3870 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f45d41012f0 tx=0x7f45c0009260 comp rx=0 tx=0).stop 2024-05-08T22:07:23.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.989+0000 7f45d9600700 1 -- 172.21.15.131:0/2210705158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45d4101f70 msgr2=0x7f45d40ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:23.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.989+0000 7f45d9600700 1 --2- 172.21.15.131:0/2210705158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45d4101f70 0x7f45d40ffbc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f45c400e9c0 tx=0x7f45c400ecd0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.990+0000 7f45d9600700 1 -- 172.21.15.131:0/2210705158 shutdown_connections 2024-05-08T22:07:23.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.990+0000 7f45d9600700 1 --2- 172.21.15.131:0/2210705158 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f45b80a13b0 0x7f45b80a3870 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.990+0000 7f45d9600700 1 --2- 172.21.15.131:0/2210705158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d410b390 0x7f45d4100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.990+0000 7f45d9600700 1 --2- 172.21.15.131:0/2210705158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45d4101f70 0x7f45d40ffbc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.990+0000 7f45d9600700 1 --2- 172.21.15.131:0/2210705158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d4102890 0x7f45d4100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:23.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.990+0000 7f45d9600700 1 -- 172.21.15.131:0/2210705158 >> 172.21.15.131:0/2210705158 conn(0x7f45d40fb770 msgr2=0x7f45d40fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:23.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.990+0000 7f45d9600700 1 -- 172.21.15.131:0/2210705158 shutdown_connections 2024-05-08T22:07:23.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:23.990+0000 7f45d9600700 1 -- 172.21.15.131:0/2210705158 wait complete. 2024-05-08T22:07:24.010 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:07:24.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.208+0000 7f112c800700 1 -- 172.21.15.131:0/33213728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1124106280 msgr2=0x7f112410de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:24.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.208+0000 7f112c800700 1 --2- 172.21.15.131:0/33213728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1124106280 0x7f112410de00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f112000b0a0 tx=0x7f112000b3b0 comp rx=0 tx=0).stop 2024-05-08T22:07:24.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.208+0000 7f112c800700 1 -- 172.21.15.131:0/33213728 shutdown_connections 2024-05-08T22:07:24.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.208+0000 7f112c800700 1 --2- 172.21.15.131:0/33213728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1124106280 0x7f112410de00 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:24.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.208+0000 7f112c800700 1 --2- 172.21.15.131:0/33213728 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1124101160 0x7f1124101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:24.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.208+0000 7f112c800700 1 --2- 172.21.15.131:0/33213728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1124101b10 0x7f1124105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:24.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.208+0000 7f112c800700 1 -- 172.21.15.131:0/33213728 >> 172.21.15.131:0/33213728 conn(0x7f11240fc9c0 msgr2=0x7f11240fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:24.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.208+0000 7f112c800700 1 -- 172.21.15.131:0/33213728 shutdown_connections 2024-05-08T22:07:24.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.208+0000 7f112c800700 1 -- 172.21.15.131:0/33213728 wait complete. 2024-05-08T22:07:24.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.209+0000 7f112c800700 1 Processor -- start 2024-05-08T22:07:24.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.209+0000 7f112c800700 1 -- start start 2024-05-08T22:07:24.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.210+0000 7f112c800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1124101160 0x7f1124197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:24.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.210+0000 7f112c800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1124106280 0x7f1124197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:24.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.210+0000 7f112c800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f112419be60 0x7f112419c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:24.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.210+0000 7f112c800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112419c8e0 con 0x7f1124101160 2024-05-08T22:07:24.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.210+0000 7f112c800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112419ca50 con 0x7f112419be60 2024-05-08T22:07:24.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.210+0000 7f112c800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112419cc10 con 0x7f1124106280 2024-05-08T22:07:24.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.210+0000 7f1129e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1124101160 0x7f1124197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:24.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.210+0000 7f1129e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1124101160 0x7f1124197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50676/0 (socket says 172.21.15.131:50676) 2024-05-08T22:07:24.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.210+0000 7f1129e00700 1 -- 172.21.15.131:0/4061176932 learned_addr learned my addr 172.21.15.131:0/4061176932 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:24.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.210+0000 7f1129400700 1 --2- 172.21.15.131:0/4061176932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1124106280 0x7f1124197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:24.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.210+0000 7f112a800700 1 --2- 172.21.15.131:0/4061176932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f112419be60 0x7f112419c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:24.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.210+0000 7f1129e00700 1 -- 172.21.15.131:0/4061176932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1124106280 msgr2=0x7f1124197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:24.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.210+0000 7f1129e00700 1 --2- 172.21.15.131:0/4061176932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1124106280 0x7f1124197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:24.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.210+0000 7f1129e00700 1 -- 172.21.15.131:0/4061176932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f112419be60 msgr2=0x7f112419c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:24.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.210+0000 7f1129e00700 1 --2- 172.21.15.131:0/4061176932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f112419be60 0x7f112419c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:24.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.210+0000 7f1129e00700 1 -- 172.21.15.131:0/4061176932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1120009d00 con 0x7f1124101160 2024-05-08T22:07:24.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.211+0000 7f1129e00700 1 --2- 172.21.15.131:0/4061176932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1124101160 0x7f1124197580 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f111800e9a0 tx=0x7f111800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:24.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.211+0000 7f111ea00700 1 -- 172.21.15.131:0/4061176932 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1118009790 con 0x7f1124101160 2024-05-08T22:07:24.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.211+0000 7f112c800700 1 -- 172.21.15.131:0/4061176932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11241a4170 con 0x7f1124101160 2024-05-08T22:07:24.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.211+0000 7f111ea00700 1 -- 172.21.15.131:0/4061176932 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1118010430 con 0x7f1124101160 2024-05-08T22:07:24.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.211+0000 7f111ea00700 1 -- 172.21.15.131:0/4061176932 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1118018680 con 0x7f1124101160 2024-05-08T22:07:24.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.211+0000 7f112c800700 1 -- 172.21.15.131:0/4061176932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11241a4650 con 0x7f1124101160 2024-05-08T22:07:24.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.212+0000 7f112c800700 1 -- 172.21.15.131:0/4061176932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1124109550 con 0x7f1124101160 2024-05-08T22:07:24.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.214+0000 7f111ea00700 1 -- 172.21.15.131:0/4061176932 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7f11180105a0 con 0x7f1124101160 2024-05-08T22:07:24.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.215+0000 7f111ea00700 1 --2- 172.21.15.131:0/4061176932 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f110c0a13b0 0x7f110c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:24.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.215+0000 7f111ea00700 1 -- 172.21.15.131:0/4061176932 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f1118014070 con 0x7f1124101160 2024-05-08T22:07:24.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.215+0000 7f1129400700 1 --2- 172.21.15.131:0/4061176932 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f110c0a13b0 0x7f110c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:24.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.216+0000 7f1129400700 1 --2- 172.21.15.131:0/4061176932 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f110c0a13b0 0x7f110c0a3870 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1124198ad0 tx=0x7f1114009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:24.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.219+0000 7f111ea00700 1 -- 172.21.15.131:0/4061176932 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1118052d60 con 0x7f1124101160 2024-05-08T22:07:24.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:24 smithi131 ceph-mon[37260]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:07:24.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:24 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3407164876' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:07:24.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:24 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2210705158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:07:24.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.422+0000 7f112c800700 1 -- 172.21.15.131:0/4061176932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1124198880 con 0x7f1124101160 2024-05-08T22:07:24.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.423+0000 7f111ea00700 1 -- 172.21.15.131:0/4061176932 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1118096370 con 0x7f1124101160 2024-05-08T22:07:24.424 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-08T22:07:24.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.426+0000 7f112c800700 1 -- 172.21.15.131:0/4061176932 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f110c0a13b0 msgr2=0x7f110c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:24.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.426+0000 7f112c800700 1 --2- 172.21.15.131:0/4061176932 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f110c0a13b0 0x7f110c0a3870 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1124198ad0 tx=0x7f1114009260 comp rx=0 tx=0).stop 2024-05-08T22:07:24.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.426+0000 7f112c800700 1 -- 172.21.15.131:0/4061176932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1124101160 msgr2=0x7f1124197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:24.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.426+0000 7f112c800700 1 --2- 172.21.15.131:0/4061176932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1124101160 0x7f1124197580 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f111800e9a0 tx=0x7f111800ecb0 comp rx=0 tx=0).stop 2024-05-08T22:07:24.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.426+0000 7f112c800700 1 -- 172.21.15.131:0/4061176932 shutdown_connections 2024-05-08T22:07:24.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.426+0000 7f112c800700 1 --2- 172.21.15.131:0/4061176932 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f110c0a13b0 0x7f110c0a3870 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:24.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.426+0000 7f112c800700 1 --2- 172.21.15.131:0/4061176932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f112419be60 0x7f112419c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:24.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.426+0000 7f112c800700 1 --2- 172.21.15.131:0/4061176932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1124101160 0x7f1124197580 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:24.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.426+0000 7f112c800700 1 --2- 172.21.15.131:0/4061176932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1124106280 0x7f1124197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:24.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.426+0000 7f112c800700 1 -- 172.21.15.131:0/4061176932 >> 172.21.15.131:0/4061176932 conn(0x7f11240fc9c0 msgr2=0x7f11240fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:24.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.426+0000 7f112c800700 1 -- 172.21.15.131:0/4061176932 shutdown_connections 2024-05-08T22:07:24.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.426+0000 7f112c800700 1 -- 172.21.15.131:0/4061176932 wait complete. 2024-05-08T22:07:24.446 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-08T22:07:24.446 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:07:24.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:24 smithi157 ceph-mon[77746]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:07:24.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:24 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3407164876' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:07:24.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:24 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2210705158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:07:24.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:24 smithi132 ceph-mon[78180]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:07:24.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:24 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3407164876' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:07:24.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:24 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2210705158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:07:24.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.638+0000 7f819d800700 1 -- 172.21.15.131:0/892087063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8198101a00 msgr2=0x7f8198105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:24.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.638+0000 7f819d800700 1 --2- 172.21.15.131:0/892087063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8198101a00 0x7f8198105a40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8184009990 tx=0x7f8184009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:24.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.638+0000 7f819d800700 1 -- 172.21.15.131:0/892087063 shutdown_connections 2024-05-08T22:07:24.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.638+0000 7f819d800700 1 --2- 172.21.15.131:0/892087063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8198106170 0x7f819810dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:24.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.638+0000 7f819d800700 1 --2- 172.21.15.131:0/892087063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8198101050 0x7f8198101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:24.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.638+0000 7f819d800700 1 --2- 172.21.15.131:0/892087063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8198101a00 0x7f8198105a40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:24.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.638+0000 7f819d800700 1 -- 172.21.15.131:0/892087063 >> 172.21.15.131:0/892087063 conn(0x7f81980fc8f0 msgr2=0x7f81980fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:24.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.638+0000 7f819d800700 1 -- 172.21.15.131:0/892087063 shutdown_connections 2024-05-08T22:07:24.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.638+0000 7f819d800700 1 -- 172.21.15.131:0/892087063 wait complete. 2024-05-08T22:07:24.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.639+0000 7f819d800700 1 Processor -- start 2024-05-08T22:07:24.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.639+0000 7f819d800700 1 -- start start 2024-05-08T22:07:24.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.639+0000 7f819d800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8198101050 0x7f8198199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:24.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.639+0000 7f819d800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8198106170 0x7f8198199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:24.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.640+0000 7f819d800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f819819de20 0x7f819819e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:24.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.640+0000 7f819d800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819819e8a0 con 0x7f8198106170 2024-05-08T22:07:24.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.640+0000 7f819d800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819819ea10 con 0x7f8198101050 2024-05-08T22:07:24.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.640+0000 7f819d800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819819ebd0 con 0x7f819819de20 2024-05-08T22:07:24.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.640+0000 7f8196000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8198106170 0x7f8198199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:24.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.640+0000 7f8196000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8198106170 0x7f8198199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50686/0 (socket says 172.21.15.131:50686) 2024-05-08T22:07:24.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.640+0000 7f8196000700 1 -- 172.21.15.131:0/756939378 learned_addr learned my addr 172.21.15.131:0/756939378 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:24.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.640+0000 7f8197400700 1 --2- 172.21.15.131:0/756939378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f819819de20 0x7f819819e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:24.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.640+0000 7f8196a00700 1 --2- 172.21.15.131:0/756939378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8198101050 0x7f8198199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:24.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.640+0000 7f8196000700 1 -- 172.21.15.131:0/756939378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f819819de20 msgr2=0x7f819819e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:24.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.640+0000 7f8196000700 1 --2- 172.21.15.131:0/756939378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f819819de20 0x7f819819e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:24.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.640+0000 7f8196000700 1 -- 172.21.15.131:0/756939378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8198101050 msgr2=0x7f8198199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:24.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.640+0000 7f8196000700 1 --2- 172.21.15.131:0/756939378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8198101050 0x7f8198199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:24.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.640+0000 7f8196000700 1 -- 172.21.15.131:0/756939378 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8190009c40 con 0x7f8198106170 2024-05-08T22:07:24.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.640+0000 7f8196a00700 1 --2- 172.21.15.131:0/756939378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8198101050 0x7f8198199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:07:24.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.640+0000 7f8196000700 1 --2- 172.21.15.131:0/756939378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8198106170 0x7f8198199a80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8184000c00 tx=0x7f81840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:24.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.641+0000 7f818f400700 1 -- 172.21.15.131:0/756939378 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818401d070 con 0x7f8198106170 2024-05-08T22:07:24.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.641+0000 7f818f400700 1 -- 172.21.15.131:0/756939378 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8184022470 con 0x7f8198106170 2024-05-08T22:07:24.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.641+0000 7f819d800700 1 -- 172.21.15.131:0/756939378 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81840095f0 con 0x7f8198106170 2024-05-08T22:07:24.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.641+0000 7f818f400700 1 -- 172.21.15.131:0/756939378 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8184017650 con 0x7f8198106170 2024-05-08T22:07:24.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.641+0000 7f819d800700 1 -- 172.21.15.131:0/756939378 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81981a6310 con 0x7f8198106170 2024-05-08T22:07:24.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.642+0000 7f819d800700 1 -- 172.21.15.131:0/756939378 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8198102520 con 0x7f8198106170 2024-05-08T22:07:24.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.644+0000 7f818f400700 1 -- 172.21.15.131:0/756939378 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7f818400fad0 con 0x7f8198106170 2024-05-08T22:07:24.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.645+0000 7f818f400700 1 --2- 172.21.15.131:0/756939378 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f817c0a13b0 0x7f817c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:24.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.645+0000 7f818f400700 1 -- 172.21.15.131:0/756939378 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f81840ce600 con 0x7f8198106170 2024-05-08T22:07:24.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.645+0000 7f8196a00700 1 --2- 172.21.15.131:0/756939378 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f817c0a13b0 0x7f817c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:24.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.646+0000 7f8196a00700 1 --2- 172.21.15.131:0/756939378 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f817c0a13b0 0x7f817c0a3870 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8188009cb0 tx=0x7f8188009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:24.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.649+0000 7f818f400700 1 -- 172.21.15.131:0/756939378 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f81840974a0 con 0x7f8198106170 2024-05-08T22:07:24.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.904+0000 7f819d800700 1 -- 172.21.15.131:0/756939378 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8198101a50 con 0x7f8198106170 2024-05-08T22:07:24.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.905+0000 7f818f400700 1 -- 172.21.15.131:0/756939378 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10889 (secure 0 0 0) 0x7f818409aac0 con 0x7f8198106170 2024-05-08T22:07:24.906 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:07:24.906 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:05:14.034908+0000","modified":"2024-05-08T22:07:19.165885+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:07:24.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.908+0000 7f819d800700 1 -- 172.21.15.131:0/756939378 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f817c0a13b0 msgr2=0x7f817c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:24.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.908+0000 7f819d800700 1 --2- 172.21.15.131:0/756939378 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f817c0a13b0 0x7f817c0a3870 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8188009cb0 tx=0x7f8188009260 comp rx=0 tx=0).stop 2024-05-08T22:07:24.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.908+0000 7f819d800700 1 -- 172.21.15.131:0/756939378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8198106170 msgr2=0x7f8198199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:24.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.908+0000 7f819d800700 1 --2- 172.21.15.131:0/756939378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8198106170 0x7f8198199a80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8184000c00 tx=0x7f81840057f0 comp rx=0 tx=0).stop 2024-05-08T22:07:24.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.908+0000 7f819d800700 1 -- 172.21.15.131:0/756939378 shutdown_connections 2024-05-08T22:07:24.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.908+0000 7f819d800700 1 --2- 172.21.15.131:0/756939378 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f817c0a13b0 0x7f817c0a3870 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:24.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.908+0000 7f819d800700 1 --2- 172.21.15.131:0/756939378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f819819de20 0x7f819819e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:24.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.908+0000 7f819d800700 1 --2- 172.21.15.131:0/756939378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8198101050 0x7f8198199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:24.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.908+0000 7f819d800700 1 --2- 172.21.15.131:0/756939378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8198106170 0x7f8198199a80 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:24.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.908+0000 7f819d800700 1 -- 172.21.15.131:0/756939378 >> 172.21.15.131:0/756939378 conn(0x7f81980fc8f0 msgr2=0x7f81980fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:24.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.908+0000 7f819d800700 1 -- 172.21.15.131:0/756939378 shutdown_connections 2024-05-08T22:07:24.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:24.908+0000 7f819d800700 1 -- 172.21.15.131:0/756939378 wait complete. 2024-05-08T22:07:24.910 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 24 2024-05-08T22:07:24.928 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-08T22:07:24.929 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-08T22:07:24.929 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:07:25.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.121+0000 7f0efe800700 1 -- 172.21.15.131:0/1864903969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef81061a0 msgr2=0x7f0ef810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:25.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.121+0000 7f0efe800700 1 --2- 172.21.15.131:0/1864903969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef81061a0 0x7f0ef810dd20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0ef000b0a0 tx=0x7f0ef000b3b0 comp rx=0 tx=0).stop 2024-05-08T22:07:25.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.121+0000 7f0efe800700 1 -- 172.21.15.131:0/1864903969 shutdown_connections 2024-05-08T22:07:25.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.121+0000 7f0efe800700 1 --2- 172.21.15.131:0/1864903969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef81061a0 0x7f0ef810dd20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:25.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.121+0000 7f0efe800700 1 --2- 172.21.15.131:0/1864903969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ef8101080 0x7f0ef8101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:25.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.121+0000 7f0efe800700 1 --2- 172.21.15.131:0/1864903969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ef8101a30 0x7f0ef8105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:25.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.121+0000 7f0efe800700 1 -- 172.21.15.131:0/1864903969 >> 172.21.15.131:0/1864903969 conn(0x7f0ef80fc940 msgr2=0x7f0ef80fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:25.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.121+0000 7f0efe800700 1 -- 172.21.15.131:0/1864903969 shutdown_connections 2024-05-08T22:07:25.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.122+0000 7f0efe800700 1 -- 172.21.15.131:0/1864903969 wait complete. 2024-05-08T22:07:25.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.123+0000 7f0efe800700 1 Processor -- start 2024-05-08T22:07:25.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.123+0000 7f0efe800700 1 -- start start 2024-05-08T22:07:25.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.123+0000 7f0efe800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef8101080 0x7f0ef8199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:25.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.123+0000 7f0efe800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ef81061a0 0x7f0ef8199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:25.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.123+0000 7f0efe800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ef819df40 0x7f0ef819e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:25.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.123+0000 7f0efe800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef819e9c0 con 0x7f0ef8101080 2024-05-08T22:07:25.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.123+0000 7f0efe800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef819eb30 con 0x7f0ef819df40 2024-05-08T22:07:25.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.123+0000 7f0efe800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef819ecf0 con 0x7f0ef81061a0 2024-05-08T22:07:25.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.123+0000 7f0ef7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef8101080 0x7f0ef8199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:25.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.123+0000 7f0ef7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef8101080 0x7f0ef8199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50704/0 (socket says 172.21.15.131:50704) 2024-05-08T22:07:25.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.123+0000 7f0ef7400700 1 -- 172.21.15.131:0/2510761900 learned_addr learned my addr 172.21.15.131:0/2510761900 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:25.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.123+0000 7f0ef6a00700 1 --2- 172.21.15.131:0/2510761900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ef81061a0 0x7f0ef8199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:25.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.123+0000 7f0ef7e00700 1 --2- 172.21.15.131:0/2510761900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ef819df40 0x7f0ef819e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:25.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.124+0000 7f0ef7400700 1 -- 172.21.15.131:0/2510761900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ef81061a0 msgr2=0x7f0ef8199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:25.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.124+0000 7f0ef7400700 1 --2- 172.21.15.131:0/2510761900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ef81061a0 0x7f0ef8199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:25.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.124+0000 7f0ef7400700 1 -- 172.21.15.131:0/2510761900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ef819df40 msgr2=0x7f0ef819e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:25.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.124+0000 7f0ef7400700 1 --2- 172.21.15.131:0/2510761900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ef819df40 0x7f0ef819e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:25.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.124+0000 7f0ef7400700 1 -- 172.21.15.131:0/2510761900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef0009d00 con 0x7f0ef8101080 2024-05-08T22:07:25.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.124+0000 7f0ef7400700 1 --2- 172.21.15.131:0/2510761900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef8101080 0x7f0ef8199660 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0ee800e9c0 tx=0x7f0ee800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:25.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.124+0000 7f0eefe00700 1 -- 172.21.15.131:0/2510761900 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee80097b0 con 0x7f0ef8101080 2024-05-08T22:07:25.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.124+0000 7f0eefe00700 1 -- 172.21.15.131:0/2510761900 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ee800ee60 con 0x7f0ef8101080 2024-05-08T22:07:25.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.124+0000 7f0efe800700 1 -- 172.21.15.131:0/2510761900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef81a6250 con 0x7f0ef8101080 2024-05-08T22:07:25.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.124+0000 7f0eefe00700 1 -- 172.21.15.131:0/2510761900 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee80054f0 con 0x7f0ef8101080 2024-05-08T22:07:25.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.124+0000 7f0efe800700 1 -- 172.21.15.131:0/2510761900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef81a6730 con 0x7f0ef8101080 2024-05-08T22:07:25.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.125+0000 7f0efe800700 1 -- 172.21.15.131:0/2510761900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef8102550 con 0x7f0ef8101080 2024-05-08T22:07:25.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.127+0000 7f0eefe00700 1 -- 172.21.15.131:0/2510761900 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7f0ee8004750 con 0x7f0ef8101080 2024-05-08T22:07:25.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.128+0000 7f0eefe00700 1 --2- 172.21.15.131:0/2510761900 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f0edc0a1430 0x7f0edc0a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:25.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.129+0000 7f0eefe00700 1 -- 172.21.15.131:0/2510761900 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f0ee8014070 con 0x7f0ef8101080 2024-05-08T22:07:25.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.129+0000 7f0ef6a00700 1 --2- 172.21.15.131:0/2510761900 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f0edc0a1430 0x7f0edc0a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:25.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.129+0000 7f0ef6a00700 1 --2- 172.21.15.131:0/2510761900 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f0edc0a1430 0x7f0edc0a38f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0ef819abb0 tx=0x7f0ee4009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:25.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.132+0000 7f0eefe00700 1 -- 172.21.15.131:0/2510761900 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0ee8096330 con 0x7f0ef8101080 2024-05-08T22:07:25.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.338+0000 7f0efe800700 1 -- 172.21.15.131:0/2510761900 --> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0ef8101a30 con 0x7f0edc0a1430 2024-05-08T22:07:25.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.354+0000 7f0eefe00700 1 -- 172.21.15.131:0/2510761900 <== mgr.15366 v2:172.21.15.131:6800/1157588501 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426525 (secure 0 0 0) 0x7f0ef8101a30 con 0x7f0edc0a1430 2024-05-08T22:07:25.363 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:07:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.361+0000 7f0efe800700 1 -- 172.21.15.131:0/2510761900 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f0edc0a1430 msgr2=0x7f0edc0a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.361+0000 7f0efe800700 1 --2- 172.21.15.131:0/2510761900 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f0edc0a1430 0x7f0edc0a38f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0ef819abb0 tx=0x7f0ee4009260 comp rx=0 tx=0).stop 2024-05-08T22:07:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.361+0000 7f0efe800700 1 -- 172.21.15.131:0/2510761900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef8101080 msgr2=0x7f0ef8199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.361+0000 7f0efe800700 1 --2- 172.21.15.131:0/2510761900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef8101080 0x7f0ef8199660 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0ee800e9c0 tx=0x7f0ee800ecd0 comp rx=0 tx=0).stop 2024-05-08T22:07:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.361+0000 7f0efe800700 1 -- 172.21.15.131:0/2510761900 shutdown_connections 2024-05-08T22:07:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.361+0000 7f0efe800700 1 --2- 172.21.15.131:0/2510761900 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f0edc0a1430 0x7f0edc0a38f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.361+0000 7f0efe800700 1 --2- 172.21.15.131:0/2510761900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ef819df40 0x7f0ef819e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.361+0000 7f0efe800700 1 --2- 172.21.15.131:0/2510761900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef8101080 0x7f0ef8199660 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.361+0000 7f0efe800700 1 --2- 172.21.15.131:0/2510761900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ef81061a0 0x7f0ef8199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.361+0000 7f0efe800700 1 -- 172.21.15.131:0/2510761900 >> 172.21.15.131:0/2510761900 conn(0x7f0ef80fc940 msgr2=0x7f0ef80fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.361+0000 7f0efe800700 1 -- 172.21.15.131:0/2510761900 shutdown_connections 2024-05-08T22:07:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.361+0000 7f0efe800700 1 -- 172.21.15.131:0/2510761900 wait complete. 2024-05-08T22:07:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-08T22:07:25.382 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-05-08T22:07:25.114979+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190120,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":984,"num_read_kb":813,"num_write":486,"num_write_kb":4381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181430,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":875,"ondisk_log_size":875,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373032,"kb_used_data":11944,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124568792,"statfs":{"total":1151940427776,"available":1151558443008,"internally_reserved":0,"allocated":12230656,"data_stored":9524439,"data_compressed":502086,"data_compressed_allocated":675840,"data_compressed_original":4202496,"omap_allocated":19131,"internal_metadata":369603909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3699,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":134,"num_read_kb":229,"num_write":28,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.919574"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.174991+0000","last_change":"2024-05-08T22:05:14.061628+0000","last_active":"2024-05-08T22:07:17.174991+0000","last_peered":"2024-05-08T22:07:17.174991+0000","last_clean":"2024-05-08T22:07:17.174991+0000","last_became_active":"2024-05-08T22:05:14.060986+0000","last_became_peered":"2024-05-08T22:05:14.060986+0000","last_unstale":"2024-05-08T22:07:17.174991+0000","last_undegraded":"2024-05-08T22:07:17.174991+0000","last_fullsized":"2024-05-08T22:07:17.174991+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:51.643118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.174788+0000","last_change":"2024-05-08T22:05:14.073300+0000","last_active":"2024-05-08T22:07:17.174788+0000","last_peered":"2024-05-08T22:07:17.174788+0000","last_clean":"2024-05-08T22:07:17.174788+0000","last_became_active":"2024-05-08T22:05:14.069058+0000","last_became_peered":"2024-05-08T22:05:14.069058+0000","last_unstale":"2024-05-08T22:07:17.174788+0000","last_undegraded":"2024-05-08T22:07:17.174788+0000","last_fullsized":"2024-05-08T22:07:17.174788+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:15.834956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.175346+0000","last_change":"2024-05-08T22:05:14.071676+0000","last_active":"2024-05-08T22:07:17.175346+0000","last_peered":"2024-05-08T22:07:17.175346+0000","last_clean":"2024-05-08T22:07:17.175346+0000","last_became_active":"2024-05-08T22:05:14.058345+0000","last_became_peered":"2024-05-08T22:05:14.058345+0000","last_unstale":"2024-05-08T22:07:17.175346+0000","last_undegraded":"2024-05-08T22:07:17.175346+0000","last_fullsized":"2024-05-08T22:07:17.175346+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:09.239058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.155221+0000","last_change":"2024-05-08T22:05:14.067170+0000","last_active":"2024-05-08T22:07:17.155221+0000","last_peered":"2024-05-08T22:07:17.155221+0000","last_clean":"2024-05-08T22:07:17.155221+0000","last_became_active":"2024-05-08T22:05:14.066850+0000","last_became_peered":"2024-05-08T22:05:14.066850+0000","last_unstale":"2024-05-08T22:07:17.155221+0000","last_undegraded":"2024-05-08T22:07:17.155221+0000","last_fullsized":"2024-05-08T22:07:17.155221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:28.439786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.155731+0000","last_change":"2024-05-08T22:05:14.065177+0000","last_active":"2024-05-08T22:07:17.155731+0000","last_peered":"2024-05-08T22:07:17.155731+0000","last_clean":"2024-05-08T22:07:17.155731+0000","last_became_active":"2024-05-08T22:05:14.064833+0000","last_became_peered":"2024-05-08T22:05:14.064833+0000","last_unstale":"2024-05-08T22:07:17.155731+0000","last_undegraded":"2024-05-08T22:07:17.155731+0000","last_fullsized":"2024-05-08T22:07:17.155731+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:11.764154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.179822+0000","last_change":"2024-05-08T22:05:14.061610+0000","last_active":"2024-05-08T22:07:17.179822+0000","last_peered":"2024-05-08T22:07:17.179822+0000","last_clean":"2024-05-08T22:07:17.179822+0000","last_became_active":"2024-05-08T22:05:14.061277+0000","last_became_peered":"2024-05-08T22:05:14.061277+0000","last_unstale":"2024-05-08T22:07:17.179822+0000","last_undegraded":"2024-05-08T22:07:17.179822+0000","last_fullsized":"2024-05-08T22:07:17.179822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:19.154980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.176639+0000","last_change":"2024-05-08T22:05:14.063941+0000","last_active":"2024-05-08T22:07:17.176639+0000","last_peered":"2024-05-08T22:07:17.176639+0000","last_clean":"2024-05-08T22:07:17.176639+0000","last_became_active":"2024-05-08T22:05:14.063579+0000","last_became_peered":"2024-05-08T22:05:14.063579+0000","last_unstale":"2024-05-08T22:07:17.176639+0000","last_undegraded":"2024-05-08T22:07:17.176639+0000","last_fullsized":"2024-05-08T22:07:17.176639+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:27.679067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.179515+0000","last_change":"2024-05-08T22:05:14.060366+0000","last_active":"2024-05-08T22:07:17.179515+0000","last_peered":"2024-05-08T22:07:17.179515+0000","last_clean":"2024-05-08T22:07:17.179515+0000","last_became_active":"2024-05-08T22:05:14.059818+0000","last_became_peered":"2024-05-08T22:05:14.059818+0000","last_unstale":"2024-05-08T22:07:17.179515+0000","last_undegraded":"2024-05-08T22:07:17.179515+0000","last_fullsized":"2024-05-08T22:07:17.179515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:53.118508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.154256+0000","last_change":"2024-05-08T22:05:14.057185+0000","last_active":"2024-05-08T22:07:17.154256+0000","last_peered":"2024-05-08T22:07:17.154256+0000","last_clean":"2024-05-08T22:07:17.154256+0000","last_became_active":"2024-05-08T22:05:14.056889+0000","last_became_peered":"2024-05-08T22:05:14.056889+0000","last_unstale":"2024-05-08T22:07:17.154256+0000","last_undegraded":"2024-05-08T22:07:17.154256+0000","last_fullsized":"2024-05-08T22:07:17.154256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:35.372705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.146313+0000","last_change":"2024-05-08T22:05:14.062282+0000","last_active":"2024-05-08T22:07:17.146313+0000","last_peered":"2024-05-08T22:07:17.146313+0000","last_clean":"2024-05-08T22:07:17.146313+0000","last_became_active":"2024-05-08T22:05:14.061574+0000","last_became_peered":"2024-05-08T22:05:14.061574+0000","last_unstale":"2024-05-08T22:07:17.146313+0000","last_undegraded":"2024-05-08T22:07:17.146313+0000","last_fullsized":"2024-05-08T22:07:17.146313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:16:32.873590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.153206+0000","last_change":"2024-05-08T22:05:14.061267+0000","last_active":"2024-05-08T22:07:17.153206+0000","last_peered":"2024-05-08T22:07:17.153206+0000","last_clean":"2024-05-08T22:07:17.153206+0000","last_became_active":"2024-05-08T22:05:14.060943+0000","last_became_peered":"2024-05-08T22:05:14.060943+0000","last_unstale":"2024-05-08T22:07:17.153206+0000","last_undegraded":"2024-05-08T22:07:17.153206+0000","last_fullsized":"2024-05-08T22:07:17.153206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:03.831138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.155172+0000","last_change":"2024-05-08T22:05:14.067136+0000","last_active":"2024-05-08T22:07:17.155172+0000","last_peered":"2024-05-08T22:07:17.155172+0000","last_clean":"2024-05-08T22:07:17.155172+0000","last_became_active":"2024-05-08T22:05:14.066810+0000","last_became_peered":"2024-05-08T22:05:14.066810+0000","last_unstale":"2024-05-08T22:07:17.155172+0000","last_undegraded":"2024-05-08T22:07:17.155172+0000","last_fullsized":"2024-05-08T22:07:17.155172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:43.001024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.157400+0000","last_change":"2024-05-08T22:05:14.062119+0000","last_active":"2024-05-08T22:07:17.157400+0000","last_peered":"2024-05-08T22:07:17.157400+0000","last_clean":"2024-05-08T22:07:17.157400+0000","last_became_active":"2024-05-08T22:05:14.058977+0000","last_became_peered":"2024-05-08T22:05:14.058977+0000","last_unstale":"2024-05-08T22:07:17.157400+0000","last_undegraded":"2024-05-08T22:07:17.157400+0000","last_fullsized":"2024-05-08T22:07:17.157400+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:03.657985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.163070+0000","last_change":"2024-05-08T22:05:14.067363+0000","last_active":"2024-05-08T22:07:17.163070+0000","last_peered":"2024-05-08T22:07:17.163070+0000","last_clean":"2024-05-08T22:07:17.163070+0000","last_became_active":"2024-05-08T22:05:14.066119+0000","last_became_peered":"2024-05-08T22:05:14.066119+0000","last_unstale":"2024-05-08T22:07:17.163070+0000","last_undegraded":"2024-05-08T22:07:17.163070+0000","last_fullsized":"2024-05-08T22:07:17.163070+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:43.426750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.174135+0000","last_change":"2024-05-08T22:05:14.073925+0000","last_active":"2024-05-08T22:07:17.174135+0000","last_peered":"2024-05-08T22:07:17.174135+0000","last_clean":"2024-05-08T22:07:17.174135+0000","last_became_active":"2024-05-08T22:05:14.072287+0000","last_became_peered":"2024-05-08T22:05:14.072287+0000","last_unstale":"2024-05-08T22:07:17.174135+0000","last_undegraded":"2024-05-08T22:07:17.174135+0000","last_fullsized":"2024-05-08T22:07:17.174135+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:13.537938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.172678+0000","last_change":"2024-05-08T22:05:14.057274+0000","last_active":"2024-05-08T22:07:17.172678+0000","last_peered":"2024-05-08T22:07:17.172678+0000","last_clean":"2024-05-08T22:07:17.172678+0000","last_became_active":"2024-05-08T22:05:14.056989+0000","last_became_peered":"2024-05-08T22:05:14.056989+0000","last_unstale":"2024-05-08T22:07:17.172678+0000","last_undegraded":"2024-05-08T22:07:17.172678+0000","last_fullsized":"2024-05-08T22:07:17.172678+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:32.316406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.174918+0000","last_change":"2024-05-08T22:05:14.069263+0000","last_active":"2024-05-08T22:07:17.174918+0000","last_peered":"2024-05-08T22:07:17.174918+0000","last_clean":"2024-05-08T22:07:17.174918+0000","last_became_active":"2024-05-08T22:05:14.068960+0000","last_became_peered":"2024-05-08T22:05:14.068960+0000","last_unstale":"2024-05-08T22:07:17.174918+0000","last_undegraded":"2024-05-08T22:07:17.174918+0000","last_fullsized":"2024-05-08T22:07:17.174918+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:59:42.829110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.175370+0000","last_change":"2024-05-08T22:05:14.049393+0000","last_active":"2024-05-08T22:07:17.175370+0000","last_peered":"2024-05-08T22:07:17.175370+0000","last_clean":"2024-05-08T22:07:17.175370+0000","last_became_active":"2024-05-08T22:05:14.049094+0000","last_became_peered":"2024-05-08T22:05:14.049094+0000","last_unstale":"2024-05-08T22:07:17.175370+0000","last_undegraded":"2024-05-08T22:07:17.175370+0000","last_fullsized":"2024-05-08T22:07:17.175370+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:42.106568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.176257+0000","last_change":"2024-05-08T22:05:14.055436+0000","last_active":"2024-05-08T22:07:17.176257+0000","last_peered":"2024-05-08T22:07:17.176257+0000","last_clean":"2024-05-08T22:07:17.176257+0000","last_became_active":"2024-05-08T22:05:14.055100+0000","last_became_peered":"2024-05-08T22:05:14.055100+0000","last_unstale":"2024-05-08T22:07:17.176257+0000","last_undegraded":"2024-05-08T22:07:17.176257+0000","last_fullsized":"2024-05-08T22:07:17.176257+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:52.833780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.447942+0000","last_change":"2024-05-08T22:05:18.532086+0000","last_active":"2024-05-08T22:06:42.447942+0000","last_peered":"2024-05-08T22:06:42.447942+0000","last_clean":"2024-05-08T22:06:42.447942+0000","last_became_active":"2024-05-08T22:05:18.531485+0000","last_became_peered":"2024-05-08T22:05:18.531485+0000","last_unstale":"2024-05-08T22:06:42.447942+0000","last_undegraded":"2024-05-08T22:06:42.447942+0000","last_fullsized":"2024-05-08T22:06:42.447942+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:40:36.050367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.103820+0000","last_change":"2024-05-08T22:05:13.549611+0000","last_active":"2024-05-08T22:07:17.103820+0000","last_peered":"2024-05-08T22:07:17.103820+0000","last_clean":"2024-05-08T22:07:17.103820+0000","last_became_active":"2024-05-08T22:05:13.549140+0000","last_became_peered":"2024-05-08T22:05:13.549140+0000","last_unstale":"2024-05-08T22:07:17.103820+0000","last_undegraded":"2024-05-08T22:07:17.103820+0000","last_fullsized":"2024-05-08T22:07:17.103820+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:54.785123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.166838+0000","last_change":"2024-05-08T22:06:42.448887+0000","last_active":"2024-05-08T22:07:17.166838+0000","last_peered":"2024-05-08T22:07:17.166838+0000","last_clean":"2024-05-08T22:07:17.166838+0000","last_became_active":"2024-05-08T22:06:42.448391+0000","last_became_peered":"2024-05-08T22:06:42.448391+0000","last_unstale":"2024-05-08T22:07:17.166838+0000","last_undegraded":"2024-05-08T22:07:17.166838+0000","last_fullsized":"2024-05-08T22:07:17.166838+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:51.859658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976371+0000","last_change":"2024-05-08T22:05:42.147633+0000","last_active":"2024-05-08T22:06:41.976371+0000","last_peered":"2024-05-08T22:06:41.976371+0000","last_clean":"2024-05-08T22:06:41.976371+0000","last_became_active":"2024-05-08T22:05:42.147284+0000","last_became_peered":"2024-05-08T22:05:42.147284+0000","last_unstale":"2024-05-08T22:06:41.976371+0000","last_undegraded":"2024-05-08T22:06:41.976371+0000","last_fullsized":"2024-05-08T22:06:41.976371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:32.864756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.139965+0000","last_change":"2024-05-08T22:05:13.551351+0000","last_active":"2024-05-08T22:07:17.139965+0000","last_peered":"2024-05-08T22:07:17.139965+0000","last_clean":"2024-05-08T22:07:17.139965+0000","last_became_active":"2024-05-08T22:05:13.551014+0000","last_became_peered":"2024-05-08T22:05:13.551014+0000","last_unstale":"2024-05-08T22:07:17.139965+0000","last_undegraded":"2024-05-08T22:07:17.139965+0000","last_fullsized":"2024-05-08T22:07:17.139965+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:07:22.405251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.180951+0000","last_change":"2024-05-08T22:05:14.067270+0000","last_active":"2024-05-08T22:07:17.180951+0000","last_peered":"2024-05-08T22:07:17.180951+0000","last_clean":"2024-05-08T22:07:17.180951+0000","last_became_active":"2024-05-08T22:05:14.065496+0000","last_became_peered":"2024-05-08T22:05:14.065496+0000","last_unstale":"2024-05-08T22:07:17.180951+0000","last_undegraded":"2024-05-08T22:07:17.180951+0000","last_fullsized":"2024-05-08T22:07:17.180951+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:07.087880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976487+0000","last_change":"2024-05-08T22:05:18.526557+0000","last_active":"2024-05-08T22:06:41.976487+0000","last_peered":"2024-05-08T22:06:41.976487+0000","last_clean":"2024-05-08T22:06:41.976487+0000","last_became_active":"2024-05-08T22:05:18.526129+0000","last_became_peered":"2024-05-08T22:05:18.526129+0000","last_unstale":"2024-05-08T22:06:41.976487+0000","last_undegraded":"2024-05-08T22:06:41.976487+0000","last_fullsized":"2024-05-08T22:06:41.976487+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:36.376579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.376514+0000","last_change":"2024-05-08T22:05:41.835074+0000","last_active":"2024-05-08T22:06:42.376514+0000","last_peered":"2024-05-08T22:06:42.376514+0000","last_clean":"2024-05-08T22:06:42.376514+0000","last_became_active":"2024-05-08T22:05:41.829567+0000","last_became_peered":"2024-05-08T22:05:41.829567+0000","last_unstale":"2024-05-08T22:06:42.376514+0000","last_undegraded":"2024-05-08T22:06:42.376514+0000","last_fullsized":"2024-05-08T22:06:42.376514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:15.261352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.151662+0000","last_change":"2024-05-08T22:05:14.060089+0000","last_active":"2024-05-08T22:07:17.151662+0000","last_peered":"2024-05-08T22:07:17.151662+0000","last_clean":"2024-05-08T22:07:17.151662+0000","last_became_active":"2024-05-08T22:05:14.057620+0000","last_became_peered":"2024-05-08T22:05:14.057620+0000","last_unstale":"2024-05-08T22:07:17.151662+0000","last_undegraded":"2024-05-08T22:07:17.151662+0000","last_fullsized":"2024-05-08T22:07:17.151662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:29:33.127510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976369+0000","last_change":"2024-05-08T22:05:18.529746+0000","last_active":"2024-05-08T22:06:41.976369+0000","last_peered":"2024-05-08T22:06:41.976369+0000","last_clean":"2024-05-08T22:06:41.976369+0000","last_became_active":"2024-05-08T22:05:18.529097+0000","last_became_peered":"2024-05-08T22:05:18.529097+0000","last_unstale":"2024-05-08T22:06:41.976369+0000","last_undegraded":"2024-05-08T22:06:41.976369+0000","last_fullsized":"2024-05-08T22:06:41.976369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:59:36.249787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.376187+0000","last_change":"2024-05-08T22:05:41.837103+0000","last_active":"2024-05-08T22:06:42.376187+0000","last_peered":"2024-05-08T22:06:42.376187+0000","last_clean":"2024-05-08T22:06:42.376187+0000","last_became_active":"2024-05-08T22:05:41.829464+0000","last_became_peered":"2024-05-08T22:05:41.829464+0000","last_unstale":"2024-05-08T22:06:42.376187+0000","last_undegraded":"2024-05-08T22:06:42.376187+0000","last_fullsized":"2024-05-08T22:06:42.376187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:55.747487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.151416+0000","last_change":"2024-05-08T22:05:14.064695+0000","last_active":"2024-05-08T22:07:17.151416+0000","last_peered":"2024-05-08T22:07:17.151416+0000","last_clean":"2024-05-08T22:07:17.151416+0000","last_became_active":"2024-05-08T22:05:14.064283+0000","last_became_peered":"2024-05-08T22:05:14.064283+0000","last_unstale":"2024-05-08T22:07:17.151416+0000","last_undegraded":"2024-05-08T22:07:17.151416+0000","last_fullsized":"2024-05-08T22:07:17.151416+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:06.980594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.447815+0000","last_change":"2024-05-08T22:05:18.528475+0000","last_active":"2024-05-08T22:06:42.447815+0000","last_peered":"2024-05-08T22:06:42.447815+0000","last_clean":"2024-05-08T22:06:42.447815+0000","last_became_active":"2024-05-08T22:05:18.528089+0000","last_became_peered":"2024-05-08T22:05:18.528089+0000","last_unstale":"2024-05-08T22:06:42.447815+0000","last_undegraded":"2024-05-08T22:06:42.447815+0000","last_fullsized":"2024-05-08T22:06:42.447815+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:01.508984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'8","reported_seq":138,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:21.653137+0000","last_change":"2024-05-08T22:05:13.549191+0000","last_active":"2024-05-08T22:07:21.653137+0000","last_peered":"2024-05-08T22:07:21.653137+0000","last_clean":"2024-05-08T22:07:21.653137+0000","last_became_active":"2024-05-08T22:05:13.548737+0000","last_became_peered":"2024-05-08T22:05:13.548737+0000","last_unstale":"2024-05-08T22:07:21.653137+0000","last_undegraded":"2024-05-08T22:07:21.653137+0000","last_fullsized":"2024-05-08T22:07:21.653137+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:07.355656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1501,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.180110+0000","last_change":"2024-05-08T22:06:42.018982+0000","last_active":"2024-05-08T22:07:17.180110+0000","last_peered":"2024-05-08T22:07:17.180110+0000","last_clean":"2024-05-08T22:07:17.180110+0000","last_became_active":"2024-05-08T22:06:42.018609+0000","last_became_peered":"2024-05-08T22:06:42.018609+0000","last_unstale":"2024-05-08T22:07:17.180110+0000","last_undegraded":"2024-05-08T22:07:17.180110+0000","last_fullsized":"2024-05-08T22:07:17.180110+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:37.578711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.447741+0000","last_change":"2024-05-08T22:05:18.536861+0000","last_active":"2024-05-08T22:06:42.447741+0000","last_peered":"2024-05-08T22:06:42.447741+0000","last_clean":"2024-05-08T22:06:42.447741+0000","last_became_active":"2024-05-08T22:05:18.536472+0000","last_became_peered":"2024-05-08T22:05:18.536472+0000","last_unstale":"2024-05-08T22:06:42.447741+0000","last_undegraded":"2024-05-08T22:06:42.447741+0000","last_fullsized":"2024-05-08T22:06:42.447741+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:45:21.826812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.467017+0000","last_change":"2024-05-08T22:05:13.553326+0000","last_active":"2024-05-08T22:06:42.467017+0000","last_peered":"2024-05-08T22:06:42.467017+0000","last_clean":"2024-05-08T22:06:42.467017+0000","last_became_active":"2024-05-08T22:05:13.552983+0000","last_became_peered":"2024-05-08T22:05:13.552983+0000","last_unstale":"2024-05-08T22:06:42.467017+0000","last_undegraded":"2024-05-08T22:06:42.467017+0000","last_fullsized":"2024-05-08T22:06:42.467017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:17.195253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.165741+0000","last_change":"2024-05-08T22:05:14.068617+0000","last_active":"2024-05-08T22:07:17.165741+0000","last_peered":"2024-05-08T22:07:17.165741+0000","last_clean":"2024-05-08T22:07:17.165741+0000","last_became_active":"2024-05-08T22:05:14.067400+0000","last_became_peered":"2024-05-08T22:05:14.067400+0000","last_unstale":"2024-05-08T22:07:17.165741+0000","last_undegraded":"2024-05-08T22:07:17.165741+0000","last_fullsized":"2024-05-08T22:07:17.165741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:25.488212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.130379+0000","last_change":"2024-05-08T22:05:18.538294+0000","last_active":"2024-05-08T22:07:17.130379+0000","last_peered":"2024-05-08T22:07:17.130379+0000","last_clean":"2024-05-08T22:07:17.130379+0000","last_became_active":"2024-05-08T22:05:18.537984+0000","last_became_peered":"2024-05-08T22:05:18.537984+0000","last_unstale":"2024-05-08T22:07:17.130379+0000","last_undegraded":"2024-05-08T22:07:17.130379+0000","last_fullsized":"2024-05-08T22:07:17.130379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:32.117439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.017185+0000","last_change":"2024-05-08T22:05:13.557669+0000","last_active":"2024-05-08T22:06:42.017185+0000","last_peered":"2024-05-08T22:06:42.017185+0000","last_clean":"2024-05-08T22:06:42.017185+0000","last_became_active":"2024-05-08T22:05:13.555835+0000","last_became_peered":"2024-05-08T22:05:13.555835+0000","last_unstale":"2024-05-08T22:06:42.017185+0000","last_undegraded":"2024-05-08T22:06:42.017185+0000","last_fullsized":"2024-05-08T22:06:42.017185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:02.377635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.179458+0000","last_change":"2024-05-08T22:05:14.060144+0000","last_active":"2024-05-08T22:07:17.179458+0000","last_peered":"2024-05-08T22:07:17.179458+0000","last_clean":"2024-05-08T22:07:17.179458+0000","last_became_active":"2024-05-08T22:05:14.059387+0000","last_became_peered":"2024-05-08T22:05:14.059387+0000","last_unstale":"2024-05-08T22:07:17.179458+0000","last_undegraded":"2024-05-08T22:07:17.179458+0000","last_fullsized":"2024-05-08T22:07:17.179458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:32:28.057831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.140368+0000","last_change":"2024-05-08T22:05:18.527293+0000","last_active":"2024-05-08T22:07:17.140368+0000","last_peered":"2024-05-08T22:07:17.140368+0000","last_clean":"2024-05-08T22:07:17.140368+0000","last_became_active":"2024-05-08T22:05:18.526690+0000","last_became_peered":"2024-05-08T22:05:18.526690+0000","last_unstale":"2024-05-08T22:07:17.140368+0000","last_undegraded":"2024-05-08T22:07:17.140368+0000","last_fullsized":"2024-05-08T22:07:17.140368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:50.343536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.180700+0000","last_change":"2024-05-08T22:05:13.549774+0000","last_active":"2024-05-08T22:06:42.180700+0000","last_peered":"2024-05-08T22:06:42.180700+0000","last_clean":"2024-05-08T22:06:42.180700+0000","last_became_active":"2024-05-08T22:05:13.549531+0000","last_became_peered":"2024-05-08T22:05:13.549531+0000","last_unstale":"2024-05-08T22:06:42.180700+0000","last_undegraded":"2024-05-08T22:06:42.180700+0000","last_fullsized":"2024-05-08T22:06:42.180700+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:59:23.798728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.181017+0000","last_change":"2024-05-08T22:05:14.054232+0000","last_active":"2024-05-08T22:07:17.181017+0000","last_peered":"2024-05-08T22:07:17.181017+0000","last_clean":"2024-05-08T22:07:17.181017+0000","last_became_active":"2024-05-08T22:05:14.053806+0000","last_became_peered":"2024-05-08T22:05:14.053806+0000","last_unstale":"2024-05-08T22:07:17.181017+0000","last_undegraded":"2024-05-08T22:07:17.181017+0000","last_fullsized":"2024-05-08T22:07:17.181017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:32:21.678792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.979318+0000","last_change":"2024-05-08T22:05:18.535241+0000","last_active":"2024-05-08T22:06:41.979318+0000","last_peered":"2024-05-08T22:06:41.979318+0000","last_clean":"2024-05-08T22:06:41.979318+0000","last_became_active":"2024-05-08T22:05:18.534753+0000","last_became_peered":"2024-05-08T22:05:18.534753+0000","last_unstale":"2024-05-08T22:06:41.979318+0000","last_undegraded":"2024-05-08T22:06:41.979318+0000","last_fullsized":"2024-05-08T22:06:41.979318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:46:15.138437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":129,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:21.652796+0000","last_change":"2024-05-08T22:05:13.544899+0000","last_active":"2024-05-08T22:07:21.652796+0000","last_peered":"2024-05-08T22:07:21.652796+0000","last_clean":"2024-05-08T22:07:21.652796+0000","last_became_active":"2024-05-08T22:05:13.544498+0000","last_became_peered":"2024-05-08T22:05:13.544498+0000","last_unstale":"2024-05-08T22:07:21.652796+0000","last_undegraded":"2024-05-08T22:07:21.652796+0000","last_fullsized":"2024-05-08T22:07:21.652796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:58.169648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.176371+0000","last_change":"2024-05-08T22:05:14.049377+0000","last_active":"2024-05-08T22:07:17.176371+0000","last_peered":"2024-05-08T22:07:17.176371+0000","last_clean":"2024-05-08T22:07:17.176371+0000","last_became_active":"2024-05-08T22:05:14.049039+0000","last_became_peered":"2024-05-08T22:05:14.049039+0000","last_unstale":"2024-05-08T22:07:17.176371+0000","last_undegraded":"2024-05-08T22:07:17.176371+0000","last_fullsized":"2024-05-08T22:07:17.176371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:24.415392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976341+0000","last_change":"2024-05-08T22:05:18.528912+0000","last_active":"2024-05-08T22:06:41.976341+0000","last_peered":"2024-05-08T22:06:41.976341+0000","last_clean":"2024-05-08T22:06:41.976341+0000","last_became_active":"2024-05-08T22:05:18.528433+0000","last_became_peered":"2024-05-08T22:05:18.528433+0000","last_unstale":"2024-05-08T22:06:41.976341+0000","last_undegraded":"2024-05-08T22:06:41.976341+0000","last_fullsized":"2024-05-08T22:06:41.976341+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:05.574728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.055447+0000","last_change":"2024-05-08T22:05:13.550214+0000","last_active":"2024-05-08T22:06:42.055447+0000","last_peered":"2024-05-08T22:06:42.055447+0000","last_clean":"2024-05-08T22:06:42.055447+0000","last_became_active":"2024-05-08T22:05:13.549717+0000","last_became_peered":"2024-05-08T22:05:13.549717+0000","last_unstale":"2024-05-08T22:06:42.055447+0000","last_undegraded":"2024-05-08T22:06:42.055447+0000","last_fullsized":"2024-05-08T22:06:42.055447+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:12:20.406333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.177101+0000","last_change":"2024-05-08T22:05:14.071214+0000","last_active":"2024-05-08T22:07:17.177101+0000","last_peered":"2024-05-08T22:07:17.177101+0000","last_clean":"2024-05-08T22:07:17.177101+0000","last_became_active":"2024-05-08T22:05:14.066777+0000","last_became_peered":"2024-05-08T22:05:14.066777+0000","last_unstale":"2024-05-08T22:07:17.177101+0000","last_undegraded":"2024-05-08T22:07:17.177101+0000","last_fullsized":"2024-05-08T22:07:17.177101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:53.360911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976794+0000","last_change":"2024-05-08T22:05:18.523688+0000","last_active":"2024-05-08T22:06:41.976794+0000","last_peered":"2024-05-08T22:06:41.976794+0000","last_clean":"2024-05-08T22:06:41.976794+0000","last_became_active":"2024-05-08T22:05:18.523236+0000","last_became_peered":"2024-05-08T22:05:18.523236+0000","last_unstale":"2024-05-08T22:06:41.976794+0000","last_undegraded":"2024-05-08T22:06:41.976794+0000","last_fullsized":"2024-05-08T22:06:41.976794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:34.531699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.181222+0000","last_change":"2024-05-08T22:05:13.551027+0000","last_active":"2024-05-08T22:06:42.181222+0000","last_peered":"2024-05-08T22:06:42.181222+0000","last_clean":"2024-05-08T22:06:42.181222+0000","last_became_active":"2024-05-08T22:05:13.550471+0000","last_became_peered":"2024-05-08T22:05:13.550471+0000","last_unstale":"2024-05-08T22:06:42.181222+0000","last_undegraded":"2024-05-08T22:06:42.181222+0000","last_fullsized":"2024-05-08T22:06:42.181222+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:50.900419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.156519+0000","last_change":"2024-05-08T22:05:14.065191+0000","last_active":"2024-05-08T22:07:17.156519+0000","last_peered":"2024-05-08T22:07:17.156519+0000","last_clean":"2024-05-08T22:07:17.156519+0000","last_became_active":"2024-05-08T22:05:14.064893+0000","last_became_peered":"2024-05-08T22:05:14.064893+0000","last_unstale":"2024-05-08T22:07:17.156519+0000","last_undegraded":"2024-05-08T22:07:17.156519+0000","last_fullsized":"2024-05-08T22:07:17.156519+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:11.344646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976480+0000","last_change":"2024-05-08T22:05:18.530447+0000","last_active":"2024-05-08T22:06:41.976480+0000","last_peered":"2024-05-08T22:06:41.976480+0000","last_clean":"2024-05-08T22:06:41.976480+0000","last_became_active":"2024-05-08T22:05:18.528171+0000","last_became_peered":"2024-05-08T22:05:18.528171+0000","last_unstale":"2024-05-08T22:06:41.976480+0000","last_undegraded":"2024-05-08T22:06:41.976480+0000","last_fullsized":"2024-05-08T22:06:41.976480+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:09:43.347366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'39","reported_seq":278,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:22.654579+0000","last_change":"2024-05-08T22:05:13.547289+0000","last_active":"2024-05-08T22:07:22.654579+0000","last_peered":"2024-05-08T22:07:22.654579+0000","last_clean":"2024-05-08T22:07:22.654579+0000","last_became_active":"2024-05-08T22:05:13.546860+0000","last_became_peered":"2024-05-08T22:05:13.546860+0000","last_unstale":"2024-05-08T22:07:22.654579+0000","last_undegraded":"2024-05-08T22:07:22.654579+0000","last_fullsized":"2024-05-08T22:07:22.654579+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:08:50.339361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69082,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":218,"num_read_kb":175,"num_write":40,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.146652+0000","last_change":"2024-05-08T22:05:14.057925+0000","last_active":"2024-05-08T22:07:17.146652+0000","last_peered":"2024-05-08T22:07:17.146652+0000","last_clean":"2024-05-08T22:07:17.146652+0000","last_became_active":"2024-05-08T22:05:14.057662+0000","last_became_peered":"2024-05-08T22:05:14.057662+0000","last_unstale":"2024-05-08T22:07:17.146652+0000","last_undegraded":"2024-05-08T22:07:17.146652+0000","last_fullsized":"2024-05-08T22:07:17.146652+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:49.929610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.181486+0000","last_change":"2024-05-08T22:05:18.535509+0000","last_active":"2024-05-08T22:06:42.181486+0000","last_peered":"2024-05-08T22:06:42.181486+0000","last_clean":"2024-05-08T22:06:42.181486+0000","last_became_active":"2024-05-08T22:05:18.535024+0000","last_became_peered":"2024-05-08T22:05:18.535024+0000","last_unstale":"2024-05-08T22:06:42.181486+0000","last_undegraded":"2024-05-08T22:06:42.181486+0000","last_fullsized":"2024-05-08T22:06:42.181486+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:28.098737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.447863+0000","last_change":"2024-05-08T22:05:13.552096+0000","last_active":"2024-05-08T22:06:42.447863+0000","last_peered":"2024-05-08T22:06:42.447863+0000","last_clean":"2024-05-08T22:06:42.447863+0000","last_became_active":"2024-05-08T22:05:13.551655+0000","last_became_peered":"2024-05-08T22:05:13.551655+0000","last_unstale":"2024-05-08T22:06:42.447863+0000","last_undegraded":"2024-05-08T22:06:42.447863+0000","last_fullsized":"2024-05-08T22:06:42.447863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:24.371008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.180706+0000","last_change":"2024-05-08T22:05:14.066075+0000","last_active":"2024-05-08T22:07:17.180706+0000","last_peered":"2024-05-08T22:07:17.180706+0000","last_clean":"2024-05-08T22:07:17.180706+0000","last_became_active":"2024-05-08T22:05:14.065581+0000","last_became_peered":"2024-05-08T22:05:14.065581+0000","last_unstale":"2024-05-08T22:07:17.180706+0000","last_undegraded":"2024-05-08T22:07:17.180706+0000","last_fullsized":"2024-05-08T22:07:17.180706+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:15.403714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.376646+0000","last_change":"2024-05-08T22:05:41.831782+0000","last_active":"2024-05-08T22:06:42.376646+0000","last_peered":"2024-05-08T22:06:42.376646+0000","last_clean":"2024-05-08T22:06:42.376646+0000","last_became_active":"2024-05-08T22:05:41.831335+0000","last_became_peered":"2024-05-08T22:05:41.831335+0000","last_unstale":"2024-05-08T22:06:42.376646+0000","last_undegraded":"2024-05-08T22:06:42.376646+0000","last_fullsized":"2024-05-08T22:06:42.376646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:39:34.031128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.181155+0000","last_change":"2024-05-08T22:05:13.549112+0000","last_active":"2024-05-08T22:06:42.181155+0000","last_peered":"2024-05-08T22:06:42.181155+0000","last_clean":"2024-05-08T22:06:42.181155+0000","last_became_active":"2024-05-08T22:05:13.548763+0000","last_became_peered":"2024-05-08T22:05:13.548763+0000","last_unstale":"2024-05-08T22:06:42.181155+0000","last_undegraded":"2024-05-08T22:06:42.181155+0000","last_fullsized":"2024-05-08T22:06:42.181155+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:26:56.448611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.158808+0000","last_change":"2024-05-08T22:05:14.067116+0000","last_active":"2024-05-08T22:07:17.158808+0000","last_peered":"2024-05-08T22:07:17.158808+0000","last_clean":"2024-05-08T22:07:17.158808+0000","last_became_active":"2024-05-08T22:05:14.064775+0000","last_became_peered":"2024-05-08T22:05:14.064775+0000","last_unstale":"2024-05-08T22:07:17.158808+0000","last_undegraded":"2024-05-08T22:07:17.158808+0000","last_fullsized":"2024-05-08T22:07:17.158808+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:28:32.901405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.977984+0000","last_change":"2024-05-08T22:05:18.539546+0000","last_active":"2024-05-08T22:06:41.977984+0000","last_peered":"2024-05-08T22:06:41.977984+0000","last_clean":"2024-05-08T22:06:41.977984+0000","last_became_active":"2024-05-08T22:05:18.538998+0000","last_became_peered":"2024-05-08T22:05:18.538998+0000","last_unstale":"2024-05-08T22:06:41.977984+0000","last_undegraded":"2024-05-08T22:06:41.977984+0000","last_fullsized":"2024-05-08T22:06:41.977984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:30.453881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":135,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:21.653538+0000","last_change":"2024-05-08T22:05:13.549053+0000","last_active":"2024-05-08T22:07:21.653538+0000","last_peered":"2024-05-08T22:07:21.653538+0000","last_clean":"2024-05-08T22:07:21.653538+0000","last_became_active":"2024-05-08T22:05:13.548739+0000","last_became_peered":"2024-05-08T22:05:13.548739+0000","last_unstale":"2024-05-08T22:07:21.653538+0000","last_undegraded":"2024-05-08T22:07:21.653538+0000","last_fullsized":"2024-05-08T22:07:21.653538+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:07:10.298084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1719,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.150047+0000","last_change":"2024-05-08T22:05:14.061753+0000","last_active":"2024-05-08T22:07:17.150047+0000","last_peered":"2024-05-08T22:07:17.150047+0000","last_clean":"2024-05-08T22:07:17.150047+0000","last_became_active":"2024-05-08T22:05:14.061412+0000","last_became_peered":"2024-05-08T22:05:14.061412+0000","last_unstale":"2024-05-08T22:07:17.150047+0000","last_undegraded":"2024-05-08T22:07:17.150047+0000","last_fullsized":"2024-05-08T22:07:17.150047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:54.380181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.017707+0000","last_change":"2024-05-08T22:05:18.532627+0000","last_active":"2024-05-08T22:06:42.017707+0000","last_peered":"2024-05-08T22:06:42.017707+0000","last_clean":"2024-05-08T22:06:42.017707+0000","last_became_active":"2024-05-08T22:05:18.532205+0000","last_became_peered":"2024-05-08T22:05:18.532205+0000","last_unstale":"2024-05-08T22:06:42.017707+0000","last_undegraded":"2024-05-08T22:06:42.017707+0000","last_fullsized":"2024-05-08T22:06:42.017707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:20:11.028645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":64,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976362+0000","last_change":"2024-05-08T22:05:41.841832+0000","last_active":"2024-05-08T22:06:41.976362+0000","last_peered":"2024-05-08T22:06:41.976362+0000","last_clean":"2024-05-08T22:06:41.976362+0000","last_became_active":"2024-05-08T22:05:41.829584+0000","last_became_peered":"2024-05-08T22:05:41.829584+0000","last_unstale":"2024-05-08T22:06:41.976362+0000","last_undegraded":"2024-05-08T22:06:41.976362+0000","last_fullsized":"2024-05-08T22:06:41.976362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:52:19.876215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.149577+0000","last_change":"2024-05-08T22:05:14.063027+0000","last_active":"2024-05-08T22:07:17.149577+0000","last_peered":"2024-05-08T22:07:17.149577+0000","last_clean":"2024-05-08T22:07:17.149577+0000","last_became_active":"2024-05-08T22:05:14.062682+0000","last_became_peered":"2024-05-08T22:05:14.062682+0000","last_unstale":"2024-05-08T22:07:17.149577+0000","last_undegraded":"2024-05-08T22:07:17.149577+0000","last_fullsized":"2024-05-08T22:07:17.149577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:47.248832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.180890+0000","last_change":"2024-05-08T22:05:18.528370+0000","last_active":"2024-05-08T22:06:42.180890+0000","last_peered":"2024-05-08T22:06:42.180890+0000","last_clean":"2024-05-08T22:06:42.180890+0000","last_became_active":"2024-05-08T22:05:18.527990+0000","last_became_peered":"2024-05-08T22:05:18.527990+0000","last_unstale":"2024-05-08T22:06:42.180890+0000","last_undegraded":"2024-05-08T22:06:42.180890+0000","last_fullsized":"2024-05-08T22:06:42.180890+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:07.524175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.120821+0000","last_change":"2024-05-08T22:05:13.547543+0000","last_active":"2024-05-08T22:07:17.120821+0000","last_peered":"2024-05-08T22:07:17.120821+0000","last_clean":"2024-05-08T22:07:17.120821+0000","last_became_active":"2024-05-08T22:05:13.547191+0000","last_became_peered":"2024-05-08T22:05:13.547191+0000","last_unstale":"2024-05-08T22:07:17.120821+0000","last_undegraded":"2024-05-08T22:07:17.120821+0000","last_fullsized":"2024-05-08T22:07:17.120821+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:53.213239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.150554+0000","last_change":"2024-05-08T22:05:14.062677+0000","last_active":"2024-05-08T22:07:17.150554+0000","last_peered":"2024-05-08T22:07:17.150554+0000","last_clean":"2024-05-08T22:07:17.150554+0000","last_became_active":"2024-05-08T22:05:14.062306+0000","last_became_peered":"2024-05-08T22:05:14.062306+0000","last_unstale":"2024-05-08T22:07:17.150554+0000","last_undegraded":"2024-05-08T22:07:17.150554+0000","last_fullsized":"2024-05-08T22:07:17.150554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:00.356505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.119465+0000","last_change":"2024-05-08T22:05:18.617075+0000","last_active":"2024-05-08T22:07:17.119465+0000","last_peered":"2024-05-08T22:07:17.119465+0000","last_clean":"2024-05-08T22:07:17.119465+0000","last_became_active":"2024-05-08T22:05:18.616653+0000","last_became_peered":"2024-05-08T22:05:18.616653+0000","last_unstale":"2024-05-08T22:07:17.119465+0000","last_undegraded":"2024-05-08T22:07:17.119465+0000","last_fullsized":"2024-05-08T22:07:17.119465+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:30.537064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.103554+0000","last_change":"2024-05-08T22:05:13.553457+0000","last_active":"2024-05-08T22:07:17.103554+0000","last_peered":"2024-05-08T22:07:17.103554+0000","last_clean":"2024-05-08T22:07:17.103554+0000","last_became_active":"2024-05-08T22:05:13.551583+0000","last_became_peered":"2024-05-08T22:05:13.551583+0000","last_unstale":"2024-05-08T22:07:17.103554+0000","last_undegraded":"2024-05-08T22:07:17.103554+0000","last_fullsized":"2024-05-08T22:07:17.103554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:07:05.971413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.172164+0000","last_change":"2024-05-08T22:05:14.057476+0000","last_active":"2024-05-08T22:07:17.172164+0000","last_peered":"2024-05-08T22:07:17.172164+0000","last_clean":"2024-05-08T22:07:17.172164+0000","last_became_active":"2024-05-08T22:05:14.056550+0000","last_became_peered":"2024-05-08T22:05:14.056550+0000","last_unstale":"2024-05-08T22:07:17.172164+0000","last_undegraded":"2024-05-08T22:07:17.172164+0000","last_fullsized":"2024-05-08T22:07:17.172164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:41:29.264096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.017335+0000","last_change":"2024-05-08T22:05:18.530719+0000","last_active":"2024-05-08T22:06:42.017335+0000","last_peered":"2024-05-08T22:06:42.017335+0000","last_clean":"2024-05-08T22:06:42.017335+0000","last_became_active":"2024-05-08T22:05:18.529659+0000","last_became_peered":"2024-05-08T22:05:18.529659+0000","last_unstale":"2024-05-08T22:06:42.017335+0000","last_undegraded":"2024-05-08T22:06:42.017335+0000","last_fullsized":"2024-05-08T22:06:42.017335+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:13:06.514460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.107307+0000","last_change":"2024-05-08T22:05:13.556191+0000","last_active":"2024-05-08T22:07:17.107307+0000","last_peered":"2024-05-08T22:07:17.107307+0000","last_clean":"2024-05-08T22:07:17.107307+0000","last_became_active":"2024-05-08T22:05:13.555888+0000","last_became_peered":"2024-05-08T22:05:13.555888+0000","last_unstale":"2024-05-08T22:07:17.107307+0000","last_undegraded":"2024-05-08T22:07:17.107307+0000","last_fullsized":"2024-05-08T22:07:17.107307+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:05.637980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.179969+0000","last_change":"2024-05-08T22:05:41.828023+0000","last_active":"2024-05-08T22:07:17.179969+0000","last_peered":"2024-05-08T22:07:17.179969+0000","last_clean":"2024-05-08T22:07:17.179969+0000","last_became_active":"2024-05-08T22:05:41.827550+0000","last_became_peered":"2024-05-08T22:05:41.827550+0000","last_unstale":"2024-05-08T22:07:17.179969+0000","last_undegraded":"2024-05-08T22:07:17.179969+0000","last_fullsized":"2024-05-08T22:07:17.179969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:13.234423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.017591+0000","last_change":"2024-05-08T22:05:41.828529+0000","last_active":"2024-05-08T22:06:42.017591+0000","last_peered":"2024-05-08T22:06:42.017591+0000","last_clean":"2024-05-08T22:06:42.017591+0000","last_became_active":"2024-05-08T22:05:41.828081+0000","last_became_peered":"2024-05-08T22:05:41.828081+0000","last_unstale":"2024-05-08T22:06:42.017591+0000","last_undegraded":"2024-05-08T22:06:42.017591+0000","last_fullsized":"2024-05-08T22:06:42.017591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:27.478823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.152262+0000","last_change":"2024-05-08T22:05:14.069096+0000","last_active":"2024-05-08T22:07:17.152262+0000","last_peered":"2024-05-08T22:07:17.152262+0000","last_clean":"2024-05-08T22:07:17.152262+0000","last_became_active":"2024-05-08T22:05:14.068796+0000","last_became_peered":"2024-05-08T22:05:14.068796+0000","last_unstale":"2024-05-08T22:07:17.152262+0000","last_undegraded":"2024-05-08T22:07:17.152262+0000","last_fullsized":"2024-05-08T22:07:17.152262+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:52:52.391196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":131,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:21.653111+0000","last_change":"2024-05-08T22:05:13.559286+0000","last_active":"2024-05-08T22:07:21.653111+0000","last_peered":"2024-05-08T22:07:21.653111+0000","last_clean":"2024-05-08T22:07:21.653111+0000","last_became_active":"2024-05-08T22:05:13.558922+0000","last_became_peered":"2024-05-08T22:05:13.558922+0000","last_unstale":"2024-05-08T22:07:21.653111+0000","last_undegraded":"2024-05-08T22:07:21.653111+0000","last_fullsized":"2024-05-08T22:07:21.653111+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:25:24.732422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.055193+0000","last_change":"2024-05-08T22:05:18.527031+0000","last_active":"2024-05-08T22:06:42.055193+0000","last_peered":"2024-05-08T22:06:42.055193+0000","last_clean":"2024-05-08T22:06:42.055193+0000","last_became_active":"2024-05-08T22:05:18.526573+0000","last_became_peered":"2024-05-08T22:05:18.526573+0000","last_unstale":"2024-05-08T22:06:42.055193+0000","last_undegraded":"2024-05-08T22:06:42.055193+0000","last_fullsized":"2024-05-08T22:06:42.055193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:08.492359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.178225+0000","last_change":"2024-05-08T22:06:42.278191+0000","last_active":"2024-05-08T22:07:17.178225+0000","last_peered":"2024-05-08T22:07:17.178225+0000","last_clean":"2024-05-08T22:07:17.178225+0000","last_became_active":"2024-05-08T22:06:42.277819+0000","last_became_peered":"2024-05-08T22:06:42.277819+0000","last_unstale":"2024-05-08T22:07:17.178225+0000","last_undegraded":"2024-05-08T22:07:17.178225+0000","last_fullsized":"2024-05-08T22:07:17.178225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:48.919595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.104081+0000","last_change":"2024-05-08T22:05:13.545709+0000","last_active":"2024-05-08T22:07:17.104081+0000","last_peered":"2024-05-08T22:07:17.104081+0000","last_clean":"2024-05-08T22:07:17.104081+0000","last_became_active":"2024-05-08T22:05:13.545419+0000","last_became_peered":"2024-05-08T22:05:13.545419+0000","last_unstale":"2024-05-08T22:07:17.104081+0000","last_undegraded":"2024-05-08T22:07:17.104081+0000","last_fullsized":"2024-05-08T22:07:17.104081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:44.903903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"88'148","reported_seq":384,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:19.254206+0000","last_change":"2024-05-08T22:04:20.948328+0000","last_active":"2024-05-08T22:07:19.254206+0000","last_peered":"2024-05-08T22:07:19.254206+0000","last_clean":"2024-05-08T22:07:19.254206+0000","last_became_active":"2024-05-08T22:04:20.947887+0000","last_became_peered":"2024-05-08T22:04:20.947887+0000","last_unstale":"2024-05-08T22:07:19.254206+0000","last_undegraded":"2024-05-08T22:07:19.254206+0000","last_fullsized":"2024-05-08T22:07:19.254206+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:01:41.485277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:01:41.485277+0000","last_clean_scrub_stamp":"2024-05-08T22:01:41.485277+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:43:59.208120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.181413+0000","last_change":"2024-05-08T22:05:41.824591+0000","last_active":"2024-05-08T22:06:42.181413+0000","last_peered":"2024-05-08T22:06:42.181413+0000","last_clean":"2024-05-08T22:06:42.181413+0000","last_became_active":"2024-05-08T22:05:41.824053+0000","last_became_peered":"2024-05-08T22:05:41.824053+0000","last_unstale":"2024-05-08T22:06:42.181413+0000","last_undegraded":"2024-05-08T22:06:42.181413+0000","last_fullsized":"2024-05-08T22:06:42.181413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:18:57.269459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.145622+0000","last_change":"2024-05-08T22:05:14.066818+0000","last_active":"2024-05-08T22:07:17.145622+0000","last_peered":"2024-05-08T22:07:17.145622+0000","last_clean":"2024-05-08T22:07:17.145622+0000","last_became_active":"2024-05-08T22:05:14.065152+0000","last_became_peered":"2024-05-08T22:05:14.065152+0000","last_unstale":"2024-05-08T22:07:17.145622+0000","last_undegraded":"2024-05-08T22:07:17.145622+0000","last_fullsized":"2024-05-08T22:07:17.145622+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:53:19.018852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.107386+0000","last_change":"2024-05-08T22:05:13.550365+0000","last_active":"2024-05-08T22:07:17.107386+0000","last_peered":"2024-05-08T22:07:17.107386+0000","last_clean":"2024-05-08T22:07:17.107386+0000","last_became_active":"2024-05-08T22:05:13.549677+0000","last_became_peered":"2024-05-08T22:05:13.549677+0000","last_unstale":"2024-05-08T22:07:17.107386+0000","last_undegraded":"2024-05-08T22:07:17.107386+0000","last_fullsized":"2024-05-08T22:07:17.107386+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:10.842838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.376136+0000","last_change":"2024-05-08T22:05:18.535777+0000","last_active":"2024-05-08T22:06:42.376136+0000","last_peered":"2024-05-08T22:06:42.376136+0000","last_clean":"2024-05-08T22:06:42.376136+0000","last_became_active":"2024-05-08T22:05:18.535327+0000","last_became_peered":"2024-05-08T22:05:18.535327+0000","last_unstale":"2024-05-08T22:06:42.376136+0000","last_undegraded":"2024-05-08T22:06:42.376136+0000","last_fullsized":"2024-05-08T22:06:42.376136+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:13.886989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.121005+0000","last_change":"2024-05-08T22:05:13.554045+0000","last_active":"2024-05-08T22:07:17.121005+0000","last_peered":"2024-05-08T22:07:17.121005+0000","last_clean":"2024-05-08T22:07:17.121005+0000","last_became_active":"2024-05-08T22:05:13.553315+0000","last_became_peered":"2024-05-08T22:05:13.553315+0000","last_unstale":"2024-05-08T22:07:17.121005+0000","last_undegraded":"2024-05-08T22:07:17.121005+0000","last_fullsized":"2024-05-08T22:07:17.121005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:13.131267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.182154+0000","last_change":"2024-05-08T22:05:14.065018+0000","last_active":"2024-05-08T22:07:17.182154+0000","last_peered":"2024-05-08T22:07:17.182154+0000","last_clean":"2024-05-08T22:07:17.182154+0000","last_became_active":"2024-05-08T22:05:14.064701+0000","last_became_peered":"2024-05-08T22:05:14.064701+0000","last_unstale":"2024-05-08T22:07:17.182154+0000","last_undegraded":"2024-05-08T22:07:17.182154+0000","last_fullsized":"2024-05-08T22:07:17.182154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:48:43.468373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.121062+0000","last_change":"2024-05-08T22:05:18.617853+0000","last_active":"2024-05-08T22:07:17.121062+0000","last_peered":"2024-05-08T22:07:17.121062+0000","last_clean":"2024-05-08T22:07:17.121062+0000","last_became_active":"2024-05-08T22:05:18.616255+0000","last_became_peered":"2024-05-08T22:05:18.616255+0000","last_unstale":"2024-05-08T22:07:17.121062+0000","last_undegraded":"2024-05-08T22:07:17.121062+0000","last_fullsized":"2024-05-08T22:07:17.121062+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:13.233425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'41","reported_seq":86,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:22.787863+0000","last_change":"2024-05-08T22:05:13.546274+0000","last_active":"2024-05-08T22:07:22.787863+0000","last_peered":"2024-05-08T22:07:22.787863+0000","last_clean":"2024-05-08T22:07:22.787863+0000","last_became_active":"2024-05-08T22:05:13.543651+0000","last_became_peered":"2024-05-08T22:05:13.543651+0000","last_unstale":"2024-05-08T22:07:22.787863+0000","last_undegraded":"2024-05-08T22:07:22.787863+0000","last_fullsized":"2024-05-08T22:07:22.787863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:21:58.601261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":73,"num_write_kb":835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.162082+0000","last_change":"2024-05-08T22:05:14.056535+0000","last_active":"2024-05-08T22:07:17.162082+0000","last_peered":"2024-05-08T22:07:17.162082+0000","last_clean":"2024-05-08T22:07:17.162082+0000","last_became_active":"2024-05-08T22:05:14.056211+0000","last_became_peered":"2024-05-08T22:05:14.056211+0000","last_unstale":"2024-05-08T22:07:17.162082+0000","last_undegraded":"2024-05-08T22:07:17.162082+0000","last_fullsized":"2024-05-08T22:07:17.162082+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:46:30.744490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.107088+0000","last_change":"2024-05-08T22:05:18.524939+0000","last_active":"2024-05-08T22:07:17.107088+0000","last_peered":"2024-05-08T22:07:17.107088+0000","last_clean":"2024-05-08T22:07:17.107088+0000","last_became_active":"2024-05-08T22:05:18.524518+0000","last_became_peered":"2024-05-08T22:05:18.524518+0000","last_unstale":"2024-05-08T22:07:17.107088+0000","last_undegraded":"2024-05-08T22:07:17.107088+0000","last_fullsized":"2024-05-08T22:07:17.107088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:50.192400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.977078+0000","last_change":"2024-05-08T22:05:13.554346+0000","last_active":"2024-05-08T22:06:41.977078+0000","last_peered":"2024-05-08T22:06:41.977078+0000","last_clean":"2024-05-08T22:06:41.977078+0000","last_became_active":"2024-05-08T22:05:13.554080+0000","last_became_peered":"2024-05-08T22:05:13.554080+0000","last_unstale":"2024-05-08T22:06:41.977078+0000","last_undegraded":"2024-05-08T22:06:41.977078+0000","last_fullsized":"2024-05-08T22:06:41.977078+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:29:49.695682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.181025+0000","last_change":"2024-05-08T22:05:14.071422+0000","last_active":"2024-05-08T22:07:17.181025+0000","last_peered":"2024-05-08T22:07:17.181025+0000","last_clean":"2024-05-08T22:07:17.181025+0000","last_became_active":"2024-05-08T22:05:14.070396+0000","last_became_peered":"2024-05-08T22:05:14.070396+0000","last_unstale":"2024-05-08T22:07:17.181025+0000","last_undegraded":"2024-05-08T22:07:17.181025+0000","last_fullsized":"2024-05-08T22:07:17.181025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:37.957983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976556+0000","last_change":"2024-05-08T22:05:18.617149+0000","last_active":"2024-05-08T22:06:41.976556+0000","last_peered":"2024-05-08T22:06:41.976556+0000","last_clean":"2024-05-08T22:06:41.976556+0000","last_became_active":"2024-05-08T22:05:18.616699+0000","last_became_peered":"2024-05-08T22:05:18.616699+0000","last_unstale":"2024-05-08T22:06:41.976556+0000","last_undegraded":"2024-05-08T22:06:41.976556+0000","last_fullsized":"2024-05-08T22:06:41.976556+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:51.167142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.117273+0000","last_change":"2024-05-08T22:05:13.551212+0000","last_active":"2024-05-08T22:07:17.117273+0000","last_peered":"2024-05-08T22:07:17.117273+0000","last_clean":"2024-05-08T22:07:17.117273+0000","last_became_active":"2024-05-08T22:05:13.550909+0000","last_became_peered":"2024-05-08T22:05:13.550909+0000","last_unstale":"2024-05-08T22:07:17.117273+0000","last_undegraded":"2024-05-08T22:07:17.117273+0000","last_fullsized":"2024-05-08T22:07:17.117273+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:48.657928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.123447+0000","last_change":"2024-05-08T22:05:14.053976+0000","last_active":"2024-05-08T22:07:17.123447+0000","last_peered":"2024-05-08T22:07:17.123447+0000","last_clean":"2024-05-08T22:07:17.123447+0000","last_became_active":"2024-05-08T22:05:14.053664+0000","last_became_peered":"2024-05-08T22:05:14.053664+0000","last_unstale":"2024-05-08T22:07:17.123447+0000","last_undegraded":"2024-05-08T22:07:17.123447+0000","last_fullsized":"2024-05-08T22:07:17.123447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:25:25.642768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.978021+0000","last_change":"2024-05-08T22:05:18.536771+0000","last_active":"2024-05-08T22:06:41.978021+0000","last_peered":"2024-05-08T22:06:41.978021+0000","last_clean":"2024-05-08T22:06:41.978021+0000","last_became_active":"2024-05-08T22:05:18.536260+0000","last_became_peered":"2024-05-08T22:05:18.536260+0000","last_unstale":"2024-05-08T22:06:41.978021+0000","last_undegraded":"2024-05-08T22:06:41.978021+0000","last_fullsized":"2024-05-08T22:06:41.978021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:34.414381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.979065+0000","last_change":"2024-05-08T22:05:13.540679+0000","last_active":"2024-05-08T22:06:41.979065+0000","last_peered":"2024-05-08T22:06:41.979065+0000","last_clean":"2024-05-08T22:06:41.979065+0000","last_became_active":"2024-05-08T22:05:13.539428+0000","last_became_peered":"2024-05-08T22:05:13.539428+0000","last_unstale":"2024-05-08T22:06:41.979065+0000","last_undegraded":"2024-05-08T22:06:41.979065+0000","last_fullsized":"2024-05-08T22:06:41.979065+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:59:34.455411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.153688+0000","last_change":"2024-05-08T22:05:14.061849+0000","last_active":"2024-05-08T22:07:17.153688+0000","last_peered":"2024-05-08T22:07:17.153688+0000","last_clean":"2024-05-08T22:07:17.153688+0000","last_became_active":"2024-05-08T22:05:14.061028+0000","last_became_peered":"2024-05-08T22:05:14.061028+0000","last_unstale":"2024-05-08T22:07:17.153688+0000","last_undegraded":"2024-05-08T22:07:17.153688+0000","last_fullsized":"2024-05-08T22:07:17.153688+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:11:59.489376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976549+0000","last_change":"2024-05-08T22:05:18.517994+0000","last_active":"2024-05-08T22:06:41.976549+0000","last_peered":"2024-05-08T22:06:41.976549+0000","last_clean":"2024-05-08T22:06:41.976549+0000","last_became_active":"2024-05-08T22:05:18.517655+0000","last_became_peered":"2024-05-08T22:05:18.517655+0000","last_unstale":"2024-05-08T22:06:41.976549+0000","last_undegraded":"2024-05-08T22:06:41.976549+0000","last_fullsized":"2024-05-08T22:06:41.976549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:50:08.956277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"84'1","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.447717+0000","last_change":"2024-05-08T22:05:13.552851+0000","last_active":"2024-05-08T22:06:42.447717+0000","last_peered":"2024-05-08T22:06:42.447717+0000","last_clean":"2024-05-08T22:06:42.447717+0000","last_became_active":"2024-05-08T22:05:13.552507+0000","last_became_peered":"2024-05-08T22:05:13.552507+0000","last_unstale":"2024-05-08T22:06:42.447717+0000","last_undegraded":"2024-05-08T22:06:42.447717+0000","last_fullsized":"2024-05-08T22:06:42.447717+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:29.248623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.176835+0000","last_change":"2024-05-08T22:05:14.062618+0000","last_active":"2024-05-08T22:07:17.176835+0000","last_peered":"2024-05-08T22:07:17.176835+0000","last_clean":"2024-05-08T22:07:17.176835+0000","last_became_active":"2024-05-08T22:05:14.062308+0000","last_became_peered":"2024-05-08T22:05:14.062308+0000","last_unstale":"2024-05-08T22:07:17.176835+0000","last_undegraded":"2024-05-08T22:07:17.176835+0000","last_fullsized":"2024-05-08T22:07:17.176835+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:34.195343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976573+0000","last_change":"2024-05-08T22:05:18.616846+0000","last_active":"2024-05-08T22:06:41.976573+0000","last_peered":"2024-05-08T22:06:41.976573+0000","last_clean":"2024-05-08T22:06:41.976573+0000","last_became_active":"2024-05-08T22:05:18.616369+0000","last_became_peered":"2024-05-08T22:05:18.616369+0000","last_unstale":"2024-05-08T22:06:41.976573+0000","last_undegraded":"2024-05-08T22:06:41.976573+0000","last_fullsized":"2024-05-08T22:06:41.976573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:26.128949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976543+0000","last_change":"2024-05-08T22:05:13.549712+0000","last_active":"2024-05-08T22:06:41.976543+0000","last_peered":"2024-05-08T22:06:41.976543+0000","last_clean":"2024-05-08T22:06:41.976543+0000","last_became_active":"2024-05-08T22:05:13.549508+0000","last_became_peered":"2024-05-08T22:05:13.549508+0000","last_unstale":"2024-05-08T22:06:41.976543+0000","last_undegraded":"2024-05-08T22:06:41.976543+0000","last_fullsized":"2024-05-08T22:06:41.976543+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:30:22.706389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.157838+0000","last_change":"2024-05-08T22:05:14.060920+0000","last_active":"2024-05-08T22:07:17.157838+0000","last_peered":"2024-05-08T22:07:17.157838+0000","last_clean":"2024-05-08T22:07:17.157838+0000","last_became_active":"2024-05-08T22:05:14.059914+0000","last_became_peered":"2024-05-08T22:05:14.059914+0000","last_unstale":"2024-05-08T22:07:17.157838+0000","last_undegraded":"2024-05-08T22:07:17.157838+0000","last_fullsized":"2024-05-08T22:07:17.157838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:15.513600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.276872+0000","last_change":"2024-05-08T22:05:18.534076+0000","last_active":"2024-05-08T22:06:42.276872+0000","last_peered":"2024-05-08T22:06:42.276872+0000","last_clean":"2024-05-08T22:06:42.276872+0000","last_became_active":"2024-05-08T22:05:18.533679+0000","last_became_peered":"2024-05-08T22:05:18.533679+0000","last_unstale":"2024-05-08T22:06:42.276872+0000","last_undegraded":"2024-05-08T22:06:42.276872+0000","last_fullsized":"2024-05-08T22:06:42.276872+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:08.979193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.276939+0000","last_change":"2024-05-08T22:05:41.845354+0000","last_active":"2024-05-08T22:06:42.276939+0000","last_peered":"2024-05-08T22:06:42.276939+0000","last_clean":"2024-05-08T22:06:42.276939+0000","last_became_active":"2024-05-08T22:05:41.830506+0000","last_became_peered":"2024-05-08T22:05:41.830506+0000","last_unstale":"2024-05-08T22:06:42.276939+0000","last_undegraded":"2024-05-08T22:06:42.276939+0000","last_fullsized":"2024-05-08T22:06:42.276939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:53:23.081939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.153480+0000","last_change":"2024-05-08T22:05:14.065868+0000","last_active":"2024-05-08T22:07:17.153480+0000","last_peered":"2024-05-08T22:07:17.153480+0000","last_clean":"2024-05-08T22:07:17.153480+0000","last_became_active":"2024-05-08T22:05:14.065545+0000","last_became_peered":"2024-05-08T22:05:14.065545+0000","last_unstale":"2024-05-08T22:07:17.153480+0000","last_undegraded":"2024-05-08T22:07:17.153480+0000","last_fullsized":"2024-05-08T22:07:17.153480+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:56:34.309095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.467018+0000","last_change":"2024-05-08T22:05:18.534261+0000","last_active":"2024-05-08T22:06:42.467018+0000","last_peered":"2024-05-08T22:06:42.467018+0000","last_clean":"2024-05-08T22:06:42.467018+0000","last_became_active":"2024-05-08T22:05:18.531814+0000","last_became_peered":"2024-05-08T22:05:18.531814+0000","last_unstale":"2024-05-08T22:06:42.467018+0000","last_undegraded":"2024-05-08T22:06:42.467018+0000","last_fullsized":"2024-05-08T22:06:42.467018+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:38.344546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'39","reported_seq":68,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.397200+0000","last_change":"2024-05-08T22:05:13.549344+0000","last_active":"2024-05-08T22:07:17.397200+0000","last_peered":"2024-05-08T22:07:17.397200+0000","last_clean":"2024-05-08T22:07:17.397200+0000","last_became_active":"2024-05-08T22:05:13.548993+0000","last_became_peered":"2024-05-08T22:05:13.548993+0000","last_unstale":"2024-05-08T22:07:17.397200+0000","last_undegraded":"2024-05-08T22:07:17.397200+0000","last_fullsized":"2024-05-08T22:07:17.397200+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:50.096602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.180642+0000","last_change":"2024-05-08T22:05:14.065758+0000","last_active":"2024-05-08T22:07:17.180642+0000","last_peered":"2024-05-08T22:07:17.180642+0000","last_clean":"2024-05-08T22:07:17.180642+0000","last_became_active":"2024-05-08T22:05:14.065442+0000","last_became_peered":"2024-05-08T22:05:14.065442+0000","last_unstale":"2024-05-08T22:07:17.180642+0000","last_undegraded":"2024-05-08T22:07:17.180642+0000","last_fullsized":"2024-05-08T22:07:17.180642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:43.047796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.180606+0000","last_change":"2024-05-08T22:05:18.529804+0000","last_active":"2024-05-08T22:06:42.180606+0000","last_peered":"2024-05-08T22:06:42.180606+0000","last_clean":"2024-05-08T22:06:42.180606+0000","last_became_active":"2024-05-08T22:05:18.529346+0000","last_became_peered":"2024-05-08T22:05:18.529346+0000","last_unstale":"2024-05-08T22:06:42.180606+0000","last_undegraded":"2024-05-08T22:06:42.180606+0000","last_fullsized":"2024-05-08T22:06:42.180606+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:13.565707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.017625+0000","last_change":"2024-05-08T22:05:13.551818+0000","last_active":"2024-05-08T22:06:42.017625+0000","last_peered":"2024-05-08T22:06:42.017625+0000","last_clean":"2024-05-08T22:06:42.017625+0000","last_became_active":"2024-05-08T22:05:13.551080+0000","last_became_peered":"2024-05-08T22:05:13.551080+0000","last_unstale":"2024-05-08T22:06:42.017625+0000","last_undegraded":"2024-05-08T22:06:42.017625+0000","last_fullsized":"2024-05-08T22:06:42.017625+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:31.636482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.146513+0000","last_change":"2024-05-08T22:05:14.066572+0000","last_active":"2024-05-08T22:07:17.146513+0000","last_peered":"2024-05-08T22:07:17.146513+0000","last_clean":"2024-05-08T22:07:17.146513+0000","last_became_active":"2024-05-08T22:05:14.066272+0000","last_became_peered":"2024-05-08T22:05:14.066272+0000","last_unstale":"2024-05-08T22:07:17.146513+0000","last_undegraded":"2024-05-08T22:07:17.146513+0000","last_fullsized":"2024-05-08T22:07:17.146513+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:20:27.423464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.151234+0000","last_change":"2024-05-08T22:05:14.061435+0000","last_active":"2024-05-08T22:07:17.151234+0000","last_peered":"2024-05-08T22:07:17.151234+0000","last_clean":"2024-05-08T22:07:17.151234+0000","last_became_active":"2024-05-08T22:05:14.061079+0000","last_became_peered":"2024-05-08T22:05:14.061079+0000","last_unstale":"2024-05-08T22:07:17.151234+0000","last_undegraded":"2024-05-08T22:07:17.151234+0000","last_fullsized":"2024-05-08T22:07:17.151234+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:42.684743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.147601+0000","last_change":"2024-05-08T22:05:41.828352+0000","last_active":"2024-05-08T22:07:17.147601+0000","last_peered":"2024-05-08T22:07:17.147601+0000","last_clean":"2024-05-08T22:07:17.147601+0000","last_became_active":"2024-05-08T22:05:41.827782+0000","last_became_peered":"2024-05-08T22:05:41.827782+0000","last_unstale":"2024-05-08T22:07:17.147601+0000","last_undegraded":"2024-05-08T22:07:17.147601+0000","last_fullsized":"2024-05-08T22:07:17.147601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:08.883148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.176085+0000","last_change":"2024-05-08T22:05:14.066697+0000","last_active":"2024-05-08T22:07:17.176085+0000","last_peered":"2024-05-08T22:07:17.176085+0000","last_clean":"2024-05-08T22:07:17.176085+0000","last_became_active":"2024-05-08T22:05:14.065952+0000","last_became_peered":"2024-05-08T22:05:14.065952+0000","last_unstale":"2024-05-08T22:07:17.176085+0000","last_undegraded":"2024-05-08T22:07:17.176085+0000","last_fullsized":"2024-05-08T22:07:17.176085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:03:19.006644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.179903+0000","last_change":"2024-05-08T22:05:14.072295+0000","last_active":"2024-05-08T22:07:17.179903+0000","last_peered":"2024-05-08T22:07:17.179903+0000","last_clean":"2024-05-08T22:07:17.179903+0000","last_became_active":"2024-05-08T22:05:14.065034+0000","last_became_peered":"2024-05-08T22:05:14.065034+0000","last_unstale":"2024-05-08T22:07:17.179903+0000","last_undegraded":"2024-05-08T22:07:17.179903+0000","last_fullsized":"2024-05-08T22:07:17.179903+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:08:41.380814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.147802+0000","last_change":"2024-05-08T22:05:14.067263+0000","last_active":"2024-05-08T22:07:17.147802+0000","last_peered":"2024-05-08T22:07:17.147802+0000","last_clean":"2024-05-08T22:07:17.147802+0000","last_became_active":"2024-05-08T22:05:14.066957+0000","last_became_peered":"2024-05-08T22:05:14.066957+0000","last_unstale":"2024-05-08T22:07:17.147802+0000","last_undegraded":"2024-05-08T22:07:17.147802+0000","last_fullsized":"2024-05-08T22:07:17.147802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:29:35.638489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.151853+0000","last_change":"2024-05-08T22:05:14.060561+0000","last_active":"2024-05-08T22:07:17.151853+0000","last_peered":"2024-05-08T22:07:17.151853+0000","last_clean":"2024-05-08T22:07:17.151853+0000","last_became_active":"2024-05-08T22:05:14.060150+0000","last_became_peered":"2024-05-08T22:05:14.060150+0000","last_unstale":"2024-05-08T22:07:17.151853+0000","last_undegraded":"2024-05-08T22:07:17.151853+0000","last_fullsized":"2024-05-08T22:07:17.151853+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:22.710776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.148159+0000","last_change":"2024-05-08T22:05:14.056394+0000","last_active":"2024-05-08T22:07:17.148159+0000","last_peered":"2024-05-08T22:07:17.148159+0000","last_clean":"2024-05-08T22:07:17.148159+0000","last_became_active":"2024-05-08T22:05:14.055977+0000","last_became_peered":"2024-05-08T22:05:14.055977+0000","last_unstale":"2024-05-08T22:07:17.148159+0000","last_undegraded":"2024-05-08T22:07:17.148159+0000","last_fullsized":"2024-05-08T22:07:17.148159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:56:48.249379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.161659+0000","last_change":"2024-05-08T22:06:42.448921+0000","last_active":"2024-05-08T22:07:17.161659+0000","last_peered":"2024-05-08T22:07:17.161659+0000","last_clean":"2024-05-08T22:07:17.161659+0000","last_became_active":"2024-05-08T22:06:42.448567+0000","last_became_peered":"2024-05-08T22:06:42.448567+0000","last_unstale":"2024-05-08T22:07:17.161659+0000","last_undegraded":"2024-05-08T22:07:17.161659+0000","last_fullsized":"2024-05-08T22:07:17.161659+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:52.822511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.180232+0000","last_change":"2024-05-08T22:05:14.069982+0000","last_active":"2024-05-08T22:07:17.180232+0000","last_peered":"2024-05-08T22:07:17.180232+0000","last_clean":"2024-05-08T22:07:17.180232+0000","last_became_active":"2024-05-08T22:05:14.068293+0000","last_became_peered":"2024-05-08T22:05:14.068293+0000","last_unstale":"2024-05-08T22:07:17.180232+0000","last_undegraded":"2024-05-08T22:07:17.180232+0000","last_fullsized":"2024-05-08T22:07:17.180232+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:36.396983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.163286+0000","last_change":"2024-05-08T22:05:14.058393+0000","last_active":"2024-05-08T22:07:17.163286+0000","last_peered":"2024-05-08T22:07:17.163286+0000","last_clean":"2024-05-08T22:07:17.163286+0000","last_became_active":"2024-05-08T22:05:14.058083+0000","last_became_peered":"2024-05-08T22:05:14.058083+0000","last_unstale":"2024-05-08T22:07:17.163286+0000","last_undegraded":"2024-05-08T22:07:17.163286+0000","last_fullsized":"2024-05-08T22:07:17.163286+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:10:47.074957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.166671+0000","last_change":"2024-05-08T22:05:14.056989+0000","last_active":"2024-05-08T22:07:17.166671+0000","last_peered":"2024-05-08T22:07:17.166671+0000","last_clean":"2024-05-08T22:07:17.166671+0000","last_became_active":"2024-05-08T22:05:14.056669+0000","last_became_peered":"2024-05-08T22:05:14.056669+0000","last_unstale":"2024-05-08T22:07:17.166671+0000","last_undegraded":"2024-05-08T22:07:17.166671+0000","last_fullsized":"2024-05-08T22:07:17.166671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:56:51.173885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.175205+0000","last_change":"2024-05-08T22:05:14.058635+0000","last_active":"2024-05-08T22:07:17.175205+0000","last_peered":"2024-05-08T22:07:17.175205+0000","last_clean":"2024-05-08T22:07:17.175205+0000","last_became_active":"2024-05-08T22:05:14.058243+0000","last_became_peered":"2024-05-08T22:05:14.058243+0000","last_unstale":"2024-05-08T22:07:17.175205+0000","last_undegraded":"2024-05-08T22:07:17.175205+0000","last_fullsized":"2024-05-08T22:07:17.175205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:18.142563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.146228+0000","last_change":"2024-05-08T22:05:14.060735+0000","last_active":"2024-05-08T22:07:17.146228+0000","last_peered":"2024-05-08T22:07:17.146228+0000","last_clean":"2024-05-08T22:07:17.146228+0000","last_became_active":"2024-05-08T22:05:14.060391+0000","last_became_peered":"2024-05-08T22:05:14.060391+0000","last_unstale":"2024-05-08T22:07:17.146228+0000","last_undegraded":"2024-05-08T22:07:17.146228+0000","last_fullsized":"2024-05-08T22:07:17.146228+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:30.528705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107988,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599518,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":747,"num_read_kb":483,"num_write":283,"num_write_kb":1441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":694,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1798554,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579940,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36156,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708996,"statfs":{"total":95995035648,"available":95958011904,"internally_reserved":0,"allocated":966656,"data_stored":498801,"data_compressed":26516,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36043205},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.54100000000000004,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.67000000000000004,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.48199999999999998}]},{"osd":1,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.65600000000000003,"15min":0.65300000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.878,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76800000000000002,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.373}]},{"osd":2,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.36299999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.65500000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.309},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.48599999999999999,"15min":0.47799999999999998},"min":{"1min":0.34699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72799999999999998,"5min":1.052,"15min":1.052},"last":0.27500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":4,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.53100000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.433},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.752,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.41999999999999998}]},{"osd":5,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.53200000000000003,"15min":0.52300000000000002},"min":{"1min":0.38,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82999999999999996,"5min":1.204,"15min":1.204},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.61299999999999999,"15min":0.61099999999999999},"min":{"1min":0.42299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86799999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.34899999999999998}]},{"osd":6,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.65800000000000003,"15min":0.66500000000000004},"min":{"1min":0.48099999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91500000000000004,"5min":1.278,"15min":1.278},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.65700000000000003,"15min":0.66000000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.85199999999999998,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.46899999999999997}]},{"osd":7,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.745,"15min":0.74299999999999999},"min":{"1min":0.58999999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.96899999999999997,"5min":1.415,"15min":1.415},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.74099999999999999,"15min":0.746},"min":{"1min":0.60199999999999998,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.86799999999999999,"5min":1.48,"15min":1.48},"last":0.45200000000000001}]},{"osd":8,"last update":"Wed May 8 22:07:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.67000000000000004,"15min":0.67500000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.85699999999999998,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.505},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.73099999999999998,"15min":0.73699999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.99299999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.52700000000000002}]},{"osd":9,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76300000000000001,"15min":0.76400000000000001},"min":{"1min":0.59399999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.97899999999999998,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.65300000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0369999999999999,"5min":1.5,"15min":1.5},"last":0.58399999999999996}]},{"osd":10,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.78900000000000003,"15min":0.79600000000000004},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.96399999999999997,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78100000000000003,"15min":0.78400000000000003},"min":{"1min":0.61899999999999999,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93899999999999995,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.56799999999999995}]}]},{"osd":10,"up_from":67,"seq":287762808871,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1134592,"data_stored":1287551,"data_compressed":68668,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:07:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53900000000000003,"15min":0.54000000000000004},"min":{"1min":0.40100000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.747,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.874,"15min":0.92200000000000004},"min":{"1min":0.44,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.76700000000000002,"5min":7.258,"15min":7.258},"last":0.46800000000000003}]},{"osd":1,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.437,"15min":0.438},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.46700000000000003,"15min":0.46999999999999997},"min":{"1min":0.30599999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.65400000000000003,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.27800000000000002}]},{"osd":2,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.624,"15min":0.61899999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59999999999999998,"15min":0.60199999999999998},"min":{"1min":0.40600000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76000000000000001,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.50900000000000001}]},{"osd":3,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.77500000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.61099999999999999,"15min":0.629},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73499999999999999,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.40999999999999998}]},{"osd":4,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54200000000000004,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76500000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.317},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.315,"5min":0.309,"15min":0.309},"max":{"1min":0.79500000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.32500000000000001}]},{"osd":5,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64600000000000002,"15min":0.64000000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.379},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.65600000000000003,"15min":0.66000000000000003},"min":{"1min":0.47999999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.47699999999999998}]},{"osd":6,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.71699999999999997,"15min":0.67700000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.87,"5min":7.2380000000000004,"15min":7.2380000000000004},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.69599999999999995,"15min":0.69199999999999995},"min":{"1min":0.53500000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.88200000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.45800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":8,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71299999999999997,"15min":0.72499999999999998},"min":{"1min":0.51500000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.372},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.72799999999999998,"15min":0.73399999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.436}]},{"osd":9,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.747,"15min":0.755},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.82499999999999996,"15min":0.78600000000000003},"min":{"1min":0.57499999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.92800000000000005,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.51900000000000002}]},{"osd":11,"last update":"Wed May 8 22:06:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.71399999999999997,"15min":0.70599999999999996},"min":{"1min":0.60099999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73899999999999999,"15min":0.73499999999999999},"min":{"1min":0.61899999999999999,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.49099999999999999}]}]},{"osd":9,"up_from":62,"seq":266287972395,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":896349,"data_compressed":45662,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.50900000000000001,"15min":0.52400000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.60999999999999999,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.57099999999999995,"15min":0.58999999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0409999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.47499999999999998}]},{"osd":1,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.73899999999999999,"15min":0.77100000000000002},"min":{"1min":0.47099999999999997,"5min":0.438,"15min":0.438},"max":{"1min":0.80500000000000005,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.46300000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84699999999999998,"5min":0.997,"15min":0.997},"last":0.65500000000000003}]},{"osd":2,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.75800000000000001,"15min":0.78600000000000003},"min":{"1min":0.56000000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81699999999999995,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.72199999999999998,"15min":0.73999999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.878,"5min":1.167,"15min":1.167},"last":0.63900000000000001}]},{"osd":3,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.73899999999999999,"15min":0.79100000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90300000000000002,"5min":1.024,"15min":1.024},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.60699999999999998},"min":{"1min":0.41499999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.81200000000000006,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.54100000000000004}]},{"osd":4,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.75900000000000001,"15min":0.78300000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96999999999999997,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.80000000000000004,"15min":0.83199999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.86799999999999999,"5min":1.081,"15min":1.081},"last":0.88100000000000001}]},{"osd":5,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.81299999999999994,"15min":0.85499999999999998},"min":{"1min":0.49399999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.93200000000000005,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.84699999999999998,"15min":0.88600000000000001},"min":{"1min":0.53700000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.92400000000000004,"5min":1.175,"15min":1.175},"last":0.85599999999999998}]},{"osd":6,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.71199999999999997}]},{"osd":7,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76300000000000001,"15min":0.78400000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.87,"15min":0.91900000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0109999999999999,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.79700000000000004}]},{"osd":8,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80500000000000005,"15min":0.82899999999999996},"min":{"1min":0.60599999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.181,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.90100000000000002,"15min":0.94299999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2509999999999999,"5min":1.478,"15min":1.478},"last":0.66900000000000004}]},{"osd":10,"last update":"Wed May 8 22:07:22 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.86599999999999999,"15min":0.89000000000000001},"min":{"1min":0.56299999999999994,"5min":0.495,"15min":0.495},"max":{"1min":1.2170000000000001,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.84999999999999998,"15min":0.876},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.95099999999999996,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.89900000000000002}]},{"osd":11,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.85599999999999998,"15min":0.85899999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1990000000000001,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.83999999999999997,"15min":0.84599999999999997},"min":{"1min":0.61499999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.98199999999999998,"5min":1.105,"15min":1.105},"last":0.91700000000000004}]}]},{"osd":8,"up_from":55,"seq":236223201328,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":891842,"data_compressed":45662,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.40300000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81699999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.60399999999999998,"15min":0.626},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69899999999999995,"5min":1.865,"15min":1.865},"last":0.58099999999999996}]},{"osd":1,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.60299999999999998,"15min":0.59399999999999997},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.498},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55700000000000005,"15min":0.55100000000000005},"min":{"1min":0.40799999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.42399999999999999}]},{"osd":2,"last update":"Wed May 8 22:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.437,"15min":0.42299999999999999},"min":{"1min":0.315,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.46999999999999997,"15min":0.46500000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68100000000000005,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.377}]},{"osd":3,"last update":"Wed May 8 22:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.72299999999999998,"15min":0.79300000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77900000000000003,"5min":4.8920000000000003,"15min":4.8920000000000003},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63400000000000001,"15min":0.626},"min":{"1min":0.41999999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.51500000000000001}]},{"osd":4,"last update":"Wed May 8 22:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.46000000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.64000000000000001,"15min":0.61399999999999999},"min":{"1min":0.497,"5min":0.433,"15min":0.433},"max":{"1min":0.871,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.47799999999999998}]},{"osd":5,"last update":"Wed May 8 22:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51200000000000001,"15min":0.50600000000000001},"min":{"1min":0.34399999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.72899999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.54200000000000004,"15min":0.51800000000000002},"min":{"1min":0.39900000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.35699999999999998}]},{"osd":6,"last update":"Wed May 8 22:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.83799999999999997,"15min":0.89400000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.98799999999999999,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.90600000000000003,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.61699999999999999}]},{"osd":7,"last update":"Wed May 8 22:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66400000000000003,"15min":0.66200000000000003},"min":{"1min":0.45000000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.504},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.68600000000000005,"15min":0.67000000000000004},"min":{"1min":0.51100000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.495}]},{"osd":9,"last update":"Wed May 8 22:07:05 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78300000000000003,"15min":0.77600000000000002},"min":{"1min":0.57299999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73299999999999998,"15min":0.73099999999999998},"min":{"1min":0.53600000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92300000000000004,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.56399999999999995}]},{"osd":10,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.71799999999999997,"15min":0.70499999999999996},"min":{"1min":0.55000000000000004,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94699999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.76400000000000001,"15min":0.752},"min":{"1min":0.68400000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.57399999999999995}]},{"osd":11,"last update":"Wed May 8 22:06:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.59399999999999997}]}]},{"osd":7,"up_from":48,"seq":206158430257,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":1096485,"data_compressed":62512,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.41199999999999998}]},{"osd":1,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.38800000000000001},"min":{"1min":0.29699999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.57499999999999996,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40999999999999998,"15min":0.40400000000000003},"min":{"1min":0.27900000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.55400000000000005,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.36399999999999999}]},{"osd":2,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.41599999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71499999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.57599999999999996,"15min":0.56999999999999995},"min":{"1min":0.46200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80400000000000005,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.38200000000000001}]},{"osd":3,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.41299999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76800000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68400000000000005,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.48499999999999999}]},{"osd":4,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.49399999999999999,"15min":0.49299999999999999},"min":{"1min":0.35899999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.505,"15min":0.51400000000000001},"min":{"1min":0.34599999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69499999999999995,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.39300000000000002}]},{"osd":5,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.46800000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77500000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63,"15min":0.628},"min":{"1min":0.495,"5min":0.436,"15min":0.436},"max":{"1min":0.83999999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.438}]},{"osd":6,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65800000000000003,"15min":0.66800000000000004},"min":{"1min":0.504,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.80900000000000005,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.67700000000000005,"15min":0.69299999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84099999999999997,"5min":1.002,"15min":1.002},"last":0.58099999999999996}]},{"osd":8,"last update":"Wed May 8 22:06:39 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.68000000000000005,"15min":0.69199999999999995},"min":{"1min":0.48599999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.82299999999999995,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.499},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71599999999999997,"15min":0.72599999999999998},"min":{"1min":0.54700000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.81799999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.52900000000000003}]},{"osd":9,"last update":"Wed May 8 22:07:05 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.624,"5min":0.501,"15min":0.501},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.73399999999999999,"15min":0.74199999999999999},"min":{"1min":0.59699999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.629}]},{"osd":10,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.69999999999999996,"15min":0.71099999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.82099999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.68500000000000005},"min":{"1min":0.499,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.86799999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.47399999999999998}]},{"osd":11,"last update":"Wed May 8 22:07:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.75,"15min":0.748},"min":{"1min":0.61699999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.59399999999999997}]}]},{"osd":6,"up_from":43,"seq":184683593782,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":493940,"data_compressed":26516,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:07:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.47799999999999998,"15min":0.45600000000000002},"min":{"1min":0.35899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.54000000000000004}]},{"osd":1,"last update":"Wed May 8 22:07:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60099999999999998,"15min":0.57499999999999996},"min":{"1min":0.44800000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.65100000000000002,"15min":0.61699999999999999},"min":{"1min":0.46500000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.73999999999999999}]},{"osd":2,"last update":"Wed May 8 22:07:10 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.69099999999999995,"15min":0.65800000000000003},"min":{"1min":0.51600000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.67400000000000004,"15min":0.65400000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.755}]},{"osd":3,"last update":"Wed May 8 22:07:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.42099999999999999,"5min":0.375,"15min":0.375},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.64900000000000002,"15min":0.64100000000000001},"min":{"1min":0.434,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.81299999999999994}]},{"osd":4,"last update":"Wed May 8 22:07:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.45000000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.53300000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":0.96199999999999997}]},{"osd":5,"last update":"Wed May 8 22:07:10 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.76900000000000002,"15min":0.749},"min":{"1min":0.56499999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.753,"15min":0.75},"min":{"1min":0.59399999999999997,"5min":0.438,"15min":0.438},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.044}]},{"osd":7,"last update":"Wed May 8 22:06:28 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.747,"15min":0.73999999999999999},"min":{"1min":0.54900000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.9690000000000001,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.77700000000000002,"15min":0.76500000000000001},"min":{"1min":0.63200000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.86599999999999999}]},{"osd":8,"last update":"Wed May 8 22:06:38 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.83799999999999997,"15min":0.84199999999999997},"min":{"1min":0.58199999999999996,"5min":0.496,"15min":0.496},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.55200000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.097}]},{"osd":9,"last update":"Wed May 8 22:07:04 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.85699999999999998,"15min":0.85899999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.81699999999999995,"15min":0.83499999999999996},"min":{"1min":0.51000000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.1299999999999999}]},{"osd":10,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.85799999999999998,"15min":0.86799999999999999},"min":{"1min":0.66600000000000004,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":0.95899999999999996,"5min":1.639,"15min":1.639},"last":1.165},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.82799999999999996,"15min":0.81999999999999995},"min":{"1min":0.60699999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0369999999999999,"5min":1.927,"15min":1.927},"last":1.1140000000000001}]},{"osd":11,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.91700000000000004,"15min":0.92900000000000005},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.127,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.88500000000000001,"15min":0.89800000000000002},"min":{"1min":0.68000000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.022,"5min":1.78,"15min":1.78},"last":1.0029999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":1036288,"data_stored":560994,"data_compressed":26516,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:06:31 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.55100000000000005,"15min":0.501},"min":{"1min":0.40400000000000003,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.85699999999999998,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55200000000000005,"15min":0.51800000000000002},"min":{"1min":0.44900000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76000000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.5}]},{"osd":2,"last update":"Wed May 8 22:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58099999999999996,"15min":0.50800000000000001},"min":{"1min":0.45400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.77400000000000002,"5min":0.877,"15min":0.877},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.56899999999999995,"15min":0.505},"min":{"1min":0.39300000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76200000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.36899999999999999}]},{"osd":3,"last update":"Wed May 8 22:07:08 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.57899999999999996,"15min":0.52000000000000002},"min":{"1min":0.495,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70799999999999996,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55000000000000004,"15min":0.51800000000000002},"min":{"1min":0.41799999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71099999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.44500000000000001}]},{"osd":4,"last update":"Wed May 8 22:06:31 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.42399999999999999,"15min":0.38400000000000001},"min":{"1min":0.32500000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.80600000000000005,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.45600000000000002,"15min":0.40300000000000002},"min":{"1min":0.35599999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.77600000000000002,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.52900000000000003}]},{"osd":5,"last update":"Wed May 8 22:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.622,"15min":0.56799999999999995},"min":{"1min":0.53600000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.84199999999999997,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.61799999999999999,"15min":0.58399999999999996},"min":{"1min":0.46800000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80000000000000004,"5min":1.089,"15min":1.089},"last":0.46100000000000002}]},{"osd":6,"last update":"Wed May 8 22:07:08 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.626,"15min":0.60599999999999998},"min":{"1min":0.47999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66600000000000004,"15min":0.628},"min":{"1min":0.47899999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.93400000000000005,"5min":1.05,"15min":1.05},"last":0.47399999999999998}]},{"osd":7,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58999999999999997,"15min":0.57199999999999995},"min":{"1min":0.44400000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.871,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.39100000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74199999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.34200000000000003}]},{"osd":8,"last update":"Wed May 8 22:06:42 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65200000000000002,"15min":0.65900000000000003},"min":{"1min":0.53600000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.78300000000000003,"5min":1.115,"15min":1.115},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.53600000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.92100000000000004,"5min":1.222,"15min":1.222},"last":0.53700000000000003}]},{"osd":9,"last update":"Wed May 8 22:07:04 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.70899999999999996,"15min":0.71899999999999997},"min":{"1min":0.54400000000000004,"5min":0.501,"15min":0.501},"max":{"1min":0.86699999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80100000000000005,"15min":0.79600000000000004},"min":{"1min":0.71199999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0860000000000001,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.59899999999999998}]},{"osd":10,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.76200000000000001,"15min":0.76400000000000001},"min":{"1min":0.57199999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0389999999999999,"5min":1.179,"15min":1.179},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.56699999999999995,"5min":0.435,"15min":0.435},"max":{"1min":0.94899999999999995,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.64100000000000001}]},{"osd":11,"last update":"Wed May 8 22:07:14 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.83599999999999997,"15min":0.84099999999999997},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.78000000000000003,"15min":0.78300000000000003},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.98699999999999999,"5min":0.996,"15min":0.996},"last":0.69299999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":983040,"data_stored":495328,"data_compressed":26516,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:06:29 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.60699999999999998,"15min":0.55800000000000005},"min":{"1min":0.45200000000000001,"5min":0.31,"15min":0.31},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.58399999999999996,"15min":0.53400000000000003},"min":{"1min":0.41899999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.55700000000000005}]},{"osd":2,"last update":"Wed May 8 22:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.64100000000000001,"15min":0.56599999999999995},"min":{"1min":0.45100000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61699999999999999,"15min":0.55900000000000005},"min":{"1min":0.53900000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86399999999999999,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.625}]},{"osd":3,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53600000000000003,"15min":0.53100000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66800000000000004,"15min":0.66400000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.78300000000000003}]},{"osd":4,"last update":"Wed May 8 22:06:29 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.59899999999999998,"15min":0.55800000000000005},"min":{"1min":0.52400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95999999999999996,"5min":1.218,"15min":1.218},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.70399999999999996,"15min":0.69899999999999995},"min":{"1min":0.56699999999999995,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94599999999999995,"5min":2.984,"15min":2.984},"last":0.84799999999999998}]},{"osd":5,"last update":"Wed May 8 22:06:46 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.70599999999999996,"15min":0.65600000000000003},"min":{"1min":0.501,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0760000000000001,"5min":1.129,"15min":1.129},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68400000000000005,"15min":0.66600000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.69499999999999995}]},{"osd":6,"last update":"Wed May 8 22:07:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.625,"15min":0.58899999999999997},"min":{"1min":0.40300000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90300000000000002,"5min":1.254,"15min":1.254},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70999999999999996,"15min":0.67700000000000005},"min":{"1min":0.443,"5min":0.375,"15min":0.375},"max":{"1min":1.0089999999999999,"5min":1.208,"15min":1.208},"last":0.92500000000000004}]},{"osd":7,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.72699999999999998}]},{"osd":8,"last update":"Wed May 8 22:06:41 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.83199999999999996,"15min":0.82899999999999996},"min":{"1min":0.65000000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1080000000000001,"5min":1.427,"15min":1.427},"last":1.024},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.80200000000000005,"15min":0.80500000000000005},"min":{"1min":0.64800000000000002,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.76200000000000001}]},{"osd":9,"last update":"Wed May 8 22:07:07 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78900000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.875,"15min":0.89200000000000002},"min":{"1min":0.47999999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1379999999999999,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.089}]},{"osd":10,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.61599999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1339999999999999,"5min":1.2,"15min":1.2},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.876,"15min":0.879},"min":{"1min":0.69599999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.77200000000000002}]},{"osd":11,"last update":"Wed May 8 22:06:41 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.88500000000000001,"15min":0.874},"min":{"1min":0.76100000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":1.056},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.86899999999999999,"15min":0.86099999999999999},"min":{"1min":0.70499999999999996,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.80700000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":1040384,"data_stored":561278,"data_compressed":26516,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.63100000000000001,"15min":0.55100000000000005},"min":{"1min":0.44900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":6.4640000000000004,"5min":6.4640000000000004,"15min":6.4640000000000004},"last":0.503},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.54300000000000004,"15min":0.51500000000000001},"min":{"1min":0.46600000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.47299999999999998}]},{"osd":1,"last update":"Wed May 8 22:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.56599999999999995,"15min":0.53800000000000003},"min":{"1min":0.42699999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.60599999999999998,"15min":0.56499999999999995},"min":{"1min":0.38400000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.81200000000000006}]},{"osd":3,"last update":"Wed May 8 22:07:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.59899999999999998,"15min":0.56699999999999995},"min":{"1min":0.50700000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.90100000000000002,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59699999999999998,"15min":0.56899999999999995},"min":{"1min":0.44900000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67200000000000004}]},{"osd":4,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.755,"15min":0.76900000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.70199999999999996,"15min":0.71199999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.77200000000000002}]},{"osd":5,"last update":"Wed May 8 22:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.51100000000000001,"15min":0.45800000000000002},"min":{"1min":0.41099999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80400000000000005,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.496,"15min":0.45800000000000002},"min":{"1min":0.435,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.52900000000000003}]},{"osd":6,"last update":"Wed May 8 22:07:09 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.53400000000000003,"5min":0.439,"15min":0.439},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.70999999999999996,"15min":0.68200000000000005},"min":{"1min":0.54800000000000004,"5min":0.442,"15min":0.442},"max":{"1min":0.98899999999999999,"5min":0.998,"15min":0.998},"last":0.63700000000000001}]},{"osd":7,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.875},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.84599999999999997}]},{"osd":8,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.63500000000000001,"15min":0.59299999999999997},"min":{"1min":0.505,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.626,"15min":0.58399999999999996},"min":{"1min":0.54100000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90500000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.78900000000000003}]},{"osd":9,"last update":"Wed May 8 22:07:05 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.77300000000000002,"15min":0.72699999999999998},"min":{"1min":0.73099999999999998,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.024,"5min":1.101,"15min":1.101},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.76100000000000001,"15min":0.71899999999999997},"min":{"1min":0.71099999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.996,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.80200000000000005}]},{"osd":10,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.73899999999999999,"15min":0.71299999999999997},"min":{"1min":0.65700000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.72199999999999998,"15min":0.70399999999999996},"min":{"1min":0.66800000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.83699999999999997}]},{"osd":11,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.86199999999999999,"15min":0.871},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.81200000000000006,"15min":0.82199999999999995},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.95199999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.754}]}]},{"osd":3,"up_from":24,"seq":103079215169,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":1019223,"data_compressed":51818,"data_compressed_allocated":69632,"data_compressed_original":589824,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:07:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.49099999999999999,"15min":0.41699999999999998},"min":{"1min":0.33300000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.89000000000000001,"5min":3.2530000000000001,"15min":3.2530000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.52800000000000002,"15min":0.436},"min":{"1min":0.30299999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.1619999999999999,"5min":3.4460000000000002,"15min":3.4460000000000002},"last":0.58199999999999996}]},{"osd":1,"last update":"Wed May 8 22:07:14 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.68100000000000005,"15min":0.58499999999999996},"min":{"1min":0.41399999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79300000000000004,"5min":3.7090000000000001,"15min":3.7090000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66200000000000003,"15min":0.56200000000000006},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.3899999999999999,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":0.55800000000000005}]},{"osd":2,"last update":"Wed May 8 22:07:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71099999999999997,"15min":0.621},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.258,"5min":3.577,"15min":3.577},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.71499999999999997,"15min":0.63500000000000001},"min":{"1min":0.45600000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97299999999999998,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.45800000000000002}]},{"osd":4,"last update":"Wed May 8 22:06:25 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.67700000000000005,"15min":0.61299999999999999},"min":{"1min":0.46600000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":3.8109999999999999,"5min":3.8109999999999999,"15min":3.8109999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.72199999999999998,"15min":0.64000000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.847,"5min":3.847,"15min":3.847},"last":0.78900000000000003}]},{"osd":5,"last update":"Wed May 8 22:06:44 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.73799999999999999,"15min":0.66500000000000004},"min":{"1min":0.47999999999999998,"5min":0.433,"15min":0.433},"max":{"1min":3.9260000000000002,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.70699999999999996,"15min":0.63},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.54,"5min":3.54,"15min":3.54},"last":0.628}]},{"osd":6,"last update":"Wed May 8 22:07:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69599999999999995,"15min":0.63700000000000001},"min":{"1min":0.44600000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.157,"5min":3.653,"15min":3.653},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72499999999999998,"15min":0.65000000000000002},"min":{"1min":0.47599999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96299999999999997,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.495}]},{"osd":7,"last update":"Wed May 8 22:06:25 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.73999999999999999,"15min":0.72299999999999998},"min":{"1min":0.55500000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.86399999999999999,"15min":0.84699999999999998},"min":{"1min":0.57599999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":3.887,"5min":3.887,"15min":3.887},"last":0.71799999999999997}]},{"osd":8,"last update":"Wed May 8 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.53100000000000003}]},{"osd":9,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.88600000000000001,"15min":0.87},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.179,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.873},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.80400000000000005,"15min":0.79600000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.454,"5min":3.3940000000000001,"15min":3.3940000000000001},"last":0.80800000000000005}]},{"osd":10,"last update":"Wed May 8 22:07:22 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.84699999999999998,"15min":0.83699999999999997},"min":{"1min":0.60599999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4850000000000001,"5min":3.298,"15min":3.298},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.871,"15min":0.85999999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1240000000000001,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":0.70099999999999996}]},{"osd":11,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.99399999999999999,"15min":1.032},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.204,"5min":3.6150000000000002,"15min":3.6150000000000002},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.82999999999999996,"5min":1.034,"15min":1.0680000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.014,"5min":3.9950000000000001,"15min":3.9950000000000001},"last":0.84099999999999997}]}]},{"osd":4,"up_from":31,"seq":133143986239,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":500106,"data_compressed":26516,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.44700000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.51900000000000002,"15min":0.50900000000000001},"min":{"1min":0.45000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.629}]},{"osd":1,"last update":"Wed May 8 22:06:28 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.39100000000000001,"15min":0.35699999999999998},"min":{"1min":0.33500000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.40000000000000002,"15min":0.36199999999999999},"min":{"1min":0.36299999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.24299999999999999}]},{"osd":2,"last update":"Wed May 8 22:06:28 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.53600000000000003,"15min":0.51100000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82099999999999995,"5min":0.88,"15min":0.88},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54800000000000004,"15min":0.51400000000000001},"min":{"1min":0.436,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.52200000000000002}]},{"osd":3,"last update":"Wed May 8 22:06:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.56699999999999995},"min":{"1min":0.46000000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.70499999999999996,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57299999999999995,"15min":0.55500000000000005},"min":{"1min":0.439,"5min":0.377,"15min":0.377},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.51200000000000001}]},{"osd":5,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57399999999999995,"15min":0.54000000000000004},"min":{"1min":0.49299999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77400000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59399999999999997,"15min":0.55700000000000005},"min":{"1min":0.48399999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.80200000000000005,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.53800000000000003}]},{"osd":6,"last update":"Wed May 8 22:07:07 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.65600000000000003,"15min":0.63800000000000001},"min":{"1min":0.49299999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.75600000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.628,"15min":0.61599999999999999},"min":{"1min":0.45700000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.38900000000000001}]},{"osd":7,"last update":"Wed May 8 22:06:28 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57999999999999996,"15min":0.58599999999999997},"min":{"1min":0.45900000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.46200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.503}]},{"osd":8,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.69199999999999995,"15min":0.68799999999999994},"min":{"1min":0.58699999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93000000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.60199999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.874,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.56899999999999995}]},{"osd":9,"last update":"Wed May 8 22:07:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69499999999999995,"15min":0.69799999999999995},"min":{"1min":0.53500000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.749,"15min":0.755},"min":{"1min":0.61699999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.48699999999999999}]},{"osd":10,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.73599999999999999,"15min":0.73999999999999999},"min":{"1min":0.55800000000000005,"5min":0.434,"15min":0.434},"max":{"1min":0.84799999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.627,"15min":0.625},"min":{"1min":0.50600000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.34300000000000003}]},{"osd":11,"last update":"Wed May 8 22:07:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.75,"15min":0.747},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.90200000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.747,"15min":0.747},"min":{"1min":0.60699999999999998,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":0.93400000000000005,"5min":1.052,"15min":1.052},"last":0.54800000000000004}]}]},{"osd":5,"up_from":36,"seq":154618822715,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":1060864,"data_stored":1222542,"data_compressed":68668,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.41799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.55000000000000004,"15min":0.53800000000000003},"min":{"1min":0.40000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70699999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.40600000000000003}]},{"osd":1,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.56599999999999995,"15min":0.54000000000000004},"min":{"1min":0.28799999999999998,"5min":0.248,"15min":0.248},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.55500000000000005,"15min":0.52600000000000002},"min":{"1min":0.34399999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.66800000000000004}]},{"osd":2,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.40200000000000002,"15min":0.376},"min":{"1min":0.33200000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.41499999999999998,"15min":0.39200000000000002},"min":{"1min":0.35499999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67100000000000004,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.38}]},{"osd":3,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.46500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.78100000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.58699999999999997,"15min":0.55900000000000005},"min":{"1min":0.51000000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.62}]},{"osd":4,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.378,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.58199999999999996,"15min":0.56100000000000005},"min":{"1min":0.48399999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.65000000000000002}]},{"osd":6,"last update":"Wed May 8 22:07:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61099999999999999,"15min":0.59299999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61899999999999999,"15min":0.57299999999999995},"min":{"1min":0.505,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48899999999999999}]},{"osd":7,"last update":"Wed May 8 22:06:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.60899999999999999,"15min":0.57699999999999996},"min":{"1min":0.54500000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.61799999999999999,"15min":0.58699999999999997},"min":{"1min":0.51100000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.70099999999999996}]},{"osd":8,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.54000000000000004,"15min":0.50600000000000001},"min":{"1min":0.44600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.624,"15min":0.58199999999999996},"min":{"1min":0.5,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.78300000000000003}]},{"osd":9,"last update":"Wed May 8 22:07:03 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.73299999999999998,"15min":0.70899999999999996},"min":{"1min":0.57899999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.749},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.72199999999999998,"15min":0.69799999999999995},"min":{"1min":0.623,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.63400000000000001}]},{"osd":10,"last update":"Wed May 8 22:06:25 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.752,"15min":0.74299999999999999},"min":{"1min":0.65900000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.74199999999999999,"15min":0.73299999999999998},"min":{"1min":0.66100000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.98599999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.72899999999999998}]},{"osd":11,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":69664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133955,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":203391,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:07:25.390 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:07:25.478 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:25 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/4061176932' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:07:25.478 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:25 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/756939378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:07:25.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:25 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/4061176932' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:07:25.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:25 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/756939378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:07:25.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:25 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/4061176932' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:07:25.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:25 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/756939378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:07:25.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.614+0000 7fe936600700 1 -- 172.21.15.131:0/1737691110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe930101050 msgr2=0x7fe930101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.614+0000 7fe936600700 1 --2- 172.21.15.131:0/1737691110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe930101050 0x7fe930101430 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe920009990 tx=0x7fe920009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.614+0000 7fe936600700 1 -- 172.21.15.131:0/1737691110 shutdown_connections 2024-05-08T22:07:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.614+0000 7fe936600700 1 --2- 172.21.15.131:0/1737691110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe930106170 0x7fe93010dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.614+0000 7fe936600700 1 --2- 172.21.15.131:0/1737691110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe930101050 0x7fe930101430 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.614+0000 7fe936600700 1 --2- 172.21.15.131:0/1737691110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe930101a00 0x7fe930105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.614+0000 7fe936600700 1 -- 172.21.15.131:0/1737691110 >> 172.21.15.131:0/1737691110 conn(0x7fe9300fc8f0 msgr2=0x7fe9300fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.614+0000 7fe936600700 1 -- 172.21.15.131:0/1737691110 shutdown_connections 2024-05-08T22:07:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.614+0000 7fe936600700 1 -- 172.21.15.131:0/1737691110 wait complete. 2024-05-08T22:07:25.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.615+0000 7fe936600700 1 Processor -- start 2024-05-08T22:07:25.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.616+0000 7fe936600700 1 -- start start 2024-05-08T22:07:25.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.616+0000 7fe936600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe930101050 0x7fe9300732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:25.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.616+0000 7fe936600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe930106170 0x7fe9300737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:25.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.616+0000 7fe936600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe930073d20 0x7fe9301a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:25.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.616+0000 7fe936600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9301a5b20 con 0x7fe930106170 2024-05-08T22:07:25.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.616+0000 7fe936600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9301a5c60 con 0x7fe930101050 2024-05-08T22:07:25.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.616+0000 7fe936600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9301a5e20 con 0x7fe930073d20 2024-05-08T22:07:25.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.617+0000 7fe92ea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe930106170 0x7fe9300737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:25.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.617+0000 7fe92f400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe930101050 0x7fe9300732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.617+0000 7fe92f400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe930101050 0x7fe9300732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:43584/0 (socket says 172.21.15.131:43584) 2024-05-08T22:07:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.617+0000 7fe92f400700 1 -- 172.21.15.131:0/1054289282 learned_addr learned my addr 172.21.15.131:0/1054289282 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.617+0000 7fe92ea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe930106170 0x7fe9300737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50720/0 (socket says 172.21.15.131:50720) 2024-05-08T22:07:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.617+0000 7fe92fe00700 1 --2- 172.21.15.131:0/1054289282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe930073d20 0x7fe9301a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.617+0000 7fe92f400700 1 -- 172.21.15.131:0/1054289282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe930073d20 msgr2=0x7fe9301a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.617+0000 7fe92f400700 1 --2- 172.21.15.131:0/1054289282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe930073d20 0x7fe9301a5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.617+0000 7fe92f400700 1 -- 172.21.15.131:0/1054289282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe930106170 msgr2=0x7fe9300737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.617+0000 7fe92f400700 1 --2- 172.21.15.131:0/1054289282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe930106170 0x7fe9300737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.617+0000 7fe92f400700 1 -- 172.21.15.131:0/1054289282 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9200095f0 con 0x7fe930101050 2024-05-08T22:07:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.617+0000 7fe92f400700 1 --2- 172.21.15.131:0/1054289282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe930101050 0x7fe9300732a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe920005910 tx=0x7fe920004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:25.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.617+0000 7fe927e00700 1 -- 172.21.15.131:0/1054289282 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92001d070 con 0x7fe930101050 2024-05-08T22:07:25.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.618+0000 7fe936600700 1 -- 172.21.15.131:0/1054289282 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9301a60a0 con 0x7fe930101050 2024-05-08T22:07:25.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.618+0000 7fe927e00700 1 -- 172.21.15.131:0/1054289282 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe920022470 con 0x7fe930101050 2024-05-08T22:07:25.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.618+0000 7fe927e00700 1 -- 172.21.15.131:0/1054289282 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe920017650 con 0x7fe930101050 2024-05-08T22:07:25.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.618+0000 7fe936600700 1 -- 172.21.15.131:0/1054289282 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9301a6520 con 0x7fe930101050 2024-05-08T22:07:25.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.619+0000 7fe936600700 1 -- 172.21.15.131:0/1054289282 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe930109390 con 0x7fe930101050 2024-05-08T22:07:25.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.621+0000 7fe927e00700 1 -- 172.21.15.131:0/1054289282 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7fe9200225e0 con 0x7fe930101050 2024-05-08T22:07:25.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.622+0000 7fe927e00700 1 --2- 172.21.15.131:0/1054289282 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe9140a13b0 0x7fe9140a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:25.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.622+0000 7fe927e00700 1 -- 172.21.15.131:0/1054289282 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fe9200ce9a0 con 0x7fe930101050 2024-05-08T22:07:25.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.622+0000 7fe92ea00700 1 --2- 172.21.15.131:0/1054289282 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe9140a13b0 0x7fe9140a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:25.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.623+0000 7fe92ea00700 1 --2- 172.21.15.131:0/1054289282 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe9140a13b0 0x7fe9140a3870 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe930074980 tx=0x7fe91c009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:25.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.624+0000 7fe927e00700 1 -- 172.21.15.131:0/1054289282 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe920097840 con 0x7fe930101050 2024-05-08T22:07:25.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.828+0000 7fe936600700 1 -- 172.21.15.131:0/1054289282 --> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe930102520 con 0x7fe9140a13b0 2024-05-08T22:07:25.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.841+0000 7fe927e00700 1 -- 172.21.15.131:0/1054289282 <== mgr.15366 v2:172.21.15.131:6800/1157588501 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426525 (secure 0 0 0) 0x7fe930102520 con 0x7fe9140a13b0 2024-05-08T22:07:25.849 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:07:25.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.846+0000 7fe936600700 1 -- 172.21.15.131:0/1054289282 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe9140a13b0 msgr2=0x7fe9140a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:25.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.846+0000 7fe936600700 1 --2- 172.21.15.131:0/1054289282 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe9140a13b0 0x7fe9140a3870 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe930074980 tx=0x7fe91c009260 comp rx=0 tx=0).stop 2024-05-08T22:07:25.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.846+0000 7fe936600700 1 -- 172.21.15.131:0/1054289282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe930101050 msgr2=0x7fe9300732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:25.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.847+0000 7fe936600700 1 --2- 172.21.15.131:0/1054289282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe930101050 0x7fe9300732a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe920005910 tx=0x7fe920004730 comp rx=0 tx=0).stop 2024-05-08T22:07:25.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.847+0000 7fe936600700 1 -- 172.21.15.131:0/1054289282 shutdown_connections 2024-05-08T22:07:25.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.847+0000 7fe936600700 1 --2- 172.21.15.131:0/1054289282 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe9140a13b0 0x7fe9140a3870 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:25.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.847+0000 7fe936600700 1 --2- 172.21.15.131:0/1054289282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe930073d20 0x7fe9301a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:25.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.847+0000 7fe936600700 1 --2- 172.21.15.131:0/1054289282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe930101050 0x7fe9300732a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:25.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.847+0000 7fe936600700 1 --2- 172.21.15.131:0/1054289282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe930106170 0x7fe9300737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:25.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.847+0000 7fe936600700 1 -- 172.21.15.131:0/1054289282 >> 172.21.15.131:0/1054289282 conn(0x7fe9300fc8f0 msgr2=0x7fe9300fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:25.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.847+0000 7fe936600700 1 -- 172.21.15.131:0/1054289282 shutdown_connections 2024-05-08T22:07:25.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:25.847+0000 7fe936600700 1 -- 172.21.15.131:0/1054289282 wait complete. 2024-05-08T22:07:25.853 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-08T22:07:25.867 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-05-08T22:07:25.114979+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190120,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":984,"num_read_kb":813,"num_write":486,"num_write_kb":4381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181430,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":875,"ondisk_log_size":875,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373032,"kb_used_data":11944,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124568792,"statfs":{"total":1151940427776,"available":1151558443008,"internally_reserved":0,"allocated":12230656,"data_stored":9524439,"data_compressed":502086,"data_compressed_allocated":675840,"data_compressed_original":4202496,"omap_allocated":19131,"internal_metadata":369603909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3699,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":134,"num_read_kb":229,"num_write":28,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.919574"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.174991+0000","last_change":"2024-05-08T22:05:14.061628+0000","last_active":"2024-05-08T22:07:17.174991+0000","last_peered":"2024-05-08T22:07:17.174991+0000","last_clean":"2024-05-08T22:07:17.174991+0000","last_became_active":"2024-05-08T22:05:14.060986+0000","last_became_peered":"2024-05-08T22:05:14.060986+0000","last_unstale":"2024-05-08T22:07:17.174991+0000","last_undegraded":"2024-05-08T22:07:17.174991+0000","last_fullsized":"2024-05-08T22:07:17.174991+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:51.643118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.174788+0000","last_change":"2024-05-08T22:05:14.073300+0000","last_active":"2024-05-08T22:07:17.174788+0000","last_peered":"2024-05-08T22:07:17.174788+0000","last_clean":"2024-05-08T22:07:17.174788+0000","last_became_active":"2024-05-08T22:05:14.069058+0000","last_became_peered":"2024-05-08T22:05:14.069058+0000","last_unstale":"2024-05-08T22:07:17.174788+0000","last_undegraded":"2024-05-08T22:07:17.174788+0000","last_fullsized":"2024-05-08T22:07:17.174788+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:15.834956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.175346+0000","last_change":"2024-05-08T22:05:14.071676+0000","last_active":"2024-05-08T22:07:17.175346+0000","last_peered":"2024-05-08T22:07:17.175346+0000","last_clean":"2024-05-08T22:07:17.175346+0000","last_became_active":"2024-05-08T22:05:14.058345+0000","last_became_peered":"2024-05-08T22:05:14.058345+0000","last_unstale":"2024-05-08T22:07:17.175346+0000","last_undegraded":"2024-05-08T22:07:17.175346+0000","last_fullsized":"2024-05-08T22:07:17.175346+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:09.239058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.155221+0000","last_change":"2024-05-08T22:05:14.067170+0000","last_active":"2024-05-08T22:07:17.155221+0000","last_peered":"2024-05-08T22:07:17.155221+0000","last_clean":"2024-05-08T22:07:17.155221+0000","last_became_active":"2024-05-08T22:05:14.066850+0000","last_became_peered":"2024-05-08T22:05:14.066850+0000","last_unstale":"2024-05-08T22:07:17.155221+0000","last_undegraded":"2024-05-08T22:07:17.155221+0000","last_fullsized":"2024-05-08T22:07:17.155221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:28.439786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.155731+0000","last_change":"2024-05-08T22:05:14.065177+0000","last_active":"2024-05-08T22:07:17.155731+0000","last_peered":"2024-05-08T22:07:17.155731+0000","last_clean":"2024-05-08T22:07:17.155731+0000","last_became_active":"2024-05-08T22:05:14.064833+0000","last_became_peered":"2024-05-08T22:05:14.064833+0000","last_unstale":"2024-05-08T22:07:17.155731+0000","last_undegraded":"2024-05-08T22:07:17.155731+0000","last_fullsized":"2024-05-08T22:07:17.155731+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:11.764154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.179822+0000","last_change":"2024-05-08T22:05:14.061610+0000","last_active":"2024-05-08T22:07:17.179822+0000","last_peered":"2024-05-08T22:07:17.179822+0000","last_clean":"2024-05-08T22:07:17.179822+0000","last_became_active":"2024-05-08T22:05:14.061277+0000","last_became_peered":"2024-05-08T22:05:14.061277+0000","last_unstale":"2024-05-08T22:07:17.179822+0000","last_undegraded":"2024-05-08T22:07:17.179822+0000","last_fullsized":"2024-05-08T22:07:17.179822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:19.154980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.176639+0000","last_change":"2024-05-08T22:05:14.063941+0000","last_active":"2024-05-08T22:07:17.176639+0000","last_peered":"2024-05-08T22:07:17.176639+0000","last_clean":"2024-05-08T22:07:17.176639+0000","last_became_active":"2024-05-08T22:05:14.063579+0000","last_became_peered":"2024-05-08T22:05:14.063579+0000","last_unstale":"2024-05-08T22:07:17.176639+0000","last_undegraded":"2024-05-08T22:07:17.176639+0000","last_fullsized":"2024-05-08T22:07:17.176639+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:27.679067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.179515+0000","last_change":"2024-05-08T22:05:14.060366+0000","last_active":"2024-05-08T22:07:17.179515+0000","last_peered":"2024-05-08T22:07:17.179515+0000","last_clean":"2024-05-08T22:07:17.179515+0000","last_became_active":"2024-05-08T22:05:14.059818+0000","last_became_peered":"2024-05-08T22:05:14.059818+0000","last_unstale":"2024-05-08T22:07:17.179515+0000","last_undegraded":"2024-05-08T22:07:17.179515+0000","last_fullsized":"2024-05-08T22:07:17.179515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:53.118508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.154256+0000","last_change":"2024-05-08T22:05:14.057185+0000","last_active":"2024-05-08T22:07:17.154256+0000","last_peered":"2024-05-08T22:07:17.154256+0000","last_clean":"2024-05-08T22:07:17.154256+0000","last_became_active":"2024-05-08T22:05:14.056889+0000","last_became_peered":"2024-05-08T22:05:14.056889+0000","last_unstale":"2024-05-08T22:07:17.154256+0000","last_undegraded":"2024-05-08T22:07:17.154256+0000","last_fullsized":"2024-05-08T22:07:17.154256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:35.372705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.146313+0000","last_change":"2024-05-08T22:05:14.062282+0000","last_active":"2024-05-08T22:07:17.146313+0000","last_peered":"2024-05-08T22:07:17.146313+0000","last_clean":"2024-05-08T22:07:17.146313+0000","last_became_active":"2024-05-08T22:05:14.061574+0000","last_became_peered":"2024-05-08T22:05:14.061574+0000","last_unstale":"2024-05-08T22:07:17.146313+0000","last_undegraded":"2024-05-08T22:07:17.146313+0000","last_fullsized":"2024-05-08T22:07:17.146313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:16:32.873590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.153206+0000","last_change":"2024-05-08T22:05:14.061267+0000","last_active":"2024-05-08T22:07:17.153206+0000","last_peered":"2024-05-08T22:07:17.153206+0000","last_clean":"2024-05-08T22:07:17.153206+0000","last_became_active":"2024-05-08T22:05:14.060943+0000","last_became_peered":"2024-05-08T22:05:14.060943+0000","last_unstale":"2024-05-08T22:07:17.153206+0000","last_undegraded":"2024-05-08T22:07:17.153206+0000","last_fullsized":"2024-05-08T22:07:17.153206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:03.831138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.155172+0000","last_change":"2024-05-08T22:05:14.067136+0000","last_active":"2024-05-08T22:07:17.155172+0000","last_peered":"2024-05-08T22:07:17.155172+0000","last_clean":"2024-05-08T22:07:17.155172+0000","last_became_active":"2024-05-08T22:05:14.066810+0000","last_became_peered":"2024-05-08T22:05:14.066810+0000","last_unstale":"2024-05-08T22:07:17.155172+0000","last_undegraded":"2024-05-08T22:07:17.155172+0000","last_fullsized":"2024-05-08T22:07:17.155172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:43.001024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.157400+0000","last_change":"2024-05-08T22:05:14.062119+0000","last_active":"2024-05-08T22:07:17.157400+0000","last_peered":"2024-05-08T22:07:17.157400+0000","last_clean":"2024-05-08T22:07:17.157400+0000","last_became_active":"2024-05-08T22:05:14.058977+0000","last_became_peered":"2024-05-08T22:05:14.058977+0000","last_unstale":"2024-05-08T22:07:17.157400+0000","last_undegraded":"2024-05-08T22:07:17.157400+0000","last_fullsized":"2024-05-08T22:07:17.157400+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:03.657985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.163070+0000","last_change":"2024-05-08T22:05:14.067363+0000","last_active":"2024-05-08T22:07:17.163070+0000","last_peered":"2024-05-08T22:07:17.163070+0000","last_clean":"2024-05-08T22:07:17.163070+0000","last_became_active":"2024-05-08T22:05:14.066119+0000","last_became_peered":"2024-05-08T22:05:14.066119+0000","last_unstale":"2024-05-08T22:07:17.163070+0000","last_undegraded":"2024-05-08T22:07:17.163070+0000","last_fullsized":"2024-05-08T22:07:17.163070+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:43.426750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.174135+0000","last_change":"2024-05-08T22:05:14.073925+0000","last_active":"2024-05-08T22:07:17.174135+0000","last_peered":"2024-05-08T22:07:17.174135+0000","last_clean":"2024-05-08T22:07:17.174135+0000","last_became_active":"2024-05-08T22:05:14.072287+0000","last_became_peered":"2024-05-08T22:05:14.072287+0000","last_unstale":"2024-05-08T22:07:17.174135+0000","last_undegraded":"2024-05-08T22:07:17.174135+0000","last_fullsized":"2024-05-08T22:07:17.174135+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:13.537938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.172678+0000","last_change":"2024-05-08T22:05:14.057274+0000","last_active":"2024-05-08T22:07:17.172678+0000","last_peered":"2024-05-08T22:07:17.172678+0000","last_clean":"2024-05-08T22:07:17.172678+0000","last_became_active":"2024-05-08T22:05:14.056989+0000","last_became_peered":"2024-05-08T22:05:14.056989+0000","last_unstale":"2024-05-08T22:07:17.172678+0000","last_undegraded":"2024-05-08T22:07:17.172678+0000","last_fullsized":"2024-05-08T22:07:17.172678+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:32.316406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.174918+0000","last_change":"2024-05-08T22:05:14.069263+0000","last_active":"2024-05-08T22:07:17.174918+0000","last_peered":"2024-05-08T22:07:17.174918+0000","last_clean":"2024-05-08T22:07:17.174918+0000","last_became_active":"2024-05-08T22:05:14.068960+0000","last_became_peered":"2024-05-08T22:05:14.068960+0000","last_unstale":"2024-05-08T22:07:17.174918+0000","last_undegraded":"2024-05-08T22:07:17.174918+0000","last_fullsized":"2024-05-08T22:07:17.174918+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:59:42.829110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.175370+0000","last_change":"2024-05-08T22:05:14.049393+0000","last_active":"2024-05-08T22:07:17.175370+0000","last_peered":"2024-05-08T22:07:17.175370+0000","last_clean":"2024-05-08T22:07:17.175370+0000","last_became_active":"2024-05-08T22:05:14.049094+0000","last_became_peered":"2024-05-08T22:05:14.049094+0000","last_unstale":"2024-05-08T22:07:17.175370+0000","last_undegraded":"2024-05-08T22:07:17.175370+0000","last_fullsized":"2024-05-08T22:07:17.175370+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:42.106568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.176257+0000","last_change":"2024-05-08T22:05:14.055436+0000","last_active":"2024-05-08T22:07:17.176257+0000","last_peered":"2024-05-08T22:07:17.176257+0000","last_clean":"2024-05-08T22:07:17.176257+0000","last_became_active":"2024-05-08T22:05:14.055100+0000","last_became_peered":"2024-05-08T22:05:14.055100+0000","last_unstale":"2024-05-08T22:07:17.176257+0000","last_undegraded":"2024-05-08T22:07:17.176257+0000","last_fullsized":"2024-05-08T22:07:17.176257+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:52.833780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.447942+0000","last_change":"2024-05-08T22:05:18.532086+0000","last_active":"2024-05-08T22:06:42.447942+0000","last_peered":"2024-05-08T22:06:42.447942+0000","last_clean":"2024-05-08T22:06:42.447942+0000","last_became_active":"2024-05-08T22:05:18.531485+0000","last_became_peered":"2024-05-08T22:05:18.531485+0000","last_unstale":"2024-05-08T22:06:42.447942+0000","last_undegraded":"2024-05-08T22:06:42.447942+0000","last_fullsized":"2024-05-08T22:06:42.447942+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:40:36.050367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.103820+0000","last_change":"2024-05-08T22:05:13.549611+0000","last_active":"2024-05-08T22:07:17.103820+0000","last_peered":"2024-05-08T22:07:17.103820+0000","last_clean":"2024-05-08T22:07:17.103820+0000","last_became_active":"2024-05-08T22:05:13.549140+0000","last_became_peered":"2024-05-08T22:05:13.549140+0000","last_unstale":"2024-05-08T22:07:17.103820+0000","last_undegraded":"2024-05-08T22:07:17.103820+0000","last_fullsized":"2024-05-08T22:07:17.103820+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:54.785123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.166838+0000","last_change":"2024-05-08T22:06:42.448887+0000","last_active":"2024-05-08T22:07:17.166838+0000","last_peered":"2024-05-08T22:07:17.166838+0000","last_clean":"2024-05-08T22:07:17.166838+0000","last_became_active":"2024-05-08T22:06:42.448391+0000","last_became_peered":"2024-05-08T22:06:42.448391+0000","last_unstale":"2024-05-08T22:07:17.166838+0000","last_undegraded":"2024-05-08T22:07:17.166838+0000","last_fullsized":"2024-05-08T22:07:17.166838+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:51.859658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976371+0000","last_change":"2024-05-08T22:05:42.147633+0000","last_active":"2024-05-08T22:06:41.976371+0000","last_peered":"2024-05-08T22:06:41.976371+0000","last_clean":"2024-05-08T22:06:41.976371+0000","last_became_active":"2024-05-08T22:05:42.147284+0000","last_became_peered":"2024-05-08T22:05:42.147284+0000","last_unstale":"2024-05-08T22:06:41.976371+0000","last_undegraded":"2024-05-08T22:06:41.976371+0000","last_fullsized":"2024-05-08T22:06:41.976371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:32.864756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.139965+0000","last_change":"2024-05-08T22:05:13.551351+0000","last_active":"2024-05-08T22:07:17.139965+0000","last_peered":"2024-05-08T22:07:17.139965+0000","last_clean":"2024-05-08T22:07:17.139965+0000","last_became_active":"2024-05-08T22:05:13.551014+0000","last_became_peered":"2024-05-08T22:05:13.551014+0000","last_unstale":"2024-05-08T22:07:17.139965+0000","last_undegraded":"2024-05-08T22:07:17.139965+0000","last_fullsized":"2024-05-08T22:07:17.139965+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:07:22.405251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.180951+0000","last_change":"2024-05-08T22:05:14.067270+0000","last_active":"2024-05-08T22:07:17.180951+0000","last_peered":"2024-05-08T22:07:17.180951+0000","last_clean":"2024-05-08T22:07:17.180951+0000","last_became_active":"2024-05-08T22:05:14.065496+0000","last_became_peered":"2024-05-08T22:05:14.065496+0000","last_unstale":"2024-05-08T22:07:17.180951+0000","last_undegraded":"2024-05-08T22:07:17.180951+0000","last_fullsized":"2024-05-08T22:07:17.180951+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:07.087880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976487+0000","last_change":"2024-05-08T22:05:18.526557+0000","last_active":"2024-05-08T22:06:41.976487+0000","last_peered":"2024-05-08T22:06:41.976487+0000","last_clean":"2024-05-08T22:06:41.976487+0000","last_became_active":"2024-05-08T22:05:18.526129+0000","last_became_peered":"2024-05-08T22:05:18.526129+0000","last_unstale":"2024-05-08T22:06:41.976487+0000","last_undegraded":"2024-05-08T22:06:41.976487+0000","last_fullsized":"2024-05-08T22:06:41.976487+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:36.376579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.376514+0000","last_change":"2024-05-08T22:05:41.835074+0000","last_active":"2024-05-08T22:06:42.376514+0000","last_peered":"2024-05-08T22:06:42.376514+0000","last_clean":"2024-05-08T22:06:42.376514+0000","last_became_active":"2024-05-08T22:05:41.829567+0000","last_became_peered":"2024-05-08T22:05:41.829567+0000","last_unstale":"2024-05-08T22:06:42.376514+0000","last_undegraded":"2024-05-08T22:06:42.376514+0000","last_fullsized":"2024-05-08T22:06:42.376514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:15.261352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.151662+0000","last_change":"2024-05-08T22:05:14.060089+0000","last_active":"2024-05-08T22:07:17.151662+0000","last_peered":"2024-05-08T22:07:17.151662+0000","last_clean":"2024-05-08T22:07:17.151662+0000","last_became_active":"2024-05-08T22:05:14.057620+0000","last_became_peered":"2024-05-08T22:05:14.057620+0000","last_unstale":"2024-05-08T22:07:17.151662+0000","last_undegraded":"2024-05-08T22:07:17.151662+0000","last_fullsized":"2024-05-08T22:07:17.151662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:29:33.127510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976369+0000","last_change":"2024-05-08T22:05:18.529746+0000","last_active":"2024-05-08T22:06:41.976369+0000","last_peered":"2024-05-08T22:06:41.976369+0000","last_clean":"2024-05-08T22:06:41.976369+0000","last_became_active":"2024-05-08T22:05:18.529097+0000","last_became_peered":"2024-05-08T22:05:18.529097+0000","last_unstale":"2024-05-08T22:06:41.976369+0000","last_undegraded":"2024-05-08T22:06:41.976369+0000","last_fullsized":"2024-05-08T22:06:41.976369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:59:36.249787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.376187+0000","last_change":"2024-05-08T22:05:41.837103+0000","last_active":"2024-05-08T22:06:42.376187+0000","last_peered":"2024-05-08T22:06:42.376187+0000","last_clean":"2024-05-08T22:06:42.376187+0000","last_became_active":"2024-05-08T22:05:41.829464+0000","last_became_peered":"2024-05-08T22:05:41.829464+0000","last_unstale":"2024-05-08T22:06:42.376187+0000","last_undegraded":"2024-05-08T22:06:42.376187+0000","last_fullsized":"2024-05-08T22:06:42.376187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:55.747487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.151416+0000","last_change":"2024-05-08T22:05:14.064695+0000","last_active":"2024-05-08T22:07:17.151416+0000","last_peered":"2024-05-08T22:07:17.151416+0000","last_clean":"2024-05-08T22:07:17.151416+0000","last_became_active":"2024-05-08T22:05:14.064283+0000","last_became_peered":"2024-05-08T22:05:14.064283+0000","last_unstale":"2024-05-08T22:07:17.151416+0000","last_undegraded":"2024-05-08T22:07:17.151416+0000","last_fullsized":"2024-05-08T22:07:17.151416+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:06.980594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.447815+0000","last_change":"2024-05-08T22:05:18.528475+0000","last_active":"2024-05-08T22:06:42.447815+0000","last_peered":"2024-05-08T22:06:42.447815+0000","last_clean":"2024-05-08T22:06:42.447815+0000","last_became_active":"2024-05-08T22:05:18.528089+0000","last_became_peered":"2024-05-08T22:05:18.528089+0000","last_unstale":"2024-05-08T22:06:42.447815+0000","last_undegraded":"2024-05-08T22:06:42.447815+0000","last_fullsized":"2024-05-08T22:06:42.447815+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:01.508984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'8","reported_seq":138,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:21.653137+0000","last_change":"2024-05-08T22:05:13.549191+0000","last_active":"2024-05-08T22:07:21.653137+0000","last_peered":"2024-05-08T22:07:21.653137+0000","last_clean":"2024-05-08T22:07:21.653137+0000","last_became_active":"2024-05-08T22:05:13.548737+0000","last_became_peered":"2024-05-08T22:05:13.548737+0000","last_unstale":"2024-05-08T22:07:21.653137+0000","last_undegraded":"2024-05-08T22:07:21.653137+0000","last_fullsized":"2024-05-08T22:07:21.653137+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:07.355656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1501,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.180110+0000","last_change":"2024-05-08T22:06:42.018982+0000","last_active":"2024-05-08T22:07:17.180110+0000","last_peered":"2024-05-08T22:07:17.180110+0000","last_clean":"2024-05-08T22:07:17.180110+0000","last_became_active":"2024-05-08T22:06:42.018609+0000","last_became_peered":"2024-05-08T22:06:42.018609+0000","last_unstale":"2024-05-08T22:07:17.180110+0000","last_undegraded":"2024-05-08T22:07:17.180110+0000","last_fullsized":"2024-05-08T22:07:17.180110+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:37.578711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.447741+0000","last_change":"2024-05-08T22:05:18.536861+0000","last_active":"2024-05-08T22:06:42.447741+0000","last_peered":"2024-05-08T22:06:42.447741+0000","last_clean":"2024-05-08T22:06:42.447741+0000","last_became_active":"2024-05-08T22:05:18.536472+0000","last_became_peered":"2024-05-08T22:05:18.536472+0000","last_unstale":"2024-05-08T22:06:42.447741+0000","last_undegraded":"2024-05-08T22:06:42.447741+0000","last_fullsized":"2024-05-08T22:06:42.447741+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:45:21.826812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.467017+0000","last_change":"2024-05-08T22:05:13.553326+0000","last_active":"2024-05-08T22:06:42.467017+0000","last_peered":"2024-05-08T22:06:42.467017+0000","last_clean":"2024-05-08T22:06:42.467017+0000","last_became_active":"2024-05-08T22:05:13.552983+0000","last_became_peered":"2024-05-08T22:05:13.552983+0000","last_unstale":"2024-05-08T22:06:42.467017+0000","last_undegraded":"2024-05-08T22:06:42.467017+0000","last_fullsized":"2024-05-08T22:06:42.467017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:17.195253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.165741+0000","last_change":"2024-05-08T22:05:14.068617+0000","last_active":"2024-05-08T22:07:17.165741+0000","last_peered":"2024-05-08T22:07:17.165741+0000","last_clean":"2024-05-08T22:07:17.165741+0000","last_became_active":"2024-05-08T22:05:14.067400+0000","last_became_peered":"2024-05-08T22:05:14.067400+0000","last_unstale":"2024-05-08T22:07:17.165741+0000","last_undegraded":"2024-05-08T22:07:17.165741+0000","last_fullsized":"2024-05-08T22:07:17.165741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:25.488212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.130379+0000","last_change":"2024-05-08T22:05:18.538294+0000","last_active":"2024-05-08T22:07:17.130379+0000","last_peered":"2024-05-08T22:07:17.130379+0000","last_clean":"2024-05-08T22:07:17.130379+0000","last_became_active":"2024-05-08T22:05:18.537984+0000","last_became_peered":"2024-05-08T22:05:18.537984+0000","last_unstale":"2024-05-08T22:07:17.130379+0000","last_undegraded":"2024-05-08T22:07:17.130379+0000","last_fullsized":"2024-05-08T22:07:17.130379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:32.117439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.017185+0000","last_change":"2024-05-08T22:05:13.557669+0000","last_active":"2024-05-08T22:06:42.017185+0000","last_peered":"2024-05-08T22:06:42.017185+0000","last_clean":"2024-05-08T22:06:42.017185+0000","last_became_active":"2024-05-08T22:05:13.555835+0000","last_became_peered":"2024-05-08T22:05:13.555835+0000","last_unstale":"2024-05-08T22:06:42.017185+0000","last_undegraded":"2024-05-08T22:06:42.017185+0000","last_fullsized":"2024-05-08T22:06:42.017185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:02.377635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.179458+0000","last_change":"2024-05-08T22:05:14.060144+0000","last_active":"2024-05-08T22:07:17.179458+0000","last_peered":"2024-05-08T22:07:17.179458+0000","last_clean":"2024-05-08T22:07:17.179458+0000","last_became_active":"2024-05-08T22:05:14.059387+0000","last_became_peered":"2024-05-08T22:05:14.059387+0000","last_unstale":"2024-05-08T22:07:17.179458+0000","last_undegraded":"2024-05-08T22:07:17.179458+0000","last_fullsized":"2024-05-08T22:07:17.179458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:32:28.057831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.140368+0000","last_change":"2024-05-08T22:05:18.527293+0000","last_active":"2024-05-08T22:07:17.140368+0000","last_peered":"2024-05-08T22:07:17.140368+0000","last_clean":"2024-05-08T22:07:17.140368+0000","last_became_active":"2024-05-08T22:05:18.526690+0000","last_became_peered":"2024-05-08T22:05:18.526690+0000","last_unstale":"2024-05-08T22:07:17.140368+0000","last_undegraded":"2024-05-08T22:07:17.140368+0000","last_fullsized":"2024-05-08T22:07:17.140368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:50.343536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.180700+0000","last_change":"2024-05-08T22:05:13.549774+0000","last_active":"2024-05-08T22:06:42.180700+0000","last_peered":"2024-05-08T22:06:42.180700+0000","last_clean":"2024-05-08T22:06:42.180700+0000","last_became_active":"2024-05-08T22:05:13.549531+0000","last_became_peered":"2024-05-08T22:05:13.549531+0000","last_unstale":"2024-05-08T22:06:42.180700+0000","last_undegraded":"2024-05-08T22:06:42.180700+0000","last_fullsized":"2024-05-08T22:06:42.180700+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:59:23.798728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.181017+0000","last_change":"2024-05-08T22:05:14.054232+0000","last_active":"2024-05-08T22:07:17.181017+0000","last_peered":"2024-05-08T22:07:17.181017+0000","last_clean":"2024-05-08T22:07:17.181017+0000","last_became_active":"2024-05-08T22:05:14.053806+0000","last_became_peered":"2024-05-08T22:05:14.053806+0000","last_unstale":"2024-05-08T22:07:17.181017+0000","last_undegraded":"2024-05-08T22:07:17.181017+0000","last_fullsized":"2024-05-08T22:07:17.181017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:32:21.678792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.979318+0000","last_change":"2024-05-08T22:05:18.535241+0000","last_active":"2024-05-08T22:06:41.979318+0000","last_peered":"2024-05-08T22:06:41.979318+0000","last_clean":"2024-05-08T22:06:41.979318+0000","last_became_active":"2024-05-08T22:05:18.534753+0000","last_became_peered":"2024-05-08T22:05:18.534753+0000","last_unstale":"2024-05-08T22:06:41.979318+0000","last_undegraded":"2024-05-08T22:06:41.979318+0000","last_fullsized":"2024-05-08T22:06:41.979318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:46:15.138437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":129,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:21.652796+0000","last_change":"2024-05-08T22:05:13.544899+0000","last_active":"2024-05-08T22:07:21.652796+0000","last_peered":"2024-05-08T22:07:21.652796+0000","last_clean":"2024-05-08T22:07:21.652796+0000","last_became_active":"2024-05-08T22:05:13.544498+0000","last_became_peered":"2024-05-08T22:05:13.544498+0000","last_unstale":"2024-05-08T22:07:21.652796+0000","last_undegraded":"2024-05-08T22:07:21.652796+0000","last_fullsized":"2024-05-08T22:07:21.652796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:58.169648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.176371+0000","last_change":"2024-05-08T22:05:14.049377+0000","last_active":"2024-05-08T22:07:17.176371+0000","last_peered":"2024-05-08T22:07:17.176371+0000","last_clean":"2024-05-08T22:07:17.176371+0000","last_became_active":"2024-05-08T22:05:14.049039+0000","last_became_peered":"2024-05-08T22:05:14.049039+0000","last_unstale":"2024-05-08T22:07:17.176371+0000","last_undegraded":"2024-05-08T22:07:17.176371+0000","last_fullsized":"2024-05-08T22:07:17.176371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:24.415392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976341+0000","last_change":"2024-05-08T22:05:18.528912+0000","last_active":"2024-05-08T22:06:41.976341+0000","last_peered":"2024-05-08T22:06:41.976341+0000","last_clean":"2024-05-08T22:06:41.976341+0000","last_became_active":"2024-05-08T22:05:18.528433+0000","last_became_peered":"2024-05-08T22:05:18.528433+0000","last_unstale":"2024-05-08T22:06:41.976341+0000","last_undegraded":"2024-05-08T22:06:41.976341+0000","last_fullsized":"2024-05-08T22:06:41.976341+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:05.574728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.055447+0000","last_change":"2024-05-08T22:05:13.550214+0000","last_active":"2024-05-08T22:06:42.055447+0000","last_peered":"2024-05-08T22:06:42.055447+0000","last_clean":"2024-05-08T22:06:42.055447+0000","last_became_active":"2024-05-08T22:05:13.549717+0000","last_became_peered":"2024-05-08T22:05:13.549717+0000","last_unstale":"2024-05-08T22:06:42.055447+0000","last_undegraded":"2024-05-08T22:06:42.055447+0000","last_fullsized":"2024-05-08T22:06:42.055447+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:12:20.406333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.177101+0000","last_change":"2024-05-08T22:05:14.071214+0000","last_active":"2024-05-08T22:07:17.177101+0000","last_peered":"2024-05-08T22:07:17.177101+0000","last_clean":"2024-05-08T22:07:17.177101+0000","last_became_active":"2024-05-08T22:05:14.066777+0000","last_became_peered":"2024-05-08T22:05:14.066777+0000","last_unstale":"2024-05-08T22:07:17.177101+0000","last_undegraded":"2024-05-08T22:07:17.177101+0000","last_fullsized":"2024-05-08T22:07:17.177101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:53.360911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976794+0000","last_change":"2024-05-08T22:05:18.523688+0000","last_active":"2024-05-08T22:06:41.976794+0000","last_peered":"2024-05-08T22:06:41.976794+0000","last_clean":"2024-05-08T22:06:41.976794+0000","last_became_active":"2024-05-08T22:05:18.523236+0000","last_became_peered":"2024-05-08T22:05:18.523236+0000","last_unstale":"2024-05-08T22:06:41.976794+0000","last_undegraded":"2024-05-08T22:06:41.976794+0000","last_fullsized":"2024-05-08T22:06:41.976794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:34.531699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.181222+0000","last_change":"2024-05-08T22:05:13.551027+0000","last_active":"2024-05-08T22:06:42.181222+0000","last_peered":"2024-05-08T22:06:42.181222+0000","last_clean":"2024-05-08T22:06:42.181222+0000","last_became_active":"2024-05-08T22:05:13.550471+0000","last_became_peered":"2024-05-08T22:05:13.550471+0000","last_unstale":"2024-05-08T22:06:42.181222+0000","last_undegraded":"2024-05-08T22:06:42.181222+0000","last_fullsized":"2024-05-08T22:06:42.181222+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:50.900419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.156519+0000","last_change":"2024-05-08T22:05:14.065191+0000","last_active":"2024-05-08T22:07:17.156519+0000","last_peered":"2024-05-08T22:07:17.156519+0000","last_clean":"2024-05-08T22:07:17.156519+0000","last_became_active":"2024-05-08T22:05:14.064893+0000","last_became_peered":"2024-05-08T22:05:14.064893+0000","last_unstale":"2024-05-08T22:07:17.156519+0000","last_undegraded":"2024-05-08T22:07:17.156519+0000","last_fullsized":"2024-05-08T22:07:17.156519+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:11.344646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976480+0000","last_change":"2024-05-08T22:05:18.530447+0000","last_active":"2024-05-08T22:06:41.976480+0000","last_peered":"2024-05-08T22:06:41.976480+0000","last_clean":"2024-05-08T22:06:41.976480+0000","last_became_active":"2024-05-08T22:05:18.528171+0000","last_became_peered":"2024-05-08T22:05:18.528171+0000","last_unstale":"2024-05-08T22:06:41.976480+0000","last_undegraded":"2024-05-08T22:06:41.976480+0000","last_fullsized":"2024-05-08T22:06:41.976480+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:09:43.347366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'39","reported_seq":278,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:22.654579+0000","last_change":"2024-05-08T22:05:13.547289+0000","last_active":"2024-05-08T22:07:22.654579+0000","last_peered":"2024-05-08T22:07:22.654579+0000","last_clean":"2024-05-08T22:07:22.654579+0000","last_became_active":"2024-05-08T22:05:13.546860+0000","last_became_peered":"2024-05-08T22:05:13.546860+0000","last_unstale":"2024-05-08T22:07:22.654579+0000","last_undegraded":"2024-05-08T22:07:22.654579+0000","last_fullsized":"2024-05-08T22:07:22.654579+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:08:50.339361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69082,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":218,"num_read_kb":175,"num_write":40,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.146652+0000","last_change":"2024-05-08T22:05:14.057925+0000","last_active":"2024-05-08T22:07:17.146652+0000","last_peered":"2024-05-08T22:07:17.146652+0000","last_clean":"2024-05-08T22:07:17.146652+0000","last_became_active":"2024-05-08T22:05:14.057662+0000","last_became_peered":"2024-05-08T22:05:14.057662+0000","last_unstale":"2024-05-08T22:07:17.146652+0000","last_undegraded":"2024-05-08T22:07:17.146652+0000","last_fullsized":"2024-05-08T22:07:17.146652+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:49.929610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.181486+0000","last_change":"2024-05-08T22:05:18.535509+0000","last_active":"2024-05-08T22:06:42.181486+0000","last_peered":"2024-05-08T22:06:42.181486+0000","last_clean":"2024-05-08T22:06:42.181486+0000","last_became_active":"2024-05-08T22:05:18.535024+0000","last_became_peered":"2024-05-08T22:05:18.535024+0000","last_unstale":"2024-05-08T22:06:42.181486+0000","last_undegraded":"2024-05-08T22:06:42.181486+0000","last_fullsized":"2024-05-08T22:06:42.181486+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:28.098737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.447863+0000","last_change":"2024-05-08T22:05:13.552096+0000","last_active":"2024-05-08T22:06:42.447863+0000","last_peered":"2024-05-08T22:06:42.447863+0000","last_clean":"2024-05-08T22:06:42.447863+0000","last_became_active":"2024-05-08T22:05:13.551655+0000","last_became_peered":"2024-05-08T22:05:13.551655+0000","last_unstale":"2024-05-08T22:06:42.447863+0000","last_undegraded":"2024-05-08T22:06:42.447863+0000","last_fullsized":"2024-05-08T22:06:42.447863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:24.371008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.180706+0000","last_change":"2024-05-08T22:05:14.066075+0000","last_active":"2024-05-08T22:07:17.180706+0000","last_peered":"2024-05-08T22:07:17.180706+0000","last_clean":"2024-05-08T22:07:17.180706+0000","last_became_active":"2024-05-08T22:05:14.065581+0000","last_became_peered":"2024-05-08T22:05:14.065581+0000","last_unstale":"2024-05-08T22:07:17.180706+0000","last_undegraded":"2024-05-08T22:07:17.180706+0000","last_fullsized":"2024-05-08T22:07:17.180706+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:15.403714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.376646+0000","last_change":"2024-05-08T22:05:41.831782+0000","last_active":"2024-05-08T22:06:42.376646+0000","last_peered":"2024-05-08T22:06:42.376646+0000","last_clean":"2024-05-08T22:06:42.376646+0000","last_became_active":"2024-05-08T22:05:41.831335+0000","last_became_peered":"2024-05-08T22:05:41.831335+0000","last_unstale":"2024-05-08T22:06:42.376646+0000","last_undegraded":"2024-05-08T22:06:42.376646+0000","last_fullsized":"2024-05-08T22:06:42.376646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:39:34.031128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.181155+0000","last_change":"2024-05-08T22:05:13.549112+0000","last_active":"2024-05-08T22:06:42.181155+0000","last_peered":"2024-05-08T22:06:42.181155+0000","last_clean":"2024-05-08T22:06:42.181155+0000","last_became_active":"2024-05-08T22:05:13.548763+0000","last_became_peered":"2024-05-08T22:05:13.548763+0000","last_unstale":"2024-05-08T22:06:42.181155+0000","last_undegraded":"2024-05-08T22:06:42.181155+0000","last_fullsized":"2024-05-08T22:06:42.181155+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:26:56.448611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.158808+0000","last_change":"2024-05-08T22:05:14.067116+0000","last_active":"2024-05-08T22:07:17.158808+0000","last_peered":"2024-05-08T22:07:17.158808+0000","last_clean":"2024-05-08T22:07:17.158808+0000","last_became_active":"2024-05-08T22:05:14.064775+0000","last_became_peered":"2024-05-08T22:05:14.064775+0000","last_unstale":"2024-05-08T22:07:17.158808+0000","last_undegraded":"2024-05-08T22:07:17.158808+0000","last_fullsized":"2024-05-08T22:07:17.158808+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:28:32.901405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.977984+0000","last_change":"2024-05-08T22:05:18.539546+0000","last_active":"2024-05-08T22:06:41.977984+0000","last_peered":"2024-05-08T22:06:41.977984+0000","last_clean":"2024-05-08T22:06:41.977984+0000","last_became_active":"2024-05-08T22:05:18.538998+0000","last_became_peered":"2024-05-08T22:05:18.538998+0000","last_unstale":"2024-05-08T22:06:41.977984+0000","last_undegraded":"2024-05-08T22:06:41.977984+0000","last_fullsized":"2024-05-08T22:06:41.977984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:30.453881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":135,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:21.653538+0000","last_change":"2024-05-08T22:05:13.549053+0000","last_active":"2024-05-08T22:07:21.653538+0000","last_peered":"2024-05-08T22:07:21.653538+0000","last_clean":"2024-05-08T22:07:21.653538+0000","last_became_active":"2024-05-08T22:05:13.548739+0000","last_became_peered":"2024-05-08T22:05:13.548739+0000","last_unstale":"2024-05-08T22:07:21.653538+0000","last_undegraded":"2024-05-08T22:07:21.653538+0000","last_fullsized":"2024-05-08T22:07:21.653538+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:07:10.298084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1719,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.150047+0000","last_change":"2024-05-08T22:05:14.061753+0000","last_active":"2024-05-08T22:07:17.150047+0000","last_peered":"2024-05-08T22:07:17.150047+0000","last_clean":"2024-05-08T22:07:17.150047+0000","last_became_active":"2024-05-08T22:05:14.061412+0000","last_became_peered":"2024-05-08T22:05:14.061412+0000","last_unstale":"2024-05-08T22:07:17.150047+0000","last_undegraded":"2024-05-08T22:07:17.150047+0000","last_fullsized":"2024-05-08T22:07:17.150047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:54.380181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.017707+0000","last_change":"2024-05-08T22:05:18.532627+0000","last_active":"2024-05-08T22:06:42.017707+0000","last_peered":"2024-05-08T22:06:42.017707+0000","last_clean":"2024-05-08T22:06:42.017707+0000","last_became_active":"2024-05-08T22:05:18.532205+0000","last_became_peered":"2024-05-08T22:05:18.532205+0000","last_unstale":"2024-05-08T22:06:42.017707+0000","last_undegraded":"2024-05-08T22:06:42.017707+0000","last_fullsized":"2024-05-08T22:06:42.017707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:20:11.028645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":64,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976362+0000","last_change":"2024-05-08T22:05:41.841832+0000","last_active":"2024-05-08T22:06:41.976362+0000","last_peered":"2024-05-08T22:06:41.976362+0000","last_clean":"2024-05-08T22:06:41.976362+0000","last_became_active":"2024-05-08T22:05:41.829584+0000","last_became_peered":"2024-05-08T22:05:41.829584+0000","last_unstale":"2024-05-08T22:06:41.976362+0000","last_undegraded":"2024-05-08T22:06:41.976362+0000","last_fullsized":"2024-05-08T22:06:41.976362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:52:19.876215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.149577+0000","last_change":"2024-05-08T22:05:14.063027+0000","last_active":"2024-05-08T22:07:17.149577+0000","last_peered":"2024-05-08T22:07:17.149577+0000","last_clean":"2024-05-08T22:07:17.149577+0000","last_became_active":"2024-05-08T22:05:14.062682+0000","last_became_peered":"2024-05-08T22:05:14.062682+0000","last_unstale":"2024-05-08T22:07:17.149577+0000","last_undegraded":"2024-05-08T22:07:17.149577+0000","last_fullsized":"2024-05-08T22:07:17.149577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:47.248832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.180890+0000","last_change":"2024-05-08T22:05:18.528370+0000","last_active":"2024-05-08T22:06:42.180890+0000","last_peered":"2024-05-08T22:06:42.180890+0000","last_clean":"2024-05-08T22:06:42.180890+0000","last_became_active":"2024-05-08T22:05:18.527990+0000","last_became_peered":"2024-05-08T22:05:18.527990+0000","last_unstale":"2024-05-08T22:06:42.180890+0000","last_undegraded":"2024-05-08T22:06:42.180890+0000","last_fullsized":"2024-05-08T22:06:42.180890+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:07.524175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.120821+0000","last_change":"2024-05-08T22:05:13.547543+0000","last_active":"2024-05-08T22:07:17.120821+0000","last_peered":"2024-05-08T22:07:17.120821+0000","last_clean":"2024-05-08T22:07:17.120821+0000","last_became_active":"2024-05-08T22:05:13.547191+0000","last_became_peered":"2024-05-08T22:05:13.547191+0000","last_unstale":"2024-05-08T22:07:17.120821+0000","last_undegraded":"2024-05-08T22:07:17.120821+0000","last_fullsized":"2024-05-08T22:07:17.120821+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:53.213239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.150554+0000","last_change":"2024-05-08T22:05:14.062677+0000","last_active":"2024-05-08T22:07:17.150554+0000","last_peered":"2024-05-08T22:07:17.150554+0000","last_clean":"2024-05-08T22:07:17.150554+0000","last_became_active":"2024-05-08T22:05:14.062306+0000","last_became_peered":"2024-05-08T22:05:14.062306+0000","last_unstale":"2024-05-08T22:07:17.150554+0000","last_undegraded":"2024-05-08T22:07:17.150554+0000","last_fullsized":"2024-05-08T22:07:17.150554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:00.356505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.119465+0000","last_change":"2024-05-08T22:05:18.617075+0000","last_active":"2024-05-08T22:07:17.119465+0000","last_peered":"2024-05-08T22:07:17.119465+0000","last_clean":"2024-05-08T22:07:17.119465+0000","last_became_active":"2024-05-08T22:05:18.616653+0000","last_became_peered":"2024-05-08T22:05:18.616653+0000","last_unstale":"2024-05-08T22:07:17.119465+0000","last_undegraded":"2024-05-08T22:07:17.119465+0000","last_fullsized":"2024-05-08T22:07:17.119465+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:30.537064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.103554+0000","last_change":"2024-05-08T22:05:13.553457+0000","last_active":"2024-05-08T22:07:17.103554+0000","last_peered":"2024-05-08T22:07:17.103554+0000","last_clean":"2024-05-08T22:07:17.103554+0000","last_became_active":"2024-05-08T22:05:13.551583+0000","last_became_peered":"2024-05-08T22:05:13.551583+0000","last_unstale":"2024-05-08T22:07:17.103554+0000","last_undegraded":"2024-05-08T22:07:17.103554+0000","last_fullsized":"2024-05-08T22:07:17.103554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:07:05.971413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.172164+0000","last_change":"2024-05-08T22:05:14.057476+0000","last_active":"2024-05-08T22:07:17.172164+0000","last_peered":"2024-05-08T22:07:17.172164+0000","last_clean":"2024-05-08T22:07:17.172164+0000","last_became_active":"2024-05-08T22:05:14.056550+0000","last_became_peered":"2024-05-08T22:05:14.056550+0000","last_unstale":"2024-05-08T22:07:17.172164+0000","last_undegraded":"2024-05-08T22:07:17.172164+0000","last_fullsized":"2024-05-08T22:07:17.172164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:41:29.264096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.017335+0000","last_change":"2024-05-08T22:05:18.530719+0000","last_active":"2024-05-08T22:06:42.017335+0000","last_peered":"2024-05-08T22:06:42.017335+0000","last_clean":"2024-05-08T22:06:42.017335+0000","last_became_active":"2024-05-08T22:05:18.529659+0000","last_became_peered":"2024-05-08T22:05:18.529659+0000","last_unstale":"2024-05-08T22:06:42.017335+0000","last_undegraded":"2024-05-08T22:06:42.017335+0000","last_fullsized":"2024-05-08T22:06:42.017335+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:13:06.514460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.107307+0000","last_change":"2024-05-08T22:05:13.556191+0000","last_active":"2024-05-08T22:07:17.107307+0000","last_peered":"2024-05-08T22:07:17.107307+0000","last_clean":"2024-05-08T22:07:17.107307+0000","last_became_active":"2024-05-08T22:05:13.555888+0000","last_became_peered":"2024-05-08T22:05:13.555888+0000","last_unstale":"2024-05-08T22:07:17.107307+0000","last_undegraded":"2024-05-08T22:07:17.107307+0000","last_fullsized":"2024-05-08T22:07:17.107307+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:05.637980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.179969+0000","last_change":"2024-05-08T22:05:41.828023+0000","last_active":"2024-05-08T22:07:17.179969+0000","last_peered":"2024-05-08T22:07:17.179969+0000","last_clean":"2024-05-08T22:07:17.179969+0000","last_became_active":"2024-05-08T22:05:41.827550+0000","last_became_peered":"2024-05-08T22:05:41.827550+0000","last_unstale":"2024-05-08T22:07:17.179969+0000","last_undegraded":"2024-05-08T22:07:17.179969+0000","last_fullsized":"2024-05-08T22:07:17.179969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:13.234423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.017591+0000","last_change":"2024-05-08T22:05:41.828529+0000","last_active":"2024-05-08T22:06:42.017591+0000","last_peered":"2024-05-08T22:06:42.017591+0000","last_clean":"2024-05-08T22:06:42.017591+0000","last_became_active":"2024-05-08T22:05:41.828081+0000","last_became_peered":"2024-05-08T22:05:41.828081+0000","last_unstale":"2024-05-08T22:06:42.017591+0000","last_undegraded":"2024-05-08T22:06:42.017591+0000","last_fullsized":"2024-05-08T22:06:42.017591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:27.478823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.152262+0000","last_change":"2024-05-08T22:05:14.069096+0000","last_active":"2024-05-08T22:07:17.152262+0000","last_peered":"2024-05-08T22:07:17.152262+0000","last_clean":"2024-05-08T22:07:17.152262+0000","last_became_active":"2024-05-08T22:05:14.068796+0000","last_became_peered":"2024-05-08T22:05:14.068796+0000","last_unstale":"2024-05-08T22:07:17.152262+0000","last_undegraded":"2024-05-08T22:07:17.152262+0000","last_fullsized":"2024-05-08T22:07:17.152262+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:52:52.391196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":131,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:21.653111+0000","last_change":"2024-05-08T22:05:13.559286+0000","last_active":"2024-05-08T22:07:21.653111+0000","last_peered":"2024-05-08T22:07:21.653111+0000","last_clean":"2024-05-08T22:07:21.653111+0000","last_became_active":"2024-05-08T22:05:13.558922+0000","last_became_peered":"2024-05-08T22:05:13.558922+0000","last_unstale":"2024-05-08T22:07:21.653111+0000","last_undegraded":"2024-05-08T22:07:21.653111+0000","last_fullsized":"2024-05-08T22:07:21.653111+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:25:24.732422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.055193+0000","last_change":"2024-05-08T22:05:18.527031+0000","last_active":"2024-05-08T22:06:42.055193+0000","last_peered":"2024-05-08T22:06:42.055193+0000","last_clean":"2024-05-08T22:06:42.055193+0000","last_became_active":"2024-05-08T22:05:18.526573+0000","last_became_peered":"2024-05-08T22:05:18.526573+0000","last_unstale":"2024-05-08T22:06:42.055193+0000","last_undegraded":"2024-05-08T22:06:42.055193+0000","last_fullsized":"2024-05-08T22:06:42.055193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:08.492359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.178225+0000","last_change":"2024-05-08T22:06:42.278191+0000","last_active":"2024-05-08T22:07:17.178225+0000","last_peered":"2024-05-08T22:07:17.178225+0000","last_clean":"2024-05-08T22:07:17.178225+0000","last_became_active":"2024-05-08T22:06:42.277819+0000","last_became_peered":"2024-05-08T22:06:42.277819+0000","last_unstale":"2024-05-08T22:07:17.178225+0000","last_undegraded":"2024-05-08T22:07:17.178225+0000","last_fullsized":"2024-05-08T22:07:17.178225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:48.919595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.104081+0000","last_change":"2024-05-08T22:05:13.545709+0000","last_active":"2024-05-08T22:07:17.104081+0000","last_peered":"2024-05-08T22:07:17.104081+0000","last_clean":"2024-05-08T22:07:17.104081+0000","last_became_active":"2024-05-08T22:05:13.545419+0000","last_became_peered":"2024-05-08T22:05:13.545419+0000","last_unstale":"2024-05-08T22:07:17.104081+0000","last_undegraded":"2024-05-08T22:07:17.104081+0000","last_fullsized":"2024-05-08T22:07:17.104081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:44.903903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"88'148","reported_seq":384,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:19.254206+0000","last_change":"2024-05-08T22:04:20.948328+0000","last_active":"2024-05-08T22:07:19.254206+0000","last_peered":"2024-05-08T22:07:19.254206+0000","last_clean":"2024-05-08T22:07:19.254206+0000","last_became_active":"2024-05-08T22:04:20.947887+0000","last_became_peered":"2024-05-08T22:04:20.947887+0000","last_unstale":"2024-05-08T22:07:19.254206+0000","last_undegraded":"2024-05-08T22:07:19.254206+0000","last_fullsized":"2024-05-08T22:07:19.254206+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:01:41.485277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:01:41.485277+0000","last_clean_scrub_stamp":"2024-05-08T22:01:41.485277+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:43:59.208120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.181413+0000","last_change":"2024-05-08T22:05:41.824591+0000","last_active":"2024-05-08T22:06:42.181413+0000","last_peered":"2024-05-08T22:06:42.181413+0000","last_clean":"2024-05-08T22:06:42.181413+0000","last_became_active":"2024-05-08T22:05:41.824053+0000","last_became_peered":"2024-05-08T22:05:41.824053+0000","last_unstale":"2024-05-08T22:06:42.181413+0000","last_undegraded":"2024-05-08T22:06:42.181413+0000","last_fullsized":"2024-05-08T22:06:42.181413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:18:57.269459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.145622+0000","last_change":"2024-05-08T22:05:14.066818+0000","last_active":"2024-05-08T22:07:17.145622+0000","last_peered":"2024-05-08T22:07:17.145622+0000","last_clean":"2024-05-08T22:07:17.145622+0000","last_became_active":"2024-05-08T22:05:14.065152+0000","last_became_peered":"2024-05-08T22:05:14.065152+0000","last_unstale":"2024-05-08T22:07:17.145622+0000","last_undegraded":"2024-05-08T22:07:17.145622+0000","last_fullsized":"2024-05-08T22:07:17.145622+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:53:19.018852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.107386+0000","last_change":"2024-05-08T22:05:13.550365+0000","last_active":"2024-05-08T22:07:17.107386+0000","last_peered":"2024-05-08T22:07:17.107386+0000","last_clean":"2024-05-08T22:07:17.107386+0000","last_became_active":"2024-05-08T22:05:13.549677+0000","last_became_peered":"2024-05-08T22:05:13.549677+0000","last_unstale":"2024-05-08T22:07:17.107386+0000","last_undegraded":"2024-05-08T22:07:17.107386+0000","last_fullsized":"2024-05-08T22:07:17.107386+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:10.842838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.376136+0000","last_change":"2024-05-08T22:05:18.535777+0000","last_active":"2024-05-08T22:06:42.376136+0000","last_peered":"2024-05-08T22:06:42.376136+0000","last_clean":"2024-05-08T22:06:42.376136+0000","last_became_active":"2024-05-08T22:05:18.535327+0000","last_became_peered":"2024-05-08T22:05:18.535327+0000","last_unstale":"2024-05-08T22:06:42.376136+0000","last_undegraded":"2024-05-08T22:06:42.376136+0000","last_fullsized":"2024-05-08T22:06:42.376136+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:13.886989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.121005+0000","last_change":"2024-05-08T22:05:13.554045+0000","last_active":"2024-05-08T22:07:17.121005+0000","last_peered":"2024-05-08T22:07:17.121005+0000","last_clean":"2024-05-08T22:07:17.121005+0000","last_became_active":"2024-05-08T22:05:13.553315+0000","last_became_peered":"2024-05-08T22:05:13.553315+0000","last_unstale":"2024-05-08T22:07:17.121005+0000","last_undegraded":"2024-05-08T22:07:17.121005+0000","last_fullsized":"2024-05-08T22:07:17.121005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:13.131267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.182154+0000","last_change":"2024-05-08T22:05:14.065018+0000","last_active":"2024-05-08T22:07:17.182154+0000","last_peered":"2024-05-08T22:07:17.182154+0000","last_clean":"2024-05-08T22:07:17.182154+0000","last_became_active":"2024-05-08T22:05:14.064701+0000","last_became_peered":"2024-05-08T22:05:14.064701+0000","last_unstale":"2024-05-08T22:07:17.182154+0000","last_undegraded":"2024-05-08T22:07:17.182154+0000","last_fullsized":"2024-05-08T22:07:17.182154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:48:43.468373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.121062+0000","last_change":"2024-05-08T22:05:18.617853+0000","last_active":"2024-05-08T22:07:17.121062+0000","last_peered":"2024-05-08T22:07:17.121062+0000","last_clean":"2024-05-08T22:07:17.121062+0000","last_became_active":"2024-05-08T22:05:18.616255+0000","last_became_peered":"2024-05-08T22:05:18.616255+0000","last_unstale":"2024-05-08T22:07:17.121062+0000","last_undegraded":"2024-05-08T22:07:17.121062+0000","last_fullsized":"2024-05-08T22:07:17.121062+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:13.233425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'41","reported_seq":86,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:22.787863+0000","last_change":"2024-05-08T22:05:13.546274+0000","last_active":"2024-05-08T22:07:22.787863+0000","last_peered":"2024-05-08T22:07:22.787863+0000","last_clean":"2024-05-08T22:07:22.787863+0000","last_became_active":"2024-05-08T22:05:13.543651+0000","last_became_peered":"2024-05-08T22:05:13.543651+0000","last_unstale":"2024-05-08T22:07:22.787863+0000","last_undegraded":"2024-05-08T22:07:22.787863+0000","last_fullsized":"2024-05-08T22:07:22.787863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:21:58.601261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":73,"num_write_kb":835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.162082+0000","last_change":"2024-05-08T22:05:14.056535+0000","last_active":"2024-05-08T22:07:17.162082+0000","last_peered":"2024-05-08T22:07:17.162082+0000","last_clean":"2024-05-08T22:07:17.162082+0000","last_became_active":"2024-05-08T22:05:14.056211+0000","last_became_peered":"2024-05-08T22:05:14.056211+0000","last_unstale":"2024-05-08T22:07:17.162082+0000","last_undegraded":"2024-05-08T22:07:17.162082+0000","last_fullsized":"2024-05-08T22:07:17.162082+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:46:30.744490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.107088+0000","last_change":"2024-05-08T22:05:18.524939+0000","last_active":"2024-05-08T22:07:17.107088+0000","last_peered":"2024-05-08T22:07:17.107088+0000","last_clean":"2024-05-08T22:07:17.107088+0000","last_became_active":"2024-05-08T22:05:18.524518+0000","last_became_peered":"2024-05-08T22:05:18.524518+0000","last_unstale":"2024-05-08T22:07:17.107088+0000","last_undegraded":"2024-05-08T22:07:17.107088+0000","last_fullsized":"2024-05-08T22:07:17.107088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:50.192400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.977078+0000","last_change":"2024-05-08T22:05:13.554346+0000","last_active":"2024-05-08T22:06:41.977078+0000","last_peered":"2024-05-08T22:06:41.977078+0000","last_clean":"2024-05-08T22:06:41.977078+0000","last_became_active":"2024-05-08T22:05:13.554080+0000","last_became_peered":"2024-05-08T22:05:13.554080+0000","last_unstale":"2024-05-08T22:06:41.977078+0000","last_undegraded":"2024-05-08T22:06:41.977078+0000","last_fullsized":"2024-05-08T22:06:41.977078+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:29:49.695682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.181025+0000","last_change":"2024-05-08T22:05:14.071422+0000","last_active":"2024-05-08T22:07:17.181025+0000","last_peered":"2024-05-08T22:07:17.181025+0000","last_clean":"2024-05-08T22:07:17.181025+0000","last_became_active":"2024-05-08T22:05:14.070396+0000","last_became_peered":"2024-05-08T22:05:14.070396+0000","last_unstale":"2024-05-08T22:07:17.181025+0000","last_undegraded":"2024-05-08T22:07:17.181025+0000","last_fullsized":"2024-05-08T22:07:17.181025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:37.957983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976556+0000","last_change":"2024-05-08T22:05:18.617149+0000","last_active":"2024-05-08T22:06:41.976556+0000","last_peered":"2024-05-08T22:06:41.976556+0000","last_clean":"2024-05-08T22:06:41.976556+0000","last_became_active":"2024-05-08T22:05:18.616699+0000","last_became_peered":"2024-05-08T22:05:18.616699+0000","last_unstale":"2024-05-08T22:06:41.976556+0000","last_undegraded":"2024-05-08T22:06:41.976556+0000","last_fullsized":"2024-05-08T22:06:41.976556+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:51.167142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.117273+0000","last_change":"2024-05-08T22:05:13.551212+0000","last_active":"2024-05-08T22:07:17.117273+0000","last_peered":"2024-05-08T22:07:17.117273+0000","last_clean":"2024-05-08T22:07:17.117273+0000","last_became_active":"2024-05-08T22:05:13.550909+0000","last_became_peered":"2024-05-08T22:05:13.550909+0000","last_unstale":"2024-05-08T22:07:17.117273+0000","last_undegraded":"2024-05-08T22:07:17.117273+0000","last_fullsized":"2024-05-08T22:07:17.117273+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:48.657928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.123447+0000","last_change":"2024-05-08T22:05:14.053976+0000","last_active":"2024-05-08T22:07:17.123447+0000","last_peered":"2024-05-08T22:07:17.123447+0000","last_clean":"2024-05-08T22:07:17.123447+0000","last_became_active":"2024-05-08T22:05:14.053664+0000","last_became_peered":"2024-05-08T22:05:14.053664+0000","last_unstale":"2024-05-08T22:07:17.123447+0000","last_undegraded":"2024-05-08T22:07:17.123447+0000","last_fullsized":"2024-05-08T22:07:17.123447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:25:25.642768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.978021+0000","last_change":"2024-05-08T22:05:18.536771+0000","last_active":"2024-05-08T22:06:41.978021+0000","last_peered":"2024-05-08T22:06:41.978021+0000","last_clean":"2024-05-08T22:06:41.978021+0000","last_became_active":"2024-05-08T22:05:18.536260+0000","last_became_peered":"2024-05-08T22:05:18.536260+0000","last_unstale":"2024-05-08T22:06:41.978021+0000","last_undegraded":"2024-05-08T22:06:41.978021+0000","last_fullsized":"2024-05-08T22:06:41.978021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:34.414381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.979065+0000","last_change":"2024-05-08T22:05:13.540679+0000","last_active":"2024-05-08T22:06:41.979065+0000","last_peered":"2024-05-08T22:06:41.979065+0000","last_clean":"2024-05-08T22:06:41.979065+0000","last_became_active":"2024-05-08T22:05:13.539428+0000","last_became_peered":"2024-05-08T22:05:13.539428+0000","last_unstale":"2024-05-08T22:06:41.979065+0000","last_undegraded":"2024-05-08T22:06:41.979065+0000","last_fullsized":"2024-05-08T22:06:41.979065+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:59:34.455411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.153688+0000","last_change":"2024-05-08T22:05:14.061849+0000","last_active":"2024-05-08T22:07:17.153688+0000","last_peered":"2024-05-08T22:07:17.153688+0000","last_clean":"2024-05-08T22:07:17.153688+0000","last_became_active":"2024-05-08T22:05:14.061028+0000","last_became_peered":"2024-05-08T22:05:14.061028+0000","last_unstale":"2024-05-08T22:07:17.153688+0000","last_undegraded":"2024-05-08T22:07:17.153688+0000","last_fullsized":"2024-05-08T22:07:17.153688+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:11:59.489376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976549+0000","last_change":"2024-05-08T22:05:18.517994+0000","last_active":"2024-05-08T22:06:41.976549+0000","last_peered":"2024-05-08T22:06:41.976549+0000","last_clean":"2024-05-08T22:06:41.976549+0000","last_became_active":"2024-05-08T22:05:18.517655+0000","last_became_peered":"2024-05-08T22:05:18.517655+0000","last_unstale":"2024-05-08T22:06:41.976549+0000","last_undegraded":"2024-05-08T22:06:41.976549+0000","last_fullsized":"2024-05-08T22:06:41.976549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:50:08.956277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"84'1","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.447717+0000","last_change":"2024-05-08T22:05:13.552851+0000","last_active":"2024-05-08T22:06:42.447717+0000","last_peered":"2024-05-08T22:06:42.447717+0000","last_clean":"2024-05-08T22:06:42.447717+0000","last_became_active":"2024-05-08T22:05:13.552507+0000","last_became_peered":"2024-05-08T22:05:13.552507+0000","last_unstale":"2024-05-08T22:06:42.447717+0000","last_undegraded":"2024-05-08T22:06:42.447717+0000","last_fullsized":"2024-05-08T22:06:42.447717+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:29.248623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.176835+0000","last_change":"2024-05-08T22:05:14.062618+0000","last_active":"2024-05-08T22:07:17.176835+0000","last_peered":"2024-05-08T22:07:17.176835+0000","last_clean":"2024-05-08T22:07:17.176835+0000","last_became_active":"2024-05-08T22:05:14.062308+0000","last_became_peered":"2024-05-08T22:05:14.062308+0000","last_unstale":"2024-05-08T22:07:17.176835+0000","last_undegraded":"2024-05-08T22:07:17.176835+0000","last_fullsized":"2024-05-08T22:07:17.176835+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:34.195343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976573+0000","last_change":"2024-05-08T22:05:18.616846+0000","last_active":"2024-05-08T22:06:41.976573+0000","last_peered":"2024-05-08T22:06:41.976573+0000","last_clean":"2024-05-08T22:06:41.976573+0000","last_became_active":"2024-05-08T22:05:18.616369+0000","last_became_peered":"2024-05-08T22:05:18.616369+0000","last_unstale":"2024-05-08T22:06:41.976573+0000","last_undegraded":"2024-05-08T22:06:41.976573+0000","last_fullsized":"2024-05-08T22:06:41.976573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:26.128949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976543+0000","last_change":"2024-05-08T22:05:13.549712+0000","last_active":"2024-05-08T22:06:41.976543+0000","last_peered":"2024-05-08T22:06:41.976543+0000","last_clean":"2024-05-08T22:06:41.976543+0000","last_became_active":"2024-05-08T22:05:13.549508+0000","last_became_peered":"2024-05-08T22:05:13.549508+0000","last_unstale":"2024-05-08T22:06:41.976543+0000","last_undegraded":"2024-05-08T22:06:41.976543+0000","last_fullsized":"2024-05-08T22:06:41.976543+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:30:22.706389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.157838+0000","last_change":"2024-05-08T22:05:14.060920+0000","last_active":"2024-05-08T22:07:17.157838+0000","last_peered":"2024-05-08T22:07:17.157838+0000","last_clean":"2024-05-08T22:07:17.157838+0000","last_became_active":"2024-05-08T22:05:14.059914+0000","last_became_peered":"2024-05-08T22:05:14.059914+0000","last_unstale":"2024-05-08T22:07:17.157838+0000","last_undegraded":"2024-05-08T22:07:17.157838+0000","last_fullsized":"2024-05-08T22:07:17.157838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:15.513600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.276872+0000","last_change":"2024-05-08T22:05:18.534076+0000","last_active":"2024-05-08T22:06:42.276872+0000","last_peered":"2024-05-08T22:06:42.276872+0000","last_clean":"2024-05-08T22:06:42.276872+0000","last_became_active":"2024-05-08T22:05:18.533679+0000","last_became_peered":"2024-05-08T22:05:18.533679+0000","last_unstale":"2024-05-08T22:06:42.276872+0000","last_undegraded":"2024-05-08T22:06:42.276872+0000","last_fullsized":"2024-05-08T22:06:42.276872+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:08.979193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.276939+0000","last_change":"2024-05-08T22:05:41.845354+0000","last_active":"2024-05-08T22:06:42.276939+0000","last_peered":"2024-05-08T22:06:42.276939+0000","last_clean":"2024-05-08T22:06:42.276939+0000","last_became_active":"2024-05-08T22:05:41.830506+0000","last_became_peered":"2024-05-08T22:05:41.830506+0000","last_unstale":"2024-05-08T22:06:42.276939+0000","last_undegraded":"2024-05-08T22:06:42.276939+0000","last_fullsized":"2024-05-08T22:06:42.276939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:53:23.081939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.153480+0000","last_change":"2024-05-08T22:05:14.065868+0000","last_active":"2024-05-08T22:07:17.153480+0000","last_peered":"2024-05-08T22:07:17.153480+0000","last_clean":"2024-05-08T22:07:17.153480+0000","last_became_active":"2024-05-08T22:05:14.065545+0000","last_became_peered":"2024-05-08T22:05:14.065545+0000","last_unstale":"2024-05-08T22:07:17.153480+0000","last_undegraded":"2024-05-08T22:07:17.153480+0000","last_fullsized":"2024-05-08T22:07:17.153480+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:56:34.309095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.467018+0000","last_change":"2024-05-08T22:05:18.534261+0000","last_active":"2024-05-08T22:06:42.467018+0000","last_peered":"2024-05-08T22:06:42.467018+0000","last_clean":"2024-05-08T22:06:42.467018+0000","last_became_active":"2024-05-08T22:05:18.531814+0000","last_became_peered":"2024-05-08T22:05:18.531814+0000","last_unstale":"2024-05-08T22:06:42.467018+0000","last_undegraded":"2024-05-08T22:06:42.467018+0000","last_fullsized":"2024-05-08T22:06:42.467018+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:38.344546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'39","reported_seq":68,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.397200+0000","last_change":"2024-05-08T22:05:13.549344+0000","last_active":"2024-05-08T22:07:17.397200+0000","last_peered":"2024-05-08T22:07:17.397200+0000","last_clean":"2024-05-08T22:07:17.397200+0000","last_became_active":"2024-05-08T22:05:13.548993+0000","last_became_peered":"2024-05-08T22:05:13.548993+0000","last_unstale":"2024-05-08T22:07:17.397200+0000","last_undegraded":"2024-05-08T22:07:17.397200+0000","last_fullsized":"2024-05-08T22:07:17.397200+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:50.096602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.180642+0000","last_change":"2024-05-08T22:05:14.065758+0000","last_active":"2024-05-08T22:07:17.180642+0000","last_peered":"2024-05-08T22:07:17.180642+0000","last_clean":"2024-05-08T22:07:17.180642+0000","last_became_active":"2024-05-08T22:05:14.065442+0000","last_became_peered":"2024-05-08T22:05:14.065442+0000","last_unstale":"2024-05-08T22:07:17.180642+0000","last_undegraded":"2024-05-08T22:07:17.180642+0000","last_fullsized":"2024-05-08T22:07:17.180642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:43.047796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.180606+0000","last_change":"2024-05-08T22:05:18.529804+0000","last_active":"2024-05-08T22:06:42.180606+0000","last_peered":"2024-05-08T22:06:42.180606+0000","last_clean":"2024-05-08T22:06:42.180606+0000","last_became_active":"2024-05-08T22:05:18.529346+0000","last_became_peered":"2024-05-08T22:05:18.529346+0000","last_unstale":"2024-05-08T22:06:42.180606+0000","last_undegraded":"2024-05-08T22:06:42.180606+0000","last_fullsized":"2024-05-08T22:06:42.180606+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:13.565707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.017625+0000","last_change":"2024-05-08T22:05:13.551818+0000","last_active":"2024-05-08T22:06:42.017625+0000","last_peered":"2024-05-08T22:06:42.017625+0000","last_clean":"2024-05-08T22:06:42.017625+0000","last_became_active":"2024-05-08T22:05:13.551080+0000","last_became_peered":"2024-05-08T22:05:13.551080+0000","last_unstale":"2024-05-08T22:06:42.017625+0000","last_undegraded":"2024-05-08T22:06:42.017625+0000","last_fullsized":"2024-05-08T22:06:42.017625+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:31.636482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.146513+0000","last_change":"2024-05-08T22:05:14.066572+0000","last_active":"2024-05-08T22:07:17.146513+0000","last_peered":"2024-05-08T22:07:17.146513+0000","last_clean":"2024-05-08T22:07:17.146513+0000","last_became_active":"2024-05-08T22:05:14.066272+0000","last_became_peered":"2024-05-08T22:05:14.066272+0000","last_unstale":"2024-05-08T22:07:17.146513+0000","last_undegraded":"2024-05-08T22:07:17.146513+0000","last_fullsized":"2024-05-08T22:07:17.146513+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:20:27.423464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.151234+0000","last_change":"2024-05-08T22:05:14.061435+0000","last_active":"2024-05-08T22:07:17.151234+0000","last_peered":"2024-05-08T22:07:17.151234+0000","last_clean":"2024-05-08T22:07:17.151234+0000","last_became_active":"2024-05-08T22:05:14.061079+0000","last_became_peered":"2024-05-08T22:05:14.061079+0000","last_unstale":"2024-05-08T22:07:17.151234+0000","last_undegraded":"2024-05-08T22:07:17.151234+0000","last_fullsized":"2024-05-08T22:07:17.151234+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:42.684743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.147601+0000","last_change":"2024-05-08T22:05:41.828352+0000","last_active":"2024-05-08T22:07:17.147601+0000","last_peered":"2024-05-08T22:07:17.147601+0000","last_clean":"2024-05-08T22:07:17.147601+0000","last_became_active":"2024-05-08T22:05:41.827782+0000","last_became_peered":"2024-05-08T22:05:41.827782+0000","last_unstale":"2024-05-08T22:07:17.147601+0000","last_undegraded":"2024-05-08T22:07:17.147601+0000","last_fullsized":"2024-05-08T22:07:17.147601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:08.883148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.176085+0000","last_change":"2024-05-08T22:05:14.066697+0000","last_active":"2024-05-08T22:07:17.176085+0000","last_peered":"2024-05-08T22:07:17.176085+0000","last_clean":"2024-05-08T22:07:17.176085+0000","last_became_active":"2024-05-08T22:05:14.065952+0000","last_became_peered":"2024-05-08T22:05:14.065952+0000","last_unstale":"2024-05-08T22:07:17.176085+0000","last_undegraded":"2024-05-08T22:07:17.176085+0000","last_fullsized":"2024-05-08T22:07:17.176085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:03:19.006644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.179903+0000","last_change":"2024-05-08T22:05:14.072295+0000","last_active":"2024-05-08T22:07:17.179903+0000","last_peered":"2024-05-08T22:07:17.179903+0000","last_clean":"2024-05-08T22:07:17.179903+0000","last_became_active":"2024-05-08T22:05:14.065034+0000","last_became_peered":"2024-05-08T22:05:14.065034+0000","last_unstale":"2024-05-08T22:07:17.179903+0000","last_undegraded":"2024-05-08T22:07:17.179903+0000","last_fullsized":"2024-05-08T22:07:17.179903+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:08:41.380814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.147802+0000","last_change":"2024-05-08T22:05:14.067263+0000","last_active":"2024-05-08T22:07:17.147802+0000","last_peered":"2024-05-08T22:07:17.147802+0000","last_clean":"2024-05-08T22:07:17.147802+0000","last_became_active":"2024-05-08T22:05:14.066957+0000","last_became_peered":"2024-05-08T22:05:14.066957+0000","last_unstale":"2024-05-08T22:07:17.147802+0000","last_undegraded":"2024-05-08T22:07:17.147802+0000","last_fullsized":"2024-05-08T22:07:17.147802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:29:35.638489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.151853+0000","last_change":"2024-05-08T22:05:14.060561+0000","last_active":"2024-05-08T22:07:17.151853+0000","last_peered":"2024-05-08T22:07:17.151853+0000","last_clean":"2024-05-08T22:07:17.151853+0000","last_became_active":"2024-05-08T22:05:14.060150+0000","last_became_peered":"2024-05-08T22:05:14.060150+0000","last_unstale":"2024-05-08T22:07:17.151853+0000","last_undegraded":"2024-05-08T22:07:17.151853+0000","last_fullsized":"2024-05-08T22:07:17.151853+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:22.710776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.148159+0000","last_change":"2024-05-08T22:05:14.056394+0000","last_active":"2024-05-08T22:07:17.148159+0000","last_peered":"2024-05-08T22:07:17.148159+0000","last_clean":"2024-05-08T22:07:17.148159+0000","last_became_active":"2024-05-08T22:05:14.055977+0000","last_became_peered":"2024-05-08T22:05:14.055977+0000","last_unstale":"2024-05-08T22:07:17.148159+0000","last_undegraded":"2024-05-08T22:07:17.148159+0000","last_fullsized":"2024-05-08T22:07:17.148159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:56:48.249379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.161659+0000","last_change":"2024-05-08T22:06:42.448921+0000","last_active":"2024-05-08T22:07:17.161659+0000","last_peered":"2024-05-08T22:07:17.161659+0000","last_clean":"2024-05-08T22:07:17.161659+0000","last_became_active":"2024-05-08T22:06:42.448567+0000","last_became_peered":"2024-05-08T22:06:42.448567+0000","last_unstale":"2024-05-08T22:07:17.161659+0000","last_undegraded":"2024-05-08T22:07:17.161659+0000","last_fullsized":"2024-05-08T22:07:17.161659+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:52.822511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.180232+0000","last_change":"2024-05-08T22:05:14.069982+0000","last_active":"2024-05-08T22:07:17.180232+0000","last_peered":"2024-05-08T22:07:17.180232+0000","last_clean":"2024-05-08T22:07:17.180232+0000","last_became_active":"2024-05-08T22:05:14.068293+0000","last_became_peered":"2024-05-08T22:05:14.068293+0000","last_unstale":"2024-05-08T22:07:17.180232+0000","last_undegraded":"2024-05-08T22:07:17.180232+0000","last_fullsized":"2024-05-08T22:07:17.180232+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:36.396983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.163286+0000","last_change":"2024-05-08T22:05:14.058393+0000","last_active":"2024-05-08T22:07:17.163286+0000","last_peered":"2024-05-08T22:07:17.163286+0000","last_clean":"2024-05-08T22:07:17.163286+0000","last_became_active":"2024-05-08T22:05:14.058083+0000","last_became_peered":"2024-05-08T22:05:14.058083+0000","last_unstale":"2024-05-08T22:07:17.163286+0000","last_undegraded":"2024-05-08T22:07:17.163286+0000","last_fullsized":"2024-05-08T22:07:17.163286+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:10:47.074957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.166671+0000","last_change":"2024-05-08T22:05:14.056989+0000","last_active":"2024-05-08T22:07:17.166671+0000","last_peered":"2024-05-08T22:07:17.166671+0000","last_clean":"2024-05-08T22:07:17.166671+0000","last_became_active":"2024-05-08T22:05:14.056669+0000","last_became_peered":"2024-05-08T22:05:14.056669+0000","last_unstale":"2024-05-08T22:07:17.166671+0000","last_undegraded":"2024-05-08T22:07:17.166671+0000","last_fullsized":"2024-05-08T22:07:17.166671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:56:51.173885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.175205+0000","last_change":"2024-05-08T22:05:14.058635+0000","last_active":"2024-05-08T22:07:17.175205+0000","last_peered":"2024-05-08T22:07:17.175205+0000","last_clean":"2024-05-08T22:07:17.175205+0000","last_became_active":"2024-05-08T22:05:14.058243+0000","last_became_peered":"2024-05-08T22:05:14.058243+0000","last_unstale":"2024-05-08T22:07:17.175205+0000","last_undegraded":"2024-05-08T22:07:17.175205+0000","last_fullsized":"2024-05-08T22:07:17.175205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:18.142563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.146228+0000","last_change":"2024-05-08T22:05:14.060735+0000","last_active":"2024-05-08T22:07:17.146228+0000","last_peered":"2024-05-08T22:07:17.146228+0000","last_clean":"2024-05-08T22:07:17.146228+0000","last_became_active":"2024-05-08T22:05:14.060391+0000","last_became_peered":"2024-05-08T22:05:14.060391+0000","last_unstale":"2024-05-08T22:07:17.146228+0000","last_undegraded":"2024-05-08T22:07:17.146228+0000","last_fullsized":"2024-05-08T22:07:17.146228+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:30.528705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107988,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599518,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":747,"num_read_kb":483,"num_write":283,"num_write_kb":1441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":694,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1798554,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579940,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36156,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708996,"statfs":{"total":95995035648,"available":95958011904,"internally_reserved":0,"allocated":966656,"data_stored":498801,"data_compressed":26516,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36043205},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.54100000000000004,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.67000000000000004,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.48199999999999998}]},{"osd":1,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.65600000000000003,"15min":0.65300000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.878,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76800000000000002,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.373}]},{"osd":2,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.36299999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.65500000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.309},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.48599999999999999,"15min":0.47799999999999998},"min":{"1min":0.34699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72799999999999998,"5min":1.052,"15min":1.052},"last":0.27500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":4,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.53100000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.433},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.752,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.41999999999999998}]},{"osd":5,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.53200000000000003,"15min":0.52300000000000002},"min":{"1min":0.38,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82999999999999996,"5min":1.204,"15min":1.204},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.61299999999999999,"15min":0.61099999999999999},"min":{"1min":0.42299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86799999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.34899999999999998}]},{"osd":6,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.65800000000000003,"15min":0.66500000000000004},"min":{"1min":0.48099999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91500000000000004,"5min":1.278,"15min":1.278},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.65700000000000003,"15min":0.66000000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.85199999999999998,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.46899999999999997}]},{"osd":7,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.745,"15min":0.74299999999999999},"min":{"1min":0.58999999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.96899999999999997,"5min":1.415,"15min":1.415},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.74099999999999999,"15min":0.746},"min":{"1min":0.60199999999999998,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.86799999999999999,"5min":1.48,"15min":1.48},"last":0.45200000000000001}]},{"osd":8,"last update":"Wed May 8 22:07:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.67000000000000004,"15min":0.67500000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.85699999999999998,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.505},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.73099999999999998,"15min":0.73699999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.99299999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.52700000000000002}]},{"osd":9,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76300000000000001,"15min":0.76400000000000001},"min":{"1min":0.59399999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.97899999999999998,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.65300000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0369999999999999,"5min":1.5,"15min":1.5},"last":0.58399999999999996}]},{"osd":10,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.78900000000000003,"15min":0.79600000000000004},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.96399999999999997,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78100000000000003,"15min":0.78400000000000003},"min":{"1min":0.61899999999999999,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93899999999999995,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.56799999999999995}]}]},{"osd":10,"up_from":67,"seq":287762808871,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1134592,"data_stored":1287551,"data_compressed":68668,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:07:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53900000000000003,"15min":0.54000000000000004},"min":{"1min":0.40100000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.747,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.874,"15min":0.92200000000000004},"min":{"1min":0.44,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.76700000000000002,"5min":7.258,"15min":7.258},"last":0.46800000000000003}]},{"osd":1,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.437,"15min":0.438},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.46700000000000003,"15min":0.46999999999999997},"min":{"1min":0.30599999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.65400000000000003,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.27800000000000002}]},{"osd":2,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.624,"15min":0.61899999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59999999999999998,"15min":0.60199999999999998},"min":{"1min":0.40600000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76000000000000001,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.50900000000000001}]},{"osd":3,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.77500000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.61099999999999999,"15min":0.629},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73499999999999999,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.40999999999999998}]},{"osd":4,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54200000000000004,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76500000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.317},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.315,"5min":0.309,"15min":0.309},"max":{"1min":0.79500000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.32500000000000001}]},{"osd":5,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64600000000000002,"15min":0.64000000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.379},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.65600000000000003,"15min":0.66000000000000003},"min":{"1min":0.47999999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.47699999999999998}]},{"osd":6,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.71699999999999997,"15min":0.67700000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.87,"5min":7.2380000000000004,"15min":7.2380000000000004},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.69599999999999995,"15min":0.69199999999999995},"min":{"1min":0.53500000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.88200000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.45800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":8,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71299999999999997,"15min":0.72499999999999998},"min":{"1min":0.51500000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.372},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.72799999999999998,"15min":0.73399999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.436}]},{"osd":9,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.747,"15min":0.755},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.82499999999999996,"15min":0.78600000000000003},"min":{"1min":0.57499999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.92800000000000005,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.51900000000000002}]},{"osd":11,"last update":"Wed May 8 22:06:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.71399999999999997,"15min":0.70599999999999996},"min":{"1min":0.60099999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73899999999999999,"15min":0.73499999999999999},"min":{"1min":0.61899999999999999,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.49099999999999999}]}]},{"osd":9,"up_from":62,"seq":266287972395,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":896349,"data_compressed":45662,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.50900000000000001,"15min":0.52400000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.60999999999999999,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.57099999999999995,"15min":0.58999999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0409999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.47499999999999998}]},{"osd":1,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.73899999999999999,"15min":0.77100000000000002},"min":{"1min":0.47099999999999997,"5min":0.438,"15min":0.438},"max":{"1min":0.80500000000000005,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.46300000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84699999999999998,"5min":0.997,"15min":0.997},"last":0.65500000000000003}]},{"osd":2,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.75800000000000001,"15min":0.78600000000000003},"min":{"1min":0.56000000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81699999999999995,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.72199999999999998,"15min":0.73999999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.878,"5min":1.167,"15min":1.167},"last":0.63900000000000001}]},{"osd":3,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.73899999999999999,"15min":0.79100000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90300000000000002,"5min":1.024,"15min":1.024},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.60699999999999998},"min":{"1min":0.41499999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.81200000000000006,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.54100000000000004}]},{"osd":4,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.75900000000000001,"15min":0.78300000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96999999999999997,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.80000000000000004,"15min":0.83199999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.86799999999999999,"5min":1.081,"15min":1.081},"last":0.88100000000000001}]},{"osd":5,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.81299999999999994,"15min":0.85499999999999998},"min":{"1min":0.49399999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.93200000000000005,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.84699999999999998,"15min":0.88600000000000001},"min":{"1min":0.53700000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.92400000000000004,"5min":1.175,"15min":1.175},"last":0.85599999999999998}]},{"osd":6,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.71199999999999997}]},{"osd":7,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76300000000000001,"15min":0.78400000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.87,"15min":0.91900000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0109999999999999,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.79700000000000004}]},{"osd":8,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80500000000000005,"15min":0.82899999999999996},"min":{"1min":0.60599999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.181,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.90100000000000002,"15min":0.94299999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2509999999999999,"5min":1.478,"15min":1.478},"last":0.66900000000000004}]},{"osd":10,"last update":"Wed May 8 22:07:22 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.86599999999999999,"15min":0.89000000000000001},"min":{"1min":0.56299999999999994,"5min":0.495,"15min":0.495},"max":{"1min":1.2170000000000001,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.84999999999999998,"15min":0.876},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.95099999999999996,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.89900000000000002}]},{"osd":11,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.85599999999999998,"15min":0.85899999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1990000000000001,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.83999999999999997,"15min":0.84599999999999997},"min":{"1min":0.61499999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.98199999999999998,"5min":1.105,"15min":1.105},"last":0.91700000000000004}]}]},{"osd":8,"up_from":55,"seq":236223201328,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":891842,"data_compressed":45662,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.40300000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81699999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.60399999999999998,"15min":0.626},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69899999999999995,"5min":1.865,"15min":1.865},"last":0.58099999999999996}]},{"osd":1,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.60299999999999998,"15min":0.59399999999999997},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.498},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55700000000000005,"15min":0.55100000000000005},"min":{"1min":0.40799999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.42399999999999999}]},{"osd":2,"last update":"Wed May 8 22:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.437,"15min":0.42299999999999999},"min":{"1min":0.315,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.46999999999999997,"15min":0.46500000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68100000000000005,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.377}]},{"osd":3,"last update":"Wed May 8 22:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.72299999999999998,"15min":0.79300000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77900000000000003,"5min":4.8920000000000003,"15min":4.8920000000000003},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63400000000000001,"15min":0.626},"min":{"1min":0.41999999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.51500000000000001}]},{"osd":4,"last update":"Wed May 8 22:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.46000000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.64000000000000001,"15min":0.61399999999999999},"min":{"1min":0.497,"5min":0.433,"15min":0.433},"max":{"1min":0.871,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.47799999999999998}]},{"osd":5,"last update":"Wed May 8 22:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51200000000000001,"15min":0.50600000000000001},"min":{"1min":0.34399999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.72899999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.54200000000000004,"15min":0.51800000000000002},"min":{"1min":0.39900000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.35699999999999998}]},{"osd":6,"last update":"Wed May 8 22:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.83799999999999997,"15min":0.89400000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.98799999999999999,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.90600000000000003,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.61699999999999999}]},{"osd":7,"last update":"Wed May 8 22:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66400000000000003,"15min":0.66200000000000003},"min":{"1min":0.45000000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.504},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.68600000000000005,"15min":0.67000000000000004},"min":{"1min":0.51100000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.495}]},{"osd":9,"last update":"Wed May 8 22:07:05 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78300000000000003,"15min":0.77600000000000002},"min":{"1min":0.57299999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73299999999999998,"15min":0.73099999999999998},"min":{"1min":0.53600000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92300000000000004,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.56399999999999995}]},{"osd":10,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.71799999999999997,"15min":0.70499999999999996},"min":{"1min":0.55000000000000004,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94699999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.76400000000000001,"15min":0.752},"min":{"1min":0.68400000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.57399999999999995}]},{"osd":11,"last update":"Wed May 8 22:06:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.59399999999999997}]}]},{"osd":7,"up_from":48,"seq":206158430257,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":1096485,"data_compressed":62512,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.41199999999999998}]},{"osd":1,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.38800000000000001},"min":{"1min":0.29699999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.57499999999999996,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40999999999999998,"15min":0.40400000000000003},"min":{"1min":0.27900000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.55400000000000005,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.36399999999999999}]},{"osd":2,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.41599999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71499999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.57599999999999996,"15min":0.56999999999999995},"min":{"1min":0.46200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80400000000000005,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.38200000000000001}]},{"osd":3,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.41299999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76800000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68400000000000005,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.48499999999999999}]},{"osd":4,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.49399999999999999,"15min":0.49299999999999999},"min":{"1min":0.35899999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.505,"15min":0.51400000000000001},"min":{"1min":0.34599999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69499999999999995,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.39300000000000002}]},{"osd":5,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.46800000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77500000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63,"15min":0.628},"min":{"1min":0.495,"5min":0.436,"15min":0.436},"max":{"1min":0.83999999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.438}]},{"osd":6,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65800000000000003,"15min":0.66800000000000004},"min":{"1min":0.504,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.80900000000000005,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.67700000000000005,"15min":0.69299999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84099999999999997,"5min":1.002,"15min":1.002},"last":0.58099999999999996}]},{"osd":8,"last update":"Wed May 8 22:06:39 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.68000000000000005,"15min":0.69199999999999995},"min":{"1min":0.48599999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.82299999999999995,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.499},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71599999999999997,"15min":0.72599999999999998},"min":{"1min":0.54700000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.81799999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.52900000000000003}]},{"osd":9,"last update":"Wed May 8 22:07:05 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.624,"5min":0.501,"15min":0.501},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.73399999999999999,"15min":0.74199999999999999},"min":{"1min":0.59699999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.629}]},{"osd":10,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.69999999999999996,"15min":0.71099999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.82099999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.68500000000000005},"min":{"1min":0.499,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.86799999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.47399999999999998}]},{"osd":11,"last update":"Wed May 8 22:07:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.75,"15min":0.748},"min":{"1min":0.61699999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.59399999999999997}]}]},{"osd":6,"up_from":43,"seq":184683593782,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":493940,"data_compressed":26516,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:07:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.47799999999999998,"15min":0.45600000000000002},"min":{"1min":0.35899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.54000000000000004}]},{"osd":1,"last update":"Wed May 8 22:07:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60099999999999998,"15min":0.57499999999999996},"min":{"1min":0.44800000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.65100000000000002,"15min":0.61699999999999999},"min":{"1min":0.46500000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.73999999999999999}]},{"osd":2,"last update":"Wed May 8 22:07:10 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.69099999999999995,"15min":0.65800000000000003},"min":{"1min":0.51600000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.67400000000000004,"15min":0.65400000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.755}]},{"osd":3,"last update":"Wed May 8 22:07:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.42099999999999999,"5min":0.375,"15min":0.375},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.64900000000000002,"15min":0.64100000000000001},"min":{"1min":0.434,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.81299999999999994}]},{"osd":4,"last update":"Wed May 8 22:07:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.45000000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.53300000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":0.96199999999999997}]},{"osd":5,"last update":"Wed May 8 22:07:10 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.76900000000000002,"15min":0.749},"min":{"1min":0.56499999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.753,"15min":0.75},"min":{"1min":0.59399999999999997,"5min":0.438,"15min":0.438},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.044}]},{"osd":7,"last update":"Wed May 8 22:06:28 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.747,"15min":0.73999999999999999},"min":{"1min":0.54900000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.9690000000000001,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.77700000000000002,"15min":0.76500000000000001},"min":{"1min":0.63200000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.86599999999999999}]},{"osd":8,"last update":"Wed May 8 22:06:38 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.83799999999999997,"15min":0.84199999999999997},"min":{"1min":0.58199999999999996,"5min":0.496,"15min":0.496},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.55200000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.097}]},{"osd":9,"last update":"Wed May 8 22:07:04 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.85699999999999998,"15min":0.85899999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.81699999999999995,"15min":0.83499999999999996},"min":{"1min":0.51000000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.1299999999999999}]},{"osd":10,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.85799999999999998,"15min":0.86799999999999999},"min":{"1min":0.66600000000000004,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":0.95899999999999996,"5min":1.639,"15min":1.639},"last":1.165},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.82799999999999996,"15min":0.81999999999999995},"min":{"1min":0.60699999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0369999999999999,"5min":1.927,"15min":1.927},"last":1.1140000000000001}]},{"osd":11,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.91700000000000004,"15min":0.92900000000000005},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.127,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.88500000000000001,"15min":0.89800000000000002},"min":{"1min":0.68000000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.022,"5min":1.78,"15min":1.78},"last":1.0029999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":1036288,"data_stored":560994,"data_compressed":26516,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:06:31 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.55100000000000005,"15min":0.501},"min":{"1min":0.40400000000000003,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.85699999999999998,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55200000000000005,"15min":0.51800000000000002},"min":{"1min":0.44900000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76000000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.5}]},{"osd":2,"last update":"Wed May 8 22:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58099999999999996,"15min":0.50800000000000001},"min":{"1min":0.45400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.77400000000000002,"5min":0.877,"15min":0.877},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.56899999999999995,"15min":0.505},"min":{"1min":0.39300000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76200000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.36899999999999999}]},{"osd":3,"last update":"Wed May 8 22:07:08 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.57899999999999996,"15min":0.52000000000000002},"min":{"1min":0.495,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70799999999999996,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55000000000000004,"15min":0.51800000000000002},"min":{"1min":0.41799999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71099999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.44500000000000001}]},{"osd":4,"last update":"Wed May 8 22:06:31 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.42399999999999999,"15min":0.38400000000000001},"min":{"1min":0.32500000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.80600000000000005,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.45600000000000002,"15min":0.40300000000000002},"min":{"1min":0.35599999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.77600000000000002,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.52900000000000003}]},{"osd":5,"last update":"Wed May 8 22:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.622,"15min":0.56799999999999995},"min":{"1min":0.53600000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.84199999999999997,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.61799999999999999,"15min":0.58399999999999996},"min":{"1min":0.46800000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80000000000000004,"5min":1.089,"15min":1.089},"last":0.46100000000000002}]},{"osd":6,"last update":"Wed May 8 22:07:08 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.626,"15min":0.60599999999999998},"min":{"1min":0.47999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66600000000000004,"15min":0.628},"min":{"1min":0.47899999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.93400000000000005,"5min":1.05,"15min":1.05},"last":0.47399999999999998}]},{"osd":7,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58999999999999997,"15min":0.57199999999999995},"min":{"1min":0.44400000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.871,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.39100000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74199999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.34200000000000003}]},{"osd":8,"last update":"Wed May 8 22:06:42 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65200000000000002,"15min":0.65900000000000003},"min":{"1min":0.53600000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.78300000000000003,"5min":1.115,"15min":1.115},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.53600000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.92100000000000004,"5min":1.222,"15min":1.222},"last":0.53700000000000003}]},{"osd":9,"last update":"Wed May 8 22:07:04 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.70899999999999996,"15min":0.71899999999999997},"min":{"1min":0.54400000000000004,"5min":0.501,"15min":0.501},"max":{"1min":0.86699999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80100000000000005,"15min":0.79600000000000004},"min":{"1min":0.71199999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0860000000000001,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.59899999999999998}]},{"osd":10,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.76200000000000001,"15min":0.76400000000000001},"min":{"1min":0.57199999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0389999999999999,"5min":1.179,"15min":1.179},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.56699999999999995,"5min":0.435,"15min":0.435},"max":{"1min":0.94899999999999995,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.64100000000000001}]},{"osd":11,"last update":"Wed May 8 22:07:14 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.83599999999999997,"15min":0.84099999999999997},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.78000000000000003,"15min":0.78300000000000003},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.98699999999999999,"5min":0.996,"15min":0.996},"last":0.69299999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":983040,"data_stored":495328,"data_compressed":26516,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:06:29 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.60699999999999998,"15min":0.55800000000000005},"min":{"1min":0.45200000000000001,"5min":0.31,"15min":0.31},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.58399999999999996,"15min":0.53400000000000003},"min":{"1min":0.41899999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.55700000000000005}]},{"osd":2,"last update":"Wed May 8 22:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.64100000000000001,"15min":0.56599999999999995},"min":{"1min":0.45100000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61699999999999999,"15min":0.55900000000000005},"min":{"1min":0.53900000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86399999999999999,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.625}]},{"osd":3,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53600000000000003,"15min":0.53100000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66800000000000004,"15min":0.66400000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.78300000000000003}]},{"osd":4,"last update":"Wed May 8 22:06:29 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.59899999999999998,"15min":0.55800000000000005},"min":{"1min":0.52400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95999999999999996,"5min":1.218,"15min":1.218},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.70399999999999996,"15min":0.69899999999999995},"min":{"1min":0.56699999999999995,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94599999999999995,"5min":2.984,"15min":2.984},"last":0.84799999999999998}]},{"osd":5,"last update":"Wed May 8 22:06:46 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.70599999999999996,"15min":0.65600000000000003},"min":{"1min":0.501,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0760000000000001,"5min":1.129,"15min":1.129},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68400000000000005,"15min":0.66600000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.69499999999999995}]},{"osd":6,"last update":"Wed May 8 22:07:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.625,"15min":0.58899999999999997},"min":{"1min":0.40300000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90300000000000002,"5min":1.254,"15min":1.254},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70999999999999996,"15min":0.67700000000000005},"min":{"1min":0.443,"5min":0.375,"15min":0.375},"max":{"1min":1.0089999999999999,"5min":1.208,"15min":1.208},"last":0.92500000000000004}]},{"osd":7,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.72699999999999998}]},{"osd":8,"last update":"Wed May 8 22:06:41 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.83199999999999996,"15min":0.82899999999999996},"min":{"1min":0.65000000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1080000000000001,"5min":1.427,"15min":1.427},"last":1.024},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.80200000000000005,"15min":0.80500000000000005},"min":{"1min":0.64800000000000002,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.76200000000000001}]},{"osd":9,"last update":"Wed May 8 22:07:07 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78900000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.875,"15min":0.89200000000000002},"min":{"1min":0.47999999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1379999999999999,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.089}]},{"osd":10,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.61599999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1339999999999999,"5min":1.2,"15min":1.2},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.876,"15min":0.879},"min":{"1min":0.69599999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.77200000000000002}]},{"osd":11,"last update":"Wed May 8 22:06:41 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.88500000000000001,"15min":0.874},"min":{"1min":0.76100000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":1.056},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.86899999999999999,"15min":0.86099999999999999},"min":{"1min":0.70499999999999996,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.80700000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":1040384,"data_stored":561278,"data_compressed":26516,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.63100000000000001,"15min":0.55100000000000005},"min":{"1min":0.44900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":6.4640000000000004,"5min":6.4640000000000004,"15min":6.4640000000000004},"last":0.503},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.54300000000000004,"15min":0.51500000000000001},"min":{"1min":0.46600000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.47299999999999998}]},{"osd":1,"last update":"Wed May 8 22:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.56599999999999995,"15min":0.53800000000000003},"min":{"1min":0.42699999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.60599999999999998,"15min":0.56499999999999995},"min":{"1min":0.38400000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.81200000000000006}]},{"osd":3,"last update":"Wed May 8 22:07:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.59899999999999998,"15min":0.56699999999999995},"min":{"1min":0.50700000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.90100000000000002,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59699999999999998,"15min":0.56899999999999995},"min":{"1min":0.44900000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67200000000000004}]},{"osd":4,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.755,"15min":0.76900000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.70199999999999996,"15min":0.71199999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.77200000000000002}]},{"osd":5,"last update":"Wed May 8 22:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.51100000000000001,"15min":0.45800000000000002},"min":{"1min":0.41099999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80400000000000005,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.496,"15min":0.45800000000000002},"min":{"1min":0.435,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.52900000000000003}]},{"osd":6,"last update":"Wed May 8 22:07:09 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.53400000000000003,"5min":0.439,"15min":0.439},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.70999999999999996,"15min":0.68200000000000005},"min":{"1min":0.54800000000000004,"5min":0.442,"15min":0.442},"max":{"1min":0.98899999999999999,"5min":0.998,"15min":0.998},"last":0.63700000000000001}]},{"osd":7,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.875},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.84599999999999997}]},{"osd":8,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.63500000000000001,"15min":0.59299999999999997},"min":{"1min":0.505,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.626,"15min":0.58399999999999996},"min":{"1min":0.54100000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90500000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.78900000000000003}]},{"osd":9,"last update":"Wed May 8 22:07:05 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.77300000000000002,"15min":0.72699999999999998},"min":{"1min":0.73099999999999998,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.024,"5min":1.101,"15min":1.101},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.76100000000000001,"15min":0.71899999999999997},"min":{"1min":0.71099999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.996,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.80200000000000005}]},{"osd":10,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.73899999999999999,"15min":0.71299999999999997},"min":{"1min":0.65700000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.72199999999999998,"15min":0.70399999999999996},"min":{"1min":0.66800000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.83699999999999997}]},{"osd":11,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.86199999999999999,"15min":0.871},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.81200000000000006,"15min":0.82199999999999995},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.95199999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.754}]}]},{"osd":3,"up_from":24,"seq":103079215169,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":1019223,"data_compressed":51818,"data_compressed_allocated":69632,"data_compressed_original":589824,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:07:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.49099999999999999,"15min":0.41699999999999998},"min":{"1min":0.33300000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.89000000000000001,"5min":3.2530000000000001,"15min":3.2530000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.52800000000000002,"15min":0.436},"min":{"1min":0.30299999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.1619999999999999,"5min":3.4460000000000002,"15min":3.4460000000000002},"last":0.58199999999999996}]},{"osd":1,"last update":"Wed May 8 22:07:14 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.68100000000000005,"15min":0.58499999999999996},"min":{"1min":0.41399999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79300000000000004,"5min":3.7090000000000001,"15min":3.7090000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66200000000000003,"15min":0.56200000000000006},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.3899999999999999,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":0.55800000000000005}]},{"osd":2,"last update":"Wed May 8 22:07:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71099999999999997,"15min":0.621},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.258,"5min":3.577,"15min":3.577},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.71499999999999997,"15min":0.63500000000000001},"min":{"1min":0.45600000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97299999999999998,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.45800000000000002}]},{"osd":4,"last update":"Wed May 8 22:06:25 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.67700000000000005,"15min":0.61299999999999999},"min":{"1min":0.46600000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":3.8109999999999999,"5min":3.8109999999999999,"15min":3.8109999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.72199999999999998,"15min":0.64000000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.847,"5min":3.847,"15min":3.847},"last":0.78900000000000003}]},{"osd":5,"last update":"Wed May 8 22:06:44 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.73799999999999999,"15min":0.66500000000000004},"min":{"1min":0.47999999999999998,"5min":0.433,"15min":0.433},"max":{"1min":3.9260000000000002,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.70699999999999996,"15min":0.63},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.54,"5min":3.54,"15min":3.54},"last":0.628}]},{"osd":6,"last update":"Wed May 8 22:07:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69599999999999995,"15min":0.63700000000000001},"min":{"1min":0.44600000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.157,"5min":3.653,"15min":3.653},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72499999999999998,"15min":0.65000000000000002},"min":{"1min":0.47599999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96299999999999997,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.495}]},{"osd":7,"last update":"Wed May 8 22:06:25 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.73999999999999999,"15min":0.72299999999999998},"min":{"1min":0.55500000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.86399999999999999,"15min":0.84699999999999998},"min":{"1min":0.57599999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":3.887,"5min":3.887,"15min":3.887},"last":0.71799999999999997}]},{"osd":8,"last update":"Wed May 8 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.53100000000000003}]},{"osd":9,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.88600000000000001,"15min":0.87},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.179,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.873},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.80400000000000005,"15min":0.79600000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.454,"5min":3.3940000000000001,"15min":3.3940000000000001},"last":0.80800000000000005}]},{"osd":10,"last update":"Wed May 8 22:07:22 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.84699999999999998,"15min":0.83699999999999997},"min":{"1min":0.60599999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4850000000000001,"5min":3.298,"15min":3.298},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.871,"15min":0.85999999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1240000000000001,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":0.70099999999999996}]},{"osd":11,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.99399999999999999,"15min":1.032},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.204,"5min":3.6150000000000002,"15min":3.6150000000000002},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.82999999999999996,"5min":1.034,"15min":1.0680000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.014,"5min":3.9950000000000001,"15min":3.9950000000000001},"last":0.84099999999999997}]}]},{"osd":4,"up_from":31,"seq":133143986239,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":500106,"data_compressed":26516,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.44700000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.51900000000000002,"15min":0.50900000000000001},"min":{"1min":0.45000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.629}]},{"osd":1,"last update":"Wed May 8 22:06:28 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.39100000000000001,"15min":0.35699999999999998},"min":{"1min":0.33500000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.40000000000000002,"15min":0.36199999999999999},"min":{"1min":0.36299999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.24299999999999999}]},{"osd":2,"last update":"Wed May 8 22:06:28 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.53600000000000003,"15min":0.51100000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82099999999999995,"5min":0.88,"15min":0.88},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54800000000000004,"15min":0.51400000000000001},"min":{"1min":0.436,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.52200000000000002}]},{"osd":3,"last update":"Wed May 8 22:06:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.56699999999999995},"min":{"1min":0.46000000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.70499999999999996,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57299999999999995,"15min":0.55500000000000005},"min":{"1min":0.439,"5min":0.377,"15min":0.377},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.51200000000000001}]},{"osd":5,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57399999999999995,"15min":0.54000000000000004},"min":{"1min":0.49299999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77400000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59399999999999997,"15min":0.55700000000000005},"min":{"1min":0.48399999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.80200000000000005,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.53800000000000003}]},{"osd":6,"last update":"Wed May 8 22:07:07 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.65600000000000003,"15min":0.63800000000000001},"min":{"1min":0.49299999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.75600000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.628,"15min":0.61599999999999999},"min":{"1min":0.45700000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.38900000000000001}]},{"osd":7,"last update":"Wed May 8 22:06:28 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57999999999999996,"15min":0.58599999999999997},"min":{"1min":0.45900000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.46200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.503}]},{"osd":8,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.69199999999999995,"15min":0.68799999999999994},"min":{"1min":0.58699999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93000000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.60199999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.874,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.56899999999999995}]},{"osd":9,"last update":"Wed May 8 22:07:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69499999999999995,"15min":0.69799999999999995},"min":{"1min":0.53500000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.749,"15min":0.755},"min":{"1min":0.61699999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.48699999999999999}]},{"osd":10,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.73599999999999999,"15min":0.73999999999999999},"min":{"1min":0.55800000000000005,"5min":0.434,"15min":0.434},"max":{"1min":0.84799999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.627,"15min":0.625},"min":{"1min":0.50600000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.34300000000000003}]},{"osd":11,"last update":"Wed May 8 22:07:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.75,"15min":0.747},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.90200000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.747,"15min":0.747},"min":{"1min":0.60699999999999998,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":0.93400000000000005,"5min":1.052,"15min":1.052},"last":0.54800000000000004}]}]},{"osd":5,"up_from":36,"seq":154618822715,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":1060864,"data_stored":1222542,"data_compressed":68668,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.41799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.55000000000000004,"15min":0.53800000000000003},"min":{"1min":0.40000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70699999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.40600000000000003}]},{"osd":1,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.56599999999999995,"15min":0.54000000000000004},"min":{"1min":0.28799999999999998,"5min":0.248,"15min":0.248},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.55500000000000005,"15min":0.52600000000000002},"min":{"1min":0.34399999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.66800000000000004}]},{"osd":2,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.40200000000000002,"15min":0.376},"min":{"1min":0.33200000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.41499999999999998,"15min":0.39200000000000002},"min":{"1min":0.35499999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67100000000000004,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.38}]},{"osd":3,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.46500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.78100000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.58699999999999997,"15min":0.55900000000000005},"min":{"1min":0.51000000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.62}]},{"osd":4,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.378,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.58199999999999996,"15min":0.56100000000000005},"min":{"1min":0.48399999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.65000000000000002}]},{"osd":6,"last update":"Wed May 8 22:07:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61099999999999999,"15min":0.59299999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61899999999999999,"15min":0.57299999999999995},"min":{"1min":0.505,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48899999999999999}]},{"osd":7,"last update":"Wed May 8 22:06:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.60899999999999999,"15min":0.57699999999999996},"min":{"1min":0.54500000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.61799999999999999,"15min":0.58699999999999997},"min":{"1min":0.51100000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.70099999999999996}]},{"osd":8,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.54000000000000004,"15min":0.50600000000000001},"min":{"1min":0.44600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.624,"15min":0.58199999999999996},"min":{"1min":0.5,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.78300000000000003}]},{"osd":9,"last update":"Wed May 8 22:07:03 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.73299999999999998,"15min":0.70899999999999996},"min":{"1min":0.57899999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.749},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.72199999999999998,"15min":0.69799999999999995},"min":{"1min":0.623,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.63400000000000001}]},{"osd":10,"last update":"Wed May 8 22:06:25 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.752,"15min":0.74299999999999999},"min":{"1min":0.65900000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.74199999999999999,"15min":0.73299999999999998},"min":{"1min":0.66100000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.98599999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.72899999999999998}]},{"osd":11,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":69664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133955,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":203391,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:07:25.877 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-08T22:07:25.877 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:07:26.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.100+0000 7f3f9be00700 1 -- 172.21.15.131:0/3905888517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f94101a00 msgr2=0x7f3f94105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:26.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.100+0000 7f3f9be00700 1 --2- 172.21.15.131:0/3905888517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f94101a00 0x7f3f94105a40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009990 tx=0x7f3f80009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:26.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.100+0000 7f3f9be00700 1 -- 172.21.15.131:0/3905888517 shutdown_connections 2024-05-08T22:07:26.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.100+0000 7f3f9be00700 1 --2- 172.21.15.131:0/3905888517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f94106170 0x7f3f9410dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:26.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.100+0000 7f3f9be00700 1 --2- 172.21.15.131:0/3905888517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f94101050 0x7f3f94101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:26.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.100+0000 7f3f9be00700 1 --2- 172.21.15.131:0/3905888517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f94101a00 0x7f3f94105a40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:26.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.100+0000 7f3f9be00700 1 -- 172.21.15.131:0/3905888517 >> 172.21.15.131:0/3905888517 conn(0x7f3f940fc8f0 msgr2=0x7f3f940fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:26.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.100+0000 7f3f9be00700 1 -- 172.21.15.131:0/3905888517 shutdown_connections 2024-05-08T22:07:26.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.100+0000 7f3f9be00700 1 -- 172.21.15.131:0/3905888517 wait complete. 2024-05-08T22:07:26.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.102+0000 7f3f9be00700 1 Processor -- start 2024-05-08T22:07:26.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.102+0000 7f3f9be00700 1 -- start start 2024-05-08T22:07:26.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.102+0000 7f3f9be00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f94101050 0x7f3f940732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:26.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.102+0000 7f3f9be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f94106170 0x7f3f940737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:26.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.103+0000 7f3f9be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f94073d20 0x7f3f941a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:26.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.103+0000 7f3f9be00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f941a5b70 con 0x7f3f94106170 2024-05-08T22:07:26.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.103+0000 7f3f9be00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f941a5cb0 con 0x7f3f94101050 2024-05-08T22:07:26.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.103+0000 7f3f9be00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f941a5e70 con 0x7f3f94073d20 2024-05-08T22:07:26.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.103+0000 7f3f98a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f94106170 0x7f3f940737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:26.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.103+0000 7f3f98a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f94106170 0x7f3f940737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50732/0 (socket says 172.21.15.131:50732) 2024-05-08T22:07:26.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.103+0000 7f3f98a00700 1 -- 172.21.15.131:0/2220513075 learned_addr learned my addr 172.21.15.131:0/2220513075 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:26.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.103+0000 7f3f99e00700 1 --2- 172.21.15.131:0/2220513075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f94073d20 0x7f3f941a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:26.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.103+0000 7f3f99400700 1 --2- 172.21.15.131:0/2220513075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f94101050 0x7f3f940732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:26.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.103+0000 7f3f98a00700 1 -- 172.21.15.131:0/2220513075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f94073d20 msgr2=0x7f3f941a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:26.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.103+0000 7f3f98a00700 1 --2- 172.21.15.131:0/2220513075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f94073d20 0x7f3f941a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:26.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.103+0000 7f3f98a00700 1 -- 172.21.15.131:0/2220513075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f94101050 msgr2=0x7f3f940732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:26.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.103+0000 7f3f98a00700 1 --2- 172.21.15.131:0/2220513075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f94101050 0x7f3f940732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:26.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.103+0000 7f3f98a00700 1 -- 172.21.15.131:0/2220513075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8c009c40 con 0x7f3f94106170 2024-05-08T22:07:26.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.103+0000 7f3f99e00700 1 --2- 172.21.15.131:0/2220513075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f94073d20 0x7f3f941a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:26.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.103+0000 7f3f98a00700 1 --2- 172.21.15.131:0/2220513075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f94106170 0x7f3f940737e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3f80000c00 tx=0x7f3f80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:26.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.104+0000 7f3f92000700 1 -- 172.21.15.131:0/2220513075 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8001d070 con 0x7f3f94106170 2024-05-08T22:07:26.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.104+0000 7f3f9be00700 1 -- 172.21.15.131:0/2220513075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f800095f0 con 0x7f3f94106170 2024-05-08T22:07:26.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.104+0000 7f3f92000700 1 -- 172.21.15.131:0/2220513075 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f80021410 con 0x7f3f94106170 2024-05-08T22:07:26.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.104+0000 7f3f9be00700 1 -- 172.21.15.131:0/2220513075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f941a63e0 con 0x7f3f94106170 2024-05-08T22:07:26.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.104+0000 7f3f92000700 1 -- 172.21.15.131:0/2220513075 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f80005220 con 0x7f3f94106170 2024-05-08T22:07:26.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.105+0000 7f3f9be00700 1 -- 172.21.15.131:0/2220513075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f94109440 con 0x7f3f94106170 2024-05-08T22:07:26.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.107+0000 7f3f92000700 1 -- 172.21.15.131:0/2220513075 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7f3f80021580 con 0x7f3f94106170 2024-05-08T22:07:26.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.108+0000 7f3f92000700 1 --2- 172.21.15.131:0/2220513075 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3f7c0a13b0 0x7f3f7c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:26.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.108+0000 7f3f92000700 1 -- 172.21.15.131:0/2220513075 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f3f800ce470 con 0x7f3f94106170 2024-05-08T22:07:26.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.108+0000 7f3f99400700 1 --2- 172.21.15.131:0/2220513075 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3f7c0a13b0 0x7f3f7c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:26.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.109+0000 7f3f99400700 1 --2- 172.21.15.131:0/2220513075 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3f7c0a13b0 0x7f3f7c0a3870 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009c60 tx=0x7f3f88009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:26.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.112+0000 7f3f92000700 1 -- 172.21.15.131:0/2220513075 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3f80097310 con 0x7f3f94106170 2024-05-08T22:07:26.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.314+0000 7f3f9be00700 1 -- 172.21.15.131:0/2220513075 --> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3f94074730 con 0x7f3f7c0a13b0 2024-05-08T22:07:26.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.327+0000 7f3f92000700 1 -- 172.21.15.131:0/2220513075 <== mgr.15366 v2:172.21.15.131:6800/1157588501 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426525 (secure 0 0 0) 0x7f3f94074730 con 0x7f3f7c0a13b0 2024-05-08T22:07:26.334 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:07:26.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.332+0000 7f3f9be00700 1 -- 172.21.15.131:0/2220513075 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3f7c0a13b0 msgr2=0x7f3f7c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:26.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.332+0000 7f3f9be00700 1 --2- 172.21.15.131:0/2220513075 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3f7c0a13b0 0x7f3f7c0a3870 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009c60 tx=0x7f3f88009260 comp rx=0 tx=0).stop 2024-05-08T22:07:26.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.332+0000 7f3f9be00700 1 -- 172.21.15.131:0/2220513075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f94106170 msgr2=0x7f3f940737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:26.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.332+0000 7f3f9be00700 1 --2- 172.21.15.131:0/2220513075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f94106170 0x7f3f940737e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3f80000c00 tx=0x7f3f80004730 comp rx=0 tx=0).stop 2024-05-08T22:07:26.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.332+0000 7f3f9be00700 1 -- 172.21.15.131:0/2220513075 shutdown_connections 2024-05-08T22:07:26.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.332+0000 7f3f9be00700 1 --2- 172.21.15.131:0/2220513075 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3f7c0a13b0 0x7f3f7c0a3870 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:26.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.332+0000 7f3f9be00700 1 --2- 172.21.15.131:0/2220513075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f94073d20 0x7f3f941a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:26.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.332+0000 7f3f9be00700 1 --2- 172.21.15.131:0/2220513075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f94101050 0x7f3f940732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:26.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.332+0000 7f3f9be00700 1 --2- 172.21.15.131:0/2220513075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f94106170 0x7f3f940737e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:26.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.332+0000 7f3f9be00700 1 -- 172.21.15.131:0/2220513075 >> 172.21.15.131:0/2220513075 conn(0x7f3f940fc8f0 msgr2=0x7f3f940fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:26.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.332+0000 7f3f9be00700 1 -- 172.21.15.131:0/2220513075 shutdown_connections 2024-05-08T22:07:26.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.332+0000 7f3f9be00700 1 -- 172.21.15.131:0/2220513075 wait complete. 2024-05-08T22:07:26.339 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-08T22:07:26.352 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-05-08T22:07:25.114979+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190120,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":984,"num_read_kb":813,"num_write":486,"num_write_kb":4381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181430,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":875,"ondisk_log_size":875,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373032,"kb_used_data":11944,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124568792,"statfs":{"total":1151940427776,"available":1151558443008,"internally_reserved":0,"allocated":12230656,"data_stored":9524439,"data_compressed":502086,"data_compressed_allocated":675840,"data_compressed_original":4202496,"omap_allocated":19131,"internal_metadata":369603909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3699,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":134,"num_read_kb":229,"num_write":28,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.919574"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.174991+0000","last_change":"2024-05-08T22:05:14.061628+0000","last_active":"2024-05-08T22:07:17.174991+0000","last_peered":"2024-05-08T22:07:17.174991+0000","last_clean":"2024-05-08T22:07:17.174991+0000","last_became_active":"2024-05-08T22:05:14.060986+0000","last_became_peered":"2024-05-08T22:05:14.060986+0000","last_unstale":"2024-05-08T22:07:17.174991+0000","last_undegraded":"2024-05-08T22:07:17.174991+0000","last_fullsized":"2024-05-08T22:07:17.174991+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:51.643118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.174788+0000","last_change":"2024-05-08T22:05:14.073300+0000","last_active":"2024-05-08T22:07:17.174788+0000","last_peered":"2024-05-08T22:07:17.174788+0000","last_clean":"2024-05-08T22:07:17.174788+0000","last_became_active":"2024-05-08T22:05:14.069058+0000","last_became_peered":"2024-05-08T22:05:14.069058+0000","last_unstale":"2024-05-08T22:07:17.174788+0000","last_undegraded":"2024-05-08T22:07:17.174788+0000","last_fullsized":"2024-05-08T22:07:17.174788+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:15.834956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.175346+0000","last_change":"2024-05-08T22:05:14.071676+0000","last_active":"2024-05-08T22:07:17.175346+0000","last_peered":"2024-05-08T22:07:17.175346+0000","last_clean":"2024-05-08T22:07:17.175346+0000","last_became_active":"2024-05-08T22:05:14.058345+0000","last_became_peered":"2024-05-08T22:05:14.058345+0000","last_unstale":"2024-05-08T22:07:17.175346+0000","last_undegraded":"2024-05-08T22:07:17.175346+0000","last_fullsized":"2024-05-08T22:07:17.175346+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:09.239058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.155221+0000","last_change":"2024-05-08T22:05:14.067170+0000","last_active":"2024-05-08T22:07:17.155221+0000","last_peered":"2024-05-08T22:07:17.155221+0000","last_clean":"2024-05-08T22:07:17.155221+0000","last_became_active":"2024-05-08T22:05:14.066850+0000","last_became_peered":"2024-05-08T22:05:14.066850+0000","last_unstale":"2024-05-08T22:07:17.155221+0000","last_undegraded":"2024-05-08T22:07:17.155221+0000","last_fullsized":"2024-05-08T22:07:17.155221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:28.439786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.155731+0000","last_change":"2024-05-08T22:05:14.065177+0000","last_active":"2024-05-08T22:07:17.155731+0000","last_peered":"2024-05-08T22:07:17.155731+0000","last_clean":"2024-05-08T22:07:17.155731+0000","last_became_active":"2024-05-08T22:05:14.064833+0000","last_became_peered":"2024-05-08T22:05:14.064833+0000","last_unstale":"2024-05-08T22:07:17.155731+0000","last_undegraded":"2024-05-08T22:07:17.155731+0000","last_fullsized":"2024-05-08T22:07:17.155731+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:11.764154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.179822+0000","last_change":"2024-05-08T22:05:14.061610+0000","last_active":"2024-05-08T22:07:17.179822+0000","last_peered":"2024-05-08T22:07:17.179822+0000","last_clean":"2024-05-08T22:07:17.179822+0000","last_became_active":"2024-05-08T22:05:14.061277+0000","last_became_peered":"2024-05-08T22:05:14.061277+0000","last_unstale":"2024-05-08T22:07:17.179822+0000","last_undegraded":"2024-05-08T22:07:17.179822+0000","last_fullsized":"2024-05-08T22:07:17.179822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:19.154980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.176639+0000","last_change":"2024-05-08T22:05:14.063941+0000","last_active":"2024-05-08T22:07:17.176639+0000","last_peered":"2024-05-08T22:07:17.176639+0000","last_clean":"2024-05-08T22:07:17.176639+0000","last_became_active":"2024-05-08T22:05:14.063579+0000","last_became_peered":"2024-05-08T22:05:14.063579+0000","last_unstale":"2024-05-08T22:07:17.176639+0000","last_undegraded":"2024-05-08T22:07:17.176639+0000","last_fullsized":"2024-05-08T22:07:17.176639+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:27.679067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.179515+0000","last_change":"2024-05-08T22:05:14.060366+0000","last_active":"2024-05-08T22:07:17.179515+0000","last_peered":"2024-05-08T22:07:17.179515+0000","last_clean":"2024-05-08T22:07:17.179515+0000","last_became_active":"2024-05-08T22:05:14.059818+0000","last_became_peered":"2024-05-08T22:05:14.059818+0000","last_unstale":"2024-05-08T22:07:17.179515+0000","last_undegraded":"2024-05-08T22:07:17.179515+0000","last_fullsized":"2024-05-08T22:07:17.179515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:53.118508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.154256+0000","last_change":"2024-05-08T22:05:14.057185+0000","last_active":"2024-05-08T22:07:17.154256+0000","last_peered":"2024-05-08T22:07:17.154256+0000","last_clean":"2024-05-08T22:07:17.154256+0000","last_became_active":"2024-05-08T22:05:14.056889+0000","last_became_peered":"2024-05-08T22:05:14.056889+0000","last_unstale":"2024-05-08T22:07:17.154256+0000","last_undegraded":"2024-05-08T22:07:17.154256+0000","last_fullsized":"2024-05-08T22:07:17.154256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:35.372705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.146313+0000","last_change":"2024-05-08T22:05:14.062282+0000","last_active":"2024-05-08T22:07:17.146313+0000","last_peered":"2024-05-08T22:07:17.146313+0000","last_clean":"2024-05-08T22:07:17.146313+0000","last_became_active":"2024-05-08T22:05:14.061574+0000","last_became_peered":"2024-05-08T22:05:14.061574+0000","last_unstale":"2024-05-08T22:07:17.146313+0000","last_undegraded":"2024-05-08T22:07:17.146313+0000","last_fullsized":"2024-05-08T22:07:17.146313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:16:32.873590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.153206+0000","last_change":"2024-05-08T22:05:14.061267+0000","last_active":"2024-05-08T22:07:17.153206+0000","last_peered":"2024-05-08T22:07:17.153206+0000","last_clean":"2024-05-08T22:07:17.153206+0000","last_became_active":"2024-05-08T22:05:14.060943+0000","last_became_peered":"2024-05-08T22:05:14.060943+0000","last_unstale":"2024-05-08T22:07:17.153206+0000","last_undegraded":"2024-05-08T22:07:17.153206+0000","last_fullsized":"2024-05-08T22:07:17.153206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:03.831138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.155172+0000","last_change":"2024-05-08T22:05:14.067136+0000","last_active":"2024-05-08T22:07:17.155172+0000","last_peered":"2024-05-08T22:07:17.155172+0000","last_clean":"2024-05-08T22:07:17.155172+0000","last_became_active":"2024-05-08T22:05:14.066810+0000","last_became_peered":"2024-05-08T22:05:14.066810+0000","last_unstale":"2024-05-08T22:07:17.155172+0000","last_undegraded":"2024-05-08T22:07:17.155172+0000","last_fullsized":"2024-05-08T22:07:17.155172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:43.001024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.157400+0000","last_change":"2024-05-08T22:05:14.062119+0000","last_active":"2024-05-08T22:07:17.157400+0000","last_peered":"2024-05-08T22:07:17.157400+0000","last_clean":"2024-05-08T22:07:17.157400+0000","last_became_active":"2024-05-08T22:05:14.058977+0000","last_became_peered":"2024-05-08T22:05:14.058977+0000","last_unstale":"2024-05-08T22:07:17.157400+0000","last_undegraded":"2024-05-08T22:07:17.157400+0000","last_fullsized":"2024-05-08T22:07:17.157400+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:03.657985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.163070+0000","last_change":"2024-05-08T22:05:14.067363+0000","last_active":"2024-05-08T22:07:17.163070+0000","last_peered":"2024-05-08T22:07:17.163070+0000","last_clean":"2024-05-08T22:07:17.163070+0000","last_became_active":"2024-05-08T22:05:14.066119+0000","last_became_peered":"2024-05-08T22:05:14.066119+0000","last_unstale":"2024-05-08T22:07:17.163070+0000","last_undegraded":"2024-05-08T22:07:17.163070+0000","last_fullsized":"2024-05-08T22:07:17.163070+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:43.426750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.174135+0000","last_change":"2024-05-08T22:05:14.073925+0000","last_active":"2024-05-08T22:07:17.174135+0000","last_peered":"2024-05-08T22:07:17.174135+0000","last_clean":"2024-05-08T22:07:17.174135+0000","last_became_active":"2024-05-08T22:05:14.072287+0000","last_became_peered":"2024-05-08T22:05:14.072287+0000","last_unstale":"2024-05-08T22:07:17.174135+0000","last_undegraded":"2024-05-08T22:07:17.174135+0000","last_fullsized":"2024-05-08T22:07:17.174135+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:13.537938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.172678+0000","last_change":"2024-05-08T22:05:14.057274+0000","last_active":"2024-05-08T22:07:17.172678+0000","last_peered":"2024-05-08T22:07:17.172678+0000","last_clean":"2024-05-08T22:07:17.172678+0000","last_became_active":"2024-05-08T22:05:14.056989+0000","last_became_peered":"2024-05-08T22:05:14.056989+0000","last_unstale":"2024-05-08T22:07:17.172678+0000","last_undegraded":"2024-05-08T22:07:17.172678+0000","last_fullsized":"2024-05-08T22:07:17.172678+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:32.316406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.174918+0000","last_change":"2024-05-08T22:05:14.069263+0000","last_active":"2024-05-08T22:07:17.174918+0000","last_peered":"2024-05-08T22:07:17.174918+0000","last_clean":"2024-05-08T22:07:17.174918+0000","last_became_active":"2024-05-08T22:05:14.068960+0000","last_became_peered":"2024-05-08T22:05:14.068960+0000","last_unstale":"2024-05-08T22:07:17.174918+0000","last_undegraded":"2024-05-08T22:07:17.174918+0000","last_fullsized":"2024-05-08T22:07:17.174918+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:59:42.829110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.175370+0000","last_change":"2024-05-08T22:05:14.049393+0000","last_active":"2024-05-08T22:07:17.175370+0000","last_peered":"2024-05-08T22:07:17.175370+0000","last_clean":"2024-05-08T22:07:17.175370+0000","last_became_active":"2024-05-08T22:05:14.049094+0000","last_became_peered":"2024-05-08T22:05:14.049094+0000","last_unstale":"2024-05-08T22:07:17.175370+0000","last_undegraded":"2024-05-08T22:07:17.175370+0000","last_fullsized":"2024-05-08T22:07:17.175370+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:42.106568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.176257+0000","last_change":"2024-05-08T22:05:14.055436+0000","last_active":"2024-05-08T22:07:17.176257+0000","last_peered":"2024-05-08T22:07:17.176257+0000","last_clean":"2024-05-08T22:07:17.176257+0000","last_became_active":"2024-05-08T22:05:14.055100+0000","last_became_peered":"2024-05-08T22:05:14.055100+0000","last_unstale":"2024-05-08T22:07:17.176257+0000","last_undegraded":"2024-05-08T22:07:17.176257+0000","last_fullsized":"2024-05-08T22:07:17.176257+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:52.833780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.447942+0000","last_change":"2024-05-08T22:05:18.532086+0000","last_active":"2024-05-08T22:06:42.447942+0000","last_peered":"2024-05-08T22:06:42.447942+0000","last_clean":"2024-05-08T22:06:42.447942+0000","last_became_active":"2024-05-08T22:05:18.531485+0000","last_became_peered":"2024-05-08T22:05:18.531485+0000","last_unstale":"2024-05-08T22:06:42.447942+0000","last_undegraded":"2024-05-08T22:06:42.447942+0000","last_fullsized":"2024-05-08T22:06:42.447942+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:40:36.050367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.103820+0000","last_change":"2024-05-08T22:05:13.549611+0000","last_active":"2024-05-08T22:07:17.103820+0000","last_peered":"2024-05-08T22:07:17.103820+0000","last_clean":"2024-05-08T22:07:17.103820+0000","last_became_active":"2024-05-08T22:05:13.549140+0000","last_became_peered":"2024-05-08T22:05:13.549140+0000","last_unstale":"2024-05-08T22:07:17.103820+0000","last_undegraded":"2024-05-08T22:07:17.103820+0000","last_fullsized":"2024-05-08T22:07:17.103820+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:54.785123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.166838+0000","last_change":"2024-05-08T22:06:42.448887+0000","last_active":"2024-05-08T22:07:17.166838+0000","last_peered":"2024-05-08T22:07:17.166838+0000","last_clean":"2024-05-08T22:07:17.166838+0000","last_became_active":"2024-05-08T22:06:42.448391+0000","last_became_peered":"2024-05-08T22:06:42.448391+0000","last_unstale":"2024-05-08T22:07:17.166838+0000","last_undegraded":"2024-05-08T22:07:17.166838+0000","last_fullsized":"2024-05-08T22:07:17.166838+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:51.859658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976371+0000","last_change":"2024-05-08T22:05:42.147633+0000","last_active":"2024-05-08T22:06:41.976371+0000","last_peered":"2024-05-08T22:06:41.976371+0000","last_clean":"2024-05-08T22:06:41.976371+0000","last_became_active":"2024-05-08T22:05:42.147284+0000","last_became_peered":"2024-05-08T22:05:42.147284+0000","last_unstale":"2024-05-08T22:06:41.976371+0000","last_undegraded":"2024-05-08T22:06:41.976371+0000","last_fullsized":"2024-05-08T22:06:41.976371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:32.864756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.139965+0000","last_change":"2024-05-08T22:05:13.551351+0000","last_active":"2024-05-08T22:07:17.139965+0000","last_peered":"2024-05-08T22:07:17.139965+0000","last_clean":"2024-05-08T22:07:17.139965+0000","last_became_active":"2024-05-08T22:05:13.551014+0000","last_became_peered":"2024-05-08T22:05:13.551014+0000","last_unstale":"2024-05-08T22:07:17.139965+0000","last_undegraded":"2024-05-08T22:07:17.139965+0000","last_fullsized":"2024-05-08T22:07:17.139965+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:07:22.405251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.180951+0000","last_change":"2024-05-08T22:05:14.067270+0000","last_active":"2024-05-08T22:07:17.180951+0000","last_peered":"2024-05-08T22:07:17.180951+0000","last_clean":"2024-05-08T22:07:17.180951+0000","last_became_active":"2024-05-08T22:05:14.065496+0000","last_became_peered":"2024-05-08T22:05:14.065496+0000","last_unstale":"2024-05-08T22:07:17.180951+0000","last_undegraded":"2024-05-08T22:07:17.180951+0000","last_fullsized":"2024-05-08T22:07:17.180951+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:07.087880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976487+0000","last_change":"2024-05-08T22:05:18.526557+0000","last_active":"2024-05-08T22:06:41.976487+0000","last_peered":"2024-05-08T22:06:41.976487+0000","last_clean":"2024-05-08T22:06:41.976487+0000","last_became_active":"2024-05-08T22:05:18.526129+0000","last_became_peered":"2024-05-08T22:05:18.526129+0000","last_unstale":"2024-05-08T22:06:41.976487+0000","last_undegraded":"2024-05-08T22:06:41.976487+0000","last_fullsized":"2024-05-08T22:06:41.976487+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:36.376579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.376514+0000","last_change":"2024-05-08T22:05:41.835074+0000","last_active":"2024-05-08T22:06:42.376514+0000","last_peered":"2024-05-08T22:06:42.376514+0000","last_clean":"2024-05-08T22:06:42.376514+0000","last_became_active":"2024-05-08T22:05:41.829567+0000","last_became_peered":"2024-05-08T22:05:41.829567+0000","last_unstale":"2024-05-08T22:06:42.376514+0000","last_undegraded":"2024-05-08T22:06:42.376514+0000","last_fullsized":"2024-05-08T22:06:42.376514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:15.261352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.151662+0000","last_change":"2024-05-08T22:05:14.060089+0000","last_active":"2024-05-08T22:07:17.151662+0000","last_peered":"2024-05-08T22:07:17.151662+0000","last_clean":"2024-05-08T22:07:17.151662+0000","last_became_active":"2024-05-08T22:05:14.057620+0000","last_became_peered":"2024-05-08T22:05:14.057620+0000","last_unstale":"2024-05-08T22:07:17.151662+0000","last_undegraded":"2024-05-08T22:07:17.151662+0000","last_fullsized":"2024-05-08T22:07:17.151662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:29:33.127510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976369+0000","last_change":"2024-05-08T22:05:18.529746+0000","last_active":"2024-05-08T22:06:41.976369+0000","last_peered":"2024-05-08T22:06:41.976369+0000","last_clean":"2024-05-08T22:06:41.976369+0000","last_became_active":"2024-05-08T22:05:18.529097+0000","last_became_peered":"2024-05-08T22:05:18.529097+0000","last_unstale":"2024-05-08T22:06:41.976369+0000","last_undegraded":"2024-05-08T22:06:41.976369+0000","last_fullsized":"2024-05-08T22:06:41.976369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:59:36.249787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.376187+0000","last_change":"2024-05-08T22:05:41.837103+0000","last_active":"2024-05-08T22:06:42.376187+0000","last_peered":"2024-05-08T22:06:42.376187+0000","last_clean":"2024-05-08T22:06:42.376187+0000","last_became_active":"2024-05-08T22:05:41.829464+0000","last_became_peered":"2024-05-08T22:05:41.829464+0000","last_unstale":"2024-05-08T22:06:42.376187+0000","last_undegraded":"2024-05-08T22:06:42.376187+0000","last_fullsized":"2024-05-08T22:06:42.376187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:55.747487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.151416+0000","last_change":"2024-05-08T22:05:14.064695+0000","last_active":"2024-05-08T22:07:17.151416+0000","last_peered":"2024-05-08T22:07:17.151416+0000","last_clean":"2024-05-08T22:07:17.151416+0000","last_became_active":"2024-05-08T22:05:14.064283+0000","last_became_peered":"2024-05-08T22:05:14.064283+0000","last_unstale":"2024-05-08T22:07:17.151416+0000","last_undegraded":"2024-05-08T22:07:17.151416+0000","last_fullsized":"2024-05-08T22:07:17.151416+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:06.980594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.447815+0000","last_change":"2024-05-08T22:05:18.528475+0000","last_active":"2024-05-08T22:06:42.447815+0000","last_peered":"2024-05-08T22:06:42.447815+0000","last_clean":"2024-05-08T22:06:42.447815+0000","last_became_active":"2024-05-08T22:05:18.528089+0000","last_became_peered":"2024-05-08T22:05:18.528089+0000","last_unstale":"2024-05-08T22:06:42.447815+0000","last_undegraded":"2024-05-08T22:06:42.447815+0000","last_fullsized":"2024-05-08T22:06:42.447815+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:01.508984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'8","reported_seq":138,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:21.653137+0000","last_change":"2024-05-08T22:05:13.549191+0000","last_active":"2024-05-08T22:07:21.653137+0000","last_peered":"2024-05-08T22:07:21.653137+0000","last_clean":"2024-05-08T22:07:21.653137+0000","last_became_active":"2024-05-08T22:05:13.548737+0000","last_became_peered":"2024-05-08T22:05:13.548737+0000","last_unstale":"2024-05-08T22:07:21.653137+0000","last_undegraded":"2024-05-08T22:07:21.653137+0000","last_fullsized":"2024-05-08T22:07:21.653137+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:07.355656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1501,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.180110+0000","last_change":"2024-05-08T22:06:42.018982+0000","last_active":"2024-05-08T22:07:17.180110+0000","last_peered":"2024-05-08T22:07:17.180110+0000","last_clean":"2024-05-08T22:07:17.180110+0000","last_became_active":"2024-05-08T22:06:42.018609+0000","last_became_peered":"2024-05-08T22:06:42.018609+0000","last_unstale":"2024-05-08T22:07:17.180110+0000","last_undegraded":"2024-05-08T22:07:17.180110+0000","last_fullsized":"2024-05-08T22:07:17.180110+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:37.578711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.447741+0000","last_change":"2024-05-08T22:05:18.536861+0000","last_active":"2024-05-08T22:06:42.447741+0000","last_peered":"2024-05-08T22:06:42.447741+0000","last_clean":"2024-05-08T22:06:42.447741+0000","last_became_active":"2024-05-08T22:05:18.536472+0000","last_became_peered":"2024-05-08T22:05:18.536472+0000","last_unstale":"2024-05-08T22:06:42.447741+0000","last_undegraded":"2024-05-08T22:06:42.447741+0000","last_fullsized":"2024-05-08T22:06:42.447741+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:45:21.826812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.467017+0000","last_change":"2024-05-08T22:05:13.553326+0000","last_active":"2024-05-08T22:06:42.467017+0000","last_peered":"2024-05-08T22:06:42.467017+0000","last_clean":"2024-05-08T22:06:42.467017+0000","last_became_active":"2024-05-08T22:05:13.552983+0000","last_became_peered":"2024-05-08T22:05:13.552983+0000","last_unstale":"2024-05-08T22:06:42.467017+0000","last_undegraded":"2024-05-08T22:06:42.467017+0000","last_fullsized":"2024-05-08T22:06:42.467017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:17.195253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.165741+0000","last_change":"2024-05-08T22:05:14.068617+0000","last_active":"2024-05-08T22:07:17.165741+0000","last_peered":"2024-05-08T22:07:17.165741+0000","last_clean":"2024-05-08T22:07:17.165741+0000","last_became_active":"2024-05-08T22:05:14.067400+0000","last_became_peered":"2024-05-08T22:05:14.067400+0000","last_unstale":"2024-05-08T22:07:17.165741+0000","last_undegraded":"2024-05-08T22:07:17.165741+0000","last_fullsized":"2024-05-08T22:07:17.165741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:25.488212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.130379+0000","last_change":"2024-05-08T22:05:18.538294+0000","last_active":"2024-05-08T22:07:17.130379+0000","last_peered":"2024-05-08T22:07:17.130379+0000","last_clean":"2024-05-08T22:07:17.130379+0000","last_became_active":"2024-05-08T22:05:18.537984+0000","last_became_peered":"2024-05-08T22:05:18.537984+0000","last_unstale":"2024-05-08T22:07:17.130379+0000","last_undegraded":"2024-05-08T22:07:17.130379+0000","last_fullsized":"2024-05-08T22:07:17.130379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:32.117439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.017185+0000","last_change":"2024-05-08T22:05:13.557669+0000","last_active":"2024-05-08T22:06:42.017185+0000","last_peered":"2024-05-08T22:06:42.017185+0000","last_clean":"2024-05-08T22:06:42.017185+0000","last_became_active":"2024-05-08T22:05:13.555835+0000","last_became_peered":"2024-05-08T22:05:13.555835+0000","last_unstale":"2024-05-08T22:06:42.017185+0000","last_undegraded":"2024-05-08T22:06:42.017185+0000","last_fullsized":"2024-05-08T22:06:42.017185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:02.377635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.179458+0000","last_change":"2024-05-08T22:05:14.060144+0000","last_active":"2024-05-08T22:07:17.179458+0000","last_peered":"2024-05-08T22:07:17.179458+0000","last_clean":"2024-05-08T22:07:17.179458+0000","last_became_active":"2024-05-08T22:05:14.059387+0000","last_became_peered":"2024-05-08T22:05:14.059387+0000","last_unstale":"2024-05-08T22:07:17.179458+0000","last_undegraded":"2024-05-08T22:07:17.179458+0000","last_fullsized":"2024-05-08T22:07:17.179458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:32:28.057831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.140368+0000","last_change":"2024-05-08T22:05:18.527293+0000","last_active":"2024-05-08T22:07:17.140368+0000","last_peered":"2024-05-08T22:07:17.140368+0000","last_clean":"2024-05-08T22:07:17.140368+0000","last_became_active":"2024-05-08T22:05:18.526690+0000","last_became_peered":"2024-05-08T22:05:18.526690+0000","last_unstale":"2024-05-08T22:07:17.140368+0000","last_undegraded":"2024-05-08T22:07:17.140368+0000","last_fullsized":"2024-05-08T22:07:17.140368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:50.343536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.180700+0000","last_change":"2024-05-08T22:05:13.549774+0000","last_active":"2024-05-08T22:06:42.180700+0000","last_peered":"2024-05-08T22:06:42.180700+0000","last_clean":"2024-05-08T22:06:42.180700+0000","last_became_active":"2024-05-08T22:05:13.549531+0000","last_became_peered":"2024-05-08T22:05:13.549531+0000","last_unstale":"2024-05-08T22:06:42.180700+0000","last_undegraded":"2024-05-08T22:06:42.180700+0000","last_fullsized":"2024-05-08T22:06:42.180700+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:59:23.798728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.181017+0000","last_change":"2024-05-08T22:05:14.054232+0000","last_active":"2024-05-08T22:07:17.181017+0000","last_peered":"2024-05-08T22:07:17.181017+0000","last_clean":"2024-05-08T22:07:17.181017+0000","last_became_active":"2024-05-08T22:05:14.053806+0000","last_became_peered":"2024-05-08T22:05:14.053806+0000","last_unstale":"2024-05-08T22:07:17.181017+0000","last_undegraded":"2024-05-08T22:07:17.181017+0000","last_fullsized":"2024-05-08T22:07:17.181017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:32:21.678792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.979318+0000","last_change":"2024-05-08T22:05:18.535241+0000","last_active":"2024-05-08T22:06:41.979318+0000","last_peered":"2024-05-08T22:06:41.979318+0000","last_clean":"2024-05-08T22:06:41.979318+0000","last_became_active":"2024-05-08T22:05:18.534753+0000","last_became_peered":"2024-05-08T22:05:18.534753+0000","last_unstale":"2024-05-08T22:06:41.979318+0000","last_undegraded":"2024-05-08T22:06:41.979318+0000","last_fullsized":"2024-05-08T22:06:41.979318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:46:15.138437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":129,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:21.652796+0000","last_change":"2024-05-08T22:05:13.544899+0000","last_active":"2024-05-08T22:07:21.652796+0000","last_peered":"2024-05-08T22:07:21.652796+0000","last_clean":"2024-05-08T22:07:21.652796+0000","last_became_active":"2024-05-08T22:05:13.544498+0000","last_became_peered":"2024-05-08T22:05:13.544498+0000","last_unstale":"2024-05-08T22:07:21.652796+0000","last_undegraded":"2024-05-08T22:07:21.652796+0000","last_fullsized":"2024-05-08T22:07:21.652796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:58.169648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.176371+0000","last_change":"2024-05-08T22:05:14.049377+0000","last_active":"2024-05-08T22:07:17.176371+0000","last_peered":"2024-05-08T22:07:17.176371+0000","last_clean":"2024-05-08T22:07:17.176371+0000","last_became_active":"2024-05-08T22:05:14.049039+0000","last_became_peered":"2024-05-08T22:05:14.049039+0000","last_unstale":"2024-05-08T22:07:17.176371+0000","last_undegraded":"2024-05-08T22:07:17.176371+0000","last_fullsized":"2024-05-08T22:07:17.176371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:24.415392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976341+0000","last_change":"2024-05-08T22:05:18.528912+0000","last_active":"2024-05-08T22:06:41.976341+0000","last_peered":"2024-05-08T22:06:41.976341+0000","last_clean":"2024-05-08T22:06:41.976341+0000","last_became_active":"2024-05-08T22:05:18.528433+0000","last_became_peered":"2024-05-08T22:05:18.528433+0000","last_unstale":"2024-05-08T22:06:41.976341+0000","last_undegraded":"2024-05-08T22:06:41.976341+0000","last_fullsized":"2024-05-08T22:06:41.976341+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:05.574728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.055447+0000","last_change":"2024-05-08T22:05:13.550214+0000","last_active":"2024-05-08T22:06:42.055447+0000","last_peered":"2024-05-08T22:06:42.055447+0000","last_clean":"2024-05-08T22:06:42.055447+0000","last_became_active":"2024-05-08T22:05:13.549717+0000","last_became_peered":"2024-05-08T22:05:13.549717+0000","last_unstale":"2024-05-08T22:06:42.055447+0000","last_undegraded":"2024-05-08T22:06:42.055447+0000","last_fullsized":"2024-05-08T22:06:42.055447+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:12:20.406333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.177101+0000","last_change":"2024-05-08T22:05:14.071214+0000","last_active":"2024-05-08T22:07:17.177101+0000","last_peered":"2024-05-08T22:07:17.177101+0000","last_clean":"2024-05-08T22:07:17.177101+0000","last_became_active":"2024-05-08T22:05:14.066777+0000","last_became_peered":"2024-05-08T22:05:14.066777+0000","last_unstale":"2024-05-08T22:07:17.177101+0000","last_undegraded":"2024-05-08T22:07:17.177101+0000","last_fullsized":"2024-05-08T22:07:17.177101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:53.360911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976794+0000","last_change":"2024-05-08T22:05:18.523688+0000","last_active":"2024-05-08T22:06:41.976794+0000","last_peered":"2024-05-08T22:06:41.976794+0000","last_clean":"2024-05-08T22:06:41.976794+0000","last_became_active":"2024-05-08T22:05:18.523236+0000","last_became_peered":"2024-05-08T22:05:18.523236+0000","last_unstale":"2024-05-08T22:06:41.976794+0000","last_undegraded":"2024-05-08T22:06:41.976794+0000","last_fullsized":"2024-05-08T22:06:41.976794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:34.531699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.181222+0000","last_change":"2024-05-08T22:05:13.551027+0000","last_active":"2024-05-08T22:06:42.181222+0000","last_peered":"2024-05-08T22:06:42.181222+0000","last_clean":"2024-05-08T22:06:42.181222+0000","last_became_active":"2024-05-08T22:05:13.550471+0000","last_became_peered":"2024-05-08T22:05:13.550471+0000","last_unstale":"2024-05-08T22:06:42.181222+0000","last_undegraded":"2024-05-08T22:06:42.181222+0000","last_fullsized":"2024-05-08T22:06:42.181222+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:50.900419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.156519+0000","last_change":"2024-05-08T22:05:14.065191+0000","last_active":"2024-05-08T22:07:17.156519+0000","last_peered":"2024-05-08T22:07:17.156519+0000","last_clean":"2024-05-08T22:07:17.156519+0000","last_became_active":"2024-05-08T22:05:14.064893+0000","last_became_peered":"2024-05-08T22:05:14.064893+0000","last_unstale":"2024-05-08T22:07:17.156519+0000","last_undegraded":"2024-05-08T22:07:17.156519+0000","last_fullsized":"2024-05-08T22:07:17.156519+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:11.344646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976480+0000","last_change":"2024-05-08T22:05:18.530447+0000","last_active":"2024-05-08T22:06:41.976480+0000","last_peered":"2024-05-08T22:06:41.976480+0000","last_clean":"2024-05-08T22:06:41.976480+0000","last_became_active":"2024-05-08T22:05:18.528171+0000","last_became_peered":"2024-05-08T22:05:18.528171+0000","last_unstale":"2024-05-08T22:06:41.976480+0000","last_undegraded":"2024-05-08T22:06:41.976480+0000","last_fullsized":"2024-05-08T22:06:41.976480+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:09:43.347366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'39","reported_seq":278,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:22.654579+0000","last_change":"2024-05-08T22:05:13.547289+0000","last_active":"2024-05-08T22:07:22.654579+0000","last_peered":"2024-05-08T22:07:22.654579+0000","last_clean":"2024-05-08T22:07:22.654579+0000","last_became_active":"2024-05-08T22:05:13.546860+0000","last_became_peered":"2024-05-08T22:05:13.546860+0000","last_unstale":"2024-05-08T22:07:22.654579+0000","last_undegraded":"2024-05-08T22:07:22.654579+0000","last_fullsized":"2024-05-08T22:07:22.654579+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:08:50.339361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69082,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":218,"num_read_kb":175,"num_write":40,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.146652+0000","last_change":"2024-05-08T22:05:14.057925+0000","last_active":"2024-05-08T22:07:17.146652+0000","last_peered":"2024-05-08T22:07:17.146652+0000","last_clean":"2024-05-08T22:07:17.146652+0000","last_became_active":"2024-05-08T22:05:14.057662+0000","last_became_peered":"2024-05-08T22:05:14.057662+0000","last_unstale":"2024-05-08T22:07:17.146652+0000","last_undegraded":"2024-05-08T22:07:17.146652+0000","last_fullsized":"2024-05-08T22:07:17.146652+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:49.929610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.181486+0000","last_change":"2024-05-08T22:05:18.535509+0000","last_active":"2024-05-08T22:06:42.181486+0000","last_peered":"2024-05-08T22:06:42.181486+0000","last_clean":"2024-05-08T22:06:42.181486+0000","last_became_active":"2024-05-08T22:05:18.535024+0000","last_became_peered":"2024-05-08T22:05:18.535024+0000","last_unstale":"2024-05-08T22:06:42.181486+0000","last_undegraded":"2024-05-08T22:06:42.181486+0000","last_fullsized":"2024-05-08T22:06:42.181486+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:28.098737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.447863+0000","last_change":"2024-05-08T22:05:13.552096+0000","last_active":"2024-05-08T22:06:42.447863+0000","last_peered":"2024-05-08T22:06:42.447863+0000","last_clean":"2024-05-08T22:06:42.447863+0000","last_became_active":"2024-05-08T22:05:13.551655+0000","last_became_peered":"2024-05-08T22:05:13.551655+0000","last_unstale":"2024-05-08T22:06:42.447863+0000","last_undegraded":"2024-05-08T22:06:42.447863+0000","last_fullsized":"2024-05-08T22:06:42.447863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:24.371008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.180706+0000","last_change":"2024-05-08T22:05:14.066075+0000","last_active":"2024-05-08T22:07:17.180706+0000","last_peered":"2024-05-08T22:07:17.180706+0000","last_clean":"2024-05-08T22:07:17.180706+0000","last_became_active":"2024-05-08T22:05:14.065581+0000","last_became_peered":"2024-05-08T22:05:14.065581+0000","last_unstale":"2024-05-08T22:07:17.180706+0000","last_undegraded":"2024-05-08T22:07:17.180706+0000","last_fullsized":"2024-05-08T22:07:17.180706+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:15.403714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.376646+0000","last_change":"2024-05-08T22:05:41.831782+0000","last_active":"2024-05-08T22:06:42.376646+0000","last_peered":"2024-05-08T22:06:42.376646+0000","last_clean":"2024-05-08T22:06:42.376646+0000","last_became_active":"2024-05-08T22:05:41.831335+0000","last_became_peered":"2024-05-08T22:05:41.831335+0000","last_unstale":"2024-05-08T22:06:42.376646+0000","last_undegraded":"2024-05-08T22:06:42.376646+0000","last_fullsized":"2024-05-08T22:06:42.376646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:39:34.031128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.181155+0000","last_change":"2024-05-08T22:05:13.549112+0000","last_active":"2024-05-08T22:06:42.181155+0000","last_peered":"2024-05-08T22:06:42.181155+0000","last_clean":"2024-05-08T22:06:42.181155+0000","last_became_active":"2024-05-08T22:05:13.548763+0000","last_became_peered":"2024-05-08T22:05:13.548763+0000","last_unstale":"2024-05-08T22:06:42.181155+0000","last_undegraded":"2024-05-08T22:06:42.181155+0000","last_fullsized":"2024-05-08T22:06:42.181155+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:26:56.448611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.158808+0000","last_change":"2024-05-08T22:05:14.067116+0000","last_active":"2024-05-08T22:07:17.158808+0000","last_peered":"2024-05-08T22:07:17.158808+0000","last_clean":"2024-05-08T22:07:17.158808+0000","last_became_active":"2024-05-08T22:05:14.064775+0000","last_became_peered":"2024-05-08T22:05:14.064775+0000","last_unstale":"2024-05-08T22:07:17.158808+0000","last_undegraded":"2024-05-08T22:07:17.158808+0000","last_fullsized":"2024-05-08T22:07:17.158808+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:28:32.901405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.977984+0000","last_change":"2024-05-08T22:05:18.539546+0000","last_active":"2024-05-08T22:06:41.977984+0000","last_peered":"2024-05-08T22:06:41.977984+0000","last_clean":"2024-05-08T22:06:41.977984+0000","last_became_active":"2024-05-08T22:05:18.538998+0000","last_became_peered":"2024-05-08T22:05:18.538998+0000","last_unstale":"2024-05-08T22:06:41.977984+0000","last_undegraded":"2024-05-08T22:06:41.977984+0000","last_fullsized":"2024-05-08T22:06:41.977984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:30.453881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":135,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:21.653538+0000","last_change":"2024-05-08T22:05:13.549053+0000","last_active":"2024-05-08T22:07:21.653538+0000","last_peered":"2024-05-08T22:07:21.653538+0000","last_clean":"2024-05-08T22:07:21.653538+0000","last_became_active":"2024-05-08T22:05:13.548739+0000","last_became_peered":"2024-05-08T22:05:13.548739+0000","last_unstale":"2024-05-08T22:07:21.653538+0000","last_undegraded":"2024-05-08T22:07:21.653538+0000","last_fullsized":"2024-05-08T22:07:21.653538+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:07:10.298084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1719,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.150047+0000","last_change":"2024-05-08T22:05:14.061753+0000","last_active":"2024-05-08T22:07:17.150047+0000","last_peered":"2024-05-08T22:07:17.150047+0000","last_clean":"2024-05-08T22:07:17.150047+0000","last_became_active":"2024-05-08T22:05:14.061412+0000","last_became_peered":"2024-05-08T22:05:14.061412+0000","last_unstale":"2024-05-08T22:07:17.150047+0000","last_undegraded":"2024-05-08T22:07:17.150047+0000","last_fullsized":"2024-05-08T22:07:17.150047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:54.380181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.017707+0000","last_change":"2024-05-08T22:05:18.532627+0000","last_active":"2024-05-08T22:06:42.017707+0000","last_peered":"2024-05-08T22:06:42.017707+0000","last_clean":"2024-05-08T22:06:42.017707+0000","last_became_active":"2024-05-08T22:05:18.532205+0000","last_became_peered":"2024-05-08T22:05:18.532205+0000","last_unstale":"2024-05-08T22:06:42.017707+0000","last_undegraded":"2024-05-08T22:06:42.017707+0000","last_fullsized":"2024-05-08T22:06:42.017707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:20:11.028645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":64,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976362+0000","last_change":"2024-05-08T22:05:41.841832+0000","last_active":"2024-05-08T22:06:41.976362+0000","last_peered":"2024-05-08T22:06:41.976362+0000","last_clean":"2024-05-08T22:06:41.976362+0000","last_became_active":"2024-05-08T22:05:41.829584+0000","last_became_peered":"2024-05-08T22:05:41.829584+0000","last_unstale":"2024-05-08T22:06:41.976362+0000","last_undegraded":"2024-05-08T22:06:41.976362+0000","last_fullsized":"2024-05-08T22:06:41.976362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:52:19.876215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.149577+0000","last_change":"2024-05-08T22:05:14.063027+0000","last_active":"2024-05-08T22:07:17.149577+0000","last_peered":"2024-05-08T22:07:17.149577+0000","last_clean":"2024-05-08T22:07:17.149577+0000","last_became_active":"2024-05-08T22:05:14.062682+0000","last_became_peered":"2024-05-08T22:05:14.062682+0000","last_unstale":"2024-05-08T22:07:17.149577+0000","last_undegraded":"2024-05-08T22:07:17.149577+0000","last_fullsized":"2024-05-08T22:07:17.149577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:47.248832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.180890+0000","last_change":"2024-05-08T22:05:18.528370+0000","last_active":"2024-05-08T22:06:42.180890+0000","last_peered":"2024-05-08T22:06:42.180890+0000","last_clean":"2024-05-08T22:06:42.180890+0000","last_became_active":"2024-05-08T22:05:18.527990+0000","last_became_peered":"2024-05-08T22:05:18.527990+0000","last_unstale":"2024-05-08T22:06:42.180890+0000","last_undegraded":"2024-05-08T22:06:42.180890+0000","last_fullsized":"2024-05-08T22:06:42.180890+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:07.524175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.120821+0000","last_change":"2024-05-08T22:05:13.547543+0000","last_active":"2024-05-08T22:07:17.120821+0000","last_peered":"2024-05-08T22:07:17.120821+0000","last_clean":"2024-05-08T22:07:17.120821+0000","last_became_active":"2024-05-08T22:05:13.547191+0000","last_became_peered":"2024-05-08T22:05:13.547191+0000","last_unstale":"2024-05-08T22:07:17.120821+0000","last_undegraded":"2024-05-08T22:07:17.120821+0000","last_fullsized":"2024-05-08T22:07:17.120821+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:53.213239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.150554+0000","last_change":"2024-05-08T22:05:14.062677+0000","last_active":"2024-05-08T22:07:17.150554+0000","last_peered":"2024-05-08T22:07:17.150554+0000","last_clean":"2024-05-08T22:07:17.150554+0000","last_became_active":"2024-05-08T22:05:14.062306+0000","last_became_peered":"2024-05-08T22:05:14.062306+0000","last_unstale":"2024-05-08T22:07:17.150554+0000","last_undegraded":"2024-05-08T22:07:17.150554+0000","last_fullsized":"2024-05-08T22:07:17.150554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:00.356505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.119465+0000","last_change":"2024-05-08T22:05:18.617075+0000","last_active":"2024-05-08T22:07:17.119465+0000","last_peered":"2024-05-08T22:07:17.119465+0000","last_clean":"2024-05-08T22:07:17.119465+0000","last_became_active":"2024-05-08T22:05:18.616653+0000","last_became_peered":"2024-05-08T22:05:18.616653+0000","last_unstale":"2024-05-08T22:07:17.119465+0000","last_undegraded":"2024-05-08T22:07:17.119465+0000","last_fullsized":"2024-05-08T22:07:17.119465+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:30.537064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.103554+0000","last_change":"2024-05-08T22:05:13.553457+0000","last_active":"2024-05-08T22:07:17.103554+0000","last_peered":"2024-05-08T22:07:17.103554+0000","last_clean":"2024-05-08T22:07:17.103554+0000","last_became_active":"2024-05-08T22:05:13.551583+0000","last_became_peered":"2024-05-08T22:05:13.551583+0000","last_unstale":"2024-05-08T22:07:17.103554+0000","last_undegraded":"2024-05-08T22:07:17.103554+0000","last_fullsized":"2024-05-08T22:07:17.103554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:07:05.971413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.172164+0000","last_change":"2024-05-08T22:05:14.057476+0000","last_active":"2024-05-08T22:07:17.172164+0000","last_peered":"2024-05-08T22:07:17.172164+0000","last_clean":"2024-05-08T22:07:17.172164+0000","last_became_active":"2024-05-08T22:05:14.056550+0000","last_became_peered":"2024-05-08T22:05:14.056550+0000","last_unstale":"2024-05-08T22:07:17.172164+0000","last_undegraded":"2024-05-08T22:07:17.172164+0000","last_fullsized":"2024-05-08T22:07:17.172164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:41:29.264096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.017335+0000","last_change":"2024-05-08T22:05:18.530719+0000","last_active":"2024-05-08T22:06:42.017335+0000","last_peered":"2024-05-08T22:06:42.017335+0000","last_clean":"2024-05-08T22:06:42.017335+0000","last_became_active":"2024-05-08T22:05:18.529659+0000","last_became_peered":"2024-05-08T22:05:18.529659+0000","last_unstale":"2024-05-08T22:06:42.017335+0000","last_undegraded":"2024-05-08T22:06:42.017335+0000","last_fullsized":"2024-05-08T22:06:42.017335+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:13:06.514460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.107307+0000","last_change":"2024-05-08T22:05:13.556191+0000","last_active":"2024-05-08T22:07:17.107307+0000","last_peered":"2024-05-08T22:07:17.107307+0000","last_clean":"2024-05-08T22:07:17.107307+0000","last_became_active":"2024-05-08T22:05:13.555888+0000","last_became_peered":"2024-05-08T22:05:13.555888+0000","last_unstale":"2024-05-08T22:07:17.107307+0000","last_undegraded":"2024-05-08T22:07:17.107307+0000","last_fullsized":"2024-05-08T22:07:17.107307+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:05.637980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.179969+0000","last_change":"2024-05-08T22:05:41.828023+0000","last_active":"2024-05-08T22:07:17.179969+0000","last_peered":"2024-05-08T22:07:17.179969+0000","last_clean":"2024-05-08T22:07:17.179969+0000","last_became_active":"2024-05-08T22:05:41.827550+0000","last_became_peered":"2024-05-08T22:05:41.827550+0000","last_unstale":"2024-05-08T22:07:17.179969+0000","last_undegraded":"2024-05-08T22:07:17.179969+0000","last_fullsized":"2024-05-08T22:07:17.179969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:13.234423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.017591+0000","last_change":"2024-05-08T22:05:41.828529+0000","last_active":"2024-05-08T22:06:42.017591+0000","last_peered":"2024-05-08T22:06:42.017591+0000","last_clean":"2024-05-08T22:06:42.017591+0000","last_became_active":"2024-05-08T22:05:41.828081+0000","last_became_peered":"2024-05-08T22:05:41.828081+0000","last_unstale":"2024-05-08T22:06:42.017591+0000","last_undegraded":"2024-05-08T22:06:42.017591+0000","last_fullsized":"2024-05-08T22:06:42.017591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:27.478823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.152262+0000","last_change":"2024-05-08T22:05:14.069096+0000","last_active":"2024-05-08T22:07:17.152262+0000","last_peered":"2024-05-08T22:07:17.152262+0000","last_clean":"2024-05-08T22:07:17.152262+0000","last_became_active":"2024-05-08T22:05:14.068796+0000","last_became_peered":"2024-05-08T22:05:14.068796+0000","last_unstale":"2024-05-08T22:07:17.152262+0000","last_undegraded":"2024-05-08T22:07:17.152262+0000","last_fullsized":"2024-05-08T22:07:17.152262+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:52:52.391196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":131,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:21.653111+0000","last_change":"2024-05-08T22:05:13.559286+0000","last_active":"2024-05-08T22:07:21.653111+0000","last_peered":"2024-05-08T22:07:21.653111+0000","last_clean":"2024-05-08T22:07:21.653111+0000","last_became_active":"2024-05-08T22:05:13.558922+0000","last_became_peered":"2024-05-08T22:05:13.558922+0000","last_unstale":"2024-05-08T22:07:21.653111+0000","last_undegraded":"2024-05-08T22:07:21.653111+0000","last_fullsized":"2024-05-08T22:07:21.653111+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:25:24.732422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.055193+0000","last_change":"2024-05-08T22:05:18.527031+0000","last_active":"2024-05-08T22:06:42.055193+0000","last_peered":"2024-05-08T22:06:42.055193+0000","last_clean":"2024-05-08T22:06:42.055193+0000","last_became_active":"2024-05-08T22:05:18.526573+0000","last_became_peered":"2024-05-08T22:05:18.526573+0000","last_unstale":"2024-05-08T22:06:42.055193+0000","last_undegraded":"2024-05-08T22:06:42.055193+0000","last_fullsized":"2024-05-08T22:06:42.055193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:08.492359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.178225+0000","last_change":"2024-05-08T22:06:42.278191+0000","last_active":"2024-05-08T22:07:17.178225+0000","last_peered":"2024-05-08T22:07:17.178225+0000","last_clean":"2024-05-08T22:07:17.178225+0000","last_became_active":"2024-05-08T22:06:42.277819+0000","last_became_peered":"2024-05-08T22:06:42.277819+0000","last_unstale":"2024-05-08T22:07:17.178225+0000","last_undegraded":"2024-05-08T22:07:17.178225+0000","last_fullsized":"2024-05-08T22:07:17.178225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:48.919595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.104081+0000","last_change":"2024-05-08T22:05:13.545709+0000","last_active":"2024-05-08T22:07:17.104081+0000","last_peered":"2024-05-08T22:07:17.104081+0000","last_clean":"2024-05-08T22:07:17.104081+0000","last_became_active":"2024-05-08T22:05:13.545419+0000","last_became_peered":"2024-05-08T22:05:13.545419+0000","last_unstale":"2024-05-08T22:07:17.104081+0000","last_undegraded":"2024-05-08T22:07:17.104081+0000","last_fullsized":"2024-05-08T22:07:17.104081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:44.903903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"88'148","reported_seq":384,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:19.254206+0000","last_change":"2024-05-08T22:04:20.948328+0000","last_active":"2024-05-08T22:07:19.254206+0000","last_peered":"2024-05-08T22:07:19.254206+0000","last_clean":"2024-05-08T22:07:19.254206+0000","last_became_active":"2024-05-08T22:04:20.947887+0000","last_became_peered":"2024-05-08T22:04:20.947887+0000","last_unstale":"2024-05-08T22:07:19.254206+0000","last_undegraded":"2024-05-08T22:07:19.254206+0000","last_fullsized":"2024-05-08T22:07:19.254206+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:01:41.485277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:01:41.485277+0000","last_clean_scrub_stamp":"2024-05-08T22:01:41.485277+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:43:59.208120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.181413+0000","last_change":"2024-05-08T22:05:41.824591+0000","last_active":"2024-05-08T22:06:42.181413+0000","last_peered":"2024-05-08T22:06:42.181413+0000","last_clean":"2024-05-08T22:06:42.181413+0000","last_became_active":"2024-05-08T22:05:41.824053+0000","last_became_peered":"2024-05-08T22:05:41.824053+0000","last_unstale":"2024-05-08T22:06:42.181413+0000","last_undegraded":"2024-05-08T22:06:42.181413+0000","last_fullsized":"2024-05-08T22:06:42.181413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:18:57.269459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.145622+0000","last_change":"2024-05-08T22:05:14.066818+0000","last_active":"2024-05-08T22:07:17.145622+0000","last_peered":"2024-05-08T22:07:17.145622+0000","last_clean":"2024-05-08T22:07:17.145622+0000","last_became_active":"2024-05-08T22:05:14.065152+0000","last_became_peered":"2024-05-08T22:05:14.065152+0000","last_unstale":"2024-05-08T22:07:17.145622+0000","last_undegraded":"2024-05-08T22:07:17.145622+0000","last_fullsized":"2024-05-08T22:07:17.145622+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:53:19.018852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.107386+0000","last_change":"2024-05-08T22:05:13.550365+0000","last_active":"2024-05-08T22:07:17.107386+0000","last_peered":"2024-05-08T22:07:17.107386+0000","last_clean":"2024-05-08T22:07:17.107386+0000","last_became_active":"2024-05-08T22:05:13.549677+0000","last_became_peered":"2024-05-08T22:05:13.549677+0000","last_unstale":"2024-05-08T22:07:17.107386+0000","last_undegraded":"2024-05-08T22:07:17.107386+0000","last_fullsized":"2024-05-08T22:07:17.107386+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:10.842838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.376136+0000","last_change":"2024-05-08T22:05:18.535777+0000","last_active":"2024-05-08T22:06:42.376136+0000","last_peered":"2024-05-08T22:06:42.376136+0000","last_clean":"2024-05-08T22:06:42.376136+0000","last_became_active":"2024-05-08T22:05:18.535327+0000","last_became_peered":"2024-05-08T22:05:18.535327+0000","last_unstale":"2024-05-08T22:06:42.376136+0000","last_undegraded":"2024-05-08T22:06:42.376136+0000","last_fullsized":"2024-05-08T22:06:42.376136+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:13.886989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.121005+0000","last_change":"2024-05-08T22:05:13.554045+0000","last_active":"2024-05-08T22:07:17.121005+0000","last_peered":"2024-05-08T22:07:17.121005+0000","last_clean":"2024-05-08T22:07:17.121005+0000","last_became_active":"2024-05-08T22:05:13.553315+0000","last_became_peered":"2024-05-08T22:05:13.553315+0000","last_unstale":"2024-05-08T22:07:17.121005+0000","last_undegraded":"2024-05-08T22:07:17.121005+0000","last_fullsized":"2024-05-08T22:07:17.121005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:13.131267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.182154+0000","last_change":"2024-05-08T22:05:14.065018+0000","last_active":"2024-05-08T22:07:17.182154+0000","last_peered":"2024-05-08T22:07:17.182154+0000","last_clean":"2024-05-08T22:07:17.182154+0000","last_became_active":"2024-05-08T22:05:14.064701+0000","last_became_peered":"2024-05-08T22:05:14.064701+0000","last_unstale":"2024-05-08T22:07:17.182154+0000","last_undegraded":"2024-05-08T22:07:17.182154+0000","last_fullsized":"2024-05-08T22:07:17.182154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:48:43.468373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.121062+0000","last_change":"2024-05-08T22:05:18.617853+0000","last_active":"2024-05-08T22:07:17.121062+0000","last_peered":"2024-05-08T22:07:17.121062+0000","last_clean":"2024-05-08T22:07:17.121062+0000","last_became_active":"2024-05-08T22:05:18.616255+0000","last_became_peered":"2024-05-08T22:05:18.616255+0000","last_unstale":"2024-05-08T22:07:17.121062+0000","last_undegraded":"2024-05-08T22:07:17.121062+0000","last_fullsized":"2024-05-08T22:07:17.121062+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:13.233425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'41","reported_seq":86,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:22.787863+0000","last_change":"2024-05-08T22:05:13.546274+0000","last_active":"2024-05-08T22:07:22.787863+0000","last_peered":"2024-05-08T22:07:22.787863+0000","last_clean":"2024-05-08T22:07:22.787863+0000","last_became_active":"2024-05-08T22:05:13.543651+0000","last_became_peered":"2024-05-08T22:05:13.543651+0000","last_unstale":"2024-05-08T22:07:22.787863+0000","last_undegraded":"2024-05-08T22:07:22.787863+0000","last_fullsized":"2024-05-08T22:07:22.787863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:21:58.601261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":73,"num_write_kb":835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.162082+0000","last_change":"2024-05-08T22:05:14.056535+0000","last_active":"2024-05-08T22:07:17.162082+0000","last_peered":"2024-05-08T22:07:17.162082+0000","last_clean":"2024-05-08T22:07:17.162082+0000","last_became_active":"2024-05-08T22:05:14.056211+0000","last_became_peered":"2024-05-08T22:05:14.056211+0000","last_unstale":"2024-05-08T22:07:17.162082+0000","last_undegraded":"2024-05-08T22:07:17.162082+0000","last_fullsized":"2024-05-08T22:07:17.162082+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:46:30.744490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.107088+0000","last_change":"2024-05-08T22:05:18.524939+0000","last_active":"2024-05-08T22:07:17.107088+0000","last_peered":"2024-05-08T22:07:17.107088+0000","last_clean":"2024-05-08T22:07:17.107088+0000","last_became_active":"2024-05-08T22:05:18.524518+0000","last_became_peered":"2024-05-08T22:05:18.524518+0000","last_unstale":"2024-05-08T22:07:17.107088+0000","last_undegraded":"2024-05-08T22:07:17.107088+0000","last_fullsized":"2024-05-08T22:07:17.107088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:50.192400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.977078+0000","last_change":"2024-05-08T22:05:13.554346+0000","last_active":"2024-05-08T22:06:41.977078+0000","last_peered":"2024-05-08T22:06:41.977078+0000","last_clean":"2024-05-08T22:06:41.977078+0000","last_became_active":"2024-05-08T22:05:13.554080+0000","last_became_peered":"2024-05-08T22:05:13.554080+0000","last_unstale":"2024-05-08T22:06:41.977078+0000","last_undegraded":"2024-05-08T22:06:41.977078+0000","last_fullsized":"2024-05-08T22:06:41.977078+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:29:49.695682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.181025+0000","last_change":"2024-05-08T22:05:14.071422+0000","last_active":"2024-05-08T22:07:17.181025+0000","last_peered":"2024-05-08T22:07:17.181025+0000","last_clean":"2024-05-08T22:07:17.181025+0000","last_became_active":"2024-05-08T22:05:14.070396+0000","last_became_peered":"2024-05-08T22:05:14.070396+0000","last_unstale":"2024-05-08T22:07:17.181025+0000","last_undegraded":"2024-05-08T22:07:17.181025+0000","last_fullsized":"2024-05-08T22:07:17.181025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:37.957983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976556+0000","last_change":"2024-05-08T22:05:18.617149+0000","last_active":"2024-05-08T22:06:41.976556+0000","last_peered":"2024-05-08T22:06:41.976556+0000","last_clean":"2024-05-08T22:06:41.976556+0000","last_became_active":"2024-05-08T22:05:18.616699+0000","last_became_peered":"2024-05-08T22:05:18.616699+0000","last_unstale":"2024-05-08T22:06:41.976556+0000","last_undegraded":"2024-05-08T22:06:41.976556+0000","last_fullsized":"2024-05-08T22:06:41.976556+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:51.167142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.117273+0000","last_change":"2024-05-08T22:05:13.551212+0000","last_active":"2024-05-08T22:07:17.117273+0000","last_peered":"2024-05-08T22:07:17.117273+0000","last_clean":"2024-05-08T22:07:17.117273+0000","last_became_active":"2024-05-08T22:05:13.550909+0000","last_became_peered":"2024-05-08T22:05:13.550909+0000","last_unstale":"2024-05-08T22:07:17.117273+0000","last_undegraded":"2024-05-08T22:07:17.117273+0000","last_fullsized":"2024-05-08T22:07:17.117273+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:48.657928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.123447+0000","last_change":"2024-05-08T22:05:14.053976+0000","last_active":"2024-05-08T22:07:17.123447+0000","last_peered":"2024-05-08T22:07:17.123447+0000","last_clean":"2024-05-08T22:07:17.123447+0000","last_became_active":"2024-05-08T22:05:14.053664+0000","last_became_peered":"2024-05-08T22:05:14.053664+0000","last_unstale":"2024-05-08T22:07:17.123447+0000","last_undegraded":"2024-05-08T22:07:17.123447+0000","last_fullsized":"2024-05-08T22:07:17.123447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:25:25.642768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.978021+0000","last_change":"2024-05-08T22:05:18.536771+0000","last_active":"2024-05-08T22:06:41.978021+0000","last_peered":"2024-05-08T22:06:41.978021+0000","last_clean":"2024-05-08T22:06:41.978021+0000","last_became_active":"2024-05-08T22:05:18.536260+0000","last_became_peered":"2024-05-08T22:05:18.536260+0000","last_unstale":"2024-05-08T22:06:41.978021+0000","last_undegraded":"2024-05-08T22:06:41.978021+0000","last_fullsized":"2024-05-08T22:06:41.978021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:34.414381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.979065+0000","last_change":"2024-05-08T22:05:13.540679+0000","last_active":"2024-05-08T22:06:41.979065+0000","last_peered":"2024-05-08T22:06:41.979065+0000","last_clean":"2024-05-08T22:06:41.979065+0000","last_became_active":"2024-05-08T22:05:13.539428+0000","last_became_peered":"2024-05-08T22:05:13.539428+0000","last_unstale":"2024-05-08T22:06:41.979065+0000","last_undegraded":"2024-05-08T22:06:41.979065+0000","last_fullsized":"2024-05-08T22:06:41.979065+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:59:34.455411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.153688+0000","last_change":"2024-05-08T22:05:14.061849+0000","last_active":"2024-05-08T22:07:17.153688+0000","last_peered":"2024-05-08T22:07:17.153688+0000","last_clean":"2024-05-08T22:07:17.153688+0000","last_became_active":"2024-05-08T22:05:14.061028+0000","last_became_peered":"2024-05-08T22:05:14.061028+0000","last_unstale":"2024-05-08T22:07:17.153688+0000","last_undegraded":"2024-05-08T22:07:17.153688+0000","last_fullsized":"2024-05-08T22:07:17.153688+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:11:59.489376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976549+0000","last_change":"2024-05-08T22:05:18.517994+0000","last_active":"2024-05-08T22:06:41.976549+0000","last_peered":"2024-05-08T22:06:41.976549+0000","last_clean":"2024-05-08T22:06:41.976549+0000","last_became_active":"2024-05-08T22:05:18.517655+0000","last_became_peered":"2024-05-08T22:05:18.517655+0000","last_unstale":"2024-05-08T22:06:41.976549+0000","last_undegraded":"2024-05-08T22:06:41.976549+0000","last_fullsized":"2024-05-08T22:06:41.976549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:50:08.956277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"84'1","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.447717+0000","last_change":"2024-05-08T22:05:13.552851+0000","last_active":"2024-05-08T22:06:42.447717+0000","last_peered":"2024-05-08T22:06:42.447717+0000","last_clean":"2024-05-08T22:06:42.447717+0000","last_became_active":"2024-05-08T22:05:13.552507+0000","last_became_peered":"2024-05-08T22:05:13.552507+0000","last_unstale":"2024-05-08T22:06:42.447717+0000","last_undegraded":"2024-05-08T22:06:42.447717+0000","last_fullsized":"2024-05-08T22:06:42.447717+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:29.248623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.176835+0000","last_change":"2024-05-08T22:05:14.062618+0000","last_active":"2024-05-08T22:07:17.176835+0000","last_peered":"2024-05-08T22:07:17.176835+0000","last_clean":"2024-05-08T22:07:17.176835+0000","last_became_active":"2024-05-08T22:05:14.062308+0000","last_became_peered":"2024-05-08T22:05:14.062308+0000","last_unstale":"2024-05-08T22:07:17.176835+0000","last_undegraded":"2024-05-08T22:07:17.176835+0000","last_fullsized":"2024-05-08T22:07:17.176835+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:34.195343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976573+0000","last_change":"2024-05-08T22:05:18.616846+0000","last_active":"2024-05-08T22:06:41.976573+0000","last_peered":"2024-05-08T22:06:41.976573+0000","last_clean":"2024-05-08T22:06:41.976573+0000","last_became_active":"2024-05-08T22:05:18.616369+0000","last_became_peered":"2024-05-08T22:05:18.616369+0000","last_unstale":"2024-05-08T22:06:41.976573+0000","last_undegraded":"2024-05-08T22:06:41.976573+0000","last_fullsized":"2024-05-08T22:06:41.976573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:26.128949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:41.976543+0000","last_change":"2024-05-08T22:05:13.549712+0000","last_active":"2024-05-08T22:06:41.976543+0000","last_peered":"2024-05-08T22:06:41.976543+0000","last_clean":"2024-05-08T22:06:41.976543+0000","last_became_active":"2024-05-08T22:05:13.549508+0000","last_became_peered":"2024-05-08T22:05:13.549508+0000","last_unstale":"2024-05-08T22:06:41.976543+0000","last_undegraded":"2024-05-08T22:06:41.976543+0000","last_fullsized":"2024-05-08T22:06:41.976543+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:30:22.706389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.157838+0000","last_change":"2024-05-08T22:05:14.060920+0000","last_active":"2024-05-08T22:07:17.157838+0000","last_peered":"2024-05-08T22:07:17.157838+0000","last_clean":"2024-05-08T22:07:17.157838+0000","last_became_active":"2024-05-08T22:05:14.059914+0000","last_became_peered":"2024-05-08T22:05:14.059914+0000","last_unstale":"2024-05-08T22:07:17.157838+0000","last_undegraded":"2024-05-08T22:07:17.157838+0000","last_fullsized":"2024-05-08T22:07:17.157838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:15.513600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.276872+0000","last_change":"2024-05-08T22:05:18.534076+0000","last_active":"2024-05-08T22:06:42.276872+0000","last_peered":"2024-05-08T22:06:42.276872+0000","last_clean":"2024-05-08T22:06:42.276872+0000","last_became_active":"2024-05-08T22:05:18.533679+0000","last_became_peered":"2024-05-08T22:05:18.533679+0000","last_unstale":"2024-05-08T22:06:42.276872+0000","last_undegraded":"2024-05-08T22:06:42.276872+0000","last_fullsized":"2024-05-08T22:06:42.276872+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:08.979193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.276939+0000","last_change":"2024-05-08T22:05:41.845354+0000","last_active":"2024-05-08T22:06:42.276939+0000","last_peered":"2024-05-08T22:06:42.276939+0000","last_clean":"2024-05-08T22:06:42.276939+0000","last_became_active":"2024-05-08T22:05:41.830506+0000","last_became_peered":"2024-05-08T22:05:41.830506+0000","last_unstale":"2024-05-08T22:06:42.276939+0000","last_undegraded":"2024-05-08T22:06:42.276939+0000","last_fullsized":"2024-05-08T22:06:42.276939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:53:23.081939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.153480+0000","last_change":"2024-05-08T22:05:14.065868+0000","last_active":"2024-05-08T22:07:17.153480+0000","last_peered":"2024-05-08T22:07:17.153480+0000","last_clean":"2024-05-08T22:07:17.153480+0000","last_became_active":"2024-05-08T22:05:14.065545+0000","last_became_peered":"2024-05-08T22:05:14.065545+0000","last_unstale":"2024-05-08T22:07:17.153480+0000","last_undegraded":"2024-05-08T22:07:17.153480+0000","last_fullsized":"2024-05-08T22:07:17.153480+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:56:34.309095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.467018+0000","last_change":"2024-05-08T22:05:18.534261+0000","last_active":"2024-05-08T22:06:42.467018+0000","last_peered":"2024-05-08T22:06:42.467018+0000","last_clean":"2024-05-08T22:06:42.467018+0000","last_became_active":"2024-05-08T22:05:18.531814+0000","last_became_peered":"2024-05-08T22:05:18.531814+0000","last_unstale":"2024-05-08T22:06:42.467018+0000","last_undegraded":"2024-05-08T22:06:42.467018+0000","last_fullsized":"2024-05-08T22:06:42.467018+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:38.344546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'39","reported_seq":68,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.397200+0000","last_change":"2024-05-08T22:05:13.549344+0000","last_active":"2024-05-08T22:07:17.397200+0000","last_peered":"2024-05-08T22:07:17.397200+0000","last_clean":"2024-05-08T22:07:17.397200+0000","last_became_active":"2024-05-08T22:05:13.548993+0000","last_became_peered":"2024-05-08T22:05:13.548993+0000","last_unstale":"2024-05-08T22:07:17.397200+0000","last_undegraded":"2024-05-08T22:07:17.397200+0000","last_fullsized":"2024-05-08T22:07:17.397200+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:50.096602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.180642+0000","last_change":"2024-05-08T22:05:14.065758+0000","last_active":"2024-05-08T22:07:17.180642+0000","last_peered":"2024-05-08T22:07:17.180642+0000","last_clean":"2024-05-08T22:07:17.180642+0000","last_became_active":"2024-05-08T22:05:14.065442+0000","last_became_peered":"2024-05-08T22:05:14.065442+0000","last_unstale":"2024-05-08T22:07:17.180642+0000","last_undegraded":"2024-05-08T22:07:17.180642+0000","last_fullsized":"2024-05-08T22:07:17.180642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:43.047796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.180606+0000","last_change":"2024-05-08T22:05:18.529804+0000","last_active":"2024-05-08T22:06:42.180606+0000","last_peered":"2024-05-08T22:06:42.180606+0000","last_clean":"2024-05-08T22:06:42.180606+0000","last_became_active":"2024-05-08T22:05:18.529346+0000","last_became_peered":"2024-05-08T22:05:18.529346+0000","last_unstale":"2024-05-08T22:06:42.180606+0000","last_undegraded":"2024-05-08T22:06:42.180606+0000","last_fullsized":"2024-05-08T22:06:42.180606+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:17.499131+0000","last_clean_scrub_stamp":"2024-05-08T22:05:17.499131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:13.565707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T22:06:42.017625+0000","last_change":"2024-05-08T22:05:13.551818+0000","last_active":"2024-05-08T22:06:42.017625+0000","last_peered":"2024-05-08T22:06:42.017625+0000","last_clean":"2024-05-08T22:06:42.017625+0000","last_became_active":"2024-05-08T22:05:13.551080+0000","last_became_peered":"2024-05-08T22:05:13.551080+0000","last_unstale":"2024-05-08T22:06:42.017625+0000","last_undegraded":"2024-05-08T22:06:42.017625+0000","last_fullsized":"2024-05-08T22:06:42.017625+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:12.519986+0000","last_clean_scrub_stamp":"2024-05-08T22:05:12.519986+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:31.636482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.146513+0000","last_change":"2024-05-08T22:05:14.066572+0000","last_active":"2024-05-08T22:07:17.146513+0000","last_peered":"2024-05-08T22:07:17.146513+0000","last_clean":"2024-05-08T22:07:17.146513+0000","last_became_active":"2024-05-08T22:05:14.066272+0000","last_became_peered":"2024-05-08T22:05:14.066272+0000","last_unstale":"2024-05-08T22:07:17.146513+0000","last_undegraded":"2024-05-08T22:07:17.146513+0000","last_fullsized":"2024-05-08T22:07:17.146513+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:20:27.423464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.151234+0000","last_change":"2024-05-08T22:05:14.061435+0000","last_active":"2024-05-08T22:07:17.151234+0000","last_peered":"2024-05-08T22:07:17.151234+0000","last_clean":"2024-05-08T22:07:17.151234+0000","last_became_active":"2024-05-08T22:05:14.061079+0000","last_became_peered":"2024-05-08T22:05:14.061079+0000","last_unstale":"2024-05-08T22:07:17.151234+0000","last_undegraded":"2024-05-08T22:07:17.151234+0000","last_fullsized":"2024-05-08T22:07:17.151234+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:42.684743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.147601+0000","last_change":"2024-05-08T22:05:41.828352+0000","last_active":"2024-05-08T22:07:17.147601+0000","last_peered":"2024-05-08T22:07:17.147601+0000","last_clean":"2024-05-08T22:07:17.147601+0000","last_became_active":"2024-05-08T22:05:41.827782+0000","last_became_peered":"2024-05-08T22:05:41.827782+0000","last_unstale":"2024-05-08T22:07:17.147601+0000","last_undegraded":"2024-05-08T22:07:17.147601+0000","last_fullsized":"2024-05-08T22:07:17.147601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:08.883148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.176085+0000","last_change":"2024-05-08T22:05:14.066697+0000","last_active":"2024-05-08T22:07:17.176085+0000","last_peered":"2024-05-08T22:07:17.176085+0000","last_clean":"2024-05-08T22:07:17.176085+0000","last_became_active":"2024-05-08T22:05:14.065952+0000","last_became_peered":"2024-05-08T22:05:14.065952+0000","last_unstale":"2024-05-08T22:07:17.176085+0000","last_undegraded":"2024-05-08T22:07:17.176085+0000","last_fullsized":"2024-05-08T22:07:17.176085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:03:19.006644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.179903+0000","last_change":"2024-05-08T22:05:14.072295+0000","last_active":"2024-05-08T22:07:17.179903+0000","last_peered":"2024-05-08T22:07:17.179903+0000","last_clean":"2024-05-08T22:07:17.179903+0000","last_became_active":"2024-05-08T22:05:14.065034+0000","last_became_peered":"2024-05-08T22:05:14.065034+0000","last_unstale":"2024-05-08T22:07:17.179903+0000","last_undegraded":"2024-05-08T22:07:17.179903+0000","last_fullsized":"2024-05-08T22:07:17.179903+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:08:41.380814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.147802+0000","last_change":"2024-05-08T22:05:14.067263+0000","last_active":"2024-05-08T22:07:17.147802+0000","last_peered":"2024-05-08T22:07:17.147802+0000","last_clean":"2024-05-08T22:07:17.147802+0000","last_became_active":"2024-05-08T22:05:14.066957+0000","last_became_peered":"2024-05-08T22:05:14.066957+0000","last_unstale":"2024-05-08T22:07:17.147802+0000","last_undegraded":"2024-05-08T22:07:17.147802+0000","last_fullsized":"2024-05-08T22:07:17.147802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:29:35.638489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.151853+0000","last_change":"2024-05-08T22:05:14.060561+0000","last_active":"2024-05-08T22:07:17.151853+0000","last_peered":"2024-05-08T22:07:17.151853+0000","last_clean":"2024-05-08T22:07:17.151853+0000","last_became_active":"2024-05-08T22:05:14.060150+0000","last_became_peered":"2024-05-08T22:05:14.060150+0000","last_unstale":"2024-05-08T22:07:17.151853+0000","last_undegraded":"2024-05-08T22:07:17.151853+0000","last_fullsized":"2024-05-08T22:07:17.151853+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:22.710776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.148159+0000","last_change":"2024-05-08T22:05:14.056394+0000","last_active":"2024-05-08T22:07:17.148159+0000","last_peered":"2024-05-08T22:07:17.148159+0000","last_clean":"2024-05-08T22:07:17.148159+0000","last_became_active":"2024-05-08T22:05:14.055977+0000","last_became_peered":"2024-05-08T22:05:14.055977+0000","last_unstale":"2024-05-08T22:07:17.148159+0000","last_undegraded":"2024-05-08T22:07:17.148159+0000","last_fullsized":"2024-05-08T22:07:17.148159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:56:48.249379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.161659+0000","last_change":"2024-05-08T22:06:42.448921+0000","last_active":"2024-05-08T22:07:17.161659+0000","last_peered":"2024-05-08T22:07:17.161659+0000","last_clean":"2024-05-08T22:07:17.161659+0000","last_became_active":"2024-05-08T22:06:42.448567+0000","last_became_peered":"2024-05-08T22:06:42.448567+0000","last_unstale":"2024-05-08T22:07:17.161659+0000","last_undegraded":"2024-05-08T22:07:17.161659+0000","last_fullsized":"2024-05-08T22:07:17.161659+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:52.822511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.180232+0000","last_change":"2024-05-08T22:05:14.069982+0000","last_active":"2024-05-08T22:07:17.180232+0000","last_peered":"2024-05-08T22:07:17.180232+0000","last_clean":"2024-05-08T22:07:17.180232+0000","last_became_active":"2024-05-08T22:05:14.068293+0000","last_became_peered":"2024-05-08T22:05:14.068293+0000","last_unstale":"2024-05-08T22:07:17.180232+0000","last_undegraded":"2024-05-08T22:07:17.180232+0000","last_fullsized":"2024-05-08T22:07:17.180232+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:36.396983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.163286+0000","last_change":"2024-05-08T22:05:14.058393+0000","last_active":"2024-05-08T22:07:17.163286+0000","last_peered":"2024-05-08T22:07:17.163286+0000","last_clean":"2024-05-08T22:07:17.163286+0000","last_became_active":"2024-05-08T22:05:14.058083+0000","last_became_peered":"2024-05-08T22:05:14.058083+0000","last_unstale":"2024-05-08T22:07:17.163286+0000","last_undegraded":"2024-05-08T22:07:17.163286+0000","last_fullsized":"2024-05-08T22:07:17.163286+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:10:47.074957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.166671+0000","last_change":"2024-05-08T22:05:14.056989+0000","last_active":"2024-05-08T22:07:17.166671+0000","last_peered":"2024-05-08T22:07:17.166671+0000","last_clean":"2024-05-08T22:07:17.166671+0000","last_became_active":"2024-05-08T22:05:14.056669+0000","last_became_peered":"2024-05-08T22:05:14.056669+0000","last_unstale":"2024-05-08T22:07:17.166671+0000","last_undegraded":"2024-05-08T22:07:17.166671+0000","last_fullsized":"2024-05-08T22:07:17.166671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:56:51.173885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.175205+0000","last_change":"2024-05-08T22:05:14.058635+0000","last_active":"2024-05-08T22:07:17.175205+0000","last_peered":"2024-05-08T22:07:17.175205+0000","last_clean":"2024-05-08T22:07:17.175205+0000","last_became_active":"2024-05-08T22:05:14.058243+0000","last_became_peered":"2024-05-08T22:05:14.058243+0000","last_unstale":"2024-05-08T22:07:17.175205+0000","last_undegraded":"2024-05-08T22:07:17.175205+0000","last_fullsized":"2024-05-08T22:07:17.175205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:18.142563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T22:07:17.146228+0000","last_change":"2024-05-08T22:05:14.060735+0000","last_active":"2024-05-08T22:07:17.146228+0000","last_peered":"2024-05-08T22:07:17.146228+0000","last_clean":"2024-05-08T22:07:17.146228+0000","last_became_active":"2024-05-08T22:05:14.060391+0000","last_became_peered":"2024-05-08T22:05:14.060391+0000","last_unstale":"2024-05-08T22:07:17.146228+0000","last_undegraded":"2024-05-08T22:07:17.146228+0000","last_fullsized":"2024-05-08T22:07:17.146228+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:05:13.522431+0000","last_clean_scrub_stamp":"2024-05-08T22:05:13.522431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:30.528705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107988,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599518,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":747,"num_read_kb":483,"num_write":283,"num_write_kb":1441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":694,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1798554,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579940,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36156,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708996,"statfs":{"total":95995035648,"available":95958011904,"internally_reserved":0,"allocated":966656,"data_stored":498801,"data_compressed":26516,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36043205},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.54100000000000004,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.67000000000000004,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.48199999999999998}]},{"osd":1,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.65600000000000003,"15min":0.65300000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.878,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76800000000000002,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.373}]},{"osd":2,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.36299999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.65500000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.309},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.48599999999999999,"15min":0.47799999999999998},"min":{"1min":0.34699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72799999999999998,"5min":1.052,"15min":1.052},"last":0.27500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":4,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.53100000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.433},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.752,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.41999999999999998}]},{"osd":5,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.53200000000000003,"15min":0.52300000000000002},"min":{"1min":0.38,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82999999999999996,"5min":1.204,"15min":1.204},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.61299999999999999,"15min":0.61099999999999999},"min":{"1min":0.42299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86799999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.34899999999999998}]},{"osd":6,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.65800000000000003,"15min":0.66500000000000004},"min":{"1min":0.48099999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91500000000000004,"5min":1.278,"15min":1.278},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.65700000000000003,"15min":0.66000000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.85199999999999998,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.46899999999999997}]},{"osd":7,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.745,"15min":0.74299999999999999},"min":{"1min":0.58999999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.96899999999999997,"5min":1.415,"15min":1.415},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.74099999999999999,"15min":0.746},"min":{"1min":0.60199999999999998,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.86799999999999999,"5min":1.48,"15min":1.48},"last":0.45200000000000001}]},{"osd":8,"last update":"Wed May 8 22:07:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.67000000000000004,"15min":0.67500000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.85699999999999998,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.505},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.73099999999999998,"15min":0.73699999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.99299999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.52700000000000002}]},{"osd":9,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76300000000000001,"15min":0.76400000000000001},"min":{"1min":0.59399999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.97899999999999998,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.65300000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0369999999999999,"5min":1.5,"15min":1.5},"last":0.58399999999999996}]},{"osd":10,"last update":"Wed May 8 22:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.78900000000000003,"15min":0.79600000000000004},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.96399999999999997,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78100000000000003,"15min":0.78400000000000003},"min":{"1min":0.61899999999999999,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93899999999999995,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.56799999999999995}]}]},{"osd":10,"up_from":67,"seq":287762808871,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1134592,"data_stored":1287551,"data_compressed":68668,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:07:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53900000000000003,"15min":0.54000000000000004},"min":{"1min":0.40100000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.747,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.874,"15min":0.92200000000000004},"min":{"1min":0.44,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.76700000000000002,"5min":7.258,"15min":7.258},"last":0.46800000000000003}]},{"osd":1,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.437,"15min":0.438},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.46700000000000003,"15min":0.46999999999999997},"min":{"1min":0.30599999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.65400000000000003,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.27800000000000002}]},{"osd":2,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.624,"15min":0.61899999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59999999999999998,"15min":0.60199999999999998},"min":{"1min":0.40600000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76000000000000001,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.50900000000000001}]},{"osd":3,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.77500000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.61099999999999999,"15min":0.629},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73499999999999999,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.40999999999999998}]},{"osd":4,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54200000000000004,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76500000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.317},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.315,"5min":0.309,"15min":0.309},"max":{"1min":0.79500000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.32500000000000001}]},{"osd":5,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64600000000000002,"15min":0.64000000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.379},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.65600000000000003,"15min":0.66000000000000003},"min":{"1min":0.47999999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.47699999999999998}]},{"osd":6,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.71699999999999997,"15min":0.67700000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.87,"5min":7.2380000000000004,"15min":7.2380000000000004},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.69599999999999995,"15min":0.69199999999999995},"min":{"1min":0.53500000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.88200000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.45800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":8,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71299999999999997,"15min":0.72499999999999998},"min":{"1min":0.51500000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.372},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.72799999999999998,"15min":0.73399999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.436}]},{"osd":9,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.747,"15min":0.755},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.82499999999999996,"15min":0.78600000000000003},"min":{"1min":0.57499999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.92800000000000005,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.51900000000000002}]},{"osd":11,"last update":"Wed May 8 22:06:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.71399999999999997,"15min":0.70599999999999996},"min":{"1min":0.60099999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73899999999999999,"15min":0.73499999999999999},"min":{"1min":0.61899999999999999,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.49099999999999999}]}]},{"osd":9,"up_from":62,"seq":266287972395,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":896349,"data_compressed":45662,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.50900000000000001,"15min":0.52400000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.60999999999999999,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.57099999999999995,"15min":0.58999999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0409999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.47499999999999998}]},{"osd":1,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.73899999999999999,"15min":0.77100000000000002},"min":{"1min":0.47099999999999997,"5min":0.438,"15min":0.438},"max":{"1min":0.80500000000000005,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.46300000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84699999999999998,"5min":0.997,"15min":0.997},"last":0.65500000000000003}]},{"osd":2,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.75800000000000001,"15min":0.78600000000000003},"min":{"1min":0.56000000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81699999999999995,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.72199999999999998,"15min":0.73999999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.878,"5min":1.167,"15min":1.167},"last":0.63900000000000001}]},{"osd":3,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.73899999999999999,"15min":0.79100000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90300000000000002,"5min":1.024,"15min":1.024},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.60699999999999998},"min":{"1min":0.41499999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.81200000000000006,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.54100000000000004}]},{"osd":4,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.75900000000000001,"15min":0.78300000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96999999999999997,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.80000000000000004,"15min":0.83199999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.86799999999999999,"5min":1.081,"15min":1.081},"last":0.88100000000000001}]},{"osd":5,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.81299999999999994,"15min":0.85499999999999998},"min":{"1min":0.49399999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.93200000000000005,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.84699999999999998,"15min":0.88600000000000001},"min":{"1min":0.53700000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.92400000000000004,"5min":1.175,"15min":1.175},"last":0.85599999999999998}]},{"osd":6,"last update":"Wed May 8 22:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.71199999999999997}]},{"osd":7,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76300000000000001,"15min":0.78400000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.87,"15min":0.91900000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0109999999999999,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.79700000000000004}]},{"osd":8,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80500000000000005,"15min":0.82899999999999996},"min":{"1min":0.60599999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.181,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.90100000000000002,"15min":0.94299999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2509999999999999,"5min":1.478,"15min":1.478},"last":0.66900000000000004}]},{"osd":10,"last update":"Wed May 8 22:07:22 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.86599999999999999,"15min":0.89000000000000001},"min":{"1min":0.56299999999999994,"5min":0.495,"15min":0.495},"max":{"1min":1.2170000000000001,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.84999999999999998,"15min":0.876},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.95099999999999996,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.89900000000000002}]},{"osd":11,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.85599999999999998,"15min":0.85899999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1990000000000001,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.83999999999999997,"15min":0.84599999999999997},"min":{"1min":0.61499999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.98199999999999998,"5min":1.105,"15min":1.105},"last":0.91700000000000004}]}]},{"osd":8,"up_from":55,"seq":236223201328,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":891842,"data_compressed":45662,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.40300000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81699999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.60399999999999998,"15min":0.626},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69899999999999995,"5min":1.865,"15min":1.865},"last":0.58099999999999996}]},{"osd":1,"last update":"Wed May 8 22:05:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.60299999999999998,"15min":0.59399999999999997},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.498},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55700000000000005,"15min":0.55100000000000005},"min":{"1min":0.40799999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.42399999999999999}]},{"osd":2,"last update":"Wed May 8 22:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.437,"15min":0.42299999999999999},"min":{"1min":0.315,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.46999999999999997,"15min":0.46500000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68100000000000005,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.377}]},{"osd":3,"last update":"Wed May 8 22:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.72299999999999998,"15min":0.79300000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77900000000000003,"5min":4.8920000000000003,"15min":4.8920000000000003},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63400000000000001,"15min":0.626},"min":{"1min":0.41999999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.51500000000000001}]},{"osd":4,"last update":"Wed May 8 22:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.46000000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.64000000000000001,"15min":0.61399999999999999},"min":{"1min":0.497,"5min":0.433,"15min":0.433},"max":{"1min":0.871,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.47799999999999998}]},{"osd":5,"last update":"Wed May 8 22:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51200000000000001,"15min":0.50600000000000001},"min":{"1min":0.34399999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.72899999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.54200000000000004,"15min":0.51800000000000002},"min":{"1min":0.39900000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.35699999999999998}]},{"osd":6,"last update":"Wed May 8 22:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.83799999999999997,"15min":0.89400000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.98799999999999999,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.90600000000000003,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.61699999999999999}]},{"osd":7,"last update":"Wed May 8 22:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66400000000000003,"15min":0.66200000000000003},"min":{"1min":0.45000000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.504},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.68600000000000005,"15min":0.67000000000000004},"min":{"1min":0.51100000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.495}]},{"osd":9,"last update":"Wed May 8 22:07:05 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78300000000000003,"15min":0.77600000000000002},"min":{"1min":0.57299999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73299999999999998,"15min":0.73099999999999998},"min":{"1min":0.53600000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92300000000000004,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.56399999999999995}]},{"osd":10,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.71799999999999997,"15min":0.70499999999999996},"min":{"1min":0.55000000000000004,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94699999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.76400000000000001,"15min":0.752},"min":{"1min":0.68400000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.57399999999999995}]},{"osd":11,"last update":"Wed May 8 22:06:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.59399999999999997}]}]},{"osd":7,"up_from":48,"seq":206158430257,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":1096485,"data_compressed":62512,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.41199999999999998}]},{"osd":1,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.38800000000000001},"min":{"1min":0.29699999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.57499999999999996,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40999999999999998,"15min":0.40400000000000003},"min":{"1min":0.27900000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.55400000000000005,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.36399999999999999}]},{"osd":2,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.41599999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71499999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.57599999999999996,"15min":0.56999999999999995},"min":{"1min":0.46200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80400000000000005,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.38200000000000001}]},{"osd":3,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.41299999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76800000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68400000000000005,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.48499999999999999}]},{"osd":4,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.49399999999999999,"15min":0.49299999999999999},"min":{"1min":0.35899999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.505,"15min":0.51400000000000001},"min":{"1min":0.34599999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69499999999999995,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.39300000000000002}]},{"osd":5,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.46800000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77500000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63,"15min":0.628},"min":{"1min":0.495,"5min":0.436,"15min":0.436},"max":{"1min":0.83999999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.438}]},{"osd":6,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65800000000000003,"15min":0.66800000000000004},"min":{"1min":0.504,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.80900000000000005,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.67700000000000005,"15min":0.69299999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84099999999999997,"5min":1.002,"15min":1.002},"last":0.58099999999999996}]},{"osd":8,"last update":"Wed May 8 22:06:39 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.68000000000000005,"15min":0.69199999999999995},"min":{"1min":0.48599999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.82299999999999995,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.499},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71599999999999997,"15min":0.72599999999999998},"min":{"1min":0.54700000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.81799999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.52900000000000003}]},{"osd":9,"last update":"Wed May 8 22:07:05 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.624,"5min":0.501,"15min":0.501},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.73399999999999999,"15min":0.74199999999999999},"min":{"1min":0.59699999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.629}]},{"osd":10,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.69999999999999996,"15min":0.71099999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.82099999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.68500000000000005},"min":{"1min":0.499,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.86799999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.47399999999999998}]},{"osd":11,"last update":"Wed May 8 22:07:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.75,"15min":0.748},"min":{"1min":0.61699999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.59399999999999997}]}]},{"osd":6,"up_from":43,"seq":184683593782,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":493940,"data_compressed":26516,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:07:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.47799999999999998,"15min":0.45600000000000002},"min":{"1min":0.35899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.54000000000000004}]},{"osd":1,"last update":"Wed May 8 22:07:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60099999999999998,"15min":0.57499999999999996},"min":{"1min":0.44800000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.65100000000000002,"15min":0.61699999999999999},"min":{"1min":0.46500000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.73999999999999999}]},{"osd":2,"last update":"Wed May 8 22:07:10 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.69099999999999995,"15min":0.65800000000000003},"min":{"1min":0.51600000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.67400000000000004,"15min":0.65400000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.755}]},{"osd":3,"last update":"Wed May 8 22:07:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.42099999999999999,"5min":0.375,"15min":0.375},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.64900000000000002,"15min":0.64100000000000001},"min":{"1min":0.434,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.81299999999999994}]},{"osd":4,"last update":"Wed May 8 22:07:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.45000000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.53300000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":0.96199999999999997}]},{"osd":5,"last update":"Wed May 8 22:07:10 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.76900000000000002,"15min":0.749},"min":{"1min":0.56499999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.753,"15min":0.75},"min":{"1min":0.59399999999999997,"5min":0.438,"15min":0.438},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.044}]},{"osd":7,"last update":"Wed May 8 22:06:28 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.747,"15min":0.73999999999999999},"min":{"1min":0.54900000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.9690000000000001,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.77700000000000002,"15min":0.76500000000000001},"min":{"1min":0.63200000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.86599999999999999}]},{"osd":8,"last update":"Wed May 8 22:06:38 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.83799999999999997,"15min":0.84199999999999997},"min":{"1min":0.58199999999999996,"5min":0.496,"15min":0.496},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.55200000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.097}]},{"osd":9,"last update":"Wed May 8 22:07:04 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.85699999999999998,"15min":0.85899999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.81699999999999995,"15min":0.83499999999999996},"min":{"1min":0.51000000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.1299999999999999}]},{"osd":10,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.85799999999999998,"15min":0.86799999999999999},"min":{"1min":0.66600000000000004,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":0.95899999999999996,"5min":1.639,"15min":1.639},"last":1.165},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.82799999999999996,"15min":0.81999999999999995},"min":{"1min":0.60699999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0369999999999999,"5min":1.927,"15min":1.927},"last":1.1140000000000001}]},{"osd":11,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.91700000000000004,"15min":0.92900000000000005},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.127,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.88500000000000001,"15min":0.89800000000000002},"min":{"1min":0.68000000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.022,"5min":1.78,"15min":1.78},"last":1.0029999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":1036288,"data_stored":560994,"data_compressed":26516,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:06:31 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.55100000000000005,"15min":0.501},"min":{"1min":0.40400000000000003,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.85699999999999998,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55200000000000005,"15min":0.51800000000000002},"min":{"1min":0.44900000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76000000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.5}]},{"osd":2,"last update":"Wed May 8 22:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58099999999999996,"15min":0.50800000000000001},"min":{"1min":0.45400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.77400000000000002,"5min":0.877,"15min":0.877},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.56899999999999995,"15min":0.505},"min":{"1min":0.39300000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76200000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.36899999999999999}]},{"osd":3,"last update":"Wed May 8 22:07:08 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.57899999999999996,"15min":0.52000000000000002},"min":{"1min":0.495,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70799999999999996,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55000000000000004,"15min":0.51800000000000002},"min":{"1min":0.41799999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71099999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.44500000000000001}]},{"osd":4,"last update":"Wed May 8 22:06:31 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.42399999999999999,"15min":0.38400000000000001},"min":{"1min":0.32500000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.80600000000000005,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.45600000000000002,"15min":0.40300000000000002},"min":{"1min":0.35599999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.77600000000000002,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.52900000000000003}]},{"osd":5,"last update":"Wed May 8 22:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.622,"15min":0.56799999999999995},"min":{"1min":0.53600000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.84199999999999997,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.61799999999999999,"15min":0.58399999999999996},"min":{"1min":0.46800000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80000000000000004,"5min":1.089,"15min":1.089},"last":0.46100000000000002}]},{"osd":6,"last update":"Wed May 8 22:07:08 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.626,"15min":0.60599999999999998},"min":{"1min":0.47999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66600000000000004,"15min":0.628},"min":{"1min":0.47899999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.93400000000000005,"5min":1.05,"15min":1.05},"last":0.47399999999999998}]},{"osd":7,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58999999999999997,"15min":0.57199999999999995},"min":{"1min":0.44400000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.871,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.39100000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74199999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.34200000000000003}]},{"osd":8,"last update":"Wed May 8 22:06:42 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65200000000000002,"15min":0.65900000000000003},"min":{"1min":0.53600000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.78300000000000003,"5min":1.115,"15min":1.115},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.53600000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.92100000000000004,"5min":1.222,"15min":1.222},"last":0.53700000000000003}]},{"osd":9,"last update":"Wed May 8 22:07:04 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.70899999999999996,"15min":0.71899999999999997},"min":{"1min":0.54400000000000004,"5min":0.501,"15min":0.501},"max":{"1min":0.86699999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80100000000000005,"15min":0.79600000000000004},"min":{"1min":0.71199999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0860000000000001,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.59899999999999998}]},{"osd":10,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.76200000000000001,"15min":0.76400000000000001},"min":{"1min":0.57199999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0389999999999999,"5min":1.179,"15min":1.179},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.56699999999999995,"5min":0.435,"15min":0.435},"max":{"1min":0.94899999999999995,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.64100000000000001}]},{"osd":11,"last update":"Wed May 8 22:07:14 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.83599999999999997,"15min":0.84099999999999997},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.78000000000000003,"15min":0.78300000000000003},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.98699999999999999,"5min":0.996,"15min":0.996},"last":0.69299999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":983040,"data_stored":495328,"data_compressed":26516,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:06:29 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.60699999999999998,"15min":0.55800000000000005},"min":{"1min":0.45200000000000001,"5min":0.31,"15min":0.31},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.58399999999999996,"15min":0.53400000000000003},"min":{"1min":0.41899999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.55700000000000005}]},{"osd":2,"last update":"Wed May 8 22:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.64100000000000001,"15min":0.56599999999999995},"min":{"1min":0.45100000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61699999999999999,"15min":0.55900000000000005},"min":{"1min":0.53900000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86399999999999999,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.625}]},{"osd":3,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53600000000000003,"15min":0.53100000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66800000000000004,"15min":0.66400000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.78300000000000003}]},{"osd":4,"last update":"Wed May 8 22:06:29 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.59899999999999998,"15min":0.55800000000000005},"min":{"1min":0.52400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95999999999999996,"5min":1.218,"15min":1.218},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.70399999999999996,"15min":0.69899999999999995},"min":{"1min":0.56699999999999995,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94599999999999995,"5min":2.984,"15min":2.984},"last":0.84799999999999998}]},{"osd":5,"last update":"Wed May 8 22:06:46 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.70599999999999996,"15min":0.65600000000000003},"min":{"1min":0.501,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0760000000000001,"5min":1.129,"15min":1.129},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68400000000000005,"15min":0.66600000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.69499999999999995}]},{"osd":6,"last update":"Wed May 8 22:07:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.625,"15min":0.58899999999999997},"min":{"1min":0.40300000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90300000000000002,"5min":1.254,"15min":1.254},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70999999999999996,"15min":0.67700000000000005},"min":{"1min":0.443,"5min":0.375,"15min":0.375},"max":{"1min":1.0089999999999999,"5min":1.208,"15min":1.208},"last":0.92500000000000004}]},{"osd":7,"last update":"Wed May 8 22:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.72699999999999998}]},{"osd":8,"last update":"Wed May 8 22:06:41 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.83199999999999996,"15min":0.82899999999999996},"min":{"1min":0.65000000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1080000000000001,"5min":1.427,"15min":1.427},"last":1.024},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.80200000000000005,"15min":0.80500000000000005},"min":{"1min":0.64800000000000002,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.76200000000000001}]},{"osd":9,"last update":"Wed May 8 22:07:07 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78900000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.875,"15min":0.89200000000000002},"min":{"1min":0.47999999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1379999999999999,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.089}]},{"osd":10,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.61599999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1339999999999999,"5min":1.2,"15min":1.2},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.876,"15min":0.879},"min":{"1min":0.69599999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.77200000000000002}]},{"osd":11,"last update":"Wed May 8 22:06:41 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.88500000000000001,"15min":0.874},"min":{"1min":0.76100000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":1.056},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.86899999999999999,"15min":0.86099999999999999},"min":{"1min":0.70499999999999996,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.80700000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":1040384,"data_stored":561278,"data_compressed":26516,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.63100000000000001,"15min":0.55100000000000005},"min":{"1min":0.44900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":6.4640000000000004,"5min":6.4640000000000004,"15min":6.4640000000000004},"last":0.503},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.54300000000000004,"15min":0.51500000000000001},"min":{"1min":0.46600000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.47299999999999998}]},{"osd":1,"last update":"Wed May 8 22:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.56599999999999995,"15min":0.53800000000000003},"min":{"1min":0.42699999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.60599999999999998,"15min":0.56499999999999995},"min":{"1min":0.38400000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.81200000000000006}]},{"osd":3,"last update":"Wed May 8 22:07:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.59899999999999998,"15min":0.56699999999999995},"min":{"1min":0.50700000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.90100000000000002,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59699999999999998,"15min":0.56899999999999995},"min":{"1min":0.44900000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67200000000000004}]},{"osd":4,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.755,"15min":0.76900000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.70199999999999996,"15min":0.71199999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.77200000000000002}]},{"osd":5,"last update":"Wed May 8 22:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.51100000000000001,"15min":0.45800000000000002},"min":{"1min":0.41099999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80400000000000005,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.496,"15min":0.45800000000000002},"min":{"1min":0.435,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.52900000000000003}]},{"osd":6,"last update":"Wed May 8 22:07:09 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.53400000000000003,"5min":0.439,"15min":0.439},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.70999999999999996,"15min":0.68200000000000005},"min":{"1min":0.54800000000000004,"5min":0.442,"15min":0.442},"max":{"1min":0.98899999999999999,"5min":0.998,"15min":0.998},"last":0.63700000000000001}]},{"osd":7,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.875},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.84599999999999997}]},{"osd":8,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.63500000000000001,"15min":0.59299999999999997},"min":{"1min":0.505,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.626,"15min":0.58399999999999996},"min":{"1min":0.54100000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90500000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.78900000000000003}]},{"osd":9,"last update":"Wed May 8 22:07:05 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.77300000000000002,"15min":0.72699999999999998},"min":{"1min":0.73099999999999998,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.024,"5min":1.101,"15min":1.101},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.76100000000000001,"15min":0.71899999999999997},"min":{"1min":0.71099999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.996,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.80200000000000005}]},{"osd":10,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.73899999999999999,"15min":0.71299999999999997},"min":{"1min":0.65700000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.72199999999999998,"15min":0.70399999999999996},"min":{"1min":0.66800000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.83699999999999997}]},{"osd":11,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.86199999999999999,"15min":0.871},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.81200000000000006,"15min":0.82199999999999995},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.95199999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.754}]}]},{"osd":3,"up_from":24,"seq":103079215169,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":1019223,"data_compressed":51818,"data_compressed_allocated":69632,"data_compressed_original":589824,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:07:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.49099999999999999,"15min":0.41699999999999998},"min":{"1min":0.33300000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.89000000000000001,"5min":3.2530000000000001,"15min":3.2530000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.52800000000000002,"15min":0.436},"min":{"1min":0.30299999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.1619999999999999,"5min":3.4460000000000002,"15min":3.4460000000000002},"last":0.58199999999999996}]},{"osd":1,"last update":"Wed May 8 22:07:14 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.68100000000000005,"15min":0.58499999999999996},"min":{"1min":0.41399999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79300000000000004,"5min":3.7090000000000001,"15min":3.7090000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66200000000000003,"15min":0.56200000000000006},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.3899999999999999,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":0.55800000000000005}]},{"osd":2,"last update":"Wed May 8 22:07:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71099999999999997,"15min":0.621},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.258,"5min":3.577,"15min":3.577},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.71499999999999997,"15min":0.63500000000000001},"min":{"1min":0.45600000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97299999999999998,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.45800000000000002}]},{"osd":4,"last update":"Wed May 8 22:06:25 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.67700000000000005,"15min":0.61299999999999999},"min":{"1min":0.46600000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":3.8109999999999999,"5min":3.8109999999999999,"15min":3.8109999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.72199999999999998,"15min":0.64000000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.847,"5min":3.847,"15min":3.847},"last":0.78900000000000003}]},{"osd":5,"last update":"Wed May 8 22:06:44 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.73799999999999999,"15min":0.66500000000000004},"min":{"1min":0.47999999999999998,"5min":0.433,"15min":0.433},"max":{"1min":3.9260000000000002,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.70699999999999996,"15min":0.63},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.54,"5min":3.54,"15min":3.54},"last":0.628}]},{"osd":6,"last update":"Wed May 8 22:07:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69599999999999995,"15min":0.63700000000000001},"min":{"1min":0.44600000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.157,"5min":3.653,"15min":3.653},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72499999999999998,"15min":0.65000000000000002},"min":{"1min":0.47599999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96299999999999997,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.495}]},{"osd":7,"last update":"Wed May 8 22:06:25 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.73999999999999999,"15min":0.72299999999999998},"min":{"1min":0.55500000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.86399999999999999,"15min":0.84699999999999998},"min":{"1min":0.57599999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":3.887,"5min":3.887,"15min":3.887},"last":0.71799999999999997}]},{"osd":8,"last update":"Wed May 8 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.53100000000000003}]},{"osd":9,"last update":"Wed May 8 22:07:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.88600000000000001,"15min":0.87},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.179,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.873},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.80400000000000005,"15min":0.79600000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.454,"5min":3.3940000000000001,"15min":3.3940000000000001},"last":0.80800000000000005}]},{"osd":10,"last update":"Wed May 8 22:07:22 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.84699999999999998,"15min":0.83699999999999997},"min":{"1min":0.60599999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4850000000000001,"5min":3.298,"15min":3.298},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.871,"15min":0.85999999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1240000000000001,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":0.70099999999999996}]},{"osd":11,"last update":"Wed May 8 22:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.99399999999999999,"15min":1.032},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.204,"5min":3.6150000000000002,"15min":3.6150000000000002},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.82999999999999996,"5min":1.034,"15min":1.0680000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.014,"5min":3.9950000000000001,"15min":3.9950000000000001},"last":0.84099999999999997}]}]},{"osd":4,"up_from":31,"seq":133143986239,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":500106,"data_compressed":26516,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.44700000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.51900000000000002,"15min":0.50900000000000001},"min":{"1min":0.45000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.629}]},{"osd":1,"last update":"Wed May 8 22:06:28 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.39100000000000001,"15min":0.35699999999999998},"min":{"1min":0.33500000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.40000000000000002,"15min":0.36199999999999999},"min":{"1min":0.36299999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.24299999999999999}]},{"osd":2,"last update":"Wed May 8 22:06:28 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.53600000000000003,"15min":0.51100000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82099999999999995,"5min":0.88,"15min":0.88},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54800000000000004,"15min":0.51400000000000001},"min":{"1min":0.436,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.52200000000000002}]},{"osd":3,"last update":"Wed May 8 22:06:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.56699999999999995},"min":{"1min":0.46000000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.70499999999999996,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57299999999999995,"15min":0.55500000000000005},"min":{"1min":0.439,"5min":0.377,"15min":0.377},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.51200000000000001}]},{"osd":5,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57399999999999995,"15min":0.54000000000000004},"min":{"1min":0.49299999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77400000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59399999999999997,"15min":0.55700000000000005},"min":{"1min":0.48399999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.80200000000000005,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.53800000000000003}]},{"osd":6,"last update":"Wed May 8 22:07:07 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.65600000000000003,"15min":0.63800000000000001},"min":{"1min":0.49299999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.75600000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.628,"15min":0.61599999999999999},"min":{"1min":0.45700000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.38900000000000001}]},{"osd":7,"last update":"Wed May 8 22:06:28 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57999999999999996,"15min":0.58599999999999997},"min":{"1min":0.45900000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.46200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.503}]},{"osd":8,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.69199999999999995,"15min":0.68799999999999994},"min":{"1min":0.58699999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93000000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.60199999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.874,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.56899999999999995}]},{"osd":9,"last update":"Wed May 8 22:07:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69499999999999995,"15min":0.69799999999999995},"min":{"1min":0.53500000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.749,"15min":0.755},"min":{"1min":0.61699999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.48699999999999999}]},{"osd":10,"last update":"Wed May 8 22:06:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.73599999999999999,"15min":0.73999999999999999},"min":{"1min":0.55800000000000005,"5min":0.434,"15min":0.434},"max":{"1min":0.84799999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.627,"15min":0.625},"min":{"1min":0.50600000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.34300000000000003}]},{"osd":11,"last update":"Wed May 8 22:07:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.75,"15min":0.747},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.90200000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.747,"15min":0.747},"min":{"1min":0.60699999999999998,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":0.93400000000000005,"5min":1.052,"15min":1.052},"last":0.54800000000000004}]}]},{"osd":5,"up_from":36,"seq":154618822715,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":1060864,"data_stored":1222542,"data_compressed":68668,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.41799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.55000000000000004,"15min":0.53800000000000003},"min":{"1min":0.40000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70699999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.40600000000000003}]},{"osd":1,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.56599999999999995,"15min":0.54000000000000004},"min":{"1min":0.28799999999999998,"5min":0.248,"15min":0.248},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.55500000000000005,"15min":0.52600000000000002},"min":{"1min":0.34399999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.66800000000000004}]},{"osd":2,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.40200000000000002,"15min":0.376},"min":{"1min":0.33200000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.41499999999999998,"15min":0.39200000000000002},"min":{"1min":0.35499999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67100000000000004,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.38}]},{"osd":3,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.46500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.78100000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.58699999999999997,"15min":0.55900000000000005},"min":{"1min":0.51000000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.62}]},{"osd":4,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.378,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.58199999999999996,"15min":0.56100000000000005},"min":{"1min":0.48399999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.65000000000000002}]},{"osd":6,"last update":"Wed May 8 22:07:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61099999999999999,"15min":0.59299999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61899999999999999,"15min":0.57299999999999995},"min":{"1min":0.505,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48899999999999999}]},{"osd":7,"last update":"Wed May 8 22:06:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.60899999999999999,"15min":0.57699999999999996},"min":{"1min":0.54500000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.61799999999999999,"15min":0.58699999999999997},"min":{"1min":0.51100000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.70099999999999996}]},{"osd":8,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.54000000000000004,"15min":0.50600000000000001},"min":{"1min":0.44600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.624,"15min":0.58199999999999996},"min":{"1min":0.5,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.78300000000000003}]},{"osd":9,"last update":"Wed May 8 22:07:03 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.73299999999999998,"15min":0.70899999999999996},"min":{"1min":0.57899999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.749},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.72199999999999998,"15min":0.69799999999999995},"min":{"1min":0.623,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.63400000000000001}]},{"osd":10,"last update":"Wed May 8 22:06:25 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.752,"15min":0.74299999999999999},"min":{"1min":0.65900000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.74199999999999999,"15min":0.73299999999999998},"min":{"1min":0.66100000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.98599999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.72899999999999998}]},{"osd":11,"last update":"Wed May 8 22:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":69664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133955,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":203391,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:07:26.360 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:07:26.450 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:26 smithi131 ceph-mon[37260]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.2 KiB/s wr, 23 op/s 2024-05-08T22:07:26.450 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:26 smithi131 ceph-mon[37260]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:07:26.450 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:26 smithi131 ceph-mon[37260]: from='client.25547 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:07:26.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:26 smithi157 ceph-mon[77746]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.2 KiB/s wr, 23 op/s 2024-05-08T22:07:26.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:26 smithi157 ceph-mon[77746]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:07:26.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:26 smithi157 ceph-mon[77746]: from='client.25547 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:07:26.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:26 smithi132 ceph-mon[78180]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.2 KiB/s wr, 23 op/s 2024-05-08T22:07:26.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:26 smithi132 ceph-mon[78180]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:07:26.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:26 smithi132 ceph-mon[78180]: from='client.25547 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:07:26.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.590+0000 7f7b65400700 1 -- 172.21.15.131:0/779104956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b60106170 msgr2=0x7f7b6010dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:26.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.590+0000 7f7b65400700 1 --2- 172.21.15.131:0/779104956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b60106170 0x7f7b6010dcf0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7b5800b0a0 tx=0x7f7b5800b3b0 comp rx=0 tx=0).stop 2024-05-08T22:07:26.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.590+0000 7f7b65400700 1 -- 172.21.15.131:0/779104956 shutdown_connections 2024-05-08T22:07:26.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.590+0000 7f7b65400700 1 --2- 172.21.15.131:0/779104956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b60106170 0x7f7b6010dcf0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:26.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.590+0000 7f7b65400700 1 --2- 172.21.15.131:0/779104956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b60101050 0x7f7b60101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:26.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.590+0000 7f7b65400700 1 --2- 172.21.15.131:0/779104956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b60101a00 0x7f7b60105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:26.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.590+0000 7f7b65400700 1 -- 172.21.15.131:0/779104956 >> 172.21.15.131:0/779104956 conn(0x7f7b600fc8f0 msgr2=0x7f7b600fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:26.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.590+0000 7f7b65400700 1 -- 172.21.15.131:0/779104956 shutdown_connections 2024-05-08T22:07:26.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.590+0000 7f7b65400700 1 -- 172.21.15.131:0/779104956 wait complete. 2024-05-08T22:07:26.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.592+0000 7f7b65400700 1 Processor -- start 2024-05-08T22:07:26.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.592+0000 7f7b65400700 1 -- start start 2024-05-08T22:07:26.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.592+0000 7f7b65400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b60101050 0x7f7b601996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:26.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.592+0000 7f7b65400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b60106170 0x7f7b60199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:26.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.592+0000 7f7b65400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b6019dfc0 0x7f7b6019e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:26.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.592+0000 7f7b65400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6019ea40 con 0x7f7b6019dfc0 2024-05-08T22:07:26.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.592+0000 7f7b65400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6019ebb0 con 0x7f7b60106170 2024-05-08T22:07:26.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.592+0000 7f7b65400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6019ed70 con 0x7f7b60101050 2024-05-08T22:07:26.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.593+0000 7f7b5f200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b6019dfc0 0x7f7b6019e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:26.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.593+0000 7f7b5e800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b60101050 0x7f7b601996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:26.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.593+0000 7f7b5f200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b6019dfc0 0x7f7b6019e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50752/0 (socket says 172.21.15.131:50752) 2024-05-08T22:07:26.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.593+0000 7f7b5f200700 1 -- 172.21.15.131:0/996320629 learned_addr learned my addr 172.21.15.131:0/996320629 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:26.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.593+0000 7f7b5de00700 1 --2- 172.21.15.131:0/996320629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b60106170 0x7f7b60199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:26.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.593+0000 7f7b5f200700 1 -- 172.21.15.131:0/996320629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b60101050 msgr2=0x7f7b601996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:26.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.593+0000 7f7b5f200700 1 --2- 172.21.15.131:0/996320629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b60101050 0x7f7b601996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:26.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.593+0000 7f7b5f200700 1 -- 172.21.15.131:0/996320629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b60106170 msgr2=0x7f7b60199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:26.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.593+0000 7f7b5f200700 1 --2- 172.21.15.131:0/996320629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b60106170 0x7f7b60199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:26.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.593+0000 7f7b5f200700 1 -- 172.21.15.131:0/996320629 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b58009d00 con 0x7f7b6019dfc0 2024-05-08T22:07:26.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.593+0000 7f7b5e800700 1 --2- 172.21.15.131:0/996320629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b60101050 0x7f7b601996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:26.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.593+0000 7f7b5f200700 1 --2- 172.21.15.131:0/996320629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b6019dfc0 0x7f7b6019e440 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7b5800b070 tx=0x7f7b5800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:26.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.594+0000 7f7b57400700 1 -- 172.21.15.131:0/996320629 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5800e040 con 0x7f7b6019dfc0 2024-05-08T22:07:26.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.594+0000 7f7b57400700 1 -- 172.21.15.131:0/996320629 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b5801d480 con 0x7f7b6019dfc0 2024-05-08T22:07:26.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.594+0000 7f7b65400700 1 -- 172.21.15.131:0/996320629 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b601a6270 con 0x7f7b6019dfc0 2024-05-08T22:07:26.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.594+0000 7f7b57400700 1 -- 172.21.15.131:0/996320629 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b580052a0 con 0x7f7b6019dfc0 2024-05-08T22:07:26.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.594+0000 7f7b65400700 1 -- 172.21.15.131:0/996320629 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b601a6640 con 0x7f7b6019dfc0 2024-05-08T22:07:26.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.595+0000 7f7b65400700 1 -- 172.21.15.131:0/996320629 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b60102520 con 0x7f7b6019dfc0 2024-05-08T22:07:26.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.597+0000 7f7b57400700 1 -- 172.21.15.131:0/996320629 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7f7b5801d5f0 con 0x7f7b6019dfc0 2024-05-08T22:07:26.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.598+0000 7f7b57400700 1 --2- 172.21.15.131:0/996320629 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7b440a13b0 0x7f7b440a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:26.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.598+0000 7f7b57400700 1 -- 172.21.15.131:0/996320629 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f7b580ce3a0 con 0x7f7b6019dfc0 2024-05-08T22:07:26.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.598+0000 7f7b5e800700 1 --2- 172.21.15.131:0/996320629 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7b440a13b0 0x7f7b440a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:26.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.599+0000 7f7b5e800700 1 --2- 172.21.15.131:0/996320629 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7b440a13b0 0x7f7b440a3870 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7b50011490 tx=0x7f7b50009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:26.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.602+0000 7f7b57400700 1 -- 172.21.15.131:0/996320629 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7b58097240 con 0x7f7b6019dfc0 2024-05-08T22:07:26.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.816+0000 7f7b65400700 1 -- 172.21.15.131:0/996320629 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b60101a00 con 0x7f7b6019dfc0 2024-05-08T22:07:26.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.817+0000 7f7b57400700 1 -- 172.21.15.131:0/996320629 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+25165 (secure 0 0 0) 0x7f7b5809a860 con 0x7f7b6019dfc0 2024-05-08T22:07:26.819 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:07:26.819 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":88,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:07:17.096317+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:05:12.469858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:05:13.484446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:05:17.016186+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/2704707055":"2024-05-09T22:07:17.096274+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6801/2485665042":"2024-05-09T22:07:17.096274+0000","172.21.15.131:0/126236497":"2024-05-09T22:07:17.096274+0000","172.21.15.131:0/25489954":"2024-05-09T22:07:17.096274+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/1773942606":"2024-05-09T22:07:17.096274+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/2485665042":"2024-05-09T22:07:17.096274+0000","172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3120268590":"2024-05-09T22:07:17.096274+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:07:26.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.820+0000 7f7b65400700 1 -- 172.21.15.131:0/996320629 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7b440a13b0 msgr2=0x7f7b440a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:26.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.820+0000 7f7b65400700 1 --2- 172.21.15.131:0/996320629 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7b440a13b0 0x7f7b440a3870 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7b50011490 tx=0x7f7b50009260 comp rx=0 tx=0).stop 2024-05-08T22:07:26.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.820+0000 7f7b65400700 1 -- 172.21.15.131:0/996320629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b6019dfc0 msgr2=0x7f7b6019e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:26.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.820+0000 7f7b65400700 1 --2- 172.21.15.131:0/996320629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b6019dfc0 0x7f7b6019e440 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7b5800b070 tx=0x7f7b5800bda0 comp rx=0 tx=0).stop 2024-05-08T22:07:26.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.820+0000 7f7b65400700 1 -- 172.21.15.131:0/996320629 shutdown_connections 2024-05-08T22:07:26.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.820+0000 7f7b65400700 1 --2- 172.21.15.131:0/996320629 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7b440a13b0 0x7f7b440a3870 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:26.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.820+0000 7f7b65400700 1 --2- 172.21.15.131:0/996320629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b6019dfc0 0x7f7b6019e440 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:26.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.820+0000 7f7b65400700 1 --2- 172.21.15.131:0/996320629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b60101050 0x7f7b601996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:26.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.820+0000 7f7b65400700 1 --2- 172.21.15.131:0/996320629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b60106170 0x7f7b60199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:26.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.821+0000 7f7b65400700 1 -- 172.21.15.131:0/996320629 >> 172.21.15.131:0/996320629 conn(0x7f7b600fc8f0 msgr2=0x7f7b600fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:26.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.821+0000 7f7b65400700 1 -- 172.21.15.131:0/996320629 shutdown_connections 2024-05-08T22:07:26.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:26.821+0000 7f7b65400700 1 -- 172.21.15.131:0/996320629 wait complete. 2024-05-08T22:07:26.840 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:01:41.271078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:05:12.469858+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:05:13.484446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:05:17.016186+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:07:26.841 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:07:27.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.037+0000 7f7b87e00700 1 -- 172.21.15.131:0/1593498157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b801061a0 msgr2=0x7f7b8010dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:27.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.037+0000 7f7b87e00700 1 --2- 172.21.15.131:0/1593498157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b801061a0 0x7f7b8010dd20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7b7800b0a0 tx=0x7f7b7800b3b0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.037+0000 7f7b87e00700 1 -- 172.21.15.131:0/1593498157 shutdown_connections 2024-05-08T22:07:27.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.037+0000 7f7b87e00700 1 --2- 172.21.15.131:0/1593498157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b801061a0 0x7f7b8010dd20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.037+0000 7f7b87e00700 1 --2- 172.21.15.131:0/1593498157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b80101080 0x7f7b80101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.037+0000 7f7b87e00700 1 --2- 172.21.15.131:0/1593498157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b80101a30 0x7f7b80105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.037+0000 7f7b87e00700 1 -- 172.21.15.131:0/1593498157 >> 172.21.15.131:0/1593498157 conn(0x7f7b800fc940 msgr2=0x7f7b800fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:27.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.037+0000 7f7b87e00700 1 -- 172.21.15.131:0/1593498157 shutdown_connections 2024-05-08T22:07:27.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.037+0000 7f7b87e00700 1 -- 172.21.15.131:0/1593498157 wait complete. 2024-05-08T22:07:27.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.038+0000 7f7b87e00700 1 Processor -- start 2024-05-08T22:07:27.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.039+0000 7f7b87e00700 1 -- start start 2024-05-08T22:07:27.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.039+0000 7f7b87e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b80101080 0x7f7b80197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.039+0000 7f7b87e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b801061a0 0x7f7b801979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.039+0000 7f7b87e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b8019bd50 0x7f7b8019c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.039+0000 7f7b87e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8019c7d0 con 0x7f7b8019bd50 2024-05-08T22:07:27.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.039+0000 7f7b87e00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8019c940 con 0x7f7b80101080 2024-05-08T22:07:27.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.039+0000 7f7b87e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8019cb00 con 0x7f7b801061a0 2024-05-08T22:07:27.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.040+0000 7f7b85e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b8019bd50 0x7f7b8019c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.040+0000 7f7b85400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b80101080 0x7f7b80197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.040+0000 7f7b85e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b8019bd50 0x7f7b8019c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50766/0 (socket says 172.21.15.131:50766) 2024-05-08T22:07:27.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.040+0000 7f7b85e00700 1 -- 172.21.15.131:0/3776784075 learned_addr learned my addr 172.21.15.131:0/3776784075 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:27.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.040+0000 7f7b84a00700 1 --2- 172.21.15.131:0/3776784075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b801061a0 0x7f7b801979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.040+0000 7f7b85e00700 1 -- 172.21.15.131:0/3776784075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b801061a0 msgr2=0x7f7b801979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:27.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.040+0000 7f7b85e00700 1 --2- 172.21.15.131:0/3776784075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b801061a0 0x7f7b801979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.040+0000 7f7b85e00700 1 -- 172.21.15.131:0/3776784075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b80101080 msgr2=0x7f7b80197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:27.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.040+0000 7f7b85e00700 1 --2- 172.21.15.131:0/3776784075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b80101080 0x7f7b80197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.040+0000 7f7b85e00700 1 -- 172.21.15.131:0/3776784075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6c009520 con 0x7f7b8019bd50 2024-05-08T22:07:27.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.040+0000 7f7b85400700 1 --2- 172.21.15.131:0/3776784075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b80101080 0x7f7b80197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:27.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.040+0000 7f7b85e00700 1 --2- 172.21.15.131:0/3776784075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b8019bd50 0x7f7b8019c1d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7b7800b070 tx=0x7f7b7800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:27.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.040+0000 7f7b7e000700 1 -- 172.21.15.131:0/3776784075 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7800e040 con 0x7f7b8019bd50 2024-05-08T22:07:27.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.041+0000 7f7b7e000700 1 -- 172.21.15.131:0/3776784075 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b7801d480 con 0x7f7b8019bd50 2024-05-08T22:07:27.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.041+0000 7f7b87e00700 1 -- 172.21.15.131:0/3776784075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b78009d00 con 0x7f7b8019bd50 2024-05-08T22:07:27.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.041+0000 7f7b7e000700 1 -- 172.21.15.131:0/3776784075 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b780052a0 con 0x7f7b8019bd50 2024-05-08T22:07:27.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.041+0000 7f7b87e00700 1 -- 172.21.15.131:0/3776784075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b801a4240 con 0x7f7b8019bd50 2024-05-08T22:07:27.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.042+0000 7f7b87e00700 1 -- 172.21.15.131:0/3776784075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b80109470 con 0x7f7b8019bd50 2024-05-08T22:07:27.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.044+0000 7f7b7e000700 1 -- 172.21.15.131:0/3776784075 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7f7b7801d5f0 con 0x7f7b8019bd50 2024-05-08T22:07:27.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.045+0000 7f7b7e000700 1 --2- 172.21.15.131:0/3776784075 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7b680a13a0 0x7f7b680a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.045+0000 7f7b7e000700 1 -- 172.21.15.131:0/3776784075 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f7b780ce3a0 con 0x7f7b8019bd50 2024-05-08T22:07:27.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.045+0000 7f7b85400700 1 --2- 172.21.15.131:0/3776784075 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7b680a13a0 0x7f7b680a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.046+0000 7f7b85400700 1 --2- 172.21.15.131:0/3776784075 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7b680a13a0 0x7f7b680a3860 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7b74011490 tx=0x7f7b74009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:27.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.049+0000 7f7b7e000700 1 -- 172.21.15.131:0/3776784075 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7b78097240 con 0x7f7b8019bd50 2024-05-08T22:07:27.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.258+0000 7f7b87e00700 1 -- 172.21.15.131:0/3776784075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7b80198770 con 0x7f7b8019bd50 2024-05-08T22:07:27.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.259+0000 7f7b7e000700 1 -- 172.21.15.131:0/3776784075 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7b7809a860 con 0x7f7b8019bd50 2024-05-08T22:07:27.260 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-08T22:07:27.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.261+0000 7f7b87e00700 1 -- 172.21.15.131:0/3776784075 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7b680a13a0 msgr2=0x7f7b680a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:27.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.261+0000 7f7b87e00700 1 --2- 172.21.15.131:0/3776784075 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7b680a13a0 0x7f7b680a3860 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7b74011490 tx=0x7f7b74009260 comp rx=0 tx=0).stop 2024-05-08T22:07:27.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.261+0000 7f7b87e00700 1 -- 172.21.15.131:0/3776784075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b8019bd50 msgr2=0x7f7b8019c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:27.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.261+0000 7f7b87e00700 1 --2- 172.21.15.131:0/3776784075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b8019bd50 0x7f7b8019c1d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7b7800b070 tx=0x7f7b7800bda0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.262+0000 7f7b87e00700 1 -- 172.21.15.131:0/3776784075 shutdown_connections 2024-05-08T22:07:27.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.262+0000 7f7b87e00700 1 --2- 172.21.15.131:0/3776784075 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7b680a13a0 0x7f7b680a3860 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.262+0000 7f7b87e00700 1 --2- 172.21.15.131:0/3776784075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b8019bd50 0x7f7b8019c1d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.262+0000 7f7b87e00700 1 --2- 172.21.15.131:0/3776784075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b80101080 0x7f7b80197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.262+0000 7f7b87e00700 1 --2- 172.21.15.131:0/3776784075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b801061a0 0x7f7b801979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.262+0000 7f7b87e00700 1 -- 172.21.15.131:0/3776784075 >> 172.21.15.131:0/3776784075 conn(0x7f7b800fc940 msgr2=0x7f7b800fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:27.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.262+0000 7f7b87e00700 1 -- 172.21.15.131:0/3776784075 shutdown_connections 2024-05-08T22:07:27.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.262+0000 7f7b87e00700 1 -- 172.21.15.131:0/3776784075 wait complete. 2024-05-08T22:07:27.281 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:07:27.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.479+0000 7f2cf9a00700 1 -- 172.21.15.131:0/323846302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf4101b10 msgr2=0x7f2cf4105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:27.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.479+0000 7f2cf9a00700 1 --2- 172.21.15.131:0/323846302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf4101b10 0x7f2cf4105b50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009990 tx=0x7f2cdc009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.479+0000 7f2cf9a00700 1 -- 172.21.15.131:0/323846302 shutdown_connections 2024-05-08T22:07:27.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.479+0000 7f2cf9a00700 1 --2- 172.21.15.131:0/323846302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cf4106280 0x7f2cf410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.479+0000 7f2cf9a00700 1 --2- 172.21.15.131:0/323846302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cf4101160 0x7f2cf4101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.479+0000 7f2cf9a00700 1 --2- 172.21.15.131:0/323846302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf4101b10 0x7f2cf4105b50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.479+0000 7f2cf9a00700 1 -- 172.21.15.131:0/323846302 >> 172.21.15.131:0/323846302 conn(0x7f2cf40fc9c0 msgr2=0x7f2cf40fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:27.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.479+0000 7f2cf9a00700 1 -- 172.21.15.131:0/323846302 shutdown_connections 2024-05-08T22:07:27.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.479+0000 7f2cf9a00700 1 -- 172.21.15.131:0/323846302 wait complete. 2024-05-08T22:07:27.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.480+0000 7f2cf9a00700 1 Processor -- start 2024-05-08T22:07:27.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.480+0000 7f2cf9a00700 1 -- start start 2024-05-08T22:07:27.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.480+0000 7f2cf9a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cf4101160 0x7f2cf4197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.480+0000 7f2cf9a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cf4106280 0x7f2cf41979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.480+0000 7f2cf9a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf419bd40 0x7f2cf419c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.480+0000 7f2cf9a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf419c7c0 con 0x7f2cf4101160 2024-05-08T22:07:27.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.480+0000 7f2cf9a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf419c930 con 0x7f2cf4106280 2024-05-08T22:07:27.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.480+0000 7f2cf9a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf419caf0 con 0x7f2cf419bd40 2024-05-08T22:07:27.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.481+0000 7f2cf2a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cf4101160 0x7f2cf4197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.481+0000 7f2cebe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cf4106280 0x7f2cf41979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.481+0000 7f2cf2a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cf4101160 0x7f2cf4197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50776/0 (socket says 172.21.15.131:50776) 2024-05-08T22:07:27.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.481+0000 7f2cf2a00700 1 -- 172.21.15.131:0/3729562703 learned_addr learned my addr 172.21.15.131:0/3729562703 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:27.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.481+0000 7f2cf3400700 1 --2- 172.21.15.131:0/3729562703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf419bd40 0x7f2cf419c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.481+0000 7f2cf2a00700 1 -- 172.21.15.131:0/3729562703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf419bd40 msgr2=0x7f2cf419c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:27.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.481+0000 7f2cf2a00700 1 --2- 172.21.15.131:0/3729562703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf419bd40 0x7f2cf419c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.481+0000 7f2cf2a00700 1 -- 172.21.15.131:0/3729562703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cf4106280 msgr2=0x7f2cf41979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:27.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.481+0000 7f2cf2a00700 1 --2- 172.21.15.131:0/3729562703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cf4106280 0x7f2cf41979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.481+0000 7f2cf2a00700 1 -- 172.21.15.131:0/3729562703 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc0095f0 con 0x7f2cf4101160 2024-05-08T22:07:27.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.481+0000 7f2cebe00700 1 --2- 172.21.15.131:0/3729562703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cf4106280 0x7f2cf41979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:27.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.481+0000 7f2cf2a00700 1 --2- 172.21.15.131:0/3729562703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cf4101160 0x7f2cf4197460 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400e9c0 tx=0x7f2ce400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:27.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.482+0000 7f2ceb400700 1 -- 172.21.15.131:0/3729562703 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce40097b0 con 0x7f2cf4101160 2024-05-08T22:07:27.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.482+0000 7f2ceb400700 1 -- 172.21.15.131:0/3729562703 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ce400ee60 con 0x7f2cf4101160 2024-05-08T22:07:27.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.482+0000 7f2cf9a00700 1 -- 172.21.15.131:0/3729562703 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf41a3ff0 con 0x7f2cf4101160 2024-05-08T22:07:27.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.482+0000 7f2ceb400700 1 -- 172.21.15.131:0/3729562703 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce40054f0 con 0x7f2cf4101160 2024-05-08T22:07:27.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.482+0000 7f2cf9a00700 1 -- 172.21.15.131:0/3729562703 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf41a4470 con 0x7f2cf4101160 2024-05-08T22:07:27.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.483+0000 7f2cf9a00700 1 -- 172.21.15.131:0/3729562703 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cf4109550 con 0x7f2cf4101160 2024-05-08T22:07:27.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.485+0000 7f2ceb400700 1 -- 172.21.15.131:0/3729562703 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7f2ce4004750 con 0x7f2cf4101160 2024-05-08T22:07:27.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.486+0000 7f2ceb400700 1 --2- 172.21.15.131:0/3729562703 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f2cd80a13b0 0x7f2cd80a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.486+0000 7f2ceb400700 1 -- 172.21.15.131:0/3729562703 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f2ce4014070 con 0x7f2cf4101160 2024-05-08T22:07:27.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.486+0000 7f2cebe00700 1 --2- 172.21.15.131:0/3729562703 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f2cd80a13b0 0x7f2cd80a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.487+0000 7f2cebe00700 1 --2- 172.21.15.131:0/3729562703 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f2cd80a13b0 0x7f2cd80a3870 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2cf41989b0 tx=0x7f2cdc00b260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:27.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.490+0000 7f2ceb400700 1 -- 172.21.15.131:0/3729562703 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2ce4096380 con 0x7f2cf4101160 2024-05-08T22:07:27.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:27 smithi157 ceph-mon[77746]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:07:27.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:27 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/996320629' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:07:27.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:27 smithi132 ceph-mon[78180]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:07:27.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:27 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/996320629' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:07:27.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:27 smithi131 ceph-mon[37260]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:07:27.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:27 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/996320629' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:07:27.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.697+0000 7f2cf9a00700 1 -- 172.21.15.131:0/3729562703 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2cf4198760 con 0x7f2cf4101160 2024-05-08T22:07:27.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.698+0000 7f2ceb400700 1 -- 172.21.15.131:0/3729562703 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2ce40999a0 con 0x7f2cf4101160 2024-05-08T22:07:27.699 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-08T22:07:27.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.700+0000 7f2cf9a00700 1 -- 172.21.15.131:0/3729562703 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f2cd80a13b0 msgr2=0x7f2cd80a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:27.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.700+0000 7f2cf9a00700 1 --2- 172.21.15.131:0/3729562703 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f2cd80a13b0 0x7f2cd80a3870 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2cf41989b0 tx=0x7f2cdc00b260 comp rx=0 tx=0).stop 2024-05-08T22:07:27.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.700+0000 7f2cf9a00700 1 -- 172.21.15.131:0/3729562703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cf4101160 msgr2=0x7f2cf4197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:27.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.700+0000 7f2cf9a00700 1 --2- 172.21.15.131:0/3729562703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cf4101160 0x7f2cf4197460 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400e9c0 tx=0x7f2ce400ecd0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.701+0000 7f2cf9a00700 1 -- 172.21.15.131:0/3729562703 shutdown_connections 2024-05-08T22:07:27.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.701+0000 7f2cf9a00700 1 --2- 172.21.15.131:0/3729562703 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f2cd80a13b0 0x7f2cd80a3870 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.701+0000 7f2cf9a00700 1 --2- 172.21.15.131:0/3729562703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cf419bd40 0x7f2cf419c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.701+0000 7f2cf9a00700 1 --2- 172.21.15.131:0/3729562703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cf4101160 0x7f2cf4197460 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.701+0000 7f2cf9a00700 1 --2- 172.21.15.131:0/3729562703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cf4106280 0x7f2cf41979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.701+0000 7f2cf9a00700 1 -- 172.21.15.131:0/3729562703 >> 172.21.15.131:0/3729562703 conn(0x7f2cf40fc9c0 msgr2=0x7f2cf40fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:27.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.701+0000 7f2cf9a00700 1 -- 172.21.15.131:0/3729562703 shutdown_connections 2024-05-08T22:07:27.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.701+0000 7f2cf9a00700 1 -- 172.21.15.131:0/3729562703 wait complete. 2024-05-08T22:07:27.721 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:07:27.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.917+0000 7fedc7800700 1 -- 172.21.15.131:0/1457470560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedc0101160 msgr2=0x7fedc0101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:27.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.917+0000 7fedc7800700 1 --2- 172.21.15.131:0/1457470560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedc0101160 0x7fedc0101540 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fedb40098f0 tx=0x7fedb4009c00 comp rx=0 tx=0).stop 2024-05-08T22:07:27.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.917+0000 7fedc7800700 1 -- 172.21.15.131:0/1457470560 shutdown_connections 2024-05-08T22:07:27.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.917+0000 7fedc7800700 1 --2- 172.21.15.131:0/1457470560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedc0106280 0x7fedc010de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.917+0000 7fedc7800700 1 --2- 172.21.15.131:0/1457470560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedc0101160 0x7fedc0101540 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.917+0000 7fedc7800700 1 --2- 172.21.15.131:0/1457470560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedc0101b10 0x7fedc0105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.917+0000 7fedc7800700 1 -- 172.21.15.131:0/1457470560 >> 172.21.15.131:0/1457470560 conn(0x7fedc00fc9c0 msgr2=0x7fedc00fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:27.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.917+0000 7fedc7800700 1 -- 172.21.15.131:0/1457470560 shutdown_connections 2024-05-08T22:07:27.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.917+0000 7fedc7800700 1 -- 172.21.15.131:0/1457470560 wait complete. 2024-05-08T22:07:27.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.918+0000 7fedc7800700 1 Processor -- start 2024-05-08T22:07:27.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.918+0000 7fedc7800700 1 -- start start 2024-05-08T22:07:27.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.919+0000 7fedc7800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedc0101160 0x7fedc01996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.919+0000 7fedc7800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedc0106280 0x7fedc0199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.919+0000 7fedc7800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedc019dfa0 0x7fedc019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.919+0000 7fedc7800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc019ea20 con 0x7fedc019dfa0 2024-05-08T22:07:27.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.919+0000 7fedc7800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc019eb90 con 0x7fedc0101160 2024-05-08T22:07:27.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.919+0000 7fedc7800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc019ed50 con 0x7fedc0106280 2024-05-08T22:07:27.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.919+0000 7fedc5800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedc019dfa0 0x7fedc019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.919+0000 7fedc4e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedc0101160 0x7fedc01996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.919+0000 7fedc4e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedc0101160 0x7fedc01996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:43668/0 (socket says 172.21.15.131:43668) 2024-05-08T22:07:27.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.919+0000 7fedc4e00700 1 -- 172.21.15.131:0/1218406774 learned_addr learned my addr 172.21.15.131:0/1218406774 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:27.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.919+0000 7fedbfe00700 1 --2- 172.21.15.131:0/1218406774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedc0106280 0x7fedc0199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.919+0000 7fedc4e00700 1 -- 172.21.15.131:0/1218406774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedc0106280 msgr2=0x7fedc0199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:27.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.919+0000 7fedc4e00700 1 --2- 172.21.15.131:0/1218406774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedc0106280 0x7fedc0199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.919+0000 7fedc4e00700 1 -- 172.21.15.131:0/1218406774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedc019dfa0 msgr2=0x7fedc019e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:27.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.919+0000 7fedc4e00700 1 --2- 172.21.15.131:0/1218406774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedc019dfa0 0x7fedc019e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:27.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.919+0000 7fedc4e00700 1 -- 172.21.15.131:0/1218406774 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb4009520 con 0x7fedc0101160 2024-05-08T22:07:27.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.919+0000 7fedbfe00700 1 --2- 172.21.15.131:0/1218406774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedc0106280 0x7fedc0199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:07:27.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.920+0000 7fedc4e00700 1 --2- 172.21.15.131:0/1218406774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedc0101160 0x7fedc01996c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fedb4005950 tx=0x7fedb400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:27.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.920+0000 7fedbd600700 1 -- 172.21.15.131:0/1218406774 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb401c070 con 0x7fedc0101160 2024-05-08T22:07:27.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.920+0000 7fedc7800700 1 -- 172.21.15.131:0/1218406774 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc01a6250 con 0x7fedc0101160 2024-05-08T22:07:27.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.920+0000 7fedbd600700 1 -- 172.21.15.131:0/1218406774 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedb400bb20 con 0x7fedc0101160 2024-05-08T22:07:27.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.920+0000 7fedbd600700 1 -- 172.21.15.131:0/1218406774 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb40038e0 con 0x7fedc0101160 2024-05-08T22:07:27.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.920+0000 7fedc7800700 1 -- 172.21.15.131:0/1218406774 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc01a6720 con 0x7fedc0101160 2024-05-08T22:07:27.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.921+0000 7fedc7800700 1 -- 172.21.15.131:0/1218406774 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc0102630 con 0x7fedc0101160 2024-05-08T22:07:27.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.923+0000 7fedbd600700 1 -- 172.21.15.131:0/1218406774 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7fedb4020030 con 0x7fedc0101160 2024-05-08T22:07:27.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.924+0000 7fedbd600700 1 --2- 172.21.15.131:0/1218406774 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7feda80a13b0 0x7feda80a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:27.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.925+0000 7fedbd600700 1 -- 172.21.15.131:0/1218406774 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fedb40ce620 con 0x7fedc0101160 2024-05-08T22:07:27.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.925+0000 7fedbfe00700 1 --2- 172.21.15.131:0/1218406774 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7feda80a13b0 0x7feda80a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:27.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.925+0000 7fedbfe00700 1 --2- 172.21.15.131:0/1218406774 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7feda80a13b0 0x7feda80a3870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fedc019ac10 tx=0x7fedac009310 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:27.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:27.927+0000 7fedbd600700 1 -- 172.21.15.131:0/1218406774 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fedb40974c0 con 0x7fedc0101160 2024-05-08T22:07:28.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.128+0000 7fedc7800700 1 -- 172.21.15.131:0/1218406774 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fedc019a9c0 con 0x7fedc0101160 2024-05-08T22:07:28.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.129+0000 7fedbd600700 1 -- 172.21.15.131:0/1218406774 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fedb409aae0 con 0x7fedc0101160 2024-05-08T22:07:28.130 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-08T22:07:28.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.131+0000 7fedc7800700 1 -- 172.21.15.131:0/1218406774 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7feda80a13b0 msgr2=0x7feda80a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:28.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.131+0000 7fedc7800700 1 --2- 172.21.15.131:0/1218406774 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7feda80a13b0 0x7feda80a3870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fedc019ac10 tx=0x7fedac009310 comp rx=0 tx=0).stop 2024-05-08T22:07:28.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.131+0000 7fedc7800700 1 -- 172.21.15.131:0/1218406774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedc0101160 msgr2=0x7fedc01996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:28.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.131+0000 7fedc7800700 1 --2- 172.21.15.131:0/1218406774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedc0101160 0x7fedc01996c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fedb4005950 tx=0x7fedb400fa00 comp rx=0 tx=0).stop 2024-05-08T22:07:28.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.132+0000 7fedc7800700 1 -- 172.21.15.131:0/1218406774 shutdown_connections 2024-05-08T22:07:28.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.132+0000 7fedc7800700 1 --2- 172.21.15.131:0/1218406774 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7feda80a13b0 0x7feda80a3870 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:28.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.132+0000 7fedc7800700 1 --2- 172.21.15.131:0/1218406774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedc019dfa0 0x7fedc019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:28.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.132+0000 7fedc7800700 1 --2- 172.21.15.131:0/1218406774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedc0101160 0x7fedc01996c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:28.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.132+0000 7fedc7800700 1 --2- 172.21.15.131:0/1218406774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedc0106280 0x7fedc0199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:28.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.132+0000 7fedc7800700 1 -- 172.21.15.131:0/1218406774 >> 172.21.15.131:0/1218406774 conn(0x7fedc00fc9c0 msgr2=0x7fedc00fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:28.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.132+0000 7fedc7800700 1 -- 172.21.15.131:0/1218406774 shutdown_connections 2024-05-08T22:07:28.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.132+0000 7fedc7800700 1 -- 172.21.15.131:0/1218406774 wait complete. 2024-05-08T22:07:28.152 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:07:28.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.351+0000 7f2ce7800700 1 -- 172.21.15.131:0/1461208794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce0106280 msgr2=0x7f2ce010de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:28.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.351+0000 7f2ce7800700 1 --2- 172.21.15.131:0/1461208794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce0106280 0x7f2ce010de00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800b0a0 tx=0x7f2cd800b3b0 comp rx=0 tx=0).stop 2024-05-08T22:07:28.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.352+0000 7f2ce7800700 1 -- 172.21.15.131:0/1461208794 shutdown_connections 2024-05-08T22:07:28.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.352+0000 7f2ce7800700 1 --2- 172.21.15.131:0/1461208794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce0106280 0x7f2ce010de00 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:28.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.352+0000 7f2ce7800700 1 --2- 172.21.15.131:0/1461208794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce0101160 0x7f2ce0101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:28.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.352+0000 7f2ce7800700 1 --2- 172.21.15.131:0/1461208794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ce0101b10 0x7f2ce0105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:28.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.352+0000 7f2ce7800700 1 -- 172.21.15.131:0/1461208794 >> 172.21.15.131:0/1461208794 conn(0x7f2ce00fc9c0 msgr2=0x7f2ce00fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:28.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.352+0000 7f2ce7800700 1 -- 172.21.15.131:0/1461208794 shutdown_connections 2024-05-08T22:07:28.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.352+0000 7f2ce7800700 1 -- 172.21.15.131:0/1461208794 wait complete. 2024-05-08T22:07:28.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.353+0000 7f2ce7800700 1 Processor -- start 2024-05-08T22:07:28.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.353+0000 7f2ce7800700 1 -- start start 2024-05-08T22:07:28.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.353+0000 7f2ce7800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce0101160 0x7f2ce0197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:28.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.353+0000 7f2ce7800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ce0106280 0x7f2ce0197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:28.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.353+0000 7f2ce7800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce019be30 0x7f2ce019c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:28.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.353+0000 7f2ce7800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce019c8b0 con 0x7f2ce019be30 2024-05-08T22:07:28.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.353+0000 7f2ce7800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce019ca20 con 0x7f2ce0106280 2024-05-08T22:07:28.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.353+0000 7f2ce7800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce019cbe0 con 0x7f2ce0101160 2024-05-08T22:07:28.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.354+0000 7f2ce5800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce019be30 0x7f2ce019c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:28.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.354+0000 7f2ce5800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce019be30 0x7f2ce019c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50818/0 (socket says 172.21.15.131:50818) 2024-05-08T22:07:28.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.354+0000 7f2ce5800700 1 -- 172.21.15.131:0/61273544 learned_addr learned my addr 172.21.15.131:0/61273544 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:28.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.354+0000 7f2cdfe00700 1 --2- 172.21.15.131:0/61273544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ce0106280 0x7f2ce0197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:28.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.354+0000 7f2ce4e00700 1 --2- 172.21.15.131:0/61273544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce0101160 0x7f2ce0197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:28.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.354+0000 7f2ce5800700 1 -- 172.21.15.131:0/61273544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce0101160 msgr2=0x7f2ce0197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:28.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.354+0000 7f2ce5800700 1 --2- 172.21.15.131:0/61273544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce0101160 0x7f2ce0197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:28.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.354+0000 7f2ce5800700 1 -- 172.21.15.131:0/61273544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ce0106280 msgr2=0x7f2ce0197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:28.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.354+0000 7f2ce5800700 1 --2- 172.21.15.131:0/61273544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ce0106280 0x7f2ce0197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:28.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.354+0000 7f2ce5800700 1 -- 172.21.15.131:0/61273544 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd4009520 con 0x7f2ce019be30 2024-05-08T22:07:28.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.354+0000 7f2ce4e00700 1 --2- 172.21.15.131:0/61273544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce0101160 0x7f2ce0197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:07:28.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.354+0000 7f2ce5800700 1 --2- 172.21.15.131:0/61273544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce019be30 0x7f2ce019c2b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800b070 tx=0x7f2cd800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:28.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.354+0000 7f2cdd600700 1 -- 172.21.15.131:0/61273544 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd800e040 con 0x7f2ce019be30 2024-05-08T22:07:28.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.354+0000 7f2cdd600700 1 -- 172.21.15.131:0/61273544 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cd801d480 con 0x7f2ce019be30 2024-05-08T22:07:28.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.354+0000 7f2ce7800700 1 -- 172.21.15.131:0/61273544 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd8009d00 con 0x7f2ce019be30 2024-05-08T22:07:28.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.355+0000 7f2cdd600700 1 -- 172.21.15.131:0/61273544 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd80052a0 con 0x7f2ce019be30 2024-05-08T22:07:28.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.355+0000 7f2ce7800700 1 -- 172.21.15.131:0/61273544 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce01a4370 con 0x7f2ce019be30 2024-05-08T22:07:28.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.356+0000 7f2ce7800700 1 -- 172.21.15.131:0/61273544 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce0109550 con 0x7f2ce019be30 2024-05-08T22:07:28.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.358+0000 7f2cdd600700 1 -- 172.21.15.131:0/61273544 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7f2cd801d5f0 con 0x7f2ce019be30 2024-05-08T22:07:28.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.360+0000 7f2cdd600700 1 --2- 172.21.15.131:0/61273544 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f2cc80a1430 0x7f2cc80a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:28.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.360+0000 7f2cdd600700 1 -- 172.21.15.131:0/61273544 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f2cd80ce660 con 0x7f2ce019be30 2024-05-08T22:07:28.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.360+0000 7f2ce4e00700 1 --2- 172.21.15.131:0/61273544 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f2cc80a1430 0x7f2cc80a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:28.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.361+0000 7f2ce4e00700 1 --2- 172.21.15.131:0/61273544 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f2cc80a1430 0x7f2cc80a38f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009e40 tx=0x7f2cd4009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:28.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.363+0000 7f2cdd600700 1 -- 172.21.15.131:0/61273544 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2cd8097560 con 0x7f2ce019be30 2024-05-08T22:07:28.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:28 smithi157 ceph-mon[77746]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 918 B/s wr, 18 op/s 2024-05-08T22:07:28.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:28 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3776784075' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:07:28.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:28 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3729562703' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:07:28.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:28 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1218406774' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:07:28.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.570+0000 7f2ce7800700 1 -- 172.21.15.131:0/61273544 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2ce0198850 con 0x7f2ce019be30 2024-05-08T22:07:28.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.571+0000 7f2cdd600700 1 -- 172.21.15.131:0/61273544 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2cd809ab80 con 0x7f2ce019be30 2024-05-08T22:07:28.572 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-08T22:07:28.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.573+0000 7f2ce7800700 1 -- 172.21.15.131:0/61273544 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f2cc80a1430 msgr2=0x7f2cc80a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:28.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.573+0000 7f2ce7800700 1 --2- 172.21.15.131:0/61273544 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f2cc80a1430 0x7f2cc80a38f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009e40 tx=0x7f2cd4009260 comp rx=0 tx=0).stop 2024-05-08T22:07:28.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.573+0000 7f2ce7800700 1 -- 172.21.15.131:0/61273544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce019be30 msgr2=0x7f2ce019c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:28.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.573+0000 7f2ce7800700 1 --2- 172.21.15.131:0/61273544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce019be30 0x7f2ce019c2b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800b070 tx=0x7f2cd800bda0 comp rx=0 tx=0).stop 2024-05-08T22:07:28.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.573+0000 7f2ce7800700 1 -- 172.21.15.131:0/61273544 shutdown_connections 2024-05-08T22:07:28.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.573+0000 7f2ce7800700 1 --2- 172.21.15.131:0/61273544 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f2cc80a1430 0x7f2cc80a38f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:28.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.574+0000 7f2ce7800700 1 --2- 172.21.15.131:0/61273544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce019be30 0x7f2ce019c2b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:28.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.574+0000 7f2ce7800700 1 --2- 172.21.15.131:0/61273544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce0101160 0x7f2ce0197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:28.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.574+0000 7f2ce7800700 1 --2- 172.21.15.131:0/61273544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ce0106280 0x7f2ce0197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:28.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.574+0000 7f2ce7800700 1 -- 172.21.15.131:0/61273544 >> 172.21.15.131:0/61273544 conn(0x7f2ce00fc9c0 msgr2=0x7f2ce00fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:28.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.574+0000 7f2ce7800700 1 -- 172.21.15.131:0/61273544 shutdown_connections 2024-05-08T22:07:28.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.574+0000 7f2ce7800700 1 -- 172.21.15.131:0/61273544 wait complete. 2024-05-08T22:07:28.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:28 smithi132 ceph-mon[78180]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 918 B/s wr, 18 op/s 2024-05-08T22:07:28.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:28 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3776784075' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:07:28.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:28 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3729562703' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:07:28.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:28 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1218406774' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:07:28.591 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:28 smithi131 ceph-mon[37260]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 918 B/s wr, 18 op/s 2024-05-08T22:07:28.591 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:28 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3776784075' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:07:28.591 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:28 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3729562703' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:07:28.591 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:28 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1218406774' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:07:28.593 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:07:28.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.788+0000 7fdb09a00700 1 -- 172.21.15.131:0/3230801509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb04101050 msgr2=0x7fdb04101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:28.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.788+0000 7fdb09a00700 1 --2- 172.21.15.131:0/3230801509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb04101050 0x7fdb04101430 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40098f0 tx=0x7fdaf4009c00 comp rx=0 tx=0).stop 2024-05-08T22:07:28.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.788+0000 7fdb09a00700 1 -- 172.21.15.131:0/3230801509 shutdown_connections 2024-05-08T22:07:28.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.788+0000 7fdb09a00700 1 --2- 172.21.15.131:0/3230801509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb04106170 0x7fdb0410dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:28.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.788+0000 7fdb09a00700 1 --2- 172.21.15.131:0/3230801509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb04101050 0x7fdb04101430 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:28.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.788+0000 7fdb09a00700 1 --2- 172.21.15.131:0/3230801509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb04101a00 0x7fdb04105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:28.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.788+0000 7fdb09a00700 1 -- 172.21.15.131:0/3230801509 >> 172.21.15.131:0/3230801509 conn(0x7fdb040fc8f0 msgr2=0x7fdb040fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:28.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.788+0000 7fdb09a00700 1 -- 172.21.15.131:0/3230801509 shutdown_connections 2024-05-08T22:07:28.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.788+0000 7fdb09a00700 1 -- 172.21.15.131:0/3230801509 wait complete. 2024-05-08T22:07:28.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.789+0000 7fdb09a00700 1 Processor -- start 2024-05-08T22:07:28.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.789+0000 7fdb09a00700 1 -- start start 2024-05-08T22:07:28.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.789+0000 7fdb09a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb04101050 0x7fdb040732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:28.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.789+0000 7fdb09a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb04106170 0x7fdb040737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:28.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.789+0000 7fdb09a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb04073d20 0x7fdb041a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:28.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.789+0000 7fdb09a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb041a5b30 con 0x7fdb04073d20 2024-05-08T22:07:28.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.790+0000 7fdb09a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb041a5ca0 con 0x7fdb04101050 2024-05-08T22:07:28.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.790+0000 7fdb09a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb041a5e60 con 0x7fdb04106170 2024-05-08T22:07:28.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.790+0000 7fdb02000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb04106170 0x7fdb040737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:28.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.790+0000 7fdb02000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb04106170 0x7fdb040737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:48676/0 (socket says 172.21.15.131:48676) 2024-05-08T22:07:28.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.790+0000 7fdb03400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb04073d20 0x7fdb041a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:28.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.790+0000 7fdb02000700 1 -- 172.21.15.131:0/3436169957 learned_addr learned my addr 172.21.15.131:0/3436169957 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:28.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.790+0000 7fdb02a00700 1 --2- 172.21.15.131:0/3436169957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb04101050 0x7fdb040732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:28.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.790+0000 7fdb02000700 1 -- 172.21.15.131:0/3436169957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb04101050 msgr2=0x7fdb040732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:28.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.790+0000 7fdb02000700 1 --2- 172.21.15.131:0/3436169957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb04101050 0x7fdb040732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:28.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.790+0000 7fdb02000700 1 -- 172.21.15.131:0/3436169957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb04073d20 msgr2=0x7fdb041a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:28.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.790+0000 7fdb02000700 1 --2- 172.21.15.131:0/3436169957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb04073d20 0x7fdb041a5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:28.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.790+0000 7fdb02000700 1 -- 172.21.15.131:0/3436169957 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdafc009d00 con 0x7fdb04106170 2024-05-08T22:07:28.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.790+0000 7fdb02000700 1 --2- 172.21.15.131:0/3436169957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb04106170 0x7fdb040737e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000e9f0 tx=0x7fdaf000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:28.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.791+0000 7fdafb400700 1 -- 172.21.15.131:0/3436169957 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0009750 con 0x7fdb04106170 2024-05-08T22:07:28.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.791+0000 7fdafb400700 1 -- 172.21.15.131:0/3436169957 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdaf000c7c0 con 0x7fdb04106170 2024-05-08T22:07:28.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.791+0000 7fdb09a00700 1 -- 172.21.15.131:0/3436169957 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf4009520 con 0x7fdb04106170 2024-05-08T22:07:28.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.791+0000 7fdafb400700 1 -- 172.21.15.131:0/3436169957 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf00054d0 con 0x7fdb04106170 2024-05-08T22:07:28.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.791+0000 7fdb09a00700 1 -- 172.21.15.131:0/3436169957 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb041a6470 con 0x7fdb04106170 2024-05-08T22:07:28.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.792+0000 7fdb09a00700 1 -- 172.21.15.131:0/3436169957 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb041093e0 con 0x7fdb04106170 2024-05-08T22:07:28.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.794+0000 7fdafb400700 1 -- 172.21.15.131:0/3436169957 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7fdaf000c930 con 0x7fdb04106170 2024-05-08T22:07:28.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.795+0000 7fdafb400700 1 --2- 172.21.15.131:0/3436169957 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fdae80a13b0 0x7fdae80a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:28.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.795+0000 7fdafb400700 1 -- 172.21.15.131:0/3436169957 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fdaf0014070 con 0x7fdb04106170 2024-05-08T22:07:28.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.795+0000 7fdb02a00700 1 --2- 172.21.15.131:0/3436169957 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fdae80a13b0 0x7fdae80a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:28.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.796+0000 7fdb02a00700 1 --2- 172.21.15.131:0/3436169957 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fdae80a13b0 0x7fdae80a3870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40098c0 tx=0x7fdaf4019040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:28.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:28.798+0000 7fdafb400700 1 -- 172.21.15.131:0/3436169957 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdaf0096570 con 0x7fdb04106170 2024-05-08T22:07:29.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.051+0000 7fdb09a00700 1 -- 172.21.15.131:0/3436169957 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb04074780 con 0x7fdb04106170 2024-05-08T22:07:29.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.052+0000 7fdafb400700 1 -- 172.21.15.131:0/3436169957 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10889 (secure 0 0 0) 0x7fdaf0099b90 con 0x7fdb04106170 2024-05-08T22:07:29.053 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:07:29.054 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:05:14.034908+0000","modified":"2024-05-08T22:07:19.165885+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:07:29.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.055+0000 7fdb09a00700 1 -- 172.21.15.131:0/3436169957 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fdae80a13b0 msgr2=0x7fdae80a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:29.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.055+0000 7fdb09a00700 1 --2- 172.21.15.131:0/3436169957 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fdae80a13b0 0x7fdae80a3870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40098c0 tx=0x7fdaf4019040 comp rx=0 tx=0).stop 2024-05-08T22:07:29.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.055+0000 7fdb09a00700 1 -- 172.21.15.131:0/3436169957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb04106170 msgr2=0x7fdb040737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:29.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.055+0000 7fdb09a00700 1 --2- 172.21.15.131:0/3436169957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb04106170 0x7fdb040737e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000e9f0 tx=0x7fdaf000ed00 comp rx=0 tx=0).stop 2024-05-08T22:07:29.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.055+0000 7fdb09a00700 1 -- 172.21.15.131:0/3436169957 shutdown_connections 2024-05-08T22:07:29.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.055+0000 7fdb09a00700 1 --2- 172.21.15.131:0/3436169957 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fdae80a13b0 0x7fdae80a3870 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.055+0000 7fdb09a00700 1 --2- 172.21.15.131:0/3436169957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb04073d20 0x7fdb041a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.055+0000 7fdb09a00700 1 --2- 172.21.15.131:0/3436169957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb04101050 0x7fdb040732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.055+0000 7fdb09a00700 1 --2- 172.21.15.131:0/3436169957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb04106170 0x7fdb040737e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.055+0000 7fdb09a00700 1 -- 172.21.15.131:0/3436169957 >> 172.21.15.131:0/3436169957 conn(0x7fdb040fc8f0 msgr2=0x7fdb040fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:29.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.055+0000 7fdb09a00700 1 -- 172.21.15.131:0/3436169957 shutdown_connections 2024-05-08T22:07:29.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.055+0000 7fdb09a00700 1 -- 172.21.15.131:0/3436169957 wait complete. 2024-05-08T22:07:29.057 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 24 2024-05-08T22:07:29.075 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:07:29.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.271+0000 7f43dbe00700 1 -- 172.21.15.131:0/839056052 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43d4101060 msgr2=0x7f43d4101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:29.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.271+0000 7f43dbe00700 1 --2- 172.21.15.131:0/839056052 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43d4101060 0x7f43d4101440 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f43c80098f0 tx=0x7f43c8009c00 comp rx=0 tx=0).stop 2024-05-08T22:07:29.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.271+0000 7f43dbe00700 1 -- 172.21.15.131:0/839056052 shutdown_connections 2024-05-08T22:07:29.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.271+0000 7f43dbe00700 1 --2- 172.21.15.131:0/839056052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43d4106180 0x7f43d410dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.271+0000 7f43dbe00700 1 --2- 172.21.15.131:0/839056052 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43d4101060 0x7f43d4101440 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.271+0000 7f43dbe00700 1 --2- 172.21.15.131:0/839056052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43d4101a10 0x7f43d4105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.271+0000 7f43dbe00700 1 -- 172.21.15.131:0/839056052 >> 172.21.15.131:0/839056052 conn(0x7f43d40fc960 msgr2=0x7f43d40fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:29.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.271+0000 7f43dbe00700 1 -- 172.21.15.131:0/839056052 shutdown_connections 2024-05-08T22:07:29.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.271+0000 7f43dbe00700 1 -- 172.21.15.131:0/839056052 wait complete. 2024-05-08T22:07:29.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.272+0000 7f43dbe00700 1 Processor -- start 2024-05-08T22:07:29.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.272+0000 7f43dbe00700 1 -- start start 2024-05-08T22:07:29.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.272+0000 7f43dbe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43d4101060 0x7f43d4199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.273+0000 7f43dbe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43d4106180 0x7f43d4199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.273+0000 7f43dbe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43d419ddf0 0x7f43d419e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.273+0000 7f43dbe00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d419e870 con 0x7f43d4101060 2024-05-08T22:07:29.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.273+0000 7f43dbe00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d419e9e0 con 0x7f43d419ddf0 2024-05-08T22:07:29.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.273+0000 7f43dbe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d419eba0 con 0x7f43d4106180 2024-05-08T22:07:29.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.273+0000 7f43d9400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43d4101060 0x7f43d4199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.273+0000 7f43d8a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43d4106180 0x7f43d4199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.273+0000 7f43d8a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43d4106180 0x7f43d4199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:48708/0 (socket says 172.21.15.131:48708) 2024-05-08T22:07:29.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.273+0000 7f43d8a00700 1 -- 172.21.15.131:0/2099209562 learned_addr learned my addr 172.21.15.131:0/2099209562 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:29.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.273+0000 7f43d9e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43d419ddf0 0x7f43d419e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.273+0000 7f43d8a00700 1 -- 172.21.15.131:0/2099209562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43d419ddf0 msgr2=0x7f43d419e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:29.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.273+0000 7f43d8a00700 1 --2- 172.21.15.131:0/2099209562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43d419ddf0 0x7f43d419e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.273+0000 7f43d8a00700 1 -- 172.21.15.131:0/2099209562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43d4101060 msgr2=0x7f43d4199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:29.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.273+0000 7f43d8a00700 1 --2- 172.21.15.131:0/2099209562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43d4101060 0x7f43d4199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.273+0000 7f43d8a00700 1 -- 172.21.15.131:0/2099209562 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc009d00 con 0x7f43d4106180 2024-05-08T22:07:29.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.274+0000 7f43d8a00700 1 --2- 172.21.15.131:0/2099209562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43d4106180 0x7f43d4199aa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f43c000e9f0 tx=0x7f43c000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:29.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.274+0000 7f43d2000700 1 -- 172.21.15.131:0/2099209562 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c0009750 con 0x7f43d4106180 2024-05-08T22:07:29.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.274+0000 7f43d2000700 1 -- 172.21.15.131:0/2099209562 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43c000c7c0 con 0x7f43d4106180 2024-05-08T22:07:29.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.274+0000 7f43dbe00700 1 -- 172.21.15.131:0/2099209562 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c8009520 con 0x7f43d4106180 2024-05-08T22:07:29.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.274+0000 7f43d2000700 1 -- 172.21.15.131:0/2099209562 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c00054d0 con 0x7f43d4106180 2024-05-08T22:07:29.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.274+0000 7f43dbe00700 1 -- 172.21.15.131:0/2099209562 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d41a6450 con 0x7f43d4106180 2024-05-08T22:07:29.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.275+0000 7f43dbe00700 1 -- 172.21.15.131:0/2099209562 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d4102530 con 0x7f43d4106180 2024-05-08T22:07:29.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.277+0000 7f43d2000700 1 -- 172.21.15.131:0/2099209562 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7f43c000c930 con 0x7f43d4106180 2024-05-08T22:07:29.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.278+0000 7f43d2000700 1 --2- 172.21.15.131:0/2099209562 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f43bc0a1430 0x7f43bc0a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.278+0000 7f43d2000700 1 -- 172.21.15.131:0/2099209562 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f43c0014070 con 0x7f43d4106180 2024-05-08T22:07:29.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.278+0000 7f43d9400700 1 --2- 172.21.15.131:0/2099209562 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f43bc0a1430 0x7f43bc0a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.279+0000 7f43d9400700 1 --2- 172.21.15.131:0/2099209562 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f43bc0a1430 0x7f43bc0a38f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f43c80098c0 tx=0x7f43c8019040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:29.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.282+0000 7f43d2000700 1 -- 172.21.15.131:0/2099209562 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f43c0096330 con 0x7f43d4106180 2024-05-08T22:07:29.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:29 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/61273544' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:07:29.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:29 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3436169957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:07:29.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.490+0000 7f43dbe00700 1 -- 172.21.15.131:0/2099209562 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43d4101a10 con 0x7f43d4106180 2024-05-08T22:07:29.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.492+0000 7f43d2000700 1 -- 172.21.15.131:0/2099209562 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+25165 (secure 0 0 0) 0x7f43c0099950 con 0x7f43d4106180 2024-05-08T22:07:29.493 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:07:29.494 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":88,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:07:17.096317+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:05:12.469858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:05:13.484446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:05:17.016186+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/2704707055":"2024-05-09T22:07:17.096274+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6801/2485665042":"2024-05-09T22:07:17.096274+0000","172.21.15.131:0/126236497":"2024-05-09T22:07:17.096274+0000","172.21.15.131:0/25489954":"2024-05-09T22:07:17.096274+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/1773942606":"2024-05-09T22:07:17.096274+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/2485665042":"2024-05-09T22:07:17.096274+0000","172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3120268590":"2024-05-09T22:07:17.096274+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:07:29.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.495+0000 7f43dbe00700 1 -- 172.21.15.131:0/2099209562 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f43bc0a1430 msgr2=0x7f43bc0a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:29.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.495+0000 7f43dbe00700 1 --2- 172.21.15.131:0/2099209562 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f43bc0a1430 0x7f43bc0a38f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f43c80098c0 tx=0x7f43c8019040 comp rx=0 tx=0).stop 2024-05-08T22:07:29.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.495+0000 7f43dbe00700 1 -- 172.21.15.131:0/2099209562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43d4106180 msgr2=0x7f43d4199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:29.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.495+0000 7f43dbe00700 1 --2- 172.21.15.131:0/2099209562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43d4106180 0x7f43d4199aa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f43c000e9f0 tx=0x7f43c000ed00 comp rx=0 tx=0).stop 2024-05-08T22:07:29.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.495+0000 7f43dbe00700 1 -- 172.21.15.131:0/2099209562 shutdown_connections 2024-05-08T22:07:29.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.495+0000 7f43dbe00700 1 --2- 172.21.15.131:0/2099209562 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f43bc0a1430 0x7f43bc0a38f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.495+0000 7f43dbe00700 1 --2- 172.21.15.131:0/2099209562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43d419ddf0 0x7f43d419e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.495+0000 7f43dbe00700 1 --2- 172.21.15.131:0/2099209562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43d4101060 0x7f43d4199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.495+0000 7f43dbe00700 1 --2- 172.21.15.131:0/2099209562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43d4106180 0x7f43d4199aa0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.495+0000 7f43dbe00700 1 -- 172.21.15.131:0/2099209562 >> 172.21.15.131:0/2099209562 conn(0x7f43d40fc960 msgr2=0x7f43d40fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:29.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.495+0000 7f43dbe00700 1 -- 172.21.15.131:0/2099209562 shutdown_connections 2024-05-08T22:07:29.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.495+0000 7f43dbe00700 1 -- 172.21.15.131:0/2099209562 wait complete. 2024-05-08T22:07:29.515 DEBUG:tasks.fwd_scrub:Yielding 2024-05-08T22:07:29.516 INFO:teuthology.run_tasks:Running task exec... 2024-05-08T22:07:29.525 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T22:07:29.525 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi131.front.sepia.ceph.com 2024-05-08T22:07:29.525 DEBUG:teuthology.orchestra.run.smithi131:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-08T22:07:29.526 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-08T22:07:29.526 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T22:07:29.526 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T22:07:29.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:29 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/61273544' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:07:29.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:29 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3436169957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:07:29.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:29 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/61273544' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:07:29.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:29 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3436169957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:07:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.726+0000 7f8abbe00700 1 -- 172.21.15.131:0/2819664885 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ab4073c60 msgr2=0x7f8ab4111960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.726+0000 7f8abbe00700 1 --2- 172.21.15.131:0/2819664885 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ab4073c60 0x7f8ab4111960 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0009990 tx=0x7f8aa0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.726+0000 7f8abbe00700 1 -- 172.21.15.131:0/2819664885 shutdown_connections 2024-05-08T22:07:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.726+0000 7f8abbe00700 1 --2- 172.21.15.131:0/2819664885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ab4111ea0 0x7f8ab4114290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.726+0000 7f8abbe00700 1 --2- 172.21.15.131:0/2819664885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab4073340 0x7f8ab4073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.726+0000 7f8abbe00700 1 --2- 172.21.15.131:0/2819664885 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ab4073c60 0x7f8ab4111960 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.726+0000 7f8abbe00700 1 -- 172.21.15.131:0/2819664885 >> 172.21.15.131:0/2819664885 conn(0x7f8ab40fc9a0 msgr2=0x7f8ab40fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:29.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.726+0000 7f8abbe00700 1 -- 172.21.15.131:0/2819664885 shutdown_connections 2024-05-08T22:07:29.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.726+0000 7f8abbe00700 1 -- 172.21.15.131:0/2819664885 wait complete. 2024-05-08T22:07:29.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.727+0000 7f8abbe00700 1 Processor -- start 2024-05-08T22:07:29.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.727+0000 7f8abbe00700 1 -- start start 2024-05-08T22:07:29.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.727+0000 7f8abbe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ab4073340 0x7f8ab41a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.727+0000 7f8abbe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab4073c60 0x7f8ab41a3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.727+0000 7f8abbe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ab4111ea0 0x7f8ab419d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.727+0000 7f8abbe00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab419de90 con 0x7f8ab4073340 2024-05-08T22:07:29.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.727+0000 7f8abbe00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab419e000 con 0x7f8ab4111ea0 2024-05-08T22:07:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.727+0000 7f8abbe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab419e1c0 con 0x7f8ab4073c60 2024-05-08T22:07:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.728+0000 7f8ab9400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ab4073340 0x7f8ab41a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.728+0000 7f8ab9400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ab4073340 0x7f8ab41a36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50882/0 (socket says 172.21.15.131:50882) 2024-05-08T22:07:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.728+0000 7f8ab9400700 1 -- 172.21.15.131:0/3533964268 learned_addr learned my addr 172.21.15.131:0/3533964268 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.728+0000 7f8ab8a00700 1 --2- 172.21.15.131:0/3533964268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab4073c60 0x7f8ab41a3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.728+0000 7f8ab9e00700 1 --2- 172.21.15.131:0/3533964268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ab4111ea0 0x7f8ab419d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.728+0000 7f8ab9400700 1 -- 172.21.15.131:0/3533964268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab4073c60 msgr2=0x7f8ab41a3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.728+0000 7f8ab9400700 1 --2- 172.21.15.131:0/3533964268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab4073c60 0x7f8ab41a3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.728+0000 7f8ab9400700 1 -- 172.21.15.131:0/3533964268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ab4111ea0 msgr2=0x7f8ab419d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.728+0000 7f8ab9400700 1 --2- 172.21.15.131:0/3533964268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ab4111ea0 0x7f8ab419d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.728+0000 7f8ab9400700 1 -- 172.21.15.131:0/3533964268 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa00095f0 con 0x7f8ab4073340 2024-05-08T22:07:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.728+0000 7f8ab9e00700 1 --2- 172.21.15.131:0/3533964268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ab4111ea0 0x7f8ab419d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:07:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.728+0000 7f8ab9400700 1 --2- 172.21.15.131:0/3533964268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ab4073340 0x7f8ab41a36f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800e9c0 tx=0x7f8aa800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.729+0000 7f8ab2000700 1 -- 172.21.15.131:0/3533964268 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa80097b0 con 0x7f8ab4073340 2024-05-08T22:07:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.729+0000 7f8abbe00700 1 -- 172.21.15.131:0/3533964268 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab419e4a0 con 0x7f8ab4073340 2024-05-08T22:07:29.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.729+0000 7f8ab2000700 1 -- 172.21.15.131:0/3533964268 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8aa8004cf0 con 0x7f8ab4073340 2024-05-08T22:07:29.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.729+0000 7f8ab2000700 1 -- 172.21.15.131:0/3533964268 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa80055e0 con 0x7f8ab4073340 2024-05-08T22:07:29.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.729+0000 7f8abbe00700 1 -- 172.21.15.131:0/3533964268 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab41aa6e0 con 0x7f8ab4073340 2024-05-08T22:07:29.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.730+0000 7f8abbe00700 1 -- 172.21.15.131:0/3533964268 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab41043a0 con 0x7f8ab4073340 2024-05-08T22:07:29.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.732+0000 7f8ab2000700 1 -- 172.21.15.131:0/3533964268 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7f8aa800cc60 con 0x7f8ab4073340 2024-05-08T22:07:29.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.733+0000 7fa28ac00700 1 -- 172.21.15.131:0/4250322532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa284068c90 msgr2=0x7fa284105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:29.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.733+0000 7fa28ac00700 1 --2- 172.21.15.131:0/4250322532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa284068c90 0x7fa284105b60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa270009920 tx=0x7fa270009c30 comp rx=0 tx=0).stop 2024-05-08T22:07:29.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.733+0000 7fa28ac00700 1 -- 172.21.15.131:0/4250322532 shutdown_connections 2024-05-08T22:07:29.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.733+0000 7fa28ac00700 1 --2- 172.21.15.131:0/4250322532 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa284106290 0x7fa28410de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.733+0000 7fa28ac00700 1 --2- 172.21.15.131:0/4250322532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2840682e0 0x7fa2840686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.733+0000 7fa28ac00700 1 --2- 172.21.15.131:0/4250322532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa284068c90 0x7fa284105b60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.733+0000 7fa28ac00700 1 -- 172.21.15.131:0/4250322532 >> 172.21.15.131:0/4250322532 conn(0x7fa284076f40 msgr2=0x7fa284077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:29.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.733+0000 7fa28ac00700 1 -- 172.21.15.131:0/4250322532 shutdown_connections 2024-05-08T22:07:29.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.733+0000 7fa28ac00700 1 -- 172.21.15.131:0/4250322532 wait complete. 2024-05-08T22:07:29.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.734+0000 7f8ab2000700 1 --2- 172.21.15.131:0/3533964268 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8a9c0a13b0 0x7f8a9c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.734+0000 7f8ab2000700 1 -- 172.21.15.131:0/3533964268 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f8aa8014070 con 0x7f8ab4073340 2024-05-08T22:07:29.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.734+0000 7fa28ac00700 1 Processor -- start 2024-05-08T22:07:29.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.734+0000 7f8ab8a00700 1 --2- 172.21.15.131:0/3533964268 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8a9c0a13b0 0x7f8a9c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.734+0000 7fa28ac00700 1 -- start start 2024-05-08T22:07:29.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.735+0000 7fa28ac00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2840682e0 0x7fa284199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.735+0000 7fa28ac00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa284106290 0x7fa284199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.735+0000 7f8ab8a00700 1 --2- 172.21.15.131:0/3533964268 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8a9c0a13b0 0x7f8a9c0a3870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8ab419ec40 tx=0x7f8aa000b260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:29.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.735+0000 7fa28ac00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa28419df70 0x7fa28419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.735+0000 7fa28ac00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28419e9f0 con 0x7fa2840682e0 2024-05-08T22:07:29.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.735+0000 7fa28ac00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28419eb60 con 0x7fa28419df70 2024-05-08T22:07:29.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.735+0000 7fa28ac00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28419ed20 con 0x7fa284106290 2024-05-08T22:07:29.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.735+0000 7fa283e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2840682e0 0x7fa284199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.735+0000 7fa283400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa284106290 0x7fa284199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.735+0000 7fa288c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa28419df70 0x7fa28419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.735+0000 7fa288c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa28419df70 0x7fa28419e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:43784/0 (socket says 172.21.15.131:43784) 2024-05-08T22:07:29.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.735+0000 7fa283400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa284106290 0x7fa284199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:48742/0 (socket says 172.21.15.131:48742) 2024-05-08T22:07:29.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.735+0000 7fa288c00700 1 -- 172.21.15.131:0/1912526586 learned_addr learned my addr 172.21.15.131:0/1912526586 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:29.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.735+0000 7fa283e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2840682e0 0x7fa284199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50890/0 (socket says 172.21.15.131:50890) 2024-05-08T22:07:29.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.736+0000 7fa288c00700 1 -- 172.21.15.131:0/1912526586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa284106290 msgr2=0x7fa284199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:29.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.736+0000 7fa288c00700 1 --2- 172.21.15.131:0/1912526586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa284106290 0x7fa284199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.736+0000 7fa288c00700 1 -- 172.21.15.131:0/1912526586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2840682e0 msgr2=0x7fa284199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:29.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.736+0000 7fa288c00700 1 --2- 172.21.15.131:0/1912526586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2840682e0 0x7fa284199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.736+0000 7fa288c00700 1 -- 172.21.15.131:0/1912526586 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa270009520 con 0x7fa28419df70 2024-05-08T22:07:29.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.736+0000 7fa288c00700 1 --2- 172.21.15.131:0/1912526586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa28419df70 0x7fa28419e3f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00c110 tx=0x7fa27c00c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:29.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.736+0000 7fa280c00700 1 -- 172.21.15.131:0/1912526586 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c017070 con 0x7fa28419df70 2024-05-08T22:07:29.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.736+0000 7fa280c00700 1 -- 172.21.15.131:0/1912526586 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa27c00e050 con 0x7fa28419df70 2024-05-08T22:07:29.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.736+0000 7fa280c00700 1 -- 172.21.15.131:0/1912526586 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0078f0 con 0x7fa28419df70 2024-05-08T22:07:29.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.736+0000 7fa28ac00700 1 -- 172.21.15.131:0/1912526586 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2841a6220 con 0x7fa28419df70 2024-05-08T22:07:29.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.736+0000 7fa28ac00700 1 -- 172.21.15.131:0/1912526586 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2841a66b0 con 0x7fa28419df70 2024-05-08T22:07:29.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.739+0000 7f8ab2000700 1 -- 172.21.15.131:0/3533964268 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8aa8096590 con 0x7f8ab4073340 2024-05-08T22:07:29.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.740+0000 7fa280c00700 1 -- 172.21.15.131:0/1912526586 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7fa27c007a50 con 0x7fa28419df70 2024-05-08T22:07:29.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.741+0000 7fa29e6dcb80 1 Processor -- start 2024-05-08T22:07:29.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.741+0000 7fa29e6dcb80 1 -- start start 2024-05-08T22:07:29.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.741+0000 7fa29e6dcb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56017907ee80 0x56017907f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.741+0000 7fa29e6dcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017907f7a0 0x5601790793d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.741+0000 7fa29e6dcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560179079910 0x560179079d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.741+0000 7fa29e6dcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560178fd01d0 con 0x560179079910 2024-05-08T22:07:29.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.741+0000 7fa29e6dcb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560178fcdb90 con 0x56017907ee80 2024-05-08T22:07:29.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.741+0000 7fa29e6dcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560178fc9480 con 0x56017907f7a0 2024-05-08T22:07:29.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.742+0000 7fa283400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56017907ee80 0x56017907f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.742+0000 7fa283400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56017907ee80 0x56017907f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:43792/0 (socket says 172.21.15.131:43792) 2024-05-08T22:07:29.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.742+0000 7fa283400700 1 -- 172.21.15.131:0/1616998537 learned_addr learned my addr 172.21.15.131:0/1616998537 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:29.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.742+0000 7fa283e00700 1 --2- 172.21.15.131:0/1616998537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017907f7a0 0x5601790793d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.742+0000 7fa283400700 1 --2- 172.21.15.131:0/1616998537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560179079910 0x560179079d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.742+0000 7fa283400700 1 -- 172.21.15.131:0/1616998537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017907f7a0 msgr2=0x5601790793d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:29.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.742+0000 7fa283400700 1 --2- 172.21.15.131:0/1616998537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017907f7a0 0x5601790793d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.742+0000 7fa283400700 1 -- 172.21.15.131:0/1616998537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560179079910 msgr2=0x560179079d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:29.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.742+0000 7fa283400700 1 --2- 172.21.15.131:0/1616998537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560179079910 0x560179079d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.742+0000 7fa283400700 1 -- 172.21.15.131:0/1616998537 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560178e4a430 con 0x56017907ee80 2024-05-08T22:07:29.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.742+0000 7fa283e00700 1 --2- 172.21.15.131:0/1616998537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017907f7a0 0x5601790793d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:29.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.742+0000 7fa280c00700 1 --2- 172.21.15.131:0/1912526586 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa2680a13b0 0x7fa2680a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.743+0000 7fa283400700 1 --2- 172.21.15.131:0/1616998537 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56017907ee80 0x56017907f260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa270009920 tx=0x7fa270019040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62a7b123d4200c33 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:29.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.743+0000 7fa280c00700 1 -- 172.21.15.131:0/1912526586 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fa27c0cccf0 con 0x7fa28419df70 2024-05-08T22:07:29.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.743+0000 7fa279200700 1 -- 172.21.15.131:0/1616998537 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27000f460 con 0x56017907ee80 2024-05-08T22:07:29.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.743+0000 7fa279200700 1 -- 172.21.15.131:0/1616998537 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa270005110 con 0x56017907ee80 2024-05-08T22:07:29.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.743+0000 7fa279200700 1 -- 172.21.15.131:0/1616998537 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27000b7c0 con 0x56017907ee80 2024-05-08T22:07:29.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.743+0000 7fa283e00700 1 --2- 172.21.15.131:0/1912526586 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa2680a13b0 0x7fa2680a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.744+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/1616998537 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56017907ee80 msgr2=0x56017907f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:29.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.744+0000 7fa283e00700 1 --2- 172.21.15.131:0/1912526586 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa2680a13b0 0x7fa2680a3870 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x560179075c50 tx=0x7fa274009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:29.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.744+0000 7fa29e6dcb80 1 --2- 172.21.15.131:0/1616998537 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56017907ee80 0x56017907f260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa270009920 tx=0x7fa270019040 comp rx=0 tx=0).stop 2024-05-08T22:07:29.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.744+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/1616998537 shutdown_connections 2024-05-08T22:07:29.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.744+0000 7fa29e6dcb80 1 --2- 172.21.15.131:0/1616998537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560179079910 0x560179079d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.744+0000 7fa29e6dcb80 1 --2- 172.21.15.131:0/1616998537 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56017907ee80 0x56017907f260 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.744+0000 7fa29e6dcb80 1 --2- 172.21.15.131:0/1616998537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017907f7a0 0x5601790793d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.744+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/1616998537 >> 172.21.15.131:0/1616998537 conn(0x560178fd7290 msgr2=0x560179073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:29.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.744+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/1616998537 shutdown_connections 2024-05-08T22:07:29.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.744+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/1616998537 wait complete. 2024-05-08T22:07:29.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.745+0000 7fa29e6dcb80 1 -- start start 2024-05-08T22:07:29.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.745+0000 7fa29e6dcb80 1 Processor -- start 2024-05-08T22:07:29.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.746+0000 7fa29e6dcb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560179079910 0x560179082c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.746+0000 7fa29e6dcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017907ee80 0x560179083140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.746+0000 7fa29e6dcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017907f7a0 0x560179083680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.746+0000 7fa29e6dcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560178f655e0 con 0x56017907ee80 2024-05-08T22:07:29.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.746+0000 7fa29e6dcb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560178f65280 con 0x560179079910 2024-05-08T22:07:29.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.746+0000 7fa29e6dcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560178e4c720 con 0x56017907f7a0 2024-05-08T22:07:29.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.746+0000 7fa29e6dcb80 10 client.0 _ll_register_callbacks cb 0x560178f6ebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:07:29.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.746+0000 7fa283400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017907ee80 0x560179083140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.746+0000 7fa283e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560179079910 0x560179082c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.746+0000 7fa283400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017907ee80 0x560179083140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50898/0 (socket says 172.21.15.131:50898) 2024-05-08T22:07:29.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.746+0000 7fa283400700 1 -- 172.21.15.131:0/59209815 learned_addr learned my addr 172.21.15.131:0/59209815 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:29.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.746+0000 7fa288c00700 1 --2- 172.21.15.131:0/59209815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017907f7a0 0x560179083680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.746+0000 7fa283400700 1 -- 172.21.15.131:0/59209815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017907f7a0 msgr2=0x560179083680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:29.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.746+0000 7fa283400700 1 --2- 172.21.15.131:0/59209815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017907f7a0 0x560179083680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.746+0000 7fa283400700 1 -- 172.21.15.131:0/59209815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560179079910 msgr2=0x560179082c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:29.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.746+0000 7fa283400700 1 --2- 172.21.15.131:0/59209815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560179079910 0x560179082c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.746+0000 7fa283400700 1 -- 172.21.15.131:0/59209815 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2740098e0 con 0x56017907ee80 2024-05-08T22:07:29.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.747+0000 7fa283400700 1 --2- 172.21.15.131:0/59209815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017907ee80 0x560179083140 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa270005530 tx=0x7fa270009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45ab8e62eda7e62 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:29.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.747+0000 7fa26d600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:07:29.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.747+0000 7fa26d600700 1 -- 172.21.15.131:0/59209815 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27000f670 con 0x56017907ee80 2024-05-08T22:07:29.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.747+0000 7fa26d600700 1 -- 172.21.15.131:0/59209815 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fa27000f7d0 con 0x56017907ee80 2024-05-08T22:07:29.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.747+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/59209815 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560178f222f0 con 0x56017907ee80 2024-05-08T22:07:29.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.747+0000 7fa26d600700 1 -- 172.21.15.131:0/59209815 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa27000b560 con 0x56017907ee80 2024-05-08T22:07:29.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.747+0000 7fa29e6dcb80 10 client.15570 fetch_fsmap learned FSMap version 24 2024-05-08T22:07:29.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.747+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/59209815 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fa27001f040 con 0x56017907ee80 2024-05-08T22:07:29.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.747+0000 7fa26d600700 1 -- 172.21.15.131:0/59209815 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560178f222f0 con 0x56017907ee80 2024-05-08T22:07:29.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.748+0000 7fa26d600700 1 -- 172.21.15.131:0/59209815 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa270015020 con 0x56017907ee80 2024-05-08T22:07:29.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.748+0000 7fa29e6dcb80 10 client.15570 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:07:29.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.748+0000 7fa29e6dcb80 10 client.15570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:07:29.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.748+0000 7fa29e6dcb80 20 client.15570 populate_metadata read hostname 'smithi131' 2024-05-08T22:07:29.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.748+0000 7fa29e6dcb80 1 --2- 172.21.15.131:0/59209815 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560179092dc0 0x5601790951e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.748+0000 7fa29e6dcb80 4 client.15570 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:07:29.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.748+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/59209815 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560179048890 con 0x560179092dc0 2024-05-08T22:07:29.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.748+0000 7fa283e00700 1 --2- 172.21.15.131:0/59209815 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560179092dc0 0x5601790951e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.749+0000 7fa26d600700 10 client.15570 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:07:29.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.749+0000 7fa283e00700 1 --2- 172.21.15.131:0/59209815 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560179092dc0 0x5601790951e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=176382ed2ec3bacc server_cookie=cf8d681c5b2a1320 in_seq=0 out_seq=0 2024-05-08T22:07:29.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.749+0000 7fa26d600700 1 -- 172.21.15.131:0/59209815 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560179048890 con 0x560179092dc0 2024-05-08T22:07:29.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.749+0000 7fa26d600700 10 client.15570 handle_command_reply: tid=0 2024-05-08T22:07:29.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.750+0000 7fa29e6dcb80 1 client.15570 shutdown 2024-05-08T22:07:29.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.750+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/59209815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017907ee80 msgr2=0x560179083140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:29.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.750+0000 7fa29e6dcb80 1 --2- 172.21.15.131:0/59209815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017907ee80 0x560179083140 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa270005530 tx=0x7fa270009c30 comp rx=0 tx=0).stop 2024-05-08T22:07:29.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.750+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/59209815 shutdown_connections 2024-05-08T22:07:29.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.750+0000 7fa29e6dcb80 1 --2- 172.21.15.131:0/59209815 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560179092dc0 0x5601790951e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.750+0000 7fa29e6dcb80 1 --2- 172.21.15.131:0/59209815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017907f7a0 0x560179083680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.750+0000 7fa29e6dcb80 1 --2- 172.21.15.131:0/59209815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560179079910 0x560179082c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.750+0000 7fa26d600700 0 client.15570 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:07:29.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.750+0000 7fa29e6dcb80 1 --2- 172.21.15.131:0/59209815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017907ee80 0x560179083140 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.750+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/59209815 >> 172.21.15.131:0/59209815 conn(0x560178fd7290 msgr2=0x560179073620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:29.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.750+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/59209815 shutdown_connections 2024-05-08T22:07:29.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.750+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/59209815 wait complete. 2024-05-08T22:07:29.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.750+0000 7fa29e6dcb80 20 client.15570 trim_cache size 0 max 16384 2024-05-08T22:07:29.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.768+0000 7fa29e6dcb80 1 Processor -- start 2024-05-08T22:07:29.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.768+0000 7fa29e6dcb80 1 -- start start 2024-05-08T22:07:29.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.768+0000 7fa29e6dcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560179079910 0x56017907e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.768+0000 7fa29e6dcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017907ee80 0x560179073da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.768+0000 7fa29e6dcb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56017907f7a0 0x560179081be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.768+0000 7fa29e6dcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560178f222f0 con 0x56017907ee80 2024-05-08T22:07:29.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.768+0000 7fa29e6dcb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017908fdf0 con 0x56017907f7a0 2024-05-08T22:07:29.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.768+0000 7fa29e6dcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560179084270 con 0x560179079910 2024-05-08T22:07:29.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.769+0000 7fa283400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017907ee80 0x560179073da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.769+0000 7fa283e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560179079910 0x56017907e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.769+0000 7fa283e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560179079910 0x56017907e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:48772/0 (socket says 172.21.15.131:48772) 2024-05-08T22:07:29.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.769+0000 7fa283400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017907ee80 0x560179073da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50902/0 (socket says 172.21.15.131:50902) 2024-05-08T22:07:29.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.769+0000 7fa283e00700 1 -- 172.21.15.131:0/3258407281 learned_addr learned my addr 172.21.15.131:0/3258407281 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:29.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.769+0000 7fa288c00700 1 --2- 172.21.15.131:0/3258407281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56017907f7a0 0x560179081be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.769+0000 7fa283e00700 1 -- 172.21.15.131:0/3258407281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56017907f7a0 msgr2=0x560179081be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:29.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.769+0000 7fa283e00700 1 --2- 172.21.15.131:0/3258407281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56017907f7a0 0x560179081be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.769+0000 7fa283e00700 1 -- 172.21.15.131:0/3258407281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017907ee80 msgr2=0x560179073da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:29.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.769+0000 7fa283e00700 1 --2- 172.21.15.131:0/3258407281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017907ee80 0x560179073da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.769+0000 7fa283e00700 1 -- 172.21.15.131:0/3258407281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560178df6be0 con 0x560179079910 2024-05-08T22:07:29.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.769+0000 7fa283400700 1 --2- 172.21.15.131:0/3258407281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017907ee80 0x560179073da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:29.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.769+0000 7fa283e00700 1 --2- 172.21.15.131:0/3258407281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560179079910 0x56017907e550 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x560179095af0 tx=0x7fa27401b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61effb351cec6906 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:29.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.770+0000 7fa279c00700 1 -- 172.21.15.131:0/3258407281 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa274025040 con 0x560179079910 2024-05-08T22:07:29.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.770+0000 7fa279c00700 1 -- 172.21.15.131:0/3258407281 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa27402a860 con 0x560179079910 2024-05-08T22:07:29.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.770+0000 7fa279c00700 1 -- 172.21.15.131:0/3258407281 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa274017070 con 0x560179079910 2024-05-08T22:07:29.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.770+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/3258407281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560179079910 msgr2=0x56017907e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:29.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.770+0000 7fa29e6dcb80 1 --2- 172.21.15.131:0/3258407281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560179079910 0x56017907e550 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x560179095af0 tx=0x7fa27401b000 comp rx=0 tx=0).stop 2024-05-08T22:07:29.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.771+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/3258407281 shutdown_connections 2024-05-08T22:07:29.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.771+0000 7fa29e6dcb80 1 --2- 172.21.15.131:0/3258407281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56017907f7a0 0x560179081be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.771+0000 7fa29e6dcb80 1 --2- 172.21.15.131:0/3258407281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560179079910 0x56017907e550 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.771+0000 7fa29e6dcb80 1 --2- 172.21.15.131:0/3258407281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017907ee80 0x560179073da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.771+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/3258407281 >> 172.21.15.131:0/3258407281 conn(0x560178fd7290 msgr2=0x5601790814d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:29.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.771+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/3258407281 shutdown_connections 2024-05-08T22:07:29.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.771+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/3258407281 wait complete. 2024-05-08T22:07:29.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.771+0000 7fa29e6dcb80 1 -- start start 2024-05-08T22:07:29.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.771+0000 7fa29e6dcb80 1 Processor -- start 2024-05-08T22:07:29.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.772+0000 7fa29e6dcb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560179079910 0x560179088b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.772+0000 7fa29e6dcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017907ee80 0x560179089080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.772+0000 7fa29e6dcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017907f7a0 0x5601790895c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.772+0000 7fa29e6dcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017908aad0 con 0x56017907ee80 2024-05-08T22:07:29.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.772+0000 7fa29e6dcb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601790e92d0 con 0x560179079910 2024-05-08T22:07:29.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.772+0000 7fa29e6dcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601790e9410 con 0x56017907f7a0 2024-05-08T22:07:29.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.772+0000 7fa29e6dcb80 10 client.0 _ll_register_callbacks cb 0x560178f6ebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:07:29.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.772+0000 7fa283400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017907ee80 0x560179089080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.772+0000 7fa283e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560179079910 0x560179088b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.772+0000 7fa283400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017907ee80 0x560179089080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50912/0 (socket says 172.21.15.131:50912) 2024-05-08T22:07:29.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.772+0000 7fa283400700 1 -- 172.21.15.131:0/147247239 learned_addr learned my addr 172.21.15.131:0/147247239 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:29.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.772+0000 7fa288c00700 1 --2- 172.21.15.131:0/147247239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017907f7a0 0x5601790895c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.772+0000 7fa283e00700 1 -- 172.21.15.131:0/147247239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017907f7a0 msgr2=0x5601790895c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:29.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.772+0000 7fa283e00700 1 --2- 172.21.15.131:0/147247239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017907f7a0 0x5601790895c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.772+0000 7fa283e00700 1 -- 172.21.15.131:0/147247239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017907ee80 msgr2=0x560179089080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:29.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.772+0000 7fa283e00700 1 --2- 172.21.15.131:0/147247239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017907ee80 0x560179089080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.772+0000 7fa283e00700 1 -- 172.21.15.131:0/147247239 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27001f040 con 0x560179079910 2024-05-08T22:07:29.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.773+0000 7fa283e00700 1 --2- 172.21.15.131:0/147247239 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560179079910 0x560179088b40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa274019040 tx=0x7fa274004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a2843f2ca6a70bb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:29.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.773+0000 7fa26d600700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:07:29.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.773+0000 7fa26d600700 1 -- 172.21.15.131:0/147247239 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa274017050 con 0x560179079910 2024-05-08T22:07:29.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.773+0000 7fa26d600700 1 -- 172.21.15.131:0/147247239 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fa2740044b0 con 0x560179079910 2024-05-08T22:07:29.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.773+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/147247239 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa270005110 con 0x560179079910 2024-05-08T22:07:29.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.773+0000 7fa26d600700 1 -- 172.21.15.131:0/147247239 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa274025040 con 0x560179079910 2024-05-08T22:07:29.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.773+0000 7fa29e6dcb80 10 client.25619 fetch_fsmap learned FSMap version 24 2024-05-08T22:07:29.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.773+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/147247239 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fa2740041e0 con 0x560179079910 2024-05-08T22:07:29.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.774+0000 7fa26d600700 1 -- 172.21.15.131:0/147247239 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa270005110 con 0x560179079910 2024-05-08T22:07:29.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.774+0000 7fa26d600700 1 -- 172.21.15.131:0/147247239 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa274030020 con 0x560179079910 2024-05-08T22:07:29.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.774+0000 7fa29e6dcb80 10 client.25619 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:07:29.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.774+0000 7fa29e6dcb80 10 client.25619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:07:29.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.774+0000 7fa29e6dcb80 20 client.25619 populate_metadata read hostname 'smithi131' 2024-05-08T22:07:29.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.774+0000 7fa29e6dcb80 1 --2- 172.21.15.131:0/147247239 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5601790eaa70 0x5601790ece90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:29.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.774+0000 7fa29e6dcb80 4 client.25619 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T22:07:29.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.774+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/147247239 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560178fc1f40 con 0x5601790eaa70 2024-05-08T22:07:29.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.774+0000 7fa283400700 1 --2- 172.21.15.131:0/147247239 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5601790eaa70 0x5601790ece90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:29.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.775+0000 7fa26d600700 10 client.25619 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:07:29.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.775+0000 7fa283400700 1 --2- 172.21.15.131:0/147247239 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5601790eaa70 0x5601790ece90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a248b76a1f04cff server_cookie=cd9a28e9d182f507 in_seq=0 out_seq=0 2024-05-08T22:07:29.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.776+0000 7fa26d600700 1 -- 172.21.15.131:0/147247239 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560178fc1f40 con 0x5601790eaa70 2024-05-08T22:07:29.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.776+0000 7fa26d600700 10 client.25619 handle_command_reply: tid=0 2024-05-08T22:07:29.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.776+0000 7fa29e6dcb80 1 client.25619 shutdown 2024-05-08T22:07:29.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.776+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/147247239 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560179079910 msgr2=0x560179088b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:29.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.776+0000 7fa29e6dcb80 1 --2- 172.21.15.131:0/147247239 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560179079910 0x560179088b40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa274019040 tx=0x7fa274004750 comp rx=0 tx=0).stop 2024-05-08T22:07:29.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.776+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/147247239 shutdown_connections 2024-05-08T22:07:29.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.776+0000 7fa29e6dcb80 1 --2- 172.21.15.131:0/147247239 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5601790eaa70 0x5601790ece90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.776+0000 7fa29e6dcb80 1 --2- 172.21.15.131:0/147247239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017907f7a0 0x5601790895c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.776+0000 7fa29e6dcb80 1 --2- 172.21.15.131:0/147247239 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560179079910 0x560179088b40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.776+0000 7fa26d600700 0 client.25619 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:07:29.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.776+0000 7fa29e6dcb80 1 --2- 172.21.15.131:0/147247239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017907ee80 0x560179089080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.776+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/147247239 >> 172.21.15.131:0/147247239 conn(0x560178fd7290 msgr2=0x5601790833d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:29.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.776+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/147247239 shutdown_connections 2024-05-08T22:07:29.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.776+0000 7fa29e6dcb80 1 -- 172.21.15.131:0/147247239 wait complete. 2024-05-08T22:07:29.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.776+0000 7fa29e6dcb80 20 client.25619 trim_cache size 0 max 16384 2024-05-08T22:07:29.782 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:07:29.782 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-08T22:07:29.782 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "3a8ed3b9-29e9-46c4-b86f-ad39967dab74", 2024-05-08T22:07:29.782 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-08T22:07:29.782 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:07:29.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.777+0000 7fa28ac00700 1 -- 172.21.15.131:0/1912526586 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa2680a13b0 msgr2=0x7fa2680a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:29.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.777+0000 7fa28ac00700 1 --2- 172.21.15.131:0/1912526586 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa2680a13b0 0x7fa2680a3870 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x560179075c50 tx=0x7fa274009260 comp rx=0 tx=0).stop 2024-05-08T22:07:29.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.777+0000 7fa28ac00700 1 -- 172.21.15.131:0/1912526586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa28419df70 msgr2=0x7fa28419e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:29.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.777+0000 7fa28ac00700 1 --2- 172.21.15.131:0/1912526586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa28419df70 0x7fa28419e3f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00c110 tx=0x7fa27c00c420 comp rx=0 tx=0).stop 2024-05-08T22:07:29.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.777+0000 7fa28ac00700 1 -- 172.21.15.131:0/1912526586 shutdown_connections 2024-05-08T22:07:29.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.777+0000 7fa28ac00700 1 --2- 172.21.15.131:0/1912526586 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa2680a13b0 0x7fa2680a3870 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.777+0000 7fa28ac00700 1 --2- 172.21.15.131:0/1912526586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa28419df70 0x7fa28419e3f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.777+0000 7fa28ac00700 1 --2- 172.21.15.131:0/1912526586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2840682e0 0x7fa284199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.777+0000 7fa28ac00700 1 --2- 172.21.15.131:0/1912526586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa284106290 0x7fa284199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.777+0000 7fa28ac00700 1 -- 172.21.15.131:0/1912526586 >> 172.21.15.131:0/1912526586 conn(0x7fa284076f40 msgr2=0x7fa28410c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:29.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.777+0000 7fa28ac00700 1 -- 172.21.15.131:0/1912526586 shutdown_connections 2024-05-08T22:07:29.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.777+0000 7fa28ac00700 1 -- 172.21.15.131:0/1912526586 wait complete. 2024-05-08T22:07:29.798 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:07:29.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.987+0000 7f8abbe00700 1 -- 172.21.15.131:0/3533964268 --> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ab419ed50 con 0x7f8a9c0a13b0 2024-05-08T22:07:29.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.997+0000 7fc026e00700 1 -- 172.21.15.131:0/3115661107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc020101160 msgr2=0x7fc020101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:29.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.997+0000 7fc026e00700 1 --2- 172.21.15.131:0/3115661107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc020101160 0x7fc020101540 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc0100098f0 tx=0x7fc010009c00 comp rx=0 tx=0).stop 2024-05-08T22:07:29.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.997+0000 7fc026e00700 1 -- 172.21.15.131:0/3115661107 shutdown_connections 2024-05-08T22:07:29.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.997+0000 7fc026e00700 1 --2- 172.21.15.131:0/3115661107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc020106280 0x7fc02010de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.997+0000 7fc026e00700 1 --2- 172.21.15.131:0/3115661107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc020101160 0x7fc020101540 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.997+0000 7fc026e00700 1 --2- 172.21.15.131:0/3115661107 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc020101b10 0x7fc020105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:29.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.997+0000 7fc026e00700 1 -- 172.21.15.131:0/3115661107 >> 172.21.15.131:0/3115661107 conn(0x7fc0200fc9c0 msgr2=0x7fc0200fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:29.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.997+0000 7fc026e00700 1 -- 172.21.15.131:0/3115661107 shutdown_connections 2024-05-08T22:07:29.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.997+0000 7fc026e00700 1 -- 172.21.15.131:0/3115661107 wait complete. 2024-05-08T22:07:29.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.999+0000 7fc026e00700 1 Processor -- start 2024-05-08T22:07:30.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:29.999+0000 7fc026e00700 1 -- start start 2024-05-08T22:07:30.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.000+0000 7fc026e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc020101160 0x7fc0200732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.000+0000 7fc026e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc020106280 0x7fc0200737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.000+0000 7fc026e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc020073d20 0x7fc0201a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.000+0000 7fc026e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0201a5c30 con 0x7fc020073d20 2024-05-08T22:07:30.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.000+0000 7fc026e00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0201a5da0 con 0x7fc020101160 2024-05-08T22:07:30.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.000+0000 7fc026e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0201a5f60 con 0x7fc020106280 2024-05-08T22:07:30.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.000+0000 7fc01f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc020106280 0x7fc0200737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:30.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.000+0000 7fc01fe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc020101160 0x7fc0200732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:30.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.000+0000 7fc01f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc020106280 0x7fc0200737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:48792/0 (socket says 172.21.15.131:48792) 2024-05-08T22:07:30.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.000+0000 7fc01f400700 1 -- 172.21.15.131:0/630523440 learned_addr learned my addr 172.21.15.131:0/630523440 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:30.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.000+0000 7fc024e00700 1 --2- 172.21.15.131:0/630523440 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc020073d20 0x7fc0201a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:30.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.000+0000 7fc01f400700 1 -- 172.21.15.131:0/630523440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc020101160 msgr2=0x7fc0200732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:30.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.000+0000 7fc01f400700 1 --2- 172.21.15.131:0/630523440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc020101160 0x7fc0200732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.000+0000 7fc01f400700 1 -- 172.21.15.131:0/630523440 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc020073d20 msgr2=0x7fc0201a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:30.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.000+0000 7fc01f400700 1 --2- 172.21.15.131:0/630523440 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc020073d20 0x7fc0201a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.000+0000 7fc01f400700 1 -- 172.21.15.131:0/630523440 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc010009520 con 0x7fc020106280 2024-05-08T22:07:30.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.001+0000 7fc01f400700 1 --2- 172.21.15.131:0/630523440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc020106280 0x7fc0200737e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00e9f0 tx=0x7fc00c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:30.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.001+0000 7fc01cc00700 1 -- 172.21.15.131:0/630523440 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c009690 con 0x7fc020106280 2024-05-08T22:07:30.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.001+0000 7fc026e00700 1 -- 172.21.15.131:0/630523440 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0201a6240 con 0x7fc020106280 2024-05-08T22:07:30.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.001+0000 7fc01cc00700 1 -- 172.21.15.131:0/630523440 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc00c00ce80 con 0x7fc020106280 2024-05-08T22:07:30.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.001+0000 7fc01cc00700 1 -- 172.21.15.131:0/630523440 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c018420 con 0x7fc020106280 2024-05-08T22:07:30.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.001+0000 7fc026e00700 1 -- 172.21.15.131:0/630523440 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0201a67a0 con 0x7fc020106280 2024-05-08T22:07:30.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.006+0000 7fc01cc00700 1 -- 172.21.15.131:0/630523440 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7fc00c00c9c0 con 0x7fc020106280 2024-05-08T22:07:30.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.007+0000 7fc03a8ccb80 1 Processor -- start 2024-05-08T22:07:30.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.007+0000 7fc01cc00700 1 --2- 172.21.15.131:0/630523440 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc0040a13b0 0x7fc0040a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.007+0000 7fc01cc00700 1 -- 172.21.15.131:0/630523440 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fc00c014070 con 0x7fc020106280 2024-05-08T22:07:30.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.008+0000 7fc03a8ccb80 1 -- start start 2024-05-08T22:07:30.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.008+0000 7fc03a8ccb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560de1ce1610 0x560de1ce19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.008+0000 7fc03a8ccb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560de1ce1f30 0x560de1cdbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.008+0000 7fc03a8ccb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560de1cdc0a0 0x560de1cdc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.008+0000 7fc03a8ccb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560de1c2a150 con 0x560de1ce1610 2024-05-08T22:07:30.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.008+0000 7fc03a8ccb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560de1c27b10 con 0x560de1cdc0a0 2024-05-08T22:07:30.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.008+0000 7fc03a8ccb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560de1c23400 con 0x560de1ce1f30 2024-05-08T22:07:30.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.009+0000 7fc01fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560de1ce1610 0x560de1ce19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:30.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.009+0000 7fc01fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560de1ce1610 0x560de1ce19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50942/0 (socket says 172.21.15.131:50942) 2024-05-08T22:07:30.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.009+0000 7fc01fe00700 1 -- 172.21.15.131:0/58571399 learned_addr learned my addr 172.21.15.131:0/58571399 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:30.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.009+0000 7fc024e00700 1 --2- 172.21.15.131:0/630523440 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc0040a13b0 0x7fc0040a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:30.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.009+0000 7fc01fe00700 1 -- 172.21.15.131:0/58571399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560de1ce1f30 msgr2=0x560de1cdbb60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:07:30.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.009+0000 7fc01fe00700 1 --2- 172.21.15.131:0/58571399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560de1ce1f30 0x560de1cdbb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.009+0000 7fc01fe00700 1 -- 172.21.15.131:0/58571399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560de1cdc0a0 msgr2=0x560de1cdc4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:07:30.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.009+0000 7fc01fe00700 1 --2- 172.21.15.131:0/58571399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560de1cdc0a0 0x560de1cdc4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.009+0000 7fc01fe00700 1 -- 172.21.15.131:0/58571399 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560de1aa43b0 con 0x560de1ce1610 2024-05-08T22:07:30.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.010+0000 7fc01fe00700 1 --2- 172.21.15.131:0/58571399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560de1ce1610 0x560de1ce19f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc010009230 tx=0x7fc010005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59b09cf94d97013 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:30.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.010+0000 7fc024e00700 1 --2- 172.21.15.131:0/630523440 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc0040a13b0 0x7fc0040a3870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc018003f90 tx=0x7fc018009100 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:30.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.010+0000 7fc015600700 1 -- 172.21.15.131:0/58571399 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01000fe70 con 0x560de1ce1610 2024-05-08T22:07:30.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.010+0000 7fc015600700 1 -- 172.21.15.131:0/58571399 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc01001d040 con 0x560de1ce1610 2024-05-08T22:07:30.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.010+0000 7fc015600700 1 -- 172.21.15.131:0/58571399 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0100052a0 con 0x560de1ce1610 2024-05-08T22:07:30.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.011+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/58571399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560de1ce1610 msgr2=0x560de1ce19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:30.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.011+0000 7fc03a8ccb80 1 --2- 172.21.15.131:0/58571399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560de1ce1610 0x560de1ce19f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc010009230 tx=0x7fc010005dc0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.011+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/58571399 shutdown_connections 2024-05-08T22:07:30.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.011+0000 7fc03a8ccb80 1 --2- 172.21.15.131:0/58571399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560de1cdc0a0 0x560de1cdc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.011+0000 7fc03a8ccb80 1 --2- 172.21.15.131:0/58571399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560de1ce1610 0x560de1ce19f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.011+0000 7fc03a8ccb80 1 --2- 172.21.15.131:0/58571399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560de1ce1f30 0x560de1cdbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.011+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/58571399 >> 172.21.15.131:0/58571399 conn(0x560de1c31210 msgr2=0x560de1ccddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:30.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.012+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/58571399 shutdown_connections 2024-05-08T22:07:30.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.012+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/58571399 wait complete. 2024-05-08T22:07:30.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.012+0000 7fc03a8ccb80 1 -- start start 2024-05-08T22:07:30.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.013+0000 7fc03a8ccb80 1 Processor -- start 2024-05-08T22:07:30.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.013+0000 7fc03a8ccb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560de1cdc0a0 0x560de1cd5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.013+0000 7fc03a8ccb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560de1ce1610 0x560de1cd5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.013+0000 7fc03a8ccb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560de1ce1f30 0x560de1cdb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.013+0000 7fc03a8ccb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560de1bbf560 con 0x560de1cdc0a0 2024-05-08T22:07:30.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.013+0000 7fc03a8ccb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560de1bbf200 con 0x560de1ce1f30 2024-05-08T22:07:30.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.013+0000 7fc03a8ccb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560de1aa66a0 con 0x560de1ce1610 2024-05-08T22:07:30.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.013+0000 7fc03a8ccb80 10 client.0 _ll_register_callbacks cb 0x560de1bc8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:07:30.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.013+0000 7fc024e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560de1ce1f30 0x560de1cdb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:30.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.013+0000 7fc024e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560de1ce1f30 0x560de1cdb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:43864/0 (socket says 172.21.15.131:43864) 2024-05-08T22:07:30.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.013+0000 7fc01fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560de1cdc0a0 0x560de1cd5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:30.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.013+0000 7fc024e00700 1 -- 172.21.15.131:0/26544933 learned_addr learned my addr 172.21.15.131:0/26544933 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:30.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.013+0000 7fc01f400700 1 --2- 172.21.15.131:0/26544933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560de1ce1610 0x560de1cd5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:30.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.014+0000 7fc024e00700 1 -- 172.21.15.131:0/26544933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560de1ce1610 msgr2=0x560de1cd5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:30.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.014+0000 7fc024e00700 1 --2- 172.21.15.131:0/26544933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560de1ce1610 0x560de1cd5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.014+0000 7fc024e00700 1 -- 172.21.15.131:0/26544933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560de1cdc0a0 msgr2=0x560de1cd5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:30.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.014+0000 7fc024e00700 1 --2- 172.21.15.131:0/26544933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560de1cdc0a0 0x560de1cd5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.014+0000 7fc024e00700 1 -- 172.21.15.131:0/26544933 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc02010e670 con 0x560de1ce1f30 2024-05-08T22:07:30.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.014+0000 7fc01fe00700 1 --2- 172.21.15.131:0/26544933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560de1cdc0a0 0x560de1cd5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:30.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.014+0000 7fc024e00700 1 --2- 172.21.15.131:0/26544933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560de1ce1f30 0x560de1cdb5b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc0180156c0 tx=0x7fc018004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d57f9c1c83c92139 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:30.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.014+0000 7fc00a000700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:07:30.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.015+0000 7fc00a000700 1 -- 172.21.15.131:0/26544933 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc018007620 con 0x560de1ce1f30 2024-05-08T22:07:30.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.015+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/26544933 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560de1b7c270 con 0x560de1ce1f30 2024-05-08T22:07:30.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.015+0000 7fc00a000700 1 -- 172.21.15.131:0/26544933 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fc018007780 con 0x560de1ce1f30 2024-05-08T22:07:30.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.015+0000 7fc00a000700 1 -- 172.21.15.131:0/26544933 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc018015e60 con 0x560de1ce1f30 2024-05-08T22:07:30.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.015+0000 7fc03a8ccb80 10 client.25628 fetch_fsmap learned FSMap version 24 2024-05-08T22:07:30.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.015+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/26544933 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560de1a50b60 con 0x560de1ce1f30 2024-05-08T22:07:30.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.015+0000 7fc00a000700 1 -- 172.21.15.131:0/26544933 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560de1b7c270 con 0x560de1ce1f30 2024-05-08T22:07:30.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.016+0000 7fc00a000700 1 -- 172.21.15.131:0/26544933 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc018020b10 con 0x560de1ce1f30 2024-05-08T22:07:30.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.016+0000 7fc03a8ccb80 10 client.25628 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:07:30.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.016+0000 7fc03a8ccb80 10 client.25628 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:07:30.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.016+0000 7fc03a8ccb80 20 client.25628 populate_metadata read hostname 'smithi131' 2024-05-08T22:07:30.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.016+0000 7fc03a8ccb80 1 --2- 172.21.15.131:0/26544933 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560de1cfe2a0 0x560de1d006c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.016+0000 7fc03a8ccb80 4 client.25628 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:07:30.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.016+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/26544933 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560de1ca2810 con 0x560de1cfe2a0 2024-05-08T22:07:30.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.016+0000 7fc01fe00700 1 --2- 172.21.15.131:0/26544933 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560de1cfe2a0 0x560de1d006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:30.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.017+0000 7fc00a000700 10 client.25628 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:07:30.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.017+0000 7fc01fe00700 1 --2- 172.21.15.131:0/26544933 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560de1cfe2a0 0x560de1d006c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f3efd86a3f27443 server_cookie=dad463ddf082e920 in_seq=0 out_seq=0 2024-05-08T22:07:30.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.017+0000 7fc00a000700 1 -- 172.21.15.131:0/26544933 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560de1ca2810 con 0x560de1cfe2a0 2024-05-08T22:07:30.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.017+0000 7fc00a000700 10 client.25628 handle_command_reply: tid=0 2024-05-08T22:07:30.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.017+0000 7fc03a8ccb80 1 client.25628 shutdown 2024-05-08T22:07:30.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.018+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/26544933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560de1ce1f30 msgr2=0x560de1cdb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:30.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.018+0000 7fc03a8ccb80 1 --2- 172.21.15.131:0/26544933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560de1ce1f30 0x560de1cdb5b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc0180156c0 tx=0x7fc018004b40 comp rx=0 tx=0).stop 2024-05-08T22:07:30.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.018+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/26544933 shutdown_connections 2024-05-08T22:07:30.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.018+0000 7fc03a8ccb80 1 --2- 172.21.15.131:0/26544933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560de1ce1f30 0x560de1cdb5b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.018+0000 7fc03a8ccb80 1 --2- 172.21.15.131:0/26544933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560de1cdc0a0 0x560de1cd5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.018+0000 7fc03a8ccb80 1 --2- 172.21.15.131:0/26544933 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560de1cfe2a0 0x560de1d006c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.018+0000 7fc03a8ccb80 1 --2- 172.21.15.131:0/26544933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560de1ce1610 0x560de1cd5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.018+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/26544933 >> 172.21.15.131:0/26544933 conn(0x560de1c31210 msgr2=0x560de1ccd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:30.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.018+0000 7fc00a000700 0 client.25628 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:07:30.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.018+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/26544933 shutdown_connections 2024-05-08T22:07:30.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.018+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/26544933 wait complete. 2024-05-08T22:07:30.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.018+0000 7fc03a8ccb80 20 client.25628 trim_cache size 0 max 16384 2024-05-08T22:07:30.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.041+0000 7fc03a8ccb80 1 Processor -- start 2024-05-08T22:07:30.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.042+0000 7fc03a8ccb80 1 -- start start 2024-05-08T22:07:30.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.042+0000 7fc03a8ccb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560de1cdc0a0 0x560de1cdb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.042+0000 7fc03a8ccb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560de1ce1610 0x560de1ce0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.042+0000 7fc03a8ccb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560de1ce1f30 0x560de1ce7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.042+0000 7fc03a8ccb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560de1d058d0 con 0x560de1ce1610 2024-05-08T22:07:30.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.042+0000 7fc03a8ccb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560de1cdb920 con 0x560de1cdc0a0 2024-05-08T22:07:30.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.042+0000 7fc03a8ccb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560de1cd0970 con 0x560de1ce1f30 2024-05-08T22:07:30.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.042+0000 7fc01fe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560de1cdc0a0 0x560de1cdb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:30.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.042+0000 7fc01fe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560de1cdc0a0 0x560de1cdb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:43870/0 (socket says 172.21.15.131:43870) 2024-05-08T22:07:30.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.042+0000 7fc01fe00700 1 -- 172.21.15.131:0/4072714018 learned_addr learned my addr 172.21.15.131:0/4072714018 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:30.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.042+0000 7fc01f400700 1 --2- 172.21.15.131:0/4072714018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560de1ce1610 0x560de1ce0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:30.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.042+0000 7fc024e00700 1 --2- 172.21.15.131:0/4072714018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560de1ce1f30 0x560de1ce7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:30.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.043+0000 7fc01fe00700 1 -- 172.21.15.131:0/4072714018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560de1ce1f30 msgr2=0x560de1ce7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:30.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.043+0000 7fc01fe00700 1 --2- 172.21.15.131:0/4072714018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560de1ce1f30 0x560de1ce7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.043+0000 7fc01fe00700 1 -- 172.21.15.131:0/4072714018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560de1ce1610 msgr2=0x560de1ce0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:30.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.043+0000 7fc01fe00700 1 --2- 172.21.15.131:0/4072714018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560de1ce1610 0x560de1ce0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.043+0000 7fc01fe00700 1 -- 172.21.15.131:0/4072714018 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560de1be9560 con 0x560de1cdc0a0 2024-05-08T22:07:30.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.043+0000 7fc01f400700 1 --2- 172.21.15.131:0/4072714018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560de1ce1610 0x560de1ce0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:07:30.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.043+0000 7fc024e00700 1 --2- 172.21.15.131:0/4072714018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560de1ce1f30 0x560de1ce7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:30.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.043+0000 7fc01fe00700 1 --2- 172.21.15.131:0/4072714018 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560de1cdc0a0 0x560de1cdb3e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc01000b9b0 tx=0x7fc01000b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aec7f210d6b6247 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:30.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.043+0000 7fc016000700 1 -- 172.21.15.131:0/4072714018 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01000f460 con 0x560de1cdc0a0 2024-05-08T22:07:30.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.043+0000 7fc016000700 1 -- 172.21.15.131:0/4072714018 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0100050c0 con 0x560de1cdc0a0 2024-05-08T22:07:30.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.043+0000 7fc016000700 1 -- 172.21.15.131:0/4072714018 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0100038c0 con 0x560de1cdc0a0 2024-05-08T22:07:30.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.044+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/4072714018 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560de1cdc0a0 msgr2=0x560de1cdb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:30.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.044+0000 7fc03a8ccb80 1 --2- 172.21.15.131:0/4072714018 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560de1cdc0a0 0x560de1cdb3e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc01000b9b0 tx=0x7fc01000b940 comp rx=0 tx=0).stop 2024-05-08T22:07:30.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.044+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/4072714018 shutdown_connections 2024-05-08T22:07:30.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.044+0000 7fc03a8ccb80 1 --2- 172.21.15.131:0/4072714018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560de1ce1f30 0x560de1ce7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.044+0000 7fc03a8ccb80 1 --2- 172.21.15.131:0/4072714018 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560de1cdc0a0 0x560de1cdb3e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.044+0000 7fc03a8ccb80 1 --2- 172.21.15.131:0/4072714018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560de1ce1610 0x560de1ce0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.044+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/4072714018 >> 172.21.15.131:0/4072714018 conn(0x560de1c31210 msgr2=0x560de1cdc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:30.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.044+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/4072714018 shutdown_connections 2024-05-08T22:07:30.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.044+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/4072714018 wait complete. 2024-05-08T22:07:30.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.045+0000 7fc03a8ccb80 1 -- start start 2024-05-08T22:07:30.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.045+0000 7fc03a8ccb80 1 Processor -- start 2024-05-08T22:07:30.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.045+0000 7fc03a8ccb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560de1cdc0a0 0x560de1cdb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.045+0000 7fc03a8ccb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560de1ce1610 0x560de1cd69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.046+0000 7fc03a8ccb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560de1ce1f30 0x560de1cd6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.046+0000 7fc03a8ccb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560de1cd76a0 con 0x560de1ce1f30 2024-05-08T22:07:30.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.046+0000 7fc03a8ccb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560de1cd7810 con 0x560de1cdc0a0 2024-05-08T22:07:30.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.046+0000 7fc03a8ccb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560de1d537c0 con 0x560de1ce1610 2024-05-08T22:07:30.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.046+0000 7fc03a8ccb80 10 client.0 _ll_register_callbacks cb 0x560de1bc8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:07:30.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.046+0000 7fc01fe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560de1cdc0a0 0x560de1cdb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:30.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.046+0000 7fc01fe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560de1cdc0a0 0x560de1cdb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:43878/0 (socket says 172.21.15.131:43878) 2024-05-08T22:07:30.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.046+0000 7fc01fe00700 1 -- 172.21.15.131:0/3676179534 learned_addr learned my addr 172.21.15.131:0/3676179534 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:30.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.046+0000 7fc01fe00700 1 -- 172.21.15.131:0/3676179534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560de1ce1610 msgr2=0x560de1cd69b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:07:30.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.046+0000 7fc01fe00700 1 --2- 172.21.15.131:0/3676179534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560de1ce1610 0x560de1cd69b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.046+0000 7fc01fe00700 1 -- 172.21.15.131:0/3676179534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560de1ce1f30 msgr2=0x560de1cd6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:07:30.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.046+0000 7fc01fe00700 1 --2- 172.21.15.131:0/3676179534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560de1ce1f30 0x560de1cd6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.046+0000 7fc01fe00700 1 -- 172.21.15.131:0/3676179534 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560de1a50b60 con 0x560de1cdc0a0 2024-05-08T22:07:30.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.047+0000 7fc01fe00700 1 --2- 172.21.15.131:0/3676179534 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560de1cdc0a0 0x560de1cdb450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc010009440 tx=0x7fc01000b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d488aad55ef00cb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:30.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.047+0000 7fc00a000700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:07:30.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.047+0000 7fc00a000700 1 -- 172.21.15.131:0/3676179534 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01000f660 con 0x560de1cdc0a0 2024-05-08T22:07:30.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.047+0000 7fc00a000700 1 -- 172.21.15.131:0/3676179534 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fc01000f7c0 con 0x560de1cdc0a0 2024-05-08T22:07:30.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.047+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/3676179534 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc01001d040 con 0x560de1cdc0a0 2024-05-08T22:07:30.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.048+0000 7fc00a000700 1 -- 172.21.15.131:0/3676179534 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0100056b0 con 0x560de1cdc0a0 2024-05-08T22:07:30.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.048+0000 7fc03a8ccb80 10 client.25634 fetch_fsmap learned FSMap version 24 2024-05-08T22:07:30.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.048+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/3676179534 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560de1aa43b0 con 0x560de1cdc0a0 2024-05-08T22:07:30.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.048+0000 7fc00a000700 1 -- 172.21.15.131:0/3676179534 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc01001d040 con 0x560de1cdc0a0 2024-05-08T22:07:30.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.048+0000 7fc00a000700 1 -- 172.21.15.131:0/3676179534 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc010009c70 con 0x560de1cdc0a0 2024-05-08T22:07:30.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.048+0000 7fc03a8ccb80 10 client.25634 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:07:30.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.048+0000 7fc03a8ccb80 10 client.25634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:07:30.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.048+0000 7fc03a8ccb80 20 client.25634 populate_metadata read hostname 'smithi131' 2024-05-08T22:07:30.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.048+0000 7fc03a8ccb80 1 --2- 172.21.15.131:0/3676179534 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560de1d54b40 0x560de1d56f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.048+0000 7fc03a8ccb80 4 client.25634 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:07:30.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.048+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/3676179534 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560de1c1bec0 con 0x560de1d54b40 2024-05-08T22:07:30.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.049+0000 7fc01f400700 1 --2- 172.21.15.131:0/3676179534 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560de1d54b40 0x560de1d56f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:30.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.049+0000 7fc01f400700 1 --2- 172.21.15.131:0/3676179534 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560de1d54b40 0x560de1d56f60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=808c8d0f14d1c2bd server_cookie=ddc37d76a3b99d5d in_seq=0 out_seq=0 2024-05-08T22:07:30.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.049+0000 7fc00a000700 10 client.25634 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:07:30.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.049+0000 7fc00a000700 1 -- 172.21.15.131:0/3676179534 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560de1c1bec0 con 0x560de1d54b40 2024-05-08T22:07:30.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.049+0000 7fc00a000700 10 client.25634 handle_command_reply: tid=0 2024-05-08T22:07:30.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.049+0000 7fc03a8ccb80 1 client.25634 shutdown 2024-05-08T22:07:30.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.049+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/3676179534 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560de1cdc0a0 msgr2=0x560de1cdb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:30.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.049+0000 7fc03a8ccb80 1 --2- 172.21.15.131:0/3676179534 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560de1cdc0a0 0x560de1cdb450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc010009440 tx=0x7fc01000b850 comp rx=0 tx=0).stop 2024-05-08T22:07:30.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.049+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/3676179534 shutdown_connections 2024-05-08T22:07:30.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.049+0000 7fc03a8ccb80 1 --2- 172.21.15.131:0/3676179534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560de1ce1f30 0x560de1cd6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.049+0000 7fc03a8ccb80 1 --2- 172.21.15.131:0/3676179534 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560de1cdc0a0 0x560de1cdb450 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.049+0000 7fc03a8ccb80 1 --2- 172.21.15.131:0/3676179534 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560de1d54b40 0x560de1d56f60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.050+0000 7fc03a8ccb80 1 --2- 172.21.15.131:0/3676179534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560de1ce1610 0x560de1cd69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.050+0000 7fc00a000700 0 client.25634 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:07:30.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.050+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/3676179534 >> 172.21.15.131:0/3676179534 conn(0x560de1c31210 msgr2=0x560de1ce70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:30.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.050+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/3676179534 shutdown_connections 2024-05-08T22:07:30.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.050+0000 7fc03a8ccb80 1 -- 172.21.15.131:0/3676179534 wait complete. 2024-05-08T22:07:30.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.050+0000 7fc03a8ccb80 20 client.25634 trim_cache size 0 max 16384 2024-05-08T22:07:30.056 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:07:30.057 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T22:07:30.057 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-08T22:07:30.057 INFO:teuthology.orchestra.run.smithi131.stdout: "3a8ed3b9-29e9-46c4-b86f-ad39967dab74": { 2024-05-08T22:07:30.057 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-08T22:07:30.057 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "3a8ed3b9-29e9-46c4-b86f-ad39967dab74", 2024-05-08T22:07:30.057 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-08T22:07:30.057 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:07:30.057 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:07:30.057 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:07:30.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.050+0000 7fc026e00700 1 -- 172.21.15.131:0/630523440 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc0040a13b0 msgr2=0x7fc0040a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:30.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.050+0000 7fc026e00700 1 --2- 172.21.15.131:0/630523440 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc0040a13b0 0x7fc0040a3870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc018003f90 tx=0x7fc018009100 comp rx=0 tx=0).stop 2024-05-08T22:07:30.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.051+0000 7fc026e00700 1 -- 172.21.15.131:0/630523440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc020106280 msgr2=0x7fc0200737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:30.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.051+0000 7fc026e00700 1 --2- 172.21.15.131:0/630523440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc020106280 0x7fc0200737e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00e9f0 tx=0x7fc00c00ed00 comp rx=0 tx=0).stop 2024-05-08T22:07:30.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.051+0000 7fc026e00700 1 -- 172.21.15.131:0/630523440 shutdown_connections 2024-05-08T22:07:30.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.051+0000 7fc026e00700 1 --2- 172.21.15.131:0/630523440 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc0040a13b0 0x7fc0040a3870 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.051+0000 7fc026e00700 1 --2- 172.21.15.131:0/630523440 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc020073d20 0x7fc0201a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.051+0000 7fc026e00700 1 --2- 172.21.15.131:0/630523440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc020101160 0x7fc0200732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.051+0000 7fc026e00700 1 --2- 172.21.15.131:0/630523440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc020106280 0x7fc0200737e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.051+0000 7fc026e00700 1 -- 172.21.15.131:0/630523440 >> 172.21.15.131:0/630523440 conn(0x7fc0200fc9c0 msgr2=0x7fc0200fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:30.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.051+0000 7fc026e00700 1 -- 172.21.15.131:0/630523440 shutdown_connections 2024-05-08T22:07:30.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.051+0000 7fc026e00700 1 -- 172.21.15.131:0/630523440 wait complete. 2024-05-08T22:07:30.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.064+0000 7f8ab2000700 1 -- 172.21.15.131:0/3533964268 <== mgr.15366 v2:172.21.15.131:6800/1157588501 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f8ab419ed50 con 0x7f8a9c0a13b0 2024-05-08T22:07:30.065 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule set for path / 2024-05-08T22:07:30.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.068+0000 7f8abbe00700 1 -- 172.21.15.131:0/3533964268 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8a9c0a13b0 msgr2=0x7f8a9c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:30.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.068+0000 7f8abbe00700 1 --2- 172.21.15.131:0/3533964268 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8a9c0a13b0 0x7f8a9c0a3870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8ab419ec40 tx=0x7f8aa000b260 comp rx=0 tx=0).stop 2024-05-08T22:07:30.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.068+0000 7f8abbe00700 1 -- 172.21.15.131:0/3533964268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ab4073340 msgr2=0x7f8ab41a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:30.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.068+0000 7f8abbe00700 1 --2- 172.21.15.131:0/3533964268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ab4073340 0x7f8ab41a36f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800e9c0 tx=0x7f8aa800ecd0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.068+0000 7f8abbe00700 1 -- 172.21.15.131:0/3533964268 shutdown_connections 2024-05-08T22:07:30.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.068+0000 7f8abbe00700 1 --2- 172.21.15.131:0/3533964268 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8a9c0a13b0 0x7f8a9c0a3870 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8ab419ec40 tx=0x7f8aa000b260 comp rx=0 tx=0).stop 2024-05-08T22:07:30.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.068+0000 7f8abbe00700 1 --2- 172.21.15.131:0/3533964268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ab4111ea0 0x7f8ab419d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.068+0000 7f8abbe00700 1 --2- 172.21.15.131:0/3533964268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ab4073340 0x7f8ab41a36f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.068+0000 7f8abbe00700 1 --2- 172.21.15.131:0/3533964268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab4073c60 0x7f8ab41a3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.068+0000 7f8abbe00700 1 -- 172.21.15.131:0/3533964268 >> 172.21.15.131:0/3533964268 conn(0x7f8ab40fc9a0 msgr2=0x7f8ab40fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:30.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.069+0000 7f8abbe00700 1 -- 172.21.15.131:0/3533964268 shutdown_connections 2024-05-08T22:07:30.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.069+0000 7f8abbe00700 1 -- 172.21.15.131:0/3533964268 wait complete. 2024-05-08T22:07:30.075 INFO:tasks.cephfs.filesystem:scrub status for tag:3a8ed3b9-29e9-46c4-b86f-ad39967dab74 - {'path': '/', 'tag': '3a8ed3b9-29e9-46c4-b86f-ad39967dab74', 'options': 'recursive,force'} 2024-05-08T22:07:30.089 DEBUG:teuthology.orchestra.run.smithi131:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-08T22:07:30.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.321+0000 7f11eb000700 1 -- 172.21.15.131:0/2248407154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11e41018f0 msgr2=0x7f11e4105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:30.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.321+0000 7f11eb000700 1 --2- 172.21.15.131:0/2248407154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11e41018f0 0x7f11e4105930 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f11d0009990 tx=0x7f11d0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.321+0000 7f11eb000700 1 -- 172.21.15.131:0/2248407154 shutdown_connections 2024-05-08T22:07:30.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.321+0000 7f11eb000700 1 --2- 172.21.15.131:0/2248407154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11e4106060 0x7f11e410dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.321+0000 7f11eb000700 1 --2- 172.21.15.131:0/2248407154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11e4100f40 0x7f11e4101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.321+0000 7f11eb000700 1 --2- 172.21.15.131:0/2248407154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11e41018f0 0x7f11e4105930 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.321+0000 7f11eb000700 1 -- 172.21.15.131:0/2248407154 >> 172.21.15.131:0/2248407154 conn(0x7f11e40fc800 msgr2=0x7f11e40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:30.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.321+0000 7f11eb000700 1 -- 172.21.15.131:0/2248407154 shutdown_connections 2024-05-08T22:07:30.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.322+0000 7f11eb000700 1 -- 172.21.15.131:0/2248407154 wait complete. 2024-05-08T22:07:30.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.322+0000 7f11eb000700 1 Processor -- start 2024-05-08T22:07:30.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.323+0000 7f11eb000700 1 -- start start 2024-05-08T22:07:30.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.323+0000 7f11eb000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11e4100f40 0x7f11e4199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.323+0000 7f11eb000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11e4106060 0x7f11e4199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.323+0000 7f11eb000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11e419dd20 0x7f11e419e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.323+0000 7f11eb000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e419e7a0 con 0x7f11e419dd20 2024-05-08T22:07:30.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.323+0000 7f11eb000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e419e910 con 0x7f11e4106060 2024-05-08T22:07:30.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.323+0000 7f11eb000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e419ead0 con 0x7f11e4100f40 2024-05-08T22:07:30.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.323+0000 7f11ea600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11e419dd20 0x7f11e419e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:30.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.323+0000 7f11e9200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11e4106060 0x7f11e4199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:30.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.323+0000 7f11e9c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11e4100f40 0x7f11e4199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:30.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.323+0000 7f11e9200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11e4106060 0x7f11e4199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:43898/0 (socket says 172.21.15.131:43898) 2024-05-08T22:07:30.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.323+0000 7f11e9200700 1 -- 172.21.15.131:0/1481136358 learned_addr learned my addr 172.21.15.131:0/1481136358 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:30.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.324+0000 7f11e9200700 1 -- 172.21.15.131:0/1481136358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11e4100f40 msgr2=0x7f11e4199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:30.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.324+0000 7f11e9200700 1 --2- 172.21.15.131:0/1481136358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11e4100f40 0x7f11e4199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.324+0000 7f11e9200700 1 -- 172.21.15.131:0/1481136358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11e419dd20 msgr2=0x7f11e419e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:30.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.324+0000 7f11e9200700 1 --2- 172.21.15.131:0/1481136358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11e419dd20 0x7f11e419e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.324+0000 7f11e9200700 1 -- 172.21.15.131:0/1481136358 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d00095f0 con 0x7f11e4106060 2024-05-08T22:07:30.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.324+0000 7f11e9200700 1 --2- 172.21.15.131:0/1481136358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11e4106060 0x7f11e4199980 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f11d0000c00 tx=0x7f11d0005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:30.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.324+0000 7f11e2000700 1 -- 172.21.15.131:0/1481136358 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d001d070 con 0x7f11e4106060 2024-05-08T22:07:30.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.324+0000 7f11e2000700 1 -- 172.21.15.131:0/1481136358 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11d0022470 con 0x7f11e4106060 2024-05-08T22:07:30.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.324+0000 7f11eb000700 1 -- 172.21.15.131:0/1481136358 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e41a5fd0 con 0x7f11e4106060 2024-05-08T22:07:30.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.324+0000 7f11e2000700 1 -- 172.21.15.131:0/1481136358 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d0017650 con 0x7f11e4106060 2024-05-08T22:07:30.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.324+0000 7f11eb000700 1 -- 172.21.15.131:0/1481136358 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e41a6390 con 0x7f11e4106060 2024-05-08T22:07:30.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.325+0000 7f11eb000700 1 -- 172.21.15.131:0/1481136358 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e4102410 con 0x7f11e4106060 2024-05-08T22:07:30.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.327+0000 7f11e2000700 1 -- 172.21.15.131:0/1481136358 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7f11d000fad0 con 0x7f11e4106060 2024-05-08T22:07:30.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.328+0000 7f11e2000700 1 --2- 172.21.15.131:0/1481136358 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f11cc0a13b0 0x7f11cc0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.329+0000 7f11e2000700 1 -- 172.21.15.131:0/1481136358 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f11d00ce720 con 0x7f11e4106060 2024-05-08T22:07:30.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.329+0000 7f11e9c00700 1 --2- 172.21.15.131:0/1481136358 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f11cc0a13b0 0x7f11cc0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:30.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.329+0000 7f11e9c00700 1 --2- 172.21.15.131:0/1481136358 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f11cc0a13b0 0x7f11cc0a3870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009c60 tx=0x7f11d8009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:30.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.331+0000 7f11e2000700 1 -- 172.21.15.131:0/1481136358 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f11d00975c0 con 0x7f11e4106060 2024-05-08T22:07:30.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:30 smithi131 ceph-mon[37260]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 750 B/s wr, 17 op/s 2024-05-08T22:07:30.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:30 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2099209562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:07:30.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:30 smithi131 ceph-mon[37260]: scrub queued for path: / 2024-05-08T22:07:30.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:30 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:07:30.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:30 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:07:30.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:30 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:07:30.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:30 smithi131 ceph-mon[37260]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:07:30.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:30 smithi157 ceph-mon[77746]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 750 B/s wr, 17 op/s 2024-05-08T22:07:30.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:30 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2099209562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:07:30.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:30 smithi157 ceph-mon[77746]: scrub queued for path: / 2024-05-08T22:07:30.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:30 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:07:30.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:30 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:07:30.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:30 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:07:30.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:30 smithi157 ceph-mon[77746]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:07:30.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.569+0000 7f11eb000700 1 -- 172.21.15.131:0/1481136358 --> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f11e41018f0 con 0x7f11cc0a13b0 2024-05-08T22:07:30.589 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:30 smithi132 ceph-mon[78180]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 750 B/s wr, 17 op/s 2024-05-08T22:07:30.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:30 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2099209562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:07:30.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:30 smithi132 ceph-mon[78180]: scrub queued for path: / 2024-05-08T22:07:30.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:30 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:07:30.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:30 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:07:30.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:30 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:07:30.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:30 smithi132 ceph-mon[78180]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:07:30.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.635+0000 7f11e2000700 1 -- 172.21.15.131:0/1481136358 <== mgr.15366 v2:172.21.15.131:6800/1157588501 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f11e41018f0 con 0x7f11cc0a13b0 2024-05-08T22:07:30.636 INFO:teuthology.orchestra.run.smithi131.stdout:Retention added to path / 2024-05-08T22:07:30.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.637+0000 7f11eb000700 1 -- 172.21.15.131:0/1481136358 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f11cc0a13b0 msgr2=0x7f11cc0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:30.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.637+0000 7f11eb000700 1 --2- 172.21.15.131:0/1481136358 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f11cc0a13b0 0x7f11cc0a3870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009c60 tx=0x7f11d8009260 comp rx=0 tx=0).stop 2024-05-08T22:07:30.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.637+0000 7f11eb000700 1 -- 172.21.15.131:0/1481136358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11e4106060 msgr2=0x7f11e4199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:30.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.637+0000 7f11eb000700 1 --2- 172.21.15.131:0/1481136358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11e4106060 0x7f11e4199980 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f11d0000c00 tx=0x7f11d0005740 comp rx=0 tx=0).stop 2024-05-08T22:07:30.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.637+0000 7f11eb000700 1 -- 172.21.15.131:0/1481136358 shutdown_connections 2024-05-08T22:07:30.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.637+0000 7f11eb000700 1 --2- 172.21.15.131:0/1481136358 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f11cc0a13b0 0x7f11cc0a3870 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.637+0000 7f11eb000700 1 --2- 172.21.15.131:0/1481136358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11e419dd20 0x7f11e419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.637+0000 7f11eb000700 1 --2- 172.21.15.131:0/1481136358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11e4100f40 0x7f11e4199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.637+0000 7f11eb000700 1 --2- 172.21.15.131:0/1481136358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11e4106060 0x7f11e4199980 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.638+0000 7f11eb000700 1 -- 172.21.15.131:0/1481136358 >> 172.21.15.131:0/1481136358 conn(0x7f11e40fc800 msgr2=0x7f11e40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:30.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.638+0000 7f11eb000700 1 -- 172.21.15.131:0/1481136358 shutdown_connections 2024-05-08T22:07:30.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.638+0000 7f11eb000700 1 -- 172.21.15.131:0/1481136358 wait complete. 2024-05-08T22:07:30.658 DEBUG:teuthology.orchestra.run.smithi131:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-08T22:07:30.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.848+0000 7fb52ac00700 1 -- 172.21.15.131:0/331603480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb524101aa0 msgr2=0x7fb524105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:30.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.848+0000 7fb52ac00700 1 --2- 172.21.15.131:0/331603480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb524101aa0 0x7fb524105ae0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb510009990 tx=0x7fb510009ca0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.848+0000 7fb52ac00700 1 -- 172.21.15.131:0/331603480 shutdown_connections 2024-05-08T22:07:30.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.848+0000 7fb52ac00700 1 --2- 172.21.15.131:0/331603480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb524106210 0x7fb52410dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.848+0000 7fb52ac00700 1 --2- 172.21.15.131:0/331603480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5241010f0 0x7fb5241014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.848+0000 7fb52ac00700 1 --2- 172.21.15.131:0/331603480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb524101aa0 0x7fb524105ae0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.848+0000 7fb52ac00700 1 -- 172.21.15.131:0/331603480 >> 172.21.15.131:0/331603480 conn(0x7fb5240fc970 msgr2=0x7fb5240fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:30.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.848+0000 7fb52ac00700 1 -- 172.21.15.131:0/331603480 shutdown_connections 2024-05-08T22:07:30.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.848+0000 7fb52ac00700 1 -- 172.21.15.131:0/331603480 wait complete. 2024-05-08T22:07:30.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.850+0000 7fb52ac00700 1 Processor -- start 2024-05-08T22:07:30.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.850+0000 7fb52ac00700 1 -- start start 2024-05-08T22:07:30.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.850+0000 7fb52ac00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5241010f0 0x7fb524197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.850+0000 7fb52ac00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb524106210 0x7fb524197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.850+0000 7fb52ac00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb52419bbd0 0x7fb52419c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.850+0000 7fb52ac00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52419c650 con 0x7fb52419bbd0 2024-05-08T22:07:30.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.850+0000 7fb52ac00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52419c7c0 con 0x7fb5241010f0 2024-05-08T22:07:30.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.850+0000 7fb52ac00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52419c980 con 0x7fb524106210 2024-05-08T22:07:30.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.851+0000 7fb528c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb52419bbd0 0x7fb52419c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:30.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.851+0000 7fb523e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5241010f0 0x7fb524197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:30.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.851+0000 7fb528c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb52419bbd0 0x7fb52419c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51002/0 (socket says 172.21.15.131:51002) 2024-05-08T22:07:30.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.851+0000 7fb528c00700 1 -- 172.21.15.131:0/627119614 learned_addr learned my addr 172.21.15.131:0/627119614 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:30.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.851+0000 7fb523400700 1 --2- 172.21.15.131:0/627119614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb524106210 0x7fb524197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:30.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.851+0000 7fb528c00700 1 -- 172.21.15.131:0/627119614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb524106210 msgr2=0x7fb524197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:30.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.851+0000 7fb528c00700 1 --2- 172.21.15.131:0/627119614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb524106210 0x7fb524197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.851+0000 7fb528c00700 1 -- 172.21.15.131:0/627119614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5241010f0 msgr2=0x7fb524197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:30.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.851+0000 7fb528c00700 1 --2- 172.21.15.131:0/627119614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5241010f0 0x7fb524197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:30.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.851+0000 7fb528c00700 1 -- 172.21.15.131:0/627119614 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5100095f0 con 0x7fb52419bbd0 2024-05-08T22:07:30.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.851+0000 7fb523e00700 1 --2- 172.21.15.131:0/627119614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5241010f0 0x7fb524197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:07:30.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.851+0000 7fb528c00700 1 --2- 172.21.15.131:0/627119614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb52419bbd0 0x7fb52419c050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00ef30 tx=0x7fb51c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:30.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.852+0000 7fb520c00700 1 -- 172.21.15.131:0/627119614 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c018070 con 0x7fb52419bbd0 2024-05-08T22:07:30.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.852+0000 7fb520c00700 1 -- 172.21.15.131:0/627119614 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb51c00f040 con 0x7fb52419bbd0 2024-05-08T22:07:30.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.852+0000 7fb52ac00700 1 -- 172.21.15.131:0/627119614 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5241a3e80 con 0x7fb52419bbd0 2024-05-08T22:07:30.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.852+0000 7fb520c00700 1 -- 172.21.15.131:0/627119614 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c00cd00 con 0x7fb52419bbd0 2024-05-08T22:07:30.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.852+0000 7fb52ac00700 1 -- 172.21.15.131:0/627119614 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5241a4360 con 0x7fb52419bbd0 2024-05-08T22:07:30.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.853+0000 7fb52ac00700 1 -- 172.21.15.131:0/627119614 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5241094e0 con 0x7fb52419bbd0 2024-05-08T22:07:30.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.855+0000 7fb520c00700 1 -- 172.21.15.131:0/627119614 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7fb51c003a00 con 0x7fb52419bbd0 2024-05-08T22:07:30.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.856+0000 7fb520c00700 1 --2- 172.21.15.131:0/627119614 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb5080a1430 0x7fb5080a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:30.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.856+0000 7fb520c00700 1 -- 172.21.15.131:0/627119614 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fb51c0cce10 con 0x7fb52419bbd0 2024-05-08T22:07:30.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.856+0000 7fb523e00700 1 --2- 172.21.15.131:0/627119614 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb5080a1430 0x7fb5080a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:30.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.857+0000 7fb523e00700 1 --2- 172.21.15.131:0/627119614 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb5080a1430 0x7fb5080a38f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb514011490 tx=0x7fb514009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:30.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:30.860+0000 7fb520c00700 1 -- 172.21.15.131:0/627119614 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb51c0d0030 con 0x7fb52419bbd0 2024-05-08T22:07:31.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.094+0000 7fb52ac00700 1 -- 172.21.15.131:0/627119614 --> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fb524101aa0 con 0x7fb5080a1430 2024-05-08T22:07:31.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.102+0000 7fb520c00700 1 -- 172.21.15.131:0/627119614 <== mgr.15366 v2:172.21.15.131:6800/1157588501 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7fb524101aa0 con 0x7fb5080a1430 2024-05-08T22:07:31.103 INFO:teuthology.orchestra.run.smithi131.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-08T00:00:00", "created": "2024-05-08T22:07:29", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-08T22:07:31.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.105+0000 7fb52ac00700 1 -- 172.21.15.131:0/627119614 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb5080a1430 msgr2=0x7fb5080a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:31.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.105+0000 7fb52ac00700 1 --2- 172.21.15.131:0/627119614 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb5080a1430 0x7fb5080a38f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb514011490 tx=0x7fb514009260 comp rx=0 tx=0).stop 2024-05-08T22:07:31.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.105+0000 7fb52ac00700 1 -- 172.21.15.131:0/627119614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb52419bbd0 msgr2=0x7fb52419c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:31.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.105+0000 7fb52ac00700 1 --2- 172.21.15.131:0/627119614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb52419bbd0 0x7fb52419c050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00ef30 tx=0x7fb51c00c3a0 comp rx=0 tx=0).stop 2024-05-08T22:07:31.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.105+0000 7fb52ac00700 1 -- 172.21.15.131:0/627119614 shutdown_connections 2024-05-08T22:07:31.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.105+0000 7fb52ac00700 1 --2- 172.21.15.131:0/627119614 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb5080a1430 0x7fb5080a38f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:31.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.105+0000 7fb52ac00700 1 --2- 172.21.15.131:0/627119614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb52419bbd0 0x7fb52419c050 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:31.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.105+0000 7fb52ac00700 1 --2- 172.21.15.131:0/627119614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5241010f0 0x7fb524197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:31.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.105+0000 7fb52ac00700 1 --2- 172.21.15.131:0/627119614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb524106210 0x7fb524197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:31.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.105+0000 7fb52ac00700 1 -- 172.21.15.131:0/627119614 >> 172.21.15.131:0/627119614 conn(0x7fb5240fc970 msgr2=0x7fb5240fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:31.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.105+0000 7fb52ac00700 1 -- 172.21.15.131:0/627119614 shutdown_connections 2024-05-08T22:07:31.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.105+0000 7fb52ac00700 1 -- 172.21.15.131:0/627119614 wait complete. 2024-05-08T22:07:31.125 DEBUG:teuthology.orchestra.run.smithi131:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-08T22:07:31.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.317+0000 7f3211200700 1 -- 172.21.15.131:0/4205260446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f320c106220 msgr2=0x7f320c10dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:31.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.317+0000 7f3211200700 1 --2- 172.21.15.131:0/4205260446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f320c106220 0x7f320c10dda0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f320400b0a0 tx=0x7f320400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:07:31.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.317+0000 7f3211200700 1 -- 172.21.15.131:0/4205260446 shutdown_connections 2024-05-08T22:07:31.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.317+0000 7f3211200700 1 --2- 172.21.15.131:0/4205260446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f320c106220 0x7f320c10dda0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:31.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.317+0000 7f3211200700 1 --2- 172.21.15.131:0/4205260446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f320c101100 0x7f320c1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:31.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.317+0000 7f3211200700 1 --2- 172.21.15.131:0/4205260446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f320c101ab0 0x7f320c105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:31.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.317+0000 7f3211200700 1 -- 172.21.15.131:0/4205260446 >> 172.21.15.131:0/4205260446 conn(0x7f320c0fc9a0 msgr2=0x7f320c0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:31.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.317+0000 7f3211200700 1 -- 172.21.15.131:0/4205260446 shutdown_connections 2024-05-08T22:07:31.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.317+0000 7f3211200700 1 -- 172.21.15.131:0/4205260446 wait complete. 2024-05-08T22:07:31.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.318+0000 7f3211200700 1 Processor -- start 2024-05-08T22:07:31.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.318+0000 7f3211200700 1 -- start start 2024-05-08T22:07:31.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.319+0000 7f3211200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f320c101100 0x7f320c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:31.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.319+0000 7f3211200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f320c106220 0x7f320c199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:31.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.319+0000 7f3211200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f320c19dfc0 0x7f320c19e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:31.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.319+0000 7f3211200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c19ea40 con 0x7f320c101100 2024-05-08T22:07:31.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.319+0000 7f3211200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c19ebb0 con 0x7f320c19dfc0 2024-05-08T22:07:31.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.319+0000 7f3211200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c19ed70 con 0x7f320c106220 2024-05-08T22:07:31.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.319+0000 7f3209e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f320c101100 0x7f320c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:31.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.319+0000 7f3209e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f320c101100 0x7f320c199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51026/0 (socket says 172.21.15.131:51026) 2024-05-08T22:07:31.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.319+0000 7f3209e00700 1 -- 172.21.15.131:0/2414145229 learned_addr learned my addr 172.21.15.131:0/2414145229 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:07:31.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.319+0000 7f3209400700 1 --2- 172.21.15.131:0/2414145229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f320c106220 0x7f320c199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:31.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.319+0000 7f320a800700 1 --2- 172.21.15.131:0/2414145229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f320c19dfc0 0x7f320c19e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:31.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.319+0000 7f3209e00700 1 -- 172.21.15.131:0/2414145229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f320c106220 msgr2=0x7f320c199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:31.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.319+0000 7f3209e00700 1 --2- 172.21.15.131:0/2414145229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f320c106220 0x7f320c199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:31.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.319+0000 7f3209e00700 1 -- 172.21.15.131:0/2414145229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f320c19dfc0 msgr2=0x7f320c19e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:31.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.319+0000 7f3209e00700 1 --2- 172.21.15.131:0/2414145229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f320c19dfc0 0x7f320c19e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:31.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.319+0000 7f3209e00700 1 -- 172.21.15.131:0/2414145229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3204009d00 con 0x7f320c101100 2024-05-08T22:07:31.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.320+0000 7f3209e00700 1 --2- 172.21.15.131:0/2414145229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f320c101100 0x7f320c199730 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00e9c0 tx=0x7f31fc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:31.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.320+0000 7f3202a00700 1 -- 172.21.15.131:0/2414145229 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0097b0 con 0x7f320c101100 2024-05-08T22:07:31.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.320+0000 7f3202a00700 1 -- 172.21.15.131:0/2414145229 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31fc00ee60 con 0x7f320c101100 2024-05-08T22:07:31.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.320+0000 7f3211200700 1 -- 172.21.15.131:0/2414145229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320c1a62d0 con 0x7f320c101100 2024-05-08T22:07:31.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.320+0000 7f3202a00700 1 -- 172.21.15.131:0/2414145229 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0054f0 con 0x7f320c101100 2024-05-08T22:07:31.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.320+0000 7f3211200700 1 -- 172.21.15.131:0/2414145229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320c1a6760 con 0x7f320c101100 2024-05-08T22:07:31.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.321+0000 7f3211200700 1 -- 172.21.15.131:0/2414145229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f320c1025d0 con 0x7f320c101100 2024-05-08T22:07:31.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.323+0000 7f3202a00700 1 -- 172.21.15.131:0/2414145229 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7f31fc004750 con 0x7f320c101100 2024-05-08T22:07:31.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.324+0000 7f3202a00700 1 --2- 172.21.15.131:0/2414145229 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f31f00a13b0 0x7f31f00a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:07:31.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.325+0000 7f3202a00700 1 -- 172.21.15.131:0/2414145229 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f31fc014070 con 0x7f320c101100 2024-05-08T22:07:31.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.325+0000 7f3209400700 1 --2- 172.21.15.131:0/2414145229 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f31f00a13b0 0x7f31f00a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:07:31.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.325+0000 7f3209400700 1 --2- 172.21.15.131:0/2414145229 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f31f00a13b0 0x7f31f00a3870 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f320c19ac30 tx=0x7f31f8009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:07:31.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.331+0000 7f3202a00700 1 -- 172.21.15.131:0/2414145229 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f31fc0963b0 con 0x7f320c101100 2024-05-08T22:07:31.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:31 smithi131 ceph-mon[37260]: from='client.25640 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:07:31.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:31 smithi157 ceph-mon[77746]: from='client.25640 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:07:31.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.567+0000 7f3211200700 1 -- 172.21.15.131:0/2414145229 --> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f320c101ab0 con 0x7f31f00a13b0 2024-05-08T22:07:31.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.575+0000 7f3202a00700 1 -- 172.21.15.131:0/2414145229 <== mgr.15366 v2:172.21.15.131:6800/1157588501 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f320c101ab0 con 0x7f31f00a13b0 2024-05-08T22:07:31.576 INFO:teuthology.orchestra.run.smithi131.stdout:/ 1m 6m3h 2024-05-08T22:07:31.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.577+0000 7f3211200700 1 -- 172.21.15.131:0/2414145229 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f31f00a13b0 msgr2=0x7f31f00a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:31.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.577+0000 7f3211200700 1 --2- 172.21.15.131:0/2414145229 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f31f00a13b0 0x7f31f00a3870 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f320c19ac30 tx=0x7f31f8009260 comp rx=0 tx=0).stop 2024-05-08T22:07:31.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.577+0000 7f3211200700 1 -- 172.21.15.131:0/2414145229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f320c101100 msgr2=0x7f320c199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:07:31.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.577+0000 7f3211200700 1 --2- 172.21.15.131:0/2414145229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f320c101100 0x7f320c199730 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00e9c0 tx=0x7f31fc00ecd0 comp rx=0 tx=0).stop 2024-05-08T22:07:31.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.577+0000 7f3211200700 1 -- 172.21.15.131:0/2414145229 shutdown_connections 2024-05-08T22:07:31.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.577+0000 7f3211200700 1 --2- 172.21.15.131:0/2414145229 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f31f00a13b0 0x7f31f00a3870 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:31.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.577+0000 7f3211200700 1 --2- 172.21.15.131:0/2414145229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f320c19dfc0 0x7f320c19e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:31.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.577+0000 7f3211200700 1 --2- 172.21.15.131:0/2414145229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f320c101100 0x7f320c199730 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:31.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.577+0000 7f3211200700 1 --2- 172.21.15.131:0/2414145229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f320c106220 0x7f320c199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:07:31.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.577+0000 7f3211200700 1 -- 172.21.15.131:0/2414145229 >> 172.21.15.131:0/2414145229 conn(0x7f320c0fc9a0 msgr2=0x7f320c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:07:31.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.577+0000 7f3211200700 1 -- 172.21.15.131:0/2414145229 shutdown_connections 2024-05-08T22:07:31.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:07:31.577+0000 7f3211200700 1 -- 172.21.15.131:0/2414145229 wait complete. 2024-05-08T22:07:31.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:31 smithi132 ceph-mon[78180]: from='client.25640 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:07:31.597 DEBUG:teuthology.orchestra.run.smithi131:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-08T22:07:31.665 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-08T22:07:31.676 INFO:teuthology.run_tasks:Running task workunit... 2024-05-08T22:07:31.691 INFO:tasks.workunit:Pulling workunits from ref d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:07:31.691 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-08T22:07:31.691 INFO:tasks.workunit:timeout=3h 2024-05-08T22:07:31.691 INFO:tasks.workunit:cleanup=True 2024-05-08T22:07:31.691 DEBUG:teuthology.orchestra.run.smithi131:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-08T22:07:31.734 INFO:teuthology.orchestra.run.smithi131.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-08T22:07:31.734 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-08T22:07:31.734 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 7bh/123d Inode: 1099511628283 Links: 2 2024-05-08T22:07:31.734 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:07:31.735 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:cephfs_t:s0 2024-05-08T22:07:31.735 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-05-08 22:05:41.461243124 +0000 2024-05-08T22:07:31.735 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-05-08 22:05:41.461243124 +0000 2024-05-08T22:07:31.735 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-05-08 22:07:17.231849969 +0000 2024-05-08T22:07:31.735 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-05-08 22:05:41.461243124 +0000 2024-05-08T22:07:31.735 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-08T22:07:31.735 DEBUG:teuthology.orchestra.run.smithi131:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-08T22:07:31.853 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:07:31.898 INFO:tasks.workunit.client.0.smithi131.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-08T22:07:32.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:32 smithi157 ceph-mon[77746]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:07:32.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:32 smithi157 ceph-mon[77746]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 682 B/s wr, 16 op/s 2024-05-08T22:07:32.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:32 smithi157 ceph-mon[77746]: from='client.15609 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:07:32.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:32 smithi132 ceph-mon[78180]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:07:32.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:32 smithi132 ceph-mon[78180]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 682 B/s wr, 16 op/s 2024-05-08T22:07:32.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:32 smithi132 ceph-mon[78180]: from='client.15609 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:07:32.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:32 smithi131 ceph-mon[37260]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:07:32.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:32 smithi131 ceph-mon[37260]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 682 B/s wr, 16 op/s 2024-05-08T22:07:32.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:32 smithi131 ceph-mon[37260]: from='client.15609 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:07:34.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:34 smithi157 ceph-mon[77746]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 682 B/s wr, 16 op/s 2024-05-08T22:07:34.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:34 smithi132 ceph-mon[78180]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 682 B/s wr, 16 op/s 2024-05-08T22:07:34.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:34 smithi131 ceph-mon[37260]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 682 B/s wr, 16 op/s 2024-05-08T22:07:36.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:36 smithi157 ceph-mon[77746]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 50 op/s 2024-05-08T22:07:36.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:36 smithi132 ceph-mon[78180]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 50 op/s 2024-05-08T22:07:36.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:36 smithi131 ceph-mon[37260]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 50 op/s 2024-05-08T22:07:38.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:38 smithi157 ceph-mon[77746]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-05-08T22:07:38.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:38 smithi132 ceph-mon[78180]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-05-08T22:07:38.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:38 smithi131 ceph-mon[37260]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-05-08T22:07:40.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:40 smithi157 ceph-mon[77746]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-05-08T22:07:40.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:40 smithi132 ceph-mon[78180]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-05-08T22:07:40.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:40 smithi131 ceph-mon[37260]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-05-08T22:07:42.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:42 smithi157 ceph-mon[77746]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-05-08T22:07:42.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:42 smithi132 ceph-mon[78180]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-05-08T22:07:42.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:42 smithi131 ceph-mon[37260]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-05-08T22:07:43.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:43 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:07:43.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:43 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:07:43.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:43 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:07:44.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:44 smithi157 ceph-mon[77746]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-05-08T22:07:44.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:44 smithi132 ceph-mon[78180]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-05-08T22:07:44.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:44 smithi131 ceph-mon[37260]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-05-08T22:07:46.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:46 smithi157 ceph-mon[77746]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-05-08T22:07:46.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:46 smithi132 ceph-mon[78180]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-05-08T22:07:46.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:46 smithi131 ceph-mon[37260]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-05-08T22:07:48.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:48 smithi157 ceph-mon[77746]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:07:48.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:48 smithi132 ceph-mon[78180]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:07:48.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:48 smithi131 ceph-mon[37260]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:07:50.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:50 smithi157 ceph-mon[77746]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:07:50.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:50 smithi132 ceph-mon[78180]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:07:50.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:50 smithi131 ceph-mon[37260]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:07:52.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:52 smithi157 ceph-mon[77746]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:07:52.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:52 smithi132 ceph-mon[78180]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:07:52.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:52 smithi131 ceph-mon[37260]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:07:54.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:54 smithi157 ceph-mon[77746]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:07:54.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:54 smithi132 ceph-mon[78180]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:07:54.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:54 smithi131 ceph-mon[37260]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:07:56.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:56 smithi157 ceph-mon[77746]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:07:56.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:56 smithi132 ceph-mon[78180]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:07:56.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:56 smithi131 ceph-mon[37260]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:07:58.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:07:58 smithi157 ceph-mon[77746]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:07:58.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:07:58 smithi132 ceph-mon[78180]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:07:58.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:07:58 smithi131 ceph-mon[37260]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:08:00.077 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:08:00.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.304+0000 7fc464200700 1 -- 172.21.15.131:0/3581131691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc45c101b30 msgr2=0x7fc45c105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:00.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.304+0000 7fc464200700 1 --2- 172.21.15.131:0/3581131691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc45c101b30 0x7fc45c105b70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc448009990 tx=0x7fc448009ca0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.304+0000 7fc464200700 1 -- 172.21.15.131:0/3581131691 shutdown_connections 2024-05-08T22:08:00.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.304+0000 7fc464200700 1 --2- 172.21.15.131:0/3581131691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc45c1062a0 0x7fc45c10de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.304+0000 7fc464200700 1 --2- 172.21.15.131:0/3581131691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc45c101180 0x7fc45c101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.304+0000 7fc464200700 1 --2- 172.21.15.131:0/3581131691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc45c101b30 0x7fc45c105b70 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.304+0000 7fc464200700 1 -- 172.21.15.131:0/3581131691 >> 172.21.15.131:0/3581131691 conn(0x7fc45c0fc9e0 msgr2=0x7fc45c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:00.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.304+0000 7fc464200700 1 -- 172.21.15.131:0/3581131691 shutdown_connections 2024-05-08T22:08:00.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.304+0000 7fc464200700 1 -- 172.21.15.131:0/3581131691 wait complete. 2024-05-08T22:08:00.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.306+0000 7fc464200700 1 Processor -- start 2024-05-08T22:08:00.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.306+0000 7fc464200700 1 -- start start 2024-05-08T22:08:00.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.306+0000 7fc464200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc45c101180 0x7fc45c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:00.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.306+0000 7fc464200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc45c1062a0 0x7fc45c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:00.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.306+0000 7fc464200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc45c073d20 0x7fc45c1a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:00.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.306+0000 7fc464200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c1a5be0 con 0x7fc45c1062a0 2024-05-08T22:08:00.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.306+0000 7fc464200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c1a5d50 con 0x7fc45c101180 2024-05-08T22:08:00.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.306+0000 7fc464200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c1a5f10 con 0x7fc45c073d20 2024-05-08T22:08:00.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.307+0000 7fc460e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc45c1062a0 0x7fc45c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:00.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.307+0000 7fc461800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc45c101180 0x7fc45c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:00.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.307+0000 7fc460e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc45c1062a0 0x7fc45c0737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35092/0 (socket says 172.21.15.131:35092) 2024-05-08T22:08:00.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.307+0000 7fc460e00700 1 -- 172.21.15.131:0/4057113366 learned_addr learned my addr 172.21.15.131:0/4057113366 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:00.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.307+0000 7fc462200700 1 --2- 172.21.15.131:0/4057113366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc45c073d20 0x7fc45c1a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:00.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.307+0000 7fc460e00700 1 -- 172.21.15.131:0/4057113366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc45c073d20 msgr2=0x7fc45c1a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:00.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.307+0000 7fc460e00700 1 --2- 172.21.15.131:0/4057113366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc45c073d20 0x7fc45c1a5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.307+0000 7fc460e00700 1 -- 172.21.15.131:0/4057113366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc45c101180 msgr2=0x7fc45c0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:00.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.307+0000 7fc460e00700 1 --2- 172.21.15.131:0/4057113366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc45c101180 0x7fc45c0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.307+0000 7fc460e00700 1 -- 172.21.15.131:0/4057113366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4480095f0 con 0x7fc45c1062a0 2024-05-08T22:08:00.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.307+0000 7fc461800700 1 --2- 172.21.15.131:0/4057113366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc45c101180 0x7fc45c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:00.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.307+0000 7fc460e00700 1 --2- 172.21.15.131:0/4057113366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc45c1062a0 0x7fc45c0737e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc448000c00 tx=0x7fc448005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:00.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.308+0000 7fc45a000700 1 -- 172.21.15.131:0/4057113366 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44801d070 con 0x7fc45c1062a0 2024-05-08T22:08:00.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.308+0000 7fc45a000700 1 -- 172.21.15.131:0/4057113366 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc448022470 con 0x7fc45c1062a0 2024-05-08T22:08:00.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.308+0000 7fc464200700 1 -- 172.21.15.131:0/4057113366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc45c1a6190 con 0x7fc45c1062a0 2024-05-08T22:08:00.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.308+0000 7fc45a000700 1 -- 172.21.15.131:0/4057113366 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc448017650 con 0x7fc45c1062a0 2024-05-08T22:08:00.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.308+0000 7fc464200700 1 -- 172.21.15.131:0/4057113366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc45c1a6610 con 0x7fc45c1062a0 2024-05-08T22:08:00.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.311+0000 7fc45a000700 1 -- 172.21.15.131:0/4057113366 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7fc44800fad0 con 0x7fc45c1062a0 2024-05-08T22:08:00.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.312+0000 7fc477cacb80 1 Processor -- start 2024-05-08T22:08:00.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.312+0000 7fc477cacb80 1 -- start start 2024-05-08T22:08:00.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.312+0000 7fc477cacb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55826930be00 0x55826930c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:00.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.312+0000 7fc477cacb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55826930c720 0x558269306350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:00.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.312+0000 7fc477cacb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558269306890 0x558269306cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:00.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.312+0000 7fc477cacb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55826925d150 con 0x558269306890 2024-05-08T22:08:00.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.312+0000 7fc477cacb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55826925ab10 con 0x55826930c720 2024-05-08T22:08:00.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.312+0000 7fc45a000700 1 --2- 172.21.15.131:0/4057113366 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc4440a1430 0x7fc4440a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:00.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.312+0000 7fc477cacb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558269256400 con 0x55826930be00 2024-05-08T22:08:00.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.313+0000 7fc461800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55826930c720 0x558269306350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:00.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.313+0000 7fc462200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55826930be00 0x55826930c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:00.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.313+0000 7fc461800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55826930c720 0x558269306350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:35028/0 (socket says 172.21.15.131:35028) 2024-05-08T22:08:00.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.313+0000 7fc462200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55826930be00 0x55826930c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46598/0 (socket says 172.21.15.131:46598) 2024-05-08T22:08:00.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.313+0000 7fc461800700 1 -- 172.21.15.131:0/3154787782 learned_addr learned my addr 172.21.15.131:0/3154787782 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:00.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.313+0000 7fc45a000700 1 -- 172.21.15.131:0/4057113366 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fc4480ce6d0 con 0x7fc45c1062a0 2024-05-08T22:08:00.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.313+0000 7fc462200700 1 --2- 172.21.15.131:0/4057113366 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc4440a1430 0x7fc4440a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:00.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.313+0000 7fc460e00700 1 --2- 172.21.15.131:0/3154787782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558269306890 0x558269306cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:00.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.313+0000 7fc461800700 1 -- 172.21.15.131:0/3154787782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55826930be00 msgr2=0x55826930c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:00.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.313+0000 7fc461800700 1 --2- 172.21.15.131:0/3154787782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55826930be00 0x55826930c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.313+0000 7fc461800700 1 -- 172.21.15.131:0/3154787782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558269306890 msgr2=0x558269306cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:00.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.313+0000 7fc461800700 1 --2- 172.21.15.131:0/3154787782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558269306890 0x558269306cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.313+0000 7fc461800700 1 -- 172.21.15.131:0/3154787782 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582690d73b0 con 0x55826930c720 2024-05-08T22:08:00.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.313+0000 7fc461800700 1 --2- 172.21.15.131:0/3154787782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55826930c720 0x558269306350 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc4500115c0 tx=0x7fc4500118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d821d99fcc4b936 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:00.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.313+0000 7fc462200700 1 --2- 172.21.15.131:0/4057113366 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc4440a1430 0x7fc4440a38f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc454009ec0 tx=0x7fc45400b040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:00.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.314+0000 7fc44ea00700 1 -- 172.21.15.131:0/3154787782 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc450019630 con 0x55826930c720 2024-05-08T22:08:00.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.314+0000 7fc44ea00700 1 -- 172.21.15.131:0/3154787782 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc450019c50 con 0x55826930c720 2024-05-08T22:08:00.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.314+0000 7fc44ea00700 1 -- 172.21.15.131:0/3154787782 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45000ec70 con 0x55826930c720 2024-05-08T22:08:00.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.314+0000 7fc477cacb80 1 -- 172.21.15.131:0/3154787782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55826930c720 msgr2=0x558269306350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:00.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.314+0000 7fc477cacb80 1 --2- 172.21.15.131:0/3154787782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55826930c720 0x558269306350 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc4500115c0 tx=0x7fc4500118d0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.314+0000 7fc477cacb80 1 -- 172.21.15.131:0/3154787782 shutdown_connections 2024-05-08T22:08:00.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.314+0000 7fc477cacb80 1 --2- 172.21.15.131:0/3154787782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558269306890 0x558269306cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.314+0000 7fc477cacb80 1 --2- 172.21.15.131:0/3154787782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55826930be00 0x55826930c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.314+0000 7fc477cacb80 1 --2- 172.21.15.131:0/3154787782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55826930c720 0x558269306350 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.314+0000 7fc477cacb80 1 -- 172.21.15.131:0/3154787782 >> 172.21.15.131:0/3154787782 conn(0x558269264210 msgr2=0x558269300df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:00.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.314+0000 7fc477cacb80 1 -- 172.21.15.131:0/3154787782 shutdown_connections 2024-05-08T22:08:00.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.314+0000 7fc477cacb80 1 -- 172.21.15.131:0/3154787782 wait complete. 2024-05-08T22:08:00.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.315+0000 7fc477cacb80 1 -- start start 2024-05-08T22:08:00.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.315+0000 7fc477cacb80 1 Processor -- start 2024-05-08T22:08:00.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.316+0000 7fc477cacb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558269306890 0x55826930fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:00.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.316+0000 7fc477cacb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55826930be00 0x5582693100c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:00.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.316+0000 7fc477cacb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55826930c720 0x558269310600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:00.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.316+0000 7fc477cacb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582691f2560 con 0x558269306890 2024-05-08T22:08:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.316+0000 7fc477cacb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582691f2200 con 0x55826930c720 2024-05-08T22:08:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.316+0000 7fc477cacb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582690d96a0 con 0x55826930be00 2024-05-08T22:08:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.316+0000 7fc477cacb80 10 client.0 _ll_register_callbacks cb 0x5582691fbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.316+0000 7fc461800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558269306890 0x55826930fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.316+0000 7fc461800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558269306890 0x55826930fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35110/0 (socket says 172.21.15.131:35110) 2024-05-08T22:08:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.316+0000 7fc461800700 1 -- 172.21.15.131:0/399060425 learned_addr learned my addr 172.21.15.131:0/399060425 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.316+0000 7fc462200700 1 --2- 172.21.15.131:0/399060425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55826930c720 0x558269310600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.317+0000 7fc460e00700 1 --2- 172.21.15.131:0/399060425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55826930be00 0x5582693100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.317+0000 7fc461800700 1 -- 172.21.15.131:0/399060425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55826930be00 msgr2=0x5582693100c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.317+0000 7fc461800700 1 --2- 172.21.15.131:0/399060425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55826930be00 0x5582693100c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.317+0000 7fc461800700 1 -- 172.21.15.131:0/399060425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55826930c720 msgr2=0x558269310600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.317+0000 7fc461800700 1 --2- 172.21.15.131:0/399060425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55826930c720 0x558269310600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.317+0000 7fc461800700 1 -- 172.21.15.131:0/399060425 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc450009dd0 con 0x558269306890 2024-05-08T22:08:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.317+0000 7fc462200700 1 --2- 172.21.15.131:0/399060425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55826930c720 0x558269310600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.317+0000 7fc461800700 1 --2- 172.21.15.131:0/399060425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558269306890 0x55826930fb80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc450004690 tx=0x7fc45000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58c4c77a5b2fea3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:00.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.317+0000 7fc443400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:08:00.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.317+0000 7fc443400700 1 -- 172.21.15.131:0/399060425 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc450019630 con 0x558269306890 2024-05-08T22:08:00.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.317+0000 7fc443400700 1 -- 172.21.15.131:0/399060425 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fc450005ce0 con 0x558269306890 2024-05-08T22:08:00.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.317+0000 7fc477cacb80 1 -- 172.21.15.131:0/399060425 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5582691af270 con 0x558269306890 2024-05-08T22:08:00.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.318+0000 7fc443400700 1 -- 172.21.15.131:0/399060425 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc45000e610 con 0x558269306890 2024-05-08T22:08:00.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.318+0000 7fc477cacb80 10 client.15621 fetch_fsmap learned FSMap version 24 2024-05-08T22:08:00.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.318+0000 7fc477cacb80 1 -- 172.21.15.131:0/399060425 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558269083b60 con 0x558269306890 2024-05-08T22:08:00.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.318+0000 7fc443400700 1 -- 172.21.15.131:0/399060425 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5582691af270 con 0x558269306890 2024-05-08T22:08:00.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.318+0000 7fc443400700 1 -- 172.21.15.131:0/399060425 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc450004870 con 0x558269306890 2024-05-08T22:08:00.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.318+0000 7fc477cacb80 10 client.15621 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:08:00.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.318+0000 7fc477cacb80 10 client.15621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:08:00.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.318+0000 7fc477cacb80 20 client.15621 populate_metadata read hostname 'smithi131' 2024-05-08T22:08:00.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.318+0000 7fc477cacb80 1 --2- 172.21.15.131:0/399060425 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5582693201b0 0x5582693225a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:00.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.318+0000 7fc477cacb80 4 client.15621 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:08:00.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.318+0000 7fc477cacb80 1 -- 172.21.15.131:0/399060425 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582692d5810 con 0x5582693201b0 2024-05-08T22:08:00.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.319+0000 7fc460e00700 1 --2- 172.21.15.131:0/399060425 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5582693201b0 0x5582693225a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:00.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.319+0000 7fc443400700 10 client.15621 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:00.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.319+0000 7fc460e00700 1 --2- 172.21.15.131:0/399060425 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5582693201b0 0x5582693225a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dedaf6b6778e2956 server_cookie=6e39852acbc738bd in_seq=0 out_seq=0 2024-05-08T22:08:00.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.320+0000 7fc443400700 1 -- 172.21.15.131:0/399060425 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5582692d5810 con 0x5582693201b0 2024-05-08T22:08:00.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.320+0000 7fc443400700 10 client.15621 handle_command_reply: tid=0 2024-05-08T22:08:00.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.320+0000 7fc477cacb80 1 client.15621 shutdown 2024-05-08T22:08:00.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.320+0000 7fc477cacb80 1 -- 172.21.15.131:0/399060425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558269306890 msgr2=0x55826930fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:00.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.320+0000 7fc477cacb80 1 --2- 172.21.15.131:0/399060425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558269306890 0x55826930fb80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc450004690 tx=0x7fc45000eef0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.320+0000 7fc477cacb80 1 -- 172.21.15.131:0/399060425 shutdown_connections 2024-05-08T22:08:00.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.321+0000 7fc477cacb80 1 --2- 172.21.15.131:0/399060425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55826930c720 0x558269310600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.321+0000 7fc477cacb80 1 --2- 172.21.15.131:0/399060425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558269306890 0x55826930fb80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.321+0000 7fc477cacb80 1 --2- 172.21.15.131:0/399060425 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5582693201b0 0x5582693225a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.321+0000 7fc477cacb80 1 --2- 172.21.15.131:0/399060425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55826930be00 0x5582693100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.321+0000 7fc443400700 0 client.15621 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.321+0000 7fc477cacb80 1 -- 172.21.15.131:0/399060425 >> 172.21.15.131:0/399060425 conn(0x558269264210 msgr2=0x5582693005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.321+0000 7fc477cacb80 1 -- 172.21.15.131:0/399060425 shutdown_connections 2024-05-08T22:08:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.321+0000 7fc477cacb80 1 -- 172.21.15.131:0/399060425 wait complete. 2024-05-08T22:08:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.321+0000 7fc477cacb80 20 client.15621 trim_cache size 0 max 16384 2024-05-08T22:08:00.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.338+0000 7fc477cacb80 1 Processor -- start 2024-05-08T22:08:00.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.338+0000 7fc477cacb80 1 -- start start 2024-05-08T22:08:00.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.339+0000 7fc477cacb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558269306890 0x558269306350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:00.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.339+0000 7fc477cacb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55826930be00 0x558269300d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:00.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.339+0000 7fc477cacb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55826930c720 0x5582693100d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:00.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.339+0000 7fc477cacb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582692fb840 con 0x558269306890 2024-05-08T22:08:00.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.339+0000 7fc477cacb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582692fddc0 con 0x55826930be00 2024-05-08T22:08:00.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.339+0000 7fc477cacb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558269302aa0 con 0x55826930c720 2024-05-08T22:08:00.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.339+0000 7fc461800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558269306890 0x558269306350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:00.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.339+0000 7fc461800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558269306890 0x558269306350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35124/0 (socket says 172.21.15.131:35124) 2024-05-08T22:08:00.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.339+0000 7fc461800700 1 -- 172.21.15.131:0/2688234149 learned_addr learned my addr 172.21.15.131:0/2688234149 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:00.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.339+0000 7fc460e00700 1 --2- 172.21.15.131:0/2688234149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55826930be00 0x558269300d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:00.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.339+0000 7fc462200700 1 --2- 172.21.15.131:0/2688234149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55826930c720 0x5582693100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:00.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.339+0000 7fc461800700 1 -- 172.21.15.131:0/2688234149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55826930c720 msgr2=0x5582693100d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:00.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.339+0000 7fc461800700 1 --2- 172.21.15.131:0/2688234149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55826930c720 0x5582693100d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.339+0000 7fc461800700 1 -- 172.21.15.131:0/2688234149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55826930be00 msgr2=0x558269300d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:00.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.339+0000 7fc461800700 1 --2- 172.21.15.131:0/2688234149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55826930be00 0x558269300d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.339+0000 7fc461800700 1 -- 172.21.15.131:0/2688234149 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55826921c560 con 0x558269306890 2024-05-08T22:08:00.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.339+0000 7fc461800700 1 --2- 172.21.15.131:0/2688234149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558269306890 0x558269306350 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc450011510 tx=0x7fc45000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ef2f49905a1833 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:00.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.340+0000 7fc44f400700 1 -- 172.21.15.131:0/2688234149 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc450011cd0 con 0x558269306890 2024-05-08T22:08:00.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.340+0000 7fc44f400700 1 -- 172.21.15.131:0/2688234149 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc450005c80 con 0x558269306890 2024-05-08T22:08:00.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.340+0000 7fc44f400700 1 -- 172.21.15.131:0/2688234149 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4500051a0 con 0x558269306890 2024-05-08T22:08:00.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.340+0000 7fc477cacb80 1 -- 172.21.15.131:0/2688234149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558269306890 msgr2=0x558269306350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:00.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.340+0000 7fc477cacb80 1 --2- 172.21.15.131:0/2688234149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558269306890 0x558269306350 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc450011510 tx=0x7fc45000ef60 comp rx=0 tx=0).stop 2024-05-08T22:08:00.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.340+0000 7fc477cacb80 1 -- 172.21.15.131:0/2688234149 shutdown_connections 2024-05-08T22:08:00.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.340+0000 7fc477cacb80 1 --2- 172.21.15.131:0/2688234149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55826930c720 0x5582693100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.340+0000 7fc477cacb80 1 --2- 172.21.15.131:0/2688234149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558269306890 0x558269306350 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.340+0000 7fc477cacb80 1 --2- 172.21.15.131:0/2688234149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55826930be00 0x558269300d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.340+0000 7fc477cacb80 1 -- 172.21.15.131:0/2688234149 >> 172.21.15.131:0/2688234149 conn(0x558269264210 msgr2=0x55826930e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:00.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.340+0000 7fc477cacb80 1 -- 172.21.15.131:0/2688234149 shutdown_connections 2024-05-08T22:08:00.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.340+0000 7fc477cacb80 1 -- 172.21.15.131:0/2688234149 wait complete. 2024-05-08T22:08:00.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.341+0000 7fc477cacb80 1 -- start start 2024-05-08T22:08:00.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.341+0000 7fc477cacb80 1 Processor -- start 2024-05-08T22:08:00.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.341+0000 7fc477cacb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558269306890 0x558269315150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:00.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.341+0000 7fc477cacb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55826930be00 0x558269315690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:00.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.341+0000 7fc477cacb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826930c720 0x558269315bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:00.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.341+0000 7fc477cacb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558269375680 con 0x55826930c720 2024-05-08T22:08:00.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.341+0000 7fc477cacb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582693757c0 con 0x558269306890 2024-05-08T22:08:00.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.341+0000 7fc477cacb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558269375900 con 0x55826930be00 2024-05-08T22:08:00.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.341+0000 7fc477cacb80 10 client.0 _ll_register_callbacks cb 0x5582691fbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:00.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.342+0000 7fc462200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826930c720 0x558269315bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:00.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.342+0000 7fc462200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826930c720 0x558269315bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35130/0 (socket says 172.21.15.131:35130) 2024-05-08T22:08:00.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.342+0000 7fc462200700 1 -- 172.21.15.131:0/1577933916 learned_addr learned my addr 172.21.15.131:0/1577933916 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:00.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.342+0000 7fc461800700 1 --2- 172.21.15.131:0/1577933916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558269306890 0x558269315150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:00.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.342+0000 7fc460e00700 1 --2- 172.21.15.131:0/1577933916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55826930be00 0x558269315690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:00.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.342+0000 7fc462200700 1 -- 172.21.15.131:0/1577933916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55826930be00 msgr2=0x558269315690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:00.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.342+0000 7fc462200700 1 --2- 172.21.15.131:0/1577933916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55826930be00 0x558269315690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.342+0000 7fc462200700 1 -- 172.21.15.131:0/1577933916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558269306890 msgr2=0x558269315150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:00.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.342+0000 7fc462200700 1 --2- 172.21.15.131:0/1577933916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558269306890 0x558269315150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.342+0000 7fc462200700 1 -- 172.21.15.131:0/1577933916 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558269083b60 con 0x55826930c720 2024-05-08T22:08:00.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.342+0000 7fc461800700 1 --2- 172.21.15.131:0/1577933916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558269306890 0x558269315150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:00.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.342+0000 7fc462200700 1 --2- 172.21.15.131:0/1577933916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826930c720 0x558269315bd0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc454008510 tx=0x7fc454004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6de183722be683 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:00.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.342+0000 7fc443400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:08:00.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.342+0000 7fc443400700 1 -- 172.21.15.131:0/1577933916 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc454021040 con 0x55826930c720 2024-05-08T22:08:00.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.343+0000 7fc443400700 1 -- 172.21.15.131:0/1577933916 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fc45401ae40 con 0x55826930c720 2024-05-08T22:08:00.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.343+0000 7fc477cacb80 1 -- 172.21.15.131:0/1577933916 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc450019c50 con 0x55826930c720 2024-05-08T22:08:00.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.343+0000 7fc443400700 1 -- 172.21.15.131:0/1577933916 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc45400e070 con 0x55826930c720 2024-05-08T22:08:00.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.343+0000 7fc477cacb80 10 client.15627 fetch_fsmap learned FSMap version 24 2024-05-08T22:08:00.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.343+0000 7fc477cacb80 1 -- 172.21.15.131:0/1577933916 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5582690dbcd0 con 0x55826930c720 2024-05-08T22:08:00.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.343+0000 7fc443400700 1 -- 172.21.15.131:0/1577933916 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc450019c50 con 0x55826930c720 2024-05-08T22:08:00.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.343+0000 7fc443400700 1 -- 172.21.15.131:0/1577933916 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc454028b40 con 0x55826930c720 2024-05-08T22:08:00.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.343+0000 7fc477cacb80 10 client.15627 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:08:00.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.343+0000 7fc477cacb80 10 client.15627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:08:00.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.343+0000 7fc477cacb80 20 client.15627 populate_metadata read hostname 'smithi131' 2024-05-08T22:08:00.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.343+0000 7fc477cacb80 1 --2- 172.21.15.131:0/1577933916 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5582693770f0 0x558269379510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:00.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.343+0000 7fc477cacb80 4 client.15627 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:08:00.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.343+0000 7fc477cacb80 1 -- 172.21.15.131:0/1577933916 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55826924eec0 con 0x5582693770f0 2024-05-08T22:08:00.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.344+0000 7fc461800700 1 --2- 172.21.15.131:0/1577933916 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5582693770f0 0x558269379510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:00.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.344+0000 7fc461800700 1 --2- 172.21.15.131:0/1577933916 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5582693770f0 0x558269379510 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebf6106a9f8b0275 server_cookie=2b0c4487b89dd51f in_seq=0 out_seq=0 2024-05-08T22:08:00.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.344+0000 7fc443400700 10 client.15627 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:00.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.344+0000 7fc443400700 1 -- 172.21.15.131:0/1577933916 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55826924eec0 con 0x5582693770f0 2024-05-08T22:08:00.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.344+0000 7fc443400700 10 client.15627 handle_command_reply: tid=0 2024-05-08T22:08:00.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.344+0000 7fc477cacb80 1 client.15627 shutdown 2024-05-08T22:08:00.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.344+0000 7fc477cacb80 1 -- 172.21.15.131:0/1577933916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826930c720 msgr2=0x558269315bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:00.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.344+0000 7fc477cacb80 1 --2- 172.21.15.131:0/1577933916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826930c720 0x558269315bd0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc454008510 tx=0x7fc454004b40 comp rx=0 tx=0).stop 2024-05-08T22:08:00.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.345+0000 7fc477cacb80 1 -- 172.21.15.131:0/1577933916 shutdown_connections 2024-05-08T22:08:00.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.345+0000 7fc477cacb80 1 --2- 172.21.15.131:0/1577933916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826930c720 0x558269315bd0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.345+0000 7fc477cacb80 1 --2- 172.21.15.131:0/1577933916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558269306890 0x558269315150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.345+0000 7fc477cacb80 1 --2- 172.21.15.131:0/1577933916 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5582693770f0 0x558269379510 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.345+0000 7fc477cacb80 1 --2- 172.21.15.131:0/1577933916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55826930be00 0x558269315690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.345+0000 7fc443400700 0 client.15627 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:00.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.345+0000 7fc477cacb80 1 -- 172.21.15.131:0/1577933916 >> 172.21.15.131:0/1577933916 conn(0x558269264210 msgr2=0x55826930b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:00.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.345+0000 7fc477cacb80 1 -- 172.21.15.131:0/1577933916 shutdown_connections 2024-05-08T22:08:00.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.345+0000 7fc477cacb80 1 -- 172.21.15.131:0/1577933916 wait complete. 2024-05-08T22:08:00.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.345+0000 7fc477cacb80 20 client.15627 trim_cache size 0 max 16384 2024-05-08T22:08:00.351 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:08:00.351 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-08T22:08:00.351 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-08T22:08:00.351 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:08:00.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.345+0000 7fc464200700 1 -- 172.21.15.131:0/4057113366 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc4440a1430 msgr2=0x7fc4440a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:00.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.345+0000 7fc464200700 1 --2- 172.21.15.131:0/4057113366 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc4440a1430 0x7fc4440a38f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc454009ec0 tx=0x7fc45400b040 comp rx=0 tx=0).stop 2024-05-08T22:08:00.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.345+0000 7fc464200700 1 -- 172.21.15.131:0/4057113366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc45c1062a0 msgr2=0x7fc45c0737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:00.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.345+0000 7fc464200700 1 --2- 172.21.15.131:0/4057113366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc45c1062a0 0x7fc45c0737e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc448000c00 tx=0x7fc448005740 comp rx=0 tx=0).stop 2024-05-08T22:08:00.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.346+0000 7fc464200700 1 -- 172.21.15.131:0/4057113366 shutdown_connections 2024-05-08T22:08:00.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.346+0000 7fc464200700 1 --2- 172.21.15.131:0/4057113366 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc4440a1430 0x7fc4440a38f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.346+0000 7fc464200700 1 --2- 172.21.15.131:0/4057113366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc45c073d20 0x7fc45c1a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.346+0000 7fc464200700 1 --2- 172.21.15.131:0/4057113366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc45c101180 0x7fc45c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.346+0000 7fc464200700 1 --2- 172.21.15.131:0/4057113366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc45c1062a0 0x7fc45c0737e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.346+0000 7fc464200700 1 -- 172.21.15.131:0/4057113366 >> 172.21.15.131:0/4057113366 conn(0x7fc45c0fc9e0 msgr2=0x7fc45c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:00.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.346+0000 7fc464200700 1 -- 172.21.15.131:0/4057113366 shutdown_connections 2024-05-08T22:08:00.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.346+0000 7fc464200700 1 -- 172.21.15.131:0/4057113366 wait complete. 2024-05-08T22:08:00.365 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T22:08:00.365 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:08:00.422 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:00 smithi131 ceph-mon[37260]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:08:00.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:00 smithi157 ceph-mon[77746]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:08:00.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.559+0000 7f45a7200700 1 -- 172.21.15.131:0/4172622578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a01019d0 msgr2=0x7f45a0105a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:00.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.559+0000 7f45a7200700 1 --2- 172.21.15.131:0/4172622578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a01019d0 0x7f45a0105a10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f45900098f0 tx=0x7f4590009c00 comp rx=0 tx=0).stop 2024-05-08T22:08:00.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.559+0000 7f45a7200700 1 -- 172.21.15.131:0/4172622578 shutdown_connections 2024-05-08T22:08:00.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.559+0000 7f45a7200700 1 --2- 172.21.15.131:0/4172622578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45a0106140 0x7f45a010dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.559+0000 7f45a7200700 1 --2- 172.21.15.131:0/4172622578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45a0101020 0x7f45a0101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.559+0000 7f45a7200700 1 --2- 172.21.15.131:0/4172622578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a01019d0 0x7f45a0105a10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.559+0000 7f45a7200700 1 -- 172.21.15.131:0/4172622578 >> 172.21.15.131:0/4172622578 conn(0x7f45a00fc8c0 msgr2=0x7f45a00fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:00.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.559+0000 7f45a7200700 1 -- 172.21.15.131:0/4172622578 shutdown_connections 2024-05-08T22:08:00.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.559+0000 7f45a7200700 1 -- 172.21.15.131:0/4172622578 wait complete. 2024-05-08T22:08:00.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.560+0000 7f45a7200700 1 Processor -- start 2024-05-08T22:08:00.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.560+0000 7f45a7200700 1 -- start start 2024-05-08T22:08:00.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.561+0000 7f45a7200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a0101020 0x7f45a0199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:00.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.561+0000 7f45a7200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45a0106140 0x7f45a0199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:00.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.561+0000 7f45a7200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45a019de50 0x7f45a019e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:00.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.561+0000 7f45a7200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a019e8d0 con 0x7f45a019de50 2024-05-08T22:08:00.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.561+0000 7f45a7200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a019ea40 con 0x7f45a0106140 2024-05-08T22:08:00.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.561+0000 7f45a7200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a019ec00 con 0x7f45a0101020 2024-05-08T22:08:00.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.561+0000 7f45a5e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a0101020 0x7f45a0199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:00.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.561+0000 7f45a6800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45a019de50 0x7f45a019e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:00.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.561+0000 7f45a5e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a0101020 0x7f45a0199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46652/0 (socket says 172.21.15.131:46652) 2024-05-08T22:08:00.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.561+0000 7f45a5e00700 1 -- 172.21.15.131:0/3250505628 learned_addr learned my addr 172.21.15.131:0/3250505628 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:00.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.561+0000 7f45a5400700 1 --2- 172.21.15.131:0/3250505628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45a0106140 0x7f45a0199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:00.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.561+0000 7f45a5e00700 1 -- 172.21.15.131:0/3250505628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45a0106140 msgr2=0x7f45a0199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:00.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.561+0000 7f45a5e00700 1 --2- 172.21.15.131:0/3250505628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45a0106140 0x7f45a0199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.561+0000 7f45a5e00700 1 -- 172.21.15.131:0/3250505628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45a019de50 msgr2=0x7f45a019e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:00.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.561+0000 7f45a5e00700 1 --2- 172.21.15.131:0/3250505628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45a019de50 0x7f45a019e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.561+0000 7f45a5e00700 1 -- 172.21.15.131:0/3250505628 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45940095f0 con 0x7f45a0101020 2024-05-08T22:08:00.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.562+0000 7f45a5e00700 1 --2- 172.21.15.131:0/3250505628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a0101020 0x7f45a0199570 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f459400ec10 tx=0x7f459400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:00.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.562+0000 7f459aa00700 1 -- 172.21.15.131:0/3250505628 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459400cc90 con 0x7f45a0101020 2024-05-08T22:08:00.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.562+0000 7f45a7200700 1 -- 172.21.15.131:0/3250505628 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4590009520 con 0x7f45a0101020 2024-05-08T22:08:00.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.562+0000 7f459aa00700 1 -- 172.21.15.131:0/3250505628 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45940044e0 con 0x7f45a0101020 2024-05-08T22:08:00.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.562+0000 7f459aa00700 1 -- 172.21.15.131:0/3250505628 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4594010430 con 0x7f45a0101020 2024-05-08T22:08:00.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.562+0000 7f45a7200700 1 -- 172.21.15.131:0/3250505628 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a01a64b0 con 0x7f45a0101020 2024-05-08T22:08:00.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.563+0000 7f45a7200700 1 -- 172.21.15.131:0/3250505628 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a01024f0 con 0x7f45a0101020 2024-05-08T22:08:00.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.565+0000 7f459aa00700 1 -- 172.21.15.131:0/3250505628 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7f4594003fb0 con 0x7f45a0101020 2024-05-08T22:08:00.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.566+0000 7f459aa00700 1 --2- 172.21.15.131:0/3250505628 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f45880a13b0 0x7f45880a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:00.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.566+0000 7f459aa00700 1 -- 172.21.15.131:0/3250505628 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f4594014070 con 0x7f45a0101020 2024-05-08T22:08:00.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.566+0000 7f45a5400700 1 --2- 172.21.15.131:0/3250505628 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f45880a13b0 0x7f45880a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:00.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.567+0000 7f45a5400700 1 --2- 172.21.15.131:0/3250505628 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f45880a13b0 0x7f45880a3870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4590005e90 tx=0x7f4590019040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:00.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.569+0000 7f459aa00700 1 -- 172.21.15.131:0/3250505628 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4594096700 con 0x7f45a0101020 2024-05-08T22:08:00.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:00 smithi132 ceph-mon[78180]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:08:00.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.841+0000 7f45a7200700 1 -- 172.21.15.131:0/3250505628 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45a019a870 con 0x7f45a0101020 2024-05-08T22:08:00.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.842+0000 7f459aa00700 1 -- 172.21.15.131:0/3250505628 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10889 (secure 0 0 0) 0x7f4594099d20 con 0x7f45a0101020 2024-05-08T22:08:00.844 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:08:00.844 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:05:14.034908+0000","modified":"2024-05-08T22:07:19.165885+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:08:00.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.845+0000 7f45a7200700 1 -- 172.21.15.131:0/3250505628 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f45880a13b0 msgr2=0x7f45880a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.845+0000 7f45a7200700 1 --2- 172.21.15.131:0/3250505628 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f45880a13b0 0x7f45880a3870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4590005e90 tx=0x7f4590019040 comp rx=0 tx=0).stop 2024-05-08T22:08:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.845+0000 7f45a7200700 1 -- 172.21.15.131:0/3250505628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a0101020 msgr2=0x7f45a0199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.845+0000 7f45a7200700 1 --2- 172.21.15.131:0/3250505628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a0101020 0x7f45a0199570 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f459400ec10 tx=0x7f459400ef20 comp rx=0 tx=0).stop 2024-05-08T22:08:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.845+0000 7f45a7200700 1 -- 172.21.15.131:0/3250505628 shutdown_connections 2024-05-08T22:08:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.845+0000 7f45a7200700 1 --2- 172.21.15.131:0/3250505628 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f45880a13b0 0x7f45880a3870 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.845+0000 7f45a7200700 1 --2- 172.21.15.131:0/3250505628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45a019de50 0x7f45a019e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.845+0000 7f45a7200700 1 --2- 172.21.15.131:0/3250505628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a0101020 0x7f45a0199570 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.845+0000 7f45a7200700 1 --2- 172.21.15.131:0/3250505628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45a0106140 0x7f45a0199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.845+0000 7f45a7200700 1 -- 172.21.15.131:0/3250505628 >> 172.21.15.131:0/3250505628 conn(0x7f45a00fc8c0 msgr2=0x7f45a00fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.845+0000 7f45a7200700 1 -- 172.21.15.131:0/3250505628 shutdown_connections 2024-05-08T22:08:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:00.846+0000 7f45a7200700 1 -- 172.21.15.131:0/3250505628 wait complete. 2024-05-08T22:08:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 24 2024-05-08T22:08:00.863 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:08:00 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:08:00.646+0000 7fec69800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:08:00.863 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:08:00 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:08:00.646+0000 7fec69800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:08:00.863 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:08:00 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:08:00.646+0000 7fec69800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:08:00.864 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:08:00 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:08:00.646+0000 7fec69800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:08:00.864 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:08:00 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:08:00.646+0000 7fec69800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:08:00.866 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T22:08:01.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.061+0000 7f6b8aa00700 1 -- 172.21.15.131:0/100139791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b84101160 msgr2=0x7f6b84101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.061+0000 7f6b8aa00700 1 --2- 172.21.15.131:0/100139791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b84101160 0x7f6b84101540 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009990 tx=0x7f6b74009ca0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.061+0000 7f6b8aa00700 1 -- 172.21.15.131:0/100139791 shutdown_connections 2024-05-08T22:08:01.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.061+0000 7f6b8aa00700 1 --2- 172.21.15.131:0/100139791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b84106280 0x7f6b8410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.061+0000 7f6b8aa00700 1 --2- 172.21.15.131:0/100139791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b84101160 0x7f6b84101540 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.061+0000 7f6b8aa00700 1 --2- 172.21.15.131:0/100139791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b84101b10 0x7f6b84105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.061+0000 7f6b8aa00700 1 -- 172.21.15.131:0/100139791 >> 172.21.15.131:0/100139791 conn(0x7f6b840fc9c0 msgr2=0x7f6b840fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:01.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.061+0000 7f6b8aa00700 1 -- 172.21.15.131:0/100139791 shutdown_connections 2024-05-08T22:08:01.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.061+0000 7f6b8aa00700 1 -- 172.21.15.131:0/100139791 wait complete. 2024-05-08T22:08:01.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.062+0000 7f6b8aa00700 1 Processor -- start 2024-05-08T22:08:01.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.062+0000 7f6b8aa00700 1 -- start start 2024-05-08T22:08:01.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.063+0000 7f6b8aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b84101160 0x7f6b840732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.063+0000 7f6b8aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b84106280 0x7f6b840737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.063+0000 7f6b8aa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b84073d20 0x7f6b841a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.063+0000 7f6b8aa00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b841a5af0 con 0x7f6b84106280 2024-05-08T22:08:01.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.063+0000 7f6b8aa00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b841a5c60 con 0x7f6b84073d20 2024-05-08T22:08:01.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.063+0000 7f6b8aa00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b841a5e20 con 0x7f6b84101160 2024-05-08T22:08:01.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.063+0000 7f6b83e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b84101160 0x7f6b840732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.063+0000 7f6b83e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b84101160 0x7f6b840732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46692/0 (socket says 172.21.15.131:46692) 2024-05-08T22:08:01.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.063+0000 7f6b83e00700 1 -- 172.21.15.131:0/3289703681 learned_addr learned my addr 172.21.15.131:0/3289703681 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:01.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.063+0000 7f6b83400700 1 --2- 172.21.15.131:0/3289703681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b84106280 0x7f6b840737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.063+0000 7f6b88a00700 1 --2- 172.21.15.131:0/3289703681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b84073d20 0x7f6b841a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.063+0000 7f6b83e00700 1 -- 172.21.15.131:0/3289703681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b84073d20 msgr2=0x7f6b841a5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.063+0000 7f6b83e00700 1 --2- 172.21.15.131:0/3289703681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b84073d20 0x7f6b841a5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.063+0000 7f6b83e00700 1 -- 172.21.15.131:0/3289703681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b84106280 msgr2=0x7f6b840737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.063+0000 7f6b83e00700 1 --2- 172.21.15.131:0/3289703681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b84106280 0x7f6b840737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.063+0000 7f6b83e00700 1 -- 172.21.15.131:0/3289703681 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b740095f0 con 0x7f6b84101160 2024-05-08T22:08:01.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.063+0000 7f6b83400700 1 --2- 172.21.15.131:0/3289703681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b84106280 0x7f6b840737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:01.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.064+0000 7f6b83e00700 1 --2- 172.21.15.131:0/3289703681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b84101160 0x7f6b840732a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6b74004760 tx=0x7f6b74004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:01.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.064+0000 7f6b80c00700 1 -- 172.21.15.131:0/3289703681 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7401d070 con 0x7f6b84101160 2024-05-08T22:08:01.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.064+0000 7f6b80c00700 1 -- 172.21.15.131:0/3289703681 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b74022470 con 0x7f6b84101160 2024-05-08T22:08:01.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.064+0000 7f6b8aa00700 1 -- 172.21.15.131:0/3289703681 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b841a60a0 con 0x7f6b84101160 2024-05-08T22:08:01.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.064+0000 7f6b80c00700 1 -- 172.21.15.131:0/3289703681 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b74017650 con 0x7f6b84101160 2024-05-08T22:08:01.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.064+0000 7f6b8aa00700 1 -- 172.21.15.131:0/3289703681 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b841a6520 con 0x7f6b84101160 2024-05-08T22:08:01.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.068+0000 7f6b80c00700 1 -- 172.21.15.131:0/3289703681 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 23) v1 ==== 136101+0+0 (secure 0 0 0) 0x7f6b740225e0 con 0x7f6b84101160 2024-05-08T22:08:01.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.069+0000 7f6b9e43eb80 1 Processor -- start 2024-05-08T22:08:01.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.069+0000 7f6b9e43eb80 1 -- start start 2024-05-08T22:08:01.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.070+0000 7f6b9e43eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623cfb6ce00 0x5623cfb6d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.070+0000 7f6b9e43eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623cfb6d720 0x5623cfb67350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.070+0000 7f6b9e43eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623cfb67890 0x5623cfb67cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.070+0000 7f6b9e43eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623cfabe150 con 0x5623cfb6d720 2024-05-08T22:08:01.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.070+0000 7f6b9e43eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623cfabbb10 con 0x5623cfb6ce00 2024-05-08T22:08:01.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.070+0000 7f6b9e43eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623cfab7400 con 0x5623cfb67890 2024-05-08T22:08:01.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.070+0000 7f6b80c00700 1 --2- 172.21.15.131:0/3289703681 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f6b680a13b0 0x7f6b680a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.070+0000 7f6b83e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623cfb6d720 0x5623cfb67350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.070+0000 7f6b80c00700 1 -- 172.21.15.131:0/3289703681 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f6b740ce4b0 con 0x7f6b84101160 2024-05-08T22:08:01.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.070+0000 7f6b83e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623cfb6d720 0x5623cfb67350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35204/0 (socket says 172.21.15.131:35204) 2024-05-08T22:08:01.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.070+0000 7f6b83e00700 1 -- 172.21.15.131:0/571369872 learned_addr learned my addr 172.21.15.131:0/571369872 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:01.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.070+0000 7f6b83400700 1 --2- 172.21.15.131:0/571369872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623cfb67890 0x5623cfb67cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.070+0000 7f6b88a00700 1 --2- 172.21.15.131:0/571369872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623cfb6ce00 0x5623cfb6d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.070+0000 7f6b88a00700 1 --2- 172.21.15.131:0/3289703681 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f6b680a13b0 0x7f6b680a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.071+0000 7f6b83400700 1 -- 172.21.15.131:0/571369872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623cfb6ce00 msgr2=0x5623cfb6d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:01.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.071+0000 7f6b83400700 1 --2- 172.21.15.131:0/571369872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623cfb6ce00 0x5623cfb6d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.071+0000 7f6b83400700 1 -- 172.21.15.131:0/571369872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623cfb6d720 msgr2=0x5623cfb67350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:01.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.071+0000 7f6b83400700 1 --2- 172.21.15.131:0/571369872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623cfb6d720 0x5623cfb67350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.071+0000 7f6b83400700 1 -- 172.21.15.131:0/571369872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623cf9383b0 con 0x5623cfb67890 2024-05-08T22:08:01.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.071+0000 7f6b83e00700 1 --2- 172.21.15.131:0/571369872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623cfb6d720 0x5623cfb67350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:01.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.071+0000 7f6b83400700 1 --2- 172.21.15.131:0/571369872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623cfb67890 0x5623cfb67cd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6b8406a580 tx=0x7f6b700117a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=671dc82d0c0c9606 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:01.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.071+0000 7f6b88a00700 1 --2- 172.21.15.131:0/3289703681 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f6b680a13b0 0x7f6b680a3870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009ec0 tx=0x7f6b7c00b040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:01.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.071+0000 7f6b79200700 1 -- 172.21.15.131:0/571369872 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70004310 con 0x5623cfb67890 2024-05-08T22:08:01.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.071+0000 7f6b79200700 1 -- 172.21.15.131:0/571369872 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b70004470 con 0x5623cfb67890 2024-05-08T22:08:01.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.071+0000 7f6b79200700 1 -- 172.21.15.131:0/571369872 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70003680 con 0x5623cfb67890 2024-05-08T22:08:01.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.072+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/571369872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623cfb67890 msgr2=0x5623cfb67cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.072+0000 7f6b9e43eb80 1 --2- 172.21.15.131:0/571369872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623cfb67890 0x5623cfb67cd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6b8406a580 tx=0x7f6b700117a0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.072+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/571369872 shutdown_connections 2024-05-08T22:08:01.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.072+0000 7f6b9e43eb80 1 --2- 172.21.15.131:0/571369872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623cfb67890 0x5623cfb67cd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.072+0000 7f6b9e43eb80 1 --2- 172.21.15.131:0/571369872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623cfb6ce00 0x5623cfb6d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.072+0000 7f6b9e43eb80 1 --2- 172.21.15.131:0/571369872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623cfb6d720 0x5623cfb67350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.072+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/571369872 >> 172.21.15.131:0/571369872 conn(0x5623cfac5210 msgr2=0x5623cfb61df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:01.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.072+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/571369872 shutdown_connections 2024-05-08T22:08:01.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.072+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/571369872 wait complete. 2024-05-08T22:08:01.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.073+0000 7f6b9e43eb80 1 -- start start 2024-05-08T22:08:01.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.073+0000 7f6b9e43eb80 1 Processor -- start 2024-05-08T22:08:01.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.074+0000 7f6b9e43eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623cfb67890 0x5623cfb70ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.074+0000 7f6b9e43eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623cfb6ce00 0x5623cfb71010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.074+0000 7f6b9e43eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623cfb6d720 0x5623cfb71550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.074+0000 7f6b9e43eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623cfa53560 con 0x5623cfb67890 2024-05-08T22:08:01.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.074+0000 7f6b9e43eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623cfa53200 con 0x5623cfb6d720 2024-05-08T22:08:01.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.074+0000 7f6b9e43eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623cf93a6a0 con 0x5623cfb6ce00 2024-05-08T22:08:01.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.074+0000 7f6b9e43eb80 10 client.0 _ll_register_callbacks cb 0x5623cfa5cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:01.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.074+0000 7f6b83400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623cfb6ce00 0x5623cfb71010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.074+0000 7f6b83e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623cfb67890 0x5623cfb70ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.074+0000 7f6b83400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623cfb6ce00 0x5623cfb71010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46700/0 (socket says 172.21.15.131:46700) 2024-05-08T22:08:01.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.074+0000 7f6b83400700 1 -- 172.21.15.131:0/4126085998 learned_addr learned my addr 172.21.15.131:0/4126085998 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:01.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.074+0000 7f6b88a00700 1 --2- 172.21.15.131:0/4126085998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623cfb6d720 0x5623cfb71550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.074+0000 7f6b83400700 1 -- 172.21.15.131:0/4126085998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623cfb6d720 msgr2=0x5623cfb71550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:01.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.074+0000 7f6b83400700 1 --2- 172.21.15.131:0/4126085998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623cfb6d720 0x5623cfb71550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.074+0000 7f6b83400700 1 -- 172.21.15.131:0/4126085998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623cfb67890 msgr2=0x5623cfb70ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:01.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.074+0000 7f6b83400700 1 --2- 172.21.15.131:0/4126085998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623cfb67890 0x5623cfb70ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.074+0000 7f6b83400700 1 -- 172.21.15.131:0/4126085998 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b841a6520 con 0x5623cfb6ce00 2024-05-08T22:08:01.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.075+0000 7f6b83400700 1 --2- 172.21.15.131:0/4126085998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623cfb6ce00 0x5623cfb71010 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6b70005310 tx=0x7f6b70005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=656de1888c2dbb3e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:01.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.075+0000 7f6b6d600700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:08:01.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.075+0000 7f6b6d600700 1 -- 172.21.15.131:0/4126085998 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70003680 con 0x5623cfb6ce00 2024-05-08T22:08:01.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.075+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/4126085998 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623cfa10270 con 0x5623cfb6ce00 2024-05-08T22:08:01.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.075+0000 7f6b6d600700 1 -- 172.21.15.131:0/4126085998 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f6b700037e0 con 0x5623cfb6ce00 2024-05-08T22:08:01.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.075+0000 7f6b6d600700 1 -- 172.21.15.131:0/4126085998 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b700039a0 con 0x5623cfb6ce00 2024-05-08T22:08:01.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.075+0000 7f6b9e43eb80 10 client.25705 fetch_fsmap learned FSMap version 24 2024-05-08T22:08:01.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.075+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/4126085998 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f6b70009c90 con 0x5623cfb6ce00 2024-05-08T22:08:01.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.076+0000 7f6b6d600700 1 -- 172.21.15.131:0/4126085998 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5623cfa10270 con 0x5623cfb6ce00 2024-05-08T22:08:01.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.076+0000 7f6b6d600700 1 -- 172.21.15.131:0/4126085998 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6b70022400 con 0x5623cfb6ce00 2024-05-08T22:08:01.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.076+0000 7f6b9e43eb80 10 client.25705 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:08:01.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.076+0000 7f6b9e43eb80 10 client.25705 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:08:01.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.076+0000 7f6b9e43eb80 20 client.25705 populate_metadata read hostname 'smithi131' 2024-05-08T22:08:01.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.076+0000 7f6b9e43eb80 1 --2- 172.21.15.131:0/4126085998 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x5623cfb810f0 0x5623cfb83510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.076+0000 7f6b9e43eb80 4 client.25705 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:08:01.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.076+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/4126085998 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623cfb36810 con 0x5623cfb810f0 2024-05-08T22:08:01.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.076+0000 7f6b83e00700 1 --2- 172.21.15.131:0/4126085998 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x5623cfb810f0 0x5623cfb83510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.077+0000 7f6b6d600700 10 client.25705 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:08:01.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.077+0000 7f6b83e00700 1 --2- 172.21.15.131:0/4126085998 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x5623cfb810f0 0x5623cfb83510 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c1724ce4e8667e77 server_cookie=4c57b12e710557d9 in_seq=0 out_seq=0 2024-05-08T22:08:01.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.077+0000 7f6b6d600700 1 -- 172.21.15.131:0/4126085998 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5623cfb36810 con 0x5623cfb810f0 2024-05-08T22:08:01.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.077+0000 7f6b6d600700 10 client.25705 handle_command_reply: tid=0 2024-05-08T22:08:01.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.077+0000 7f6b9e43eb80 1 client.25705 shutdown 2024-05-08T22:08:01.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.078+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/4126085998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623cfb6ce00 msgr2=0x5623cfb71010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.078+0000 7f6b9e43eb80 1 --2- 172.21.15.131:0/4126085998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623cfb6ce00 0x5623cfb71010 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6b70005310 tx=0x7f6b70005e20 comp rx=0 tx=0).stop 2024-05-08T22:08:01.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.078+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/4126085998 shutdown_connections 2024-05-08T22:08:01.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.078+0000 7f6b9e43eb80 1 --2- 172.21.15.131:0/4126085998 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x5623cfb810f0 0x5623cfb83510 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.078+0000 7f6b9e43eb80 1 --2- 172.21.15.131:0/4126085998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623cfb6d720 0x5623cfb71550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.078+0000 7f6b9e43eb80 1 --2- 172.21.15.131:0/4126085998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623cfb67890 0x5623cfb70ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.078+0000 7f6b9e43eb80 1 --2- 172.21.15.131:0/4126085998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623cfb6ce00 0x5623cfb71010 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.078+0000 7f6b6d600700 0 client.25705 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:08:01.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.078+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/4126085998 >> 172.21.15.131:0/4126085998 conn(0x5623cfac5210 msgr2=0x5623cfb615a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:01.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.078+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/4126085998 shutdown_connections 2024-05-08T22:08:01.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.078+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/4126085998 wait complete. 2024-05-08T22:08:01.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.078+0000 7f6b9e43eb80 20 client.25705 trim_cache size 0 max 16384 2024-05-08T22:08:01.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.092+0000 7f6b9e43eb80 1 Processor -- start 2024-05-08T22:08:01.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.092+0000 7f6b9e43eb80 1 -- start start 2024-05-08T22:08:01.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.093+0000 7f6b9e43eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623cfb67890 0x5623cfb67350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.093+0000 7f6b9e43eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623cfb6ce00 0x5623cfb834e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.093+0000 7f6b9e43eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623cfb6d720 0x5623cfb67c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.093+0000 7f6b9e43eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623cfb6ca50 con 0x5623cfb6ce00 2024-05-08T22:08:01.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.093+0000 7f6b9e43eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623cfb88700 con 0x5623cfb6d720 2024-05-08T22:08:01.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.093+0000 7f6b9e43eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623cfb5c800 con 0x5623cfb67890 2024-05-08T22:08:01.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.093+0000 7f6b83e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623cfb67890 0x5623cfb67350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.093+0000 7f6b83400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623cfb6ce00 0x5623cfb834e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.093+0000 7f6b83e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623cfb67890 0x5623cfb67350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46714/0 (socket says 172.21.15.131:46714) 2024-05-08T22:08:01.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.093+0000 7f6b83e00700 1 -- 172.21.15.131:0/4081902433 learned_addr learned my addr 172.21.15.131:0/4081902433 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:01.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.093+0000 7f6b88a00700 1 --2- 172.21.15.131:0/4081902433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623cfb6d720 0x5623cfb67c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.093+0000 7f6b83e00700 1 -- 172.21.15.131:0/4081902433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623cfb6d720 msgr2=0x5623cfb67c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:01.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.093+0000 7f6b83e00700 1 --2- 172.21.15.131:0/4081902433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623cfb6d720 0x5623cfb67c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.093+0000 7f6b83e00700 1 -- 172.21.15.131:0/4081902433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623cfb6ce00 msgr2=0x5623cfb834e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:01.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.093+0000 7f6b83e00700 1 --2- 172.21.15.131:0/4081902433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623cfb6ce00 0x5623cfb834e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.093+0000 7f6b83e00700 1 -- 172.21.15.131:0/4081902433 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623cf8e4b60 con 0x5623cfb67890 2024-05-08T22:08:01.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.093+0000 7f6b83400700 1 --2- 172.21.15.131:0/4081902433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623cfb6ce00 0x5623cfb834e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:01.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.094+0000 7f6b83e00700 1 --2- 172.21.15.131:0/4081902433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623cfb67890 0x5623cfb67350 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6b7409afa0 tx=0x7f6b740a1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2cf81e65f7a589 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:01.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.094+0000 7f6b79c00700 1 -- 172.21.15.131:0/4081902433 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7409a970 con 0x5623cfb67890 2024-05-08T22:08:01.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.094+0000 7f6b79c00700 1 -- 172.21.15.131:0/4081902433 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b7409b710 con 0x5623cfb67890 2024-05-08T22:08:01.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.094+0000 7f6b79c00700 1 -- 172.21.15.131:0/4081902433 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7401f2d0 con 0x5623cfb67890 2024-05-08T22:08:01.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.094+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/4081902433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623cfb67890 msgr2=0x5623cfb67350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.094+0000 7f6b9e43eb80 1 --2- 172.21.15.131:0/4081902433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623cfb67890 0x5623cfb67350 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6b7409afa0 tx=0x7f6b740a1040 comp rx=0 tx=0).stop 2024-05-08T22:08:01.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.094+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/4081902433 shutdown_connections 2024-05-08T22:08:01.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.094+0000 7f6b9e43eb80 1 --2- 172.21.15.131:0/4081902433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623cfb6d720 0x5623cfb67c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.094+0000 7f6b9e43eb80 1 --2- 172.21.15.131:0/4081902433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623cfb67890 0x5623cfb67350 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.094+0000 7f6b9e43eb80 1 --2- 172.21.15.131:0/4081902433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623cfb6ce00 0x5623cfb834e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.095+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/4081902433 >> 172.21.15.131:0/4081902433 conn(0x5623cfac5210 msgr2=0x5623cfb6bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:01.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.095+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/4081902433 shutdown_connections 2024-05-08T22:08:01.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.095+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/4081902433 wait complete. 2024-05-08T22:08:01.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.095+0000 7f6b9e43eb80 1 -- start start 2024-05-08T22:08:01.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.095+0000 7f6b9e43eb80 1 Processor -- start 2024-05-08T22:08:01.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.096+0000 7f6b9e43eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623cfb67890 0x5623cfb78370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.096+0000 7f6b9e43eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623cfb6ce00 0x5623cfb74140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.096+0000 7f6b9e43eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623cfb6d720 0x5623cfb74680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.096+0000 7f6b9e43eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623cfb74c10 con 0x5623cfb6ce00 2024-05-08T22:08:01.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.096+0000 7f6b9e43eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623cfb74d80 con 0x5623cfb6d720 2024-05-08T22:08:01.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.096+0000 7f6b9e43eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623cfb74f40 con 0x5623cfb67890 2024-05-08T22:08:01.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.096+0000 7f6b9e43eb80 10 client.0 _ll_register_callbacks cb 0x5623cfa5cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:01.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.096+0000 7f6b83400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623cfb6ce00 0x5623cfb74140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.096+0000 7f6b88a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623cfb6d720 0x5623cfb74680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.096+0000 7f6b83400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623cfb6ce00 0x5623cfb74140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35236/0 (socket says 172.21.15.131:35236) 2024-05-08T22:08:01.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.096+0000 7f6b83400700 1 -- 172.21.15.131:0/2074404526 learned_addr learned my addr 172.21.15.131:0/2074404526 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:01.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.096+0000 7f6b83e00700 1 --2- 172.21.15.131:0/2074404526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623cfb67890 0x5623cfb78370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.097+0000 7f6b83400700 1 -- 172.21.15.131:0/2074404526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623cfb67890 msgr2=0x5623cfb78370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:01.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.097+0000 7f6b83400700 1 --2- 172.21.15.131:0/2074404526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623cfb67890 0x5623cfb78370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.097+0000 7f6b83400700 1 -- 172.21.15.131:0/2074404526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623cfb6d720 msgr2=0x5623cfb74680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:01.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.097+0000 7f6b83400700 1 --2- 172.21.15.131:0/2074404526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623cfb6d720 0x5623cfb74680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.097+0000 7f6b83400700 1 -- 172.21.15.131:0/2074404526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b70009c90 con 0x5623cfb6ce00 2024-05-08T22:08:01.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.097+0000 7f6b88a00700 1 --2- 172.21.15.131:0/2074404526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623cfb6d720 0x5623cfb74680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:01.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.097+0000 7f6b83400700 1 --2- 172.21.15.131:0/2074404526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623cfb6ce00 0x5623cfb74140 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6b70005850 tx=0x7f6b70009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db006aa684b578c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:01.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.097+0000 7f6b6d600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:08:01.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.097+0000 7f6b6d600700 1 -- 172.21.15.131:0/2074404526 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b700054c0 con 0x5623cfb6ce00 2024-05-08T22:08:01.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.097+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/2074404526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b70004470 con 0x5623cfb6ce00 2024-05-08T22:08:01.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.097+0000 7f6b6d600700 1 -- 172.21.15.131:0/2074404526 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f6b70022a00 con 0x5623cfb6ce00 2024-05-08T22:08:01.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.098+0000 7f6b6d600700 1 -- 172.21.15.131:0/2074404526 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b70011020 con 0x5623cfb6ce00 2024-05-08T22:08:01.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.098+0000 7f6b9e43eb80 10 client.15657 fetch_fsmap learned FSMap version 24 2024-05-08T22:08:01.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.098+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/2074404526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f6b841a60a0 con 0x5623cfb6ce00 2024-05-08T22:08:01.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.098+0000 7f6b6d600700 1 -- 172.21.15.131:0/2074404526 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b70004470 con 0x5623cfb6ce00 2024-05-08T22:08:01.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.098+0000 7f6b6d600700 1 -- 172.21.15.131:0/2074404526 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6b70004780 con 0x5623cfb6ce00 2024-05-08T22:08:01.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.098+0000 7f6b9e43eb80 10 client.15657 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:08:01.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.098+0000 7f6b9e43eb80 10 client.15657 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:08:01.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.098+0000 7f6b9e43eb80 20 client.15657 populate_metadata read hostname 'smithi131' 2024-05-08T22:08:01.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.098+0000 7f6b9e43eb80 1 --2- 172.21.15.131:0/2074404526 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x5623cfbd7860 0x5623cfbd9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.098+0000 7f6b9e43eb80 4 client.15657 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:08:01.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.098+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/2074404526 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5623cfaafec0 con 0x5623cfbd7860 2024-05-08T22:08:01.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.098+0000 7f6b83e00700 1 --2- 172.21.15.131:0/2074404526 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x5623cfbd7860 0x5623cfbd9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.099+0000 7f6b83e00700 1 --2- 172.21.15.131:0/2074404526 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x5623cfbd7860 0x5623cfbd9c80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e78e485abe50ed5 server_cookie=f4bc9970410cb932 in_seq=0 out_seq=0 2024-05-08T22:08:01.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.099+0000 7f6b6d600700 10 client.15657 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:08:01.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.099+0000 7f6b6d600700 1 -- 172.21.15.131:0/2074404526 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5623cfaafec0 con 0x5623cfbd7860 2024-05-08T22:08:01.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.099+0000 7f6b6d600700 10 client.15657 handle_command_reply: tid=0 2024-05-08T22:08:01.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.099+0000 7f6b9e43eb80 1 client.15657 shutdown 2024-05-08T22:08:01.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.099+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/2074404526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623cfb6ce00 msgr2=0x5623cfb74140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.099+0000 7f6b9e43eb80 1 --2- 172.21.15.131:0/2074404526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623cfb6ce00 0x5623cfb74140 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6b70005850 tx=0x7f6b70009fd0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.099+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/2074404526 shutdown_connections 2024-05-08T22:08:01.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.099+0000 7f6b9e43eb80 1 --2- 172.21.15.131:0/2074404526 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x5623cfbd7860 0x5623cfbd9c80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.099+0000 7f6b9e43eb80 1 --2- 172.21.15.131:0/2074404526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623cfb6d720 0x5623cfb74680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.099+0000 7f6b9e43eb80 1 --2- 172.21.15.131:0/2074404526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623cfb67890 0x5623cfb78370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.099+0000 7f6b9e43eb80 1 --2- 172.21.15.131:0/2074404526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623cfb6ce00 0x5623cfb74140 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.099+0000 7f6b6d600700 0 client.15657 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:08:01.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.099+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/2074404526 >> 172.21.15.131:0/2074404526 conn(0x5623cfac5210 msgr2=0x5623cfb6f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:01.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.099+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/2074404526 shutdown_connections 2024-05-08T22:08:01.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.100+0000 7f6b9e43eb80 1 -- 172.21.15.131:0/2074404526 wait complete. 2024-05-08T22:08:01.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.100+0000 7f6b9e43eb80 20 client.15657 trim_cache size 0 max 16384 2024-05-08T22:08:01.108 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:08:01.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.100+0000 7f6b8aa00700 1 -- 172.21.15.131:0/3289703681 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f6b680a13b0 msgr2=0x7f6b680a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.100+0000 7f6b8aa00700 1 --2- 172.21.15.131:0/3289703681 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f6b680a13b0 0x7f6b680a3870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009ec0 tx=0x7f6b7c00b040 comp rx=0 tx=0).stop 2024-05-08T22:08:01.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.100+0000 7f6b8aa00700 1 -- 172.21.15.131:0/3289703681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b84101160 msgr2=0x7f6b840732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.100+0000 7f6b8aa00700 1 --2- 172.21.15.131:0/3289703681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b84101160 0x7f6b840732a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6b74004760 tx=0x7f6b74004790 comp rx=0 tx=0).stop 2024-05-08T22:08:01.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.100+0000 7f6b8aa00700 1 -- 172.21.15.131:0/3289703681 shutdown_connections 2024-05-08T22:08:01.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.100+0000 7f6b8aa00700 1 --2- 172.21.15.131:0/3289703681 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f6b680a13b0 0x7f6b680a3870 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.100+0000 7f6b8aa00700 1 --2- 172.21.15.131:0/3289703681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b84073d20 0x7f6b841a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.100+0000 7f6b8aa00700 1 --2- 172.21.15.131:0/3289703681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b84101160 0x7f6b840732a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.100+0000 7f6b8aa00700 1 --2- 172.21.15.131:0/3289703681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b84106280 0x7f6b840737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.100+0000 7f6b8aa00700 1 -- 172.21.15.131:0/3289703681 >> 172.21.15.131:0/3289703681 conn(0x7f6b840fc9c0 msgr2=0x7f6b840fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:01.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.100+0000 7f6b8aa00700 1 -- 172.21.15.131:0/3289703681 shutdown_connections 2024-05-08T22:08:01.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.101+0000 7f6b8aa00700 1 -- 172.21.15.131:0/3289703681 wait complete. 2024-05-08T22:08:01.120 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T22:08:01.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.320+0000 7f636ca00700 1 -- 172.21.15.131:0/908368696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63681060b0 msgr2=0x7f636810dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.320+0000 7f636ca00700 1 --2- 172.21.15.131:0/908368696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63681060b0 0x7f636810dc30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f636000b0a0 tx=0x7f636000b3b0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.320+0000 7f636ca00700 1 -- 172.21.15.131:0/908368696 shutdown_connections 2024-05-08T22:08:01.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.320+0000 7f636ca00700 1 --2- 172.21.15.131:0/908368696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63681060b0 0x7f636810dc30 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.320+0000 7f636ca00700 1 --2- 172.21.15.131:0/908368696 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6368100f90 0x7f6368101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.320+0000 7f636ca00700 1 --2- 172.21.15.131:0/908368696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6368101940 0x7f6368105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.320+0000 7f636ca00700 1 -- 172.21.15.131:0/908368696 >> 172.21.15.131:0/908368696 conn(0x7f63680fc810 msgr2=0x7f63680fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:01.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.320+0000 7f636ca00700 1 -- 172.21.15.131:0/908368696 shutdown_connections 2024-05-08T22:08:01.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.320+0000 7f636ca00700 1 -- 172.21.15.131:0/908368696 wait complete. 2024-05-08T22:08:01.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.321+0000 7f636ca00700 1 Processor -- start 2024-05-08T22:08:01.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.321+0000 7f636ca00700 1 -- start start 2024-05-08T22:08:01.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.322+0000 7f636ca00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6368100f90 0x7f6368199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.322+0000 7f636ca00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63681060b0 0x7f6368199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.322+0000 7f636ca00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f636819dea0 0x7f636819e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.322+0000 7f636ca00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636819e920 con 0x7f636819dea0 2024-05-08T22:08:01.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.322+0000 7f636ca00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636819ea90 con 0x7f6368100f90 2024-05-08T22:08:01.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.322+0000 7f636ca00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636819ec50 con 0x7f63681060b0 2024-05-08T22:08:01.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.322+0000 7f6367e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f636819dea0 0x7f636819e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.322+0000 7f6366a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63681060b0 0x7f6368199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.322+0000 7f6367400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6368100f90 0x7f6368199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.322+0000 7f6366a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63681060b0 0x7f6368199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46732/0 (socket says 172.21.15.131:46732) 2024-05-08T22:08:01.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.322+0000 7f6367e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f636819dea0 0x7f636819e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35254/0 (socket says 172.21.15.131:35254) 2024-05-08T22:08:01.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.322+0000 7f6366a00700 1 -- 172.21.15.131:0/3033351347 learned_addr learned my addr 172.21.15.131:0/3033351347 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:01.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.322+0000 7f6366a00700 1 -- 172.21.15.131:0/3033351347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6368100f90 msgr2=0x7f6368199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.322+0000 7f6366a00700 1 --2- 172.21.15.131:0/3033351347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6368100f90 0x7f6368199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.322+0000 7f6366a00700 1 -- 172.21.15.131:0/3033351347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f636819dea0 msgr2=0x7f636819e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.322+0000 7f6366a00700 1 --2- 172.21.15.131:0/3033351347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f636819dea0 0x7f636819e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.322+0000 7f6366a00700 1 -- 172.21.15.131:0/3033351347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6360009d00 con 0x7f63681060b0 2024-05-08T22:08:01.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.323+0000 7f6367400700 1 --2- 172.21.15.131:0/3033351347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6368100f90 0x7f6368199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:01.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.323+0000 7f6366a00700 1 --2- 172.21.15.131:0/3033351347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63681060b0 0x7f6368199b50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f635400e9f0 tx=0x7f635400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:01.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.323+0000 7f635fe00700 1 -- 172.21.15.131:0/3033351347 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6354009750 con 0x7f63681060b0 2024-05-08T22:08:01.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.323+0000 7f635fe00700 1 -- 172.21.15.131:0/3033351347 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f635400c7c0 con 0x7f63681060b0 2024-05-08T22:08:01.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.323+0000 7f635fe00700 1 -- 172.21.15.131:0/3033351347 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63540054d0 con 0x7f63681060b0 2024-05-08T22:08:01.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.323+0000 7f636ca00700 1 -- 172.21.15.131:0/3033351347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63681a61b0 con 0x7f63681060b0 2024-05-08T22:08:01.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.323+0000 7f636ca00700 1 -- 172.21.15.131:0/3033351347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63681a6640 con 0x7f63681060b0 2024-05-08T22:08:01.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.327+0000 7f635fe00700 1 -- 172.21.15.131:0/3033351347 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 24) v1 ==== 136158+0+0 (secure 0 0 0) 0x7f635400c930 con 0x7f63681060b0 2024-05-08T22:08:01.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.329+0000 7f6381a31b80 1 Processor -- start 2024-05-08T22:08:01.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.329+0000 7f635fe00700 1 --2- 172.21.15.131:0/3033351347 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f634c0a14c0 0x7f634c0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.329+0000 7f6381a31b80 1 -- start start 2024-05-08T22:08:01.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.329+0000 7f635fe00700 1 -- 172.21.15.131:0/3033351347 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f6354014070 con 0x7f63681060b0 2024-05-08T22:08:01.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.329+0000 7f6367e00700 1 --2- 172.21.15.131:0/3033351347 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f634c0a14c0 0x7f634c0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.329+0000 7f6381a31b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69e05050 0x55bb69e05430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.329+0000 7f6381a31b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb69e05970 0x55bb69dff350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.329+0000 7f6381a31b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb69dff890 0x55bb69dffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.329+0000 7f6381a31b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb69d56150 con 0x55bb69e05970 2024-05-08T22:08:01.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.329+0000 7f6381a31b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb69d53b10 con 0x55bb69dff890 2024-05-08T22:08:01.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.329+0000 7f6381a31b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb69d4f400 con 0x55bb69e05050 2024-05-08T22:08:01.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.330+0000 7f6367400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69e05050 0x55bb69e05430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.330+0000 7f6366a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb69e05970 0x55bb69dff350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.330+0000 7f6367400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69e05050 0x55bb69e05430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46740/0 (socket says 172.21.15.131:46740) 2024-05-08T22:08:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.330+0000 7f6367400700 1 -- 172.21.15.131:0/3037653232 learned_addr learned my addr 172.21.15.131:0/3037653232 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.330+0000 7f6367e00700 1 --2- 172.21.15.131:0/3037653232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb69dff890 0x55bb69dffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.330+0000 7f6367e00700 1 --2- 172.21.15.131:0/3033351347 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f634c0a14c0 0x7f634c0a3980 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f636000b780 tx=0x7f6360006210 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.330+0000 7f6367400700 1 -- 172.21.15.131:0/3037653232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb69dff890 msgr2=0x55bb69dffcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.330+0000 7f6367400700 1 --2- 172.21.15.131:0/3037653232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb69dff890 0x55bb69dffcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.330+0000 7f6367400700 1 -- 172.21.15.131:0/3037653232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb69e05970 msgr2=0x55bb69dff350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.330+0000 7f6367400700 1 --2- 172.21.15.131:0/3037653232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb69e05970 0x55bb69dff350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.330+0000 7f6367400700 1 -- 172.21.15.131:0/3037653232 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb69bd03b0 con 0x55bb69e05050 2024-05-08T22:08:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.330+0000 7f6367400700 1 --2- 172.21.15.131:0/3037653232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69e05050 0x55bb69e05430 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f63580115f0 tx=0x7f6358011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6809c04b103af1f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.331+0000 7f6353e00700 1 -- 172.21.15.131:0/3037653232 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6358019630 con 0x55bb69e05050 2024-05-08T22:08:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.331+0000 7f6353e00700 1 -- 172.21.15.131:0/3037653232 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6358019c50 con 0x55bb69e05050 2024-05-08T22:08:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.331+0000 7f6353e00700 1 -- 172.21.15.131:0/3037653232 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635800ec90 con 0x55bb69e05050 2024-05-08T22:08:01.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.331+0000 7f6381a31b80 1 -- 172.21.15.131:0/3037653232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69e05050 msgr2=0x55bb69e05430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.331+0000 7f6381a31b80 1 --2- 172.21.15.131:0/3037653232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69e05050 0x55bb69e05430 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f63580115f0 tx=0x7f6358011900 comp rx=0 tx=0).stop 2024-05-08T22:08:01.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.331+0000 7f6381a31b80 1 -- 172.21.15.131:0/3037653232 shutdown_connections 2024-05-08T22:08:01.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.331+0000 7f6381a31b80 1 --2- 172.21.15.131:0/3037653232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb69dff890 0x55bb69dffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.331+0000 7f6381a31b80 1 --2- 172.21.15.131:0/3037653232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69e05050 0x55bb69e05430 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.331+0000 7f6381a31b80 1 --2- 172.21.15.131:0/3037653232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb69e05970 0x55bb69dff350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.331+0000 7f6381a31b80 1 -- 172.21.15.131:0/3037653232 >> 172.21.15.131:0/3037653232 conn(0x55bb69d5d210 msgr2=0x55bb69dfa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:01.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.331+0000 7f6381a31b80 1 -- 172.21.15.131:0/3037653232 shutdown_connections 2024-05-08T22:08:01.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.331+0000 7f6381a31b80 1 -- 172.21.15.131:0/3037653232 wait complete. 2024-05-08T22:08:01.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.332+0000 7f6381a31b80 1 -- start start 2024-05-08T22:08:01.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.332+0000 7f6381a31b80 1 Processor -- start 2024-05-08T22:08:01.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.333+0000 7f6381a31b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb69dff890 0x55bb69e0b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.333+0000 7f6381a31b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb69e05050 0x55bb69e08b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.333+0000 7f6381a31b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69e05970 0x55bb69e092b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.333+0000 7f6381a31b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb69ceb560 con 0x55bb69dff890 2024-05-08T22:08:01.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.333+0000 7f6381a31b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb69ceb200 con 0x55bb69e05050 2024-05-08T22:08:01.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.333+0000 7f6381a31b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb69bd26a0 con 0x55bb69e05970 2024-05-08T22:08:01.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.333+0000 7f6381a31b80 10 client.0 _ll_register_callbacks cb 0x55bb69cf4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:01.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.333+0000 7f6367400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb69dff890 0x55bb69e0b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.333+0000 7f6367e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69e05970 0x55bb69e092b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.333+0000 7f6367e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69e05970 0x55bb69e092b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46754/0 (socket says 172.21.15.131:46754) 2024-05-08T22:08:01.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.333+0000 7f6367e00700 1 -- 172.21.15.131:0/3229105681 learned_addr learned my addr 172.21.15.131:0/3229105681 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.333+0000 7f6366a00700 1 --2- 172.21.15.131:0/3229105681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb69e05050 0x55bb69e08b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.333+0000 7f6367e00700 1 -- 172.21.15.131:0/3229105681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb69e05050 msgr2=0x55bb69e08b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.333+0000 7f6367e00700 1 --2- 172.21.15.131:0/3229105681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb69e05050 0x55bb69e08b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.333+0000 7f6367e00700 1 -- 172.21.15.131:0/3229105681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb69dff890 msgr2=0x55bb69e0b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.333+0000 7f6367e00700 1 --2- 172.21.15.131:0/3229105681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb69dff890 0x55bb69e0b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.333+0000 7f6367e00700 1 -- 172.21.15.131:0/3229105681 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6358009dd0 con 0x55bb69e05970 2024-05-08T22:08:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.333+0000 7f6367400700 1 --2- 172.21.15.131:0/3229105681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb69dff890 0x55bb69e0b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.334+0000 7f6367e00700 1 --2- 172.21.15.131:0/3229105681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69e05970 0x55bb69e092b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6360015000 tx=0x7f63600099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44748fe4ef551403 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.334+0000 7f6350c00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:08:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.334+0000 7f6350c00700 1 -- 172.21.15.131:0/3229105681 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636000e040 con 0x55bb69e05970 2024-05-08T22:08:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.334+0000 7f6350c00700 1 -- 172.21.15.131:0/3229105681 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f636001c440 con 0x55bb69e05970 2024-05-08T22:08:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.334+0000 7f6381a31b80 1 -- 172.21.15.131:0/3229105681 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb69ca8270 con 0x55bb69e05970 2024-05-08T22:08:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.334+0000 7f6350c00700 1 -- 172.21.15.131:0/3229105681 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6360003e90 con 0x55bb69e05970 2024-05-08T22:08:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.334+0000 7f6381a31b80 10 client.25723 fetch_fsmap learned FSMap version 24 2024-05-08T22:08:01.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.334+0000 7f6381a31b80 1 -- 172.21.15.131:0/3229105681 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bb69b7cb60 con 0x55bb69e05970 2024-05-08T22:08:01.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.334+0000 7f6350c00700 1 -- 172.21.15.131:0/3229105681 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bb69ca8270 con 0x55bb69e05970 2024-05-08T22:08:01.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.335+0000 7f6350c00700 1 -- 172.21.15.131:0/3229105681 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f63600051e0 con 0x55bb69e05970 2024-05-08T22:08:01.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.335+0000 7f6381a31b80 10 client.25723 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:08:01.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.335+0000 7f6381a31b80 10 client.25723 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:08:01.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.335+0000 7f6381a31b80 20 client.25723 populate_metadata read hostname 'smithi131' 2024-05-08T22:08:01.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.335+0000 7f6381a31b80 1 --2- 172.21.15.131:0/3229105681 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55bb69e192d0 0x55bb69e1b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.335+0000 7f6381a31b80 4 client.25723 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:08:01.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.335+0000 7f6381a31b80 1 -- 172.21.15.131:0/3229105681 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb69dce810 con 0x55bb69e192d0 2024-05-08T22:08:01.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.335+0000 7f6367400700 1 --2- 172.21.15.131:0/3229105681 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55bb69e192d0 0x55bb69e1b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.336+0000 7f6350c00700 10 client.25723 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:08:01.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.335+0000 7f6367400700 1 --2- 172.21.15.131:0/3229105681 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55bb69e192d0 0x55bb69e1b6f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8366e789243c5aee server_cookie=e6fa7f82f3aca3ee in_seq=0 out_seq=0 2024-05-08T22:08:01.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.336+0000 7f6350c00700 1 -- 172.21.15.131:0/3229105681 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bb69dce810 con 0x55bb69e192d0 2024-05-08T22:08:01.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.336+0000 7f6350c00700 10 client.25723 handle_command_reply: tid=0 2024-05-08T22:08:01.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.336+0000 7f6381a31b80 1 client.25723 shutdown 2024-05-08T22:08:01.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.336+0000 7f6381a31b80 1 -- 172.21.15.131:0/3229105681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69e05970 msgr2=0x55bb69e092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.336+0000 7f6381a31b80 1 --2- 172.21.15.131:0/3229105681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69e05970 0x55bb69e092b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6360015000 tx=0x7f63600099e0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.336+0000 7f6381a31b80 1 -- 172.21.15.131:0/3229105681 shutdown_connections 2024-05-08T22:08:01.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.336+0000 7f6381a31b80 1 --2- 172.21.15.131:0/3229105681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69e05970 0x55bb69e092b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.336+0000 7f6381a31b80 1 --2- 172.21.15.131:0/3229105681 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55bb69e192d0 0x55bb69e1b6f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.336+0000 7f6381a31b80 1 --2- 172.21.15.131:0/3229105681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb69dff890 0x55bb69e0b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.337+0000 7f6350c00700 0 client.25723 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:08:01.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.337+0000 7f6381a31b80 1 --2- 172.21.15.131:0/3229105681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb69e05050 0x55bb69e08b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.337+0000 7f6381a31b80 1 -- 172.21.15.131:0/3229105681 >> 172.21.15.131:0/3229105681 conn(0x55bb69d5d210 msgr2=0x55bb69dfa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:01.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.337+0000 7f6381a31b80 1 -- 172.21.15.131:0/3229105681 shutdown_connections 2024-05-08T22:08:01.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.337+0000 7f6381a31b80 1 -- 172.21.15.131:0/3229105681 wait complete. 2024-05-08T22:08:01.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.337+0000 7f6381a31b80 20 client.25723 trim_cache size 0 max 16384 2024-05-08T22:08:01.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.350+0000 7f6381a31b80 1 Processor -- start 2024-05-08T22:08:01.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.350+0000 7f6381a31b80 1 -- start start 2024-05-08T22:08:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.351+0000 7f6381a31b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69dff890 0x55bb69e05430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.351+0000 7f6381a31b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb69e05050 0x55bb69e14de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.351+0000 7f6381a31b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb69e05970 0x55bb69dfb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.351+0000 7f6381a31b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb69e156d0 con 0x55bb69e05970 2024-05-08T22:08:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.351+0000 7f6381a31b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb69df2040 con 0x55bb69e05050 2024-05-08T22:08:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.351+0000 7f6381a31b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb69e03550 con 0x55bb69dff890 2024-05-08T22:08:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.351+0000 7f6367400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69dff890 0x55bb69e05430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.351+0000 7f6367e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb69e05970 0x55bb69dfb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.351+0000 7f6367400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69dff890 0x55bb69e05430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46760/0 (socket says 172.21.15.131:46760) 2024-05-08T22:08:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.351+0000 7f6367400700 1 -- 172.21.15.131:0/2061982131 learned_addr learned my addr 172.21.15.131:0/2061982131 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.351+0000 7f6366a00700 1 --2- 172.21.15.131:0/2061982131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb69e05050 0x55bb69e14de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.351+0000 7f6367400700 1 -- 172.21.15.131:0/2061982131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb69e05050 msgr2=0x55bb69e14de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.351+0000 7f6367400700 1 --2- 172.21.15.131:0/2061982131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb69e05050 0x55bb69e14de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.351+0000 7f6367400700 1 -- 172.21.15.131:0/2061982131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb69e05970 msgr2=0x55bb69dfb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.351+0000 7f6367400700 1 --2- 172.21.15.131:0/2061982131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb69e05970 0x55bb69dfb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.351+0000 7f6367400700 1 -- 172.21.15.131:0/2061982131 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb69d15560 con 0x55bb69dff890 2024-05-08T22:08:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.351+0000 7f6366a00700 1 --2- 172.21.15.131:0/2061982131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb69e05050 0x55bb69e14de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:08:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.352+0000 7f6367400700 1 --2- 172.21.15.131:0/2061982131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69dff890 0x55bb69e05430 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6358011510 tx=0x7f635800e830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b2071d511a1961b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.352+0000 7f635d200700 1 -- 172.21.15.131:0/2061982131 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635802cde0 con 0x55bb69dff890 2024-05-08T22:08:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.352+0000 7f635d200700 1 -- 172.21.15.131:0/2061982131 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f635800e4b0 con 0x55bb69dff890 2024-05-08T22:08:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.352+0000 7f635d200700 1 -- 172.21.15.131:0/2061982131 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635800e8a0 con 0x55bb69dff890 2024-05-08T22:08:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.353+0000 7f6381a31b80 1 -- 172.21.15.131:0/2061982131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69dff890 msgr2=0x55bb69e05430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.353+0000 7f6381a31b80 1 --2- 172.21.15.131:0/2061982131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69dff890 0x55bb69e05430 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6358011510 tx=0x7f635800e830 comp rx=0 tx=0).stop 2024-05-08T22:08:01.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.353+0000 7f6381a31b80 1 -- 172.21.15.131:0/2061982131 shutdown_connections 2024-05-08T22:08:01.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.353+0000 7f6381a31b80 1 --2- 172.21.15.131:0/2061982131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb69e05970 0x55bb69dfb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.353+0000 7f6381a31b80 1 --2- 172.21.15.131:0/2061982131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69dff890 0x55bb69e05430 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.353+0000 7f6381a31b80 1 --2- 172.21.15.131:0/2061982131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb69e05050 0x55bb69e14de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.353+0000 7f6381a31b80 1 -- 172.21.15.131:0/2061982131 >> 172.21.15.131:0/2061982131 conn(0x55bb69d5d210 msgr2=0x55bb69dffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:01.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.353+0000 7f6381a31b80 1 -- 172.21.15.131:0/2061982131 shutdown_connections 2024-05-08T22:08:01.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.353+0000 7f6381a31b80 1 -- 172.21.15.131:0/2061982131 wait complete. 2024-05-08T22:08:01.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.353+0000 7f6381a31b80 1 -- start start 2024-05-08T22:08:01.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.354+0000 7f6381a31b80 1 Processor -- start 2024-05-08T22:08:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.354+0000 7f6381a31b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb69dff890 0x55bb69e10520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.354+0000 7f6381a31b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69e05050 0x55bb69e00ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.354+0000 7f6381a31b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb69e05970 0x55bb69e01020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.354+0000 7f6381a31b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb69e01560 con 0x55bb69dff890 2024-05-08T22:08:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.354+0000 7f6381a31b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb69e016a0 con 0x55bb69e05970 2024-05-08T22:08:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.354+0000 7f6381a31b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb69e01860 con 0x55bb69e05050 2024-05-08T22:08:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.354+0000 7f6381a31b80 10 client.0 _ll_register_callbacks cb 0x55bb69cf4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.354+0000 7f6367400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb69dff890 0x55bb69e10520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.354+0000 7f6367400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb69dff890 0x55bb69e10520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35284/0 (socket says 172.21.15.131:35284) 2024-05-08T22:08:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.354+0000 7f6367400700 1 -- 172.21.15.131:0/355234516 learned_addr learned my addr 172.21.15.131:0/355234516 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.355+0000 7f6366a00700 1 --2- 172.21.15.131:0/355234516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69e05050 0x55bb69e00ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.355+0000 7f6367e00700 1 --2- 172.21.15.131:0/355234516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb69e05970 0x55bb69e01020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.355+0000 7f6367400700 1 -- 172.21.15.131:0/355234516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69e05050 msgr2=0x55bb69e00ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.355+0000 7f6367400700 1 --2- 172.21.15.131:0/355234516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69e05050 0x55bb69e00ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.355+0000 7f6367400700 1 -- 172.21.15.131:0/355234516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb69e05970 msgr2=0x55bb69e01020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.355+0000 7f6367400700 1 --2- 172.21.15.131:0/355234516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb69e05970 0x55bb69e01020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.355+0000 7f6367400700 1 -- 172.21.15.131:0/355234516 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb69bd03b0 con 0x55bb69dff890 2024-05-08T22:08:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.355+0000 7f6367400700 1 --2- 172.21.15.131:0/355234516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb69dff890 0x55bb69e10520 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6358011db0 tx=0x7f6358011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ebe934f8879fd52 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.355+0000 7f6350c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:08:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.355+0000 7f6350c00700 1 -- 172.21.15.131:0/355234516 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63580055d0 con 0x55bb69dff890 2024-05-08T22:08:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.355+0000 7f6350c00700 1 -- 172.21.15.131:0/355234516 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f635802cdf0 con 0x55bb69dff890 2024-05-08T22:08:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.355+0000 7f6381a31b80 1 -- 172.21.15.131:0/355234516 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb69d47ec0 con 0x55bb69dff890 2024-05-08T22:08:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.356+0000 7f6350c00700 1 -- 172.21.15.131:0/355234516 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f635802c3b0 con 0x55bb69dff890 2024-05-08T22:08:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.356+0000 7f6381a31b80 10 client.15675 fetch_fsmap learned FSMap version 24 2024-05-08T22:08:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.356+0000 7f6381a31b80 1 -- 172.21.15.131:0/355234516 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bb69bd4cd0 con 0x55bb69dff890 2024-05-08T22:08:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.356+0000 7f6350c00700 1 -- 172.21.15.131:0/355234516 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bb69d47ec0 con 0x55bb69dff890 2024-05-08T22:08:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.356+0000 7f6350c00700 1 -- 172.21.15.131:0/355234516 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6358019430 con 0x55bb69dff890 2024-05-08T22:08:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.356+0000 7f6381a31b80 10 client.15675 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:08:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.356+0000 7f6381a31b80 10 client.15675 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:08:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.356+0000 7f6381a31b80 20 client.15675 populate_metadata read hostname 'smithi131' 2024-05-08T22:08:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.356+0000 7f6381a31b80 1 --2- 172.21.15.131:0/355234516 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55bb69e6f9c0 0x55bb69e71de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.356+0000 7f6381a31b80 4 client.15675 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:08:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.356+0000 7f6381a31b80 1 -- 172.21.15.131:0/355234516 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bb69b9dcf0 con 0x55bb69e6f9c0 2024-05-08T22:08:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.356+0000 7f6366a00700 1 --2- 172.21.15.131:0/355234516 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55bb69e6f9c0 0x55bb69e71de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.357+0000 7f6366a00700 1 --2- 172.21.15.131:0/355234516 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55bb69e6f9c0 0x55bb69e71de0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=432e0ddd7c69a609 server_cookie=828ea197b9ed8dc1 in_seq=0 out_seq=0 2024-05-08T22:08:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.357+0000 7f6350c00700 10 client.15675 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:08:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.357+0000 7f6350c00700 1 -- 172.21.15.131:0/355234516 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bb69b9dcf0 con 0x55bb69e6f9c0 2024-05-08T22:08:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.357+0000 7f6350c00700 10 client.15675 handle_command_reply: tid=0 2024-05-08T22:08:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.357+0000 7f6381a31b80 1 client.15675 shutdown 2024-05-08T22:08:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.357+0000 7f6381a31b80 1 -- 172.21.15.131:0/355234516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb69dff890 msgr2=0x55bb69e10520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.357+0000 7f6381a31b80 1 --2- 172.21.15.131:0/355234516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb69dff890 0x55bb69e10520 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6358011db0 tx=0x7f6358011de0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.357+0000 7f6381a31b80 1 -- 172.21.15.131:0/355234516 shutdown_connections 2024-05-08T22:08:01.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.357+0000 7f6381a31b80 1 --2- 172.21.15.131:0/355234516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb69e05970 0x55bb69e01020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.357+0000 7f6381a31b80 1 --2- 172.21.15.131:0/355234516 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55bb69e6f9c0 0x55bb69e71de0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.357+0000 7f6381a31b80 1 --2- 172.21.15.131:0/355234516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb69dff890 0x55bb69e10520 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.357+0000 7f6381a31b80 1 --2- 172.21.15.131:0/355234516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb69e05050 0x55bb69e00ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.357+0000 7f6350c00700 0 client.15675 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:08:01.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.357+0000 7f6381a31b80 1 -- 172.21.15.131:0/355234516 >> 172.21.15.131:0/355234516 conn(0x55bb69d5d210 msgr2=0x55bb69dffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:01.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.357+0000 7f6381a31b80 1 -- 172.21.15.131:0/355234516 shutdown_connections 2024-05-08T22:08:01.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.358+0000 7f6381a31b80 1 -- 172.21.15.131:0/355234516 wait complete. 2024-05-08T22:08:01.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.358+0000 7f6381a31b80 20 client.15675 trim_cache size 0 max 16384 2024-05-08T22:08:01.363 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:08:01.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.358+0000 7f636ca00700 1 -- 172.21.15.131:0/3033351347 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f634c0a14c0 msgr2=0x7f634c0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.358+0000 7f636ca00700 1 --2- 172.21.15.131:0/3033351347 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f634c0a14c0 0x7f634c0a3980 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f636000b780 tx=0x7f6360006210 comp rx=0 tx=0).stop 2024-05-08T22:08:01.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.358+0000 7f636ca00700 1 -- 172.21.15.131:0/3033351347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63681060b0 msgr2=0x7f6368199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.358+0000 7f636ca00700 1 --2- 172.21.15.131:0/3033351347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63681060b0 0x7f6368199b50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f635400e9f0 tx=0x7f635400ed00 comp rx=0 tx=0).stop 2024-05-08T22:08:01.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.358+0000 7f636ca00700 1 -- 172.21.15.131:0/3033351347 shutdown_connections 2024-05-08T22:08:01.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.358+0000 7f636ca00700 1 --2- 172.21.15.131:0/3033351347 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f634c0a14c0 0x7f634c0a3980 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.358+0000 7f636ca00700 1 --2- 172.21.15.131:0/3033351347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f636819dea0 0x7f636819e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.358+0000 7f636ca00700 1 --2- 172.21.15.131:0/3033351347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6368100f90 0x7f6368199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.358+0000 7f636ca00700 1 --2- 172.21.15.131:0/3033351347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63681060b0 0x7f6368199b50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.358+0000 7f636ca00700 1 -- 172.21.15.131:0/3033351347 >> 172.21.15.131:0/3033351347 conn(0x7f63680fc810 msgr2=0x7f63680fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:01.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.358+0000 7f636ca00700 1 -- 172.21.15.131:0/3033351347 shutdown_connections 2024-05-08T22:08:01.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.359+0000 7f636ca00700 1 -- 172.21.15.131:0/3033351347 wait complete. 2024-05-08T22:08:01.375 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:01 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3250505628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:08:01.378 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T22:08:01.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.576+0000 7fd2f4400700 1 -- 172.21.15.131:0/1343562402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2ec101180 msgr2=0x7fd2ec101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.576+0000 7fd2f4400700 1 --2- 172.21.15.131:0/1343562402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2ec101180 0x7fd2ec101560 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0009990 tx=0x7fd2e0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.576+0000 7fd2f4400700 1 -- 172.21.15.131:0/1343562402 shutdown_connections 2024-05-08T22:08:01.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.576+0000 7fd2f4400700 1 --2- 172.21.15.131:0/1343562402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2ec1062a0 0x7fd2ec10de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.576+0000 7fd2f4400700 1 --2- 172.21.15.131:0/1343562402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2ec101180 0x7fd2ec101560 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.576+0000 7fd2f4400700 1 --2- 172.21.15.131:0/1343562402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2ec101b30 0x7fd2ec105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.576+0000 7fd2f4400700 1 -- 172.21.15.131:0/1343562402 >> 172.21.15.131:0/1343562402 conn(0x7fd2ec0fc9e0 msgr2=0x7fd2ec0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:01.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.576+0000 7fd2f4400700 1 -- 172.21.15.131:0/1343562402 shutdown_connections 2024-05-08T22:08:01.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.576+0000 7fd2f4400700 1 -- 172.21.15.131:0/1343562402 wait complete. 2024-05-08T22:08:01.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.577+0000 7fd2f4400700 1 Processor -- start 2024-05-08T22:08:01.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.577+0000 7fd2f4400700 1 -- start start 2024-05-08T22:08:01.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.578+0000 7fd2f4400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2ec101180 0x7fd2ec199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.578+0000 7fd2f4400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2ec1062a0 0x7fd2ec199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.578+0000 7fd2f4400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2ec19df40 0x7fd2ec19e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.578+0000 7fd2f4400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec19e9c0 con 0x7fd2ec19df40 2024-05-08T22:08:01.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.578+0000 7fd2f4400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec19eb30 con 0x7fd2ec1062a0 2024-05-08T22:08:01.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.578+0000 7fd2f4400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec19ecf0 con 0x7fd2ec101180 2024-05-08T22:08:01.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.578+0000 7fd2f1a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2ec101180 0x7fd2ec199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.578+0000 7fd2f1a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2ec101180 0x7fd2ec199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46800/0 (socket says 172.21.15.131:46800) 2024-05-08T22:08:01.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.578+0000 7fd2f1a00700 1 -- 172.21.15.131:0/254929562 learned_addr learned my addr 172.21.15.131:0/254929562 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:01.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.578+0000 7fd2f2400700 1 --2- 172.21.15.131:0/254929562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2ec19df40 0x7fd2ec19e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.578+0000 7fd2f1000700 1 --2- 172.21.15.131:0/254929562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2ec1062a0 0x7fd2ec199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.578+0000 7fd2f1a00700 1 -- 172.21.15.131:0/254929562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2ec1062a0 msgr2=0x7fd2ec199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.578+0000 7fd2f1a00700 1 --2- 172.21.15.131:0/254929562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2ec1062a0 0x7fd2ec199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.578+0000 7fd2f1a00700 1 -- 172.21.15.131:0/254929562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2ec19df40 msgr2=0x7fd2ec19e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.578+0000 7fd2f1a00700 1 --2- 172.21.15.131:0/254929562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2ec19df40 0x7fd2ec19e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.578+0000 7fd2f1a00700 1 -- 172.21.15.131:0/254929562 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e00095f0 con 0x7fd2ec101180 2024-05-08T22:08:01.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.579+0000 7fd2f1a00700 1 --2- 172.21.15.131:0/254929562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2ec101180 0x7fd2ec199660 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0005910 tx=0x7fd2e0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:01.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.579+0000 7fd2ea000700 1 -- 172.21.15.131:0/254929562 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e001d070 con 0x7fd2ec101180 2024-05-08T22:08:01.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.579+0000 7fd2ea000700 1 -- 172.21.15.131:0/254929562 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2e0022470 con 0x7fd2ec101180 2024-05-08T22:08:01.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.579+0000 7fd2f4400700 1 -- 172.21.15.131:0/254929562 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2ec1a61f0 con 0x7fd2ec101180 2024-05-08T22:08:01.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.579+0000 7fd2ea000700 1 -- 172.21.15.131:0/254929562 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0017650 con 0x7fd2ec101180 2024-05-08T22:08:01.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.579+0000 7fd2f4400700 1 -- 172.21.15.131:0/254929562 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ec1a6610 con 0x7fd2ec101180 2024-05-08T22:08:01.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.582+0000 7fd2ea000700 1 -- 172.21.15.131:0/254929562 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 24) v1 ==== 136158+0+0 (secure 0 0 0) 0x7fd2e00225e0 con 0x7fd2ec101180 2024-05-08T22:08:01.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.583+0000 7fd2ea000700 1 --2- 172.21.15.131:0/254929562 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fd2d40a1440 0x7fd2d40a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.583+0000 7fd2ea000700 1 -- 172.21.15.131:0/254929562 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fd2e00ce5a0 con 0x7fd2ec101180 2024-05-08T22:08:01.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.583+0000 7fd2f1000700 1 --2- 172.21.15.131:0/254929562 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fd2d40a1440 0x7fd2d40a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.583+0000 7fd307f23b80 1 Processor -- start 2024-05-08T22:08:01.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.584+0000 7fd307f23b80 1 -- start start 2024-05-08T22:08:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.584+0000 7fd307f23b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f259e00 0x55f45f25a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.584+0000 7fd307f23b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f45f25a720 0x55f45f254350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.584+0000 7fd307f23b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f45f254890 0x55f45f254cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.584+0000 7fd307f23b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f45f1ab150 con 0x55f45f25a720 2024-05-08T22:08:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.584+0000 7fd307f23b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f45f1a8b10 con 0x55f45f259e00 2024-05-08T22:08:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.584+0000 7fd307f23b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f45f1a4400 con 0x55f45f254890 2024-05-08T22:08:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.584+0000 7fd2f1000700 1 --2- 172.21.15.131:0/254929562 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fd2d40a1440 0x7fd2d40a3900 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec19abb0 tx=0x7fd2d8009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.584+0000 7fd2f1a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f259e00 0x55f45f25a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.584+0000 7fd2f1000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f45f25a720 0x55f45f254350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.584+0000 7fd2f1a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f259e00 0x55f45f25a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:35202/0 (socket says 172.21.15.131:35202) 2024-05-08T22:08:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.584+0000 7fd2f1a00700 1 -- 172.21.15.131:0/1259598487 learned_addr learned my addr 172.21.15.131:0/1259598487 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.584+0000 7fd2f2400700 1 --2- 172.21.15.131:0/1259598487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f45f254890 0x55f45f254cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.585+0000 7fd2f1a00700 1 -- 172.21.15.131:0/1259598487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f45f254890 msgr2=0x55f45f254cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:01.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.585+0000 7fd2f1a00700 1 --2- 172.21.15.131:0/1259598487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f45f254890 0x55f45f254cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.585+0000 7fd2f1a00700 1 -- 172.21.15.131:0/1259598487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f45f25a720 msgr2=0x55f45f254350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:01.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.585+0000 7fd2f1a00700 1 --2- 172.21.15.131:0/1259598487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f45f25a720 0x55f45f254350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.585+0000 7fd2f1a00700 1 -- 172.21.15.131:0/1259598487 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f45f0253b0 con 0x55f45f259e00 2024-05-08T22:08:01.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.585+0000 7fd2f1a00700 1 --2- 172.21.15.131:0/1259598487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f259e00 0x55f45f25a1e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd2e000f740 tx=0x7fd2e0009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba3f17361f20657d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:01.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.585+0000 7fd2dea00700 1 -- 172.21.15.131:0/1259598487 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e009abf0 con 0x55f45f259e00 2024-05-08T22:08:01.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.585+0000 7fd2dea00700 1 -- 172.21.15.131:0/1259598487 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2e009b990 con 0x55f45f259e00 2024-05-08T22:08:01.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.585+0000 7fd2dea00700 1 -- 172.21.15.131:0/1259598487 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e009a9e0 con 0x55f45f259e00 2024-05-08T22:08:01.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.586+0000 7fd307f23b80 1 -- 172.21.15.131:0/1259598487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f259e00 msgr2=0x55f45f25a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.586+0000 7fd307f23b80 1 --2- 172.21.15.131:0/1259598487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f259e00 0x55f45f25a1e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd2e000f740 tx=0x7fd2e0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.586+0000 7fd307f23b80 1 -- 172.21.15.131:0/1259598487 shutdown_connections 2024-05-08T22:08:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.586+0000 7fd307f23b80 1 --2- 172.21.15.131:0/1259598487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f45f254890 0x55f45f254cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.586+0000 7fd307f23b80 1 --2- 172.21.15.131:0/1259598487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f259e00 0x55f45f25a1e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.586+0000 7fd307f23b80 1 --2- 172.21.15.131:0/1259598487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f45f25a720 0x55f45f254350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.586+0000 7fd307f23b80 1 -- 172.21.15.131:0/1259598487 >> 172.21.15.131:0/1259598487 conn(0x55f45f1b2210 msgr2=0x55f45f24edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.586+0000 7fd307f23b80 1 -- 172.21.15.131:0/1259598487 shutdown_connections 2024-05-08T22:08:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.586+0000 7fd307f23b80 1 -- 172.21.15.131:0/1259598487 wait complete. 2024-05-08T22:08:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.587+0000 7fd307f23b80 1 -- start start 2024-05-08T22:08:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.587+0000 7fd307f23b80 1 Processor -- start 2024-05-08T22:08:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.587+0000 7fd307f23b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f254890 0x55f45f260760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.587+0000 7fd307f23b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f45f259e00 0x55f45f25da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.587+0000 7fd307f23b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f45f25a720 0x55f45f25e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.587+0000 7fd307f23b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f45f140560 con 0x55f45f259e00 2024-05-08T22:08:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.587+0000 7fd307f23b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f45f140200 con 0x55f45f254890 2024-05-08T22:08:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.587+0000 7fd307f23b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f45f0276a0 con 0x55f45f25a720 2024-05-08T22:08:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.587+0000 7fd307f23b80 10 client.0 _ll_register_callbacks cb 0x55f45f149b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.588+0000 7fd2f1000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f45f259e00 0x55f45f25da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.588+0000 7fd2f1a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f254890 0x55f45f260760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.588+0000 7fd2f2400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f45f25a720 0x55f45f25e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.588+0000 7fd2f1000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f45f259e00 0x55f45f25da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35316/0 (socket says 172.21.15.131:35316) 2024-05-08T22:08:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.588+0000 7fd2f1000700 1 -- 172.21.15.131:0/2254767968 learned_addr learned my addr 172.21.15.131:0/2254767968 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.588+0000 7fd2f1a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f254890 0x55f45f260760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:35212/0 (socket says 172.21.15.131:35212) 2024-05-08T22:08:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.588+0000 7fd2f1000700 1 -- 172.21.15.131:0/2254767968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f45f25a720 msgr2=0x55f45f25e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.588+0000 7fd2f1000700 1 --2- 172.21.15.131:0/2254767968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f45f25a720 0x55f45f25e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.588+0000 7fd2f1000700 1 -- 172.21.15.131:0/2254767968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f254890 msgr2=0x55f45f260760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.588+0000 7fd2f1000700 1 --2- 172.21.15.131:0/2254767968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f254890 0x55f45f260760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.588+0000 7fd2f1000700 1 -- 172.21.15.131:0/2254767968 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ec1a6610 con 0x55f45f259e00 2024-05-08T22:08:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.588+0000 7fd2f1a00700 1 --2- 172.21.15.131:0/2254767968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f254890 0x55f45f260760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.588+0000 7fd2f2400700 1 --2- 172.21.15.131:0/2254767968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f45f25a720 0x55f45f25e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.588+0000 7fd2f1000700 1 --2- 172.21.15.131:0/2254767968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f45f259e00 0x55f45f25da50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd2d801d830 tx=0x7fd2d80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a00e196ffb1a1a18 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.588+0000 7fd2d3400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:08:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.589+0000 7fd2d3400700 1 -- 172.21.15.131:0/2254767968 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8022070 con 0x55f45f259e00 2024-05-08T22:08:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.589+0000 7fd2d3400700 1 -- 172.21.15.131:0/2254767968 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fd2d8004830 con 0x55f45f259e00 2024-05-08T22:08:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.589+0000 7fd307f23b80 1 -- 172.21.15.131:0/2254767968 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f45f0fd270 con 0x55f45f259e00 2024-05-08T22:08:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.589+0000 7fd2d3400700 1 -- 172.21.15.131:0/2254767968 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2d80049f0 con 0x55f45f259e00 2024-05-08T22:08:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.589+0000 7fd307f23b80 10 client.15687 fetch_fsmap learned FSMap version 24 2024-05-08T22:08:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.589+0000 7fd307f23b80 1 -- 172.21.15.131:0/2254767968 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f45efd1b60 con 0x55f45f259e00 2024-05-08T22:08:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.589+0000 7fd2d3400700 1 -- 172.21.15.131:0/2254767968 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f45f0fd270 con 0x55f45f259e00 2024-05-08T22:08:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.589+0000 7fd2d3400700 1 -- 172.21.15.131:0/2254767968 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd2d801c770 con 0x55f45f259e00 2024-05-08T22:08:01.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.589+0000 7fd307f23b80 10 client.15687 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:08:01.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.589+0000 7fd307f23b80 10 client.15687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:08:01.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.589+0000 7fd307f23b80 20 client.15687 populate_metadata read hostname 'smithi131' 2024-05-08T22:08:01.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.589+0000 7fd307f23b80 1 --2- 172.21.15.131:0/2254767968 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55f45f26def0 0x55f45f270310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.589+0000 7fd307f23b80 4 client.15687 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:08:01.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.589+0000 7fd307f23b80 1 -- 172.21.15.131:0/2254767968 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f45f223810 con 0x55f45f26def0 2024-05-08T22:08:01.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.590+0000 7fd2f1a00700 1 --2- 172.21.15.131:0/2254767968 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55f45f26def0 0x55f45f270310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.595 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:01 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3250505628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:08:01.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.590+0000 7fd2f1a00700 1 --2- 172.21.15.131:0/2254767968 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55f45f26def0 0x55f45f270310 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5538bd11ab283b38 server_cookie=e7ec470b72ab2bdb in_seq=0 out_seq=0 2024-05-08T22:08:01.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.590+0000 7fd2d3400700 10 client.15687 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:01.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.591+0000 7fd2d3400700 1 -- 172.21.15.131:0/2254767968 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f45f223810 con 0x55f45f26def0 2024-05-08T22:08:01.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.591+0000 7fd2d3400700 10 client.15687 handle_command_reply: tid=0 2024-05-08T22:08:01.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.591+0000 7fd307f23b80 1 client.15687 shutdown 2024-05-08T22:08:01.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.591+0000 7fd307f23b80 1 -- 172.21.15.131:0/2254767968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f45f259e00 msgr2=0x55f45f25da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.591+0000 7fd307f23b80 1 --2- 172.21.15.131:0/2254767968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f45f259e00 0x55f45f25da50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd2d801d830 tx=0x7fd2d80047c0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.591+0000 7fd307f23b80 1 -- 172.21.15.131:0/2254767968 shutdown_connections 2024-05-08T22:08:01.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.591+0000 7fd307f23b80 1 --2- 172.21.15.131:0/2254767968 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55f45f26def0 0x55f45f270310 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.591+0000 7fd307f23b80 1 --2- 172.21.15.131:0/2254767968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f45f25a720 0x55f45f25e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.591+0000 7fd307f23b80 1 --2- 172.21.15.131:0/2254767968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f254890 0x55f45f260760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.591+0000 7fd307f23b80 1 --2- 172.21.15.131:0/2254767968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f45f259e00 0x55f45f25da50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.591+0000 7fd307f23b80 1 -- 172.21.15.131:0/2254767968 >> 172.21.15.131:0/2254767968 conn(0x55f45f1b2210 msgr2=0x55f45f24e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:01.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.591+0000 7fd2d3400700 0 client.15687 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:01.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.591+0000 7fd307f23b80 1 -- 172.21.15.131:0/2254767968 shutdown_connections 2024-05-08T22:08:01.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.591+0000 7fd307f23b80 1 -- 172.21.15.131:0/2254767968 wait complete. 2024-05-08T22:08:01.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.591+0000 7fd307f23b80 20 client.15687 trim_cache size 0 max 16384 2024-05-08T22:08:01.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.605+0000 7fd307f23b80 1 Processor -- start 2024-05-08T22:08:01.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.605+0000 7fd307f23b80 1 -- start start 2024-05-08T22:08:01.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.605+0000 7fd307f23b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f45f254890 0x55f45f254350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.605+0000 7fd307f23b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f259e00 0x55f45f24ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.605+0000 7fd307f23b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f45f25a720 0x55f45f264ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.605+0000 7fd307f23b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f45f26b570 con 0x55f45f254890 2024-05-08T22:08:01.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.605+0000 7fd307f23b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f45f24bdc0 con 0x55f45f259e00 2024-05-08T22:08:01.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.605+0000 7fd307f23b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f45f249a90 con 0x55f45f25a720 2024-05-08T22:08:01.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.605+0000 7fd2f1a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f45f254890 0x55f45f254350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.606+0000 7fd2f1a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f45f254890 0x55f45f254350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35320/0 (socket says 172.21.15.131:35320) 2024-05-08T22:08:01.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.606+0000 7fd2f1a00700 1 -- 172.21.15.131:0/980048746 learned_addr learned my addr 172.21.15.131:0/980048746 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:01.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.606+0000 7fd2f1000700 1 --2- 172.21.15.131:0/980048746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f259e00 0x55f45f24ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.606+0000 7fd2f2400700 1 --2- 172.21.15.131:0/980048746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f45f25a720 0x55f45f264ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.606+0000 7fd2f1a00700 1 -- 172.21.15.131:0/980048746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f45f25a720 msgr2=0x55f45f264ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:01.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.606+0000 7fd2f1a00700 1 --2- 172.21.15.131:0/980048746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f45f25a720 0x55f45f264ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.606+0000 7fd2f1a00700 1 -- 172.21.15.131:0/980048746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f259e00 msgr2=0x55f45f24ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:01.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.606+0000 7fd2f1a00700 1 --2- 172.21.15.131:0/980048746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f259e00 0x55f45f24ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.606+0000 7fd2f1a00700 1 -- 172.21.15.131:0/980048746 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f45f254f40 con 0x55f45f254890 2024-05-08T22:08:01.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.606+0000 7fd2f1a00700 1 --2- 172.21.15.131:0/980048746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f45f254890 0x55f45f254350 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0097d00 tx=0x7fd2e0022f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1677b94e426f14e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:01.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.606+0000 7fd2df400700 1 -- 172.21.15.131:0/980048746 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e00b7040 con 0x55f45f254890 2024-05-08T22:08:01.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.606+0000 7fd2df400700 1 -- 172.21.15.131:0/980048746 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2e009abf0 con 0x55f45f254890 2024-05-08T22:08:01.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.606+0000 7fd2df400700 1 -- 172.21.15.131:0/980048746 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e009b290 con 0x55f45f254890 2024-05-08T22:08:01.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.607+0000 7fd307f23b80 1 -- 172.21.15.131:0/980048746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f45f254890 msgr2=0x55f45f254350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.607+0000 7fd307f23b80 1 --2- 172.21.15.131:0/980048746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f45f254890 0x55f45f254350 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0097d00 tx=0x7fd2e0022f90 comp rx=0 tx=0).stop 2024-05-08T22:08:01.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.607+0000 7fd307f23b80 1 -- 172.21.15.131:0/980048746 shutdown_connections 2024-05-08T22:08:01.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.607+0000 7fd307f23b80 1 --2- 172.21.15.131:0/980048746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f45f25a720 0x55f45f264ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.607+0000 7fd307f23b80 1 --2- 172.21.15.131:0/980048746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f45f254890 0x55f45f254350 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.607+0000 7fd307f23b80 1 --2- 172.21.15.131:0/980048746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f259e00 0x55f45f24ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.607+0000 7fd307f23b80 1 -- 172.21.15.131:0/980048746 >> 172.21.15.131:0/980048746 conn(0x55f45f1b2210 msgr2=0x55f45f269990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:01.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.607+0000 7fd307f23b80 1 -- 172.21.15.131:0/980048746 shutdown_connections 2024-05-08T22:08:01.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.607+0000 7fd307f23b80 1 -- 172.21.15.131:0/980048746 wait complete. 2024-05-08T22:08:01.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.607+0000 7fd307f23b80 1 -- start start 2024-05-08T22:08:01.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.608+0000 7fd307f23b80 1 Processor -- start 2024-05-08T22:08:01.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.608+0000 7fd307f23b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f45f254890 0x55f45f263110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.608+0000 7fd307f23b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f45f259e00 0x55f45f263650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.608+0000 7fd307f23b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f25a720 0x55f45f2c33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.608+0000 7fd307f23b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f45f263d80 con 0x55f45f254890 2024-05-08T22:08:01.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.608+0000 7fd307f23b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f45f2c3930 con 0x55f45f25a720 2024-05-08T22:08:01.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.608+0000 7fd307f23b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f45f2c3ac0 con 0x55f45f259e00 2024-05-08T22:08:01.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.608+0000 7fd307f23b80 10 client.0 _ll_register_callbacks cb 0x55f45f149b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:01.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.608+0000 7fd2f1a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f45f254890 0x55f45f263110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.608+0000 7fd2f2400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f25a720 0x55f45f2c33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.608+0000 7fd2f2400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f25a720 0x55f45f2c33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:35228/0 (socket says 172.21.15.131:35228) 2024-05-08T22:08:01.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.608+0000 7fd2f2400700 1 -- 172.21.15.131:0/1573166983 learned_addr learned my addr 172.21.15.131:0/1573166983 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:01.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.608+0000 7fd2f1000700 1 --2- 172.21.15.131:0/1573166983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f45f259e00 0x55f45f263650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.609+0000 7fd2f2400700 1 -- 172.21.15.131:0/1573166983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f45f259e00 msgr2=0x55f45f263650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:01.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.609+0000 7fd2f2400700 1 --2- 172.21.15.131:0/1573166983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f45f259e00 0x55f45f263650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.609+0000 7fd2f2400700 1 -- 172.21.15.131:0/1573166983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f45f254890 msgr2=0x55f45f263110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:01.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.609+0000 7fd2f2400700 1 --2- 172.21.15.131:0/1573166983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f45f254890 0x55f45f263110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.609+0000 7fd2f2400700 1 -- 172.21.15.131:0/1573166983 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f45f0253b0 con 0x55f45f25a720 2024-05-08T22:08:01.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.609+0000 7fd2f1000700 1 --2- 172.21.15.131:0/1573166983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f45f259e00 0x55f45f263650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:08:01.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.609+0000 7fd2f1a00700 1 --2- 172.21.15.131:0/1573166983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f45f254890 0x55f45f263110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:01.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.609+0000 7fd2f2400700 1 --2- 172.21.15.131:0/1573166983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f25a720 0x55f45f2c33f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4009790 tx=0x7fd2e40077e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8145abe605524b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:01.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.609+0000 7fd2d3400700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:08:01.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.609+0000 7fd2d3400700 1 -- 172.21.15.131:0/1573166983 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e401c070 con 0x55f45f25a720 2024-05-08T22:08:01.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.609+0000 7fd307f23b80 1 -- 172.21.15.131:0/1573166983 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2e009b990 con 0x55f45f25a720 2024-05-08T22:08:01.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.609+0000 7fd2d3400700 1 -- 172.21.15.131:0/1573166983 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fd2e4003a00 con 0x55f45f25a720 2024-05-08T22:08:01.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.609+0000 7fd2d3400700 1 -- 172.21.15.131:0/1573166983 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2e400e040 con 0x55f45f25a720 2024-05-08T22:08:01.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.609+0000 7fd307f23b80 10 client.25733 fetch_fsmap learned FSMap version 24 2024-05-08T22:08:01.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.610+0000 7fd307f23b80 1 -- 172.21.15.131:0/1573166983 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f45f16a560 con 0x55f45f25a720 2024-05-08T22:08:01.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.610+0000 7fd2d3400700 1 -- 172.21.15.131:0/1573166983 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2e009b990 con 0x55f45f25a720 2024-05-08T22:08:01.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.610+0000 7fd2d3400700 1 -- 172.21.15.131:0/1573166983 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd2e4016400 con 0x55f45f25a720 2024-05-08T22:08:01.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.610+0000 7fd307f23b80 10 client.25733 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:08:01.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.610+0000 7fd307f23b80 10 client.25733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:08:01.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.610+0000 7fd307f23b80 20 client.25733 populate_metadata read hostname 'smithi131' 2024-05-08T22:08:01.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.610+0000 7fd307f23b80 1 --2- 172.21.15.131:0/1573166983 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55f45f2c5360 0x55f45f2c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:01.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.610+0000 7fd307f23b80 4 client.25733 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:08:01.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.610+0000 7fd307f23b80 1 -- 172.21.15.131:0/1573166983 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f45f19cec0 con 0x55f45f2c5360 2024-05-08T22:08:01.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.610+0000 7fd2f1a00700 1 --2- 172.21.15.131:0/1573166983 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55f45f2c5360 0x55f45f2c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:01.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.611+0000 7fd2f1a00700 1 --2- 172.21.15.131:0/1573166983 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55f45f2c5360 0x55f45f2c7780 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d3947d2c2512237 server_cookie=8699d852222cd350 in_seq=0 out_seq=0 2024-05-08T22:08:01.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.611+0000 7fd2d3400700 10 client.25733 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:01.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.611+0000 7fd2d3400700 1 -- 172.21.15.131:0/1573166983 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f45f19cec0 con 0x55f45f2c5360 2024-05-08T22:08:01.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.611+0000 7fd2d3400700 10 client.25733 handle_command_reply: tid=0 2024-05-08T22:08:01.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.611+0000 7fd307f23b80 1 client.25733 shutdown 2024-05-08T22:08:01.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.611+0000 7fd307f23b80 1 -- 172.21.15.131:0/1573166983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f25a720 msgr2=0x55f45f2c33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.611+0000 7fd307f23b80 1 --2- 172.21.15.131:0/1573166983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f25a720 0x55f45f2c33f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4009790 tx=0x7fd2e40077e0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.611+0000 7fd307f23b80 1 -- 172.21.15.131:0/1573166983 shutdown_connections 2024-05-08T22:08:01.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.611+0000 7fd307f23b80 1 --2- 172.21.15.131:0/1573166983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f45f25a720 0x55f45f2c33f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.611+0000 7fd307f23b80 1 --2- 172.21.15.131:0/1573166983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f45f254890 0x55f45f263110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.611+0000 7fd307f23b80 1 --2- 172.21.15.131:0/1573166983 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55f45f2c5360 0x55f45f2c7780 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.611+0000 7fd307f23b80 1 --2- 172.21.15.131:0/1573166983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f45f259e00 0x55f45f263650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.611+0000 7fd2d3400700 0 client.25733 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:01.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.611+0000 7fd307f23b80 1 -- 172.21.15.131:0/1573166983 >> 172.21.15.131:0/1573166983 conn(0x55f45f1b2210 msgr2=0x55f45f25c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:01.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.611+0000 7fd307f23b80 1 -- 172.21.15.131:0/1573166983 shutdown_connections 2024-05-08T22:08:01.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.611+0000 7fd307f23b80 1 -- 172.21.15.131:0/1573166983 wait complete. 2024-05-08T22:08:01.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.612+0000 7fd307f23b80 20 client.25733 trim_cache size 0 max 16384 2024-05-08T22:08:01.620 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:08:01.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.612+0000 7fd2f4400700 1 -- 172.21.15.131:0/254929562 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fd2d40a1440 msgr2=0x7fd2d40a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.612+0000 7fd2f4400700 1 --2- 172.21.15.131:0/254929562 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fd2d40a1440 0x7fd2d40a3900 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec19abb0 tx=0x7fd2d8009260 comp rx=0 tx=0).stop 2024-05-08T22:08:01.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.612+0000 7fd2f4400700 1 -- 172.21.15.131:0/254929562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2ec101180 msgr2=0x7fd2ec199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:01.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.612+0000 7fd2f4400700 1 --2- 172.21.15.131:0/254929562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2ec101180 0x7fd2ec199660 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0005910 tx=0x7fd2e0004730 comp rx=0 tx=0).stop 2024-05-08T22:08:01.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.612+0000 7fd2f4400700 1 -- 172.21.15.131:0/254929562 shutdown_connections 2024-05-08T22:08:01.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.612+0000 7fd2f4400700 1 --2- 172.21.15.131:0/254929562 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fd2d40a1440 0x7fd2d40a3900 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.612+0000 7fd2f4400700 1 --2- 172.21.15.131:0/254929562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2ec19df40 0x7fd2ec19e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.612+0000 7fd2f4400700 1 --2- 172.21.15.131:0/254929562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2ec101180 0x7fd2ec199660 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.612+0000 7fd2f4400700 1 --2- 172.21.15.131:0/254929562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2ec1062a0 0x7fd2ec199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:01.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.612+0000 7fd2f4400700 1 -- 172.21.15.131:0/254929562 >> 172.21.15.131:0/254929562 conn(0x7fd2ec0fc9e0 msgr2=0x7fd2ec0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:01.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.612+0000 7fd2f4400700 1 -- 172.21.15.131:0/254929562 shutdown_connections 2024-05-08T22:08:01.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:01.612+0000 7fd2f4400700 1 -- 172.21.15.131:0/254929562 wait complete. 2024-05-08T22:08:01.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:01 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3250505628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:08:02.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:02 smithi132 ceph-mon[78180]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:08:02.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:02 smithi132 ceph-mon[78180]: mgrmap e24: x(active, since 44s), standbys: y, z 2024-05-08T22:08:02.632 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T22:08:02.633 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T22:08:02.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:02 smithi131 ceph-mon[37260]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:08:02.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:02 smithi131 ceph-mon[37260]: mgrmap e24: x(active, since 44s), standbys: y, z 2024-05-08T22:08:02.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:02 smithi157 ceph-mon[77746]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:08:02.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:02 smithi157 ceph-mon[77746]: mgrmap e24: x(active, since 44s), standbys: y, z 2024-05-08T22:08:02.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.840+0000 7f3136800700 1 -- 172.21.15.131:0/1653927757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31301031c0 msgr2=0x7f31301035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:02.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.840+0000 7f3136800700 1 --2- 172.21.15.131:0/1653927757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31301031c0 0x7f31301035a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3124009990 tx=0x7f3124009ca0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.840+0000 7f3136800700 1 -- 172.21.15.131:0/1653927757 shutdown_connections 2024-05-08T22:08:02.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.840+0000 7f3136800700 1 --2- 172.21.15.131:0/1653927757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31301082e0 0x7f313010fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.840+0000 7f3136800700 1 --2- 172.21.15.131:0/1653927757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31301031c0 0x7f31301035a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.840+0000 7f3136800700 1 --2- 172.21.15.131:0/1653927757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3130103b70 0x7f3130107bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.840+0000 7f3136800700 1 -- 172.21.15.131:0/1653927757 >> 172.21.15.131:0/1653927757 conn(0x7f31300fea00 msgr2=0x7f3130100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:02.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.840+0000 7f3136800700 1 -- 172.21.15.131:0/1653927757 shutdown_connections 2024-05-08T22:08:02.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.841+0000 7f3136800700 1 -- 172.21.15.131:0/1653927757 wait complete. 2024-05-08T22:08:02.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.841+0000 7f3136800700 1 Processor -- start 2024-05-08T22:08:02.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.842+0000 7f3136800700 1 -- start start 2024-05-08T22:08:02.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.842+0000 7f3136800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31301031c0 0x7f3130199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:02.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.842+0000 7f3136800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31301082e0 0x7f3130199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:02.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.842+0000 7f3136800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f313019ddf0 0x7f313019e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:02.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.842+0000 7f3136800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313019e870 con 0x7f31301082e0 2024-05-08T22:08:02.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.842+0000 7f3136800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313019e9e0 con 0x7f313019ddf0 2024-05-08T22:08:02.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.842+0000 7f3136800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313019eba0 con 0x7f31301031c0 2024-05-08T22:08:02.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.842+0000 7f3135400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31301031c0 0x7f3130199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:02.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.842+0000 7f3134a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31301082e0 0x7f3130199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:02.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.842+0000 7f3135400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31301031c0 0x7f3130199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46878/0 (socket says 172.21.15.131:46878) 2024-05-08T22:08:02.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.842+0000 7f3135400700 1 -- 172.21.15.131:0/2966134032 learned_addr learned my addr 172.21.15.131:0/2966134032 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:02.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.842+0000 7f3135e00700 1 --2- 172.21.15.131:0/2966134032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f313019ddf0 0x7f313019e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:02.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.843+0000 7f3135400700 1 -- 172.21.15.131:0/2966134032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f313019ddf0 msgr2=0x7f313019e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:02.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.843+0000 7f3135400700 1 --2- 172.21.15.131:0/2966134032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f313019ddf0 0x7f313019e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.843+0000 7f3135400700 1 -- 172.21.15.131:0/2966134032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31301082e0 msgr2=0x7f3130199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:02.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.843+0000 7f3135400700 1 --2- 172.21.15.131:0/2966134032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31301082e0 0x7f3130199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.843+0000 7f3135400700 1 -- 172.21.15.131:0/2966134032 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31240095f0 con 0x7f31301031c0 2024-05-08T22:08:02.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.843+0000 7f3134a00700 1 --2- 172.21.15.131:0/2966134032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31301082e0 0x7f3130199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:02.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.843+0000 7f3135400700 1 --2- 172.21.15.131:0/2966134032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31301031c0 0x7f3130199510 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3124005910 tx=0x7f3124004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:02.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.843+0000 7f312e000700 1 -- 172.21.15.131:0/2966134032 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312401d070 con 0x7f31301031c0 2024-05-08T22:08:02.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.843+0000 7f3136800700 1 -- 172.21.15.131:0/2966134032 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31301a60a0 con 0x7f31301031c0 2024-05-08T22:08:02.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.843+0000 7f312e000700 1 -- 172.21.15.131:0/2966134032 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3124022470 con 0x7f31301031c0 2024-05-08T22:08:02.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.843+0000 7f312e000700 1 -- 172.21.15.131:0/2966134032 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3124017650 con 0x7f31301031c0 2024-05-08T22:08:02.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.843+0000 7f3136800700 1 -- 172.21.15.131:0/2966134032 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31301a6520 con 0x7f31301031c0 2024-05-08T22:08:02.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.847+0000 7f312e000700 1 -- 172.21.15.131:0/2966134032 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 24) v1 ==== 136158+0+0 (secure 0 0 0) 0x7f31240225e0 con 0x7f31301031c0 2024-05-08T22:08:02.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.848+0000 7f314b9f0b80 1 Processor -- start 2024-05-08T22:08:02.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.848+0000 7f314b9f0b80 1 -- start start 2024-05-08T22:08:02.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.848+0000 7f312e000700 1 --2- 172.21.15.131:0/2966134032 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f31180a14c0 0x7f31180a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:02.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.848+0000 7f314b9f0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d567ae0d0 0x564d567ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:02.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.848+0000 7f314b9f0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d567ae9f0 0x564d567a83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:02.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.848+0000 7f312e000700 1 -- 172.21.15.131:0/2966134032 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f31240ce520 con 0x7f31301031c0 2024-05-08T22:08:02.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.848+0000 7f314b9f0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d567a8910 0x564d567a8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:02.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.848+0000 7f314b9f0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d566ff1d0 con 0x564d567ae9f0 2024-05-08T22:08:02.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.848+0000 7f314b9f0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d566fcb90 con 0x564d567ae0d0 2024-05-08T22:08:02.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.848+0000 7f314b9f0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d566f8480 con 0x564d567a8910 2024-05-08T22:08:02.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.848+0000 7f3135e00700 1 --2- 172.21.15.131:0/2966134032 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f31180a14c0 0x7f31180a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:02.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.849+0000 7f3135400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d567ae0d0 0x564d567ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:02.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.849+0000 7f3134a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d567ae9f0 0x564d567a83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:02.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.849+0000 7f3135400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d567ae0d0 0x564d567ae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:35264/0 (socket says 172.21.15.131:35264) 2024-05-08T22:08:02.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.849+0000 7f3135400700 1 -- 172.21.15.131:0/2986016623 learned_addr learned my addr 172.21.15.131:0/2986016623 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:02.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.849+0000 7f3135e00700 1 --2- 172.21.15.131:0/2986016623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d567a8910 0x564d567a8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:02.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.849+0000 7f3135400700 1 -- 172.21.15.131:0/2986016623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d567a8910 msgr2=0x564d567a8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:02.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.849+0000 7f3135400700 1 --2- 172.21.15.131:0/2986016623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d567a8910 0x564d567a8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.849+0000 7f3135400700 1 -- 172.21.15.131:0/2986016623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d567ae9f0 msgr2=0x564d567a83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:02.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.849+0000 7f3135400700 1 --2- 172.21.15.131:0/2986016623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d567ae9f0 0x564d567a83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.849+0000 7f3135400700 1 -- 172.21.15.131:0/2986016623 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d56579430 con 0x564d567ae0d0 2024-05-08T22:08:02.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.849+0000 7f3135e00700 1 --2- 172.21.15.131:0/2966134032 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f31180a14c0 0x7f31180a3980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3128009ec0 tx=0x7f312800fc40 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:02.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.849+0000 7f3135400700 1 --2- 172.21.15.131:0/2986016623 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d567ae0d0 0x564d567ae4b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f312400f740 tx=0x7f3124009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96e250d9eccd5354 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:02.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.849+0000 7f3122600700 1 -- 172.21.15.131:0/2986016623 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312409aa50 con 0x564d567ae0d0 2024-05-08T22:08:02.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.849+0000 7f3122600700 1 -- 172.21.15.131:0/2986016623 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f312409b7f0 con 0x564d567ae0d0 2024-05-08T22:08:02.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.849+0000 7f3122600700 1 -- 172.21.15.131:0/2986016623 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31240229e0 con 0x564d567ae0d0 2024-05-08T22:08:02.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.850+0000 7f314b9f0b80 1 -- 172.21.15.131:0/2986016623 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d567ae0d0 msgr2=0x564d567ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:02.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.850+0000 7f314b9f0b80 1 --2- 172.21.15.131:0/2986016623 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d567ae0d0 0x564d567ae4b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f312400f740 tx=0x7f3124009310 comp rx=0 tx=0).stop 2024-05-08T22:08:02.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.850+0000 7f314b9f0b80 1 -- 172.21.15.131:0/2986016623 shutdown_connections 2024-05-08T22:08:02.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.850+0000 7f314b9f0b80 1 --2- 172.21.15.131:0/2986016623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d567a8910 0x564d567a8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.850+0000 7f314b9f0b80 1 --2- 172.21.15.131:0/2986016623 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d567ae0d0 0x564d567ae4b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.850+0000 7f314b9f0b80 1 --2- 172.21.15.131:0/2986016623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d567ae9f0 0x564d567a83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.850+0000 7f314b9f0b80 1 -- 172.21.15.131:0/2986016623 >> 172.21.15.131:0/2986016623 conn(0x564d56706290 msgr2=0x564d567a3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:02.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.850+0000 7f314b9f0b80 1 -- 172.21.15.131:0/2986016623 shutdown_connections 2024-05-08T22:08:02.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.850+0000 7f314b9f0b80 1 -- 172.21.15.131:0/2986016623 wait complete. 2024-05-08T22:08:02.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.851+0000 7f314b9f0b80 1 -- start start 2024-05-08T22:08:02.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.851+0000 7f314b9f0b80 1 Processor -- start 2024-05-08T22:08:02.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.851+0000 7f314b9f0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d567a8910 0x564d567b4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:02.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.851+0000 7f314b9f0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d567ae0d0 0x564d567b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:02.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.851+0000 7f314b9f0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d567ae9f0 0x564d567b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:02.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.851+0000 7f314b9f0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d566945e0 con 0x564d567ae9f0 2024-05-08T22:08:02.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.851+0000 7f314b9f0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d56694280 con 0x564d567ae0d0 2024-05-08T22:08:02.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.851+0000 7f314b9f0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d5657b720 con 0x564d567a8910 2024-05-08T22:08:02.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.852+0000 7f314b9f0b80 10 client.0 _ll_register_callbacks cb 0x564d5669dbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:02.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.852+0000 7f3135400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d567a8910 0x564d567b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:02.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.852+0000 7f3135e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d567ae9f0 0x564d567b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:02.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.852+0000 7f3135400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d567a8910 0x564d567b4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46904/0 (socket says 172.21.15.131:46904) 2024-05-08T22:08:02.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.852+0000 7f3135400700 1 -- 172.21.15.131:0/4274414637 learned_addr learned my addr 172.21.15.131:0/4274414637 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:02.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.852+0000 7f3134a00700 1 --2- 172.21.15.131:0/4274414637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d567ae0d0 0x564d567b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:02.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.852+0000 7f3135400700 1 -- 172.21.15.131:0/4274414637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d567ae0d0 msgr2=0x564d567b1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:02.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.852+0000 7f3135400700 1 --2- 172.21.15.131:0/4274414637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d567ae0d0 0x564d567b1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.852+0000 7f3135400700 1 -- 172.21.15.131:0/4274414637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d567ae9f0 msgr2=0x564d567b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:02.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.852+0000 7f3135400700 1 --2- 172.21.15.131:0/4274414637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d567ae9f0 0x564d567b22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.852+0000 7f3135400700 1 -- 172.21.15.131:0/4274414637 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31301a6520 con 0x564d567a8910 2024-05-08T22:08:02.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.852+0000 7f3135400700 1 --2- 172.21.15.131:0/4274414637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d567a8910 0x564d567b4950 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3124005cd0 tx=0x7f3124009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19e6623ce0ddf8da server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:02.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.852+0000 7f3116a00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:08:02.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.853+0000 7f3116a00700 1 -- 172.21.15.131:0/4274414637 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312409ef20 con 0x564d567a8910 2024-05-08T22:08:02.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.853+0000 7f314b9f0b80 1 -- 172.21.15.131:0/4274414637 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d566512f0 con 0x564d567a8910 2024-05-08T22:08:02.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.853+0000 7f3116a00700 1 -- 172.21.15.131:0/4274414637 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f312409fe40 con 0x564d567a8910 2024-05-08T22:08:02.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.853+0000 7f3116a00700 1 -- 172.21.15.131:0/4274414637 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f312409b0f0 con 0x564d567a8910 2024-05-08T22:08:02.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.853+0000 7f314b9f0b80 10 client.25759 fetch_fsmap learned FSMap version 24 2024-05-08T22:08:02.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.853+0000 7f314b9f0b80 1 -- 172.21.15.131:0/4274414637 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564d56525be0 con 0x564d567a8910 2024-05-08T22:08:02.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.853+0000 7f3116a00700 1 -- 172.21.15.131:0/4274414637 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564d566512f0 con 0x564d567a8910 2024-05-08T22:08:02.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.853+0000 7f3116a00700 1 -- 172.21.15.131:0/4274414637 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f31240a3050 con 0x564d567a8910 2024-05-08T22:08:02.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.853+0000 7f314b9f0b80 10 client.25759 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:08:02.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.853+0000 7f314b9f0b80 10 client.25759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:08:02.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.853+0000 7f314b9f0b80 20 client.25759 populate_metadata read hostname 'smithi131' 2024-05-08T22:08:02.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.853+0000 7f314b9f0b80 1 --2- 172.21.15.131:0/4274414637 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x564d567c1af0 0x564d567c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:02.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.853+0000 7f314b9f0b80 4 client.25759 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:08:02.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.853+0000 7f314b9f0b80 1 -- 172.21.15.131:0/4274414637 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d56777890 con 0x564d567c1af0 2024-05-08T22:08:02.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.853+0000 7f3134a00700 1 --2- 172.21.15.131:0/4274414637 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x564d567c1af0 0x564d567c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:02.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.854+0000 7f3116a00700 10 client.25759 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:02.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.854+0000 7f3134a00700 1 --2- 172.21.15.131:0/4274414637 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x564d567c1af0 0x564d567c3ee0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91cd1a04199bf241 server_cookie=5094da6df6cec70d in_seq=0 out_seq=0 2024-05-08T22:08:02.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.855+0000 7f3116a00700 1 -- 172.21.15.131:0/4274414637 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564d56777890 con 0x564d567c1af0 2024-05-08T22:08:02.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.855+0000 7f3116a00700 10 client.25759 handle_command_reply: tid=0 2024-05-08T22:08:02.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.855+0000 7f314b9f0b80 1 client.25759 shutdown 2024-05-08T22:08:02.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.855+0000 7f314b9f0b80 1 -- 172.21.15.131:0/4274414637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d567a8910 msgr2=0x564d567b4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:02.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.855+0000 7f314b9f0b80 1 --2- 172.21.15.131:0/4274414637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d567a8910 0x564d567b4950 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3124005cd0 tx=0x7f3124009ca0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.855+0000 7f314b9f0b80 1 -- 172.21.15.131:0/4274414637 shutdown_connections 2024-05-08T22:08:02.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.855+0000 7f314b9f0b80 1 --2- 172.21.15.131:0/4274414637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d567ae9f0 0x564d567b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.855+0000 7f314b9f0b80 1 --2- 172.21.15.131:0/4274414637 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x564d567c1af0 0x564d567c3ee0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.855+0000 7f314b9f0b80 1 --2- 172.21.15.131:0/4274414637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d567a8910 0x564d567b4950 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.855+0000 7f314b9f0b80 1 --2- 172.21.15.131:0/4274414637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d567ae0d0 0x564d567b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.855+0000 7f3116a00700 0 client.25759 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:02.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.855+0000 7f314b9f0b80 1 -- 172.21.15.131:0/4274414637 >> 172.21.15.131:0/4274414637 conn(0x564d56706290 msgr2=0x564d567a3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:02.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.855+0000 7f314b9f0b80 1 -- 172.21.15.131:0/4274414637 shutdown_connections 2024-05-08T22:08:02.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.855+0000 7f314b9f0b80 1 -- 172.21.15.131:0/4274414637 wait complete. 2024-05-08T22:08:02.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.855+0000 7f314b9f0b80 20 client.25759 trim_cache size 0 max 16384 2024-05-08T22:08:02.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.872+0000 7f314b9f0b80 1 Processor -- start 2024-05-08T22:08:02.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.872+0000 7f314b9f0b80 1 -- start start 2024-05-08T22:08:02.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.873+0000 7f314b9f0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d567a8910 0x564d567a83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:02.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.873+0000 7f314b9f0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d567ae0d0 0x564d567b4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:02.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.873+0000 7f314b9f0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d567ae9f0 0x564d567bda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:02.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.873+0000 7f314b9f0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d567c8f70 con 0x564d567ae9f0 2024-05-08T22:08:02.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.873+0000 7f314b9f0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d567ae7b0 con 0x564d567ae0d0 2024-05-08T22:08:02.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.873+0000 7f314b9f0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d5679b0c0 con 0x564d567a8910 2024-05-08T22:08:02.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.873+0000 7f3135400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d567a8910 0x564d567a83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:02.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.873+0000 7f3135e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d567ae9f0 0x564d567bda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:02.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.873+0000 7f3135400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d567a8910 0x564d567a83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46908/0 (socket says 172.21.15.131:46908) 2024-05-08T22:08:02.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.873+0000 7f3135400700 1 -- 172.21.15.131:0/3642806031 learned_addr learned my addr 172.21.15.131:0/3642806031 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:02.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.873+0000 7f3134a00700 1 --2- 172.21.15.131:0/3642806031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d567ae0d0 0x564d567b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:02.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.873+0000 7f3135400700 1 -- 172.21.15.131:0/3642806031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d567ae0d0 msgr2=0x564d567b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:02.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.873+0000 7f3135400700 1 --2- 172.21.15.131:0/3642806031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d567ae0d0 0x564d567b4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.873+0000 7f3135400700 1 -- 172.21.15.131:0/3642806031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d567ae9f0 msgr2=0x564d567bda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:02.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.873+0000 7f3135400700 1 --2- 172.21.15.131:0/3642806031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d567ae9f0 0x564d567bda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.873+0000 7f3135400700 1 -- 172.21.15.131:0/3642806031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d566be5e0 con 0x564d567a8910 2024-05-08T22:08:02.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.873+0000 7f3135e00700 1 --2- 172.21.15.131:0/3642806031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d567ae9f0 0x564d567bda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:02.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.874+0000 7f3135400700 1 --2- 172.21.15.131:0/3642806031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d567a8910 0x564d567a83d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3124000c00 tx=0x7f3124022f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38c1e79921913c64 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:02.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.874+0000 7f3123000700 1 -- 172.21.15.131:0/3642806031 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3124084800 con 0x564d567a8910 2024-05-08T22:08:02.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.874+0000 7f3123000700 1 -- 172.21.15.131:0/3642806031 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31240cbad0 con 0x564d567a8910 2024-05-08T22:08:02.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.874+0000 7f3123000700 1 -- 172.21.15.131:0/3642806031 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31240a34e0 con 0x564d567a8910 2024-05-08T22:08:02.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.874+0000 7f314b9f0b80 1 -- 172.21.15.131:0/3642806031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d567a8910 msgr2=0x564d567a83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:02.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.874+0000 7f314b9f0b80 1 --2- 172.21.15.131:0/3642806031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d567a8910 0x564d567a83d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3124000c00 tx=0x7f3124022f90 comp rx=0 tx=0).stop 2024-05-08T22:08:02.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.874+0000 7f314b9f0b80 1 -- 172.21.15.131:0/3642806031 shutdown_connections 2024-05-08T22:08:02.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.874+0000 7f314b9f0b80 1 --2- 172.21.15.131:0/3642806031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d567ae9f0 0x564d567bda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.875+0000 7f314b9f0b80 1 --2- 172.21.15.131:0/3642806031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d567a8910 0x564d567a83d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.875+0000 7f314b9f0b80 1 --2- 172.21.15.131:0/3642806031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d567ae0d0 0x564d567b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.875+0000 7f314b9f0b80 1 -- 172.21.15.131:0/3642806031 >> 172.21.15.131:0/3642806031 conn(0x564d56706290 msgr2=0x564d567abd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:02.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.875+0000 7f314b9f0b80 1 -- 172.21.15.131:0/3642806031 shutdown_connections 2024-05-08T22:08:02.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.875+0000 7f314b9f0b80 1 -- 172.21.15.131:0/3642806031 wait complete. 2024-05-08T22:08:02.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.875+0000 7f314b9f0b80 1 -- start start 2024-05-08T22:08:02.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.875+0000 7f314b9f0b80 1 Processor -- start 2024-05-08T22:08:02.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.876+0000 7f314b9f0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d567a8910 0x564d567aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:02.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.876+0000 7f314b9f0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d567ae0d0 0x564d567aaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:02.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.876+0000 7f314b9f0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d567ae9f0 0x564d567aaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:02.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.876+0000 7f314b9f0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d567b9870 con 0x564d567a8910 2024-05-08T22:08:02.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.876+0000 7f314b9f0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d56817e60 con 0x564d567ae9f0 2024-05-08T22:08:02.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.876+0000 7f314b9f0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d56817fd0 con 0x564d567ae0d0 2024-05-08T22:08:02.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.876+0000 7f314b9f0b80 10 client.0 _ll_register_callbacks cb 0x564d5669dbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:02.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.876+0000 7f3135400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d567a8910 0x564d567aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:02.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.876+0000 7f3134a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d567ae0d0 0x564d567aaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:02.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.876+0000 7f3135400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d567a8910 0x564d567aa4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35388/0 (socket says 172.21.15.131:35388) 2024-05-08T22:08:02.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.876+0000 7f3135400700 1 -- 172.21.15.131:0/3377455583 learned_addr learned my addr 172.21.15.131:0/3377455583 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:02.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.876+0000 7f3135e00700 1 --2- 172.21.15.131:0/3377455583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d567ae9f0 0x564d567aaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:02.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.876+0000 7f3134a00700 1 -- 172.21.15.131:0/3377455583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d567ae9f0 msgr2=0x564d567aaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:02.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.876+0000 7f3134a00700 1 --2- 172.21.15.131:0/3377455583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d567ae9f0 0x564d567aaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.876+0000 7f3134a00700 1 -- 172.21.15.131:0/3377455583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d567a8910 msgr2=0x564d567aa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:02.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.876+0000 7f3134a00700 1 --2- 172.21.15.131:0/3377455583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d567a8910 0x564d567aa4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.876+0000 7f3134a00700 1 -- 172.21.15.131:0/3377455583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312800fdc0 con 0x564d567ae0d0 2024-05-08T22:08:02.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.876+0000 7f3135400700 1 --2- 172.21.15.131:0/3377455583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d567a8910 0x564d567aa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:02.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.877+0000 7f3134a00700 1 --2- 172.21.15.131:0/3377455583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d567ae0d0 0x564d567aaa00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x564d567b3210 tx=0x7f311c01f6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d9e7f25cae2ebac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:02.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.877+0000 7f3116a00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:08:02.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.877+0000 7f3116a00700 1 -- 172.21.15.131:0/3377455583 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c017050 con 0x564d567ae0d0 2024-05-08T22:08:02.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.877+0000 7f3116a00700 1 -- 172.21.15.131:0/3377455583 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f311c012b80 con 0x564d567ae0d0 2024-05-08T22:08:02.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.877+0000 7f314b9f0b80 1 -- 172.21.15.131:0/3377455583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f312409b7f0 con 0x564d567ae0d0 2024-05-08T22:08:02.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.877+0000 7f3116a00700 1 -- 172.21.15.131:0/3377455583 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f311c012d40 con 0x564d567ae0d0 2024-05-08T22:08:02.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.877+0000 7f314b9f0b80 10 client.25765 fetch_fsmap learned FSMap version 24 2024-05-08T22:08:02.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.877+0000 7f314b9f0b80 1 -- 172.21.15.131:0/3377455583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f3124082070 con 0x564d567ae0d0 2024-05-08T22:08:02.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.877+0000 7f3116a00700 1 -- 172.21.15.131:0/3377455583 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f312409b7f0 con 0x564d567ae0d0 2024-05-08T22:08:02.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.877+0000 7f3116a00700 1 -- 172.21.15.131:0/3377455583 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f311c0274f0 con 0x564d567ae0d0 2024-05-08T22:08:02.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.877+0000 7f314b9f0b80 10 client.25765 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:08:02.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.877+0000 7f314b9f0b80 10 client.25765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:08:02.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.878+0000 7f314b9f0b80 20 client.25765 populate_metadata read hostname 'smithi131' 2024-05-08T22:08:02.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.878+0000 7f314b9f0b80 1 --2- 172.21.15.131:0/3377455583 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x564d56819840 0x564d5681bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:02.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.878+0000 7f314b9f0b80 4 client.25765 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T22:08:02.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.878+0000 7f314b9f0b80 1 -- 172.21.15.131:0/3377455583 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564d566f0f40 con 0x564d56819840 2024-05-08T22:08:02.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.878+0000 7f3135400700 1 --2- 172.21.15.131:0/3377455583 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x564d56819840 0x564d5681bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:02.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.878+0000 7f3116a00700 10 client.25765 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:02.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.878+0000 7f3135400700 1 --2- 172.21.15.131:0/3377455583 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x564d56819840 0x564d5681bc60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6c2f670d87a7c72 server_cookie=ce1b5d6794b85a52 in_seq=0 out_seq=0 2024-05-08T22:08:02.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.879+0000 7f3116a00700 1 -- 172.21.15.131:0/3377455583 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564d566f0f40 con 0x564d56819840 2024-05-08T22:08:02.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.879+0000 7f3116a00700 10 client.25765 handle_command_reply: tid=0 2024-05-08T22:08:02.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.879+0000 7f314b9f0b80 1 client.25765 shutdown 2024-05-08T22:08:02.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.880+0000 7f314b9f0b80 1 -- 172.21.15.131:0/3377455583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d567ae0d0 msgr2=0x564d567aaa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:02.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.880+0000 7f314b9f0b80 1 --2- 172.21.15.131:0/3377455583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d567ae0d0 0x564d567aaa00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x564d567b3210 tx=0x7f311c01f6e0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.880+0000 7f314b9f0b80 1 -- 172.21.15.131:0/3377455583 shutdown_connections 2024-05-08T22:08:02.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.880+0000 7f314b9f0b80 1 --2- 172.21.15.131:0/3377455583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d567ae9f0 0x564d567aaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.880+0000 7f314b9f0b80 1 --2- 172.21.15.131:0/3377455583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d567a8910 0x564d567aa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.880+0000 7f314b9f0b80 1 --2- 172.21.15.131:0/3377455583 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x564d56819840 0x564d5681bc60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.880+0000 7f314b9f0b80 1 --2- 172.21.15.131:0/3377455583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d567ae0d0 0x564d567aaa00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.880+0000 7f314b9f0b80 1 -- 172.21.15.131:0/3377455583 >> 172.21.15.131:0/3377455583 conn(0x564d56706290 msgr2=0x564d567bda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:02.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.880+0000 7f3116a00700 0 client.25765 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:02.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.880+0000 7f314b9f0b80 1 -- 172.21.15.131:0/3377455583 shutdown_connections 2024-05-08T22:08:02.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.880+0000 7f314b9f0b80 1 -- 172.21.15.131:0/3377455583 wait complete. 2024-05-08T22:08:02.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.880+0000 7f314b9f0b80 20 client.25765 trim_cache size 0 max 16384 2024-05-08T22:08:02.886 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:08:02.886 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-08T22:08:02.886 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "db302458-2ae9-4d61-bac6-f5ed08d07d7b", 2024-05-08T22:08:02.886 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-08T22:08:02.886 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:08:02.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.881+0000 7f3136800700 1 -- 172.21.15.131:0/2966134032 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f31180a14c0 msgr2=0x7f31180a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:02.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.881+0000 7f3136800700 1 --2- 172.21.15.131:0/2966134032 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f31180a14c0 0x7f31180a3980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3128009ec0 tx=0x7f312800fc40 comp rx=0 tx=0).stop 2024-05-08T22:08:02.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.881+0000 7f3136800700 1 -- 172.21.15.131:0/2966134032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31301031c0 msgr2=0x7f3130199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:02.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.881+0000 7f3136800700 1 --2- 172.21.15.131:0/2966134032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31301031c0 0x7f3130199510 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3124005910 tx=0x7f3124004730 comp rx=0 tx=0).stop 2024-05-08T22:08:02.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.881+0000 7f3136800700 1 -- 172.21.15.131:0/2966134032 shutdown_connections 2024-05-08T22:08:02.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.881+0000 7f3136800700 1 --2- 172.21.15.131:0/2966134032 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f31180a14c0 0x7f31180a3980 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.881+0000 7f3136800700 1 --2- 172.21.15.131:0/2966134032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f313019ddf0 0x7f313019e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.881+0000 7f3136800700 1 --2- 172.21.15.131:0/2966134032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31301031c0 0x7f3130199510 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.881+0000 7f3136800700 1 --2- 172.21.15.131:0/2966134032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31301082e0 0x7f3130199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:02.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.881+0000 7f3136800700 1 -- 172.21.15.131:0/2966134032 >> 172.21.15.131:0/2966134032 conn(0x7f31300fea00 msgr2=0x7f31300ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:02.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.881+0000 7f3136800700 1 -- 172.21.15.131:0/2966134032 shutdown_connections 2024-05-08T22:08:02.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:02.881+0000 7f3136800700 1 -- 172.21.15.131:0/2966134032 wait complete. 2024-05-08T22:08:02.900 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:08:03.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.137+0000 7fd647000700 1 -- 172.21.15.131:0/4049283579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd640101b30 msgr2=0x7fd640105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:03.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.137+0000 7fd647000700 1 --2- 172.21.15.131:0/4049283579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd640101b30 0x7fd640105b70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd62c009990 tx=0x7fd62c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.138+0000 7fd647000700 1 -- 172.21.15.131:0/4049283579 shutdown_connections 2024-05-08T22:08:03.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.138+0000 7fd647000700 1 --2- 172.21.15.131:0/4049283579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6401062a0 0x7fd64010de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.138+0000 7fd647000700 1 --2- 172.21.15.131:0/4049283579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd640101180 0x7fd640101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.138+0000 7fd647000700 1 --2- 172.21.15.131:0/4049283579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd640101b30 0x7fd640105b70 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.138+0000 7fd647000700 1 -- 172.21.15.131:0/4049283579 >> 172.21.15.131:0/4049283579 conn(0x7fd6400fc9e0 msgr2=0x7fd6400fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:03.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.138+0000 7fd647000700 1 -- 172.21.15.131:0/4049283579 shutdown_connections 2024-05-08T22:08:03.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.138+0000 7fd647000700 1 -- 172.21.15.131:0/4049283579 wait complete. 2024-05-08T22:08:03.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.139+0000 7fd647000700 1 Processor -- start 2024-05-08T22:08:03.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.139+0000 7fd647000700 1 -- start start 2024-05-08T22:08:03.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.139+0000 7fd647000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd640101180 0x7fd640197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:03.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.139+0000 7fd647000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6401062a0 0x7fd640197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:03.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.139+0000 7fd647000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd64019bd00 0x7fd64019c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:03.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.139+0000 7fd647000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64019c780 con 0x7fd6401062a0 2024-05-08T22:08:03.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.139+0000 7fd647000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64019c8f0 con 0x7fd640101180 2024-05-08T22:08:03.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.139+0000 7fd647000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64019cab0 con 0x7fd64019bd00 2024-05-08T22:08:03.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.140+0000 7fd63f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6401062a0 0x7fd640197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:03.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.140+0000 7fd63f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6401062a0 0x7fd640197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35408/0 (socket says 172.21.15.131:35408) 2024-05-08T22:08:03.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.140+0000 7fd63f400700 1 -- 172.21.15.131:0/2367678715 learned_addr learned my addr 172.21.15.131:0/2367678715 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:03.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.140+0000 7fd645000700 1 --2- 172.21.15.131:0/2367678715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd64019bd00 0x7fd64019c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:03.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.140+0000 7fd63fe00700 1 --2- 172.21.15.131:0/2367678715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd640101180 0x7fd640197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:03.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.140+0000 7fd63f400700 1 -- 172.21.15.131:0/2367678715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd64019bd00 msgr2=0x7fd64019c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:03.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.140+0000 7fd63f400700 1 --2- 172.21.15.131:0/2367678715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd64019bd00 0x7fd64019c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.140+0000 7fd63f400700 1 -- 172.21.15.131:0/2367678715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd640101180 msgr2=0x7fd640197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:03.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.140+0000 7fd63f400700 1 --2- 172.21.15.131:0/2367678715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd640101180 0x7fd640197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.140+0000 7fd63f400700 1 -- 172.21.15.131:0/2367678715 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd630009520 con 0x7fd6401062a0 2024-05-08T22:08:03.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.140+0000 7fd645000700 1 --2- 172.21.15.131:0/2367678715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd64019bd00 0x7fd64019c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:03.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.140+0000 7fd63f400700 1 --2- 172.21.15.131:0/2367678715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6401062a0 0x7fd640197960 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd62c000c00 tx=0x7fd62c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:03.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.141+0000 7fd63cc00700 1 -- 172.21.15.131:0/2367678715 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c01d070 con 0x7fd6401062a0 2024-05-08T22:08:03.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.141+0000 7fd63cc00700 1 -- 172.21.15.131:0/2367678715 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd62c022470 con 0x7fd6401062a0 2024-05-08T22:08:03.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.141+0000 7fd647000700 1 -- 172.21.15.131:0/2367678715 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd62c0095f0 con 0x7fd6401062a0 2024-05-08T22:08:03.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.141+0000 7fd63cc00700 1 -- 172.21.15.131:0/2367678715 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c017650 con 0x7fd6401062a0 2024-05-08T22:08:03.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.141+0000 7fd647000700 1 -- 172.21.15.131:0/2367678715 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6401a41e0 con 0x7fd6401062a0 2024-05-08T22:08:03.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.144+0000 7fd63cc00700 1 -- 172.21.15.131:0/2367678715 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 24) v1 ==== 136158+0+0 (secure 0 0 0) 0x7fd62c00fad0 con 0x7fd6401062a0 2024-05-08T22:08:03.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.145+0000 7fd65aad5b80 1 Processor -- start 2024-05-08T22:08:03.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.145+0000 7fd63cc00700 1 --2- 172.21.15.131:0/2367678715 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fd6240a1440 0x7fd6240a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:03.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.145+0000 7fd65aad5b80 1 -- start start 2024-05-08T22:08:03.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.145+0000 7fd63cc00700 1 -- 172.21.15.131:0/2367678715 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fd62c0ce600 con 0x7fd6401062a0 2024-05-08T22:08:03.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.145+0000 7fd645000700 1 --2- 172.21.15.131:0/2367678715 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fd6240a1440 0x7fd6240a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:03.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.145+0000 7fd65aad5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a906342e00 0x55a9063431e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:03.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.145+0000 7fd65aad5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a906343720 0x55a90633d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:03.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.145+0000 7fd65aad5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90633d890 0x55a90633dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:03.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.145+0000 7fd65aad5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a906294150 con 0x55a906343720 2024-05-08T22:08:03.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.145+0000 7fd65aad5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a906291b10 con 0x55a906342e00 2024-05-08T22:08:03.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.145+0000 7fd65aad5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a90628d400 con 0x55a90633d890 2024-05-08T22:08:03.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.146+0000 7fd63f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a906343720 0x55a90633d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:03.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.146+0000 7fd63f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a906343720 0x55a90633d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35418/0 (socket says 172.21.15.131:35418) 2024-05-08T22:08:03.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.146+0000 7fd63f400700 1 -- 172.21.15.131:0/1338312415 learned_addr learned my addr 172.21.15.131:0/1338312415 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:03.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.146+0000 7fd63fe00700 1 --2- 172.21.15.131:0/1338312415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a906342e00 0x55a9063431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:03.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.146+0000 7fd645000700 1 --2- 172.21.15.131:0/1338312415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90633d890 0x55a90633dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:03.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.146+0000 7fd645000700 1 --2- 172.21.15.131:0/2367678715 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fd6240a1440 0x7fd6240a3900 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd638003e80 tx=0x7fd63800b040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:03.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.146+0000 7fd63f400700 1 -- 172.21.15.131:0/1338312415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90633d890 msgr2=0x55a90633dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:03.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.146+0000 7fd63f400700 1 --2- 172.21.15.131:0/1338312415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90633d890 0x55a90633dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.146+0000 7fd63f400700 1 -- 172.21.15.131:0/1338312415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a906342e00 msgr2=0x55a9063431e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:03.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.146+0000 7fd63f400700 1 --2- 172.21.15.131:0/1338312415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a906342e00 0x55a9063431e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.146+0000 7fd63f400700 1 -- 172.21.15.131:0/1338312415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a90610e3b0 con 0x55a906343720 2024-05-08T22:08:03.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.146+0000 7fd63f400700 1 --2- 172.21.15.131:0/1338312415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a906343720 0x55a90633d350 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd62c09b280 tx=0x7fd62c00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251c5b5abcf26106 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:03.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.147+0000 7fd635600700 1 -- 172.21.15.131:0/1338312415 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c0b0030 con 0x55a906343720 2024-05-08T22:08:03.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.147+0000 7fd635600700 1 -- 172.21.15.131:0/1338312415 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd62c09ae30 con 0x55a906343720 2024-05-08T22:08:03.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.147+0000 7fd635600700 1 -- 172.21.15.131:0/1338312415 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c022850 con 0x55a906343720 2024-05-08T22:08:03.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.147+0000 7fd65aad5b80 1 -- 172.21.15.131:0/1338312415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a906343720 msgr2=0x55a90633d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:03.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.147+0000 7fd65aad5b80 1 --2- 172.21.15.131:0/1338312415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a906343720 0x55a90633d350 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd62c09b280 tx=0x7fd62c00bfd0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.147+0000 7fd65aad5b80 1 -- 172.21.15.131:0/1338312415 shutdown_connections 2024-05-08T22:08:03.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.147+0000 7fd65aad5b80 1 --2- 172.21.15.131:0/1338312415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90633d890 0x55a90633dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.147+0000 7fd65aad5b80 1 --2- 172.21.15.131:0/1338312415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a906342e00 0x55a9063431e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.147+0000 7fd65aad5b80 1 --2- 172.21.15.131:0/1338312415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a906343720 0x55a90633d350 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.147+0000 7fd65aad5b80 1 -- 172.21.15.131:0/1338312415 >> 172.21.15.131:0/1338312415 conn(0x55a90629b210 msgr2=0x55a906337df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:03.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.148+0000 7fd65aad5b80 1 -- 172.21.15.131:0/1338312415 shutdown_connections 2024-05-08T22:08:03.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.148+0000 7fd65aad5b80 1 -- 172.21.15.131:0/1338312415 wait complete. 2024-05-08T22:08:03.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.148+0000 7fd65aad5b80 1 -- start start 2024-05-08T22:08:03.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.148+0000 7fd65aad5b80 1 Processor -- start 2024-05-08T22:08:03.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.149+0000 7fd65aad5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a90633d890 0x55a9063496e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:03.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.149+0000 7fd65aad5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a906342e00 0x55a906346a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:03.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.149+0000 7fd65aad5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a906343720 0x55a906347180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:03.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.149+0000 7fd65aad5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a906229560 con 0x55a906342e00 2024-05-08T22:08:03.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.149+0000 7fd65aad5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a906229200 con 0x55a90633d890 2024-05-08T22:08:03.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.149+0000 7fd65aad5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9061106a0 con 0x55a906343720 2024-05-08T22:08:03.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.149+0000 7fd65aad5b80 10 client.0 _ll_register_callbacks cb 0x55a906232b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:03.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.149+0000 7fd63f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a906342e00 0x55a906346a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:03.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.149+0000 7fd645000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a906343720 0x55a906347180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:03.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.149+0000 7fd63f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a906342e00 0x55a906346a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35422/0 (socket says 172.21.15.131:35422) 2024-05-08T22:08:03.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.149+0000 7fd63f400700 1 -- 172.21.15.131:0/1622770519 learned_addr learned my addr 172.21.15.131:0/1622770519 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:03.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.149+0000 7fd63fe00700 1 --2- 172.21.15.131:0/1622770519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a90633d890 0x55a9063496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:03.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.149+0000 7fd63f400700 1 -- 172.21.15.131:0/1622770519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a906343720 msgr2=0x55a906347180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:03.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.149+0000 7fd63f400700 1 --2- 172.21.15.131:0/1622770519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a906343720 0x55a906347180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.149+0000 7fd63f400700 1 -- 172.21.15.131:0/1622770519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a90633d890 msgr2=0x55a9063496e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:03.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.149+0000 7fd63f400700 1 --2- 172.21.15.131:0/1622770519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a90633d890 0x55a9063496e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.149+0000 7fd63f400700 1 -- 172.21.15.131:0/1622770519 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6401a41e0 con 0x55a906342e00 2024-05-08T22:08:03.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.149+0000 7fd645000700 1 --2- 172.21.15.131:0/1622770519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a906343720 0x55a906347180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:03.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.150+0000 7fd63f400700 1 --2- 172.21.15.131:0/1622770519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a906342e00 0x55a906346a40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd62c006010 tx=0x7fd62c00ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226c6b726e530b8f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:03.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.150+0000 7fd62a000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:08:03.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.150+0000 7fd62a000700 1 -- 172.21.15.131:0/1622770519 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c0b0040 con 0x55a906342e00 2024-05-08T22:08:03.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.150+0000 7fd62a000700 1 -- 172.21.15.131:0/1622770519 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fd62c0b1070 con 0x55a906342e00 2024-05-08T22:08:03.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.150+0000 7fd65aad5b80 1 -- 172.21.15.131:0/1622770519 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9061e6270 con 0x55a906342e00 2024-05-08T22:08:03.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.150+0000 7fd62a000700 1 -- 172.21.15.131:0/1622770519 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd62c09b2f0 con 0x55a906342e00 2024-05-08T22:08:03.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.150+0000 7fd65aad5b80 10 client.15723 fetch_fsmap learned FSMap version 24 2024-05-08T22:08:03.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.150+0000 7fd65aad5b80 1 -- 172.21.15.131:0/1622770519 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a9060bab60 con 0x55a906342e00 2024-05-08T22:08:03.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.150+0000 7fd62a000700 1 -- 172.21.15.131:0/1622770519 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a9061e6270 con 0x55a906342e00 2024-05-08T22:08:03.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.150+0000 7fd62a000700 1 -- 172.21.15.131:0/1622770519 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd62c0a3250 con 0x55a906342e00 2024-05-08T22:08:03.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.151+0000 7fd65aad5b80 10 client.15723 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:08:03.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.151+0000 7fd65aad5b80 10 client.15723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:08:03.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.151+0000 7fd65aad5b80 20 client.15723 populate_metadata read hostname 'smithi131' 2024-05-08T22:08:03.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.151+0000 7fd65aad5b80 1 --2- 172.21.15.131:0/1622770519 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55a9063569c0 0x55a906358db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:03.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.151+0000 7fd65aad5b80 4 client.15723 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:08:03.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.151+0000 7fd65aad5b80 1 -- 172.21.15.131:0/1622770519 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a90630c810 con 0x55a9063569c0 2024-05-08T22:08:03.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.151+0000 7fd63fe00700 1 --2- 172.21.15.131:0/1622770519 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55a9063569c0 0x55a906358db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:03.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.151+0000 7fd63fe00700 1 --2- 172.21.15.131:0/1622770519 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55a9063569c0 0x55a906358db0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=315613b84ef9d2bd server_cookie=9b41182cb0326ece in_seq=0 out_seq=0 2024-05-08T22:08:03.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.151+0000 7fd62a000700 10 client.15723 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:03.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.152+0000 7fd62a000700 1 -- 172.21.15.131:0/1622770519 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a90630c810 con 0x55a9063569c0 2024-05-08T22:08:03.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.152+0000 7fd62a000700 10 client.15723 handle_command_reply: tid=0 2024-05-08T22:08:03.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.152+0000 7fd65aad5b80 1 client.15723 shutdown 2024-05-08T22:08:03.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.152+0000 7fd65aad5b80 1 -- 172.21.15.131:0/1622770519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a906342e00 msgr2=0x55a906346a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:03.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.152+0000 7fd65aad5b80 1 --2- 172.21.15.131:0/1622770519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a906342e00 0x55a906346a40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd62c006010 tx=0x7fd62c00ff90 comp rx=0 tx=0).stop 2024-05-08T22:08:03.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.152+0000 7fd65aad5b80 1 -- 172.21.15.131:0/1622770519 shutdown_connections 2024-05-08T22:08:03.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.152+0000 7fd65aad5b80 1 --2- 172.21.15.131:0/1622770519 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55a9063569c0 0x55a906358db0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.152+0000 7fd65aad5b80 1 --2- 172.21.15.131:0/1622770519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a906343720 0x55a906347180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.152+0000 7fd65aad5b80 1 --2- 172.21.15.131:0/1622770519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a90633d890 0x55a9063496e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.152+0000 7fd62a000700 0 client.15723 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:03.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.152+0000 7fd65aad5b80 1 --2- 172.21.15.131:0/1622770519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a906342e00 0x55a906346a40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.152+0000 7fd65aad5b80 1 -- 172.21.15.131:0/1622770519 >> 172.21.15.131:0/1622770519 conn(0x55a90629b210 msgr2=0x55a9063375a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:03.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.153+0000 7fd65aad5b80 1 -- 172.21.15.131:0/1622770519 shutdown_connections 2024-05-08T22:08:03.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.153+0000 7fd65aad5b80 1 -- 172.21.15.131:0/1622770519 wait complete. 2024-05-08T22:08:03.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.153+0000 7fd65aad5b80 20 client.15723 trim_cache size 0 max 16384 2024-05-08T22:08:03.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.170+0000 7fd65aad5b80 1 Processor -- start 2024-05-08T22:08:03.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.170+0000 7fd65aad5b80 1 -- start start 2024-05-08T22:08:03.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.171+0000 7fd65aad5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90633d890 0x55a9063399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:03.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.171+0000 7fd65aad5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a906342e00 0x55a906337d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:03.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.171+0000 7fd65aad5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a906343720 0x55a90634ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:03.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.171+0000 7fd65aad5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9063539a0 con 0x55a90633d890 2024-05-08T22:08:03.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.171+0000 7fd65aad5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a906347ed0 con 0x55a906343720 2024-05-08T22:08:03.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.171+0000 7fd65aad5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a906334dc0 con 0x55a906342e00 2024-05-08T22:08:03.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.171+0000 7fd63fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90633d890 0x55a9063399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:03.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.171+0000 7fd63fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90633d890 0x55a9063399d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35430/0 (socket says 172.21.15.131:35430) 2024-05-08T22:08:03.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.171+0000 7fd63fe00700 1 -- 172.21.15.131:0/2891341161 learned_addr learned my addr 172.21.15.131:0/2891341161 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:03.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.171+0000 7fd645000700 1 --2- 172.21.15.131:0/2891341161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a906343720 0x55a90634ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:03.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.171+0000 7fd63f400700 1 --2- 172.21.15.131:0/2891341161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a906342e00 0x55a906337d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:03.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.171+0000 7fd63fe00700 1 -- 172.21.15.131:0/2891341161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a906342e00 msgr2=0x55a906337d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:03.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.171+0000 7fd63fe00700 1 --2- 172.21.15.131:0/2891341161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a906342e00 0x55a906337d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.171+0000 7fd63fe00700 1 -- 172.21.15.131:0/2891341161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a906343720 msgr2=0x55a90634ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:03.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.171+0000 7fd63fe00700 1 --2- 172.21.15.131:0/2891341161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a906343720 0x55a90634ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.171+0000 7fd63fe00700 1 -- 172.21.15.131:0/2891341161 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a906253560 con 0x55a90633d890 2024-05-08T22:08:03.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.171+0000 7fd63f400700 1 --2- 172.21.15.131:0/2891341161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a906342e00 0x55a906337d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:08:03.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.171+0000 7fd645000700 1 --2- 172.21.15.131:0/2891341161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a906343720 0x55a90634ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:08:03.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.172+0000 7fd63fe00700 1 --2- 172.21.15.131:0/2891341161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90633d890 0x55a9063399d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd63000a010 tx=0x7fd630009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9186b4a27f61bf4a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:03.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.172+0000 7fd636000700 1 -- 172.21.15.131:0/2891341161 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd630017040 con 0x55a90633d890 2024-05-08T22:08:03.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.172+0000 7fd636000700 1 -- 172.21.15.131:0/2891341161 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd630018040 con 0x55a90633d890 2024-05-08T22:08:03.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.172+0000 7fd636000700 1 -- 172.21.15.131:0/2891341161 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6300055f0 con 0x55a90633d890 2024-05-08T22:08:03.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.172+0000 7fd65aad5b80 1 -- 172.21.15.131:0/2891341161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90633d890 msgr2=0x55a9063399d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:03.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.172+0000 7fd65aad5b80 1 --2- 172.21.15.131:0/2891341161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90633d890 0x55a9063399d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd63000a010 tx=0x7fd630009260 comp rx=0 tx=0).stop 2024-05-08T22:08:03.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.173+0000 7fd65aad5b80 1 -- 172.21.15.131:0/2891341161 shutdown_connections 2024-05-08T22:08:03.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.173+0000 7fd65aad5b80 1 --2- 172.21.15.131:0/2891341161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a906343720 0x55a90634ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.173+0000 7fd65aad5b80 1 --2- 172.21.15.131:0/2891341161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a90633d890 0x55a9063399d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.173+0000 7fd65aad5b80 1 --2- 172.21.15.131:0/2891341161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a906342e00 0x55a906337d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.173+0000 7fd65aad5b80 1 -- 172.21.15.131:0/2891341161 >> 172.21.15.131:0/2891341161 conn(0x55a90629b210 msgr2=0x55a90633dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:03.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.173+0000 7fd65aad5b80 1 -- 172.21.15.131:0/2891341161 shutdown_connections 2024-05-08T22:08:03.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.173+0000 7fd65aad5b80 1 -- 172.21.15.131:0/2891341161 wait complete. 2024-05-08T22:08:03.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.173+0000 7fd65aad5b80 1 -- start start 2024-05-08T22:08:03.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.173+0000 7fd65aad5b80 1 Processor -- start 2024-05-08T22:08:03.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.174+0000 7fd65aad5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90633d890 0x55a906340340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:03.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.174+0000 7fd65aad5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a906342e00 0x55a90634dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:03.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.174+0000 7fd65aad5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a906343720 0x55a90633e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:03.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.174+0000 7fd65aad5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a90634e820 con 0x55a906342e00 2024-05-08T22:08:03.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.174+0000 7fd65aad5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a90633ee70 con 0x55a906343720 2024-05-08T22:08:03.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.174+0000 7fd65aad5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a90633efe0 con 0x55a90633d890 2024-05-08T22:08:03.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.174+0000 7fd65aad5b80 10 client.0 _ll_register_callbacks cb 0x55a906232b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:03.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.174+0000 7fd63f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a906342e00 0x55a90634dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:03.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.174+0000 7fd63fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90633d890 0x55a906340340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:03.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.174+0000 7fd63f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a906342e00 0x55a90634dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35444/0 (socket says 172.21.15.131:35444) 2024-05-08T22:08:03.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.174+0000 7fd63fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90633d890 0x55a906340340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46962/0 (socket says 172.21.15.131:46962) 2024-05-08T22:08:03.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.174+0000 7fd63f400700 1 -- 172.21.15.131:0/2146484014 learned_addr learned my addr 172.21.15.131:0/2146484014 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:03.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.174+0000 7fd645000700 1 --2- 172.21.15.131:0/2146484014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a906343720 0x55a90633e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:03.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.174+0000 7fd63fe00700 1 -- 172.21.15.131:0/2146484014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a906343720 msgr2=0x55a90633e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:03.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.174+0000 7fd63fe00700 1 --2- 172.21.15.131:0/2146484014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a906343720 0x55a90633e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.174+0000 7fd63fe00700 1 -- 172.21.15.131:0/2146484014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a906342e00 msgr2=0x55a90634dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:03.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.175+0000 7fd63fe00700 1 --2- 172.21.15.131:0/2146484014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a906342e00 0x55a90634dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.175+0000 7fd63fe00700 1 -- 172.21.15.131:0/2146484014 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6300124b0 con 0x55a90633d890 2024-05-08T22:08:03.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.175+0000 7fd63f400700 1 --2- 172.21.15.131:0/2146484014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a906342e00 0x55a90634dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:03.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.175+0000 7fd63fe00700 1 --2- 172.21.15.131:0/2146484014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90633d890 0x55a906340340 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd630009990 tx=0x7fd630009a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccb4a3561bcf9b53 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:03.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.175+0000 7fd62a000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:08:03.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.175+0000 7fd62a000700 1 -- 172.21.15.131:0/2146484014 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd630005270 con 0x55a90633d890 2024-05-08T22:08:03.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.175+0000 7fd65aad5b80 1 -- 172.21.15.131:0/2146484014 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd62c09ae30 con 0x55a90633d890 2024-05-08T22:08:03.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.175+0000 7fd62a000700 1 -- 172.21.15.131:0/2146484014 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fd6300053d0 con 0x55a90633d890 2024-05-08T22:08:03.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.175+0000 7fd62a000700 1 -- 172.21.15.131:0/2146484014 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd630005590 con 0x55a90633d890 2024-05-08T22:08:03.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.175+0000 7fd65aad5b80 10 client.25783 fetch_fsmap learned FSMap version 24 2024-05-08T22:08:03.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.175+0000 7fd65aad5b80 1 -- 172.21.15.131:0/2146484014 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a906112cd0 con 0x55a90633d890 2024-05-08T22:08:03.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.176+0000 7fd62a000700 1 -- 172.21.15.131:0/2146484014 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd62c09ae30 con 0x55a90633d890 2024-05-08T22:08:03.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.176+0000 7fd62a000700 1 -- 172.21.15.131:0/2146484014 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd6300213f0 con 0x55a90633d890 2024-05-08T22:08:03.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.176+0000 7fd65aad5b80 10 client.25783 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:08:03.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.176+0000 7fd65aad5b80 10 client.25783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:08:03.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.176+0000 7fd65aad5b80 20 client.25783 populate_metadata read hostname 'smithi131' 2024-05-08T22:08:03.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.176+0000 7fd65aad5b80 1 --2- 172.21.15.131:0/2146484014 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55a9063adc60 0x55a9063b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:03.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.176+0000 7fd65aad5b80 4 client.25783 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:08:03.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.176+0000 7fd65aad5b80 1 -- 172.21.15.131:0/2146484014 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a906285ec0 con 0x55a9063adc60 2024-05-08T22:08:03.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.176+0000 7fd63f400700 1 --2- 172.21.15.131:0/2146484014 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55a9063adc60 0x55a9063b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:03.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.177+0000 7fd62a000700 10 client.25783 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:03.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.177+0000 7fd63f400700 1 --2- 172.21.15.131:0/2146484014 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55a9063adc60 0x55a9063b0080 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b72305885bf73c00 server_cookie=b55856761fca2750 in_seq=0 out_seq=0 2024-05-08T22:08:03.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.177+0000 7fd62a000700 1 -- 172.21.15.131:0/2146484014 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a906285ec0 con 0x55a9063adc60 2024-05-08T22:08:03.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.177+0000 7fd62a000700 10 client.25783 handle_command_reply: tid=0 2024-05-08T22:08:03.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.177+0000 7fd65aad5b80 1 client.25783 shutdown 2024-05-08T22:08:03.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.177+0000 7fd65aad5b80 1 -- 172.21.15.131:0/2146484014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90633d890 msgr2=0x55a906340340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:03.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.177+0000 7fd65aad5b80 1 --2- 172.21.15.131:0/2146484014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90633d890 0x55a906340340 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd630009990 tx=0x7fd630009a70 comp rx=0 tx=0).stop 2024-05-08T22:08:03.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.177+0000 7fd65aad5b80 1 -- 172.21.15.131:0/2146484014 shutdown_connections 2024-05-08T22:08:03.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.177+0000 7fd65aad5b80 1 --2- 172.21.15.131:0/2146484014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a906343720 0x55a90633e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.177+0000 7fd65aad5b80 1 --2- 172.21.15.131:0/2146484014 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55a9063adc60 0x55a9063b0080 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.177+0000 7fd65aad5b80 1 --2- 172.21.15.131:0/2146484014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a90633d890 0x55a906340340 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.177+0000 7fd62a000700 0 client.25783 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:03.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.177+0000 7fd65aad5b80 1 --2- 172.21.15.131:0/2146484014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a906342e00 0x55a90634dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.177+0000 7fd65aad5b80 1 -- 172.21.15.131:0/2146484014 >> 172.21.15.131:0/2146484014 conn(0x55a90629b210 msgr2=0x55a906340c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:03.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.177+0000 7fd65aad5b80 1 -- 172.21.15.131:0/2146484014 shutdown_connections 2024-05-08T22:08:03.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.177+0000 7fd65aad5b80 1 -- 172.21.15.131:0/2146484014 wait complete. 2024-05-08T22:08:03.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.177+0000 7fd65aad5b80 20 client.25783 trim_cache size 0 max 16384 2024-05-08T22:08:03.184 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:08:03.184 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T22:08:03.184 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-08T22:08:03.184 INFO:teuthology.orchestra.run.smithi131.stdout: "db302458-2ae9-4d61-bac6-f5ed08d07d7b": { 2024-05-08T22:08:03.184 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-08T22:08:03.184 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "db302458-2ae9-4d61-bac6-f5ed08d07d7b", 2024-05-08T22:08:03.184 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-08T22:08:03.184 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:08:03.185 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:08:03.185 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:08:03.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.178+0000 7fd647000700 1 -- 172.21.15.131:0/2367678715 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fd6240a1440 msgr2=0x7fd6240a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:03.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.178+0000 7fd647000700 1 --2- 172.21.15.131:0/2367678715 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fd6240a1440 0x7fd6240a3900 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd638003e80 tx=0x7fd63800b040 comp rx=0 tx=0).stop 2024-05-08T22:08:03.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.178+0000 7fd647000700 1 -- 172.21.15.131:0/2367678715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6401062a0 msgr2=0x7fd640197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:03.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.178+0000 7fd647000700 1 --2- 172.21.15.131:0/2367678715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6401062a0 0x7fd640197960 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd62c000c00 tx=0x7fd62c005740 comp rx=0 tx=0).stop 2024-05-08T22:08:03.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.178+0000 7fd647000700 1 -- 172.21.15.131:0/2367678715 shutdown_connections 2024-05-08T22:08:03.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.178+0000 7fd647000700 1 --2- 172.21.15.131:0/2367678715 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fd6240a1440 0x7fd6240a3900 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.178+0000 7fd647000700 1 --2- 172.21.15.131:0/2367678715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd64019bd00 0x7fd64019c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.178+0000 7fd647000700 1 --2- 172.21.15.131:0/2367678715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd640101180 0x7fd640197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.178+0000 7fd647000700 1 --2- 172.21.15.131:0/2367678715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6401062a0 0x7fd640197960 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:03.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.178+0000 7fd647000700 1 -- 172.21.15.131:0/2367678715 >> 172.21.15.131:0/2367678715 conn(0x7fd6400fc9e0 msgr2=0x7fd6400fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:03.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.178+0000 7fd647000700 1 -- 172.21.15.131:0/2367678715 shutdown_connections 2024-05-08T22:08:03.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:03.178+0000 7fd647000700 1 -- 172.21.15.131:0/2367678715 wait complete. 2024-05-08T22:08:03.198 INFO:tasks.cephfs.filesystem:scrub status for tag:db302458-2ae9-4d61-bac6-f5ed08d07d7b - {'path': '/', 'tag': 'db302458-2ae9-4d61-bac6-f5ed08d07d7b', 'options': 'recursive,force'} 2024-05-08T22:08:04.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:04 smithi132 ceph-mon[78180]: scrub queued for path: / 2024-05-08T22:08:04.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:04 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:08:04.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:04 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:08:04.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:04 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:08:04.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:04 smithi132 ceph-mon[78180]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:08:04.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:04 smithi131 ceph-mon[37260]: scrub queued for path: / 2024-05-08T22:08:04.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:04 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:08:04.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:04 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:08:04.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:04 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:08:04.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:04 smithi131 ceph-mon[37260]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:08:04.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:04 smithi157 ceph-mon[77746]: scrub queued for path: / 2024-05-08T22:08:04.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:04 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:08:04.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:04 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:08:04.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:04 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:08:04.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:04 smithi157 ceph-mon[77746]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:08:06.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:06 smithi132 ceph-mon[78180]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-05-08T22:08:06.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:06 smithi131 ceph-mon[37260]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-05-08T22:08:06.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:06 smithi157 ceph-mon[77746]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-05-08T22:08:08.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:08 smithi132 ceph-mon[78180]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-05-08T22:08:08.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:08 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:08:08.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:08 smithi131 ceph-mon[37260]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-05-08T22:08:08.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:08 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:08:08.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:08 smithi157 ceph-mon[77746]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-05-08T22:08:08.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:08 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:08:10.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:10 smithi132 ceph-mon[78180]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-05-08T22:08:10.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:10 smithi131 ceph-mon[37260]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-05-08T22:08:10.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:10 smithi157 ceph-mon[77746]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-05-08T22:08:12.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:12 smithi132 ceph-mon[78180]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-05-08T22:08:12.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:12 smithi131 ceph-mon[37260]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-05-08T22:08:12.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:12 smithi157 ceph-mon[77746]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-05-08T22:08:14.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:14 smithi132 ceph-mon[78180]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-05-08T22:08:14.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:14 smithi131 ceph-mon[37260]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-05-08T22:08:14.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:14 smithi157 ceph-mon[77746]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-05-08T22:08:16.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:16 smithi132 ceph-mon[78180]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-05-08T22:08:16.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:16 smithi131 ceph-mon[37260]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-05-08T22:08:16.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:16 smithi157 ceph-mon[77746]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-05-08T22:08:18.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:18 smithi132 ceph-mon[78180]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-08T22:08:18.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:18 smithi131 ceph-mon[37260]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-08T22:08:18.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:18 smithi157 ceph-mon[77746]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-08T22:08:20.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:20 smithi132 ceph-mon[78180]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-08T22:08:20.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:20 smithi131 ceph-mon[37260]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-08T22:08:20.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:20 smithi157 ceph-mon[77746]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-08T22:08:22.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:22 smithi132 ceph-mon[78180]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:08:22.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:22 smithi131 ceph-mon[37260]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:08:22.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:22 smithi157 ceph-mon[77746]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:08:24.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:24 smithi132 ceph-mon[78180]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:08:24.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:24 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:08:24.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:24 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:08:24.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:24 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:08:24.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:24 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:08:24.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:24 smithi131 ceph-mon[37260]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:08:24.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:24 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:08:24.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:24 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:08:24.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:24 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:08:24.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:24 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:08:24.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:24 smithi157 ceph-mon[77746]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:08:24.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:24 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:08:24.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:24 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:08:24.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:24 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:08:24.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:24 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:08:26.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:26 smithi132 ceph-mon[78180]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:08:26.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:26 smithi131 ceph-mon[37260]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:08:26.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:26 smithi157 ceph-mon[77746]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:08:28.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:28 smithi132 ceph-mon[78180]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:08:28.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:28 smithi131 ceph-mon[37260]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:08:28.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:28 smithi157 ceph-mon[77746]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:08:30.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:30 smithi132 ceph-mon[78180]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:08:30.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:30 smithi131 ceph-mon[37260]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:08:30.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:30 smithi157 ceph-mon[77746]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:08:32.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:32 smithi132 ceph-mon[78180]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:08:32.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:32 smithi131 ceph-mon[37260]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:08:32.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:32 smithi157 ceph-mon[77746]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:08:33.199 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:08:33.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.429+0000 7f880a000700 1 -- 172.21.15.131:0/986375328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8804073c90 msgr2=0x7f880410d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:33.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.429+0000 7f880a000700 1 --2- 172.21.15.131:0/986375328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8804073c90 0x7f880410d0c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f87f0009990 tx=0x7f87f0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.429+0000 7f880a000700 1 -- 172.21.15.131:0/986375328 shutdown_connections 2024-05-08T22:08:33.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.429+0000 7f880a000700 1 --2- 172.21.15.131:0/986375328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f880410d600 0x7f880410fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.429+0000 7f880a000700 1 --2- 172.21.15.131:0/986375328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8804073370 0x7f8804073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.429+0000 7f880a000700 1 --2- 172.21.15.131:0/986375328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8804073c90 0x7f880410d0c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.429+0000 7f880a000700 1 -- 172.21.15.131:0/986375328 >> 172.21.15.131:0/986375328 conn(0x7f88040fc870 msgr2=0x7f88040fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:33.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.429+0000 7f880a000700 1 -- 172.21.15.131:0/986375328 shutdown_connections 2024-05-08T22:08:33.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.429+0000 7f880a000700 1 -- 172.21.15.131:0/986375328 wait complete. 2024-05-08T22:08:33.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.430+0000 7f880a000700 1 Processor -- start 2024-05-08T22:08:33.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.430+0000 7f880a000700 1 -- start start 2024-05-08T22:08:33.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.431+0000 7f880a000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8804073370 0x7f88041994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.431+0000 7f880a000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8804073c90 0x7f88041999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.431+0000 7f880a000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f880410d600 0x7f880419dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.431+0000 7f880a000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880419e3c0 con 0x7f8804073370 2024-05-08T22:08:33.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.431+0000 7f880a000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880419e530 con 0x7f880410d600 2024-05-08T22:08:33.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.431+0000 7f880a000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880419e6f0 con 0x7f8804073c90 2024-05-08T22:08:33.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.431+0000 7f8803e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8804073c90 0x7f88041999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.431+0000 7f8803e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8804073c90 0x7f88041999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:38550/0 (socket says 172.21.15.131:38550) 2024-05-08T22:08:33.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.431+0000 7f8803e00700 1 -- 172.21.15.131:0/396407763 learned_addr learned my addr 172.21.15.131:0/396407763 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:33.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.431+0000 7f8808c00700 1 --2- 172.21.15.131:0/396407763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8804073370 0x7f88041994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.431+0000 7f8809600700 1 --2- 172.21.15.131:0/396407763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f880410d600 0x7f880419dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.432+0000 7f8803e00700 1 -- 172.21.15.131:0/396407763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f880410d600 msgr2=0x7f880419dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:33.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.432+0000 7f8803e00700 1 --2- 172.21.15.131:0/396407763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f880410d600 0x7f880419dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.432+0000 7f8803e00700 1 -- 172.21.15.131:0/396407763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8804073370 msgr2=0x7f88041994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:33.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.432+0000 7f8803e00700 1 --2- 172.21.15.131:0/396407763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8804073370 0x7f88041994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.432+0000 7f8803e00700 1 -- 172.21.15.131:0/396407763 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f00095f0 con 0x7f8804073c90 2024-05-08T22:08:33.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.432+0000 7f8803e00700 1 --2- 172.21.15.131:0/396407763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8804073c90 0x7f88041999e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f87f0005910 tx=0x7f87f0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:33.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.432+0000 7f8801600700 1 -- 172.21.15.131:0/396407763 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f001d070 con 0x7f8804073c90 2024-05-08T22:08:33.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.433+0000 7f8801600700 1 -- 172.21.15.131:0/396407763 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87f0022470 con 0x7f8804073c90 2024-05-08T22:08:33.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.433+0000 7f880a000700 1 -- 172.21.15.131:0/396407763 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f880419e970 con 0x7f8804073c90 2024-05-08T22:08:33.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.433+0000 7f8801600700 1 -- 172.21.15.131:0/396407763 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0017650 con 0x7f8804073c90 2024-05-08T22:08:33.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.433+0000 7f880a000700 1 -- 172.21.15.131:0/396407763 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88041a6210 con 0x7f8804073c90 2024-05-08T22:08:33.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.437+0000 7f8801600700 1 -- 172.21.15.131:0/396407763 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 24) v1 ==== 136158+0+0 (secure 0 0 0) 0x7f87f00225e0 con 0x7f8804073c90 2024-05-08T22:08:33.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.438+0000 7f881f16bb80 1 Processor -- start 2024-05-08T22:08:33.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.438+0000 7f881f16bb80 1 -- start start 2024-05-08T22:08:33.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.438+0000 7f881f16bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b25321050 0x557b25321430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.438+0000 7f881f16bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b25321970 0x557b2531b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.438+0000 7f881f16bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b2531b890 0x557b2531bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.438+0000 7f881f16bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b25272150 con 0x557b25321970 2024-05-08T22:08:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.438+0000 7f881f16bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b2526fb10 con 0x557b2531b890 2024-05-08T22:08:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.438+0000 7f881f16bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b2526b400 con 0x557b25321050 2024-05-08T22:08:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.439+0000 7f8801600700 1 --2- 172.21.15.131:0/396407763 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f87ec0a1440 0x7f87ec0a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.439+0000 7f8808c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b25321970 0x557b2531b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.439+0000 7f8809600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b25321050 0x557b25321430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.439+0000 7f8801600700 1 -- 172.21.15.131:0/396407763 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f87f00ce620 con 0x7f8804073c90 2024-05-08T22:08:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.439+0000 7f8809600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b25321050 0x557b25321430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:38560/0 (socket says 172.21.15.131:38560) 2024-05-08T22:08:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.439+0000 7f8809600700 1 -- 172.21.15.131:0/508666279 learned_addr learned my addr 172.21.15.131:0/508666279 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.439+0000 7f8803e00700 1 --2- 172.21.15.131:0/508666279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b2531b890 0x557b2531bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.439+0000 7f8809600700 1 --2- 172.21.15.131:0/396407763 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f87ec0a1440 0x7f87ec0a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.439+0000 7f8808c00700 1 -- 172.21.15.131:0/508666279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b25321050 msgr2=0x557b25321430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.439+0000 7f8808c00700 1 --2- 172.21.15.131:0/508666279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b25321050 0x557b25321430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.439+0000 7f8808c00700 1 -- 172.21.15.131:0/508666279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b2531b890 msgr2=0x557b2531bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.439+0000 7f8808c00700 1 --2- 172.21.15.131:0/508666279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b2531b890 0x557b2531bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.440+0000 7f8808c00700 1 -- 172.21.15.131:0/508666279 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b250ec3b0 con 0x557b25321970 2024-05-08T22:08:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.440+0000 7f8808c00700 1 --2- 172.21.15.131:0/508666279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b25321970 0x557b2531b350 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f87f8007bf0 tx=0x7f87f8007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e0af0f2288f70e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.440+0000 7f87f5c00700 1 -- 172.21.15.131:0/508666279 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f8010c40 con 0x557b25321970 2024-05-08T22:08:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.440+0000 7f87f5c00700 1 -- 172.21.15.131:0/508666279 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87f80044f0 con 0x557b25321970 2024-05-08T22:08:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.440+0000 7f87f5c00700 1 -- 172.21.15.131:0/508666279 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f8005220 con 0x557b25321970 2024-05-08T22:08:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.440+0000 7f8809600700 1 --2- 172.21.15.131:0/396407763 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f87ec0a1440 0x7f87ec0a3900 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f87fc009fd0 tx=0x7f87fc00b040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.441+0000 7f881f16bb80 1 -- 172.21.15.131:0/508666279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b25321970 msgr2=0x557b2531b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.441+0000 7f881f16bb80 1 --2- 172.21.15.131:0/508666279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b25321970 0x557b2531b350 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f87f8007bf0 tx=0x7f87f8007f00 comp rx=0 tx=0).stop 2024-05-08T22:08:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.441+0000 7f881f16bb80 1 -- 172.21.15.131:0/508666279 shutdown_connections 2024-05-08T22:08:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.441+0000 7f881f16bb80 1 --2- 172.21.15.131:0/508666279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b2531b890 0x557b2531bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.441+0000 7f881f16bb80 1 --2- 172.21.15.131:0/508666279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b25321050 0x557b25321430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.441+0000 7f881f16bb80 1 --2- 172.21.15.131:0/508666279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b25321970 0x557b2531b350 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.441+0000 7f881f16bb80 1 -- 172.21.15.131:0/508666279 >> 172.21.15.131:0/508666279 conn(0x557b25279210 msgr2=0x557b253162d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:33.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.441+0000 7f881f16bb80 1 -- 172.21.15.131:0/508666279 shutdown_connections 2024-05-08T22:08:33.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.441+0000 7f881f16bb80 1 -- 172.21.15.131:0/508666279 wait complete. 2024-05-08T22:08:33.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.442+0000 7f881f16bb80 1 -- start start 2024-05-08T22:08:33.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.442+0000 7f881f16bb80 1 Processor -- start 2024-05-08T22:08:33.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.443+0000 7f881f16bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b2531b890 0x557b253277f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.443+0000 7f881f16bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b25321050 0x557b25324ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.443+0000 7f881f16bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b25321970 0x557b25325110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.443+0000 7f881f16bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b25207560 con 0x557b25321050 2024-05-08T22:08:33.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.443+0000 7f881f16bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b25207200 con 0x557b2531b890 2024-05-08T22:08:33.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.443+0000 7f881f16bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b250ee6a0 con 0x557b25321970 2024-05-08T22:08:33.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.443+0000 7f881f16bb80 10 client.0 _ll_register_callbacks cb 0x557b25210b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:33.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.443+0000 7f8808c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b2531b890 0x557b253277f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.443+0000 7f8808c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b2531b890 0x557b253277f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:40184/0 (socket says 172.21.15.131:40184) 2024-05-08T22:08:33.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.443+0000 7f8808c00700 1 -- 172.21.15.131:0/3929382064 learned_addr learned my addr 172.21.15.131:0/3929382064 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:33.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.444+0000 7f8803e00700 1 --2- 172.21.15.131:0/3929382064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b25321050 0x557b25324ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.444+0000 7f8809600700 1 --2- 172.21.15.131:0/3929382064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b25321970 0x557b25325110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.444+0000 7f8808c00700 1 -- 172.21.15.131:0/3929382064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b25321970 msgr2=0x557b25325110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:33.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.444+0000 7f8808c00700 1 --2- 172.21.15.131:0/3929382064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b25321970 0x557b25325110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.444+0000 7f8808c00700 1 -- 172.21.15.131:0/3929382064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b25321050 msgr2=0x557b25324ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:33.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.444+0000 7f8808c00700 1 --2- 172.21.15.131:0/3929382064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b25321050 0x557b25324ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.444+0000 7f8808c00700 1 -- 172.21.15.131:0/3929382064 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f80078a0 con 0x557b2531b890 2024-05-08T22:08:33.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.444+0000 7f8809600700 1 --2- 172.21.15.131:0/3929382064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b25321970 0x557b25325110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:08:33.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.444+0000 7f8803e00700 1 --2- 172.21.15.131:0/3929382064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b25321050 0x557b25324ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:33.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.444+0000 7f8808c00700 1 --2- 172.21.15.131:0/3929382064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b2531b890 0x557b253277f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f87f80106b0 tx=0x7f87f8010af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d384852ff4929ae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:33.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.444+0000 7f87ea000700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:08:33.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.445+0000 7f87ea000700 1 -- 172.21.15.131:0/3929382064 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f801f070 con 0x557b2531b890 2024-05-08T22:08:33.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.445+0000 7f87ea000700 1 -- 172.21.15.131:0/3929382064 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f87f8004cd0 con 0x557b2531b890 2024-05-08T22:08:33.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.445+0000 7f881f16bb80 1 -- 172.21.15.131:0/3929382064 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b251c4270 con 0x557b2531b890 2024-05-08T22:08:33.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.445+0000 7f87ea000700 1 -- 172.21.15.131:0/3929382064 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87f8004ed0 con 0x557b2531b890 2024-05-08T22:08:33.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.445+0000 7f881f16bb80 10 client.25781 fetch_fsmap learned FSMap version 24 2024-05-08T22:08:33.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.445+0000 7f881f16bb80 1 -- 172.21.15.131:0/3929382064 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557b25098b60 con 0x557b2531b890 2024-05-08T22:08:33.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.445+0000 7f87ea000700 1 -- 172.21.15.131:0/3929382064 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557b251c4270 con 0x557b2531b890 2024-05-08T22:08:33.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.446+0000 7f87ea000700 1 -- 172.21.15.131:0/3929382064 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f87f8011840 con 0x557b2531b890 2024-05-08T22:08:33.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.446+0000 7f881f16bb80 10 client.25781 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:08:33.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.446+0000 7f881f16bb80 10 client.25781 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:08:33.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.446+0000 7f881f16bb80 20 client.25781 populate_metadata read hostname 'smithi131' 2024-05-08T22:08:33.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.446+0000 7f881f16bb80 1 --2- 172.21.15.131:0/3929382064 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x557b25335150 0x557b25337540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.446+0000 7f881f16bb80 4 client.25781 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:08:33.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.446+0000 7f881f16bb80 1 -- 172.21.15.131:0/3929382064 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b252ea810 con 0x557b25335150 2024-05-08T22:08:33.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.446+0000 7f8803e00700 1 --2- 172.21.15.131:0/3929382064 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x557b25335150 0x557b25337540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.447+0000 7f87ea000700 10 client.25781 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:33.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.447+0000 7f8803e00700 1 --2- 172.21.15.131:0/3929382064 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x557b25335150 0x557b25337540 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a5e4afe7fa297d3 server_cookie=d76aa6ca5831437d in_seq=0 out_seq=0 2024-05-08T22:08:33.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.447+0000 7f87ea000700 1 -- 172.21.15.131:0/3929382064 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557b252ea810 con 0x557b25335150 2024-05-08T22:08:33.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.447+0000 7f87ea000700 10 client.25781 handle_command_reply: tid=0 2024-05-08T22:08:33.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.447+0000 7f881f16bb80 1 client.25781 shutdown 2024-05-08T22:08:33.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.448+0000 7f881f16bb80 1 -- 172.21.15.131:0/3929382064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b2531b890 msgr2=0x557b253277f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:33.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.448+0000 7f881f16bb80 1 --2- 172.21.15.131:0/3929382064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b2531b890 0x557b253277f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f87f80106b0 tx=0x7f87f8010af0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.448+0000 7f881f16bb80 1 -- 172.21.15.131:0/3929382064 shutdown_connections 2024-05-08T22:08:33.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.448+0000 7f881f16bb80 1 --2- 172.21.15.131:0/3929382064 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x557b25335150 0x557b25337540 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.448+0000 7f881f16bb80 1 --2- 172.21.15.131:0/3929382064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b25321970 0x557b25325110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.448+0000 7f881f16bb80 1 --2- 172.21.15.131:0/3929382064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b2531b890 0x557b253277f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.448+0000 7f881f16bb80 1 --2- 172.21.15.131:0/3929382064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b25321050 0x557b25324ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.448+0000 7f881f16bb80 1 -- 172.21.15.131:0/3929382064 >> 172.21.15.131:0/3929382064 conn(0x557b25279210 msgr2=0x557b253160b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:33.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.448+0000 7f87ea000700 0 client.25781 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:33.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.448+0000 7f881f16bb80 1 -- 172.21.15.131:0/3929382064 shutdown_connections 2024-05-08T22:08:33.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.448+0000 7f881f16bb80 1 -- 172.21.15.131:0/3929382064 wait complete. 2024-05-08T22:08:33.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.448+0000 7f881f16bb80 20 client.25781 trim_cache size 0 max 16384 2024-05-08T22:08:33.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.469+0000 7f881f16bb80 1 Processor -- start 2024-05-08T22:08:33.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.469+0000 7f881f16bb80 1 -- start start 2024-05-08T22:08:33.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.469+0000 7f881f16bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b2531b890 0x557b2531b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.469+0000 7f881f16bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b25321050 0x557b2533be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.469+0000 7f881f16bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b25321970 0x557b2532c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.469+0000 7f881f16bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b25315390 con 0x557b2531b890 2024-05-08T22:08:33.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.469+0000 7f881f16bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b25325af0 con 0x557b25321970 2024-05-08T22:08:33.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.469+0000 7f881f16bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b253217c0 con 0x557b25321050 2024-05-08T22:08:33.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.470+0000 7f8808c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b2531b890 0x557b2531b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.470+0000 7f8803e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b25321050 0x557b2533be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.470+0000 7f8808c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b2531b890 0x557b2531b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52744/0 (socket says 172.21.15.131:52744) 2024-05-08T22:08:33.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.470+0000 7f8808c00700 1 -- 172.21.15.131:0/969077975 learned_addr learned my addr 172.21.15.131:0/969077975 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:33.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.470+0000 7f8809600700 1 --2- 172.21.15.131:0/969077975 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b25321970 0x557b2532c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.470+0000 7f8808c00700 1 -- 172.21.15.131:0/969077975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b25321050 msgr2=0x557b2533be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:33.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.470+0000 7f8808c00700 1 --2- 172.21.15.131:0/969077975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b25321050 0x557b2533be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.470+0000 7f8808c00700 1 -- 172.21.15.131:0/969077975 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b25321970 msgr2=0x557b2532c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:33.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.470+0000 7f8808c00700 1 --2- 172.21.15.131:0/969077975 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b25321970 0x557b2532c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.470+0000 7f8808c00700 1 -- 172.21.15.131:0/969077975 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b253124e0 con 0x557b2531b890 2024-05-08T22:08:33.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.470+0000 7f8803e00700 1 --2- 172.21.15.131:0/969077975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b25321050 0x557b2533be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:33.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.470+0000 7f8808c00700 1 --2- 172.21.15.131:0/969077975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b2531b890 0x557b2531b350 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f87f8007fd0 tx=0x7f87f80071c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8fd56447c62b7d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:33.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.471+0000 7f87f6600700 1 -- 172.21.15.131:0/969077975 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f801f070 con 0x557b2531b890 2024-05-08T22:08:33.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.471+0000 7f87f6600700 1 -- 172.21.15.131:0/969077975 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87f8004aa0 con 0x557b2531b890 2024-05-08T22:08:33.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.471+0000 7f87f6600700 1 -- 172.21.15.131:0/969077975 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f8018600 con 0x557b2531b890 2024-05-08T22:08:33.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.472+0000 7f881f16bb80 1 -- 172.21.15.131:0/969077975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b2531b890 msgr2=0x557b2531b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:33.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.472+0000 7f881f16bb80 1 --2- 172.21.15.131:0/969077975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b2531b890 0x557b2531b350 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f87f8007fd0 tx=0x7f87f80071c0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.472+0000 7f881f16bb80 1 -- 172.21.15.131:0/969077975 shutdown_connections 2024-05-08T22:08:33.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.472+0000 7f881f16bb80 1 --2- 172.21.15.131:0/969077975 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b25321970 0x557b2532c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.472+0000 7f881f16bb80 1 --2- 172.21.15.131:0/969077975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b2531b890 0x557b2531b350 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.472+0000 7f881f16bb80 1 --2- 172.21.15.131:0/969077975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b25321050 0x557b2533be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.472+0000 7f881f16bb80 1 -- 172.21.15.131:0/969077975 >> 172.21.15.131:0/969077975 conn(0x557b25279210 msgr2=0x557b2531eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:33.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.472+0000 7f881f16bb80 1 -- 172.21.15.131:0/969077975 shutdown_connections 2024-05-08T22:08:33.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.472+0000 7f881f16bb80 1 -- 172.21.15.131:0/969077975 wait complete. 2024-05-08T22:08:33.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.472+0000 7f881f16bb80 1 -- start start 2024-05-08T22:08:33.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.473+0000 7f881f16bb80 1 Processor -- start 2024-05-08T22:08:33.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.473+0000 7f881f16bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b2531b890 0x557b2532c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.473+0000 7f881f16bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b25321050 0x557b2532c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.473+0000 7f881f16bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b25321970 0x557b2531cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.473+0000 7f881f16bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b2531d2e0 con 0x557b25321050 2024-05-08T22:08:33.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.474+0000 7f881f16bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b2531d450 con 0x557b2531b890 2024-05-08T22:08:33.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.474+0000 7f881f16bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b2531d610 con 0x557b25321970 2024-05-08T22:08:33.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.474+0000 7f881f16bb80 10 client.0 _ll_register_callbacks cb 0x557b25210b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:33.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.474+0000 7f8808c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b2531b890 0x557b2532c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.474+0000 7f8803e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b25321050 0x557b2532c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.474+0000 7f8808c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b2531b890 0x557b2532c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:40200/0 (socket says 172.21.15.131:40200) 2024-05-08T22:08:33.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.474+0000 7f8808c00700 1 -- 172.21.15.131:0/3766597724 learned_addr learned my addr 172.21.15.131:0/3766597724 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:33.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.474+0000 7f8809600700 1 --2- 172.21.15.131:0/3766597724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b25321970 0x557b2531cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.474+0000 7f8808c00700 1 -- 172.21.15.131:0/3766597724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b25321970 msgr2=0x557b2531cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:33.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.474+0000 7f8808c00700 1 --2- 172.21.15.131:0/3766597724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b25321970 0x557b2531cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.474+0000 7f8808c00700 1 -- 172.21.15.131:0/3766597724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b25321050 msgr2=0x557b2532c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:33.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.474+0000 7f8808c00700 1 --2- 172.21.15.131:0/3766597724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b25321050 0x557b2532c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.474+0000 7f8808c00700 1 -- 172.21.15.131:0/3766597724 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88041a6210 con 0x557b2531b890 2024-05-08T22:08:33.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.474+0000 7f8803e00700 1 --2- 172.21.15.131:0/3766597724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b25321050 0x557b2532c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:33.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.475+0000 7f8808c00700 1 --2- 172.21.15.131:0/3766597724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b2531b890 0x557b2532c360 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f87f8010b30 tx=0x7f87f80075f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff740d5a3a6503e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:33.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.475+0000 7f87ea000700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:08:33.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.475+0000 7f87ea000700 1 -- 172.21.15.131:0/3766597724 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f801f040 con 0x557b2531b890 2024-05-08T22:08:33.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.475+0000 7f87ea000700 1 -- 172.21.15.131:0/3766597724 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f87f8011850 con 0x557b2531b890 2024-05-08T22:08:33.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.475+0000 7f881f16bb80 1 -- 172.21.15.131:0/3766597724 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87f80044f0 con 0x557b2531b890 2024-05-08T22:08:33.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.475+0000 7f87ea000700 1 -- 172.21.15.131:0/3766597724 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87f8011a30 con 0x557b2531b890 2024-05-08T22:08:33.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.475+0000 7f881f16bb80 10 client.25787 fetch_fsmap learned FSMap version 24 2024-05-08T22:08:33.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.475+0000 7f881f16bb80 1 -- 172.21.15.131:0/3766597724 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557b25098b60 con 0x557b2531b890 2024-05-08T22:08:33.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.476+0000 7f87ea000700 1 -- 172.21.15.131:0/3766597724 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87f80044f0 con 0x557b2531b890 2024-05-08T22:08:33.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.476+0000 7f87ea000700 1 -- 172.21.15.131:0/3766597724 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f87f80103d0 con 0x557b2531b890 2024-05-08T22:08:33.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.476+0000 7f881f16bb80 10 client.25787 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:08:33.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.476+0000 7f881f16bb80 10 client.25787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:08:33.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.476+0000 7f881f16bb80 20 client.25787 populate_metadata read hostname 'smithi131' 2024-05-08T22:08:33.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.476+0000 7f881f16bb80 1 --2- 172.21.15.131:0/3766597724 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x557b2538b710 0x557b2538db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.476+0000 7f881f16bb80 4 client.25787 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:08:33.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.476+0000 7f881f16bb80 1 -- 172.21.15.131:0/3766597724 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557b25263ec0 con 0x557b2538b710 2024-05-08T22:08:33.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.477+0000 7f8803e00700 1 --2- 172.21.15.131:0/3766597724 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x557b2538b710 0x557b2538db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.477+0000 7f87ea000700 10 client.25787 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:33.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.477+0000 7f8803e00700 1 --2- 172.21.15.131:0/3766597724 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x557b2538b710 0x557b2538db30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce38cf1aae69d12e server_cookie=dfe2de204f424d09 in_seq=0 out_seq=0 2024-05-08T22:08:33.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.477+0000 7f87ea000700 1 -- 172.21.15.131:0/3766597724 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557b25263ec0 con 0x557b2538b710 2024-05-08T22:08:33.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.477+0000 7f87ea000700 10 client.25787 handle_command_reply: tid=0 2024-05-08T22:08:33.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.477+0000 7f881f16bb80 1 client.25787 shutdown 2024-05-08T22:08:33.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.478+0000 7f881f16bb80 1 -- 172.21.15.131:0/3766597724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b2531b890 msgr2=0x557b2532c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:33.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.478+0000 7f881f16bb80 1 --2- 172.21.15.131:0/3766597724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b2531b890 0x557b2532c360 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f87f8010b30 tx=0x7f87f80075f0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.478+0000 7f881f16bb80 1 -- 172.21.15.131:0/3766597724 shutdown_connections 2024-05-08T22:08:33.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.478+0000 7f881f16bb80 1 --2- 172.21.15.131:0/3766597724 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x557b2538b710 0x557b2538db30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.478+0000 7f881f16bb80 1 --2- 172.21.15.131:0/3766597724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b25321970 0x557b2531cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.478+0000 7f881f16bb80 1 --2- 172.21.15.131:0/3766597724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b2531b890 0x557b2532c360 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.478+0000 7f87ea000700 0 client.25787 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:33.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.478+0000 7f881f16bb80 1 --2- 172.21.15.131:0/3766597724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557b25321050 0x557b2532c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.478+0000 7f881f16bb80 1 -- 172.21.15.131:0/3766597724 >> 172.21.15.131:0/3766597724 conn(0x557b25279210 msgr2=0x557b25330e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:33.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.478+0000 7f881f16bb80 1 -- 172.21.15.131:0/3766597724 shutdown_connections 2024-05-08T22:08:33.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.478+0000 7f881f16bb80 1 -- 172.21.15.131:0/3766597724 wait complete. 2024-05-08T22:08:33.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.478+0000 7f881f16bb80 20 client.25787 trim_cache size 0 max 16384 2024-05-08T22:08:33.481 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:08:33.482 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-08T22:08:33.482 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-08T22:08:33.482 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:08:33.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.479+0000 7f880a000700 1 -- 172.21.15.131:0/396407763 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f87ec0a1440 msgr2=0x7f87ec0a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:33.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.479+0000 7f880a000700 1 --2- 172.21.15.131:0/396407763 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f87ec0a1440 0x7f87ec0a3900 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f87fc009fd0 tx=0x7f87fc00b040 comp rx=0 tx=0).stop 2024-05-08T22:08:33.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.479+0000 7f880a000700 1 -- 172.21.15.131:0/396407763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8804073c90 msgr2=0x7f88041999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:33.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.479+0000 7f880a000700 1 --2- 172.21.15.131:0/396407763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8804073c90 0x7f88041999e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f87f0005910 tx=0x7f87f0004730 comp rx=0 tx=0).stop 2024-05-08T22:08:33.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.479+0000 7f880a000700 1 -- 172.21.15.131:0/396407763 shutdown_connections 2024-05-08T22:08:33.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.479+0000 7f880a000700 1 --2- 172.21.15.131:0/396407763 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f87ec0a1440 0x7f87ec0a3900 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.479+0000 7f880a000700 1 --2- 172.21.15.131:0/396407763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f880410d600 0x7f880419dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.479+0000 7f880a000700 1 --2- 172.21.15.131:0/396407763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8804073370 0x7f88041994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.479+0000 7f880a000700 1 --2- 172.21.15.131:0/396407763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8804073c90 0x7f88041999e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.479+0000 7f880a000700 1 -- 172.21.15.131:0/396407763 >> 172.21.15.131:0/396407763 conn(0x7f88040fc870 msgr2=0x7f88040fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:33.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.479+0000 7f880a000700 1 -- 172.21.15.131:0/396407763 shutdown_connections 2024-05-08T22:08:33.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.479+0000 7f880a000700 1 -- 172.21.15.131:0/396407763 wait complete. 2024-05-08T22:08:33.505 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T22:08:33.505 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:08:33.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.762+0000 7f00d0a00700 1 -- 172.21.15.131:0/485858688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00cc10b390 msgr2=0x7f00cc10d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:33.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.762+0000 7f00d0a00700 1 --2- 172.21.15.131:0/485858688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00cc10b390 0x7f00cc10d850 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f00c400b0a0 tx=0x7f00c400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.762+0000 7f00d0a00700 1 -- 172.21.15.131:0/485858688 shutdown_connections 2024-05-08T22:08:33.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.762+0000 7f00d0a00700 1 --2- 172.21.15.131:0/485858688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00cc10b390 0x7f00cc10d850 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.762+0000 7f00d0a00700 1 --2- 172.21.15.131:0/485858688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00cc101f70 0x7f00cc102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.762+0000 7f00d0a00700 1 --2- 172.21.15.131:0/485858688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00cc102890 0x7f00cc10ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.762+0000 7f00d0a00700 1 -- 172.21.15.131:0/485858688 >> 172.21.15.131:0/485858688 conn(0x7f00cc0fb790 msgr2=0x7f00cc0fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:33.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.762+0000 7f00d0a00700 1 -- 172.21.15.131:0/485858688 shutdown_connections 2024-05-08T22:08:33.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.762+0000 7f00d0a00700 1 -- 172.21.15.131:0/485858688 wait complete. 2024-05-08T22:08:33.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.763+0000 7f00d0a00700 1 Processor -- start 2024-05-08T22:08:33.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.763+0000 7f00d0a00700 1 -- start start 2024-05-08T22:08:33.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.763+0000 7f00d0a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00cc101f70 0x7f00cc0ffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.763+0000 7f00d0a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00cc102890 0x7f00cc100100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.763+0000 7f00d0a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00cc10b390 0x7f00cc100640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.763+0000 7f00d0a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc1a57d0 con 0x7f00cc101f70 2024-05-08T22:08:33.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.763+0000 7f00d0a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc1a5940 con 0x7f00cc10b390 2024-05-08T22:08:33.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.763+0000 7f00d0a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc1a5b00 con 0x7f00cc102890 2024-05-08T22:08:33.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.764+0000 7f00cb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00cc101f70 0x7f00cc0ffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.764+0000 7f00cb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00cc101f70 0x7f00cc0ffbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52754/0 (socket says 172.21.15.131:52754) 2024-05-08T22:08:33.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.764+0000 7f00cb400700 1 -- 172.21.15.131:0/4147742059 learned_addr learned my addr 172.21.15.131:0/4147742059 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:33.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.764+0000 7f00caa00700 1 --2- 172.21.15.131:0/4147742059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00cc102890 0x7f00cc100100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.764+0000 7f00cbe00700 1 --2- 172.21.15.131:0/4147742059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00cc10b390 0x7f00cc100640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.764+0000 7f00cb400700 1 -- 172.21.15.131:0/4147742059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00cc102890 msgr2=0x7f00cc100100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:33.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.764+0000 7f00cb400700 1 --2- 172.21.15.131:0/4147742059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00cc102890 0x7f00cc100100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.764+0000 7f00cb400700 1 -- 172.21.15.131:0/4147742059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00cc10b390 msgr2=0x7f00cc100640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:33.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.764+0000 7f00cb400700 1 --2- 172.21.15.131:0/4147742059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00cc10b390 0x7f00cc100640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:33.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.764+0000 7f00cb400700 1 -- 172.21.15.131:0/4147742059 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c4009d00 con 0x7f00cc101f70 2024-05-08T22:08:33.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.764+0000 7f00cbe00700 1 --2- 172.21.15.131:0/4147742059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00cc10b390 0x7f00cc100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:08:33.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.765+0000 7f00cb400700 1 --2- 172.21.15.131:0/4147742059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00cc101f70 0x7f00cc0ffbc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00e9a0 tx=0x7f00bc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:33.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.765+0000 7f00c3e00700 1 -- 172.21.15.131:0/4147742059 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc009790 con 0x7f00cc101f70 2024-05-08T22:08:33.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.765+0000 7f00c3e00700 1 -- 172.21.15.131:0/4147742059 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00bc00cbd0 con 0x7f00cc101f70 2024-05-08T22:08:33.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.765+0000 7f00c3e00700 1 -- 172.21.15.131:0/4147742059 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc018680 con 0x7f00cc101f70 2024-05-08T22:08:33.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.765+0000 7f00d0a00700 1 -- 172.21.15.131:0/4147742059 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00cc1a5de0 con 0x7f00cc101f70 2024-05-08T22:08:33.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.765+0000 7f00d0a00700 1 -- 172.21.15.131:0/4147742059 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00cc1a6210 con 0x7f00cc101f70 2024-05-08T22:08:33.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.767+0000 7f00d0a00700 1 -- 172.21.15.131:0/4147742059 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00cc1064c0 con 0x7f00cc101f70 2024-05-08T22:08:33.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.768+0000 7f00c3e00700 1 -- 172.21.15.131:0/4147742059 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 24) v1 ==== 136158+0+0 (secure 0 0 0) 0x7f00bc00cd40 con 0x7f00cc101f70 2024-05-08T22:08:33.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.770+0000 7f00c3e00700 1 --2- 172.21.15.131:0/4147742059 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f00b00a14c0 0x7f00b00a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:33.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.771+0000 7f00c3e00700 1 -- 172.21.15.131:0/4147742059 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f00bc014070 con 0x7f00cc101f70 2024-05-08T22:08:33.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.771+0000 7f00caa00700 1 --2- 172.21.15.131:0/4147742059 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f00b00a14c0 0x7f00b00a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:33.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.771+0000 7f00caa00700 1 --2- 172.21.15.131:0/4147742059 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f00b00a14c0 0x7f00b00a3980 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f00cc1012a0 tx=0x7f00b8009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:33.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:33.777+0000 7f00c3e00700 1 -- 172.21.15.131:0/4147742059 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f00bc052d60 con 0x7f00cc101f70 2024-05-08T22:08:34.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.056+0000 7f00d0a00700 1 -- 172.21.15.131:0/4147742059 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00cc101050 con 0x7f00cc101f70 2024-05-08T22:08:34.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.057+0000 7f00c3e00700 1 -- 172.21.15.131:0/4147742059 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10889 (secure 0 0 0) 0x7f00bc0963b0 con 0x7f00cc101f70 2024-05-08T22:08:34.058 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:08:34.059 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:05:14.034908+0000","modified":"2024-05-08T22:07:19.165885+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:08:34.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.061+0000 7f00d0a00700 1 -- 172.21.15.131:0/4147742059 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f00b00a14c0 msgr2=0x7f00b00a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:34.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.061+0000 7f00d0a00700 1 --2- 172.21.15.131:0/4147742059 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f00b00a14c0 0x7f00b00a3980 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f00cc1012a0 tx=0x7f00b8009260 comp rx=0 tx=0).stop 2024-05-08T22:08:34.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.061+0000 7f00d0a00700 1 -- 172.21.15.131:0/4147742059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00cc101f70 msgr2=0x7f00cc0ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:34.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.061+0000 7f00d0a00700 1 --2- 172.21.15.131:0/4147742059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00cc101f70 0x7f00cc0ffbc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00e9a0 tx=0x7f00bc00ecb0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.061+0000 7f00d0a00700 1 -- 172.21.15.131:0/4147742059 shutdown_connections 2024-05-08T22:08:34.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.061+0000 7f00d0a00700 1 --2- 172.21.15.131:0/4147742059 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f00b00a14c0 0x7f00b00a3980 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.061+0000 7f00d0a00700 1 --2- 172.21.15.131:0/4147742059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00cc10b390 0x7f00cc100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.061+0000 7f00d0a00700 1 --2- 172.21.15.131:0/4147742059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00cc101f70 0x7f00cc0ffbc0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.061+0000 7f00d0a00700 1 --2- 172.21.15.131:0/4147742059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00cc102890 0x7f00cc100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.061+0000 7f00d0a00700 1 -- 172.21.15.131:0/4147742059 >> 172.21.15.131:0/4147742059 conn(0x7f00cc0fb790 msgr2=0x7f00cc0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:34.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.061+0000 7f00d0a00700 1 -- 172.21.15.131:0/4147742059 shutdown_connections 2024-05-08T22:08:34.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.062+0000 7f00d0a00700 1 -- 172.21.15.131:0/4147742059 wait complete. 2024-05-08T22:08:34.064 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 24 2024-05-08T22:08:34.087 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T22:08:34.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.338+0000 7faa0b200700 1 -- 172.21.15.131:0/2335655402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa04073340 msgr2=0x7faa04073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:34.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.338+0000 7faa0b200700 1 --2- 172.21.15.131:0/2335655402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa04073340 0x7faa04073720 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009990 tx=0x7fa9f8009ca0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.338+0000 7faa0b200700 1 -- 172.21.15.131:0/2335655402 shutdown_connections 2024-05-08T22:08:34.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.338+0000 7faa0b200700 1 --2- 172.21.15.131:0/2335655402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa04111d30 0x7faa04114120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.338+0000 7faa0b200700 1 --2- 172.21.15.131:0/2335655402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa04073340 0x7faa04073720 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.338+0000 7faa0b200700 1 --2- 172.21.15.131:0/2335655402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa04073c60 0x7faa041117f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.338+0000 7faa0b200700 1 -- 172.21.15.131:0/2335655402 >> 172.21.15.131:0/2335655402 conn(0x7faa040fc810 msgr2=0x7faa040fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:34.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.338+0000 7faa0b200700 1 -- 172.21.15.131:0/2335655402 shutdown_connections 2024-05-08T22:08:34.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.339+0000 7faa0b200700 1 -- 172.21.15.131:0/2335655402 wait complete. 2024-05-08T22:08:34.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.340+0000 7faa0b200700 1 Processor -- start 2024-05-08T22:08:34.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.340+0000 7faa0b200700 1 -- start start 2024-05-08T22:08:34.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.340+0000 7faa0b200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa04073340 0x7faa0419d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.340+0000 7faa0b200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa04073c60 0x7faa0419dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.340+0000 7faa0b200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa04111d30 0x7faa041a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.340+0000 7faa0b200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa041a2620 con 0x7faa04073340 2024-05-08T22:08:34.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.340+0000 7faa0b200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa041a2790 con 0x7faa04111d30 2024-05-08T22:08:34.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.340+0000 7faa0b200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa041a2950 con 0x7faa04073c60 2024-05-08T22:08:34.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.341+0000 7faa09e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa04073340 0x7faa0419d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.341+0000 7faa09400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa04073c60 0x7faa0419dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.341+0000 7faa0a800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa04111d30 0x7faa041a1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.341+0000 7faa09400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa04073c60 0x7faa0419dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:47540/0 (socket says 172.21.15.131:47540) 2024-05-08T22:08:34.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.341+0000 7faa09400700 1 -- 172.21.15.131:0/2118074600 learned_addr learned my addr 172.21.15.131:0/2118074600 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:34.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.341+0000 7faa09400700 1 -- 172.21.15.131:0/2118074600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa04111d30 msgr2=0x7faa041a1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:34.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.341+0000 7faa09400700 1 --2- 172.21.15.131:0/2118074600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa04111d30 0x7faa041a1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.341+0000 7faa09400700 1 -- 172.21.15.131:0/2118074600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa04073340 msgr2=0x7faa0419d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:34.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.341+0000 7faa09400700 1 --2- 172.21.15.131:0/2118074600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa04073340 0x7faa0419d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.341+0000 7faa09400700 1 -- 172.21.15.131:0/2118074600 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f80095f0 con 0x7faa04073c60 2024-05-08T22:08:34.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.342+0000 7faa09400700 1 --2- 172.21.15.131:0/2118074600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa04073c60 0x7faa0419dc90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa9f000c9b0 tx=0x7fa9f000ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:34.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.342+0000 7fa9fea00700 1 -- 172.21.15.131:0/2118074600 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0007b00 con 0x7faa04073c60 2024-05-08T22:08:34.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.342+0000 7fa9fea00700 1 -- 172.21.15.131:0/2118074600 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9f000ce50 con 0x7faa04073c60 2024-05-08T22:08:34.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.342+0000 7fa9fea00700 1 -- 172.21.15.131:0/2118074600 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0018660 con 0x7faa04073c60 2024-05-08T22:08:34.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.343+0000 7faa0b200700 1 -- 172.21.15.131:0/2118074600 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa041a2c30 con 0x7faa04073c60 2024-05-08T22:08:34.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.343+0000 7faa0b200700 1 -- 172.21.15.131:0/2118074600 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa041aa470 con 0x7faa04073c60 2024-05-08T22:08:34.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.346+0000 7fa9fea00700 1 -- 172.21.15.131:0/2118074600 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 24) v1 ==== 136158+0+0 (secure 0 0 0) 0x7fa9f000f450 con 0x7faa04073c60 2024-05-08T22:08:34.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.348+0000 7fa9fea00700 1 --2- 172.21.15.131:0/2118074600 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa9ec0a14c0 0x7fa9ec0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.348+0000 7fa9fea00700 1 -- 172.21.15.131:0/2118074600 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fa9f00cd0d0 con 0x7faa04073c60 2024-05-08T22:08:34.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.348+0000 7faa09e00700 1 --2- 172.21.15.131:0/2118074600 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa9ec0a14c0 0x7fa9ec0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.349+0000 7faa202b4b80 1 Processor -- start 2024-05-08T22:08:34.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.349+0000 7faa09e00700 1 --2- 172.21.15.131:0/2118074600 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa9ec0a14c0 0x7fa9ec0a3980 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8006010 tx=0x7fa9f800b540 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:34.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.349+0000 7faa202b4b80 1 -- start start 2024-05-08T22:08:34.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.350+0000 7faa202b4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c477168050 0x55c477168430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.350+0000 7faa202b4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c477168970 0x55c477162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.350+0000 7faa202b4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c477162890 0x55c477162cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.350+0000 7faa202b4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4770b9150 con 0x55c477168050 2024-05-08T22:08:34.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.350+0000 7faa202b4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4770b6b10 con 0x55c477162890 2024-05-08T22:08:34.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.350+0000 7faa202b4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4770b2400 con 0x55c477168970 2024-05-08T22:08:34.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.350+0000 7faa09e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c477168050 0x55c477168430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.350+0000 7faa09e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c477168050 0x55c477168430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36288/0 (socket says 172.21.15.131:36288) 2024-05-08T22:08:34.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.350+0000 7faa09e00700 1 -- 172.21.15.131:0/179566775 learned_addr learned my addr 172.21.15.131:0/179566775 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:34.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.350+0000 7faa0a800700 1 --2- 172.21.15.131:0/179566775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c477162890 0x55c477162cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.350+0000 7faa09400700 1 --2- 172.21.15.131:0/179566775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c477168970 0x55c477162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.351+0000 7faa09e00700 1 -- 172.21.15.131:0/179566775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c477168970 msgr2=0x55c477162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:34.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.351+0000 7faa09e00700 1 --2- 172.21.15.131:0/179566775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c477168970 0x55c477162350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.351+0000 7faa09e00700 1 -- 172.21.15.131:0/179566775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c477162890 msgr2=0x55c477162cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:34.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.351+0000 7faa09e00700 1 --2- 172.21.15.131:0/179566775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c477162890 0x55c477162cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.351+0000 7faa09e00700 1 -- 172.21.15.131:0/179566775 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c476f333b0 con 0x55c477168050 2024-05-08T22:08:34.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.351+0000 7faa09e00700 1 --2- 172.21.15.131:0/179566775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c477168050 0x55c477168430 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800b9d0 tx=0x7fa9f801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f64926289fb5b79 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:34.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.351+0000 7fa9f7000700 1 -- 172.21.15.131:0/179566775 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8029070 con 0x55c477168050 2024-05-08T22:08:34.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.351+0000 7fa9f7000700 1 -- 172.21.15.131:0/179566775 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9f801e040 con 0x55c477168050 2024-05-08T22:08:34.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.351+0000 7fa9f7000700 1 -- 172.21.15.131:0/179566775 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f800f830 con 0x55c477168050 2024-05-08T22:08:34.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.352+0000 7faa202b4b80 1 -- 172.21.15.131:0/179566775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c477168050 msgr2=0x55c477168430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:34.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.352+0000 7faa202b4b80 1 --2- 172.21.15.131:0/179566775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c477168050 0x55c477168430 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800b9d0 tx=0x7fa9f801f000 comp rx=0 tx=0).stop 2024-05-08T22:08:34.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.353+0000 7faa202b4b80 1 -- 172.21.15.131:0/179566775 shutdown_connections 2024-05-08T22:08:34.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.353+0000 7faa202b4b80 1 --2- 172.21.15.131:0/179566775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c477162890 0x55c477162cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.353+0000 7faa202b4b80 1 --2- 172.21.15.131:0/179566775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c477168050 0x55c477168430 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.353+0000 7faa202b4b80 1 --2- 172.21.15.131:0/179566775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c477168970 0x55c477162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.353+0000 7faa202b4b80 1 -- 172.21.15.131:0/179566775 >> 172.21.15.131:0/179566775 conn(0x55c4770c0210 msgr2=0x55c47715d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:34.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.353+0000 7faa202b4b80 1 -- 172.21.15.131:0/179566775 shutdown_connections 2024-05-08T22:08:34.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.353+0000 7faa202b4b80 1 -- 172.21.15.131:0/179566775 wait complete. 2024-05-08T22:08:34.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.353+0000 7faa202b4b80 1 -- start start 2024-05-08T22:08:34.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.354+0000 7faa202b4b80 1 Processor -- start 2024-05-08T22:08:34.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.354+0000 7faa202b4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c477162890 0x55c47716e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.354+0000 7faa202b4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c477168050 0x55c47716bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.354+0000 7faa202b4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c477168970 0x55c47716c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.354+0000 7faa202b4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47704e560 con 0x55c477168050 2024-05-08T22:08:34.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.354+0000 7faa202b4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47704e200 con 0x55c477168970 2024-05-08T22:08:34.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.354+0000 7faa202b4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c476f356a0 con 0x55c477162890 2024-05-08T22:08:34.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.354+0000 7faa202b4b80 10 client.0 _ll_register_callbacks cb 0x55c477057b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:34.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.355+0000 7faa09400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c477168050 0x55c47716bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.355+0000 7faa09400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c477168050 0x55c47716bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36296/0 (socket says 172.21.15.131:36296) 2024-05-08T22:08:34.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.355+0000 7faa09400700 1 -- 172.21.15.131:0/4260449477 learned_addr learned my addr 172.21.15.131:0/4260449477 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:34.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.355+0000 7faa0a800700 1 --2- 172.21.15.131:0/4260449477 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c477168970 0x55c47716c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.355+0000 7faa09e00700 1 --2- 172.21.15.131:0/4260449477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c477162890 0x55c47716e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.355+0000 7faa09400700 1 -- 172.21.15.131:0/4260449477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c477162890 msgr2=0x55c47716e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:34.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.355+0000 7faa09400700 1 --2- 172.21.15.131:0/4260449477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c477162890 0x55c47716e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.355+0000 7faa09400700 1 -- 172.21.15.131:0/4260449477 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c477168970 msgr2=0x55c47716c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:34.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.355+0000 7faa09400700 1 --2- 172.21.15.131:0/4260449477 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c477168970 0x55c47716c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.355+0000 7faa09400700 1 -- 172.21.15.131:0/4260449477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa04114ad0 con 0x55c477168050 2024-05-08T22:08:34.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.355+0000 7faa09e00700 1 --2- 172.21.15.131:0/4260449477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c477162890 0x55c47716e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:34.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.356+0000 7faa09400700 1 --2- 172.21.15.131:0/4260449477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c477168050 0x55c47716bc10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0099910 tx=0x7fa9f0007c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6b7de3a27f86be server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:34.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.356+0000 7fa9eb400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:08:34.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.356+0000 7fa9eb400700 1 -- 172.21.15.131:0/4260449477 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f00b6070 con 0x55c477168050 2024-05-08T22:08:34.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.356+0000 7fa9eb400700 1 -- 172.21.15.131:0/4260449477 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fa9f0016510 con 0x55c477168050 2024-05-08T22:08:34.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.356+0000 7faa202b4b80 1 -- 172.21.15.131:0/4260449477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c47700b270 con 0x55c477168050 2024-05-08T22:08:34.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.356+0000 7fa9eb400700 1 -- 172.21.15.131:0/4260449477 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9f0099c10 con 0x55c477168050 2024-05-08T22:08:34.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.356+0000 7faa202b4b80 10 client.15765 fetch_fsmap learned FSMap version 24 2024-05-08T22:08:34.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.356+0000 7faa202b4b80 1 -- 172.21.15.131:0/4260449477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c476edfb60 con 0x55c477168050 2024-05-08T22:08:34.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.357+0000 7fa9eb400700 1 -- 172.21.15.131:0/4260449477 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c47700b270 con 0x55c477168050 2024-05-08T22:08:34.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.357+0000 7fa9eb400700 1 -- 172.21.15.131:0/4260449477 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa9f0099e70 con 0x55c477168050 2024-05-08T22:08:34.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.357+0000 7faa202b4b80 10 client.15765 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:08:34.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.357+0000 7faa202b4b80 10 client.15765 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:08:34.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.357+0000 7faa202b4b80 20 client.15765 populate_metadata read hostname 'smithi131' 2024-05-08T22:08:34.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.357+0000 7faa202b4b80 1 --2- 172.21.15.131:0/4260449477 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55c47717c140 0x55c47717e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.357+0000 7faa202b4b80 4 client.15765 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:08:34.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.357+0000 7faa202b4b80 1 -- 172.21.15.131:0/4260449477 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c477131810 con 0x55c47717c140 2024-05-08T22:08:34.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.357+0000 7faa09e00700 1 --2- 172.21.15.131:0/4260449477 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55c47717c140 0x55c47717e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.358+0000 7faa09e00700 1 --2- 172.21.15.131:0/4260449477 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55c47717c140 0x55c47717e560 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=426fc71ee735750f server_cookie=d7400038f8a0a8c2 in_seq=0 out_seq=0 2024-05-08T22:08:34.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.358+0000 7fa9eb400700 10 client.15765 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:08:34.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.358+0000 7fa9eb400700 1 -- 172.21.15.131:0/4260449477 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c477131810 con 0x55c47717c140 2024-05-08T22:08:34.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.358+0000 7fa9eb400700 10 client.15765 handle_command_reply: tid=0 2024-05-08T22:08:34.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.359+0000 7faa202b4b80 1 client.15765 shutdown 2024-05-08T22:08:34.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.359+0000 7faa202b4b80 1 -- 172.21.15.131:0/4260449477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c477168050 msgr2=0x55c47716bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:34.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.359+0000 7faa202b4b80 1 --2- 172.21.15.131:0/4260449477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c477168050 0x55c47716bc10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0099910 tx=0x7fa9f0007c60 comp rx=0 tx=0).stop 2024-05-08T22:08:34.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.359+0000 7faa202b4b80 1 -- 172.21.15.131:0/4260449477 shutdown_connections 2024-05-08T22:08:34.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.359+0000 7faa202b4b80 1 --2- 172.21.15.131:0/4260449477 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55c47717c140 0x55c47717e560 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.359+0000 7faa202b4b80 1 --2- 172.21.15.131:0/4260449477 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c477168970 0x55c47716c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.359+0000 7faa202b4b80 1 --2- 172.21.15.131:0/4260449477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c477162890 0x55c47716e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.359+0000 7faa202b4b80 1 --2- 172.21.15.131:0/4260449477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c477168050 0x55c47716bc10 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.359+0000 7faa202b4b80 1 -- 172.21.15.131:0/4260449477 >> 172.21.15.131:0/4260449477 conn(0x55c4770c0210 msgr2=0x55c47715d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:34.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.359+0000 7fa9eb400700 0 client.15765 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:08:34.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.359+0000 7faa202b4b80 1 -- 172.21.15.131:0/4260449477 shutdown_connections 2024-05-08T22:08:34.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.359+0000 7faa202b4b80 1 -- 172.21.15.131:0/4260449477 wait complete. 2024-05-08T22:08:34.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.360+0000 7faa202b4b80 20 client.15765 trim_cache size 0 max 16384 2024-05-08T22:08:34.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.384+0000 7faa202b4b80 1 Processor -- start 2024-05-08T22:08:34.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.384+0000 7faa202b4b80 1 -- start start 2024-05-08T22:08:34.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.384+0000 7faa202b4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c477162890 0x55c477162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.384+0000 7faa202b4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c477168050 0x55c47717e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.384+0000 7faa202b4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c477168970 0x55c47716bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.384+0000 7faa202b4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47715ee20 con 0x55c477168970 2024-05-08T22:08:34.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.384+0000 7faa202b4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47716cb20 con 0x55c477168050 2024-05-08T22:08:34.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.384+0000 7faa202b4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c477168730 con 0x55c477162890 2024-05-08T22:08:34.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.385+0000 7faa09400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c477168050 0x55c47717e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.385+0000 7faa09e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c477162890 0x55c477162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.385+0000 7faa09400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c477168050 0x55c47717e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:41458/0 (socket says 172.21.15.131:41458) 2024-05-08T22:08:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.385+0000 7faa09400700 1 -- 172.21.15.131:0/3167556751 learned_addr learned my addr 172.21.15.131:0/3167556751 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.385+0000 7faa0a800700 1 --2- 172.21.15.131:0/3167556751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c477168970 0x55c47716bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.385+0000 7faa09e00700 1 -- 172.21.15.131:0/3167556751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c477168050 msgr2=0x55c47717e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.385+0000 7faa09e00700 1 --2- 172.21.15.131:0/3167556751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c477168050 0x55c47717e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.385+0000 7faa09e00700 1 -- 172.21.15.131:0/3167556751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c477168970 msgr2=0x55c47716bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.385+0000 7faa09e00700 1 --2- 172.21.15.131:0/3167556751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c477168970 0x55c47716bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.385+0000 7faa09e00700 1 -- 172.21.15.131:0/3167556751 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c477078560 con 0x55c477162890 2024-05-08T22:08:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.385+0000 7faa09e00700 1 --2- 172.21.15.131:0/3167556751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c477162890 0x55c477162350 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8005630 tx=0x7fa9f80058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e3440bdf83c518e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.386+0000 7fa9f7a00700 1 -- 172.21.15.131:0/3167556751 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8029040 con 0x55c477162890 2024-05-08T22:08:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.386+0000 7fa9f7a00700 1 -- 172.21.15.131:0/3167556751 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9f800bb00 con 0x55c477162890 2024-05-08T22:08:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.386+0000 7fa9f7a00700 1 -- 172.21.15.131:0/3167556751 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f800f6a0 con 0x55c477162890 2024-05-08T22:08:34.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.387+0000 7faa202b4b80 1 -- 172.21.15.131:0/3167556751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c477162890 msgr2=0x55c477162350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:34.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.387+0000 7faa202b4b80 1 --2- 172.21.15.131:0/3167556751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c477162890 0x55c477162350 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8005630 tx=0x7fa9f80058e0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.387+0000 7faa202b4b80 1 -- 172.21.15.131:0/3167556751 shutdown_connections 2024-05-08T22:08:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.387+0000 7faa202b4b80 1 --2- 172.21.15.131:0/3167556751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c477168970 0x55c47716bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.387+0000 7faa202b4b80 1 --2- 172.21.15.131:0/3167556751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c477162890 0x55c477162350 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.387+0000 7faa202b4b80 1 --2- 172.21.15.131:0/3167556751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c477168050 0x55c47717e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.387+0000 7faa202b4b80 1 -- 172.21.15.131:0/3167556751 >> 172.21.15.131:0/3167556751 conn(0x55c4770c0210 msgr2=0x55c47716b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.387+0000 7faa202b4b80 1 -- 172.21.15.131:0/3167556751 shutdown_connections 2024-05-08T22:08:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.387+0000 7faa202b4b80 1 -- 172.21.15.131:0/3167556751 wait complete. 2024-05-08T22:08:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.387+0000 7faa202b4b80 1 -- start start 2024-05-08T22:08:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.388+0000 7faa202b4b80 1 Processor -- start 2024-05-08T22:08:34.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.388+0000 7faa202b4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c477162890 0x55c4771732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.388+0000 7faa202b4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c477168050 0x55c4771737e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.388+0000 7faa202b4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c477168970 0x55c47716dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.388+0000 7faa202b4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47716e2c0 con 0x55c477168050 2024-05-08T22:08:34.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.388+0000 7faa202b4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47716e430 con 0x55c477162890 2024-05-08T22:08:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.388+0000 7faa202b4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47716e5f0 con 0x55c477168970 2024-05-08T22:08:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.388+0000 7faa202b4b80 10 client.0 _ll_register_callbacks cb 0x55c477057b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.389+0000 7faa09e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c477162890 0x55c4771732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.389+0000 7faa09e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c477162890 0x55c4771732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:41462/0 (socket says 172.21.15.131:41462) 2024-05-08T22:08:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.389+0000 7faa09e00700 1 -- 172.21.15.131:0/736895646 learned_addr learned my addr 172.21.15.131:0/736895646 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.389+0000 7faa0a800700 1 --2- 172.21.15.131:0/736895646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c477168970 0x55c47716dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.389+0000 7faa09400700 1 --2- 172.21.15.131:0/736895646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c477168050 0x55c4771737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.389+0000 7faa0a800700 1 -- 172.21.15.131:0/736895646 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c477162890 msgr2=0x55c4771732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.389+0000 7faa0a800700 1 --2- 172.21.15.131:0/736895646 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c477162890 0x55c4771732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.389+0000 7faa0a800700 1 -- 172.21.15.131:0/736895646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c477168050 msgr2=0x55c4771737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.389+0000 7faa0a800700 1 --2- 172.21.15.131:0/736895646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c477168050 0x55c4771737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.389+0000 7faa0a800700 1 -- 172.21.15.131:0/736895646 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c476f333b0 con 0x55c477168970 2024-05-08T22:08:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.389+0000 7faa09400700 1 --2- 172.21.15.131:0/736895646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c477168050 0x55c4771737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:34.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:34 smithi131 ceph-mon[37260]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:08:34.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:34 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/4147742059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:08:34.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.390+0000 7faa0a800700 1 --2- 172.21.15.131:0/736895646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c477168970 0x55c47716dd80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faa00016f00 tx=0x7faa00016f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84ba4b0db243ee17 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:34.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.390+0000 7fa9eb400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:08:34.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.390+0000 7fa9eb400700 1 -- 172.21.15.131:0/736895646 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0001b070 con 0x55c477168970 2024-05-08T22:08:34.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.390+0000 7fa9eb400700 1 -- 172.21.15.131:0/736895646 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7faa00015ad0 con 0x55c477168970 2024-05-08T22:08:34.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.390+0000 7faa202b4b80 1 -- 172.21.15.131:0/736895646 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9f801e040 con 0x55c477168970 2024-05-08T22:08:34.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.391+0000 7fa9eb400700 1 -- 172.21.15.131:0/736895646 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa00015c90 con 0x55c477168970 2024-05-08T22:08:34.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.391+0000 7faa202b4b80 10 client.25819 fetch_fsmap learned FSMap version 24 2024-05-08T22:08:34.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.391+0000 7faa202b4b80 1 -- 172.21.15.131:0/736895646 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c476f37cd0 con 0x55c477168970 2024-05-08T22:08:34.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.391+0000 7fa9eb400700 1 -- 172.21.15.131:0/736895646 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9f801e040 con 0x55c477168970 2024-05-08T22:08:34.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.391+0000 7fa9eb400700 1 -- 172.21.15.131:0/736895646 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faa0001f3c0 con 0x55c477168970 2024-05-08T22:08:34.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.391+0000 7faa202b4b80 10 client.25819 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:08:34.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.391+0000 7faa202b4b80 10 client.25819 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:08:34.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.391+0000 7faa202b4b80 20 client.25819 populate_metadata read hostname 'smithi131' 2024-05-08T22:08:34.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.391+0000 7faa202b4b80 1 --2- 172.21.15.131:0/736895646 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55c4771d2640 0x55c4771d4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.391+0000 7faa202b4b80 4 client.25819 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:08:34.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.391+0000 7faa202b4b80 1 -- 172.21.15.131:0/736895646 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c4770aaec0 con 0x55c4771d2640 2024-05-08T22:08:34.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.392+0000 7faa09e00700 1 --2- 172.21.15.131:0/736895646 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55c4771d2640 0x55c4771d4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.392+0000 7fa9eb400700 10 client.25819 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:08:34.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.392+0000 7faa09e00700 1 --2- 172.21.15.131:0/736895646 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55c4771d2640 0x55c4771d4a60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a7fbb73e5d8d7f4 server_cookie=4460bce6c4a1dadd in_seq=0 out_seq=0 2024-05-08T22:08:34.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.393+0000 7fa9eb400700 1 -- 172.21.15.131:0/736895646 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c4770aaec0 con 0x55c4771d2640 2024-05-08T22:08:34.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.393+0000 7fa9eb400700 10 client.25819 handle_command_reply: tid=0 2024-05-08T22:08:34.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.393+0000 7faa202b4b80 1 client.25819 shutdown 2024-05-08T22:08:34.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.393+0000 7faa202b4b80 1 -- 172.21.15.131:0/736895646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c477168970 msgr2=0x55c47716dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:34.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.393+0000 7faa202b4b80 1 --2- 172.21.15.131:0/736895646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c477168970 0x55c47716dd80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faa00016f00 tx=0x7faa00016f30 comp rx=0 tx=0).stop 2024-05-08T22:08:34.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.393+0000 7faa202b4b80 1 -- 172.21.15.131:0/736895646 shutdown_connections 2024-05-08T22:08:34.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.393+0000 7faa202b4b80 1 --2- 172.21.15.131:0/736895646 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55c4771d2640 0x55c4771d4a60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.393+0000 7faa202b4b80 1 --2- 172.21.15.131:0/736895646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c477168970 0x55c47716dd80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.393+0000 7faa202b4b80 1 --2- 172.21.15.131:0/736895646 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c477162890 0x55c4771732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.393+0000 7faa202b4b80 1 --2- 172.21.15.131:0/736895646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c477168050 0x55c4771737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.393+0000 7faa202b4b80 1 -- 172.21.15.131:0/736895646 >> 172.21.15.131:0/736895646 conn(0x55c4770c0210 msgr2=0x55c477159210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:34.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.393+0000 7fa9eb400700 0 client.25819 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:08:34.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.393+0000 7faa202b4b80 1 -- 172.21.15.131:0/736895646 shutdown_connections 2024-05-08T22:08:34.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.393+0000 7faa202b4b80 1 -- 172.21.15.131:0/736895646 wait complete. 2024-05-08T22:08:34.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.394+0000 7faa202b4b80 20 client.25819 trim_cache size 0 max 16384 2024-05-08T22:08:34.397 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:08:34.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.394+0000 7faa0b200700 1 -- 172.21.15.131:0/2118074600 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa9ec0a14c0 msgr2=0x7fa9ec0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:34.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.394+0000 7faa0b200700 1 --2- 172.21.15.131:0/2118074600 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa9ec0a14c0 0x7fa9ec0a3980 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8006010 tx=0x7fa9f800b540 comp rx=0 tx=0).stop 2024-05-08T22:08:34.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.394+0000 7faa0b200700 1 -- 172.21.15.131:0/2118074600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa04073c60 msgr2=0x7faa0419dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:34.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.394+0000 7faa0b200700 1 --2- 172.21.15.131:0/2118074600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa04073c60 0x7faa0419dc90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa9f000c9b0 tx=0x7fa9f000ccc0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.394+0000 7faa0b200700 1 -- 172.21.15.131:0/2118074600 shutdown_connections 2024-05-08T22:08:34.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.394+0000 7faa0b200700 1 --2- 172.21.15.131:0/2118074600 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa9ec0a14c0 0x7fa9ec0a3980 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.394+0000 7faa0b200700 1 --2- 172.21.15.131:0/2118074600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa04111d30 0x7faa041a1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.394+0000 7faa0b200700 1 --2- 172.21.15.131:0/2118074600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa04073340 0x7faa0419d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.394+0000 7faa0b200700 1 --2- 172.21.15.131:0/2118074600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa04073c60 0x7faa0419dc90 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.394+0000 7faa0b200700 1 -- 172.21.15.131:0/2118074600 >> 172.21.15.131:0/2118074600 conn(0x7faa040fc810 msgr2=0x7faa040fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:34.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.394+0000 7faa0b200700 1 -- 172.21.15.131:0/2118074600 shutdown_connections 2024-05-08T22:08:34.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.395+0000 7faa0b200700 1 -- 172.21.15.131:0/2118074600 wait complete. 2024-05-08T22:08:34.421 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T22:08:34.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:34 smithi132 ceph-mon[78180]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:08:34.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:34 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/4147742059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:08:34.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.665+0000 7f40dca00700 1 -- 172.21.15.131:0/203114800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40d8101940 msgr2=0x7f40d8105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:34.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.665+0000 7f40dca00700 1 --2- 172.21.15.131:0/203114800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40d8101940 0x7f40d8105980 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009990 tx=0x7f40c4009ca0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.665+0000 7f40dca00700 1 -- 172.21.15.131:0/203114800 shutdown_connections 2024-05-08T22:08:34.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.665+0000 7f40dca00700 1 --2- 172.21.15.131:0/203114800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40d81060b0 0x7f40d810dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.665+0000 7f40dca00700 1 --2- 172.21.15.131:0/203114800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40d8100f90 0x7f40d8101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.665+0000 7f40dca00700 1 --2- 172.21.15.131:0/203114800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40d8101940 0x7f40d8105980 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.665+0000 7f40dca00700 1 -- 172.21.15.131:0/203114800 >> 172.21.15.131:0/203114800 conn(0x7f40d80fc810 msgr2=0x7f40d80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:34.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.665+0000 7f40dca00700 1 -- 172.21.15.131:0/203114800 shutdown_connections 2024-05-08T22:08:34.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.665+0000 7f40dca00700 1 -- 172.21.15.131:0/203114800 wait complete. 2024-05-08T22:08:34.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.666+0000 7f40dca00700 1 Processor -- start 2024-05-08T22:08:34.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.667+0000 7f40dca00700 1 -- start start 2024-05-08T22:08:34.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.667+0000 7f40dca00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40d8100f90 0x7f40d81994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.667+0000 7f40dca00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40d81060b0 0x7f40d8199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.667+0000 7f40dca00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40d819dd60 0x7f40d819e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.667+0000 7f40dca00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d819e7e0 con 0x7f40d81060b0 2024-05-08T22:08:34.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.667+0000 7f40dca00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d819e950 con 0x7f40d819dd60 2024-05-08T22:08:34.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.667+0000 7f40dca00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d819eb10 con 0x7f40d8100f90 2024-05-08T22:08:34.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.667+0000 7f40d6800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40d8100f90 0x7f40d81994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.667+0000 7f40d6800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40d8100f90 0x7f40d81994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:47582/0 (socket says 172.21.15.131:47582) 2024-05-08T22:08:34.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.667+0000 7f40d6800700 1 -- 172.21.15.131:0/2305775847 learned_addr learned my addr 172.21.15.131:0/2305775847 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:34.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.667+0000 7f40d7200700 1 --2- 172.21.15.131:0/2305775847 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40d819dd60 0x7f40d819e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.668+0000 7f40d5e00700 1 --2- 172.21.15.131:0/2305775847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40d81060b0 0x7f40d8199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.668+0000 7f40d6800700 1 -- 172.21.15.131:0/2305775847 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40d819dd60 msgr2=0x7f40d819e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:34.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.668+0000 7f40d6800700 1 --2- 172.21.15.131:0/2305775847 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40d819dd60 0x7f40d819e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.668+0000 7f40d6800700 1 -- 172.21.15.131:0/2305775847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40d81060b0 msgr2=0x7f40d8199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:34.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.668+0000 7f40d6800700 1 --2- 172.21.15.131:0/2305775847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40d81060b0 0x7f40d8199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.668+0000 7f40d6800700 1 -- 172.21.15.131:0/2305775847 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c40095f0 con 0x7f40d8100f90 2024-05-08T22:08:34.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.668+0000 7f40d5e00700 1 --2- 172.21.15.131:0/2305775847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40d81060b0 0x7f40d8199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:08:34.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.668+0000 7f40d6800700 1 --2- 172.21.15.131:0/2305775847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40d8100f90 0x7f40d81994d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f40c800e9a0 tx=0x7f40c800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:34.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.668+0000 7f40cf400700 1 -- 172.21.15.131:0/2305775847 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8009790 con 0x7f40d8100f90 2024-05-08T22:08:34.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.668+0000 7f40cf400700 1 -- 172.21.15.131:0/2305775847 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40c800cbd0 con 0x7f40d8100f90 2024-05-08T22:08:34.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.668+0000 7f40dca00700 1 -- 172.21.15.131:0/2305775847 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d81a6070 con 0x7f40d8100f90 2024-05-08T22:08:34.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.668+0000 7f40cf400700 1 -- 172.21.15.131:0/2305775847 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8018680 con 0x7f40d8100f90 2024-05-08T22:08:34.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.669+0000 7f40dca00700 1 -- 172.21.15.131:0/2305775847 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d81a6550 con 0x7f40d8100f90 2024-05-08T22:08:34.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.673+0000 7f40cf400700 1 -- 172.21.15.131:0/2305775847 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 24) v1 ==== 136158+0+0 (secure 0 0 0) 0x7f40c800cd40 con 0x7f40d8100f90 2024-05-08T22:08:34.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.675+0000 7f40f0e8eb80 1 Processor -- start 2024-05-08T22:08:34.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.676+0000 7f40f0e8eb80 1 -- start start 2024-05-08T22:08:34.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.676+0000 7f40cf400700 1 --2- 172.21.15.131:0/2305775847 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f40bc0a14c0 0x7f40bc0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.676+0000 7f40cf400700 1 -- 172.21.15.131:0/2305775847 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f40c8014070 con 0x7f40d8100f90 2024-05-08T22:08:34.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.676+0000 7f40f0e8eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75db1810 0x560f75db1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.676+0000 7f40f0e8eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f75db2130 0x560f75dabb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.677+0000 7f40d7200700 1 --2- 172.21.15.131:0/2305775847 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f40bc0a14c0 0x7f40bc0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.677+0000 7f40f0e8eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f75dac0a0 0x560f75dac500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.677+0000 7f40f0e8eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f75cfa150 con 0x560f75dac0a0 2024-05-08T22:08:34.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.677+0000 7f40f0e8eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f75cf7b10 con 0x560f75db2130 2024-05-08T22:08:34.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.677+0000 7f40d6800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75db1810 0x560f75db1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.677+0000 7f40f0e8eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f75cf3400 con 0x560f75db1810 2024-05-08T22:08:34.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.677+0000 7f40d6800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75db1810 0x560f75db1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:47594/0 (socket says 172.21.15.131:47594) 2024-05-08T22:08:34.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.677+0000 7f40d6800700 1 -- 172.21.15.131:0/3630810034 learned_addr learned my addr 172.21.15.131:0/3630810034 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:34.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.677+0000 7f40d5e00700 1 --2- 172.21.15.131:0/3630810034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f75db2130 0x560f75dabb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.677+0000 7f40d7200700 1 --2- 172.21.15.131:0/3630810034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f75dac0a0 0x560f75dac500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.678+0000 7f40d6800700 1 -- 172.21.15.131:0/3630810034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f75db2130 msgr2=0x560f75dabb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:34.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.678+0000 7f40d6800700 1 --2- 172.21.15.131:0/3630810034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f75db2130 0x560f75dabb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.678+0000 7f40d6800700 1 -- 172.21.15.131:0/3630810034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f75dac0a0 msgr2=0x560f75dac500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:34.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.678+0000 7f40d6800700 1 --2- 172.21.15.131:0/3630810034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f75dac0a0 0x560f75dac500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.678+0000 7f40d6800700 1 -- 172.21.15.131:0/3630810034 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f75b743b0 con 0x560f75db1810 2024-05-08T22:08:34.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.678+0000 7f40d5e00700 1 --2- 172.21.15.131:0/3630810034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f75db2130 0x560f75dabb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:08:34.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.678+0000 7f40d6800700 1 --2- 172.21.15.131:0/3630810034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75db1810 0x560f75db1bf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f40c8096750 tx=0x7f40c8099ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b31a5587bf641638 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:34.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.678+0000 7f40d7200700 1 --2- 172.21.15.131:0/2305775847 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f40bc0a14c0 0x7f40bc0a3980 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f40d000f870 tx=0x7f40d000fc30 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:34.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.678+0000 7f40c3400700 1 -- 172.21.15.131:0/3630810034 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c809e080 con 0x560f75db1810 2024-05-08T22:08:34.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.678+0000 7f40c3400700 1 -- 172.21.15.131:0/3630810034 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40c80a26d0 con 0x560f75db1810 2024-05-08T22:08:34.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.679+0000 7f40c3400700 1 -- 172.21.15.131:0/3630810034 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8099740 con 0x560f75db1810 2024-05-08T22:08:34.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.680+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/3630810034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75db1810 msgr2=0x560f75db1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:34.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.680+0000 7f40f0e8eb80 1 --2- 172.21.15.131:0/3630810034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75db1810 0x560f75db1bf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f40c8096750 tx=0x7f40c8099ea0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.680+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/3630810034 shutdown_connections 2024-05-08T22:08:34.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.680+0000 7f40f0e8eb80 1 --2- 172.21.15.131:0/3630810034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f75dac0a0 0x560f75dac500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.680+0000 7f40f0e8eb80 1 --2- 172.21.15.131:0/3630810034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75db1810 0x560f75db1bf0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.680+0000 7f40f0e8eb80 1 --2- 172.21.15.131:0/3630810034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f75db2130 0x560f75dabb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.680+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/3630810034 >> 172.21.15.131:0/3630810034 conn(0x560f75d01210 msgr2=0x560f75d9e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:34.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.680+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/3630810034 shutdown_connections 2024-05-08T22:08:34.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.680+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/3630810034 wait complete. 2024-05-08T22:08:34.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.681+0000 7f40f0e8eb80 1 -- start start 2024-05-08T22:08:34.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.681+0000 7f40f0e8eb80 1 Processor -- start 2024-05-08T22:08:34.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.682+0000 7f40f0e8eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f75dac0a0 0x560f75db5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.682+0000 7f40f0e8eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75db1810 0x560f75db5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.682+0000 7f40f0e8eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f75db2130 0x560f75db5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.682+0000 7f40f0e8eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f75c8f560 con 0x560f75dac0a0 2024-05-08T22:08:34.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.682+0000 7f40f0e8eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f75c8f200 con 0x560f75db2130 2024-05-08T22:08:34.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.682+0000 7f40f0e8eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f75b766a0 con 0x560f75db1810 2024-05-08T22:08:34.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.682+0000 7f40f0e8eb80 10 client.0 _ll_register_callbacks cb 0x560f75c98b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:34.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.682+0000 7f40d7200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f75db2130 0x560f75db5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.683+0000 7f40d7200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f75db2130 0x560f75db5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:41504/0 (socket says 172.21.15.131:41504) 2024-05-08T22:08:34.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.683+0000 7f40d7200700 1 -- 172.21.15.131:0/2653108058 learned_addr learned my addr 172.21.15.131:0/2653108058 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:34.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.683+0000 7f40d5e00700 1 --2- 172.21.15.131:0/2653108058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75db1810 0x560f75db5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.683+0000 7f40d6800700 1 --2- 172.21.15.131:0/2653108058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f75dac0a0 0x560f75db5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.683+0000 7f40d5e00700 1 -- 172.21.15.131:0/2653108058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f75db2130 msgr2=0x560f75db5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:34.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.683+0000 7f40d5e00700 1 --2- 172.21.15.131:0/2653108058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f75db2130 0x560f75db5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.683+0000 7f40d5e00700 1 -- 172.21.15.131:0/2653108058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f75dac0a0 msgr2=0x560f75db5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:34.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.683+0000 7f40d5e00700 1 --2- 172.21.15.131:0/2653108058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f75dac0a0 0x560f75db5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.683+0000 7f40d5e00700 1 -- 172.21.15.131:0/2653108058 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d81a6550 con 0x560f75db1810 2024-05-08T22:08:34.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.683+0000 7f40d5e00700 1 --2- 172.21.15.131:0/2653108058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75db1810 0x560f75db5a70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f40c4000c00 tx=0x7f40c400b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e4cbdf547655343 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:34.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.684+0000 7f40b7e00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:08:34.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.684+0000 7f40b7e00700 1 -- 172.21.15.131:0/2653108058 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c400f660 con 0x560f75db1810 2024-05-08T22:08:34.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.684+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/2653108058 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f75c4c270 con 0x560f75db1810 2024-05-08T22:08:34.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.684+0000 7f40b7e00700 1 -- 172.21.15.131:0/2653108058 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f40c400f7c0 con 0x560f75db1810 2024-05-08T22:08:34.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.684+0000 7f40b7e00700 1 -- 172.21.15.131:0/2653108058 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40c400f980 con 0x560f75db1810 2024-05-08T22:08:34.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.684+0000 7f40f0e8eb80 10 client.25831 fetch_fsmap learned FSMap version 24 2024-05-08T22:08:34.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.684+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/2653108058 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560f75b20b60 con 0x560f75db1810 2024-05-08T22:08:34.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.685+0000 7f40b7e00700 1 -- 172.21.15.131:0/2653108058 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560f75c4c270 con 0x560f75db1810 2024-05-08T22:08:34.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.685+0000 7f40b7e00700 1 -- 172.21.15.131:0/2653108058 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f40c4023b10 con 0x560f75db1810 2024-05-08T22:08:34.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.685+0000 7f40f0e8eb80 10 client.25831 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:08:34.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.685+0000 7f40f0e8eb80 10 client.25831 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:08:34.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.685+0000 7f40f0e8eb80 20 client.25831 populate_metadata read hostname 'smithi131' 2024-05-08T22:08:34.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.685+0000 7f40f0e8eb80 1 --2- 172.21.15.131:0/2653108058 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x560f75dc5b20 0x560f75dc7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.685+0000 7f40f0e8eb80 4 client.25831 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:08:34.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.685+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/2653108058 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f75d72810 con 0x560f75dc5b20 2024-05-08T22:08:34.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.686+0000 7f40d6800700 1 --2- 172.21.15.131:0/2653108058 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x560f75dc5b20 0x560f75dc7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.686+0000 7f40d6800700 1 --2- 172.21.15.131:0/2653108058 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x560f75dc5b20 0x560f75dc7f40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d159e66104e3a245 server_cookie=47ee71cdcf2a5e48 in_seq=0 out_seq=0 2024-05-08T22:08:34.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.686+0000 7f40b7e00700 10 client.25831 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:08:34.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.687+0000 7f40b7e00700 1 -- 172.21.15.131:0/2653108058 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560f75d72810 con 0x560f75dc5b20 2024-05-08T22:08:34.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.687+0000 7f40b7e00700 10 client.25831 handle_command_reply: tid=0 2024-05-08T22:08:34.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.687+0000 7f40f0e8eb80 1 client.25831 shutdown 2024-05-08T22:08:34.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.687+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/2653108058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75db1810 msgr2=0x560f75db5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:34.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.687+0000 7f40f0e8eb80 1 --2- 172.21.15.131:0/2653108058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75db1810 0x560f75db5a70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f40c4000c00 tx=0x7f40c400b920 comp rx=0 tx=0).stop 2024-05-08T22:08:34.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.687+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/2653108058 shutdown_connections 2024-05-08T22:08:34.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.687+0000 7f40f0e8eb80 1 --2- 172.21.15.131:0/2653108058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f75db2130 0x560f75db5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.687+0000 7f40f0e8eb80 1 --2- 172.21.15.131:0/2653108058 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x560f75dc5b20 0x560f75dc7f40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.687+0000 7f40f0e8eb80 1 --2- 172.21.15.131:0/2653108058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f75dac0a0 0x560f75db5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.687+0000 7f40f0e8eb80 1 --2- 172.21.15.131:0/2653108058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75db1810 0x560f75db5a70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.687+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/2653108058 >> 172.21.15.131:0/2653108058 conn(0x560f75d01210 msgr2=0x560f75d9e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:34.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.687+0000 7f40b7e00700 0 client.25831 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:08:34.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.688+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/2653108058 shutdown_connections 2024-05-08T22:08:34.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.688+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/2653108058 wait complete. 2024-05-08T22:08:34.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.688+0000 7f40f0e8eb80 20 client.25831 trim_cache size 0 max 16384 2024-05-08T22:08:34.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.713+0000 7f40f0e8eb80 1 Processor -- start 2024-05-08T22:08:34.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.713+0000 7f40f0e8eb80 1 -- start start 2024-05-08T22:08:34.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.713+0000 7f40f0e8eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75dac0a0 0x560f75dabb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.713+0000 7f40f0e8eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f75db2130 0x560f75dc1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.713+0000 7f40f0e8eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f75db7d70 0x560f75d9cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.713+0000 7f40f0e8eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f75db6b30 con 0x560f75db7d70 2024-05-08T22:08:34.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.713+0000 7f40f0e8eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f75db1f90 con 0x560f75db2130 2024-05-08T22:08:34.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.713+0000 7f40f0e8eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f75dacc60 con 0x560f75dac0a0 2024-05-08T22:08:34.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.714+0000 7f40d5e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f75db2130 0x560f75dc1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.714+0000 7f40d6800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75dac0a0 0x560f75dabb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.714+0000 7f40d7200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f75db7d70 0x560f75d9cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.714+0000 7f40d6800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75dac0a0 0x560f75dabb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:47612/0 (socket says 172.21.15.131:47612) 2024-05-08T22:08:34.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.714+0000 7f40d6800700 1 -- 172.21.15.131:0/3161349889 learned_addr learned my addr 172.21.15.131:0/3161349889 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:34.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.714+0000 7f40d6800700 1 -- 172.21.15.131:0/3161349889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f75db2130 msgr2=0x560f75dc1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:34.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.714+0000 7f40d6800700 1 --2- 172.21.15.131:0/3161349889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f75db2130 0x560f75dc1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.714+0000 7f40d6800700 1 -- 172.21.15.131:0/3161349889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f75db7d70 msgr2=0x560f75d9cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:34.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.714+0000 7f40d6800700 1 --2- 172.21.15.131:0/3161349889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f75db7d70 0x560f75d9cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.714+0000 7f40d6800700 1 -- 172.21.15.131:0/3161349889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f75cb9560 con 0x560f75dac0a0 2024-05-08T22:08:34.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.714+0000 7f40d6800700 1 --2- 172.21.15.131:0/3161349889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75dac0a0 0x560f75dabb60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f40c80985a0 tx=0x7f40c809a150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cba93128f313923f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:34.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.715+0000 7f40c3e00700 1 -- 172.21.15.131:0/3161349889 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c80b7040 con 0x560f75dac0a0 2024-05-08T22:08:34.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.715+0000 7f40c3e00700 1 -- 172.21.15.131:0/3161349889 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40c80985f0 con 0x560f75dac0a0 2024-05-08T22:08:34.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.715+0000 7f40c3e00700 1 -- 172.21.15.131:0/3161349889 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8099740 con 0x560f75dac0a0 2024-05-08T22:08:34.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.716+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/3161349889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75dac0a0 msgr2=0x560f75dabb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:34.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.716+0000 7f40f0e8eb80 1 --2- 172.21.15.131:0/3161349889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75dac0a0 0x560f75dabb60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f40c80985a0 tx=0x7f40c809a150 comp rx=0 tx=0).stop 2024-05-08T22:08:34.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.716+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/3161349889 shutdown_connections 2024-05-08T22:08:34.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.716+0000 7f40f0e8eb80 1 --2- 172.21.15.131:0/3161349889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f75db7d70 0x560f75d9cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.716+0000 7f40f0e8eb80 1 --2- 172.21.15.131:0/3161349889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75dac0a0 0x560f75dabb60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.716+0000 7f40f0e8eb80 1 --2- 172.21.15.131:0/3161349889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f75db2130 0x560f75dc1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.716+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/3161349889 >> 172.21.15.131:0/3161349889 conn(0x560f75d01210 msgr2=0x560f75d9a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:34.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.716+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/3161349889 shutdown_connections 2024-05-08T22:08:34.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.716+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/3161349889 wait complete. 2024-05-08T22:08:34.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.717+0000 7f40f0e8eb80 1 -- start start 2024-05-08T22:08:34.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.717+0000 7f40f0e8eb80 1 Processor -- start 2024-05-08T22:08:34.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.718+0000 7f40f0e8eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f75dac0a0 0x560f75dbab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.718+0000 7f40f0e8eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75db2130 0x560f75dbb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.718+0000 7f40f0e8eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f75db7d70 0x560f75dbb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.718+0000 7f40f0e8eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f75e1b020 con 0x560f75dac0a0 2024-05-08T22:08:34.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.718+0000 7f40f0e8eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f75e1b160 con 0x560f75db7d70 2024-05-08T22:08:34.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.718+0000 7f40f0e8eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f75e1b2a0 con 0x560f75db2130 2024-05-08T22:08:34.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.718+0000 7f40f0e8eb80 10 client.0 _ll_register_callbacks cb 0x560f75c98b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:34.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.718+0000 7f40d6800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f75dac0a0 0x560f75dbab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.718+0000 7f40d5e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75db2130 0x560f75dbb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.718+0000 7f40d7200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f75db7d70 0x560f75dbb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.718+0000 7f40d6800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f75dac0a0 0x560f75dbab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36358/0 (socket says 172.21.15.131:36358) 2024-05-08T22:08:34.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.718+0000 7f40d6800700 1 -- 172.21.15.131:0/391403735 learned_addr learned my addr 172.21.15.131:0/391403735 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:34.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.719+0000 7f40d5e00700 1 -- 172.21.15.131:0/391403735 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f75db7d70 msgr2=0x560f75dbb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:34.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.719+0000 7f40d5e00700 1 --2- 172.21.15.131:0/391403735 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f75db7d70 0x560f75dbb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.719+0000 7f40d5e00700 1 -- 172.21.15.131:0/391403735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f75dac0a0 msgr2=0x560f75dbab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:34.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.719+0000 7f40d5e00700 1 --2- 172.21.15.131:0/391403735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f75dac0a0 0x560f75dbab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.719+0000 7f40d5e00700 1 -- 172.21.15.131:0/391403735 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f75b743b0 con 0x560f75db2130 2024-05-08T22:08:34.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.719+0000 7f40d5e00700 1 --2- 172.21.15.131:0/391403735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75db2130 0x560f75dbb040 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x560f75db65c0 tx=0x7f40c4017480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a1bb7b69b07d776 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:34.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.719+0000 7f40b7e00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:08:34.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.719+0000 7f40b7e00700 1 -- 172.21.15.131:0/391403735 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c4016c60 con 0x560f75db2130 2024-05-08T22:08:34.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.719+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/391403735 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40c80a26d0 con 0x560f75db2130 2024-05-08T22:08:34.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.719+0000 7f40b7e00700 1 -- 172.21.15.131:0/391403735 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f40c4017540 con 0x560f75db2130 2024-05-08T22:08:34.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.720+0000 7f40b7e00700 1 -- 172.21.15.131:0/391403735 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40c400bc70 con 0x560f75db2130 2024-05-08T22:08:34.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.720+0000 7f40f0e8eb80 10 client.25837 fetch_fsmap learned FSMap version 24 2024-05-08T22:08:34.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.720+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/391403735 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560f75b78cd0 con 0x560f75db2130 2024-05-08T22:08:34.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.720+0000 7f40b7e00700 1 -- 172.21.15.131:0/391403735 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40c80a26d0 con 0x560f75db2130 2024-05-08T22:08:34.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.720+0000 7f40b7e00700 1 -- 172.21.15.131:0/391403735 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f40c40099c0 con 0x560f75db2130 2024-05-08T22:08:34.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.720+0000 7f40f0e8eb80 10 client.25837 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:08:34.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.720+0000 7f40f0e8eb80 10 client.25837 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:08:34.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.720+0000 7f40f0e8eb80 20 client.25837 populate_metadata read hostname 'smithi131' 2024-05-08T22:08:34.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.720+0000 7f40f0e8eb80 1 --2- 172.21.15.131:0/391403735 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x560f75e1c520 0x560f75e1e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:34.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.720+0000 7f40f0e8eb80 4 client.25837 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:08:34.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.720+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/391403735 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560f75cebec0 con 0x560f75e1c520 2024-05-08T22:08:34.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.721+0000 7f40d6800700 1 --2- 172.21.15.131:0/391403735 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x560f75e1c520 0x560f75e1e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:34.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.721+0000 7f40d6800700 1 --2- 172.21.15.131:0/391403735 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x560f75e1c520 0x560f75e1e940 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cbbb82b458198725 server_cookie=8f447043ef3d9e28 in_seq=0 out_seq=0 2024-05-08T22:08:34.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.722+0000 7f40b7e00700 10 client.25837 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:08:34.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.722+0000 7f40b7e00700 1 -- 172.21.15.131:0/391403735 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560f75cebec0 con 0x560f75e1c520 2024-05-08T22:08:34.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.722+0000 7f40b7e00700 10 client.25837 handle_command_reply: tid=0 2024-05-08T22:08:34.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.722+0000 7f40f0e8eb80 1 client.25837 shutdown 2024-05-08T22:08:34.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.722+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/391403735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75db2130 msgr2=0x560f75dbb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:34.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.722+0000 7f40f0e8eb80 1 --2- 172.21.15.131:0/391403735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75db2130 0x560f75dbb040 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x560f75db65c0 tx=0x7f40c4017480 comp rx=0 tx=0).stop 2024-05-08T22:08:34.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.722+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/391403735 shutdown_connections 2024-05-08T22:08:34.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.722+0000 7f40f0e8eb80 1 --2- 172.21.15.131:0/391403735 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f75db7d70 0x560f75dbb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.722+0000 7f40f0e8eb80 1 --2- 172.21.15.131:0/391403735 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x560f75e1c520 0x560f75e1e940 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.722+0000 7f40f0e8eb80 1 --2- 172.21.15.131:0/391403735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f75dac0a0 0x560f75dbab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.722+0000 7f40f0e8eb80 1 --2- 172.21.15.131:0/391403735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f75db2130 0x560f75dbb040 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.722+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/391403735 >> 172.21.15.131:0/391403735 conn(0x560f75d01210 msgr2=0x560f75daf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:34.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.722+0000 7f40b7e00700 0 client.25837 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:08:34.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.722+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/391403735 shutdown_connections 2024-05-08T22:08:34.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.722+0000 7f40f0e8eb80 1 -- 172.21.15.131:0/391403735 wait complete. 2024-05-08T22:08:34.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.723+0000 7f40f0e8eb80 20 client.25837 trim_cache size 0 max 16384 2024-05-08T22:08:34.727 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:08:34.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.723+0000 7f40dca00700 1 -- 172.21.15.131:0/2305775847 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f40bc0a14c0 msgr2=0x7f40bc0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:34.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.723+0000 7f40dca00700 1 --2- 172.21.15.131:0/2305775847 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f40bc0a14c0 0x7f40bc0a3980 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f40d000f870 tx=0x7f40d000fc30 comp rx=0 tx=0).stop 2024-05-08T22:08:34.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.724+0000 7f40dca00700 1 -- 172.21.15.131:0/2305775847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40d8100f90 msgr2=0x7f40d81994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:34.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.724+0000 7f40dca00700 1 --2- 172.21.15.131:0/2305775847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40d8100f90 0x7f40d81994d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f40c800e9a0 tx=0x7f40c800ecb0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.724+0000 7f40dca00700 1 -- 172.21.15.131:0/2305775847 shutdown_connections 2024-05-08T22:08:34.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.724+0000 7f40dca00700 1 --2- 172.21.15.131:0/2305775847 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f40bc0a14c0 0x7f40bc0a3980 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.724+0000 7f40dca00700 1 --2- 172.21.15.131:0/2305775847 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40d819dd60 0x7f40d819e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.724+0000 7f40dca00700 1 --2- 172.21.15.131:0/2305775847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40d8100f90 0x7f40d81994d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.724+0000 7f40dca00700 1 --2- 172.21.15.131:0/2305775847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40d81060b0 0x7f40d8199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:34.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.724+0000 7f40dca00700 1 -- 172.21.15.131:0/2305775847 >> 172.21.15.131:0/2305775847 conn(0x7f40d80fc810 msgr2=0x7f40d80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:34.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.724+0000 7f40dca00700 1 -- 172.21.15.131:0/2305775847 shutdown_connections 2024-05-08T22:08:34.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:34.724+0000 7f40dca00700 1 -- 172.21.15.131:0/2305775847 wait complete. 2024-05-08T22:08:34.747 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T22:08:34.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:34 smithi157 ceph-mon[77746]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:08:34.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:34 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/4147742059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:08:35.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.033+0000 7fb237000700 1 -- 172.21.15.131:0/2376330636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb230111f00 msgr2=0x7fb2301142f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:35.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.033+0000 7fb237000700 1 --2- 172.21.15.131:0/2376330636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb230111f00 0x7fb2301142f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb228009f90 tx=0x7fb22800b330 comp rx=0 tx=0).stop 2024-05-08T22:08:35.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.033+0000 7fb237000700 1 -- 172.21.15.131:0/2376330636 shutdown_connections 2024-05-08T22:08:35.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.033+0000 7fb237000700 1 --2- 172.21.15.131:0/2376330636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb230111f00 0x7fb2301142f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.033+0000 7fb237000700 1 --2- 172.21.15.131:0/2376330636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb230073340 0x7fb230073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.033+0000 7fb237000700 1 --2- 172.21.15.131:0/2376330636 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb230073c60 0x7fb2301119c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.033+0000 7fb237000700 1 -- 172.21.15.131:0/2376330636 >> 172.21.15.131:0/2376330636 conn(0x7fb2300fc9c0 msgr2=0x7fb2300fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:35.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.033+0000 7fb237000700 1 -- 172.21.15.131:0/2376330636 shutdown_connections 2024-05-08T22:08:35.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.033+0000 7fb237000700 1 -- 172.21.15.131:0/2376330636 wait complete. 2024-05-08T22:08:35.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.035+0000 7fb237000700 1 Processor -- start 2024-05-08T22:08:35.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.035+0000 7fb237000700 1 -- start start 2024-05-08T22:08:35.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.036+0000 7fb237000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb230073340 0x7fb2301a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:35.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.036+0000 7fb237000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb230073c60 0x7fb2301a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:35.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.036+0000 7fb237000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb230111f00 0x7fb23019d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:35.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.036+0000 7fb237000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23019e010 con 0x7fb230073c60 2024-05-08T22:08:35.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.036+0000 7fb237000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23019e180 con 0x7fb230111f00 2024-05-08T22:08:35.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.036+0000 7fb237000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23019e340 con 0x7fb230073340 2024-05-08T22:08:35.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.036+0000 7fb22fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb230073340 0x7fb2301a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:35.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.036+0000 7fb22fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb230073340 0x7fb2301a3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:47632/0 (socket says 172.21.15.131:47632) 2024-05-08T22:08:35.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.036+0000 7fb22fe00700 1 -- 172.21.15.131:0/2744986000 learned_addr learned my addr 172.21.15.131:0/2744986000 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:35.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.036+0000 7fb235000700 1 --2- 172.21.15.131:0/2744986000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb230111f00 0x7fb23019d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:35.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.036+0000 7fb22fe00700 1 -- 172.21.15.131:0/2744986000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb230111f00 msgr2=0x7fb23019d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:35.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.036+0000 7fb22fe00700 1 --2- 172.21.15.131:0/2744986000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb230111f00 0x7fb23019d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.036+0000 7fb22fe00700 1 -- 172.21.15.131:0/2744986000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb230073c60 msgr2=0x7fb2301a3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:35.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.037+0000 7fb22f400700 1 --2- 172.21.15.131:0/2744986000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb230073c60 0x7fb2301a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:35.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.037+0000 7fb22fe00700 1 --2- 172.21.15.131:0/2744986000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb230073c60 0x7fb2301a3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.037+0000 7fb22fe00700 1 -- 172.21.15.131:0/2744986000 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21c009520 con 0x7fb230073340 2024-05-08T22:08:35.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.037+0000 7fb235000700 1 --2- 172.21.15.131:0/2744986000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb230111f00 0x7fb23019d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:35.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.037+0000 7fb22f400700 1 --2- 172.21.15.131:0/2744986000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb230073c60 0x7fb2301a3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:08:35.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.037+0000 7fb22fe00700 1 --2- 172.21.15.131:0/2744986000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb230073340 0x7fb2301a3880 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb22000e9c0 tx=0x7fb22000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:35.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.037+0000 7fb22cc00700 1 -- 172.21.15.131:0/2744986000 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22000cbd0 con 0x7fb230073340 2024-05-08T22:08:35.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.037+0000 7fb22cc00700 1 -- 172.21.15.131:0/2744986000 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb22000eeb0 con 0x7fb230073340 2024-05-08T22:08:35.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.037+0000 7fb237000700 1 -- 172.21.15.131:0/2744986000 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb228009c40 con 0x7fb230073340 2024-05-08T22:08:35.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.037+0000 7fb22cc00700 1 -- 172.21.15.131:0/2744986000 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb220005580 con 0x7fb230073340 2024-05-08T22:08:35.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.038+0000 7fb237000700 1 -- 172.21.15.131:0/2744986000 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb23019e810 con 0x7fb230073340 2024-05-08T22:08:35.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.042+0000 7fb22cc00700 1 -- 172.21.15.131:0/2744986000 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 24) v1 ==== 136158+0+0 (secure 0 0 0) 0x7fb220004750 con 0x7fb230073340 2024-05-08T22:08:35.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.045+0000 7fb22cc00700 1 --2- 172.21.15.131:0/2744986000 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb2140a14c0 0x7fb2140a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:35.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.045+0000 7fb22cc00700 1 -- 172.21.15.131:0/2744986000 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fb220014070 con 0x7fb230073340 2024-05-08T22:08:35.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.045+0000 7fb24aa89b80 1 Processor -- start 2024-05-08T22:08:35.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.045+0000 7fb22f400700 1 --2- 172.21.15.131:0/2744986000 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb2140a14c0 0x7fb2140a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:35.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.046+0000 7fb24aa89b80 1 -- start start 2024-05-08T22:08:35.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.047+0000 7fb24aa89b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317dee00 0x562d317df1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:35.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.047+0000 7fb22f400700 1 --2- 172.21.15.131:0/2744986000 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb2140a14c0 0x7fb2140a3980 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb23019eb70 tx=0x7fb21c009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:35.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.047+0000 7fb24aa89b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d317df720 0x562d317d9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:35.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.047+0000 7fb24aa89b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d317d9890 0x562d317d9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:35.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.047+0000 7fb24aa89b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d31730150 con 0x562d317d9890 2024-05-08T22:08:35.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.047+0000 7fb24aa89b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3172db10 con 0x562d317df720 2024-05-08T22:08:35.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.047+0000 7fb24aa89b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d31729400 con 0x562d317dee00 2024-05-08T22:08:35.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.047+0000 7fb22fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317dee00 0x562d317df1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:35.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.047+0000 7fb22f400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d317df720 0x562d317d9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.047+0000 7fb22fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317dee00 0x562d317df1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:47648/0 (socket says 172.21.15.131:47648) 2024-05-08T22:08:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.047+0000 7fb22fe00700 1 -- 172.21.15.131:0/2158871818 learned_addr learned my addr 172.21.15.131:0/2158871818 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.048+0000 7fb235000700 1 --2- 172.21.15.131:0/2158871818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d317d9890 0x562d317d9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.048+0000 7fb22f400700 1 -- 172.21.15.131:0/2158871818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317dee00 msgr2=0x562d317df1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.048+0000 7fb22f400700 1 --2- 172.21.15.131:0/2158871818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317dee00 0x562d317df1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.048+0000 7fb22f400700 1 -- 172.21.15.131:0/2158871818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d317d9890 msgr2=0x562d317d9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.048+0000 7fb22f400700 1 --2- 172.21.15.131:0/2158871818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d317d9890 0x562d317d9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.048+0000 7fb22f400700 1 -- 172.21.15.131:0/2158871818 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d315aa3b0 con 0x562d317df720 2024-05-08T22:08:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.048+0000 7fb22f400700 1 --2- 172.21.15.131:0/2158871818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d317df720 0x562d317d9350 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb21c00fd70 tx=0x7fb21c012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6951355effcabebc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.048+0000 7fb235000700 1 --2- 172.21.15.131:0/2158871818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d317d9890 0x562d317d9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:08:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.049+0000 7fb225600700 1 -- 172.21.15.131:0/2158871818 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c011bb0 con 0x562d317df720 2024-05-08T22:08:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.049+0000 7fb225600700 1 -- 172.21.15.131:0/2158871818 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb21c011d10 con 0x562d317df720 2024-05-08T22:08:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.049+0000 7fb225600700 1 -- 172.21.15.131:0/2158871818 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c01d5d0 con 0x562d317df720 2024-05-08T22:08:35.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.050+0000 7fb24aa89b80 1 -- 172.21.15.131:0/2158871818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d317df720 msgr2=0x562d317d9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:35.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.050+0000 7fb24aa89b80 1 --2- 172.21.15.131:0/2158871818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d317df720 0x562d317d9350 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb21c00fd70 tx=0x7fb21c012000 comp rx=0 tx=0).stop 2024-05-08T22:08:35.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.050+0000 7fb24aa89b80 1 -- 172.21.15.131:0/2158871818 shutdown_connections 2024-05-08T22:08:35.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.050+0000 7fb24aa89b80 1 --2- 172.21.15.131:0/2158871818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d317d9890 0x562d317d9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.050+0000 7fb24aa89b80 1 --2- 172.21.15.131:0/2158871818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317dee00 0x562d317df1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.050+0000 7fb24aa89b80 1 --2- 172.21.15.131:0/2158871818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d317df720 0x562d317d9350 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.050+0000 7fb24aa89b80 1 -- 172.21.15.131:0/2158871818 >> 172.21.15.131:0/2158871818 conn(0x562d31737210 msgr2=0x562d317d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:35.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.050+0000 7fb24aa89b80 1 -- 172.21.15.131:0/2158871818 shutdown_connections 2024-05-08T22:08:35.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.050+0000 7fb24aa89b80 1 -- 172.21.15.131:0/2158871818 wait complete. 2024-05-08T22:08:35.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.051+0000 7fb24aa89b80 1 -- start start 2024-05-08T22:08:35.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.052+0000 7fb24aa89b80 1 Processor -- start 2024-05-08T22:08:35.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.052+0000 7fb24aa89b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317d9890 0x562d317e2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:35.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.052+0000 7fb24aa89b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d317dee00 0x562d317e30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:35.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.052+0000 7fb24aa89b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d317df720 0x562d317e3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:35.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.052+0000 7fb24aa89b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d316c5560 con 0x562d317dee00 2024-05-08T22:08:35.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.052+0000 7fb24aa89b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d316c5200 con 0x562d317df720 2024-05-08T22:08:35.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.052+0000 7fb24aa89b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d315ac6a0 con 0x562d317d9890 2024-05-08T22:08:35.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.052+0000 7fb24aa89b80 10 client.0 _ll_register_callbacks cb 0x562d316ceb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:35.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.053+0000 7fb22f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d317dee00 0x562d317e30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:35.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.053+0000 7fb22fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317d9890 0x562d317e2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:35.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.053+0000 7fb22fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317d9890 0x562d317e2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:47664/0 (socket says 172.21.15.131:47664) 2024-05-08T22:08:35.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.053+0000 7fb22fe00700 1 -- 172.21.15.131:0/84165527 learned_addr learned my addr 172.21.15.131:0/84165527 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:35.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.053+0000 7fb235000700 1 --2- 172.21.15.131:0/84165527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d317df720 0x562d317e3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:35.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.053+0000 7fb22fe00700 1 -- 172.21.15.131:0/84165527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d317df720 msgr2=0x562d317e3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:35.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.053+0000 7fb22fe00700 1 --2- 172.21.15.131:0/84165527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d317df720 0x562d317e3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.053+0000 7fb22fe00700 1 -- 172.21.15.131:0/84165527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d317dee00 msgr2=0x562d317e30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:35.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.054+0000 7fb22fe00700 1 --2- 172.21.15.131:0/84165527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d317dee00 0x562d317e30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.054+0000 7fb22fe00700 1 -- 172.21.15.131:0/84165527 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d31556b60 con 0x562d317d9890 2024-05-08T22:08:35.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.054+0000 7fb22f400700 1 --2- 172.21.15.131:0/84165527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d317dee00 0x562d317e30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:35.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.054+0000 7fb22fe00700 1 --2- 172.21.15.131:0/84165527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317d9890 0x562d317e2b90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb2200ca900 tx=0x7fb22009a1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fec1fd42e8b63bc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:35.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.054+0000 7fb21a000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:08:35.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.054+0000 7fb21a000700 1 -- 172.21.15.131:0/84165527 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2200a2650 con 0x562d317d9890 2024-05-08T22:08:35.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.054+0000 7fb21a000700 1 -- 172.21.15.131:0/84165527 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fb2200996c0 con 0x562d317d9890 2024-05-08T22:08:35.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.055+0000 7fb24aa89b80 1 -- 172.21.15.131:0/84165527 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d31682270 con 0x562d317d9890 2024-05-08T22:08:35.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.055+0000 7fb21a000700 1 -- 172.21.15.131:0/84165527 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb220099880 con 0x562d317d9890 2024-05-08T22:08:35.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.055+0000 7fb24aa89b80 10 client.25849 fetch_fsmap learned FSMap version 24 2024-05-08T22:08:35.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.055+0000 7fb24aa89b80 1 -- 172.21.15.131:0/84165527 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562d316ef560 con 0x562d317d9890 2024-05-08T22:08:35.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.056+0000 7fb21a000700 1 -- 172.21.15.131:0/84165527 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562d31682270 con 0x562d317d9890 2024-05-08T22:08:35.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.056+0000 7fb21a000700 1 -- 172.21.15.131:0/84165527 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb22007a020 con 0x562d317d9890 2024-05-08T22:08:35.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.056+0000 7fb24aa89b80 10 client.25849 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:08:35.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.056+0000 7fb24aa89b80 10 client.25849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:08:35.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.056+0000 7fb24aa89b80 20 client.25849 populate_metadata read hostname 'smithi131' 2024-05-08T22:08:35.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.056+0000 7fb24aa89b80 1 --2- 172.21.15.131:0/84165527 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562d317f32c0 0x562d317f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:35.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.056+0000 7fb24aa89b80 4 client.25849 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:08:35.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.056+0000 7fb24aa89b80 1 -- 172.21.15.131:0/84165527 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d317a8810 con 0x562d317f32c0 2024-05-08T22:08:35.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.057+0000 7fb22f400700 1 --2- 172.21.15.131:0/84165527 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562d317f32c0 0x562d317f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:35.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.057+0000 7fb21a000700 10 client.25849 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:35.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.057+0000 7fb22f400700 1 --2- 172.21.15.131:0/84165527 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562d317f32c0 0x562d317f56b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b03f22a540fd7651 server_cookie=7a896b113d84ad7f in_seq=0 out_seq=0 2024-05-08T22:08:35.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.058+0000 7fb21a000700 1 -- 172.21.15.131:0/84165527 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562d317a8810 con 0x562d317f32c0 2024-05-08T22:08:35.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.058+0000 7fb21a000700 10 client.25849 handle_command_reply: tid=0 2024-05-08T22:08:35.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.058+0000 7fb24aa89b80 1 client.25849 shutdown 2024-05-08T22:08:35.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.058+0000 7fb24aa89b80 1 -- 172.21.15.131:0/84165527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317d9890 msgr2=0x562d317e2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:35.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.058+0000 7fb24aa89b80 1 --2- 172.21.15.131:0/84165527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317d9890 0x562d317e2b90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb2200ca900 tx=0x7fb22009a1d0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.059+0000 7fb24aa89b80 1 -- 172.21.15.131:0/84165527 shutdown_connections 2024-05-08T22:08:35.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.059+0000 7fb24aa89b80 1 --2- 172.21.15.131:0/84165527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d317df720 0x562d317e3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.059+0000 7fb24aa89b80 1 --2- 172.21.15.131:0/84165527 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562d317f32c0 0x562d317f56b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.059+0000 7fb24aa89b80 1 --2- 172.21.15.131:0/84165527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317d9890 0x562d317e2b90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.059+0000 7fb24aa89b80 1 --2- 172.21.15.131:0/84165527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d317dee00 0x562d317e30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.059+0000 7fb21a000700 0 client.25849 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:35.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.059+0000 7fb24aa89b80 1 -- 172.21.15.131:0/84165527 >> 172.21.15.131:0/84165527 conn(0x562d31737210 msgr2=0x562d317d35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:35.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.059+0000 7fb24aa89b80 1 -- 172.21.15.131:0/84165527 shutdown_connections 2024-05-08T22:08:35.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.059+0000 7fb24aa89b80 1 -- 172.21.15.131:0/84165527 wait complete. 2024-05-08T22:08:35.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.060+0000 7fb24aa89b80 20 client.25849 trim_cache size 0 max 16384 2024-05-08T22:08:35.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.086+0000 7fb24aa89b80 1 Processor -- start 2024-05-08T22:08:35.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.086+0000 7fb24aa89b80 1 -- start start 2024-05-08T22:08:35.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.086+0000 7fb24aa89b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317d9890 0x562d317d9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:35.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.086+0000 7fb24aa89b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d317dee00 0x562d317eec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:35.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.086+0000 7fb24aa89b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d317df720 0x562d317d3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:35.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.086+0000 7fb24aa89b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d317e4190 con 0x562d317dee00 2024-05-08T22:08:35.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.086+0000 7fb24aa89b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d317d0dc0 con 0x562d317df720 2024-05-08T22:08:35.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.086+0000 7fb24aa89b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d317d5e50 con 0x562d317d9890 2024-05-08T22:08:35.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.087+0000 7fb235000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d317df720 0x562d317d3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:35.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.087+0000 7fb235000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d317df720 0x562d317d3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:41564/0 (socket says 172.21.15.131:41564) 2024-05-08T22:08:35.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.087+0000 7fb235000700 1 -- 172.21.15.131:0/1268081691 learned_addr learned my addr 172.21.15.131:0/1268081691 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:35.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.087+0000 7fb22fe00700 1 --2- 172.21.15.131:0/1268081691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317d9890 0x562d317d9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:35.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.087+0000 7fb22f400700 1 --2- 172.21.15.131:0/1268081691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d317dee00 0x562d317eec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:35.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.087+0000 7fb22fe00700 1 -- 172.21.15.131:0/1268081691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d317df720 msgr2=0x562d317d3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:35.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.087+0000 7fb22fe00700 1 --2- 172.21.15.131:0/1268081691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d317df720 0x562d317d3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.087+0000 7fb22fe00700 1 -- 172.21.15.131:0/1268081691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d317dee00 msgr2=0x562d317eec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:35.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.087+0000 7fb22fe00700 1 --2- 172.21.15.131:0/1268081691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d317dee00 0x562d317eec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.087+0000 7fb22fe00700 1 -- 172.21.15.131:0/1268081691 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d315aecd0 con 0x562d317d9890 2024-05-08T22:08:35.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.087+0000 7fb22f400700 1 --2- 172.21.15.131:0/1268081691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d317dee00 0x562d317eec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:08:35.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.087+0000 7fb22fe00700 1 --2- 172.21.15.131:0/1268081691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317d9890 0x562d317d9350 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb220018000 tx=0x7fb220009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=991078542c586e7a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:35.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.088+0000 7fb226000700 1 -- 172.21.15.131:0/1268081691 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb220099d10 con 0x562d317d9890 2024-05-08T22:08:35.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.088+0000 7fb226000700 1 -- 172.21.15.131:0/1268081691 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2200b82a0 con 0x562d317d9890 2024-05-08T22:08:35.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.088+0000 7fb226000700 1 -- 172.21.15.131:0/1268081691 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2200a2420 con 0x562d317d9890 2024-05-08T22:08:35.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.089+0000 7fb24aa89b80 1 -- 172.21.15.131:0/1268081691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317d9890 msgr2=0x562d317d9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:35.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.089+0000 7fb24aa89b80 1 --2- 172.21.15.131:0/1268081691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317d9890 0x562d317d9350 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb220018000 tx=0x7fb220009550 comp rx=0 tx=0).stop 2024-05-08T22:08:35.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.089+0000 7fb24aa89b80 1 -- 172.21.15.131:0/1268081691 shutdown_connections 2024-05-08T22:08:35.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.089+0000 7fb24aa89b80 1 --2- 172.21.15.131:0/1268081691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d317df720 0x562d317d3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.089+0000 7fb24aa89b80 1 --2- 172.21.15.131:0/1268081691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317d9890 0x562d317d9350 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.089+0000 7fb24aa89b80 1 --2- 172.21.15.131:0/1268081691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d317dee00 0x562d317eec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.089+0000 7fb24aa89b80 1 -- 172.21.15.131:0/1268081691 >> 172.21.15.131:0/1268081691 conn(0x562d31737210 msgr2=0x562d317d0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:35.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.089+0000 7fb24aa89b80 1 -- 172.21.15.131:0/1268081691 shutdown_connections 2024-05-08T22:08:35.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.089+0000 7fb24aa89b80 1 -- 172.21.15.131:0/1268081691 wait complete. 2024-05-08T22:08:35.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.090+0000 7fb24aa89b80 1 -- start start 2024-05-08T22:08:35.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.090+0000 7fb24aa89b80 1 Processor -- start 2024-05-08T22:08:35.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.091+0000 7fb24aa89b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317d9890 0x562d317ea1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:35.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.091+0000 7fb24aa89b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d317dee00 0x562d317ea6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:35.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.091+0000 7fb24aa89b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d317df720 0x562d317e81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:35.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.091+0000 7fb24aa89b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d317e88b0 con 0x562d317dee00 2024-05-08T22:08:35.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.091+0000 7fb24aa89b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d317e8a20 con 0x562d317df720 2024-05-08T22:08:35.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.091+0000 7fb24aa89b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d317e8be0 con 0x562d317d9890 2024-05-08T22:08:35.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.091+0000 7fb24aa89b80 10 client.0 _ll_register_callbacks cb 0x562d316ceb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:35.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.091+0000 7fb22fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317d9890 0x562d317ea1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:35.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.091+0000 7fb22fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317d9890 0x562d317ea1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:47690/0 (socket says 172.21.15.131:47690) 2024-05-08T22:08:35.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.091+0000 7fb22fe00700 1 -- 172.21.15.131:0/2403470262 learned_addr learned my addr 172.21.15.131:0/2403470262 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:35.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.091+0000 7fb235000700 1 --2- 172.21.15.131:0/2403470262 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d317df720 0x562d317e81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:35.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.092+0000 7fb22f400700 1 --2- 172.21.15.131:0/2403470262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d317dee00 0x562d317ea6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:35.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.092+0000 7fb235000700 1 -- 172.21.15.131:0/2403470262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317d9890 msgr2=0x562d317ea1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:35.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.092+0000 7fb235000700 1 --2- 172.21.15.131:0/2403470262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317d9890 0x562d317ea1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.092+0000 7fb235000700 1 -- 172.21.15.131:0/2403470262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d317dee00 msgr2=0x562d317ea6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:35.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.092+0000 7fb235000700 1 --2- 172.21.15.131:0/2403470262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d317dee00 0x562d317ea6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.092+0000 7fb235000700 1 -- 172.21.15.131:0/2403470262 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d316ef560 con 0x562d317df720 2024-05-08T22:08:35.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.092+0000 7fb22fe00700 1 --2- 172.21.15.131:0/2403470262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317d9890 0x562d317ea1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:08:35.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.092+0000 7fb22f400700 1 --2- 172.21.15.131:0/2403470262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d317dee00 0x562d317ea6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:08:35.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.092+0000 7fb235000700 1 --2- 172.21.15.131:0/2403470262 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d317df720 0x562d317e81a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb22800b6c0 tx=0x7fb228003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=add813a052838d68 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:35.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.092+0000 7fb21a000700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:08:35.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.093+0000 7fb21a000700 1 -- 172.21.15.131:0/2403470262 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22800e040 con 0x562d317df720 2024-05-08T22:08:35.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.093+0000 7fb21a000700 1 -- 172.21.15.131:0/2403470262 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fb228007bd0 con 0x562d317df720 2024-05-08T22:08:35.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.093+0000 7fb24aa89b80 1 -- 172.21.15.131:0/2403470262 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb21c011d10 con 0x562d317df720 2024-05-08T22:08:35.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.093+0000 7fb21a000700 1 -- 172.21.15.131:0/2403470262 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb228007dd0 con 0x562d317df720 2024-05-08T22:08:35.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.094+0000 7fb24aa89b80 10 client.25844 fetch_fsmap learned FSMap version 24 2024-05-08T22:08:35.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.094+0000 7fb24aa89b80 1 -- 172.21.15.131:0/2403470262 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562d31555030 con 0x562d317df720 2024-05-08T22:08:35.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.094+0000 7fb21a000700 1 -- 172.21.15.131:0/2403470262 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb22802f020 con 0x562d317df720 2024-05-08T22:08:35.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.094+0000 7fb21a000700 1 -- 172.21.15.131:0/2403470262 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb22800b050 con 0x562d317df720 2024-05-08T22:08:35.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.094+0000 7fb24aa89b80 10 client.25844 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:08:35.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.094+0000 7fb24aa89b80 10 client.25844 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:08:35.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.094+0000 7fb24aa89b80 20 client.25844 populate_metadata read hostname 'smithi131' 2024-05-08T22:08:35.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.094+0000 7fb24aa89b80 1 --2- 172.21.15.131:0/2403470262 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562d31849a50 0x562d3184be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:35.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.095+0000 7fb24aa89b80 4 client.25844 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:08:35.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.095+0000 7fb24aa89b80 1 -- 172.21.15.131:0/2403470262 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562d31721ec0 con 0x562d31849a50 2024-05-08T22:08:35.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.095+0000 7fb22fe00700 1 --2- 172.21.15.131:0/2403470262 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562d31849a50 0x562d3184be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:35.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.095+0000 7fb22fe00700 1 --2- 172.21.15.131:0/2403470262 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562d31849a50 0x562d3184be70 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b470498b9e1fcdc server_cookie=5a501a30188fc6c2 in_seq=0 out_seq=0 2024-05-08T22:08:35.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.095+0000 7fb21a000700 10 client.25844 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:35.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.096+0000 7fb21a000700 1 -- 172.21.15.131:0/2403470262 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562d31721ec0 con 0x562d31849a50 2024-05-08T22:08:35.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.096+0000 7fb21a000700 10 client.25844 handle_command_reply: tid=0 2024-05-08T22:08:35.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.096+0000 7fb24aa89b80 1 client.25844 shutdown 2024-05-08T22:08:35.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.096+0000 7fb24aa89b80 1 -- 172.21.15.131:0/2403470262 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d317df720 msgr2=0x562d317e81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:35.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.096+0000 7fb24aa89b80 1 --2- 172.21.15.131:0/2403470262 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d317df720 0x562d317e81a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb22800b6c0 tx=0x7fb228003f90 comp rx=0 tx=0).stop 2024-05-08T22:08:35.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.096+0000 7fb24aa89b80 1 -- 172.21.15.131:0/2403470262 shutdown_connections 2024-05-08T22:08:35.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.096+0000 7fb24aa89b80 1 --2- 172.21.15.131:0/2403470262 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d317df720 0x562d317e81a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.096+0000 7fb24aa89b80 1 --2- 172.21.15.131:0/2403470262 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562d31849a50 0x562d3184be70 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.096+0000 7fb24aa89b80 1 --2- 172.21.15.131:0/2403470262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d317d9890 0x562d317ea1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.096+0000 7fb24aa89b80 1 --2- 172.21.15.131:0/2403470262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d317dee00 0x562d317ea6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.096+0000 7fb21a000700 0 client.25844 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:35.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.096+0000 7fb24aa89b80 1 -- 172.21.15.131:0/2403470262 >> 172.21.15.131:0/2403470262 conn(0x562d31737210 msgr2=0x562d317e12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:35.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.097+0000 7fb24aa89b80 1 -- 172.21.15.131:0/2403470262 shutdown_connections 2024-05-08T22:08:35.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.097+0000 7fb24aa89b80 1 -- 172.21.15.131:0/2403470262 wait complete. 2024-05-08T22:08:35.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.097+0000 7fb24aa89b80 20 client.25844 trim_cache size 0 max 16384 2024-05-08T22:08:35.100 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:08:35.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.098+0000 7fb237000700 1 -- 172.21.15.131:0/2744986000 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb2140a14c0 msgr2=0x7fb2140a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:35.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.098+0000 7fb237000700 1 --2- 172.21.15.131:0/2744986000 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb2140a14c0 0x7fb2140a3980 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb23019eb70 tx=0x7fb21c009260 comp rx=0 tx=0).stop 2024-05-08T22:08:35.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.098+0000 7fb237000700 1 -- 172.21.15.131:0/2744986000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb230073340 msgr2=0x7fb2301a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:35.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.098+0000 7fb237000700 1 --2- 172.21.15.131:0/2744986000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb230073340 0x7fb2301a3880 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb22000e9c0 tx=0x7fb22000ecd0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.098+0000 7fb237000700 1 -- 172.21.15.131:0/2744986000 shutdown_connections 2024-05-08T22:08:35.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.098+0000 7fb237000700 1 --2- 172.21.15.131:0/2744986000 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb2140a14c0 0x7fb2140a3980 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.098+0000 7fb237000700 1 --2- 172.21.15.131:0/2744986000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb230111f00 0x7fb23019d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.098+0000 7fb237000700 1 --2- 172.21.15.131:0/2744986000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb230073340 0x7fb2301a3880 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.098+0000 7fb237000700 1 --2- 172.21.15.131:0/2744986000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb230073c60 0x7fb2301a3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:35.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.098+0000 7fb237000700 1 -- 172.21.15.131:0/2744986000 >> 172.21.15.131:0/2744986000 conn(0x7fb2300fc9c0 msgr2=0x7fb2300fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:35.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.098+0000 7fb237000700 1 -- 172.21.15.131:0/2744986000 shutdown_connections 2024-05-08T22:08:35.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:35.098+0000 7fb237000700 1 -- 172.21.15.131:0/2744986000 wait complete. 2024-05-08T22:08:36.130 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T22:08:36.131 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T22:08:36.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.341+0000 7fc2ca200700 1 -- 172.21.15.131:0/3204103852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2c410c720 msgr2=0x7fc2c41142a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:36.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.341+0000 7fc2ca200700 1 --2- 172.21.15.131:0/3204103852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2c410c720 0x7fc2c41142a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00b0a0 tx=0x7fc2bc00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.341+0000 7fc2ca200700 1 -- 172.21.15.131:0/3204103852 shutdown_connections 2024-05-08T22:08:36.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.341+0000 7fc2ca200700 1 --2- 172.21.15.131:0/3204103852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2c410c720 0x7fc2c41142a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.341+0000 7fc2ca200700 1 --2- 172.21.15.131:0/3204103852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2c4107750 0x7fc2c4107b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.341+0000 7fc2ca200700 1 --2- 172.21.15.131:0/3204103852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2c4108100 0x7fc2c410bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.341+0000 7fc2ca200700 1 -- 172.21.15.131:0/3204103852 >> 172.21.15.131:0/3204103852 conn(0x7fc2c4076dc0 msgr2=0x7fc2c4105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:36.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.341+0000 7fc2ca200700 1 -- 172.21.15.131:0/3204103852 shutdown_connections 2024-05-08T22:08:36.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.341+0000 7fc2ca200700 1 -- 172.21.15.131:0/3204103852 wait complete. 2024-05-08T22:08:36.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.342+0000 7fc2ca200700 1 Processor -- start 2024-05-08T22:08:36.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.343+0000 7fc2ca200700 1 -- start start 2024-05-08T22:08:36.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.343+0000 7fc2ca200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2c4107750 0x7fc2c41a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.343+0000 7fc2ca200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2c410c720 0x7fc2c41a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.343+0000 7fc2ca200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2c419d800 0x7fc2c419dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.343+0000 7fc2ca200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c419e280 con 0x7fc2c410c720 2024-05-08T22:08:36.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.343+0000 7fc2ca200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c419e3c0 con 0x7fc2c419d800 2024-05-08T22:08:36.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.343+0000 7fc2ca200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c419e580 con 0x7fc2c4107750 2024-05-08T22:08:36.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.344+0000 7fc2c9800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2c419d800 0x7fc2c419dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.344+0000 7fc2c9800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2c419d800 0x7fc2c419dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:41578/0 (socket says 172.21.15.131:41578) 2024-05-08T22:08:36.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.344+0000 7fc2c9800700 1 -- 172.21.15.131:0/1939553826 learned_addr learned my addr 172.21.15.131:0/1939553826 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:36.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.344+0000 7fc2c3e00700 1 --2- 172.21.15.131:0/1939553826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2c410c720 0x7fc2c41a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.344+0000 7fc2c9800700 1 -- 172.21.15.131:0/1939553826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2c4107750 msgr2=0x7fc2c41a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:36.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.344+0000 7fc2c9800700 1 --2- 172.21.15.131:0/1939553826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2c4107750 0x7fc2c41a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.344+0000 7fc2c9800700 1 -- 172.21.15.131:0/1939553826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2c410c720 msgr2=0x7fc2c41a3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:36.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.344+0000 7fc2c9800700 1 --2- 172.21.15.131:0/1939553826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2c410c720 0x7fc2c41a3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.344+0000 7fc2c9800700 1 -- 172.21.15.131:0/1939553826 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2bc009d00 con 0x7fc2c419d800 2024-05-08T22:08:36.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.344+0000 7fc2c3e00700 1 --2- 172.21.15.131:0/1939553826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2c410c720 0x7fc2c41a3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:08:36.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.345+0000 7fc2c9800700 1 --2- 172.21.15.131:0/1939553826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2c419d800 0x7fc2c419dcb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00b070 tx=0x7fc2bc00bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:36.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.345+0000 7fc2c1600700 1 -- 172.21.15.131:0/1939553826 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc00e040 con 0x7fc2c419d800 2024-05-08T22:08:36.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.345+0000 7fc2c1600700 1 -- 172.21.15.131:0/1939553826 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2bc01d480 con 0x7fc2c419d800 2024-05-08T22:08:36.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.345+0000 7fc2c1600700 1 -- 172.21.15.131:0/1939553826 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0052a0 con 0x7fc2c419d800 2024-05-08T22:08:36.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.345+0000 7fc2ca200700 1 -- 172.21.15.131:0/1939553826 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c41aa4e0 con 0x7fc2c419d800 2024-05-08T22:08:36.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.345+0000 7fc2ca200700 1 -- 172.21.15.131:0/1939553826 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c41aa850 con 0x7fc2c419d800 2024-05-08T22:08:36.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.349+0000 7fc2c1600700 1 -- 172.21.15.131:0/1939553826 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) v1 ==== 136158+0+0 (secure 0 0 0) 0x7fc2bc01d5f0 con 0x7fc2c419d800 2024-05-08T22:08:36.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.351+0000 7fc2df2bdb80 1 Processor -- start 2024-05-08T22:08:36.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.351+0000 7fc2df2bdb80 1 -- start start 2024-05-08T22:08:36.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.352+0000 7fc2c1600700 1 --2- 172.21.15.131:0/1939553826 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc2ac0a14c0 0x7fc2ac0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.352+0000 7fc2df2bdb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe200d0 0x55dabbe204b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.352+0000 7fc2df2bdb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabbe209f0 0x55dabbe1a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.352+0000 7fc2df2bdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dabbe1a910 0x55dabbe1ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.352+0000 7fc2c1600700 1 -- 172.21.15.131:0/1939553826 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fc2bc0ce170 con 0x7fc2c419d800 2024-05-08T22:08:36.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.353+0000 7fc2df2bdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dabbd711d0 con 0x55dabbe1a910 2024-05-08T22:08:36.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.353+0000 7fc2df2bdb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dabbd6eb90 con 0x55dabbe209f0 2024-05-08T22:08:36.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.353+0000 7fc2df2bdb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dabbd6a480 con 0x55dabbe200d0 2024-05-08T22:08:36.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.353+0000 7fc2c8e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe200d0 0x55dabbe204b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.353+0000 7fc2c8e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe200d0 0x55dabbe204b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:47708/0 (socket says 172.21.15.131:47708) 2024-05-08T22:08:36.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.353+0000 7fc2c8e00700 1 -- 172.21.15.131:0/1144689234 learned_addr learned my addr 172.21.15.131:0/1144689234 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:36.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.353+0000 7fc2c9800700 1 --2- 172.21.15.131:0/1144689234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dabbe1a910 0x55dabbe1ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.353+0000 7fc2c8e00700 1 -- 172.21.15.131:0/1144689234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabbe209f0 msgr2=0x55dabbe1a3d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:08:36.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.353+0000 7fc2c8e00700 1 --2- 172.21.15.131:0/1144689234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabbe209f0 0x55dabbe1a3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.353+0000 7fc2c8e00700 1 -- 172.21.15.131:0/1144689234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dabbe1a910 msgr2=0x55dabbe1ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:36.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.353+0000 7fc2c8e00700 1 --2- 172.21.15.131:0/1144689234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dabbe1a910 0x55dabbe1ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.353+0000 7fc2c8e00700 1 -- 172.21.15.131:0/1144689234 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dabbbeb430 con 0x55dabbe200d0 2024-05-08T22:08:36.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.353+0000 7fc2c3e00700 1 --2- 172.21.15.131:0/1939553826 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc2ac0a14c0 0x7fc2ac0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.354+0000 7fc2c9800700 1 --2- 172.21.15.131:0/1144689234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dabbe1a910 0x55dabbe1ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:08:36.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.354+0000 7fc2c8e00700 1 --2- 172.21.15.131:0/1144689234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe200d0 0x55dabbe204b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009cd0 tx=0x7fc2b800e350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af6c318f12a54177 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:36.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.354+0000 7fc2b6000700 1 -- 172.21.15.131:0/1144689234 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b800ec80 con 0x55dabbe200d0 2024-05-08T22:08:36.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.354+0000 7fc2b6000700 1 -- 172.21.15.131:0/1144689234 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2b800ede0 con 0x55dabbe200d0 2024-05-08T22:08:36.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.354+0000 7fc2b6000700 1 -- 172.21.15.131:0/1144689234 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8005700 con 0x55dabbe200d0 2024-05-08T22:08:36.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.355+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/1144689234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe200d0 msgr2=0x55dabbe204b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:36.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.355+0000 7fc2c3e00700 1 --2- 172.21.15.131:0/1939553826 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc2ac0a14c0 0x7fc2ac0a3980 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc2c419ec20 tx=0x7fc2b0009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:36.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.355+0000 7fc2df2bdb80 1 --2- 172.21.15.131:0/1144689234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe200d0 0x55dabbe204b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009cd0 tx=0x7fc2b800e350 comp rx=0 tx=0).stop 2024-05-08T22:08:36.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.355+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/1144689234 shutdown_connections 2024-05-08T22:08:36.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.355+0000 7fc2df2bdb80 1 --2- 172.21.15.131:0/1144689234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dabbe1a910 0x55dabbe1ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.355+0000 7fc2df2bdb80 1 --2- 172.21.15.131:0/1144689234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe200d0 0x55dabbe204b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.355+0000 7fc2df2bdb80 1 --2- 172.21.15.131:0/1144689234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabbe209f0 0x55dabbe1a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.355+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/1144689234 >> 172.21.15.131:0/1144689234 conn(0x55dabbd78290 msgr2=0x55dabbe15350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:36.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.356+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/1144689234 shutdown_connections 2024-05-08T22:08:36.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.356+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/1144689234 wait complete. 2024-05-08T22:08:36.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.356+0000 7fc2df2bdb80 1 -- start start 2024-05-08T22:08:36.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.357+0000 7fc2df2bdb80 1 Processor -- start 2024-05-08T22:08:36.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.357+0000 7fc2df2bdb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe1a910 0x55dabbe268d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.357+0000 7fc2df2bdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dabbe200d0 0x55dabbe23bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.357+0000 7fc2df2bdb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabbe209f0 0x55dabbe241a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.357+0000 7fc2df2bdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dabbd065e0 con 0x55dabbe200d0 2024-05-08T22:08:36.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.358+0000 7fc2df2bdb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dabbd06280 con 0x55dabbe209f0 2024-05-08T22:08:36.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.358+0000 7fc2df2bdb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dabbbed720 con 0x55dabbe1a910 2024-05-08T22:08:36.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.358+0000 7fc2df2bdb80 10 client.0 _ll_register_callbacks cb 0x55dabbd0fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:36.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.358+0000 7fc2c8e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe1a910 0x55dabbe268d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.358+0000 7fc2c8e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe1a910 0x55dabbe268d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:47710/0 (socket says 172.21.15.131:47710) 2024-05-08T22:08:36.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.358+0000 7fc2c8e00700 1 -- 172.21.15.131:0/1962161323 learned_addr learned my addr 172.21.15.131:0/1962161323 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:36.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.359+0000 7fc2c9800700 1 --2- 172.21.15.131:0/1962161323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabbe209f0 0x55dabbe241a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.359+0000 7fc2c3e00700 1 --2- 172.21.15.131:0/1962161323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dabbe200d0 0x55dabbe23bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.359+0000 7fc2c8e00700 1 -- 172.21.15.131:0/1962161323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabbe209f0 msgr2=0x55dabbe241a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:36.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.359+0000 7fc2c8e00700 1 --2- 172.21.15.131:0/1962161323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabbe209f0 0x55dabbe241a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.359+0000 7fc2c8e00700 1 -- 172.21.15.131:0/1962161323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dabbe200d0 msgr2=0x55dabbe23bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:36.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.359+0000 7fc2c8e00700 1 --2- 172.21.15.131:0/1962161323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dabbe200d0 0x55dabbe23bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.359+0000 7fc2c8e00700 1 -- 172.21.15.131:0/1962161323 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b8009980 con 0x55dabbe1a910 2024-05-08T22:08:36.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.359+0000 7fc2c8e00700 1 --2- 172.21.15.131:0/1962161323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe1a910 0x55dabbe268d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc2b800ef90 tx=0x7fc2b8005630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5298f896a76e3a53 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:36.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.359+0000 7fc2aaa00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:08:36.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.360+0000 7fc2aaa00700 1 -- 172.21.15.131:0/1962161323 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8004a30 con 0x55dabbe1a910 2024-05-08T22:08:36.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.360+0000 7fc2aaa00700 1 -- 172.21.15.131:0/1962161323 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fc2b8014070 con 0x55dabbe1a910 2024-05-08T22:08:36.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.360+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/1962161323 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dabbcc32f0 con 0x55dabbe1a910 2024-05-08T22:08:36.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.360+0000 7fc2aaa00700 1 -- 172.21.15.131:0/1962161323 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2b8004b90 con 0x55dabbe1a910 2024-05-08T22:08:36.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.360+0000 7fc2df2bdb80 10 client.25864 fetch_fsmap learned FSMap version 24 2024-05-08T22:08:36.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.360+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/1962161323 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55dabbb97be0 con 0x55dabbe1a910 2024-05-08T22:08:36.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.361+0000 7fc2aaa00700 1 -- 172.21.15.131:0/1962161323 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dabbcc32f0 con 0x55dabbe1a910 2024-05-08T22:08:36.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.361+0000 7fc2aaa00700 1 -- 172.21.15.131:0/1962161323 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc2b8010430 con 0x55dabbe1a910 2024-05-08T22:08:36.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.361+0000 7fc2df2bdb80 10 client.25864 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:08:36.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.361+0000 7fc2df2bdb80 10 client.25864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:08:36.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.361+0000 7fc2df2bdb80 20 client.25864 populate_metadata read hostname 'smithi131' 2024-05-08T22:08:36.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.361+0000 7fc2df2bdb80 1 --2- 172.21.15.131:0/1962161323 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55dabbe34310 0x55dabbe36730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.361+0000 7fc2df2bdb80 4 client.25864 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:08:36.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.361+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/1962161323 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dabbde9890 con 0x55dabbe34310 2024-05-08T22:08:36.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.362+0000 7fc2c3e00700 1 --2- 172.21.15.131:0/1962161323 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55dabbe34310 0x55dabbe36730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.362+0000 7fc2aaa00700 10 client.25864 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:36.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.362+0000 7fc2c3e00700 1 --2- 172.21.15.131:0/1962161323 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55dabbe34310 0x55dabbe36730 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50b20e8070e54c04 server_cookie=dd74eeeada12c4a in_seq=0 out_seq=0 2024-05-08T22:08:36.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.363+0000 7fc2aaa00700 1 -- 172.21.15.131:0/1962161323 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dabbde9890 con 0x55dabbe34310 2024-05-08T22:08:36.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.363+0000 7fc2aaa00700 10 client.25864 handle_command_reply: tid=0 2024-05-08T22:08:36.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.363+0000 7fc2df2bdb80 1 client.25864 shutdown 2024-05-08T22:08:36.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.363+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/1962161323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe1a910 msgr2=0x55dabbe268d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:36.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.363+0000 7fc2df2bdb80 1 --2- 172.21.15.131:0/1962161323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe1a910 0x55dabbe268d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc2b800ef90 tx=0x7fc2b8005630 comp rx=0 tx=0).stop 2024-05-08T22:08:36.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.363+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/1962161323 shutdown_connections 2024-05-08T22:08:36.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.363+0000 7fc2df2bdb80 1 --2- 172.21.15.131:0/1962161323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabbe209f0 0x55dabbe241a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.363+0000 7fc2df2bdb80 1 --2- 172.21.15.131:0/1962161323 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55dabbe34310 0x55dabbe36730 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.364+0000 7fc2df2bdb80 1 --2- 172.21.15.131:0/1962161323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe1a910 0x55dabbe268d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.364+0000 7fc2df2bdb80 1 --2- 172.21.15.131:0/1962161323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dabbe200d0 0x55dabbe23bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.364+0000 7fc2aaa00700 0 client.25864 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:36.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.364+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/1962161323 >> 172.21.15.131:0/1962161323 conn(0x55dabbd78290 msgr2=0x55dabbe15130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:36.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.364+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/1962161323 shutdown_connections 2024-05-08T22:08:36.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.364+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/1962161323 wait complete. 2024-05-08T22:08:36.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.364+0000 7fc2df2bdb80 20 client.25864 trim_cache size 0 max 16384 2024-05-08T22:08:36.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.388+0000 7fc2df2bdb80 1 Processor -- start 2024-05-08T22:08:36.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.388+0000 7fc2df2bdb80 1 -- start start 2024-05-08T22:08:36.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.389+0000 7fc2df2bdb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe1a910 0x55dabbe1a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.389+0000 7fc2df2bdb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabbe200d0 0x55dabbe21e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.389+0000 7fc2df2bdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dabbe209f0 0x55dabbe2b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.389+0000 7fc2df2bdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dabbe1fd30 con 0x55dabbe209f0 2024-05-08T22:08:36.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.389+0000 7fc2df2bdb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dabbe162b0 con 0x55dabbe200d0 2024-05-08T22:08:36.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.389+0000 7fc2df2bdb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dabbe26eb0 con 0x55dabbe1a910 2024-05-08T22:08:36.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.389+0000 7fc2c8e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe1a910 0x55dabbe1a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.389+0000 7fc2c8e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe1a910 0x55dabbe1a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:47712/0 (socket says 172.21.15.131:47712) 2024-05-08T22:08:36.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.389+0000 7fc2c8e00700 1 -- 172.21.15.131:0/2331250594 learned_addr learned my addr 172.21.15.131:0/2331250594 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:36.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.389+0000 7fc2c3e00700 1 --2- 172.21.15.131:0/2331250594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabbe200d0 0x55dabbe21e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.390+0000 7fc2c8e00700 1 -- 172.21.15.131:0/2331250594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabbe200d0 msgr2=0x55dabbe21e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:36.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.390+0000 7fc2c8e00700 1 --2- 172.21.15.131:0/2331250594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabbe200d0 0x55dabbe21e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.390+0000 7fc2c8e00700 1 -- 172.21.15.131:0/2331250594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dabbe209f0 msgr2=0x55dabbe2b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:36.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.390+0000 7fc2c8e00700 1 --2- 172.21.15.131:0/2331250594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dabbe209f0 0x55dabbe2b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.390+0000 7fc2c8e00700 1 -- 172.21.15.131:0/2331250594 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dabbd305e0 con 0x55dabbe1a910 2024-05-08T22:08:36.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.390+0000 7fc2c8e00700 1 --2- 172.21.15.131:0/2331250594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe1a910 0x55dabbe1a3d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8010830 tx=0x7fc2b800e350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5edcf6dbfe33577f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:36.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.390+0000 7fc2b6a00700 1 -- 172.21.15.131:0/2331250594 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8005090 con 0x55dabbe1a910 2024-05-08T22:08:36.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.390+0000 7fc2b6a00700 1 -- 172.21.15.131:0/2331250594 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2b80058e0 con 0x55dabbe1a910 2024-05-08T22:08:36.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.390+0000 7fc2b6a00700 1 -- 172.21.15.131:0/2331250594 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b802a840 con 0x55dabbe1a910 2024-05-08T22:08:36.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.391+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/2331250594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe1a910 msgr2=0x55dabbe1a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:36.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.391+0000 7fc2df2bdb80 1 --2- 172.21.15.131:0/2331250594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe1a910 0x55dabbe1a3d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8010830 tx=0x7fc2b800e350 comp rx=0 tx=0).stop 2024-05-08T22:08:36.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.391+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/2331250594 shutdown_connections 2024-05-08T22:08:36.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.391+0000 7fc2df2bdb80 1 --2- 172.21.15.131:0/2331250594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dabbe209f0 0x55dabbe2b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.391+0000 7fc2df2bdb80 1 --2- 172.21.15.131:0/2331250594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe1a910 0x55dabbe1a3d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.391+0000 7fc2df2bdb80 1 --2- 172.21.15.131:0/2331250594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabbe200d0 0x55dabbe21e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.391+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/2331250594 >> 172.21.15.131:0/2331250594 conn(0x55dabbd78290 msgr2=0x55dabbe2fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:36.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.391+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/2331250594 shutdown_connections 2024-05-08T22:08:36.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.391+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/2331250594 wait complete. 2024-05-08T22:08:36.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.392+0000 7fc2df2bdb80 1 -- start start 2024-05-08T22:08:36.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.392+0000 7fc2df2bdb80 1 Processor -- start 2024-05-08T22:08:36.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.393+0000 7fc2df2bdb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe1a910 0x55dabbe1c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.393+0000 7fc2df2bdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dabbe200d0 0x55dabbe1cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.394+0000 7fc2df2bdb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabbe209f0 0x55dabbe1d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.394+0000 7fc2df2bdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dabbe8a7d0 con 0x55dabbe200d0 2024-05-08T22:08:36.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.394+0000 7fc2df2bdb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dabbe8a940 con 0x55dabbe209f0 2024-05-08T22:08:36.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.394+0000 7fc2df2bdb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dabbe8ab00 con 0x55dabbe1a910 2024-05-08T22:08:36.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.394+0000 7fc2df2bdb80 10 client.0 _ll_register_callbacks cb 0x55dabbd0fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:36.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.394+0000 7fc2c8e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe1a910 0x55dabbe1c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.394+0000 7fc2c8e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe1a910 0x55dabbe1c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:47724/0 (socket says 172.21.15.131:47724) 2024-05-08T22:08:36.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.394+0000 7fc2c8e00700 1 -- 172.21.15.131:0/3571481489 learned_addr learned my addr 172.21.15.131:0/3571481489 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:36.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.394+0000 7fc2c3e00700 1 --2- 172.21.15.131:0/3571481489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dabbe200d0 0x55dabbe1cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.394+0000 7fc2c9800700 1 --2- 172.21.15.131:0/3571481489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabbe209f0 0x55dabbe1d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.394+0000 7fc2c8e00700 1 -- 172.21.15.131:0/3571481489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabbe209f0 msgr2=0x55dabbe1d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:36.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.394+0000 7fc2c8e00700 1 --2- 172.21.15.131:0/3571481489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabbe209f0 0x55dabbe1d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.394+0000 7fc2c8e00700 1 -- 172.21.15.131:0/3571481489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dabbe200d0 msgr2=0x55dabbe1cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:36.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.394+0000 7fc2c8e00700 1 --2- 172.21.15.131:0/3571481489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dabbe200d0 0x55dabbe1cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.394+0000 7fc2c8e00700 1 -- 172.21.15.131:0/3571481489 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dabbb97be0 con 0x55dabbe1a910 2024-05-08T22:08:36.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.395+0000 7fc2c9800700 1 --2- 172.21.15.131:0/3571481489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabbe209f0 0x55dabbe1d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:08:36.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.395+0000 7fc2c8e00700 1 --2- 172.21.15.131:0/3571481489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe1a910 0x55dabbe1c5a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc2b800edb0 tx=0x7fc2b8005d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cd353ef990a8302 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:36.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.395+0000 7fc2aaa00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:08:36.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.395+0000 7fc2aaa00700 1 -- 172.21.15.131:0/3571481489 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8004c40 con 0x55dabbe1a910 2024-05-08T22:08:36.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.395+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/3571481489 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2b800ede0 con 0x55dabbe1a910 2024-05-08T22:08:36.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.395+0000 7fc2aaa00700 1 -- 172.21.15.131:0/3571481489 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fc2b8000e40 con 0x55dabbe1a910 2024-05-08T22:08:36.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.396+0000 7fc2aaa00700 1 -- 172.21.15.131:0/3571481489 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2b8010530 con 0x55dabbe1a910 2024-05-08T22:08:36.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.396+0000 7fc2df2bdb80 10 client.25870 fetch_fsmap learned FSMap version 24 2024-05-08T22:08:36.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.396+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/3571481489 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55dabbbefd50 con 0x55dabbe1a910 2024-05-08T22:08:36.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.396+0000 7fc2aaa00700 1 -- 172.21.15.131:0/3571481489 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2b800ede0 con 0x55dabbe1a910 2024-05-08T22:08:36.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.396+0000 7fc2aaa00700 1 -- 172.21.15.131:0/3571481489 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc2b803ec10 con 0x55dabbe1a910 2024-05-08T22:08:36.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.396+0000 7fc2df2bdb80 10 client.25870 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:08:36.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.396+0000 7fc2df2bdb80 10 client.25870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:08:36.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.396+0000 7fc2df2bdb80 20 client.25870 populate_metadata read hostname 'smithi131' 2024-05-08T22:08:36.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.396+0000 7fc2df2bdb80 1 --2- 172.21.15.131:0/3571481489 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55dabbe8be10 0x55dabbe8e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.396+0000 7fc2df2bdb80 4 client.25870 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T22:08:36.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.396+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/3571481489 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55dabbd62f40 con 0x55dabbe8be10 2024-05-08T22:08:36.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.397+0000 7fc2c3e00700 1 --2- 172.21.15.131:0/3571481489 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55dabbe8be10 0x55dabbe8e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.397+0000 7fc2aaa00700 10 client.25870 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:36.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.397+0000 7fc2c3e00700 1 --2- 172.21.15.131:0/3571481489 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55dabbe8be10 0x55dabbe8e200 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2f4333590cde24e server_cookie=4fe1b08e94974d3a in_seq=0 out_seq=0 2024-05-08T22:08:36.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.398+0000 7fc2aaa00700 1 -- 172.21.15.131:0/3571481489 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55dabbd62f40 con 0x55dabbe8be10 2024-05-08T22:08:36.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.398+0000 7fc2aaa00700 10 client.25870 handle_command_reply: tid=0 2024-05-08T22:08:36.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.399+0000 7fc2df2bdb80 1 client.25870 shutdown 2024-05-08T22:08:36.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.399+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/3571481489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe1a910 msgr2=0x55dabbe1c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:36.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.399+0000 7fc2df2bdb80 1 --2- 172.21.15.131:0/3571481489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe1a910 0x55dabbe1c5a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc2b800edb0 tx=0x7fc2b8005d10 comp rx=0 tx=0).stop 2024-05-08T22:08:36.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.399+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/3571481489 shutdown_connections 2024-05-08T22:08:36.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.399+0000 7fc2df2bdb80 1 --2- 172.21.15.131:0/3571481489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabbe209f0 0x55dabbe1d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.399+0000 7fc2df2bdb80 1 --2- 172.21.15.131:0/3571481489 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55dabbe8be10 0x55dabbe8e200 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.399+0000 7fc2df2bdb80 1 --2- 172.21.15.131:0/3571481489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dabbe1a910 0x55dabbe1c5a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.399+0000 7fc2df2bdb80 1 --2- 172.21.15.131:0/3571481489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dabbe200d0 0x55dabbe1cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.399+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/3571481489 >> 172.21.15.131:0/3571481489 conn(0x55dabbd78290 msgr2=0x55dabbe1ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:36.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.399+0000 7fc2aaa00700 0 client.25870 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:36.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.399+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/3571481489 shutdown_connections 2024-05-08T22:08:36.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.400+0000 7fc2df2bdb80 1 -- 172.21.15.131:0/3571481489 wait complete. 2024-05-08T22:08:36.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.400+0000 7fc2df2bdb80 20 client.25870 trim_cache size 0 max 16384 2024-05-08T22:08:36.402 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:08:36.402 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-08T22:08:36.402 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "e2f0ba16-3caa-4ec8-9ed2-571d17c6ac2d", 2024-05-08T22:08:36.402 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-08T22:08:36.403 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:08:36.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.401+0000 7fc2ca200700 1 -- 172.21.15.131:0/1939553826 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc2ac0a14c0 msgr2=0x7fc2ac0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:36.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.401+0000 7fc2ca200700 1 --2- 172.21.15.131:0/1939553826 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc2ac0a14c0 0x7fc2ac0a3980 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc2c419ec20 tx=0x7fc2b0009260 comp rx=0 tx=0).stop 2024-05-08T22:08:36.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.401+0000 7fc2ca200700 1 -- 172.21.15.131:0/1939553826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2c419d800 msgr2=0x7fc2c419dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:36.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.401+0000 7fc2ca200700 1 --2- 172.21.15.131:0/1939553826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2c419d800 0x7fc2c419dcb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00b070 tx=0x7fc2bc00bda0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.401+0000 7fc2ca200700 1 -- 172.21.15.131:0/1939553826 shutdown_connections 2024-05-08T22:08:36.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.401+0000 7fc2ca200700 1 --2- 172.21.15.131:0/1939553826 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc2ac0a14c0 0x7fc2ac0a3980 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.401+0000 7fc2ca200700 1 --2- 172.21.15.131:0/1939553826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2c419d800 0x7fc2c419dcb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.401+0000 7fc2ca200700 1 --2- 172.21.15.131:0/1939553826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2c4107750 0x7fc2c41a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.401+0000 7fc2ca200700 1 --2- 172.21.15.131:0/1939553826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2c410c720 0x7fc2c41a3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.401+0000 7fc2ca200700 1 -- 172.21.15.131:0/1939553826 >> 172.21.15.131:0/1939553826 conn(0x7fc2c4076dc0 msgr2=0x7fc2c4078a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:36.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.402+0000 7fc2ca200700 1 -- 172.21.15.131:0/1939553826 shutdown_connections 2024-05-08T22:08:36.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.402+0000 7fc2ca200700 1 -- 172.21.15.131:0/1939553826 wait complete. 2024-05-08T22:08:36.422 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:08:36.482 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:36 smithi131 ceph-mon[37260]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:08:36.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:36 smithi132 ceph-mon[78180]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:08:36.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.626+0000 7fb0f3200700 1 -- 172.21.15.131:0/4070653843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0f4102890 msgr2=0x7fb0f410ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:36.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.626+0000 7fb0f3200700 1 --2- 172.21.15.131:0/4070653843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0f4102890 0x7fb0f410ad70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009990 tx=0x7fb0e0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.626+0000 7fb0f3200700 1 -- 172.21.15.131:0/4070653843 shutdown_connections 2024-05-08T22:08:36.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.626+0000 7fb0f3200700 1 --2- 172.21.15.131:0/4070653843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0f410b390 0x7fb0f410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.626+0000 7fb0f3200700 1 --2- 172.21.15.131:0/4070653843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0f4101f70 0x7fb0f4102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.626+0000 7fb0f3200700 1 --2- 172.21.15.131:0/4070653843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0f4102890 0x7fb0f410ad70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.626+0000 7fb0f3200700 1 -- 172.21.15.131:0/4070653843 >> 172.21.15.131:0/4070653843 conn(0x7fb0f40fb770 msgr2=0x7fb0f40fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:36.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.626+0000 7fb0f3200700 1 -- 172.21.15.131:0/4070653843 shutdown_connections 2024-05-08T22:08:36.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.626+0000 7fb0f3200700 1 -- 172.21.15.131:0/4070653843 wait complete. 2024-05-08T22:08:36.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.627+0000 7fb0f3200700 1 Processor -- start 2024-05-08T22:08:36.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.628+0000 7fb0f3200700 1 -- start start 2024-05-08T22:08:36.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.628+0000 7fb0f3200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0f4101f70 0x7fb0f4197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.628+0000 7fb0f3200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0f4102890 0x7fb0f41978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.628+0000 7fb0f3200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0f410b390 0x7fb0f419bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.628+0000 7fb0f3200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f419c1f0 con 0x7fb0f4102890 2024-05-08T22:08:36.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.628+0000 7fb0f3200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f419c360 con 0x7fb0f410b390 2024-05-08T22:08:36.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.628+0000 7fb0f3200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f419c520 con 0x7fb0f4101f70 2024-05-08T22:08:36.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.629+0000 7fb0f2800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0f410b390 0x7fb0f419bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.629+0000 7fb0f1e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0f4101f70 0x7fb0f4197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.629+0000 7fb0f1e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0f4101f70 0x7fb0f4197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:47744/0 (socket says 172.21.15.131:47744) 2024-05-08T22:08:36.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.629+0000 7fb0f1e00700 1 -- 172.21.15.131:0/2685182146 learned_addr learned my addr 172.21.15.131:0/2685182146 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:36.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.629+0000 7fb0f1400700 1 --2- 172.21.15.131:0/2685182146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0f4102890 0x7fb0f41978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.629+0000 7fb0f1e00700 1 -- 172.21.15.131:0/2685182146 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0f410b390 msgr2=0x7fb0f419bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:36.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.629+0000 7fb0f1e00700 1 --2- 172.21.15.131:0/2685182146 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0f410b390 0x7fb0f419bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.629+0000 7fb0f1e00700 1 -- 172.21.15.131:0/2685182146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0f4102890 msgr2=0x7fb0f41978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:36.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.629+0000 7fb0f1e00700 1 --2- 172.21.15.131:0/2685182146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0f4102890 0x7fb0f41978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.629+0000 7fb0f1e00700 1 -- 172.21.15.131:0/2685182146 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e00095f0 con 0x7fb0f4101f70 2024-05-08T22:08:36.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.629+0000 7fb0f1e00700 1 --2- 172.21.15.131:0/2685182146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0f4101f70 0x7fb0f4197390 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400e9a0 tx=0x7fb0e400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:36.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.630+0000 7fb0eaa00700 1 -- 172.21.15.131:0/2685182146 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4009790 con 0x7fb0f4101f70 2024-05-08T22:08:36.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.630+0000 7fb0eaa00700 1 -- 172.21.15.131:0/2685182146 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0e400cbd0 con 0x7fb0f4101f70 2024-05-08T22:08:36.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.630+0000 7fb0f3200700 1 -- 172.21.15.131:0/2685182146 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f419c800 con 0x7fb0f4101f70 2024-05-08T22:08:36.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.630+0000 7fb0eaa00700 1 -- 172.21.15.131:0/2685182146 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4018680 con 0x7fb0f4101f70 2024-05-08T22:08:36.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.630+0000 7fb0f3200700 1 -- 172.21.15.131:0/2685182146 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f40ffcd0 con 0x7fb0f4101f70 2024-05-08T22:08:36.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.635+0000 7fb0eaa00700 1 -- 172.21.15.131:0/2685182146 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 24) v1 ==== 136158+0+0 (secure 0 0 0) 0x7fb0e40187e0 con 0x7fb0f4101f70 2024-05-08T22:08:36.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.635+0000 7fb10c5f4b80 1 Processor -- start 2024-05-08T22:08:36.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.636+0000 7fb10c5f4b80 1 -- start start 2024-05-08T22:08:36.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.636+0000 7fb10c5f4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c8d4050 0x56076c8d4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.636+0000 7fb10c5f4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56076c8d4970 0x56076c8ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.636+0000 7fb10c5f4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56076c8ce890 0x56076c8cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.636+0000 7fb10c5f4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076c825150 con 0x56076c8ce890 2024-05-08T22:08:36.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.636+0000 7fb10c5f4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076c822b10 con 0x56076c8d4970 2024-05-08T22:08:36.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.636+0000 7fb10c5f4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076c81e400 con 0x56076c8d4050 2024-05-08T22:08:36.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.636+0000 7fb0f2800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c8d4050 0x56076c8d4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.636+0000 7fb0f2800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c8d4050 0x56076c8d4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:47760/0 (socket says 172.21.15.131:47760) 2024-05-08T22:08:36.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.636+0000 7fb0f2800700 1 -- 172.21.15.131:0/2359917963 learned_addr learned my addr 172.21.15.131:0/2359917963 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:36.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.636+0000 7fb0f1e00700 1 --2- 172.21.15.131:0/2359917963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56076c8d4970 0x56076c8ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.637+0000 7fb0f2800700 1 -- 172.21.15.131:0/2359917963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56076c8d4970 msgr2=0x56076c8ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:36.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.637+0000 7fb0f2800700 1 --2- 172.21.15.131:0/2359917963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56076c8d4970 0x56076c8ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.637+0000 7fb0f2800700 1 -- 172.21.15.131:0/2359917963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56076c8ce890 msgr2=0x56076c8cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:36.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.637+0000 7fb0f2800700 1 --2- 172.21.15.131:0/2359917963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56076c8ce890 0x56076c8cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.637+0000 7fb0f2800700 1 -- 172.21.15.131:0/2359917963 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56076c69f3b0 con 0x56076c8d4050 2024-05-08T22:08:36.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.637+0000 7fb0f2800700 1 --2- 172.21.15.131:0/2359917963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c8d4050 0x56076c8d4430 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec011680 tx=0x7fb0ec009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3497f006f3bfb045 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:36.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.637+0000 7fb0eaa00700 1 --2- 172.21.15.131:0/2685182146 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb0d80a14c0 0x7fb0d80a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.637+0000 7fb0df000700 1 -- 172.21.15.131:0/2359917963 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec00e040 con 0x56076c8d4050 2024-05-08T22:08:36.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.638+0000 7fb0df000700 1 -- 172.21.15.131:0/2359917963 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0ec00f040 con 0x56076c8d4050 2024-05-08T22:08:36.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.638+0000 7fb0eaa00700 1 -- 172.21.15.131:0/2685182146 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fb0e4014070 con 0x7fb0f4101f70 2024-05-08T22:08:36.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.638+0000 7fb0df000700 1 -- 172.21.15.131:0/2359917963 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec004840 con 0x56076c8d4050 2024-05-08T22:08:36.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.638+0000 7fb0f1e00700 1 --2- 172.21.15.131:0/2685182146 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb0d80a14c0 0x7fb0d80a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.639+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/2359917963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c8d4050 msgr2=0x56076c8d4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:36.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.639+0000 7fb0f1e00700 1 --2- 172.21.15.131:0/2685182146 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb0d80a14c0 0x7fb0d80a3980 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40ca6f0 tx=0x7fb0e40d0dc0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:36.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.639+0000 7fb10c5f4b80 1 --2- 172.21.15.131:0/2359917963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c8d4050 0x56076c8d4430 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec011680 tx=0x7fb0ec009920 comp rx=0 tx=0).stop 2024-05-08T22:08:36.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.639+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/2359917963 shutdown_connections 2024-05-08T22:08:36.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.639+0000 7fb10c5f4b80 1 --2- 172.21.15.131:0/2359917963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56076c8ce890 0x56076c8cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.639+0000 7fb10c5f4b80 1 --2- 172.21.15.131:0/2359917963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c8d4050 0x56076c8d4430 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.639+0000 7fb10c5f4b80 1 --2- 172.21.15.131:0/2359917963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56076c8d4970 0x56076c8ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.639+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/2359917963 >> 172.21.15.131:0/2359917963 conn(0x56076c82c210 msgr2=0x56076c8c92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:36.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.639+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/2359917963 shutdown_connections 2024-05-08T22:08:36.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.639+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/2359917963 wait complete. 2024-05-08T22:08:36.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.639+0000 7fb10c5f4b80 1 -- start start 2024-05-08T22:08:36.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.640+0000 7fb10c5f4b80 1 Processor -- start 2024-05-08T22:08:36.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.640+0000 7fb10c5f4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c8ce890 0x56076c8da970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.640+0000 7fb10c5f4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56076c8d4050 0x56076c8d7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.640+0000 7fb10c5f4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56076c8d4970 0x56076c8d83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.640+0000 7fb10c5f4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076c7ba560 con 0x56076c8d4050 2024-05-08T22:08:36.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.640+0000 7fb10c5f4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076c7ba200 con 0x56076c8d4970 2024-05-08T22:08:36.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.640+0000 7fb10c5f4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076c6a16a0 con 0x56076c8ce890 2024-05-08T22:08:36.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.640+0000 7fb10c5f4b80 10 client.0 _ll_register_callbacks cb 0x56076c7c3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:36.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.641+0000 7fb0f2800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56076c8d4970 0x56076c8d83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.641+0000 7fb0f2800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56076c8d4970 0x56076c8d83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:41652/0 (socket says 172.21.15.131:41652) 2024-05-08T22:08:36.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.641+0000 7fb0f2800700 1 -- 172.21.15.131:0/1291344852 learned_addr learned my addr 172.21.15.131:0/1291344852 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:36.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.641+0000 7fb0f2800700 1 --2- 172.21.15.131:0/1291344852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c8ce890 0x56076c8da970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.641+0000 7fb0f1400700 1 --2- 172.21.15.131:0/1291344852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56076c8d4050 0x56076c8d7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.641+0000 7fb0f2800700 1 -- 172.21.15.131:0/1291344852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56076c8d4970 msgr2=0x56076c8d83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:36.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.641+0000 7fb0f2800700 1 --2- 172.21.15.131:0/1291344852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56076c8d4970 0x56076c8d83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.641+0000 7fb0f2800700 1 -- 172.21.15.131:0/1291344852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56076c8d4050 msgr2=0x56076c8d7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:36.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.641+0000 7fb0f2800700 1 --2- 172.21.15.131:0/1291344852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56076c8d4050 0x56076c8d7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.641+0000 7fb0f2800700 1 -- 172.21.15.131:0/1291344852 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0ec011330 con 0x56076c8ce890 2024-05-08T22:08:36.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.641+0000 7fb0f1400700 1 --2- 172.21.15.131:0/1291344852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56076c8d4050 0x56076c8d7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:08:36.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.641+0000 7fb0f2800700 1 --2- 172.21.15.131:0/1291344852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c8ce890 0x56076c8da970 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009530 tx=0x7fb0ec004340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad96b76dcd710f0c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:36.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.641+0000 7fb0d3400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:08:36.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.642+0000 7fb0d3400700 1 -- 172.21.15.131:0/1291344852 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec00e040 con 0x56076c8ce890 2024-05-08T22:08:36.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.642+0000 7fb0d3400700 1 -- 172.21.15.131:0/1291344852 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fb0ec007500 con 0x56076c8ce890 2024-05-08T22:08:36.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.642+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/1291344852 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56076c777270 con 0x56076c8ce890 2024-05-08T22:08:36.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.642+0000 7fb0d3400700 1 -- 172.21.15.131:0/1291344852 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0ec004010 con 0x56076c8ce890 2024-05-08T22:08:36.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.642+0000 7fb10c5f4b80 10 client.25882 fetch_fsmap learned FSMap version 24 2024-05-08T22:08:36.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.642+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/1291344852 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56076c64bb60 con 0x56076c8ce890 2024-05-08T22:08:36.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.642+0000 7fb0d3400700 1 -- 172.21.15.131:0/1291344852 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56076c777270 con 0x56076c8ce890 2024-05-08T22:08:36.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.643+0000 7fb0d3400700 1 -- 172.21.15.131:0/1291344852 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb0ec015640 con 0x56076c8ce890 2024-05-08T22:08:36.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.643+0000 7fb10c5f4b80 10 client.25882 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:08:36.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.643+0000 7fb10c5f4b80 10 client.25882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:08:36.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.643+0000 7fb10c5f4b80 20 client.25882 populate_metadata read hostname 'smithi131' 2024-05-08T22:08:36.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.643+0000 7fb10c5f4b80 1 --2- 172.21.15.131:0/1291344852 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x56076c8f0980 0x56076c8f2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.643+0000 7fb10c5f4b80 4 client.25882 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:08:36.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.643+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/1291344852 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56076c89d810 con 0x56076c8f0980 2024-05-08T22:08:36.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.643+0000 7fb0f1400700 1 --2- 172.21.15.131:0/1291344852 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x56076c8f0980 0x56076c8f2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.644+0000 7fb0f1400700 1 --2- 172.21.15.131:0/1291344852 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x56076c8f0980 0x56076c8f2da0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e20202b15bf91420 server_cookie=3c972570dbda671d in_seq=0 out_seq=0 2024-05-08T22:08:36.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.644+0000 7fb0d3400700 10 client.25882 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:36.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.644+0000 7fb0d3400700 1 -- 172.21.15.131:0/1291344852 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56076c89d810 con 0x56076c8f0980 2024-05-08T22:08:36.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.644+0000 7fb0d3400700 10 client.25882 handle_command_reply: tid=0 2024-05-08T22:08:36.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.645+0000 7fb10c5f4b80 1 client.25882 shutdown 2024-05-08T22:08:36.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.645+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/1291344852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c8ce890 msgr2=0x56076c8da970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:36.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.645+0000 7fb10c5f4b80 1 --2- 172.21.15.131:0/1291344852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c8ce890 0x56076c8da970 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009530 tx=0x7fb0ec004340 comp rx=0 tx=0).stop 2024-05-08T22:08:36.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.645+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/1291344852 shutdown_connections 2024-05-08T22:08:36.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.645+0000 7fb10c5f4b80 1 --2- 172.21.15.131:0/1291344852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56076c8d4970 0x56076c8d83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.645+0000 7fb10c5f4b80 1 --2- 172.21.15.131:0/1291344852 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x56076c8f0980 0x56076c8f2da0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.645+0000 7fb10c5f4b80 1 --2- 172.21.15.131:0/1291344852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c8ce890 0x56076c8da970 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.645+0000 7fb10c5f4b80 1 --2- 172.21.15.131:0/1291344852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56076c8d4050 0x56076c8d7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.645+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/1291344852 >> 172.21.15.131:0/1291344852 conn(0x56076c82c210 msgr2=0x56076c8c90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:36.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.645+0000 7fb0d3400700 0 client.25882 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:36.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.645+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/1291344852 shutdown_connections 2024-05-08T22:08:36.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.645+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/1291344852 wait complete. 2024-05-08T22:08:36.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.645+0000 7fb10c5f4b80 20 client.25882 trim_cache size 0 max 16384 2024-05-08T22:08:36.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.663+0000 7fb10c5f4b80 1 Processor -- start 2024-05-08T22:08:36.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.663+0000 7fb10c5f4b80 1 -- start start 2024-05-08T22:08:36.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.664+0000 7fb10c5f4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56076c8ce890 0x56076c8f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.664+0000 7fb10c5f4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56076c8d4050 0x56076c8da4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.664+0000 7fb10c5f4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c8d4970 0x56076c8da9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.664+0000 7fb10c5f4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076c8ce350 con 0x56076c8ce890 2024-05-08T22:08:36.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.664+0000 7fb10c5f4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076c8c97c0 con 0x56076c8d4050 2024-05-08T22:08:36.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.664+0000 7fb10c5f4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076c8c1040 con 0x56076c8d4970 2024-05-08T22:08:36.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.664+0000 7fb0f2800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56076c8ce890 0x56076c8f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.664+0000 7fb0f2800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56076c8ce890 0x56076c8f2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36532/0 (socket says 172.21.15.131:36532) 2024-05-08T22:08:36.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.664+0000 7fb0f2800700 1 -- 172.21.15.131:0/3063869826 learned_addr learned my addr 172.21.15.131:0/3063869826 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:36.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.664+0000 7fb0f1400700 1 --2- 172.21.15.131:0/3063869826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56076c8d4050 0x56076c8da4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.664+0000 7fb0f2800700 1 --2- 172.21.15.131:0/3063869826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c8d4970 0x56076c8da9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.664+0000 7fb0f1400700 1 -- 172.21.15.131:0/3063869826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c8d4970 msgr2=0x56076c8da9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:36.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.664+0000 7fb0f1400700 1 --2- 172.21.15.131:0/3063869826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c8d4970 0x56076c8da9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.664+0000 7fb0f1400700 1 -- 172.21.15.131:0/3063869826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56076c8ce890 msgr2=0x56076c8f2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:36.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.664+0000 7fb0f1400700 1 --2- 172.21.15.131:0/3063869826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56076c8ce890 0x56076c8f2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.665+0000 7fb0f1400700 1 -- 172.21.15.131:0/3063869826 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56076c7e4560 con 0x56076c8d4050 2024-05-08T22:08:36.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.665+0000 7fb0f1400700 1 --2- 172.21.15.131:0/3063869826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56076c8d4050 0x56076c8da4a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0000c00 tx=0x7fb0e0005980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffa512895e37909c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:36.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.665+0000 7fb0dfa00700 1 -- 172.21.15.131:0/3063869826 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e00253a0 con 0x56076c8d4050 2024-05-08T22:08:36.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.665+0000 7fb0dfa00700 1 -- 172.21.15.131:0/3063869826 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0e000b750 con 0x56076c8d4050 2024-05-08T22:08:36.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.665+0000 7fb0dfa00700 1 -- 172.21.15.131:0/3063869826 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0004760 con 0x56076c8d4050 2024-05-08T22:08:36.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.666+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/3063869826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56076c8d4050 msgr2=0x56076c8da4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:36.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.666+0000 7fb10c5f4b80 1 --2- 172.21.15.131:0/3063869826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56076c8d4050 0x56076c8da4a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0000c00 tx=0x7fb0e0005980 comp rx=0 tx=0).stop 2024-05-08T22:08:36.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.666+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/3063869826 shutdown_connections 2024-05-08T22:08:36.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.666+0000 7fb10c5f4b80 1 --2- 172.21.15.131:0/3063869826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c8d4970 0x56076c8da9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.666+0000 7fb10c5f4b80 1 --2- 172.21.15.131:0/3063869826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56076c8ce890 0x56076c8f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.666+0000 7fb10c5f4b80 1 --2- 172.21.15.131:0/3063869826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56076c8d4050 0x56076c8da4a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.666+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/3063869826 >> 172.21.15.131:0/3063869826 conn(0x56076c82c210 msgr2=0x56076c8c7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:36.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.666+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/3063869826 shutdown_connections 2024-05-08T22:08:36.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.666+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/3063869826 wait complete. 2024-05-08T22:08:36.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.666+0000 7fb10c5f4b80 1 -- start start 2024-05-08T22:08:36.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.667+0000 7fb10c5f4b80 1 Processor -- start 2024-05-08T22:08:36.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.667+0000 7fb10c5f4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c8ce890 0x56076c8e80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.667+0000 7fb10c5f4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56076c8d4050 0x56076c8d0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.667+0000 7fb10c5f4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56076c8d4970 0x56076c8d1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.667+0000 7fb10c5f4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076c8cfa90 con 0x56076c8d4050 2024-05-08T22:08:36.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.667+0000 7fb10c5f4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076c8cfc00 con 0x56076c8d4970 2024-05-08T22:08:36.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.667+0000 7fb10c5f4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076c8cfdc0 con 0x56076c8ce890 2024-05-08T22:08:36.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.667+0000 7fb10c5f4b80 10 client.0 _ll_register_callbacks cb 0x56076c7c3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:08:36.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.668+0000 7fb0f1400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56076c8d4050 0x56076c8d0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.668+0000 7fb0f1400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56076c8d4050 0x56076c8d0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36534/0 (socket says 172.21.15.131:36534) 2024-05-08T22:08:36.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.668+0000 7fb0f1400700 1 -- 172.21.15.131:0/2704010351 learned_addr learned my addr 172.21.15.131:0/2704010351 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:08:36.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.668+0000 7fb0f2800700 1 --2- 172.21.15.131:0/2704010351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c8ce890 0x56076c8e80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.668+0000 7fb0f2800700 1 --2- 172.21.15.131:0/2704010351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56076c8d4970 0x56076c8d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.668+0000 7fb0f2800700 1 -- 172.21.15.131:0/2704010351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56076c8d4970 msgr2=0x56076c8d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:36.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.668+0000 7fb0f2800700 1 --2- 172.21.15.131:0/2704010351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56076c8d4970 0x56076c8d1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.668+0000 7fb0f2800700 1 -- 172.21.15.131:0/2704010351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56076c8d4050 msgr2=0x56076c8d0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:08:36.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.668+0000 7fb0f2800700 1 --2- 172.21.15.131:0/2704010351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56076c8d4050 0x56076c8d0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.668+0000 7fb0f2800700 1 -- 172.21.15.131:0/2704010351 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f410e510 con 0x56076c8ce890 2024-05-08T22:08:36.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.668+0000 7fb0f2800700 1 --2- 172.21.15.131:0/2704010351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c8ce890 0x56076c8e80e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x56076c893ca0 tx=0x7fb0ec021040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b65270e6639d5e56 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:08:36.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.668+0000 7fb0d3400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:08:36.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.668+0000 7fb0d3400700 1 -- 172.21.15.131:0/2704010351 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec00e020 con 0x56076c8ce890 2024-05-08T22:08:36.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.669+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/2704010351 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0ec00f040 con 0x56076c8ce890 2024-05-08T22:08:36.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.669+0000 7fb0d3400700 1 -- 172.21.15.131:0/2704010351 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fb0ec015c70 con 0x56076c8ce890 2024-05-08T22:08:36.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.669+0000 7fb0d3400700 1 -- 172.21.15.131:0/2704010351 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0ec004980 con 0x56076c8ce890 2024-05-08T22:08:36.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.669+0000 7fb10c5f4b80 10 client.25888 fetch_fsmap learned FSMap version 24 2024-05-08T22:08:36.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.669+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/2704010351 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56076c6a3cd0 con 0x56076c8ce890 2024-05-08T22:08:36.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.669+0000 7fb0d3400700 1 -- 172.21.15.131:0/2704010351 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0ec00f040 con 0x56076c8ce890 2024-05-08T22:08:36.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.669+0000 7fb0d3400700 1 -- 172.21.15.131:0/2704010351 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb0ec034070 con 0x56076c8ce890 2024-05-08T22:08:36.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.669+0000 7fb10c5f4b80 10 client.25888 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:08:36.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.669+0000 7fb10c5f4b80 10 client.25888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:08:36.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.669+0000 7fb10c5f4b80 20 client.25888 populate_metadata read hostname 'smithi131' 2024-05-08T22:08:36.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.669+0000 7fb10c5f4b80 1 --2- 172.21.15.131:0/2704010351 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x56076c9476c0 0x56076c949ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:08:36.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.669+0000 7fb10c5f4b80 4 client.25888 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:08:36.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.669+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/2704010351 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56076c816ec0 con 0x56076c9476c0 2024-05-08T22:08:36.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.670+0000 7fb0f1400700 1 --2- 172.21.15.131:0/2704010351 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x56076c9476c0 0x56076c949ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:08:36.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.670+0000 7fb0f1400700 1 --2- 172.21.15.131:0/2704010351 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x56076c9476c0 0x56076c949ae0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96181274346958ae server_cookie=e39f567bd553edd1 in_seq=0 out_seq=0 2024-05-08T22:08:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.670+0000 7fb0d3400700 10 client.25888 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.671+0000 7fb0d3400700 1 -- 172.21.15.131:0/2704010351 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56076c816ec0 con 0x56076c9476c0 2024-05-08T22:08:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.671+0000 7fb0d3400700 10 client.25888 handle_command_reply: tid=0 2024-05-08T22:08:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.671+0000 7fb10c5f4b80 1 client.25888 shutdown 2024-05-08T22:08:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.671+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/2704010351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c8ce890 msgr2=0x56076c8e80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.671+0000 7fb10c5f4b80 1 --2- 172.21.15.131:0/2704010351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c8ce890 0x56076c8e80e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x56076c893ca0 tx=0x7fb0ec021040 comp rx=0 tx=0).stop 2024-05-08T22:08:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.671+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/2704010351 shutdown_connections 2024-05-08T22:08:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.671+0000 7fb10c5f4b80 1 --2- 172.21.15.131:0/2704010351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56076c8d4970 0x56076c8d1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.671+0000 7fb10c5f4b80 1 --2- 172.21.15.131:0/2704010351 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x56076c9476c0 0x56076c949ae0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.671+0000 7fb0d3400700 0 client.25888 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:08:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.671+0000 7fb10c5f4b80 1 --2- 172.21.15.131:0/2704010351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c8ce890 0x56076c8e80e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.671+0000 7fb10c5f4b80 1 --2- 172.21.15.131:0/2704010351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56076c8d4050 0x56076c8d0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.671+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/2704010351 >> 172.21.15.131:0/2704010351 conn(0x56076c82c210 msgr2=0x56076c8d1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.671+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/2704010351 shutdown_connections 2024-05-08T22:08:36.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.671+0000 7fb10c5f4b80 1 -- 172.21.15.131:0/2704010351 wait complete. 2024-05-08T22:08:36.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.671+0000 7fb10c5f4b80 20 client.25888 trim_cache size 0 max 16384 2024-05-08T22:08:36.675 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:08:36.675 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T22:08:36.675 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-08T22:08:36.675 INFO:teuthology.orchestra.run.smithi131.stdout: "e2f0ba16-3caa-4ec8-9ed2-571d17c6ac2d": { 2024-05-08T22:08:36.675 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-08T22:08:36.675 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "e2f0ba16-3caa-4ec8-9ed2-571d17c6ac2d", 2024-05-08T22:08:36.675 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-08T22:08:36.675 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:08:36.675 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:08:36.676 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:08:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.672+0000 7fb0f3200700 1 -- 172.21.15.131:0/2685182146 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb0d80a14c0 msgr2=0x7fb0d80a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.672+0000 7fb0f3200700 1 --2- 172.21.15.131:0/2685182146 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb0d80a14c0 0x7fb0d80a3980 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40ca6f0 tx=0x7fb0e40d0dc0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.672+0000 7fb0f3200700 1 -- 172.21.15.131:0/2685182146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0f4101f70 msgr2=0x7fb0f4197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:08:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.672+0000 7fb0f3200700 1 --2- 172.21.15.131:0/2685182146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0f4101f70 0x7fb0f4197390 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400e9a0 tx=0x7fb0e400ecb0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.672+0000 7fb0f3200700 1 -- 172.21.15.131:0/2685182146 shutdown_connections 2024-05-08T22:08:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.672+0000 7fb0f3200700 1 --2- 172.21.15.131:0/2685182146 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb0d80a14c0 0x7fb0d80a3980 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.672+0000 7fb0f3200700 1 --2- 172.21.15.131:0/2685182146 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0f410b390 0x7fb0f419bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.672+0000 7fb0f3200700 1 --2- 172.21.15.131:0/2685182146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0f4101f70 0x7fb0f4197390 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.672+0000 7fb0f3200700 1 --2- 172.21.15.131:0/2685182146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0f4102890 0x7fb0f41978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:08:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.672+0000 7fb0f3200700 1 -- 172.21.15.131:0/2685182146 >> 172.21.15.131:0/2685182146 conn(0x7fb0f40fb770 msgr2=0x7fb0f40fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:08:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.672+0000 7fb0f3200700 1 -- 172.21.15.131:0/2685182146 shutdown_connections 2024-05-08T22:08:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:08:36.672+0000 7fb0f3200700 1 -- 172.21.15.131:0/2685182146 wait complete. 2024-05-08T22:08:36.692 INFO:tasks.cephfs.filesystem:scrub status for tag:e2f0ba16-3caa-4ec8-9ed2-571d17c6ac2d - {'path': '/', 'tag': 'e2f0ba16-3caa-4ec8-9ed2-571d17c6ac2d', 'options': 'recursive,force'} 2024-05-08T22:08:36.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:36 smithi157 ceph-mon[77746]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:08:37.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:37 smithi131 ceph-mon[37260]: scrub queued for path: / 2024-05-08T22:08:37.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:37 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:08:37.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:37 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:08:37.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:37 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:08:37.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:37 smithi157 ceph-mon[77746]: scrub queued for path: / 2024-05-08T22:08:37.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:37 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:08:37.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:37 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:08:37.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:37 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:08:37.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:37 smithi132 ceph-mon[78180]: scrub queued for path: / 2024-05-08T22:08:37.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:37 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:08:37.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:37 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:08:37.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:37 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:08:38.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:38 smithi131 ceph-mon[37260]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:08:38.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:38 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:08:38.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:38 smithi157 ceph-mon[77746]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:08:38.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:38 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:08:38.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:38 smithi132 ceph-mon[78180]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:08:38.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:38 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:08:40.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:40 smithi131 ceph-mon[37260]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-08T22:08:40.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:40 smithi157 ceph-mon[77746]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-08T22:08:40.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:40 smithi132 ceph-mon[78180]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-08T22:08:42.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:42 smithi131 ceph-mon[37260]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T22:08:42.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:42 smithi157 ceph-mon[77746]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T22:08:42.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:42 smithi132 ceph-mon[78180]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T22:08:44.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:44 smithi131 ceph-mon[37260]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T22:08:44.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:44 smithi157 ceph-mon[77746]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T22:08:44.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:44 smithi132 ceph-mon[78180]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T22:08:46.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:46 smithi131 ceph-mon[37260]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-08T22:08:46.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:46 smithi157 ceph-mon[77746]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-08T22:08:46.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:46 smithi132 ceph-mon[78180]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-08T22:08:48.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:48 smithi131 ceph-mon[37260]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T22:08:48.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:48 smithi157 ceph-mon[77746]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T22:08:48.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:48 smithi132 ceph-mon[78180]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T22:08:50.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:50 smithi131 ceph-mon[37260]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-08T22:08:50.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:50 smithi131 ceph-mon[37260]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-05-08T22:08:50.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:50 smithi157 ceph-mon[77746]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-08T22:08:50.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:50 smithi157 ceph-mon[77746]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-05-08T22:08:50.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:50 smithi132 ceph-mon[78180]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-08T22:08:50.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:50 smithi132 ceph-mon[78180]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-05-08T22:08:52.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:52 smithi131 ceph-mon[37260]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-08T22:08:52.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:52 smithi157 ceph-mon[77746]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-08T22:08:52.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:52 smithi132 ceph-mon[78180]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-08T22:08:54.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:54 smithi132 ceph-mon[78180]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-08T22:08:54.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:54 smithi131 ceph-mon[37260]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-08T22:08:55.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:54 smithi157 ceph-mon[77746]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-08T22:08:56.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:56 smithi132 ceph-mon[78180]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-08T22:08:56.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:56 smithi131 ceph-mon[37260]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-08T22:08:57.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:56 smithi157 ceph-mon[77746]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-08T22:08:58.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:08:58 smithi132 ceph-mon[78180]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-08T22:08:58.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:08:58 smithi131 ceph-mon[37260]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-08T22:08:59.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:08:58 smithi157 ceph-mon[77746]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-08T22:08:59.828 INFO:tasks.workunit.client.0.smithi131.stderr:Updating files: 63% (7790/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-08T22:09:00.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:00 smithi132 ceph-mon[78180]: pgmap v54: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-08T22:09:00.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:00 smithi131 ceph-mon[37260]: pgmap v54: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-08T22:09:00.997 INFO:tasks.workunit.client.0.smithi131.stderr:Note: switching to 'd21e4f961318b56f0d915caa2490c3711bf13c34'. 2024-05-08T22:09:00.998 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-05-08T22:09:00.998 INFO:tasks.workunit.client.0.smithi131.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-08T22:09:00.998 INFO:tasks.workunit.client.0.smithi131.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-08T22:09:00.998 INFO:tasks.workunit.client.0.smithi131.stderr:state without impacting any branches by switching back to a branch. 2024-05-08T22:09:00.998 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-05-08T22:09:00.998 INFO:tasks.workunit.client.0.smithi131.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-08T22:09:00.998 INFO:tasks.workunit.client.0.smithi131.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-08T22:09:00.998 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-05-08T22:09:00.998 INFO:tasks.workunit.client.0.smithi131.stderr: git switch -c 2024-05-08T22:09:00.998 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-05-08T22:09:00.998 INFO:tasks.workunit.client.0.smithi131.stderr:Or undo this operation with: 2024-05-08T22:09:00.998 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-05-08T22:09:00.999 INFO:tasks.workunit.client.0.smithi131.stderr: git switch - 2024-05-08T22:09:00.999 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-05-08T22:09:00.999 INFO:tasks.workunit.client.0.smithi131.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-08T22:09:00.999 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-05-08T22:09:00.999 INFO:tasks.workunit.client.0.smithi131.stderr:HEAD is now at d21e4f96131 Merge PR #57343 into wip-pdonnell-testing-20240508.150423-reef 2024-05-08T22:09:01.014 DEBUG:teuthology.orchestra.run.smithi131:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-08T22:09:01.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:00 smithi157 ceph-mon[77746]: pgmap v54: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-08T22:09:01.080 INFO:tasks.workunit.client.0.smithi131.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-08T22:09:01.085 INFO:tasks.workunit.client.0.smithi131.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-08T22:09:01.085 INFO:tasks.workunit.client.0.smithi131.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-08T22:09:01.872 INFO:tasks.workunit.client.0.smithi131.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-08T22:09:01.956 INFO:tasks.workunit.client.0.smithi131.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-08T22:09:02.019 INFO:tasks.workunit.client.0.smithi131.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-08T22:09:02.023 INFO:tasks.workunit.client.0.smithi131.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-08T22:09:02.023 INFO:tasks.workunit.client.0.smithi131.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-08T22:09:02.087 INFO:tasks.workunit.client.0.smithi131.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-08T22:09:02.094 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-08T22:09:02.094 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-08T22:09:02.136 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2024-05-08T22:09:02.137 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2024-05-08T22:09:02.138 DEBUG:teuthology.orchestra.run.smithi131:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d21e4f961318b56f0d915caa2490c3711bf13c34 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2024-05-08T22:09:02.232 INFO:tasks.workunit.client.0.smithi131.stdout:getting blogbench 2024-05-08T22:09:02.278 INFO:tasks.workunit.client.0.smithi131.stderr:--2024-05-08 22:09:02-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-05-08T22:09:02.279 INFO:tasks.workunit.client.0.smithi131.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-05-08T22:09:02.301 INFO:tasks.workunit.client.0.smithi131.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-05-08T22:09:02.321 INFO:tasks.workunit.client.0.smithi131.stderr:HTTP request sent, awaiting response... 200 OK 2024-05-08T22:09:02.322 INFO:tasks.workunit.client.0.smithi131.stderr:Length: 103355 (101K) [application/octet-stream] 2024-05-08T22:09:02.323 INFO:tasks.workunit.client.0.smithi131.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2024-05-08T22:09:02.327 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-05-08T22:09:02.364 INFO:tasks.workunit.client.0.smithi131.stderr: 0K .......... .......... .......... .......... .......... 49% 1.25M 0s 2024-05-08T22:09:02.385 INFO:tasks.workunit.client.0.smithi131.stderr: 50K .......... .......... .......... .......... .......... 99% 2.38M 0s 2024-05-08T22:09:02.385 INFO:tasks.workunit.client.0.smithi131.stderr: 100K 100% 1779G=0.06s 2024-05-08T22:09:02.385 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-05-08T22:09:02.385 INFO:tasks.workunit.client.0.smithi131.stderr:2024-05-08 22:09:02 (1.65 MB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2024-05-08T22:09:02.385 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-05-08T22:09:02.420 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/ 2024-05-08T22:09:02.422 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/m4/ 2024-05-08T22:09:02.423 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/m4/acx_pthread.m4 2024-05-08T22:09:02.424 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/man/ 2024-05-08T22:09:02.425 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/man/blogbench.8 2024-05-08T22:09:02.427 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/man/Makefile.am 2024-05-08T22:09:02.427 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/man/Makefile.in 2024-05-08T22:09:02.427 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/src/ 2024-05-08T22:09:02.429 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/src/mysnprintf.c 2024-05-08T22:09:02.432 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/src/mysnprintf.h 2024-05-08T22:09:02.432 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/src/blogbench.c 2024-05-08T22:09:02.432 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/src/blogbench.h 2024-05-08T22:09:02.432 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/src/fakesnprintf.c 2024-05-08T22:09:02.432 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/src/fakesnprintf.h 2024-05-08T22:09:02.432 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/src/blogbench_p.h 2024-05-08T22:09:02.432 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/src/bsd-getopt_long.c 2024-05-08T22:09:02.432 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/src/bsd-getopt_long.h 2024-05-08T22:09:02.433 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/src/Makefile.am 2024-05-08T22:09:02.433 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/src/Makefile.in 2024-05-08T22:09:02.433 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/src/process.c 2024-05-08T22:09:02.434 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/src/globals.h 2024-05-08T22:09:02.434 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/src/commenter.c 2024-05-08T22:09:02.434 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/src/rewriter.c 2024-05-08T22:09:02.434 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/src/reader.c 2024-05-08T22:09:02.434 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/src/writer.c 2024-05-08T22:09:02.434 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/src/helpers.c 2024-05-08T22:09:02.434 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/NEWS 2024-05-08T22:09:02.443 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/compile 2024-05-08T22:09:02.443 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/depcomp 2024-05-08T22:09:02.443 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/aclocal.m4 2024-05-08T22:09:02.444 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/README 2024-05-08T22:09:02.444 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/configure 2024-05-08T22:09:02.447 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/configure.ac 2024-05-08T22:09:02.447 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/config.guess 2024-05-08T22:09:02.448 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/install-sh 2024-05-08T22:09:02.448 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/config.sub 2024-05-08T22:09:02.448 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/missing 2024-05-08T22:09:02.448 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/mkinstalldirs 2024-05-08T22:09:02.448 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/Makefile.am 2024-05-08T22:09:02.449 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/Makefile.in 2024-05-08T22:09:02.449 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/config.h.in 2024-05-08T22:09:02.449 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/AUTHORS 2024-05-08T22:09:02.449 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/INSTALL 2024-05-08T22:09:02.449 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/ChangeLog 2024-05-08T22:09:02.449 INFO:tasks.workunit.client.0.smithi131.stdout:blogbench-1.0/COPYING 2024-05-08T22:09:02.449 INFO:tasks.workunit.client.0.smithi131.stdout:making blogbench 2024-05-08T22:09:02.707 INFO:tasks.workunit.client.0.smithi131.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-05-08T22:09:02.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:02 smithi132 ceph-mon[78180]: pgmap v55: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-08T22:09:02.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:02 smithi131 ceph-mon[37260]: pgmap v55: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-08T22:09:03.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:02 smithi157 ceph-mon[77746]: pgmap v55: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-08T22:09:03.718 INFO:tasks.workunit.client.0.smithi131.stdout:checking whether build environment is sane... yes 2024-05-08T22:09:03.737 INFO:tasks.workunit.client.0.smithi131.stdout:checking for gawk... gawk 2024-05-08T22:09:03.790 INFO:tasks.workunit.client.0.smithi131.stdout:checking whether make sets $(MAKE)... yes 2024-05-08T22:09:03.811 INFO:tasks.workunit.client.0.smithi131.stdout:checking for gcc... gcc 2024-05-08T22:09:03.904 INFO:tasks.workunit.client.0.smithi131.stdout:checking for C compiler default output file name... a.out 2024-05-08T22:09:03.909 INFO:tasks.workunit.client.0.smithi131.stdout:checking whether the C compiler works... yes 2024-05-08T22:09:03.913 INFO:tasks.workunit.client.0.smithi131.stdout:checking whether we are cross compiling... no 2024-05-08T22:09:03.974 INFO:tasks.workunit.client.0.smithi131.stdout:checking for suffix of executables... 2024-05-08T22:09:04.037 INFO:tasks.workunit.client.0.smithi131.stdout:checking for suffix of object files... o 2024-05-08T22:09:04.096 INFO:tasks.workunit.client.0.smithi131.stdout:checking whether we are using the GNU C compiler... yes 2024-05-08T22:09:04.157 INFO:tasks.workunit.client.0.smithi131.stdout:checking whether gcc accepts -g... yes 2024-05-08T22:09:04.299 INFO:tasks.workunit.client.0.smithi131.stdout:checking for gcc option to accept ANSI C... none needed 2024-05-08T22:09:04.362 INFO:tasks.workunit.client.0.smithi131.stdout:checking for style of include used by make... GNU 2024-05-08T22:09:04.487 INFO:tasks.workunit.client.0.smithi131.stdout:checking dependency style of gcc... gcc3 2024-05-08T22:09:04.487 INFO:tasks.workunit.client.0.smithi131.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-05-08T22:09:04.579 INFO:tasks.workunit.client.0.smithi131.stdout:checking how to run the C preprocessor... gcc -E 2024-05-08T22:09:04.646 INFO:tasks.workunit.client.0.smithi131.stdout:checking for egrep... grep -E 2024-05-08T22:09:04.663 INFO:tasks.workunit.client.0.smithi131.stdout:checking for AIX... no 2024-05-08T22:09:04.762 INFO:tasks.workunit.client.0.smithi131.stdout:checking for library containing strerror... none required 2024-05-08T22:09:04.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:04 smithi132 ceph-mon[78180]: pgmap v56: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-08T22:09:04.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:04 smithi131 ceph-mon[37260]: pgmap v56: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-08T22:09:05.009 INFO:tasks.workunit.client.0.smithi131.stdout:checking for ANSI C header files... yes 2024-05-08T22:09:05.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:04 smithi157 ceph-mon[77746]: pgmap v56: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-08T22:09:05.112 INFO:tasks.workunit.client.0.smithi131.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-05-08T22:09:05.195 INFO:tasks.workunit.client.0.smithi131.stdout:checking for sys/types.h... yes 2024-05-08T22:09:05.282 INFO:tasks.workunit.client.0.smithi131.stdout:checking for sys/stat.h... yes 2024-05-08T22:09:05.368 INFO:tasks.workunit.client.0.smithi131.stdout:checking for stdlib.h... yes 2024-05-08T22:09:05.456 INFO:tasks.workunit.client.0.smithi131.stdout:checking for string.h... yes 2024-05-08T22:09:05.542 INFO:tasks.workunit.client.0.smithi131.stdout:checking for memory.h... yes 2024-05-08T22:09:05.630 INFO:tasks.workunit.client.0.smithi131.stdout:checking for strings.h... yes 2024-05-08T22:09:05.719 INFO:tasks.workunit.client.0.smithi131.stdout:checking for inttypes.h... yes 2024-05-08T22:09:05.807 INFO:tasks.workunit.client.0.smithi131.stdout:checking for stdint.h... yes 2024-05-08T22:09:05.898 INFO:tasks.workunit.client.0.smithi131.stdout:checking for unistd.h... yes 2024-05-08T22:09:05.921 INFO:tasks.workunit.client.0.smithi131.stdout:checking for memory.h... (cached) yes 2024-05-08T22:09:05.936 INFO:tasks.workunit.client.0.smithi131.stdout:checking for strings.h... (cached) yes 2024-05-08T22:09:05.951 INFO:tasks.workunit.client.0.smithi131.stdout:checking for unistd.h... (cached) yes 2024-05-08T22:09:06.038 INFO:tasks.workunit.client.0.smithi131.stdout:checking locale.h usability... yes 2024-05-08T22:09:06.071 INFO:tasks.workunit.client.0.smithi131.stdout:checking locale.h presence... yes 2024-05-08T22:09:06.073 INFO:tasks.workunit.client.0.smithi131.stdout:checking for locale.h... yes 2024-05-08T22:09:06.100 INFO:tasks.workunit.client.0.smithi131.stdout:checking for stdlib.h... (cached) yes 2024-05-08T22:09:06.115 INFO:tasks.workunit.client.0.smithi131.stdout:checking for string.h... (cached) yes 2024-05-08T22:09:06.255 INFO:tasks.workunit.client.0.smithi131.stdout:checking sys/socket.h usability... yes 2024-05-08T22:09:06.288 INFO:tasks.workunit.client.0.smithi131.stdout:checking sys/socket.h presence... yes 2024-05-08T22:09:06.290 INFO:tasks.workunit.client.0.smithi131.stdout:checking for sys/socket.h... yes 2024-05-08T22:09:06.380 INFO:tasks.workunit.client.0.smithi131.stdout:checking sys/time.h usability... yes 2024-05-08T22:09:06.413 INFO:tasks.workunit.client.0.smithi131.stdout:checking sys/time.h presence... yes 2024-05-08T22:09:06.415 INFO:tasks.workunit.client.0.smithi131.stdout:checking for sys/time.h... yes 2024-05-08T22:09:06.506 INFO:tasks.workunit.client.0.smithi131.stdout:checking fcntl.h usability... yes 2024-05-08T22:09:06.538 INFO:tasks.workunit.client.0.smithi131.stdout:checking fcntl.h presence... yes 2024-05-08T22:09:06.540 INFO:tasks.workunit.client.0.smithi131.stdout:checking for fcntl.h... yes 2024-05-08T22:09:06.659 INFO:tasks.workunit.client.0.smithi131.stdout:checking sys/fcntl.h usability... yes 2024-05-08T22:09:06.691 INFO:tasks.workunit.client.0.smithi131.stdout:checking sys/fcntl.h presence... yes 2024-05-08T22:09:06.693 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:09:06.694 INFO:tasks.workunit.client.0.smithi131.stdout:checking for sys/fcntl.h... yes 2024-05-08T22:09:06.824 INFO:tasks.workunit.client.0.smithi131.stdout:checking sys/param.h usability... yes 2024-05-08T22:09:06.839 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:06 smithi132 ceph-mon[78180]: pgmap v57: 129 pgs: 129 active+clean; 1.6 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 40 KiB/s wr, 23 op/s 2024-05-08T22:09:06.859 INFO:tasks.workunit.client.0.smithi131.stdout:checking sys/param.h presence... yes 2024-05-08T22:09:06.861 INFO:tasks.workunit.client.0.smithi131.stdout:checking for sys/param.h... yes 2024-05-08T22:09:06.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:06 smithi131 ceph-mon[37260]: pgmap v57: 129 pgs: 129 active+clean; 1.6 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 40 KiB/s wr, 23 op/s 2024-05-08T22:09:06.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.895+0000 7fc88d400700 1 -- 172.21.15.131:0/4052355021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc888073c60 msgr2=0x7fc8881119e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:06.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.895+0000 7fc88d400700 1 --2- 172.21.15.131:0/4052355021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc888073c60 0x7fc8881119e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc874009990 tx=0x7fc874009ca0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.896+0000 7fc88d400700 1 -- 172.21.15.131:0/4052355021 shutdown_connections 2024-05-08T22:09:06.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.896+0000 7fc88d400700 1 --2- 172.21.15.131:0/4052355021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc888111f20 0x7fc888114310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.896+0000 7fc88d400700 1 --2- 172.21.15.131:0/4052355021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc888073340 0x7fc888073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.896+0000 7fc88d400700 1 --2- 172.21.15.131:0/4052355021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc888073c60 0x7fc8881119e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.896+0000 7fc88d400700 1 -- 172.21.15.131:0/4052355021 >> 172.21.15.131:0/4052355021 conn(0x7fc8880fc9e0 msgr2=0x7fc8880fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:06.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.896+0000 7fc88d400700 1 -- 172.21.15.131:0/4052355021 shutdown_connections 2024-05-08T22:09:06.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.896+0000 7fc88d400700 1 -- 172.21.15.131:0/4052355021 wait complete. 2024-05-08T22:09:06.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.897+0000 7fc88d400700 1 Processor -- start 2024-05-08T22:09:06.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.897+0000 7fc88d400700 1 -- start start 2024-05-08T22:09:06.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.897+0000 7fc88d400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc888073340 0x7fc8881a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:06.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.897+0000 7fc88d400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc888073c60 0x7fc8881a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:06.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.897+0000 7fc88d400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc888111f20 0x7fc88819d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:06.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.897+0000 7fc88d400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88819df40 con 0x7fc888111f20 2024-05-08T22:09:06.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.897+0000 7fc88d400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88819e0b0 con 0x7fc888073c60 2024-05-08T22:09:06.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.897+0000 7fc88d400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88819e270 con 0x7fc888073340 2024-05-08T22:09:06.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.897+0000 7fc887200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc888111f20 0x7fc88819d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:06.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.898+0000 7fc887200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc888111f20 0x7fc88819d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51706/0 (socket says 172.21.15.131:51706) 2024-05-08T22:09:06.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.898+0000 7fc887200700 1 -- 172.21.15.131:0/4570671 learned_addr learned my addr 172.21.15.131:0/4570671 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:06.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.898+0000 7fc886800700 1 --2- 172.21.15.131:0/4570671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc888073340 0x7fc8881a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:06.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.898+0000 7fc885e00700 1 --2- 172.21.15.131:0/4570671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc888073c60 0x7fc8881a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:06.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.898+0000 7fc887200700 1 -- 172.21.15.131:0/4570671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc888073340 msgr2=0x7fc8881a37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:06.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.898+0000 7fc887200700 1 --2- 172.21.15.131:0/4570671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc888073340 0x7fc8881a37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.898+0000 7fc887200700 1 -- 172.21.15.131:0/4570671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc888073c60 msgr2=0x7fc8881a3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:06.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.898+0000 7fc887200700 1 --2- 172.21.15.131:0/4570671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc888073c60 0x7fc8881a3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.898+0000 7fc887200700 1 -- 172.21.15.131:0/4570671 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8740095f0 con 0x7fc888111f20 2024-05-08T22:09:06.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.898+0000 7fc887200700 1 --2- 172.21.15.131:0/4570671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc888111f20 0x7fc88819d8e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc88000ef90 tx=0x7fc88000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:06.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.900+0000 7fc87f400700 1 -- 172.21.15.131:0/4570671 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc880018070 con 0x7fc888111f20 2024-05-08T22:09:06.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.900+0000 7fc87f400700 1 -- 172.21.15.131:0/4570671 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc88000f040 con 0x7fc888111f20 2024-05-08T22:09:06.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.900+0000 7fc88d400700 1 -- 172.21.15.131:0/4570671 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc88819e4f0 con 0x7fc888111f20 2024-05-08T22:09:06.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.900+0000 7fc87f400700 1 -- 172.21.15.131:0/4570671 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88000cda0 con 0x7fc888111f20 2024-05-08T22:09:06.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.900+0000 7fc88d400700 1 -- 172.21.15.131:0/4570671 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8881aa7a0 con 0x7fc888111f20 2024-05-08T22:09:06.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.903+0000 7fc87f400700 1 -- 172.21.15.131:0/4570671 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fc880003a00 con 0x7fc888111f20 2024-05-08T22:09:06.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.904+0000 7fc8a0e89b80 1 Processor -- start 2024-05-08T22:09:06.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.905+0000 7fc87f400700 1 --2- 172.21.15.131:0/4570671 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc86c0a1310 0x7fc86c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:06.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.905+0000 7fc8a0e89b80 1 -- start start 2024-05-08T22:09:06.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.905+0000 7fc87f400700 1 -- 172.21.15.131:0/4570671 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fc8800ccba0 con 0x7fc888111f20 2024-05-08T22:09:06.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.905+0000 7fc885e00700 1 --2- 172.21.15.131:0/4570671 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc86c0a1310 0x7fc86c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:06.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.905+0000 7fc8a0e89b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee08a31e00 0x55ee08a321e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:06.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.905+0000 7fc8a0e89b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a32720 0x55ee08a2c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:06.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.905+0000 7fc8a0e89b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee08a2c890 0x55ee08a2ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:06.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.905+0000 7fc8a0e89b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee08983150 con 0x55ee08a31e00 2024-05-08T22:09:06.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.905+0000 7fc8a0e89b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee08980b10 con 0x55ee08a32720 2024-05-08T22:09:06.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.905+0000 7fc8a0e89b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee0897c400 con 0x55ee08a2c890 2024-05-08T22:09:06.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.906+0000 7fc885e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a32720 0x55ee08a2c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:06.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.906+0000 7fc886800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee08a31e00 0x55ee08a321e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:06.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.906+0000 7fc885e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a32720 0x55ee08a2c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:38160/0 (socket says 172.21.15.131:38160) 2024-05-08T22:09:06.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.906+0000 7fc885e00700 1 -- 172.21.15.131:0/606305540 learned_addr learned my addr 172.21.15.131:0/606305540 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:06.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.906+0000 7fc887200700 1 --2- 172.21.15.131:0/606305540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee08a2c890 0x55ee08a2ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:06.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.906+0000 7fc886800700 1 -- 172.21.15.131:0/606305540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee08a2c890 msgr2=0x55ee08a2ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:06.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.906+0000 7fc886800700 1 --2- 172.21.15.131:0/606305540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee08a2c890 0x55ee08a2ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.906+0000 7fc886800700 1 -- 172.21.15.131:0/606305540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a32720 msgr2=0x55ee08a2c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:06.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.906+0000 7fc886800700 1 --2- 172.21.15.131:0/606305540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a32720 0x55ee08a2c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.906+0000 7fc886800700 1 -- 172.21.15.131:0/606305540 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee087fd3b0 con 0x55ee08a31e00 2024-05-08T22:09:06.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.906+0000 7fc887200700 1 --2- 172.21.15.131:0/606305540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee08a2c890 0x55ee08a2ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:09:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.906+0000 7fc885e00700 1 --2- 172.21.15.131:0/4570671 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc86c0a1310 0x7fc86c0a37d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc874009990 tx=0x7fc874005c20 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.906+0000 7fc886800700 1 --2- 172.21.15.131:0/606305540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee08a31e00 0x55ee08a321e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc878009c90 tx=0x7fc878011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d9f9ddafe2a48b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.907+0000 7fc873400700 1 -- 172.21.15.131:0/606305540 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc878011d20 con 0x55ee08a31e00 2024-05-08T22:09:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.907+0000 7fc873400700 1 -- 172.21.15.131:0/606305540 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc878011e80 con 0x55ee08a31e00 2024-05-08T22:09:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.907+0000 7fc873400700 1 -- 172.21.15.131:0/606305540 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc878004ea0 con 0x55ee08a31e00 2024-05-08T22:09:06.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.907+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/606305540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee08a31e00 msgr2=0x55ee08a321e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:06.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.907+0000 7fc8a0e89b80 1 --2- 172.21.15.131:0/606305540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee08a31e00 0x55ee08a321e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc878009c90 tx=0x7fc878011440 comp rx=0 tx=0).stop 2024-05-08T22:09:06.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.908+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/606305540 shutdown_connections 2024-05-08T22:09:06.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.908+0000 7fc8a0e89b80 1 --2- 172.21.15.131:0/606305540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee08a2c890 0x55ee08a2ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.908+0000 7fc8a0e89b80 1 --2- 172.21.15.131:0/606305540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee08a31e00 0x55ee08a321e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.908+0000 7fc8a0e89b80 1 --2- 172.21.15.131:0/606305540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a32720 0x55ee08a2c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.908+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/606305540 >> 172.21.15.131:0/606305540 conn(0x55ee0898a210 msgr2=0x55ee08a26df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:06.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.908+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/606305540 shutdown_connections 2024-05-08T22:09:06.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.908+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/606305540 wait complete. 2024-05-08T22:09:06.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.908+0000 7fc8a0e89b80 1 -- start start 2024-05-08T22:09:06.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.908+0000 7fc8a0e89b80 1 Processor -- start 2024-05-08T22:09:06.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.909+0000 7fc8a0e89b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a2c890 0x55ee08a38700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:06.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.909+0000 7fc8a0e89b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee08a31e00 0x55ee08a359f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:06.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.909+0000 7fc8a0e89b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee08a32720 0x55ee08a360e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:06.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.909+0000 7fc8a0e89b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee08918560 con 0x55ee08a31e00 2024-05-08T22:09:06.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.909+0000 7fc8a0e89b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee08918200 con 0x55ee08a2c890 2024-05-08T22:09:06.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.909+0000 7fc8a0e89b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee087ff6a0 con 0x55ee08a32720 2024-05-08T22:09:06.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.909+0000 7fc8a0e89b80 10 client.0 _ll_register_callbacks cb 0x55ee08921b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:06.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.909+0000 7fc886800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a2c890 0x55ee08a38700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:06.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.909+0000 7fc886800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a2c890 0x55ee08a38700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:38176/0 (socket says 172.21.15.131:38176) 2024-05-08T22:09:06.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.909+0000 7fc886800700 1 -- 172.21.15.131:0/2838670793 learned_addr learned my addr 172.21.15.131:0/2838670793 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:06.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.909+0000 7fc885e00700 1 --2- 172.21.15.131:0/2838670793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee08a31e00 0x55ee08a359f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:06.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.909+0000 7fc887200700 1 --2- 172.21.15.131:0/2838670793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee08a32720 0x55ee08a360e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:06.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.909+0000 7fc886800700 1 -- 172.21.15.131:0/2838670793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee08a32720 msgr2=0x55ee08a360e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:06.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.909+0000 7fc886800700 1 --2- 172.21.15.131:0/2838670793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee08a32720 0x55ee08a360e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.909+0000 7fc886800700 1 -- 172.21.15.131:0/2838670793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee08a31e00 msgr2=0x55ee08a359f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:06.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.909+0000 7fc886800700 1 --2- 172.21.15.131:0/2838670793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee08a31e00 0x55ee08a359f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.909+0000 7fc886800700 1 -- 172.21.15.131:0/2838670793 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8780098c0 con 0x55ee08a2c890 2024-05-08T22:09:06.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.909+0000 7fc885e00700 1 --2- 172.21.15.131:0/2838670793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee08a31e00 0x55ee08a359f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:09:06.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.910+0000 7fc886800700 1 --2- 172.21.15.131:0/2838670793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a2c890 0x55ee08a38700 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc878009770 tx=0x7fc878004960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31d262700f307ac7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:06.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.910+0000 7fc867e00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:09:06.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.910+0000 7fc867e00700 1 -- 172.21.15.131:0/2838670793 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8780042e0 con 0x55ee08a2c890 2024-05-08T22:09:06.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.910+0000 7fc867e00700 1 -- 172.21.15.131:0/2838670793 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fc878004440 con 0x55ee08a2c890 2024-05-08T22:09:06.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.910+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/2838670793 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee089fb810 con 0x55ee08a2c890 2024-05-08T22:09:06.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.910+0000 7fc867e00700 1 -- 172.21.15.131:0/2838670793 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc878003680 con 0x55ee08a2c890 2024-05-08T22:09:06.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.910+0000 7fc8a0e89b80 10 client.25886 fetch_fsmap learned FSMap version 24 2024-05-08T22:09:06.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.910+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/2838670793 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ee087a9b60 con 0x55ee08a2c890 2024-05-08T22:09:06.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.910+0000 7fc867e00700 1 -- 172.21.15.131:0/2838670793 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ee089fb810 con 0x55ee08a2c890 2024-05-08T22:09:06.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.911+0000 7fc867e00700 1 -- 172.21.15.131:0/2838670793 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc8780051e0 con 0x55ee08a2c890 2024-05-08T22:09:06.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.911+0000 7fc8a0e89b80 10 client.25886 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:09:06.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.911+0000 7fc8a0e89b80 10 client.25886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:06.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.911+0000 7fc8a0e89b80 20 client.25886 populate_metadata read hostname 'smithi131' 2024-05-08T22:09:06.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.911+0000 7fc8a0e89b80 1 --2- 172.21.15.131:0/2838670793 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ee08a46040 0x55ee08a48430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:06.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.911+0000 7fc8a0e89b80 4 client.25886 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:09:06.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.911+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/2838670793 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee08974ec0 con 0x55ee08a46040 2024-05-08T22:09:06.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.911+0000 7fc885e00700 1 --2- 172.21.15.131:0/2838670793 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ee08a46040 0x55ee08a48430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:06.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.912+0000 7fc885e00700 1 --2- 172.21.15.131:0/2838670793 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ee08a46040 0x55ee08a48430 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b18af99befe9be66 server_cookie=20647868c6a8ec3b in_seq=0 out_seq=0 2024-05-08T22:09:06.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.912+0000 7fc867e00700 10 client.25886 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:06.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.912+0000 7fc867e00700 1 -- 172.21.15.131:0/2838670793 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ee08974ec0 con 0x55ee08a46040 2024-05-08T22:09:06.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.912+0000 7fc867e00700 10 client.25886 handle_command_reply: tid=0 2024-05-08T22:09:06.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.912+0000 7fc8a0e89b80 1 client.25886 shutdown 2024-05-08T22:09:06.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.912+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/2838670793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a2c890 msgr2=0x55ee08a38700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:06.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.912+0000 7fc8a0e89b80 1 --2- 172.21.15.131:0/2838670793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a2c890 0x55ee08a38700 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc878009770 tx=0x7fc878004960 comp rx=0 tx=0).stop 2024-05-08T22:09:06.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.912+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/2838670793 shutdown_connections 2024-05-08T22:09:06.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.912+0000 7fc8a0e89b80 1 --2- 172.21.15.131:0/2838670793 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ee08a46040 0x55ee08a48430 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.913+0000 7fc8a0e89b80 1 --2- 172.21.15.131:0/2838670793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee08a32720 0x55ee08a360e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.913+0000 7fc867e00700 0 client.25886 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:06.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.913+0000 7fc8a0e89b80 1 --2- 172.21.15.131:0/2838670793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a2c890 0x55ee08a38700 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.913+0000 7fc8a0e89b80 1 --2- 172.21.15.131:0/2838670793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee08a31e00 0x55ee08a359f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.913+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/2838670793 >> 172.21.15.131:0/2838670793 conn(0x55ee0898a210 msgr2=0x55ee08a265a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:06.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.913+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/2838670793 shutdown_connections 2024-05-08T22:09:06.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.913+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/2838670793 wait complete. 2024-05-08T22:09:06.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.913+0000 7fc8a0e89b80 20 client.25886 trim_cache size 0 max 16384 2024-05-08T22:09:06.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.944+0000 7fc8a0e89b80 1 Processor -- start 2024-05-08T22:09:06.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.944+0000 7fc8a0e89b80 1 -- start start 2024-05-08T22:09:06.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.944+0000 7fc8a0e89b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a2c890 0x55ee08a26d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:06.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.944+0000 7fc8a0e89b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee08a31e00 0x55ee08a35280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:06.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.944+0000 7fc8a0e89b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee08a32720 0x55ee08a357c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:06.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.944+0000 7fc8a0e89b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee08a366e0 con 0x55ee08a32720 2024-05-08T22:09:06.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.944+0000 7fc8a0e89b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee08a36fc0 con 0x55ee08a2c890 2024-05-08T22:09:06.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.944+0000 7fc8a0e89b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee08a45e20 con 0x55ee08a31e00 2024-05-08T22:09:06.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.945+0000 7fc886800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a2c890 0x55ee08a26d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:06.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.945+0000 7fc886800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a2c890 0x55ee08a26d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:38192/0 (socket says 172.21.15.131:38192) 2024-05-08T22:09:06.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.945+0000 7fc886800700 1 -- 172.21.15.131:0/3520122613 learned_addr learned my addr 172.21.15.131:0/3520122613 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:06.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.945+0000 7fc887200700 1 --2- 172.21.15.131:0/3520122613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee08a32720 0x55ee08a357c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:06.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.945+0000 7fc885e00700 1 --2- 172.21.15.131:0/3520122613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee08a31e00 0x55ee08a35280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:06.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.945+0000 7fc886800700 1 -- 172.21.15.131:0/3520122613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee08a31e00 msgr2=0x55ee08a35280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:06.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.945+0000 7fc886800700 1 --2- 172.21.15.131:0/3520122613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee08a31e00 0x55ee08a35280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.945+0000 7fc886800700 1 -- 172.21.15.131:0/3520122613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee08a32720 msgr2=0x55ee08a357c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:06.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.945+0000 7fc886800700 1 --2- 172.21.15.131:0/3520122613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee08a32720 0x55ee08a357c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.945+0000 7fc886800700 1 -- 172.21.15.131:0/3520122613 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee08942560 con 0x55ee08a2c890 2024-05-08T22:09:06.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.945+0000 7fc885e00700 1 --2- 172.21.15.131:0/3520122613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee08a31e00 0x55ee08a35280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:09:06.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.945+0000 7fc887200700 1 --2- 172.21.15.131:0/3520122613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee08a32720 0x55ee08a357c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:09:06.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.945+0000 7fc886800700 1 --2- 172.21.15.131:0/3520122613 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a2c890 0x55ee08a26d20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc878001eb0 tx=0x7fc878011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9bfd2cefa563c45 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:06.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.946+0000 7fc873e00700 1 -- 172.21.15.131:0/3520122613 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc878015030 con 0x55ee08a2c890 2024-05-08T22:09:06.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.946+0000 7fc873e00700 1 -- 172.21.15.131:0/3520122613 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc878022400 con 0x55ee08a2c890 2024-05-08T22:09:06.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.946+0000 7fc873e00700 1 -- 172.21.15.131:0/3520122613 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8780038e0 con 0x55ee08a2c890 2024-05-08T22:09:06.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.946+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/3520122613 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a2c890 msgr2=0x55ee08a26d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:06.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.946+0000 7fc8a0e89b80 1 --2- 172.21.15.131:0/3520122613 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a2c890 0x55ee08a26d20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc878001eb0 tx=0x7fc878011440 comp rx=0 tx=0).stop 2024-05-08T22:09:06.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.946+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/3520122613 shutdown_connections 2024-05-08T22:09:06.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.946+0000 7fc8a0e89b80 1 --2- 172.21.15.131:0/3520122613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee08a32720 0x55ee08a357c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.946+0000 7fc8a0e89b80 1 --2- 172.21.15.131:0/3520122613 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a2c890 0x55ee08a26d20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.946+0000 7fc8a0e89b80 1 --2- 172.21.15.131:0/3520122613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee08a31e00 0x55ee08a35280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.946+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/3520122613 >> 172.21.15.131:0/3520122613 conn(0x55ee0898a210 msgr2=0x55ee08a41c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:06.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.947+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/3520122613 shutdown_connections 2024-05-08T22:09:06.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.947+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/3520122613 wait complete. 2024-05-08T22:09:06.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.947+0000 7fc8a0e89b80 1 -- start start 2024-05-08T22:09:06.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.947+0000 7fc8a0e89b80 1 Processor -- start 2024-05-08T22:09:06.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.948+0000 7fc8a0e89b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee08a2c890 0x55ee08a3d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:06.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.948+0000 7fc8a0e89b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee08a31e00 0x55ee08a3d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:06.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.948+0000 7fc8a0e89b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a32720 0x55ee08a2d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:06.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.948+0000 7fc8a0e89b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee08a2dda0 con 0x55ee08a31e00 2024-05-08T22:09:06.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.948+0000 7fc8a0e89b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee08a2df10 con 0x55ee08a32720 2024-05-08T22:09:06.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.948+0000 7fc8a0e89b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee08a2e0d0 con 0x55ee08a2c890 2024-05-08T22:09:06.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.948+0000 7fc8a0e89b80 10 client.0 _ll_register_callbacks cb 0x55ee08921b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:06.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.948+0000 7fc886800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee08a2c890 0x55ee08a3d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:06.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.948+0000 7fc885e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee08a31e00 0x55ee08a3d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:06.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.948+0000 7fc886800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee08a2c890 0x55ee08a3d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60568/0 (socket says 172.21.15.131:60568) 2024-05-08T22:09:06.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.948+0000 7fc886800700 1 -- 172.21.15.131:0/3544801053 learned_addr learned my addr 172.21.15.131:0/3544801053 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:06.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.948+0000 7fc887200700 1 --2- 172.21.15.131:0/3544801053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a32720 0x55ee08a2d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:06.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.948+0000 7fc886800700 1 -- 172.21.15.131:0/3544801053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a32720 msgr2=0x55ee08a2d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.948+0000 7fc886800700 1 --2- 172.21.15.131:0/3544801053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a32720 0x55ee08a2d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.948+0000 7fc886800700 1 -- 172.21.15.131:0/3544801053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee08a31e00 msgr2=0x55ee08a3d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.948+0000 7fc886800700 1 --2- 172.21.15.131:0/3544801053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee08a31e00 0x55ee08a3d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.948+0000 7fc886800700 1 -- 172.21.15.131:0/3544801053 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee087a9b60 con 0x55ee08a2c890 2024-05-08T22:09:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.948+0000 7fc885e00700 1 --2- 172.21.15.131:0/3544801053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee08a31e00 0x55ee08a3d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:09:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.948+0000 7fc887200700 1 --2- 172.21.15.131:0/3544801053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a32720 0x55ee08a2d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:09:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.949+0000 7fc886800700 1 --2- 172.21.15.131:0/3544801053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee08a2c890 0x55ee08a3d170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc878016040 tx=0x7fc878004590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35684ee3d087bdf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.949+0000 7fc867e00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:09:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.949+0000 7fc867e00700 1 -- 172.21.15.131:0/3544801053 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8780228d0 con 0x55ee08a2c890 2024-05-08T22:09:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.949+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/3544801053 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee088d5270 con 0x55ee08a2c890 2024-05-08T22:09:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.949+0000 7fc867e00700 1 -- 172.21.15.131:0/3544801053 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fc878000e40 con 0x55ee08a2c890 2024-05-08T22:09:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.949+0000 7fc867e00700 1 -- 172.21.15.131:0/3544801053 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc878022a30 con 0x55ee08a2c890 2024-05-08T22:09:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.949+0000 7fc8a0e89b80 10 client.25906 fetch_fsmap learned FSMap version 24 2024-05-08T22:09:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.949+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/3544801053 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ee08801cd0 con 0x55ee08a2c890 2024-05-08T22:09:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.950+0000 7fc867e00700 1 -- 172.21.15.131:0/3544801053 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ee088d5270 con 0x55ee08a2c890 2024-05-08T22:09:06.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.950+0000 7fc867e00700 1 -- 172.21.15.131:0/3544801053 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc878004bd0 con 0x55ee08a2c890 2024-05-08T22:09:06.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.950+0000 7fc8a0e89b80 10 client.25906 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:09:06.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.950+0000 7fc8a0e89b80 10 client.25906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:06.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.950+0000 7fc8a0e89b80 20 client.25906 populate_metadata read hostname 'smithi131' 2024-05-08T22:09:06.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.950+0000 7fc8a0e89b80 1 --2- 172.21.15.131:0/3544801053 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ee08a9cf60 0x55ee08a9f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:06.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.950+0000 7fc8a0e89b80 4 client.25906 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:09:06.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.950+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/3544801053 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ee087cacf0 con 0x55ee08a9cf60 2024-05-08T22:09:06.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.950+0000 7fc885e00700 1 --2- 172.21.15.131:0/3544801053 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ee08a9cf60 0x55ee08a9f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:06.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.951+0000 7fc885e00700 1 --2- 172.21.15.131:0/3544801053 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ee08a9cf60 0x55ee08a9f350 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdc52d8e61527a40 server_cookie=31f5491d7551f192 in_seq=0 out_seq=0 2024-05-08T22:09:06.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.951+0000 7fc867e00700 10 client.25906 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:06.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.951+0000 7fc867e00700 1 -- 172.21.15.131:0/3544801053 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ee087cacf0 con 0x55ee08a9cf60 2024-05-08T22:09:06.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.951+0000 7fc867e00700 10 client.25906 handle_command_reply: tid=0 2024-05-08T22:09:06.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.951+0000 7fc8a0e89b80 1 client.25906 shutdown 2024-05-08T22:09:06.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.951+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/3544801053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee08a2c890 msgr2=0x55ee08a3d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:06.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.951+0000 7fc8a0e89b80 1 --2- 172.21.15.131:0/3544801053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee08a2c890 0x55ee08a3d170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc878016040 tx=0x7fc878004590 comp rx=0 tx=0).stop 2024-05-08T22:09:06.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.951+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/3544801053 shutdown_connections 2024-05-08T22:09:06.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.951+0000 7fc8a0e89b80 1 --2- 172.21.15.131:0/3544801053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee08a32720 0x55ee08a2d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.951+0000 7fc8a0e89b80 1 --2- 172.21.15.131:0/3544801053 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ee08a9cf60 0x55ee08a9f350 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.951+0000 7fc8a0e89b80 1 --2- 172.21.15.131:0/3544801053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee08a2c890 0x55ee08a3d170 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.951+0000 7fc8a0e89b80 1 --2- 172.21.15.131:0/3544801053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ee08a31e00 0x55ee08a3d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.951+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/3544801053 >> 172.21.15.131:0/3544801053 conn(0x55ee0898a210 msgr2=0x55ee08a2b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:06.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.951+0000 7fc867e00700 0 client.25906 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:06.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.951+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/3544801053 shutdown_connections 2024-05-08T22:09:06.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.952+0000 7fc8a0e89b80 1 -- 172.21.15.131:0/3544801053 wait complete. 2024-05-08T22:09:06.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.952+0000 7fc8a0e89b80 20 client.25906 trim_cache size 0 max 16384 2024-05-08T22:09:06.956 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:09:06.957 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-08T22:09:06.957 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-08T22:09:06.957 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:09:06.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.952+0000 7fc88d400700 1 -- 172.21.15.131:0/4570671 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc86c0a1310 msgr2=0x7fc86c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:06.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.952+0000 7fc88d400700 1 --2- 172.21.15.131:0/4570671 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc86c0a1310 0x7fc86c0a37d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc874009990 tx=0x7fc874005c20 comp rx=0 tx=0).stop 2024-05-08T22:09:06.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.952+0000 7fc88d400700 1 -- 172.21.15.131:0/4570671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc888111f20 msgr2=0x7fc88819d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:06.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.952+0000 7fc88d400700 1 --2- 172.21.15.131:0/4570671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc888111f20 0x7fc88819d8e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc88000ef90 tx=0x7fc88000c3a0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.952+0000 7fc88d400700 1 -- 172.21.15.131:0/4570671 shutdown_connections 2024-05-08T22:09:06.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.952+0000 7fc88d400700 1 --2- 172.21.15.131:0/4570671 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc86c0a1310 0x7fc86c0a37d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.952+0000 7fc88d400700 1 --2- 172.21.15.131:0/4570671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc888111f20 0x7fc88819d8e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.952+0000 7fc88d400700 1 --2- 172.21.15.131:0/4570671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc888073340 0x7fc8881a37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.952+0000 7fc88d400700 1 --2- 172.21.15.131:0/4570671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc888073c60 0x7fc8881a3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:06.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.952+0000 7fc88d400700 1 -- 172.21.15.131:0/4570671 >> 172.21.15.131:0/4570671 conn(0x7fc8880fc9e0 msgr2=0x7fc8880fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:06.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.953+0000 7fc88d400700 1 -- 172.21.15.131:0/4570671 shutdown_connections 2024-05-08T22:09:06.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:06.953+0000 7fc88d400700 1 -- 172.21.15.131:0/4570671 wait complete. 2024-05-08T22:09:06.960 INFO:tasks.workunit.client.0.smithi131.stdout:checking ioctl.h usability... no 2024-05-08T22:09:06.972 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T22:09:06.973 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:09:06.993 INFO:tasks.workunit.client.0.smithi131.stdout:checking ioctl.h presence... no 2024-05-08T22:09:06.997 INFO:tasks.workunit.client.0.smithi131.stdout:checking for ioctl.h... no 2024-05-08T22:09:07.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:06 smithi157 ceph-mon[77746]: pgmap v57: 129 pgs: 129 active+clean; 1.6 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 40 KiB/s wr, 23 op/s 2024-05-08T22:09:07.083 INFO:tasks.workunit.client.0.smithi131.stdout:checking sys/ioctl.h usability... yes 2024-05-08T22:09:07.114 INFO:tasks.workunit.client.0.smithi131.stdout:checking sys/ioctl.h presence... yes 2024-05-08T22:09:07.117 INFO:tasks.workunit.client.0.smithi131.stdout:checking for sys/ioctl.h... yes 2024-05-08T22:09:07.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.194+0000 7f2f48e00700 1 -- 172.21.15.131:0/3997141600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f44101050 msgr2=0x7f2f44101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:07.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.194+0000 7f2f48e00700 1 --2- 172.21.15.131:0/3997141600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f44101050 0x7f2f44101430 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009990 tx=0x7f2f34009ca0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.195+0000 7f2f48e00700 1 -- 172.21.15.131:0/3997141600 shutdown_connections 2024-05-08T22:09:07.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.195+0000 7f2f48e00700 1 --2- 172.21.15.131:0/3997141600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f44106170 0x7f2f4410dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.195+0000 7f2f48e00700 1 --2- 172.21.15.131:0/3997141600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f44101050 0x7f2f44101430 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.195+0000 7f2f48e00700 1 --2- 172.21.15.131:0/3997141600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f44101a00 0x7f2f44105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.195+0000 7f2f48e00700 1 -- 172.21.15.131:0/3997141600 >> 172.21.15.131:0/3997141600 conn(0x7f2f440fc8f0 msgr2=0x7f2f440fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:07.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.195+0000 7f2f48e00700 1 -- 172.21.15.131:0/3997141600 shutdown_connections 2024-05-08T22:09:07.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.195+0000 7f2f48e00700 1 -- 172.21.15.131:0/3997141600 wait complete. 2024-05-08T22:09:07.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.196+0000 7f2f48e00700 1 Processor -- start 2024-05-08T22:09:07.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.196+0000 7f2f48e00700 1 -- start start 2024-05-08T22:09:07.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.197+0000 7f2f48e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f44101050 0x7f2f44197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.197+0000 7f2f48e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f44106170 0x7f2f44197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.197+0000 7f2f48e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4419bbb0 0x7f2f4419c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.197+0000 7f2f48e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4419c630 con 0x7f2f44106170 2024-05-08T22:09:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.197+0000 7f2f48e00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4419c7a0 con 0x7f2f4419bbb0 2024-05-08T22:09:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.197+0000 7f2f48e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4419c960 con 0x7f2f44101050 2024-05-08T22:09:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.197+0000 7f2f41e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f44101050 0x7f2f44197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.197+0000 7f2f41e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f44101050 0x7f2f44197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60582/0 (socket says 172.21.15.131:60582) 2024-05-08T22:09:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.197+0000 7f2f41e00700 1 -- 172.21.15.131:0/543275021 learned_addr learned my addr 172.21.15.131:0/543275021 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.197+0000 7f2f41400700 1 --2- 172.21.15.131:0/543275021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f44106170 0x7f2f44197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.197+0000 7f2f42800700 1 --2- 172.21.15.131:0/543275021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4419bbb0 0x7f2f4419c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.198+0000 7f2f41e00700 1 -- 172.21.15.131:0/543275021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4419bbb0 msgr2=0x7f2f4419c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.198+0000 7f2f41e00700 1 --2- 172.21.15.131:0/543275021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4419bbb0 0x7f2f4419c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.198+0000 7f2f41e00700 1 -- 172.21.15.131:0/543275021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f44106170 msgr2=0x7f2f44197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.198+0000 7f2f41e00700 1 --2- 172.21.15.131:0/543275021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f44106170 0x7f2f44197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.198+0000 7f2f41e00700 1 -- 172.21.15.131:0/543275021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f340095f0 con 0x7f2f44101050 2024-05-08T22:09:07.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.198+0000 7f2f41400700 1 --2- 172.21.15.131:0/543275021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f44106170 0x7f2f44197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:09:07.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.198+0000 7f2f41e00700 1 --2- 172.21.15.131:0/543275021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f44101050 0x7f2f44197320 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2f34005910 tx=0x7f2f34004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:07.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.198+0000 7f2f3aa00700 1 -- 172.21.15.131:0/543275021 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3401d070 con 0x7f2f44101050 2024-05-08T22:09:07.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.198+0000 7f2f3aa00700 1 -- 172.21.15.131:0/543275021 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f34022470 con 0x7f2f44101050 2024-05-08T22:09:07.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.198+0000 7f2f48e00700 1 -- 172.21.15.131:0/543275021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f441a3e60 con 0x7f2f44101050 2024-05-08T22:09:07.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.198+0000 7f2f3aa00700 1 -- 172.21.15.131:0/543275021 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34017650 con 0x7f2f44101050 2024-05-08T22:09:07.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.198+0000 7f2f48e00700 1 -- 172.21.15.131:0/543275021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f441a4280 con 0x7f2f44101050 2024-05-08T22:09:07.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.199+0000 7f2f48e00700 1 -- 172.21.15.131:0/543275021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f44109390 con 0x7f2f44101050 2024-05-08T22:09:07.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.202+0000 7f2f3aa00700 1 -- 172.21.15.131:0/543275021 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f2f340225e0 con 0x7f2f44101050 2024-05-08T22:09:07.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.203+0000 7f2f3aa00700 1 --2- 172.21.15.131:0/543275021 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f2f280a1290 0x7f2f280a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:07.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.203+0000 7f2f3aa00700 1 -- 172.21.15.131:0/543275021 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f2f340ce650 con 0x7f2f44101050 2024-05-08T22:09:07.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.203+0000 7f2f41400700 1 --2- 172.21.15.131:0/543275021 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f2f280a1290 0x7f2f280a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:07.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.204+0000 7f2f41400700 1 --2- 172.21.15.131:0/543275021 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f2f280a1290 0x7f2f280a3750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2f44198820 tx=0x7f2f30009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:07.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.205+0000 7f2f3aa00700 1 -- 172.21.15.131:0/543275021 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2f34097740 con 0x7f2f44101050 2024-05-08T22:09:07.209 INFO:tasks.workunit.client.0.smithi131.stdout:checking return type of signal handlers... void 2024-05-08T22:09:07.290 INFO:tasks.workunit.client.0.smithi131.stdout:checking for size_t... yes 2024-05-08T22:09:07.360 INFO:tasks.workunit.client.0.smithi131.stdout:checking for dev_t... yes 2024-05-08T22:09:07.438 INFO:tasks.workunit.client.0.smithi131.stdout:checking for short... yes 2024-05-08T22:09:07.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.493+0000 7f2f48e00700 1 -- 172.21.15.131:0/543275021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f44198670 con 0x7f2f44101050 2024-05-08T22:09:07.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.494+0000 7f2f3aa00700 1 -- 172.21.15.131:0/543275021 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10889 (secure 0 0 0) 0x7f2f3409ad60 con 0x7f2f44101050 2024-05-08T22:09:07.495 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:09:07.495 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:05:14.034908+0000","modified":"2024-05-08T22:07:19.165885+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:09:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.497+0000 7f2f48e00700 1 -- 172.21.15.131:0/543275021 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f2f280a1290 msgr2=0x7f2f280a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.497+0000 7f2f48e00700 1 --2- 172.21.15.131:0/543275021 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f2f280a1290 0x7f2f280a3750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2f44198820 tx=0x7f2f30009260 comp rx=0 tx=0).stop 2024-05-08T22:09:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.497+0000 7f2f48e00700 1 -- 172.21.15.131:0/543275021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f44101050 msgr2=0x7f2f44197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.497+0000 7f2f48e00700 1 --2- 172.21.15.131:0/543275021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f44101050 0x7f2f44197320 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2f34005910 tx=0x7f2f34004730 comp rx=0 tx=0).stop 2024-05-08T22:09:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.498+0000 7f2f48e00700 1 -- 172.21.15.131:0/543275021 shutdown_connections 2024-05-08T22:09:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.498+0000 7f2f48e00700 1 --2- 172.21.15.131:0/543275021 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f2f280a1290 0x7f2f280a3750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.498+0000 7f2f48e00700 1 --2- 172.21.15.131:0/543275021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4419bbb0 0x7f2f4419c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.498+0000 7f2f48e00700 1 --2- 172.21.15.131:0/543275021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f44101050 0x7f2f44197320 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.498+0000 7f2f48e00700 1 --2- 172.21.15.131:0/543275021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f44106170 0x7f2f44197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.498+0000 7f2f48e00700 1 -- 172.21.15.131:0/543275021 >> 172.21.15.131:0/543275021 conn(0x7f2f440fc8f0 msgr2=0x7f2f440fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.498+0000 7f2f48e00700 1 -- 172.21.15.131:0/543275021 shutdown_connections 2024-05-08T22:09:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.498+0000 7f2f48e00700 1 -- 172.21.15.131:0/543275021 wait complete. 2024-05-08T22:09:07.500 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 24 2024-05-08T22:09:07.520 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T22:09:07.554 INFO:tasks.workunit.client.0.smithi131.stdout:checking size of short... 2 2024-05-08T22:09:07.669 INFO:tasks.workunit.client.0.smithi131.stdout:checking for int... yes 2024-05-08T22:09:07.778 INFO:tasks.workunit.client.0.smithi131.stdout:checking size of int... 4 2024-05-08T22:09:07.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.795+0000 7f40a8400700 1 -- 172.21.15.131:0/2806533598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40a0101180 msgr2=0x7f40a0101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:07.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.795+0000 7f40a8400700 1 --2- 172.21.15.131:0/2806533598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40a0101180 0x7f40a0101560 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4094009990 tx=0x7f4094009ca0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.795+0000 7f40a8400700 1 -- 172.21.15.131:0/2806533598 shutdown_connections 2024-05-08T22:09:07.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.795+0000 7f40a8400700 1 --2- 172.21.15.131:0/2806533598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40a01062a0 0x7f40a010de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.795+0000 7f40a8400700 1 --2- 172.21.15.131:0/2806533598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40a0101180 0x7f40a0101560 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.795+0000 7f40a8400700 1 --2- 172.21.15.131:0/2806533598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40a0101b30 0x7f40a0105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.795+0000 7f40a8400700 1 -- 172.21.15.131:0/2806533598 >> 172.21.15.131:0/2806533598 conn(0x7f40a00fc9e0 msgr2=0x7f40a00fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:07.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.796+0000 7f40a8400700 1 -- 172.21.15.131:0/2806533598 shutdown_connections 2024-05-08T22:09:07.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.796+0000 7f40a8400700 1 -- 172.21.15.131:0/2806533598 wait complete. 2024-05-08T22:09:07.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.797+0000 7f40a8400700 1 Processor -- start 2024-05-08T22:09:07.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.797+0000 7f40a8400700 1 -- start start 2024-05-08T22:09:07.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.797+0000 7f40a8400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40a0101180 0x7f40a00732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:07.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.797+0000 7f40a8400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40a01062a0 0x7f40a00737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:07.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.797+0000 7f40a8400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40a0073d20 0x7f40a01a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:07.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.797+0000 7f40a8400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a01a5bd0 con 0x7f40a01062a0 2024-05-08T22:09:07.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.797+0000 7f40a8400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a01a5d40 con 0x7f40a0073d20 2024-05-08T22:09:07.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.797+0000 7f40a8400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a01a5f00 con 0x7f40a0101180 2024-05-08T22:09:07.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.797+0000 7f40a5a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40a0101180 0x7f40a00732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:07.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.797+0000 7f40a5a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40a0101180 0x7f40a00732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60598/0 (socket says 172.21.15.131:60598) 2024-05-08T22:09:07.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.797+0000 7f40a5a00700 1 -- 172.21.15.131:0/2688980610 learned_addr learned my addr 172.21.15.131:0/2688980610 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:07.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.797+0000 7f40a6400700 1 --2- 172.21.15.131:0/2688980610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40a0073d20 0x7f40a01a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:07.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.797+0000 7f40a5000700 1 --2- 172.21.15.131:0/2688980610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40a01062a0 0x7f40a00737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:07.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.798+0000 7f40a5a00700 1 -- 172.21.15.131:0/2688980610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40a0073d20 msgr2=0x7f40a01a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:07.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.798+0000 7f40a5a00700 1 --2- 172.21.15.131:0/2688980610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40a0073d20 0x7f40a01a5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.798+0000 7f40a5a00700 1 -- 172.21.15.131:0/2688980610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40a01062a0 msgr2=0x7f40a00737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:07.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.798+0000 7f40a5a00700 1 --2- 172.21.15.131:0/2688980610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40a01062a0 0x7f40a00737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.798+0000 7f40a5a00700 1 -- 172.21.15.131:0/2688980610 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40940095f0 con 0x7f40a0101180 2024-05-08T22:09:07.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.798+0000 7f40a5a00700 1 --2- 172.21.15.131:0/2688980610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40a0101180 0x7f40a00732a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4094000c00 tx=0x7f4094005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:07.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.798+0000 7f409e000700 1 -- 172.21.15.131:0/2688980610 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409401d070 con 0x7f40a0101180 2024-05-08T22:09:07.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.798+0000 7f409e000700 1 -- 172.21.15.131:0/2688980610 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4094022470 con 0x7f40a0101180 2024-05-08T22:09:07.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.798+0000 7f40a8400700 1 -- 172.21.15.131:0/2688980610 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a01a6180 con 0x7f40a0101180 2024-05-08T22:09:07.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.798+0000 7f409e000700 1 -- 172.21.15.131:0/2688980610 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4094017650 con 0x7f40a0101180 2024-05-08T22:09:07.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.798+0000 7f40a8400700 1 -- 172.21.15.131:0/2688980610 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a01a65a0 con 0x7f40a0101180 2024-05-08T22:09:07.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.803+0000 7f409e000700 1 -- 172.21.15.131:0/2688980610 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f40940225e0 con 0x7f40a0101180 2024-05-08T22:09:07.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.804+0000 7f40bbffab80 1 Processor -- start 2024-05-08T22:09:07.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.804+0000 7f40bbffab80 1 -- start start 2024-05-08T22:09:07.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.804+0000 7f40bbffab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721bbe00 0x557e721bc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:07.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.804+0000 7f40bbffab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557e721bc720 0x557e721b6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:07.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.804+0000 7f40bbffab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e721b6890 0x557e721b6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:07.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.804+0000 7f40bbffab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7210d150 con 0x557e721bc720 2024-05-08T22:09:07.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.804+0000 7f40bbffab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7210ab10 con 0x557e721b6890 2024-05-08T22:09:07.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.804+0000 7f40bbffab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e72106400 con 0x557e721bbe00 2024-05-08T22:09:07.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.804+0000 7f409e000700 1 --2- 172.21.15.131:0/2688980610 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f40880a1290 0x7f40880a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:07.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.805+0000 7f40a5a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557e721bc720 0x557e721b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:07.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.805+0000 7f40a6400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721bbe00 0x557e721bc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:07.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.805+0000 7f409e000700 1 -- 172.21.15.131:0/2688980610 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f40940ce2e0 con 0x7f40a0101180 2024-05-08T22:09:07.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.805+0000 7f40a6400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721bbe00 0x557e721bc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60610/0 (socket says 172.21.15.131:60610) 2024-05-08T22:09:07.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.805+0000 7f40a6400700 1 -- 172.21.15.131:0/3576031256 learned_addr learned my addr 172.21.15.131:0/3576031256 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:07.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.805+0000 7f40a5000700 1 --2- 172.21.15.131:0/3576031256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e721b6890 0x557e721b6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:07.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.805+0000 7f40a6400700 1 --2- 172.21.15.131:0/2688980610 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f40880a1290 0x7f40880a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:07.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.805+0000 7f40a5000700 1 -- 172.21.15.131:0/3576031256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721bbe00 msgr2=0x557e721bc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:07.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.805+0000 7f40a5000700 1 --2- 172.21.15.131:0/3576031256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721bbe00 0x557e721bc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.805+0000 7f40a5000700 1 -- 172.21.15.131:0/3576031256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557e721bc720 msgr2=0x557e721b6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:07.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.805+0000 7f40a5000700 1 --2- 172.21.15.131:0/3576031256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557e721bc720 0x557e721b6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.805+0000 7f40a5000700 1 -- 172.21.15.131:0/3576031256 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e71f873b0 con 0x557e721b6890 2024-05-08T22:09:07.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.806+0000 7f40a5000700 1 --2- 172.21.15.131:0/3576031256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e721b6890 0x557e721b6cd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f40a006a5d0 tx=0x7f408c009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a1896f7e05e512c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:07.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.806+0000 7f40a6400700 1 --2- 172.21.15.131:0/2688980610 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f40880a1290 0x7f40880a3750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4098009fd0 tx=0x7f409800b040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:07.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.806+0000 7f4092a00700 1 -- 172.21.15.131:0/3576031256 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c011c80 con 0x557e721b6890 2024-05-08T22:09:07.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.806+0000 7f4092a00700 1 -- 172.21.15.131:0/3576031256 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f408c004510 con 0x557e721b6890 2024-05-08T22:09:07.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.806+0000 7f4092a00700 1 -- 172.21.15.131:0/3576031256 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c005280 con 0x557e721b6890 2024-05-08T22:09:07.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.807+0000 7f40bbffab80 1 -- 172.21.15.131:0/3576031256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e721b6890 msgr2=0x557e721b6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:07.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.807+0000 7f40bbffab80 1 --2- 172.21.15.131:0/3576031256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e721b6890 0x557e721b6cd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f40a006a5d0 tx=0x7f408c009f60 comp rx=0 tx=0).stop 2024-05-08T22:09:07.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.807+0000 7f40bbffab80 1 -- 172.21.15.131:0/3576031256 shutdown_connections 2024-05-08T22:09:07.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.807+0000 7f40bbffab80 1 --2- 172.21.15.131:0/3576031256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e721b6890 0x557e721b6cd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.807+0000 7f40bbffab80 1 --2- 172.21.15.131:0/3576031256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721bbe00 0x557e721bc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.807+0000 7f40bbffab80 1 --2- 172.21.15.131:0/3576031256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557e721bc720 0x557e721b6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.807+0000 7f40bbffab80 1 -- 172.21.15.131:0/3576031256 >> 172.21.15.131:0/3576031256 conn(0x557e72114210 msgr2=0x557e721b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:07.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.807+0000 7f40bbffab80 1 -- 172.21.15.131:0/3576031256 shutdown_connections 2024-05-08T22:09:07.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.807+0000 7f40bbffab80 1 -- 172.21.15.131:0/3576031256 wait complete. 2024-05-08T22:09:07.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.807+0000 7f40bbffab80 1 -- start start 2024-05-08T22:09:07.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.808+0000 7f40bbffab80 1 Processor -- start 2024-05-08T22:09:07.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.808+0000 7f40bbffab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721b6890 0x557e721bfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:07.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.808+0000 7f40bbffab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e721bbe00 0x557e721c0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:07.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.808+0000 7f40bbffab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557e721bc720 0x557e721c0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:07.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.809+0000 7f40bbffab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e720a2560 con 0x557e721bc720 2024-05-08T22:09:07.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.809+0000 7f40bbffab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e720a2200 con 0x557e721bbe00 2024-05-08T22:09:07.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.809+0000 7f40bbffab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e71f896a0 con 0x557e721b6890 2024-05-08T22:09:07.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.809+0000 7f40bbffab80 10 client.0 _ll_register_callbacks cb 0x557e720abb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:07.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.809+0000 7f40a5a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721b6890 0x557e721bfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:07.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.809+0000 7f40a5a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721b6890 0x557e721bfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60612/0 (socket says 172.21.15.131:60612) 2024-05-08T22:09:07.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.809+0000 7f40a5a00700 1 -- 172.21.15.131:0/1842924736 learned_addr learned my addr 172.21.15.131:0/1842924736 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:07.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.809+0000 7f40a5000700 1 --2- 172.21.15.131:0/1842924736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e721bbe00 0x557e721c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:07.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.809+0000 7f40a6400700 1 --2- 172.21.15.131:0/1842924736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557e721bc720 0x557e721c0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:07.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.809+0000 7f40a5a00700 1 -- 172.21.15.131:0/1842924736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e721bbe00 msgr2=0x557e721c0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:07.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.809+0000 7f40a5a00700 1 --2- 172.21.15.131:0/1842924736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e721bbe00 0x557e721c0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.809+0000 7f40a5a00700 1 -- 172.21.15.131:0/1842924736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557e721bc720 msgr2=0x557e721c0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:07.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.809+0000 7f40a5a00700 1 --2- 172.21.15.131:0/1842924736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557e721bc720 0x557e721c0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.809+0000 7f40a5a00700 1 -- 172.21.15.131:0/1842924736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f408c0098e0 con 0x557e721b6890 2024-05-08T22:09:07.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.809+0000 7f40a6400700 1 --2- 172.21.15.131:0/1842924736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557e721bc720 0x557e721c0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:09:07.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.809+0000 7f40a5a00700 1 --2- 172.21.15.131:0/1842924736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721b6890 0x557e721bfad0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f409400f740 tx=0x7f4094014490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bcbf8fb895f5b6c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:07.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.809+0000 7f4087400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:09:07.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.810+0000 7f4087400700 1 -- 172.21.15.131:0/1842924736 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40940ce170 con 0x557e721b6890 2024-05-08T22:09:07.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.810+0000 7f4087400700 1 -- 172.21.15.131:0/1842924736 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f409409b910 con 0x557e721b6890 2024-05-08T22:09:07.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.810+0000 7f40bbffab80 1 -- 172.21.15.131:0/1842924736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e7205f270 con 0x557e721b6890 2024-05-08T22:09:07.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.810+0000 7f4087400700 1 -- 172.21.15.131:0/1842924736 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4094097550 con 0x557e721b6890 2024-05-08T22:09:07.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.810+0000 7f40bbffab80 10 client.25924 fetch_fsmap learned FSMap version 24 2024-05-08T22:09:07.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.810+0000 7f40bbffab80 1 -- 172.21.15.131:0/1842924736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557e71f33b60 con 0x557e721b6890 2024-05-08T22:09:07.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.810+0000 7f4087400700 1 -- 172.21.15.131:0/1842924736 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557e7205f270 con 0x557e721b6890 2024-05-08T22:09:07.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.810+0000 7f4087400700 1 -- 172.21.15.131:0/1842924736 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f40940a10b0 con 0x557e721b6890 2024-05-08T22:09:07.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.810+0000 7f40bbffab80 10 client.25924 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:09:07.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.810+0000 7f40bbffab80 10 client.25924 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:09:07.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.810+0000 7f40bbffab80 20 client.25924 populate_metadata read hostname 'smithi131' 2024-05-08T22:09:07.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.810+0000 7f40bbffab80 1 --2- 172.21.15.131:0/1842924736 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x557e721cfe40 0x557e721d2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:07.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.810+0000 7f40bbffab80 4 client.25924 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:09:07.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.810+0000 7f40bbffab80 1 -- 172.21.15.131:0/1842924736 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e72185810 con 0x557e721cfe40 2024-05-08T22:09:07.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.811+0000 7f40a5000700 1 --2- 172.21.15.131:0/1842924736 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x557e721cfe40 0x557e721d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:07.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.811+0000 7f4087400700 10 client.25924 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:09:07.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.811+0000 7f40a5000700 1 --2- 172.21.15.131:0/1842924736 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x557e721cfe40 0x557e721d2230 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e622f9186b9d706 server_cookie=c977751a5d780a0 in_seq=0 out_seq=0 2024-05-08T22:09:07.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.812+0000 7f4087400700 1 -- 172.21.15.131:0/1842924736 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557e72185810 con 0x557e721cfe40 2024-05-08T22:09:07.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.812+0000 7f4087400700 10 client.25924 handle_command_reply: tid=0 2024-05-08T22:09:07.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.812+0000 7f40bbffab80 1 client.25924 shutdown 2024-05-08T22:09:07.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.812+0000 7f40bbffab80 1 -- 172.21.15.131:0/1842924736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721b6890 msgr2=0x557e721bfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:07.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.812+0000 7f40bbffab80 1 --2- 172.21.15.131:0/1842924736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721b6890 0x557e721bfad0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f409400f740 tx=0x7f4094014490 comp rx=0 tx=0).stop 2024-05-08T22:09:07.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.812+0000 7f40bbffab80 1 -- 172.21.15.131:0/1842924736 shutdown_connections 2024-05-08T22:09:07.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.812+0000 7f40bbffab80 1 --2- 172.21.15.131:0/1842924736 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x557e721cfe40 0x557e721d2230 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.812+0000 7f40bbffab80 1 --2- 172.21.15.131:0/1842924736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557e721bc720 0x557e721c0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.812+0000 7f4087400700 0 client.25924 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:09:07.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.812+0000 7f40bbffab80 1 --2- 172.21.15.131:0/1842924736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721b6890 0x557e721bfad0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.812+0000 7f40bbffab80 1 --2- 172.21.15.131:0/1842924736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e721bbe00 0x557e721c0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.812+0000 7f40bbffab80 1 -- 172.21.15.131:0/1842924736 >> 172.21.15.131:0/1842924736 conn(0x557e72114210 msgr2=0x557e721b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:07.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.812+0000 7f40bbffab80 1 -- 172.21.15.131:0/1842924736 shutdown_connections 2024-05-08T22:09:07.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.813+0000 7f40bbffab80 1 -- 172.21.15.131:0/1842924736 wait complete. 2024-05-08T22:09:07.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.813+0000 7f40bbffab80 20 client.25924 trim_cache size 0 max 16384 2024-05-08T22:09:07.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.826+0000 7f40bbffab80 1 Processor -- start 2024-05-08T22:09:07.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.826+0000 7f40bbffab80 1 -- start start 2024-05-08T22:09:07.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.827+0000 7f40bbffab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721b6890 0x557e721b6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:07.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.827+0000 7f40bbffab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557e721bbe00 0x557e721b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:07.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.827+0000 7f40bbffab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e721bc720 0x557e721b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:07.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.827+0000 7f40bbffab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e721bc580 con 0x557e721bbe00 2024-05-08T22:09:07.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.827+0000 7f40bbffab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e721addc0 con 0x557e721bc720 2024-05-08T22:09:07.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.827+0000 7f40bbffab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e721d73a0 con 0x557e721b6890 2024-05-08T22:09:07.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.827+0000 7f40a5a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721b6890 0x557e721b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:07.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.827+0000 7f40a5000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557e721bbe00 0x557e721b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:07.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.827+0000 7f40a5a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721b6890 0x557e721b6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60628/0 (socket says 172.21.15.131:60628) 2024-05-08T22:09:07.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.827+0000 7f40a5a00700 1 -- 172.21.15.131:0/1416610319 learned_addr learned my addr 172.21.15.131:0/1416610319 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:07.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.827+0000 7f40a6400700 1 --2- 172.21.15.131:0/1416610319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e721bc720 0x557e721b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:07.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.827+0000 7f40a5a00700 1 -- 172.21.15.131:0/1416610319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e721bc720 msgr2=0x557e721b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:07.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.827+0000 7f40a5a00700 1 --2- 172.21.15.131:0/1416610319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e721bc720 0x557e721b0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.827+0000 7f40a5a00700 1 -- 172.21.15.131:0/1416610319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557e721bbe00 msgr2=0x557e721b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:07.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.827+0000 7f40a5a00700 1 --2- 172.21.15.131:0/1416610319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557e721bbe00 0x557e721b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.827+0000 7f40a5a00700 1 -- 172.21.15.131:0/1416610319 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e720cc560 con 0x557e721b6890 2024-05-08T22:09:07.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.827+0000 7f40a6400700 1 --2- 172.21.15.131:0/1416610319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e721bc720 0x557e721b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:09:07.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.828+0000 7f40a5a00700 1 --2- 172.21.15.131:0/1416610319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721b6890 0x557e721b6350 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f409409b1a0 tx=0x7f40940ce5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6e38121a8da308f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:07.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.828+0000 7f4093400700 1 -- 172.21.15.131:0/1416610319 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40940ce170 con 0x557e721b6890 2024-05-08T22:09:07.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.828+0000 7f4093400700 1 -- 172.21.15.131:0/1416610319 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40940142a0 con 0x557e721b6890 2024-05-08T22:09:07.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.828+0000 7f4093400700 1 -- 172.21.15.131:0/1416610319 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409409b950 con 0x557e721b6890 2024-05-08T22:09:07.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.829+0000 7f40bbffab80 1 -- 172.21.15.131:0/1416610319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721b6890 msgr2=0x557e721b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:07.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.829+0000 7f40bbffab80 1 --2- 172.21.15.131:0/1416610319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721b6890 0x557e721b6350 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f409409b1a0 tx=0x7f40940ce5a0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.829+0000 7f40bbffab80 1 -- 172.21.15.131:0/1416610319 shutdown_connections 2024-05-08T22:09:07.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.829+0000 7f40bbffab80 1 --2- 172.21.15.131:0/1416610319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e721bc720 0x557e721b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.829+0000 7f40bbffab80 1 --2- 172.21.15.131:0/1416610319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721b6890 0x557e721b6350 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.829+0000 7f40bbffab80 1 --2- 172.21.15.131:0/1416610319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557e721bbe00 0x557e721b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.829+0000 7f40bbffab80 1 -- 172.21.15.131:0/1416610319 >> 172.21.15.131:0/1416610319 conn(0x557e72114210 msgr2=0x557e721ad210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:07.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.829+0000 7f40bbffab80 1 -- 172.21.15.131:0/1416610319 shutdown_connections 2024-05-08T22:09:07.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.829+0000 7f40bbffab80 1 -- 172.21.15.131:0/1416610319 wait complete. 2024-05-08T22:09:07.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.829+0000 7f40bbffab80 1 -- start start 2024-05-08T22:09:07.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.829+0000 7f40bbffab80 1 Processor -- start 2024-05-08T22:09:07.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.830+0000 7f40bbffab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e721b6890 0x557e721c7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:07.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.830+0000 7f40bbffab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721bbe00 0x557e721c4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:07.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.830+0000 7f40bbffab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557e721bc720 0x557e721c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:07.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.830+0000 7f40bbffab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e721c51a0 con 0x557e721bc720 2024-05-08T22:09:07.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.830+0000 7f40bbffab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e721c5310 con 0x557e721b6890 2024-05-08T22:09:07.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.830+0000 7f40bbffab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e721c54d0 con 0x557e721bbe00 2024-05-08T22:09:07.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.830+0000 7f40bbffab80 10 client.0 _ll_register_callbacks cb 0x557e720abb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:07.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.830+0000 7f40a5000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721bbe00 0x557e721c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:07.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.831+0000 7f40a6400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557e721bc720 0x557e721c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:07.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.831+0000 7f40a5000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721bbe00 0x557e721c4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60632/0 (socket says 172.21.15.131:60632) 2024-05-08T22:09:07.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.831+0000 7f40a5000700 1 -- 172.21.15.131:0/166789260 learned_addr learned my addr 172.21.15.131:0/166789260 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:07.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.831+0000 7f40a5a00700 1 --2- 172.21.15.131:0/166789260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e721b6890 0x557e721c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:07.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.831+0000 7f40a5000700 1 -- 172.21.15.131:0/166789260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e721b6890 msgr2=0x557e721c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:07.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.831+0000 7f40a5000700 1 --2- 172.21.15.131:0/166789260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e721b6890 0x557e721c7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.831+0000 7f40a5000700 1 -- 172.21.15.131:0/166789260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557e721bc720 msgr2=0x557e721c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:07.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.831+0000 7f40a5000700 1 --2- 172.21.15.131:0/166789260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557e721bc720 0x557e721c4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.831+0000 7f40a5000700 1 -- 172.21.15.131:0/166789260 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e71f33b60 con 0x557e721bbe00 2024-05-08T22:09:07.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.831+0000 7f40a5000700 1 --2- 172.21.15.131:0/166789260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721bbe00 0x557e721c4720 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f40a006a5d0 tx=0x7f408c0047a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee218a64ff877698 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:07.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.831+0000 7f4087400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:09:07.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.831+0000 7f4087400700 1 -- 172.21.15.131:0/166789260 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c02a070 con 0x557e721bbe00 2024-05-08T22:09:07.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.832+0000 7f40bbffab80 1 -- 172.21.15.131:0/166789260 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f408c004510 con 0x557e721bbe00 2024-05-08T22:09:07.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.832+0000 7f4087400700 1 -- 172.21.15.131:0/166789260 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f408c011a80 con 0x557e721bbe00 2024-05-08T22:09:07.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.832+0000 7f4087400700 1 -- 172.21.15.131:0/166789260 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f408c004b50 con 0x557e721bbe00 2024-05-08T22:09:07.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.832+0000 7f40bbffab80 10 client.25930 fetch_fsmap learned FSMap version 24 2024-05-08T22:09:07.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.832+0000 7f40bbffab80 1 -- 172.21.15.131:0/166789260 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557e71f8bcd0 con 0x557e721bbe00 2024-05-08T22:09:07.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.832+0000 7f4087400700 1 -- 172.21.15.131:0/166789260 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f408c004510 con 0x557e721bbe00 2024-05-08T22:09:07.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.832+0000 7f4087400700 1 -- 172.21.15.131:0/166789260 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f408c005580 con 0x557e721bbe00 2024-05-08T22:09:07.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.832+0000 7f40bbffab80 10 client.25930 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:09:07.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.832+0000 7f40bbffab80 10 client.25930 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:09:07.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.832+0000 7f40bbffab80 20 client.25930 populate_metadata read hostname 'smithi131' 2024-05-08T22:09:07.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.832+0000 7f40bbffab80 1 --2- 172.21.15.131:0/166789260 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x557e722267f0 0x557e72228c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:07.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.832+0000 7f40bbffab80 4 client.25930 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:09:07.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.832+0000 7f40bbffab80 1 -- 172.21.15.131:0/166789260 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557e720feec0 con 0x557e722267f0 2024-05-08T22:09:07.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.833+0000 7f40a5a00700 1 --2- 172.21.15.131:0/166789260 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x557e722267f0 0x557e72228c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:07.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.833+0000 7f4087400700 10 client.25930 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:09:07.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.833+0000 7f40a5a00700 1 --2- 172.21.15.131:0/166789260 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x557e722267f0 0x557e72228c10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e322886fd3d039d9 server_cookie=5b0e78caba01b9a in_seq=0 out_seq=0 2024-05-08T22:09:07.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.833+0000 7f4087400700 1 -- 172.21.15.131:0/166789260 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557e720feec0 con 0x557e722267f0 2024-05-08T22:09:07.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.833+0000 7f4087400700 10 client.25930 handle_command_reply: tid=0 2024-05-08T22:09:07.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.833+0000 7f40bbffab80 1 client.25930 shutdown 2024-05-08T22:09:07.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.834+0000 7f40bbffab80 1 -- 172.21.15.131:0/166789260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721bbe00 msgr2=0x557e721c4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:07.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.834+0000 7f40bbffab80 1 --2- 172.21.15.131:0/166789260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721bbe00 0x557e721c4720 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f40a006a5d0 tx=0x7f408c0047a0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.834+0000 7f40bbffab80 1 -- 172.21.15.131:0/166789260 shutdown_connections 2024-05-08T22:09:07.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.834+0000 7f40bbffab80 1 --2- 172.21.15.131:0/166789260 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x557e722267f0 0x557e72228c10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.834+0000 7f40bbffab80 1 --2- 172.21.15.131:0/166789260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557e721bc720 0x557e721c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.834+0000 7f40bbffab80 1 --2- 172.21.15.131:0/166789260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e721b6890 0x557e721c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.834+0000 7f40bbffab80 1 --2- 172.21.15.131:0/166789260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557e721bbe00 0x557e721c4720 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.834+0000 7f4087400700 0 client.25930 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:09:07.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.834+0000 7f40bbffab80 1 -- 172.21.15.131:0/166789260 >> 172.21.15.131:0/166789260 conn(0x557e72114210 msgr2=0x557e721c03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:07.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.834+0000 7f40bbffab80 1 -- 172.21.15.131:0/166789260 shutdown_connections 2024-05-08T22:09:07.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.834+0000 7f40bbffab80 1 -- 172.21.15.131:0/166789260 wait complete. 2024-05-08T22:09:07.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.834+0000 7f40bbffab80 20 client.25930 trim_cache size 0 max 16384 2024-05-08T22:09:07.840 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:09:07.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.835+0000 7f40a8400700 1 -- 172.21.15.131:0/2688980610 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f40880a1290 msgr2=0x7f40880a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:07.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.835+0000 7f40a8400700 1 --2- 172.21.15.131:0/2688980610 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f40880a1290 0x7f40880a3750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4098009fd0 tx=0x7f409800b040 comp rx=0 tx=0).stop 2024-05-08T22:09:07.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.835+0000 7f40a8400700 1 -- 172.21.15.131:0/2688980610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40a0101180 msgr2=0x7f40a00732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:07.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.835+0000 7f40a8400700 1 --2- 172.21.15.131:0/2688980610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40a0101180 0x7f40a00732a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4094000c00 tx=0x7f4094005740 comp rx=0 tx=0).stop 2024-05-08T22:09:07.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.835+0000 7f40a8400700 1 -- 172.21.15.131:0/2688980610 shutdown_connections 2024-05-08T22:09:07.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.835+0000 7f40a8400700 1 --2- 172.21.15.131:0/2688980610 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f40880a1290 0x7f40880a3750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.835+0000 7f40a8400700 1 --2- 172.21.15.131:0/2688980610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40a0073d20 0x7f40a01a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.835+0000 7f40a8400700 1 --2- 172.21.15.131:0/2688980610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40a0101180 0x7f40a00732a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.835+0000 7f40a8400700 1 --2- 172.21.15.131:0/2688980610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40a01062a0 0x7f40a00737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:07.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.835+0000 7f40a8400700 1 -- 172.21.15.131:0/2688980610 >> 172.21.15.131:0/2688980610 conn(0x7f40a00fc9e0 msgr2=0x7f40a00fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:07.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.835+0000 7f40a8400700 1 -- 172.21.15.131:0/2688980610 shutdown_connections 2024-05-08T22:09:07.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:07.835+0000 7f40a8400700 1 -- 172.21.15.131:0/2688980610 wait complete. 2024-05-08T22:09:07.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:07 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/543275021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:09:07.853 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:07 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/543275021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:09:07.856 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T22:09:07.869 INFO:tasks.workunit.client.0.smithi131.stdout:checking for long... yes 2024-05-08T22:09:07.973 INFO:tasks.workunit.client.0.smithi131.stdout:checking size of long... 8 2024-05-08T22:09:08.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:07 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/543275021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:09:08.051 INFO:tasks.workunit.client.0.smithi131.stdout:checking for long long... yes 2024-05-08T22:09:08.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.060+0000 7f035a800700 1 -- 172.21.15.131:0/985880514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0354068c90 msgr2=0x7f0354105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:08.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.060+0000 7f035a800700 1 --2- 172.21.15.131:0/985880514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0354068c90 0x7f0354105b60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f03400099c0 tx=0x7f0340009cd0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.060+0000 7f035a800700 1 -- 172.21.15.131:0/985880514 shutdown_connections 2024-05-08T22:09:08.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.060+0000 7f035a800700 1 --2- 172.21.15.131:0/985880514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0354106290 0x7f035410de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.060+0000 7f035a800700 1 --2- 172.21.15.131:0/985880514 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03540682e0 0x7f03540686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.060+0000 7f035a800700 1 --2- 172.21.15.131:0/985880514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0354068c90 0x7f0354105b60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.060+0000 7f035a800700 1 -- 172.21.15.131:0/985880514 >> 172.21.15.131:0/985880514 conn(0x7f0354076f40 msgr2=0x7f0354077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:08.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.060+0000 7f035a800700 1 -- 172.21.15.131:0/985880514 shutdown_connections 2024-05-08T22:09:08.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.060+0000 7f035a800700 1 -- 172.21.15.131:0/985880514 wait complete. 2024-05-08T22:09:08.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.061+0000 7f035a800700 1 Processor -- start 2024-05-08T22:09:08.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.061+0000 7f035a800700 1 -- start start 2024-05-08T22:09:08.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.062+0000 7f035a800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03540682e0 0x7f03540ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.062+0000 7f035a800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0354106290 0x7f03541003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.062+0000 7f035a800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03541008e0 0x7f03541a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.062+0000 7f035a800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03541a5bc0 con 0x7f0354106290 2024-05-08T22:09:08.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.062+0000 7f035a800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03541a5d30 con 0x7f03540682e0 2024-05-08T22:09:08.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.062+0000 7f035a800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03541a5ef0 con 0x7f03541008e0 2024-05-08T22:09:08.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.062+0000 7f0352a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0354106290 0x7f03541003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.062+0000 7f0353e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03541008e0 0x7f03541a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.062+0000 7f0352a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0354106290 0x7f03541003a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51840/0 (socket says 172.21.15.131:51840) 2024-05-08T22:09:08.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.062+0000 7f0352a00700 1 -- 172.21.15.131:0/1966159415 learned_addr learned my addr 172.21.15.131:0/1966159415 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:08.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.062+0000 7f0353400700 1 --2- 172.21.15.131:0/1966159415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03540682e0 0x7f03540ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.062+0000 7f0353e00700 1 -- 172.21.15.131:0/1966159415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03540682e0 msgr2=0x7f03540ffe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:08.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.062+0000 7f0353e00700 1 --2- 172.21.15.131:0/1966159415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03540682e0 0x7f03540ffe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.062+0000 7f0353e00700 1 -- 172.21.15.131:0/1966159415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0354106290 msgr2=0x7f03541003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:08.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.062+0000 7f0353e00700 1 --2- 172.21.15.131:0/1966159415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0354106290 0x7f03541003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.062+0000 7f0353e00700 1 -- 172.21.15.131:0/1966159415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0344009520 con 0x7f03541008e0 2024-05-08T22:09:08.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.062+0000 7f0353400700 1 --2- 172.21.15.131:0/1966159415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03540682e0 0x7f03540ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:09:08.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.062+0000 7f0352a00700 1 --2- 172.21.15.131:0/1966159415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0354106290 0x7f03541003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:09:08.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.063+0000 7f0353e00700 1 --2- 172.21.15.131:0/1966159415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03541008e0 0x7f03541a55f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f034c007f10 tx=0x7f034c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:08.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.063+0000 7f034be00700 1 -- 172.21.15.131:0/1966159415 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c009ea0 con 0x7f03541008e0 2024-05-08T22:09:08.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.063+0000 7f035a800700 1 -- 172.21.15.131:0/1966159415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03400095f0 con 0x7f03541008e0 2024-05-08T22:09:08.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.063+0000 7f034be00700 1 -- 172.21.15.131:0/1966159415 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f034c00f040 con 0x7f03541008e0 2024-05-08T22:09:08.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.063+0000 7f034be00700 1 -- 172.21.15.131:0/1966159415 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c004350 con 0x7f03541008e0 2024-05-08T22:09:08.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.063+0000 7f035a800700 1 -- 172.21.15.131:0/1966159415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03541a65a0 con 0x7f03541008e0 2024-05-08T22:09:08.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.066+0000 7f034be00700 1 -- 172.21.15.131:0/1966159415 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f034c00db20 con 0x7f03541008e0 2024-05-08T22:09:08.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.067+0000 7f036e31ab80 1 Processor -- start 2024-05-08T22:09:08.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.068+0000 7f034be00700 1 --2- 172.21.15.131:0/1966159415 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f03380a1290 0x7f03380a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.068+0000 7f036e31ab80 1 -- start start 2024-05-08T22:09:08.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.068+0000 7f034be00700 1 -- 172.21.15.131:0/1966159415 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f034c0cc940 con 0x7f03541008e0 2024-05-08T22:09:08.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.068+0000 7f036e31ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638105e3e00 0x5638105e41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.068+0000 7f036e31ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5638105e4720 0x5638105de350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.068+0000 7f036e31ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638105de890 0x5638105decd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.068+0000 7f036e31ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563810535150 con 0x5638105e3e00 2024-05-08T22:09:08.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.068+0000 7f036e31ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563810532b10 con 0x5638105e4720 2024-05-08T22:09:08.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.068+0000 7f036e31ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56381052e400 con 0x5638105de890 2024-05-08T22:09:08.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.068+0000 7f0352a00700 1 --2- 172.21.15.131:0/1966159415 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f03380a1290 0x7f03380a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.068+0000 7f0353400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638105e3e00 0x5638105e41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.068+0000 7f0353400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638105e3e00 0x5638105e41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51852/0 (socket says 172.21.15.131:51852) 2024-05-08T22:09:08.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.068+0000 7f0353400700 1 -- 172.21.15.131:0/3790228717 learned_addr learned my addr 172.21.15.131:0/3790228717 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:08.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.069+0000 7f0353e00700 1 --2- 172.21.15.131:0/3790228717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638105de890 0x5638105decd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.069+0000 7f0352a00700 1 --2- 172.21.15.131:0/3790228717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5638105e4720 0x5638105de350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.069+0000 7f0353400700 1 -- 172.21.15.131:0/3790228717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638105de890 msgr2=0x5638105decd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:08.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.069+0000 7f0353400700 1 --2- 172.21.15.131:0/3790228717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638105de890 0x5638105decd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.069+0000 7f0353400700 1 -- 172.21.15.131:0/3790228717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5638105e4720 msgr2=0x5638105de350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:08.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.069+0000 7f0353400700 1 --2- 172.21.15.131:0/3790228717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5638105e4720 0x5638105de350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.069+0000 7f0353400700 1 -- 172.21.15.131:0/3790228717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638103af3b0 con 0x5638105e3e00 2024-05-08T22:09:08.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.069+0000 7f0353400700 1 --2- 172.21.15.131:0/3790228717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638105e3e00 0x5638105e41e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0344011760 tx=0x7f0344011a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38bac3d9f897df server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:08.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.069+0000 7f0352a00700 1 --2- 172.21.15.131:0/1966159415 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f03380a1290 0x7f03380a3750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x5638105e4b80 tx=0x7f0340005fd0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:08.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.069+0000 7f0348c00700 1 -- 172.21.15.131:0/3790228717 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034400f4e0 con 0x5638105e3e00 2024-05-08T22:09:08.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.070+0000 7f0348c00700 1 -- 172.21.15.131:0/3790228717 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f034400fb00 con 0x5638105e3e00 2024-05-08T22:09:08.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.070+0000 7f0348c00700 1 -- 172.21.15.131:0/3790228717 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344019b70 con 0x5638105e3e00 2024-05-08T22:09:08.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.070+0000 7f036e31ab80 1 -- 172.21.15.131:0/3790228717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638105e3e00 msgr2=0x5638105e41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:08.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.070+0000 7f036e31ab80 1 --2- 172.21.15.131:0/3790228717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638105e3e00 0x5638105e41e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0344011760 tx=0x7f0344011a70 comp rx=0 tx=0).stop 2024-05-08T22:09:08.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.070+0000 7f036e31ab80 1 -- 172.21.15.131:0/3790228717 shutdown_connections 2024-05-08T22:09:08.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.070+0000 7f036e31ab80 1 --2- 172.21.15.131:0/3790228717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638105de890 0x5638105decd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.070+0000 7f036e31ab80 1 --2- 172.21.15.131:0/3790228717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638105e3e00 0x5638105e41e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.070+0000 7f036e31ab80 1 --2- 172.21.15.131:0/3790228717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5638105e4720 0x5638105de350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.070+0000 7f036e31ab80 1 -- 172.21.15.131:0/3790228717 >> 172.21.15.131:0/3790228717 conn(0x56381053c210 msgr2=0x5638105d8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:08.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.070+0000 7f036e31ab80 1 -- 172.21.15.131:0/3790228717 shutdown_connections 2024-05-08T22:09:08.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.071+0000 7f036e31ab80 1 -- 172.21.15.131:0/3790228717 wait complete. 2024-05-08T22:09:08.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.071+0000 7f036e31ab80 1 -- start start 2024-05-08T22:09:08.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.071+0000 7f036e31ab80 1 Processor -- start 2024-05-08T22:09:08.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.072+0000 7f036e31ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638105de890 0x5638105e7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.072+0000 7f036e31ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638105e3e00 0x5638105e7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.072+0000 7f036e31ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5638105e4720 0x5638105e84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.072+0000 7f036e31ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638104ca560 con 0x5638105de890 2024-05-08T22:09:08.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.072+0000 7f036e31ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638104ca200 con 0x5638105e4720 2024-05-08T22:09:08.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.072+0000 7f036e31ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638103b16a0 con 0x5638105e3e00 2024-05-08T22:09:08.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.072+0000 7f036e31ab80 10 client.0 _ll_register_callbacks cb 0x5638104d3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:08.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.072+0000 7f0352a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638105e3e00 0x5638105e7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.072+0000 7f0353400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638105de890 0x5638105e7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.072+0000 7f0352a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638105e3e00 0x5638105e7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60668/0 (socket says 172.21.15.131:60668) 2024-05-08T22:09:08.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.072+0000 7f0352a00700 1 -- 172.21.15.131:0/3414485842 learned_addr learned my addr 172.21.15.131:0/3414485842 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:08.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.072+0000 7f0353e00700 1 --2- 172.21.15.131:0/3414485842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5638105e4720 0x5638105e84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.072+0000 7f0353400700 1 -- 172.21.15.131:0/3414485842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638105e3e00 msgr2=0x5638105e7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:08.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.072+0000 7f0353400700 1 --2- 172.21.15.131:0/3414485842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638105e3e00 0x5638105e7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.072+0000 7f0353400700 1 -- 172.21.15.131:0/3414485842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5638105e4720 msgr2=0x5638105e84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:08.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.072+0000 7f0353400700 1 --2- 172.21.15.131:0/3414485842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5638105e4720 0x5638105e84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.072+0000 7f0353400700 1 -- 172.21.15.131:0/3414485842 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0344011440 con 0x5638105de890 2024-05-08T22:09:08.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.073+0000 7f0353400700 1 --2- 172.21.15.131:0/3414485842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638105de890 0x5638105e7a50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0344011dc0 tx=0x7f0344005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9a7d73fedd6c21 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:08.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.073+0000 7f033d600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:09:08.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.073+0000 7f033d600700 1 -- 172.21.15.131:0/3414485842 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034400f870 con 0x5638105de890 2024-05-08T22:09:08.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.073+0000 7f033d600700 1 -- 172.21.15.131:0/3414485842 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f0344004040 con 0x5638105de890 2024-05-08T22:09:08.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.073+0000 7f036e31ab80 1 -- 172.21.15.131:0/3414485842 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563810487270 con 0x5638105de890 2024-05-08T22:09:08.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.073+0000 7f033d600700 1 -- 172.21.15.131:0/3414485842 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0344004200 con 0x5638105de890 2024-05-08T22:09:08.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.073+0000 7f036e31ab80 10 client.15876 fetch_fsmap learned FSMap version 24 2024-05-08T22:09:08.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.073+0000 7f036e31ab80 1 -- 172.21.15.131:0/3414485842 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56381035bb60 con 0x5638105de890 2024-05-08T22:09:08.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.074+0000 7f033d600700 1 -- 172.21.15.131:0/3414485842 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563810487270 con 0x5638105de890 2024-05-08T22:09:08.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.074+0000 7f033d600700 1 -- 172.21.15.131:0/3414485842 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0344003710 con 0x5638105de890 2024-05-08T22:09:08.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.074+0000 7f036e31ab80 10 client.15876 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:09:08.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.074+0000 7f036e31ab80 10 client.15876 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:09:08.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.074+0000 7f036e31ab80 20 client.15876 populate_metadata read hostname 'smithi131' 2024-05-08T22:09:08.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.074+0000 7f036e31ab80 1 --2- 172.21.15.131:0/3414485842 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x5638105f80e0 0x5638105fa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.074+0000 7f036e31ab80 4 client.15876 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:09:08.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.074+0000 7f036e31ab80 1 -- 172.21.15.131:0/3414485842 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638105ad810 con 0x5638105f80e0 2024-05-08T22:09:08.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.074+0000 7f0352a00700 1 --2- 172.21.15.131:0/3414485842 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x5638105f80e0 0x5638105fa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.075+0000 7f0352a00700 1 --2- 172.21.15.131:0/3414485842 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x5638105f80e0 0x5638105fa500 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f85771df889019fa server_cookie=2bd0a01fec3f597e in_seq=0 out_seq=0 2024-05-08T22:09:08.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.075+0000 7f033d600700 10 client.15876 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:09:08.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.075+0000 7f033d600700 1 -- 172.21.15.131:0/3414485842 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5638105ad810 con 0x5638105f80e0 2024-05-08T22:09:08.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.075+0000 7f033d600700 10 client.15876 handle_command_reply: tid=0 2024-05-08T22:09:08.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.075+0000 7f036e31ab80 1 client.15876 shutdown 2024-05-08T22:09:08.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.076+0000 7f036e31ab80 1 -- 172.21.15.131:0/3414485842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638105de890 msgr2=0x5638105e7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:08.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.076+0000 7f036e31ab80 1 --2- 172.21.15.131:0/3414485842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638105de890 0x5638105e7a50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0344011dc0 tx=0x7f0344005e30 comp rx=0 tx=0).stop 2024-05-08T22:09:08.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.076+0000 7f036e31ab80 1 -- 172.21.15.131:0/3414485842 shutdown_connections 2024-05-08T22:09:08.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.076+0000 7f036e31ab80 1 --2- 172.21.15.131:0/3414485842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5638105e4720 0x5638105e84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.076+0000 7f036e31ab80 1 --2- 172.21.15.131:0/3414485842 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x5638105f80e0 0x5638105fa500 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.076+0000 7f036e31ab80 1 --2- 172.21.15.131:0/3414485842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638105de890 0x5638105e7a50 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.076+0000 7f036e31ab80 1 --2- 172.21.15.131:0/3414485842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638105e3e00 0x5638105e7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.076+0000 7f036e31ab80 1 -- 172.21.15.131:0/3414485842 >> 172.21.15.131:0/3414485842 conn(0x56381053c210 msgr2=0x5638105d85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:08.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.076+0000 7f033d600700 0 client.15876 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:09:08.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.076+0000 7f036e31ab80 1 -- 172.21.15.131:0/3414485842 shutdown_connections 2024-05-08T22:09:08.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.076+0000 7f036e31ab80 1 -- 172.21.15.131:0/3414485842 wait complete. 2024-05-08T22:09:08.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.076+0000 7f036e31ab80 20 client.15876 trim_cache size 0 max 16384 2024-05-08T22:09:08.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.090+0000 7f036e31ab80 1 Processor -- start 2024-05-08T22:09:08.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.090+0000 7f036e31ab80 1 -- start start 2024-05-08T22:09:08.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.090+0000 7f036e31ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638105de890 0x5638105fa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.090+0000 7f036e31ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638105e3e00 0x5638105dec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.090+0000 7f036e31ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5638105e4720 0x5638105d8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.090+0000 7f036e31ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638105e90c0 con 0x5638105de890 2024-05-08T22:09:08.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.090+0000 7f036e31ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638105eac50 con 0x5638105e4720 2024-05-08T22:09:08.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.090+0000 7f036e31ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638105d5dc0 con 0x5638105e3e00 2024-05-08T22:09:08.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.091+0000 7f0353400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638105de890 0x5638105fa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.091+0000 7f0353e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5638105e4720 0x5638105d8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.091+0000 7f0353e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5638105e4720 0x5638105d8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:38322/0 (socket says 172.21.15.131:38322) 2024-05-08T22:09:08.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.091+0000 7f0353400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638105de890 0x5638105fa4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51868/0 (socket says 172.21.15.131:51868) 2024-05-08T22:09:08.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.091+0000 7f0353e00700 1 -- 172.21.15.131:0/994565775 learned_addr learned my addr 172.21.15.131:0/994565775 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:08.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.091+0000 7f0352a00700 1 --2- 172.21.15.131:0/994565775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638105e3e00 0x5638105dec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.091+0000 7f0353400700 1 -- 172.21.15.131:0/994565775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638105e3e00 msgr2=0x5638105dec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:08.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.091+0000 7f0353400700 1 --2- 172.21.15.131:0/994565775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638105e3e00 0x5638105dec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.091+0000 7f0353400700 1 -- 172.21.15.131:0/994565775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5638105e4720 msgr2=0x5638105d8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:08.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.091+0000 7f0353400700 1 --2- 172.21.15.131:0/994565775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5638105e4720 0x5638105d8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.091+0000 7f0353400700 1 -- 172.21.15.131:0/994565775 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638104f4560 con 0x5638105de890 2024-05-08T22:09:08.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.091+0000 7f0353400700 1 --2- 172.21.15.131:0/994565775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638105de890 0x5638105fa4d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0344010010 tx=0x7f0344005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb7684c3b66c08e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:08.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.091+0000 7f0349600700 1 -- 172.21.15.131:0/994565775 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344003cc0 con 0x5638105de890 2024-05-08T22:09:08.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.092+0000 7f0349600700 1 -- 172.21.15.131:0/994565775 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0344005590 con 0x5638105de890 2024-05-08T22:09:08.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.092+0000 7f0349600700 1 -- 172.21.15.131:0/994565775 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344019bc0 con 0x5638105de890 2024-05-08T22:09:08.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.092+0000 7f036e31ab80 1 -- 172.21.15.131:0/994565775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638105de890 msgr2=0x5638105fa4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:08.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.092+0000 7f036e31ab80 1 --2- 172.21.15.131:0/994565775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638105de890 0x5638105fa4d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0344010010 tx=0x7f0344005a00 comp rx=0 tx=0).stop 2024-05-08T22:09:08.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.092+0000 7f036e31ab80 1 -- 172.21.15.131:0/994565775 shutdown_connections 2024-05-08T22:09:08.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.092+0000 7f036e31ab80 1 --2- 172.21.15.131:0/994565775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5638105e4720 0x5638105d8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.092+0000 7f036e31ab80 1 --2- 172.21.15.131:0/994565775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638105de890 0x5638105fa4d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.092+0000 7f036e31ab80 1 --2- 172.21.15.131:0/994565775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638105e3e00 0x5638105dec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.092+0000 7f036e31ab80 1 -- 172.21.15.131:0/994565775 >> 172.21.15.131:0/994565775 conn(0x56381053c210 msgr2=0x5638105e1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:08.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.092+0000 7f036e31ab80 1 -- 172.21.15.131:0/994565775 shutdown_connections 2024-05-08T22:09:08.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.093+0000 7f036e31ab80 1 -- 172.21.15.131:0/994565775 wait complete. 2024-05-08T22:09:08.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.093+0000 7f036e31ab80 1 -- start start 2024-05-08T22:09:08.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.093+0000 7f036e31ab80 1 Processor -- start 2024-05-08T22:09:08.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.094+0000 7f036e31ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638105de890 0x5638105ef180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.094+0000 7f036e31ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5638105e3e00 0x5638105ef6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.094+0000 7f036e31ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638105e4720 0x5638105eb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.094+0000 7f036e31ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638105eba20 con 0x5638105e4720 2024-05-08T22:09:08.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.094+0000 7f036e31ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638105ebb90 con 0x5638105e3e00 2024-05-08T22:09:08.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.094+0000 7f036e31ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638105ebd50 con 0x5638105de890 2024-05-08T22:09:08.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.094+0000 7f036e31ab80 10 client.0 _ll_register_callbacks cb 0x5638104d3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:08.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.094+0000 7f0352a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5638105e3e00 0x5638105ef6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.094+0000 7f0353400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638105de890 0x5638105ef180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.094+0000 7f0352a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5638105e3e00 0x5638105ef6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:38328/0 (socket says 172.21.15.131:38328) 2024-05-08T22:09:08.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.094+0000 7f0352a00700 1 -- 172.21.15.131:0/2758557453 learned_addr learned my addr 172.21.15.131:0/2758557453 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:08.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.094+0000 7f0353e00700 1 --2- 172.21.15.131:0/2758557453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638105e4720 0x5638105eb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.095+0000 7f0352a00700 1 -- 172.21.15.131:0/2758557453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638105de890 msgr2=0x5638105ef180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:08.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.095+0000 7f0352a00700 1 --2- 172.21.15.131:0/2758557453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638105de890 0x5638105ef180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.095+0000 7f0352a00700 1 -- 172.21.15.131:0/2758557453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638105e4720 msgr2=0x5638105eb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:08.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.095+0000 7f0352a00700 1 --2- 172.21.15.131:0/2758557453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638105e4720 0x5638105eb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.095+0000 7f0352a00700 1 -- 172.21.15.131:0/2758557453 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56381035bb60 con 0x5638105e3e00 2024-05-08T22:09:08.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.095+0000 7f0352a00700 1 --2- 172.21.15.131:0/2758557453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5638105e3e00 0x5638105ef6c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f03400092e0 tx=0x7f0340009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=290c6a63b80858aa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:08.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.095+0000 7f033d600700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:09:08.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.095+0000 7f033d600700 1 -- 172.21.15.131:0/2758557453 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034002f040 con 0x5638105e3e00 2024-05-08T22:09:08.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.095+0000 7f033d600700 1 -- 172.21.15.131:0/2758557453 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f0340003890 con 0x5638105e3e00 2024-05-08T22:09:08.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.095+0000 7f036e31ab80 1 -- 172.21.15.131:0/2758557453 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f034400fb00 con 0x5638105e3e00 2024-05-08T22:09:08.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.095+0000 7f033d600700 1 -- 172.21.15.131:0/2758557453 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f034001d070 con 0x5638105e3e00 2024-05-08T22:09:08.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.095+0000 7f036e31ab80 10 client.25934 fetch_fsmap learned FSMap version 24 2024-05-08T22:09:08.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.095+0000 7f036e31ab80 1 -- 172.21.15.131:0/2758557453 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5638103b3cd0 con 0x5638105e3e00 2024-05-08T22:09:08.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.096+0000 7f033d600700 1 -- 172.21.15.131:0/2758557453 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f034400fb00 con 0x5638105e3e00 2024-05-08T22:09:08.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.096+0000 7f033d600700 1 -- 172.21.15.131:0/2758557453 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f034003d8c0 con 0x5638105e3e00 2024-05-08T22:09:08.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.096+0000 7f036e31ab80 10 client.25934 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:09:08.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.096+0000 7f036e31ab80 10 client.25934 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:09:08.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.096+0000 7f036e31ab80 20 client.25934 populate_metadata read hostname 'smithi131' 2024-05-08T22:09:08.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.096+0000 7f036e31ab80 1 --2- 172.21.15.131:0/2758557453 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x56381064e9f0 0x563810650e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.096+0000 7f036e31ab80 4 client.25934 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:09:08.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.096+0000 7f036e31ab80 1 -- 172.21.15.131:0/2758557453 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563810526ec0 con 0x56381064e9f0 2024-05-08T22:09:08.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.096+0000 7f0353400700 1 --2- 172.21.15.131:0/2758557453 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x56381064e9f0 0x563810650e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.097+0000 7f0353400700 1 --2- 172.21.15.131:0/2758557453 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x56381064e9f0 0x563810650e10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3acbf2c1dc2e31b0 server_cookie=198e1864998ed66a in_seq=0 out_seq=0 2024-05-08T22:09:08.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.097+0000 7f033d600700 10 client.25934 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:09:08.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.097+0000 7f033d600700 1 -- 172.21.15.131:0/2758557453 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563810526ec0 con 0x56381064e9f0 2024-05-08T22:09:08.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.097+0000 7f033d600700 10 client.25934 handle_command_reply: tid=0 2024-05-08T22:09:08.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.097+0000 7f036e31ab80 1 client.25934 shutdown 2024-05-08T22:09:08.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.097+0000 7f036e31ab80 1 -- 172.21.15.131:0/2758557453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5638105e3e00 msgr2=0x5638105ef6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:08.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.097+0000 7f036e31ab80 1 --2- 172.21.15.131:0/2758557453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5638105e3e00 0x5638105ef6c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f03400092e0 tx=0x7f0340009310 comp rx=0 tx=0).stop 2024-05-08T22:09:08.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.097+0000 7f036e31ab80 1 -- 172.21.15.131:0/2758557453 shutdown_connections 2024-05-08T22:09:08.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.097+0000 7f036e31ab80 1 --2- 172.21.15.131:0/2758557453 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x56381064e9f0 0x563810650e10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.097+0000 7f036e31ab80 1 --2- 172.21.15.131:0/2758557453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638105e4720 0x5638105eb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.097+0000 7f036e31ab80 1 --2- 172.21.15.131:0/2758557453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638105de890 0x5638105ef180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.097+0000 7f033d600700 0 client.25934 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:09:08.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.097+0000 7f036e31ab80 1 --2- 172.21.15.131:0/2758557453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5638105e3e00 0x5638105ef6c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.097+0000 7f036e31ab80 1 -- 172.21.15.131:0/2758557453 >> 172.21.15.131:0/2758557453 conn(0x56381053c210 msgr2=0x5638105e1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:08.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.097+0000 7f036e31ab80 1 -- 172.21.15.131:0/2758557453 shutdown_connections 2024-05-08T22:09:08.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.097+0000 7f036e31ab80 1 -- 172.21.15.131:0/2758557453 wait complete. 2024-05-08T22:09:08.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.098+0000 7f036e31ab80 20 client.25934 trim_cache size 0 max 16384 2024-05-08T22:09:08.112 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:09:08.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.098+0000 7f035a800700 1 -- 172.21.15.131:0/1966159415 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f03380a1290 msgr2=0x7f03380a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:08.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.098+0000 7f035a800700 1 --2- 172.21.15.131:0/1966159415 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f03380a1290 0x7f03380a3750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x5638105e4b80 tx=0x7f0340005fd0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.098+0000 7f035a800700 1 -- 172.21.15.131:0/1966159415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03541008e0 msgr2=0x7f03541a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:08.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.098+0000 7f035a800700 1 --2- 172.21.15.131:0/1966159415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03541008e0 0x7f03541a55f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f034c007f10 tx=0x7f034c00d3b0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.098+0000 7f035a800700 1 -- 172.21.15.131:0/1966159415 shutdown_connections 2024-05-08T22:09:08.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.098+0000 7f035a800700 1 --2- 172.21.15.131:0/1966159415 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f03380a1290 0x7f03380a3750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.098+0000 7f035a800700 1 --2- 172.21.15.131:0/1966159415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03541008e0 0x7f03541a55f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.098+0000 7f035a800700 1 --2- 172.21.15.131:0/1966159415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03540682e0 0x7f03540ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.098+0000 7f035a800700 1 --2- 172.21.15.131:0/1966159415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0354106290 0x7f03541003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.098+0000 7f035a800700 1 -- 172.21.15.131:0/1966159415 >> 172.21.15.131:0/1966159415 conn(0x7f0354076f40 msgr2=0x7f03540fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:08.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.098+0000 7f035a800700 1 -- 172.21.15.131:0/1966159415 shutdown_connections 2024-05-08T22:09:08.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.098+0000 7f035a800700 1 -- 172.21.15.131:0/1966159415 wait complete. 2024-05-08T22:09:08.119 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T22:09:08.166 INFO:tasks.workunit.client.0.smithi131.stdout:checking size of long long... 8 2024-05-08T22:09:08.293 INFO:tasks.workunit.client.0.smithi131.stdout:checking for ssize_t... yes 2024-05-08T22:09:08.362 INFO:tasks.workunit.client.0.smithi131.stdout:checking for sig_atomic_t... yes 2024-05-08T22:09:08.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.416+0000 7f8f70600700 1 -- 172.21.15.131:0/1584509181 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f681011e0 msgr2=0x7f8f681015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:08.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.416+0000 7f8f70600700 1 --2- 172.21.15.131:0/1584509181 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f681011e0 0x7f8f681015c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009990 tx=0x7f8f5c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.417+0000 7f8f70600700 1 -- 172.21.15.131:0/1584509181 shutdown_connections 2024-05-08T22:09:08.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.417+0000 7f8f70600700 1 --2- 172.21.15.131:0/1584509181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f68106300 0x7f8f6810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.417+0000 7f8f70600700 1 --2- 172.21.15.131:0/1584509181 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f681011e0 0x7f8f681015c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.417+0000 7f8f70600700 1 --2- 172.21.15.131:0/1584509181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f68101b90 0x7f8f68105bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.417+0000 7f8f70600700 1 -- 172.21.15.131:0/1584509181 >> 172.21.15.131:0/1584509181 conn(0x7f8f680fca80 msgr2=0x7f8f680feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:08.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.417+0000 7f8f70600700 1 -- 172.21.15.131:0/1584509181 shutdown_connections 2024-05-08T22:09:08.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.417+0000 7f8f70600700 1 -- 172.21.15.131:0/1584509181 wait complete. 2024-05-08T22:09:08.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.418+0000 7f8f70600700 1 Processor -- start 2024-05-08T22:09:08.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.418+0000 7f8f70600700 1 -- start start 2024-05-08T22:09:08.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.418+0000 7f8f70600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f681011e0 0x7f8f6819f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.418+0000 7f8f70600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f68106300 0x7f8f6819f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.418+0000 7f8f70600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f681995c0 0x7f8f68199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.418+0000 7f8f70600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6819a010 con 0x7f8f681011e0 2024-05-08T22:09:08.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.418+0000 7f8f70600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6819a150 con 0x7f8f681995c0 2024-05-08T22:09:08.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.418+0000 7f8f70600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6819a310 con 0x7f8f68106300 2024-05-08T22:09:08.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.419+0000 7f8f6dc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f681011e0 0x7f8f6819f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.419+0000 7f8f6dc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f681011e0 0x7f8f6819f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51896/0 (socket says 172.21.15.131:51896) 2024-05-08T22:09:08.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.419+0000 7f8f6e600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f681995c0 0x7f8f68199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.419+0000 7f8f6d200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f68106300 0x7f8f6819f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.419+0000 7f8f6dc00700 1 -- 172.21.15.131:0/3547683173 learned_addr learned my addr 172.21.15.131:0/3547683173 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:08.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.419+0000 7f8f6e600700 1 -- 172.21.15.131:0/3547683173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f68106300 msgr2=0x7f8f6819f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:08.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.419+0000 7f8f6e600700 1 --2- 172.21.15.131:0/3547683173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f68106300 0x7f8f6819f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.419+0000 7f8f6e600700 1 -- 172.21.15.131:0/3547683173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f681011e0 msgr2=0x7f8f6819f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:08.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.419+0000 7f8f6e600700 1 --2- 172.21.15.131:0/3547683173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f681011e0 0x7f8f6819f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.419+0000 7f8f6e600700 1 -- 172.21.15.131:0/3547683173 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5c0095f0 con 0x7f8f681995c0 2024-05-08T22:09:08.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.419+0000 7f8f6e600700 1 --2- 172.21.15.131:0/3547683173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f681995c0 0x7f8f68199a40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000efc0 tx=0x7f8f6000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:08.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.420+0000 7f8f66000700 1 -- 172.21.15.131:0/3547683173 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f60018070 con 0x7f8f681995c0 2024-05-08T22:09:08.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.420+0000 7f8f66000700 1 -- 172.21.15.131:0/3547683173 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f6000f040 con 0x7f8f681995c0 2024-05-08T22:09:08.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.420+0000 7f8f66000700 1 -- 172.21.15.131:0/3547683173 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6000cd40 con 0x7f8f681995c0 2024-05-08T22:09:08.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.420+0000 7f8f70600700 1 -- 172.21.15.131:0/3547683173 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f681a62f0 con 0x7f8f681995c0 2024-05-08T22:09:08.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.420+0000 7f8f70600700 1 -- 172.21.15.131:0/3547683173 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f681a6780 con 0x7f8f681995c0 2024-05-08T22:09:08.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.424+0000 7f8f840ceb80 1 Processor -- start 2024-05-08T22:09:08.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.424+0000 7f8f66000700 1 -- 172.21.15.131:0/3547683173 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f8f60003a00 con 0x7f8f681995c0 2024-05-08T22:09:08.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.425+0000 7f8f840ceb80 1 -- start start 2024-05-08T22:09:08.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.425+0000 7f8f840ceb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897bee00 0x55b0897bf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.425+0000 7f8f840ceb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0897bf720 0x55b0897b9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.425+0000 7f8f840ceb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0897b9890 0x55b0897b9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.425+0000 7f8f840ceb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b089710150 con 0x55b0897b9890 2024-05-08T22:09:08.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.425+0000 7f8f840ceb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08970db10 con 0x55b0897bee00 2024-05-08T22:09:08.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.425+0000 7f8f840ceb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b089709400 con 0x55b0897bf720 2024-05-08T22:09:08.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.425+0000 7f8f6dc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0897bf720 0x55b0897b9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.425+0000 7f8f6dc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0897bf720 0x55b0897b9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60704/0 (socket says 172.21.15.131:60704) 2024-05-08T22:09:08.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.425+0000 7f8f6d200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897bee00 0x55b0897bf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.425+0000 7f8f6dc00700 1 -- 172.21.15.131:0/3768222341 learned_addr learned my addr 172.21.15.131:0/3768222341 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:08.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.425+0000 7f8f6d200700 1 --2- 172.21.15.131:0/3768222341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0897b9890 0x55b0897b9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.426+0000 7f8f6d200700 1 -- 172.21.15.131:0/3768222341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0897bf720 msgr2=0x55b0897b9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:08.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.426+0000 7f8f6d200700 1 --2- 172.21.15.131:0/3768222341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0897bf720 0x55b0897b9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.426+0000 7f8f6d200700 1 -- 172.21.15.131:0/3768222341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0897b9890 msgr2=0x55b0897b9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:08.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.426+0000 7f8f6d200700 1 --2- 172.21.15.131:0/3768222341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0897b9890 0x55b0897b9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.426+0000 7f8f6d200700 1 -- 172.21.15.131:0/3768222341 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b08958a3b0 con 0x55b0897bee00 2024-05-08T22:09:08.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.426+0000 7f8f6d200700 1 --2- 172.21.15.131:0/3768222341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897bee00 0x55b0897bf1e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8f6819acf0 tx=0x7f8f54004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6bb5eb833a49376 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:08.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.426+0000 7f8f66000700 1 --2- 172.21.15.131:0/3547683173 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8f500a1310 0x7f8f500a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.426+0000 7f8f5aa00700 1 -- 172.21.15.131:0/3768222341 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f54004a30 con 0x55b0897bee00 2024-05-08T22:09:08.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.426+0000 7f8f5aa00700 1 -- 172.21.15.131:0/3768222341 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f540054e0 con 0x55b0897bee00 2024-05-08T22:09:08.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.426+0000 7f8f5aa00700 1 -- 172.21.15.131:0/3768222341 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5401e660 con 0x55b0897bee00 2024-05-08T22:09:08.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.426+0000 7f8f66000700 1 -- 172.21.15.131:0/3547683173 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f8f600ccb50 con 0x7f8f681995c0 2024-05-08T22:09:08.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.426+0000 7f8f6dc00700 1 --2- 172.21.15.131:0/3547683173 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8f500a1310 0x7f8f500a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.427+0000 7f8f840ceb80 1 -- 172.21.15.131:0/3768222341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897bee00 msgr2=0x55b0897bf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:08.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.427+0000 7f8f840ceb80 1 --2- 172.21.15.131:0/3768222341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897bee00 0x55b0897bf1e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8f6819acf0 tx=0x7f8f54004750 comp rx=0 tx=0).stop 2024-05-08T22:09:08.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.427+0000 7f8f840ceb80 1 -- 172.21.15.131:0/3768222341 shutdown_connections 2024-05-08T22:09:08.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.427+0000 7f8f6dc00700 1 --2- 172.21.15.131:0/3547683173 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8f500a1310 0x7f8f500a37d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55b0897b5bd0 tx=0x7f8f5c00b260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:08.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.427+0000 7f8f840ceb80 1 --2- 172.21.15.131:0/3768222341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0897b9890 0x55b0897b9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.427+0000 7f8f840ceb80 1 --2- 172.21.15.131:0/3768222341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897bee00 0x55b0897bf1e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.427+0000 7f8f840ceb80 1 --2- 172.21.15.131:0/3768222341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0897bf720 0x55b0897b9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.427+0000 7f8f840ceb80 1 -- 172.21.15.131:0/3768222341 >> 172.21.15.131:0/3768222341 conn(0x55b089717210 msgr2=0x55b0897b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:08.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.427+0000 7f8f840ceb80 1 -- 172.21.15.131:0/3768222341 shutdown_connections 2024-05-08T22:09:08.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.427+0000 7f8f840ceb80 1 -- 172.21.15.131:0/3768222341 wait complete. 2024-05-08T22:09:08.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.428+0000 7f8f840ceb80 1 -- start start 2024-05-08T22:09:08.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.428+0000 7f8f840ceb80 1 Processor -- start 2024-05-08T22:09:08.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.428+0000 7f8f840ceb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897b9890 0x55b0897c2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.428+0000 7f8f840ceb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0897bee00 0x55b0897c30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.428+0000 7f8f840ceb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0897bf720 0x55b0897c3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.428+0000 7f8f840ceb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0896a5560 con 0x55b0897bf720 2024-05-08T22:09:08.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.428+0000 7f8f840ceb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0896a5200 con 0x55b0897b9890 2024-05-08T22:09:08.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.428+0000 7f8f840ceb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08958c6a0 con 0x55b0897bee00 2024-05-08T22:09:08.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.429+0000 7f8f840ceb80 10 client.0 _ll_register_callbacks cb 0x55b0896aeb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:08.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.429+0000 7f8f6dc00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897b9890 0x55b0897c2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.429+0000 7f8f6d200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0897bee00 0x55b0897c30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.429+0000 7f8f6dc00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897b9890 0x55b0897c2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:38376/0 (socket says 172.21.15.131:38376) 2024-05-08T22:09:08.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.429+0000 7f8f6dc00700 1 -- 172.21.15.131:0/3880527442 learned_addr learned my addr 172.21.15.131:0/3880527442 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:08.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.429+0000 7f8f6e600700 1 --2- 172.21.15.131:0/3880527442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0897bf720 0x55b0897c3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.429+0000 7f8f6d200700 1 -- 172.21.15.131:0/3880527442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897b9890 msgr2=0x55b0897c2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:08.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.429+0000 7f8f6d200700 1 --2- 172.21.15.131:0/3880527442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897b9890 0x55b0897c2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.429+0000 7f8f6d200700 1 -- 172.21.15.131:0/3880527442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0897bf720 msgr2=0x55b0897c3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:08.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.429+0000 7f8f6d200700 1 --2- 172.21.15.131:0/3880527442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0897bf720 0x55b0897c3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.429+0000 7f8f6d200700 1 -- 172.21.15.131:0/3880527442 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f54010620 con 0x55b0897bee00 2024-05-08T22:09:08.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.430+0000 7f8f6d200700 1 --2- 172.21.15.131:0/3880527442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0897bee00 0x55b0897c30c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8f54005920 tx=0x7f8f54009f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f7c4e1815e63311 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:08.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.430+0000 7f8f4f400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:09:08.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.430+0000 7f8f4f400700 1 -- 172.21.15.131:0/3880527442 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5402d070 con 0x55b0897bee00 2024-05-08T22:09:08.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.430+0000 7f8f840ceb80 1 -- 172.21.15.131:0/3880527442 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b089662270 con 0x55b0897bee00 2024-05-08T22:09:08.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.430+0000 7f8f4f400700 1 -- 172.21.15.131:0/3880527442 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f8f54005020 con 0x55b0897bee00 2024-05-08T22:09:08.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.430+0000 7f8f4f400700 1 -- 172.21.15.131:0/3880527442 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f540104b0 con 0x55b0897bee00 2024-05-08T22:09:08.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.430+0000 7f8f840ceb80 10 client.25960 fetch_fsmap learned FSMap version 24 2024-05-08T22:09:08.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.430+0000 7f8f840ceb80 1 -- 172.21.15.131:0/3880527442 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b089536b60 con 0x55b0897bee00 2024-05-08T22:09:08.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.430+0000 7f8f4f400700 1 -- 172.21.15.131:0/3880527442 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b089662270 con 0x55b0897bee00 2024-05-08T22:09:08.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.431+0000 7f8f4f400700 1 -- 172.21.15.131:0/3880527442 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8f54030020 con 0x55b0897bee00 2024-05-08T22:09:08.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.431+0000 7f8f840ceb80 10 client.25960 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:09:08.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.431+0000 7f8f840ceb80 10 client.25960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:08.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.431+0000 7f8f840ceb80 20 client.25960 populate_metadata read hostname 'smithi131' 2024-05-08T22:09:08.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.431+0000 7f8f840ceb80 1 --2- 172.21.15.131:0/3880527442 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b0897d3050 0x55b0897d5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.431+0000 7f8f840ceb80 4 client.25960 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:09:08.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.431+0000 7f8f840ceb80 1 -- 172.21.15.131:0/3880527442 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b089788810 con 0x55b0897d3050 2024-05-08T22:09:08.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.431+0000 7f8f6dc00700 1 --2- 172.21.15.131:0/3880527442 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b0897d3050 0x55b0897d5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.432+0000 7f8f6dc00700 1 --2- 172.21.15.131:0/3880527442 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b0897d3050 0x55b0897d5470 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8574d1cfdb2428ea server_cookie=c0989fd045e8f82b in_seq=0 out_seq=0 2024-05-08T22:09:08.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.432+0000 7f8f4f400700 10 client.25960 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:08.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.432+0000 7f8f4f400700 1 -- 172.21.15.131:0/3880527442 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b089788810 con 0x55b0897d3050 2024-05-08T22:09:08.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.432+0000 7f8f4f400700 10 client.25960 handle_command_reply: tid=0 2024-05-08T22:09:08.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.432+0000 7f8f840ceb80 1 client.25960 shutdown 2024-05-08T22:09:08.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.433+0000 7f8f840ceb80 1 -- 172.21.15.131:0/3880527442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0897bee00 msgr2=0x55b0897c30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:08.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.433+0000 7f8f840ceb80 1 --2- 172.21.15.131:0/3880527442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0897bee00 0x55b0897c30c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8f54005920 tx=0x7f8f54009f10 comp rx=0 tx=0).stop 2024-05-08T22:09:08.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.433+0000 7f8f840ceb80 1 -- 172.21.15.131:0/3880527442 shutdown_connections 2024-05-08T22:09:08.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.433+0000 7f8f840ceb80 1 --2- 172.21.15.131:0/3880527442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0897bf720 0x55b0897c3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.433+0000 7f8f840ceb80 1 --2- 172.21.15.131:0/3880527442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897b9890 0x55b0897c2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.433+0000 7f8f840ceb80 1 --2- 172.21.15.131:0/3880527442 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b0897d3050 0x55b0897d5470 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.433+0000 7f8f840ceb80 1 --2- 172.21.15.131:0/3880527442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0897bee00 0x55b0897c30c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.433+0000 7f8f4f400700 0 client.25960 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:08.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.433+0000 7f8f840ceb80 1 -- 172.21.15.131:0/3880527442 >> 172.21.15.131:0/3880527442 conn(0x55b089717210 msgr2=0x55b0897b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:08.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.433+0000 7f8f840ceb80 1 -- 172.21.15.131:0/3880527442 shutdown_connections 2024-05-08T22:09:08.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.433+0000 7f8f840ceb80 1 -- 172.21.15.131:0/3880527442 wait complete. 2024-05-08T22:09:08.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.433+0000 7f8f840ceb80 20 client.25960 trim_cache size 0 max 16384 2024-05-08T22:09:08.442 INFO:tasks.workunit.client.0.smithi131.stdout:checking whether gcc needs -traditional... no 2024-05-08T22:09:08.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.447+0000 7f8f840ceb80 1 Processor -- start 2024-05-08T22:09:08.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.447+0000 7f8f840ceb80 1 -- start start 2024-05-08T22:09:08.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.447+0000 7f8f840ceb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897b9890 0x55b0897b9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.447+0000 7f8f840ceb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0897bee00 0x55b0897be350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.447+0000 7f8f840ceb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0897bf720 0x55b0897b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.447+0000 7f8f840ceb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0897be8d0 con 0x55b0897bf720 2024-05-08T22:09:08.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.447+0000 7f8f840ceb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0897c4220 con 0x55b0897b9890 2024-05-08T22:09:08.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.447+0000 7f8f840ceb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0897b5070 con 0x55b0897bee00 2024-05-08T22:09:08.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.448+0000 7f8f6dc00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897b9890 0x55b0897b9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.448+0000 7f8f6d200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0897bee00 0x55b0897be350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.448+0000 7f8f6dc00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897b9890 0x55b0897b9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:38390/0 (socket says 172.21.15.131:38390) 2024-05-08T22:09:08.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.448+0000 7f8f6dc00700 1 -- 172.21.15.131:0/2215804234 learned_addr learned my addr 172.21.15.131:0/2215804234 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:08.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.448+0000 7f8f6e600700 1 --2- 172.21.15.131:0/2215804234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0897bf720 0x55b0897b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.448+0000 7f8f6dc00700 1 -- 172.21.15.131:0/2215804234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0897bee00 msgr2=0x55b0897be350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:08.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.448+0000 7f8f6dc00700 1 --2- 172.21.15.131:0/2215804234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0897bee00 0x55b0897be350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.448+0000 7f8f6dc00700 1 -- 172.21.15.131:0/2215804234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0897bf720 msgr2=0x55b0897b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:08.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.448+0000 7f8f6dc00700 1 --2- 172.21.15.131:0/2215804234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0897bf720 0x55b0897b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.448+0000 7f8f6dc00700 1 -- 172.21.15.131:0/2215804234 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0896cf560 con 0x55b0897b9890 2024-05-08T22:09:08.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.448+0000 7f8f6dc00700 1 --2- 172.21.15.131:0/2215804234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897b9890 0x55b0897b9350 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c005950 tx=0x7f8f5c023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a1c4e530843796d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:08.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.448+0000 7f8f5b400700 1 -- 172.21.15.131:0/2215804234 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c004860 con 0x55b0897b9890 2024-05-08T22:09:08.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.449+0000 7f8f5b400700 1 -- 172.21.15.131:0/2215804234 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f5c01e040 con 0x55b0897b9890 2024-05-08T22:09:08.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.449+0000 7f8f5b400700 1 -- 172.21.15.131:0/2215804234 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c037800 con 0x55b0897b9890 2024-05-08T22:09:08.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.449+0000 7f8f840ceb80 1 -- 172.21.15.131:0/2215804234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897b9890 msgr2=0x55b0897b9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:08.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.449+0000 7f8f840ceb80 1 --2- 172.21.15.131:0/2215804234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897b9890 0x55b0897b9350 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c005950 tx=0x7f8f5c023000 comp rx=0 tx=0).stop 2024-05-08T22:09:08.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.450+0000 7f8f840ceb80 1 -- 172.21.15.131:0/2215804234 shutdown_connections 2024-05-08T22:09:08.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.450+0000 7f8f840ceb80 1 --2- 172.21.15.131:0/2215804234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0897bf720 0x55b0897b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.450+0000 7f8f840ceb80 1 --2- 172.21.15.131:0/2215804234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897b9890 0x55b0897b9350 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.450+0000 7f8f840ceb80 1 --2- 172.21.15.131:0/2215804234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0897bee00 0x55b0897be350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.450+0000 7f8f840ceb80 1 -- 172.21.15.131:0/2215804234 >> 172.21.15.131:0/2215804234 conn(0x55b089717210 msgr2=0x55b0897bdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:08.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.450+0000 7f8f840ceb80 1 -- 172.21.15.131:0/2215804234 shutdown_connections 2024-05-08T22:09:08.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.450+0000 7f8f840ceb80 1 -- 172.21.15.131:0/2215804234 wait complete. 2024-05-08T22:09:08.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.450+0000 7f8f840ceb80 1 -- start start 2024-05-08T22:09:08.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.450+0000 7f8f840ceb80 1 Processor -- start 2024-05-08T22:09:08.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.451+0000 7f8f840ceb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897b9890 0x55b0897c8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.451+0000 7f8f840ceb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0897bee00 0x55b0897c8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.451+0000 7f8f840ceb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0897bf720 0x55b0897c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.451+0000 7f8f840ceb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b089828630 con 0x55b0897bee00 2024-05-08T22:09:08.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.451+0000 7f8f840ceb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0898287a0 con 0x55b0897b9890 2024-05-08T22:09:08.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.451+0000 7f8f840ceb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b089828960 con 0x55b0897bf720 2024-05-08T22:09:08.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.451+0000 7f8f840ceb80 10 client.0 _ll_register_callbacks cb 0x55b0896aeb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:08.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.451+0000 7f8f6dc00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897b9890 0x55b0897c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.451+0000 7f8f6d200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0897bee00 0x55b0897c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.451+0000 7f8f6e600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0897bf720 0x55b0897c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.451+0000 7f8f6dc00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897b9890 0x55b0897c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:38392/0 (socket says 172.21.15.131:38392) 2024-05-08T22:09:08.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.451+0000 7f8f6dc00700 1 -- 172.21.15.131:0/3428748795 learned_addr learned my addr 172.21.15.131:0/3428748795 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:08.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.451+0000 7f8f6dc00700 1 -- 172.21.15.131:0/3428748795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0897bf720 msgr2=0x55b0897c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:08.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.451+0000 7f8f6dc00700 1 --2- 172.21.15.131:0/3428748795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0897bf720 0x55b0897c8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.451+0000 7f8f6dc00700 1 -- 172.21.15.131:0/3428748795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0897bee00 msgr2=0x55b0897c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:08.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.451+0000 7f8f6dc00700 1 --2- 172.21.15.131:0/3428748795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0897bee00 0x55b0897c8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.451+0000 7f8f6dc00700 1 -- 172.21.15.131:0/3428748795 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6810e6f0 con 0x55b0897b9890 2024-05-08T22:09:08.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.451+0000 7f8f6e600700 1 --2- 172.21.15.131:0/3428748795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0897bf720 0x55b0897c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:09:08.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.452+0000 7f8f6dc00700 1 --2- 172.21.15.131:0/3428748795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897b9890 0x55b0897c8150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c000c00 tx=0x7f8f5c0371f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52c383d478fb346b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:08.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.452+0000 7f8f4f400700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:09:08.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.452+0000 7f8f4f400700 1 -- 172.21.15.131:0/3428748795 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c037850 con 0x55b0897b9890 2024-05-08T22:09:08.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.452+0000 7f8f4f400700 1 -- 172.21.15.131:0/3428748795 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f8f5c004860 con 0x55b0897b9890 2024-05-08T22:09:08.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.452+0000 7f8f840ceb80 1 -- 172.21.15.131:0/3428748795 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f540054e0 con 0x55b0897b9890 2024-05-08T22:09:08.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.452+0000 7f8f4f400700 1 -- 172.21.15.131:0/3428748795 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f5c00be90 con 0x55b0897b9890 2024-05-08T22:09:08.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.452+0000 7f8f840ceb80 10 client.25952 fetch_fsmap learned FSMap version 24 2024-05-08T22:09:08.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.452+0000 7f8f840ceb80 1 -- 172.21.15.131:0/3428748795 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b08958ecd0 con 0x55b0897b9890 2024-05-08T22:09:08.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.452+0000 7f8f4f400700 1 -- 172.21.15.131:0/3428748795 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f540054e0 con 0x55b0897b9890 2024-05-08T22:09:08.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.453+0000 7f8f4f400700 1 -- 172.21.15.131:0/3428748795 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8f5c040070 con 0x55b0897b9890 2024-05-08T22:09:08.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.453+0000 7f8f840ceb80 10 client.25952 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:09:08.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.453+0000 7f8f840ceb80 10 client.25952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:08.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.453+0000 7f8f840ceb80 20 client.25952 populate_metadata read hostname 'smithi131' 2024-05-08T22:09:08.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.453+0000 7f8f840ceb80 1 --2- 172.21.15.131:0/3428748795 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b089829c80 0x55b08982c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:08.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.453+0000 7f8f840ceb80 4 client.25952 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:09:08.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.453+0000 7f8f840ceb80 1 -- 172.21.15.131:0/3428748795 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b089701ec0 con 0x55b089829c80 2024-05-08T22:09:08.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.453+0000 7f8f6d200700 1 --2- 172.21.15.131:0/3428748795 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b089829c80 0x55b08982c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:08.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.453+0000 7f8f6d200700 1 --2- 172.21.15.131:0/3428748795 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b089829c80 0x55b08982c0a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee56d753554d3c0 server_cookie=ff8c6dae3abbd835 in_seq=0 out_seq=0 2024-05-08T22:09:08.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.453+0000 7f8f4f400700 10 client.25952 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:08.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.454+0000 7f8f4f400700 1 -- 172.21.15.131:0/3428748795 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b089701ec0 con 0x55b089829c80 2024-05-08T22:09:08.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.454+0000 7f8f4f400700 10 client.25952 handle_command_reply: tid=0 2024-05-08T22:09:08.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.454+0000 7f8f840ceb80 1 client.25952 shutdown 2024-05-08T22:09:08.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.454+0000 7f8f840ceb80 1 -- 172.21.15.131:0/3428748795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897b9890 msgr2=0x55b0897c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:08.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.454+0000 7f8f840ceb80 1 --2- 172.21.15.131:0/3428748795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897b9890 0x55b0897c8150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c000c00 tx=0x7f8f5c0371f0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.454+0000 7f8f840ceb80 1 -- 172.21.15.131:0/3428748795 shutdown_connections 2024-05-08T22:09:08.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.454+0000 7f8f840ceb80 1 --2- 172.21.15.131:0/3428748795 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b089829c80 0x55b08982c0a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.454+0000 7f8f840ceb80 1 --2- 172.21.15.131:0/3428748795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0897bf720 0x55b0897c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.454+0000 7f8f840ceb80 1 --2- 172.21.15.131:0/3428748795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0897b9890 0x55b0897c8150 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.454+0000 7f8f4f400700 0 client.25952 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:08.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.454+0000 7f8f840ceb80 1 --2- 172.21.15.131:0/3428748795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b0897bee00 0x55b0897c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.454+0000 7f8f840ceb80 1 -- 172.21.15.131:0/3428748795 >> 172.21.15.131:0/3428748795 conn(0x55b089717210 msgr2=0x55b0897c1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:08.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.454+0000 7f8f840ceb80 1 -- 172.21.15.131:0/3428748795 shutdown_connections 2024-05-08T22:09:08.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.454+0000 7f8f840ceb80 1 -- 172.21.15.131:0/3428748795 wait complete. 2024-05-08T22:09:08.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.454+0000 7f8f840ceb80 20 client.25952 trim_cache size 0 max 16384 2024-05-08T22:09:08.460 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:09:08.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.455+0000 7f8f70600700 1 -- 172.21.15.131:0/3547683173 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8f500a1310 msgr2=0x7f8f500a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:08.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.455+0000 7f8f70600700 1 --2- 172.21.15.131:0/3547683173 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8f500a1310 0x7f8f500a37d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55b0897b5bd0 tx=0x7f8f5c00b260 comp rx=0 tx=0).stop 2024-05-08T22:09:08.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.455+0000 7f8f70600700 1 -- 172.21.15.131:0/3547683173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f681995c0 msgr2=0x7f8f68199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:08.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.455+0000 7f8f70600700 1 --2- 172.21.15.131:0/3547683173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f681995c0 0x7f8f68199a40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000efc0 tx=0x7f8f6000c3a0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.455+0000 7f8f70600700 1 -- 172.21.15.131:0/3547683173 shutdown_connections 2024-05-08T22:09:08.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.455+0000 7f8f70600700 1 --2- 172.21.15.131:0/3547683173 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8f500a1310 0x7f8f500a37d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.455+0000 7f8f70600700 1 --2- 172.21.15.131:0/3547683173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f681995c0 0x7f8f68199a40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.455+0000 7f8f70600700 1 --2- 172.21.15.131:0/3547683173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f681011e0 0x7f8f6819f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.455+0000 7f8f70600700 1 --2- 172.21.15.131:0/3547683173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f68106300 0x7f8f6819f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:08.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.455+0000 7f8f70600700 1 -- 172.21.15.131:0/3547683173 >> 172.21.15.131:0/3547683173 conn(0x7f8f680fca80 msgr2=0x7f8f680fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:08.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.455+0000 7f8f70600700 1 -- 172.21.15.131:0/3547683173 shutdown_connections 2024-05-08T22:09:08.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:08.455+0000 7f8f70600700 1 -- 172.21.15.131:0/3547683173 wait complete. 2024-05-08T22:09:08.518 INFO:tasks.workunit.client.0.smithi131.stdout:checking for an ANSI C-conforming const... yes 2024-05-08T22:09:08.565 INFO:tasks.workunit.client.0.smithi131.stdout:checking for inline... inline 2024-05-08T22:09:08.659 INFO:tasks.workunit.client.0.smithi131.stdout:checking for working alloca.h... yes 2024-05-08T22:09:08.742 INFO:tasks.workunit.client.0.smithi131.stdout:checking for alloca... yes 2024-05-08T22:09:08.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:08 smithi132 ceph-mon[78180]: pgmap v58: 129 pgs: 129 active+clean; 1.6 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-05-08T22:09:08.843 INFO:tasks.workunit.client.0.smithi131.stdout:checking for working memcmp... yes 2024-05-08T22:09:08.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:08 smithi131 ceph-mon[37260]: pgmap v58: 129 pgs: 129 active+clean; 1.6 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-05-08T22:09:08.938 INFO:tasks.workunit.client.0.smithi131.stdout:checking for strftime... yes 2024-05-08T22:09:09.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:08 smithi157 ceph-mon[77746]: pgmap v58: 129 pgs: 129 active+clean; 1.6 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-05-08T22:09:09.054 INFO:tasks.workunit.client.0.smithi131.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2024-05-08T22:09:09.151 INFO:tasks.workunit.client.0.smithi131.stdout:checking whether stat accepts an empty string... no 2024-05-08T22:09:09.259 INFO:tasks.workunit.client.0.smithi131.stdout:checking for vprintf... yes 2024-05-08T22:09:09.355 INFO:tasks.workunit.client.0.smithi131.stdout:checking for _doprnt... no 2024-05-08T22:09:09.410 INFO:tasks.workunit.client.0.smithi131.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-05-08T22:09:09.442 INFO:tasks.workunit.client.0.smithi131.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-05-08T22:09:09.477 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T22:09:09.477 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T22:09:09.558 INFO:tasks.workunit.client.0.smithi131.stdout:checking for the pthreads library -lpthreads... no 2024-05-08T22:09:09.645 INFO:tasks.workunit.client.0.smithi131.stdout:checking whether pthreads work without any flags... no 2024-05-08T22:09:09.674 INFO:tasks.workunit.client.0.smithi131.stdout:checking whether pthreads work with -Kthread... no 2024-05-08T22:09:09.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.683+0000 7f7517a00700 1 -- 172.21.15.131:0/3476927446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7510073c60 msgr2=0x7f75101119c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:09.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.683+0000 7f7517a00700 1 --2- 172.21.15.131:0/3476927446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7510073c60 0x7f75101119c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009990 tx=0x7f74fc009ca0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.683+0000 7f7517a00700 1 -- 172.21.15.131:0/3476927446 shutdown_connections 2024-05-08T22:09:09.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.683+0000 7f7517a00700 1 --2- 172.21.15.131:0/3476927446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7510111f00 0x7f75101142f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.683+0000 7f7517a00700 1 --2- 172.21.15.131:0/3476927446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7510073340 0x7f7510073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.683+0000 7f7517a00700 1 --2- 172.21.15.131:0/3476927446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7510073c60 0x7f75101119c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.683+0000 7f7517a00700 1 -- 172.21.15.131:0/3476927446 >> 172.21.15.131:0/3476927446 conn(0x7f75100fc9c0 msgr2=0x7f75100fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:09.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.683+0000 7f7517a00700 1 -- 172.21.15.131:0/3476927446 shutdown_connections 2024-05-08T22:09:09.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.684+0000 7f7517a00700 1 -- 172.21.15.131:0/3476927446 wait complete. 2024-05-08T22:09:09.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.685+0000 7f7517a00700 1 Processor -- start 2024-05-08T22:09:09.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.685+0000 7f7517a00700 1 -- start start 2024-05-08T22:09:09.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.686+0000 7f7517a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7510073340 0x7f75101a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.686+0000 7f7517a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7510073c60 0x7f75101a3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.686+0000 7f7517a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7510111f00 0x7f751019d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.686+0000 7f7517a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751019df80 con 0x7f7510073c60 2024-05-08T22:09:09.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.686+0000 7f7517a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751019e0f0 con 0x7f7510073340 2024-05-08T22:09:09.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.686+0000 7f7517a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751019e2b0 con 0x7f7510111f00 2024-05-08T22:09:09.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.686+0000 7f750fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7510073c60 0x7f75101a3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.686+0000 7f7515000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7510073340 0x7f75101a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.686+0000 7f7515a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7510111f00 0x7f751019d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.686+0000 7f750fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7510073c60 0x7f75101a3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51932/0 (socket says 172.21.15.131:51932) 2024-05-08T22:09:09.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.686+0000 7f750fe00700 1 -- 172.21.15.131:0/2182530866 learned_addr learned my addr 172.21.15.131:0/2182530866 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:09.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.686+0000 7f7515000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7510073340 0x7f75101a37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:38412/0 (socket says 172.21.15.131:38412) 2024-05-08T22:09:09.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.687+0000 7f7515000700 1 -- 172.21.15.131:0/2182530866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7510111f00 msgr2=0x7f751019d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:09.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.687+0000 7f7515000700 1 --2- 172.21.15.131:0/2182530866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7510111f00 0x7f751019d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.687+0000 7f7515000700 1 -- 172.21.15.131:0/2182530866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7510073c60 msgr2=0x7f75101a3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:09.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.687+0000 7f7515000700 1 --2- 172.21.15.131:0/2182530866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7510073c60 0x7f75101a3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.687+0000 7f7515000700 1 -- 172.21.15.131:0/2182530866 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7508009c40 con 0x7f7510073340 2024-05-08T22:09:09.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.687+0000 7f7515000700 1 --2- 172.21.15.131:0/2182530866 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7510073340 0x7f75101a37a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f750400e9f0 tx=0x7f750400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:09.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.687+0000 7f750d600700 1 -- 172.21.15.131:0/2182530866 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75040096b0 con 0x7f7510073340 2024-05-08T22:09:09.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.687+0000 7f750d600700 1 -- 172.21.15.131:0/2182530866 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7504004cf0 con 0x7f7510073340 2024-05-08T22:09:09.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.687+0000 7f750d600700 1 -- 172.21.15.131:0/2182530866 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750400cdb0 con 0x7f7510073340 2024-05-08T22:09:09.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.687+0000 7f7517a00700 1 -- 172.21.15.131:0/2182530866 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74fc0095f0 con 0x7f7510073340 2024-05-08T22:09:09.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.687+0000 7f7517a00700 1 -- 172.21.15.131:0/2182530866 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75101aa5b0 con 0x7f7510073340 2024-05-08T22:09:09.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.691+0000 7f750d600700 1 -- 172.21.15.131:0/2182530866 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f7504004320 con 0x7f7510073340 2024-05-08T22:09:09.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.692+0000 7f752b582b80 1 Processor -- start 2024-05-08T22:09:09.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.692+0000 7f750d600700 1 --2- 172.21.15.131:0/2182530866 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f74f80a1310 0x7f74f80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.692+0000 7f752b582b80 1 -- start start 2024-05-08T22:09:09.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.692+0000 7f750d600700 1 -- 172.21.15.131:0/2182530866 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f7504014070 con 0x7f7510073340 2024-05-08T22:09:09.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.692+0000 7f7515a00700 1 --2- 172.21.15.131:0/2182530866 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f74f80a1310 0x7f74f80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.693+0000 7f752b582b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6a452ae80 0x55e6a452b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.693+0000 7f752b582b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e6a452b7a0 0x55e6a45253d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.693+0000 7f752b582b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6a4525910 0x55e6a4525d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.693+0000 7f752b582b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a447c1d0 con 0x55e6a452ae80 2024-05-08T22:09:09.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.693+0000 7f752b582b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a4479b90 con 0x55e6a452b7a0 2024-05-08T22:09:09.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.693+0000 7f752b582b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a4475480 con 0x55e6a4525910 2024-05-08T22:09:09.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.693+0000 7f750fe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e6a452b7a0 0x55e6a45253d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.693+0000 7f750fe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e6a452b7a0 0x55e6a45253d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:38416/0 (socket says 172.21.15.131:38416) 2024-05-08T22:09:09.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.693+0000 7f750fe00700 1 -- 172.21.15.131:0/1127150430 learned_addr learned my addr 172.21.15.131:0/1127150430 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:09.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.693+0000 7f7515000700 1 --2- 172.21.15.131:0/1127150430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6a452ae80 0x55e6a452b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.693+0000 7f7515a00700 1 --2- 172.21.15.131:0/2182530866 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f74f80a1310 0x7f74f80a37d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7508009fd0 tx=0x7f750800b040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:09.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.693+0000 7f7515a00700 1 --2- 172.21.15.131:0/1127150430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6a4525910 0x55e6a4525d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.693+0000 7f750fe00700 1 -- 172.21.15.131:0/1127150430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6a4525910 msgr2=0x55e6a4525d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:09.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.693+0000 7f750fe00700 1 --2- 172.21.15.131:0/1127150430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6a4525910 0x55e6a4525d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.693+0000 7f750fe00700 1 -- 172.21.15.131:0/1127150430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6a452ae80 msgr2=0x55e6a452b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:09.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.693+0000 7f750fe00700 1 --2- 172.21.15.131:0/1127150430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6a452ae80 0x55e6a452b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.693+0000 7f750fe00700 1 -- 172.21.15.131:0/1127150430 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6a42f6430 con 0x55e6a452b7a0 2024-05-08T22:09:09.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.693+0000 7f7515a00700 1 --2- 172.21.15.131:0/1127150430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6a4525910 0x55e6a4525d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:09:09.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.694+0000 7f750fe00700 1 --2- 172.21.15.131:0/1127150430 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e6a452b7a0 0x55e6a45253d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f74fc005f50 tx=0x7f74fc00b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaad3c867c0f0ea4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.694+0000 7f7502000700 1 -- 172.21.15.131:0/1127150430 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74fc01d070 con 0x55e6a452b7a0 2024-05-08T22:09:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.694+0000 7f7502000700 1 -- 172.21.15.131:0/1127150430 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74fc01e040 con 0x55e6a452b7a0 2024-05-08T22:09:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.694+0000 7f7502000700 1 -- 172.21.15.131:0/1127150430 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74fc004920 con 0x55e6a452b7a0 2024-05-08T22:09:09.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.695+0000 7f752b582b80 1 -- 172.21.15.131:0/1127150430 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e6a452b7a0 msgr2=0x55e6a45253d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:09.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.695+0000 7f752b582b80 1 --2- 172.21.15.131:0/1127150430 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e6a452b7a0 0x55e6a45253d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f74fc005f50 tx=0x7f74fc00b610 comp rx=0 tx=0).stop 2024-05-08T22:09:09.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.695+0000 7f752b582b80 1 -- 172.21.15.131:0/1127150430 shutdown_connections 2024-05-08T22:09:09.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.695+0000 7f752b582b80 1 --2- 172.21.15.131:0/1127150430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6a4525910 0x55e6a4525d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.695+0000 7f752b582b80 1 --2- 172.21.15.131:0/1127150430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6a452ae80 0x55e6a452b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.695+0000 7f752b582b80 1 --2- 172.21.15.131:0/1127150430 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e6a452b7a0 0x55e6a45253d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.695+0000 7f752b582b80 1 -- 172.21.15.131:0/1127150430 >> 172.21.15.131:0/1127150430 conn(0x55e6a4483290 msgr2=0x55e6a451fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:09.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.695+0000 7f752b582b80 1 -- 172.21.15.131:0/1127150430 shutdown_connections 2024-05-08T22:09:09.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.695+0000 7f752b582b80 1 -- 172.21.15.131:0/1127150430 wait complete. 2024-05-08T22:09:09.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.696+0000 7f752b582b80 1 -- start start 2024-05-08T22:09:09.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.696+0000 7f752b582b80 1 Processor -- start 2024-05-08T22:09:09.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.696+0000 7f752b582b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e6a4525910 0x55e6a452eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.696+0000 7f752b582b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6a452ae80 0x55e6a452f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.696+0000 7f752b582b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6a452b7a0 0x55e6a452f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.696+0000 7f752b582b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a44115e0 con 0x55e6a452ae80 2024-05-08T22:09:09.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.696+0000 7f752b582b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a4411280 con 0x55e6a4525910 2024-05-08T22:09:09.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.696+0000 7f752b582b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a42f8720 con 0x55e6a452b7a0 2024-05-08T22:09:09.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.696+0000 7f752b582b80 10 client.0 _ll_register_callbacks cb 0x55e6a441abb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:09.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.697+0000 7f750fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6a452ae80 0x55e6a452f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.697+0000 7f750fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6a452ae80 0x55e6a452f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51956/0 (socket says 172.21.15.131:51956) 2024-05-08T22:09:09.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.697+0000 7f750fe00700 1 -- 172.21.15.131:0/2146519556 learned_addr learned my addr 172.21.15.131:0/2146519556 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:09.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.697+0000 7f7515a00700 1 --2- 172.21.15.131:0/2146519556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6a452b7a0 0x55e6a452f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.697+0000 7f7515000700 1 --2- 172.21.15.131:0/2146519556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e6a4525910 0x55e6a452eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.697+0000 7f750fe00700 1 -- 172.21.15.131:0/2146519556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6a452b7a0 msgr2=0x55e6a452f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:09.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.697+0000 7f750fe00700 1 --2- 172.21.15.131:0/2146519556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6a452b7a0 0x55e6a452f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.697+0000 7f750fe00700 1 -- 172.21.15.131:0/2146519556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e6a4525910 msgr2=0x55e6a452eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:09.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.697+0000 7f750fe00700 1 --2- 172.21.15.131:0/2146519556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e6a4525910 0x55e6a452eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.697+0000 7f750fe00700 1 -- 172.21.15.131:0/2146519556 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7510114ca0 con 0x55e6a452ae80 2024-05-08T22:09:09.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.697+0000 7f750fe00700 1 --2- 172.21.15.131:0/2146519556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6a452ae80 0x55e6a452f030 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f74fc005f20 tx=0x7f74fc00b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c53bd790a3ed44 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:09.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.697+0000 7f74f6a00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:09:09.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.698+0000 7f74f6a00700 1 -- 172.21.15.131:0/2146519556 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74fc032070 con 0x55e6a452ae80 2024-05-08T22:09:09.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.698+0000 7f74f6a00700 1 -- 172.21.15.131:0/2146519556 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f74fc004500 con 0x55e6a452ae80 2024-05-08T22:09:09.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.698+0000 7f752b582b80 1 -- 172.21.15.131:0/2146519556 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6a43ce2f0 con 0x55e6a452ae80 2024-05-08T22:09:09.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.698+0000 7f74f6a00700 1 -- 172.21.15.131:0/2146519556 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74fc01d070 con 0x55e6a452ae80 2024-05-08T22:09:09.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.698+0000 7f752b582b80 10 client.15912 fetch_fsmap learned FSMap version 24 2024-05-08T22:09:09.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.698+0000 7f752b582b80 1 -- 172.21.15.131:0/2146519556 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e6a42a2be0 con 0x55e6a452ae80 2024-05-08T22:09:09.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.698+0000 7f74f6a00700 1 -- 172.21.15.131:0/2146519556 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e6a43ce2f0 con 0x55e6a452ae80 2024-05-08T22:09:09.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.698+0000 7f74f6a00700 1 -- 172.21.15.131:0/2146519556 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f74fc035020 con 0x55e6a452ae80 2024-05-08T22:09:09.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.699+0000 7f752b582b80 10 client.15912 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:09:09.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.699+0000 7f752b582b80 10 client.15912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:09.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.699+0000 7f752b582b80 20 client.15912 populate_metadata read hostname 'smithi131' 2024-05-08T22:09:09.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.699+0000 7f752b582b80 1 --2- 172.21.15.131:0/2146519556 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55e6a453ed40 0x55e6a4541130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.699+0000 7f752b582b80 4 client.15912 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:09:09.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.699+0000 7f752b582b80 1 -- 172.21.15.131:0/2146519556 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e6a44f4890 con 0x55e6a453ed40 2024-05-08T22:09:09.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.699+0000 7f7515000700 1 --2- 172.21.15.131:0/2146519556 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55e6a453ed40 0x55e6a4541130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.700+0000 7f74f6a00700 10 client.15912 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:09.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.700+0000 7f7515000700 1 --2- 172.21.15.131:0/2146519556 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55e6a453ed40 0x55e6a4541130 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c4648c0238df4fd server_cookie=7f553dcd6d45d0c2 in_seq=0 out_seq=0 2024-05-08T22:09:09.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.700+0000 7f74f6a00700 1 -- 172.21.15.131:0/2146519556 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e6a44f4890 con 0x55e6a453ed40 2024-05-08T22:09:09.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.700+0000 7f74f6a00700 10 client.15912 handle_command_reply: tid=0 2024-05-08T22:09:09.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.700+0000 7f752b582b80 1 client.15912 shutdown 2024-05-08T22:09:09.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.700+0000 7f752b582b80 1 -- 172.21.15.131:0/2146519556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6a452ae80 msgr2=0x55e6a452f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:09.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.700+0000 7f752b582b80 1 --2- 172.21.15.131:0/2146519556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6a452ae80 0x55e6a452f030 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f74fc005f20 tx=0x7f74fc00b980 comp rx=0 tx=0).stop 2024-05-08T22:09:09.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.700+0000 7f752b582b80 1 -- 172.21.15.131:0/2146519556 shutdown_connections 2024-05-08T22:09:09.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.700+0000 7f752b582b80 1 --2- 172.21.15.131:0/2146519556 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55e6a453ed40 0x55e6a4541130 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.700+0000 7f752b582b80 1 --2- 172.21.15.131:0/2146519556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6a452b7a0 0x55e6a452f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.700+0000 7f74f6a00700 0 client.15912 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:09.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.700+0000 7f752b582b80 1 --2- 172.21.15.131:0/2146519556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e6a4525910 0x55e6a452eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.700+0000 7f752b582b80 1 --2- 172.21.15.131:0/2146519556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6a452ae80 0x55e6a452f030 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.700+0000 7f752b582b80 1 -- 172.21.15.131:0/2146519556 >> 172.21.15.131:0/2146519556 conn(0x55e6a4483290 msgr2=0x55e6a451f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:09.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.701+0000 7f752b582b80 1 -- 172.21.15.131:0/2146519556 shutdown_connections 2024-05-08T22:09:09.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.701+0000 7f752b582b80 1 -- 172.21.15.131:0/2146519556 wait complete. 2024-05-08T22:09:09.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.701+0000 7f752b582b80 20 client.15912 trim_cache size 0 max 16384 2024-05-08T22:09:09.706 INFO:tasks.workunit.client.0.smithi131.stdout:checking whether pthreads work with -kthread... no 2024-05-08T22:09:09.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.719+0000 7f752b582b80 1 Processor -- start 2024-05-08T22:09:09.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.719+0000 7f752b582b80 1 -- start start 2024-05-08T22:09:09.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.719+0000 7f752b582b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6a4525910 0x55e6a452a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.719+0000 7f752b582b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6a452ae80 0x55e6a451fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.720+0000 7f752b582b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e6a452b7a0 0x55e6a452f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.720+0000 7f752b582b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a4521730 con 0x55e6a452ae80 2024-05-08T22:09:09.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.720+0000 7f752b582b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a4546370 con 0x55e6a452b7a0 2024-05-08T22:09:09.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.720+0000 7f752b582b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a4521b20 con 0x55e6a4525910 2024-05-08T22:09:09.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.720+0000 7f750fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6a452ae80 0x55e6a451fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.720+0000 7f7515000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6a4525910 0x55e6a452a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.720+0000 7f7515000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6a4525910 0x55e6a452a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60752/0 (socket says 172.21.15.131:60752) 2024-05-08T22:09:09.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.720+0000 7f7515a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e6a452b7a0 0x55e6a452f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.720+0000 7f7515000700 1 -- 172.21.15.131:0/377286551 learned_addr learned my addr 172.21.15.131:0/377286551 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:09.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.720+0000 7f750fe00700 1 --2- 172.21.15.131:0/377286551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6a452ae80 0x55e6a451fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51962/0 (socket says 172.21.15.131:51962) 2024-05-08T22:09:09.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.720+0000 7f7515000700 1 -- 172.21.15.131:0/377286551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e6a452b7a0 msgr2=0x55e6a452f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:09.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.720+0000 7f7515000700 1 --2- 172.21.15.131:0/377286551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e6a452b7a0 0x55e6a452f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.720+0000 7f7515000700 1 -- 172.21.15.131:0/377286551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6a452ae80 msgr2=0x55e6a451fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:09.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.720+0000 7f7515000700 1 --2- 172.21.15.131:0/377286551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6a452ae80 0x55e6a451fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.720+0000 7f7515000700 1 -- 172.21.15.131:0/377286551 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6a451c560 con 0x55e6a4525910 2024-05-08T22:09:09.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.720+0000 7f7515a00700 1 --2- 172.21.15.131:0/377286551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e6a452b7a0 0x55e6a452f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:09:09.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.720+0000 7f7515000700 1 --2- 172.21.15.131:0/377286551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6a4525910 0x55e6a452a330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7504099df0 tx=0x7f7504005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7af7f6ae6f26621 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:09.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.721+0000 7f7502a00700 1 -- 172.21.15.131:0/377286551 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750409a560 con 0x55e6a4525910 2024-05-08T22:09:09.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.721+0000 7f7502a00700 1 -- 172.21.15.131:0/377286551 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75040b1070 con 0x55e6a4525910 2024-05-08T22:09:09.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.721+0000 7f7502a00700 1 -- 172.21.15.131:0/377286551 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750409a320 con 0x55e6a4525910 2024-05-08T22:09:09.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.721+0000 7f752b582b80 1 -- 172.21.15.131:0/377286551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6a4525910 msgr2=0x55e6a452a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:09.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.721+0000 7f752b582b80 1 --2- 172.21.15.131:0/377286551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6a4525910 0x55e6a452a330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7504099df0 tx=0x7f7504005b20 comp rx=0 tx=0).stop 2024-05-08T22:09:09.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.722+0000 7f752b582b80 1 -- 172.21.15.131:0/377286551 shutdown_connections 2024-05-08T22:09:09.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.722+0000 7f752b582b80 1 --2- 172.21.15.131:0/377286551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e6a452b7a0 0x55e6a452f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.722+0000 7f752b582b80 1 --2- 172.21.15.131:0/377286551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6a4525910 0x55e6a452a330 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.722+0000 7f752b582b80 1 --2- 172.21.15.131:0/377286551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6a452ae80 0x55e6a451fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.722+0000 7f752b582b80 1 -- 172.21.15.131:0/377286551 >> 172.21.15.131:0/377286551 conn(0x55e6a4483290 msgr2=0x55e6a4529c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:09.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.722+0000 7f752b582b80 1 -- 172.21.15.131:0/377286551 shutdown_connections 2024-05-08T22:09:09.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.722+0000 7f752b582b80 1 -- 172.21.15.131:0/377286551 wait complete. 2024-05-08T22:09:09.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.722+0000 7f752b582b80 1 -- start start 2024-05-08T22:09:09.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.722+0000 7f752b582b80 1 Processor -- start 2024-05-08T22:09:09.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.723+0000 7f752b582b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6a4525910 0x55e6a4535780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.723+0000 7f752b582b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e6a452ae80 0x55e6a4535cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.723+0000 7f752b582b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6a452b7a0 0x55e6a458e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.723+0000 7f752b582b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a458e720 con 0x55e6a452b7a0 2024-05-08T22:09:09.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.723+0000 7f752b582b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a458e890 con 0x55e6a452ae80 2024-05-08T22:09:09.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.723+0000 7f752b582b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a458ea50 con 0x55e6a4525910 2024-05-08T22:09:09.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.723+0000 7f752b582b80 10 client.0 _ll_register_callbacks cb 0x55e6a441abb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:09.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.723+0000 7f7515000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6a4525910 0x55e6a4535780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.723+0000 7f7515a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6a452b7a0 0x55e6a458e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.723+0000 7f750fe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e6a452ae80 0x55e6a4535cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.723+0000 7f7515000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6a4525910 0x55e6a4535780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60758/0 (socket says 172.21.15.131:60758) 2024-05-08T22:09:09.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.723+0000 7f750fe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e6a452ae80 0x55e6a4535cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:38440/0 (socket says 172.21.15.131:38440) 2024-05-08T22:09:09.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.723+0000 7f7515000700 1 -- 172.21.15.131:0/226213027 learned_addr learned my addr 172.21.15.131:0/226213027 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:09.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.723+0000 7f750fe00700 1 -- 172.21.15.131:0/226213027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6a4525910 msgr2=0x55e6a4535780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:09.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.723+0000 7f750fe00700 1 --2- 172.21.15.131:0/226213027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6a4525910 0x55e6a4535780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.724+0000 7f750fe00700 1 -- 172.21.15.131:0/226213027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6a452b7a0 msgr2=0x55e6a458e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:09.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.724+0000 7f750fe00700 1 --2- 172.21.15.131:0/226213027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6a452b7a0 0x55e6a458e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.724+0000 7f750fe00700 1 -- 172.21.15.131:0/226213027 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7508003bd0 con 0x55e6a452ae80 2024-05-08T22:09:09.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.724+0000 7f750fe00700 1 --2- 172.21.15.131:0/226213027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e6a452ae80 0x55e6a4535cc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009ce0 tx=0x7f74fc02ffe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b49bf73e849d3a04 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:09.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.724+0000 7f74f6a00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:09:09.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.724+0000 7f74f6a00700 1 -- 172.21.15.131:0/226213027 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74fc032030 con 0x55e6a452ae80 2024-05-08T22:09:09.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.724+0000 7f74f6a00700 1 -- 172.21.15.131:0/226213027 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f74fc033070 con 0x55e6a452ae80 2024-05-08T22:09:09.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.724+0000 7f752b582b80 1 -- 172.21.15.131:0/226213027 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74fc01e040 con 0x55e6a452ae80 2024-05-08T22:09:09.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.724+0000 7f74f6a00700 1 -- 172.21.15.131:0/226213027 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74fc01d040 con 0x55e6a452ae80 2024-05-08T22:09:09.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.724+0000 7f752b582b80 10 client.25970 fetch_fsmap learned FSMap version 24 2024-05-08T22:09:09.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.724+0000 7f752b582b80 1 -- 172.21.15.131:0/226213027 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f750409ae70 con 0x55e6a452ae80 2024-05-08T22:09:09.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.724+0000 7f74f6a00700 1 -- 172.21.15.131:0/226213027 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74fc01e040 con 0x55e6a452ae80 2024-05-08T22:09:09.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.725+0000 7f74f6a00700 1 -- 172.21.15.131:0/226213027 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f74fc005040 con 0x55e6a452ae80 2024-05-08T22:09:09.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.725+0000 7f752b582b80 10 client.25970 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:09:09.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.725+0000 7f752b582b80 10 client.25970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:09.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.725+0000 7f752b582b80 20 client.25970 populate_metadata read hostname 'smithi131' 2024-05-08T22:09:09.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.725+0000 7f752b582b80 1 --2- 172.21.15.131:0/226213027 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55e6a4597090 0x55e6a45994b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.725+0000 7f752b582b80 4 client.25970 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T22:09:09.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.725+0000 7f752b582b80 1 -- 172.21.15.131:0/226213027 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e6a446df40 con 0x55e6a4597090 2024-05-08T22:09:09.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.725+0000 7f7515000700 1 --2- 172.21.15.131:0/226213027 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55e6a4597090 0x55e6a45994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.725+0000 7f7515000700 1 --2- 172.21.15.131:0/226213027 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55e6a4597090 0x55e6a45994b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=348004572f3ac0f2 server_cookie=c4afcb774ff20fa3 in_seq=0 out_seq=0 2024-05-08T22:09:09.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.725+0000 7f74f6a00700 10 client.25970 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:09.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.726+0000 7f74f6a00700 1 -- 172.21.15.131:0/226213027 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e6a446df40 con 0x55e6a4597090 2024-05-08T22:09:09.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.726+0000 7f74f6a00700 10 client.25970 handle_command_reply: tid=0 2024-05-08T22:09:09.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.726+0000 7f752b582b80 1 client.25970 shutdown 2024-05-08T22:09:09.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.727+0000 7f752b582b80 1 -- 172.21.15.131:0/226213027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e6a452ae80 msgr2=0x55e6a4535cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:09.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.727+0000 7f752b582b80 1 --2- 172.21.15.131:0/226213027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e6a452ae80 0x55e6a4535cc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009ce0 tx=0x7f74fc02ffe0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.727+0000 7f752b582b80 1 -- 172.21.15.131:0/226213027 shutdown_connections 2024-05-08T22:09:09.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.727+0000 7f752b582b80 1 --2- 172.21.15.131:0/226213027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6a452b7a0 0x55e6a458e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.727+0000 7f752b582b80 1 --2- 172.21.15.131:0/226213027 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55e6a4597090 0x55e6a45994b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.727+0000 7f752b582b80 1 --2- 172.21.15.131:0/226213027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6a4525910 0x55e6a4535780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.727+0000 7f752b582b80 1 --2- 172.21.15.131:0/226213027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e6a452ae80 0x55e6a4535cc0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.727+0000 7f752b582b80 1 -- 172.21.15.131:0/226213027 >> 172.21.15.131:0/226213027 conn(0x55e6a4483290 msgr2=0x55e6a4530390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:09.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.727+0000 7f74f6a00700 0 client.25970 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:09.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.727+0000 7f752b582b80 1 -- 172.21.15.131:0/226213027 shutdown_connections 2024-05-08T22:09:09.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.727+0000 7f752b582b80 1 -- 172.21.15.131:0/226213027 wait complete. 2024-05-08T22:09:09.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.727+0000 7f752b582b80 20 client.25970 trim_cache size 0 max 16384 2024-05-08T22:09:09.732 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:09:09.732 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-08T22:09:09.732 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "479b4292-010e-4bdd-9584-940c0e0b36b4", 2024-05-08T22:09:09.732 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-08T22:09:09.732 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:09:09.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.728+0000 7f7517a00700 1 -- 172.21.15.131:0/2182530866 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f74f80a1310 msgr2=0x7f74f80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:09.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.728+0000 7f7517a00700 1 --2- 172.21.15.131:0/2182530866 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f74f80a1310 0x7f74f80a37d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7508009fd0 tx=0x7f750800b040 comp rx=0 tx=0).stop 2024-05-08T22:09:09.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.728+0000 7f7517a00700 1 -- 172.21.15.131:0/2182530866 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7510073340 msgr2=0x7f75101a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:09.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.728+0000 7f7517a00700 1 --2- 172.21.15.131:0/2182530866 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7510073340 0x7f75101a37a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f750400e9f0 tx=0x7f750400ed00 comp rx=0 tx=0).stop 2024-05-08T22:09:09.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.728+0000 7f7517a00700 1 -- 172.21.15.131:0/2182530866 shutdown_connections 2024-05-08T22:09:09.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.728+0000 7f7517a00700 1 --2- 172.21.15.131:0/2182530866 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f74f80a1310 0x7f74f80a37d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.728+0000 7f7517a00700 1 --2- 172.21.15.131:0/2182530866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7510111f00 0x7f751019d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.728+0000 7f7517a00700 1 --2- 172.21.15.131:0/2182530866 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7510073340 0x7f75101a37a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.728+0000 7f7517a00700 1 --2- 172.21.15.131:0/2182530866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7510073c60 0x7f75101a3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.728+0000 7f7517a00700 1 -- 172.21.15.131:0/2182530866 >> 172.21.15.131:0/2182530866 conn(0x7f75100fc9c0 msgr2=0x7f75100fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:09.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.728+0000 7f7517a00700 1 -- 172.21.15.131:0/2182530866 shutdown_connections 2024-05-08T22:09:09.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.728+0000 7f7517a00700 1 -- 172.21.15.131:0/2182530866 wait complete. 2024-05-08T22:09:09.749 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:09:09.805 INFO:tasks.workunit.client.0.smithi131.stdout:checking for the pthreads library -llthread... no 2024-05-08T22:09:09.896 INFO:tasks.workunit.client.0.smithi131.stdout:checking whether pthreads work with -pthread... yes 2024-05-08T22:09:09.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.952+0000 7faac2400700 1 -- 172.21.15.131:0/1393533837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faabc073cf0 msgr2=0x7faabc10d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:09.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.952+0000 7faac2400700 1 --2- 172.21.15.131:0/1393533837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faabc073cf0 0x7faabc10d210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faaa8009990 tx=0x7faaa8009ca0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.952+0000 7faac2400700 1 -- 172.21.15.131:0/1393533837 shutdown_connections 2024-05-08T22:09:09.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.952+0000 7faac2400700 1 --2- 172.21.15.131:0/1393533837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faabc10d750 0x7faabc10fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.952+0000 7faac2400700 1 --2- 172.21.15.131:0/1393533837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faabc0733d0 0x7faabc0737b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.952+0000 7faac2400700 1 --2- 172.21.15.131:0/1393533837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faabc073cf0 0x7faabc10d210 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.952+0000 7faac2400700 1 -- 172.21.15.131:0/1393533837 >> 172.21.15.131:0/1393533837 conn(0x7faabc0fc9e0 msgr2=0x7faabc0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:09.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.952+0000 7faac2400700 1 -- 172.21.15.131:0/1393533837 shutdown_connections 2024-05-08T22:09:09.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.952+0000 7faac2400700 1 -- 172.21.15.131:0/1393533837 wait complete. 2024-05-08T22:09:09.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.953+0000 7faac2400700 1 Processor -- start 2024-05-08T22:09:09.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.953+0000 7faac2400700 1 -- start start 2024-05-08T22:09:09.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.954+0000 7faac2400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faabc0733d0 0x7faabc199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.954+0000 7faac2400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faabc073cf0 0x7faabc199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.954+0000 7faac2400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faabc10d750 0x7faabc19df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.954+0000 7faac2400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc19e570 con 0x7faabc073cf0 2024-05-08T22:09:09.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.954+0000 7faac2400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc19e6e0 con 0x7faabc0733d0 2024-05-08T22:09:09.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.954+0000 7faac2400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc19e8a0 con 0x7faabc10d750 2024-05-08T22:09:09.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.954+0000 7faabb400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faabc0733d0 0x7faabc199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.954+0000 7faabb400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faabc0733d0 0x7faabc199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:38450/0 (socket says 172.21.15.131:38450) 2024-05-08T22:09:09.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.954+0000 7faabb400700 1 -- 172.21.15.131:0/3249863570 learned_addr learned my addr 172.21.15.131:0/3249863570 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:09.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.954+0000 7faabbe00700 1 --2- 172.21.15.131:0/3249863570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faabc10d750 0x7faabc19df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.954+0000 7faabaa00700 1 --2- 172.21.15.131:0/3249863570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faabc073cf0 0x7faabc199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.954+0000 7faabb400700 1 -- 172.21.15.131:0/3249863570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faabc10d750 msgr2=0x7faabc19df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:09.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.954+0000 7faabb400700 1 --2- 172.21.15.131:0/3249863570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faabc10d750 0x7faabc19df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.954+0000 7faabb400700 1 -- 172.21.15.131:0/3249863570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faabc073cf0 msgr2=0x7faabc199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:09.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.954+0000 7faabb400700 1 --2- 172.21.15.131:0/3249863570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faabc073cf0 0x7faabc199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.954+0000 7faabb400700 1 -- 172.21.15.131:0/3249863570 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa80095f0 con 0x7faabc0733d0 2024-05-08T22:09:09.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.954+0000 7faabaa00700 1 --2- 172.21.15.131:0/3249863570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faabc073cf0 0x7faabc199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:09:09.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.955+0000 7faabb400700 1 --2- 172.21.15.131:0/3249863570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faabc0733d0 0x7faabc199650 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faaac00e9c0 tx=0x7faaac00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:09.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.955+0000 7faab3e00700 1 -- 172.21.15.131:0/3249863570 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac0097b0 con 0x7faabc0733d0 2024-05-08T22:09:09.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.955+0000 7faac2400700 1 -- 172.21.15.131:0/3249863570 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faabc19eb80 con 0x7faabc0733d0 2024-05-08T22:09:09.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.955+0000 7faab3e00700 1 -- 172.21.15.131:0/3249863570 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faaac004cf0 con 0x7faabc0733d0 2024-05-08T22:09:09.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.955+0000 7faab3e00700 1 -- 172.21.15.131:0/3249863570 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac005520 con 0x7faabc0733d0 2024-05-08T22:09:09.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.955+0000 7faac2400700 1 -- 172.21.15.131:0/3249863570 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faabc1a63c0 con 0x7faabc0733d0 2024-05-08T22:09:09.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.958+0000 7faab3e00700 1 -- 172.21.15.131:0/3249863570 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7faaac00cc60 con 0x7faabc0733d0 2024-05-08T22:09:09.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.959+0000 7faab3e00700 1 --2- 172.21.15.131:0/3249863570 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7faaa00a1310 0x7faaa00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.960+0000 7faab3e00700 1 -- 172.21.15.131:0/3249863570 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7faaac014070 con 0x7faabc0733d0 2024-05-08T22:09:09.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.960+0000 7faad5eefb80 1 Processor -- start 2024-05-08T22:09:09.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.960+0000 7faabaa00700 1 --2- 172.21.15.131:0/3249863570 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7faaa00a1310 0x7faaa00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.960+0000 7faad5eefb80 1 -- start start 2024-05-08T22:09:09.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.960+0000 7faad5eefb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d53576e00 0x562d535771e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.960+0000 7faad5eefb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d53577720 0x562d53571350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.960+0000 7faad5eefb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d53571890 0x562d53571cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.960+0000 7faad5eefb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d534c8150 con 0x562d53577720 2024-05-08T22:09:09.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.960+0000 7faad5eefb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d534c5b10 con 0x562d53576e00 2024-05-08T22:09:09.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.960+0000 7faad5eefb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d534c1400 con 0x562d53571890 2024-05-08T22:09:09.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.961+0000 7faabb400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d53576e00 0x562d535771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.961+0000 7faabb400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d53576e00 0x562d535771e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:38458/0 (socket says 172.21.15.131:38458) 2024-05-08T22:09:09.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.961+0000 7faabb400700 1 -- 172.21.15.131:0/912658104 learned_addr learned my addr 172.21.15.131:0/912658104 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:09.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.961+0000 7faabaa00700 1 --2- 172.21.15.131:0/3249863570 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7faaa00a1310 0x7faaa00a37d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faaa8006010 tx=0x7faaa800b260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:09.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.961+0000 7faabaa00700 1 --2- 172.21.15.131:0/912658104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d53577720 0x562d53571350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.961+0000 7faabbe00700 1 --2- 172.21.15.131:0/912658104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d53571890 0x562d53571cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.961+0000 7faabb400700 1 -- 172.21.15.131:0/912658104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d53571890 msgr2=0x562d53571cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:09.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.961+0000 7faabb400700 1 --2- 172.21.15.131:0/912658104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d53571890 0x562d53571cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.961+0000 7faabb400700 1 -- 172.21.15.131:0/912658104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d53577720 msgr2=0x562d53571350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:09.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.961+0000 7faabb400700 1 --2- 172.21.15.131:0/912658104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d53577720 0x562d53571350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.961+0000 7faabb400700 1 -- 172.21.15.131:0/912658104 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d533423b0 con 0x562d53576e00 2024-05-08T22:09:09.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.961+0000 7faabaa00700 1 --2- 172.21.15.131:0/912658104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d53577720 0x562d53571350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:09:09.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.961+0000 7faabbe00700 1 --2- 172.21.15.131:0/912658104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d53571890 0x562d53571cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:09:09.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.961+0000 7faabb400700 1 --2- 172.21.15.131:0/912658104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d53576e00 0x562d535771e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faaac096ac0 tx=0x7faaac099a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=749f379d2f7d6be6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:09.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.961+0000 7faab0c00700 1 -- 172.21.15.131:0/912658104 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac0cd160 con 0x562d53576e00 2024-05-08T22:09:09.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.961+0000 7faab0c00700 1 -- 172.21.15.131:0/912658104 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faaac016e80 con 0x562d53576e00 2024-05-08T22:09:09.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.961+0000 7faab0c00700 1 -- 172.21.15.131:0/912658104 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac09a510 con 0x562d53576e00 2024-05-08T22:09:09.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.962+0000 7faad5eefb80 1 -- 172.21.15.131:0/912658104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d53576e00 msgr2=0x562d535771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:09.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.962+0000 7faad5eefb80 1 --2- 172.21.15.131:0/912658104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d53576e00 0x562d535771e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faaac096ac0 tx=0x7faaac099a30 comp rx=0 tx=0).stop 2024-05-08T22:09:09.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.962+0000 7faad5eefb80 1 -- 172.21.15.131:0/912658104 shutdown_connections 2024-05-08T22:09:09.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.962+0000 7faad5eefb80 1 --2- 172.21.15.131:0/912658104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d53571890 0x562d53571cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.962+0000 7faad5eefb80 1 --2- 172.21.15.131:0/912658104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d53576e00 0x562d535771e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.962+0000 7faad5eefb80 1 --2- 172.21.15.131:0/912658104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d53577720 0x562d53571350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.962+0000 7faad5eefb80 1 -- 172.21.15.131:0/912658104 >> 172.21.15.131:0/912658104 conn(0x562d534cf210 msgr2=0x562d5356bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:09.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.962+0000 7faad5eefb80 1 -- 172.21.15.131:0/912658104 shutdown_connections 2024-05-08T22:09:09.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.963+0000 7faad5eefb80 1 -- 172.21.15.131:0/912658104 wait complete. 2024-05-08T22:09:09.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.963+0000 7faad5eefb80 1 -- start start 2024-05-08T22:09:09.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.963+0000 7faad5eefb80 1 Processor -- start 2024-05-08T22:09:09.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.964+0000 7faad5eefb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d53571890 0x562d5357aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.964+0000 7faad5eefb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d53576e00 0x562d5357afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.964+0000 7faad5eefb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d53577720 0x562d5357b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.964+0000 7faad5eefb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5345d560 con 0x562d53571890 2024-05-08T22:09:09.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.964+0000 7faad5eefb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5345d200 con 0x562d53576e00 2024-05-08T22:09:09.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.964+0000 7faad5eefb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d533446a0 con 0x562d53577720 2024-05-08T22:09:09.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.964+0000 7faad5eefb80 10 client.0 _ll_register_callbacks cb 0x562d53466b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:09.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.964+0000 7faabb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d53571890 0x562d5357aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.964+0000 7faabbe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d53577720 0x562d5357b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.964+0000 7faabb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d53571890 0x562d5357aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52018/0 (socket says 172.21.15.131:52018) 2024-05-08T22:09:09.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.964+0000 7faabb400700 1 -- 172.21.15.131:0/2610518712 learned_addr learned my addr 172.21.15.131:0/2610518712 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:09.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.964+0000 7faabaa00700 1 --2- 172.21.15.131:0/2610518712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d53576e00 0x562d5357afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.964+0000 7faabbe00700 1 -- 172.21.15.131:0/2610518712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d53576e00 msgr2=0x562d5357afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:09.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.964+0000 7faabbe00700 1 --2- 172.21.15.131:0/2610518712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d53576e00 0x562d5357afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.964+0000 7faabbe00700 1 -- 172.21.15.131:0/2610518712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d53571890 msgr2=0x562d5357aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:09.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.965+0000 7faabbe00700 1 --2- 172.21.15.131:0/2610518712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d53571890 0x562d5357aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.965+0000 7faabbe00700 1 -- 172.21.15.131:0/2610518712 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faabc1a63c0 con 0x562d53577720 2024-05-08T22:09:09.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.965+0000 7faabb400700 1 --2- 172.21.15.131:0/2610518712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d53571890 0x562d5357aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:09:09.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.965+0000 7faabbe00700 1 --2- 172.21.15.131:0/2610518712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d53577720 0x562d5357b520 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7faab4009f60 tx=0x7faab4009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1e07f636c17d2f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:09.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.965+0000 7faaa5600700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:09:09.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.965+0000 7faaa5600700 1 -- 172.21.15.131:0/2610518712 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab40048c0 con 0x562d53577720 2024-05-08T22:09:09.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.965+0000 7faad5eefb80 1 -- 172.21.15.131:0/2610518712 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d5341a270 con 0x562d53577720 2024-05-08T22:09:09.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.965+0000 7faaa5600700 1 -- 172.21.15.131:0/2610518712 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7faab4015600 con 0x562d53577720 2024-05-08T22:09:09.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.965+0000 7faaa5600700 1 -- 172.21.15.131:0/2610518712 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7faab40157c0 con 0x562d53577720 2024-05-08T22:09:09.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.966+0000 7faad5eefb80 10 client.25990 fetch_fsmap learned FSMap version 24 2024-05-08T22:09:09.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.966+0000 7faad5eefb80 1 -- 172.21.15.131:0/2610518712 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562d532eeb60 con 0x562d53577720 2024-05-08T22:09:09.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.966+0000 7faaa5600700 1 -- 172.21.15.131:0/2610518712 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562d5341a270 con 0x562d53577720 2024-05-08T22:09:09.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.966+0000 7faaa5600700 1 -- 172.21.15.131:0/2610518712 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faab4014070 con 0x562d53577720 2024-05-08T22:09:09.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.966+0000 7faad5eefb80 10 client.25990 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:09:09.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.966+0000 7faad5eefb80 10 client.25990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:09.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.966+0000 7faad5eefb80 20 client.25990 populate_metadata read hostname 'smithi131' 2024-05-08T22:09:09.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.966+0000 7faad5eefb80 1 --2- 172.21.15.131:0/2610518712 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562d5358b160 0x562d5358d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.966+0000 7faad5eefb80 4 client.25990 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:09:09.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.966+0000 7faad5eefb80 1 -- 172.21.15.131:0/2610518712 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d53540810 con 0x562d5358b160 2024-05-08T22:09:09.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.967+0000 7faabb400700 1 --2- 172.21.15.131:0/2610518712 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562d5358b160 0x562d5358d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.967+0000 7faabb400700 1 --2- 172.21.15.131:0/2610518712 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562d5358b160 0x562d5358d580 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d90ac7f041714c24 server_cookie=dffce1a1a6929513 in_seq=0 out_seq=0 2024-05-08T22:09:09.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.967+0000 7faaa5600700 10 client.25990 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:09.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.967+0000 7faaa5600700 1 -- 172.21.15.131:0/2610518712 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562d53540810 con 0x562d5358b160 2024-05-08T22:09:09.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.967+0000 7faaa5600700 10 client.25990 handle_command_reply: tid=0 2024-05-08T22:09:09.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.968+0000 7faad5eefb80 1 client.25990 shutdown 2024-05-08T22:09:09.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.968+0000 7faad5eefb80 1 -- 172.21.15.131:0/2610518712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d53577720 msgr2=0x562d5357b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:09.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.968+0000 7faad5eefb80 1 --2- 172.21.15.131:0/2610518712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d53577720 0x562d5357b520 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7faab4009f60 tx=0x7faab4009f90 comp rx=0 tx=0).stop 2024-05-08T22:09:09.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.968+0000 7faad5eefb80 1 -- 172.21.15.131:0/2610518712 shutdown_connections 2024-05-08T22:09:09.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.968+0000 7faad5eefb80 1 --2- 172.21.15.131:0/2610518712 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562d5358b160 0x562d5358d580 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.968+0000 7faad5eefb80 1 --2- 172.21.15.131:0/2610518712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d53577720 0x562d5357b520 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.968+0000 7faad5eefb80 1 --2- 172.21.15.131:0/2610518712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d53571890 0x562d5357aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.968+0000 7faad5eefb80 1 --2- 172.21.15.131:0/2610518712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d53576e00 0x562d5357afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.968+0000 7faaa5600700 0 client.25990 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:09.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.968+0000 7faad5eefb80 1 -- 172.21.15.131:0/2610518712 >> 172.21.15.131:0/2610518712 conn(0x562d534cf210 msgr2=0x562d5356b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:09.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.968+0000 7faad5eefb80 1 -- 172.21.15.131:0/2610518712 shutdown_connections 2024-05-08T22:09:09.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.968+0000 7faad5eefb80 1 -- 172.21.15.131:0/2610518712 wait complete. 2024-05-08T22:09:09.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.968+0000 7faad5eefb80 20 client.25990 trim_cache size 0 max 16384 2024-05-08T22:09:09.980 INFO:tasks.workunit.client.0.smithi131.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2024-05-08T22:09:09.980 INFO:tasks.workunit.client.0.smithi131.stdout:checking if more special flags are required for pthreads... no 2024-05-08T22:09:09.980 INFO:tasks.workunit.client.0.smithi131.stdout:checking for cc_r... gcc 2024-05-08T22:09:09.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.988+0000 7faad5eefb80 1 Processor -- start 2024-05-08T22:09:09.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.988+0000 7faad5eefb80 1 -- start start 2024-05-08T22:09:09.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.988+0000 7faad5eefb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d53571890 0x562d53571350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.988+0000 7faad5eefb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d53576e00 0x562d53571c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.988+0000 7faad5eefb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d53577720 0x562d5356bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.988+0000 7faad5eefb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d53568dc0 con 0x562d53576e00 2024-05-08T22:09:09.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.988+0000 7faad5eefb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5356de50 con 0x562d53571890 2024-05-08T22:09:09.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.988+0000 7faad5eefb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d53577560 con 0x562d53577720 2024-05-08T22:09:09.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.988+0000 7faabaa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d53576e00 0x562d53571c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.988+0000 7faabb400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d53571890 0x562d53571350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.989+0000 7faabaa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d53576e00 0x562d53571c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52030/0 (socket says 172.21.15.131:52030) 2024-05-08T22:09:09.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.989+0000 7faabaa00700 1 -- 172.21.15.131:0/2172839770 learned_addr learned my addr 172.21.15.131:0/2172839770 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:09.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.989+0000 7faabbe00700 1 --2- 172.21.15.131:0/2172839770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d53577720 0x562d5356bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.989+0000 7faabb400700 1 -- 172.21.15.131:0/2172839770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d53577720 msgr2=0x562d5356bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:09.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.989+0000 7faabb400700 1 --2- 172.21.15.131:0/2172839770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d53577720 0x562d5356bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.989+0000 7faabb400700 1 -- 172.21.15.131:0/2172839770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d53576e00 msgr2=0x562d53571c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:09.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.989+0000 7faabb400700 1 --2- 172.21.15.131:0/2172839770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d53576e00 0x562d53571c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.989+0000 7faabb400700 1 -- 172.21.15.131:0/2172839770 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d53487560 con 0x562d53571890 2024-05-08T22:09:09.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.989+0000 7faabb400700 1 --2- 172.21.15.131:0/2172839770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d53571890 0x562d53571350 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7faaac099d50 tx=0x7faaac00cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7fa41ff3e1f9c43 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:09.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.989+0000 7faab1600700 1 -- 172.21.15.131:0/2172839770 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac07a050 con 0x562d53571890 2024-05-08T22:09:09.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.989+0000 7faab1600700 1 -- 172.21.15.131:0/2172839770 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faaac09de80 con 0x562d53571890 2024-05-08T22:09:09.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.989+0000 7faab1600700 1 -- 172.21.15.131:0/2172839770 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac07ea90 con 0x562d53571890 2024-05-08T22:09:09.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.990+0000 7faad5eefb80 1 -- 172.21.15.131:0/2172839770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d53571890 msgr2=0x562d53571350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:09.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.990+0000 7faad5eefb80 1 --2- 172.21.15.131:0/2172839770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d53571890 0x562d53571350 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7faaac099d50 tx=0x7faaac00cbd0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.990+0000 7faad5eefb80 1 -- 172.21.15.131:0/2172839770 shutdown_connections 2024-05-08T22:09:09.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.990+0000 7faad5eefb80 1 --2- 172.21.15.131:0/2172839770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d53577720 0x562d5356bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.990+0000 7faad5eefb80 1 --2- 172.21.15.131:0/2172839770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d53571890 0x562d53571350 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.990+0000 7faad5eefb80 1 --2- 172.21.15.131:0/2172839770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d53576e00 0x562d53571c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.990+0000 7faad5eefb80 1 -- 172.21.15.131:0/2172839770 >> 172.21.15.131:0/2172839770 conn(0x562d534cf210 msgr2=0x562d53568210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:09.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.990+0000 7faad5eefb80 1 -- 172.21.15.131:0/2172839770 shutdown_connections 2024-05-08T22:09:09.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.990+0000 7faad5eefb80 1 -- 172.21.15.131:0/2172839770 wait complete. 2024-05-08T22:09:09.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.991+0000 7faad5eefb80 1 -- start start 2024-05-08T22:09:09.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.991+0000 7faad5eefb80 1 Processor -- start 2024-05-08T22:09:09.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.992+0000 7faad5eefb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d53571890 0x562d53582370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.992+0000 7faad5eefb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d53576e00 0x562d5357e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.992+0000 7faad5eefb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d53577720 0x562d5357e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.992+0000 7faad5eefb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5357ebc0 con 0x562d53576e00 2024-05-08T22:09:09.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.992+0000 7faad5eefb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5357ed30 con 0x562d53577720 2024-05-08T22:09:09.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.992+0000 7faad5eefb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5357eef0 con 0x562d53571890 2024-05-08T22:09:09.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.992+0000 7faad5eefb80 10 client.0 _ll_register_callbacks cb 0x562d53466b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:09.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.992+0000 7faabaa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d53576e00 0x562d5357e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.992+0000 7faabb400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d53571890 0x562d53582370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.992+0000 7faabbe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d53577720 0x562d5357e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.992+0000 7faabb400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d53571890 0x562d53582370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60812/0 (socket says 172.21.15.131:60812) 2024-05-08T22:09:09.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.992+0000 7faabbe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d53577720 0x562d5357e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:38476/0 (socket says 172.21.15.131:38476) 2024-05-08T22:09:09.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.992+0000 7faabb400700 1 -- 172.21.15.131:0/332757023 learned_addr learned my addr 172.21.15.131:0/332757023 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:09.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.992+0000 7faabbe00700 1 -- 172.21.15.131:0/332757023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d53571890 msgr2=0x562d53582370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:09.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.992+0000 7faabbe00700 1 --2- 172.21.15.131:0/332757023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d53571890 0x562d53582370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.992+0000 7faabbe00700 1 -- 172.21.15.131:0/332757023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d53576e00 msgr2=0x562d5357e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:09.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.992+0000 7faabbe00700 1 --2- 172.21.15.131:0/332757023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d53576e00 0x562d5357e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:09.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.992+0000 7faabbe00700 1 -- 172.21.15.131:0/332757023 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d533423b0 con 0x562d53577720 2024-05-08T22:09:09.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.993+0000 7faabbe00700 1 --2- 172.21.15.131:0/332757023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d53577720 0x562d5357e680 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faab4000f80 tx=0x7faab40046b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c99ddf91110d3236 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:09.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.993+0000 7faaa5600700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:09:09.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.993+0000 7faaa5600700 1 -- 172.21.15.131:0/332757023 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab40075e0 con 0x562d53577720 2024-05-08T22:09:09.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.993+0000 7faad5eefb80 1 -- 172.21.15.131:0/332757023 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faaac016e80 con 0x562d53577720 2024-05-08T22:09:09.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.993+0000 7faaa5600700 1 -- 172.21.15.131:0/332757023 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7faab4007740 con 0x562d53577720 2024-05-08T22:09:09.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.993+0000 7faaa5600700 1 -- 172.21.15.131:0/332757023 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7faab4007900 con 0x562d53577720 2024-05-08T22:09:09.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.993+0000 7faad5eefb80 10 client.25988 fetch_fsmap learned FSMap version 24 2024-05-08T22:09:09.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.993+0000 7faad5eefb80 1 -- 172.21.15.131:0/332757023 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562d53346cd0 con 0x562d53577720 2024-05-08T22:09:09.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.993+0000 7faaa5600700 1 -- 172.21.15.131:0/332757023 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faaac016e80 con 0x562d53577720 2024-05-08T22:09:09.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.994+0000 7faaa5600700 1 -- 172.21.15.131:0/332757023 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faab4004040 con 0x562d53577720 2024-05-08T22:09:09.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.994+0000 7faad5eefb80 10 client.25988 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:09:09.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.994+0000 7faad5eefb80 10 client.25988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:09.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.994+0000 7faad5eefb80 20 client.25988 populate_metadata read hostname 'smithi131' 2024-05-08T22:09:09.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.994+0000 7faad5eefb80 1 --2- 172.21.15.131:0/332757023 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562d535e1a60 0x562d535e3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:09.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.994+0000 7faad5eefb80 4 client.25988 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:09:09.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.994+0000 7faad5eefb80 1 -- 172.21.15.131:0/332757023 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562d534b9ec0 con 0x562d535e1a60 2024-05-08T22:09:09.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.994+0000 7faabb400700 1 --2- 172.21.15.131:0/332757023 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562d535e1a60 0x562d535e3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:09.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.994+0000 7faabb400700 1 --2- 172.21.15.131:0/332757023 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562d535e1a60 0x562d535e3e80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e18265cae9d1ed1 server_cookie=5c978dccd16049d5 in_seq=0 out_seq=0 2024-05-08T22:09:09.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.994+0000 7faaa5600700 10 client.25988 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:09.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.996+0000 7faaa5600700 1 -- 172.21.15.131:0/332757023 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562d534b9ec0 con 0x562d535e1a60 2024-05-08T22:09:09.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.996+0000 7faaa5600700 10 client.25988 handle_command_reply: tid=0 2024-05-08T22:09:09.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.996+0000 7faad5eefb80 1 client.25988 shutdown 2024-05-08T22:09:09.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.996+0000 7faad5eefb80 1 -- 172.21.15.131:0/332757023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d53577720 msgr2=0x562d5357e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:09.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.996+0000 7faad5eefb80 1 --2- 172.21.15.131:0/332757023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d53577720 0x562d5357e680 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faab4000f80 tx=0x7faab40046b0 comp rx=0 tx=0).stop 2024-05-08T22:09:10.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.996+0000 7faad5eefb80 1 -- 172.21.15.131:0/332757023 shutdown_connections 2024-05-08T22:09:10.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.996+0000 7faad5eefb80 1 --2- 172.21.15.131:0/332757023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d53577720 0x562d5357e680 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:10.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.996+0000 7faad5eefb80 1 --2- 172.21.15.131:0/332757023 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562d535e1a60 0x562d535e3e80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:10.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.996+0000 7faad5eefb80 1 --2- 172.21.15.131:0/332757023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d53571890 0x562d53582370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:10.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.996+0000 7faaa5600700 0 client.25988 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:10.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.996+0000 7faad5eefb80 1 --2- 172.21.15.131:0/332757023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d53576e00 0x562d5357e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:10.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.996+0000 7faad5eefb80 1 -- 172.21.15.131:0/332757023 >> 172.21.15.131:0/332757023 conn(0x562d534cf210 msgr2=0x562d53575bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:10.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.996+0000 7faad5eefb80 1 -- 172.21.15.131:0/332757023 shutdown_connections 2024-05-08T22:09:10.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.996+0000 7faad5eefb80 1 -- 172.21.15.131:0/332757023 wait complete. 2024-05-08T22:09:10.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.997+0000 7faad5eefb80 20 client.25988 trim_cache size 0 max 16384 2024-05-08T22:09:10.000 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:09:10.001 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T22:09:10.001 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-08T22:09:10.001 INFO:teuthology.orchestra.run.smithi131.stdout: "479b4292-010e-4bdd-9584-940c0e0b36b4": { 2024-05-08T22:09:10.001 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-08T22:09:10.001 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "479b4292-010e-4bdd-9584-940c0e0b36b4", 2024-05-08T22:09:10.001 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-08T22:09:10.001 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:09:10.001 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:09:10.001 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:09:10.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.997+0000 7faac2400700 1 -- 172.21.15.131:0/3249863570 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7faaa00a1310 msgr2=0x7faaa00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:10.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.997+0000 7faac2400700 1 --2- 172.21.15.131:0/3249863570 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7faaa00a1310 0x7faaa00a37d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faaa8006010 tx=0x7faaa800b260 comp rx=0 tx=0).stop 2024-05-08T22:09:10.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.997+0000 7faac2400700 1 -- 172.21.15.131:0/3249863570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faabc0733d0 msgr2=0x7faabc199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:10.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.997+0000 7faac2400700 1 --2- 172.21.15.131:0/3249863570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faabc0733d0 0x7faabc199650 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faaac00e9c0 tx=0x7faaac00ecd0 comp rx=0 tx=0).stop 2024-05-08T22:09:10.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.997+0000 7faac2400700 1 -- 172.21.15.131:0/3249863570 shutdown_connections 2024-05-08T22:09:10.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.997+0000 7faac2400700 1 --2- 172.21.15.131:0/3249863570 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7faaa00a1310 0x7faaa00a37d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:10.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.997+0000 7faac2400700 1 --2- 172.21.15.131:0/3249863570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faabc10d750 0x7faabc19df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:10.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.997+0000 7faac2400700 1 --2- 172.21.15.131:0/3249863570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faabc0733d0 0x7faabc199650 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:10.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.997+0000 7faac2400700 1 --2- 172.21.15.131:0/3249863570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faabc073cf0 0x7faabc199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:10.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.997+0000 7faac2400700 1 -- 172.21.15.131:0/3249863570 >> 172.21.15.131:0/3249863570 conn(0x7faabc0fc9e0 msgr2=0x7faabc0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:10.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.997+0000 7faac2400700 1 -- 172.21.15.131:0/3249863570 shutdown_connections 2024-05-08T22:09:10.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:09.997+0000 7faac2400700 1 -- 172.21.15.131:0/3249863570 wait complete. 2024-05-08T22:09:10.017 INFO:tasks.cephfs.filesystem:scrub status for tag:479b4292-010e-4bdd-9584-940c0e0b36b4 - {'path': '/', 'tag': '479b4292-010e-4bdd-9584-940c0e0b36b4', 'options': 'recursive,force'} 2024-05-08T22:09:10.094 INFO:tasks.workunit.client.0.smithi131.stdout:checking for setrlimit... yes 2024-05-08T22:09:10.199 INFO:tasks.workunit.client.0.smithi131.stdout:checking for setproctitle... no 2024-05-08T22:09:10.298 INFO:tasks.workunit.client.0.smithi131.stdout:checking for getopt_long... yes 2024-05-08T22:09:10.407 INFO:tasks.workunit.client.0.smithi131.stdout:checking for strtoull... yes 2024-05-08T22:09:10.516 INFO:tasks.workunit.client.0.smithi131.stdout:checking for strtoq... yes 2024-05-08T22:09:10.624 INFO:tasks.workunit.client.0.smithi131.stdout:checking for snprintf... yes 2024-05-08T22:09:10.730 INFO:tasks.workunit.client.0.smithi131.stdout:checking for vsnprintf... yes 2024-05-08T22:09:10.836 INFO:tasks.workunit.client.0.smithi131.stdout:checking for vfprintf... yes 2024-05-08T22:09:10.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:10 smithi132 ceph-mon[78180]: pgmap v59: 129 pgs: 129 active+clean; 4.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 268 KiB/s wr, 39 op/s 2024-05-08T22:09:10.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:10 smithi132 ceph-mon[78180]: scrub queued for path: / 2024-05-08T22:09:10.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:10 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:09:10.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:10 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:09:10.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:10 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:09:10.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:10 smithi131 ceph-mon[37260]: pgmap v59: 129 pgs: 129 active+clean; 4.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 268 KiB/s wr, 39 op/s 2024-05-08T22:09:10.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:10 smithi131 ceph-mon[37260]: scrub queued for path: / 2024-05-08T22:09:10.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:10 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:09:10.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:10 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:09:10.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:10 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:09:10.942 INFO:tasks.workunit.client.0.smithi131.stdout:checking for gethostname... yes 2024-05-08T22:09:11.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:10 smithi157 ceph-mon[77746]: pgmap v59: 129 pgs: 129 active+clean; 4.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 268 KiB/s wr, 39 op/s 2024-05-08T22:09:11.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:10 smithi157 ceph-mon[77746]: scrub queued for path: / 2024-05-08T22:09:11.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:10 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:09:11.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:10 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:09:11.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:10 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:09:11.047 INFO:tasks.workunit.client.0.smithi131.stdout:checking for setlocale... yes 2024-05-08T22:09:11.156 INFO:tasks.workunit.client.0.smithi131.stdout:checking for sync... yes 2024-05-08T22:09:11.261 INFO:tasks.workunit.client.0.smithi131.stdout:checking whether snprintf is C99 conformant... done 2024-05-08T22:09:11.284 INFO:tasks.workunit.client.0.smithi131.stdout:configure: creating ./config.status 2024-05-08T22:09:11.569 INFO:tasks.workunit.client.0.smithi131.stdout:config.status: creating Makefile 2024-05-08T22:09:11.610 INFO:tasks.workunit.client.0.smithi131.stdout:config.status: creating src/Makefile 2024-05-08T22:09:11.650 INFO:tasks.workunit.client.0.smithi131.stdout:config.status: creating man/Makefile 2024-05-08T22:09:11.668 INFO:tasks.workunit.client.0.smithi131.stdout:config.status: creating config.h 2024-05-08T22:09:11.750 INFO:tasks.workunit.client.0.smithi131.stdout:config.status: executing depfiles commands 2024-05-08T22:09:12.051 INFO:tasks.workunit.client.0.smithi131.stdout:make all-recursive 2024-05-08T22:09:12.053 INFO:tasks.workunit.client.0.smithi131.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-05-08T22:09:12.059 INFO:tasks.workunit.client.0.smithi131.stdout:Making all in src 2024-05-08T22:09:12.068 INFO:tasks.workunit.client.0.smithi131.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-05-08T22:09:12.069 INFO:tasks.workunit.client.0.smithi131.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2024-05-08T22:09:12.069 INFO:tasks.workunit.client.0.smithi131.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2024-05-08T22:09:12.231 INFO:tasks.workunit.client.0.smithi131.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2024-05-08T22:09:12.231 INFO:tasks.workunit.client.0.smithi131.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2024-05-08T22:09:12.326 INFO:tasks.workunit.client.0.smithi131.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2024-05-08T22:09:12.326 INFO:tasks.workunit.client.0.smithi131.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2024-05-08T22:09:12.402 INFO:tasks.workunit.client.0.smithi131.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2024-05-08T22:09:12.402 INFO:tasks.workunit.client.0.smithi131.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2024-05-08T22:09:12.476 INFO:tasks.workunit.client.0.smithi131.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2024-05-08T22:09:12.476 INFO:tasks.workunit.client.0.smithi131.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2024-05-08T22:09:12.549 INFO:tasks.workunit.client.0.smithi131.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2024-05-08T22:09:12.550 INFO:tasks.workunit.client.0.smithi131.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2024-05-08T22:09:12.621 INFO:tasks.workunit.client.0.smithi131.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2024-05-08T22:09:12.621 INFO:tasks.workunit.client.0.smithi131.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2024-05-08T22:09:12.733 INFO:tasks.workunit.client.0.smithi131.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2024-05-08T22:09:12.734 INFO:tasks.workunit.client.0.smithi131.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2024-05-08T22:09:12.802 INFO:tasks.workunit.client.0.smithi131.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2024-05-08T22:09:12.802 INFO:tasks.workunit.client.0.smithi131.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2024-05-08T22:09:12.839 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:12 smithi132 ceph-mon[78180]: pgmap v60: 129 pgs: 129 active+clean; 4.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 268 KiB/s wr, 37 op/s 2024-05-08T22:09:12.857 INFO:tasks.workunit.client.0.smithi131.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2024-05-08T22:09:12.857 INFO:tasks.workunit.client.0.smithi131.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2024-05-08T22:09:12.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:12 smithi131 ceph-mon[37260]: pgmap v60: 129 pgs: 129 active+clean; 4.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 268 KiB/s wr, 37 op/s 2024-05-08T22:09:12.917 INFO:tasks.workunit.client.0.smithi131.stdout:gcc -g -O2 -pthread -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2024-05-08T22:09:12.946 INFO:tasks.workunit.client.0.smithi131.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-05-08T22:09:12.947 INFO:tasks.workunit.client.0.smithi131.stdout:Making all in man 2024-05-08T22:09:12.957 INFO:tasks.workunit.client.0.smithi131.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-05-08T22:09:12.957 INFO:tasks.workunit.client.0.smithi131.stdout:make[2]: Nothing to be done for 'all'. 2024-05-08T22:09:12.957 INFO:tasks.workunit.client.0.smithi131.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-05-08T22:09:12.960 INFO:tasks.workunit.client.0.smithi131.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-05-08T22:09:12.963 INFO:tasks.workunit.client.0.smithi131.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-05-08T22:09:12.963 INFO:tasks.workunit.client.0.smithi131.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-05-08T22:09:12.967 INFO:tasks.workunit.client.0.smithi131.stdout:running blogbench 2024-05-08T22:09:12.969 INFO:tasks.workunit.client.0.smithi131.stdout: 2024-05-08T22:09:12.969 INFO:tasks.workunit.client.0.smithi131.stdout:Frequency = 10 secs 2024-05-08T22:09:12.969 INFO:tasks.workunit.client.0.smithi131.stdout:Scratch dir = [blogtest_in] 2024-05-08T22:09:13.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:12 smithi157 ceph-mon[77746]: pgmap v60: 129 pgs: 129 active+clean; 4.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 268 KiB/s wr, 37 op/s 2024-05-08T22:09:13.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:13 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:09:13.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:13 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:09:14.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:13 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:09:14.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:14 smithi132 ceph-mon[78180]: pgmap v61: 129 pgs: 129 active+clean; 4.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 268 KiB/s wr, 37 op/s 2024-05-08T22:09:14.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:14 smithi131 ceph-mon[37260]: pgmap v61: 129 pgs: 129 active+clean; 4.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 268 KiB/s wr, 37 op/s 2024-05-08T22:09:15.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:14 smithi157 ceph-mon[77746]: pgmap v61: 129 pgs: 129 active+clean; 4.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 268 KiB/s wr, 37 op/s 2024-05-08T22:09:15.211 INFO:tasks.workunit.client.0.smithi131.stdout:Spawning 3 writers... 2024-05-08T22:09:15.211 INFO:tasks.workunit.client.0.smithi131.stdout:Spawning 1 rewriters... 2024-05-08T22:09:15.212 INFO:tasks.workunit.client.0.smithi131.stdout:Spawning 5 commenters... 2024-05-08T22:09:15.212 INFO:tasks.workunit.client.0.smithi131.stdout:Spawning 100 readers... 2024-05-08T22:09:15.220 INFO:tasks.workunit.client.0.smithi131.stdout:Benchmarking for 30 iterations. 2024-05-08T22:09:15.220 INFO:tasks.workunit.client.0.smithi131.stdout:The test will run during 5 minutes. 2024-05-08T22:09:15.220 INFO:tasks.workunit.client.0.smithi131.stdout: 2024-05-08T22:09:15.220 INFO:tasks.workunit.client.0.smithi131.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2024-05-08T22:09:16.839 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:16 smithi132 ceph-mon[78180]: pgmap v62: 129 pgs: 129 active+clean; 6.2 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 498 KiB/s wr, 70 op/s 2024-05-08T22:09:16.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:16 smithi131 ceph-mon[37260]: pgmap v62: 129 pgs: 129 active+clean; 6.2 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 498 KiB/s wr, 70 op/s 2024-05-08T22:09:17.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:16 smithi157 ceph-mon[77746]: pgmap v62: 129 pgs: 129 active+clean; 6.2 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 498 KiB/s wr, 70 op/s 2024-05-08T22:09:18.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:18 smithi132 ceph-mon[78180]: pgmap v63: 129 pgs: 129 active+clean; 6.2 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 458 KiB/s wr, 52 op/s 2024-05-08T22:09:18.899 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:18 smithi131 ceph-mon[37260]: pgmap v63: 129 pgs: 129 active+clean; 6.2 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 458 KiB/s wr, 52 op/s 2024-05-08T22:09:19.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:18 smithi157 ceph-mon[77746]: pgmap v63: 129 pgs: 129 active+clean; 6.2 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 458 KiB/s wr, 52 op/s 2024-05-08T22:09:20.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:20 smithi132 ceph-mon[78180]: pgmap v64: 129 pgs: 129 active+clean; 12 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.0 MiB/s wr, 64 op/s 2024-05-08T22:09:20.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:20 smithi131 ceph-mon[37260]: pgmap v64: 129 pgs: 129 active+clean; 12 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.0 MiB/s wr, 64 op/s 2024-05-08T22:09:21.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:20 smithi157 ceph-mon[77746]: pgmap v64: 129 pgs: 129 active+clean; 12 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.0 MiB/s wr, 64 op/s 2024-05-08T22:09:22.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:22 smithi131 ceph-mon[37260]: pgmap v65: 129 pgs: 129 active+clean; 12 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 807 KiB/s wr, 45 op/s 2024-05-08T22:09:23.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:22 smithi157 ceph-mon[77746]: pgmap v65: 129 pgs: 129 active+clean; 12 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 807 KiB/s wr, 45 op/s 2024-05-08T22:09:23.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:22 smithi132 ceph-mon[78180]: pgmap v65: 129 pgs: 129 active+clean; 12 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 807 KiB/s wr, 45 op/s 2024-05-08T22:09:24.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:24 smithi131 ceph-mon[37260]: pgmap v66: 129 pgs: 129 active+clean; 12 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 807 KiB/s wr, 45 op/s 2024-05-08T22:09:24.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:24 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:09:25.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:24 smithi157 ceph-mon[77746]: pgmap v66: 129 pgs: 129 active+clean; 12 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 807 KiB/s wr, 45 op/s 2024-05-08T22:09:25.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:24 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:09:25.089 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:24 smithi132 ceph-mon[78180]: pgmap v66: 129 pgs: 129 active+clean; 12 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 807 KiB/s wr, 45 op/s 2024-05-08T22:09:25.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:24 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:09:25.221 INFO:tasks.workunit.client.0.smithi131.stdout: 17 280659 923 193969 842 67717 976 2024-05-08T22:09:27.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:26 smithi157 ceph-mon[77746]: pgmap v67: 129 pgs: 129 active+clean; 37 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 139 op/s 2024-05-08T22:09:27.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:26 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:09:27.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:26 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:09:27.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:26 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:09:27.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:26 smithi132 ceph-mon[78180]: pgmap v67: 129 pgs: 129 active+clean; 37 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 139 op/s 2024-05-08T22:09:27.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:26 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:09:27.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:26 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:09:27.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:26 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:09:27.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:26 smithi131 ceph-mon[37260]: pgmap v67: 129 pgs: 129 active+clean; 37 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 139 op/s 2024-05-08T22:09:27.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:26 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:09:27.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:26 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:09:27.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:26 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:09:29.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:28 smithi157 ceph-mon[77746]: pgmap v68: 129 pgs: 129 active+clean; 37 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 106 op/s 2024-05-08T22:09:29.089 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:28 smithi132 ceph-mon[78180]: pgmap v68: 129 pgs: 129 active+clean; 37 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 106 op/s 2024-05-08T22:09:29.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:28 smithi131 ceph-mon[37260]: pgmap v68: 129 pgs: 129 active+clean; 37 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 106 op/s 2024-05-08T22:09:31.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:30 smithi157 ceph-mon[77746]: pgmap v69: 129 pgs: 129 active+clean; 62 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 215 op/s 2024-05-08T22:09:31.089 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:30 smithi132 ceph-mon[78180]: pgmap v69: 129 pgs: 129 active+clean; 62 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 215 op/s 2024-05-08T22:09:31.151 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:30 smithi131 ceph-mon[37260]: pgmap v69: 129 pgs: 129 active+clean; 62 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 215 op/s 2024-05-08T22:09:33.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:32 smithi157 ceph-mon[77746]: pgmap v70: 129 pgs: 129 active+clean; 62 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 203 op/s 2024-05-08T22:09:33.089 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:32 smithi132 ceph-mon[78180]: pgmap v70: 129 pgs: 129 active+clean; 62 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 203 op/s 2024-05-08T22:09:33.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:32 smithi131 ceph-mon[37260]: pgmap v70: 129 pgs: 129 active+clean; 62 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 203 op/s 2024-05-08T22:09:35.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:34 smithi157 ceph-mon[77746]: pgmap v71: 129 pgs: 129 active+clean; 62 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 203 op/s 2024-05-08T22:09:35.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:34 smithi157 ceph-mon[77746]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T22:09:35.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:34 smithi132 ceph-mon[78180]: pgmap v71: 129 pgs: 129 active+clean; 62 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 203 op/s 2024-05-08T22:09:35.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:34 smithi132 ceph-mon[78180]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T22:09:35.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:34 smithi131 ceph-mon[37260]: pgmap v71: 129 pgs: 129 active+clean; 62 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 203 op/s 2024-05-08T22:09:35.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:34 smithi131 ceph-mon[37260]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T22:09:35.220 INFO:tasks.workunit.client.0.smithi131.stdout: 26 497109 557 360005 472 153471 1115 2024-05-08T22:09:37.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:36 smithi157 ceph-mon[77746]: pgmap v72: 129 pgs: 129 active+clean; 69 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 230 op/s 2024-05-08T22:09:37.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:36 smithi132 ceph-mon[78180]: pgmap v72: 129 pgs: 129 active+clean; 69 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 230 op/s 2024-05-08T22:09:37.150 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:36 smithi131 ceph-mon[37260]: pgmap v72: 129 pgs: 129 active+clean; 69 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 230 op/s 2024-05-08T22:09:39.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:38 smithi157 ceph-mon[77746]: pgmap v73: 129 pgs: 129 active+clean; 69 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 137 op/s 2024-05-08T22:09:39.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:38 smithi132 ceph-mon[78180]: pgmap v73: 129 pgs: 129 active+clean; 69 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 137 op/s 2024-05-08T22:09:39.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:38 smithi131 ceph-mon[37260]: pgmap v73: 129 pgs: 129 active+clean; 69 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 137 op/s 2024-05-08T22:09:40.018 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:09:40.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.809+0000 7fe8b6800700 1 -- 172.21.15.131:0/1905925738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8b0073c60 msgr2=0x7fe8b0111a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:40.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.809+0000 7fe8b6800700 1 --2- 172.21.15.131:0/1905925738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8b0073c60 0x7fe8b0111a40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe8980098f0 tx=0x7fe898009c00 comp rx=0 tx=0).stop 2024-05-08T22:09:40.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.809+0000 7fe8b6800700 1 -- 172.21.15.131:0/1905925738 shutdown_connections 2024-05-08T22:09:40.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.809+0000 7fe8b6800700 1 --2- 172.21.15.131:0/1905925738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8b0111f80 0x7fe8b0114370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.809+0000 7fe8b6800700 1 --2- 172.21.15.131:0/1905925738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8b0073340 0x7fe8b0073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.809+0000 7fe8b6800700 1 --2- 172.21.15.131:0/1905925738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8b0073c60 0x7fe8b0111a40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.809+0000 7fe8b6800700 1 -- 172.21.15.131:0/1905925738 >> 172.21.15.131:0/1905925738 conn(0x7fe8b00fca80 msgr2=0x7fe8b00feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:40.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.809+0000 7fe8b6800700 1 -- 172.21.15.131:0/1905925738 shutdown_connections 2024-05-08T22:09:40.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.809+0000 7fe8b6800700 1 -- 172.21.15.131:0/1905925738 wait complete. 2024-05-08T22:09:40.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.810+0000 7fe8b6800700 1 Processor -- start 2024-05-08T22:09:40.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.810+0000 7fe8b6800700 1 -- start start 2024-05-08T22:09:40.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.811+0000 7fe8b6800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8b0073340 0x7fe8b010d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.811+0000 7fe8b6800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8b0073c60 0x7fe8b010dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.811+0000 7fe8b6800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8b0111f80 0x7fe8b0199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.811+0000 7fe8b6800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b019a450 con 0x7fe8b0111f80 2024-05-08T22:09:40.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.811+0000 7fe8b6800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b019a5c0 con 0x7fe8b0073c60 2024-05-08T22:09:40.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.811+0000 7fe8b6800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b019a780 con 0x7fe8b0073340 2024-05-08T22:09:40.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.811+0000 7fe8afe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8b0111f80 0x7fe8b0199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.812+0000 7fe8afe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8b0111f80 0x7fe8b0199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58078/0 (socket says 172.21.15.131:58078) 2024-05-08T22:09:40.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.812+0000 7fe8afe00700 1 -- 172.21.15.131:0/2307429347 learned_addr learned my addr 172.21.15.131:0/2307429347 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:40.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.812+0000 7fe8aea00700 1 --2- 172.21.15.131:0/2307429347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8b0073c60 0x7fe8b010dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.812+0000 7fe8aea00700 1 -- 172.21.15.131:0/2307429347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8b0073340 msgr2=0x7fe8b010d7f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:09:40.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.813+0000 7fe8aea00700 1 --2- 172.21.15.131:0/2307429347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8b0073340 0x7fe8b010d7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.813+0000 7fe8aea00700 1 -- 172.21.15.131:0/2307429347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8b0111f80 msgr2=0x7fe8b0199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:40.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.813+0000 7fe8aea00700 1 --2- 172.21.15.131:0/2307429347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8b0111f80 0x7fe8b0199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.813+0000 7fe8aea00700 1 -- 172.21.15.131:0/2307429347 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe898009520 con 0x7fe8b0073c60 2024-05-08T22:09:40.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.813+0000 7fe8afe00700 1 --2- 172.21.15.131:0/2307429347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8b0111f80 0x7fe8b0199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:09:40.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.813+0000 7fe8aea00700 1 --2- 172.21.15.131:0/2307429347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8b0073c60 0x7fe8b010dd30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe898000c00 tx=0x7fe89800fab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:40.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.813+0000 7fe8a7e00700 1 -- 172.21.15.131:0/2307429347 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89801c070 con 0x7fe8b0073c60 2024-05-08T22:09:40.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.813+0000 7fe8a7e00700 1 -- 172.21.15.131:0/2307429347 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe89800bbd0 con 0x7fe8b0073c60 2024-05-08T22:09:40.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.813+0000 7fe8b6800700 1 -- 172.21.15.131:0/2307429347 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b019aa00 con 0x7fe8b0073c60 2024-05-08T22:09:40.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.813+0000 7fe8a7e00700 1 -- 172.21.15.131:0/2307429347 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe898003aa0 con 0x7fe8b0073c60 2024-05-08T22:09:40.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.813+0000 7fe8b6800700 1 -- 172.21.15.131:0/2307429347 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b01a20a0 con 0x7fe8b0073c60 2024-05-08T22:09:40.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.817+0000 7fe8a7e00700 1 -- 172.21.15.131:0/2307429347 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fe898020030 con 0x7fe8b0073c60 2024-05-08T22:09:40.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.820+0000 7fe8a7e00700 1 --2- 172.21.15.131:0/2307429347 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe8940a12d0 0x7fe8940a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.820+0000 7fe8a7e00700 1 -- 172.21.15.131:0/2307429347 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fe8980cdfa0 con 0x7fe8b0073c60 2024-05-08T22:09:40.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.820+0000 7fe8af400700 1 --2- 172.21.15.131:0/2307429347 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe8940a12d0 0x7fe8940a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.821+0000 7fe8ca3a6b80 1 Processor -- start 2024-05-08T22:09:40.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.821+0000 7fe8ca3a6b80 1 -- start start 2024-05-08T22:09:40.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.821+0000 7fe8af400700 1 --2- 172.21.15.131:0/2307429347 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe8940a12d0 0x7fe8940a3790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0005fd0 tx=0x7fe8a001a320 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:40.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.821+0000 7fe8ca3a6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffe4f0ae00 0x55ffe4f0b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.821+0000 7fe8ca3a6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffe4f0b720 0x55ffe4f05350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.821+0000 7fe8ca3a6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffe4f05890 0x55ffe4f05cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.821+0000 7fe8ca3a6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffe4e5c150 con 0x55ffe4f05890 2024-05-08T22:09:40.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.821+0000 7fe8ca3a6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffe4e59b10 con 0x55ffe4f0b720 2024-05-08T22:09:40.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.821+0000 7fe8ca3a6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffe4e55400 con 0x55ffe4f0ae00 2024-05-08T22:09:40.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.822+0000 7fe8af400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffe4f0ae00 0x55ffe4f0b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.822+0000 7fe8af400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffe4f0ae00 0x55ffe4f0b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37892/0 (socket says 172.21.15.131:37892) 2024-05-08T22:09:40.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.822+0000 7fe8af400700 1 -- 172.21.15.131:0/3747720859 learned_addr learned my addr 172.21.15.131:0/3747720859 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:40.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.822+0000 7fe8afe00700 1 --2- 172.21.15.131:0/3747720859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffe4f05890 0x55ffe4f05cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.822+0000 7fe8af400700 1 -- 172.21.15.131:0/3747720859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffe4f0b720 msgr2=0x55ffe4f05350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:40.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.822+0000 7fe8af400700 1 --2- 172.21.15.131:0/3747720859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffe4f0b720 0x55ffe4f05350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.822+0000 7fe8af400700 1 -- 172.21.15.131:0/3747720859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffe4f05890 msgr2=0x55ffe4f05cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:40.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.822+0000 7fe8af400700 1 --2- 172.21.15.131:0/3747720859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffe4f05890 0x55ffe4f05cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.822+0000 7fe8af400700 1 -- 172.21.15.131:0/3747720859 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffe4cd63b0 con 0x55ffe4f0ae00 2024-05-08T22:09:40.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.822+0000 7fe8afe00700 1 --2- 172.21.15.131:0/3747720859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffe4f05890 0x55ffe4f05cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:09:40.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.823+0000 7fe8af400700 1 --2- 172.21.15.131:0/3747720859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffe4f0ae00 0x55ffe4f0b1e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe8a001fb60 tx=0x7fe8a0022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=994ca8148a2d7c37 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:40.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.823+0000 7fe8a4c00700 1 -- 172.21.15.131:0/3747720859 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0021ce0 con 0x55ffe4f0ae00 2024-05-08T22:09:40.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.823+0000 7fe8a4c00700 1 -- 172.21.15.131:0/3747720859 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8a0021e40 con 0x55ffe4f0ae00 2024-05-08T22:09:40.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.823+0000 7fe8a4c00700 1 -- 172.21.15.131:0/3747720859 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a002d7a0 con 0x55ffe4f0ae00 2024-05-08T22:09:40.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.824+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/3747720859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffe4f0ae00 msgr2=0x55ffe4f0b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:40.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.824+0000 7fe8ca3a6b80 1 --2- 172.21.15.131:0/3747720859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffe4f0ae00 0x55ffe4f0b1e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe8a001fb60 tx=0x7fe8a0022000 comp rx=0 tx=0).stop 2024-05-08T22:09:40.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.824+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/3747720859 shutdown_connections 2024-05-08T22:09:40.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.824+0000 7fe8ca3a6b80 1 --2- 172.21.15.131:0/3747720859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffe4f05890 0x55ffe4f05cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.824+0000 7fe8ca3a6b80 1 --2- 172.21.15.131:0/3747720859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffe4f0ae00 0x55ffe4f0b1e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.824+0000 7fe8ca3a6b80 1 --2- 172.21.15.131:0/3747720859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffe4f0b720 0x55ffe4f05350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.824+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/3747720859 >> 172.21.15.131:0/3747720859 conn(0x55ffe4e63210 msgr2=0x55ffe4effdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:40.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.824+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/3747720859 shutdown_connections 2024-05-08T22:09:40.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.824+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/3747720859 wait complete. 2024-05-08T22:09:40.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.825+0000 7fe8ca3a6b80 1 -- start start 2024-05-08T22:09:40.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.825+0000 7fe8ca3a6b80 1 Processor -- start 2024-05-08T22:09:40.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.826+0000 7fe8ca3a6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffe4f05890 0x55ffe4f0ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.826+0000 7fe8ca3a6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffe4f0ae00 0x55ffe4f0f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.826+0000 7fe8ca3a6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffe4f0b720 0x55ffe4f0f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.826+0000 7fe8ca3a6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffe4df1560 con 0x55ffe4f0ae00 2024-05-08T22:09:40.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.826+0000 7fe8ca3a6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffe4df1200 con 0x55ffe4f0b720 2024-05-08T22:09:40.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.826+0000 7fe8ca3a6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffe4cd86a0 con 0x55ffe4f05890 2024-05-08T22:09:40.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.826+0000 7fe8ca3a6b80 10 client.0 _ll_register_callbacks cb 0x55ffe4dfab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:40.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.827+0000 7fe8af400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffe4f05890 0x55ffe4f0ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.827+0000 7fe8af400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffe4f05890 0x55ffe4f0ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37904/0 (socket says 172.21.15.131:37904) 2024-05-08T22:09:40.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.827+0000 7fe8af400700 1 -- 172.21.15.131:0/1873689022 learned_addr learned my addr 172.21.15.131:0/1873689022 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:40.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.827+0000 7fe8afe00700 1 --2- 172.21.15.131:0/1873689022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffe4f0b720 0x55ffe4f0f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.827+0000 7fe8aea00700 1 --2- 172.21.15.131:0/1873689022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffe4f0ae00 0x55ffe4f0f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.827+0000 7fe8afe00700 1 -- 172.21.15.131:0/1873689022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffe4f05890 msgr2=0x55ffe4f0ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:40.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.827+0000 7fe8afe00700 1 --2- 172.21.15.131:0/1873689022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffe4f05890 0x55ffe4f0ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.827+0000 7fe8afe00700 1 -- 172.21.15.131:0/1873689022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffe4f0ae00 msgr2=0x55ffe4f0f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:40.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.827+0000 7fe8afe00700 1 --2- 172.21.15.131:0/1873689022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffe4f0ae00 0x55ffe4f0f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.827+0000 7fe8afe00700 1 -- 172.21.15.131:0/1873689022 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a001f810 con 0x55ffe4f0b720 2024-05-08T22:09:40.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.829+0000 7fe8afe00700 1 --2- 172.21.15.131:0/1873689022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffe4f0b720 0x55ffe4f0f630 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8008f50 tx=0x7fe8a8008f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9548d235eaa0ca48 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:40.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.829+0000 7fe89d600700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:09:40.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.829+0000 7fe89d600700 1 -- 172.21.15.131:0/1873689022 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8019070 con 0x55ffe4f0b720 2024-05-08T22:09:40.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.829+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/1873689022 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ffe4dae270 con 0x55ffe4f0b720 2024-05-08T22:09:40.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.829+0000 7fe89d600700 1 -- 172.21.15.131:0/1873689022 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fe8a8004bb0 con 0x55ffe4f0b720 2024-05-08T22:09:40.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.829+0000 7fe89d600700 1 -- 172.21.15.131:0/1873689022 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8a800f8f0 con 0x55ffe4f0b720 2024-05-08T22:09:40.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.830+0000 7fe8ca3a6b80 10 client.25997 fetch_fsmap learned FSMap version 25 2024-05-08T22:09:40.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.830+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/1873689022 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ffe4c82b60 con 0x55ffe4f0b720 2024-05-08T22:09:40.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.831+0000 7fe89d600700 1 -- 172.21.15.131:0/1873689022 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ffe4dae270 con 0x55ffe4f0b720 2024-05-08T22:09:40.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.832+0000 7fe89d600700 1 -- 172.21.15.131:0/1873689022 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fe8a801c020 con 0x55ffe4f0b720 2024-05-08T22:09:40.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.832+0000 7fe8ca3a6b80 10 client.25997 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T22:09:40.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.832+0000 7fe8ca3a6b80 10 client.25997 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:40.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.832+0000 7fe8ca3a6b80 20 client.25997 populate_metadata read hostname 'smithi131' 2024-05-08T22:09:40.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.832+0000 7fe8ca3a6b80 1 --2- 172.21.15.131:0/1873689022 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ffe4f1f200 0x55ffe4f21620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.832+0000 7fe8ca3a6b80 4 client.25997 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:09:40.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.832+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/1873689022 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffe4ed4810 con 0x55ffe4f1f200 2024-05-08T22:09:40.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.832+0000 7fe8af400700 1 --2- 172.21.15.131:0/1873689022 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ffe4f1f200 0x55ffe4f21620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.833+0000 7fe8af400700 1 --2- 172.21.15.131:0/1873689022 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ffe4f1f200 0x55ffe4f21620 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d28bd22068fd3dc server_cookie=14add0876bf3f488 in_seq=0 out_seq=0 2024-05-08T22:09:40.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.833+0000 7fe89d600700 10 client.25997 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:40.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.835+0000 7fe89d600700 1 -- 172.21.15.131:0/1873689022 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ffe4ed4810 con 0x55ffe4f1f200 2024-05-08T22:09:40.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.835+0000 7fe89d600700 10 client.25997 handle_command_reply: tid=0 2024-05-08T22:09:40.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.835+0000 7fe8ca3a6b80 1 client.25997 shutdown 2024-05-08T22:09:40.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.836+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/1873689022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffe4f0b720 msgr2=0x55ffe4f0f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:40.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.836+0000 7fe8ca3a6b80 1 --2- 172.21.15.131:0/1873689022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffe4f0b720 0x55ffe4f0f630 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8008f50 tx=0x7fe8a8008f80 comp rx=0 tx=0).stop 2024-05-08T22:09:40.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.836+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/1873689022 shutdown_connections 2024-05-08T22:09:40.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.836+0000 7fe8ca3a6b80 1 --2- 172.21.15.131:0/1873689022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffe4f0b720 0x55ffe4f0f630 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.836+0000 7fe8ca3a6b80 1 --2- 172.21.15.131:0/1873689022 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ffe4f1f200 0x55ffe4f21620 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.836+0000 7fe8ca3a6b80 1 --2- 172.21.15.131:0/1873689022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffe4f05890 0x55ffe4f0ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.836+0000 7fe8ca3a6b80 1 --2- 172.21.15.131:0/1873689022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffe4f0ae00 0x55ffe4f0f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.836+0000 7fe89d600700 0 client.25997 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:40.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.836+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/1873689022 >> 172.21.15.131:0/1873689022 conn(0x55ffe4e63210 msgr2=0x55ffe4eff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:40.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.836+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/1873689022 shutdown_connections 2024-05-08T22:09:40.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.836+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/1873689022 wait complete. 2024-05-08T22:09:40.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.836+0000 7fe8ca3a6b80 20 client.25997 trim_cache size 0 max 16384 2024-05-08T22:09:40.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.855+0000 7fe8ca3a6b80 1 Processor -- start 2024-05-08T22:09:40.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.855+0000 7fe8ca3a6b80 1 -- start start 2024-05-08T22:09:40.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.856+0000 7fe8ca3a6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffe4f05890 0x55ffe4f05350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.856+0000 7fe8ca3a6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffe4f0ae00 0x55ffe4f01a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.856+0000 7fe8ca3a6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffe4f0b720 0x55ffe4effd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.856+0000 7fe8ca3a6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffe4f11720 con 0x55ffe4f0b720 2024-05-08T22:09:40.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.856+0000 7fe8ca3a6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffe4efcdc0 con 0x55ffe4f0ae00 2024-05-08T22:09:40.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.856+0000 7fe8ca3a6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffe4f06280 con 0x55ffe4f05890 2024-05-08T22:09:40.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.856+0000 7fe8aea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffe4f0ae00 0x55ffe4f01a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.856+0000 7fe8af400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffe4f05890 0x55ffe4f05350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.856+0000 7fe8aea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffe4f0ae00 0x55ffe4f01a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:57920/0 (socket says 172.21.15.131:57920) 2024-05-08T22:09:40.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.856+0000 7fe8aea00700 1 -- 172.21.15.131:0/202883672 learned_addr learned my addr 172.21.15.131:0/202883672 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:40.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.856+0000 7fe8afe00700 1 --2- 172.21.15.131:0/202883672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffe4f0b720 0x55ffe4effd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.856+0000 7fe8aea00700 1 -- 172.21.15.131:0/202883672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffe4f05890 msgr2=0x55ffe4f05350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:40.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.856+0000 7fe8aea00700 1 --2- 172.21.15.131:0/202883672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffe4f05890 0x55ffe4f05350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.856+0000 7fe8aea00700 1 -- 172.21.15.131:0/202883672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffe4f0b720 msgr2=0x55ffe4effd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:40.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.856+0000 7fe8aea00700 1 --2- 172.21.15.131:0/202883672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffe4f0b720 0x55ffe4effd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.857+0000 7fe8aea00700 1 -- 172.21.15.131:0/202883672 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffe4e1b560 con 0x55ffe4f0ae00 2024-05-08T22:09:40.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.857+0000 7fe8af400700 1 --2- 172.21.15.131:0/202883672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffe4f05890 0x55ffe4f05350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:09:40.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.857+0000 7fe8aea00700 1 --2- 172.21.15.131:0/202883672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffe4f0ae00 0x55ffe4f01a00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe8980977c0 tx=0x7fe89809ac40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a6383564cf9f9b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:40.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.857+0000 7fe8a5600700 1 -- 172.21.15.131:0/202883672 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89809b450 con 0x55ffe4f0ae00 2024-05-08T22:09:40.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.857+0000 7fe8a5600700 1 -- 172.21.15.131:0/202883672 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe89809ad50 con 0x55ffe4f0ae00 2024-05-08T22:09:40.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.857+0000 7fe8a5600700 1 -- 172.21.15.131:0/202883672 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe898097460 con 0x55ffe4f0ae00 2024-05-08T22:09:40.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.858+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/202883672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffe4f0ae00 msgr2=0x55ffe4f01a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:40.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.858+0000 7fe8ca3a6b80 1 --2- 172.21.15.131:0/202883672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffe4f0ae00 0x55ffe4f01a00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe8980977c0 tx=0x7fe89809ac40 comp rx=0 tx=0).stop 2024-05-08T22:09:40.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.858+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/202883672 shutdown_connections 2024-05-08T22:09:40.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.858+0000 7fe8ca3a6b80 1 --2- 172.21.15.131:0/202883672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffe4f0b720 0x55ffe4effd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.858+0000 7fe8ca3a6b80 1 --2- 172.21.15.131:0/202883672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffe4f05890 0x55ffe4f05350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.858+0000 7fe8ca3a6b80 1 --2- 172.21.15.131:0/202883672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffe4f0ae00 0x55ffe4f01a00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.858+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/202883672 >> 172.21.15.131:0/202883672 conn(0x55ffe4e63210 msgr2=0x55ffe4f0f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:40.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.858+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/202883672 shutdown_connections 2024-05-08T22:09:40.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.858+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/202883672 wait complete. 2024-05-08T22:09:40.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.858+0000 7fe8ca3a6b80 1 -- start start 2024-05-08T22:09:40.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.859+0000 7fe8ca3a6b80 1 Processor -- start 2024-05-08T22:09:40.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.859+0000 7fe8ca3a6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffe4f05890 0x55ffe4f69690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.859+0000 7fe8ca3a6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffe4f0ae00 0x55ffe4f69bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.859+0000 7fe8ca3a6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffe4f0b720 0x55ffe4f6a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.859+0000 7fe8ca3a6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffe4f74840 con 0x55ffe4f0ae00 2024-05-08T22:09:40.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.859+0000 7fe8ca3a6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffe4f749b0 con 0x55ffe4f0b720 2024-05-08T22:09:40.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.859+0000 7fe8ca3a6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffe4f74b70 con 0x55ffe4f05890 2024-05-08T22:09:40.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.859+0000 7fe8ca3a6b80 10 client.0 _ll_register_callbacks cb 0x55ffe4dfab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:40.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.859+0000 7fe8afe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffe4f0b720 0x55ffe4f6a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.859+0000 7fe8afe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffe4f0b720 0x55ffe4f6a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:57924/0 (socket says 172.21.15.131:57924) 2024-05-08T22:09:40.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.859+0000 7fe8af400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffe4f05890 0x55ffe4f69690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.859+0000 7fe8afe00700 1 -- 172.21.15.131:0/3423230934 learned_addr learned my addr 172.21.15.131:0/3423230934 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:40.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.860+0000 7fe8aea00700 1 --2- 172.21.15.131:0/3423230934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffe4f0ae00 0x55ffe4f69bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.860+0000 7fe8afe00700 1 -- 172.21.15.131:0/3423230934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffe4f05890 msgr2=0x55ffe4f69690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:40.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.860+0000 7fe8afe00700 1 --2- 172.21.15.131:0/3423230934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffe4f05890 0x55ffe4f69690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.860+0000 7fe8afe00700 1 -- 172.21.15.131:0/3423230934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffe4f0ae00 msgr2=0x55ffe4f69bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:40.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.860+0000 7fe8afe00700 1 --2- 172.21.15.131:0/3423230934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffe4f0ae00 0x55ffe4f69bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.860+0000 7fe8afe00700 1 -- 172.21.15.131:0/3423230934 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b01a20a0 con 0x55ffe4f0b720 2024-05-08T22:09:40.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.861+0000 7fe8afe00700 1 --2- 172.21.15.131:0/3423230934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffe4f0b720 0x55ffe4f6a110 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8008f20 tx=0x7fe8a800fbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3a4a022ab8bbd86 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:40.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.861+0000 7fe89d600700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:09:40.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.861+0000 7fe89d600700 1 -- 172.21.15.131:0/3423230934 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a802a070 con 0x55ffe4f0b720 2024-05-08T22:09:40.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.861+0000 7fe89d600700 1 -- 172.21.15.131:0/3423230934 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fe8a8007a40 con 0x55ffe4f0b720 2024-05-08T22:09:40.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.861+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/3423230934 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8a0021e40 con 0x55ffe4f0b720 2024-05-08T22:09:40.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.862+0000 7fe89d600700 1 -- 172.21.15.131:0/3423230934 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8a8019040 con 0x55ffe4f0b720 2024-05-08T22:09:40.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.862+0000 7fe8ca3a6b80 10 client.26003 fetch_fsmap learned FSMap version 25 2024-05-08T22:09:40.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.862+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/3423230934 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ffe4cdacd0 con 0x55ffe4f0b720 2024-05-08T22:09:40.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.862+0000 7fe89d600700 1 -- 172.21.15.131:0/3423230934 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8a0021e40 con 0x55ffe4f0b720 2024-05-08T22:09:40.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.862+0000 7fe89d600700 1 -- 172.21.15.131:0/3423230934 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fe8a802d020 con 0x55ffe4f0b720 2024-05-08T22:09:40.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.862+0000 7fe8ca3a6b80 10 client.26003 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T22:09:40.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.862+0000 7fe8ca3a6b80 10 client.26003 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:40.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.862+0000 7fe8ca3a6b80 20 client.26003 populate_metadata read hostname 'smithi131' 2024-05-08T22:09:40.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.862+0000 7fe8ca3a6b80 1 --2- 172.21.15.131:0/3423230934 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ffe4f763d0 0x55ffe4f787f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:40.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.862+0000 7fe8ca3a6b80 4 client.26003 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:09:40.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.862+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/3423230934 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ffe4e4dec0 con 0x55ffe4f763d0 2024-05-08T22:09:40.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.863+0000 7fe8af400700 1 --2- 172.21.15.131:0/3423230934 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ffe4f763d0 0x55ffe4f787f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:40.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.863+0000 7fe8af400700 1 --2- 172.21.15.131:0/3423230934 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ffe4f763d0 0x55ffe4f787f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8d727550754a85c server_cookie=f0492c1366b880bf in_seq=0 out_seq=0 2024-05-08T22:09:40.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.863+0000 7fe89d600700 10 client.26003 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:40.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.866+0000 7fe89d600700 1 -- 172.21.15.131:0/3423230934 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ffe4e4dec0 con 0x55ffe4f763d0 2024-05-08T22:09:40.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.866+0000 7fe89d600700 10 client.26003 handle_command_reply: tid=0 2024-05-08T22:09:40.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.866+0000 7fe8ca3a6b80 1 client.26003 shutdown 2024-05-08T22:09:40.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.866+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/3423230934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffe4f0b720 msgr2=0x55ffe4f6a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:40.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.866+0000 7fe8ca3a6b80 1 --2- 172.21.15.131:0/3423230934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffe4f0b720 0x55ffe4f6a110 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8008f20 tx=0x7fe8a800fbd0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.866+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/3423230934 shutdown_connections 2024-05-08T22:09:40.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.866+0000 7fe8ca3a6b80 1 --2- 172.21.15.131:0/3423230934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffe4f0b720 0x55ffe4f6a110 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.866+0000 7fe8ca3a6b80 1 --2- 172.21.15.131:0/3423230934 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ffe4f763d0 0x55ffe4f787f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.866+0000 7fe8ca3a6b80 1 --2- 172.21.15.131:0/3423230934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ffe4f05890 0x55ffe4f69690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.866+0000 7fe8ca3a6b80 1 --2- 172.21.15.131:0/3423230934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ffe4f0ae00 0x55ffe4f69bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.866+0000 7fe89d600700 0 client.26003 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:40.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.866+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/3423230934 >> 172.21.15.131:0/3423230934 conn(0x55ffe4e63210 msgr2=0x55ffe4f09c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:40.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.867+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/3423230934 shutdown_connections 2024-05-08T22:09:40.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.867+0000 7fe8ca3a6b80 1 -- 172.21.15.131:0/3423230934 wait complete. 2024-05-08T22:09:40.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.867+0000 7fe8ca3a6b80 20 client.26003 trim_cache size 0 max 16384 2024-05-08T22:09:40.869 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:09:40.869 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-08T22:09:40.869 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-08T22:09:40.869 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:09:40.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.868+0000 7fe8b6800700 1 -- 172.21.15.131:0/2307429347 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe8940a12d0 msgr2=0x7fe8940a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:40.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.868+0000 7fe8b6800700 1 --2- 172.21.15.131:0/2307429347 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe8940a12d0 0x7fe8940a3790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0005fd0 tx=0x7fe8a001a320 comp rx=0 tx=0).stop 2024-05-08T22:09:40.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.868+0000 7fe8b6800700 1 -- 172.21.15.131:0/2307429347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8b0073c60 msgr2=0x7fe8b010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:40.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.868+0000 7fe8b6800700 1 --2- 172.21.15.131:0/2307429347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8b0073c60 0x7fe8b010dd30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe898000c00 tx=0x7fe89800fab0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.868+0000 7fe8b6800700 1 -- 172.21.15.131:0/2307429347 shutdown_connections 2024-05-08T22:09:40.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.868+0000 7fe8b6800700 1 --2- 172.21.15.131:0/2307429347 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe8940a12d0 0x7fe8940a3790 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.868+0000 7fe8b6800700 1 --2- 172.21.15.131:0/2307429347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8b0111f80 0x7fe8b0199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.868+0000 7fe8b6800700 1 --2- 172.21.15.131:0/2307429347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8b0073340 0x7fe8b010d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.868+0000 7fe8b6800700 1 --2- 172.21.15.131:0/2307429347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8b0073c60 0x7fe8b010dd30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:40.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.868+0000 7fe8b6800700 1 -- 172.21.15.131:0/2307429347 >> 172.21.15.131:0/2307429347 conn(0x7fe8b00fca80 msgr2=0x7fe8b00fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:40.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.868+0000 7fe8b6800700 1 -- 172.21.15.131:0/2307429347 shutdown_connections 2024-05-08T22:09:40.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:40.868+0000 7fe8b6800700 1 -- 172.21.15.131:0/2307429347 wait complete. 2024-05-08T22:09:40.890 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T22:09:40.890 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:09:41.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:40 smithi157 ceph-mon[77746]: pgmap v74: 129 pgs: 129 active+clean; 78 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 187 op/s 2024-05-08T22:09:41.031 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:40 smithi131 ceph-mon[37260]: pgmap v74: 129 pgs: 129 active+clean; 78 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 187 op/s 2024-05-08T22:09:41.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:40 smithi132 ceph-mon[78180]: pgmap v74: 129 pgs: 129 active+clean; 78 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 187 op/s 2024-05-08T22:09:41.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.237+0000 7f1f4b800700 1 -- 172.21.15.131:0/516618390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f44101a00 msgr2=0x7f1f44105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:41.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.237+0000 7f1f4b800700 1 --2- 172.21.15.131:0/516618390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f44101a00 0x7f1f44105a40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009990 tx=0x7f1f30009ca0 comp rx=0 tx=0).stop 2024-05-08T22:09:41.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.237+0000 7f1f4b800700 1 -- 172.21.15.131:0/516618390 shutdown_connections 2024-05-08T22:09:41.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.237+0000 7f1f4b800700 1 --2- 172.21.15.131:0/516618390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f44106170 0x7f1f4410dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:41.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.237+0000 7f1f4b800700 1 --2- 172.21.15.131:0/516618390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f44101050 0x7f1f44101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:41.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.237+0000 7f1f4b800700 1 --2- 172.21.15.131:0/516618390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f44101a00 0x7f1f44105a40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:41.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.237+0000 7f1f4b800700 1 -- 172.21.15.131:0/516618390 >> 172.21.15.131:0/516618390 conn(0x7f1f440fc8f0 msgr2=0x7f1f440fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:41.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.238+0000 7f1f4b800700 1 -- 172.21.15.131:0/516618390 shutdown_connections 2024-05-08T22:09:41.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.238+0000 7f1f4b800700 1 -- 172.21.15.131:0/516618390 wait complete. 2024-05-08T22:09:41.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.239+0000 7f1f4b800700 1 Processor -- start 2024-05-08T22:09:41.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.239+0000 7f1f4b800700 1 -- start start 2024-05-08T22:09:41.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.239+0000 7f1f4b800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f44101050 0x7f1f440732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:41.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.239+0000 7f1f4b800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f44106170 0x7f1f440737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:41.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.239+0000 7f1f4b800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f44073d20 0x7f1f441a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:41.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.239+0000 7f1f4b800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f441a5b20 con 0x7f1f44106170 2024-05-08T22:09:41.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.239+0000 7f1f4b800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f441a5c60 con 0x7f1f44073d20 2024-05-08T22:09:41.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.239+0000 7f1f4b800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f441a5e20 con 0x7f1f44101050 2024-05-08T22:09:41.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.240+0000 7f1f43e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f44106170 0x7f1f440737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:41.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.240+0000 7f1f48e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f44101050 0x7f1f440732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:41.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.240+0000 7f1f43e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f44106170 0x7f1f440737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58132/0 (socket says 172.21.15.131:58132) 2024-05-08T22:09:41.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.240+0000 7f1f43e00700 1 -- 172.21.15.131:0/1586114421 learned_addr learned my addr 172.21.15.131:0/1586114421 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:41.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.240+0000 7f1f49800700 1 --2- 172.21.15.131:0/1586114421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f44073d20 0x7f1f441a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:41.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.240+0000 7f1f43e00700 1 -- 172.21.15.131:0/1586114421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f44101050 msgr2=0x7f1f440732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:41.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.240+0000 7f1f43e00700 1 --2- 172.21.15.131:0/1586114421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f44101050 0x7f1f440732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:41.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.240+0000 7f1f43e00700 1 -- 172.21.15.131:0/1586114421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f44073d20 msgr2=0x7f1f441a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:41.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.240+0000 7f1f43e00700 1 --2- 172.21.15.131:0/1586114421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f44073d20 0x7f1f441a5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:41.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.240+0000 7f1f43e00700 1 -- 172.21.15.131:0/1586114421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f300095f0 con 0x7f1f44106170 2024-05-08T22:09:41.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.240+0000 7f1f48e00700 1 --2- 172.21.15.131:0/1586114421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f44101050 0x7f1f440732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:09:41.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.240+0000 7f1f49800700 1 --2- 172.21.15.131:0/1586114421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f44073d20 0x7f1f441a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:09:41.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.241+0000 7f1f43e00700 1 --2- 172.21.15.131:0/1586114421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f44106170 0x7f1f440737e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1f30005770 tx=0x7f1f300057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:41.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.241+0000 7f1f41600700 1 -- 172.21.15.131:0/1586114421 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3001d070 con 0x7f1f44106170 2024-05-08T22:09:41.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.241+0000 7f1f41600700 1 -- 172.21.15.131:0/1586114421 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f30021410 con 0x7f1f44106170 2024-05-08T22:09:41.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.241+0000 7f1f4b800700 1 -- 172.21.15.131:0/1586114421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f441a60a0 con 0x7f1f44106170 2024-05-08T22:09:41.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.241+0000 7f1f41600700 1 -- 172.21.15.131:0/1586114421 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f30017650 con 0x7f1f44106170 2024-05-08T22:09:41.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.241+0000 7f1f4b800700 1 -- 172.21.15.131:0/1586114421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f441a6600 con 0x7f1f44106170 2024-05-08T22:09:41.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.242+0000 7f1f4b800700 1 -- 172.21.15.131:0/1586114421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f44109440 con 0x7f1f44106170 2024-05-08T22:09:41.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.245+0000 7f1f41600700 1 -- 172.21.15.131:0/1586114421 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f1f3000fb00 con 0x7f1f44106170 2024-05-08T22:09:41.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.246+0000 7f1f41600700 1 --2- 172.21.15.131:0/1586114421 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1f2c0a1310 0x7f1f2c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:41.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.246+0000 7f1f41600700 1 -- 172.21.15.131:0/1586114421 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f1f300ce800 con 0x7f1f44106170 2024-05-08T22:09:41.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.246+0000 7f1f48e00700 1 --2- 172.21.15.131:0/1586114421 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1f2c0a1310 0x7f1f2c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:41.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.248+0000 7f1f48e00700 1 --2- 172.21.15.131:0/1586114421 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1f2c0a1310 0x7f1f2c0a37d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1f38011490 tx=0x7f1f38009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:41.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:41.250+0000 7f1f41600700 1 -- 172.21.15.131:0/1586114421 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1f300978f0 con 0x7f1f44106170 2024-05-08T22:09:43.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:42.993+0000 7f1f4b800700 1 -- 172.21.15.131:0/1586114421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f44074730 con 0x7f1f44106170 2024-05-08T22:09:43.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:43.007+0000 7f1f41600700 1 -- 172.21.15.131:0/1586114421 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10890 (secure 0 0 0) 0x7f1f3009af10 con 0x7f1f44106170 2024-05-08T22:09:43.025 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:42 smithi157 ceph-mon[77746]: pgmap v75: 129 pgs: 129 active+clean; 78 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 77 op/s 2024-05-08T22:09:43.033 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:09:43.033 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:05:14.034908+0000","modified":"2024-05-08T22:09:33.712734+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:09:43.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:43.073+0000 7f1f4b800700 1 -- 172.21.15.131:0/1586114421 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1f2c0a1310 msgr2=0x7f1f2c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:43.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:43.073+0000 7f1f4b800700 1 --2- 172.21.15.131:0/1586114421 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1f2c0a1310 0x7f1f2c0a37d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1f38011490 tx=0x7f1f38009260 comp rx=0 tx=0).stop 2024-05-08T22:09:43.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:43.073+0000 7f1f4b800700 1 -- 172.21.15.131:0/1586114421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f44106170 msgr2=0x7f1f440737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:43.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:43.073+0000 7f1f4b800700 1 --2- 172.21.15.131:0/1586114421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f44106170 0x7f1f440737e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1f30005770 tx=0x7f1f300057a0 comp rx=0 tx=0).stop 2024-05-08T22:09:43.089 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:42 smithi132 ceph-mon[78180]: pgmap v75: 129 pgs: 129 active+clean; 78 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 77 op/s 2024-05-08T22:09:43.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:43.093+0000 7f1f4b800700 1 -- 172.21.15.131:0/1586114421 shutdown_connections 2024-05-08T22:09:43.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:43.093+0000 7f1f4b800700 1 --2- 172.21.15.131:0/1586114421 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1f2c0a1310 0x7f1f2c0a37d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:43.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:43.093+0000 7f1f4b800700 1 --2- 172.21.15.131:0/1586114421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f44073d20 0x7f1f441a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:43.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:43.093+0000 7f1f4b800700 1 --2- 172.21.15.131:0/1586114421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f44101050 0x7f1f440732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:43.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:43.093+0000 7f1f4b800700 1 --2- 172.21.15.131:0/1586114421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f44106170 0x7f1f440737e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:43.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:43.093+0000 7f1f4b800700 1 -- 172.21.15.131:0/1586114421 >> 172.21.15.131:0/1586114421 conn(0x7f1f440fc8f0 msgr2=0x7f1f440fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:43.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:43.126+0000 7f1f4b800700 1 -- 172.21.15.131:0/1586114421 shutdown_connections 2024-05-08T22:09:43.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:43.126+0000 7f1f4b800700 1 -- 172.21.15.131:0/1586114421 wait complete. 2024-05-08T22:09:43.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:42 smithi131 ceph-mon[37260]: pgmap v75: 129 pgs: 129 active+clean; 78 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 77 op/s 2024-05-08T22:09:43.158 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 25 2024-05-08T22:09:43.411 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T22:09:44.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:43 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1586114421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:09:44.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:43 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1586114421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:09:44.112 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:43 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1586114421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:09:44.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.474+0000 7f58a9600700 1 -- 172.21.15.131:0/2892070152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a4105ed0 msgr2=0x7f58a41062b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:44.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.474+0000 7f58a9600700 1 --2- 172.21.15.131:0/2892070152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a4105ed0 0x7f58a41062b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f58940099c0 tx=0x7f5894009cd0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.474+0000 7f58a9600700 1 -- 172.21.15.131:0/2892070152 shutdown_connections 2024-05-08T22:09:44.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.474+0000 7f58a9600700 1 --2- 172.21.15.131:0/2892070152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58a40689f0 0x7f58a410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.474+0000 7f58a9600700 1 --2- 172.21.15.131:0/2892070152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a4105ed0 0x7f58a41062b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.474+0000 7f58a9600700 1 --2- 172.21.15.131:0/2892070152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58a4068030 0x7f58a40684b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.474+0000 7f58a9600700 1 -- 172.21.15.131:0/2892070152 >> 172.21.15.131:0/2892070152 conn(0x7f58a4076f40 msgr2=0x7f58a4077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:44.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.475+0000 7f58a9600700 1 -- 172.21.15.131:0/2892070152 shutdown_connections 2024-05-08T22:09:44.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.475+0000 7f58a9600700 1 -- 172.21.15.131:0/2892070152 wait complete. 2024-05-08T22:09:44.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.476+0000 7f58a9600700 1 Processor -- start 2024-05-08T22:09:44.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.476+0000 7f58a9600700 1 -- start start 2024-05-08T22:09:44.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.477+0000 7f58a9600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a4068030 0x7f58a40ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.477+0000 7f58a9600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58a40689f0 0x7f58a41003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.477+0000 7f58a9600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58a4105ed0 0x7f58a41008e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.477+0000 7f58a9600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a4101e30 con 0x7f58a40689f0 2024-05-08T22:09:44.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.477+0000 7f58a9600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a419b560 con 0x7f58a4068030 2024-05-08T22:09:44.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.477+0000 7f58a9600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a419b720 con 0x7f58a4105ed0 2024-05-08T22:09:44.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.477+0000 7f58a2a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a4068030 0x7f58a40ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:44.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.477+0000 7f58a2a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a4068030 0x7f58a40ffe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:33332/0 (socket says 172.21.15.131:33332) 2024-05-08T22:09:44.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.477+0000 7f58a2a00700 1 -- 172.21.15.131:0/1553406928 learned_addr learned my addr 172.21.15.131:0/1553406928 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:44.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.478+0000 7f58a2000700 1 --2- 172.21.15.131:0/1553406928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58a40689f0 0x7f58a41003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:44.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.478+0000 7f58a2a00700 1 -- 172.21.15.131:0/1553406928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58a4105ed0 msgr2=0x7f58a41008e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T22:09:44.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.478+0000 7f58a2a00700 1 --2- 172.21.15.131:0/1553406928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58a4105ed0 0x7f58a41008e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.478+0000 7f58a2a00700 1 -- 172.21.15.131:0/1553406928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58a40689f0 msgr2=0x7f58a41003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:44.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.478+0000 7f58a2a00700 1 --2- 172.21.15.131:0/1553406928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58a40689f0 0x7f58a41003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.478+0000 7f58a2a00700 1 -- 172.21.15.131:0/1553406928 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58940095f0 con 0x7f58a4068030 2024-05-08T22:09:44.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.478+0000 7f58a2000700 1 --2- 172.21.15.131:0/1553406928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58a40689f0 0x7f58a41003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:09:44.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.478+0000 7f58a2a00700 1 --2- 172.21.15.131:0/1553406928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a4068030 0x7f58a40ffe60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f589400b9a0 tx=0x7f5894005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:44.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.478+0000 7f589b400700 1 -- 172.21.15.131:0/1553406928 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589401d070 con 0x7f58a4068030 2024-05-08T22:09:44.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.478+0000 7f589b400700 1 -- 172.21.15.131:0/1553406928 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f589400beb0 con 0x7f58a4068030 2024-05-08T22:09:44.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.478+0000 7f589b400700 1 -- 172.21.15.131:0/1553406928 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5894003720 con 0x7f58a4068030 2024-05-08T22:09:44.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.478+0000 7f58a9600700 1 -- 172.21.15.131:0/1553406928 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a419b9a0 con 0x7f58a4068030 2024-05-08T22:09:44.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.479+0000 7f58a9600700 1 -- 172.21.15.131:0/1553406928 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a419bda0 con 0x7f58a4068030 2024-05-08T22:09:44.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.483+0000 7f589b400700 1 -- 172.21.15.131:0/1553406928 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f5894004830 con 0x7f58a4068030 2024-05-08T22:09:44.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.485+0000 7f589b400700 1 --2- 172.21.15.131:0/1553406928 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f58880a1310 0x7f58880a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.485+0000 7f589b400700 1 -- 172.21.15.131:0/1553406928 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f58940cdb10 con 0x7f58a4068030 2024-05-08T22:09:44.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.485+0000 7f58bd1ccb80 1 Processor -- start 2024-05-08T22:09:44.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.485+0000 7f58bd1ccb80 1 -- start start 2024-05-08T22:09:44.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.486+0000 7f58a2000700 1 --2- 172.21.15.131:0/1553406928 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f58880a1310 0x7f58880a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:44.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.486+0000 7f58bd1ccb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559085b20e00 0x559085b211e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.486+0000 7f58bd1ccb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559085b21720 0x559085b1b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.486+0000 7f58bd1ccb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559085b1b890 0x559085b1bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.486+0000 7f58bd1ccb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559085a72150 con 0x559085b20e00 2024-05-08T22:09:44.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.486+0000 7f58bd1ccb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559085a6fb10 con 0x559085b21720 2024-05-08T22:09:44.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.486+0000 7f58bd1ccb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559085a6b400 con 0x559085b1b890 2024-05-08T22:09:44.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.486+0000 7f58a2000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559085b21720 0x559085b1b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:44.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.487+0000 7f58a2000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559085b21720 0x559085b1b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:33346/0 (socket says 172.21.15.131:33346) 2024-05-08T22:09:44.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.487+0000 7f58a2000700 1 -- 172.21.15.131:0/3233088982 learned_addr learned my addr 172.21.15.131:0/3233088982 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:44.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.487+0000 7f58a3400700 1 --2- 172.21.15.131:0/3233088982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559085b1b890 0x559085b1bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:44.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.487+0000 7f58a2000700 1 --2- 172.21.15.131:0/1553406928 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f58880a1310 0x7f58880a37d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f58a4101590 tx=0x7f5890009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:44.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.487+0000 7f58a2a00700 1 --2- 172.21.15.131:0/3233088982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559085b20e00 0x559085b211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:44.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.487+0000 7f58a2000700 1 -- 172.21.15.131:0/3233088982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559085b1b890 msgr2=0x559085b1bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:44.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.487+0000 7f58a2000700 1 --2- 172.21.15.131:0/3233088982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559085b1b890 0x559085b1bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.487+0000 7f58a2000700 1 -- 172.21.15.131:0/3233088982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559085b20e00 msgr2=0x559085b211e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:44.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.487+0000 7f58a2000700 1 --2- 172.21.15.131:0/3233088982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559085b20e00 0x559085b211e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.487+0000 7f58a2000700 1 -- 172.21.15.131:0/3233088982 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590858ec3b0 con 0x559085b21720 2024-05-08T22:09:44.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.487+0000 7f58a3400700 1 --2- 172.21.15.131:0/3233088982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559085b1b890 0x559085b1bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:09:44.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.487+0000 7f58a2a00700 1 --2- 172.21.15.131:0/3233088982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559085b20e00 0x559085b211e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:09:44.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.488+0000 7f58a2000700 1 --2- 172.21.15.131:0/3233088982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559085b21720 0x559085b1b350 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5890010d70 tx=0x7f589001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a6f740a2c9a4de6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:44.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.488+0000 7f588f400700 1 -- 172.21.15.131:0/3233088982 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890018040 con 0x559085b21720 2024-05-08T22:09:44.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.488+0000 7f588f400700 1 -- 172.21.15.131:0/3233088982 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5890009870 con 0x559085b21720 2024-05-08T22:09:44.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.488+0000 7f588f400700 1 -- 172.21.15.131:0/3233088982 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589001edf0 con 0x559085b21720 2024-05-08T22:09:44.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.489+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/3233088982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559085b21720 msgr2=0x559085b1b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:44.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.489+0000 7f58bd1ccb80 1 --2- 172.21.15.131:0/3233088982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559085b21720 0x559085b1b350 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5890010d70 tx=0x7f589001b000 comp rx=0 tx=0).stop 2024-05-08T22:09:44.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.489+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/3233088982 shutdown_connections 2024-05-08T22:09:44.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.489+0000 7f58bd1ccb80 1 --2- 172.21.15.131:0/3233088982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559085b1b890 0x559085b1bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.489+0000 7f58bd1ccb80 1 --2- 172.21.15.131:0/3233088982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559085b20e00 0x559085b211e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.489+0000 7f58bd1ccb80 1 --2- 172.21.15.131:0/3233088982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559085b21720 0x559085b1b350 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.489+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/3233088982 >> 172.21.15.131:0/3233088982 conn(0x559085a79210 msgr2=0x559085b15df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:44.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.489+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/3233088982 shutdown_connections 2024-05-08T22:09:44.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.490+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/3233088982 wait complete. 2024-05-08T22:09:44.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.491+0000 7f58bd1ccb80 1 -- start start 2024-05-08T22:09:44.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.491+0000 7f58bd1ccb80 1 Processor -- start 2024-05-08T22:09:44.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.491+0000 7f58bd1ccb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559085b1b890 0x559085b24af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.491+0000 7f58bd1ccb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559085b20e00 0x559085b25030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.491+0000 7f58bd1ccb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559085b21720 0x559085b25570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.491+0000 7f58bd1ccb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559085a07560 con 0x559085b1b890 2024-05-08T22:09:44.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.491+0000 7f58bd1ccb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559085a07200 con 0x559085b20e00 2024-05-08T22:09:44.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.491+0000 7f58bd1ccb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590858ee6a0 con 0x559085b21720 2024-05-08T22:09:44.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.491+0000 7f58bd1ccb80 10 client.0 _ll_register_callbacks cb 0x559085a10b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:44.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.492+0000 7f58a2000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559085b20e00 0x559085b25030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:44.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.492+0000 7f58a2000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559085b20e00 0x559085b25030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:33356/0 (socket says 172.21.15.131:33356) 2024-05-08T22:09:44.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.492+0000 7f58a2000700 1 -- 172.21.15.131:0/3425756432 learned_addr learned my addr 172.21.15.131:0/3425756432 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:44.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.492+0000 7f58a3400700 1 --2- 172.21.15.131:0/3425756432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559085b21720 0x559085b25570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:44.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.492+0000 7f58a2000700 1 -- 172.21.15.131:0/3425756432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559085b21720 msgr2=0x559085b25570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:44.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.492+0000 7f58a2000700 1 --2- 172.21.15.131:0/3425756432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559085b21720 0x559085b25570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.492+0000 7f58a2000700 1 -- 172.21.15.131:0/3425756432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559085b1b890 msgr2=0x559085b24af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:44.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.492+0000 7f58a2000700 1 --2- 172.21.15.131:0/3425756432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559085b1b890 0x559085b24af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.492+0000 7f58a2000700 1 -- 172.21.15.131:0/3425756432 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5890010a50 con 0x559085b20e00 2024-05-08T22:09:44.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.492+0000 7f58a2000700 1 --2- 172.21.15.131:0/3425756432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559085b20e00 0x559085b25030 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f589001e690 tx=0x7f589001fb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=843bb491c6aaf9bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:44.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.492+0000 7f5883e00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:09:44.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.493+0000 7f5883e00700 1 -- 172.21.15.131:0/3425756432 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589001ff10 con 0x559085b20e00 2024-05-08T22:09:44.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.493+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/3425756432 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559085aea810 con 0x559085b20e00 2024-05-08T22:09:44.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.493+0000 7f5883e00700 1 -- 172.21.15.131:0/3425756432 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f5890012ae0 con 0x559085b20e00 2024-05-08T22:09:44.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.493+0000 7f5883e00700 1 -- 172.21.15.131:0/3425756432 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5890012ca0 con 0x559085b20e00 2024-05-08T22:09:44.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.493+0000 7f58bd1ccb80 10 client.26021 fetch_fsmap learned FSMap version 25 2024-05-08T22:09:44.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.493+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/3425756432 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559085898b60 con 0x559085b20e00 2024-05-08T22:09:44.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.493+0000 7f5883e00700 1 -- 172.21.15.131:0/3425756432 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559085aea810 con 0x559085b20e00 2024-05-08T22:09:44.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.493+0000 7f5883e00700 1 -- 172.21.15.131:0/3425756432 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f5890024020 con 0x559085b20e00 2024-05-08T22:09:44.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.494+0000 7f58bd1ccb80 10 client.26021 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T22:09:44.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.494+0000 7f58bd1ccb80 10 client.26021 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:09:44.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.494+0000 7f58bd1ccb80 20 client.26021 populate_metadata read hostname 'smithi131' 2024-05-08T22:09:44.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.494+0000 7f58bd1ccb80 1 --2- 172.21.15.131:0/3425756432 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x559085b34ff0 0x559085b37410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.494+0000 7f58bd1ccb80 4 client.26021 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:09:44.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.494+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/3425756432 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559085a63ec0 con 0x559085b34ff0 2024-05-08T22:09:44.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.494+0000 7f58a2a00700 1 --2- 172.21.15.131:0/3425756432 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x559085b34ff0 0x559085b37410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.494+0000 7f58a2a00700 1 --2- 172.21.15.131:0/3425756432 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x559085b34ff0 0x559085b37410 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6e2818eddc5ab809 server_cookie=589edac667e03261 in_seq=0 out_seq=0 2024-05-08T22:09:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.494+0000 7f5883e00700 10 client.26021 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:09:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.495+0000 7f5883e00700 1 -- 172.21.15.131:0/3425756432 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559085a63ec0 con 0x559085b34ff0 2024-05-08T22:09:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.495+0000 7f5883e00700 10 client.26021 handle_command_reply: tid=0 2024-05-08T22:09:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.495+0000 7f58bd1ccb80 1 client.26021 shutdown 2024-05-08T22:09:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.496+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/3425756432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559085b20e00 msgr2=0x559085b25030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.496+0000 7f58bd1ccb80 1 --2- 172.21.15.131:0/3425756432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559085b20e00 0x559085b25030 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f589001e690 tx=0x7f589001fb30 comp rx=0 tx=0).stop 2024-05-08T22:09:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.496+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/3425756432 shutdown_connections 2024-05-08T22:09:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.496+0000 7f58bd1ccb80 1 --2- 172.21.15.131:0/3425756432 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x559085b34ff0 0x559085b37410 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.496+0000 7f58bd1ccb80 1 --2- 172.21.15.131:0/3425756432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559085b21720 0x559085b25570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.496+0000 7f58bd1ccb80 1 --2- 172.21.15.131:0/3425756432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559085b1b890 0x559085b24af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.496+0000 7f58bd1ccb80 1 --2- 172.21.15.131:0/3425756432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559085b20e00 0x559085b25030 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.496+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/3425756432 >> 172.21.15.131:0/3425756432 conn(0x559085a79210 msgr2=0x559085b155a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.496+0000 7f5883e00700 0 client.26021 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:09:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.496+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/3425756432 shutdown_connections 2024-05-08T22:09:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.496+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/3425756432 wait complete. 2024-05-08T22:09:44.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.496+0000 7f58bd1ccb80 20 client.26021 trim_cache size 0 max 16384 2024-05-08T22:09:44.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.511+0000 7f58bd1ccb80 1 Processor -- start 2024-05-08T22:09:44.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.511+0000 7f58bd1ccb80 1 -- start start 2024-05-08T22:09:44.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.512+0000 7f58bd1ccb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559085b1b890 0x559085b179d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.512+0000 7f58bd1ccb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559085b20e00 0x559085b15d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.512+0000 7f58bd1ccb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559085b21720 0x559085b25390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.512+0000 7f58bd1ccb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559085b3c620 con 0x559085b1b890 2024-05-08T22:09:44.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.512+0000 7f58bd1ccb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559085b0e040 con 0x559085b21720 2024-05-08T22:09:44.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.512+0000 7f58bd1ccb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559085b16260 con 0x559085b20e00 2024-05-08T22:09:44.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.512+0000 7f58a2a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559085b1b890 0x559085b179d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:44.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.512+0000 7f58a2000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559085b20e00 0x559085b15d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:44.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.512+0000 7f58a2000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559085b20e00 0x559085b15d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51466/0 (socket says 172.21.15.131:51466) 2024-05-08T22:09:44.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.512+0000 7f58a2000700 1 -- 172.21.15.131:0/1161028775 learned_addr learned my addr 172.21.15.131:0/1161028775 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:44.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.512+0000 7f58a2000700 1 -- 172.21.15.131:0/1161028775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559085b21720 msgr2=0x559085b25390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:44.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.512+0000 7f58a2000700 1 --2- 172.21.15.131:0/1161028775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559085b21720 0x559085b25390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.512+0000 7f58a2000700 1 -- 172.21.15.131:0/1161028775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559085b1b890 msgr2=0x559085b179d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:44.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.512+0000 7f58a2000700 1 --2- 172.21.15.131:0/1161028775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559085b1b890 0x559085b179d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.512+0000 7f58a2000700 1 -- 172.21.15.131:0/1161028775 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559085a31560 con 0x559085b20e00 2024-05-08T22:09:44.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.513+0000 7f58a2000700 1 --2- 172.21.15.131:0/1161028775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559085b20e00 0x559085b15d20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5890003b00 tx=0x7f589001fc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2d497a05b0d6876 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:44.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.513+0000 7f588fe00700 1 -- 172.21.15.131:0/1161028775 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890003c20 con 0x559085b20e00 2024-05-08T22:09:44.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.513+0000 7f588fe00700 1 -- 172.21.15.131:0/1161028775 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5890003d80 con 0x559085b20e00 2024-05-08T22:09:44.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.513+0000 7f588fe00700 1 -- 172.21.15.131:0/1161028775 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890012730 con 0x559085b20e00 2024-05-08T22:09:44.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.514+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/1161028775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559085b20e00 msgr2=0x559085b15d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:44.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.514+0000 7f58bd1ccb80 1 --2- 172.21.15.131:0/1161028775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559085b20e00 0x559085b15d20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5890003b00 tx=0x7f589001fc50 comp rx=0 tx=0).stop 2024-05-08T22:09:44.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.516+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/1161028775 shutdown_connections 2024-05-08T22:09:44.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.516+0000 7f58bd1ccb80 1 --2- 172.21.15.131:0/1161028775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559085b21720 0x559085b25390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.516+0000 7f58bd1ccb80 1 --2- 172.21.15.131:0/1161028775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559085b1b890 0x559085b179d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.516+0000 7f58bd1ccb80 1 --2- 172.21.15.131:0/1161028775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559085b20e00 0x559085b15d20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.516+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/1161028775 >> 172.21.15.131:0/1161028775 conn(0x559085a79210 msgr2=0x559085b1bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:44.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.516+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/1161028775 shutdown_connections 2024-05-08T22:09:44.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.516+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/1161028775 wait complete. 2024-05-08T22:09:44.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.516+0000 7f58bd1ccb80 1 -- start start 2024-05-08T22:09:44.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.517+0000 7f58bd1ccb80 1 Processor -- start 2024-05-08T22:09:44.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.517+0000 7f58bd1ccb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559085b1b890 0x559085b2c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.517+0000 7f58bd1ccb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559085b20e00 0x559085b2c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.517+0000 7f58bd1ccb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559085b21720 0x559085b29720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.518+0000 7f58bd1ccb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559085b29df0 con 0x559085b20e00 2024-05-08T22:09:44.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.518+0000 7f58bd1ccb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559085b29f60 con 0x559085b21720 2024-05-08T22:09:44.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.518+0000 7f58bd1ccb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559085b2a120 con 0x559085b1b890 2024-05-08T22:09:44.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.518+0000 7f58bd1ccb80 10 client.0 _ll_register_callbacks cb 0x559085a10b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:44.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.518+0000 7f58a2a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559085b1b890 0x559085b2c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:44.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.518+0000 7f58a2000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559085b20e00 0x559085b2c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:44.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.518+0000 7f58a2a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559085b1b890 0x559085b2c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51480/0 (socket says 172.21.15.131:51480) 2024-05-08T22:09:44.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.518+0000 7f58a2a00700 1 -- 172.21.15.131:0/2218576598 learned_addr learned my addr 172.21.15.131:0/2218576598 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:44.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.518+0000 7f58a3400700 1 --2- 172.21.15.131:0/2218576598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559085b21720 0x559085b29720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:44.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.518+0000 7f58a2000700 1 -- 172.21.15.131:0/2218576598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559085b1b890 msgr2=0x559085b2c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:44.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.518+0000 7f58a2000700 1 --2- 172.21.15.131:0/2218576598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559085b1b890 0x559085b2c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.518+0000 7f58a2000700 1 -- 172.21.15.131:0/2218576598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559085b21720 msgr2=0x559085b29720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:44.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.518+0000 7f58a2000700 1 --2- 172.21.15.131:0/2218576598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559085b21720 0x559085b29720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.518+0000 7f58a2000700 1 -- 172.21.15.131:0/2218576598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559085898b60 con 0x559085b20e00 2024-05-08T22:09:44.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.519+0000 7f58a2000700 1 --2- 172.21.15.131:0/2218576598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559085b20e00 0x559085b2c680 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x559085b16ba0 tx=0x7f589001e740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b75990c7d680b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:44.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.519+0000 7f5883e00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:09:44.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.519+0000 7f5883e00700 1 -- 172.21.15.131:0/2218576598 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890003c20 con 0x559085b20e00 2024-05-08T22:09:44.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.519+0000 7f5883e00700 1 -- 172.21.15.131:0/2218576598 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f589001ea90 con 0x559085b20e00 2024-05-08T22:09:44.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.519+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/2218576598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5590859c4270 con 0x559085b20e00 2024-05-08T22:09:44.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.519+0000 7f5883e00700 1 -- 172.21.15.131:0/2218576598 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f589001ec70 con 0x559085b20e00 2024-05-08T22:09:44.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.519+0000 7f58bd1ccb80 10 client.15960 fetch_fsmap learned FSMap version 25 2024-05-08T22:09:44.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.519+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/2218576598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5590858f0cd0 con 0x559085b20e00 2024-05-08T22:09:44.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.520+0000 7f5883e00700 1 -- 172.21.15.131:0/2218576598 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5890050080 con 0x559085b20e00 2024-05-08T22:09:44.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.520+0000 7f5883e00700 1 -- 172.21.15.131:0/2218576598 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f5890012450 con 0x559085b20e00 2024-05-08T22:09:44.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.520+0000 7f58bd1ccb80 10 client.15960 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T22:09:44.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.520+0000 7f58bd1ccb80 10 client.15960 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:09:44.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.520+0000 7f58bd1ccb80 20 client.15960 populate_metadata read hostname 'smithi131' 2024-05-08T22:09:44.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.520+0000 7f58bd1ccb80 1 --2- 172.21.15.131:0/2218576598 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x559085b8b390 0x559085b8d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.520+0000 7f58bd1ccb80 4 client.15960 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:09:44.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.520+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/2218576598 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5590858b9cf0 con 0x559085b8b390 2024-05-08T22:09:44.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.520+0000 7f58a2a00700 1 --2- 172.21.15.131:0/2218576598 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x559085b8b390 0x559085b8d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:44.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.521+0000 7f58a2a00700 1 --2- 172.21.15.131:0/2218576598 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x559085b8b390 0x559085b8d780 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ce076688579ee0c8 server_cookie=b1eff8ce3bb25ae7 in_seq=0 out_seq=0 2024-05-08T22:09:44.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.521+0000 7f5883e00700 10 client.15960 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:09:44.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.521+0000 7f5883e00700 1 -- 172.21.15.131:0/2218576598 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5590858b9cf0 con 0x559085b8b390 2024-05-08T22:09:44.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.521+0000 7f5883e00700 10 client.15960 handle_command_reply: tid=0 2024-05-08T22:09:44.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.521+0000 7f58bd1ccb80 1 client.15960 shutdown 2024-05-08T22:09:44.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.522+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/2218576598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559085b20e00 msgr2=0x559085b2c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:44.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.522+0000 7f58bd1ccb80 1 --2- 172.21.15.131:0/2218576598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559085b20e00 0x559085b2c680 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x559085b16ba0 tx=0x7f589001e740 comp rx=0 tx=0).stop 2024-05-08T22:09:44.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.522+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/2218576598 shutdown_connections 2024-05-08T22:09:44.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.522+0000 7f58bd1ccb80 1 --2- 172.21.15.131:0/2218576598 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x559085b8b390 0x559085b8d780 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.522+0000 7f58bd1ccb80 1 --2- 172.21.15.131:0/2218576598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559085b21720 0x559085b29720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.522+0000 7f58bd1ccb80 1 --2- 172.21.15.131:0/2218576598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559085b1b890 0x559085b2c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.522+0000 7f5883e00700 0 client.15960 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:09:44.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.522+0000 7f58bd1ccb80 1 --2- 172.21.15.131:0/2218576598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559085b20e00 0x559085b2c680 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.522+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/2218576598 >> 172.21.15.131:0/2218576598 conn(0x559085a79210 msgr2=0x559085b253d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:44.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.522+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/2218576598 shutdown_connections 2024-05-08T22:09:44.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.522+0000 7f58bd1ccb80 1 -- 172.21.15.131:0/2218576598 wait complete. 2024-05-08T22:09:44.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.523+0000 7f58bd1ccb80 20 client.15960 trim_cache size 0 max 16384 2024-05-08T22:09:44.525 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:09:44.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.523+0000 7f58a9600700 1 -- 172.21.15.131:0/1553406928 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f58880a1310 msgr2=0x7f58880a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:44.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.523+0000 7f58a9600700 1 --2- 172.21.15.131:0/1553406928 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f58880a1310 0x7f58880a37d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f58a4101590 tx=0x7f5890009260 comp rx=0 tx=0).stop 2024-05-08T22:09:44.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.523+0000 7f58a9600700 1 -- 172.21.15.131:0/1553406928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a4068030 msgr2=0x7f58a40ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:44.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.523+0000 7f58a9600700 1 --2- 172.21.15.131:0/1553406928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a4068030 0x7f58a40ffe60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f589400b9a0 tx=0x7f5894005f10 comp rx=0 tx=0).stop 2024-05-08T22:09:44.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.523+0000 7f58a9600700 1 -- 172.21.15.131:0/1553406928 shutdown_connections 2024-05-08T22:09:44.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.523+0000 7f58a9600700 1 --2- 172.21.15.131:0/1553406928 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f58880a1310 0x7f58880a37d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.523+0000 7f58a9600700 1 --2- 172.21.15.131:0/1553406928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58a4105ed0 0x7f58a41008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.523+0000 7f58a9600700 1 --2- 172.21.15.131:0/1553406928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a4068030 0x7f58a40ffe60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.523+0000 7f58a9600700 1 --2- 172.21.15.131:0/1553406928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58a40689f0 0x7f58a41003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.523+0000 7f58a9600700 1 -- 172.21.15.131:0/1553406928 >> 172.21.15.131:0/1553406928 conn(0x7f58a4076f40 msgr2=0x7f58a40fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:44.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.524+0000 7f58a9600700 1 -- 172.21.15.131:0/1553406928 shutdown_connections 2024-05-08T22:09:44.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.524+0000 7f58a9600700 1 -- 172.21.15.131:0/1553406928 wait complete. 2024-05-08T22:09:44.546 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T22:09:44.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.879+0000 7feab4c00700 1 -- 172.21.15.131:0/605844997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feab006d2b0 msgr2=0x7feab0107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:44.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.879+0000 7feab4c00700 1 --2- 172.21.15.131:0/605844997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feab006d2b0 0x7feab0107fa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fea98009990 tx=0x7fea98009ca0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.885+0000 7feab4c00700 1 -- 172.21.15.131:0/605844997 shutdown_connections 2024-05-08T22:09:44.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.885+0000 7feab4c00700 1 --2- 172.21.15.131:0/605844997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feab0113ce0 0x7feab01160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.885+0000 7feab4c00700 1 --2- 172.21.15.131:0/605844997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feab006d2b0 0x7feab0107fa0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.885+0000 7feab4c00700 1 --2- 172.21.15.131:0/605844997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feab0108570 0x7feab01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.885+0000 7feab4c00700 1 -- 172.21.15.131:0/605844997 >> 172.21.15.131:0/605844997 conn(0x7feab006be80 msgr2=0x7feab006c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:44.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.888+0000 7feab4c00700 1 -- 172.21.15.131:0/605844997 shutdown_connections 2024-05-08T22:09:44.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:44 smithi131 ceph-mon[37260]: pgmap v76: 129 pgs: 129 active+clean; 78 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 77 op/s 2024-05-08T22:09:44.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.888+0000 7feab4c00700 1 -- 172.21.15.131:0/605844997 wait complete. 2024-05-08T22:09:44.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.891+0000 7feab4c00700 1 Processor -- start 2024-05-08T22:09:44.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.892+0000 7feab4c00700 1 -- start start 2024-05-08T22:09:44.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.894+0000 7feab4c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feab006d2b0 0x7feab010ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.894+0000 7feab4c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feab0108570 0x7feab010f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.894+0000 7feaade00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feab006d2b0 0x7feab010ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:44.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.895+0000 7feaade00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feab006d2b0 0x7feab010ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51496/0 (socket says 172.21.15.131:51496) 2024-05-08T22:09:44.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.895+0000 7feaade00700 1 -- 172.21.15.131:0/2916252828 learned_addr learned my addr 172.21.15.131:0/2916252828 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:44.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.895+0000 7feab4c00700 1 --2- 172.21.15.131:0/2916252828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feab0113ce0 0x7feab0109e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.895+0000 7feab4c00700 1 -- 172.21.15.131:0/2916252828 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab010a650 con 0x7feab0108570 2024-05-08T22:09:44.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.895+0000 7feab4c00700 1 -- 172.21.15.131:0/2916252828 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab010a7c0 con 0x7feab0113ce0 2024-05-08T22:09:44.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.895+0000 7feab4c00700 1 -- 172.21.15.131:0/2916252828 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab010a980 con 0x7feab006d2b0 2024-05-08T22:09:44.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.895+0000 7feaae800700 1 --2- 172.21.15.131:0/2916252828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feab0113ce0 0x7feab0109e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:44.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.896+0000 7feaade00700 1 -- 172.21.15.131:0/2916252828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feab0113ce0 msgr2=0x7feab0109e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:44.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.896+0000 7feaade00700 1 --2- 172.21.15.131:0/2916252828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feab0113ce0 0x7feab0109e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.896+0000 7feaade00700 1 -- 172.21.15.131:0/2916252828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feab0108570 msgr2=0x7feab010f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:44.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.896+0000 7feaade00700 1 --2- 172.21.15.131:0/2916252828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feab0108570 0x7feab010f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.896+0000 7feaade00700 1 -- 172.21.15.131:0/2916252828 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea980095f0 con 0x7feab006d2b0 2024-05-08T22:09:44.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.896+0000 7feaae800700 1 --2- 172.21.15.131:0/2916252828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feab0113ce0 0x7feab0109e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:09:44.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.900+0000 7feaade00700 1 --2- 172.21.15.131:0/2916252828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feab006d2b0 0x7feab010ed00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fea98005b40 tx=0x7fea98003b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:44.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.900+0000 7feaa7400700 1 -- 172.21.15.131:0/2916252828 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9801d070 con 0x7feab006d2b0 2024-05-08T22:09:44.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.900+0000 7feaa7400700 1 -- 172.21.15.131:0/2916252828 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea98005580 con 0x7feab006d2b0 2024-05-08T22:09:44.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.900+0000 7feaa7400700 1 -- 172.21.15.131:0/2916252828 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9800fe90 con 0x7feab006d2b0 2024-05-08T22:09:44.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.904+0000 7feab4c00700 1 -- 172.21.15.131:0/2916252828 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab010ac00 con 0x7feab006d2b0 2024-05-08T22:09:44.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.904+0000 7feab4c00700 1 -- 172.21.15.131:0/2916252828 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab0112c50 con 0x7feab006d2b0 2024-05-08T22:09:44.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.915+0000 7feaa7400700 1 -- 172.21.15.131:0/2916252828 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fea9802b440 con 0x7feab006d2b0 2024-05-08T22:09:44.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.917+0000 7feaa7400700 1 --2- 172.21.15.131:0/2916252828 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fea940a1390 0x7fea940a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.917+0000 7feaa7400700 1 -- 172.21.15.131:0/2916252828 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fea980ce360 con 0x7feab006d2b0 2024-05-08T22:09:44.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.918+0000 7feaa5400700 1 --2- 172.21.15.131:0/2916252828 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fea940a1390 0x7fea940a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:44.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.922+0000 7feac864ab80 1 Processor -- start 2024-05-08T22:09:44.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.922+0000 7feac864ab80 1 -- start start 2024-05-08T22:09:44.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.923+0000 7feac864ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56142499d050 0x56142499d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.923+0000 7feac864ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d970 0x561424997350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.923+0000 7feac864ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561424997890 0x561424997cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.923+0000 7feac864ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614248ee150 con 0x561424997890 2024-05-08T22:09:44.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.923+0000 7feac864ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614248ebb10 con 0x56142499d970 2024-05-08T22:09:44.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.923+0000 7feac864ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614248e7400 con 0x56142499d050 2024-05-08T22:09:44.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.923+0000 7feaa5400700 1 --2- 172.21.15.131:0/2916252828 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fea940a1390 0x7fea940a3850 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7feab010b590 tx=0x7feaa0009230 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:44.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.924+0000 7feaa5400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d970 0x561424997350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:44.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.924+0000 7feaa5400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d970 0x561424997350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:33398/0 (socket says 172.21.15.131:33398) 2024-05-08T22:09:44.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.924+0000 7feaa5400700 1 -- 172.21.15.131:0/2172245421 learned_addr learned my addr 172.21.15.131:0/2172245421 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:44.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.925+0000 7feaae800700 1 --2- 172.21.15.131:0/2172245421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561424997890 0x561424997cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:44.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.929+0000 7feaae800700 1 -- 172.21.15.131:0/2172245421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56142499d050 msgr2=0x56142499d430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:09:44.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.929+0000 7feaae800700 1 --2- 172.21.15.131:0/2172245421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56142499d050 0x56142499d430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.929+0000 7feaae800700 1 -- 172.21.15.131:0/2172245421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d970 msgr2=0x561424997350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:44.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.929+0000 7feaae800700 1 --2- 172.21.15.131:0/2172245421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d970 0x561424997350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.929+0000 7feaae800700 1 -- 172.21.15.131:0/2172245421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614247683b0 con 0x561424997890 2024-05-08T22:09:44.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.930+0000 7feaae800700 1 --2- 172.21.15.131:0/2172245421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561424997890 0x561424997cd0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7feab006a210 tx=0x7feaa800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cba21ce8d1d1f25 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:44.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.930+0000 7fea9f000700 1 -- 172.21.15.131:0/2172245421 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa80125f0 con 0x561424997890 2024-05-08T22:09:44.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.930+0000 7fea9f000700 1 -- 172.21.15.131:0/2172245421 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaa8012c10 con 0x561424997890 2024-05-08T22:09:44.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.930+0000 7fea9f000700 1 -- 172.21.15.131:0/2172245421 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa80125f0 con 0x561424997890 2024-05-08T22:09:44.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.931+0000 7feaa5400700 1 --2- 172.21.15.131:0/2172245421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d970 0x561424997350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:09:44.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.933+0000 7feac864ab80 1 -- 172.21.15.131:0/2172245421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561424997890 msgr2=0x561424997cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:44.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.933+0000 7feac864ab80 1 --2- 172.21.15.131:0/2172245421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561424997890 0x561424997cd0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7feab006a210 tx=0x7feaa800bab0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.933+0000 7feac864ab80 1 -- 172.21.15.131:0/2172245421 shutdown_connections 2024-05-08T22:09:44.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.933+0000 7feac864ab80 1 --2- 172.21.15.131:0/2172245421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561424997890 0x561424997cd0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.933+0000 7feac864ab80 1 --2- 172.21.15.131:0/2172245421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56142499d050 0x56142499d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.933+0000 7feac864ab80 1 --2- 172.21.15.131:0/2172245421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d970 0x561424997350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.933+0000 7feac864ab80 1 -- 172.21.15.131:0/2172245421 >> 172.21.15.131:0/2172245421 conn(0x5614248f5210 msgr2=0x5614249922d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:44.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.934+0000 7feac864ab80 1 -- 172.21.15.131:0/2172245421 shutdown_connections 2024-05-08T22:09:44.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.934+0000 7feac864ab80 1 -- 172.21.15.131:0/2172245421 wait complete. 2024-05-08T22:09:44.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.936+0000 7feac864ab80 1 -- start start 2024-05-08T22:09:44.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.936+0000 7feac864ab80 1 Processor -- start 2024-05-08T22:09:44.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.939+0000 7feac864ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561424997890 0x5614249a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.939+0000 7feac864ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56142499d050 0x5614249a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.940+0000 7feac864ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d970 0x5614249a1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.940+0000 7feaa5400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56142499d050 0x5614249a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:44.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.940+0000 7feaa5400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56142499d050 0x5614249a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39364/0 (socket says 172.21.15.131:39364) 2024-05-08T22:09:44.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.941+0000 7feaa5400700 1 -- 172.21.15.131:0/3624434138 learned_addr learned my addr 172.21.15.131:0/3624434138 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:44.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.941+0000 7feaade00700 1 --2- 172.21.15.131:0/3624434138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561424997890 0x5614249a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:44.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.941+0000 7feaae800700 1 --2- 172.21.15.131:0/3624434138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d970 0x5614249a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:44.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.941+0000 7feac864ab80 1 -- 172.21.15.131:0/3624434138 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561424883560 con 0x56142499d050 2024-05-08T22:09:44.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.941+0000 7feac864ab80 1 -- 172.21.15.131:0/3624434138 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561424883200 con 0x56142499d970 2024-05-08T22:09:44.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.941+0000 7feac864ab80 1 -- 172.21.15.131:0/3624434138 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56142476a6a0 con 0x561424997890 2024-05-08T22:09:44.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.942+0000 7feaa5400700 1 -- 172.21.15.131:0/3624434138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561424997890 msgr2=0x5614249a38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:44.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.942+0000 7feaa5400700 1 --2- 172.21.15.131:0/3624434138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561424997890 0x5614249a38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.942+0000 7feaa5400700 1 -- 172.21.15.131:0/3624434138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d970 msgr2=0x5614249a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:44.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.942+0000 7feaa5400700 1 --2- 172.21.15.131:0/3624434138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d970 0x5614249a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.942+0000 7feaa5400700 1 -- 172.21.15.131:0/3624434138 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa800e040 con 0x56142499d050 2024-05-08T22:09:44.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.943+0000 7feaa5400700 1 --2- 172.21.15.131:0/3624434138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56142499d050 0x5614249a0bc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7feaa0019b60 tx=0x7feaa0004bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa0416733cc41fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:44.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.943+0000 7fea93400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:09:44.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.943+0000 7fea93400700 1 -- 172.21.15.131:0/3624434138 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa0017ad0 con 0x56142499d050 2024-05-08T22:09:44.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.943+0000 7feac864ab80 10 client.0 _ll_register_callbacks cb 0x56142488cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:44.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.943+0000 7feac864ab80 1 -- 172.21.15.131:0/3624434138 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561424840270 con 0x56142499d050 2024-05-08T22:09:44.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.944+0000 7feaae800700 1 --2- 172.21.15.131:0/3624434138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d970 0x5614249a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:09:44.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.944+0000 7fea93400700 1 -- 172.21.15.131:0/3624434138 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7feaa0017c30 con 0x56142499d050 2024-05-08T22:09:44.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.945+0000 7fea93400700 1 -- 172.21.15.131:0/3624434138 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7feaa0017df0 con 0x56142499d050 2024-05-08T22:09:44.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.945+0000 7feac864ab80 10 client.15966 fetch_fsmap learned FSMap version 25 2024-05-08T22:09:44.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.945+0000 7feac864ab80 1 -- 172.21.15.131:0/3624434138 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x561424714b60 con 0x56142499d050 2024-05-08T22:09:44.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.947+0000 7fea93400700 1 -- 172.21.15.131:0/3624434138 <== mon.0 v2:172.21.15.131:3300/0 4 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7feaa0021410 con 0x56142499d050 2024-05-08T22:09:44.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.947+0000 7feac864ab80 10 client.15966 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T22:09:44.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.947+0000 7feac864ab80 10 client.15966 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:09:44.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.947+0000 7feac864ab80 20 client.15966 populate_metadata read hostname 'smithi131' 2024-05-08T22:09:44.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.947+0000 7feac864ab80 1 --2- 172.21.15.131:0/3624434138 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x5614249b0fb0 0x5614249b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:44.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.947+0000 7feac864ab80 4 client.15966 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:09:44.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.947+0000 7feac864ab80 1 -- 172.21.15.131:0/3624434138 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561424966810 con 0x5614249b0fb0 2024-05-08T22:09:44.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.949+0000 7feaade00700 1 --2- 172.21.15.131:0/3624434138 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x5614249b0fb0 0x5614249b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:44.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.951+0000 7feaade00700 1 --2- 172.21.15.131:0/3624434138 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x5614249b0fb0 0x5614249b3410 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=346f50f09e99301a server_cookie=1a8b00474d53080c in_seq=0 out_seq=0 2024-05-08T22:09:44.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.952+0000 7fea93400700 10 client.15966 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:09:44.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.954+0000 7fea93400700 1 -- 172.21.15.131:0/3624434138 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561424966810 con 0x5614249b0fb0 2024-05-08T22:09:44.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.954+0000 7fea93400700 10 client.15966 handle_command_reply: tid=0 2024-05-08T22:09:44.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.954+0000 7feac864ab80 1 client.15966 shutdown 2024-05-08T22:09:44.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.962+0000 7feac864ab80 1 -- 172.21.15.131:0/3624434138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56142499d050 msgr2=0x5614249a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:44.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.962+0000 7feac864ab80 1 --2- 172.21.15.131:0/3624434138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56142499d050 0x5614249a0bc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7feaa0019b60 tx=0x7feaa0004bf0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.966+0000 7feac864ab80 1 -- 172.21.15.131:0/3624434138 shutdown_connections 2024-05-08T22:09:44.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.966+0000 7feac864ab80 1 --2- 172.21.15.131:0/3624434138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d970 0x5614249a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.966+0000 7feac864ab80 1 --2- 172.21.15.131:0/3624434138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561424997890 0x5614249a38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.966+0000 7feac864ab80 1 --2- 172.21.15.131:0/3624434138 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x5614249b0fb0 0x5614249b3410 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.966+0000 7feac864ab80 1 --2- 172.21.15.131:0/3624434138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56142499d050 0x5614249a0bc0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:44.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.966+0000 7fea93400700 0 client.15966 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:09:44.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.966+0000 7feac864ab80 1 -- 172.21.15.131:0/3624434138 >> 172.21.15.131:0/3624434138 conn(0x5614248f5210 msgr2=0x5614249920b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:44.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.969+0000 7feac864ab80 1 -- 172.21.15.131:0/3624434138 shutdown_connections 2024-05-08T22:09:44.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.969+0000 7feac864ab80 1 -- 172.21.15.131:0/3624434138 wait complete. 2024-05-08T22:09:44.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:44.976+0000 7feac864ab80 20 client.15966 trim_cache size 0 max 16384 2024-05-08T22:09:45.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:44 smithi157 ceph-mon[77746]: pgmap v76: 129 pgs: 129 active+clean; 78 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 77 op/s 2024-05-08T22:09:45.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.039+0000 7feac864ab80 1 Processor -- start 2024-05-08T22:09:45.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.042+0000 7feac864ab80 1 -- start start 2024-05-08T22:09:45.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.043+0000 7feac864ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561424997890 0x561424997350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:45.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.044+0000 7feac864ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d050 0x5614249b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:45.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.044+0000 7feac864ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56142499d970 0x5614249ace10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:45.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.044+0000 7feac864ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614249927c0 con 0x56142499d970 2024-05-08T22:09:45.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.044+0000 7feac864ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614249a2190 con 0x56142499d050 2024-05-08T22:09:45.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.044+0000 7feac864ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56142498a040 con 0x561424997890 2024-05-08T22:09:45.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.044+0000 7feaa5400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d050 0x5614249b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:45.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.044+0000 7feaa5400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d050 0x5614249b7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:33422/0 (socket says 172.21.15.131:33422) 2024-05-08T22:09:45.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.044+0000 7feaa5400700 1 -- 172.21.15.131:0/3378689602 learned_addr learned my addr 172.21.15.131:0/3378689602 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:45.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.044+0000 7feaade00700 1 --2- 172.21.15.131:0/3378689602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561424997890 0x561424997350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:45.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.044+0000 7feaa5400700 1 -- 172.21.15.131:0/3378689602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561424997890 msgr2=0x561424997350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:45.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.044+0000 7feaa5400700 1 --2- 172.21.15.131:0/3378689602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561424997890 0x561424997350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.044+0000 7feaa5400700 1 -- 172.21.15.131:0/3378689602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56142499d970 msgr2=0x5614249ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:45.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.044+0000 7feaa5400700 1 --2- 172.21.15.131:0/3378689602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56142499d970 0x5614249ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.044+0000 7feaa5400700 1 -- 172.21.15.131:0/3378689602 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614248ad560 con 0x56142499d050 2024-05-08T22:09:45.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.045+0000 7feaade00700 1 --2- 172.21.15.131:0/3378689602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561424997890 0x561424997350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:09:45.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.045+0000 7feaa5400700 1 --2- 172.21.15.131:0/3378689602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d050 0x5614249b7db0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7feaa000e010 tx=0x7feaa001b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b37da2192c5e868 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:45.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.045+0000 7fea9fa00700 1 -- 172.21.15.131:0/3378689602 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa0017680 con 0x56142499d050 2024-05-08T22:09:45.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.045+0000 7fea9fa00700 1 -- 172.21.15.131:0/3378689602 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561424840270 con 0x56142499d050 2024-05-08T22:09:45.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.045+0000 7fea9fa00700 1 -- 172.21.15.131:0/3378689602 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa0017ad0 con 0x56142499d050 2024-05-08T22:09:45.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.046+0000 7feac864ab80 1 -- 172.21.15.131:0/3378689602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d050 msgr2=0x5614249b7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:45.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.046+0000 7feac864ab80 1 --2- 172.21.15.131:0/3378689602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d050 0x5614249b7db0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7feaa000e010 tx=0x7feaa001b040 comp rx=0 tx=0).stop 2024-05-08T22:09:45.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.047+0000 7feac864ab80 1 -- 172.21.15.131:0/3378689602 shutdown_connections 2024-05-08T22:09:45.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.047+0000 7feac864ab80 1 --2- 172.21.15.131:0/3378689602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56142499d970 0x5614249ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.047+0000 7feac864ab80 1 --2- 172.21.15.131:0/3378689602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561424997890 0x561424997350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.047+0000 7feac864ab80 1 --2- 172.21.15.131:0/3378689602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d050 0x5614249b7db0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.047+0000 7feac864ab80 1 -- 172.21.15.131:0/3378689602 >> 172.21.15.131:0/3378689602 conn(0x5614248f5210 msgr2=0x5614249920b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:45.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.047+0000 7feac864ab80 1 -- 172.21.15.131:0/3378689602 shutdown_connections 2024-05-08T22:09:45.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.047+0000 7feac864ab80 1 -- 172.21.15.131:0/3378689602 wait complete. 2024-05-08T22:09:45.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.048+0000 7feac864ab80 1 -- start start 2024-05-08T22:09:45.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.048+0000 7feac864ab80 1 Processor -- start 2024-05-08T22:09:45.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.049+0000 7feac864ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561424997890 0x5614249a8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:45.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.050+0000 7feac864ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56142499d050 0x561424999100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:45.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.050+0000 7feac864ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d970 0x561424999640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:45.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.050+0000 7feac864ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561424999d50 con 0x561424997890 2024-05-08T22:09:45.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.050+0000 7feac864ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561424999ec0 con 0x56142499d970 2024-05-08T22:09:45.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.050+0000 7feac864ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561424a07530 con 0x56142499d050 2024-05-08T22:09:45.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.050+0000 7feac864ab80 10 client.0 _ll_register_callbacks cb 0x56142488cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.050+0000 7feaae800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d970 0x561424999640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.050+0000 7feaae800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d970 0x561424999640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:33426/0 (socket says 172.21.15.131:33426) 2024-05-08T22:09:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.050+0000 7feaae800700 1 -- 172.21.15.131:0/1010008652 learned_addr learned my addr 172.21.15.131:0/1010008652 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.050+0000 7feaade00700 1 --2- 172.21.15.131:0/1010008652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561424997890 0x5614249a8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.050+0000 7feaa5400700 1 --2- 172.21.15.131:0/1010008652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56142499d050 0x561424999100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.051+0000 7feaa5400700 1 -- 172.21.15.131:0/1010008652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d970 msgr2=0x561424999640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.051+0000 7feaa5400700 1 --2- 172.21.15.131:0/1010008652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d970 0x561424999640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.051+0000 7feaa5400700 1 -- 172.21.15.131:0/1010008652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561424997890 msgr2=0x5614249a8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:45.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.051+0000 7feaa5400700 1 --2- 172.21.15.131:0/1010008652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561424997890 0x5614249a8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.051+0000 7feaa5400700 1 -- 172.21.15.131:0/1010008652 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561424714b60 con 0x56142499d050 2024-05-08T22:09:45.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.051+0000 7feaae800700 1 --2- 172.21.15.131:0/1010008652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d970 0x561424999640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:09:45.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.052+0000 7feaade00700 1 --2- 172.21.15.131:0/1010008652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561424997890 0x5614249a8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:09:45.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.053+0000 7feaa5400700 1 --2- 172.21.15.131:0/1010008652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56142499d050 0x561424999100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x561424998150 tx=0x7feaa00218d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b131490970b88ef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:45.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.053+0000 7fea93400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:09:45.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.053+0000 7fea93400700 1 -- 172.21.15.131:0/1010008652 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa0021b10 con 0x56142499d050 2024-05-08T22:09:45.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.053+0000 7fea93400700 1 -- 172.21.15.131:0/1010008652 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7feaa003d070 con 0x56142499d050 2024-05-08T22:09:45.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.053+0000 7feac864ab80 1 -- 172.21.15.131:0/1010008652 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feaa8012c10 con 0x56142499d050 2024-05-08T22:09:45.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.054+0000 7fea93400700 1 -- 172.21.15.131:0/1010008652 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7feaa0021c70 con 0x56142499d050 2024-05-08T22:09:45.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.055+0000 7feac864ab80 10 client.26044 fetch_fsmap learned FSMap version 25 2024-05-08T22:09:45.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.055+0000 7feac864ab80 1 -- 172.21.15.131:0/1010008652 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56142476ccd0 con 0x56142499d050 2024-05-08T22:09:45.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.055+0000 7fea93400700 1 -- 172.21.15.131:0/1010008652 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaa8012c10 con 0x56142499d050 2024-05-08T22:09:45.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.055+0000 7fea93400700 1 -- 172.21.15.131:0/1010008652 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7feaa0019c40 con 0x56142499d050 2024-05-08T22:09:45.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.055+0000 7feac864ab80 10 client.26044 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T22:09:45.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.055+0000 7feac864ab80 10 client.26044 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:09:45.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.055+0000 7feac864ab80 20 client.26044 populate_metadata read hostname 'smithi131' 2024-05-08T22:09:45.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.055+0000 7feac864ab80 1 --2- 172.21.15.131:0/1010008652 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x561424a08800 0x561424a0abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:45.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.055+0000 7feac864ab80 4 client.26044 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:09:45.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.055+0000 7feac864ab80 1 -- 172.21.15.131:0/1010008652 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5614248dfec0 con 0x561424a08800 2024-05-08T22:09:45.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.056+0000 7feaade00700 1 --2- 172.21.15.131:0/1010008652 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x561424a08800 0x561424a0abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:45.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.056+0000 7feaade00700 1 --2- 172.21.15.131:0/1010008652 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x561424a08800 0x561424a0abf0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=10f7cc46b52972ef server_cookie=d3b5c3f335d3a88a in_seq=0 out_seq=0 2024-05-08T22:09:45.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.056+0000 7fea93400700 10 client.26044 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:09:45.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.057+0000 7fea93400700 1 -- 172.21.15.131:0/1010008652 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5614248dfec0 con 0x561424a08800 2024-05-08T22:09:45.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.057+0000 7fea93400700 10 client.26044 handle_command_reply: tid=0 2024-05-08T22:09:45.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.057+0000 7feac864ab80 1 client.26044 shutdown 2024-05-08T22:09:45.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.060+0000 7feac864ab80 1 -- 172.21.15.131:0/1010008652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56142499d050 msgr2=0x561424999100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:45.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.060+0000 7feac864ab80 1 --2- 172.21.15.131:0/1010008652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56142499d050 0x561424999100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x561424998150 tx=0x7feaa00218d0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.062+0000 7feac864ab80 1 -- 172.21.15.131:0/1010008652 shutdown_connections 2024-05-08T22:09:45.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.062+0000 7feac864ab80 1 --2- 172.21.15.131:0/1010008652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56142499d970 0x561424999640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.062+0000 7feac864ab80 1 --2- 172.21.15.131:0/1010008652 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x561424a08800 0x561424a0abf0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.062+0000 7feac864ab80 1 --2- 172.21.15.131:0/1010008652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561424997890 0x5614249a8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.062+0000 7feac864ab80 1 --2- 172.21.15.131:0/1010008652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56142499d050 0x561424999100 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.062+0000 7feac864ab80 1 -- 172.21.15.131:0/1010008652 >> 172.21.15.131:0/1010008652 conn(0x5614248f5210 msgr2=0x56142499abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:45.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.062+0000 7fea93400700 0 client.26044 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:09:45.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.062+0000 7feac864ab80 1 -- 172.21.15.131:0/1010008652 shutdown_connections 2024-05-08T22:09:45.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.062+0000 7feac864ab80 1 -- 172.21.15.131:0/1010008652 wait complete. 2024-05-08T22:09:45.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.063+0000 7feac864ab80 20 client.26044 trim_cache size 0 max 16384 2024-05-08T22:09:45.066 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:09:45.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.064+0000 7feab4c00700 1 -- 172.21.15.131:0/2916252828 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fea940a1390 msgr2=0x7fea940a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:45.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.064+0000 7feab4c00700 1 --2- 172.21.15.131:0/2916252828 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fea940a1390 0x7fea940a3850 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7feab010b590 tx=0x7feaa0009230 comp rx=0 tx=0).stop 2024-05-08T22:09:45.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.064+0000 7feab4c00700 1 -- 172.21.15.131:0/2916252828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feab006d2b0 msgr2=0x7feab010ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:45.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.064+0000 7feab4c00700 1 --2- 172.21.15.131:0/2916252828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feab006d2b0 0x7feab010ed00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fea98005b40 tx=0x7fea98003b40 comp rx=0 tx=0).stop 2024-05-08T22:09:45.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.064+0000 7feab4c00700 1 -- 172.21.15.131:0/2916252828 shutdown_connections 2024-05-08T22:09:45.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.064+0000 7feab4c00700 1 --2- 172.21.15.131:0/2916252828 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fea940a1390 0x7fea940a3850 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.064+0000 7feab4c00700 1 --2- 172.21.15.131:0/2916252828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feab0113ce0 0x7feab0109e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.064+0000 7feab4c00700 1 --2- 172.21.15.131:0/2916252828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feab006d2b0 0x7feab010ed00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.064+0000 7feab4c00700 1 --2- 172.21.15.131:0/2916252828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feab0108570 0x7feab010f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.064+0000 7feab4c00700 1 -- 172.21.15.131:0/2916252828 >> 172.21.15.131:0/2916252828 conn(0x7feab006be80 msgr2=0x7feab006f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:45.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.064+0000 7feab4c00700 1 -- 172.21.15.131:0/2916252828 shutdown_connections 2024-05-08T22:09:45.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.065+0000 7feab4c00700 1 -- 172.21.15.131:0/2916252828 wait complete. 2024-05-08T22:09:45.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:44 smithi132 ceph-mon[78180]: pgmap v76: 129 pgs: 129 active+clean; 78 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 77 op/s 2024-05-08T22:09:45.095 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T22:09:45.220 INFO:tasks.workunit.client.0.smithi131.stdout: 35 580985 711 396213 398 212319 1377 2024-05-08T22:09:45.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.343+0000 7f2abca00700 1 -- 172.21.15.131:0/1280918404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ab8068050 msgr2=0x7f2ab8068430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:45.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.343+0000 7f2abca00700 1 --2- 172.21.15.131:0/1280918404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ab8068050 0x7f2ab8068430 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2aa80098f0 tx=0x7f2aa8009c00 comp rx=0 tx=0).stop 2024-05-08T22:09:45.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.343+0000 7f2abca00700 1 -- 172.21.15.131:0/1280918404 shutdown_connections 2024-05-08T22:09:45.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.343+0000 7f2abca00700 1 --2- 172.21.15.131:0/1280918404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ab810d940 0x7f2ab810fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.343+0000 7f2abca00700 1 --2- 172.21.15.131:0/1280918404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ab8068050 0x7f2ab8068430 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.343+0000 7f2abca00700 1 --2- 172.21.15.131:0/1280918404 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ab8068970 0x7f2ab810d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.343+0000 7f2abca00700 1 -- 172.21.15.131:0/1280918404 >> 172.21.15.131:0/1280918404 conn(0x7f2ab8076f40 msgr2=0x7f2ab8077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:45.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.343+0000 7f2abca00700 1 -- 172.21.15.131:0/1280918404 shutdown_connections 2024-05-08T22:09:45.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.343+0000 7f2abca00700 1 -- 172.21.15.131:0/1280918404 wait complete. 2024-05-08T22:09:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.344+0000 7f2abca00700 1 Processor -- start 2024-05-08T22:09:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.344+0000 7f2abca00700 1 -- start start 2024-05-08T22:09:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.345+0000 7f2abca00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ab8068050 0x7f2ab8199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.345+0000 7f2abca00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ab8068970 0x7f2ab8199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.345+0000 7f2abca00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ab810d940 0x7f2ab819df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.345+0000 7f2abca00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab819e5a0 con 0x7f2ab810d940 2024-05-08T22:09:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.345+0000 7f2abca00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab819e710 con 0x7f2ab8068970 2024-05-08T22:09:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.345+0000 7f2abca00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab819e8d0 con 0x7f2ab8068050 2024-05-08T22:09:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.345+0000 7f2ab5e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ab8068050 0x7f2ab8199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.345+0000 7f2ab5e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ab8068050 0x7f2ab8199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51538/0 (socket says 172.21.15.131:51538) 2024-05-08T22:09:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.345+0000 7f2ab5e00700 1 -- 172.21.15.131:0/103743218 learned_addr learned my addr 172.21.15.131:0/103743218 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.345+0000 7f2ab6800700 1 --2- 172.21.15.131:0/103743218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ab810d940 0x7f2ab819df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.345+0000 7f2ab5400700 1 --2- 172.21.15.131:0/103743218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ab8068970 0x7f2ab8199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.345+0000 7f2ab5e00700 1 -- 172.21.15.131:0/103743218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ab8068970 msgr2=0x7f2ab8199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.345+0000 7f2ab5e00700 1 --2- 172.21.15.131:0/103743218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ab8068970 0x7f2ab8199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.345+0000 7f2ab5e00700 1 -- 172.21.15.131:0/103743218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ab810d940 msgr2=0x7f2ab819df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.345+0000 7f2ab5e00700 1 --2- 172.21.15.131:0/103743218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ab810d940 0x7f2ab819df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.345+0000 7f2ab5e00700 1 -- 172.21.15.131:0/103743218 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa40095f0 con 0x7f2ab8068050 2024-05-08T22:09:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.345+0000 7f2ab6800700 1 --2- 172.21.15.131:0/103743218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ab810d940 0x7f2ab819df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:09:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.346+0000 7f2ab5e00700 1 --2- 172.21.15.131:0/103743218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ab8068050 0x7f2ab8199680 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8005950 tx=0x7f2aa800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.346+0000 7f2aaea00700 1 -- 172.21.15.131:0/103743218 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa801c070 con 0x7f2ab8068050 2024-05-08T22:09:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.346+0000 7f2aaea00700 1 -- 172.21.15.131:0/103743218 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2aa800bb20 con 0x7f2ab8068050 2024-05-08T22:09:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.346+0000 7f2abca00700 1 -- 172.21.15.131:0/103743218 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa8009520 con 0x7f2ab8068050 2024-05-08T22:09:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.346+0000 7f2aaea00700 1 -- 172.21.15.131:0/103743218 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa80038e0 con 0x7f2ab8068050 2024-05-08T22:09:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.346+0000 7f2abca00700 1 -- 172.21.15.131:0/103743218 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab81a6210 con 0x7f2ab8068050 2024-05-08T22:09:45.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.349+0000 7f2aaea00700 1 -- 172.21.15.131:0/103743218 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f2aa8020030 con 0x7f2ab8068050 2024-05-08T22:09:45.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.351+0000 7f2aaea00700 1 --2- 172.21.15.131:0/103743218 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f2a9c0a1310 0x7f2a9c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:45.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.351+0000 7f2aaea00700 1 -- 172.21.15.131:0/103743218 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f2aa80cdf00 con 0x7f2ab8068050 2024-05-08T22:09:45.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.351+0000 7f2ab5400700 1 --2- 172.21.15.131:0/103743218 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f2a9c0a1310 0x7f2a9c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:45.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.352+0000 7f2ab5400700 1 --2- 172.21.15.131:0/103743218 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f2a9c0a1310 0x7f2a9c0a37d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2ab819abd0 tx=0x7f2aa4009310 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:45.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.352+0000 7f2ad0502b80 1 Processor -- start 2024-05-08T22:09:45.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.353+0000 7f2ad0502b80 1 -- start start 2024-05-08T22:09:45.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.353+0000 7f2ad0502b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c464a6e00 0x556c464a71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:45.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.353+0000 7f2ad0502b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c464a7720 0x556c464a1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:45.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.353+0000 7f2ad0502b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c464a1890 0x556c464a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:45.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.353+0000 7f2ad0502b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c463f8150 con 0x556c464a7720 2024-05-08T22:09:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.353+0000 7f2ad0502b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c463f5b10 con 0x556c464a6e00 2024-05-08T22:09:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.353+0000 7f2ad0502b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c463f1400 con 0x556c464a1890 2024-05-08T22:09:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.353+0000 7f2ab5e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c464a6e00 0x556c464a71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.354+0000 7f2ab5e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c464a6e00 0x556c464a71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:33458/0 (socket says 172.21.15.131:33458) 2024-05-08T22:09:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.354+0000 7f2ab5e00700 1 -- 172.21.15.131:0/2565260998 learned_addr learned my addr 172.21.15.131:0/2565260998 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.354+0000 7f2ab5400700 1 --2- 172.21.15.131:0/2565260998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c464a7720 0x556c464a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.354+0000 7f2ab6800700 1 --2- 172.21.15.131:0/2565260998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c464a1890 0x556c464a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.354+0000 7f2ab5e00700 1 -- 172.21.15.131:0/2565260998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c464a1890 msgr2=0x556c464a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.354+0000 7f2ab5e00700 1 --2- 172.21.15.131:0/2565260998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c464a1890 0x556c464a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.354+0000 7f2ab5e00700 1 -- 172.21.15.131:0/2565260998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c464a7720 msgr2=0x556c464a1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.354+0000 7f2ab5e00700 1 --2- 172.21.15.131:0/2565260998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c464a7720 0x556c464a1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.354+0000 7f2ab5e00700 1 -- 172.21.15.131:0/2565260998 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c462723b0 con 0x556c464a6e00 2024-05-08T22:09:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.354+0000 7f2ab6800700 1 --2- 172.21.15.131:0/2565260998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c464a1890 0x556c464a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:09:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.354+0000 7f2ab5e00700 1 --2- 172.21.15.131:0/2565260998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c464a6e00 0x556c464a71e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8097720 tx=0x7f2aa8005570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0343356255fa162 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.354+0000 7f2aa3000700 1 -- 172.21.15.131:0/2565260998 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa809eae0 con 0x556c464a6e00 2024-05-08T22:09:45.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.354+0000 7f2aa3000700 1 -- 172.21.15.131:0/2565260998 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2aa80a30a0 con 0x556c464a6e00 2024-05-08T22:09:45.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.354+0000 7f2aa3000700 1 -- 172.21.15.131:0/2565260998 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa80053d0 con 0x556c464a6e00 2024-05-08T22:09:45.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.355+0000 7f2ad0502b80 1 -- 172.21.15.131:0/2565260998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c464a6e00 msgr2=0x556c464a71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:45.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.355+0000 7f2ad0502b80 1 --2- 172.21.15.131:0/2565260998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c464a6e00 0x556c464a71e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8097720 tx=0x7f2aa8005570 comp rx=0 tx=0).stop 2024-05-08T22:09:45.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.355+0000 7f2ad0502b80 1 -- 172.21.15.131:0/2565260998 shutdown_connections 2024-05-08T22:09:45.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.355+0000 7f2ad0502b80 1 --2- 172.21.15.131:0/2565260998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c464a1890 0x556c464a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.355+0000 7f2ad0502b80 1 --2- 172.21.15.131:0/2565260998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c464a6e00 0x556c464a71e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.355+0000 7f2ad0502b80 1 --2- 172.21.15.131:0/2565260998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c464a7720 0x556c464a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.355+0000 7f2ad0502b80 1 -- 172.21.15.131:0/2565260998 >> 172.21.15.131:0/2565260998 conn(0x556c463ff210 msgr2=0x556c4649bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.355+0000 7f2ad0502b80 1 -- 172.21.15.131:0/2565260998 shutdown_connections 2024-05-08T22:09:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.356+0000 7f2ad0502b80 1 -- 172.21.15.131:0/2565260998 wait complete. 2024-05-08T22:09:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.356+0000 7f2ad0502b80 1 -- start start 2024-05-08T22:09:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.356+0000 7f2ad0502b80 1 Processor -- start 2024-05-08T22:09:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.357+0000 7f2ad0502b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c464a1890 0x556c464aaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.357+0000 7f2ad0502b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c464a6e00 0x556c464ab000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.357+0000 7f2ad0502b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c464a7720 0x556c464ab540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.357+0000 7f2ad0502b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4638d560 con 0x556c464a7720 2024-05-08T22:09:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.357+0000 7f2ad0502b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4638d200 con 0x556c464a1890 2024-05-08T22:09:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.357+0000 7f2ad0502b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c462746a0 con 0x556c464a6e00 2024-05-08T22:09:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.357+0000 7f2ad0502b80 10 client.0 _ll_register_callbacks cb 0x556c46396b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.357+0000 7f2ab5e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c464a1890 0x556c464aaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.357+0000 7f2ab5e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c464a1890 0x556c464aaac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:33464/0 (socket says 172.21.15.131:33464) 2024-05-08T22:09:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.357+0000 7f2ab5e00700 1 -- 172.21.15.131:0/2999888294 learned_addr learned my addr 172.21.15.131:0/2999888294 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.358+0000 7f2ab5400700 1 --2- 172.21.15.131:0/2999888294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c464a6e00 0x556c464ab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.358+0000 7f2ab5e00700 1 -- 172.21.15.131:0/2999888294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c464a6e00 msgr2=0x556c464ab000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.358+0000 7f2ab5e00700 1 --2- 172.21.15.131:0/2999888294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c464a6e00 0x556c464ab000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.358+0000 7f2ab5e00700 1 -- 172.21.15.131:0/2999888294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c464a7720 msgr2=0x556c464ab540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.358+0000 7f2ab5e00700 1 --2- 172.21.15.131:0/2999888294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c464a7720 0x556c464ab540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.358+0000 7f2ab5e00700 1 -- 172.21.15.131:0/2999888294 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab81a6210 con 0x556c464a1890 2024-05-08T22:09:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.358+0000 7f2ab5e00700 1 --2- 172.21.15.131:0/2999888294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c464a1890 0x556c464aaac0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2aa809ac40 tx=0x7f2aa800fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a838eec3c2c1bc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.358+0000 7f2a97400700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:09:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.358+0000 7f2a97400700 1 -- 172.21.15.131:0/2999888294 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa80051d0 con 0x556c464a1890 2024-05-08T22:09:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.358+0000 7f2a97400700 1 -- 172.21.15.131:0/2999888294 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f2aa8005330 con 0x556c464a1890 2024-05-08T22:09:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.358+0000 7f2ad0502b80 1 -- 172.21.15.131:0/2999888294 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c4634a270 con 0x556c464a1890 2024-05-08T22:09:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.358+0000 7f2a97400700 1 -- 172.21.15.131:0/2999888294 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2aa8021380 con 0x556c464a1890 2024-05-08T22:09:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.358+0000 7f2ad0502b80 10 client.26048 fetch_fsmap learned FSMap version 25 2024-05-08T22:09:45.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.358+0000 7f2ad0502b80 1 -- 172.21.15.131:0/2999888294 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556c4621eb60 con 0x556c464a1890 2024-05-08T22:09:45.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.359+0000 7f2a97400700 1 -- 172.21.15.131:0/2999888294 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556c4634a270 con 0x556c464a1890 2024-05-08T22:09:45.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.359+0000 7f2a97400700 1 -- 172.21.15.131:0/2999888294 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f2aa809a1a0 con 0x556c464a1890 2024-05-08T22:09:45.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.359+0000 7f2ad0502b80 10 client.26048 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T22:09:45.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.359+0000 7f2ad0502b80 10 client.26048 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:45.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.359+0000 7f2ad0502b80 20 client.26048 populate_metadata read hostname 'smithi131' 2024-05-08T22:09:45.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.359+0000 7f2ad0502b80 1 --2- 172.21.15.131:0/2999888294 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x556c464ba940 0x556c464bcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:45.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.359+0000 7f2ad0502b80 4 client.26048 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:09:45.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.359+0000 7f2ad0502b80 1 -- 172.21.15.131:0/2999888294 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c46470810 con 0x556c464ba940 2024-05-08T22:09:45.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.359+0000 7f2ab5400700 1 --2- 172.21.15.131:0/2999888294 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x556c464ba940 0x556c464bcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:45.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.360+0000 7f2ab5400700 1 --2- 172.21.15.131:0/2999888294 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x556c464ba940 0x556c464bcd30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53a4bb1af95e4333 server_cookie=7315db5873916f60 in_seq=0 out_seq=0 2024-05-08T22:09:45.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.360+0000 7f2a97400700 10 client.26048 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:45.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.362+0000 7f2a97400700 1 -- 172.21.15.131:0/2999888294 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556c46470810 con 0x556c464ba940 2024-05-08T22:09:45.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.362+0000 7f2a97400700 10 client.26048 handle_command_reply: tid=0 2024-05-08T22:09:45.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.362+0000 7f2ad0502b80 1 client.26048 shutdown 2024-05-08T22:09:45.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.363+0000 7f2ad0502b80 1 -- 172.21.15.131:0/2999888294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c464a1890 msgr2=0x556c464aaac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:45.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.363+0000 7f2ad0502b80 1 --2- 172.21.15.131:0/2999888294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c464a1890 0x556c464aaac0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2aa809ac40 tx=0x7f2aa800fa70 comp rx=0 tx=0).stop 2024-05-08T22:09:45.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.363+0000 7f2ad0502b80 1 -- 172.21.15.131:0/2999888294 shutdown_connections 2024-05-08T22:09:45.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.363+0000 7f2ad0502b80 1 --2- 172.21.15.131:0/2999888294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c464a7720 0x556c464ab540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.363+0000 7f2ad0502b80 1 --2- 172.21.15.131:0/2999888294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c464a1890 0x556c464aaac0 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2aa809ac40 tx=0x7f2aa800fa70 comp rx=0 tx=0).stop 2024-05-08T22:09:45.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.363+0000 7f2ad0502b80 1 --2- 172.21.15.131:0/2999888294 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x556c464ba940 0x556c464bcd30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.363+0000 7f2ad0502b80 1 --2- 172.21.15.131:0/2999888294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c464a6e00 0x556c464ab000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.363+0000 7f2a97400700 0 client.26048 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:45.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.363+0000 7f2ad0502b80 1 -- 172.21.15.131:0/2999888294 >> 172.21.15.131:0/2999888294 conn(0x556c463ff210 msgr2=0x556c4649b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:45.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.363+0000 7f2ad0502b80 1 -- 172.21.15.131:0/2999888294 shutdown_connections 2024-05-08T22:09:45.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.363+0000 7f2ad0502b80 1 -- 172.21.15.131:0/2999888294 wait complete. 2024-05-08T22:09:45.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.363+0000 7f2ad0502b80 20 client.26048 trim_cache size 0 max 16384 2024-05-08T22:09:45.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.388+0000 7f2ad0502b80 1 Processor -- start 2024-05-08T22:09:45.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.389+0000 7f2ad0502b80 1 -- start start 2024-05-08T22:09:45.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.389+0000 7f2ad0502b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c464a6e00 0x556c464a1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:45.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.389+0000 7f2ad0502b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c464a7720 0x556c464a6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:45.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.389+0000 7f2ad0502b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c46498210 0x556c4649bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:45.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.389+0000 7f2ad0502b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c464a22d0 con 0x556c464a6e00 2024-05-08T22:09:45.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.389+0000 7f2ad0502b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4649c9a0 con 0x556c46498210 2024-05-08T22:09:45.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.389+0000 7f2ad0502b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c46498dc0 con 0x556c464a7720 2024-05-08T22:09:45.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.390+0000 7f2ab6800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c46498210 0x556c4649bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:45.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.390+0000 7f2ab5400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c464a7720 0x556c464a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:45.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.390+0000 7f2ab5400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c464a7720 0x556c464a6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51576/0 (socket says 172.21.15.131:51576) 2024-05-08T22:09:45.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.390+0000 7f2ab5400700 1 -- 172.21.15.131:0/3013663941 learned_addr learned my addr 172.21.15.131:0/3013663941 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:45.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.390+0000 7f2ab5e00700 1 --2- 172.21.15.131:0/3013663941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c464a6e00 0x556c464a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:45.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.391+0000 7f2ab5e00700 1 -- 172.21.15.131:0/3013663941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c464a7720 msgr2=0x556c464a6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:45.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.391+0000 7f2ab5e00700 1 --2- 172.21.15.131:0/3013663941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c464a7720 0x556c464a6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.391+0000 7f2ab5e00700 1 -- 172.21.15.131:0/3013663941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c46498210 msgr2=0x556c4649bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:45.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.391+0000 7f2ab5e00700 1 --2- 172.21.15.131:0/3013663941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c46498210 0x556c4649bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.391+0000 7f2ab5e00700 1 -- 172.21.15.131:0/3013663941 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c463b7560 con 0x556c464a6e00 2024-05-08T22:09:45.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.392+0000 7f2ab5e00700 1 --2- 172.21.15.131:0/3013663941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c464a6e00 0x556c464a1350 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2aa809a9b0 tx=0x7f2aa8019fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ec8789a9bd323f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:45.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.392+0000 7f2aa3a00700 1 -- 172.21.15.131:0/3013663941 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa809fa00 con 0x556c464a6e00 2024-05-08T22:09:45.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.392+0000 7f2aa3a00700 1 -- 172.21.15.131:0/3013663941 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2aa80a2cb0 con 0x556c464a6e00 2024-05-08T22:09:45.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.392+0000 7f2aa3a00700 1 -- 172.21.15.131:0/3013663941 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa809a1a0 con 0x556c464a6e00 2024-05-08T22:09:45.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.393+0000 7f2ad0502b80 1 -- 172.21.15.131:0/3013663941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c464a6e00 msgr2=0x556c464a1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:45.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.393+0000 7f2ad0502b80 1 --2- 172.21.15.131:0/3013663941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c464a6e00 0x556c464a1350 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2aa809a9b0 tx=0x7f2aa8019fd0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.394+0000 7f2ad0502b80 1 -- 172.21.15.131:0/3013663941 shutdown_connections 2024-05-08T22:09:45.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.394+0000 7f2ad0502b80 1 --2- 172.21.15.131:0/3013663941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c46498210 0x556c4649bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.394+0000 7f2ad0502b80 1 --2- 172.21.15.131:0/3013663941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c464a6e00 0x556c464a1350 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.394+0000 7f2ad0502b80 1 --2- 172.21.15.131:0/3013663941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c464a7720 0x556c464a6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.394+0000 7f2ad0502b80 1 -- 172.21.15.131:0/3013663941 >> 172.21.15.131:0/3013663941 conn(0x556c463ff210 msgr2=0x556c464a9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:45.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.394+0000 7f2ad0502b80 1 -- 172.21.15.131:0/3013663941 shutdown_connections 2024-05-08T22:09:45.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.394+0000 7f2ad0502b80 1 -- 172.21.15.131:0/3013663941 wait complete. 2024-05-08T22:09:45.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.395+0000 7f2ad0502b80 1 -- start start 2024-05-08T22:09:45.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.395+0000 7f2ad0502b80 1 Processor -- start 2024-05-08T22:09:45.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.396+0000 7f2ad0502b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c46498210 0x556c465148d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:45.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.396+0000 7f2ad0502b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c464a6e00 0x556c46514e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:45.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.396+0000 7f2ad0502b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c464a7720 0x556c464b0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:45.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.396+0000 7f2ab5400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c464a6e00 0x556c46514e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:45.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.396+0000 7f2ab5400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c464a6e00 0x556c46514e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:33484/0 (socket says 172.21.15.131:33484) 2024-05-08T22:09:45.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.396+0000 7f2ab5400700 1 -- 172.21.15.131:0/1322093933 learned_addr learned my addr 172.21.15.131:0/1322093933 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:45.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.396+0000 7f2ad0502b80 1 -- 172.21.15.131:0/1322093933 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c464b0770 con 0x556c464a7720 2024-05-08T22:09:45.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.396+0000 7f2ad0502b80 1 -- 172.21.15.131:0/1322093933 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c464b08e0 con 0x556c464a6e00 2024-05-08T22:09:45.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.396+0000 7f2ad0502b80 1 -- 172.21.15.131:0/1322093933 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c464b0aa0 con 0x556c46498210 2024-05-08T22:09:45.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.396+0000 7f2ad0502b80 10 client.0 _ll_register_callbacks cb 0x556c46396b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:45.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.396+0000 7f2ab5e00700 1 --2- 172.21.15.131:0/1322093933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c46498210 0x556c465148d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:45.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.396+0000 7f2ab6800700 1 --2- 172.21.15.131:0/1322093933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c464a7720 0x556c464b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:45.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.396+0000 7f2ab5400700 1 -- 172.21.15.131:0/1322093933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c46498210 msgr2=0x556c465148d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:45.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.396+0000 7f2ab5400700 1 --2- 172.21.15.131:0/1322093933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c46498210 0x556c465148d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.396+0000 7f2ab5400700 1 -- 172.21.15.131:0/1322093933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c464a7720 msgr2=0x556c464b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:45.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.396+0000 7f2ab5400700 1 --2- 172.21.15.131:0/1322093933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c464a7720 0x556c464b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.396+0000 7f2ab5400700 1 -- 172.21.15.131:0/1322093933 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c4621eb60 con 0x556c464a6e00 2024-05-08T22:09:45.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.396+0000 7f2ab6800700 1 --2- 172.21.15.131:0/1322093933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c464a7720 0x556c464b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:09:45.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.397+0000 7f2ab5400700 1 --2- 172.21.15.131:0/1322093933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c464a6e00 0x556c46514e10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4005fd0 tx=0x7f2aa4013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5cfe2e6ef369375 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:45.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.397+0000 7f2a97400700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:09:45.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.397+0000 7f2a97400700 1 -- 172.21.15.131:0/1322093933 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4023040 con 0x556c464a6e00 2024-05-08T22:09:45.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.397+0000 7f2ad0502b80 1 -- 172.21.15.131:0/1322093933 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2aa80a30a0 con 0x556c464a6e00 2024-05-08T22:09:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.397+0000 7f2a97400700 1 -- 172.21.15.131:0/1322093933 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f2aa402e070 con 0x556c464a6e00 2024-05-08T22:09:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.397+0000 7f2a97400700 1 -- 172.21.15.131:0/1322093933 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2aa4028570 con 0x556c464a6e00 2024-05-08T22:09:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.398+0000 7f2ad0502b80 10 client.26054 fetch_fsmap learned FSMap version 25 2024-05-08T22:09:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.398+0000 7f2ad0502b80 1 -- 172.21.15.131:0/1322093933 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556c46276cd0 con 0x556c464a6e00 2024-05-08T22:09:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.398+0000 7f2a97400700 1 -- 172.21.15.131:0/1322093933 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2aa80a30a0 con 0x556c464a6e00 2024-05-08T22:09:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.399+0000 7f2a97400700 1 -- 172.21.15.131:0/1322093933 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f2aa4032420 con 0x556c464a6e00 2024-05-08T22:09:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.399+0000 7f2ad0502b80 10 client.26054 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T22:09:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.399+0000 7f2ad0502b80 10 client.26054 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.399+0000 7f2ad0502b80 20 client.26054 populate_metadata read hostname 'smithi131' 2024-05-08T22:09:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.399+0000 7f2ad0502b80 1 --2- 172.21.15.131:0/1322093933 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x556c4652b730 0x556c4652db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.399+0000 7f2ad0502b80 4 client.26054 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:09:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.399+0000 7f2ad0502b80 1 -- 172.21.15.131:0/1322093933 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556c463e9ec0 con 0x556c4652b730 2024-05-08T22:09:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.399+0000 7f2ab5e00700 1 --2- 172.21.15.131:0/1322093933 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x556c4652b730 0x556c4652db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.400+0000 7f2a97400700 10 client.26054 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:45.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.400+0000 7f2ab5e00700 1 --2- 172.21.15.131:0/1322093933 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x556c4652b730 0x556c4652db50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1c2708d81d048a4 server_cookie=c1ea9d499c516b25 in_seq=0 out_seq=0 2024-05-08T22:09:45.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.403+0000 7f2a97400700 1 -- 172.21.15.131:0/1322093933 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556c463e9ec0 con 0x556c4652b730 2024-05-08T22:09:45.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.403+0000 7f2a97400700 10 client.26054 handle_command_reply: tid=0 2024-05-08T22:09:45.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.403+0000 7f2ad0502b80 1 client.26054 shutdown 2024-05-08T22:09:45.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.406+0000 7f2ad0502b80 1 -- 172.21.15.131:0/1322093933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c464a6e00 msgr2=0x556c46514e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:45.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.406+0000 7f2ad0502b80 1 --2- 172.21.15.131:0/1322093933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c464a6e00 0x556c46514e10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4005fd0 tx=0x7f2aa4013040 comp rx=0 tx=0).stop 2024-05-08T22:09:45.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.406+0000 7f2ad0502b80 1 -- 172.21.15.131:0/1322093933 shutdown_connections 2024-05-08T22:09:45.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.406+0000 7f2ad0502b80 1 --2- 172.21.15.131:0/1322093933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c464a7720 0x556c464b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.406+0000 7f2ad0502b80 1 --2- 172.21.15.131:0/1322093933 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x556c4652b730 0x556c4652db50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.406+0000 7f2ad0502b80 1 --2- 172.21.15.131:0/1322093933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c46498210 0x556c465148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.406+0000 7f2ad0502b80 1 --2- 172.21.15.131:0/1322093933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c464a6e00 0x556c46514e10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.406+0000 7f2ad0502b80 1 -- 172.21.15.131:0/1322093933 >> 172.21.15.131:0/1322093933 conn(0x556c463ff210 msgr2=0x556c464ab3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:45.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.407+0000 7f2a97400700 0 client.26054 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:45.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.407+0000 7f2ad0502b80 1 -- 172.21.15.131:0/1322093933 shutdown_connections 2024-05-08T22:09:45.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.407+0000 7f2ad0502b80 1 -- 172.21.15.131:0/1322093933 wait complete. 2024-05-08T22:09:45.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.407+0000 7f2ad0502b80 20 client.26054 trim_cache size 0 max 16384 2024-05-08T22:09:45.408 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:09:45.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.408+0000 7f2abca00700 1 -- 172.21.15.131:0/103743218 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f2a9c0a1310 msgr2=0x7f2a9c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:45.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.408+0000 7f2abca00700 1 --2- 172.21.15.131:0/103743218 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f2a9c0a1310 0x7f2a9c0a37d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2ab819abd0 tx=0x7f2aa4009310 comp rx=0 tx=0).stop 2024-05-08T22:09:45.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.408+0000 7f2abca00700 1 -- 172.21.15.131:0/103743218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ab8068050 msgr2=0x7f2ab8199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:45.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.408+0000 7f2abca00700 1 --2- 172.21.15.131:0/103743218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ab8068050 0x7f2ab8199680 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8005950 tx=0x7f2aa800fa00 comp rx=0 tx=0).stop 2024-05-08T22:09:45.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.408+0000 7f2abca00700 1 -- 172.21.15.131:0/103743218 shutdown_connections 2024-05-08T22:09:45.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.408+0000 7f2abca00700 1 --2- 172.21.15.131:0/103743218 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f2a9c0a1310 0x7f2a9c0a37d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.408+0000 7f2abca00700 1 --2- 172.21.15.131:0/103743218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ab810d940 0x7f2ab819df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.408+0000 7f2abca00700 1 --2- 172.21.15.131:0/103743218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ab8068050 0x7f2ab8199680 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.408+0000 7f2abca00700 1 --2- 172.21.15.131:0/103743218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ab8068970 0x7f2ab8199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:45.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.408+0000 7f2abca00700 1 -- 172.21.15.131:0/103743218 >> 172.21.15.131:0/103743218 conn(0x7f2ab8076f40 msgr2=0x7f2ab810f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:45.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.409+0000 7f2abca00700 1 -- 172.21.15.131:0/103743218 shutdown_connections 2024-05-08T22:09:45.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:45.409+0000 7f2abca00700 1 -- 172.21.15.131:0/103743218 wait complete. 2024-05-08T22:09:46.440 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T22:09:46.441 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T22:09:47.089 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:46 smithi132 ceph-mon[78180]: pgmap v77: 129 pgs: 129 active+clean; 88 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 106 op/s 2024-05-08T22:09:47.172 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:46 smithi131 ceph-mon[37260]: pgmap v77: 129 pgs: 129 active+clean; 88 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 106 op/s 2024-05-08T22:09:47.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:46 smithi157 ceph-mon[77746]: pgmap v77: 129 pgs: 129 active+clean; 88 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 106 op/s 2024-05-08T22:09:49.089 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:48 smithi132 ceph-mon[78180]: pgmap v78: 129 pgs: 129 active+clean; 88 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.7 MiB/s wr, 79 op/s 2024-05-08T22:09:49.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:48 smithi131 ceph-mon[37260]: pgmap v78: 129 pgs: 129 active+clean; 88 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.7 MiB/s wr, 79 op/s 2024-05-08T22:09:49.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:48 smithi157 ceph-mon[77746]: pgmap v78: 129 pgs: 129 active+clean; 88 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.7 MiB/s wr, 79 op/s 2024-05-08T22:09:49.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.658+0000 7f3444e00700 1 -- 172.21.15.131:0/380911977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3440073cf0 msgr2=0x7f344010d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:49.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.658+0000 7f3444e00700 1 --2- 172.21.15.131:0/380911977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3440073cf0 0x7f344010d1f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f342c009990 tx=0x7f342c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.658+0000 7f3444e00700 1 -- 172.21.15.131:0/380911977 shutdown_connections 2024-05-08T22:09:49.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.658+0000 7f3444e00700 1 --2- 172.21.15.131:0/380911977 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f344010d730 0x7f344010fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.658+0000 7f3444e00700 1 --2- 172.21.15.131:0/380911977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34400733d0 0x7f34400737b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.658+0000 7f3444e00700 1 --2- 172.21.15.131:0/380911977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3440073cf0 0x7f344010d1f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.658+0000 7f3444e00700 1 -- 172.21.15.131:0/380911977 >> 172.21.15.131:0/380911977 conn(0x7f34400fc9c0 msgr2=0x7f34400fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:49.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.658+0000 7f3444e00700 1 -- 172.21.15.131:0/380911977 shutdown_connections 2024-05-08T22:09:49.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.658+0000 7f3444e00700 1 -- 172.21.15.131:0/380911977 wait complete. 2024-05-08T22:09:49.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.659+0000 7f3444e00700 1 Processor -- start 2024-05-08T22:09:49.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.660+0000 7f3444e00700 1 -- start start 2024-05-08T22:09:49.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.660+0000 7f3444e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34400733d0 0x7f34401996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:49.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.660+0000 7f3444e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3440073cf0 0x7f3440199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:49.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.660+0000 7f3444e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f344010d730 0x7f344019df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:49.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.660+0000 7f3444e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344019e580 con 0x7f344010d730 2024-05-08T22:09:49.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.660+0000 7f3444e00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344019e6f0 con 0x7f34400733d0 2024-05-08T22:09:49.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.660+0000 7f3444e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344019e8b0 con 0x7f3440073cf0 2024-05-08T22:09:49.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.660+0000 7f343de00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34400733d0 0x7f34401996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:49.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.660+0000 7f343d400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3440073cf0 0x7f3440199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:49.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.660+0000 7f343de00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34400733d0 0x7f34401996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:33498/0 (socket says 172.21.15.131:33498) 2024-05-08T22:09:49.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.660+0000 7f343de00700 1 -- 172.21.15.131:0/758978207 learned_addr learned my addr 172.21.15.131:0/758978207 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:49.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.661+0000 7f343e800700 1 --2- 172.21.15.131:0/758978207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f344010d730 0x7f344019df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:49.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.661+0000 7f343de00700 1 -- 172.21.15.131:0/758978207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3440073cf0 msgr2=0x7f3440199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:49.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.661+0000 7f343de00700 1 --2- 172.21.15.131:0/758978207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3440073cf0 0x7f3440199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.661+0000 7f343de00700 1 -- 172.21.15.131:0/758978207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f344010d730 msgr2=0x7f344019df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:49.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.661+0000 7f343de00700 1 --2- 172.21.15.131:0/758978207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f344010d730 0x7f344019df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.661+0000 7f343de00700 1 -- 172.21.15.131:0/758978207 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342c0095f0 con 0x7f34400733d0 2024-05-08T22:09:49.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.661+0000 7f343e800700 1 --2- 172.21.15.131:0/758978207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f344010d730 0x7f344019df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:09:49.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.661+0000 7f343de00700 1 --2- 172.21.15.131:0/758978207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34400733d0 0x7f34401996b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f343000d980 tx=0x7f343000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:49.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.661+0000 7f3436a00700 1 -- 172.21.15.131:0/758978207 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430004a30 con 0x7f34400733d0 2024-05-08T22:09:49.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.661+0000 7f3436a00700 1 -- 172.21.15.131:0/758978207 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34300054e0 con 0x7f34400733d0 2024-05-08T22:09:49.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.661+0000 7f3444e00700 1 -- 172.21.15.131:0/758978207 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f344019eb90 con 0x7f34400733d0 2024-05-08T22:09:49.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.661+0000 7f3436a00700 1 -- 172.21.15.131:0/758978207 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430009c90 con 0x7f34400733d0 2024-05-08T22:09:49.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.662+0000 7f3444e00700 1 -- 172.21.15.131:0/758978207 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34401a63d0 con 0x7f34400733d0 2024-05-08T22:09:49.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.666+0000 7f3436a00700 1 -- 172.21.15.131:0/758978207 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f3430010460 con 0x7f34400733d0 2024-05-08T22:09:49.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.667+0000 7f3436a00700 1 --2- 172.21.15.131:0/758978207 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f34240a1290 0x7f34240a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:49.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.667+0000 7f3436a00700 1 -- 172.21.15.131:0/758978207 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f34300ccff0 con 0x7f34400733d0 2024-05-08T22:09:49.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.667+0000 7f34589c6b80 1 Processor -- start 2024-05-08T22:09:49.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.668+0000 7f34589c6b80 1 -- start start 2024-05-08T22:09:49.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.668+0000 7f343d400700 1 --2- 172.21.15.131:0/758978207 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f34240a1290 0x7f34240a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:49.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.668+0000 7f34589c6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab28d4690 0x55fab28d4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:49.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.668+0000 7f34589c6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab28d4fb0 0x55fab28cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:49.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.668+0000 7f34589c6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fab28cf120 0x55fab28cf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:49.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.668+0000 7f34589c6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab281d1d0 con 0x55fab28cf120 2024-05-08T22:09:49.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.668+0000 7f34589c6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab281ab90 con 0x55fab28d4690 2024-05-08T22:09:49.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.668+0000 7f34589c6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab2816480 con 0x55fab28d4fb0 2024-05-08T22:09:49.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.668+0000 7f343de00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab28d4690 0x55fab28d4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:49.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.668+0000 7f343de00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab28d4690 0x55fab28d4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:33512/0 (socket says 172.21.15.131:33512) 2024-05-08T22:09:49.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.668+0000 7f343de00700 1 -- 172.21.15.131:0/3077760289 learned_addr learned my addr 172.21.15.131:0/3077760289 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:49.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.669+0000 7f343d400700 1 --2- 172.21.15.131:0/3077760289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab28d4fb0 0x55fab28cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:49.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.669+0000 7f343de00700 1 -- 172.21.15.131:0/3077760289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab28d4fb0 msgr2=0x55fab28cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:49.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.669+0000 7f343de00700 1 --2- 172.21.15.131:0/3077760289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab28d4fb0 0x55fab28cebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.669+0000 7f343de00700 1 -- 172.21.15.131:0/3077760289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fab28cf120 msgr2=0x55fab28cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:49.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.669+0000 7f343de00700 1 --2- 172.21.15.131:0/3077760289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fab28cf120 0x55fab28cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.669+0000 7f343de00700 1 -- 172.21.15.131:0/3077760289 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fab2697430 con 0x55fab28d4690 2024-05-08T22:09:49.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.669+0000 7f343d400700 1 --2- 172.21.15.131:0/758978207 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f34240a1290 0x7f34240a3750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f344019abb0 tx=0x7f342c00be00 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:49.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.669+0000 7f343de00700 1 --2- 172.21.15.131:0/3077760289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab28d4690 0x55fab28d4a70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f34300a1c80 tx=0x7f34300056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f5aaff138403498 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:49.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.669+0000 7f342b400700 1 -- 172.21.15.131:0/3077760289 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430099d30 con 0x55fab28d4690 2024-05-08T22:09:49.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.669+0000 7f342b400700 1 -- 172.21.15.131:0/3077760289 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f343009db60 con 0x55fab28d4690 2024-05-08T22:09:49.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.669+0000 7f342b400700 1 -- 172.21.15.131:0/3077760289 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430099f90 con 0x55fab28d4690 2024-05-08T22:09:49.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.670+0000 7f34589c6b80 1 -- 172.21.15.131:0/3077760289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab28d4690 msgr2=0x55fab28d4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:49.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.670+0000 7f34589c6b80 1 --2- 172.21.15.131:0/3077760289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab28d4690 0x55fab28d4a70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f34300a1c80 tx=0x7f34300056d0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.671+0000 7f34589c6b80 1 -- 172.21.15.131:0/3077760289 shutdown_connections 2024-05-08T22:09:49.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.671+0000 7f34589c6b80 1 --2- 172.21.15.131:0/3077760289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fab28cf120 0x55fab28cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.671+0000 7f34589c6b80 1 --2- 172.21.15.131:0/3077760289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab28d4690 0x55fab28d4a70 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.671+0000 7f34589c6b80 1 --2- 172.21.15.131:0/3077760289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab28d4fb0 0x55fab28cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.671+0000 7f34589c6b80 1 -- 172.21.15.131:0/3077760289 >> 172.21.15.131:0/3077760289 conn(0x55fab2824290 msgr2=0x55fab28c0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:49.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.671+0000 7f34589c6b80 1 -- 172.21.15.131:0/3077760289 shutdown_connections 2024-05-08T22:09:49.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.671+0000 7f34589c6b80 1 -- 172.21.15.131:0/3077760289 wait complete. 2024-05-08T22:09:49.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.671+0000 7f34589c6b80 1 -- start start 2024-05-08T22:09:49.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.672+0000 7f34589c6b80 1 Processor -- start 2024-05-08T22:09:49.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.672+0000 7f34589c6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab28cf120 0x55fab28d83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:49.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.672+0000 7f34589c6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab28d4690 0x55fab28d8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:49.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.672+0000 7f34589c6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fab28d4fb0 0x55fab28d8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:49.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.672+0000 7f34589c6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab27b25e0 con 0x55fab28d4fb0 2024-05-08T22:09:49.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.672+0000 7f34589c6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab27b2280 con 0x55fab28d4690 2024-05-08T22:09:49.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.672+0000 7f34589c6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab2699720 con 0x55fab28cf120 2024-05-08T22:09:49.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.672+0000 7f34589c6b80 10 client.0 _ll_register_callbacks cb 0x55fab27bbbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:49.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.673+0000 7f343d400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab28d4690 0x55fab28d8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:49.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.673+0000 7f343d400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab28d4690 0x55fab28d8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:33522/0 (socket says 172.21.15.131:33522) 2024-05-08T22:09:49.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.673+0000 7f343d400700 1 -- 172.21.15.131:0/3127467636 learned_addr learned my addr 172.21.15.131:0/3127467636 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:49.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.673+0000 7f343de00700 1 --2- 172.21.15.131:0/3127467636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab28cf120 0x55fab28d83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:49.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.673+0000 7f343e800700 1 --2- 172.21.15.131:0/3127467636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fab28d4fb0 0x55fab28d8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:49.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.673+0000 7f343d400700 1 -- 172.21.15.131:0/3127467636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab28cf120 msgr2=0x55fab28d83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:49.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.673+0000 7f343d400700 1 --2- 172.21.15.131:0/3127467636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab28cf120 0x55fab28d83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.673+0000 7f343d400700 1 -- 172.21.15.131:0/3127467636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fab28d4fb0 msgr2=0x55fab28d8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:49.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.673+0000 7f343d400700 1 --2- 172.21.15.131:0/3127467636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fab28d4fb0 0x55fab28d8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.673+0000 7f343d400700 1 -- 172.21.15.131:0/3127467636 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34401a63d0 con 0x55fab28d4690 2024-05-08T22:09:49.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.673+0000 7f343de00700 1 --2- 172.21.15.131:0/3127467636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab28cf120 0x55fab28d83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:09:49.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.673+0000 7f343d400700 1 --2- 172.21.15.131:0/3127467636 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab28d4690 0x55fab28d8930 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f342c015000 tx=0x7f342c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b495988cc07155f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:49.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.673+0000 7f341fe00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:09:49.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.674+0000 7f341fe00700 1 -- 172.21.15.131:0/3127467636 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c02e070 con 0x55fab28d4690 2024-05-08T22:09:49.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.674+0000 7f34589c6b80 1 -- 172.21.15.131:0/3127467636 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fab276f2f0 con 0x55fab28d4690 2024-05-08T22:09:49.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.674+0000 7f341fe00700 1 -- 172.21.15.131:0/3127467636 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f342c0039a0 con 0x55fab28d4690 2024-05-08T22:09:49.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.674+0000 7f341fe00700 1 -- 172.21.15.131:0/3127467636 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f342c003b60 con 0x55fab28d4690 2024-05-08T22:09:49.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.674+0000 7f34589c6b80 10 client.26066 fetch_fsmap learned FSMap version 25 2024-05-08T22:09:49.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.674+0000 7f34589c6b80 1 -- 172.21.15.131:0/3127467636 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fab2643be0 con 0x55fab28d4690 2024-05-08T22:09:49.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.674+0000 7f341fe00700 1 -- 172.21.15.131:0/3127467636 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fab276f2f0 con 0x55fab28d4690 2024-05-08T22:09:49.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.675+0000 7f341fe00700 1 -- 172.21.15.131:0/3127467636 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f342c032410 con 0x55fab28d4690 2024-05-08T22:09:49.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.675+0000 7f34589c6b80 10 client.26066 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T22:09:49.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.675+0000 7f34589c6b80 10 client.26066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:49.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.675+0000 7f34589c6b80 20 client.26066 populate_metadata read hostname 'smithi131' 2024-05-08T22:09:49.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.675+0000 7f34589c6b80 1 --2- 172.21.15.131:0/3127467636 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55fab28e8ab0 0x55fab28eaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:49.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.675+0000 7f34589c6b80 4 client.26066 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:09:49.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.675+0000 7f34589c6b80 1 -- 172.21.15.131:0/3127467636 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fab2895890 con 0x55fab28e8ab0 2024-05-08T22:09:49.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.675+0000 7f343de00700 1 --2- 172.21.15.131:0/3127467636 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55fab28e8ab0 0x55fab28eaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:49.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.676+0000 7f341fe00700 10 client.26066 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:49.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.676+0000 7f343de00700 1 --2- 172.21.15.131:0/3127467636 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55fab28e8ab0 0x55fab28eaed0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9218e47e78212978 server_cookie=cced74bbbf631364 in_seq=0 out_seq=0 2024-05-08T22:09:49.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.678+0000 7f341fe00700 1 -- 172.21.15.131:0/3127467636 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fab2895890 con 0x55fab28e8ab0 2024-05-08T22:09:49.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.678+0000 7f341fe00700 10 client.26066 handle_command_reply: tid=0 2024-05-08T22:09:49.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.678+0000 7f34589c6b80 1 client.26066 shutdown 2024-05-08T22:09:49.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.679+0000 7f34589c6b80 1 -- 172.21.15.131:0/3127467636 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab28d4690 msgr2=0x55fab28d8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:49.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.679+0000 7f34589c6b80 1 --2- 172.21.15.131:0/3127467636 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab28d4690 0x55fab28d8930 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f342c015000 tx=0x7f342c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.679+0000 7f34589c6b80 1 -- 172.21.15.131:0/3127467636 shutdown_connections 2024-05-08T22:09:49.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.679+0000 7f34589c6b80 1 --2- 172.21.15.131:0/3127467636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fab28d4fb0 0x55fab28d8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.679+0000 7f34589c6b80 1 --2- 172.21.15.131:0/3127467636 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55fab28e8ab0 0x55fab28eaed0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.679+0000 7f34589c6b80 1 --2- 172.21.15.131:0/3127467636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab28cf120 0x55fab28d83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.679+0000 7f34589c6b80 1 --2- 172.21.15.131:0/3127467636 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab28d4690 0x55fab28d8930 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.679+0000 7f341fe00700 0 client.26066 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:49.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.679+0000 7f34589c6b80 1 -- 172.21.15.131:0/3127467636 >> 172.21.15.131:0/3127467636 conn(0x55fab2824290 msgr2=0x55fab28c0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:49.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.679+0000 7f34589c6b80 1 -- 172.21.15.131:0/3127467636 shutdown_connections 2024-05-08T22:09:49.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.679+0000 7f34589c6b80 1 -- 172.21.15.131:0/3127467636 wait complete. 2024-05-08T22:09:49.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.679+0000 7f34589c6b80 20 client.26066 trim_cache size 0 max 16384 2024-05-08T22:09:49.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.916+0000 7f34589c6b80 1 Processor -- start 2024-05-08T22:09:49.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.926+0000 7f34589c6b80 1 -- start start 2024-05-08T22:09:49.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.927+0000 7f34589c6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab28cf120 0x55fab28cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:49.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.927+0000 7f34589c6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab28d4690 0x55fab28d3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:49.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.927+0000 7f34589c6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fab28d4fb0 0x55fab28c0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:49.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.927+0000 7f34589c6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab28e61e0 con 0x55fab28d4fb0 2024-05-08T22:09:49.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.927+0000 7f34589c6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab28daf60 con 0x55fab28cf120 2024-05-08T22:09:49.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.927+0000 7f34589c6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab28bde40 con 0x55fab28d4690 2024-05-08T22:09:49.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.928+0000 7f343e800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fab28d4fb0 0x55fab28c0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:49.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.929+0000 7f343e800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fab28d4fb0 0x55fab28c0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39492/0 (socket says 172.21.15.131:39492) 2024-05-08T22:09:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.929+0000 7f343e800700 1 -- 172.21.15.131:0/1714372648 learned_addr learned my addr 172.21.15.131:0/1714372648 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.929+0000 7f343d400700 1 --2- 172.21.15.131:0/1714372648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab28d4690 0x55fab28d3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.929+0000 7f343d400700 1 -- 172.21.15.131:0/1714372648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab28cf120 msgr2=0x55fab28cebe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:09:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.929+0000 7f343d400700 1 --2- 172.21.15.131:0/1714372648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab28cf120 0x55fab28cebe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.929+0000 7f343d400700 1 -- 172.21.15.131:0/1714372648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fab28d4fb0 msgr2=0x55fab28c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:49.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.930+0000 7f343d400700 1 --2- 172.21.15.131:0/1714372648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fab28d4fb0 0x55fab28c0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.930+0000 7f343d400700 1 -- 172.21.15.131:0/1714372648 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fab27dc5e0 con 0x55fab28d4690 2024-05-08T22:09:49.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.930+0000 7f343e800700 1 --2- 172.21.15.131:0/1714372648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fab28d4fb0 0x55fab28c0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:09:49.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.931+0000 7f343d400700 1 --2- 172.21.15.131:0/1714372648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab28d4690 0x55fab28d3bd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f342c009990 tx=0x7f342c0329c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f56ae78ca56ab5d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:49.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.936+0000 7f342be00700 1 -- 172.21.15.131:0/1714372648 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c02e070 con 0x55fab28d4690 2024-05-08T22:09:49.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.936+0000 7f342be00700 1 -- 172.21.15.131:0/1714372648 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f342c01e070 con 0x55fab28d4690 2024-05-08T22:09:49.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.937+0000 7f34589c6b80 1 -- 172.21.15.131:0/1714372648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab28d4690 msgr2=0x55fab28d3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:49.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.937+0000 7f34589c6b80 1 --2- 172.21.15.131:0/1714372648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab28d4690 0x55fab28d3bd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f342c009990 tx=0x7f342c0329c0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.941+0000 7f34589c6b80 1 -- 172.21.15.131:0/1714372648 shutdown_connections 2024-05-08T22:09:49.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.941+0000 7f34589c6b80 1 --2- 172.21.15.131:0/1714372648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fab28d4fb0 0x55fab28c0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.941+0000 7f34589c6b80 1 --2- 172.21.15.131:0/1714372648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab28cf120 0x55fab28cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.941+0000 7f34589c6b80 1 --2- 172.21.15.131:0/1714372648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab28d4690 0x55fab28d3bd0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.941+0000 7f34589c6b80 1 -- 172.21.15.131:0/1714372648 >> 172.21.15.131:0/1714372648 conn(0x55fab2824290 msgr2=0x55fab28d34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:49.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.941+0000 7f34589c6b80 1 -- 172.21.15.131:0/1714372648 shutdown_connections 2024-05-08T22:09:49.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.941+0000 7f34589c6b80 1 -- 172.21.15.131:0/1714372648 wait complete. 2024-05-08T22:09:49.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.942+0000 7f34589c6b80 1 -- start start 2024-05-08T22:09:49.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.942+0000 7f34589c6b80 1 Processor -- start 2024-05-08T22:09:49.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.943+0000 7f34589c6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab28cf120 0x55fab28de5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:49.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.943+0000 7f34589c6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fab28d4690 0x55fab28deb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:49.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.944+0000 7f343de00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab28cf120 0x55fab28de5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:49.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.944+0000 7f343de00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab28cf120 0x55fab28de5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51636/0 (socket says 172.21.15.131:51636) 2024-05-08T22:09:49.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.945+0000 7f34589c6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab28d4fb0 0x55fab28df050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:49.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.945+0000 7f343d400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fab28d4690 0x55fab28deb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:49.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.945+0000 7f343d400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fab28d4690 0x55fab28deb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39508/0 (socket says 172.21.15.131:39508) 2024-05-08T22:09:49.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.948+0000 7f34589c6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab2939110 con 0x55fab28d4690 2024-05-08T22:09:49.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.948+0000 7f34589c6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab2939280 con 0x55fab28d4fb0 2024-05-08T22:09:49.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.948+0000 7f34589c6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab2939440 con 0x55fab28cf120 2024-05-08T22:09:49.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.948+0000 7f34589c6b80 10 client.0 _ll_register_callbacks cb 0x55fab27bbbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:49.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.952+0000 7f343e800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab28d4fb0 0x55fab28df050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:49.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.952+0000 7f343e800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab28d4fb0 0x55fab28df050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:33528/0 (socket says 172.21.15.131:33528) 2024-05-08T22:09:49.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.952+0000 7f343e800700 1 -- 172.21.15.131:0/1022025655 learned_addr learned my addr 172.21.15.131:0/1022025655 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:49.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.955+0000 7f343d400700 1 -- 172.21.15.131:0/1022025655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab28cf120 msgr2=0x55fab28de5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:49.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.955+0000 7f343d400700 1 --2- 172.21.15.131:0/1022025655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab28cf120 0x55fab28de5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.955+0000 7f343d400700 1 -- 172.21.15.131:0/1022025655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab28d4fb0 msgr2=0x55fab28df050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:49.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.955+0000 7f343d400700 1 --2- 172.21.15.131:0/1022025655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab28d4fb0 0x55fab28df050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:49.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.955+0000 7f343d400700 1 -- 172.21.15.131:0/1022025655 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3438009c00 con 0x55fab28d4690 2024-05-08T22:09:49.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.956+0000 7f343d400700 1 --2- 172.21.15.131:0/1022025655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fab28d4690 0x55fab28deb10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f342c009990 tx=0x7f342c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323c918c03fd6d4c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:49.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.956+0000 7f341fe00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:09:49.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.956+0000 7f341fe00700 1 -- 172.21.15.131:0/1022025655 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c02e040 con 0x55fab28d4690 2024-05-08T22:09:49.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.956+0000 7f341fe00700 1 -- 172.21.15.131:0/1022025655 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f342c003ba0 con 0x55fab28d4690 2024-05-08T22:09:49.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.957+0000 7f34589c6b80 1 -- 172.21.15.131:0/1022025655 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f343009db60 con 0x55fab28d4690 2024-05-08T22:09:49.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.957+0000 7f341fe00700 1 -- 172.21.15.131:0/1022025655 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f342c003d80 con 0x55fab28d4690 2024-05-08T22:09:49.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.957+0000 7f34589c6b80 10 client.15993 fetch_fsmap learned FSMap version 25 2024-05-08T22:09:49.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.957+0000 7f34589c6b80 1 -- 172.21.15.131:0/1022025655 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fab2643be0 con 0x55fab28d4690 2024-05-08T22:09:49.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.958+0000 7f341fe00700 1 -- 172.21.15.131:0/1022025655 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f343009db60 con 0x55fab28d4690 2024-05-08T22:09:49.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.958+0000 7f341fe00700 1 -- 172.21.15.131:0/1022025655 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f342c03a070 con 0x55fab28d4690 2024-05-08T22:09:49.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.958+0000 7f34589c6b80 10 client.15993 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T22:09:49.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.958+0000 7f34589c6b80 10 client.15993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:49.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.958+0000 7f34589c6b80 20 client.15993 populate_metadata read hostname 'smithi131' 2024-05-08T22:09:49.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.958+0000 7f34589c6b80 1 --2- 172.21.15.131:0/1022025655 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55fab293a770 0x55fab28cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:49.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.958+0000 7f34589c6b80 4 client.15993 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T22:09:49.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.958+0000 7f34589c6b80 1 -- 172.21.15.131:0/1022025655 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55fab280ef40 con 0x55fab293a770 2024-05-08T22:09:49.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.959+0000 7f343de00700 1 --2- 172.21.15.131:0/1022025655 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55fab293a770 0x55fab28cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:49.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.961+0000 7f341fe00700 10 client.15993 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:49.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:49.961+0000 7f343de00700 1 --2- 172.21.15.131:0/1022025655 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55fab293a770 0x55fab28cb3f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97f35a6f73a19a23 server_cookie=371123f8b05d8cc3 in_seq=0 out_seq=0 2024-05-08T22:09:50.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.404+0000 7f341fe00700 1 -- 172.21.15.131:0/1022025655 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fab280ef40 con 0x55fab293a770 2024-05-08T22:09:50.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.404+0000 7f341fe00700 10 client.15993 handle_command_reply: tid=0 2024-05-08T22:09:50.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.404+0000 7f34589c6b80 1 client.15993 shutdown 2024-05-08T22:09:50.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.404+0000 7f34589c6b80 1 -- 172.21.15.131:0/1022025655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fab28d4690 msgr2=0x55fab28deb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:50.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.404+0000 7f34589c6b80 1 --2- 172.21.15.131:0/1022025655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fab28d4690 0x55fab28deb10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f342c009990 tx=0x7f342c005800 comp rx=0 tx=0).stop 2024-05-08T22:09:50.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.405+0000 7f34589c6b80 1 -- 172.21.15.131:0/1022025655 shutdown_connections 2024-05-08T22:09:50.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.406+0000 7f34589c6b80 1 --2- 172.21.15.131:0/1022025655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab28d4fb0 0x55fab28df050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:50.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.406+0000 7f34589c6b80 1 --2- 172.21.15.131:0/1022025655 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55fab293a770 0x55fab28cb3f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:50.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.406+0000 7f34589c6b80 1 --2- 172.21.15.131:0/1022025655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab28cf120 0x55fab28de5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:50.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.406+0000 7f34589c6b80 1 --2- 172.21.15.131:0/1022025655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fab28d4690 0x55fab28deb10 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:50.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.406+0000 7f34589c6b80 1 -- 172.21.15.131:0/1022025655 >> 172.21.15.131:0/1022025655 conn(0x55fab2824290 msgr2=0x55fab28d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:50.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.406+0000 7f341fe00700 0 client.15993 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:50.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.407+0000 7f34589c6b80 1 -- 172.21.15.131:0/1022025655 shutdown_connections 2024-05-08T22:09:50.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.407+0000 7f34589c6b80 1 -- 172.21.15.131:0/1022025655 wait complete. 2024-05-08T22:09:50.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.408+0000 7f34589c6b80 20 client.15993 trim_cache size 0 max 16384 2024-05-08T22:09:50.409 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:09:50.410 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-08T22:09:50.410 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "5412d246-753f-4286-8c74-a6f748feebcb", 2024-05-08T22:09:50.410 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-08T22:09:50.410 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:09:50.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.410+0000 7f3444e00700 1 -- 172.21.15.131:0/758978207 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f34240a1290 msgr2=0x7f34240a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:50.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.410+0000 7f3444e00700 1 --2- 172.21.15.131:0/758978207 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f34240a1290 0x7f34240a3750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f344019abb0 tx=0x7f342c00be00 comp rx=0 tx=0).stop 2024-05-08T22:09:50.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.410+0000 7f3444e00700 1 -- 172.21.15.131:0/758978207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34400733d0 msgr2=0x7f34401996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:50.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.410+0000 7f3444e00700 1 --2- 172.21.15.131:0/758978207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34400733d0 0x7f34401996b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f343000d980 tx=0x7f343000dc90 comp rx=0 tx=0).stop 2024-05-08T22:09:50.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.410+0000 7f3444e00700 1 -- 172.21.15.131:0/758978207 shutdown_connections 2024-05-08T22:09:50.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.410+0000 7f3444e00700 1 --2- 172.21.15.131:0/758978207 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f34240a1290 0x7f34240a3750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:50.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.410+0000 7f3444e00700 1 --2- 172.21.15.131:0/758978207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f344010d730 0x7f344019df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:50.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.410+0000 7f3444e00700 1 --2- 172.21.15.131:0/758978207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34400733d0 0x7f34401996b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:50.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.410+0000 7f3444e00700 1 --2- 172.21.15.131:0/758978207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3440073cf0 0x7f3440199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:50.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.410+0000 7f3444e00700 1 -- 172.21.15.131:0/758978207 >> 172.21.15.131:0/758978207 conn(0x7f34400fc9c0 msgr2=0x7f34400fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:50.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.410+0000 7f3444e00700 1 -- 172.21.15.131:0/758978207 shutdown_connections 2024-05-08T22:09:50.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:50.411+0000 7f3444e00700 1 -- 172.21.15.131:0/758978207 wait complete. 2024-05-08T22:09:50.444 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:09:51.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.050+0000 7fe07f400700 1 -- 172.21.15.131:0/3346324515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe07811a780 msgr2=0x7fe07811cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:51.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.050+0000 7fe07f400700 1 --2- 172.21.15.131:0/3346324515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe07811a780 0x7fe07811cb70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe07000b0a0 tx=0x7fe07000b3b0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.067+0000 7fe07f400700 1 -- 172.21.15.131:0/3346324515 shutdown_connections 2024-05-08T22:09:51.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.067+0000 7fe07f400700 1 --2- 172.21.15.131:0/3346324515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe07811a780 0x7fe07811cb70 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.067+0000 7fe07f400700 1 --2- 172.21.15.131:0/3346324515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe07810f6a0 0x7fe078107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.067+0000 7fe07f400700 1 --2- 172.21.15.131:0/3346324515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe07810fb10 0x7fe0781084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.067+0000 7fe07f400700 1 -- 172.21.15.131:0/3346324515 >> 172.21.15.131:0/3346324515 conn(0x7fe07806bff0 msgr2=0x7fe07806c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:51.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.068+0000 7fe07f400700 1 -- 172.21.15.131:0/3346324515 shutdown_connections 2024-05-08T22:09:51.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.068+0000 7fe07f400700 1 -- 172.21.15.131:0/3346324515 wait complete. 2024-05-08T22:09:51.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.077+0000 7fe07f400700 1 Processor -- start 2024-05-08T22:09:51.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.078+0000 7fe07f400700 1 -- start start 2024-05-08T22:09:51.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.079+0000 7fe07f400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe07810f6a0 0x7fe0781a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:51.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.079+0000 7fe07f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe07810fb10 0x7fe0781a6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:51.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.079+0000 7fe07ca00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe07810f6a0 0x7fe0781a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:51.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.079+0000 7fe07ca00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe07810f6a0 0x7fe0781a6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:33544/0 (socket says 172.21.15.131:33544) 2024-05-08T22:09:51.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.081+0000 7fe07f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe07811a780 0x7fe0781aabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:51.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.083+0000 7fe07d400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe07811a780 0x7fe0781aabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:51.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.083+0000 7fe07d400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe07811a780 0x7fe0781aabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39516/0 (socket says 172.21.15.131:39516) 2024-05-08T22:09:51.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.084+0000 7fe07f400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0781ab230 con 0x7fe07811a780 2024-05-08T22:09:51.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.084+0000 7fe07f400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0781ab3a0 con 0x7fe07810f6a0 2024-05-08T22:09:51.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.084+0000 7fe07f400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0781ab560 con 0x7fe07810fb10 2024-05-08T22:09:51.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.087+0000 7fe077e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe07810fb10 0x7fe0781a6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:51.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.087+0000 7fe077e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe07810fb10 0x7fe0781a6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51652/0 (socket says 172.21.15.131:51652) 2024-05-08T22:09:51.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.087+0000 7fe077e00700 1 -- 172.21.15.131:0/603165296 learned_addr learned my addr 172.21.15.131:0/603165296 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:51.089 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:50 smithi132 ceph-mon[78180]: pgmap v79: 129 pgs: 129 active+clean; 149 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.1 MiB/s wr, 388 op/s 2024-05-08T22:09:51.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.090+0000 7fe07d400700 1 -- 172.21.15.131:0/603165296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe07810fb10 msgr2=0x7fe0781a6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:51.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.090+0000 7fe07d400700 1 --2- 172.21.15.131:0/603165296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe07810fb10 0x7fe0781a6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.090+0000 7fe07d400700 1 -- 172.21.15.131:0/603165296 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe07810f6a0 msgr2=0x7fe0781a6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:51.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.090+0000 7fe07d400700 1 --2- 172.21.15.131:0/603165296 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe07810f6a0 0x7fe0781a6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.090+0000 7fe07d400700 1 -- 172.21.15.131:0/603165296 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe070009d00 con 0x7fe07811a780 2024-05-08T22:09:51.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.091+0000 7fe07d400700 1 --2- 172.21.15.131:0/603165296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe07811a780 0x7fe0781aabf0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe07000b070 tx=0x7fe070007b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:51.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.092+0000 7fe075600700 1 -- 172.21.15.131:0/603165296 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07000e040 con 0x7fe07811a780 2024-05-08T22:09:51.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.093+0000 7fe075600700 1 -- 172.21.15.131:0/603165296 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe07001de10 con 0x7fe07811a780 2024-05-08T22:09:51.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.093+0000 7fe075600700 1 -- 172.21.15.131:0/603165296 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07001b450 con 0x7fe07811a780 2024-05-08T22:09:51.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.093+0000 7fe07f400700 1 -- 172.21.15.131:0/603165296 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0781ab7e0 con 0x7fe07811a780 2024-05-08T22:09:51.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.095+0000 7fe07f400700 1 -- 172.21.15.131:0/603165296 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0781b3080 con 0x7fe07811a780 2024-05-08T22:09:51.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.096+0000 7fe07ca00700 1 --2- 172.21.15.131:0/603165296 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe07810f6a0 0x7fe0781a6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:09:51.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.102+0000 7fe075600700 1 -- 172.21.15.131:0/603165296 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fe07001b5b0 con 0x7fe07811a780 2024-05-08T22:09:51.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.104+0000 7fe075600700 1 --2- 172.21.15.131:0/603165296 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe0600a1350 0x7fe0600a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:51.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.106+0000 7fe07ca00700 1 --2- 172.21.15.131:0/603165296 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe0600a1350 0x7fe0600a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:51.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.107+0000 7fe075600700 1 -- 172.21.15.131:0/603165296 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fe07009d130 con 0x7fe07811a780 2024-05-08T22:09:51.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.108+0000 7fe07ca00700 1 --2- 172.21.15.131:0/603165296 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe0600a1350 0x7fe0600a3810 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe06c006fd0 tx=0x7fe06c008040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:51.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.114+0000 7fe092fedb80 1 Processor -- start 2024-05-08T22:09:51.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.115+0000 7fe092fedb80 1 -- start start 2024-05-08T22:09:51.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.116+0000 7fe092fedb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ba0b602e00 0x55ba0b6031e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:51.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.117+0000 7fe092fedb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b603720 0x55ba0b5fd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:51.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.117+0000 7fe092fedb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba0b5fd890 0x55ba0b5fdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:51.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.117+0000 7fe07ca00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ba0b602e00 0x55ba0b6031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:51.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.117+0000 7fe07ca00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ba0b602e00 0x55ba0b6031e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:33558/0 (socket says 172.21.15.131:33558) 2024-05-08T22:09:51.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.117+0000 7fe077e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b603720 0x55ba0b5fd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:51.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.117+0000 7fe077e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b603720 0x55ba0b5fd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51664/0 (socket says 172.21.15.131:51664) 2024-05-08T22:09:51.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.118+0000 7fe07d400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba0b5fd890 0x55ba0b5fdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:51.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.118+0000 7fe07d400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba0b5fd890 0x55ba0b5fdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39530/0 (socket says 172.21.15.131:39530) 2024-05-08T22:09:51.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.118+0000 7fe07ca00700 1 -- 172.21.15.131:0/370296038 learned_addr learned my addr 172.21.15.131:0/370296038 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:51.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.119+0000 7fe092fedb80 1 -- 172.21.15.131:0/370296038 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0b554150 con 0x55ba0b5fd890 2024-05-08T22:09:51.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.119+0000 7fe092fedb80 1 -- 172.21.15.131:0/370296038 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0b551b10 con 0x55ba0b602e00 2024-05-08T22:09:51.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.119+0000 7fe092fedb80 1 -- 172.21.15.131:0/370296038 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0b54d400 con 0x55ba0b603720 2024-05-08T22:09:51.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.119+0000 7fe07d400700 1 -- 172.21.15.131:0/370296038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b603720 msgr2=0x55ba0b5fd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:51.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.119+0000 7fe07d400700 1 --2- 172.21.15.131:0/370296038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b603720 0x55ba0b5fd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.119+0000 7fe07d400700 1 -- 172.21.15.131:0/370296038 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ba0b602e00 msgr2=0x55ba0b6031e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:51.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.119+0000 7fe07d400700 1 --2- 172.21.15.131:0/370296038 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ba0b602e00 0x55ba0b6031e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.119+0000 7fe07d400700 1 -- 172.21.15.131:0/370296038 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba0b3ce3b0 con 0x55ba0b5fd890 2024-05-08T22:09:51.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.120+0000 7fe07d400700 1 --2- 172.21.15.131:0/370296038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba0b5fd890 0x55ba0b5fdcd0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe0700ccfe0 tx=0x7fe0700a0a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483d3cc2b7cc686b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:51.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.121+0000 7fe077e00700 1 --2- 172.21.15.131:0/370296038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b603720 0x55ba0b5fd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-08T22:09:51.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.121+0000 7fe07ca00700 1 --2- 172.21.15.131:0/370296038 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ba0b602e00 0x55ba0b6031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-08T22:09:51.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.125+0000 7fe069c00700 1 -- 172.21.15.131:0/370296038 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07009c570 con 0x55ba0b5fd890 2024-05-08T22:09:51.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.125+0000 7fe069c00700 1 -- 172.21.15.131:0/370296038 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe07009c390 con 0x55ba0b5fd890 2024-05-08T22:09:51.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.125+0000 7fe069c00700 1 -- 172.21.15.131:0/370296038 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0700a4eb0 con 0x55ba0b5fd890 2024-05-08T22:09:51.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.127+0000 7fe092fedb80 1 -- 172.21.15.131:0/370296038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba0b5fd890 msgr2=0x55ba0b5fdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:51.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.127+0000 7fe092fedb80 1 --2- 172.21.15.131:0/370296038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba0b5fd890 0x55ba0b5fdcd0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe0700ccfe0 tx=0x7fe0700a0a50 comp rx=0 tx=0).stop 2024-05-08T22:09:51.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.138+0000 7fe092fedb80 1 -- 172.21.15.131:0/370296038 shutdown_connections 2024-05-08T22:09:51.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.138+0000 7fe092fedb80 1 --2- 172.21.15.131:0/370296038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba0b5fd890 0x55ba0b5fdcd0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.138+0000 7fe092fedb80 1 --2- 172.21.15.131:0/370296038 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ba0b602e00 0x55ba0b6031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.138+0000 7fe092fedb80 1 --2- 172.21.15.131:0/370296038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b603720 0x55ba0b5fd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.138+0000 7fe092fedb80 1 -- 172.21.15.131:0/370296038 >> 172.21.15.131:0/370296038 conn(0x55ba0b55b210 msgr2=0x55ba0b5f7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:51.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:50 smithi131 ceph-mon[37260]: pgmap v79: 129 pgs: 129 active+clean; 149 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.1 MiB/s wr, 388 op/s 2024-05-08T22:09:51.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.141+0000 7fe092fedb80 1 -- 172.21.15.131:0/370296038 shutdown_connections 2024-05-08T22:09:51.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.141+0000 7fe092fedb80 1 -- 172.21.15.131:0/370296038 wait complete. 2024-05-08T22:09:51.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.142+0000 7fe092fedb80 1 -- start start 2024-05-08T22:09:51.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.143+0000 7fe092fedb80 1 Processor -- start 2024-05-08T22:09:51.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.145+0000 7fe092fedb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ba0b5fd890 0x55ba0b606b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:51.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.145+0000 7fe092fedb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba0b602e00 0x55ba0b607040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:51.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.147+0000 7fe092fedb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b603720 0x55ba0b607580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:51.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.147+0000 7fe077e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba0b602e00 0x55ba0b607040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:51.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.147+0000 7fe07d400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b603720 0x55ba0b607580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:51.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.147+0000 7fe07d400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b603720 0x55ba0b607580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51674/0 (socket says 172.21.15.131:51674) 2024-05-08T22:09:51.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.147+0000 7fe07d400700 1 -- 172.21.15.131:0/2199545469 learned_addr learned my addr 172.21.15.131:0/2199545469 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:51.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.148+0000 7fe092fedb80 1 -- 172.21.15.131:0/2199545469 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0b4e9560 con 0x55ba0b602e00 2024-05-08T22:09:51.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.148+0000 7fe092fedb80 1 -- 172.21.15.131:0/2199545469 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0b4e9200 con 0x55ba0b5fd890 2024-05-08T22:09:51.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.148+0000 7fe092fedb80 1 -- 172.21.15.131:0/2199545469 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0b3d06a0 con 0x55ba0b603720 2024-05-08T22:09:51.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.148+0000 7fe092fedb80 10 client.0 _ll_register_callbacks cb 0x55ba0b4f2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:51.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.149+0000 7fe077e00700 1 -- 172.21.15.131:0/2199545469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b603720 msgr2=0x55ba0b607580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:51.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.149+0000 7fe077e00700 1 --2- 172.21.15.131:0/2199545469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b603720 0x55ba0b607580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.149+0000 7fe077e00700 1 -- 172.21.15.131:0/2199545469 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ba0b5fd890 msgr2=0x55ba0b606b00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:09:51.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.149+0000 7fe077e00700 1 --2- 172.21.15.131:0/2199545469 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ba0b5fd890 0x55ba0b606b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.149+0000 7fe077e00700 1 -- 172.21.15.131:0/2199545469 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0781b3080 con 0x55ba0b602e00 2024-05-08T22:09:51.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.151+0000 7fe07d400700 1 --2- 172.21.15.131:0/2199545469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b603720 0x55ba0b607580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:09:51.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.154+0000 7fe077e00700 1 --2- 172.21.15.131:0/2199545469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba0b602e00 0x55ba0b607040 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55ba0b603b80 tx=0x7fe064010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89d327cee49ed67 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:51.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.155+0000 7fe05e000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:09:51.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.155+0000 7fe05e000700 1 -- 172.21.15.131:0/2199545469 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe064014800 con 0x55ba0b602e00 2024-05-08T22:09:51.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.155+0000 7fe05e000700 1 -- 172.21.15.131:0/2199545469 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fe064014960 con 0x55ba0b602e00 2024-05-08T22:09:51.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.155+0000 7fe092fedb80 1 -- 172.21.15.131:0/2199545469 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba0b4a6270 con 0x55ba0b602e00 2024-05-08T22:09:51.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.158+0000 7fe05e000700 1 -- 172.21.15.131:0/2199545469 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe064010ea0 con 0x55ba0b602e00 2024-05-08T22:09:51.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.158+0000 7fe092fedb80 10 client.16005 fetch_fsmap learned FSMap version 25 2024-05-08T22:09:51.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.158+0000 7fe092fedb80 1 -- 172.21.15.131:0/2199545469 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ba0b37ab60 con 0x55ba0b602e00 2024-05-08T22:09:51.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.161+0000 7fe05e000700 1 -- 172.21.15.131:0/2199545469 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ba0b4a6270 con 0x55ba0b602e00 2024-05-08T22:09:51.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.161+0000 7fe05e000700 1 -- 172.21.15.131:0/2199545469 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fe06401dd00 con 0x55ba0b602e00 2024-05-08T22:09:51.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.161+0000 7fe092fedb80 10 client.16005 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T22:09:51.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.161+0000 7fe092fedb80 10 client.16005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:51.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.161+0000 7fe092fedb80 20 client.16005 populate_metadata read hostname 'smithi131' 2024-05-08T22:09:51.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.161+0000 7fe092fedb80 1 --2- 172.21.15.131:0/2199545469 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ba0b616f60 0x55ba0b619380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:51.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.161+0000 7fe092fedb80 4 client.16005 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:09:51.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.161+0000 7fe092fedb80 1 -- 172.21.15.131:0/2199545469 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba0b5cc810 con 0x55ba0b616f60 2024-05-08T22:09:51.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.165+0000 7fe07ca00700 1 --2- 172.21.15.131:0/2199545469 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ba0b616f60 0x55ba0b619380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:51.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.166+0000 7fe05e000700 10 client.16005 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:51.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.167+0000 7fe07ca00700 1 --2- 172.21.15.131:0/2199545469 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ba0b616f60 0x55ba0b619380 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40344eb153649a9e server_cookie=1f9c799235717ded in_seq=0 out_seq=0 2024-05-08T22:09:51.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.170+0000 7fe05e000700 1 -- 172.21.15.131:0/2199545469 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ba0b5cc810 con 0x55ba0b616f60 2024-05-08T22:09:51.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.170+0000 7fe05e000700 10 client.16005 handle_command_reply: tid=0 2024-05-08T22:09:51.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.170+0000 7fe092fedb80 1 client.16005 shutdown 2024-05-08T22:09:51.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.171+0000 7fe092fedb80 1 -- 172.21.15.131:0/2199545469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba0b602e00 msgr2=0x55ba0b607040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:51.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.171+0000 7fe092fedb80 1 --2- 172.21.15.131:0/2199545469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba0b602e00 0x55ba0b607040 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55ba0b603b80 tx=0x7fe064010800 comp rx=0 tx=0).stop 2024-05-08T22:09:51.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.171+0000 7fe092fedb80 1 -- 172.21.15.131:0/2199545469 shutdown_connections 2024-05-08T22:09:51.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.172+0000 7fe092fedb80 1 --2- 172.21.15.131:0/2199545469 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ba0b616f60 0x55ba0b619380 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.172+0000 7fe092fedb80 1 --2- 172.21.15.131:0/2199545469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b603720 0x55ba0b607580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.172+0000 7fe092fedb80 1 --2- 172.21.15.131:0/2199545469 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ba0b5fd890 0x55ba0b606b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.172+0000 7fe05e000700 0 client.16005 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:51.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.172+0000 7fe092fedb80 1 --2- 172.21.15.131:0/2199545469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba0b602e00 0x55ba0b607040 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.172+0000 7fe092fedb80 1 -- 172.21.15.131:0/2199545469 >> 172.21.15.131:0/2199545469 conn(0x55ba0b55b210 msgr2=0x55ba0b5f75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:51.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.173+0000 7fe092fedb80 1 -- 172.21.15.131:0/2199545469 shutdown_connections 2024-05-08T22:09:51.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.174+0000 7fe092fedb80 1 -- 172.21.15.131:0/2199545469 wait complete. 2024-05-08T22:09:51.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.183+0000 7fe092fedb80 20 client.16005 trim_cache size 0 max 16384 2024-05-08T22:09:51.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:50 smithi157 ceph-mon[77746]: pgmap v79: 129 pgs: 129 active+clean; 149 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.1 MiB/s wr, 388 op/s 2024-05-08T22:09:51.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.358+0000 7fe092fedb80 1 Processor -- start 2024-05-08T22:09:51.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.359+0000 7fe092fedb80 1 -- start start 2024-05-08T22:09:51.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.360+0000 7fe092fedb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba0b5fd890 0x55ba0b5fd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:51.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.360+0000 7fe092fedb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b602e00 0x55ba0b5f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:51.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.360+0000 7fe092fedb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ba0b603720 0x55ba0b601bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:51.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.360+0000 7fe092fedb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0b6080e0 con 0x55ba0b5fd890 2024-05-08T22:09:51.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.360+0000 7fe092fedb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0b607e30 con 0x55ba0b603720 2024-05-08T22:09:51.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.360+0000 7fe092fedb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0b6068f0 con 0x55ba0b602e00 2024-05-08T22:09:51.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.362+0000 7fe077e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b602e00 0x55ba0b5f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:51.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.362+0000 7fe07ca00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba0b5fd890 0x55ba0b5fd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:51.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.362+0000 7fe077e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b602e00 0x55ba0b5f7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51684/0 (socket says 172.21.15.131:51684) 2024-05-08T22:09:51.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.362+0000 7fe077e00700 1 -- 172.21.15.131:0/2593936828 learned_addr learned my addr 172.21.15.131:0/2593936828 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:51.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.363+0000 7fe07d400700 1 --2- 172.21.15.131:0/2593936828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ba0b603720 0x55ba0b601bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:51.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.363+0000 7fe07ca00700 1 -- 172.21.15.131:0/2593936828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b602e00 msgr2=0x55ba0b5f7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:51.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.363+0000 7fe07ca00700 1 --2- 172.21.15.131:0/2593936828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b602e00 0x55ba0b5f7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.363+0000 7fe07ca00700 1 -- 172.21.15.131:0/2593936828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ba0b603720 msgr2=0x55ba0b601bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:51.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.363+0000 7fe07ca00700 1 --2- 172.21.15.131:0/2593936828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ba0b603720 0x55ba0b601bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.363+0000 7fe07ca00700 1 -- 172.21.15.131:0/2593936828 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba0b513560 con 0x55ba0b5fd890 2024-05-08T22:09:51.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.363+0000 7fe07d400700 1 --2- 172.21.15.131:0/2593936828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ba0b603720 0x55ba0b601bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:09:51.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.368+0000 7fe07ca00700 1 --2- 172.21.15.131:0/2593936828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba0b5fd890 0x55ba0b5fd350 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x55ba0b617340 tx=0x7fe06c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5f680ad9d58b3d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:51.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.368+0000 7fe06a600700 1 -- 172.21.15.131:0/2593936828 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c022620 con 0x55ba0b5fd890 2024-05-08T22:09:51.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.368+0000 7fe06a600700 1 -- 172.21.15.131:0/2593936828 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe06c019040 con 0x55ba0b5fd890 2024-05-08T22:09:51.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.369+0000 7fe06a600700 1 -- 172.21.15.131:0/2593936828 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c013420 con 0x55ba0b5fd890 2024-05-08T22:09:51.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.371+0000 7fe092fedb80 1 -- 172.21.15.131:0/2593936828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba0b5fd890 msgr2=0x55ba0b5fd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:51.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.371+0000 7fe092fedb80 1 --2- 172.21.15.131:0/2593936828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba0b5fd890 0x55ba0b5fd350 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x55ba0b617340 tx=0x7fe06c012000 comp rx=0 tx=0).stop 2024-05-08T22:09:51.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.381+0000 7fe092fedb80 1 -- 172.21.15.131:0/2593936828 shutdown_connections 2024-05-08T22:09:51.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.381+0000 7fe092fedb80 1 --2- 172.21.15.131:0/2593936828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ba0b603720 0x55ba0b601bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.381+0000 7fe092fedb80 1 --2- 172.21.15.131:0/2593936828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba0b5fd890 0x55ba0b5fd350 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.381+0000 7fe092fedb80 1 --2- 172.21.15.131:0/2593936828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b602e00 0x55ba0b5f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.381+0000 7fe092fedb80 1 -- 172.21.15.131:0/2593936828 >> 172.21.15.131:0/2593936828 conn(0x55ba0b55b210 msgr2=0x55ba0b6053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:51.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.381+0000 7fe092fedb80 1 -- 172.21.15.131:0/2593936828 shutdown_connections 2024-05-08T22:09:51.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.381+0000 7fe092fedb80 1 -- 172.21.15.131:0/2593936828 wait complete. 2024-05-08T22:09:51.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.382+0000 7fe092fedb80 1 -- start start 2024-05-08T22:09:51.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.382+0000 7fe092fedb80 1 Processor -- start 2024-05-08T22:09:51.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.383+0000 7fe092fedb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b5fd890 0x55ba0b60ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:51.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.383+0000 7fe092fedb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ba0b602e00 0x55ba0b60e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:51.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.383+0000 7fe07ca00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b5fd890 0x55ba0b60ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:51.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.383+0000 7fe07ca00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b5fd890 0x55ba0b60ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51694/0 (socket says 172.21.15.131:51694) 2024-05-08T22:09:51.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.386+0000 7fe092fedb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba0b603720 0x55ba0b60c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:51.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.386+0000 7fe092fedb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0b60cb60 con 0x55ba0b603720 2024-05-08T22:09:51.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.386+0000 7fe092fedb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0b60ccd0 con 0x55ba0b602e00 2024-05-08T22:09:51.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.386+0000 7fe092fedb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0b60ce90 con 0x55ba0b5fd890 2024-05-08T22:09:51.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.386+0000 7fe092fedb80 10 client.0 _ll_register_callbacks cb 0x55ba0b4f2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:09:51.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.386+0000 7fe07ca00700 1 -- 172.21.15.131:0/2335249636 learned_addr learned my addr 172.21.15.131:0/2335249636 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:09:51.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.386+0000 7fe077e00700 1 --2- 172.21.15.131:0/2335249636 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ba0b602e00 0x55ba0b60e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:51.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.386+0000 7fe07d400700 1 --2- 172.21.15.131:0/2335249636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba0b603720 0x55ba0b60c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:51.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.387+0000 7fe07ca00700 1 -- 172.21.15.131:0/2335249636 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ba0b602e00 msgr2=0x55ba0b60e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:51.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.387+0000 7fe07ca00700 1 --2- 172.21.15.131:0/2335249636 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ba0b602e00 0x55ba0b60e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.387+0000 7fe07ca00700 1 -- 172.21.15.131:0/2335249636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba0b603720 msgr2=0x55ba0b60c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:09:51.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.387+0000 7fe07ca00700 1 --2- 172.21.15.131:0/2335249636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba0b603720 0x55ba0b60c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.387+0000 7fe07ca00700 1 -- 172.21.15.131:0/2335249636 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe06c00c890 con 0x55ba0b5fd890 2024-05-08T22:09:51.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.387+0000 7fe077e00700 1 --2- 172.21.15.131:0/2335249636 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ba0b602e00 0x55ba0b60e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:09:51.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.387+0000 7fe07d400700 1 --2- 172.21.15.131:0/2335249636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba0b603720 0x55ba0b60c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:09:51.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.387+0000 7fe07ca00700 1 --2- 172.21.15.131:0/2335249636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b5fd890 0x55ba0b60ded0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe06c025040 tx=0x7fe06c02be80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4427347ec3333492 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:09:51.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.387+0000 7fe05e000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:09:51.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.388+0000 7fe05e000700 1 -- 172.21.15.131:0/2335249636 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c0228b0 con 0x55ba0b5fd890 2024-05-08T22:09:51.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.388+0000 7fe05e000700 1 -- 172.21.15.131:0/2335249636 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fe06c005a20 con 0x55ba0b5fd890 2024-05-08T22:09:51.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.389+0000 7fe092fedb80 1 -- 172.21.15.131:0/2335249636 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe07009c390 con 0x55ba0b5fd890 2024-05-08T22:09:51.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.394+0000 7fe05e000700 1 -- 172.21.15.131:0/2335249636 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe06c005be0 con 0x55ba0b5fd890 2024-05-08T22:09:51.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.394+0000 7fe092fedb80 10 client.26086 fetch_fsmap learned FSMap version 25 2024-05-08T22:09:51.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.394+0000 7fe092fedb80 1 -- 172.21.15.131:0/2335249636 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ba0b3d2cd0 con 0x55ba0b5fd890 2024-05-08T22:09:51.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.394+0000 7fe05e000700 1 -- 172.21.15.131:0/2335249636 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe07009c390 con 0x55ba0b5fd890 2024-05-08T22:09:51.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.394+0000 7fe05e000700 1 -- 172.21.15.131:0/2335249636 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fe06c0044c0 con 0x55ba0b5fd890 2024-05-08T22:09:51.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.394+0000 7fe092fedb80 10 client.26086 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T22:09:51.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.394+0000 7fe092fedb80 10 client.26086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:09:51.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.394+0000 7fe092fedb80 20 client.26086 populate_metadata read hostname 'smithi131' 2024-05-08T22:09:51.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.396+0000 7fe092fedb80 1 --2- 172.21.15.131:0/2335249636 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ba0b668520 0x55ba0b668930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:09:51.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.396+0000 7fe092fedb80 4 client.26086 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:09:51.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.396+0000 7fe092fedb80 1 -- 172.21.15.131:0/2335249636 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ba0b545ec0 con 0x55ba0b668520 2024-05-08T22:09:51.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.398+0000 7fe077e00700 1 --2- 172.21.15.131:0/2335249636 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ba0b668520 0x55ba0b668930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:09:51.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.398+0000 7fe077e00700 1 --2- 172.21.15.131:0/2335249636 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ba0b668520 0x55ba0b668930 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aacb4adae61470b0 server_cookie=d49892e7874c239 in_seq=0 out_seq=0 2024-05-08T22:09:51.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.402+0000 7fe05e000700 10 client.26086 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:51.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.402+0000 7fe05e000700 1 -- 172.21.15.131:0/2335249636 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55ba0b545ec0 con 0x55ba0b668520 2024-05-08T22:09:51.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.402+0000 7fe05e000700 10 client.26086 handle_command_reply: tid=0 2024-05-08T22:09:51.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.402+0000 7fe092fedb80 1 client.26086 shutdown 2024-05-08T22:09:51.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.403+0000 7fe092fedb80 1 -- 172.21.15.131:0/2335249636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b5fd890 msgr2=0x55ba0b60ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:51.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.403+0000 7fe092fedb80 1 --2- 172.21.15.131:0/2335249636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b5fd890 0x55ba0b60ded0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe06c025040 tx=0x7fe06c02be80 comp rx=0 tx=0).stop 2024-05-08T22:09:51.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.406+0000 7fe092fedb80 1 -- 172.21.15.131:0/2335249636 shutdown_connections 2024-05-08T22:09:51.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.407+0000 7fe092fedb80 1 --2- 172.21.15.131:0/2335249636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ba0b603720 0x55ba0b60c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.407+0000 7fe092fedb80 1 --2- 172.21.15.131:0/2335249636 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ba0b668520 0x55ba0b668930 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.407+0000 7fe05e000700 0 client.26086 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:09:51.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.407+0000 7fe092fedb80 1 --2- 172.21.15.131:0/2335249636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba0b5fd890 0x55ba0b60ded0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.407+0000 7fe092fedb80 1 --2- 172.21.15.131:0/2335249636 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ba0b602e00 0x55ba0b60e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.407+0000 7fe092fedb80 1 -- 172.21.15.131:0/2335249636 >> 172.21.15.131:0/2335249636 conn(0x55ba0b55b210 msgr2=0x55ba0b601bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:51.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.411+0000 7fe092fedb80 1 -- 172.21.15.131:0/2335249636 shutdown_connections 2024-05-08T22:09:51.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.411+0000 7fe092fedb80 1 -- 172.21.15.131:0/2335249636 wait complete. 2024-05-08T22:09:51.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.411+0000 7fe092fedb80 20 client.26086 trim_cache size 0 max 16384 2024-05-08T22:09:51.412 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:09:51.413 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "scrub active (3 inodes in the stack)", 2024-05-08T22:09:51.413 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-08T22:09:51.413 INFO:teuthology.orchestra.run.smithi131.stdout: "5412d246-753f-4286-8c74-a6f748feebcb": { 2024-05-08T22:09:51.413 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-08T22:09:51.413 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "5412d246-753f-4286-8c74-a6f748feebcb", 2024-05-08T22:09:51.413 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-08T22:09:51.413 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:09:51.413 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:09:51.413 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:09:51.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.412+0000 7fe07f400700 1 -- 172.21.15.131:0/603165296 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe0600a1350 msgr2=0x7fe0600a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:51.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.412+0000 7fe07f400700 1 --2- 172.21.15.131:0/603165296 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe0600a1350 0x7fe0600a3810 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe06c006fd0 tx=0x7fe06c008040 comp rx=0 tx=0).stop 2024-05-08T22:09:51.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.413+0000 7fe07f400700 1 -- 172.21.15.131:0/603165296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe07811a780 msgr2=0x7fe0781aabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:09:51.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.413+0000 7fe07f400700 1 --2- 172.21.15.131:0/603165296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe07811a780 0x7fe0781aabf0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe07000b070 tx=0x7fe070007b60 comp rx=0 tx=0).stop 2024-05-08T22:09:51.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.413+0000 7fe07f400700 1 -- 172.21.15.131:0/603165296 shutdown_connections 2024-05-08T22:09:51.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.413+0000 7fe07f400700 1 --2- 172.21.15.131:0/603165296 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe0600a1350 0x7fe0600a3810 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.413+0000 7fe07f400700 1 --2- 172.21.15.131:0/603165296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe07811a780 0x7fe0781aabf0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.413+0000 7fe07f400700 1 --2- 172.21.15.131:0/603165296 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe07810f6a0 0x7fe0781a6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.413+0000 7fe07f400700 1 --2- 172.21.15.131:0/603165296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe07810fb10 0x7fe0781a6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:09:51.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.413+0000 7fe07f400700 1 -- 172.21.15.131:0/603165296 >> 172.21.15.131:0/603165296 conn(0x7fe07806bff0 msgr2=0x7fe078118770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:09:51.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.415+0000 7fe07f400700 1 -- 172.21.15.131:0/603165296 shutdown_connections 2024-05-08T22:09:51.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:09:51.416+0000 7fe07f400700 1 -- 172.21.15.131:0/603165296 wait complete. 2024-05-08T22:09:51.521 INFO:tasks.cephfs.filesystem:scrub status for tag:5412d246-753f-4286-8c74-a6f748feebcb - {'path': '/', 'tag': '5412d246-753f-4286-8c74-a6f748feebcb', 'options': 'recursive,force'} 2024-05-08T22:09:52.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:51 smithi132 ceph-mon[78180]: scrub queued for path: / 2024-05-08T22:09:52.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:51 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:09:52.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:51 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:09:52.150 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:51 smithi131 ceph-mon[37260]: scrub queued for path: / 2024-05-08T22:09:52.150 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:51 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:09:52.151 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:51 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:09:52.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:51 smithi157 ceph-mon[77746]: scrub queued for path: / 2024-05-08T22:09:52.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:51 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:09:52.277 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:51 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:09:53.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:52 smithi132 ceph-mon[78180]: pgmap v80: 129 pgs: 129 active+clean; 149 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.3 MiB/s wr, 337 op/s 2024-05-08T22:09:53.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:52 smithi131 ceph-mon[37260]: pgmap v80: 129 pgs: 129 active+clean; 149 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.3 MiB/s wr, 337 op/s 2024-05-08T22:09:53.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:52 smithi157 ceph-mon[77746]: pgmap v80: 129 pgs: 129 active+clean; 149 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.3 MiB/s wr, 337 op/s 2024-05-08T22:09:55.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:54 smithi132 ceph-mon[78180]: pgmap v81: 129 pgs: 129 active+clean; 149 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.3 MiB/s wr, 337 op/s 2024-05-08T22:09:55.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:54 smithi131 ceph-mon[37260]: pgmap v81: 129 pgs: 129 active+clean; 149 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.3 MiB/s wr, 337 op/s 2024-05-08T22:09:55.222 INFO:tasks.workunit.client.0.smithi131.stdout: 40 493790 320 311259 347 207741 992 2024-05-08T22:09:55.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:54 smithi157 ceph-mon[77746]: pgmap v81: 129 pgs: 129 active+clean; 149 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.3 MiB/s wr, 337 op/s 2024-05-08T22:09:57.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:56 smithi132 ceph-mon[78180]: pgmap v82: 129 pgs: 129 active+clean; 185 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 693 op/s 2024-05-08T22:09:57.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:56 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:09:57.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:56 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:09:57.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:56 smithi131 ceph-mon[37260]: pgmap v82: 129 pgs: 129 active+clean; 185 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 693 op/s 2024-05-08T22:09:57.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:56 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:09:57.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:56 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:09:57.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:56 smithi157 ceph-mon[77746]: pgmap v82: 129 pgs: 129 active+clean; 185 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 693 op/s 2024-05-08T22:09:57.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:56 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:09:57.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:56 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:09:59.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:09:58 smithi132 ceph-mon[78180]: pgmap v83: 129 pgs: 129 active+clean; 185 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.3 MiB/s wr, 664 op/s 2024-05-08T22:09:59.147 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:09:58 smithi131 ceph-mon[37260]: pgmap v83: 129 pgs: 129 active+clean; 185 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.3 MiB/s wr, 664 op/s 2024-05-08T22:09:59.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:09:58 smithi157 ceph-mon[77746]: pgmap v83: 129 pgs: 129 active+clean; 185 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.3 MiB/s wr, 664 op/s 2024-05-08T22:10:01.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:00 smithi132 ceph-mon[78180]: pgmap v84: 129 pgs: 129 active+clean; 192 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 10 MiB/s wr, 755 op/s 2024-05-08T22:10:01.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:00 smithi132 ceph-mon[78180]: overall HEALTH_OK 2024-05-08T22:10:01.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:00 smithi131 ceph-mon[37260]: pgmap v84: 129 pgs: 129 active+clean; 192 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 10 MiB/s wr, 755 op/s 2024-05-08T22:10:01.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:00 smithi131 ceph-mon[37260]: overall HEALTH_OK 2024-05-08T22:10:01.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:00 smithi157 ceph-mon[77746]: pgmap v84: 129 pgs: 129 active+clean; 192 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 10 MiB/s wr, 755 op/s 2024-05-08T22:10:01.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:00 smithi157 ceph-mon[77746]: overall HEALTH_OK 2024-05-08T22:10:03.146 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:02 smithi131 ceph-mon[37260]: pgmap v85: 129 pgs: 129 active+clean; 192 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 446 op/s 2024-05-08T22:10:03.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:02 smithi157 ceph-mon[77746]: pgmap v85: 129 pgs: 129 active+clean; 192 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 446 op/s 2024-05-08T22:10:03.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:02 smithi132 ceph-mon[78180]: pgmap v85: 129 pgs: 129 active+clean; 192 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 446 op/s 2024-05-08T22:10:05.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:04 smithi131 ceph-mon[37260]: pgmap v86: 129 pgs: 129 active+clean; 192 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 446 op/s 2024-05-08T22:10:05.232 INFO:tasks.workunit.client.0.smithi131.stdout: 47 661482 482 414463 422 313040 1317 2024-05-08T22:10:05.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:04 smithi157 ceph-mon[77746]: pgmap v86: 129 pgs: 129 active+clean; 192 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 446 op/s 2024-05-08T22:10:05.339 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:04 smithi132 ceph-mon[78180]: pgmap v86: 129 pgs: 129 active+clean; 192 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 446 op/s 2024-05-08T22:10:07.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:06 smithi131 ceph-mon[37260]: pgmap v87: 129 pgs: 129 active+clean; 201 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.0 MiB/s wr, 484 op/s 2024-05-08T22:10:07.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:06 smithi157 ceph-mon[77746]: pgmap v87: 129 pgs: 129 active+clean; 201 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.0 MiB/s wr, 484 op/s 2024-05-08T22:10:07.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:06 smithi132 ceph-mon[78180]: pgmap v87: 129 pgs: 129 active+clean; 201 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.0 MiB/s wr, 484 op/s 2024-05-08T22:10:09.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:08 smithi131 ceph-mon[37260]: pgmap v88: 129 pgs: 129 active+clean; 201 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 129 op/s 2024-05-08T22:10:09.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:08 smithi157 ceph-mon[77746]: pgmap v88: 129 pgs: 129 active+clean; 201 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 129 op/s 2024-05-08T22:10:09.339 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:08 smithi132 ceph-mon[78180]: pgmap v88: 129 pgs: 129 active+clean; 201 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 129 op/s 2024-05-08T22:10:11.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:10 smithi131 ceph-mon[37260]: pgmap v89: 129 pgs: 129 active+clean; 214 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 159 op/s 2024-05-08T22:10:11.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:10 smithi157 ceph-mon[77746]: pgmap v89: 129 pgs: 129 active+clean; 214 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 159 op/s 2024-05-08T22:10:11.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:10 smithi132 ceph-mon[78180]: pgmap v89: 129 pgs: 129 active+clean; 214 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 159 op/s 2024-05-08T22:10:13.147 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:12 smithi131 ceph-mon[37260]: pgmap v90: 129 pgs: 129 active+clean; 214 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 68 op/s 2024-05-08T22:10:13.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:12 smithi157 ceph-mon[77746]: pgmap v90: 129 pgs: 129 active+clean; 214 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 68 op/s 2024-05-08T22:10:13.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:12 smithi132 ceph-mon[78180]: pgmap v90: 129 pgs: 129 active+clean; 214 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 68 op/s 2024-05-08T22:10:15.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:14 smithi131 ceph-mon[37260]: pgmap v91: 129 pgs: 129 active+clean; 214 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 68 op/s 2024-05-08T22:10:15.233 INFO:tasks.workunit.client.0.smithi131.stdout: 60 660787 512 409428 401 333012 1304 2024-05-08T22:10:15.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:14 smithi157 ceph-mon[77746]: pgmap v91: 129 pgs: 129 active+clean; 214 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 68 op/s 2024-05-08T22:10:15.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:14 smithi132 ceph-mon[78180]: pgmap v91: 129 pgs: 129 active+clean; 214 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 68 op/s 2024-05-08T22:10:17.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:16 smithi131 ceph-mon[37260]: pgmap v92: 129 pgs: 129 active+clean; 230 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 136 op/s 2024-05-08T22:10:17.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:16 smithi157 ceph-mon[77746]: pgmap v92: 129 pgs: 129 active+clean; 230 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 136 op/s 2024-05-08T22:10:17.339 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:16 smithi132 ceph-mon[78180]: pgmap v92: 129 pgs: 129 active+clean; 230 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 136 op/s 2024-05-08T22:10:19.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:18 smithi157 ceph-mon[77746]: pgmap v93: 129 pgs: 129 active+clean; 230 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 97 op/s 2024-05-08T22:10:19.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:18 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:10:19.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:18 smithi132 ceph-mon[78180]: pgmap v93: 129 pgs: 129 active+clean; 230 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 97 op/s 2024-05-08T22:10:19.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:18 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:10:19.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:18 smithi131 ceph-mon[37260]: pgmap v93: 129 pgs: 129 active+clean; 230 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 97 op/s 2024-05-08T22:10:19.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:18 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:10:21.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:20 smithi157 ceph-mon[77746]: pgmap v94: 129 pgs: 129 active+clean; 251 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.4 MiB/s wr, 196 op/s 2024-05-08T22:10:21.339 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:20 smithi132 ceph-mon[78180]: pgmap v94: 129 pgs: 129 active+clean; 251 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.4 MiB/s wr, 196 op/s 2024-05-08T22:10:21.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:20 smithi131 ceph-mon[37260]: pgmap v94: 129 pgs: 129 active+clean; 251 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.4 MiB/s wr, 196 op/s 2024-05-08T22:10:21.523 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:10:23.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:22 smithi157 ceph-mon[77746]: pgmap v95: 129 pgs: 129 active+clean; 257 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 220 op/s 2024-05-08T22:10:23.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:22 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:10:23.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:22 smithi132 ceph-mon[78180]: pgmap v95: 129 pgs: 129 active+clean; 257 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 220 op/s 2024-05-08T22:10:23.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:22 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:10:23.396 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:22 smithi131 ceph-mon[37260]: pgmap v95: 129 pgs: 129 active+clean; 257 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 220 op/s 2024-05-08T22:10:23.397 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:22 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:10:24.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:23 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:10:24.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:23 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:10:24.398 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:23 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:10:24.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.874+0000 7f4a96800700 1 -- 172.21.15.131:0/1229673807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9010d080 msgr2=0x7f4a9010d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:24.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.874+0000 7f4a96800700 1 --2- 172.21.15.131:0/1229673807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9010d080 0x7f4a9010d460 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4a84009990 tx=0x7f4a84009ca0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.884+0000 7f4a96800700 1 -- 172.21.15.131:0/1229673807 shutdown_connections 2024-05-08T22:10:24.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.884+0000 7f4a96800700 1 --2- 172.21.15.131:0/1229673807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a9011a430 0x7f4a9011c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.884+0000 7f4a96800700 1 --2- 172.21.15.131:0/1229673807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9010d080 0x7f4a9010d460 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.885+0000 7f4a96800700 1 --2- 172.21.15.131:0/1229673807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a9010d9a0 0x7f4a90119ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.886+0000 7f4a96800700 1 -- 172.21.15.131:0/1229673807 >> 172.21.15.131:0/1229673807 conn(0x7f4a9006be10 msgr2=0x7f4a9006c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:24.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.891+0000 7f4a96800700 1 -- 172.21.15.131:0/1229673807 shutdown_connections 2024-05-08T22:10:24.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.893+0000 7f4a96800700 1 -- 172.21.15.131:0/1229673807 wait complete. 2024-05-08T22:10:24.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.896+0000 7f4a96800700 1 Processor -- start 2024-05-08T22:10:24.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.896+0000 7f4a96800700 1 -- start start 2024-05-08T22:10:24.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.898+0000 7f4a96800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a9010d080 0x7f4a901a6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:24.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.899+0000 7f4a96800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9010d9a0 0x7f4a901a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:24.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.901+0000 7f4a94a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9010d9a0 0x7f4a901a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:24.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.901+0000 7f4a94a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9010d9a0 0x7f4a901a6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55058/0 (socket says 172.21.15.131:55058) 2024-05-08T22:10:24.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.902+0000 7f4a96800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a9011a430 0x7f4a901aa840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:24.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.902+0000 7f4a94a00700 1 -- 172.21.15.131:0/879735515 learned_addr learned my addr 172.21.15.131:0/879735515 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:24.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.902+0000 7f4a95e00700 1 --2- 172.21.15.131:0/879735515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a9011a430 0x7f4a901aa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:24.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.903+0000 7f4a96800700 1 -- 172.21.15.131:0/879735515 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a901aae40 con 0x7f4a9010d9a0 2024-05-08T22:10:24.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.903+0000 7f4a96800700 1 -- 172.21.15.131:0/879735515 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a901aafb0 con 0x7f4a9010d080 2024-05-08T22:10:24.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.903+0000 7f4a96800700 1 -- 172.21.15.131:0/879735515 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a901ab170 con 0x7f4a9011a430 2024-05-08T22:10:24.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.903+0000 7f4a95400700 1 --2- 172.21.15.131:0/879735515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a9010d080 0x7f4a901a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:24.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.908+0000 7f4a95400700 1 -- 172.21.15.131:0/879735515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a9011a430 msgr2=0x7f4a901aa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:24.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.908+0000 7f4a95400700 1 --2- 172.21.15.131:0/879735515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a9011a430 0x7f4a901aa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.908+0000 7f4a95400700 1 -- 172.21.15.131:0/879735515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9010d9a0 msgr2=0x7f4a901a6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:24.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.908+0000 7f4a95400700 1 --2- 172.21.15.131:0/879735515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9010d9a0 0x7f4a901a6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.908+0000 7f4a95400700 1 -- 172.21.15.131:0/879735515 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a840095f0 con 0x7f4a9010d080 2024-05-08T22:10:24.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.909+0000 7f4a94a00700 1 --2- 172.21.15.131:0/879735515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9010d9a0 0x7f4a901a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:10:24.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.909+0000 7f4a95400700 1 --2- 172.21.15.131:0/879735515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a9010d080 0x7f4a901a6020 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4a84000c00 tx=0x7f4a840037c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:24.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.909+0000 7f4a95e00700 1 --2- 172.21.15.131:0/879735515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a9011a430 0x7f4a901aa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:10:24.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.909+0000 7f4a8e000700 1 -- 172.21.15.131:0/879735515 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8401d070 con 0x7f4a9010d080 2024-05-08T22:10:24.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.909+0000 7f4a8e000700 1 -- 172.21.15.131:0/879735515 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a84005580 con 0x7f4a9010d080 2024-05-08T22:10:24.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.909+0000 7f4a8e000700 1 -- 172.21.15.131:0/879735515 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8400fda0 con 0x7f4a9010d080 2024-05-08T22:10:24.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.911+0000 7f4a96800700 1 -- 172.21.15.131:0/879735515 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a901ab3f0 con 0x7f4a9010d080 2024-05-08T22:10:24.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.911+0000 7f4a96800700 1 -- 172.21.15.131:0/879735515 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9010f6d0 con 0x7f4a9010d080 2024-05-08T22:10:24.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.932+0000 7f4a8e000700 1 -- 172.21.15.131:0/879735515 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f4a8400f460 con 0x7f4a9010d080 2024-05-08T22:10:24.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.934+0000 7f4a8e000700 1 --2- 172.21.15.131:0/879735515 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f4a780a14a0 0x7f4a780a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:24.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.934+0000 7f4a8e000700 1 -- 172.21.15.131:0/879735515 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f4a840cdfa0 con 0x7f4a9010d080 2024-05-08T22:10:24.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.936+0000 7f4aab8f6b80 1 Processor -- start 2024-05-08T22:10:24.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.939+0000 7f4a94a00700 1 --2- 172.21.15.131:0/879735515 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f4a780a14a0 0x7f4a780a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:24.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.940+0000 7f4a94a00700 1 --2- 172.21.15.131:0/879735515 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f4a780a14a0 0x7f4a780a3960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4a901a74b0 tx=0x7f4a7c00b3a0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:24.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.944+0000 7f4aab8f6b80 1 -- start start 2024-05-08T22:10:24.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.945+0000 7f4aab8f6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a1de6f050 0x557a1de6f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:24.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.945+0000 7f4aab8f6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f970 0x557a1de69350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:24.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.945+0000 7f4aab8f6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a1de69890 0x557a1de69cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:24.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.946+0000 7f4a95400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a1de6f050 0x557a1de6f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:24.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.946+0000 7f4a95400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a1de6f050 0x557a1de6f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:50882/0 (socket says 172.21.15.131:50882) 2024-05-08T22:10:24.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.947+0000 7f4a94a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f970 0x557a1de69350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:24.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.947+0000 7f4a94a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f970 0x557a1de69350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55064/0 (socket says 172.21.15.131:55064) 2024-05-08T22:10:24.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.947+0000 7f4a95400700 1 -- 172.21.15.131:0/3180337037 learned_addr learned my addr 172.21.15.131:0/3180337037 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:24.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.947+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/3180337037 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a1ddc0150 con 0x557a1de6f970 2024-05-08T22:10:24.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.948+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/3180337037 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a1ddbdb10 con 0x557a1de69890 2024-05-08T22:10:24.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.948+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/3180337037 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a1ddb9400 con 0x557a1de6f050 2024-05-08T22:10:24.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.950+0000 7f4a94a00700 1 -- 172.21.15.131:0/3180337037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a1de6f050 msgr2=0x557a1de6f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:24.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.950+0000 7f4a94a00700 1 --2- 172.21.15.131:0/3180337037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a1de6f050 0x557a1de6f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.950+0000 7f4a94a00700 1 -- 172.21.15.131:0/3180337037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a1de69890 msgr2=0x557a1de69cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:24.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.950+0000 7f4a94a00700 1 --2- 172.21.15.131:0/3180337037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a1de69890 0x557a1de69cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.950+0000 7f4a94a00700 1 -- 172.21.15.131:0/3180337037 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a1dc3a3b0 con 0x557a1de6f970 2024-05-08T22:10:24.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.960+0000 7f4a94a00700 1 --2- 172.21.15.131:0/3180337037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f970 0x557a1de69350 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00fed0 tx=0x7f4a7c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471b06e9e25acb87 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:24.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.963+0000 7f4a82600700 1 -- 172.21.15.131:0/3180337037 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c011e60 con 0x557a1de6f970 2024-05-08T22:10:24.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.963+0000 7f4a82600700 1 -- 172.21.15.131:0/3180337037 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a7c00b550 con 0x557a1de6f970 2024-05-08T22:10:24.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.963+0000 7f4a82600700 1 -- 172.21.15.131:0/3180337037 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c01e450 con 0x557a1de6f970 2024-05-08T22:10:24.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.965+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/3180337037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f970 msgr2=0x557a1de69350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:24.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.965+0000 7f4aab8f6b80 1 --2- 172.21.15.131:0/3180337037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f970 0x557a1de69350 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00fed0 tx=0x7f4a7c012000 comp rx=0 tx=0).stop 2024-05-08T22:10:24.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.966+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/3180337037 shutdown_connections 2024-05-08T22:10:24.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.966+0000 7f4aab8f6b80 1 --2- 172.21.15.131:0/3180337037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a1de69890 0x557a1de69cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.966+0000 7f4aab8f6b80 1 --2- 172.21.15.131:0/3180337037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a1de6f050 0x557a1de6f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.966+0000 7f4aab8f6b80 1 --2- 172.21.15.131:0/3180337037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f970 0x557a1de69350 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.966+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/3180337037 >> 172.21.15.131:0/3180337037 conn(0x557a1ddc7210 msgr2=0x557a1de642d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:24.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.975+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/3180337037 shutdown_connections 2024-05-08T22:10:24.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.977+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/3180337037 wait complete. 2024-05-08T22:10:24.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.977+0000 7f4aab8f6b80 1 -- start start 2024-05-08T22:10:24.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.978+0000 7f4aab8f6b80 1 Processor -- start 2024-05-08T22:10:24.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.979+0000 7f4aab8f6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a1de69890 0x557a1de75910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:24.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.979+0000 7f4aab8f6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f050 0x557a1de72c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:24.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.980+0000 7f4a94a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f050 0x557a1de72c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:24.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.980+0000 7f4a94a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f050 0x557a1de72c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55068/0 (socket says 172.21.15.131:55068) 2024-05-08T22:10:24.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.986+0000 7f4aab8f6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a1de6f970 0x557a1de73340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:24.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.986+0000 7f4aab8f6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a1dd55560 con 0x557a1de6f050 2024-05-08T22:10:24.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.986+0000 7f4aab8f6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a1dd55200 con 0x557a1de69890 2024-05-08T22:10:24.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.986+0000 7f4aab8f6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a1dc3c6a0 con 0x557a1de6f970 2024-05-08T22:10:24.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.986+0000 7f4aab8f6b80 10 client.0 _ll_register_callbacks cb 0x557a1dd5eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:10:24.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.986+0000 7f4a95400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a1de69890 0x557a1de75910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:24.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.986+0000 7f4a95400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a1de69890 0x557a1de75910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:59994/0 (socket says 172.21.15.131:59994) 2024-05-08T22:10:24.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.986+0000 7f4a95400700 1 -- 172.21.15.131:0/2618907346 learned_addr learned my addr 172.21.15.131:0/2618907346 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:24.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.986+0000 7f4a95e00700 1 --2- 172.21.15.131:0/2618907346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a1de6f970 0x557a1de73340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:24.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.988+0000 7f4a95400700 1 -- 172.21.15.131:0/2618907346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a1de6f970 msgr2=0x557a1de73340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:24.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.988+0000 7f4a95400700 1 --2- 172.21.15.131:0/2618907346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a1de6f970 0x557a1de73340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.988+0000 7f4a95400700 1 -- 172.21.15.131:0/2618907346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f050 msgr2=0x557a1de72c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:24.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.988+0000 7f4a95400700 1 --2- 172.21.15.131:0/2618907346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f050 0x557a1de72c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:24.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.988+0000 7f4a95400700 1 -- 172.21.15.131:0/2618907346 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a7c00fbb0 con 0x557a1de69890 2024-05-08T22:10:24.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.990+0000 7f4a94a00700 1 --2- 172.21.15.131:0/2618907346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f050 0x557a1de72c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-08T22:10:24.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.992+0000 7f4a95400700 1 --2- 172.21.15.131:0/2618907346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a1de69890 0x557a1de75910 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4a84028000 tx=0x7f4a8409abd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec053cfb67bc176f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:24.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.992+0000 7f4a76a00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:10:24.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.992+0000 7f4a76a00700 1 -- 172.21.15.131:0/2618907346 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a84016e80 con 0x557a1de69890 2024-05-08T22:10:24.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.992+0000 7f4a76a00700 1 -- 172.21.15.131:0/2618907346 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f4a8409b450 con 0x557a1de69890 2024-05-08T22:10:24.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.992+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/2618907346 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a1dd12270 con 0x557a1de69890 2024-05-08T22:10:24.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.992+0000 7f4a76a00700 1 -- 172.21.15.131:0/2618907346 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a8409ad50 con 0x557a1de69890 2024-05-08T22:10:24.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.992+0000 7f4aab8f6b80 10 client.26081 fetch_fsmap learned FSMap version 25 2024-05-08T22:10:24.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.992+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/2618907346 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557a1dbe6b60 con 0x557a1de69890 2024-05-08T22:10:24.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.992+0000 7f4a76a00700 1 -- 172.21.15.131:0/2618907346 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a840a8020 con 0x557a1de69890 2024-05-08T22:10:24.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.994+0000 7f4a76a00700 1 -- 172.21.15.131:0/2618907346 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f4a840a2cb0 con 0x557a1de69890 2024-05-08T22:10:24.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.994+0000 7f4aab8f6b80 10 client.26081 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T22:10:24.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.994+0000 7f4aab8f6b80 10 client.26081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:10:24.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.994+0000 7f4aab8f6b80 20 client.26081 populate_metadata read hostname 'smithi131' 2024-05-08T22:10:24.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.994+0000 7f4aab8f6b80 1 --2- 172.21.15.131:0/2618907346 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x557a1de82e80 0x557a1de852a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:24.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.994+0000 7f4aab8f6b80 4 client.26081 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:10:24.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.994+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/2618907346 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a1dd12270 con 0x557a1de82e80 2024-05-08T22:10:24.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.995+0000 7f4a94a00700 1 --2- 172.21.15.131:0/2618907346 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x557a1de82e80 0x557a1de852a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:24.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.995+0000 7f4a94a00700 1 --2- 172.21.15.131:0/2618907346 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x557a1de82e80 0x557a1de852a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6a1e574033967a5 server_cookie=262414cc7d135309 in_seq=0 out_seq=0 2024-05-08T22:10:24.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.996+0000 7f4a76a00700 10 client.26081 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:10:24.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.998+0000 7f4a76a00700 1 -- 172.21.15.131:0/2618907346 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557a1dd12270 con 0x557a1de82e80 2024-05-08T22:10:24.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.998+0000 7f4a76a00700 10 client.26081 handle_command_reply: tid=0 2024-05-08T22:10:24.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.998+0000 7f4aab8f6b80 1 client.26081 shutdown 2024-05-08T22:10:24.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.998+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/2618907346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a1de69890 msgr2=0x557a1de75910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:24.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:24.998+0000 7f4aab8f6b80 1 --2- 172.21.15.131:0/2618907346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a1de69890 0x557a1de75910 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4a84028000 tx=0x7f4a8409abd0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.003+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/2618907346 shutdown_connections 2024-05-08T22:10:25.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.003+0000 7f4aab8f6b80 1 --2- 172.21.15.131:0/2618907346 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x557a1de82e80 0x557a1de852a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.003+0000 7f4aab8f6b80 1 --2- 172.21.15.131:0/2618907346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a1de6f970 0x557a1de73340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.003+0000 7f4aab8f6b80 1 --2- 172.21.15.131:0/2618907346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a1de69890 0x557a1de75910 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.003+0000 7f4aab8f6b80 1 --2- 172.21.15.131:0/2618907346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f050 0x557a1de72c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.003+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/2618907346 >> 172.21.15.131:0/2618907346 conn(0x557a1ddc7210 msgr2=0x557a1de640b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:25.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.003+0000 7f4a76a00700 0 client.26081 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:10:25.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.018+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/2618907346 shutdown_connections 2024-05-08T22:10:25.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.018+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/2618907346 wait complete. 2024-05-08T22:10:25.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.020+0000 7f4aab8f6b80 20 client.26081 trim_cache size 0 max 16384 2024-05-08T22:10:25.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.159+0000 7f4aab8f6b80 1 Processor -- start 2024-05-08T22:10:25.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.163+0000 7f4aab8f6b80 1 -- start start 2024-05-08T22:10:25.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.164+0000 7f4aab8f6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a1de69890 0x557a1de89c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:25.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.164+0000 7f4aab8f6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a1de6f050 0x557a1de75520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:25.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.164+0000 7f4aab8f6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f970 0x557a1de75a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:25.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.164+0000 7f4a94a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a1de6f050 0x557a1de75520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:25.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.164+0000 7f4a94a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a1de6f050 0x557a1de75520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:50900/0 (socket says 172.21.15.131:50900) 2024-05-08T22:10:25.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.165+0000 7f4a94a00700 1 -- 172.21.15.131:0/3139747206 learned_addr learned my addr 172.21.15.131:0/3139747206 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:25.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.165+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/3139747206 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a1de647c0 con 0x557a1de6f970 2024-05-08T22:10:25.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.165+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/3139747206 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a1de5c040 con 0x557a1de69890 2024-05-08T22:10:25.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.165+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/3139747206 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a1de64dc0 con 0x557a1de6f050 2024-05-08T22:10:25.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.165+0000 7f4a95e00700 1 --2- 172.21.15.131:0/3139747206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f970 0x557a1de75a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:25.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.165+0000 7f4a94a00700 1 -- 172.21.15.131:0/3139747206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a1de69890 msgr2=0x557a1de89c00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:10:25.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.165+0000 7f4a94a00700 1 --2- 172.21.15.131:0/3139747206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a1de69890 0x557a1de89c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.165+0000 7f4a94a00700 1 -- 172.21.15.131:0/3139747206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f970 msgr2=0x557a1de75a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:25.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.165+0000 7f4a94a00700 1 --2- 172.21.15.131:0/3139747206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f970 0x557a1de75a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.165+0000 7f4a94a00700 1 -- 172.21.15.131:0/3139747206 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a1dd7f560 con 0x557a1de6f050 2024-05-08T22:10:25.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.165+0000 7f4a95e00700 1 --2- 172.21.15.131:0/3139747206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f970 0x557a1de75a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:10:25.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.165+0000 7f4a94a00700 1 --2- 172.21.15.131:0/3139747206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a1de6f050 0x557a1de75520 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c003a50 tx=0x7f4a7c003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cae7ed6f558e115a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.172+0000 7f4a83000700 1 -- 172.21.15.131:0/3139747206 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c003820 con 0x557a1de6f050 2024-05-08T22:10:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.172+0000 7f4a83000700 1 -- 172.21.15.131:0/3139747206 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a7c02c040 con 0x557a1de6f050 2024-05-08T22:10:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.172+0000 7f4a83000700 1 -- 172.21.15.131:0/3139747206 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c005290 con 0x557a1de6f050 2024-05-08T22:10:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.174+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/3139747206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a1de6f050 msgr2=0x557a1de75520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.174+0000 7f4aab8f6b80 1 --2- 172.21.15.131:0/3139747206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a1de6f050 0x557a1de75520 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c003a50 tx=0x7f4a7c003b20 comp rx=0 tx=0).stop 2024-05-08T22:10:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.175+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/3139747206 shutdown_connections 2024-05-08T22:10:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.175+0000 7f4aab8f6b80 1 --2- 172.21.15.131:0/3139747206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f970 0x557a1de75a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.175+0000 7f4aab8f6b80 1 --2- 172.21.15.131:0/3139747206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a1de69890 0x557a1de89c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.175+0000 7f4aab8f6b80 1 --2- 172.21.15.131:0/3139747206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a1de6f050 0x557a1de75520 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.175+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/3139747206 >> 172.21.15.131:0/3139747206 conn(0x557a1ddc7210 msgr2=0x557a1de69c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.175+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/3139747206 shutdown_connections 2024-05-08T22:10:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.175+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/3139747206 wait complete. 2024-05-08T22:10:25.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.177+0000 7f4aab8f6b80 1 -- start start 2024-05-08T22:10:25.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.177+0000 7f4aab8f6b80 1 Processor -- start 2024-05-08T22:10:25.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.179+0000 7f4aab8f6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a1de69890 0x557a1de7a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:25.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.179+0000 7f4aab8f6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f050 0x557a1de7a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:25.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.179+0000 7f4a95400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a1de69890 0x557a1de7a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:25.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.179+0000 7f4a95400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a1de69890 0x557a1de7a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:60016/0 (socket says 172.21.15.131:60016) 2024-05-08T22:10:25.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.180+0000 7f4a95400700 1 -- 172.21.15.131:0/2385640039 learned_addr learned my addr 172.21.15.131:0/2385640039 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:25.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.180+0000 7f4aab8f6b80 1 --2- 172.21.15.131:0/2385640039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a1de6f970 0x557a1de6aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:25.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.180+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/2385640039 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a1de6af40 con 0x557a1de6f050 2024-05-08T22:10:25.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.180+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/2385640039 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a1de6b0b0 con 0x557a1de69890 2024-05-08T22:10:25.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.180+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/2385640039 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a1de6b270 con 0x557a1de6f970 2024-05-08T22:10:25.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.181+0000 7f4aab8f6b80 10 client.0 _ll_register_callbacks cb 0x557a1dd5eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:10:25.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.181+0000 7f4a95e00700 1 --2- 172.21.15.131:0/2385640039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a1de6f970 0x557a1de6aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:25.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.181+0000 7f4a94a00700 1 --2- 172.21.15.131:0/2385640039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f050 0x557a1de7a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:25.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.181+0000 7f4a95400700 1 -- 172.21.15.131:0/2385640039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a1de6f970 msgr2=0x557a1de6aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:25.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.181+0000 7f4a95400700 1 --2- 172.21.15.131:0/2385640039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a1de6f970 0x557a1de6aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.181+0000 7f4a95400700 1 -- 172.21.15.131:0/2385640039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f050 msgr2=0x557a1de7a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:25.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.181+0000 7f4a95400700 1 --2- 172.21.15.131:0/2385640039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f050 0x557a1de7a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.181+0000 7f4a95400700 1 -- 172.21.15.131:0/2385640039 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a1dc3a3b0 con 0x557a1de69890 2024-05-08T22:10:25.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.182+0000 7f4a94a00700 1 --2- 172.21.15.131:0/2385640039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f050 0x557a1de7a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:10:25.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.182+0000 7f4a95400700 1 --2- 172.21.15.131:0/2385640039 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a1de69890 0x557a1de7a360 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4a8409aa50 tx=0x7f4a8409a9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17be4bc83dc9db77 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:25.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.182+0000 7f4a76a00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:10:25.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.182+0000 7f4a76a00700 1 -- 172.21.15.131:0/2385640039 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8409a450 con 0x557a1de69890 2024-05-08T22:10:25.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.182+0000 7f4a76a00700 1 -- 172.21.15.131:0/2385640039 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f4a840a9080 con 0x557a1de69890 2024-05-08T22:10:25.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.182+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/2385640039 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a7c00b550 con 0x557a1de69890 2024-05-08T22:10:25.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.183+0000 7f4a76a00700 1 -- 172.21.15.131:0/2385640039 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a840aeb90 con 0x557a1de69890 2024-05-08T22:10:25.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.183+0000 7f4aab8f6b80 10 client.26084 fetch_fsmap learned FSMap version 25 2024-05-08T22:10:25.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.183+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/2385640039 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557a1dc3ecd0 con 0x557a1de69890 2024-05-08T22:10:25.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.183+0000 7f4a76a00700 1 -- 172.21.15.131:0/2385640039 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a840a8040 con 0x557a1de69890 2024-05-08T22:10:25.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.183+0000 7f4a76a00700 1 -- 172.21.15.131:0/2385640039 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f4a84096b40 con 0x557a1de69890 2024-05-08T22:10:25.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.183+0000 7f4aab8f6b80 10 client.26084 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T22:10:25.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.183+0000 7f4aab8f6b80 10 client.26084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:10:25.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.183+0000 7f4aab8f6b80 20 client.26084 populate_metadata read hostname 'smithi131' 2024-05-08T22:10:25.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.183+0000 7f4aab8f6b80 1 --2- 172.21.15.131:0/2385640039 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x557a1ded9cc0 0x557a1dedc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:25.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.184+0000 7f4aab8f6b80 4 client.26084 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:10:25.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.184+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/2385640039 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f4a7c00b550 con 0x557a1ded9cc0 2024-05-08T22:10:25.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.184+0000 7f4a94a00700 1 --2- 172.21.15.131:0/2385640039 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x557a1ded9cc0 0x557a1dedc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:25.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.185+0000 7f4a76a00700 10 client.26084 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:10:25.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.185+0000 7f4a94a00700 1 --2- 172.21.15.131:0/2385640039 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x557a1ded9cc0 0x557a1dedc0b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9bb2967364d5e3f server_cookie=5e47fedf983db40f in_seq=0 out_seq=0 2024-05-08T22:10:25.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.188+0000 7f4a76a00700 1 -- 172.21.15.131:0/2385640039 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f4a7c00b550 con 0x557a1ded9cc0 2024-05-08T22:10:25.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.188+0000 7f4a76a00700 10 client.26084 handle_command_reply: tid=0 2024-05-08T22:10:25.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.188+0000 7f4aab8f6b80 1 client.26084 shutdown 2024-05-08T22:10:25.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.189+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/2385640039 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a1de69890 msgr2=0x557a1de7a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:25.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.189+0000 7f4aab8f6b80 1 --2- 172.21.15.131:0/2385640039 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a1de69890 0x557a1de7a360 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4a8409aa50 tx=0x7f4a8409a9b0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.189+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/2385640039 shutdown_connections 2024-05-08T22:10:25.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.189+0000 7f4aab8f6b80 1 --2- 172.21.15.131:0/2385640039 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x557a1ded9cc0 0x557a1dedc0b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.189+0000 7f4aab8f6b80 1 --2- 172.21.15.131:0/2385640039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557a1de6f970 0x557a1de6aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.189+0000 7f4aab8f6b80 1 --2- 172.21.15.131:0/2385640039 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a1de69890 0x557a1de7a360 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.189+0000 7f4aab8f6b80 1 --2- 172.21.15.131:0/2385640039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1de6f050 0x557a1de7a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.189+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/2385640039 >> 172.21.15.131:0/2385640039 conn(0x557a1ddc7210 msgr2=0x557a1de89c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:25.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.189+0000 7f4a76a00700 0 client.26084 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:10:25.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.190+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/2385640039 shutdown_connections 2024-05-08T22:10:25.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.190+0000 7f4aab8f6b80 1 -- 172.21.15.131:0/2385640039 wait complete. 2024-05-08T22:10:25.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.190+0000 7f4aab8f6b80 20 client.26084 trim_cache size 0 max 16384 2024-05-08T22:10:25.192 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:10:25.192 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-08T22:10:25.192 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-08T22:10:25.192 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:10:25.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.192+0000 7f4a96800700 1 -- 172.21.15.131:0/879735515 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f4a780a14a0 msgr2=0x7f4a780a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:25.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.192+0000 7f4a96800700 1 --2- 172.21.15.131:0/879735515 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f4a780a14a0 0x7f4a780a3960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4a901a74b0 tx=0x7f4a7c00b3a0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.192+0000 7f4a96800700 1 -- 172.21.15.131:0/879735515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a9010d080 msgr2=0x7f4a901a6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:25.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.192+0000 7f4a96800700 1 --2- 172.21.15.131:0/879735515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a9010d080 0x7f4a901a6020 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4a84000c00 tx=0x7f4a840037c0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.192+0000 7f4a96800700 1 -- 172.21.15.131:0/879735515 shutdown_connections 2024-05-08T22:10:25.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.192+0000 7f4a96800700 1 --2- 172.21.15.131:0/879735515 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f4a780a14a0 0x7f4a780a3960 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.192+0000 7f4a96800700 1 --2- 172.21.15.131:0/879735515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a9011a430 0x7f4a901aa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.192+0000 7f4a96800700 1 --2- 172.21.15.131:0/879735515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a9010d080 0x7f4a901a6020 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.192+0000 7f4a96800700 1 --2- 172.21.15.131:0/879735515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9010d9a0 0x7f4a901a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.192+0000 7f4a96800700 1 -- 172.21.15.131:0/879735515 >> 172.21.15.131:0/879735515 conn(0x7f4a9006be10 msgr2=0x7f4a90118fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:25.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.192+0000 7f4a96800700 1 -- 172.21.15.131:0/879735515 shutdown_connections 2024-05-08T22:10:25.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.192+0000 7f4a96800700 1 -- 172.21.15.131:0/879735515 wait complete. 2024-05-08T22:10:25.229 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:24 smithi131 ceph-mon[37260]: pgmap v96: 129 pgs: 129 active+clean; 257 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 220 op/s 2024-05-08T22:10:25.233 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T22:10:25.234 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:10:25.235 INFO:tasks.workunit.client.0.smithi131.stdout: 65 529975 401 336295 325 289472 1240 2024-05-08T22:10:25.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:24 smithi157 ceph-mon[77746]: pgmap v96: 129 pgs: 129 active+clean; 257 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 220 op/s 2024-05-08T22:10:25.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:24 smithi132 ceph-mon[78180]: pgmap v96: 129 pgs: 129 active+clean; 257 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 220 op/s 2024-05-08T22:10:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.777+0000 7f0d11e00700 1 -- 172.21.15.131:0/1396399433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d0c101050 msgr2=0x7f0d0c101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.777+0000 7f0d11e00700 1 --2- 172.21.15.131:0/1396399433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d0c101050 0x7f0d0c101430 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0098f0 tx=0x7f0cfc009c00 comp rx=0 tx=0).stop 2024-05-08T22:10:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.778+0000 7f0d11e00700 1 -- 172.21.15.131:0/1396399433 shutdown_connections 2024-05-08T22:10:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.778+0000 7f0d11e00700 1 --2- 172.21.15.131:0/1396399433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d0c106170 0x7f0d0c10dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.778+0000 7f0d11e00700 1 --2- 172.21.15.131:0/1396399433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d0c101050 0x7f0d0c101430 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.778+0000 7f0d11e00700 1 --2- 172.21.15.131:0/1396399433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d0c101a00 0x7f0d0c105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.778+0000 7f0d11e00700 1 -- 172.21.15.131:0/1396399433 >> 172.21.15.131:0/1396399433 conn(0x7f0d0c0fc8f0 msgr2=0x7f0d0c0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.778+0000 7f0d11e00700 1 -- 172.21.15.131:0/1396399433 shutdown_connections 2024-05-08T22:10:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.778+0000 7f0d11e00700 1 -- 172.21.15.131:0/1396399433 wait complete. 2024-05-08T22:10:25.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.779+0000 7f0d11e00700 1 Processor -- start 2024-05-08T22:10:25.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.779+0000 7f0d11e00700 1 -- start start 2024-05-08T22:10:25.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.779+0000 7f0d11e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d0c101050 0x7f0d0c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:25.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.779+0000 7f0d11e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d0c106170 0x7f0d0c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:25.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.779+0000 7f0d11e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d0c073d20 0x7f0d0c1a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:25.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.779+0000 7f0d11e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c1a5ae0 con 0x7f0d0c073d20 2024-05-08T22:10:25.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.779+0000 7f0d11e00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c1a5c20 con 0x7f0d0c106170 2024-05-08T22:10:25.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.779+0000 7f0d11e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c1a5de0 con 0x7f0d0c101050 2024-05-08T22:10:25.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.779+0000 7f0d0a000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d0c106170 0x7f0d0c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:25.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.779+0000 7f0d0b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d0c073d20 0x7f0d0c1a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:25.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.780+0000 7f0d0a000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d0c106170 0x7f0d0c0737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:60026/0 (socket says 172.21.15.131:60026) 2024-05-08T22:10:25.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.780+0000 7f0d0b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d0c073d20 0x7f0d0c1a5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55098/0 (socket says 172.21.15.131:55098) 2024-05-08T22:10:25.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.780+0000 7f0d0a000700 1 -- 172.21.15.131:0/2969733400 learned_addr learned my addr 172.21.15.131:0/2969733400 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:25.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.780+0000 7f0d0aa00700 1 --2- 172.21.15.131:0/2969733400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d0c101050 0x7f0d0c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:25.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.780+0000 7f0d0a000700 1 -- 172.21.15.131:0/2969733400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d0c101050 msgr2=0x7f0d0c0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:25.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.780+0000 7f0d0a000700 1 --2- 172.21.15.131:0/2969733400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d0c101050 0x7f0d0c0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.780+0000 7f0d0a000700 1 -- 172.21.15.131:0/2969733400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d0c073d20 msgr2=0x7f0d0c1a5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:25.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.780+0000 7f0d0a000700 1 --2- 172.21.15.131:0/2969733400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d0c073d20 0x7f0d0c1a5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:25.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.780+0000 7f0d0a000700 1 -- 172.21.15.131:0/2969733400 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc009520 con 0x7f0d0c106170 2024-05-08T22:10:25.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.780+0000 7f0d0a000700 1 --2- 172.21.15.131:0/2969733400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d0c106170 0x7f0d0c0737e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800e9c0 tx=0x7f0cf800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:25.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.780+0000 7f0d10a00700 1 -- 172.21.15.131:0/2969733400 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8004a30 con 0x7f0d0c106170 2024-05-08T22:10:25.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.780+0000 7f0d10a00700 1 -- 172.21.15.131:0/2969733400 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cf800cc80 con 0x7f0d0c106170 2024-05-08T22:10:25.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.780+0000 7f0d11e00700 1 -- 172.21.15.131:0/2969733400 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d0c1a60c0 con 0x7f0d0c106170 2024-05-08T22:10:25.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.780+0000 7f0d10a00700 1 -- 172.21.15.131:0/2969733400 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8005560 con 0x7f0d0c106170 2024-05-08T22:10:25.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.781+0000 7f0d11e00700 1 -- 172.21.15.131:0/2969733400 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d0c1a65d0 con 0x7f0d0c106170 2024-05-08T22:10:25.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.782+0000 7f0d11e00700 1 -- 172.21.15.131:0/2969733400 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d0c109390 con 0x7f0d0c106170 2024-05-08T22:10:25.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.783+0000 7f0d10a00700 1 -- 172.21.15.131:0/2969733400 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f0cf800c7c0 con 0x7f0d0c106170 2024-05-08T22:10:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.785+0000 7f0d10a00700 1 --2- 172.21.15.131:0/2969733400 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f0cf00a1310 0x7f0cf00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.785+0000 7f0d10a00700 1 -- 172.21.15.131:0/2969733400 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f0cf8014070 con 0x7f0d0c106170 2024-05-08T22:10:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.785+0000 7f0d0aa00700 1 --2- 172.21.15.131:0/2969733400 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f0cf00a1310 0x7f0cf00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.786+0000 7f0d0aa00700 1 --2- 172.21.15.131:0/2969733400 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f0cf00a1310 0x7f0cf00a37d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009fd0 tx=0x7f0cfc019040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:25.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:25.788+0000 7f0d10a00700 1 -- 172.21.15.131:0/2969733400 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0cf8096140 con 0x7f0d0c106170 2024-05-08T22:10:27.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:26 smithi157 ceph-mon[77746]: pgmap v97: 129 pgs: 129 active+clean; 312 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.9 MiB/s wr, 563 op/s 2024-05-08T22:10:27.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:26 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:10:27.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:26 smithi132 ceph-mon[78180]: pgmap v97: 129 pgs: 129 active+clean; 312 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.9 MiB/s wr, 563 op/s 2024-05-08T22:10:27.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:26 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:10:27.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:26 smithi131 ceph-mon[37260]: pgmap v97: 129 pgs: 129 active+clean; 312 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.9 MiB/s wr, 563 op/s 2024-05-08T22:10:27.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:26 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:10:27.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:27.536+0000 7f0d11e00700 1 -- 172.21.15.131:0/2969733400 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d0c074730 con 0x7f0d0c106170 2024-05-08T22:10:27.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:27.554+0000 7f0d10a00700 1 -- 172.21.15.131:0/2969733400 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10890 (secure 0 0 0) 0x7f0cf8099760 con 0x7f0d0c106170 2024-05-08T22:10:27.583 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:10:27.583 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:05:14.034908+0000","modified":"2024-05-08T22:09:33.712734+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:10:27.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:27.622+0000 7f0d11e00700 1 -- 172.21.15.131:0/2969733400 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f0cf00a1310 msgr2=0x7f0cf00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:27.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:27.622+0000 7f0d11e00700 1 --2- 172.21.15.131:0/2969733400 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f0cf00a1310 0x7f0cf00a37d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009fd0 tx=0x7f0cfc019040 comp rx=0 tx=0).stop 2024-05-08T22:10:27.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:27.622+0000 7f0d11e00700 1 -- 172.21.15.131:0/2969733400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d0c106170 msgr2=0x7f0d0c0737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:27.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:27.622+0000 7f0d11e00700 1 --2- 172.21.15.131:0/2969733400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d0c106170 0x7f0d0c0737e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800e9c0 tx=0x7f0cf800ecd0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:27.646+0000 7f0d11e00700 1 -- 172.21.15.131:0/2969733400 shutdown_connections 2024-05-08T22:10:27.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:27.646+0000 7f0d11e00700 1 --2- 172.21.15.131:0/2969733400 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f0cf00a1310 0x7f0cf00a37d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:27.646+0000 7f0d11e00700 1 --2- 172.21.15.131:0/2969733400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d0c073d20 0x7f0d0c1a5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:27.646+0000 7f0d11e00700 1 --2- 172.21.15.131:0/2969733400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d0c101050 0x7f0d0c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:27.646+0000 7f0d11e00700 1 --2- 172.21.15.131:0/2969733400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d0c106170 0x7f0d0c0737e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:27.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:27.646+0000 7f0d11e00700 1 -- 172.21.15.131:0/2969733400 >> 172.21.15.131:0/2969733400 conn(0x7f0d0c0fc8f0 msgr2=0x7f0d0c0fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:27.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:27.653+0000 7f0d11e00700 1 -- 172.21.15.131:0/2969733400 shutdown_connections 2024-05-08T22:10:27.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:27.677+0000 7f0d11e00700 1 -- 172.21.15.131:0/2969733400 wait complete. 2024-05-08T22:10:27.741 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 25 2024-05-08T22:10:28.182 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T22:10:28.482 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:28 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:10:28.482 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:28 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:10:28.482 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:28 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:10:28.485 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:28 smithi131 ceph-mon[37260]: pgmap v98: 129 pgs: 129 active+clean; 318 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.0 MiB/s wr, 542 op/s 2024-05-08T22:10:28.485 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:28 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:10:28.485 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:28 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2969733400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:10:28.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:28 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:10:28.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:28 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:10:28.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:28 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:10:28.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:28 smithi157 ceph-mon[77746]: pgmap v98: 129 pgs: 129 active+clean; 318 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.0 MiB/s wr, 542 op/s 2024-05-08T22:10:28.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:28 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:10:28.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:28 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2969733400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:10:28.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:28 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:10:28.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:28 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:10:28.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:28 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:10:28.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:28 smithi132 ceph-mon[78180]: pgmap v98: 129 pgs: 129 active+clean; 318 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.0 MiB/s wr, 542 op/s 2024-05-08T22:10:28.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:28 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:10:28.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:28 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2969733400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:10:30.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.474+0000 7f31fb000700 1 -- 172.21.15.131:0/1756087820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31f411a430 msgr2=0x7f31f411c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:30.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.474+0000 7f31fb000700 1 --2- 172.21.15.131:0/1756087820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31f411a430 0x7f31f411c820 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009990 tx=0x7f31e0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.474+0000 7f31fb000700 1 -- 172.21.15.131:0/1756087820 shutdown_connections 2024-05-08T22:10:30.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.475+0000 7f31fb000700 1 --2- 172.21.15.131:0/1756087820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31f411a430 0x7f31f411c820 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.475+0000 7f31fb000700 1 --2- 172.21.15.131:0/1756087820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31f410d080 0x7f31f410d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.475+0000 7f31fb000700 1 --2- 172.21.15.131:0/1756087820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31f410d9a0 0x7f31f4119ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.475+0000 7f31fb000700 1 -- 172.21.15.131:0/1756087820 >> 172.21.15.131:0/1756087820 conn(0x7f31f406be10 msgr2=0x7f31f406c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:30.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.481+0000 7f31fb000700 1 -- 172.21.15.131:0/1756087820 shutdown_connections 2024-05-08T22:10:30.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.481+0000 7f31fb000700 1 -- 172.21.15.131:0/1756087820 wait complete. 2024-05-08T22:10:30.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.485+0000 7f31fb000700 1 Processor -- start 2024-05-08T22:10:30.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.487+0000 7f31fb000700 1 -- start start 2024-05-08T22:10:30.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.488+0000 7f31fb000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31f410d080 0x7f31f41a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:30.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.490+0000 7f31f9a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31f410d080 0x7f31f41a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:30.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.490+0000 7f31f9a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31f410d080 0x7f31f41a5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55122/0 (socket says 172.21.15.131:55122) 2024-05-08T22:10:30.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.490+0000 7f31f9a00700 1 -- 172.21.15.131:0/1026111267 learned_addr learned my addr 172.21.15.131:0/1026111267 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:30.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.490+0000 7f31fb000700 1 --2- 172.21.15.131:0/1026111267 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31f410d9a0 0x7f31f41a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:30.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.490+0000 7f31fb000700 1 --2- 172.21.15.131:0/1026111267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31f411a430 0x7f31f41aa720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:30.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.491+0000 7f31f3400700 1 --2- 172.21.15.131:0/1026111267 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31f410d9a0 0x7f31f41a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:30.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.491+0000 7f31f3e00700 1 --2- 172.21.15.131:0/1026111267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31f411a430 0x7f31f41aa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:30.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.491+0000 7f31fb000700 1 -- 172.21.15.131:0/1026111267 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f41aad60 con 0x7f31f410d080 2024-05-08T22:10:30.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.491+0000 7f31fb000700 1 -- 172.21.15.131:0/1026111267 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f41aaed0 con 0x7f31f410d9a0 2024-05-08T22:10:30.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.491+0000 7f31fb000700 1 -- 172.21.15.131:0/1026111267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f41ab090 con 0x7f31f411a430 2024-05-08T22:10:30.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.492+0000 7f31f9a00700 1 -- 172.21.15.131:0/1026111267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31f411a430 msgr2=0x7f31f41aa720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:30.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.492+0000 7f31f9a00700 1 --2- 172.21.15.131:0/1026111267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31f411a430 0x7f31f41aa720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.492+0000 7f31f9a00700 1 -- 172.21.15.131:0/1026111267 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31f410d9a0 msgr2=0x7f31f41a6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:30.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.492+0000 7f31f9a00700 1 --2- 172.21.15.131:0/1026111267 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31f410d9a0 0x7f31f41a6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.492+0000 7f31f9a00700 1 -- 172.21.15.131:0/1026111267 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e00095f0 con 0x7f31f410d080 2024-05-08T22:10:30.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.495+0000 7f31f9a00700 1 --2- 172.21.15.131:0/1026111267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31f410d080 0x7f31f41a5e40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f31ec001440 tx=0x7f31ec007580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:30.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.496+0000 7f31f0c00700 1 -- 172.21.15.131:0/1026111267 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec007de0 con 0x7f31f410d080 2024-05-08T22:10:30.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.496+0000 7f31f0c00700 1 -- 172.21.15.131:0/1026111267 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31ec00a7f0 con 0x7f31f410d080 2024-05-08T22:10:30.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.498+0000 7f31f0c00700 1 -- 172.21.15.131:0/1026111267 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec007de0 con 0x7f31f410d080 2024-05-08T22:10:30.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.499+0000 7f31fb000700 1 -- 172.21.15.131:0/1026111267 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f41ab370 con 0x7f31f410d080 2024-05-08T22:10:30.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.512+0000 7f31fb000700 1 -- 172.21.15.131:0/1026111267 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f41b2bb0 con 0x7f31f410d080 2024-05-08T22:10:30.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:30 smithi157 ceph-mon[77746]: pgmap v99: 129 pgs: 129 active+clean; 322 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.5 MiB/s wr, 593 op/s 2024-05-08T22:10:30.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.537+0000 7f320eae8b80 1 Processor -- start 2024-05-08T22:10:30.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.538+0000 7f320eae8b80 1 -- start start 2024-05-08T22:10:30.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.539+0000 7f320eae8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91050 0x555ce0d91430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:30.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.540+0000 7f31f3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91050 0x555ce0d91430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:30.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.540+0000 7f31f3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91050 0x555ce0d91430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55134/0 (socket says 172.21.15.131:55134) 2024-05-08T22:10:30.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.540+0000 7f320eae8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555ce0d91970 0x555ce0d8b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:30.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.540+0000 7f320eae8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ce0d8b890 0x555ce0d8bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:30.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.541+0000 7f31f3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ce0d8b890 0x555ce0d8bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:30.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.541+0000 7f31f3e00700 1 -- 172.21.15.131:0/3732482672 learned_addr learned my addr 172.21.15.131:0/3732482672 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:30.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.541+0000 7f320eae8b80 1 -- 172.21.15.131:0/3732482672 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce0ce2150 con 0x555ce0d91050 2024-05-08T22:10:30.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.541+0000 7f320eae8b80 1 -- 172.21.15.131:0/3732482672 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce0cdfb10 con 0x555ce0d91970 2024-05-08T22:10:30.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.541+0000 7f320eae8b80 1 -- 172.21.15.131:0/3732482672 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce0cdb400 con 0x555ce0d8b890 2024-05-08T22:10:30.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.542+0000 7f31f3e00700 1 -- 172.21.15.131:0/3732482672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ce0d8b890 msgr2=0x555ce0d8bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:30.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.542+0000 7f31f3e00700 1 --2- 172.21.15.131:0/3732482672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ce0d8b890 0x555ce0d8bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.542+0000 7f31f3e00700 1 -- 172.21.15.131:0/3732482672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555ce0d91970 msgr2=0x555ce0d8b350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:10:30.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.542+0000 7f31f3e00700 1 --2- 172.21.15.131:0/3732482672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555ce0d91970 0x555ce0d8b350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.542+0000 7f31f3e00700 1 -- 172.21.15.131:0/3732482672 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ce0b5c3b0 con 0x555ce0d91050 2024-05-08T22:10:30.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.543+0000 7f31f3e00700 1 --2- 172.21.15.131:0/3732482672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91050 0x555ce0d91430 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009990 tx=0x7f31e0005cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba4f9178929c48c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:30.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.547+0000 7f31f0c00700 1 -- 172.21.15.131:0/1026111267 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f31ec015430 con 0x7f31f410d080 2024-05-08T22:10:30.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.551+0000 7f31f0c00700 1 --2- 172.21.15.131:0/1026111267 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f31dc0a1410 0x7f31dc0a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:30.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.558+0000 7f31e5600700 1 -- 172.21.15.131:0/3732482672 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e001d030 con 0x555ce0d91050 2024-05-08T22:10:30.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.558+0000 7f31e5600700 1 -- 172.21.15.131:0/3732482672 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31e001e040 con 0x555ce0d91050 2024-05-08T22:10:30.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.558+0000 7f31e5600700 1 -- 172.21.15.131:0/3732482672 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e00056b0 con 0x555ce0d91050 2024-05-08T22:10:30.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.567+0000 7f31f0c00700 1 -- 172.21.15.131:0/1026111267 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f31ec0ce220 con 0x7f31f410d080 2024-05-08T22:10:30.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.568+0000 7f31f9a00700 1 --2- 172.21.15.131:0/1026111267 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f31dc0a1410 0x7f31dc0a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:30.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.570+0000 7f31f9a00700 1 --2- 172.21.15.131:0/1026111267 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f31dc0a1410 0x7f31dc0a38d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f31f41aaea0 tx=0x7f31ec095040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:30.589 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:30 smithi132 ceph-mon[78180]: pgmap v99: 129 pgs: 129 active+clean; 322 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.5 MiB/s wr, 593 op/s 2024-05-08T22:10:30.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.574+0000 7f320eae8b80 1 -- 172.21.15.131:0/3732482672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91050 msgr2=0x555ce0d91430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:30.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.574+0000 7f320eae8b80 1 --2- 172.21.15.131:0/3732482672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91050 0x555ce0d91430 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009990 tx=0x7f31e0005cb0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.590+0000 7f320eae8b80 1 -- 172.21.15.131:0/3732482672 shutdown_connections 2024-05-08T22:10:30.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.590+0000 7f320eae8b80 1 --2- 172.21.15.131:0/3732482672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ce0d8b890 0x555ce0d8bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.590+0000 7f320eae8b80 1 --2- 172.21.15.131:0/3732482672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91050 0x555ce0d91430 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.590+0000 7f320eae8b80 1 --2- 172.21.15.131:0/3732482672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555ce0d91970 0x555ce0d8b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.590+0000 7f320eae8b80 1 -- 172.21.15.131:0/3732482672 >> 172.21.15.131:0/3732482672 conn(0x555ce0ce9210 msgr2=0x555ce0d862d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:30.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.601+0000 7f320eae8b80 1 -- 172.21.15.131:0/3732482672 shutdown_connections 2024-05-08T22:10:30.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.601+0000 7f320eae8b80 1 -- 172.21.15.131:0/3732482672 wait complete. 2024-05-08T22:10:30.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.602+0000 7f320eae8b80 1 -- start start 2024-05-08T22:10:30.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.603+0000 7f320eae8b80 1 Processor -- start 2024-05-08T22:10:30.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.613+0000 7f320eae8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ce0d8b890 0x555ce0d94cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:30.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.618+0000 7f320eae8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555ce0d91050 0x555ce0d951f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.618+0000 7f320eae8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91970 0x555ce0d95730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.618+0000 7f320eae8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce0c77560 con 0x555ce0d91970 2024-05-08T22:10:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.618+0000 7f320eae8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce0c77200 con 0x555ce0d91050 2024-05-08T22:10:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.618+0000 7f320eae8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce0b5e6a0 con 0x555ce0d8b890 2024-05-08T22:10:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.618+0000 7f320eae8b80 10 client.0 _ll_register_callbacks cb 0x555ce0c80b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:10:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.618+0000 7f31f3400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555ce0d91050 0x555ce0d951f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.618+0000 7f31f3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ce0d8b890 0x555ce0d94cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.618+0000 7f31f3400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555ce0d91050 0x555ce0d951f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:60046/0 (socket says 172.21.15.131:60046) 2024-05-08T22:10:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.618+0000 7f31f3400700 1 -- 172.21.15.131:0/3596393597 learned_addr learned my addr 172.21.15.131:0/3596393597 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.619+0000 7f31f3e00700 1 --2- 172.21.15.131:0/3596393597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91970 0x555ce0d95730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.619+0000 7f31f3400700 1 -- 172.21.15.131:0/3596393597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ce0d8b890 msgr2=0x555ce0d94cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.619+0000 7f31f3400700 1 --2- 172.21.15.131:0/3596393597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ce0d8b890 0x555ce0d94cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.619+0000 7f31f3400700 1 -- 172.21.15.131:0/3596393597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91970 msgr2=0x555ce0d95730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.619+0000 7f31f3400700 1 --2- 172.21.15.131:0/3596393597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91970 0x555ce0d95730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.619+0000 7f31f3400700 1 -- 172.21.15.131:0/3596393597 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f411d280 con 0x555ce0d91050 2024-05-08T22:10:30.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.619+0000 7f31f3e00700 1 --2- 172.21.15.131:0/3596393597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91970 0x555ce0d95730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:10:30.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.620+0000 7f31f3400700 1 --2- 172.21.15.131:0/3596393597 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555ce0d91050 0x555ce0d951f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f31e8007f20 tx=0x7f31e8012730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a10f0c0fc1881d60 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:30.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.620+0000 7f31da000700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:10:30.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:30 smithi131 ceph-mon[37260]: pgmap v99: 129 pgs: 129 active+clean; 322 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.5 MiB/s wr, 593 op/s 2024-05-08T22:10:30.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.636+0000 7f31da000700 1 -- 172.21.15.131:0/3596393597 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e8004460 con 0x555ce0d91050 2024-05-08T22:10:30.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.636+0000 7f31da000700 1 -- 172.21.15.131:0/3596393597 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f31e8003680 con 0x555ce0d91050 2024-05-08T22:10:30.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.656+0000 7f320eae8b80 1 -- 172.21.15.131:0/3596393597 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ce0c34270 con 0x555ce0d91050 2024-05-08T22:10:30.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.666+0000 7f31da000700 1 -- 172.21.15.131:0/3596393597 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31e8003840 con 0x555ce0d91050 2024-05-08T22:10:30.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.670+0000 7f320eae8b80 10 client.26096 fetch_fsmap learned FSMap version 25 2024-05-08T22:10:30.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.670+0000 7f320eae8b80 1 -- 172.21.15.131:0/3596393597 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555ce0b08b60 con 0x555ce0d91050 2024-05-08T22:10:30.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.674+0000 7f31da000700 1 -- 172.21.15.131:0/3596393597 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555ce0c34270 con 0x555ce0d91050 2024-05-08T22:10:30.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.674+0000 7f31da000700 1 -- 172.21.15.131:0/3596393597 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f31e801bbe0 con 0x555ce0d91050 2024-05-08T22:10:30.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.675+0000 7f320eae8b80 10 client.26096 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T22:10:30.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.675+0000 7f320eae8b80 10 client.26096 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:10:30.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.675+0000 7f320eae8b80 20 client.26096 populate_metadata read hostname 'smithi131' 2024-05-08T22:10:30.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.675+0000 7f320eae8b80 1 --2- 172.21.15.131:0/3596393597 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x555ce0da52a0 0x555ce0da76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:30.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.675+0000 7f320eae8b80 4 client.26096 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:10:30.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.675+0000 7f320eae8b80 1 -- 172.21.15.131:0/3596393597 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ce0d5a810 con 0x555ce0da52a0 2024-05-08T22:10:30.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.679+0000 7f31f3e00700 1 --2- 172.21.15.131:0/3596393597 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x555ce0da52a0 0x555ce0da76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.680+0000 7f31f3e00700 1 --2- 172.21.15.131:0/3596393597 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x555ce0da52a0 0x555ce0da76c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d33e501072feff71 server_cookie=a48428662ee2956d in_seq=0 out_seq=0 2024-05-08T22:10:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.680+0000 7f31da000700 10 client.26096 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:10:30.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.688+0000 7f31da000700 1 -- 172.21.15.131:0/3596393597 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555ce0d5a810 con 0x555ce0da52a0 2024-05-08T22:10:30.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.688+0000 7f31da000700 10 client.26096 handle_command_reply: tid=0 2024-05-08T22:10:30.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.688+0000 7f320eae8b80 1 client.26096 shutdown 2024-05-08T22:10:30.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.696+0000 7f320eae8b80 1 -- 172.21.15.131:0/3596393597 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555ce0d91050 msgr2=0x555ce0d951f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:30.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.696+0000 7f320eae8b80 1 --2- 172.21.15.131:0/3596393597 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555ce0d91050 0x555ce0d951f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f31e8007f20 tx=0x7f31e8012730 comp rx=0 tx=0).stop 2024-05-08T22:10:30.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.696+0000 7f320eae8b80 1 -- 172.21.15.131:0/3596393597 shutdown_connections 2024-05-08T22:10:30.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.696+0000 7f320eae8b80 1 --2- 172.21.15.131:0/3596393597 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x555ce0da52a0 0x555ce0da76c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.696+0000 7f320eae8b80 1 --2- 172.21.15.131:0/3596393597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91970 0x555ce0d95730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.696+0000 7f320eae8b80 1 --2- 172.21.15.131:0/3596393597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ce0d8b890 0x555ce0d94cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.696+0000 7f320eae8b80 1 --2- 172.21.15.131:0/3596393597 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555ce0d91050 0x555ce0d951f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.696+0000 7f320eae8b80 1 -- 172.21.15.131:0/3596393597 >> 172.21.15.131:0/3596393597 conn(0x555ce0ce9210 msgr2=0x555ce0d860b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:30.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.697+0000 7f31da000700 0 client.26096 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:10:30.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.697+0000 7f320eae8b80 1 -- 172.21.15.131:0/3596393597 shutdown_connections 2024-05-08T22:10:30.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.697+0000 7f320eae8b80 1 -- 172.21.15.131:0/3596393597 wait complete. 2024-05-08T22:10:30.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.698+0000 7f320eae8b80 20 client.26096 trim_cache size 0 max 16384 2024-05-08T22:10:30.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.818+0000 7f320eae8b80 1 Processor -- start 2024-05-08T22:10:30.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.831+0000 7f320eae8b80 1 -- start start 2024-05-08T22:10:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.832+0000 7f320eae8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ce0d8b890 0x555ce0d8b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.832+0000 7f320eae8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91050 0x555ce0d9c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.832+0000 7f320eae8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555ce0d91970 0x555ce0d9c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.832+0000 7f320eae8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce0d80a90 con 0x555ce0d91050 2024-05-08T22:10:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.832+0000 7f320eae8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce0d962b0 con 0x555ce0d91970 2024-05-08T22:10:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.832+0000 7f320eae8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce0d82df0 con 0x555ce0d8b890 2024-05-08T22:10:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.833+0000 7f31f3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91050 0x555ce0d9c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.833+0000 7f31f3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91050 0x555ce0d9c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55150/0 (socket says 172.21.15.131:55150) 2024-05-08T22:10:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.833+0000 7f31f3400700 1 -- 172.21.15.131:0/670234437 learned_addr learned my addr 172.21.15.131:0/670234437 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.834+0000 7f31f3400700 1 -- 172.21.15.131:0/670234437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ce0d8b890 msgr2=0x555ce0d8b350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:10:30.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.834+0000 7f31f3400700 1 --2- 172.21.15.131:0/670234437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ce0d8b890 0x555ce0d8b350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.834+0000 7f31f3400700 1 -- 172.21.15.131:0/670234437 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555ce0d91970 msgr2=0x555ce0d9c850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:10:30.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.834+0000 7f31f3400700 1 --2- 172.21.15.131:0/670234437 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555ce0d91970 0x555ce0d9c850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.834+0000 7f31f3400700 1 -- 172.21.15.131:0/670234437 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ce0ca1560 con 0x555ce0d91050 2024-05-08T22:10:30.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.834+0000 7f31f3400700 1 --2- 172.21.15.131:0/670234437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91050 0x555ce0d9c310 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f31e8015040 tx=0x7f31e8005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c2f4d925a89541 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:30.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.842+0000 7f31e6000700 1 -- 172.21.15.131:0/670234437 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e8025630 con 0x555ce0d91050 2024-05-08T22:10:30.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.842+0000 7f31e6000700 1 -- 172.21.15.131:0/670234437 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31e800f3c0 con 0x555ce0d91050 2024-05-08T22:10:30.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.842+0000 7f31e6000700 1 -- 172.21.15.131:0/670234437 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e800f810 con 0x555ce0d91050 2024-05-08T22:10:30.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.852+0000 7f320eae8b80 1 -- 172.21.15.131:0/670234437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91050 msgr2=0x555ce0d9c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:30.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.852+0000 7f320eae8b80 1 --2- 172.21.15.131:0/670234437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91050 0x555ce0d9c310 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f31e8015040 tx=0x7f31e8005f60 comp rx=0 tx=0).stop 2024-05-08T22:10:30.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.862+0000 7f320eae8b80 1 -- 172.21.15.131:0/670234437 shutdown_connections 2024-05-08T22:10:30.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.862+0000 7f320eae8b80 1 --2- 172.21.15.131:0/670234437 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555ce0d91970 0x555ce0d9c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.862+0000 7f320eae8b80 1 --2- 172.21.15.131:0/670234437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ce0d8b890 0x555ce0d8b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.862+0000 7f320eae8b80 1 --2- 172.21.15.131:0/670234437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91050 0x555ce0d9c310 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.862+0000 7f320eae8b80 1 -- 172.21.15.131:0/670234437 >> 172.21.15.131:0/670234437 conn(0x555ce0ce9210 msgr2=0x555ce0d8bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:30.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.866+0000 7f320eae8b80 1 -- 172.21.15.131:0/670234437 shutdown_connections 2024-05-08T22:10:30.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.872+0000 7f320eae8b80 1 -- 172.21.15.131:0/670234437 wait complete. 2024-05-08T22:10:30.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.873+0000 7f320eae8b80 1 -- start start 2024-05-08T22:10:30.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.890+0000 7f320eae8b80 1 Processor -- start 2024-05-08T22:10:30.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.895+0000 7f320eae8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ce0d8b890 0x555ce0d984e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:30.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.895+0000 7f320eae8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91050 0x555ce0d98a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:30.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.896+0000 7f320eae8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555ce0d91970 0x555ce0dfa790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:30.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.896+0000 7f320eae8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce0d99130 con 0x555ce0d91050 2024-05-08T22:10:30.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.896+0000 7f320eae8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce0dfacd0 con 0x555ce0d91970 2024-05-08T22:10:30.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.896+0000 7f320eae8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce0dfae40 con 0x555ce0d8b890 2024-05-08T22:10:30.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.896+0000 7f320eae8b80 10 client.0 _ll_register_callbacks cb 0x555ce0c80b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:10:30.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.897+0000 7f31f3e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555ce0d91970 0x555ce0dfa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:30.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.897+0000 7f31f3e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555ce0d91970 0x555ce0dfa790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:60056/0 (socket says 172.21.15.131:60056) 2024-05-08T22:10:30.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.897+0000 7f31f3e00700 1 -- 172.21.15.131:0/2538239275 learned_addr learned my addr 172.21.15.131:0/2538239275 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:30.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.897+0000 7f31f3e00700 1 --2- 172.21.15.131:0/2538239275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ce0d8b890 0x555ce0d984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:30.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.897+0000 7f31f3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91050 0x555ce0d98a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:30.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.905+0000 7f31f3400700 1 -- 172.21.15.131:0/2538239275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ce0d8b890 msgr2=0x555ce0d984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:30.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.905+0000 7f31f3400700 1 --2- 172.21.15.131:0/2538239275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ce0d8b890 0x555ce0d984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.905+0000 7f31f3400700 1 -- 172.21.15.131:0/2538239275 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555ce0d91970 msgr2=0x555ce0dfa790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:30.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.905+0000 7f31f3400700 1 --2- 172.21.15.131:0/2538239275 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555ce0d91970 0x555ce0dfa790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.905+0000 7f31f3400700 1 -- 172.21.15.131:0/2538239275 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ce0b08b60 con 0x555ce0d91050 2024-05-08T22:10:30.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.905+0000 7f31f3400700 1 --2- 172.21.15.131:0/2538239275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91050 0x555ce0d98a20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x555ce0d92e10 tx=0x7f31e8005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74eda92e6a477320 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:30.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.915+0000 7f31da000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:10:30.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.915+0000 7f31da000700 1 -- 172.21.15.131:0/2538239275 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e80054d0 con 0x555ce0d91050 2024-05-08T22:10:30.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.915+0000 7f31da000700 1 -- 172.21.15.131:0/2538239275 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f31e8005630 con 0x555ce0d91050 2024-05-08T22:10:30.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.922+0000 7f320eae8b80 1 -- 172.21.15.131:0/2538239275 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f31e001e040 con 0x555ce0d91050 2024-05-08T22:10:30.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.929+0000 7f31da000700 1 -- 172.21.15.131:0/2538239275 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31e8003680 con 0x555ce0d91050 2024-05-08T22:10:30.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.929+0000 7f320eae8b80 10 client.16041 fetch_fsmap learned FSMap version 25 2024-05-08T22:10:30.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.929+0000 7f320eae8b80 1 -- 172.21.15.131:0/2538239275 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555ce0b60cd0 con 0x555ce0d91050 2024-05-08T22:10:30.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.933+0000 7f31da000700 1 -- 172.21.15.131:0/2538239275 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31e001e040 con 0x555ce0d91050 2024-05-08T22:10:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.933+0000 7f31da000700 1 -- 172.21.15.131:0/2538239275 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f31e8050050 con 0x555ce0d91050 2024-05-08T22:10:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.933+0000 7f320eae8b80 10 client.16041 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T22:10:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.933+0000 7f320eae8b80 10 client.16041 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:10:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.933+0000 7f320eae8b80 20 client.16041 populate_metadata read hostname 'smithi131' 2024-05-08T22:10:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.933+0000 7f320eae8b80 1 --2- 172.21.15.131:0/2538239275 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x555ce0dfc110 0x555ce0dfe500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.933+0000 7f31f3e00700 1 --2- 172.21.15.131:0/2538239275 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x555ce0dfc110 0x555ce0dfe500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.934+0000 7f31f3e00700 1 --2- 172.21.15.131:0/2538239275 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x555ce0dfc110 0x555ce0dfe500 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ce875f38f8fbce7 server_cookie=5f42ab41a6e9af64 in_seq=0 out_seq=0 2024-05-08T22:10:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.934+0000 7f320eae8b80 4 client.16041 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:10:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.934+0000 7f320eae8b80 1 -- 172.21.15.131:0/2538239275 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555ce0cd3ec0 con 0x555ce0dfc110 2024-05-08T22:10:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.937+0000 7f31da000700 10 client.16041 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:10:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.937+0000 7f31da000700 1 -- 172.21.15.131:0/2538239275 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555ce0cd3ec0 con 0x555ce0dfc110 2024-05-08T22:10:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.937+0000 7f31da000700 10 client.16041 handle_command_reply: tid=0 2024-05-08T22:10:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.937+0000 7f320eae8b80 1 client.16041 shutdown 2024-05-08T22:10:30.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.937+0000 7f320eae8b80 1 -- 172.21.15.131:0/2538239275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91050 msgr2=0x555ce0d98a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:30.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.937+0000 7f320eae8b80 1 --2- 172.21.15.131:0/2538239275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91050 0x555ce0d98a20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x555ce0d92e10 tx=0x7f31e8005960 comp rx=0 tx=0).stop 2024-05-08T22:10:30.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.938+0000 7f320eae8b80 1 -- 172.21.15.131:0/2538239275 shutdown_connections 2024-05-08T22:10:30.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.938+0000 7f320eae8b80 1 --2- 172.21.15.131:0/2538239275 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x555ce0dfc110 0x555ce0dfe500 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.938+0000 7f320eae8b80 1 --2- 172.21.15.131:0/2538239275 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555ce0d91970 0x555ce0dfa790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.938+0000 7f320eae8b80 1 --2- 172.21.15.131:0/2538239275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ce0d8b890 0x555ce0d984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.938+0000 7f320eae8b80 1 --2- 172.21.15.131:0/2538239275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ce0d91050 0x555ce0d98a20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.938+0000 7f320eae8b80 1 -- 172.21.15.131:0/2538239275 >> 172.21.15.131:0/2538239275 conn(0x555ce0ce9210 msgr2=0x555ce0d8ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:30.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.938+0000 7f31da000700 0 client.16041 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:10:30.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.939+0000 7f320eae8b80 1 -- 172.21.15.131:0/2538239275 shutdown_connections 2024-05-08T22:10:30.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.939+0000 7f320eae8b80 1 -- 172.21.15.131:0/2538239275 wait complete. 2024-05-08T22:10:30.942 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:10:30.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.940+0000 7f320eae8b80 20 client.16041 trim_cache size 0 max 16384 2024-05-08T22:10:30.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.944+0000 7f31fb000700 1 -- 172.21.15.131:0/1026111267 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f31dc0a1410 msgr2=0x7f31dc0a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:30.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.944+0000 7f31fb000700 1 --2- 172.21.15.131:0/1026111267 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f31dc0a1410 0x7f31dc0a38d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f31f41aaea0 tx=0x7f31ec095040 comp rx=0 tx=0).stop 2024-05-08T22:10:30.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.944+0000 7f31fb000700 1 -- 172.21.15.131:0/1026111267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31f410d080 msgr2=0x7f31f41a5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:30.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.944+0000 7f31fb000700 1 --2- 172.21.15.131:0/1026111267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31f410d080 0x7f31f41a5e40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f31ec001440 tx=0x7f31ec007580 comp rx=0 tx=0).stop 2024-05-08T22:10:30.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.945+0000 7f31fb000700 1 -- 172.21.15.131:0/1026111267 shutdown_connections 2024-05-08T22:10:30.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.945+0000 7f31fb000700 1 --2- 172.21.15.131:0/1026111267 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f31dc0a1410 0x7f31dc0a38d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.945+0000 7f31fb000700 1 --2- 172.21.15.131:0/1026111267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31f411a430 0x7f31f41aa720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.945+0000 7f31fb000700 1 --2- 172.21.15.131:0/1026111267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31f410d080 0x7f31f41a5e40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.945+0000 7f31fb000700 1 --2- 172.21.15.131:0/1026111267 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31f410d9a0 0x7f31f41a6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:30.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.945+0000 7f31fb000700 1 -- 172.21.15.131:0/1026111267 >> 172.21.15.131:0/1026111267 conn(0x7f31f406be10 msgr2=0x7f31f40719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:30.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.954+0000 7f31fb000700 1 -- 172.21.15.131:0/1026111267 shutdown_connections 2024-05-08T22:10:30.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:30.954+0000 7f31fb000700 1 -- 172.21.15.131:0/1026111267 wait complete. 2024-05-08T22:10:31.264 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T22:10:32.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:32 smithi131 ceph-mon[37260]: pgmap v100: 129 pgs: 129 active+clean; 326 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.1 MiB/s wr, 510 op/s 2024-05-08T22:10:32.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:32 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:10:32.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:32 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:10:32.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:32 smithi157 ceph-mon[77746]: pgmap v100: 129 pgs: 129 active+clean; 326 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.1 MiB/s wr, 510 op/s 2024-05-08T22:10:32.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:32 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:10:32.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:32 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:10:32.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:32 smithi132 ceph-mon[78180]: pgmap v100: 129 pgs: 129 active+clean; 326 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.1 MiB/s wr, 510 op/s 2024-05-08T22:10:32.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:32 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:10:32.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:32 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:10:34.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:34 smithi132 ceph-mon[78180]: pgmap v101: 129 pgs: 129 active+clean; 326 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 456 op/s 2024-05-08T22:10:34.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:34 smithi132 ceph-mon[78180]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T22:10:34.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:34 smithi131 ceph-mon[37260]: pgmap v101: 129 pgs: 129 active+clean; 326 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 456 op/s 2024-05-08T22:10:34.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:34 smithi131 ceph-mon[37260]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T22:10:34.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:34 smithi157 ceph-mon[77746]: pgmap v101: 129 pgs: 129 active+clean; 326 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 456 op/s 2024-05-08T22:10:34.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:34 smithi157 ceph-mon[77746]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T22:10:34.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.808+0000 7f50ed600700 1 -- 172.21.15.131:0/1783173159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50e810fb10 msgr2=0x7f50e81084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:34.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.808+0000 7f50ed600700 1 --2- 172.21.15.131:0/1783173159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50e810fb10 0x7f50e81084e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f50d8009940 tx=0x7f50d8009c50 comp rx=0 tx=0).stop 2024-05-08T22:10:34.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.824+0000 7f50ed600700 1 -- 172.21.15.131:0/1783173159 shutdown_connections 2024-05-08T22:10:34.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.824+0000 7f50ed600700 1 --2- 172.21.15.131:0/1783173159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e811a780 0x7f50e811cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:34.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.824+0000 7f50ed600700 1 --2- 172.21.15.131:0/1783173159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50e810f6a0 0x7f50e8107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:34.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.824+0000 7f50ed600700 1 --2- 172.21.15.131:0/1783173159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50e810fb10 0x7f50e81084e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:34.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.824+0000 7f50ed600700 1 -- 172.21.15.131:0/1783173159 >> 172.21.15.131:0/1783173159 conn(0x7f50e806bff0 msgr2=0x7f50e806c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:34.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.835+0000 7f50ed600700 1 -- 172.21.15.131:0/1783173159 shutdown_connections 2024-05-08T22:10:34.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.837+0000 7f50ed600700 1 -- 172.21.15.131:0/1783173159 wait complete. 2024-05-08T22:10:34.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.849+0000 7f50ed600700 1 Processor -- start 2024-05-08T22:10:34.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.857+0000 7f50ed600700 1 -- start start 2024-05-08T22:10:34.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.860+0000 7f50ed600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50e810f6a0 0x7f50e81a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:34.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.860+0000 7f50ed600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e810fb10 0x7f50e81a6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:34.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.860+0000 7f50e6a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50e810f6a0 0x7f50e81a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:34.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.860+0000 7f50e6a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50e810f6a0 0x7f50e81a6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:60700/0 (socket says 172.21.15.131:60700) 2024-05-08T22:10:34.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.861+0000 7f50e6a00700 1 -- 172.21.15.131:0/1674343597 learned_addr learned my addr 172.21.15.131:0/1674343597 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:34.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.861+0000 7f50e6000700 1 --2- 172.21.15.131:0/1674343597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e810fb10 0x7f50e81a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:34.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.863+0000 7f50ed600700 1 --2- 172.21.15.131:0/1674343597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50e811a780 0x7f50e81aab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:34.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.863+0000 7f50ed600700 1 -- 172.21.15.131:0/1674343597 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e81ab160 con 0x7f50e811a780 2024-05-08T22:10:34.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.863+0000 7f50ed600700 1 -- 172.21.15.131:0/1674343597 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e81ab2d0 con 0x7f50e810f6a0 2024-05-08T22:10:34.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.863+0000 7f50ed600700 1 -- 172.21.15.131:0/1674343597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e81ab490 con 0x7f50e810fb10 2024-05-08T22:10:34.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.863+0000 7f50e7400700 1 --2- 172.21.15.131:0/1674343597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50e811a780 0x7f50e81aab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:34.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.864+0000 7f50e6000700 1 -- 172.21.15.131:0/1674343597 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50e810f6a0 msgr2=0x7f50e81a6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:34.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.864+0000 7f50e6000700 1 --2- 172.21.15.131:0/1674343597 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50e810f6a0 0x7f50e81a6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:34.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.864+0000 7f50e6000700 1 -- 172.21.15.131:0/1674343597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50e811a780 msgr2=0x7f50e81aab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:34.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.864+0000 7f50e6000700 1 --2- 172.21.15.131:0/1674343597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50e811a780 0x7f50e81aab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:34.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.865+0000 7f50e6000700 1 -- 172.21.15.131:0/1674343597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d8009520 con 0x7f50e810fb10 2024-05-08T22:10:34.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.866+0000 7f50e7400700 1 --2- 172.21.15.131:0/1674343597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50e811a780 0x7f50e81aab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:10:34.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.870+0000 7f50e6000700 1 --2- 172.21.15.131:0/1674343597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e810fb10 0x7f50e81a6780 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f50d8009860 tx=0x7f50d80055e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:34.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.871+0000 7f50df400700 1 -- 172.21.15.131:0/1674343597 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d800bbe0 con 0x7f50e810fb10 2024-05-08T22:10:34.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.871+0000 7f50ed600700 1 -- 172.21.15.131:0/1674343597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e81ab710 con 0x7f50e810fb10 2024-05-08T22:10:34.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.871+0000 7f50ed600700 1 -- 172.21.15.131:0/1674343597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e8110140 con 0x7f50e810fb10 2024-05-08T22:10:34.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.876+0000 7f50df400700 1 -- 172.21.15.131:0/1674343597 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50d800bd40 con 0x7f50e810fb10 2024-05-08T22:10:34.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.876+0000 7f50df400700 1 -- 172.21.15.131:0/1674343597 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d800bbe0 con 0x7f50e810fb10 2024-05-08T22:10:34.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.933+0000 7f50df400700 1 -- 172.21.15.131:0/1674343597 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f50d8003680 con 0x7f50e810fb10 2024-05-08T22:10:34.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.935+0000 7f50df400700 1 --2- 172.21.15.131:0/1674343597 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f50cc0a13a0 0x7f50cc0a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:34.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.936+0000 7f50df400700 1 -- 172.21.15.131:0/1674343597 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f50d80df000 con 0x7f50e810fb10 2024-05-08T22:10:34.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.940+0000 7f50e6a00700 1 --2- 172.21.15.131:0/1674343597 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f50cc0a13a0 0x7f50cc0a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:34.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.944+0000 7f51010bfb80 1 Processor -- start 2024-05-08T22:10:34.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.946+0000 7f50e6a00700 1 --2- 172.21.15.131:0/1674343597 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f50cc0a13a0 0x7f50cc0a3860 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f50d400bd40 tx=0x7f50d400b2c0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:34.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.977+0000 7f51010bfb80 1 -- start start 2024-05-08T22:10:34.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.977+0000 7f51010bfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c8cd2fe00 0x560c8cd301e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:34.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.977+0000 7f51010bfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8cd30720 0x560c8cd2a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:34.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.977+0000 7f51010bfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c8cd2a890 0x560c8cd2acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:34.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.977+0000 7f51010bfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8cc81150 con 0x560c8cd2fe00 2024-05-08T22:10:34.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.977+0000 7f51010bfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8cc7eb10 con 0x560c8cd2a890 2024-05-08T22:10:34.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.977+0000 7f51010bfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8cc7a400 con 0x560c8cd30720 2024-05-08T22:10:34.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.985+0000 7f50e6a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c8cd2fe00 0x560c8cd301e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:34.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.985+0000 7f50e6a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c8cd2fe00 0x560c8cd301e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51856/0 (socket says 172.21.15.131:51856) 2024-05-08T22:10:34.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.985+0000 7f50e6a00700 1 -- 172.21.15.131:0/1137657104 learned_addr learned my addr 172.21.15.131:0/1137657104 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:34.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.986+0000 7f50e6a00700 1 -- 172.21.15.131:0/1137657104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8cd30720 msgr2=0x560c8cd2a350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:10:34.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.986+0000 7f50e6a00700 1 --2- 172.21.15.131:0/1137657104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8cd30720 0x560c8cd2a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:34.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.986+0000 7f50e6a00700 1 -- 172.21.15.131:0/1137657104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c8cd2a890 msgr2=0x560c8cd2acd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:10:34.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.986+0000 7f50e6a00700 1 --2- 172.21.15.131:0/1137657104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c8cd2a890 0x560c8cd2acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:34.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.986+0000 7f50e6a00700 1 -- 172.21.15.131:0/1137657104 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c8cafb3b0 con 0x560c8cd2fe00 2024-05-08T22:10:34.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.987+0000 7f50e6a00700 1 --2- 172.21.15.131:0/1137657104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c8cd2fe00 0x560c8cd301e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f50d400ff50 tx=0x7f50d4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14a38261a325f9e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:34.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.993+0000 7f50d3400700 1 -- 172.21.15.131:0/1137657104 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d4011c60 con 0x560c8cd2fe00 2024-05-08T22:10:34.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.993+0000 7f50d3400700 1 -- 172.21.15.131:0/1137657104 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50d4011dc0 con 0x560c8cd2fe00 2024-05-08T22:10:34.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:34.993+0000 7f50d3400700 1 -- 172.21.15.131:0/1137657104 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d401d8f0 con 0x560c8cd2fe00 2024-05-08T22:10:35.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.004+0000 7f51010bfb80 1 -- 172.21.15.131:0/1137657104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c8cd2fe00 msgr2=0x560c8cd301e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:35.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.004+0000 7f51010bfb80 1 --2- 172.21.15.131:0/1137657104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c8cd2fe00 0x560c8cd301e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f50d400ff50 tx=0x7f50d4012000 comp rx=0 tx=0).stop 2024-05-08T22:10:35.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.010+0000 7f51010bfb80 1 -- 172.21.15.131:0/1137657104 shutdown_connections 2024-05-08T22:10:35.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.010+0000 7f51010bfb80 1 --2- 172.21.15.131:0/1137657104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c8cd2a890 0x560c8cd2acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.010+0000 7f51010bfb80 1 --2- 172.21.15.131:0/1137657104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c8cd2fe00 0x560c8cd301e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.010+0000 7f51010bfb80 1 --2- 172.21.15.131:0/1137657104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8cd30720 0x560c8cd2a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.010+0000 7f51010bfb80 1 -- 172.21.15.131:0/1137657104 >> 172.21.15.131:0/1137657104 conn(0x560c8cc88210 msgr2=0x560c8cd24df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:35.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.012+0000 7f51010bfb80 1 -- 172.21.15.131:0/1137657104 shutdown_connections 2024-05-08T22:10:35.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.013+0000 7f51010bfb80 1 -- 172.21.15.131:0/1137657104 wait complete. 2024-05-08T22:10:35.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.015+0000 7f51010bfb80 1 -- start start 2024-05-08T22:10:35.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.020+0000 7f51010bfb80 1 Processor -- start 2024-05-08T22:10:35.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.034+0000 7f51010bfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8cd2a890 0x560c8cd33ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:35.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.034+0000 7f51010bfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c8cd2fe00 0x560c8cd34010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:35.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.040+0000 7f51010bfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c8cd30720 0x560c8cd34550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:35.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.040+0000 7f51010bfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8cc16560 con 0x560c8cd2fe00 2024-05-08T22:10:35.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.040+0000 7f51010bfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8cc16200 con 0x560c8cd30720 2024-05-08T22:10:35.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.040+0000 7f51010bfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8cafd6a0 con 0x560c8cd2a890 2024-05-08T22:10:35.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.040+0000 7f51010bfb80 10 client.0 _ll_register_callbacks cb 0x560c8cc1fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:10:35.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.041+0000 7f50e6000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c8cd2fe00 0x560c8cd34010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:35.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.041+0000 7f50e6000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c8cd2fe00 0x560c8cd34010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51858/0 (socket says 172.21.15.131:51858) 2024-05-08T22:10:35.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.041+0000 7f50e6000700 1 -- 172.21.15.131:0/127421390 learned_addr learned my addr 172.21.15.131:0/127421390 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:35.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.041+0000 7f50e6000700 1 -- 172.21.15.131:0/127421390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8cd2a890 msgr2=0x560c8cd33ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:10:35.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.041+0000 7f50e6000700 1 --2- 172.21.15.131:0/127421390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8cd2a890 0x560c8cd33ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.041+0000 7f50e6000700 1 -- 172.21.15.131:0/127421390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c8cd30720 msgr2=0x560c8cd34550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:10:35.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.041+0000 7f50e6000700 1 --2- 172.21.15.131:0/127421390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c8cd30720 0x560c8cd34550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.041+0000 7f50e6000700 1 -- 172.21.15.131:0/127421390 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d400fbe0 con 0x560c8cd2fe00 2024-05-08T22:10:35.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.042+0000 7f50e6000700 1 --2- 172.21.15.131:0/127421390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c8cd2fe00 0x560c8cd34010 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f50d80dc9c0 tx=0x7f50d80ab950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c380a9015d7807 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:35.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.057+0000 7f50c7e00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:10:35.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.057+0000 7f50c7e00700 1 -- 172.21.15.131:0/127421390 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d80ac270 con 0x560c8cd2fe00 2024-05-08T22:10:35.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.057+0000 7f50c7e00700 1 -- 172.21.15.131:0/127421390 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f50d80abe10 con 0x560c8cd2fe00 2024-05-08T22:10:35.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.072+0000 7f51010bfb80 1 -- 172.21.15.131:0/127421390 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c8cbd3270 con 0x560c8cd2fe00 2024-05-08T22:10:35.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.084+0000 7f50c7e00700 1 -- 172.21.15.131:0/127421390 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50d80afc40 con 0x560c8cd2fe00 2024-05-08T22:10:35.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.104+0000 7f51010bfb80 10 client.16050 fetch_fsmap learned FSMap version 26 2024-05-08T22:10:35.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.104+0000 7f51010bfb80 1 -- 172.21.15.131:0/127421390 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560c8caa7b60 con 0x560c8cd2fe00 2024-05-08T22:10:35.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.108+0000 7f50c7e00700 1 -- 172.21.15.131:0/127421390 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560c8cbd3270 con 0x560c8cd2fe00 2024-05-08T22:10:35.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.108+0000 7f50c7e00700 1 -- 172.21.15.131:0/127421390 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f50d80c9f70 con 0x560c8cd2fe00 2024-05-08T22:10:35.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.109+0000 7f51010bfb80 10 client.16050 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:10:35.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.109+0000 7f51010bfb80 10 client.16050 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:10:35.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.109+0000 7f51010bfb80 20 client.16050 populate_metadata read hostname 'smithi131' 2024-05-08T22:10:35.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.109+0000 7f51010bfb80 1 --2- 172.21.15.131:0/127421390 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x560c8cd43d70 0x560c8cd46160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:35.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.109+0000 7f51010bfb80 4 client.16050 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:10:35.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.109+0000 7f51010bfb80 1 -- 172.21.15.131:0/127421390 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c8ccf9810 con 0x560c8cd43d70 2024-05-08T22:10:35.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.110+0000 7f50e6a00700 1 --2- 172.21.15.131:0/127421390 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x560c8cd43d70 0x560c8cd46160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:35.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.111+0000 7f50e6a00700 1 --2- 172.21.15.131:0/127421390 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x560c8cd43d70 0x560c8cd46160 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=758aefa092ca26b5 server_cookie=fb59b25a0e356ff9 in_seq=0 out_seq=0 2024-05-08T22:10:35.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.111+0000 7f50c7e00700 10 client.16050 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:10:35.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.112+0000 7f50c7e00700 1 -- 172.21.15.131:0/127421390 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560c8ccf9810 con 0x560c8cd43d70 2024-05-08T22:10:35.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.112+0000 7f50c7e00700 10 client.16050 handle_command_reply: tid=0 2024-05-08T22:10:35.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.112+0000 7f51010bfb80 1 client.16050 shutdown 2024-05-08T22:10:35.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.113+0000 7f51010bfb80 1 -- 172.21.15.131:0/127421390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c8cd2fe00 msgr2=0x560c8cd34010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:35.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.113+0000 7f51010bfb80 1 --2- 172.21.15.131:0/127421390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c8cd2fe00 0x560c8cd34010 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f50d80dc9c0 tx=0x7f50d80ab950 comp rx=0 tx=0).stop 2024-05-08T22:10:35.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.114+0000 7f51010bfb80 1 -- 172.21.15.131:0/127421390 shutdown_connections 2024-05-08T22:10:35.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.114+0000 7f51010bfb80 1 --2- 172.21.15.131:0/127421390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c8cd30720 0x560c8cd34550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.114+0000 7f51010bfb80 1 --2- 172.21.15.131:0/127421390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8cd2a890 0x560c8cd33ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.114+0000 7f51010bfb80 1 --2- 172.21.15.131:0/127421390 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x560c8cd43d70 0x560c8cd46160 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.114+0000 7f51010bfb80 1 --2- 172.21.15.131:0/127421390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c8cd2fe00 0x560c8cd34010 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.114+0000 7f51010bfb80 1 -- 172.21.15.131:0/127421390 >> 172.21.15.131:0/127421390 conn(0x560c8cc88210 msgr2=0x560c8cd245a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:35.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.114+0000 7f50c7e00700 0 client.16050 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:10:35.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.115+0000 7f51010bfb80 1 -- 172.21.15.131:0/127421390 shutdown_connections 2024-05-08T22:10:35.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.115+0000 7f51010bfb80 1 -- 172.21.15.131:0/127421390 wait complete. 2024-05-08T22:10:35.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.116+0000 7f51010bfb80 20 client.16050 trim_cache size 0 max 16384 2024-05-08T22:10:35.233 INFO:tasks.workunit.client.0.smithi131.stdout: 75 604265 431 387802 440 339742 1310 2024-05-08T22:10:35.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.276+0000 7f51010bfb80 1 Processor -- start 2024-05-08T22:10:35.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.290+0000 7f51010bfb80 1 -- start start 2024-05-08T22:10:35.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.291+0000 7f51010bfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c8cd2a890 0x560c8cd2a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:35.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.291+0000 7f51010bfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c8cd2fe00 0x560c8cd2f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:35.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.291+0000 7f51010bfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8cd30720 0x560c8cd333d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:35.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.292+0000 7f51010bfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8cd33910 con 0x560c8cd2a890 2024-05-08T22:10:35.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.292+0000 7f51010bfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8cd35170 con 0x560c8cd2fe00 2024-05-08T22:10:35.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.292+0000 7f51010bfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8cd4b370 con 0x560c8cd30720 2024-05-08T22:10:35.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.292+0000 7f50e6000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c8cd2fe00 0x560c8cd2f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:35.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.292+0000 7f50e6000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c8cd2fe00 0x560c8cd2f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:60714/0 (socket says 172.21.15.131:60714) 2024-05-08T22:10:35.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.292+0000 7f50e6000700 1 -- 172.21.15.131:0/3484549481 learned_addr learned my addr 172.21.15.131:0/3484549481 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:35.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.292+0000 7f50e7400700 1 --2- 172.21.15.131:0/3484549481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8cd30720 0x560c8cd333d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:35.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.292+0000 7f50e6000700 1 -- 172.21.15.131:0/3484549481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8cd30720 msgr2=0x560c8cd333d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:35.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.292+0000 7f50e6000700 1 --2- 172.21.15.131:0/3484549481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8cd30720 0x560c8cd333d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.292+0000 7f50e6000700 1 -- 172.21.15.131:0/3484549481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c8cd2a890 msgr2=0x560c8cd2a350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:10:35.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.292+0000 7f50e6000700 1 --2- 172.21.15.131:0/3484549481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c8cd2a890 0x560c8cd2a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.292+0000 7f50e6000700 1 -- 172.21.15.131:0/3484549481 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c8cc40560 con 0x560c8cd2fe00 2024-05-08T22:10:35.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.293+0000 7f50e6000700 1 --2- 172.21.15.131:0/3484549481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c8cd2fe00 0x560c8cd2f430 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f50d8000c00 tx=0x7f50d800bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bc7f348994b6eb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:35.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.307+0000 7f50d3e00700 1 -- 172.21.15.131:0/3484549481 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d80afc40 con 0x560c8cd2fe00 2024-05-08T22:10:35.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.307+0000 7f50d3e00700 1 -- 172.21.15.131:0/3484549481 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50d80b4200 con 0x560c8cd2fe00 2024-05-08T22:10:35.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.307+0000 7f50d3e00700 1 -- 172.21.15.131:0/3484549481 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d80ac070 con 0x560c8cd2fe00 2024-05-08T22:10:35.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.314+0000 7f51010bfb80 1 -- 172.21.15.131:0/3484549481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c8cd2fe00 msgr2=0x560c8cd2f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:35.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.314+0000 7f51010bfb80 1 --2- 172.21.15.131:0/3484549481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c8cd2fe00 0x560c8cd2f430 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f50d8000c00 tx=0x7f50d800bf90 comp rx=0 tx=0).stop 2024-05-08T22:10:35.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.352+0000 7f51010bfb80 1 -- 172.21.15.131:0/3484549481 shutdown_connections 2024-05-08T22:10:35.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.352+0000 7f51010bfb80 1 --2- 172.21.15.131:0/3484549481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8cd30720 0x560c8cd333d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.352+0000 7f51010bfb80 1 --2- 172.21.15.131:0/3484549481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c8cd2a890 0x560c8cd2a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.352+0000 7f51010bfb80 1 --2- 172.21.15.131:0/3484549481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c8cd2fe00 0x560c8cd2f430 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.352+0000 7f51010bfb80 1 -- 172.21.15.131:0/3484549481 >> 172.21.15.131:0/3484549481 conn(0x560c8cc88210 msgr2=0x560c8cd30b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:35.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.352+0000 7f51010bfb80 1 -- 172.21.15.131:0/3484549481 shutdown_connections 2024-05-08T22:10:35.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.352+0000 7f51010bfb80 1 -- 172.21.15.131:0/3484549481 wait complete. 2024-05-08T22:10:35.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.353+0000 7f51010bfb80 1 -- start start 2024-05-08T22:10:35.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.353+0000 7f51010bfb80 1 Processor -- start 2024-05-08T22:10:35.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.357+0000 7f51010bfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8cd2a890 0x560c8cd393a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:35.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.357+0000 7f51010bfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c8cd2fe00 0x560c8cd398e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:35.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.357+0000 7f51010bfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c8cd30720 0x560c8cd991c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:35.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.357+0000 7f51010bfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8cd39fc0 con 0x560c8cd30720 2024-05-08T22:10:35.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.357+0000 7f51010bfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8cd99700 con 0x560c8cd2fe00 2024-05-08T22:10:35.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.357+0000 7f51010bfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8cd99840 con 0x560c8cd2a890 2024-05-08T22:10:35.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.357+0000 7f51010bfb80 10 client.0 _ll_register_callbacks cb 0x560c8cc1fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:10:35.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.358+0000 7f50e7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c8cd30720 0x560c8cd991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:35.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.358+0000 7f50e7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c8cd30720 0x560c8cd991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51860/0 (socket says 172.21.15.131:51860) 2024-05-08T22:10:35.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.358+0000 7f50e7400700 1 -- 172.21.15.131:0/2209169794 learned_addr learned my addr 172.21.15.131:0/2209169794 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:35.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.359+0000 7f50e7400700 1 -- 172.21.15.131:0/2209169794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8cd2a890 msgr2=0x560c8cd393a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:10:35.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.359+0000 7f50e7400700 1 --2- 172.21.15.131:0/2209169794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8cd2a890 0x560c8cd393a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.359+0000 7f50e7400700 1 -- 172.21.15.131:0/2209169794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c8cd2fe00 msgr2=0x560c8cd398e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:10:35.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.359+0000 7f50e7400700 1 --2- 172.21.15.131:0/2209169794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c8cd2fe00 0x560c8cd398e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.359+0000 7f50e7400700 1 -- 172.21.15.131:0/2209169794 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d400fbe0 con 0x560c8cd30720 2024-05-08T22:10:35.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.359+0000 7f50e7400700 1 --2- 172.21.15.131:0/2209169794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c8cd30720 0x560c8cd991c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f50e0010a10 tx=0x7f50e0010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d7bd356e301a74 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:35.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.378+0000 7f50c7e00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:10:35.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.378+0000 7f50c7e00700 1 -- 172.21.15.131:0/2209169794 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e000eaa0 con 0x560c8cd30720 2024-05-08T22:10:35.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.378+0000 7f50c7e00700 1 -- 172.21.15.131:0/2209169794 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f50e000ec00 con 0x560c8cd30720 2024-05-08T22:10:35.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.392+0000 7f51010bfb80 1 -- 172.21.15.131:0/2209169794 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50d4011dc0 con 0x560c8cd30720 2024-05-08T22:10:35.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.407+0000 7f50c7e00700 1 -- 172.21.15.131:0/2209169794 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50e000ede0 con 0x560c8cd30720 2024-05-08T22:10:35.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.409+0000 7f51010bfb80 10 client.16053 fetch_fsmap learned FSMap version 26 2024-05-08T22:10:35.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.409+0000 7f51010bfb80 1 -- 172.21.15.131:0/2209169794 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560c8caffcd0 con 0x560c8cd30720 2024-05-08T22:10:35.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.410+0000 7f50c7e00700 1 -- 172.21.15.131:0/2209169794 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50d4011dc0 con 0x560c8cd30720 2024-05-08T22:10:35.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.410+0000 7f50c7e00700 1 -- 172.21.15.131:0/2209169794 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f50e0012040 con 0x560c8cd30720 2024-05-08T22:10:35.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.410+0000 7f51010bfb80 10 client.16053 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:10:35.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.410+0000 7f51010bfb80 10 client.16053 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:10:35.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.410+0000 7f51010bfb80 20 client.16053 populate_metadata read hostname 'smithi131' 2024-05-08T22:10:35.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.410+0000 7f51010bfb80 1 --2- 172.21.15.131:0/2209169794 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x560c8cd9b0a0 0x560c8cd9d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:35.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.410+0000 7f51010bfb80 4 client.16053 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:10:35.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.410+0000 7f51010bfb80 1 -- 172.21.15.131:0/2209169794 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560c8cc72ec0 con 0x560c8cd9b0a0 2024-05-08T22:10:35.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.411+0000 7f50e6a00700 1 --2- 172.21.15.131:0/2209169794 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x560c8cd9b0a0 0x560c8cd9d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:35.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.412+0000 7f50e6a00700 1 --2- 172.21.15.131:0/2209169794 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x560c8cd9b0a0 0x560c8cd9d4c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aaadacc3b59eb746 server_cookie=e310d05d3d7f8cee in_seq=0 out_seq=0 2024-05-08T22:10:35.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.412+0000 7f50c7e00700 10 client.16053 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:10:35.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.418+0000 7f50c7e00700 1 -- 172.21.15.131:0/2209169794 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560c8cc72ec0 con 0x560c8cd9b0a0 2024-05-08T22:10:35.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.418+0000 7f50c7e00700 10 client.16053 handle_command_reply: tid=0 2024-05-08T22:10:35.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.428+0000 7f51010bfb80 1 client.16053 shutdown 2024-05-08T22:10:35.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.428+0000 7f51010bfb80 1 -- 172.21.15.131:0/2209169794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c8cd30720 msgr2=0x560c8cd991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:35.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.428+0000 7f51010bfb80 1 --2- 172.21.15.131:0/2209169794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c8cd30720 0x560c8cd991c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f50e0010a10 tx=0x7f50e0010dd0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.431+0000 7f51010bfb80 1 -- 172.21.15.131:0/2209169794 shutdown_connections 2024-05-08T22:10:35.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.431+0000 7f51010bfb80 1 --2- 172.21.15.131:0/2209169794 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x560c8cd9b0a0 0x560c8cd9d4c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.431+0000 7f51010bfb80 1 --2- 172.21.15.131:0/2209169794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c8cd30720 0x560c8cd991c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.431+0000 7f51010bfb80 1 --2- 172.21.15.131:0/2209169794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c8cd2a890 0x560c8cd393a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.431+0000 7f51010bfb80 1 --2- 172.21.15.131:0/2209169794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c8cd2fe00 0x560c8cd398e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.431+0000 7f51010bfb80 1 -- 172.21.15.131:0/2209169794 >> 172.21.15.131:0/2209169794 conn(0x560c8cc88210 msgr2=0x560c8cd31390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.435+0000 7f50c7e00700 0 client.16053 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:10:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.435+0000 7f51010bfb80 1 -- 172.21.15.131:0/2209169794 shutdown_connections 2024-05-08T22:10:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.436+0000 7f51010bfb80 1 -- 172.21.15.131:0/2209169794 wait complete. 2024-05-08T22:10:35.444 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:10:35.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.440+0000 7f51010bfb80 20 client.16053 trim_cache size 0 max 16384 2024-05-08T22:10:35.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.463+0000 7f50ed600700 1 -- 172.21.15.131:0/1674343597 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f50cc0a13a0 msgr2=0x7f50cc0a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:35.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.463+0000 7f50ed600700 1 --2- 172.21.15.131:0/1674343597 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f50cc0a13a0 0x7f50cc0a3860 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f50d400bd40 tx=0x7f50d400b2c0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.463+0000 7f50ed600700 1 -- 172.21.15.131:0/1674343597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e810fb10 msgr2=0x7f50e81a6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:35.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.463+0000 7f50ed600700 1 --2- 172.21.15.131:0/1674343597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e810fb10 0x7f50e81a6780 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f50d8009860 tx=0x7f50d80055e0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.476+0000 7f50ed600700 1 -- 172.21.15.131:0/1674343597 shutdown_connections 2024-05-08T22:10:35.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.476+0000 7f50ed600700 1 --2- 172.21.15.131:0/1674343597 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f50cc0a13a0 0x7f50cc0a3860 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.476+0000 7f50ed600700 1 --2- 172.21.15.131:0/1674343597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50e811a780 0x7f50e81aab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.476+0000 7f50ed600700 1 --2- 172.21.15.131:0/1674343597 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50e810f6a0 0x7f50e81a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.476+0000 7f50ed600700 1 --2- 172.21.15.131:0/1674343597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e810fb10 0x7f50e81a6780 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:35.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.476+0000 7f50ed600700 1 -- 172.21.15.131:0/1674343597 >> 172.21.15.131:0/1674343597 conn(0x7f50e806bff0 msgr2=0x7f50e8118770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:35.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.490+0000 7f50ed600700 1 -- 172.21.15.131:0/1674343597 shutdown_connections 2024-05-08T22:10:35.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:35.491+0000 7f50ed600700 1 -- 172.21.15.131:0/1674343597 wait complete. 2024-05-08T22:10:35.958 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T22:10:36.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:36 smithi132 ceph-mon[78180]: pgmap v102: 129 pgs: 129 active+clean; 333 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.2 MiB/s wr, 492 op/s 2024-05-08T22:10:36.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:36 smithi131 ceph-mon[37260]: pgmap v102: 129 pgs: 129 active+clean; 333 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.2 MiB/s wr, 492 op/s 2024-05-08T22:10:36.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:36 smithi157 ceph-mon[77746]: pgmap v102: 129 pgs: 129 active+clean; 333 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.2 MiB/s wr, 492 op/s 2024-05-08T22:10:38.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:38 smithi132 ceph-mon[78180]: pgmap v103: 129 pgs: 129 active+clean; 336 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 153 op/s 2024-05-08T22:10:38.654 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:38 smithi131 ceph-mon[37260]: pgmap v103: 129 pgs: 129 active+clean; 336 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 153 op/s 2024-05-08T22:10:38.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:38 smithi157 ceph-mon[77746]: pgmap v103: 129 pgs: 129 active+clean; 336 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 153 op/s 2024-05-08T22:10:40.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.164+0000 7f3076a00700 1 -- 172.21.15.131:0/3715125873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f307010d080 msgr2=0x7f307010d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:40.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.164+0000 7f3076a00700 1 --2- 172.21.15.131:0/3715125873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f307010d080 0x7f307010d460 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3068009f90 tx=0x7f306800b330 comp rx=0 tx=0).stop 2024-05-08T22:10:40.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.190+0000 7f3076a00700 1 -- 172.21.15.131:0/3715125873 shutdown_connections 2024-05-08T22:10:40.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.190+0000 7f3076a00700 1 --2- 172.21.15.131:0/3715125873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f307011a430 0x7f307011c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.190+0000 7f3076a00700 1 --2- 172.21.15.131:0/3715125873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f307010d080 0x7f307010d460 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.190+0000 7f3076a00700 1 --2- 172.21.15.131:0/3715125873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f307010d9a0 0x7f3070119ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.190+0000 7f3076a00700 1 -- 172.21.15.131:0/3715125873 >> 172.21.15.131:0/3715125873 conn(0x7f307006be10 msgr2=0x7f307006c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:40.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.194+0000 7f3076a00700 1 -- 172.21.15.131:0/3715125873 shutdown_connections 2024-05-08T22:10:40.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.194+0000 7f3076a00700 1 -- 172.21.15.131:0/3715125873 wait complete. 2024-05-08T22:10:40.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.196+0000 7f3076a00700 1 Processor -- start 2024-05-08T22:10:40.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.201+0000 7f3076a00700 1 -- start start 2024-05-08T22:10:40.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.202+0000 7f3076a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f307010d080 0x7f30701a5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:40.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.202+0000 7f3076a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f307010d9a0 0x7f30701a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:40.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.202+0000 7f3076a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f307011a430 0x7f30701aa870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:40.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.202+0000 7f3076a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30701aaeb0 con 0x7f307010d9a0 2024-05-08T22:10:40.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.202+0000 7f3076a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30701ab020 con 0x7f307011a430 2024-05-08T22:10:40.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.202+0000 7f3076a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30701ab1e0 con 0x7f307010d080 2024-05-08T22:10:40.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.203+0000 7f3074c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f307010d9a0 0x7f30701a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:40.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.203+0000 7f3074c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f307010d9a0 0x7f30701a64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51878/0 (socket says 172.21.15.131:51878) 2024-05-08T22:10:40.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.203+0000 7f3074c00700 1 -- 172.21.15.131:0/2440065047 learned_addr learned my addr 172.21.15.131:0/2440065047 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:40.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.203+0000 7f3076000700 1 --2- 172.21.15.131:0/2440065047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f307011a430 0x7f30701aa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:40.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.203+0000 7f3075600700 1 --2- 172.21.15.131:0/2440065047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f307010d080 0x7f30701a5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:40.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.204+0000 7f3074c00700 1 -- 172.21.15.131:0/2440065047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f307010d080 msgr2=0x7f30701a5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:40.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.204+0000 7f3074c00700 1 --2- 172.21.15.131:0/2440065047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f307010d080 0x7f30701a5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.204+0000 7f3074c00700 1 -- 172.21.15.131:0/2440065047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f307011a430 msgr2=0x7f30701aa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:40.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.204+0000 7f3074c00700 1 --2- 172.21.15.131:0/2440065047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f307011a430 0x7f30701aa870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.204+0000 7f3074c00700 1 -- 172.21.15.131:0/2440065047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3068009c40 con 0x7f307010d9a0 2024-05-08T22:10:40.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.204+0000 7f3074c00700 1 --2- 172.21.15.131:0/2440065047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f307010d9a0 0x7f30701a64d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f306400e9c0 tx=0x7f306400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:40.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.215+0000 7f306e000700 1 -- 172.21.15.131:0/2440065047 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3064004a30 con 0x7f307010d9a0 2024-05-08T22:10:40.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.215+0000 7f306e000700 1 -- 172.21.15.131:0/2440065047 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f306400cc80 con 0x7f307010d9a0 2024-05-08T22:10:40.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.215+0000 7f306e000700 1 -- 172.21.15.131:0/2440065047 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30640054b0 con 0x7f307010d9a0 2024-05-08T22:10:40.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.226+0000 7f3076a00700 1 -- 172.21.15.131:0/2440065047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30701ab4c0 con 0x7f307010d9a0 2024-05-08T22:10:40.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.226+0000 7f3076a00700 1 -- 172.21.15.131:0/2440065047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30700793e0 con 0x7f307010d9a0 2024-05-08T22:10:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.262+0000 7f306e000700 1 -- 172.21.15.131:0/2440065047 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f3064005610 con 0x7f307010d9a0 2024-05-08T22:10:40.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.275+0000 7f308bbf2b80 1 Processor -- start 2024-05-08T22:10:40.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.278+0000 7f306e000700 1 --2- 172.21.15.131:0/2440065047 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f30580a14a0 0x7f30580a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:40.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.278+0000 7f306e000700 1 -- 172.21.15.131:0/2440065047 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f3064014070 con 0x7f307010d9a0 2024-05-08T22:10:40.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.280+0000 7f3076000700 1 --2- 172.21.15.131:0/2440065047 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f30580a14a0 0x7f30580a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:40.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.281+0000 7f3076000700 1 --2- 172.21.15.131:0/2440065047 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f30580a14a0 0x7f30580a3960 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f305c005950 tx=0x7f305c00b390 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:40.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.295+0000 7f308bbf2b80 1 -- start start 2024-05-08T22:10:40.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.295+0000 7f308bbf2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc30ba050 0x55ddc30ba430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:40.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.295+0000 7f308bbf2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddc30ba970 0x55ddc30b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:40.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.296+0000 7f308bbf2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30b4890 0x55ddc30b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:40.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.296+0000 7f308bbf2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc300b150 con 0x55ddc30b4890 2024-05-08T22:10:40.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.296+0000 7f308bbf2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc3008b10 con 0x55ddc30ba050 2024-05-08T22:10:40.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.296+0000 7f308bbf2b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc3004400 con 0x55ddc30ba970 2024-05-08T22:10:40.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.297+0000 7f3076000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30b4890 0x55ddc30b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:40.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.297+0000 7f3076000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30b4890 0x55ddc30b4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51888/0 (socket says 172.21.15.131:51888) 2024-05-08T22:10:40.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.297+0000 7f3076000700 1 -- 172.21.15.131:0/2571875151 learned_addr learned my addr 172.21.15.131:0/2571875151 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:40.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.298+0000 7f3076000700 1 -- 172.21.15.131:0/2571875151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddc30ba970 msgr2=0x55ddc30b4350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:10:40.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.298+0000 7f3076000700 1 --2- 172.21.15.131:0/2571875151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddc30ba970 0x55ddc30b4350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.298+0000 7f3076000700 1 -- 172.21.15.131:0/2571875151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc30ba050 msgr2=0x55ddc30ba430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:10:40.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.298+0000 7f3076000700 1 --2- 172.21.15.131:0/2571875151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc30ba050 0x55ddc30ba430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.298+0000 7f3076000700 1 -- 172.21.15.131:0/2571875151 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddc2e853b0 con 0x55ddc30b4890 2024-05-08T22:10:40.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.302+0000 7f3076000700 1 --2- 172.21.15.131:0/2571875151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30b4890 0x55ddc30b4cd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f305c003ad0 tx=0x7f305c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1b6f6695f0038f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:40.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.311+0000 7f3062600700 1 -- 172.21.15.131:0/2571875151 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c011ad0 con 0x55ddc30b4890 2024-05-08T22:10:40.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.311+0000 7f3062600700 1 -- 172.21.15.131:0/2571875151 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f305c015070 con 0x55ddc30b4890 2024-05-08T22:10:40.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.311+0000 7f3062600700 1 -- 172.21.15.131:0/2571875151 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c01c3b0 con 0x55ddc30b4890 2024-05-08T22:10:40.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.312+0000 7f308bbf2b80 1 -- 172.21.15.131:0/2571875151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30b4890 msgr2=0x55ddc30b4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:40.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.312+0000 7f308bbf2b80 1 --2- 172.21.15.131:0/2571875151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30b4890 0x55ddc30b4cd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f305c003ad0 tx=0x7f305c013000 comp rx=0 tx=0).stop 2024-05-08T22:10:40.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.335+0000 7f308bbf2b80 1 -- 172.21.15.131:0/2571875151 shutdown_connections 2024-05-08T22:10:40.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.335+0000 7f308bbf2b80 1 --2- 172.21.15.131:0/2571875151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30b4890 0x55ddc30b4cd0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.335+0000 7f308bbf2b80 1 --2- 172.21.15.131:0/2571875151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc30ba050 0x55ddc30ba430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.335+0000 7f308bbf2b80 1 --2- 172.21.15.131:0/2571875151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddc30ba970 0x55ddc30b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.335+0000 7f308bbf2b80 1 -- 172.21.15.131:0/2571875151 >> 172.21.15.131:0/2571875151 conn(0x55ddc3012210 msgr2=0x55ddc30af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:40.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.335+0000 7f308bbf2b80 1 -- 172.21.15.131:0/2571875151 shutdown_connections 2024-05-08T22:10:40.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.335+0000 7f308bbf2b80 1 -- 172.21.15.131:0/2571875151 wait complete. 2024-05-08T22:10:40.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.337+0000 7f308bbf2b80 1 -- start start 2024-05-08T22:10:40.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.338+0000 7f308bbf2b80 1 Processor -- start 2024-05-08T22:10:40.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.345+0000 7f308bbf2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30b4890 0x55ddc30bdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:40.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.345+0000 7f308bbf2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddc30ba050 0x55ddc30be2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:40.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.345+0000 7f3075600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30b4890 0x55ddc30bdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:40.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.345+0000 7f3075600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30b4890 0x55ddc30bdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51902/0 (socket says 172.21.15.131:51902) 2024-05-08T22:10:40.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.346+0000 7f308bbf2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc30ba970 0x55ddc30be7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:40.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.346+0000 7f3075600700 1 -- 172.21.15.131:0/2365970979 learned_addr learned my addr 172.21.15.131:0/2365970979 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:40.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.346+0000 7f3074c00700 1 --2- 172.21.15.131:0/2365970979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddc30ba050 0x55ddc30be2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:40.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.346+0000 7f3076000700 1 --2- 172.21.15.131:0/2365970979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc30ba970 0x55ddc30be7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:40.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.346+0000 7f308bbf2b80 1 -- 172.21.15.131:0/2365970979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc2fa0560 con 0x55ddc30b4890 2024-05-08T22:10:40.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.346+0000 7f308bbf2b80 1 -- 172.21.15.131:0/2365970979 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc2fa0200 con 0x55ddc30ba970 2024-05-08T22:10:40.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.346+0000 7f308bbf2b80 1 -- 172.21.15.131:0/2365970979 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc2e876a0 con 0x55ddc30ba050 2024-05-08T22:10:40.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.347+0000 7f3075600700 1 -- 172.21.15.131:0/2365970979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddc30ba050 msgr2=0x55ddc30be2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:40.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.347+0000 7f3075600700 1 --2- 172.21.15.131:0/2365970979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddc30ba050 0x55ddc30be2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.347+0000 7f3075600700 1 -- 172.21.15.131:0/2365970979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc30ba970 msgr2=0x55ddc30be7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:40.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.347+0000 7f3075600700 1 --2- 172.21.15.131:0/2365970979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc30ba970 0x55ddc30be7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.347+0000 7f3075600700 1 -- 172.21.15.131:0/2365970979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f305c003760 con 0x55ddc30b4890 2024-05-08T22:10:40.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.348+0000 7f3075600700 1 --2- 172.21.15.131:0/2365970979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30b4890 0x55ddc30bdd70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3068009550 tx=0x7f3068007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b164de85e5234cc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:40.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.349+0000 7f308bbf2b80 10 client.0 _ll_register_callbacks cb 0x55ddc2fa9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:10:40.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.349+0000 7f308bbf2b80 1 -- 172.21.15.131:0/2365970979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ddc2f5d270 con 0x55ddc30b4890 2024-05-08T22:10:40.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.364+0000 7f3056a00700 10 client.16065 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:10:40.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.364+0000 7f3056a00700 1 -- 172.21.15.131:0/2365970979 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306800e040 con 0x55ddc30b4890 2024-05-08T22:10:40.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.365+0000 7f3056a00700 1 -- 172.21.15.131:0/2365970979 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f306800baf0 con 0x55ddc30b4890 2024-05-08T22:10:40.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.365+0000 7f3056a00700 1 -- 172.21.15.131:0/2365970979 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f306800bcb0 con 0x55ddc30b4890 2024-05-08T22:10:40.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.396+0000 7f308bbf2b80 10 client.16065 fetch_fsmap learned FSMap version 26 2024-05-08T22:10:40.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.396+0000 7f308bbf2b80 1 -- 172.21.15.131:0/2365970979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=26}) v3 -- 0x55ddc2e31b60 con 0x55ddc30b4890 2024-05-08T22:10:40.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.415+0000 7f3056a00700 1 -- 172.21.15.131:0/2365970979 <== mon.0 v2:172.21.15.131:3300/0 4 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3068019070 con 0x55ddc30b4890 2024-05-08T22:10:40.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.415+0000 7f308bbf2b80 10 client.16065 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:10:40.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.415+0000 7f308bbf2b80 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:10:40.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.415+0000 7f308bbf2b80 20 client.16065 populate_metadata read hostname 'smithi131' 2024-05-08T22:10:40.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.415+0000 7f308bbf2b80 1 --2- 172.21.15.131:0/2365970979 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ddc30ce0a0 0x55ddc30d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:40.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.415+0000 7f308bbf2b80 4 client.16065 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:10:40.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.415+0000 7f308bbf2b80 1 -- 172.21.15.131:0/2365970979 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ddc3083810 con 0x55ddc30ce0a0 2024-05-08T22:10:40.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.421+0000 7f3074c00700 1 --2- 172.21.15.131:0/2365970979 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ddc30ce0a0 0x55ddc30d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:40.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.428+0000 7f3074c00700 1 --2- 172.21.15.131:0/2365970979 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ddc30ce0a0 0x55ddc30d0560 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92f835577f0d9417 server_cookie=45122ef8c54c0ac7 in_seq=0 out_seq=0 2024-05-08T22:10:40.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.428+0000 7f3056a00700 10 client.16065 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:10:40.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.433+0000 7f3056a00700 1 -- 172.21.15.131:0/2365970979 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ddc3083810 con 0x55ddc30ce0a0 2024-05-08T22:10:40.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.433+0000 7f3056a00700 10 client.16065 handle_command_reply: tid=0 2024-05-08T22:10:40.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.436+0000 7f308bbf2b80 1 client.16065 shutdown 2024-05-08T22:10:40.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.448+0000 7f308bbf2b80 1 -- 172.21.15.131:0/2365970979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30b4890 msgr2=0x55ddc30bdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:40.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.448+0000 7f308bbf2b80 1 --2- 172.21.15.131:0/2365970979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30b4890 0x55ddc30bdd70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3068009550 tx=0x7f3068007f70 comp rx=0 tx=0).stop 2024-05-08T22:10:40.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.456+0000 7f308bbf2b80 1 -- 172.21.15.131:0/2365970979 shutdown_connections 2024-05-08T22:10:40.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.456+0000 7f308bbf2b80 1 --2- 172.21.15.131:0/2365970979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc30ba970 0x55ddc30be7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.456+0000 7f308bbf2b80 1 --2- 172.21.15.131:0/2365970979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30b4890 0x55ddc30bdd70 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.456+0000 7f308bbf2b80 1 --2- 172.21.15.131:0/2365970979 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ddc30ce0a0 0x55ddc30d0560 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.456+0000 7f308bbf2b80 1 --2- 172.21.15.131:0/2365970979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddc30ba050 0x55ddc30be2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.456+0000 7f308bbf2b80 1 -- 172.21.15.131:0/2365970979 >> 172.21.15.131:0/2365970979 conn(0x55ddc3012210 msgr2=0x55ddc30af0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:40.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.456+0000 7f3056a00700 0 client.16065 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:10:40.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.478+0000 7f308bbf2b80 1 -- 172.21.15.131:0/2365970979 shutdown_connections 2024-05-08T22:10:40.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.478+0000 7f308bbf2b80 1 -- 172.21.15.131:0/2365970979 wait complete. 2024-05-08T22:10:40.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.479+0000 7f308bbf2b80 20 client.16065 trim_cache size 0 max 16384 2024-05-08T22:10:40.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.550+0000 7f308bbf2b80 1 Processor -- start 2024-05-08T22:10:40.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.554+0000 7f308bbf2b80 1 -- start start 2024-05-08T22:10:40.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.557+0000 7f308bbf2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc30b4890 0x55ddc30b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:40.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.557+0000 7f308bbf2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddc30ba050 0x55ddc30c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:40.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.557+0000 7f308bbf2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30ba970 0x55ddc30be610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:40.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.559+0000 7f3076000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30ba970 0x55ddc30be610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:40.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.559+0000 7f3076000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30ba970 0x55ddc30be610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51914/0 (socket says 172.21.15.131:51914) 2024-05-08T22:10:40.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.560+0000 7f3074c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddc30ba050 0x55ddc30c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:40.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.560+0000 7f3075600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc30b4890 0x55ddc30b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:40.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.560+0000 7f3074c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddc30ba050 0x55ddc30c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40504/0 (socket says 172.21.15.131:40504) 2024-05-08T22:10:40.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.560+0000 7f3074c00700 1 -- 172.21.15.131:0/3965437254 learned_addr learned my addr 172.21.15.131:0/3965437254 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:40.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.560+0000 7f308bbf2b80 1 -- 172.21.15.131:0/3965437254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc30bbdf0 con 0x55ddc30ba970 2024-05-08T22:10:40.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.560+0000 7f308bbf2b80 1 -- 172.21.15.131:0/3965437254 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc30b0820 con 0x55ddc30b4890 2024-05-08T22:10:40.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.560+0000 7f308bbf2b80 1 -- 172.21.15.131:0/3965437254 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc30bf340 con 0x55ddc30ba050 2024-05-08T22:10:40.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.561+0000 7f3076000700 1 -- 172.21.15.131:0/3965437254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddc30ba050 msgr2=0x55ddc30c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:40.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.561+0000 7f3076000700 1 --2- 172.21.15.131:0/3965437254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddc30ba050 0x55ddc30c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.561+0000 7f3076000700 1 -- 172.21.15.131:0/3965437254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc30b4890 msgr2=0x55ddc30b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:40.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.561+0000 7f3076000700 1 --2- 172.21.15.131:0/3965437254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc30b4890 0x55ddc30b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.561+0000 7f3076000700 1 -- 172.21.15.131:0/3965437254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddc30ab4e0 con 0x55ddc30ba970 2024-05-08T22:10:40.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.573+0000 7f3076000700 1 --2- 172.21.15.131:0/3965437254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30ba970 0x55ddc30be610 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f305c011f50 tx=0x7f305c0044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92319811a35c118 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:40.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.573+0000 7f3063000700 1 -- 172.21.15.131:0/3965437254 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c02c040 con 0x55ddc30ba970 2024-05-08T22:10:40.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.574+0000 7f3063000700 1 -- 172.21.15.131:0/3965437254 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f305c01ece0 con 0x55ddc30ba970 2024-05-08T22:10:40.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.574+0000 7f3063000700 1 -- 172.21.15.131:0/3965437254 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c011df0 con 0x55ddc30ba970 2024-05-08T22:10:40.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.580+0000 7f308bbf2b80 1 -- 172.21.15.131:0/3965437254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30ba970 msgr2=0x55ddc30be610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:40.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.580+0000 7f308bbf2b80 1 --2- 172.21.15.131:0/3965437254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30ba970 0x55ddc30be610 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f305c011f50 tx=0x7f305c0044d0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.585+0000 7f308bbf2b80 1 -- 172.21.15.131:0/3965437254 shutdown_connections 2024-05-08T22:10:40.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.585+0000 7f308bbf2b80 1 --2- 172.21.15.131:0/3965437254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30ba970 0x55ddc30be610 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.585+0000 7f308bbf2b80 1 --2- 172.21.15.131:0/3965437254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc30b4890 0x55ddc30b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.585+0000 7f308bbf2b80 1 --2- 172.21.15.131:0/3965437254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddc30ba050 0x55ddc30c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.585+0000 7f308bbf2b80 1 -- 172.21.15.131:0/3965437254 >> 172.21.15.131:0/3965437254 conn(0x55ddc3012210 msgr2=0x55ddc30d4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:40.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:40 smithi132 ceph-mon[78180]: pgmap v104: 129 pgs: 129 active+clean; 348 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 138 op/s 2024-05-08T22:10:40.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.598+0000 7f308bbf2b80 1 -- 172.21.15.131:0/3965437254 shutdown_connections 2024-05-08T22:10:40.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.598+0000 7f308bbf2b80 1 -- 172.21.15.131:0/3965437254 wait complete. 2024-05-08T22:10:40.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.599+0000 7f308bbf2b80 1 -- start start 2024-05-08T22:10:40.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.600+0000 7f308bbf2b80 1 Processor -- start 2024-05-08T22:10:40.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.608+0000 7f308bbf2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc30b4890 0x55ddc30c39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:40.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.608+0000 7f308bbf2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddc30ba050 0x55ddc30c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:40.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.608+0000 7f308bbf2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30ba970 0x55ddc30c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:40.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.610+0000 7f3076000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30ba970 0x55ddc30c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:40.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.610+0000 7f3076000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30ba970 0x55ddc30c4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51916/0 (socket says 172.21.15.131:51916) 2024-05-08T22:10:40.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.610+0000 7f308bbf2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc30c5330 con 0x55ddc30ba970 2024-05-08T22:10:40.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.610+0000 7f308bbf2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc30c54a0 con 0x55ddc30b4890 2024-05-08T22:10:40.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.610+0000 7f308bbf2b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc30c5660 con 0x55ddc30ba050 2024-05-08T22:10:40.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.610+0000 7f308bbf2b80 10 client.0 _ll_register_callbacks cb 0x55ddc2fa9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:10:40.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.611+0000 7f3074c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddc30ba050 0x55ddc30c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:40.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.611+0000 7f3074c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddc30ba050 0x55ddc30c3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40510/0 (socket says 172.21.15.131:40510) 2024-05-08T22:10:40.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.611+0000 7f3074c00700 1 -- 172.21.15.131:0/1473263648 learned_addr learned my addr 172.21.15.131:0/1473263648 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:40.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.611+0000 7f3076000700 1 -- 172.21.15.131:0/1473263648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddc30ba050 msgr2=0x55ddc30c3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:40.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.611+0000 7f3076000700 1 --2- 172.21.15.131:0/1473263648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddc30ba050 0x55ddc30c3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.611+0000 7f3076000700 1 -- 172.21.15.131:0/1473263648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc30b4890 msgr2=0x55ddc30c39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:40.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.612+0000 7f3076000700 1 --2- 172.21.15.131:0/1473263648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc30b4890 0x55ddc30c39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.612+0000 7f3076000700 1 -- 172.21.15.131:0/1473263648 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ddc2e853b0 con 0x55ddc30ba970 2024-05-08T22:10:40.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.612+0000 7f3076000700 1 --2- 172.21.15.131:0/1473263648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30ba970 0x55ddc30c4da0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f305c000c00 tx=0x7f305c003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4567faf645a7d67 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:40.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.625+0000 7f3056a00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:10:40.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.625+0000 7f3056a00700 1 -- 172.21.15.131:0/1473263648 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c02c040 con 0x55ddc30ba970 2024-05-08T22:10:40.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.625+0000 7f3056a00700 1 -- 172.21.15.131:0/1473263648 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f305c017070 con 0x55ddc30ba970 2024-05-08T22:10:40.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:40 smithi131 ceph-mon[37260]: pgmap v104: 129 pgs: 129 active+clean; 348 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 138 op/s 2024-05-08T22:10:40.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.653+0000 7f308bbf2b80 1 -- 172.21.15.131:0/1473263648 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f305c015070 con 0x55ddc30ba970 2024-05-08T22:10:40.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.653+0000 7f3056a00700 1 -- 172.21.15.131:0/1473263648 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f305c01e620 con 0x55ddc30ba970 2024-05-08T22:10:40.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.653+0000 7f308bbf2b80 10 client.16071 fetch_fsmap learned FSMap version 26 2024-05-08T22:10:40.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.653+0000 7f308bbf2b80 1 -- 172.21.15.131:0/1473263648 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55ddc2fca560 con 0x55ddc30ba970 2024-05-08T22:10:40.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.654+0000 7f3056a00700 1 -- 172.21.15.131:0/1473263648 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f305c015070 con 0x55ddc30ba970 2024-05-08T22:10:40.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.657+0000 7f3056a00700 1 -- 172.21.15.131:0/1473263648 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f305c041d00 con 0x55ddc30ba970 2024-05-08T22:10:40.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.658+0000 7f308bbf2b80 10 client.16071 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:10:40.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.658+0000 7f308bbf2b80 10 client.16071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:10:40.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.658+0000 7f308bbf2b80 20 client.16071 populate_metadata read hostname 'smithi131' 2024-05-08T22:10:40.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.658+0000 7f308bbf2b80 1 --2- 172.21.15.131:0/1473263648 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ddc3125430 0x55ddc3127820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:40.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.658+0000 7f308bbf2b80 4 client.16071 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:10:40.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.658+0000 7f308bbf2b80 1 -- 172.21.15.131:0/1473263648 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ddc2ffcec0 con 0x55ddc3125430 2024-05-08T22:10:40.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.659+0000 7f3075600700 1 --2- 172.21.15.131:0/1473263648 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ddc3125430 0x55ddc3127820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:40.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.661+0000 7f3075600700 1 --2- 172.21.15.131:0/1473263648 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ddc3125430 0x55ddc3127820 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36c82cc644c7405f server_cookie=bd3c60d199024201 in_seq=0 out_seq=0 2024-05-08T22:10:40.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.685+0000 7f3056a00700 10 client.16071 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:10:40.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.685+0000 7f3056a00700 1 -- 172.21.15.131:0/1473263648 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ddc2ffcec0 con 0x55ddc3125430 2024-05-08T22:10:40.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.685+0000 7f3056a00700 10 client.16071 handle_command_reply: tid=0 2024-05-08T22:10:40.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.685+0000 7f308bbf2b80 1 client.16071 shutdown 2024-05-08T22:10:40.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.690+0000 7f308bbf2b80 1 -- 172.21.15.131:0/1473263648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30ba970 msgr2=0x55ddc30c4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:40.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.690+0000 7f308bbf2b80 1 --2- 172.21.15.131:0/1473263648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30ba970 0x55ddc30c4da0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f305c000c00 tx=0x7f305c003f60 comp rx=0 tx=0).stop 2024-05-08T22:10:40.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.694+0000 7f308bbf2b80 1 -- 172.21.15.131:0/1473263648 shutdown_connections 2024-05-08T22:10:40.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.694+0000 7f308bbf2b80 1 --2- 172.21.15.131:0/1473263648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ddc30ba970 0x55ddc30c4da0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.694+0000 7f308bbf2b80 1 --2- 172.21.15.131:0/1473263648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc30b4890 0x55ddc30c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.694+0000 7f308bbf2b80 1 --2- 172.21.15.131:0/1473263648 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ddc3125430 0x55ddc3127820 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.694+0000 7f308bbf2b80 1 --2- 172.21.15.131:0/1473263648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ddc30ba050 0x55ddc30c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.694+0000 7f308bbf2b80 1 -- 172.21.15.131:0/1473263648 >> 172.21.15.131:0/1473263648 conn(0x55ddc3012210 msgr2=0x55ddc30b7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:40.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.694+0000 7f3056a00700 0 client.16071 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:10:40.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.694+0000 7f308bbf2b80 1 -- 172.21.15.131:0/1473263648 shutdown_connections 2024-05-08T22:10:40.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.695+0000 7f308bbf2b80 1 -- 172.21.15.131:0/1473263648 wait complete. 2024-05-08T22:10:40.711 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:10:40.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.709+0000 7f308bbf2b80 20 client.16071 trim_cache size 0 max 16384 2024-05-08T22:10:40.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.743+0000 7f3076a00700 1 -- 172.21.15.131:0/2440065047 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f30580a14a0 msgr2=0x7f30580a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:40.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.743+0000 7f3076a00700 1 --2- 172.21.15.131:0/2440065047 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f30580a14a0 0x7f30580a3960 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f305c005950 tx=0x7f305c00b390 comp rx=0 tx=0).stop 2024-05-08T22:10:40.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.743+0000 7f3076a00700 1 -- 172.21.15.131:0/2440065047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f307010d9a0 msgr2=0x7f30701a64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:40.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.743+0000 7f3076a00700 1 --2- 172.21.15.131:0/2440065047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f307010d9a0 0x7f30701a64d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f306400e9c0 tx=0x7f306400ecd0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.754+0000 7f3076a00700 1 -- 172.21.15.131:0/2440065047 shutdown_connections 2024-05-08T22:10:40.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.754+0000 7f3076a00700 1 --2- 172.21.15.131:0/2440065047 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f30580a14a0 0x7f30580a3960 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.754+0000 7f3076a00700 1 --2- 172.21.15.131:0/2440065047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f307011a430 0x7f30701aa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.754+0000 7f3076a00700 1 --2- 172.21.15.131:0/2440065047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f307010d080 0x7f30701a5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.754+0000 7f3076a00700 1 --2- 172.21.15.131:0/2440065047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f307010d9a0 0x7f30701a64d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:40.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.754+0000 7f3076a00700 1 -- 172.21.15.131:0/2440065047 >> 172.21.15.131:0/2440065047 conn(0x7f307006be10 msgr2=0x7f30700718e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:40.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.765+0000 7f3076a00700 1 -- 172.21.15.131:0/2440065047 shutdown_connections 2024-05-08T22:10:40.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:40.767+0000 7f3076a00700 1 -- 172.21.15.131:0/2440065047 wait complete. 2024-05-08T22:10:40.775 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:40 smithi157 ceph-mon[77746]: pgmap v104: 129 pgs: 129 active+clean; 348 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 138 op/s 2024-05-08T22:10:41.951 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T22:10:41.951 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T22:10:42.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:42 smithi132 ceph-mon[78180]: pgmap v105: 129 pgs: 129 active+clean; 348 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 96 op/s 2024-05-08T22:10:42.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:42 smithi131 ceph-mon[37260]: pgmap v105: 129 pgs: 129 active+clean; 348 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 96 op/s 2024-05-08T22:10:42.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:42 smithi157 ceph-mon[77746]: pgmap v105: 129 pgs: 129 active+clean; 348 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 96 op/s 2024-05-08T22:10:44.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:44 smithi132 ceph-mon[78180]: pgmap v106: 129 pgs: 129 active+clean; 348 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 81 op/s 2024-05-08T22:10:44.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:44 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:10:44.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:44 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:10:44.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:44 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:10:44.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:44 smithi131 ceph-mon[37260]: pgmap v106: 129 pgs: 129 active+clean; 348 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 81 op/s 2024-05-08T22:10:44.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:44 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:10:44.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:44 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:10:44.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:44 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:10:44.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:44 smithi157 ceph-mon[77746]: pgmap v106: 129 pgs: 129 active+clean; 348 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 81 op/s 2024-05-08T22:10:44.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:44 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:10:44.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:44 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:10:44.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:44 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:10:45.234 INFO:tasks.workunit.client.0.smithi131.stdout: 82 598204 563 378535 368 338686 1445 2024-05-08T22:10:45.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.291+0000 7f3a58600700 1 -- 172.21.15.131:0/243072854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a5010fb10 msgr2=0x7f3a501084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:45.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.291+0000 7f3a58600700 1 --2- 172.21.15.131:0/243072854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a5010fb10 0x7f3a501084e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0099e0 tx=0x7f3a3c009cf0 comp rx=0 tx=0).stop 2024-05-08T22:10:45.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.291+0000 7f3a58600700 1 -- 172.21.15.131:0/243072854 shutdown_connections 2024-05-08T22:10:45.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.291+0000 7f3a58600700 1 --2- 172.21.15.131:0/243072854 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a5011a780 0x7f3a5011cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:45.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.291+0000 7f3a58600700 1 --2- 172.21.15.131:0/243072854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a5010f6a0 0x7f3a50107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:45.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.291+0000 7f3a58600700 1 --2- 172.21.15.131:0/243072854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a5010fb10 0x7f3a501084e0 secure :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0099e0 tx=0x7f3a3c009cf0 comp rx=0 tx=0).stop 2024-05-08T22:10:45.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.291+0000 7f3a58600700 1 -- 172.21.15.131:0/243072854 >> 172.21.15.131:0/243072854 conn(0x7f3a5006bff0 msgr2=0x7f3a5006c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:45.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.292+0000 7f3a58600700 1 -- 172.21.15.131:0/243072854 shutdown_connections 2024-05-08T22:10:45.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.292+0000 7f3a58600700 1 -- 172.21.15.131:0/243072854 wait complete. 2024-05-08T22:10:45.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.298+0000 7f3a58600700 1 Processor -- start 2024-05-08T22:10:45.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.300+0000 7f3a58600700 1 -- start start 2024-05-08T22:10:45.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.300+0000 7f3a58600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a5010f6a0 0x7f3a501a6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:45.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.300+0000 7f3a58600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a5011a780 0x7f3a501a6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:45.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.301+0000 7f3a58600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a501aaf10 0x7f3a501ab390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:45.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.301+0000 7f3a58600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a501ab990 con 0x7f3a5011a780 2024-05-08T22:10:45.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.301+0000 7f3a58600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a501abb00 con 0x7f3a5010f6a0 2024-05-08T22:10:45.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.301+0000 7f3a58600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a501abcc0 con 0x7f3a501aaf10 2024-05-08T22:10:45.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.301+0000 7f3a55200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a5011a780 0x7f3a501a6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:45.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.302+0000 7f3a55200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a5011a780 0x7f3a501a6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37824/0 (socket says 172.21.15.131:37824) 2024-05-08T22:10:45.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.302+0000 7f3a55200700 1 -- 172.21.15.131:0/2975359340 learned_addr learned my addr 172.21.15.131:0/2975359340 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:45.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.303+0000 7f3a55c00700 1 --2- 172.21.15.131:0/2975359340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a5010f6a0 0x7f3a501a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:45.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.304+0000 7f3a56600700 1 --2- 172.21.15.131:0/2975359340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a501aaf10 0x7f3a501ab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:45.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.308+0000 7f3a55200700 1 -- 172.21.15.131:0/2975359340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a501aaf10 msgr2=0x7f3a501ab390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:45.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.308+0000 7f3a55200700 1 --2- 172.21.15.131:0/2975359340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a501aaf10 0x7f3a501ab390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:45.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.308+0000 7f3a55200700 1 -- 172.21.15.131:0/2975359340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a5010f6a0 msgr2=0x7f3a501a6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:45.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.308+0000 7f3a55200700 1 --2- 172.21.15.131:0/2975359340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a5010f6a0 0x7f3a501a6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:45.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.308+0000 7f3a55200700 1 -- 172.21.15.131:0/2975359340 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c0095f0 con 0x7f3a5011a780 2024-05-08T22:10:45.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.318+0000 7f3a55200700 1 --2- 172.21.15.131:0/2975359340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a5011a780 0x7f3a501a6b70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c006010 tx=0x7f3a3c003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:45.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.321+0000 7f3a4e000700 1 -- 172.21.15.131:0/2975359340 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c03f070 con 0x7f3a5011a780 2024-05-08T22:10:45.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.321+0000 7f3a4e000700 1 -- 172.21.15.131:0/2975359340 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a3c005580 con 0x7f3a5011a780 2024-05-08T22:10:45.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.321+0000 7f3a4e000700 1 -- 172.21.15.131:0/2975359340 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c00fd30 con 0x7f3a5011a780 2024-05-08T22:10:45.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.325+0000 7f3a58600700 1 -- 172.21.15.131:0/2975359340 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a500731d0 con 0x7f3a5011a780 2024-05-08T22:10:45.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.325+0000 7f3a58600700 1 -- 172.21.15.131:0/2975359340 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a50073600 con 0x7f3a5011a780 2024-05-08T22:10:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.352+0000 7f3a4e000700 1 -- 172.21.15.131:0/2975359340 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f3a3c00f460 con 0x7f3a5011a780 2024-05-08T22:10:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.355+0000 7f3a4e000700 1 --2- 172.21.15.131:0/2975359340 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3a380a13d0 0x7f3a380a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.355+0000 7f3a4e000700 1 -- 172.21.15.131:0/2975359340 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f3a3c0f0200 con 0x7f3a5011a780 2024-05-08T22:10:45.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.357+0000 7f3a55c00700 1 --2- 172.21.15.131:0/2975359340 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3a380a13d0 0x7f3a380a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:45.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.360+0000 7f3a55c00700 1 --2- 172.21.15.131:0/2975359340 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3a380a13d0 0x7f3a380a3890 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009910 tx=0x7f3a44008040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:45.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.366+0000 7f3a6c20db80 1 Processor -- start 2024-05-08T22:10:45.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.375+0000 7f3a6c20db80 1 -- start start 2024-05-08T22:10:45.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.375+0000 7f3a6c20db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f41dc1e80 0x555f41dc2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:45.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.375+0000 7f3a6c20db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555f41dc27a0 0x555f41dbc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:45.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.375+0000 7f3a6c20db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41dbc910 0x555f41dbcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:45.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.376+0000 7f3a6c20db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f41d131d0 con 0x555f41dc1e80 2024-05-08T22:10:45.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.376+0000 7f3a6c20db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f41d10b90 con 0x555f41dc27a0 2024-05-08T22:10:45.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.376+0000 7f3a6c20db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f41d0c480 con 0x555f41dbc910 2024-05-08T22:10:45.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.376+0000 7f3a56600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41dbc910 0x555f41dbcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:45.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.376+0000 7f3a56600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41dbc910 0x555f41dbcd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41036/0 (socket says 172.21.15.131:41036) 2024-05-08T22:10:45.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.376+0000 7f3a56600700 1 -- 172.21.15.131:0/4109954165 learned_addr learned my addr 172.21.15.131:0/4109954165 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:45.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.376+0000 7f3a56600700 1 -- 172.21.15.131:0/4109954165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555f41dc27a0 msgr2=0x555f41dbc3d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:10:45.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.376+0000 7f3a56600700 1 --2- 172.21.15.131:0/4109954165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555f41dc27a0 0x555f41dbc3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:45.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.376+0000 7f3a56600700 1 -- 172.21.15.131:0/4109954165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f41dc1e80 msgr2=0x555f41dc2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:45.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.376+0000 7f3a56600700 1 --2- 172.21.15.131:0/4109954165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f41dc1e80 0x555f41dc2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:45.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.376+0000 7f3a56600700 1 -- 172.21.15.131:0/4109954165 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f41b8d430 con 0x555f41dbc910 2024-05-08T22:10:45.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.419+0000 7f3a56600700 1 --2- 172.21.15.131:0/4109954165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41dbc910 0x555f41dbcd50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800b480 tx=0x7f3a48009630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a6dd01ab53d0a1a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:45.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.420+0000 7f3a42a00700 1 -- 172.21.15.131:0/4109954165 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a48010040 con 0x555f41dbc910 2024-05-08T22:10:45.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.420+0000 7f3a42a00700 1 -- 172.21.15.131:0/4109954165 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a4800be50 con 0x555f41dbc910 2024-05-08T22:10:45.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.420+0000 7f3a42a00700 1 -- 172.21.15.131:0/4109954165 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a48014bf0 con 0x555f41dbc910 2024-05-08T22:10:45.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.432+0000 7f3a6c20db80 1 -- 172.21.15.131:0/4109954165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41dbc910 msgr2=0x555f41dbcd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:45.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.432+0000 7f3a6c20db80 1 --2- 172.21.15.131:0/4109954165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41dbc910 0x555f41dbcd50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800b480 tx=0x7f3a48009630 comp rx=0 tx=0).stop 2024-05-08T22:10:45.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.461+0000 7f3a6c20db80 1 -- 172.21.15.131:0/4109954165 shutdown_connections 2024-05-08T22:10:45.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.461+0000 7f3a6c20db80 1 --2- 172.21.15.131:0/4109954165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41dbc910 0x555f41dbcd50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:45.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.461+0000 7f3a6c20db80 1 --2- 172.21.15.131:0/4109954165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f41dc1e80 0x555f41dc2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:45.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.461+0000 7f3a6c20db80 1 --2- 172.21.15.131:0/4109954165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555f41dc27a0 0x555f41dbc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:45.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.461+0000 7f3a6c20db80 1 -- 172.21.15.131:0/4109954165 >> 172.21.15.131:0/4109954165 conn(0x555f41d1a290 msgr2=0x555f41db6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:45.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.492+0000 7f3a6c20db80 1 -- 172.21.15.131:0/4109954165 shutdown_connections 2024-05-08T22:10:45.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.508+0000 7f3a6c20db80 1 -- 172.21.15.131:0/4109954165 wait complete. 2024-05-08T22:10:45.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.509+0000 7f3a6c20db80 1 -- start start 2024-05-08T22:10:45.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.509+0000 7f3a6c20db80 1 Processor -- start 2024-05-08T22:10:45.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.528+0000 7f3a6c20db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41dbc910 0x555f41dc5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:45.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.528+0000 7f3a6c20db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555f41dc1e80 0x555f41dc61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:45.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.528+0000 7f3a6c20db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f41dc27a0 0x555f41dc6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:45.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.528+0000 7f3a6c20db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f41ca85e0 con 0x555f41dc27a0 2024-05-08T22:10:45.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.528+0000 7f3a6c20db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f41ca8280 con 0x555f41dc1e80 2024-05-08T22:10:45.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.528+0000 7f3a6c20db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f41b8f720 con 0x555f41dbc910 2024-05-08T22:10:45.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.528+0000 7f3a6c20db80 10 client.0 _ll_register_callbacks cb 0x555f41cb1bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:10:45.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.529+0000 7f3a55c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41dbc910 0x555f41dc5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:45.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.529+0000 7f3a55c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41dbc910 0x555f41dc5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41046/0 (socket says 172.21.15.131:41046) 2024-05-08T22:10:45.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.529+0000 7f3a55c00700 1 -- 172.21.15.131:0/1637255499 learned_addr learned my addr 172.21.15.131:0/1637255499 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:45.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.529+0000 7f3a56600700 1 --2- 172.21.15.131:0/1637255499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f41dc27a0 0x555f41dc6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:45.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.529+0000 7f3a55c00700 1 -- 172.21.15.131:0/1637255499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555f41dc1e80 msgr2=0x555f41dc61d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:10:45.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.529+0000 7f3a55c00700 1 --2- 172.21.15.131:0/1637255499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555f41dc1e80 0x555f41dc61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:45.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.529+0000 7f3a55c00700 1 -- 172.21.15.131:0/1637255499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f41dc27a0 msgr2=0x555f41dc6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:45.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.529+0000 7f3a55c00700 1 --2- 172.21.15.131:0/1637255499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f41dc27a0 0x555f41dc6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:45.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.529+0000 7f3a55c00700 1 -- 172.21.15.131:0/1637255499 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a48009c60 con 0x555f41dbc910 2024-05-08T22:10:45.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.530+0000 7f3a55c00700 1 --2- 172.21.15.131:0/1637255499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41dbc910 0x555f41dc5c90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400dd20 tx=0x7f3a44014710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0ece8f19fb2511a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:45.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.557+0000 7f3a37400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:10:45.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.557+0000 7f3a37400700 1 -- 172.21.15.131:0/1637255499 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a44014c40 con 0x555f41dbc910 2024-05-08T22:10:45.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.557+0000 7f3a37400700 1 -- 172.21.15.131:0/1637255499 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f3a44014da0 con 0x555f41dbc910 2024-05-08T22:10:45.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.576+0000 7f3a6c20db80 1 -- 172.21.15.131:0/1637255499 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f41c652f0 con 0x555f41dbc910 2024-05-08T22:10:45.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.578+0000 7f3a37400700 1 -- 172.21.15.131:0/1637255499 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a4401f400 con 0x555f41dbc910 2024-05-08T22:10:45.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.621+0000 7f3a6c20db80 10 client.26152 fetch_fsmap learned FSMap version 26 2024-05-08T22:10:45.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.621+0000 7f3a6c20db80 1 -- 172.21.15.131:0/1637255499 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x555f41b39be0 con 0x555f41dbc910 2024-05-08T22:10:45.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.627+0000 7f3a37400700 1 -- 172.21.15.131:0/1637255499 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555f41c652f0 con 0x555f41dbc910 2024-05-08T22:10:45.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.627+0000 7f3a37400700 1 -- 172.21.15.131:0/1637255499 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3a44027070 con 0x555f41dbc910 2024-05-08T22:10:45.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.627+0000 7f3a6c20db80 10 client.26152 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:10:45.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.627+0000 7f3a6c20db80 10 client.26152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:10:45.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.627+0000 7f3a6c20db80 20 client.26152 populate_metadata read hostname 'smithi131' 2024-05-08T22:10:45.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.627+0000 7f3a6c20db80 1 --2- 172.21.15.131:0/1637255499 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x555f41dd60f0 0x555f41dd8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:45.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.627+0000 7f3a6c20db80 4 client.26152 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:10:45.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.627+0000 7f3a6c20db80 1 -- 172.21.15.131:0/1637255499 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f41d8b890 con 0x555f41dd60f0 2024-05-08T22:10:45.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.627+0000 7f3a55200700 1 --2- 172.21.15.131:0/1637255499 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x555f41dd60f0 0x555f41dd8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:45.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.632+0000 7f3a37400700 10 client.26152 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:10:45.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.632+0000 7f3a55200700 1 --2- 172.21.15.131:0/1637255499 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x555f41dd60f0 0x555f41dd8510 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6a23e4ff0cc4f5d server_cookie=c7a9ba8191838b3a in_seq=0 out_seq=0 2024-05-08T22:10:45.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.643+0000 7f3a37400700 1 -- 172.21.15.131:0/1637255499 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555f41d8b890 con 0x555f41dd60f0 2024-05-08T22:10:45.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.643+0000 7f3a37400700 10 client.26152 handle_command_reply: tid=0 2024-05-08T22:10:45.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.643+0000 7f3a6c20db80 1 client.26152 shutdown 2024-05-08T22:10:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.652+0000 7f3a6c20db80 1 -- 172.21.15.131:0/1637255499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41dbc910 msgr2=0x555f41dc5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.652+0000 7f3a6c20db80 1 --2- 172.21.15.131:0/1637255499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41dbc910 0x555f41dc5c90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400dd20 tx=0x7f3a44014710 comp rx=0 tx=0).stop 2024-05-08T22:10:45.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.656+0000 7f3a6c20db80 1 -- 172.21.15.131:0/1637255499 shutdown_connections 2024-05-08T22:10:45.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.656+0000 7f3a6c20db80 1 --2- 172.21.15.131:0/1637255499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f41dc27a0 0x555f41dc6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:45.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.656+0000 7f3a6c20db80 1 --2- 172.21.15.131:0/1637255499 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x555f41dd60f0 0x555f41dd8510 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:45.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.656+0000 7f3a6c20db80 1 --2- 172.21.15.131:0/1637255499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41dbc910 0x555f41dc5c90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:45.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.656+0000 7f3a6c20db80 1 --2- 172.21.15.131:0/1637255499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555f41dc1e80 0x555f41dc61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:45.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.656+0000 7f3a6c20db80 1 -- 172.21.15.131:0/1637255499 >> 172.21.15.131:0/1637255499 conn(0x555f41d1a290 msgr2=0x555f41db6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:45.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.656+0000 7f3a37400700 0 client.26152 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:10:45.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.671+0000 7f3a6c20db80 1 -- 172.21.15.131:0/1637255499 shutdown_connections 2024-05-08T22:10:45.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.675+0000 7f3a6c20db80 1 -- 172.21.15.131:0/1637255499 wait complete. 2024-05-08T22:10:45.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:45.678+0000 7f3a6c20db80 20 client.26152 trim_cache size 0 max 16384 2024-05-08T22:10:46.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.004+0000 7f3a6c20db80 1 Processor -- start 2024-05-08T22:10:46.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.008+0000 7f3a6c20db80 1 -- start start 2024-05-08T22:10:46.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.016+0000 7f3a6c20db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41d1a290 0x555f41dbc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:46.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.016+0000 7f3a6c20db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555f41dbc910 0x555f41db6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:46.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.016+0000 7f3a6c20db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f41dc1e80 0x555f41dc1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:46.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.016+0000 7f3a55c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41d1a290 0x555f41dbc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:46.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.017+0000 7f3a55c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41d1a290 0x555f41dbc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41054/0 (socket says 172.21.15.131:41054) 2024-05-08T22:10:46.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.019+0000 7f3a55200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555f41dbc910 0x555f41db6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:46.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.019+0000 7f3a55200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555f41dbc910 0x555f41db6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:33912/0 (socket says 172.21.15.131:33912) 2024-05-08T22:10:46.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.029+0000 7f3a6c20db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f41dc87d0 con 0x555f41dc1e80 2024-05-08T22:10:46.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.029+0000 7f3a6c20db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f41dd2ea0 con 0x555f41dbc910 2024-05-08T22:10:46.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.029+0000 7f3a6c20db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f41dc7240 con 0x555f41d1a290 2024-05-08T22:10:46.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.037+0000 7f3a55c00700 1 -- 172.21.15.131:0/604369932 learned_addr learned my addr 172.21.15.131:0/604369932 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:46.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.037+0000 7f3a56600700 1 --2- 172.21.15.131:0/604369932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f41dc1e80 0x555f41dc1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:46.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.043+0000 7f3a56600700 1 -- 172.21.15.131:0/604369932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41d1a290 msgr2=0x555f41dbc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:46.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.043+0000 7f3a56600700 1 --2- 172.21.15.131:0/604369932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41d1a290 0x555f41dbc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:46.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.043+0000 7f3a56600700 1 -- 172.21.15.131:0/604369932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555f41dbc910 msgr2=0x555f41db6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:46.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.043+0000 7f3a56600700 1 --2- 172.21.15.131:0/604369932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555f41dbc910 0x555f41db6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:46.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.043+0000 7f3a56600700 1 -- 172.21.15.131:0/604369932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f41db3560 con 0x555f41dc1e80 2024-05-08T22:10:46.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.044+0000 7f3a56600700 1 --2- 172.21.15.131:0/604369932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f41dc1e80 0x555f41dc1470 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800b480 tx=0x7f3a48014750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b3a439a0f82e16 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:46.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.053+0000 7f3a43400700 1 -- 172.21.15.131:0/604369932 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a48010070 con 0x555f41dc1e80 2024-05-08T22:10:46.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.053+0000 7f3a43400700 1 -- 172.21.15.131:0/604369932 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a48003cc0 con 0x555f41dc1e80 2024-05-08T22:10:46.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.053+0000 7f3a43400700 1 -- 172.21.15.131:0/604369932 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4800acd0 con 0x555f41dc1e80 2024-05-08T22:10:46.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.073+0000 7f3a6c20db80 1 -- 172.21.15.131:0/604369932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f41dc1e80 msgr2=0x555f41dc1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:46.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.073+0000 7f3a6c20db80 1 --2- 172.21.15.131:0/604369932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f41dc1e80 0x555f41dc1470 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800b480 tx=0x7f3a48014750 comp rx=0 tx=0).stop 2024-05-08T22:10:46.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.103+0000 7f3a6c20db80 1 -- 172.21.15.131:0/604369932 shutdown_connections 2024-05-08T22:10:46.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.103+0000 7f3a6c20db80 1 --2- 172.21.15.131:0/604369932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f41dc1e80 0x555f41dc1470 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:46.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.103+0000 7f3a6c20db80 1 --2- 172.21.15.131:0/604369932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41d1a290 0x555f41dbc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:46.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.103+0000 7f3a6c20db80 1 --2- 172.21.15.131:0/604369932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555f41dbc910 0x555f41db6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:46.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.103+0000 7f3a6c20db80 1 -- 172.21.15.131:0/604369932 >> 172.21.15.131:0/604369932 conn(0x555f41dd1f70 msgr2=0x555f41dc0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:46.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.104+0000 7f3a6c20db80 1 -- 172.21.15.131:0/604369932 shutdown_connections 2024-05-08T22:10:46.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.104+0000 7f3a6c20db80 1 -- 172.21.15.131:0/604369932 wait complete. 2024-05-08T22:10:46.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.105+0000 7f3a6c20db80 1 -- start start 2024-05-08T22:10:46.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.105+0000 7f3a6c20db80 1 Processor -- start 2024-05-08T22:10:46.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.126+0000 7f3a6c20db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41d1a290 0x555f41dcbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:46.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.126+0000 7f3a6c20db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555f41dbc910 0x555f41dcc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:46.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.130+0000 7f3a55c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41d1a290 0x555f41dcbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:46.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.130+0000 7f3a55200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555f41dbc910 0x555f41dcc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:46.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.130+0000 7f3a55200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555f41dbc910 0x555f41dcc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:33922/0 (socket says 172.21.15.131:33922) 2024-05-08T22:10:46.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.130+0000 7f3a55c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41d1a290 0x555f41dcbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41070/0 (socket says 172.21.15.131:41070) 2024-05-08T22:10:46.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.130+0000 7f3a55200700 1 -- 172.21.15.131:0/1465957551 learned_addr learned my addr 172.21.15.131:0/1465957551 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:46.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.130+0000 7f3a6c20db80 1 --2- 172.21.15.131:0/1465957551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f41dc1e80 0x555f41dcc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:46.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.133+0000 7f3a56600700 1 --2- 172.21.15.131:0/1465957551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f41dc1e80 0x555f41dcc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:46.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.136+0000 7f3a6c20db80 1 -- 172.21.15.131:0/1465957551 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f41dcdac0 con 0x555f41dc1e80 2024-05-08T22:10:46.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.136+0000 7f3a6c20db80 1 -- 172.21.15.131:0/1465957551 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f41e3d610 con 0x555f41dbc910 2024-05-08T22:10:46.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.136+0000 7f3a6c20db80 1 -- 172.21.15.131:0/1465957551 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f41e3d780 con 0x555f41d1a290 2024-05-08T22:10:46.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.137+0000 7f3a56600700 1 -- 172.21.15.131:0/1465957551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41d1a290 msgr2=0x555f41dcbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:46.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.137+0000 7f3a56600700 1 --2- 172.21.15.131:0/1465957551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41d1a290 0x555f41dcbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:46.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.137+0000 7f3a56600700 1 -- 172.21.15.131:0/1465957551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555f41dbc910 msgr2=0x555f41dcc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:46.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.137+0000 7f3a56600700 1 --2- 172.21.15.131:0/1465957551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555f41dbc910 0x555f41dcc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:46.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.137+0000 7f3a56600700 1 -- 172.21.15.131:0/1465957551 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f41b8d430 con 0x555f41dc1e80 2024-05-08T22:10:46.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.138+0000 7f3a56600700 1 --2- 172.21.15.131:0/1465957551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f41dc1e80 0x555f41dcc6c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800b450 tx=0x7f3a48014cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc2bc6674b5aa9b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:46.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.139+0000 7f3a6c20db80 10 client.0 _ll_register_callbacks cb 0x555f41cb1bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:10:46.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.139+0000 7f3a6c20db80 1 -- 172.21.15.131:0/1465957551 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a4800be50 con 0x555f41dc1e80 2024-05-08T22:10:46.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.141+0000 7f3a37400700 10 client.16086 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:10:46.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.141+0000 7f3a37400700 1 -- 172.21.15.131:0/1465957551 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a48010040 con 0x555f41dc1e80 2024-05-08T22:10:46.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.141+0000 7f3a37400700 1 -- 172.21.15.131:0/1465957551 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f3a4800a8d0 con 0x555f41dc1e80 2024-05-08T22:10:46.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.142+0000 7f3a37400700 1 -- 172.21.15.131:0/1465957551 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a48009370 con 0x555f41dc1e80 2024-05-08T22:10:46.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.144+0000 7f3a6c20db80 10 client.16086 fetch_fsmap learned FSMap version 26 2024-05-08T22:10:46.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.144+0000 7f3a6c20db80 1 -- 172.21.15.131:0/1465957551 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=26}) v3 -- 0x555f41cd25e0 con 0x555f41dc1e80 2024-05-08T22:10:46.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.145+0000 7f3a37400700 1 -- 172.21.15.131:0/1465957551 <== mon.0 v2:172.21.15.131:3300/0 4 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3a4800aa90 con 0x555f41dc1e80 2024-05-08T22:10:46.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.155+0000 7f3a6c20db80 10 client.16086 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:10:46.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.155+0000 7f3a6c20db80 10 client.16086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:10:46.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.155+0000 7f3a6c20db80 20 client.16086 populate_metadata read hostname 'smithi131' 2024-05-08T22:10:46.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.155+0000 7f3a6c20db80 1 --2- 172.21.15.131:0/1465957551 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x555f41dc27a0 0x555f41e407d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:46.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.155+0000 7f3a6c20db80 4 client.16086 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T22:10:46.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.155+0000 7f3a6c20db80 1 -- 172.21.15.131:0/1465957551 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555f41d04f40 con 0x555f41dc27a0 2024-05-08T22:10:46.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.156+0000 7f3a55c00700 1 --2- 172.21.15.131:0/1465957551 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x555f41dc27a0 0x555f41e407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:46.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.170+0000 7f3a55c00700 1 --2- 172.21.15.131:0/1465957551 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x555f41dc27a0 0x555f41e407d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=609553bed5065be1 server_cookie=a534c82ac3b2526f in_seq=0 out_seq=0 2024-05-08T22:10:46.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.179+0000 7f3a37400700 10 client.16086 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:10:46.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.231+0000 7f3a37400700 1 -- 172.21.15.131:0/1465957551 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555f41d04f40 con 0x555f41dc27a0 2024-05-08T22:10:46.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.231+0000 7f3a37400700 10 client.16086 handle_command_reply: tid=0 2024-05-08T22:10:46.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.231+0000 7f3a6c20db80 1 client.16086 shutdown 2024-05-08T22:10:46.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.257+0000 7f3a6c20db80 1 -- 172.21.15.131:0/1465957551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f41dc1e80 msgr2=0x555f41dcc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:46.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.257+0000 7f3a6c20db80 1 --2- 172.21.15.131:0/1465957551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f41dc1e80 0x555f41dcc6c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800b450 tx=0x7f3a48014cf0 comp rx=0 tx=0).stop 2024-05-08T22:10:46.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.262+0000 7f3a6c20db80 1 -- 172.21.15.131:0/1465957551 shutdown_connections 2024-05-08T22:10:46.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.262+0000 7f3a6c20db80 1 --2- 172.21.15.131:0/1465957551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f41dc1e80 0x555f41dcc6c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:46.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.262+0000 7f3a6c20db80 1 --2- 172.21.15.131:0/1465957551 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x555f41dc27a0 0x555f41e407d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:46.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.262+0000 7f3a6c20db80 1 --2- 172.21.15.131:0/1465957551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555f41d1a290 0x555f41dcbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:46.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.262+0000 7f3a6c20db80 1 --2- 172.21.15.131:0/1465957551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555f41dbc910 0x555f41dcc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:46.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.262+0000 7f3a6c20db80 1 -- 172.21.15.131:0/1465957551 >> 172.21.15.131:0/1465957551 conn(0x555f41dd1f70 msgr2=0x555f41dc4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:46.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.263+0000 7f3a37400700 0 client.16086 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:10:46.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.263+0000 7f3a6c20db80 1 -- 172.21.15.131:0/1465957551 shutdown_connections 2024-05-08T22:10:46.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.263+0000 7f3a6c20db80 1 -- 172.21.15.131:0/1465957551 wait complete. 2024-05-08T22:10:46.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.264+0000 7f3a6c20db80 20 client.16086 trim_cache size 0 max 16384 2024-05-08T22:10:46.267 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:10:46.268 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-08T22:10:46.268 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "aa3822e7-6b1d-40c9-9246-37810abecb93", 2024-05-08T22:10:46.268 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-08T22:10:46.268 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:10:46.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.302+0000 7f3a58600700 1 -- 172.21.15.131:0/2975359340 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3a380a13d0 msgr2=0x7f3a380a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:46.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.302+0000 7f3a58600700 1 --2- 172.21.15.131:0/2975359340 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3a380a13d0 0x7f3a380a3890 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009910 tx=0x7f3a44008040 comp rx=0 tx=0).stop 2024-05-08T22:10:46.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.302+0000 7f3a58600700 1 -- 172.21.15.131:0/2975359340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a5011a780 msgr2=0x7f3a501a6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:46.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.302+0000 7f3a58600700 1 --2- 172.21.15.131:0/2975359340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a5011a780 0x7f3a501a6b70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c006010 tx=0x7f3a3c003b20 comp rx=0 tx=0).stop 2024-05-08T22:10:46.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.303+0000 7f3a58600700 1 -- 172.21.15.131:0/2975359340 shutdown_connections 2024-05-08T22:10:46.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.303+0000 7f3a58600700 1 --2- 172.21.15.131:0/2975359340 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3a380a13d0 0x7f3a380a3890 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:46.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.303+0000 7f3a58600700 1 --2- 172.21.15.131:0/2975359340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a501aaf10 0x7f3a501ab390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:46.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.303+0000 7f3a58600700 1 --2- 172.21.15.131:0/2975359340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a5010f6a0 0x7f3a501a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:46.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.303+0000 7f3a58600700 1 --2- 172.21.15.131:0/2975359340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a5011a780 0x7f3a501a6b70 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:46.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.303+0000 7f3a58600700 1 -- 172.21.15.131:0/2975359340 >> 172.21.15.131:0/2975359340 conn(0x7f3a5006bff0 msgr2=0x7f3a50118770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:46.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.305+0000 7f3a58600700 1 -- 172.21.15.131:0/2975359340 shutdown_connections 2024-05-08T22:10:46.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:46.305+0000 7f3a58600700 1 -- 172.21.15.131:0/2975359340 wait complete. 2024-05-08T22:10:46.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:46 smithi132 ceph-mon[78180]: pgmap v107: 129 pgs: 129 active+clean; 361 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 132 op/s 2024-05-08T22:10:46.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:46 smithi131 ceph-mon[37260]: pgmap v107: 129 pgs: 129 active+clean; 361 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 132 op/s 2024-05-08T22:10:46.775 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:46 smithi157 ceph-mon[77746]: pgmap v107: 129 pgs: 129 active+clean; 361 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 132 op/s 2024-05-08T22:10:46.908 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:10:47.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:47 smithi132 ceph-mon[78180]: scrub queued for path: / 2024-05-08T22:10:47.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:47 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:10:47.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:47 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:10:47.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:47 smithi131 ceph-mon[37260]: scrub queued for path: / 2024-05-08T22:10:47.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:47 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:10:47.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:47 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:10:47.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:47 smithi157 ceph-mon[77746]: scrub queued for path: / 2024-05-08T22:10:47.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:47 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:10:47.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:47 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:10:48.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.462+0000 7fcee5a00700 1 -- 172.21.15.131:0/798215334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee011a780 msgr2=0x7fcee011cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:48.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.462+0000 7fcee5a00700 1 --2- 172.21.15.131:0/798215334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee011a780 0x7fcee011cb70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fced0009940 tx=0x7fced0009c50 comp rx=0 tx=0).stop 2024-05-08T22:10:48.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.483+0000 7fcee5a00700 1 -- 172.21.15.131:0/798215334 shutdown_connections 2024-05-08T22:10:48.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.483+0000 7fcee5a00700 1 --2- 172.21.15.131:0/798215334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee011a780 0x7fcee011cb70 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:48.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.483+0000 7fcee5a00700 1 --2- 172.21.15.131:0/798215334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee010f6a0 0x7fcee0107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:48.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.483+0000 7fcee5a00700 1 --2- 172.21.15.131:0/798215334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcee010fb10 0x7fcee01084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:48.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.483+0000 7fcee5a00700 1 -- 172.21.15.131:0/798215334 >> 172.21.15.131:0/798215334 conn(0x7fcee006bff0 msgr2=0x7fcee006c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:48.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.498+0000 7fcee5a00700 1 -- 172.21.15.131:0/798215334 shutdown_connections 2024-05-08T22:10:48.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.498+0000 7fcee5a00700 1 -- 172.21.15.131:0/798215334 wait complete. 2024-05-08T22:10:48.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.500+0000 7fcee5a00700 1 Processor -- start 2024-05-08T22:10:48.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.504+0000 7fcee5a00700 1 -- start start 2024-05-08T22:10:48.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.505+0000 7fcee5a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee010f6a0 0x7fcee0118c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:48.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.513+0000 7fcedea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee010f6a0 0x7fcee0118c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:48.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.513+0000 7fcedea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee010f6a0 0x7fcee0118c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37880/0 (socket says 172.21.15.131:37880) 2024-05-08T22:10:48.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.513+0000 7fcedea00700 1 -- 172.21.15.131:0/4021121773 learned_addr learned my addr 172.21.15.131:0/4021121773 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:48.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.514+0000 7fcee5a00700 1 --2- 172.21.15.131:0/4021121773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee010fb10 0x7fcee01191b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:48.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.517+0000 7fcee5a00700 1 --2- 172.21.15.131:0/4021121773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcee011a780 0x7fcee0111e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.518+0000 7fcede000700 1 --2- 172.21.15.131:0/4021121773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee010fb10 0x7fcee01191b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.521+0000 7fcee5a00700 1 -- 172.21.15.131:0/4021121773 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee0112600 con 0x7fcee010f6a0 2024-05-08T22:10:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.521+0000 7fcee5a00700 1 -- 172.21.15.131:0/4021121773 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee0112770 con 0x7fcee010fb10 2024-05-08T22:10:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.521+0000 7fcee5a00700 1 -- 172.21.15.131:0/4021121773 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee0112930 con 0x7fcee011a780 2024-05-08T22:10:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.521+0000 7fcedf400700 1 --2- 172.21.15.131:0/4021121773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcee011a780 0x7fcee0111e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.522+0000 7fcedea00700 1 -- 172.21.15.131:0/4021121773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcee011a780 msgr2=0x7fcee0111e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.522+0000 7fcedea00700 1 --2- 172.21.15.131:0/4021121773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcee011a780 0x7fcee0111e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.522+0000 7fcedea00700 1 -- 172.21.15.131:0/4021121773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee010fb10 msgr2=0x7fcee01191b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.522+0000 7fcedea00700 1 --2- 172.21.15.131:0/4021121773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee010fb10 0x7fcee01191b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.522+0000 7fcedea00700 1 -- 172.21.15.131:0/4021121773 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced0009520 con 0x7fcee010f6a0 2024-05-08T22:10:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.523+0000 7fcedea00700 1 --2- 172.21.15.131:0/4021121773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee010f6a0 0x7fcee0118c30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fced800ba50 tx=0x7fced800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:48.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.526+0000 7fced7400700 1 -- 172.21.15.131:0/4021121773 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced800c9b0 con 0x7fcee010f6a0 2024-05-08T22:10:48.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.526+0000 7fced7400700 1 -- 172.21.15.131:0/4021121773 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fced8014440 con 0x7fcee010f6a0 2024-05-08T22:10:48.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.526+0000 7fced7400700 1 -- 172.21.15.131:0/4021121773 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced80075b0 con 0x7fcee010f6a0 2024-05-08T22:10:48.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.544+0000 7fcee5a00700 1 -- 172.21.15.131:0/4021121773 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee01167c0 con 0x7fcee010f6a0 2024-05-08T22:10:48.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.544+0000 7fcee5a00700 1 -- 172.21.15.131:0/4021121773 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee0116c50 con 0x7fcee010f6a0 2024-05-08T22:10:48.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.583+0000 7fced7400700 1 -- 172.21.15.131:0/4021121773 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fced80145b0 con 0x7fcee010f6a0 2024-05-08T22:10:48.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.585+0000 7fced7400700 1 --2- 172.21.15.131:0/4021121773 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fcec40a14a0 0x7fcec40a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:48.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.585+0000 7fced7400700 1 -- 172.21.15.131:0/4021121773 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fced80cc750 con 0x7fcee010f6a0 2024-05-08T22:10:48.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.585+0000 7fcede000700 1 --2- 172.21.15.131:0/4021121773 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fcec40a14a0 0x7fcec40a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:48.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.587+0000 7fcede000700 1 --2- 172.21.15.131:0/4021121773 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fcec40a14a0 0x7fcec40a3960 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcee0113430 tx=0x7fcecc00b410 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:48.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.614+0000 7fcef95f8b80 1 Processor -- start 2024-05-08T22:10:48.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.614+0000 7fcef95f8b80 1 -- start start 2024-05-08T22:10:48.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.615+0000 7fcef95f8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c42b6bce00 0x55c42b6bd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:48.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.615+0000 7fcef95f8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c42b6bd720 0x55c42b6b7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:48.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.615+0000 7fcef95f8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c42b6b7890 0x55c42b6b7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:48.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.615+0000 7fcef95f8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42b60e150 con 0x55c42b6bce00 2024-05-08T22:10:48.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.615+0000 7fcef95f8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42b60bb10 con 0x55c42b6b7890 2024-05-08T22:10:48.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.615+0000 7fcef95f8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42b607400 con 0x55c42b6bd720 2024-05-08T22:10:48.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.615+0000 7fcedf400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c42b6b7890 0x55c42b6b7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:48.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.615+0000 7fcedf400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c42b6b7890 0x55c42b6b7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:33944/0 (socket says 172.21.15.131:33944) 2024-05-08T22:10:48.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.615+0000 7fcedf400700 1 -- 172.21.15.131:0/32786849 learned_addr learned my addr 172.21.15.131:0/32786849 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:48.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.616+0000 7fcedf400700 1 -- 172.21.15.131:0/32786849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c42b6bd720 msgr2=0x55c42b6b7350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:10:48.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.616+0000 7fcedf400700 1 --2- 172.21.15.131:0/32786849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c42b6bd720 0x55c42b6b7350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:48.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.616+0000 7fcedf400700 1 -- 172.21.15.131:0/32786849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c42b6bce00 msgr2=0x55c42b6bd1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:10:48.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.616+0000 7fcedf400700 1 --2- 172.21.15.131:0/32786849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c42b6bce00 0x55c42b6bd1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:48.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.616+0000 7fcedf400700 1 -- 172.21.15.131:0/32786849 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c42b4883b0 con 0x55c42b6b7890 2024-05-08T22:10:48.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.616+0000 7fcedf400700 1 --2- 172.21.15.131:0/32786849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c42b6b7890 0x55c42b6b7cd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fced0005d90 tx=0x7fced000f480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0d9c878308cb0fe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:48.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.636+0000 7fcecbe00700 1 -- 172.21.15.131:0/32786849 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced001c070 con 0x55c42b6b7890 2024-05-08T22:10:48.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.636+0000 7fcecbe00700 1 -- 172.21.15.131:0/32786849 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fced000fae0 con 0x55c42b6b7890 2024-05-08T22:10:48.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.636+0000 7fcecbe00700 1 -- 172.21.15.131:0/32786849 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced000baa0 con 0x55c42b6b7890 2024-05-08T22:10:48.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.642+0000 7fcef95f8b80 1 -- 172.21.15.131:0/32786849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c42b6b7890 msgr2=0x55c42b6b7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:48.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.642+0000 7fcef95f8b80 1 --2- 172.21.15.131:0/32786849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c42b6b7890 0x55c42b6b7cd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fced0005d90 tx=0x7fced000f480 comp rx=0 tx=0).stop 2024-05-08T22:10:48.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.654+0000 7fcef95f8b80 1 -- 172.21.15.131:0/32786849 shutdown_connections 2024-05-08T22:10:48.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.654+0000 7fcef95f8b80 1 --2- 172.21.15.131:0/32786849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c42b6b7890 0x55c42b6b7cd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:48.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.654+0000 7fcef95f8b80 1 --2- 172.21.15.131:0/32786849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c42b6bce00 0x55c42b6bd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:48.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.654+0000 7fcef95f8b80 1 --2- 172.21.15.131:0/32786849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c42b6bd720 0x55c42b6b7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:48.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.654+0000 7fcef95f8b80 1 -- 172.21.15.131:0/32786849 >> 172.21.15.131:0/32786849 conn(0x55c42b615210 msgr2=0x55c42b6b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:48.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.670+0000 7fcef95f8b80 1 -- 172.21.15.131:0/32786849 shutdown_connections 2024-05-08T22:10:48.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.671+0000 7fcef95f8b80 1 -- 172.21.15.131:0/32786849 wait complete. 2024-05-08T22:10:48.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.672+0000 7fcef95f8b80 1 -- start start 2024-05-08T22:10:48.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.672+0000 7fcef95f8b80 1 Processor -- start 2024-05-08T22:10:48.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.690+0000 7fcef95f8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c42b6b7890 0x55c42b6c0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:48.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.700+0000 7fcef95f8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c42b6bce00 0x55c42b6c10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:48.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.699+0000 7fcedea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c42b6b7890 0x55c42b6c0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:48.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.700+0000 7fcedea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c42b6b7890 0x55c42b6c0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:33958/0 (socket says 172.21.15.131:33958) 2024-05-08T22:10:48.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.701+0000 7fcedea00700 1 -- 172.21.15.131:0/1931973934 learned_addr learned my addr 172.21.15.131:0/1931973934 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:48.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.701+0000 7fcef95f8b80 1 --2- 172.21.15.131:0/1931973934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c42b6bd720 0x55c42b6c1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:48.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.701+0000 7fcef95f8b80 1 -- 172.21.15.131:0/1931973934 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42b5a3560 con 0x55c42b6bce00 2024-05-08T22:10:48.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.701+0000 7fcef95f8b80 1 -- 172.21.15.131:0/1931973934 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42b5a3200 con 0x55c42b6b7890 2024-05-08T22:10:48.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.701+0000 7fcef95f8b80 1 -- 172.21.15.131:0/1931973934 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42b48a6a0 con 0x55c42b6bd720 2024-05-08T22:10:48.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.701+0000 7fcef95f8b80 10 client.0 _ll_register_callbacks cb 0x55c42b5acb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:10:48.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.702+0000 7fcedf400700 1 --2- 172.21.15.131:0/1931973934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c42b6bd720 0x55c42b6c1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:48.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.702+0000 7fcedf400700 1 -- 172.21.15.131:0/1931973934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c42b6b7890 msgr2=0x55c42b6c0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:48.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.702+0000 7fcedf400700 1 --2- 172.21.15.131:0/1931973934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c42b6b7890 0x55c42b6c0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:48.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.702+0000 7fcedf400700 1 -- 172.21.15.131:0/1931973934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c42b6bce00 msgr2=0x55c42b6c10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:48.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.702+0000 7fcedf400700 1 --2- 172.21.15.131:0/1931973934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c42b6bce00 0x55c42b6c10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:48.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.702+0000 7fcedf400700 1 -- 172.21.15.131:0/1931973934 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee011d460 con 0x55c42b6bd720 2024-05-08T22:10:48.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.705+0000 7fcedea00700 1 --2- 172.21.15.131:0/1931973934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c42b6b7890 0x55c42b6c0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:10:48.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.706+0000 7fcef95f8b80 1 -- 172.21.15.131:0/1931973934 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c42b560270 con 0x55c42b6bd720 2024-05-08T22:10:48.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.715+0000 7fcedf400700 1 --2- 172.21.15.131:0/1931973934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c42b6bd720 0x55c42b6c1600 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fced000b5c0 tx=0x7fced0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68070b562cd7fbcf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:48.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.718+0000 7fcec8c00700 10 client.26164 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:10:48.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.718+0000 7fcec8c00700 1 -- 172.21.15.131:0/1931973934 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced001c070 con 0x55c42b6bd720 2024-05-08T22:10:48.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.718+0000 7fcec8c00700 1 -- 172.21.15.131:0/1931973934 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fced000bbe0 con 0x55c42b6bd720 2024-05-08T22:10:48.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.718+0000 7fcec8c00700 1 -- 172.21.15.131:0/1931973934 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fced000bda0 con 0x55c42b6bd720 2024-05-08T22:10:48.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.742+0000 7fcef95f8b80 10 client.26164 fetch_fsmap learned FSMap version 26 2024-05-08T22:10:48.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.742+0000 7fcef95f8b80 1 -- 172.21.15.131:0/1931973934 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=26}) v3 -- 0x55c42b434b60 con 0x55c42b6bd720 2024-05-08T22:10:48.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.744+0000 7fcec8c00700 1 -- 172.21.15.131:0/1931973934 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fced0027070 con 0x55c42b6bd720 2024-05-08T22:10:48.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.746+0000 7fcef95f8b80 10 client.26164 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:10:48.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.746+0000 7fcef95f8b80 10 client.26164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:10:48.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.746+0000 7fcef95f8b80 20 client.26164 populate_metadata read hostname 'smithi131' 2024-05-08T22:10:48.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.746+0000 7fcef95f8b80 1 --2- 172.21.15.131:0/1931973934 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55c42b6d0c10 0x55c42b6d3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:48.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.746+0000 7fcef95f8b80 4 client.26164 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:10:48.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.746+0000 7fcef95f8b80 1 -- 172.21.15.131:0/1931973934 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c42b686810 con 0x55c42b6d0c10 2024-05-08T22:10:48.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.747+0000 7fcedea00700 1 --2- 172.21.15.131:0/1931973934 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55c42b6d0c10 0x55c42b6d3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:48.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.757+0000 7fcedea00700 1 --2- 172.21.15.131:0/1931973934 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55c42b6d0c10 0x55c42b6d3000 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af88939e210746ea server_cookie=464f14194a25e8f1 in_seq=0 out_seq=0 2024-05-08T22:10:48.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.757+0000 7fcec8c00700 10 client.26164 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:10:48.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.761+0000 7fcec8c00700 1 -- 172.21.15.131:0/1931973934 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c42b686810 con 0x55c42b6d0c10 2024-05-08T22:10:48.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.761+0000 7fcec8c00700 10 client.26164 handle_command_reply: tid=0 2024-05-08T22:10:48.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.762+0000 7fcef95f8b80 1 client.26164 shutdown 2024-05-08T22:10:48.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.770+0000 7fcef95f8b80 1 -- 172.21.15.131:0/1931973934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c42b6bd720 msgr2=0x55c42b6c1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:48.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.770+0000 7fcef95f8b80 1 --2- 172.21.15.131:0/1931973934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c42b6bd720 0x55c42b6c1600 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fced000b5c0 tx=0x7fced0009260 comp rx=0 tx=0).stop 2024-05-08T22:10:48.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.770+0000 7fcef95f8b80 1 -- 172.21.15.131:0/1931973934 shutdown_connections 2024-05-08T22:10:48.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.770+0000 7fcef95f8b80 1 --2- 172.21.15.131:0/1931973934 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55c42b6d0c10 0x55c42b6d3000 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:48.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.770+0000 7fcef95f8b80 1 --2- 172.21.15.131:0/1931973934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c42b6bd720 0x55c42b6c1600 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:48.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.770+0000 7fcef95f8b80 1 --2- 172.21.15.131:0/1931973934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c42b6b7890 0x55c42b6c0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:48.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.770+0000 7fcef95f8b80 1 --2- 172.21.15.131:0/1931973934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c42b6bce00 0x55c42b6c10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:48.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.770+0000 7fcef95f8b80 1 -- 172.21.15.131:0/1931973934 >> 172.21.15.131:0/1931973934 conn(0x55c42b615210 msgr2=0x55c42b6b15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:48.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.770+0000 7fcec8c00700 0 client.26164 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:10:48.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.781+0000 7fcef95f8b80 1 -- 172.21.15.131:0/1931973934 shutdown_connections 2024-05-08T22:10:48.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.784+0000 7fcef95f8b80 1 -- 172.21.15.131:0/1931973934 wait complete. 2024-05-08T22:10:48.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.785+0000 7fcef95f8b80 20 client.26164 trim_cache size 0 max 16384 2024-05-08T22:10:48.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:48 smithi132 ceph-mon[78180]: pgmap v108: 129 pgs: 129 active+clean; 363 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 111 op/s 2024-05-08T22:10:48.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:48 smithi131 ceph-mon[37260]: pgmap v108: 129 pgs: 129 active+clean; 363 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 111 op/s 2024-05-08T22:10:48.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:48.993+0000 7fcef95f8b80 1 Processor -- start 2024-05-08T22:10:49.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.015+0000 7fcef95f8b80 1 -- start start 2024-05-08T22:10:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.015+0000 7fcef95f8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c42b6b7890 0x55c42b6b7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.015+0000 7fcef95f8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c42b6bce00 0x55c42b6ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.015+0000 7fcef95f8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c42b6bd720 0x55c42b6bc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.015+0000 7fcef95f8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42b6bc890 con 0x55c42b6b7890 2024-05-08T22:10:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.015+0000 7fcef95f8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42b6cdd80 con 0x55c42b6bce00 2024-05-08T22:10:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.015+0000 7fcef95f8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42b6c2180 con 0x55c42b6bd720 2024-05-08T22:10:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.016+0000 7fcedea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c42b6b7890 0x55c42b6b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.016+0000 7fcedea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c42b6b7890 0x55c42b6b7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37906/0 (socket says 172.21.15.131:37906) 2024-05-08T22:10:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.016+0000 7fcedea00700 1 -- 172.21.15.131:0/4233117274 learned_addr learned my addr 172.21.15.131:0/4233117274 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.016+0000 7fcedf400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c42b6bd720 0x55c42b6bc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.017+0000 7fcedea00700 1 -- 172.21.15.131:0/4233117274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c42b6bd720 msgr2=0x55c42b6bc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.017+0000 7fcedea00700 1 --2- 172.21.15.131:0/4233117274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c42b6bd720 0x55c42b6bc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.017+0000 7fcedea00700 1 -- 172.21.15.131:0/4233117274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c42b6bce00 msgr2=0x55c42b6ccbc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:10:49.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.017+0000 7fcedea00700 1 --2- 172.21.15.131:0/4233117274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c42b6bce00 0x55c42b6ccbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:49.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.017+0000 7fcedea00700 1 -- 172.21.15.131:0/4233117274 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c42b5cd560 con 0x55c42b6b7890 2024-05-08T22:10:49.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.017+0000 7fcedea00700 1 --2- 172.21.15.131:0/4233117274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c42b6b7890 0x55c42b6b7350 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fced8095a80 tx=0x7fced8014c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a69143147ea092 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:49.025 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:48 smithi157 ceph-mon[77746]: pgmap v108: 129 pgs: 129 active+clean; 363 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 111 op/s 2024-05-08T22:10:49.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.025+0000 7fced4c00700 1 -- 172.21.15.131:0/4233117274 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced80b3020 con 0x55c42b6b7890 2024-05-08T22:10:49.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.025+0000 7fced4c00700 1 -- 172.21.15.131:0/4233117274 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fced8099580 con 0x55c42b6b7890 2024-05-08T22:10:49.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.025+0000 7fced4c00700 1 -- 172.21.15.131:0/4233117274 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced80979f0 con 0x55c42b6b7890 2024-05-08T22:10:49.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.034+0000 7fcef95f8b80 1 -- 172.21.15.131:0/4233117274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c42b6b7890 msgr2=0x55c42b6b7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:49.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.034+0000 7fcef95f8b80 1 --2- 172.21.15.131:0/4233117274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c42b6b7890 0x55c42b6b7350 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fced8095a80 tx=0x7fced8014c70 comp rx=0 tx=0).stop 2024-05-08T22:10:49.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.039+0000 7fcef95f8b80 1 -- 172.21.15.131:0/4233117274 shutdown_connections 2024-05-08T22:10:49.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.039+0000 7fcef95f8b80 1 --2- 172.21.15.131:0/4233117274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c42b6bd720 0x55c42b6bc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:49.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.039+0000 7fcef95f8b80 1 --2- 172.21.15.131:0/4233117274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c42b6b7890 0x55c42b6b7350 secure :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fced8095a80 tx=0x7fced8014c70 comp rx=0 tx=0).stop 2024-05-08T22:10:49.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.039+0000 7fcef95f8b80 1 --2- 172.21.15.131:0/4233117274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c42b6bce00 0x55c42b6ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:49.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.039+0000 7fcef95f8b80 1 -- 172.21.15.131:0/4233117274 >> 172.21.15.131:0/4233117274 conn(0x55c42b615210 msgr2=0x55c42b6bbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:49.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.054+0000 7fcef95f8b80 1 -- 172.21.15.131:0/4233117274 shutdown_connections 2024-05-08T22:10:49.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.054+0000 7fcef95f8b80 1 -- 172.21.15.131:0/4233117274 wait complete. 2024-05-08T22:10:49.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.054+0000 7fcef95f8b80 1 -- start start 2024-05-08T22:10:49.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.055+0000 7fcef95f8b80 1 Processor -- start 2024-05-08T22:10:49.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.055+0000 7fcef95f8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c42b6bce00 0x55c42b6c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:49.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.060+0000 7fcef95f8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c42b6bd720 0x55c42b6c8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:49.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.060+0000 7fcedea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c42b6bce00 0x55c42b6c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:49.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.060+0000 7fcede000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c42b6bd720 0x55c42b6c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:49.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.060+0000 7fcede000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c42b6bd720 0x55c42b6c8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37914/0 (socket says 172.21.15.131:37914) 2024-05-08T22:10:49.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.060+0000 7fcede000700 1 -- 172.21.15.131:0/3576179135 learned_addr learned my addr 172.21.15.131:0/3576179135 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:10:49.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.061+0000 7fcef95f8b80 1 --2- 172.21.15.131:0/3576179135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c42b6c6610 0x55c42b6c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:49.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.061+0000 7fcef95f8b80 1 -- 172.21.15.131:0/3576179135 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42b6c71a0 con 0x55c42b6bd720 2024-05-08T22:10:49.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.061+0000 7fcef95f8b80 1 -- 172.21.15.131:0/3576179135 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42b6c7310 con 0x55c42b6c6610 2024-05-08T22:10:49.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.061+0000 7fcef95f8b80 1 -- 172.21.15.131:0/3576179135 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42b727090 con 0x55c42b6bce00 2024-05-08T22:10:49.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.061+0000 7fcedf400700 1 --2- 172.21.15.131:0/3576179135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c42b6c6610 0x55c42b6c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:49.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.062+0000 7fcede000700 1 -- 172.21.15.131:0/3576179135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c42b6bce00 msgr2=0x55c42b6c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:49.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.062+0000 7fcede000700 1 --2- 172.21.15.131:0/3576179135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c42b6bce00 0x55c42b6c80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:49.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.062+0000 7fcede000700 1 -- 172.21.15.131:0/3576179135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c42b6c6610 msgr2=0x55c42b6c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:10:49.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.062+0000 7fcede000700 1 --2- 172.21.15.131:0/3576179135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c42b6c6610 0x55c42b6c6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:49.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.062+0000 7fcede000700 1 -- 172.21.15.131:0/3576179135 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee0116c50 con 0x55c42b6bd720 2024-05-08T22:10:49.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.062+0000 7fcede000700 1 --2- 172.21.15.131:0/3576179135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c42b6bd720 0x55c42b6c8610 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0046e0 tx=0x7fcecc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42af3ca6f2e9291 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:10:49.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.062+0000 7fcedf400700 1 --2- 172.21.15.131:0/3576179135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c42b6c6610 0x55c42b6c6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:10:49.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.063+0000 7fcef95f8b80 10 client.0 _ll_register_callbacks cb 0x55c42b5acb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:10:49.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.063+0000 7fcef95f8b80 1 -- 172.21.15.131:0/3576179135 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fced000fae0 con 0x55c42b6bd720 2024-05-08T22:10:49.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.074+0000 7fcec8c00700 10 client.16098 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:10:49.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.074+0000 7fcec8c00700 1 -- 172.21.15.131:0/3576179135 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc011ad0 con 0x55c42b6bd720 2024-05-08T22:10:49.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.074+0000 7fcec8c00700 1 -- 172.21.15.131:0/3576179135 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fcecc017070 con 0x55c42b6bd720 2024-05-08T22:10:49.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.074+0000 7fcec8c00700 1 -- 172.21.15.131:0/3576179135 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcecc011c30 con 0x55c42b6bd720 2024-05-08T22:10:49.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.074+0000 7fcef95f8b80 10 client.16098 fetch_fsmap learned FSMap version 26 2024-05-08T22:10:49.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.074+0000 7fcef95f8b80 1 -- 172.21.15.131:0/3576179135 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=26}) v3 -- 0x55c42b48ccd0 con 0x55c42b6bd720 2024-05-08T22:10:49.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.079+0000 7fcec8c00700 1 -- 172.21.15.131:0/3576179135 <== mon.0 v2:172.21.15.131:3300/0 4 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fcecc01e420 con 0x55c42b6bd720 2024-05-08T22:10:49.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.090+0000 7fcef95f8b80 10 client.16098 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:10:49.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.090+0000 7fcef95f8b80 10 client.16098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:10:49.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.090+0000 7fcef95f8b80 20 client.16098 populate_metadata read hostname 'smithi131' 2024-05-08T22:10:49.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.090+0000 7fcef95f8b80 1 --2- 172.21.15.131:0/3576179135 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55c42b7284a0 0x55c42b72a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:10:49.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.090+0000 7fcef95f8b80 4 client.16098 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:10:49.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.090+0000 7fcef95f8b80 1 -- 172.21.15.131:0/3576179135 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c42b5ffec0 con 0x55c42b7284a0 2024-05-08T22:10:49.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.090+0000 7fcedea00700 1 --2- 172.21.15.131:0/3576179135 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55c42b7284a0 0x55c42b72a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:10:49.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.091+0000 7fcedea00700 1 --2- 172.21.15.131:0/3576179135 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55c42b7284a0 0x55c42b72a960 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9dca2a9c107210b server_cookie=aef761f2aaa42842 in_seq=0 out_seq=0 2024-05-08T22:10:49.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.091+0000 7fcec8c00700 10 client.16098 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:10:49.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.097+0000 7fcec8c00700 1 -- 172.21.15.131:0/3576179135 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x55c42b5ffec0 con 0x55c42b7284a0 2024-05-08T22:10:49.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.097+0000 7fcec8c00700 10 client.16098 handle_command_reply: tid=0 2024-05-08T22:10:49.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.097+0000 7fcef95f8b80 1 client.16098 shutdown 2024-05-08T22:10:49.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.105+0000 7fcef95f8b80 1 -- 172.21.15.131:0/3576179135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c42b6bd720 msgr2=0x55c42b6c8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:49.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.105+0000 7fcef95f8b80 1 --2- 172.21.15.131:0/3576179135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c42b6bd720 0x55c42b6c8610 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0046e0 tx=0x7fcecc004750 comp rx=0 tx=0).stop 2024-05-08T22:10:49.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.105+0000 7fcef95f8b80 1 -- 172.21.15.131:0/3576179135 shutdown_connections 2024-05-08T22:10:49.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.105+0000 7fcef95f8b80 1 --2- 172.21.15.131:0/3576179135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c42b6c6610 0x55c42b6c6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:49.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.105+0000 7fcef95f8b80 1 --2- 172.21.15.131:0/3576179135 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55c42b7284a0 0x55c42b72a960 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:49.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.105+0000 7fcef95f8b80 1 --2- 172.21.15.131:0/3576179135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c42b6bce00 0x55c42b6c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:49.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.105+0000 7fcef95f8b80 1 --2- 172.21.15.131:0/3576179135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c42b6bd720 0x55c42b6c8610 secure :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0046e0 tx=0x7fcecc004750 comp rx=0 tx=0).stop 2024-05-08T22:10:49.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.105+0000 7fcef95f8b80 1 -- 172.21.15.131:0/3576179135 >> 172.21.15.131:0/3576179135 conn(0x55c42b615210 msgr2=0x55c42b6bf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:49.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.105+0000 7fcec8c00700 0 client.16098 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:10:49.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.106+0000 7fcef95f8b80 1 -- 172.21.15.131:0/3576179135 shutdown_connections 2024-05-08T22:10:49.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.106+0000 7fcef95f8b80 1 -- 172.21.15.131:0/3576179135 wait complete. 2024-05-08T22:10:49.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.107+0000 7fcef95f8b80 20 client.16098 trim_cache size 0 max 16384 2024-05-08T22:10:49.108 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:10:49.108 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "scrub active (28 inodes in the stack)", 2024-05-08T22:10:49.108 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-08T22:10:49.109 INFO:teuthology.orchestra.run.smithi131.stdout: "aa3822e7-6b1d-40c9-9246-37810abecb93": { 2024-05-08T22:10:49.109 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-08T22:10:49.109 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "aa3822e7-6b1d-40c9-9246-37810abecb93", 2024-05-08T22:10:49.109 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-08T22:10:49.109 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:10:49.109 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:10:49.109 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:10:49.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.133+0000 7fcee5a00700 1 -- 172.21.15.131:0/4021121773 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fcec40a14a0 msgr2=0x7fcec40a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:49.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.133+0000 7fcee5a00700 1 --2- 172.21.15.131:0/4021121773 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fcec40a14a0 0x7fcec40a3960 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcee0113430 tx=0x7fcecc00b410 comp rx=0 tx=0).stop 2024-05-08T22:10:49.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.133+0000 7fcee5a00700 1 -- 172.21.15.131:0/4021121773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee010f6a0 msgr2=0x7fcee0118c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:10:49.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.133+0000 7fcee5a00700 1 --2- 172.21.15.131:0/4021121773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee010f6a0 0x7fcee0118c30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fced800ba50 tx=0x7fced800bd60 comp rx=0 tx=0).stop 2024-05-08T22:10:49.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.144+0000 7fcee5a00700 1 -- 172.21.15.131:0/4021121773 shutdown_connections 2024-05-08T22:10:49.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.144+0000 7fcee5a00700 1 --2- 172.21.15.131:0/4021121773 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fcec40a14a0 0x7fcec40a3960 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:49.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.144+0000 7fcee5a00700 1 --2- 172.21.15.131:0/4021121773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcee011a780 0x7fcee0111e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:49.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.144+0000 7fcee5a00700 1 --2- 172.21.15.131:0/4021121773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee010f6a0 0x7fcee0118c30 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:49.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.144+0000 7fcee5a00700 1 --2- 172.21.15.131:0/4021121773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee010fb10 0x7fcee01191b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:10:49.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.144+0000 7fcee5a00700 1 -- 172.21.15.131:0/4021121773 >> 172.21.15.131:0/4021121773 conn(0x7fcee006bff0 msgr2=0x7fcee006f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:10:49.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.148+0000 7fcee5a00700 1 -- 172.21.15.131:0/4021121773 shutdown_connections 2024-05-08T22:10:49.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:10:49.148+0000 7fcee5a00700 1 -- 172.21.15.131:0/4021121773 wait complete. 2024-05-08T22:10:49.363 INFO:tasks.cephfs.filesystem:scrub status for tag:aa3822e7-6b1d-40c9-9246-37810abecb93 - {'path': '/', 'tag': 'aa3822e7-6b1d-40c9-9246-37810abecb93', 'options': 'recursive,force'} 2024-05-08T22:10:50.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:50 smithi131 ceph-mon[37260]: pgmap v109: 129 pgs: 129 active+clean; 386 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.7 MiB/s wr, 255 op/s 2024-05-08T22:10:50.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:50 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:10:50.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:50 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:10:51.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:50 smithi157 ceph-mon[77746]: pgmap v109: 129 pgs: 129 active+clean; 386 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.7 MiB/s wr, 255 op/s 2024-05-08T22:10:51.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:50 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:10:51.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:50 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:10:51.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:50 smithi132 ceph-mon[78180]: pgmap v109: 129 pgs: 129 active+clean; 386 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.7 MiB/s wr, 255 op/s 2024-05-08T22:10:51.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:50 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:10:51.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:50 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:10:51.894 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:51 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:10:51.894 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:51 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:10:52.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:51 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:10:52.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:51 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:10:52.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:51 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:10:52.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:51 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:10:52.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:52 smithi131 ceph-mon[37260]: pgmap v110: 129 pgs: 129 active+clean; 388 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 266 op/s 2024-05-08T22:10:53.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:52 smithi157 ceph-mon[77746]: pgmap v110: 129 pgs: 129 active+clean; 388 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 266 op/s 2024-05-08T22:10:53.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:52 smithi132 ceph-mon[78180]: pgmap v110: 129 pgs: 129 active+clean; 388 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 266 op/s 2024-05-08T22:10:54.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:54 smithi131 ceph-mon[37260]: pgmap v111: 129 pgs: 129 active+clean; 388 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 257 op/s 2024-05-08T22:10:55.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:54 smithi157 ceph-mon[77746]: pgmap v111: 129 pgs: 129 active+clean; 388 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 257 op/s 2024-05-08T22:10:55.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:54 smithi132 ceph-mon[78180]: pgmap v111: 129 pgs: 129 active+clean; 388 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 257 op/s 2024-05-08T22:10:55.245 INFO:tasks.workunit.client.0.smithi131.stdout: 87 494063 474 305134 279 276206 1046 2024-05-08T22:10:56.895 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:56 smithi131 ceph-mon[37260]: pgmap v112: 129 pgs: 129 active+clean; 455 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.8 MiB/s wr, 628 op/s 2024-05-08T22:10:57.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:56 smithi157 ceph-mon[77746]: pgmap v112: 129 pgs: 129 active+clean; 455 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.8 MiB/s wr, 628 op/s 2024-05-08T22:10:57.089 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:56 smithi132 ceph-mon[78180]: pgmap v112: 129 pgs: 129 active+clean; 455 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.8 MiB/s wr, 628 op/s 2024-05-08T22:10:58.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:10:58 smithi131 ceph-mon[37260]: pgmap v113: 129 pgs: 129 active+clean; 467 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.7 MiB/s wr, 674 op/s 2024-05-08T22:10:59.025 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:10:58 smithi157 ceph-mon[77746]: pgmap v113: 129 pgs: 129 active+clean; 467 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.7 MiB/s wr, 674 op/s 2024-05-08T22:10:59.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:10:58 smithi132 ceph-mon[78180]: pgmap v113: 129 pgs: 129 active+clean; 467 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.7 MiB/s wr, 674 op/s 2024-05-08T22:11:00.895 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:00 smithi131 ceph-mon[37260]: pgmap v114: 129 pgs: 129 active+clean; 473 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 729 op/s 2024-05-08T22:11:01.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:00 smithi157 ceph-mon[77746]: pgmap v114: 129 pgs: 129 active+clean; 473 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 729 op/s 2024-05-08T22:11:01.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:00 smithi132 ceph-mon[78180]: pgmap v114: 129 pgs: 129 active+clean; 473 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 729 op/s 2024-05-08T22:11:02.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:02 smithi131 ceph-mon[37260]: pgmap v115: 129 pgs: 129 active+clean; 473 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.2 MiB/s wr, 584 op/s 2024-05-08T22:11:03.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:02 smithi157 ceph-mon[77746]: pgmap v115: 129 pgs: 129 active+clean; 473 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.2 MiB/s wr, 584 op/s 2024-05-08T22:11:03.089 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:02 smithi132 ceph-mon[78180]: pgmap v115: 129 pgs: 129 active+clean; 473 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.2 MiB/s wr, 584 op/s 2024-05-08T22:11:04.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:04 smithi131 ceph-mon[37260]: pgmap v116: 129 pgs: 129 active+clean; 472 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.9 MiB/s wr, 565 op/s 2024-05-08T22:11:05.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:04 smithi157 ceph-mon[77746]: pgmap v116: 129 pgs: 129 active+clean; 472 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.9 MiB/s wr, 565 op/s 2024-05-08T22:11:05.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:04 smithi132 ceph-mon[78180]: pgmap v116: 129 pgs: 129 active+clean; 472 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.9 MiB/s wr, 565 op/s 2024-05-08T22:11:05.249 INFO:tasks.workunit.client.0.smithi131.stdout: 92 653319 387 400274 393 357639 1242 2024-05-08T22:11:07.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:06 smithi157 ceph-mon[77746]: pgmap v117: 129 pgs: 129 active+clean; 481 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.9 MiB/s wr, 599 op/s 2024-05-08T22:11:07.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:06 smithi132 ceph-mon[78180]: pgmap v117: 129 pgs: 129 active+clean; 481 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.9 MiB/s wr, 599 op/s 2024-05-08T22:11:07.146 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:06 smithi131 ceph-mon[37260]: pgmap v117: 129 pgs: 129 active+clean; 481 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.9 MiB/s wr, 599 op/s 2024-05-08T22:11:09.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:08 smithi157 ceph-mon[77746]: pgmap v118: 129 pgs: 129 active+clean; 485 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 237 op/s 2024-05-08T22:11:09.089 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:08 smithi132 ceph-mon[78180]: pgmap v118: 129 pgs: 129 active+clean; 485 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 237 op/s 2024-05-08T22:11:09.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:08 smithi131 ceph-mon[37260]: pgmap v118: 129 pgs: 129 active+clean; 485 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 237 op/s 2024-05-08T22:11:11.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:10 smithi157 ceph-mon[77746]: pgmap v119: 129 pgs: 129 active+clean; 496 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 153 op/s 2024-05-08T22:11:11.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:10 smithi132 ceph-mon[78180]: pgmap v119: 129 pgs: 129 active+clean; 496 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 153 op/s 2024-05-08T22:11:11.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:10 smithi131 ceph-mon[37260]: pgmap v119: 129 pgs: 129 active+clean; 496 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 153 op/s 2024-05-08T22:11:13.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:12 smithi157 ceph-mon[77746]: pgmap v120: 129 pgs: 129 active+clean; 496 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 85 op/s 2024-05-08T22:11:13.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:12 smithi132 ceph-mon[78180]: pgmap v120: 129 pgs: 129 active+clean; 496 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 85 op/s 2024-05-08T22:11:13.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:12 smithi131 ceph-mon[37260]: pgmap v120: 129 pgs: 129 active+clean; 496 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 85 op/s 2024-05-08T22:11:15.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:14 smithi157 ceph-mon[77746]: pgmap v121: 129 pgs: 129 active+clean; 500 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 89 op/s 2024-05-08T22:11:15.089 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:14 smithi132 ceph-mon[78180]: pgmap v121: 129 pgs: 129 active+clean; 500 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 89 op/s 2024-05-08T22:11:15.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:14 smithi131 ceph-mon[37260]: pgmap v121: 129 pgs: 129 active+clean; 500 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 89 op/s 2024-05-08T22:11:15.249 INFO:tasks.workunit.client.0.smithi131.stdout: 100 648609 355 409610 273 376759 1500 2024-05-08T22:11:17.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:16 smithi157 ceph-mon[77746]: pgmap v122: 129 pgs: 129 active+clean; 510 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 95 op/s 2024-05-08T22:11:17.089 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:16 smithi132 ceph-mon[78180]: pgmap v122: 129 pgs: 129 active+clean; 510 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 95 op/s 2024-05-08T22:11:17.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:16 smithi131 ceph-mon[37260]: pgmap v122: 129 pgs: 129 active+clean; 510 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 95 op/s 2024-05-08T22:11:19.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:18 smithi157 ceph-mon[77746]: pgmap v123: 129 pgs: 129 active+clean; 510 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 80 op/s 2024-05-08T22:11:19.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:18 smithi132 ceph-mon[78180]: pgmap v123: 129 pgs: 129 active+clean; 510 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 80 op/s 2024-05-08T22:11:19.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:18 smithi131 ceph-mon[37260]: pgmap v123: 129 pgs: 129 active+clean; 510 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 80 op/s 2024-05-08T22:11:19.364 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:11:20.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.672+0000 7fa4c9c00700 1 -- 172.21.15.131:0/2227578395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4c411a780 msgr2=0x7fa4c411cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:11:20.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.672+0000 7fa4c9c00700 1 --2- 172.21.15.131:0/2227578395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4c411a780 0x7fa4c411cb70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa4b001ac00 tx=0x7fa4b001af10 comp rx=0 tx=0).stop 2024-05-08T22:11:20.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.681+0000 7fa4c9c00700 1 -- 172.21.15.131:0/2227578395 shutdown_connections 2024-05-08T22:11:20.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.681+0000 7fa4c9c00700 1 --2- 172.21.15.131:0/2227578395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4c411a780 0x7fa4c411cb70 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:20.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.681+0000 7fa4c9c00700 1 --2- 172.21.15.131:0/2227578395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c410f6a0 0x7fa4c4107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:20.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.681+0000 7fa4c9c00700 1 --2- 172.21.15.131:0/2227578395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4c410fb10 0x7fa4c41084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:20.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.681+0000 7fa4c9c00700 1 -- 172.21.15.131:0/2227578395 >> 172.21.15.131:0/2227578395 conn(0x7fa4c406bff0 msgr2=0x7fa4c406c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:11:20.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.689+0000 7fa4c9c00700 1 -- 172.21.15.131:0/2227578395 shutdown_connections 2024-05-08T22:11:20.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.693+0000 7fa4c9c00700 1 -- 172.21.15.131:0/2227578395 wait complete. 2024-05-08T22:11:20.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.694+0000 7fa4c9c00700 1 Processor -- start 2024-05-08T22:11:20.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.695+0000 7fa4c9c00700 1 -- start start 2024-05-08T22:11:20.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.695+0000 7fa4c9c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c410f6a0 0x7fa4c4115f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:20.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.695+0000 7fa4c9c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4c410fb10 0x7fa4c4116440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:20.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.695+0000 7fa4c9c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4c411a780 0x7fa4c41a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:20.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.696+0000 7fa4c9c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c41a2af0 con 0x7fa4c411a780 2024-05-08T22:11:20.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.696+0000 7fa4c9c00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c41a2c60 con 0x7fa4c410fb10 2024-05-08T22:11:20.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.696+0000 7fa4c9c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c41a2e20 con 0x7fa4c410f6a0 2024-05-08T22:11:20.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.696+0000 7fa4c2a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c410f6a0 0x7fa4c4115f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:11:20.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.696+0000 7fa4c2a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c410f6a0 0x7fa4c4115f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:58096/0 (socket says 172.21.15.131:58096) 2024-05-08T22:11:20.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.696+0000 7fa4c2a00700 1 -- 172.21.15.131:0/1148023641 learned_addr learned my addr 172.21.15.131:0/1148023641 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:11:20.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.696+0000 7fa4c3400700 1 --2- 172.21.15.131:0/1148023641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4c411a780 0x7fa4c41a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:11:20.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.697+0000 7fa4c2a00700 1 -- 172.21.15.131:0/1148023641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4c410fb10 msgr2=0x7fa4c4116440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T22:11:20.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.697+0000 7fa4c2a00700 1 --2- 172.21.15.131:0/1148023641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4c410fb10 0x7fa4c4116440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:20.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.697+0000 7fa4c2a00700 1 -- 172.21.15.131:0/1148023641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4c411a780 msgr2=0x7fa4c41a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:11:20.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.697+0000 7fa4c2a00700 1 --2- 172.21.15.131:0/1148023641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4c411a780 0x7fa4c41a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:20.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.697+0000 7fa4c2a00700 1 -- 172.21.15.131:0/1148023641 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b001a600 con 0x7fa4c410f6a0 2024-05-08T22:11:20.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.697+0000 7fa4c3400700 1 --2- 172.21.15.131:0/1148023641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4c411a780 0x7fa4c41a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:11:20.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.701+0000 7fa4c2a00700 1 --2- 172.21.15.131:0/1148023641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c410f6a0 0x7fa4c4115f00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa4b400b550 tx=0x7fa4b400b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:11:20.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.703+0000 7fa4bb400700 1 -- 172.21.15.131:0/1148023641 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4011830 con 0x7fa4c410f6a0 2024-05-08T22:11:20.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.703+0000 7fa4bb400700 1 -- 172.21.15.131:0/1148023641 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4b4011e50 con 0x7fa4c410f6a0 2024-05-08T22:11:20.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.703+0000 7fa4bb400700 1 -- 172.21.15.131:0/1148023641 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b400f2e0 con 0x7fa4c410f6a0 2024-05-08T22:11:20.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.712+0000 7fa4c9c00700 1 -- 172.21.15.131:0/1148023641 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c41a3100 con 0x7fa4c410f6a0 2024-05-08T22:11:20.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.713+0000 7fa4c9c00700 1 -- 172.21.15.131:0/1148023641 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c41aa660 con 0x7fa4c410f6a0 2024-05-08T22:11:20.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.732+0000 7fa4bb400700 1 -- 172.21.15.131:0/1148023641 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fa4b4011990 con 0x7fa4c410f6a0 2024-05-08T22:11:20.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.733+0000 7fa4dd637b80 1 Processor -- start 2024-05-08T22:11:20.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.734+0000 7fa4bb400700 1 --2- 172.21.15.131:0/1148023641 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa4a80a14a0 0x7fa4a80a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:20.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.734+0000 7fa4bb400700 1 -- 172.21.15.131:0/1148023641 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fa4b40cc980 con 0x7fa4c410f6a0 2024-05-08T22:11:20.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.742+0000 7fa4c3400700 1 --2- 172.21.15.131:0/1148023641 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa4a80a14a0 0x7fa4a80a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:11:20.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.743+0000 7fa4c3400700 1 --2- 172.21.15.131:0/1148023641 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa4a80a14a0 0x7fa4a80a3960 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa4b001ab50 tx=0x7fa4b002a040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:11:20.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.748+0000 7fa4dd637b80 1 -- start start 2024-05-08T22:11:20.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.748+0000 7fa4dd637b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556bdb2ffe00 0x556bdb3001e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:20.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.748+0000 7fa4dd637b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556bdb300720 0x556bdb2fa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:20.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.749+0000 7fa4dd637b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556bdb2fa890 0x556bdb2facd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:20.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.749+0000 7fa4dd637b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdb251150 con 0x556bdb2ffe00 2024-05-08T22:11:20.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.749+0000 7fa4dd637b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdb24eb10 con 0x556bdb300720 2024-05-08T22:11:20.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.749+0000 7fa4dd637b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdb24a400 con 0x556bdb2fa890 2024-05-08T22:11:20.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.751+0000 7fa4c3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556bdb2fa890 0x556bdb2facd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:11:20.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.751+0000 7fa4c3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556bdb2fa890 0x556bdb2facd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:58112/0 (socket says 172.21.15.131:58112) 2024-05-08T22:11:20.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.751+0000 7fa4c3400700 1 -- 172.21.15.131:0/1499938605 learned_addr learned my addr 172.21.15.131:0/1499938605 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:11:20.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.751+0000 7fa4c3400700 1 -- 172.21.15.131:0/1499938605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556bdb300720 msgr2=0x556bdb2fa350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:11:20.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.752+0000 7fa4c3400700 1 --2- 172.21.15.131:0/1499938605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556bdb300720 0x556bdb2fa350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:20.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.752+0000 7fa4c3400700 1 -- 172.21.15.131:0/1499938605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556bdb2ffe00 msgr2=0x556bdb3001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:11:20.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.752+0000 7fa4c3400700 1 --2- 172.21.15.131:0/1499938605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556bdb2ffe00 0x556bdb3001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:20.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.752+0000 7fa4c3400700 1 -- 172.21.15.131:0/1499938605 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bdb0cb3b0 con 0x556bdb2fa890 2024-05-08T22:11:20.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.752+0000 7fa4c3400700 1 --2- 172.21.15.131:0/1499938605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556bdb2fa890 0x556bdb2facd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa4b001ca60 tx=0x7fa4b0030000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=261c70a611de2c60 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:11:20.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.756+0000 7fa4afe00700 1 -- 172.21.15.131:0/1499938605 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b001ce20 con 0x556bdb2fa890 2024-05-08T22:11:20.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.756+0000 7fa4afe00700 1 -- 172.21.15.131:0/1499938605 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4b002e040 con 0x556bdb2fa890 2024-05-08T22:11:20.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.756+0000 7fa4afe00700 1 -- 172.21.15.131:0/1499938605 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0034670 con 0x556bdb2fa890 2024-05-08T22:11:20.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.760+0000 7fa4dd637b80 1 -- 172.21.15.131:0/1499938605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556bdb2fa890 msgr2=0x556bdb2facd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:11:20.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.761+0000 7fa4dd637b80 1 --2- 172.21.15.131:0/1499938605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556bdb2fa890 0x556bdb2facd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa4b001ca60 tx=0x7fa4b0030000 comp rx=0 tx=0).stop 2024-05-08T22:11:20.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.776+0000 7fa4dd637b80 1 -- 172.21.15.131:0/1499938605 shutdown_connections 2024-05-08T22:11:20.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.776+0000 7fa4dd637b80 1 --2- 172.21.15.131:0/1499938605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556bdb2fa890 0x556bdb2facd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:20.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.776+0000 7fa4dd637b80 1 --2- 172.21.15.131:0/1499938605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556bdb2ffe00 0x556bdb3001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:20.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.776+0000 7fa4dd637b80 1 --2- 172.21.15.131:0/1499938605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556bdb300720 0x556bdb2fa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:20.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.776+0000 7fa4dd637b80 1 -- 172.21.15.131:0/1499938605 >> 172.21.15.131:0/1499938605 conn(0x556bdb258210 msgr2=0x556bdb2f4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:11:20.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.791+0000 7fa4dd637b80 1 -- 172.21.15.131:0/1499938605 shutdown_connections 2024-05-08T22:11:20.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.792+0000 7fa4dd637b80 1 -- 172.21.15.131:0/1499938605 wait complete. 2024-05-08T22:11:20.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.793+0000 7fa4dd637b80 1 -- start start 2024-05-08T22:11:20.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.793+0000 7fa4dd637b80 1 Processor -- start 2024-05-08T22:11:20.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.794+0000 7fa4dd637b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556bdb2fa890 0x556bdb303d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:20.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.794+0000 7fa4dd637b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556bdb2ffe00 0x556bdb304250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:20.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.794+0000 7fa4c2a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556bdb2fa890 0x556bdb303d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:11:20.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.794+0000 7fa4c2a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556bdb2fa890 0x556bdb303d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:55188/0 (socket says 172.21.15.131:55188) 2024-05-08T22:11:20.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.798+0000 7fa4c2a00700 1 -- 172.21.15.131:0/2417566376 learned_addr learned my addr 172.21.15.131:0/2417566376 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:11:20.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.798+0000 7fa4c2000700 1 --2- 172.21.15.131:0/2417566376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556bdb2ffe00 0x556bdb304250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:11:20.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.806+0000 7fa4dd637b80 1 --2- 172.21.15.131:0/2417566376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556bdb300720 0x556bdb304790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:20.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.807+0000 7fa4dd637b80 1 -- 172.21.15.131:0/2417566376 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdb1e6560 con 0x556bdb2ffe00 2024-05-08T22:11:20.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.807+0000 7fa4dd637b80 1 -- 172.21.15.131:0/2417566376 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdb1e6200 con 0x556bdb2fa890 2024-05-08T22:11:20.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.807+0000 7fa4dd637b80 1 -- 172.21.15.131:0/2417566376 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdb0cd6a0 con 0x556bdb300720 2024-05-08T22:11:20.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.807+0000 7fa4dd637b80 10 client.0 _ll_register_callbacks cb 0x556bdb1efb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:11:20.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.808+0000 7fa4c3400700 1 --2- 172.21.15.131:0/2417566376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556bdb300720 0x556bdb304790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:11:20.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.808+0000 7fa4c2a00700 1 -- 172.21.15.131:0/2417566376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556bdb300720 msgr2=0x556bdb304790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:11:20.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.808+0000 7fa4c2a00700 1 --2- 172.21.15.131:0/2417566376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556bdb300720 0x556bdb304790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:20.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.808+0000 7fa4c2a00700 1 -- 172.21.15.131:0/2417566376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556bdb2ffe00 msgr2=0x556bdb304250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:11:20.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.808+0000 7fa4c2a00700 1 --2- 172.21.15.131:0/2417566376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556bdb2ffe00 0x556bdb304250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:20.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.808+0000 7fa4c2a00700 1 -- 172.21.15.131:0/2417566376 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c411d460 con 0x556bdb2fa890 2024-05-08T22:11:20.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.808+0000 7fa4c2000700 1 --2- 172.21.15.131:0/2417566376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556bdb2ffe00 0x556bdb304250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:11:20.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.810+0000 7fa4c2a00700 1 --2- 172.21.15.131:0/2417566376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556bdb2fa890 0x556bdb303d10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4096220 tx=0x7fa4b40991e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7aa445ef837ce2b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:11:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.838+0000 7fa4acc00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:11:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.838+0000 7fa4acc00700 1 -- 172.21.15.131:0/2417566376 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4010380 con 0x556bdb2fa890 2024-05-08T22:11:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.838+0000 7fa4acc00700 1 -- 172.21.15.131:0/2417566376 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fa4b40997b0 con 0x556bdb2fa890 2024-05-08T22:11:20.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.849+0000 7fa4dd637b80 1 -- 172.21.15.131:0/2417566376 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556bdb1a3270 con 0x556bdb2fa890 2024-05-08T22:11:20.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.850+0000 7fa4acc00700 1 -- 172.21.15.131:0/2417566376 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4b409d5e0 con 0x556bdb2fa890 2024-05-08T22:11:20.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.861+0000 7fa4dd637b80 10 client.26138 fetch_fsmap learned FSMap version 26 2024-05-08T22:11:20.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.861+0000 7fa4dd637b80 1 -- 172.21.15.131:0/2417566376 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556bdb077b60 con 0x556bdb2fa890 2024-05-08T22:11:20.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.877+0000 7fa4acc00700 1 -- 172.21.15.131:0/2417566376 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556bdb1a3270 con 0x556bdb2fa890 2024-05-08T22:11:20.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.880+0000 7fa4acc00700 1 -- 172.21.15.131:0/2417566376 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa4b409d1d0 con 0x556bdb2fa890 2024-05-08T22:11:20.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.880+0000 7fa4dd637b80 10 client.26138 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:11:20.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.880+0000 7fa4dd637b80 10 client.26138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:11:20.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.880+0000 7fa4dd637b80 20 client.26138 populate_metadata read hostname 'smithi131' 2024-05-08T22:11:20.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.880+0000 7fa4dd637b80 1 --2- 172.21.15.131:0/2417566376 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x556bdb314440 0x556bdb316860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:20.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.880+0000 7fa4dd637b80 4 client.26138 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:11:20.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.880+0000 7fa4dd637b80 1 -- 172.21.15.131:0/2417566376 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556bdb2c9810 con 0x556bdb314440 2024-05-08T22:11:20.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.882+0000 7fa4c2000700 1 --2- 172.21.15.131:0/2417566376 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x556bdb314440 0x556bdb316860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:11:20.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.891+0000 7fa4c2000700 1 --2- 172.21.15.131:0/2417566376 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x556bdb314440 0x556bdb316860 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1beba419a7171bcf server_cookie=80edc35b6fb45918 in_seq=0 out_seq=0 2024-05-08T22:11:20.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.893+0000 7fa4acc00700 10 client.26138 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:11:20.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.906+0000 7fa4acc00700 1 -- 172.21.15.131:0/2417566376 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556bdb2c9810 con 0x556bdb314440 2024-05-08T22:11:20.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.906+0000 7fa4acc00700 10 client.26138 handle_command_reply: tid=0 2024-05-08T22:11:20.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.906+0000 7fa4dd637b80 1 client.26138 shutdown 2024-05-08T22:11:20.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.912+0000 7fa4dd637b80 1 -- 172.21.15.131:0/2417566376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556bdb2fa890 msgr2=0x556bdb303d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:11:20.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.912+0000 7fa4dd637b80 1 --2- 172.21.15.131:0/2417566376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556bdb2fa890 0x556bdb303d10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4096220 tx=0x7fa4b40991e0 comp rx=0 tx=0).stop 2024-05-08T22:11:20.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.914+0000 7fa4dd637b80 1 -- 172.21.15.131:0/2417566376 shutdown_connections 2024-05-08T22:11:20.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.914+0000 7fa4dd637b80 1 --2- 172.21.15.131:0/2417566376 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x556bdb314440 0x556bdb316860 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:20.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.914+0000 7fa4dd637b80 1 --2- 172.21.15.131:0/2417566376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556bdb300720 0x556bdb304790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:20.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.914+0000 7fa4dd637b80 1 --2- 172.21.15.131:0/2417566376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556bdb2fa890 0x556bdb303d10 secure :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4096220 tx=0x7fa4b40991e0 comp rx=0 tx=0).stop 2024-05-08T22:11:20.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.914+0000 7fa4dd637b80 1 --2- 172.21.15.131:0/2417566376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556bdb2ffe00 0x556bdb304250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:20.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.914+0000 7fa4dd637b80 1 -- 172.21.15.131:0/2417566376 >> 172.21.15.131:0/2417566376 conn(0x556bdb258210 msgr2=0x556bdb2f45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:11:20.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.914+0000 7fa4acc00700 0 client.26138 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:11:20.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.941+0000 7fa4dd637b80 1 -- 172.21.15.131:0/2417566376 shutdown_connections 2024-05-08T22:11:20.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.941+0000 7fa4dd637b80 1 -- 172.21.15.131:0/2417566376 wait complete. 2024-05-08T22:11:20.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:20.960+0000 7fa4dd637b80 20 client.26138 trim_cache size 0 max 16384 2024-05-08T22:11:21.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:20 smithi157 ceph-mon[77746]: pgmap v124: 129 pgs: 129 active+clean; 518 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 304 op/s 2024-05-08T22:11:21.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:20 smithi132 ceph-mon[78180]: pgmap v124: 129 pgs: 129 active+clean; 518 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 304 op/s 2024-05-08T22:11:21.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:20 smithi131 ceph-mon[37260]: pgmap v124: 129 pgs: 129 active+clean; 518 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 304 op/s 2024-05-08T22:11:21.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.597+0000 7fa4dd637b80 1 Processor -- start 2024-05-08T22:11:21.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.634+0000 7fa4dd637b80 1 -- start start 2024-05-08T22:11:21.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.635+0000 7fa4dd637b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556bdb2ffe00 0x556bdb2fa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:21.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.635+0000 7fa4dd637b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556bdb300720 0x556bdb2f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:21.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.635+0000 7fa4dd637b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556bdb2fe5b0 0x556bdb3045b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:21.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.635+0000 7fa4dd637b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdb2f6a50 con 0x556bdb2ffe00 2024-05-08T22:11:21.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.635+0000 7fa4dd637b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdb2f5ca0 con 0x556bdb2fe5b0 2024-05-08T22:11:21.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.635+0000 7fa4dd637b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdb2fda10 con 0x556bdb300720 2024-05-08T22:11:21.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.638+0000 7fa4c3400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556bdb2fe5b0 0x556bdb3045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:11:21.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.638+0000 7fa4c3400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556bdb2fe5b0 0x556bdb3045b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:55196/0 (socket says 172.21.15.131:55196) 2024-05-08T22:11:21.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.638+0000 7fa4c3400700 1 -- 172.21.15.131:0/3002386117 learned_addr learned my addr 172.21.15.131:0/3002386117 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:11:21.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.638+0000 7fa4c2000700 1 --2- 172.21.15.131:0/3002386117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556bdb300720 0x556bdb2f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:11:21.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.639+0000 7fa4c3400700 1 -- 172.21.15.131:0/3002386117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556bdb300720 msgr2=0x556bdb2f4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:11:21.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.639+0000 7fa4c3400700 1 --2- 172.21.15.131:0/3002386117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556bdb300720 0x556bdb2f4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:21.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.639+0000 7fa4c3400700 1 -- 172.21.15.131:0/3002386117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556bdb2ffe00 msgr2=0x556bdb2fa350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:11:21.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.639+0000 7fa4c3400700 1 --2- 172.21.15.131:0/3002386117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556bdb2ffe00 0x556bdb2fa350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:21.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.639+0000 7fa4c3400700 1 -- 172.21.15.131:0/3002386117 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bdb30fe90 con 0x556bdb2fe5b0 2024-05-08T22:11:21.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.666+0000 7fa4c3400700 1 --2- 172.21.15.131:0/3002386117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556bdb2fe5b0 0x556bdb3045b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa4b001af50 tx=0x7fa4b0005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1488a874ddb4e789 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:11:21.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.667+0000 7fa4b8c00700 1 -- 172.21.15.131:0/3002386117 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b003fcc0 con 0x556bdb2fe5b0 2024-05-08T22:11:21.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.667+0000 7fa4b8c00700 1 -- 172.21.15.131:0/3002386117 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4b0003d50 con 0x556bdb2fe5b0 2024-05-08T22:11:21.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.667+0000 7fa4b8c00700 1 -- 172.21.15.131:0/3002386117 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0048830 con 0x556bdb2fe5b0 2024-05-08T22:11:21.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.681+0000 7fa4dd637b80 1 -- 172.21.15.131:0/3002386117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556bdb2fe5b0 msgr2=0x556bdb3045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:11:21.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.681+0000 7fa4dd637b80 1 --2- 172.21.15.131:0/3002386117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556bdb2fe5b0 0x556bdb3045b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa4b001af50 tx=0x7fa4b0005a00 comp rx=0 tx=0).stop 2024-05-08T22:11:21.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.710+0000 7fa4dd637b80 1 -- 172.21.15.131:0/3002386117 shutdown_connections 2024-05-08T22:11:21.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.710+0000 7fa4dd637b80 1 --2- 172.21.15.131:0/3002386117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556bdb2fe5b0 0x556bdb3045b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:21.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.710+0000 7fa4dd637b80 1 --2- 172.21.15.131:0/3002386117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556bdb2ffe00 0x556bdb2fa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:21.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.710+0000 7fa4dd637b80 1 --2- 172.21.15.131:0/3002386117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556bdb300720 0x556bdb2f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:21.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.710+0000 7fa4dd637b80 1 -- 172.21.15.131:0/3002386117 >> 172.21.15.131:0/3002386117 conn(0x556bdb258210 msgr2=0x556bdb2fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:11:21.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.714+0000 7fa4dd637b80 1 -- 172.21.15.131:0/3002386117 shutdown_connections 2024-05-08T22:11:21.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.714+0000 7fa4dd637b80 1 -- 172.21.15.131:0/3002386117 wait complete. 2024-05-08T22:11:21.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.715+0000 7fa4dd637b80 1 -- start start 2024-05-08T22:11:21.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.717+0000 7fa4dd637b80 1 Processor -- start 2024-05-08T22:11:21.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.744+0000 7fa4dd637b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556bdb2fe5b0 0x556bdb30b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:21.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.744+0000 7fa4dd637b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556bdb2ffe00 0x556bdb30b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:21.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.744+0000 7fa4dd637b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556bdb300720 0x556bdb2fb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:21.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.744+0000 7fa4dd637b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdb2fd1e0 con 0x556bdb2ffe00 2024-05-08T22:11:21.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.744+0000 7fa4dd637b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdb2fbe70 con 0x556bdb2fe5b0 2024-05-08T22:11:21.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.744+0000 7fa4dd637b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdb2fc030 con 0x556bdb300720 2024-05-08T22:11:21.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.744+0000 7fa4dd637b80 10 client.0 _ll_register_callbacks cb 0x556bdb1efb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:11:21.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.749+0000 7fa4c3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556bdb300720 0x556bdb2fb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:11:21.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.749+0000 7fa4c3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556bdb300720 0x556bdb2fb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:58120/0 (socket says 172.21.15.131:58120) 2024-05-08T22:11:21.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.750+0000 7fa4c2a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556bdb2fe5b0 0x556bdb30b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:11:21.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.750+0000 7fa4c2a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556bdb2fe5b0 0x556bdb30b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:55210/0 (socket says 172.21.15.131:55210) 2024-05-08T22:11:21.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.750+0000 7fa4c2a00700 1 -- 172.21.15.131:0/2052135157 learned_addr learned my addr 172.21.15.131:0/2052135157 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:11:21.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.750+0000 7fa4c3400700 1 -- 172.21.15.131:0/2052135157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556bdb2fe5b0 msgr2=0x556bdb30b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:11:21.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.750+0000 7fa4c3400700 1 --2- 172.21.15.131:0/2052135157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556bdb2fe5b0 0x556bdb30b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:21.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.751+0000 7fa4c3400700 1 -- 172.21.15.131:0/2052135157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556bdb2ffe00 msgr2=0x556bdb30b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:11:21.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.751+0000 7fa4c3400700 1 --2- 172.21.15.131:0/2052135157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556bdb2ffe00 0x556bdb30b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:21.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.751+0000 7fa4c3400700 1 -- 172.21.15.131:0/2052135157 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556bdb0cb3b0 con 0x556bdb300720 2024-05-08T22:11:21.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.765+0000 7fa4c3400700 1 --2- 172.21.15.131:0/2052135157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556bdb300720 0x556bdb2fb930 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa4b001ca60 tx=0x7fa4b001cfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e58f5554b41ca588 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:11:21.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.768+0000 7fa4acc00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:11:21.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.789+0000 7fa4acc00700 1 -- 172.21.15.131:0/2052135157 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b003f650 con 0x556bdb300720 2024-05-08T22:11:21.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.789+0000 7fa4acc00700 1 -- 172.21.15.131:0/2052135157 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fa4b003f7b0 con 0x556bdb300720 2024-05-08T22:11:21.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.810+0000 7fa4dd637b80 1 -- 172.21.15.131:0/2052135157 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4b002e040 con 0x556bdb300720 2024-05-08T22:11:21.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.859+0000 7fa4acc00700 1 -- 172.21.15.131:0/2052135157 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4b00484d0 con 0x556bdb300720 2024-05-08T22:11:21.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.859+0000 7fa4dd637b80 10 client.26185 fetch_fsmap learned FSMap version 26 2024-05-08T22:11:21.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.859+0000 7fa4dd637b80 1 -- 172.21.15.131:0/2052135157 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556bdb210560 con 0x556bdb300720 2024-05-08T22:11:21.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.859+0000 7fa4acc00700 1 -- 172.21.15.131:0/2052135157 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4b007a030 con 0x556bdb300720 2024-05-08T22:11:21.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.860+0000 7fa4acc00700 1 -- 172.21.15.131:0/2052135157 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa4b0048630 con 0x556bdb300720 2024-05-08T22:11:21.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.860+0000 7fa4dd637b80 10 client.26185 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:11:21.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.860+0000 7fa4dd637b80 10 client.26185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:11:21.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.860+0000 7fa4dd637b80 20 client.26185 populate_metadata read hostname 'smithi131' 2024-05-08T22:11:21.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.861+0000 7fa4dd637b80 1 --2- 172.21.15.131:0/2052135157 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x556bdb36b270 0x556bdb36d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:21.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.861+0000 7fa4dd637b80 4 client.26185 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:11:21.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.861+0000 7fa4dd637b80 1 -- 172.21.15.131:0/2052135157 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556bdb242ec0 con 0x556bdb36b270 2024-05-08T22:11:21.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.877+0000 7fa4c2a00700 1 --2- 172.21.15.131:0/2052135157 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x556bdb36b270 0x556bdb36d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:11:21.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.883+0000 7fa4c2a00700 1 --2- 172.21.15.131:0/2052135157 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x556bdb36b270 0x556bdb36d660 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74d361acf0178316 server_cookie=2c6eda9191e725a in_seq=0 out_seq=0 2024-05-08T22:11:21.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.913+0000 7fa4acc00700 10 client.26185 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:11:21.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.913+0000 7fa4acc00700 1 -- 172.21.15.131:0/2052135157 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x556bdb242ec0 con 0x556bdb36b270 2024-05-08T22:11:21.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.913+0000 7fa4acc00700 10 client.26185 handle_command_reply: tid=0 2024-05-08T22:11:21.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.913+0000 7fa4dd637b80 1 client.26185 shutdown 2024-05-08T22:11:21.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.914+0000 7fa4dd637b80 1 -- 172.21.15.131:0/2052135157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556bdb300720 msgr2=0x556bdb2fb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:11:21.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.914+0000 7fa4dd637b80 1 --2- 172.21.15.131:0/2052135157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556bdb300720 0x556bdb2fb930 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa4b001ca60 tx=0x7fa4b001cfd0 comp rx=0 tx=0).stop 2024-05-08T22:11:21.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.918+0000 7fa4dd637b80 1 -- 172.21.15.131:0/2052135157 shutdown_connections 2024-05-08T22:11:21.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.918+0000 7fa4dd637b80 1 --2- 172.21.15.131:0/2052135157 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x556bdb36b270 0x556bdb36d660 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:21.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.918+0000 7fa4dd637b80 1 --2- 172.21.15.131:0/2052135157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556bdb300720 0x556bdb2fb930 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:21.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.918+0000 7fa4dd637b80 1 --2- 172.21.15.131:0/2052135157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556bdb2fe5b0 0x556bdb30b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:21.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.918+0000 7fa4dd637b80 1 --2- 172.21.15.131:0/2052135157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556bdb2ffe00 0x556bdb30b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:21.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.918+0000 7fa4dd637b80 1 -- 172.21.15.131:0/2052135157 >> 172.21.15.131:0/2052135157 conn(0x556bdb258210 msgr2=0x556bdb302540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:11:21.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.918+0000 7fa4acc00700 0 client.26185 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:11:21.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.984+0000 7fa4dd637b80 1 -- 172.21.15.131:0/2052135157 shutdown_connections 2024-05-08T22:11:21.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.992+0000 7fa4dd637b80 1 -- 172.21.15.131:0/2052135157 wait complete. 2024-05-08T22:11:22.000 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:11:22.000 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "scrub active (5168 inodes in the stack)", 2024-05-08T22:11:22.000 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-08T22:11:22.000 INFO:teuthology.orchestra.run.smithi131.stdout: "aa3822e7-6b1d-40c9-9246-37810abecb93": { 2024-05-08T22:11:22.001 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-08T22:11:22.001 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "aa3822e7-6b1d-40c9-9246-37810abecb93", 2024-05-08T22:11:22.001 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-08T22:11:22.001 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:11:22.001 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:11:22.001 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:11:22.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:21.999+0000 7fa4dd637b80 20 client.26185 trim_cache size 0 max 16384 2024-05-08T22:11:22.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:22.003+0000 7fa4c9c00700 1 -- 172.21.15.131:0/1148023641 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa4a80a14a0 msgr2=0x7fa4a80a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:11:22.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:22.003+0000 7fa4c9c00700 1 --2- 172.21.15.131:0/1148023641 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa4a80a14a0 0x7fa4a80a3960 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa4b001ab50 tx=0x7fa4b002a040 comp rx=0 tx=0).stop 2024-05-08T22:11:22.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:22.003+0000 7fa4c9c00700 1 -- 172.21.15.131:0/1148023641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c410f6a0 msgr2=0x7fa4c4115f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:11:22.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:22.003+0000 7fa4c9c00700 1 --2- 172.21.15.131:0/1148023641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c410f6a0 0x7fa4c4115f00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa4b400b550 tx=0x7fa4b400b860 comp rx=0 tx=0).stop 2024-05-08T22:11:22.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:22.004+0000 7fa4c9c00700 1 -- 172.21.15.131:0/1148023641 shutdown_connections 2024-05-08T22:11:22.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:22.004+0000 7fa4c9c00700 1 --2- 172.21.15.131:0/1148023641 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa4a80a14a0 0x7fa4a80a3960 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:22.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:22.004+0000 7fa4c9c00700 1 --2- 172.21.15.131:0/1148023641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4c411a780 0x7fa4c41a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:22.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:22.004+0000 7fa4c9c00700 1 --2- 172.21.15.131:0/1148023641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c410f6a0 0x7fa4c4115f00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:22.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:22.004+0000 7fa4c9c00700 1 --2- 172.21.15.131:0/1148023641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4c410fb10 0x7fa4c4116440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:22.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:22.004+0000 7fa4c9c00700 1 -- 172.21.15.131:0/1148023641 >> 172.21.15.131:0/1148023641 conn(0x7fa4c406bff0 msgr2=0x7fa4c411ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:11:22.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:22.006+0000 7fa4c9c00700 1 -- 172.21.15.131:0/1148023641 shutdown_connections 2024-05-08T22:11:22.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:22.006+0000 7fa4c9c00700 1 -- 172.21.15.131:0/1148023641 wait complete. 2024-05-08T22:11:22.277 INFO:tasks.cephfs.filesystem:scrub status for tag:aa3822e7-6b1d-40c9-9246-37810abecb93 - {'path': '/', 'tag': 'aa3822e7-6b1d-40c9-9246-37810abecb93', 'options': 'recursive,force'} 2024-05-08T22:11:23.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:22 smithi157 ceph-mon[77746]: pgmap v125: 129 pgs: 129 active+clean; 518 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 308 op/s 2024-05-08T22:11:23.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:22 smithi132 ceph-mon[78180]: pgmap v125: 129 pgs: 129 active+clean; 518 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 308 op/s 2024-05-08T22:11:23.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:22 smithi131 ceph-mon[37260]: pgmap v125: 129 pgs: 129 active+clean; 518 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 308 op/s 2024-05-08T22:11:25.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:24 smithi157 ceph-mon[77746]: pgmap v126: 129 pgs: 129 active+clean; 519 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 368 op/s 2024-05-08T22:11:25.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:24 smithi132 ceph-mon[78180]: pgmap v126: 129 pgs: 129 active+clean; 519 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 368 op/s 2024-05-08T22:11:25.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:24 smithi131 ceph-mon[37260]: pgmap v126: 129 pgs: 129 active+clean; 519 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 368 op/s 2024-05-08T22:11:25.250 INFO:tasks.workunit.client.0.smithi131.stdout: 107 424493 418 266880 224 243894 895 2024-05-08T22:11:27.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:26 smithi157 ceph-mon[77746]: pgmap v127: 129 pgs: 129 active+clean; 538 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.2 MiB/s wr, 537 op/s 2024-05-08T22:11:27.089 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:26 smithi132 ceph-mon[78180]: pgmap v127: 129 pgs: 129 active+clean; 538 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.2 MiB/s wr, 537 op/s 2024-05-08T22:11:27.144 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:26 smithi131 ceph-mon[37260]: pgmap v127: 129 pgs: 129 active+clean; 538 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.2 MiB/s wr, 537 op/s 2024-05-08T22:11:29.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:28 smithi157 ceph-mon[77746]: pgmap v128: 129 pgs: 129 active+clean; 542 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 587 op/s 2024-05-08T22:11:29.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:28 smithi132 ceph-mon[78180]: pgmap v128: 129 pgs: 129 active+clean; 542 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 587 op/s 2024-05-08T22:11:29.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:28 smithi131 ceph-mon[37260]: pgmap v128: 129 pgs: 129 active+clean; 542 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 587 op/s 2024-05-08T22:11:31.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:30 smithi157 ceph-mon[77746]: pgmap v129: 129 pgs: 129 active+clean; 545 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 745 op/s 2024-05-08T22:11:31.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:30 smithi132 ceph-mon[78180]: pgmap v129: 129 pgs: 129 active+clean; 545 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 745 op/s 2024-05-08T22:11:31.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:30 smithi131 ceph-mon[37260]: pgmap v129: 129 pgs: 129 active+clean; 545 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 745 op/s 2024-05-08T22:11:33.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:32 smithi157 ceph-mon[77746]: pgmap v130: 129 pgs: 129 active+clean; 545 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 526 op/s 2024-05-08T22:11:33.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:32 smithi132 ceph-mon[78180]: pgmap v130: 129 pgs: 129 active+clean; 545 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 526 op/s 2024-05-08T22:11:33.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:32 smithi131 ceph-mon[37260]: pgmap v130: 129 pgs: 129 active+clean; 545 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 526 op/s 2024-05-08T22:11:35.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:34 smithi157 ceph-mon[77746]: pgmap v131: 129 pgs: 129 active+clean; 547 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 556 op/s 2024-05-08T22:11:35.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:34 smithi132 ceph-mon[78180]: pgmap v131: 129 pgs: 129 active+clean; 547 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 556 op/s 2024-05-08T22:11:35.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:34 smithi131 ceph-mon[37260]: pgmap v131: 129 pgs: 129 active+clean; 547 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 556 op/s 2024-05-08T22:11:35.277 INFO:tasks.workunit.client.0.smithi131.stdout: 111 395172 260 245565 229 226730 707 2024-05-08T22:11:37.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:36 smithi157 ceph-mon[77746]: pgmap v132: 129 pgs: 129 active+clean; 578 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.4 MiB/s wr, 918 op/s 2024-05-08T22:11:37.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:36 smithi132 ceph-mon[78180]: pgmap v132: 129 pgs: 129 active+clean; 578 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.4 MiB/s wr, 918 op/s 2024-05-08T22:11:37.151 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:36 smithi131 ceph-mon[37260]: pgmap v132: 129 pgs: 129 active+clean; 578 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.4 MiB/s wr, 918 op/s 2024-05-08T22:11:39.089 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:38 smithi132 ceph-mon[78180]: pgmap v133: 129 pgs: 129 active+clean; 578 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.4 MiB/s wr, 740 op/s 2024-05-08T22:11:39.147 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:38 smithi131 ceph-mon[37260]: pgmap v133: 129 pgs: 129 active+clean; 578 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.4 MiB/s wr, 740 op/s 2024-05-08T22:11:39.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:38 smithi157 ceph-mon[77746]: pgmap v133: 129 pgs: 129 active+clean; 578 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.4 MiB/s wr, 740 op/s 2024-05-08T22:11:41.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:40 smithi132 ceph-mon[78180]: pgmap v134: 129 pgs: 129 active+clean; 619 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.8 MiB/s wr, 1.02k op/s 2024-05-08T22:11:41.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:40 smithi131 ceph-mon[37260]: pgmap v134: 129 pgs: 129 active+clean; 619 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.8 MiB/s wr, 1.02k op/s 2024-05-08T22:11:41.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:40 smithi157 ceph-mon[77746]: pgmap v134: 129 pgs: 129 active+clean; 619 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.8 MiB/s wr, 1.02k op/s 2024-05-08T22:11:43.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:42 smithi132 ceph-mon[78180]: pgmap v135: 129 pgs: 129 active+clean; 621 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.1 MiB/s wr, 870 op/s 2024-05-08T22:11:43.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:42 smithi131 ceph-mon[37260]: pgmap v135: 129 pgs: 129 active+clean; 621 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.1 MiB/s wr, 870 op/s 2024-05-08T22:11:43.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:42 smithi157 ceph-mon[77746]: pgmap v135: 129 pgs: 129 active+clean; 621 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.1 MiB/s wr, 870 op/s 2024-05-08T22:11:45.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:44 smithi132 ceph-mon[78180]: pgmap v136: 129 pgs: 129 active+clean; 628 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.6 MiB/s wr, 896 op/s 2024-05-08T22:11:45.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:44 smithi131 ceph-mon[37260]: pgmap v136: 129 pgs: 129 active+clean; 628 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.6 MiB/s wr, 896 op/s 2024-05-08T22:11:45.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:44 smithi157 ceph-mon[77746]: pgmap v136: 129 pgs: 129 active+clean; 628 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.6 MiB/s wr, 896 op/s 2024-05-08T22:11:45.277 INFO:tasks.workunit.client.0.smithi131.stdout: 111 468180 347 293606 230 274433 1099 2024-05-08T22:11:46.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:45 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:11:46.147 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:45 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:11:46.278 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:45 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:11:47.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:46 smithi132 ceph-mon[78180]: pgmap v137: 129 pgs: 129 active+clean; 631 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.3 MiB/s wr, 1.12k op/s 2024-05-08T22:11:47.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:46 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:11:47.149 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:46 smithi131 ceph-mon[37260]: pgmap v137: 129 pgs: 129 active+clean; 631 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.3 MiB/s wr, 1.12k op/s 2024-05-08T22:11:47.149 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:46 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:11:47.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:46 smithi157 ceph-mon[77746]: pgmap v137: 129 pgs: 129 active+clean; 631 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.3 MiB/s wr, 1.12k op/s 2024-05-08T22:11:47.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:46 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:11:49.144 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:48 smithi131 ceph-mon[37260]: pgmap v138: 129 pgs: 129 active+clean; 631 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.9 MiB/s wr, 695 op/s 2024-05-08T22:11:49.144 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:48 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:11:49.144 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:48 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:11:49.144 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:48 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:11:49.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:48 smithi157 ceph-mon[77746]: pgmap v138: 129 pgs: 129 active+clean; 631 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.9 MiB/s wr, 695 op/s 2024-05-08T22:11:49.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:48 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:11:49.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:48 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:11:49.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:48 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:11:49.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:48 smithi132 ceph-mon[78180]: pgmap v138: 129 pgs: 129 active+clean; 631 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.9 MiB/s wr, 695 op/s 2024-05-08T22:11:49.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:48 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:11:49.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:48 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:11:49.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:48 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:11:51.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:50 smithi131 ceph-mon[37260]: pgmap v139: 129 pgs: 129 active+clean; 620 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.2 MiB/s wr, 1.01k op/s 2024-05-08T22:11:51.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:50 smithi157 ceph-mon[77746]: pgmap v139: 129 pgs: 129 active+clean; 620 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.2 MiB/s wr, 1.01k op/s 2024-05-08T22:11:51.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:50 smithi132 ceph-mon[78180]: pgmap v139: 129 pgs: 129 active+clean; 620 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.2 MiB/s wr, 1.01k op/s 2024-05-08T22:11:52.279 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:11:53.145 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:52 smithi131 ceph-mon[37260]: pgmap v140: 129 pgs: 129 active+clean; 620 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 666 op/s 2024-05-08T22:11:53.275 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:52 smithi157 ceph-mon[77746]: pgmap v140: 129 pgs: 129 active+clean; 620 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 666 op/s 2024-05-08T22:11:53.339 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:52 smithi132 ceph-mon[78180]: pgmap v140: 129 pgs: 129 active+clean; 620 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 666 op/s 2024-05-08T22:11:55.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:54 smithi131 ceph-mon[37260]: pgmap v141: 129 pgs: 129 active+clean; 620 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 635 op/s 2024-05-08T22:11:55.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:54 smithi157 ceph-mon[77746]: pgmap v141: 129 pgs: 129 active+clean; 620 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 635 op/s 2024-05-08T22:11:55.277 INFO:tasks.workunit.client.0.smithi131.stdout: 121 590230 647 374785 366 334981 1279 2024-05-08T22:11:55.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:54 smithi132 ceph-mon[78180]: pgmap v141: 129 pgs: 129 active+clean; 620 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 635 op/s 2024-05-08T22:11:56.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.898+0000 7f85f5c00700 1 -- 172.21.15.131:0/20796054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85f01082c0 msgr2=0x7f85f0108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:11:56.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.898+0000 7f85f5c00700 1 --2- 172.21.15.131:0/20796054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85f01082c0 0x7f85f0108740 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f85dc0099e0 tx=0x7f85dc009cf0 comp rx=0 tx=0).stop 2024-05-08T22:11:56.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.919+0000 7f85f5c00700 1 -- 172.21.15.131:0/20796054 shutdown_connections 2024-05-08T22:11:56.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.919+0000 7f85f5c00700 1 --2- 172.21.15.131:0/20796054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f011a560 0x7f85f011c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:56.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.919+0000 7f85f5c00700 1 --2- 172.21.15.131:0/20796054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85f010f570 0x7f85f0107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:56.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.919+0000 7f85f5c00700 1 --2- 172.21.15.131:0/20796054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85f01082c0 0x7f85f0108740 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:56.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.919+0000 7f85f5c00700 1 -- 172.21.15.131:0/20796054 >> 172.21.15.131:0/20796054 conn(0x7f85f006bee0 msgr2=0x7f85f006c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:11:56.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.919+0000 7f85f5c00700 1 -- 172.21.15.131:0/20796054 shutdown_connections 2024-05-08T22:11:56.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.923+0000 7f85f5c00700 1 -- 172.21.15.131:0/20796054 wait complete. 2024-05-08T22:11:56.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.925+0000 7f85f5c00700 1 Processor -- start 2024-05-08T22:11:56.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.929+0000 7f85f5c00700 1 -- start start 2024-05-08T22:11:56.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.935+0000 7f85f5c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85f01082c0 0x7f85f01a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:56.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.938+0000 7f85efe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85f01082c0 0x7f85f01a60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:11:56.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.938+0000 7f85efe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85f01082c0 0x7f85f01a60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60188/0 (socket says 172.21.15.131:60188) 2024-05-08T22:11:56.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.938+0000 7f85f5c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85f010f570 0x7f85f01a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:56.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.938+0000 7f85efe00700 1 -- 172.21.15.131:0/713310559 learned_addr learned my addr 172.21.15.131:0/713310559 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:11:56.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.938+0000 7f85ef400700 1 --2- 172.21.15.131:0/713310559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85f010f570 0x7f85f01a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:11:56.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.939+0000 7f85f5c00700 1 --2- 172.21.15.131:0/713310559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f011a560 0x7f85f01aa990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:56.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.940+0000 7f85f5c00700 1 -- 172.21.15.131:0/713310559 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f01aafd0 con 0x7f85f01082c0 2024-05-08T22:11:56.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.940+0000 7f85f5c00700 1 -- 172.21.15.131:0/713310559 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f01ab140 con 0x7f85f010f570 2024-05-08T22:11:56.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.940+0000 7f85f5c00700 1 -- 172.21.15.131:0/713310559 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f01ab300 con 0x7f85f011a560 2024-05-08T22:11:56.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.940+0000 7f85f5200700 1 --2- 172.21.15.131:0/713310559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f011a560 0x7f85f01aa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:11:56.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.941+0000 7f85f5200700 1 -- 172.21.15.131:0/713310559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85f010f570 msgr2=0x7f85f01a65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:11:56.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.941+0000 7f85f5200700 1 --2- 172.21.15.131:0/713310559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85f010f570 0x7f85f01a65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:56.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.941+0000 7f85f5200700 1 -- 172.21.15.131:0/713310559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85f01082c0 msgr2=0x7f85f01a60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:11:56.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.941+0000 7f85f5200700 1 --2- 172.21.15.131:0/713310559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85f01082c0 0x7f85f01a60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:56.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.941+0000 7f85f5200700 1 -- 172.21.15.131:0/713310559 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85dc0095f0 con 0x7f85f011a560 2024-05-08T22:11:56.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.942+0000 7f85efe00700 1 --2- 172.21.15.131:0/713310559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85f01082c0 0x7f85f01a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:11:56.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.953+0000 7f85f5200700 1 --2- 172.21.15.131:0/713310559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f011a560 0x7f85f01aa990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f85e800cc20 tx=0x7f85e800cf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:11:56.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.968+0000 7f85ecc00700 1 -- 172.21.15.131:0/713310559 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e800a910 con 0x7f85f011a560 2024-05-08T22:11:56.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.968+0000 7f85ecc00700 1 -- 172.21.15.131:0/713310559 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85e8023430 con 0x7f85f011a560 2024-05-08T22:11:56.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.968+0000 7f85ecc00700 1 -- 172.21.15.131:0/713310559 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e80225f0 con 0x7f85f011a560 2024-05-08T22:11:56.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.981+0000 7f85f5c00700 1 -- 172.21.15.131:0/713310559 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f01ab5e0 con 0x7f85f011a560 2024-05-08T22:11:56.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:56.982+0000 7f85f5c00700 1 -- 172.21.15.131:0/713310559 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f010ff20 con 0x7f85f011a560 2024-05-08T22:11:57.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.067+0000 7f85ecc00700 1 -- 172.21.15.131:0/713310559 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f85e80235a0 con 0x7f85f011a560 2024-05-08T22:11:57.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.069+0000 7f85ecc00700 1 --2- 172.21.15.131:0/713310559 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f85d40a14a0 0x7f85d40a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:57.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.069+0000 7f85ecc00700 1 -- 172.21.15.131:0/713310559 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f85e80dd2c0 con 0x7f85f011a560 2024-05-08T22:11:57.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.078+0000 7f85efe00700 1 --2- 172.21.15.131:0/713310559 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f85d40a14a0 0x7f85d40a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:11:57.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.086+0000 7f85efe00700 1 --2- 172.21.15.131:0/713310559 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f85d40a14a0 0x7f85d40a3960 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f85e0005950 tx=0x7f85e000b500 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:11:57.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.109+0000 7f860ad1bb80 1 Processor -- start 2024-05-08T22:11:57.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.130+0000 7f860ad1bb80 1 -- start start 2024-05-08T22:11:57.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.131+0000 7f860ad1bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a999650050 0x55a999650430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:57.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.131+0000 7f860ad1bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a999650970 0x55a99964a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:57.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.131+0000 7f860ad1bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a99964a890 0x55a99964acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:57.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.131+0000 7f860ad1bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9995a1150 con 0x55a99964a890 2024-05-08T22:11:57.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.131+0000 7f860ad1bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a99959eb10 con 0x55a999650050 2024-05-08T22:11:57.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.131+0000 7f860ad1bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a99959a400 con 0x55a999650970 2024-05-08T22:11:57.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.132+0000 7f85f5200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a99964a890 0x55a99964acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:11:57.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.132+0000 7f85f5200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a99964a890 0x55a99964acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60194/0 (socket says 172.21.15.131:60194) 2024-05-08T22:11:57.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.132+0000 7f85f5200700 1 -- 172.21.15.131:0/1748862819 learned_addr learned my addr 172.21.15.131:0/1748862819 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:11:57.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.133+0000 7f85f5200700 1 -- 172.21.15.131:0/1748862819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a999650970 msgr2=0x55a99964a350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:11:57.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.133+0000 7f85f5200700 1 --2- 172.21.15.131:0/1748862819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a999650970 0x55a99964a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:57.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.133+0000 7f85f5200700 1 -- 172.21.15.131:0/1748862819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a999650050 msgr2=0x55a999650430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:11:57.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.133+0000 7f85f5200700 1 --2- 172.21.15.131:0/1748862819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a999650050 0x55a999650430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:57.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.133+0000 7f85f5200700 1 -- 172.21.15.131:0/1748862819 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a99941b3b0 con 0x55a99964a890 2024-05-08T22:11:57.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.133+0000 7f85f5200700 1 --2- 172.21.15.131:0/1748862819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a99964a890 0x55a99964acd0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f85e80b2040 tx=0x7f85e80a9b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1b2ce1d1d50c06 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:11:57.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.139+0000 7f85e5600700 1 -- 172.21.15.131:0/1748862819 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e80adf20 con 0x55a99964a890 2024-05-08T22:11:57.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.139+0000 7f85e5600700 1 -- 172.21.15.131:0/1748862819 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85e80b2570 con 0x55a99964a890 2024-05-08T22:11:57.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.139+0000 7f85e5600700 1 -- 172.21.15.131:0/1748862819 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e80a9840 con 0x55a99964a890 2024-05-08T22:11:57.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:56 smithi131 ceph-mon[37260]: pgmap v142: 129 pgs: 129 active+clean; 625 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.5 MiB/s wr, 759 op/s 2024-05-08T22:11:57.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.140+0000 7f860ad1bb80 1 -- 172.21.15.131:0/1748862819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a99964a890 msgr2=0x55a99964acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:11:57.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.140+0000 7f860ad1bb80 1 --2- 172.21.15.131:0/1748862819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a99964a890 0x55a99964acd0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f85e80b2040 tx=0x7f85e80a9b20 comp rx=0 tx=0).stop 2024-05-08T22:11:57.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.144+0000 7f860ad1bb80 1 -- 172.21.15.131:0/1748862819 shutdown_connections 2024-05-08T22:11:57.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.144+0000 7f860ad1bb80 1 --2- 172.21.15.131:0/1748862819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a99964a890 0x55a99964acd0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:57.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.144+0000 7f860ad1bb80 1 --2- 172.21.15.131:0/1748862819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a999650050 0x55a999650430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:57.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.144+0000 7f860ad1bb80 1 --2- 172.21.15.131:0/1748862819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a999650970 0x55a99964a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:57.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.144+0000 7f860ad1bb80 1 -- 172.21.15.131:0/1748862819 >> 172.21.15.131:0/1748862819 conn(0x55a9995a8210 msgr2=0x55a9996452d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:11:57.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.145+0000 7f860ad1bb80 1 -- 172.21.15.131:0/1748862819 shutdown_connections 2024-05-08T22:11:57.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.145+0000 7f860ad1bb80 1 -- 172.21.15.131:0/1748862819 wait complete. 2024-05-08T22:11:57.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.148+0000 7f860ad1bb80 1 -- start start 2024-05-08T22:11:57.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.149+0000 7f860ad1bb80 1 Processor -- start 2024-05-08T22:11:57.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.152+0000 7f860ad1bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a99964a890 0x55a999653d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:57.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.152+0000 7f860ad1bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a999650050 0x55a999654290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:57.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.152+0000 7f85efe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a99964a890 0x55a999653d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:11:57.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.152+0000 7f85efe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a99964a890 0x55a999653d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:50682/0 (socket says 172.21.15.131:50682) 2024-05-08T22:11:57.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.152+0000 7f860ad1bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a999650970 0x55a9996547d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:57.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.152+0000 7f85ef400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a999650050 0x55a999654290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:11:57.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.152+0000 7f85ef400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a999650050 0x55a999654290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:50360/0 (socket says 172.21.15.131:50360) 2024-05-08T22:11:57.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.153+0000 7f85f5200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a999650970 0x55a9996547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:11:57.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.153+0000 7f85f5200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a999650970 0x55a9996547d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60198/0 (socket says 172.21.15.131:60198) 2024-05-08T22:11:57.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.153+0000 7f85efe00700 1 -- 172.21.15.131:0/1055121529 learned_addr learned my addr 172.21.15.131:0/1055121529 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:11:57.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.153+0000 7f860ad1bb80 1 -- 172.21.15.131:0/1055121529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a999536560 con 0x55a999650970 2024-05-08T22:11:57.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.153+0000 7f860ad1bb80 1 -- 172.21.15.131:0/1055121529 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a999536200 con 0x55a999650050 2024-05-08T22:11:57.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.153+0000 7f860ad1bb80 1 -- 172.21.15.131:0/1055121529 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a99941d6a0 con 0x55a99964a890 2024-05-08T22:11:57.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.154+0000 7f85ef400700 1 -- 172.21.15.131:0/1055121529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a99964a890 msgr2=0x55a999653d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:11:57.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.154+0000 7f85ef400700 1 --2- 172.21.15.131:0/1055121529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a99964a890 0x55a999653d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:57.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.154+0000 7f85ef400700 1 -- 172.21.15.131:0/1055121529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a999650970 msgr2=0x55a9996547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:11:57.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.154+0000 7f85ef400700 1 --2- 172.21.15.131:0/1055121529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a999650970 0x55a9996547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:57.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.154+0000 7f85ef400700 1 -- 172.21.15.131:0/1055121529 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f010ff20 con 0x55a999650050 2024-05-08T22:11:57.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.154+0000 7f860ad1bb80 10 client.0 _ll_register_callbacks cb 0x55a99953fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:11:57.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.154+0000 7f85f5200700 1 --2- 172.21.15.131:0/1055121529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a999650970 0x55a9996547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:11:57.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.155+0000 7f860ad1bb80 1 -- 172.21.15.131:0/1055121529 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9994f3270 con 0x55a999650050 2024-05-08T22:11:57.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.160+0000 7f85ef400700 1 --2- 172.21.15.131:0/1055121529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a999650050 0x55a999654290 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009930 tx=0x7f85dc003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c723878abea9ccf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:11:57.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.160+0000 7f85da000700 10 client.26153 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:11:57.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.160+0000 7f85da000700 1 -- 172.21.15.131:0/1055121529 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc01d040 con 0x55a999650050 2024-05-08T22:11:57.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.160+0000 7f85da000700 1 -- 172.21.15.131:0/1055121529 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f85dc003c70 con 0x55a999650050 2024-05-08T22:11:57.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.160+0000 7f85da000700 1 -- 172.21.15.131:0/1055121529 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85dc005070 con 0x55a999650050 2024-05-08T22:11:57.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.183+0000 7f860ad1bb80 10 client.26153 fetch_fsmap learned FSMap version 26 2024-05-08T22:11:57.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.183+0000 7f860ad1bb80 1 -- 172.21.15.131:0/1055121529 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({fsmap=26}) v3 -- 0x55a9993c7b60 con 0x55a999650050 2024-05-08T22:11:57.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.200+0000 7f85da000700 1 -- 172.21.15.131:0/1055121529 <== mon.2 v2:172.21.15.132:3300/0 4 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f85dc0051d0 con 0x55a999650050 2024-05-08T22:11:57.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.204+0000 7f860ad1bb80 10 client.26153 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:11:57.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.204+0000 7f860ad1bb80 10 client.26153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:11:57.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.204+0000 7f860ad1bb80 20 client.26153 populate_metadata read hostname 'smithi131' 2024-05-08T22:11:57.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.204+0000 7f860ad1bb80 1 --2- 172.21.15.131:0/1055121529 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55a999664190 0x55a9996665b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:57.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.204+0000 7f860ad1bb80 4 client.26153 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:11:57.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.204+0000 7f860ad1bb80 1 -- 172.21.15.131:0/1055121529 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a999619810 con 0x55a999664190 2024-05-08T22:11:57.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.206+0000 7f85efe00700 1 --2- 172.21.15.131:0/1055121529 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55a999664190 0x55a9996665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:11:57.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.207+0000 7f85efe00700 1 --2- 172.21.15.131:0/1055121529 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55a999664190 0x55a9996665b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba31d864d4270860 server_cookie=f891050d0f84eddc in_seq=0 out_seq=0 2024-05-08T22:11:57.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.207+0000 7f85da000700 10 client.26153 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:11:57.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.209+0000 7f85da000700 1 -- 172.21.15.131:0/1055121529 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a999619810 con 0x55a999664190 2024-05-08T22:11:57.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.209+0000 7f85da000700 10 client.26153 handle_command_reply: tid=0 2024-05-08T22:11:57.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.209+0000 7f860ad1bb80 1 client.26153 shutdown 2024-05-08T22:11:57.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.226+0000 7f860ad1bb80 1 -- 172.21.15.131:0/1055121529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a999650050 msgr2=0x55a999654290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:11:57.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.226+0000 7f860ad1bb80 1 --2- 172.21.15.131:0/1055121529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a999650050 0x55a999654290 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009930 tx=0x7f85dc003680 comp rx=0 tx=0).stop 2024-05-08T22:11:57.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.251+0000 7f860ad1bb80 1 -- 172.21.15.131:0/1055121529 shutdown_connections 2024-05-08T22:11:57.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.251+0000 7f860ad1bb80 1 --2- 172.21.15.131:0/1055121529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a999650970 0x55a9996547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:57.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.251+0000 7f860ad1bb80 1 --2- 172.21.15.131:0/1055121529 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55a999664190 0x55a9996665b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:57.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.251+0000 7f860ad1bb80 1 --2- 172.21.15.131:0/1055121529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a99964a890 0x55a999653d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:57.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.251+0000 7f860ad1bb80 1 --2- 172.21.15.131:0/1055121529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a999650050 0x55a999654290 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:57.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.251+0000 7f85da000700 0 client.26153 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:11:57.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.251+0000 7f860ad1bb80 1 -- 172.21.15.131:0/1055121529 >> 172.21.15.131:0/1055121529 conn(0x55a9995a8210 msgr2=0x55a9996450b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:11:57.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.259+0000 7f860ad1bb80 1 -- 172.21.15.131:0/1055121529 shutdown_connections 2024-05-08T22:11:57.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.263+0000 7f860ad1bb80 1 -- 172.21.15.131:0/1055121529 wait complete. 2024-05-08T22:11:57.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:56 smithi157 ceph-mon[77746]: pgmap v142: 129 pgs: 129 active+clean; 625 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.5 MiB/s wr, 759 op/s 2024-05-08T22:11:57.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.275+0000 7f860ad1bb80 20 client.26153 trim_cache size 0 max 16384 2024-05-08T22:11:57.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:56 smithi132 ceph-mon[78180]: pgmap v142: 129 pgs: 129 active+clean; 625 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.5 MiB/s wr, 759 op/s 2024-05-08T22:11:57.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.640+0000 7f860ad1bb80 1 Processor -- start 2024-05-08T22:11:57.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.674+0000 7f860ad1bb80 1 -- start start 2024-05-08T22:11:57.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.674+0000 7f860ad1bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a99964a890 0x55a99964a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:57.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.675+0000 7f860ad1bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a999650050 0x55a99965fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:57.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.675+0000 7f860ad1bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a999650970 0x55a99965b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:57.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.675+0000 7f860ad1bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a999646820 con 0x55a999650050 2024-05-08T22:11:57.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.675+0000 7f860ad1bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a99966b780 con 0x55a99964a890 2024-05-08T22:11:57.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.675+0000 7f860ad1bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a999646230 con 0x55a999650970 2024-05-08T22:11:57.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.675+0000 7f85f5200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a999650970 0x55a99965b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:11:57.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.675+0000 7f85f5200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a999650970 0x55a99965b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:50694/0 (socket says 172.21.15.131:50694) 2024-05-08T22:11:57.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.675+0000 7f85f5200700 1 -- 172.21.15.131:0/1209683031 learned_addr learned my addr 172.21.15.131:0/1209683031 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:11:57.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.676+0000 7f85f5200700 1 -- 172.21.15.131:0/1209683031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a99964a890 msgr2=0x55a99964a350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:11:57.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.676+0000 7f85f5200700 1 --2- 172.21.15.131:0/1209683031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a99964a890 0x55a99964a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:57.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.676+0000 7f85f5200700 1 -- 172.21.15.131:0/1209683031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a999650050 msgr2=0x55a99965fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:11:57.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.676+0000 7f85f5200700 1 --2- 172.21.15.131:0/1209683031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a999650050 0x55a99965fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:57.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.676+0000 7f85f5200700 1 -- 172.21.15.131:0/1209683031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a999560560 con 0x55a999650970 2024-05-08T22:11:57.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.676+0000 7f85f5200700 1 --2- 172.21.15.131:0/1209683031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a999650970 0x55a99965b330 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f85e80a6b60 tx=0x7f85e80a98f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cec205370ccd46d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:11:57.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.677+0000 7f85e6000700 1 -- 172.21.15.131:0/1209683031 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e80adfa0 con 0x55a999650970 2024-05-08T22:11:57.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.677+0000 7f85e6000700 1 -- 172.21.15.131:0/1209683031 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85e80a8490 con 0x55a999650970 2024-05-08T22:11:57.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.677+0000 7f85e6000700 1 -- 172.21.15.131:0/1209683031 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e80a6140 con 0x55a999650970 2024-05-08T22:11:57.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.678+0000 7f860ad1bb80 1 -- 172.21.15.131:0/1209683031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a999650970 msgr2=0x55a99965b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:11:57.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.678+0000 7f860ad1bb80 1 --2- 172.21.15.131:0/1209683031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a999650970 0x55a99965b330 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f85e80a6b60 tx=0x7f85e80a98f0 comp rx=0 tx=0).stop 2024-05-08T22:11:57.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.678+0000 7f860ad1bb80 1 -- 172.21.15.131:0/1209683031 shutdown_connections 2024-05-08T22:11:57.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.678+0000 7f860ad1bb80 1 --2- 172.21.15.131:0/1209683031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a999650970 0x55a99965b330 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:57.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.678+0000 7f860ad1bb80 1 --2- 172.21.15.131:0/1209683031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a99964a890 0x55a99964a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:57.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.678+0000 7f860ad1bb80 1 --2- 172.21.15.131:0/1209683031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a999650050 0x55a99965fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:57.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.678+0000 7f860ad1bb80 1 -- 172.21.15.131:0/1209683031 >> 172.21.15.131:0/1209683031 conn(0x55a9995a8210 msgr2=0x55a999654630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:11:57.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.679+0000 7f860ad1bb80 1 -- 172.21.15.131:0/1209683031 shutdown_connections 2024-05-08T22:11:57.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.679+0000 7f860ad1bb80 1 -- 172.21.15.131:0/1209683031 wait complete. 2024-05-08T22:11:57.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.681+0000 7f860ad1bb80 1 -- start start 2024-05-08T22:11:57.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.699+0000 7f860ad1bb80 1 Processor -- start 2024-05-08T22:11:57.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.703+0000 7f860ad1bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a99964a890 0x55a9996ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:57.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.703+0000 7f860ad1bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a999650970 0x55a9996aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:57.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.704+0000 7f860ad1bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9996af060 0x55a9996b9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.704+0000 7f85efe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a99964a890 0x55a9996ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:11:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.704+0000 7f85efe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a99964a890 0x55a9996ae5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60226/0 (socket says 172.21.15.131:60226) 2024-05-08T22:11:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.704+0000 7f860ad1bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9996b9be0 con 0x55a99964a890 2024-05-08T22:11:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.704+0000 7f860ad1bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9996b9d20 con 0x55a9996af060 2024-05-08T22:11:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.704+0000 7f860ad1bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9996b9ee0 con 0x55a999650970 2024-05-08T22:11:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.704+0000 7f860ad1bb80 10 client.0 _ll_register_callbacks cb 0x55a99953fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:11:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.705+0000 7f85efe00700 1 -- 172.21.15.131:0/125712699 learned_addr learned my addr 172.21.15.131:0/125712699 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:11:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.705+0000 7f85efe00700 1 -- 172.21.15.131:0/125712699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a999650970 msgr2=0x55a9996aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:11:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.705+0000 7f85efe00700 1 --2- 172.21.15.131:0/125712699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a999650970 0x55a9996aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.705+0000 7f85efe00700 1 -- 172.21.15.131:0/125712699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9996af060 msgr2=0x55a9996b9660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:11:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.705+0000 7f85efe00700 1 --2- 172.21.15.131:0/125712699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9996af060 0x55a9996b9660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.705+0000 7f85efe00700 1 -- 172.21.15.131:0/125712699 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a99941b3b0 con 0x55a99964a890 2024-05-08T22:11:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.706+0000 7f85efe00700 1 --2- 172.21.15.131:0/125712699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a99964a890 0x55a9996ae5e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f85e0000c00 tx=0x7f85e00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0755bf06ffb1527 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:11:57.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.758+0000 7f85da000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:11:57.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.758+0000 7f85da000700 1 -- 172.21.15.131:0/125712699 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e002a070 con 0x55a99964a890 2024-05-08T22:11:57.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.758+0000 7f85da000700 1 -- 172.21.15.131:0/125712699 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f85e0017070 con 0x55a99964a890 2024-05-08T22:11:57.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.784+0000 7f860ad1bb80 1 -- 172.21.15.131:0/125712699 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85e80b2570 con 0x55a99964a890 2024-05-08T22:11:57.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.802+0000 7f85da000700 1 -- 172.21.15.131:0/125712699 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85e0025070 con 0x55a99964a890 2024-05-08T22:11:57.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.803+0000 7f860ad1bb80 10 client.16119 fetch_fsmap learned FSMap version 26 2024-05-08T22:11:57.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.803+0000 7f860ad1bb80 1 -- 172.21.15.131:0/125712699 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a99941fcd0 con 0x55a99964a890 2024-05-08T22:11:57.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.804+0000 7f85da000700 1 -- 172.21.15.131:0/125712699 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85e80b2570 con 0x55a99964a890 2024-05-08T22:11:57.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.804+0000 7f85da000700 1 -- 172.21.15.131:0/125712699 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f85e002d030 con 0x55a99964a890 2024-05-08T22:11:57.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.804+0000 7f860ad1bb80 10 client.16119 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:11:57.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.804+0000 7f860ad1bb80 10 client.16119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:11:57.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.804+0000 7f860ad1bb80 20 client.16119 populate_metadata read hostname 'smithi131' 2024-05-08T22:11:57.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.804+0000 7f860ad1bb80 1 --2- 172.21.15.131:0/125712699 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55a9996bb6b0 0x55a9996bdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:11:57.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.804+0000 7f860ad1bb80 4 client.16119 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:11:57.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.804+0000 7f860ad1bb80 1 -- 172.21.15.131:0/125712699 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a999592ec0 con 0x55a9996bb6b0 2024-05-08T22:11:57.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.805+0000 7f85ef400700 1 --2- 172.21.15.131:0/125712699 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55a9996bb6b0 0x55a9996bdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:11:57.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.813+0000 7f85ef400700 1 --2- 172.21.15.131:0/125712699 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55a9996bb6b0 0x55a9996bdad0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cc8c2b3cdab25ae server_cookie=ae60ad75e0b8462b in_seq=0 out_seq=0 2024-05-08T22:11:57.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.813+0000 7f85da000700 10 client.16119 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:11:57.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.816+0000 7f85da000700 1 -- 172.21.15.131:0/125712699 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a999592ec0 con 0x55a9996bb6b0 2024-05-08T22:11:57.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.816+0000 7f85da000700 10 client.16119 handle_command_reply: tid=0 2024-05-08T22:11:57.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.818+0000 7f860ad1bb80 1 client.16119 shutdown 2024-05-08T22:11:57.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.819+0000 7f860ad1bb80 1 -- 172.21.15.131:0/125712699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a99964a890 msgr2=0x55a9996ae5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:11:57.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.819+0000 7f860ad1bb80 1 --2- 172.21.15.131:0/125712699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a99964a890 0x55a9996ae5e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f85e0000c00 tx=0x7f85e00047c0 comp rx=0 tx=0).stop 2024-05-08T22:11:57.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.820+0000 7f860ad1bb80 1 -- 172.21.15.131:0/125712699 shutdown_connections 2024-05-08T22:11:57.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.820+0000 7f860ad1bb80 1 --2- 172.21.15.131:0/125712699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9996af060 0x55a9996b9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:57.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.820+0000 7f860ad1bb80 1 --2- 172.21.15.131:0/125712699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a99964a890 0x55a9996ae5e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:57.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.820+0000 7f860ad1bb80 1 --2- 172.21.15.131:0/125712699 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55a9996bb6b0 0x55a9996bdad0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:57.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.820+0000 7f860ad1bb80 1 --2- 172.21.15.131:0/125712699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a999650970 0x55a9996aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:57.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.820+0000 7f860ad1bb80 1 -- 172.21.15.131:0/125712699 >> 172.21.15.131:0/125712699 conn(0x55a9995a8210 msgr2=0x55a99964de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:11:57.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.820+0000 7f85da000700 0 client.16119 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:11:57.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.833+0000 7f860ad1bb80 1 -- 172.21.15.131:0/125712699 shutdown_connections 2024-05-08T22:11:57.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.833+0000 7f860ad1bb80 1 -- 172.21.15.131:0/125712699 wait complete. 2024-05-08T22:11:57.891 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:11:57.892 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-08T22:11:57.892 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-08T22:11:57.892 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:11:57.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.886+0000 7f860ad1bb80 20 client.16119 trim_cache size 0 max 16384 2024-05-08T22:11:57.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.945+0000 7f85f5c00700 1 -- 172.21.15.131:0/713310559 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f85d40a14a0 msgr2=0x7f85d40a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:11:57.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.945+0000 7f85f5c00700 1 --2- 172.21.15.131:0/713310559 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f85d40a14a0 0x7f85d40a3960 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f85e0005950 tx=0x7f85e000b500 comp rx=0 tx=0).stop 2024-05-08T22:11:57.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.945+0000 7f85f5c00700 1 -- 172.21.15.131:0/713310559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f011a560 msgr2=0x7f85f01aa990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:11:57.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.945+0000 7f85f5c00700 1 --2- 172.21.15.131:0/713310559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f011a560 0x7f85f01aa990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f85e800cc20 tx=0x7f85e800cf30 comp rx=0 tx=0).stop 2024-05-08T22:11:58.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.999+0000 7f85f5c00700 1 -- 172.21.15.131:0/713310559 shutdown_connections 2024-05-08T22:11:58.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.999+0000 7f85f5c00700 1 --2- 172.21.15.131:0/713310559 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f85d40a14a0 0x7f85d40a3960 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:58.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.999+0000 7f85f5c00700 1 --2- 172.21.15.131:0/713310559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f011a560 0x7f85f01aa990 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:58.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.999+0000 7f85f5c00700 1 --2- 172.21.15.131:0/713310559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85f01082c0 0x7f85f01a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:58.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.999+0000 7f85f5c00700 1 --2- 172.21.15.131:0/713310559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85f010f570 0x7f85f01a65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:11:58.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:57.999+0000 7f85f5c00700 1 -- 172.21.15.131:0/713310559 >> 172.21.15.131:0/713310559 conn(0x7f85f006bee0 msgr2=0x7f85f0118550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:11:58.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:58.008+0000 7f85f5c00700 1 -- 172.21.15.131:0/713310559 shutdown_connections 2024-05-08T22:11:58.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:11:58.008+0000 7f85f5c00700 1 -- 172.21.15.131:0/713310559 wait complete. 2024-05-08T22:11:58.992 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T22:11:58.992 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:11:59.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:58 smithi131 ceph-mon[37260]: pgmap v143: 129 pgs: 129 active+clean; 625 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 488 op/s 2024-05-08T22:11:59.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:58 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:11:59.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:58 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:11:59.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:11:58 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:11:59.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:58 smithi157 ceph-mon[77746]: pgmap v143: 129 pgs: 129 active+clean; 625 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 488 op/s 2024-05-08T22:11:59.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:58 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:11:59.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:58 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:11:59.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:11:58 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:11:59.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:58 smithi132 ceph-mon[78180]: pgmap v143: 129 pgs: 129 active+clean; 625 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 488 op/s 2024-05-08T22:11:59.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:58 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:11:59.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:58 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:11:59.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:11:58 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' 2024-05-08T22:12:01.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.122+0000 7f6736600700 1 -- 172.21.15.131:0/2873938250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f673010d270 msgr2=0x7f673010d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:01.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.122+0000 7f6736600700 1 --2- 172.21.15.131:0/2873938250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f673010d270 0x7f673010d6f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f672001ac00 tx=0x7f672001af10 comp rx=0 tx=0).stop 2024-05-08T22:12:01.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.135+0000 7f6736600700 1 -- 172.21.15.131:0/2873938250 shutdown_connections 2024-05-08T22:12:01.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.135+0000 7f6736600700 1 --2- 172.21.15.131:0/2873938250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f673010dd10 0x7f6730107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:01.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.135+0000 7f6736600700 1 --2- 172.21.15.131:0/2873938250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f673010f510 0x7f673010f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:01.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.135+0000 7f6736600700 1 --2- 172.21.15.131:0/2873938250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f673010d270 0x7f673010d6f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:01.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.135+0000 7f6736600700 1 -- 172.21.15.131:0/2873938250 >> 172.21.15.131:0/2873938250 conn(0x7f673006bec0 msgr2=0x7f673006c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:01.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:00 smithi131 ceph-mon[37260]: pgmap v144: 129 pgs: 129 active+clean; 642 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 583 op/s 2024-05-08T22:12:01.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.143+0000 7f6736600700 1 -- 172.21.15.131:0/2873938250 shutdown_connections 2024-05-08T22:12:01.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.150+0000 7f6736600700 1 -- 172.21.15.131:0/2873938250 wait complete. 2024-05-08T22:12:01.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.151+0000 7f6736600700 1 Processor -- start 2024-05-08T22:12:01.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.151+0000 7f6736600700 1 -- start start 2024-05-08T22:12:01.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.152+0000 7f6736600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f673010d270 0x7f67301a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:01.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.156+0000 7f6736600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f673010dd10 0x7f67301a3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:01.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.156+0000 7f6736600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f673010f510 0x7f673019d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:01.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.156+0000 7f6736600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673019df30 con 0x7f673010dd10 2024-05-08T22:12:01.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.156+0000 7f6736600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673019e0a0 con 0x7f673010d270 2024-05-08T22:12:01.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.156+0000 7f6736600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673019e260 con 0x7f673010f510 2024-05-08T22:12:01.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.156+0000 7f6735200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f673010d270 0x7f67301a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:01.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.157+0000 7f6735200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f673010d270 0x7f67301a37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:50386/0 (socket says 172.21.15.131:50386) 2024-05-08T22:12:01.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.157+0000 7f6735200700 1 -- 172.21.15.131:0/199853400 learned_addr learned my addr 172.21.15.131:0/199853400 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:01.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.157+0000 7f6735c00700 1 --2- 172.21.15.131:0/199853400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f673010f510 0x7f673019d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:01.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.157+0000 7f672fe00700 1 --2- 172.21.15.131:0/199853400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f673010dd10 0x7f67301a3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:01.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.157+0000 7f6735200700 1 -- 172.21.15.131:0/199853400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f673010f510 msgr2=0x7f673019d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:01.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.157+0000 7f6735200700 1 --2- 172.21.15.131:0/199853400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f673010f510 0x7f673019d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:01.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.157+0000 7f6735200700 1 -- 172.21.15.131:0/199853400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f673010dd10 msgr2=0x7f67301a3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:01.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.158+0000 7f6735200700 1 --2- 172.21.15.131:0/199853400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f673010dd10 0x7f67301a3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:01.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.158+0000 7f6735200700 1 -- 172.21.15.131:0/199853400 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672001a600 con 0x7f673010d270 2024-05-08T22:12:01.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.158+0000 7f672fe00700 1 --2- 172.21.15.131:0/199853400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f673010dd10 0x7f67301a3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:12:01.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.158+0000 7f6735200700 1 --2- 172.21.15.131:0/199853400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f673010d270 0x7f67301a37a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f672800b840 tx=0x7f672800bc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:01.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.159+0000 7f672d600700 1 -- 172.21.15.131:0/199853400 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672800d890 con 0x7f673010d270 2024-05-08T22:12:01.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.159+0000 7f672d600700 1 -- 172.21.15.131:0/199853400 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f672800e430 con 0x7f673010d270 2024-05-08T22:12:01.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.159+0000 7f672d600700 1 -- 172.21.15.131:0/199853400 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728017660 con 0x7f673010d270 2024-05-08T22:12:01.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.168+0000 7f6736600700 1 -- 172.21.15.131:0/199853400 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f673019e540 con 0x7f673010d270 2024-05-08T22:12:01.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.168+0000 7f6736600700 1 -- 172.21.15.131:0/199853400 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67301aa790 con 0x7f673010d270 2024-05-08T22:12:01.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.171+0000 7f6736600700 1 -- 172.21.15.131:0/199853400 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f673006d220 con 0x7f673010d270 2024-05-08T22:12:01.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.176+0000 7f672d600700 1 -- 172.21.15.131:0/199853400 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f672800e8a0 con 0x7f673010d270 2024-05-08T22:12:01.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.179+0000 7f672d600700 1 --2- 172.21.15.131:0/199853400 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f67140a13e0 0x7f67140a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:01.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.185+0000 7f672d600700 1 -- 172.21.15.131:0/199853400 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f67280cd2c0 con 0x7f673010d270 2024-05-08T22:12:01.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.188+0000 7f672fe00700 1 --2- 172.21.15.131:0/199853400 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f67140a13e0 0x7f67140a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:01.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.190+0000 7f672fe00700 1 --2- 172.21.15.131:0/199853400 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f67140a13e0 0x7f67140a38a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6720003860 tx=0x7f672002a040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:01.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:01.193+0000 7f672d600700 1 -- 172.21.15.131:0/199853400 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6728096480 con 0x7f673010d270 2024-05-08T22:12:01.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:00 smithi157 ceph-mon[77746]: pgmap v144: 129 pgs: 129 active+clean; 642 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 583 op/s 2024-05-08T22:12:01.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:00 smithi132 ceph-mon[78180]: pgmap v144: 129 pgs: 129 active+clean; 642 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 583 op/s 2024-05-08T22:12:03.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:02 smithi131 ceph-mon[37260]: pgmap v145: 129 pgs: 129 active+clean; 644 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 304 op/s 2024-05-08T22:12:03.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:02 smithi157 ceph-mon[77746]: pgmap v145: 129 pgs: 129 active+clean; 644 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 304 op/s 2024-05-08T22:12:03.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:02 smithi132 ceph-mon[78180]: pgmap v145: 129 pgs: 129 active+clean; 644 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 304 op/s 2024-05-08T22:12:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:04.474+0000 7f6736600700 1 -- 172.21.15.131:0/199853400 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6708005180 con 0x7f673010d270 2024-05-08T22:12:04.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:04.484+0000 7f672d600700 1 -- 172.21.15.131:0/199853400 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10892 (secure 0 0 0) 0x7f6728099aa0 con 0x7f673010d270 2024-05-08T22:12:04.491 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:12:04.491 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:05:14.034908+0000","modified":"2024-05-08T22:10:33.275068+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:12:04.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:04.506+0000 7f6736600700 1 -- 172.21.15.131:0/199853400 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f67140a13e0 msgr2=0x7f67140a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:04.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:04.506+0000 7f6736600700 1 --2- 172.21.15.131:0/199853400 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f67140a13e0 0x7f67140a38a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6720003860 tx=0x7f672002a040 comp rx=0 tx=0).stop 2024-05-08T22:12:04.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:04.506+0000 7f6736600700 1 -- 172.21.15.131:0/199853400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f673010d270 msgr2=0x7f67301a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:04.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:04.506+0000 7f6736600700 1 --2- 172.21.15.131:0/199853400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f673010d270 0x7f67301a37a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f672800b840 tx=0x7f672800bc00 comp rx=0 tx=0).stop 2024-05-08T22:12:04.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:04.514+0000 7f6736600700 1 -- 172.21.15.131:0/199853400 shutdown_connections 2024-05-08T22:12:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:04.514+0000 7f6736600700 1 --2- 172.21.15.131:0/199853400 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f67140a13e0 0x7f67140a38a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:04.514+0000 7f6736600700 1 --2- 172.21.15.131:0/199853400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f673010f510 0x7f673019d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:04.514+0000 7f6736600700 1 --2- 172.21.15.131:0/199853400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f673010d270 0x7f67301a37a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:04.514+0000 7f6736600700 1 --2- 172.21.15.131:0/199853400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f673010dd10 0x7f67301a3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:04.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:04.514+0000 7f6736600700 1 -- 172.21.15.131:0/199853400 >> 172.21.15.131:0/199853400 conn(0x7f673006bec0 msgr2=0x7f673010aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:04.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:04.514+0000 7f6736600700 1 -- 172.21.15.131:0/199853400 shutdown_connections 2024-05-08T22:12:04.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:04.515+0000 7f6736600700 1 -- 172.21.15.131:0/199853400 wait complete. 2024-05-08T22:12:04.527 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 26 2024-05-08T22:12:04.693 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T22:12:04.951 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:04 smithi131 ceph-mon[37260]: pgmap v146: 129 pgs: 129 active+clean; 644 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 294 op/s 2024-05-08T22:12:04.952 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:04 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/199853400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:12:05.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:04 smithi157 ceph-mon[77746]: pgmap v146: 129 pgs: 129 active+clean; 644 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 294 op/s 2024-05-08T22:12:05.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:04 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/199853400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:12:05.280 INFO:tasks.workunit.client.0.smithi131.stdout: 127 446038 435 282983 361 253091 983 2024-05-08T22:12:05.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.292+0000 7f99bba00700 1 -- 172.21.15.131:0/2944878921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99b410d730 msgr2=0x7f99b410fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:05.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.292+0000 7f99bba00700 1 --2- 172.21.15.131:0/2944878921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99b410d730 0x7f99b410fba0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009990 tx=0x7f99a4009ca0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.296+0000 7f99bba00700 1 -- 172.21.15.131:0/2944878921 shutdown_connections 2024-05-08T22:12:05.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.296+0000 7f99bba00700 1 --2- 172.21.15.131:0/2944878921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99b410d730 0x7f99b410fba0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.296+0000 7f99bba00700 1 --2- 172.21.15.131:0/2944878921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b40733d0 0x7f99b40737b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.296+0000 7f99bba00700 1 --2- 172.21.15.131:0/2944878921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b4073cf0 0x7f99b410d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.296+0000 7f99bba00700 1 -- 172.21.15.131:0/2944878921 >> 172.21.15.131:0/2944878921 conn(0x7f99b40fc9c0 msgr2=0x7f99b40fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:05.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.299+0000 7f99bba00700 1 -- 172.21.15.131:0/2944878921 shutdown_connections 2024-05-08T22:12:05.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.299+0000 7f99bba00700 1 -- 172.21.15.131:0/2944878921 wait complete. 2024-05-08T22:12:05.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.300+0000 7f99bba00700 1 Processor -- start 2024-05-08T22:12:05.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.304+0000 7f99bba00700 1 -- start start 2024-05-08T22:12:05.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.305+0000 7f99bba00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b40733d0 0x7f99b410ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:05.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.305+0000 7f99bba00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99b4073cf0 0x7f99b4109fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:05.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.305+0000 7f99bba00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b410d730 0x7f99b410a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:05.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.305+0000 7f99bba00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b410ab70 con 0x7f99b410d730 2024-05-08T22:12:05.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.305+0000 7f99bba00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b410ace0 con 0x7f99b4073cf0 2024-05-08T22:12:05.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.305+0000 7f99bba00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b410aea0 con 0x7f99b40733d0 2024-05-08T22:12:05.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.305+0000 7f99b9000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b40733d0 0x7f99b410ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:05.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.305+0000 7f99b9000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b40733d0 0x7f99b410ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43110/0 (socket says 172.21.15.131:43110) 2024-05-08T22:12:05.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.305+0000 7f99b9000700 1 -- 172.21.15.131:0/2993721315 learned_addr learned my addr 172.21.15.131:0/2993721315 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:05.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.305+0000 7f99b9000700 1 -- 172.21.15.131:0/2993721315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99b4073cf0 msgr2=0x7f99b4109fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:05.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.305+0000 7f99b9000700 1 --2- 172.21.15.131:0/2993721315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99b4073cf0 0x7f99b4109fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.305+0000 7f99b9000700 1 -- 172.21.15.131:0/2993721315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b410d730 msgr2=0x7f99b410a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:05.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.305+0000 7f99b9000700 1 --2- 172.21.15.131:0/2993721315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b410d730 0x7f99b410a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.306+0000 7f99b9000700 1 -- 172.21.15.131:0/2993721315 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a40095f0 con 0x7f99b40733d0 2024-05-08T22:12:05.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.307+0000 7f99b9000700 1 --2- 172.21.15.131:0/2993721315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b40733d0 0x7f99b410ceb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00ba50 tx=0x7f99ac00be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:05.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.308+0000 7f99b1600700 1 -- 172.21.15.131:0/2993721315 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac00cab0 con 0x7f99b40733d0 2024-05-08T22:12:05.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.308+0000 7f99b1600700 1 -- 172.21.15.131:0/2993721315 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99ac00cc10 con 0x7f99b40733d0 2024-05-08T22:12:05.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.309+0000 7f99b1600700 1 -- 172.21.15.131:0/2993721315 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac00cab0 con 0x7f99b40733d0 2024-05-08T22:12:05.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.310+0000 7f99bba00700 1 -- 172.21.15.131:0/2993721315 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b41aa190 con 0x7f99b40733d0 2024-05-08T22:12:05.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.310+0000 7f99bba00700 1 -- 172.21.15.131:0/2993721315 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b41aa620 con 0x7f99b40733d0 2024-05-08T22:12:05.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.317+0000 7f99b1600700 1 -- 172.21.15.131:0/2993721315 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f99ac014440 con 0x7f99b40733d0 2024-05-08T22:12:05.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.321+0000 7f99b1600700 1 --2- 172.21.15.131:0/2993721315 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f99980a1390 0x7f99980a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:05.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.321+0000 7f99b1600700 1 -- 172.21.15.131:0/2993721315 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f99ac0cbf00 con 0x7f99b40733d0 2024-05-08T22:12:05.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.323+0000 7f99b3e00700 1 --2- 172.21.15.131:0/2993721315 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f99980a1390 0x7f99980a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:05.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.325+0000 7f99cf430b80 1 Processor -- start 2024-05-08T22:12:05.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.326+0000 7f99b3e00700 1 --2- 172.21.15.131:0/2993721315 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f99980a1390 0x7f99980a3850 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f99a0005fd0 tx=0x7f99a0005d00 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:05.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.328+0000 7f99cf430b80 1 -- start start 2024-05-08T22:12:05.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.329+0000 7f99cf430b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b12ee00 0x55d97b12f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:05.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.329+0000 7f99cf430b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97b12f720 0x55d97b129350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:05.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.329+0000 7f99cf430b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97b129890 0x55d97b129cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:05.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.329+0000 7f99b9000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b12ee00 0x55d97b12f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:05.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.329+0000 7f99cf430b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97b080150 con 0x55d97b12f720 2024-05-08T22:12:05.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.329+0000 7f99cf430b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97b07db10 con 0x55d97b129890 2024-05-08T22:12:05.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.329+0000 7f99cf430b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97b079400 con 0x55d97b12ee00 2024-05-08T22:12:05.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.330+0000 7f99b9000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b12ee00 0x55d97b12f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43124/0 (socket says 172.21.15.131:43124) 2024-05-08T22:12:05.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.330+0000 7f99b9000700 1 -- 172.21.15.131:0/303107114 learned_addr learned my addr 172.21.15.131:0/303107114 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:05.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.330+0000 7f99b3e00700 1 --2- 172.21.15.131:0/303107114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97b12f720 0x55d97b129350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:05.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.330+0000 7f99b9a00700 1 --2- 172.21.15.131:0/303107114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97b129890 0x55d97b129cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:05.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.331+0000 7f99b9000700 1 -- 172.21.15.131:0/303107114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97b129890 msgr2=0x55d97b129cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:05.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.331+0000 7f99b9000700 1 --2- 172.21.15.131:0/303107114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97b129890 0x55d97b129cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.331+0000 7f99b9000700 1 -- 172.21.15.131:0/303107114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97b12f720 msgr2=0x55d97b129350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:05.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.331+0000 7f99b9000700 1 --2- 172.21.15.131:0/303107114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97b12f720 0x55d97b129350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.331+0000 7f99b9000700 1 -- 172.21.15.131:0/303107114 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d97aefa3b0 con 0x55d97b12ee00 2024-05-08T22:12:05.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.331+0000 7f99b9a00700 1 --2- 172.21.15.131:0/303107114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97b129890 0x55d97b129cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:12:05.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.331+0000 7f99b9000700 1 --2- 172.21.15.131:0/303107114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b12ee00 0x55d97b12f1e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f99ac016000 tx=0x7f99ac014b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=121b16981864a429 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:05.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.331+0000 7f99b3e00700 1 --2- 172.21.15.131:0/303107114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97b12f720 0x55d97b129350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:12:05.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.333+0000 7f99aa000700 1 -- 172.21.15.131:0/303107114 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac00cab0 con 0x55d97b12ee00 2024-05-08T22:12:05.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.333+0000 7f99aa000700 1 -- 172.21.15.131:0/303107114 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99ac09cbc0 con 0x55d97b12ee00 2024-05-08T22:12:05.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.333+0000 7f99aa000700 1 -- 172.21.15.131:0/303107114 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac00cab0 con 0x55d97b12ee00 2024-05-08T22:12:05.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.334+0000 7f99cf430b80 1 -- 172.21.15.131:0/303107114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b12ee00 msgr2=0x55d97b12f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:05.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.334+0000 7f99cf430b80 1 --2- 172.21.15.131:0/303107114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b12ee00 0x55d97b12f1e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f99ac016000 tx=0x7f99ac014b80 comp rx=0 tx=0).stop 2024-05-08T22:12:05.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.336+0000 7f99cf430b80 1 -- 172.21.15.131:0/303107114 shutdown_connections 2024-05-08T22:12:05.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.337+0000 7f99cf430b80 1 --2- 172.21.15.131:0/303107114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97b129890 0x55d97b129cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.337+0000 7f99cf430b80 1 --2- 172.21.15.131:0/303107114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b12ee00 0x55d97b12f1e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.337+0000 7f99cf430b80 1 --2- 172.21.15.131:0/303107114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97b12f720 0x55d97b129350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.337+0000 7f99cf430b80 1 -- 172.21.15.131:0/303107114 >> 172.21.15.131:0/303107114 conn(0x55d97b087210 msgr2=0x55d97b123df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:05.339 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:04 smithi132 ceph-mon[78180]: pgmap v146: 129 pgs: 129 active+clean; 644 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 294 op/s 2024-05-08T22:12:05.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:04 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/199853400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:12:05.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.340+0000 7f99cf430b80 1 -- 172.21.15.131:0/303107114 shutdown_connections 2024-05-08T22:12:05.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.341+0000 7f99cf430b80 1 -- 172.21.15.131:0/303107114 wait complete. 2024-05-08T22:12:05.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.343+0000 7f99cf430b80 1 -- start start 2024-05-08T22:12:05.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.345+0000 7f99cf430b80 1 Processor -- start 2024-05-08T22:12:05.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.349+0000 7f99cf430b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b129890 0x55d97b132a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:05.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.349+0000 7f99cf430b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97b12ee00 0x55d97b132fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:05.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.349+0000 7f99cf430b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97b12f720 0x55d97b1334f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:05.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.350+0000 7f99cf430b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97b015560 con 0x55d97b12ee00 2024-05-08T22:12:05.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.350+0000 7f99cf430b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97b015200 con 0x55d97b12f720 2024-05-08T22:12:05.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.350+0000 7f99cf430b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97aefc6a0 con 0x55d97b129890 2024-05-08T22:12:05.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.350+0000 7f99b9000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b129890 0x55d97b132a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:05.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.350+0000 7f99b9000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b129890 0x55d97b132a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43138/0 (socket says 172.21.15.131:43138) 2024-05-08T22:12:05.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.350+0000 7f99b9000700 1 -- 172.21.15.131:0/2051066264 learned_addr learned my addr 172.21.15.131:0/2051066264 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:05.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.350+0000 7f99b3e00700 1 --2- 172.21.15.131:0/2051066264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97b12ee00 0x55d97b132fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:05.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.350+0000 7f99cf430b80 10 client.0 _ll_register_callbacks cb 0x55d97b01eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:12:05.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.352+0000 7f99b9000700 1 -- 172.21.15.131:0/2051066264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97b12f720 msgr2=0x55d97b1334f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:05.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.351+0000 7f99b9a00700 1 --2- 172.21.15.131:0/2051066264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97b12f720 0x55d97b1334f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:05.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.352+0000 7f99b9000700 1 --2- 172.21.15.131:0/2051066264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97b12f720 0x55d97b1334f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.352+0000 7f99b9000700 1 -- 172.21.15.131:0/2051066264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97b12ee00 msgr2=0x55d97b132fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:05.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.352+0000 7f99b9000700 1 --2- 172.21.15.131:0/2051066264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97b12ee00 0x55d97b132fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.352+0000 7f99b9000700 1 -- 172.21.15.131:0/2051066264 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b41aa620 con 0x55d97b129890 2024-05-08T22:12:05.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.352+0000 7f99b9a00700 1 --2- 172.21.15.131:0/2051066264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97b12f720 0x55d97b1334f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:12:05.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.353+0000 7f99b9000700 1 --2- 172.21.15.131:0/2051066264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b129890 0x55d97b132a70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f99ac095800 tx=0x7f99ac0951c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eff1694564aac6d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:05.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.353+0000 7f999ea00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:12:05.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.353+0000 7f99b3e00700 1 --2- 172.21.15.131:0/2051066264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97b12ee00 0x55d97b132fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:12:05.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.354+0000 7f999ea00700 1 -- 172.21.15.131:0/2051066264 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac0991f0 con 0x55d97b129890 2024-05-08T22:12:05.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.354+0000 7f999ea00700 1 -- 172.21.15.131:0/2051066264 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f99ac014900 con 0x55d97b129890 2024-05-08T22:12:05.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.355+0000 7f99cf430b80 1 -- 172.21.15.131:0/2051066264 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d97afd2270 con 0x55d97b129890 2024-05-08T22:12:05.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.356+0000 7f999ea00700 1 -- 172.21.15.131:0/2051066264 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99ac0b7180 con 0x55d97b129890 2024-05-08T22:12:05.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.356+0000 7f99cf430b80 10 client.26206 fetch_fsmap learned FSMap version 26 2024-05-08T22:12:05.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.357+0000 7f99cf430b80 1 -- 172.21.15.131:0/2051066264 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d97aea6b60 con 0x55d97b129890 2024-05-08T22:12:05.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.357+0000 7f999ea00700 1 -- 172.21.15.131:0/2051066264 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d97afd2270 con 0x55d97b129890 2024-05-08T22:12:05.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.358+0000 7f999ea00700 1 -- 172.21.15.131:0/2051066264 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f99ac094b80 con 0x55d97b129890 2024-05-08T22:12:05.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.358+0000 7f99cf430b80 10 client.26206 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:12:05.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.358+0000 7f99cf430b80 10 client.26206 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:12:05.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.359+0000 7f99cf430b80 20 client.26206 populate_metadata read hostname 'smithi131' 2024-05-08T22:12:05.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.359+0000 7f99cf430b80 1 --2- 172.21.15.131:0/2051066264 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d97b142da0 0x55d97b1451c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:05.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.359+0000 7f99cf430b80 4 client.26206 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:12:05.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.359+0000 7f99cf430b80 1 -- 172.21.15.131:0/2051066264 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d97b0f8810 con 0x55d97b142da0 2024-05-08T22:12:05.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.363+0000 7f99b3e00700 1 --2- 172.21.15.131:0/2051066264 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d97b142da0 0x55d97b1451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:05.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.364+0000 7f99b3e00700 1 --2- 172.21.15.131:0/2051066264 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d97b142da0 0x55d97b1451c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b82b913187f3a7b server_cookie=22ca83662fc7d478 in_seq=0 out_seq=0 2024-05-08T22:12:05.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.364+0000 7f999ea00700 10 client.26206 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:12:05.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.368+0000 7f999ea00700 1 -- 172.21.15.131:0/2051066264 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d97b0f8810 con 0x55d97b142da0 2024-05-08T22:12:05.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.368+0000 7f999ea00700 10 client.26206 handle_command_reply: tid=0 2024-05-08T22:12:05.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.368+0000 7f99cf430b80 1 client.26206 shutdown 2024-05-08T22:12:05.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.377+0000 7f99cf430b80 1 -- 172.21.15.131:0/2051066264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b129890 msgr2=0x55d97b132a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:05.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.377+0000 7f99cf430b80 1 --2- 172.21.15.131:0/2051066264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b129890 0x55d97b132a70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f99ac095800 tx=0x7f99ac0951c0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.378+0000 7f99cf430b80 1 -- 172.21.15.131:0/2051066264 shutdown_connections 2024-05-08T22:12:05.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.378+0000 7f99cf430b80 1 --2- 172.21.15.131:0/2051066264 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d97b142da0 0x55d97b1451c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.378+0000 7f99cf430b80 1 --2- 172.21.15.131:0/2051066264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97b12f720 0x55d97b1334f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.378+0000 7f99cf430b80 1 --2- 172.21.15.131:0/2051066264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b129890 0x55d97b132a70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.378+0000 7f99cf430b80 1 --2- 172.21.15.131:0/2051066264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97b12ee00 0x55d97b132fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.378+0000 7f999ea00700 0 client.26206 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:12:05.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.378+0000 7f99cf430b80 1 -- 172.21.15.131:0/2051066264 >> 172.21.15.131:0/2051066264 conn(0x55d97b087210 msgr2=0x55d97b1235a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:05.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.378+0000 7f99cf430b80 1 -- 172.21.15.131:0/2051066264 shutdown_connections 2024-05-08T22:12:05.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.379+0000 7f99cf430b80 1 -- 172.21.15.131:0/2051066264 wait complete. 2024-05-08T22:12:05.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.379+0000 7f99cf430b80 20 client.26206 trim_cache size 0 max 16384 2024-05-08T22:12:05.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.437+0000 7f99cf430b80 1 Processor -- start 2024-05-08T22:12:05.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.442+0000 7f99cf430b80 1 -- start start 2024-05-08T22:12:05.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.442+0000 7f99cf430b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97b129890 0x55d97b129c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:05.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.442+0000 7f99cf430b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97b12ee00 0x55d97b123d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:05.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.442+0000 7f99cf430b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b12f720 0x55d97b131a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:05.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.442+0000 7f99cf430b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97b11e800 con 0x55d97b12ee00 2024-05-08T22:12:05.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.442+0000 7f99cf430b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97b135c50 con 0x55d97b129890 2024-05-08T22:12:05.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.442+0000 7f99cf430b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97b130fe0 con 0x55d97b12f720 2024-05-08T22:12:05.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.445+0000 7f99b9a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b12f720 0x55d97b131a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.445+0000 7f99b9a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b12f720 0x55d97b131a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43154/0 (socket says 172.21.15.131:43154) 2024-05-08T22:12:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.445+0000 7f99b9a00700 1 -- 172.21.15.131:0/2117992514 learned_addr learned my addr 172.21.15.131:0/2117992514 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.445+0000 7f99b9a00700 1 -- 172.21.15.131:0/2117992514 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97b129890 msgr2=0x55d97b129c70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:12:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.445+0000 7f99b9a00700 1 --2- 172.21.15.131:0/2117992514 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97b129890 0x55d97b129c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.445+0000 7f99b9a00700 1 -- 172.21.15.131:0/2117992514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97b12ee00 msgr2=0x55d97b123d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.445+0000 7f99b9a00700 1 --2- 172.21.15.131:0/2117992514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97b12ee00 0x55d97b123d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.445+0000 7f99b9a00700 1 -- 172.21.15.131:0/2117992514 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d97b03f560 con 0x55d97b12f720 2024-05-08T22:12:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.446+0000 7f99b9a00700 1 --2- 172.21.15.131:0/2117992514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b12f720 0x55d97b131a70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f99a4003750 tx=0x7f99a4003780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6161f220c5384b07 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.447+0000 7f99aaa00700 1 -- 172.21.15.131:0/2117992514 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a401e070 con 0x55d97b12f720 2024-05-08T22:12:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.447+0000 7f99aaa00700 1 -- 172.21.15.131:0/2117992514 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99a4003860 con 0x55d97b12f720 2024-05-08T22:12:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.447+0000 7f99aaa00700 1 -- 172.21.15.131:0/2117992514 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a4005570 con 0x55d97b12f720 2024-05-08T22:12:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.450+0000 7f99cf430b80 1 -- 172.21.15.131:0/2117992514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b12f720 msgr2=0x55d97b131a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.450+0000 7f99cf430b80 1 --2- 172.21.15.131:0/2117992514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b12f720 0x55d97b131a70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f99a4003750 tx=0x7f99a4003780 comp rx=0 tx=0).stop 2024-05-08T22:12:05.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.453+0000 7f99cf430b80 1 -- 172.21.15.131:0/2117992514 shutdown_connections 2024-05-08T22:12:05.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.454+0000 7f99cf430b80 1 --2- 172.21.15.131:0/2117992514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b12f720 0x55d97b131a70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.454+0000 7f99cf430b80 1 --2- 172.21.15.131:0/2117992514 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97b129890 0x55d97b129c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.454+0000 7f99cf430b80 1 --2- 172.21.15.131:0/2117992514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97b12ee00 0x55d97b123d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.454+0000 7f99cf430b80 1 -- 172.21.15.131:0/2117992514 >> 172.21.15.131:0/2117992514 conn(0x55d97b087210 msgr2=0x55d97b131360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:05.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.457+0000 7f99cf430b80 1 -- 172.21.15.131:0/2117992514 shutdown_connections 2024-05-08T22:12:05.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.458+0000 7f99cf430b80 1 -- 172.21.15.131:0/2117992514 wait complete. 2024-05-08T22:12:05.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.458+0000 7f99cf430b80 1 -- start start 2024-05-08T22:12:05.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.459+0000 7f99cf430b80 1 Processor -- start 2024-05-08T22:12:05.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.466+0000 7f99cf430b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b129890 0x55d97b136140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:05.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.466+0000 7f99cf430b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97b12ee00 0x55d97b136680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:05.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.466+0000 7f99cf430b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97b12f720 0x55d97b136bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:05.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.466+0000 7f99cf430b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97b198290 con 0x55d97b12f720 2024-05-08T22:12:05.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.466+0000 7f99cf430b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97b1983d0 con 0x55d97b12ee00 2024-05-08T22:12:05.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.466+0000 7f99cf430b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97b198590 con 0x55d97b129890 2024-05-08T22:12:05.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.466+0000 7f99cf430b80 10 client.0 _ll_register_callbacks cb 0x55d97b01eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:12:05.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.467+0000 7f99b9a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97b12f720 0x55d97b136bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:05.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.467+0000 7f99b3e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97b12ee00 0x55d97b136680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:05.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.467+0000 7f99b3e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97b12ee00 0x55d97b136680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:41212/0 (socket says 172.21.15.131:41212) 2024-05-08T22:12:05.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.467+0000 7f99b3e00700 1 -- 172.21.15.131:0/3332736963 learned_addr learned my addr 172.21.15.131:0/3332736963 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:05.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.467+0000 7f99b9000700 1 --2- 172.21.15.131:0/3332736963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b129890 0x55d97b136140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:05.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.468+0000 7f99b3e00700 1 -- 172.21.15.131:0/3332736963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b129890 msgr2=0x55d97b136140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:05.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.468+0000 7f99b3e00700 1 --2- 172.21.15.131:0/3332736963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b129890 0x55d97b136140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.468+0000 7f99b3e00700 1 -- 172.21.15.131:0/3332736963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97b12f720 msgr2=0x55d97b136bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:05.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.468+0000 7f99b3e00700 1 --2- 172.21.15.131:0/3332736963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97b12f720 0x55d97b136bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.468+0000 7f99b3e00700 1 -- 172.21.15.131:0/3332736963 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b4110860 con 0x55d97b12ee00 2024-05-08T22:12:05.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.470+0000 7f99b3e00700 1 --2- 172.21.15.131:0/3332736963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97b12ee00 0x55d97b136680 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f99b410b6b0 tx=0x7f99a0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f13ed551635627a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:05.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.470+0000 7f999ea00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:12:05.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.470+0000 7f999ea00700 1 -- 172.21.15.131:0/3332736963 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a0023070 con 0x55d97b12ee00 2024-05-08T22:12:05.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.470+0000 7f999ea00700 1 -- 172.21.15.131:0/3332736963 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f99a001ecd0 con 0x55d97b12ee00 2024-05-08T22:12:05.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.470+0000 7f99cf430b80 1 -- 172.21.15.131:0/3332736963 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99ac09cbc0 con 0x55d97b12ee00 2024-05-08T22:12:05.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.471+0000 7f999ea00700 1 -- 172.21.15.131:0/3332736963 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99a001ee90 con 0x55d97b12ee00 2024-05-08T22:12:05.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.471+0000 7f99cf430b80 10 client.26162 fetch_fsmap learned FSMap version 26 2024-05-08T22:12:05.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.471+0000 7f99cf430b80 1 -- 172.21.15.131:0/3332736963 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d97aefecd0 con 0x55d97b12ee00 2024-05-08T22:12:05.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.473+0000 7f999ea00700 1 -- 172.21.15.131:0/3332736963 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99a001ee90 con 0x55d97b12ee00 2024-05-08T22:12:05.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.473+0000 7f999ea00700 1 -- 172.21.15.131:0/3332736963 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f99a0003680 con 0x55d97b12ee00 2024-05-08T22:12:05.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.473+0000 7f99cf430b80 10 client.26162 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:12:05.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.473+0000 7f99cf430b80 10 client.26162 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:12:05.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.473+0000 7f99cf430b80 20 client.26162 populate_metadata read hostname 'smithi131' 2024-05-08T22:12:05.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.473+0000 7f99cf430b80 1 --2- 172.21.15.131:0/3332736963 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d97b199dc0 0x55d97b19c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:05.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.473+0000 7f99cf430b80 4 client.26162 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:12:05.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.473+0000 7f99cf430b80 1 -- 172.21.15.131:0/3332736963 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d97b071ec0 con 0x55d97b199dc0 2024-05-08T22:12:05.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.474+0000 7f99b9000700 1 --2- 172.21.15.131:0/3332736963 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d97b199dc0 0x55d97b19c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:05.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.496+0000 7f99b9000700 1 --2- 172.21.15.131:0/3332736963 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d97b199dc0 0x55d97b19c1e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a293bcce1854b180 server_cookie=f6dd6557a6df22cc in_seq=0 out_seq=0 2024-05-08T22:12:05.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.496+0000 7f999ea00700 10 client.26162 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:12:05.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.499+0000 7f999ea00700 1 -- 172.21.15.131:0/3332736963 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d97b071ec0 con 0x55d97b199dc0 2024-05-08T22:12:05.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.499+0000 7f999ea00700 10 client.26162 handle_command_reply: tid=0 2024-05-08T22:12:05.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.499+0000 7f99cf430b80 1 client.26162 shutdown 2024-05-08T22:12:05.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.499+0000 7f99cf430b80 1 -- 172.21.15.131:0/3332736963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97b12ee00 msgr2=0x55d97b136680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:05.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.499+0000 7f99cf430b80 1 --2- 172.21.15.131:0/3332736963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97b12ee00 0x55d97b136680 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f99b410b6b0 tx=0x7f99a0004750 comp rx=0 tx=0).stop 2024-05-08T22:12:05.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.500+0000 7f99cf430b80 1 -- 172.21.15.131:0/3332736963 shutdown_connections 2024-05-08T22:12:05.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.500+0000 7f99cf430b80 1 --2- 172.21.15.131:0/3332736963 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d97b199dc0 0x55d97b19c1e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.500+0000 7f99cf430b80 1 --2- 172.21.15.131:0/3332736963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d97b12f720 0x55d97b136bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.500+0000 7f99cf430b80 1 --2- 172.21.15.131:0/3332736963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d97b129890 0x55d97b136140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.500+0000 7f99cf430b80 1 --2- 172.21.15.131:0/3332736963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d97b12ee00 0x55d97b136680 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.500+0000 7f999ea00700 0 client.26162 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:12:05.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.500+0000 7f99cf430b80 1 -- 172.21.15.131:0/3332736963 >> 172.21.15.131:0/3332736963 conn(0x55d97b087210 msgr2=0x55d97b130440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:05.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.502+0000 7f99cf430b80 1 -- 172.21.15.131:0/3332736963 shutdown_connections 2024-05-08T22:12:05.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.502+0000 7f99cf430b80 1 -- 172.21.15.131:0/3332736963 wait complete. 2024-05-08T22:12:05.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.504+0000 7f99cf430b80 20 client.26162 trim_cache size 0 max 16384 2024-05-08T22:12:05.506 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:12:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.507+0000 7f99bba00700 1 -- 172.21.15.131:0/2993721315 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f99980a1390 msgr2=0x7f99980a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.507+0000 7f99bba00700 1 --2- 172.21.15.131:0/2993721315 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f99980a1390 0x7f99980a3850 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f99a0005fd0 tx=0x7f99a0005d00 comp rx=0 tx=0).stop 2024-05-08T22:12:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.507+0000 7f99bba00700 1 -- 172.21.15.131:0/2993721315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b40733d0 msgr2=0x7f99b410ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.507+0000 7f99bba00700 1 --2- 172.21.15.131:0/2993721315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b40733d0 0x7f99b410ceb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00ba50 tx=0x7f99ac00be10 comp rx=0 tx=0).stop 2024-05-08T22:12:05.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.509+0000 7f99bba00700 1 -- 172.21.15.131:0/2993721315 shutdown_connections 2024-05-08T22:12:05.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.509+0000 7f99bba00700 1 --2- 172.21.15.131:0/2993721315 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f99980a1390 0x7f99980a3850 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.509+0000 7f99bba00700 1 --2- 172.21.15.131:0/2993721315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b410d730 0x7f99b410a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.509+0000 7f99bba00700 1 --2- 172.21.15.131:0/2993721315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b40733d0 0x7f99b410ceb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.509+0000 7f99bba00700 1 --2- 172.21.15.131:0/2993721315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99b4073cf0 0x7f99b4109fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:05.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.509+0000 7f99bba00700 1 -- 172.21.15.131:0/2993721315 >> 172.21.15.131:0/2993721315 conn(0x7f99b40fc9c0 msgr2=0x7f99b40fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.512+0000 7f99bba00700 1 -- 172.21.15.131:0/2993721315 shutdown_connections 2024-05-08T22:12:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:05.513+0000 7f99bba00700 1 -- 172.21.15.131:0/2993721315 wait complete. 2024-05-08T22:12:05.624 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T22:12:06.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.086+0000 7ff4fa400700 1 -- 172.21.15.131:0/2912208261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4f4101150 msgr2=0x7ff4f4101530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:06.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.086+0000 7ff4fa400700 1 --2- 172.21.15.131:0/2912208261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4f4101150 0x7ff4f4101530 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc009990 tx=0x7ff4dc009ca0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.087+0000 7ff4fa400700 1 -- 172.21.15.131:0/2912208261 shutdown_connections 2024-05-08T22:12:06.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.087+0000 7ff4fa400700 1 --2- 172.21.15.131:0/2912208261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4f4106270 0x7ff4f410ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.087+0000 7ff4fa400700 1 --2- 172.21.15.131:0/2912208261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4f4101150 0x7ff4f4101530 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.087+0000 7ff4fa400700 1 --2- 172.21.15.131:0/2912208261 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4f4101b00 0x7ff4f4105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.087+0000 7ff4fa400700 1 -- 172.21.15.131:0/2912208261 >> 172.21.15.131:0/2912208261 conn(0x7ff4f40fc9b0 msgr2=0x7ff4f40fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:06.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.087+0000 7ff4fa400700 1 -- 172.21.15.131:0/2912208261 shutdown_connections 2024-05-08T22:12:06.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.087+0000 7ff4fa400700 1 -- 172.21.15.131:0/2912208261 wait complete. 2024-05-08T22:12:06.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.091+0000 7ff4fa400700 1 Processor -- start 2024-05-08T22:12:06.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.091+0000 7ff4fa400700 1 -- start start 2024-05-08T22:12:06.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.092+0000 7ff4fa400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4f4101150 0x7ff4f41974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.092+0000 7ff4fa400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4f4106270 0x7ff4f41979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.092+0000 7ff4fa400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4f419bd80 0x7ff4f419c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.092+0000 7ff4fa400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f419c800 con 0x7ff4f4106270 2024-05-08T22:12:06.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.092+0000 7ff4fa400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f419c940 con 0x7ff4f419bd80 2024-05-08T22:12:06.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.092+0000 7ff4fa400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f419cab0 con 0x7ff4f4101150 2024-05-08T22:12:06.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.092+0000 7ff4f3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4f4101150 0x7ff4f41974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.092+0000 7ff4f3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4f4101150 0x7ff4f41974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43192/0 (socket says 172.21.15.131:43192) 2024-05-08T22:12:06.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.092+0000 7ff4f3400700 1 -- 172.21.15.131:0/2971184833 learned_addr learned my addr 172.21.15.131:0/2971184833 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:06.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.093+0000 7ff4eaa00700 1 --2- 172.21.15.131:0/2971184833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4f4106270 0x7ff4f41979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.093+0000 7ff4f3e00700 1 --2- 172.21.15.131:0/2971184833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4f419bd80 0x7ff4f419c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.094+0000 7ff4eaa00700 1 -- 172.21.15.131:0/2971184833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4f4101150 msgr2=0x7ff4f41974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:06.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.094+0000 7ff4eaa00700 1 --2- 172.21.15.131:0/2971184833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4f4101150 0x7ff4f41974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.094+0000 7ff4eaa00700 1 -- 172.21.15.131:0/2971184833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4f419bd80 msgr2=0x7ff4f419c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:06.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.094+0000 7ff4eaa00700 1 --2- 172.21.15.131:0/2971184833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4f419bd80 0x7ff4f419c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.094+0000 7ff4eaa00700 1 -- 172.21.15.131:0/2971184833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4dc0095f0 con 0x7ff4f4106270 2024-05-08T22:12:06.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.095+0000 7ff4eaa00700 1 --2- 172.21.15.131:0/2971184833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4f4106270 0x7ff4f41979e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4007d30 tx=0x7ff4e400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:06.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.096+0000 7ff4f0c00700 1 -- 172.21.15.131:0/2971184833 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e400ad40 con 0x7ff4f4106270 2024-05-08T22:12:06.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.096+0000 7ff4f0c00700 1 -- 172.21.15.131:0/2971184833 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4e4007e80 con 0x7ff4f4106270 2024-05-08T22:12:06.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.096+0000 7ff4fa400700 1 -- 172.21.15.131:0/2971184833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f406a9c0 con 0x7ff4f4106270 2024-05-08T22:12:06.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.096+0000 7ff4f0c00700 1 -- 172.21.15.131:0/2971184833 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4011700 con 0x7ff4f4106270 2024-05-08T22:12:06.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.096+0000 7ff4fa400700 1 -- 172.21.15.131:0/2971184833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f406aed0 con 0x7ff4f4106270 2024-05-08T22:12:06.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.103+0000 7ff4f0c00700 1 -- 172.21.15.131:0/2971184833 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7ff4e40129e0 con 0x7ff4f4106270 2024-05-08T22:12:06.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.103+0000 7ff50dffab80 1 Processor -- start 2024-05-08T22:12:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.104+0000 7ff50dffab80 1 -- start start 2024-05-08T22:12:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.105+0000 7ff50dffab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557344e4c7e0 0x557344e4cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.105+0000 7ff50dffab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e3e350 0x557344e3e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.106+0000 7ff50dffab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557344e3ed10 0x557344e534e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.106+0000 7ff50dffab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557344d95150 con 0x557344e3e350 2024-05-08T22:12:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.106+0000 7ff50dffab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557344d92b10 con 0x557344e3ed10 2024-05-08T22:12:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.106+0000 7ff50dffab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557344d8e400 con 0x557344e4c7e0 2024-05-08T22:12:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.106+0000 7ff4f3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e3e350 0x557344e3e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.106+0000 7ff4f3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e3e350 0x557344e3e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59288/0 (socket says 172.21.15.131:59288) 2024-05-08T22:12:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.106+0000 7ff4f3400700 1 -- 172.21.15.131:0/117976569 learned_addr learned my addr 172.21.15.131:0/117976569 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.106+0000 7ff4eaa00700 1 --2- 172.21.15.131:0/117976569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557344e3ed10 0x557344e534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.107+0000 7ff4f3e00700 1 --2- 172.21.15.131:0/117976569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557344e4c7e0 0x557344e4cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.107+0000 7ff4f0c00700 1 --2- 172.21.15.131:0/2971184833 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff4d80a1390 0x7ff4d80a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.107+0000 7ff4eaa00700 1 -- 172.21.15.131:0/117976569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557344e4c7e0 msgr2=0x557344e4cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:06.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.107+0000 7ff4eaa00700 1 --2- 172.21.15.131:0/117976569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557344e4c7e0 0x557344e4cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.107+0000 7ff4eaa00700 1 -- 172.21.15.131:0/117976569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e3e350 msgr2=0x557344e3e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:06.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.107+0000 7ff4eaa00700 1 --2- 172.21.15.131:0/117976569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e3e350 0x557344e3e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.107+0000 7ff4eaa00700 1 -- 172.21.15.131:0/117976569 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557344c0f3b0 con 0x557344e3ed10 2024-05-08T22:12:06.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.108+0000 7ff4f3e00700 1 --2- 172.21.15.131:0/117976569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557344e4c7e0 0x557344e4cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:12:06.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.108+0000 7ff4f0c00700 1 -- 172.21.15.131:0/2971184833 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7ff4e40ccc40 con 0x7ff4f4106270 2024-05-08T22:12:06.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.109+0000 7ff4f3e00700 1 --2- 172.21.15.131:0/2971184833 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff4d80a1390 0x7ff4d80a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.109+0000 7ff4eaa00700 1 --2- 172.21.15.131:0/117976569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557344e3ed10 0x557344e534e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff4e40ca0b0 tx=0x7ff4e4095c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6765c6ecf83879d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:06.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.110+0000 7ff4e8c00700 1 -- 172.21.15.131:0/117976569 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4007b10 con 0x557344e3ed10 2024-05-08T22:12:06.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.110+0000 7ff4e8c00700 1 -- 172.21.15.131:0/117976569 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4e4099840 con 0x557344e3ed10 2024-05-08T22:12:06.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.110+0000 7ff4e8c00700 1 -- 172.21.15.131:0/117976569 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e400aea0 con 0x557344e3ed10 2024-05-08T22:12:06.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.111+0000 7ff50dffab80 1 -- 172.21.15.131:0/117976569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557344e3ed10 msgr2=0x557344e534e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:06.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.111+0000 7ff4f3e00700 1 --2- 172.21.15.131:0/2971184833 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff4d80a1390 0x7ff4d80a3850 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x557344e3a920 tx=0x7ff4ec0047e0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:06.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.111+0000 7ff50dffab80 1 --2- 172.21.15.131:0/117976569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557344e3ed10 0x557344e534e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff4e40ca0b0 tx=0x7ff4e4095c60 comp rx=0 tx=0).stop 2024-05-08T22:12:06.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.112+0000 7ff50dffab80 1 -- 172.21.15.131:0/117976569 shutdown_connections 2024-05-08T22:12:06.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.112+0000 7ff50dffab80 1 --2- 172.21.15.131:0/117976569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557344e3ed10 0x557344e534e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.112+0000 7ff50dffab80 1 --2- 172.21.15.131:0/117976569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557344e4c7e0 0x557344e4cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.112+0000 7ff50dffab80 1 --2- 172.21.15.131:0/117976569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e3e350 0x557344e3e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.112+0000 7ff50dffab80 1 -- 172.21.15.131:0/117976569 >> 172.21.15.131:0/117976569 conn(0x557344d9c210 msgr2=0x557344e38d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:06.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.112+0000 7ff50dffab80 1 -- 172.21.15.131:0/117976569 shutdown_connections 2024-05-08T22:12:06.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.112+0000 7ff50dffab80 1 -- 172.21.15.131:0/117976569 wait complete. 2024-05-08T22:12:06.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.114+0000 7ff50dffab80 1 -- start start 2024-05-08T22:12:06.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.116+0000 7ff50dffab80 1 Processor -- start 2024-05-08T22:12:06.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.117+0000 7ff50dffab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e3e350 0x557344e53420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.117+0000 7ff50dffab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557344e3ed10 0x557344e50710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.117+0000 7ff50dffab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557344e4c7e0 0x557344e50d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.117+0000 7ff50dffab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557344d2a560 con 0x557344e3e350 2024-05-08T22:12:06.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.117+0000 7ff50dffab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557344d2a200 con 0x557344e4c7e0 2024-05-08T22:12:06.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.117+0000 7ff50dffab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557344c116a0 con 0x557344e3ed10 2024-05-08T22:12:06.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.117+0000 7ff50dffab80 10 client.0 _ll_register_callbacks cb 0x557344d33b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:12:06.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.118+0000 7ff4f3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e3e350 0x557344e53420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.118+0000 7ff4f3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e3e350 0x557344e53420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59298/0 (socket says 172.21.15.131:59298) 2024-05-08T22:12:06.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.118+0000 7ff4f3400700 1 -- 172.21.15.131:0/2604315552 learned_addr learned my addr 172.21.15.131:0/2604315552 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:06.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.119+0000 7ff4f3e00700 1 --2- 172.21.15.131:0/2604315552 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557344e4c7e0 0x557344e50d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.119+0000 7ff4eaa00700 1 --2- 172.21.15.131:0/2604315552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557344e3ed10 0x557344e50710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.120+0000 7ff4f3e00700 1 -- 172.21.15.131:0/2604315552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557344e3ed10 msgr2=0x557344e50710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:06.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.120+0000 7ff4f3e00700 1 --2- 172.21.15.131:0/2604315552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557344e3ed10 0x557344e50710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.120+0000 7ff4f3e00700 1 -- 172.21.15.131:0/2604315552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e3e350 msgr2=0x557344e53420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:06.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.120+0000 7ff4f3e00700 1 --2- 172.21.15.131:0/2604315552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e3e350 0x557344e53420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.120+0000 7ff4f3e00700 1 -- 172.21.15.131:0/2604315552 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e40d1040 con 0x557344e4c7e0 2024-05-08T22:12:06.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.120+0000 7ff4e1600700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:12:06.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.120+0000 7ff4f3e00700 1 --2- 172.21.15.131:0/2604315552 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557344e4c7e0 0x557344e50d90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0095a0 tx=0x7ff4ec004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c51eaa39ac22b4c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:06.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.121+0000 7ff4e1600700 1 -- 172.21.15.131:0/2604315552 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec00fe50 con 0x557344e4c7e0 2024-05-08T22:12:06.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.121+0000 7ff4e1600700 1 -- 172.21.15.131:0/2604315552 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7ff4ec01f450 con 0x557344e4c7e0 2024-05-08T22:12:06.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.121+0000 7ff50dffab80 1 -- 172.21.15.131:0/2604315552 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557344ce7270 con 0x557344e4c7e0 2024-05-08T22:12:06.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.121+0000 7ff4e1600700 1 -- 172.21.15.131:0/2604315552 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4ec01f610 con 0x557344e4c7e0 2024-05-08T22:12:06.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.122+0000 7ff50dffab80 10 client.26171 fetch_fsmap learned FSMap version 26 2024-05-08T22:12:06.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.122+0000 7ff50dffab80 1 -- 172.21.15.131:0/2604315552 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x557344bbbb60 con 0x557344e4c7e0 2024-05-08T22:12:06.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.122+0000 7ff4e1600700 1 -- 172.21.15.131:0/2604315552 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557344ce7270 con 0x557344e4c7e0 2024-05-08T22:12:06.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.122+0000 7ff4e1600700 1 -- 172.21.15.131:0/2604315552 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff4ec01e750 con 0x557344e4c7e0 2024-05-08T22:12:06.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.122+0000 7ff50dffab80 10 client.26171 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:12:06.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.122+0000 7ff50dffab80 10 client.26171 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:12:06.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.122+0000 7ff50dffab80 20 client.26171 populate_metadata read hostname 'smithi131' 2024-05-08T22:12:06.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.122+0000 7ff50dffab80 1 --2- 172.21.15.131:0/2604315552 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x557344e435f0 0x557344e45a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.122+0000 7ff50dffab80 4 client.26171 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:12:06.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.122+0000 7ff50dffab80 1 -- 172.21.15.131:0/2604315552 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557344e0d810 con 0x557344e435f0 2024-05-08T22:12:06.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.123+0000 7ff4f3400700 1 --2- 172.21.15.131:0/2604315552 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x557344e435f0 0x557344e45a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.126+0000 7ff4f3400700 1 --2- 172.21.15.131:0/2604315552 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x557344e435f0 0x557344e45a10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a7da4af8b9798eb2 server_cookie=173ec27255a52618 in_seq=0 out_seq=0 2024-05-08T22:12:06.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.127+0000 7ff4e1600700 10 client.26171 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:12:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.129+0000 7ff4e1600700 1 -- 172.21.15.131:0/2604315552 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557344e0d810 con 0x557344e435f0 2024-05-08T22:12:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.129+0000 7ff4e1600700 10 client.26171 handle_command_reply: tid=0 2024-05-08T22:12:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.129+0000 7ff50dffab80 1 client.26171 shutdown 2024-05-08T22:12:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.130+0000 7ff50dffab80 1 -- 172.21.15.131:0/2604315552 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557344e4c7e0 msgr2=0x557344e50d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.130+0000 7ff50dffab80 1 --2- 172.21.15.131:0/2604315552 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557344e4c7e0 0x557344e50d90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0095a0 tx=0x7ff4ec004770 comp rx=0 tx=0).stop 2024-05-08T22:12:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.130+0000 7ff50dffab80 1 -- 172.21.15.131:0/2604315552 shutdown_connections 2024-05-08T22:12:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.130+0000 7ff50dffab80 1 --2- 172.21.15.131:0/2604315552 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557344e4c7e0 0x557344e50d90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.130+0000 7ff50dffab80 1 --2- 172.21.15.131:0/2604315552 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x557344e435f0 0x557344e45a10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.130+0000 7ff50dffab80 1 --2- 172.21.15.131:0/2604315552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e3e350 0x557344e53420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.130+0000 7ff4e1600700 0 client.26171 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:12:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.130+0000 7ff50dffab80 1 --2- 172.21.15.131:0/2604315552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557344e3ed10 0x557344e50710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.130+0000 7ff50dffab80 1 -- 172.21.15.131:0/2604315552 >> 172.21.15.131:0/2604315552 conn(0x557344d9c210 msgr2=0x557344e38ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.131+0000 7ff50dffab80 1 -- 172.21.15.131:0/2604315552 shutdown_connections 2024-05-08T22:12:06.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.131+0000 7ff50dffab80 1 -- 172.21.15.131:0/2604315552 wait complete. 2024-05-08T22:12:06.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.131+0000 7ff50dffab80 20 client.26171 trim_cache size 0 max 16384 2024-05-08T22:12:06.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.159+0000 7ff50dffab80 1 Processor -- start 2024-05-08T22:12:06.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.159+0000 7ff50dffab80 1 -- start start 2024-05-08T22:12:06.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.160+0000 7ff50dffab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557344e3e350 0x557344e3e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.160+0000 7ff50dffab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557344e3ed10 0x557344e3f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.160+0000 7ff50dffab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e4c7e0 0x557344e4fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.160+0000 7ff50dffab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557344e3a040 con 0x557344e4c7e0 2024-05-08T22:12:06.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.160+0000 7ff50dffab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557344e3f400 con 0x557344e3ed10 2024-05-08T22:12:06.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.160+0000 7ff50dffab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557344e51ce0 con 0x557344e3e350 2024-05-08T22:12:06.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.160+0000 7ff4f3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e4c7e0 0x557344e4fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.161+0000 7ff4f3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e4c7e0 0x557344e4fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59306/0 (socket says 172.21.15.131:59306) 2024-05-08T22:12:06.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.161+0000 7ff4f3e00700 1 -- 172.21.15.131:0/4165234250 learned_addr learned my addr 172.21.15.131:0/4165234250 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:06.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.161+0000 7ff4f3400700 1 --2- 172.21.15.131:0/4165234250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557344e3e350 0x557344e3e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.161+0000 7ff4eaa00700 1 --2- 172.21.15.131:0/4165234250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557344e3ed10 0x557344e3f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.161+0000 7ff4f3400700 1 -- 172.21.15.131:0/4165234250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557344e3ed10 msgr2=0x557344e3f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:06.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.161+0000 7ff4f3400700 1 --2- 172.21.15.131:0/4165234250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557344e3ed10 0x557344e3f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.161+0000 7ff4f3400700 1 -- 172.21.15.131:0/4165234250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e4c7e0 msgr2=0x557344e4fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:06.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.161+0000 7ff4f3400700 1 --2- 172.21.15.131:0/4165234250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e4c7e0 0x557344e4fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.161+0000 7ff4f3400700 1 -- 172.21.15.131:0/4165234250 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557344d54560 con 0x557344e3e350 2024-05-08T22:12:06.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.161+0000 7ff4f3e00700 1 --2- 172.21.15.131:0/4165234250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e4c7e0 0x557344e4fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:12:06.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.162+0000 7ff4f3400700 1 --2- 172.21.15.131:0/4165234250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557344e3e350 0x557344e3e730 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x557344e462f0 tx=0x7ff4dc009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b1157ca92f153a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:06.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.163+0000 7ff4e9600700 1 -- 172.21.15.131:0/4165234250 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc005580 con 0x557344e3e350 2024-05-08T22:12:06.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.163+0000 7ff4e9600700 1 -- 172.21.15.131:0/4165234250 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4dc01e030 con 0x557344e3e350 2024-05-08T22:12:06.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.163+0000 7ff4e9600700 1 -- 172.21.15.131:0/4165234250 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc028520 con 0x557344e3e350 2024-05-08T22:12:06.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.164+0000 7ff50dffab80 1 -- 172.21.15.131:0/4165234250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557344e3e350 msgr2=0x557344e3e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:06.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.164+0000 7ff50dffab80 1 --2- 172.21.15.131:0/4165234250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557344e3e350 0x557344e3e730 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x557344e462f0 tx=0x7ff4dc009310 comp rx=0 tx=0).stop 2024-05-08T22:12:06.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.164+0000 7ff50dffab80 1 -- 172.21.15.131:0/4165234250 shutdown_connections 2024-05-08T22:12:06.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.164+0000 7ff50dffab80 1 --2- 172.21.15.131:0/4165234250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e4c7e0 0x557344e4fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.164+0000 7ff50dffab80 1 --2- 172.21.15.131:0/4165234250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557344e3e350 0x557344e3e730 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.164+0000 7ff50dffab80 1 --2- 172.21.15.131:0/4165234250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557344e3ed10 0x557344e3f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.164+0000 7ff50dffab80 1 -- 172.21.15.131:0/4165234250 >> 172.21.15.131:0/4165234250 conn(0x557344d9c210 msgr2=0x557344e4b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:06.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.164+0000 7ff50dffab80 1 -- 172.21.15.131:0/4165234250 shutdown_connections 2024-05-08T22:12:06.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.165+0000 7ff50dffab80 1 -- 172.21.15.131:0/4165234250 wait complete. 2024-05-08T22:12:06.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.166+0000 7ff50dffab80 1 -- start start 2024-05-08T22:12:06.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.166+0000 7ff50dffab80 1 Processor -- start 2024-05-08T22:12:06.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.167+0000 7ff50dffab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e3e350 0x557344e57c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.167+0000 7ff50dffab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557344e3ed10 0x557344e46520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.168+0000 7ff50dffab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557344e4c7e0 0x557344e46a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.168+0000 7ff50dffab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557344e46fa0 con 0x557344e3e350 2024-05-08T22:12:06.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.168+0000 7ff50dffab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557344e47110 con 0x557344e4c7e0 2024-05-08T22:12:06.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.168+0000 7ff50dffab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557344e472d0 con 0x557344e3ed10 2024-05-08T22:12:06.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.168+0000 7ff50dffab80 10 client.0 _ll_register_callbacks cb 0x557344d33b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:12:06.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.169+0000 7ff4f3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e3e350 0x557344e57c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.169+0000 7ff4f3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e3e350 0x557344e57c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59308/0 (socket says 172.21.15.131:59308) 2024-05-08T22:12:06.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.169+0000 7ff4f3400700 1 -- 172.21.15.131:0/2624700930 learned_addr learned my addr 172.21.15.131:0/2624700930 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:06.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.169+0000 7ff4eaa00700 1 --2- 172.21.15.131:0/2624700930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557344e3ed10 0x557344e46520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.169+0000 7ff4f3e00700 1 --2- 172.21.15.131:0/2624700930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557344e4c7e0 0x557344e46a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.169+0000 7ff4f3400700 1 -- 172.21.15.131:0/2624700930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557344e3ed10 msgr2=0x557344e46520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:06.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.169+0000 7ff4f3400700 1 --2- 172.21.15.131:0/2624700930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557344e3ed10 0x557344e46520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.169+0000 7ff4f3400700 1 -- 172.21.15.131:0/2624700930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557344e4c7e0 msgr2=0x557344e46a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:06.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.169+0000 7ff4f3400700 1 --2- 172.21.15.131:0/2624700930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557344e4c7e0 0x557344e46a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.169+0000 7ff4f3400700 1 -- 172.21.15.131:0/2624700930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557344bbbb60 con 0x557344e3e350 2024-05-08T22:12:06.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.169+0000 7ff4eaa00700 1 --2- 172.21.15.131:0/2624700930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557344e3ed10 0x557344e46520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:12:06.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.170+0000 7ff4f3400700 1 --2- 172.21.15.131:0/2624700930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e3e350 0x557344e57c20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc009ce0 tx=0x7ff4dc005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278b8d16d789f7dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:06.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.170+0000 7ff4e1600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:12:06.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.170+0000 7ff4e1600700 1 -- 172.21.15.131:0/2624700930 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc00bb80 con 0x557344e3e350 2024-05-08T22:12:06.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.171+0000 7ff4e1600700 1 -- 172.21.15.131:0/2624700930 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7ff4dc00bce0 con 0x557344e3e350 2024-05-08T22:12:06.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.171+0000 7ff50dffab80 1 -- 172.21.15.131:0/2624700930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4e4099840 con 0x557344e3e350 2024-05-08T22:12:06.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.172+0000 7ff4e1600700 1 -- 172.21.15.131:0/2624700930 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4dc00bea0 con 0x557344e3e350 2024-05-08T22:12:06.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.172+0000 7ff50dffab80 10 client.16149 fetch_fsmap learned FSMap version 26 2024-05-08T22:12:06.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.172+0000 7ff50dffab80 1 -- 172.21.15.131:0/2624700930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7ff4f410e660 con 0x557344e3e350 2024-05-08T22:12:06.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.173+0000 7ff4e1600700 1 -- 172.21.15.131:0/2624700930 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4e4099840 con 0x557344e3e350 2024-05-08T22:12:06.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.173+0000 7ff4e1600700 1 -- 172.21.15.131:0/2624700930 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff4dc0099c0 con 0x557344e3e350 2024-05-08T22:12:06.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.174+0000 7ff50dffab80 10 client.16149 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:12:06.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.174+0000 7ff50dffab80 10 client.16149 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:12:06.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.174+0000 7ff50dffab80 20 client.16149 populate_metadata read hostname 'smithi131' 2024-05-08T22:12:06.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.174+0000 7ff50dffab80 1 --2- 172.21.15.131:0/2624700930 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x557344ec0270 0x557344ec2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.174+0000 7ff50dffab80 4 client.16149 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:12:06.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.174+0000 7ff50dffab80 1 -- 172.21.15.131:0/2624700930 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557344d86ec0 con 0x557344ec0270 2024-05-08T22:12:06.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.174+0000 7ff4eaa00700 1 --2- 172.21.15.131:0/2624700930 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x557344ec0270 0x557344ec2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.176+0000 7ff4eaa00700 1 --2- 172.21.15.131:0/2624700930 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x557344ec0270 0x557344ec2660 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=399299a52900d8b2 server_cookie=509d846c684d2627 in_seq=0 out_seq=0 2024-05-08T22:12:06.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.176+0000 7ff4e1600700 10 client.16149 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:12:06.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.179+0000 7ff4e1600700 1 -- 172.21.15.131:0/2624700930 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557344d86ec0 con 0x557344ec0270 2024-05-08T22:12:06.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.179+0000 7ff4e1600700 10 client.16149 handle_command_reply: tid=0 2024-05-08T22:12:06.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.179+0000 7ff50dffab80 1 client.16149 shutdown 2024-05-08T22:12:06.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.181+0000 7ff50dffab80 1 -- 172.21.15.131:0/2624700930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e3e350 msgr2=0x557344e57c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:06.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.181+0000 7ff50dffab80 1 --2- 172.21.15.131:0/2624700930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e3e350 0x557344e57c20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc009ce0 tx=0x7ff4dc005a10 comp rx=0 tx=0).stop 2024-05-08T22:12:06.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.181+0000 7ff50dffab80 1 -- 172.21.15.131:0/2624700930 shutdown_connections 2024-05-08T22:12:06.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.181+0000 7ff50dffab80 1 --2- 172.21.15.131:0/2624700930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557344e4c7e0 0x557344e46a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.181+0000 7ff50dffab80 1 --2- 172.21.15.131:0/2624700930 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x557344ec0270 0x557344ec2660 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.181+0000 7ff50dffab80 1 --2- 172.21.15.131:0/2624700930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557344e3e350 0x557344e57c20 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.181+0000 7ff50dffab80 1 --2- 172.21.15.131:0/2624700930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557344e3ed10 0x557344e46520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.182+0000 7ff50dffab80 1 -- 172.21.15.131:0/2624700930 >> 172.21.15.131:0/2624700930 conn(0x557344d9c210 msgr2=0x557344e4a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:06.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.182+0000 7ff4e1600700 0 client.16149 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:12:06.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.182+0000 7ff50dffab80 1 -- 172.21.15.131:0/2624700930 shutdown_connections 2024-05-08T22:12:06.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.182+0000 7ff50dffab80 1 -- 172.21.15.131:0/2624700930 wait complete. 2024-05-08T22:12:06.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.184+0000 7ff50dffab80 20 client.16149 trim_cache size 0 max 16384 2024-05-08T22:12:06.186 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:12:06.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.187+0000 7ff4fa400700 1 -- 172.21.15.131:0/2971184833 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff4d80a1390 msgr2=0x7ff4d80a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.187+0000 7ff4fa400700 1 --2- 172.21.15.131:0/2971184833 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff4d80a1390 0x7ff4d80a3850 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x557344e3a920 tx=0x7ff4ec0047e0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.187+0000 7ff4fa400700 1 -- 172.21.15.131:0/2971184833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4f4106270 msgr2=0x7ff4f41979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.187+0000 7ff4fa400700 1 --2- 172.21.15.131:0/2971184833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4f4106270 0x7ff4f41979e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4007d30 tx=0x7ff4e400a340 comp rx=0 tx=0).stop 2024-05-08T22:12:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.187+0000 7ff4fa400700 1 -- 172.21.15.131:0/2971184833 shutdown_connections 2024-05-08T22:12:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.187+0000 7ff4fa400700 1 --2- 172.21.15.131:0/2971184833 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff4d80a1390 0x7ff4d80a3850 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.187+0000 7ff4fa400700 1 --2- 172.21.15.131:0/2971184833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4f419bd80 0x7ff4f419c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.187+0000 7ff4fa400700 1 --2- 172.21.15.131:0/2971184833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4f4101150 0x7ff4f41974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.187+0000 7ff4fa400700 1 --2- 172.21.15.131:0/2971184833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4f4106270 0x7ff4f41979e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.187+0000 7ff4fa400700 1 -- 172.21.15.131:0/2971184833 >> 172.21.15.131:0/2971184833 conn(0x7ff4f40fc9b0 msgr2=0x7ff4f40fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:06.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.187+0000 7ff4fa400700 1 -- 172.21.15.131:0/2971184833 shutdown_connections 2024-05-08T22:12:06.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.188+0000 7ff4fa400700 1 -- 172.21.15.131:0/2971184833 wait complete. 2024-05-08T22:12:06.222 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T22:12:06.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.598+0000 7f7713e00700 1 -- 172.21.15.131:0/3627179423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f770c006d70 msgr2=0x7f770c007150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:06.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.598+0000 7f7713e00700 1 --2- 172.21.15.131:0/3627179423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f770c006d70 0x7f770c007150 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7714064640 tx=0x7f771404f6a0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.600+0000 7f7713e00700 1 -- 172.21.15.131:0/3627179423 shutdown_connections 2024-05-08T22:12:06.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.600+0000 7f7713e00700 1 --2- 172.21.15.131:0/3627179423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f770c09b040 0x7f770c0a2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.600+0000 7f7713e00700 1 --2- 172.21.15.131:0/3627179423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f770c006d70 0x7f770c007150 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.600+0000 7f7713e00700 1 --2- 172.21.15.131:0/3627179423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f770c007720 0x7f770c09a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.600+0000 7f7713e00700 1 -- 172.21.15.131:0/3627179423 >> 172.21.15.131:0/3627179423 conn(0x7f770c00a7a0 msgr2=0x7f770c00abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:06.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.601+0000 7f7713e00700 1 -- 172.21.15.131:0/3627179423 shutdown_connections 2024-05-08T22:12:06.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.601+0000 7f7713e00700 1 -- 172.21.15.131:0/3627179423 wait complete. 2024-05-08T22:12:06.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.603+0000 7f7713e00700 1 Processor -- start 2024-05-08T22:12:06.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.604+0000 7f7713e00700 1 -- start start 2024-05-08T22:12:06.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.605+0000 7f7713e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f770c006d70 0x7f770c132fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.605+0000 7f7713e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f770c09b040 0x7f770c1334f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.605+0000 7f7713e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f770c0a0b30 0x7f770c0a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.606+0000 7f7712a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f770c006d70 0x7f770c132fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.606+0000 7f7712a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f770c006d70 0x7f770c132fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:41296/0 (socket says 172.21.15.131:41296) 2024-05-08T22:12:06.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.606+0000 7f7712a00700 1 -- 172.21.15.131:0/1090590660 learned_addr learned my addr 172.21.15.131:0/1090590660 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:06.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.606+0000 7f7713e00700 1 -- 172.21.15.131:0/1090590660 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c0a1770 con 0x7f770c09b040 2024-05-08T22:12:06.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.606+0000 7f7713e00700 1 -- 172.21.15.131:0/1090590660 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c0a18e0 con 0x7f770c006d70 2024-05-08T22:12:06.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.606+0000 7f7713e00700 1 -- 172.21.15.131:0/1090590660 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c12ea20 con 0x7f770c0a0b30 2024-05-08T22:12:06.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.606+0000 7f7719c00700 1 --2- 172.21.15.131:0/1090590660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f770c0a0b30 0x7f770c0a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.606+0000 7f7712000700 1 --2- 172.21.15.131:0/1090590660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f770c09b040 0x7f770c1334f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.606+0000 7f7712a00700 1 -- 172.21.15.131:0/1090590660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f770c0a0b30 msgr2=0x7f770c0a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:06.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.606+0000 7f7712a00700 1 --2- 172.21.15.131:0/1090590660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f770c0a0b30 0x7f770c0a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.607+0000 7f7712a00700 1 -- 172.21.15.131:0/1090590660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f770c09b040 msgr2=0x7f770c1334f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:06.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.607+0000 7f7712a00700 1 --2- 172.21.15.131:0/1090590660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f770c09b040 0x7f770c1334f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.607+0000 7f7712a00700 1 -- 172.21.15.131:0/1090590660 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771404f510 con 0x7f770c006d70 2024-05-08T22:12:06.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.607+0000 7f7719c00700 1 --2- 172.21.15.131:0/1090590660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f770c0a0b30 0x7f770c0a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:12:06.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.607+0000 7f7712a00700 1 --2- 172.21.15.131:0/1090590660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f770c006d70 0x7f770c132fb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f771404ff70 tx=0x7f7714076680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:06.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.607+0000 7f770b400700 1 -- 172.21.15.131:0/1090590660 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7714076e90 con 0x7f770c006d70 2024-05-08T22:12:06.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.607+0000 7f770b400700 1 -- 172.21.15.131:0/1090590660 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f771406a9c0 con 0x7f770c006d70 2024-05-08T22:12:06.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.607+0000 7f770b400700 1 -- 172.21.15.131:0/1090590660 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7714069880 con 0x7f770c006d70 2024-05-08T22:12:06.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.607+0000 7f7713e00700 1 -- 172.21.15.131:0/1090590660 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f770c12ec40 con 0x7f770c006d70 2024-05-08T22:12:06.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.608+0000 7f7713e00700 1 -- 172.21.15.131:0/1090590660 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f770c12f070 con 0x7f770c006d70 2024-05-08T22:12:06.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.615+0000 7f770b400700 1 -- 172.21.15.131:0/1090590660 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f7714070020 con 0x7f770c006d70 2024-05-08T22:12:06.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.618+0000 7f770b400700 1 --2- 172.21.15.131:0/1090590660 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f76f80a13d0 0x7f76f80a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.619+0000 7f770b400700 1 -- 172.21.15.131:0/1090590660 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f7714142550 con 0x7f770c006d70 2024-05-08T22:12:06.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.621+0000 7f772d681b80 1 Processor -- start 2024-05-08T22:12:06.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.623+0000 7f772d681b80 1 -- start start 2024-05-08T22:12:06.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.623+0000 7f772d681b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5146a050 0x55bc5146a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.623+0000 7f772d681b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc5146a970 0x55bc51464350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.623+0000 7f772d681b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc51464890 0x55bc51464cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.623+0000 7f772d681b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc513bb150 con 0x55bc51464890 2024-05-08T22:12:06.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.623+0000 7f772d681b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc513b8b10 con 0x55bc5146a970 2024-05-08T22:12:06.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.623+0000 7f772d681b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc513b4400 con 0x55bc5146a050 2024-05-08T22:12:06.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.624+0000 7f7712a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5146a050 0x55bc5146a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.624+0000 7f7712a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5146a050 0x55bc5146a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43260/0 (socket says 172.21.15.131:43260) 2024-05-08T22:12:06.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.624+0000 7f7712a00700 1 -- 172.21.15.131:0/1684109689 learned_addr learned my addr 172.21.15.131:0/1684109689 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:06.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.624+0000 7f7712000700 1 --2- 172.21.15.131:0/1684109689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc5146a970 0x55bc51464350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.624+0000 7f7712000700 1 --2- 172.21.15.131:0/1090590660 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f76f80a13d0 0x7f76f80a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.624+0000 7f7712a00700 1 -- 172.21.15.131:0/1684109689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc5146a970 msgr2=0x55bc51464350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:06.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.625+0000 7f7712a00700 1 --2- 172.21.15.131:0/1684109689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc5146a970 0x55bc51464350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.625+0000 7f7712a00700 1 -- 172.21.15.131:0/1684109689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc51464890 msgr2=0x55bc51464cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:06.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.625+0000 7f7712a00700 1 --2- 172.21.15.131:0/1684109689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc51464890 0x55bc51464cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.625+0000 7f7712a00700 1 -- 172.21.15.131:0/1684109689 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc512353b0 con 0x55bc5146a050 2024-05-08T22:12:06.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.625+0000 7f7712000700 1 --2- 172.21.15.131:0/1684109689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc5146a970 0x55bc51464350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:12:06.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.626+0000 7f7712a00700 1 --2- 172.21.15.131:0/1684109689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5146a050 0x55bc5146a430 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f771410ff20 tx=0x7f7714110070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e68ece3bb8eefd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:06.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.626+0000 7f7712000700 1 --2- 172.21.15.131:0/1090590660 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f76f80a13d0 0x7f76f80a3890 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55bc514655b0 tx=0x7f7704006a90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:06.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.627+0000 7f76ffe00700 1 -- 172.21.15.131:0/1684109689 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771406ae10 con 0x55bc5146a050 2024-05-08T22:12:06.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.627+0000 7f76ffe00700 1 -- 172.21.15.131:0/1684109689 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7714110920 con 0x55bc5146a050 2024-05-08T22:12:06.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.628+0000 7f76ffe00700 1 -- 172.21.15.131:0/1684109689 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7714142380 con 0x55bc5146a050 2024-05-08T22:12:06.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.629+0000 7f772d681b80 1 -- 172.21.15.131:0/1684109689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5146a050 msgr2=0x55bc5146a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:06.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.629+0000 7f772d681b80 1 --2- 172.21.15.131:0/1684109689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5146a050 0x55bc5146a430 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f771410ff20 tx=0x7f7714110070 comp rx=0 tx=0).stop 2024-05-08T22:12:06.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.629+0000 7f772d681b80 1 -- 172.21.15.131:0/1684109689 shutdown_connections 2024-05-08T22:12:06.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.629+0000 7f772d681b80 1 --2- 172.21.15.131:0/1684109689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc51464890 0x55bc51464cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.629+0000 7f772d681b80 1 --2- 172.21.15.131:0/1684109689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5146a050 0x55bc5146a430 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.629+0000 7f772d681b80 1 --2- 172.21.15.131:0/1684109689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc5146a970 0x55bc51464350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.629+0000 7f772d681b80 1 -- 172.21.15.131:0/1684109689 >> 172.21.15.131:0/1684109689 conn(0x55bc513c2210 msgr2=0x55bc5145f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:06.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.630+0000 7f772d681b80 1 -- 172.21.15.131:0/1684109689 shutdown_connections 2024-05-08T22:12:06.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.630+0000 7f772d681b80 1 -- 172.21.15.131:0/1684109689 wait complete. 2024-05-08T22:12:06.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.632+0000 7f772d681b80 1 -- start start 2024-05-08T22:12:06.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.632+0000 7f772d681b80 1 Processor -- start 2024-05-08T22:12:06.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.634+0000 7f772d681b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc51464890 0x55bc514708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.634+0000 7f772d681b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc5146a050 0x55bc5146dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.634+0000 7f772d681b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5146a970 0x55bc5146e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.634+0000 7f772d681b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc51350560 con 0x55bc5146a970 2024-05-08T22:12:06.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.634+0000 7f772d681b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc51350200 con 0x55bc5146a050 2024-05-08T22:12:06.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.634+0000 7f772d681b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc512376a0 con 0x55bc51464890 2024-05-08T22:12:06.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.635+0000 7f772d681b80 10 client.0 _ll_register_callbacks cb 0x55bc51359b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:12:06.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.635+0000 7f7712a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc51464890 0x55bc514708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.635+0000 7f7712a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc51464890 0x55bc514708c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43270/0 (socket says 172.21.15.131:43270) 2024-05-08T22:12:06.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.635+0000 7f7712a00700 1 -- 172.21.15.131:0/667854638 learned_addr learned my addr 172.21.15.131:0/667854638 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:06.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.636+0000 7f7712a00700 1 -- 172.21.15.131:0/667854638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc5146a050 msgr2=0x55bc5146dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:06.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.636+0000 7f7712a00700 1 --2- 172.21.15.131:0/667854638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc5146a050 0x55bc5146dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.636+0000 7f7712a00700 1 -- 172.21.15.131:0/667854638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5146a970 msgr2=0x55bc5146e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:06.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.636+0000 7f7712a00700 1 --2- 172.21.15.131:0/667854638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5146a970 0x55bc5146e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.636+0000 7f7712a00700 1 -- 172.21.15.131:0/667854638 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f770c12f070 con 0x55bc51464890 2024-05-08T22:12:06.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.636+0000 7f7712a00700 1 --2- 172.21.15.131:0/667854638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc51464890 0x55bc514708c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f77140fb050 tx=0x7f771413f960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aca15e0679cf3ff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:06.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.636+0000 7f76fcc00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:12:06.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.637+0000 7f76fcc00700 1 -- 172.21.15.131:0/667854638 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771410f910 con 0x55bc51464890 2024-05-08T22:12:06.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.637+0000 7f76fcc00700 1 -- 172.21.15.131:0/667854638 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f7714110220 con 0x55bc51464890 2024-05-08T22:12:06.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.637+0000 7f772d681b80 1 -- 172.21.15.131:0/667854638 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc5130d270 con 0x55bc51464890 2024-05-08T22:12:06.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.638+0000 7f76fcc00700 1 -- 172.21.15.131:0/667854638 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7714110480 con 0x55bc51464890 2024-05-08T22:12:06.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.639+0000 7f772d681b80 10 client.26236 fetch_fsmap learned FSMap version 26 2024-05-08T22:12:06.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.639+0000 7f772d681b80 1 -- 172.21.15.131:0/667854638 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55bc511e1b60 con 0x55bc51464890 2024-05-08T22:12:06.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.640+0000 7f76fcc00700 1 -- 172.21.15.131:0/667854638 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bc5130d270 con 0x55bc51464890 2024-05-08T22:12:06.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.641+0000 7f76fcc00700 1 -- 172.21.15.131:0/667854638 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f771410c220 con 0x55bc51464890 2024-05-08T22:12:06.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.642+0000 7f772d681b80 10 client.26236 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:12:06.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.642+0000 7f772d681b80 10 client.26236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:12:06.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.642+0000 7f772d681b80 20 client.26236 populate_metadata read hostname 'smithi131' 2024-05-08T22:12:06.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.642+0000 7f772d681b80 1 --2- 172.21.15.131:0/667854638 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55bc5147dda0 0x55bc514801c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.644+0000 7f772d681b80 4 client.26236 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:12:06.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.644+0000 7f772d681b80 1 -- 172.21.15.131:0/667854638 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc51433810 con 0x55bc5147dda0 2024-05-08T22:12:06.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.645+0000 7f7712000700 1 --2- 172.21.15.131:0/667854638 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55bc5147dda0 0x55bc514801c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.647+0000 7f76fcc00700 10 client.26236 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:12:06.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.647+0000 7f7712000700 1 --2- 172.21.15.131:0/667854638 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55bc5147dda0 0x55bc514801c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11b1da57a0243798 server_cookie=a79c037d48e42f0 in_seq=0 out_seq=0 2024-05-08T22:12:06.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.650+0000 7f76fcc00700 1 -- 172.21.15.131:0/667854638 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bc51433810 con 0x55bc5147dda0 2024-05-08T22:12:06.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.650+0000 7f76fcc00700 10 client.26236 handle_command_reply: tid=0 2024-05-08T22:12:06.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.650+0000 7f772d681b80 1 client.26236 shutdown 2024-05-08T22:12:06.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.651+0000 7f772d681b80 1 -- 172.21.15.131:0/667854638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc51464890 msgr2=0x55bc514708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:06.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.651+0000 7f772d681b80 1 --2- 172.21.15.131:0/667854638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc51464890 0x55bc514708c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f77140fb050 tx=0x7f771413f960 comp rx=0 tx=0).stop 2024-05-08T22:12:06.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.651+0000 7f772d681b80 1 -- 172.21.15.131:0/667854638 shutdown_connections 2024-05-08T22:12:06.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.651+0000 7f772d681b80 1 --2- 172.21.15.131:0/667854638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc5146a970 0x55bc5146e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.651+0000 7f772d681b80 1 --2- 172.21.15.131:0/667854638 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55bc5147dda0 0x55bc514801c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.651+0000 7f772d681b80 1 --2- 172.21.15.131:0/667854638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc51464890 0x55bc514708c0 secure :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f77140fb050 tx=0x7f771413f960 comp rx=0 tx=0).stop 2024-05-08T22:12:06.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.651+0000 7f772d681b80 1 --2- 172.21.15.131:0/667854638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc5146a050 0x55bc5146dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.651+0000 7f76fcc00700 0 client.26236 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:12:06.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.651+0000 7f772d681b80 1 -- 172.21.15.131:0/667854638 >> 172.21.15.131:0/667854638 conn(0x55bc513c2210 msgr2=0x55bc5145f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:06.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.654+0000 7f772d681b80 1 -- 172.21.15.131:0/667854638 shutdown_connections 2024-05-08T22:12:06.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.654+0000 7f772d681b80 1 -- 172.21.15.131:0/667854638 wait complete. 2024-05-08T22:12:06.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.657+0000 7f772d681b80 20 client.26236 trim_cache size 0 max 16384 2024-05-08T22:12:06.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.688+0000 7f772d681b80 1 Processor -- start 2024-05-08T22:12:06.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.689+0000 7f772d681b80 1 -- start start 2024-05-08T22:12:06.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.689+0000 7f772d681b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc5146a050 0x55bc5146e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.689+0000 7f772d681b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5146a970 0x55bc51467c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.689+0000 7f772d681b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc514681c0 0x55bc51484b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.689+0000 7f772d681b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5147abb0 con 0x55bc514681c0 2024-05-08T22:12:06.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.689+0000 7f772d681b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc51468620 con 0x55bc5146a050 2024-05-08T22:12:06.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.689+0000 7f772d681b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc51464350 con 0x55bc5146a970 2024-05-08T22:12:06.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.690+0000 7f7712000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5146a970 0x55bc51467c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.690+0000 7f7712000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5146a970 0x55bc51467c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43286/0 (socket says 172.21.15.131:43286) 2024-05-08T22:12:06.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.690+0000 7f7712000700 1 -- 172.21.15.131:0/268412666 learned_addr learned my addr 172.21.15.131:0/268412666 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:06.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.690+0000 7f7712a00700 1 --2- 172.21.15.131:0/268412666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc5146a050 0x55bc5146e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.690+0000 7f7712a00700 1 -- 172.21.15.131:0/268412666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5146a970 msgr2=0x55bc51467c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:06.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.690+0000 7f7712a00700 1 --2- 172.21.15.131:0/268412666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5146a970 0x55bc51467c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.691+0000 7f7712a00700 1 -- 172.21.15.131:0/268412666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc514681c0 msgr2=0x55bc51484b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:06.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.691+0000 7f7712a00700 1 --2- 172.21.15.131:0/268412666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc514681c0 0x55bc51484b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.691+0000 7f7712a00700 1 -- 172.21.15.131:0/268412666 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc5137a560 con 0x55bc5146a050 2024-05-08T22:12:06.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.691+0000 7f7712a00700 1 --2- 172.21.15.131:0/268412666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc5146a050 0x55bc5146e1b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f771410fed0 tx=0x7f7714110070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3eab79e8dcf43bef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:06.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.691+0000 7f7708c00700 1 -- 172.21.15.131:0/268412666 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7714110220 con 0x55bc5146a050 2024-05-08T22:12:06.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.691+0000 7f7708c00700 1 -- 172.21.15.131:0/268412666 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7714117870 con 0x55bc5146a050 2024-05-08T22:12:06.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.691+0000 7f7708c00700 1 -- 172.21.15.131:0/268412666 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771410f970 con 0x55bc5146a050 2024-05-08T22:12:06.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.692+0000 7f772d681b80 1 -- 172.21.15.131:0/268412666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc5146a050 msgr2=0x55bc5146e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:06.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.692+0000 7f772d681b80 1 --2- 172.21.15.131:0/268412666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc5146a050 0x55bc5146e1b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f771410fed0 tx=0x7f7714110070 comp rx=0 tx=0).stop 2024-05-08T22:12:06.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.695+0000 7f772d681b80 1 -- 172.21.15.131:0/268412666 shutdown_connections 2024-05-08T22:12:06.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.695+0000 7f772d681b80 1 --2- 172.21.15.131:0/268412666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc514681c0 0x55bc51484b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.695+0000 7f772d681b80 1 --2- 172.21.15.131:0/268412666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc5146a050 0x55bc5146e1b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.695+0000 7f772d681b80 1 --2- 172.21.15.131:0/268412666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5146a970 0x55bc51467c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.695+0000 7f772d681b80 1 -- 172.21.15.131:0/268412666 >> 172.21.15.131:0/268412666 conn(0x55bc513c2210 msgr2=0x55bc51460820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:06.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.695+0000 7f772d681b80 1 -- 172.21.15.131:0/268412666 shutdown_connections 2024-05-08T22:12:06.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.696+0000 7f772d681b80 1 -- 172.21.15.131:0/268412666 wait complete. 2024-05-08T22:12:06.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.696+0000 7f772d681b80 1 -- start start 2024-05-08T22:12:06.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.697+0000 7f772d681b80 1 Processor -- start 2024-05-08T22:12:06.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.698+0000 7f772d681b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc514681c0 0x55bc514755c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.698+0000 7f772d681b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5146a050 0x55bc51466db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.699+0000 7f772d681b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc5146a970 0x55bc514673e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.699+0000 7f772d681b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc51465c00 con 0x55bc514681c0 2024-05-08T22:12:06.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.699+0000 7f772d681b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc51465d70 con 0x55bc5146a970 2024-05-08T22:12:06.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.699+0000 7f772d681b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc51465f30 con 0x55bc5146a050 2024-05-08T22:12:06.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.699+0000 7f772d681b80 10 client.0 _ll_register_callbacks cb 0x55bc51359b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:12:06.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.699+0000 7f7712000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5146a050 0x55bc51466db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.699+0000 7f7712000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5146a050 0x55bc51466db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43290/0 (socket says 172.21.15.131:43290) 2024-05-08T22:12:06.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.699+0000 7f7712000700 1 -- 172.21.15.131:0/324889916 learned_addr learned my addr 172.21.15.131:0/324889916 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:06.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.699+0000 7f7719c00700 1 --2- 172.21.15.131:0/324889916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc5146a970 0x55bc514673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.700+0000 7f7712a00700 1 --2- 172.21.15.131:0/324889916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc514681c0 0x55bc514755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.700+0000 7f7719c00700 1 -- 172.21.15.131:0/324889916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5146a050 msgr2=0x55bc51466db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:06.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.700+0000 7f7719c00700 1 --2- 172.21.15.131:0/324889916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5146a050 0x55bc51466db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.700+0000 7f7719c00700 1 -- 172.21.15.131:0/324889916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc514681c0 msgr2=0x55bc514755c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:06.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.700+0000 7f7719c00700 1 --2- 172.21.15.131:0/324889916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc514681c0 0x55bc514755c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.700+0000 7f7719c00700 1 -- 172.21.15.131:0/324889916 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc511e1b60 con 0x55bc5146a970 2024-05-08T22:12:06.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.700+0000 7f7712a00700 1 --2- 172.21.15.131:0/324889916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc514681c0 0x55bc514755c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:12:06.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.701+0000 7f76fcc00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:12:06.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.701+0000 7f7719c00700 1 --2- 172.21.15.131:0/324889916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc5146a970 0x55bc514673e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f770000b640 tx=0x7f7700005650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91172311c7a4b993 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:06.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.702+0000 7f76fcc00700 1 -- 172.21.15.131:0/324889916 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7700004050 con 0x55bc5146a970 2024-05-08T22:12:06.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.702+0000 7f76fcc00700 1 -- 172.21.15.131:0/324889916 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f77000041b0 con 0x55bc5146a970 2024-05-08T22:12:06.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.703+0000 7f772d681b80 1 -- 172.21.15.131:0/324889916 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7714110920 con 0x55bc5146a970 2024-05-08T22:12:06.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.704+0000 7f76fcc00700 1 -- 172.21.15.131:0/324889916 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7700004050 con 0x55bc5146a970 2024-05-08T22:12:06.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.704+0000 7f772d681b80 10 client.26189 fetch_fsmap learned FSMap version 26 2024-05-08T22:12:06.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.704+0000 7f772d681b80 1 -- 172.21.15.131:0/324889916 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55bc51239cd0 con 0x55bc5146a970 2024-05-08T22:12:06.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.705+0000 7f76fcc00700 1 -- 172.21.15.131:0/324889916 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7714110920 con 0x55bc5146a970 2024-05-08T22:12:06.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.705+0000 7f76fcc00700 1 -- 172.21.15.131:0/324889916 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7700011440 con 0x55bc5146a970 2024-05-08T22:12:06.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.705+0000 7f772d681b80 10 client.26189 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:12:06.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.705+0000 7f772d681b80 10 client.26189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:12:06.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.705+0000 7f772d681b80 20 client.26189 populate_metadata read hostname 'smithi131' 2024-05-08T22:12:06.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.705+0000 7f772d681b80 1 --2- 172.21.15.131:0/324889916 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55bc514d50a0 0x55bc514d74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:06.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.705+0000 7f772d681b80 4 client.26189 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:12:06.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.705+0000 7f772d681b80 1 -- 172.21.15.131:0/324889916 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bc513acec0 con 0x55bc514d50a0 2024-05-08T22:12:06.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.707+0000 7f7712a00700 1 --2- 172.21.15.131:0/324889916 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55bc514d50a0 0x55bc514d74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:06.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.708+0000 7f7712a00700 1 --2- 172.21.15.131:0/324889916 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55bc514d50a0 0x55bc514d74c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eae4188b9e823de6 server_cookie=9e4be7d31994ab8e in_seq=0 out_seq=0 2024-05-08T22:12:06.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.708+0000 7f76fcc00700 10 client.26189 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:12:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.713+0000 7f76fcc00700 1 -- 172.21.15.131:0/324889916 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bc513acec0 con 0x55bc514d50a0 2024-05-08T22:12:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.713+0000 7f76fcc00700 10 client.26189 handle_command_reply: tid=0 2024-05-08T22:12:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.713+0000 7f772d681b80 1 client.26189 shutdown 2024-05-08T22:12:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.714+0000 7f772d681b80 1 -- 172.21.15.131:0/324889916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc5146a970 msgr2=0x55bc514673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.714+0000 7f772d681b80 1 --2- 172.21.15.131:0/324889916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc5146a970 0x55bc514673e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f770000b640 tx=0x7f7700005650 comp rx=0 tx=0).stop 2024-05-08T22:12:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.715+0000 7f772d681b80 1 -- 172.21.15.131:0/324889916 shutdown_connections 2024-05-08T22:12:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.715+0000 7f772d681b80 1 --2- 172.21.15.131:0/324889916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc5146a970 0x55bc514673e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.716+0000 7f772d681b80 1 --2- 172.21.15.131:0/324889916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc514681c0 0x55bc514755c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.716+0000 7f772d681b80 1 --2- 172.21.15.131:0/324889916 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55bc514d50a0 0x55bc514d74c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.716+0000 7f772d681b80 1 --2- 172.21.15.131:0/324889916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc5146a050 0x55bc51466db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.716+0000 7f772d681b80 1 -- 172.21.15.131:0/324889916 >> 172.21.15.131:0/324889916 conn(0x55bc513c2210 msgr2=0x55bc51484b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.716+0000 7f76fcc00700 0 client.26189 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:12:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.716+0000 7f772d681b80 1 -- 172.21.15.131:0/324889916 shutdown_connections 2024-05-08T22:12:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.716+0000 7f772d681b80 1 -- 172.21.15.131:0/324889916 wait complete. 2024-05-08T22:12:06.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.718+0000 7f772d681b80 20 client.26189 trim_cache size 0 max 16384 2024-05-08T22:12:06.721 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:12:06.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.719+0000 7f7713e00700 1 -- 172.21.15.131:0/1090590660 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f76f80a13d0 msgr2=0x7f76f80a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:06.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.719+0000 7f7713e00700 1 --2- 172.21.15.131:0/1090590660 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f76f80a13d0 0x7f76f80a3890 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55bc514655b0 tx=0x7f7704006a90 comp rx=0 tx=0).stop 2024-05-08T22:12:06.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.719+0000 7f7713e00700 1 -- 172.21.15.131:0/1090590660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f770c006d70 msgr2=0x7f770c132fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:06.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.719+0000 7f7713e00700 1 --2- 172.21.15.131:0/1090590660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f770c006d70 0x7f770c132fb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f771404ff70 tx=0x7f7714076680 comp rx=0 tx=0).stop 2024-05-08T22:12:06.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.719+0000 7f7713e00700 1 -- 172.21.15.131:0/1090590660 shutdown_connections 2024-05-08T22:12:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.719+0000 7f7713e00700 1 --2- 172.21.15.131:0/1090590660 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f76f80a13d0 0x7f76f80a3890 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.719+0000 7f7713e00700 1 --2- 172.21.15.131:0/1090590660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f770c0a0b30 0x7f770c0a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.719+0000 7f7713e00700 1 --2- 172.21.15.131:0/1090590660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f770c006d70 0x7f770c132fb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.719+0000 7f7713e00700 1 --2- 172.21.15.131:0/1090590660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f770c09b040 0x7f770c1334f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.719+0000 7f7713e00700 1 -- 172.21.15.131:0/1090590660 >> 172.21.15.131:0/1090590660 conn(0x7f770c00a7a0 msgr2=0x7f770c0999f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.720+0000 7f7713e00700 1 -- 172.21.15.131:0/1090590660 shutdown_connections 2024-05-08T22:12:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:06.720+0000 7f7713e00700 1 -- 172.21.15.131:0/1090590660 wait complete. 2024-05-08T22:12:07.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:06 smithi131 ceph-mon[37260]: pgmap v147: 129 pgs: 129 active+clean; 677 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.0 MiB/s wr, 854 op/s 2024-05-08T22:12:07.275 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:06 smithi157 ceph-mon[77746]: pgmap v147: 129 pgs: 129 active+clean; 677 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.0 MiB/s wr, 854 op/s 2024-05-08T22:12:07.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:06 smithi132 ceph-mon[78180]: pgmap v147: 129 pgs: 129 active+clean; 677 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.0 MiB/s wr, 854 op/s 2024-05-08T22:12:07.767 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T22:12:07.767 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T22:12:09.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:08 smithi131 ceph-mon[37260]: pgmap v148: 129 pgs: 129 active+clean; 677 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.1 MiB/s wr, 691 op/s 2024-05-08T22:12:09.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:08 smithi157 ceph-mon[77746]: pgmap v148: 129 pgs: 129 active+clean; 677 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.1 MiB/s wr, 691 op/s 2024-05-08T22:12:09.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:08 smithi132 ceph-mon[78180]: pgmap v148: 129 pgs: 129 active+clean; 677 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.1 MiB/s wr, 691 op/s 2024-05-08T22:12:11.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.245+0000 7f988f400700 1 -- 172.21.15.131:0/870158222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f988811a780 msgr2=0x7f988811cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:11.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.245+0000 7f988f400700 1 --2- 172.21.15.131:0/870158222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f988811a780 0x7f988811cb70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9878009990 tx=0x7f9878009ca0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.256+0000 7f988f400700 1 -- 172.21.15.131:0/870158222 shutdown_connections 2024-05-08T22:12:11.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.256+0000 7f988f400700 1 --2- 172.21.15.131:0/870158222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f988811a780 0x7f988811cb70 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.256+0000 7f988f400700 1 --2- 172.21.15.131:0/870158222 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f988810f6a0 0x7f9888107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.256+0000 7f988f400700 1 --2- 172.21.15.131:0/870158222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f988810fb10 0x7f98881084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.256+0000 7f988f400700 1 -- 172.21.15.131:0/870158222 >> 172.21.15.131:0/870158222 conn(0x7f988806bff0 msgr2=0x7f988806c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:11.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.256+0000 7f988f400700 1 -- 172.21.15.131:0/870158222 shutdown_connections 2024-05-08T22:12:11.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.262+0000 7f988f400700 1 -- 172.21.15.131:0/870158222 wait complete. 2024-05-08T22:12:11.275 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:10 smithi157 ceph-mon[77746]: pgmap v149: 129 pgs: 129 active+clean; 703 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.6 MiB/s wr, 876 op/s 2024-05-08T22:12:11.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.273+0000 7f988f400700 1 Processor -- start 2024-05-08T22:12:11.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.282+0000 7f988f400700 1 -- start start 2024-05-08T22:12:11.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.283+0000 7f988f400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f988810f6a0 0x7f9888118a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:11.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.283+0000 7f988f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f988810fb10 0x7f9888118fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:11.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.283+0000 7f988f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f988811a780 0x7f9888111b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:11.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.283+0000 7f988ca00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f988810f6a0 0x7f9888118a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:11.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.283+0000 7f988ca00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f988810f6a0 0x7f9888118a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:41328/0 (socket says 172.21.15.131:41328) 2024-05-08T22:12:11.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.283+0000 7f988d400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f988811a780 0x7f9888111b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:11.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.283+0000 7f988d400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f988811a780 0x7f9888111b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43306/0 (socket says 172.21.15.131:43306) 2024-05-08T22:12:11.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.283+0000 7f9887e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f988810fb10 0x7f9888118fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:11.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.284+0000 7f988ca00700 1 -- 172.21.15.131:0/3018782188 learned_addr learned my addr 172.21.15.131:0/3018782188 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:11.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.284+0000 7f988f400700 1 -- 172.21.15.131:0/3018782188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98881122d0 con 0x7f988810fb10 2024-05-08T22:12:11.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.289+0000 7f988f400700 1 -- 172.21.15.131:0/3018782188 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9888112440 con 0x7f988810f6a0 2024-05-08T22:12:11.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.289+0000 7f988f400700 1 -- 172.21.15.131:0/3018782188 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9888112600 con 0x7f988811a780 2024-05-08T22:12:11.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.302+0000 7f988d400700 1 -- 172.21.15.131:0/3018782188 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f988810f6a0 msgr2=0x7f9888118a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:11.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.302+0000 7f988d400700 1 --2- 172.21.15.131:0/3018782188 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f988810f6a0 0x7f9888118a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.302+0000 7f988d400700 1 -- 172.21.15.131:0/3018782188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f988810fb10 msgr2=0x7f9888118fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:11.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.302+0000 7f988d400700 1 --2- 172.21.15.131:0/3018782188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f988810fb10 0x7f9888118fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.302+0000 7f988d400700 1 -- 172.21.15.131:0/3018782188 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98780095f0 con 0x7f988811a780 2024-05-08T22:12:11.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.302+0000 7f988d400700 1 --2- 172.21.15.131:0/3018782188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f988811a780 0x7f9888111b00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f988806a300 tx=0x7f9878003e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:11.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.309+0000 7f9885600700 1 -- 172.21.15.131:0/3018782188 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987801d070 con 0x7f988811a780 2024-05-08T22:12:11.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.309+0000 7f9885600700 1 -- 172.21.15.131:0/3018782188 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f987800f460 con 0x7f988811a780 2024-05-08T22:12:11.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.309+0000 7f9885600700 1 -- 172.21.15.131:0/3018782188 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9878017650 con 0x7f988811a780 2024-05-08T22:12:11.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.309+0000 7f988f400700 1 -- 172.21.15.131:0/3018782188 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9888112820 con 0x7f988811a780 2024-05-08T22:12:11.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.309+0000 7f988f400700 1 -- 172.21.15.131:0/3018782188 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98881167a0 con 0x7f988811a780 2024-05-08T22:12:11.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.325+0000 7f9885600700 1 -- 172.21.15.131:0/3018782188 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f987800f5d0 con 0x7f988811a780 2024-05-08T22:12:11.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.328+0000 7f9885600700 1 --2- 172.21.15.131:0/3018782188 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f986c0a1420 0x7f986c0a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:11.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.329+0000 7f988ca00700 1 --2- 172.21.15.131:0/3018782188 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f986c0a1420 0x7f986c0a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:11.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.330+0000 7f988ca00700 1 --2- 172.21.15.131:0/3018782188 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f986c0a1420 0x7f986c0a38e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9880009fd0 tx=0x7f988000b040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:11.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.333+0000 7f9885600700 1 -- 172.21.15.131:0/3018782188 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f98780ce160 con 0x7f988811a780 2024-05-08T22:12:11.339 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:10 smithi132 ceph-mon[78180]: pgmap v149: 129 pgs: 129 active+clean; 703 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.6 MiB/s wr, 876 op/s 2024-05-08T22:12:11.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.372+0000 7f98a2fbcb80 1 Processor -- start 2024-05-08T22:12:11.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.380+0000 7f98a2fbcb80 1 -- start start 2024-05-08T22:12:11.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.381+0000 7f98a2fbcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ec2afe0e80 0x55ec2afe1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:11.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.381+0000 7f98a2fbcb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afe17a0 0x55ec2afdb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:11.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.381+0000 7f98a2fbcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec2afdb910 0x55ec2afdbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:11.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.381+0000 7f988ca00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ec2afe0e80 0x55ec2afe1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:11.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.382+0000 7f988ca00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ec2afe0e80 0x55ec2afe1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59394/0 (socket says 172.21.15.131:59394) 2024-05-08T22:12:11.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.384+0000 7f988d400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec2afdb910 0x55ec2afdbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:11.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.384+0000 7f98a2fbcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2af321d0 con 0x55ec2afe0e80 2024-05-08T22:12:11.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.384+0000 7f988d400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec2afdb910 0x55ec2afdbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43310/0 (socket says 172.21.15.131:43310) 2024-05-08T22:12:11.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.384+0000 7f988d400700 1 -- 172.21.15.131:0/3318142 learned_addr learned my addr 172.21.15.131:0/3318142 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:11.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.384+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/3318142 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2af2fb90 con 0x55ec2afe17a0 2024-05-08T22:12:11.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.384+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/3318142 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2af2b480 con 0x55ec2afdb910 2024-05-08T22:12:11.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.384+0000 7f9887e00700 1 --2- 172.21.15.131:0/3318142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afe17a0 0x55ec2afdb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:11.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.385+0000 7f9887e00700 1 -- 172.21.15.131:0/3318142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec2afdb910 msgr2=0x55ec2afdbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:11.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.385+0000 7f9887e00700 1 --2- 172.21.15.131:0/3318142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec2afdb910 0x55ec2afdbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.385+0000 7f9887e00700 1 -- 172.21.15.131:0/3318142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ec2afe0e80 msgr2=0x55ec2afe1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:11.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.385+0000 7f9887e00700 1 --2- 172.21.15.131:0/3318142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ec2afe0e80 0x55ec2afe1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.385+0000 7f9887e00700 1 -- 172.21.15.131:0/3318142 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec2adac430 con 0x55ec2afe17a0 2024-05-08T22:12:11.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.385+0000 7f988ca00700 1 --2- 172.21.15.131:0/3318142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ec2afe0e80 0x55ec2afe1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-08T22:12:11.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.385+0000 7f9887e00700 1 --2- 172.21.15.131:0/3318142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afe17a0 0x55ec2afdb3d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f988810ff30 tx=0x7f987400ba90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fe02c272d66c555 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:11.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:10 smithi131 ceph-mon[37260]: pgmap v149: 129 pgs: 129 active+clean; 703 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.6 MiB/s wr, 876 op/s 2024-05-08T22:12:11.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.392+0000 7f987dc00700 1 -- 172.21.15.131:0/3318142 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98740125f0 con 0x55ec2afe17a0 2024-05-08T22:12:11.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.392+0000 7f987dc00700 1 -- 172.21.15.131:0/3318142 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9874012c10 con 0x55ec2afe17a0 2024-05-08T22:12:11.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.392+0000 7f987dc00700 1 -- 172.21.15.131:0/3318142 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9874013db0 con 0x55ec2afe17a0 2024-05-08T22:12:11.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.395+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/3318142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afe17a0 msgr2=0x55ec2afdb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:11.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.395+0000 7f98a2fbcb80 1 --2- 172.21.15.131:0/3318142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afe17a0 0x55ec2afdb3d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f988810ff30 tx=0x7f987400ba90 comp rx=0 tx=0).stop 2024-05-08T22:12:11.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.416+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/3318142 shutdown_connections 2024-05-08T22:12:11.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.416+0000 7f98a2fbcb80 1 --2- 172.21.15.131:0/3318142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec2afdb910 0x55ec2afdbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.416+0000 7f98a2fbcb80 1 --2- 172.21.15.131:0/3318142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ec2afe0e80 0x55ec2afe1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.416+0000 7f98a2fbcb80 1 --2- 172.21.15.131:0/3318142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afe17a0 0x55ec2afdb3d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.416+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/3318142 >> 172.21.15.131:0/3318142 conn(0x55ec2af39290 msgr2=0x55ec2afd5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:11.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.416+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/3318142 shutdown_connections 2024-05-08T22:12:11.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.423+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/3318142 wait complete. 2024-05-08T22:12:11.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.425+0000 7f98a2fbcb80 1 -- start start 2024-05-08T22:12:11.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.426+0000 7f98a2fbcb80 1 Processor -- start 2024-05-08T22:12:11.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.433+0000 7f98a2fbcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec2afdb910 0x55ec2afe4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:11.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.433+0000 7f98a2fbcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ec2afe0e80 0x55ec2afe5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:11.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.433+0000 7f98a2fbcb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afe17a0 0x55ec2afe5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:11.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.433+0000 7f98a2fbcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2aec75e0 con 0x55ec2afe0e80 2024-05-08T22:12:11.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.433+0000 7f98a2fbcb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2aec7280 con 0x55ec2afe17a0 2024-05-08T22:12:11.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.433+0000 7f98a2fbcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2adae720 con 0x55ec2afdb910 2024-05-08T22:12:11.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.433+0000 7f98a2fbcb80 10 client.0 _ll_register_callbacks cb 0x55ec2aed0bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:12:11.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.434+0000 7f988ca00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec2afdb910 0x55ec2afe4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:11.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.434+0000 7f988ca00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec2afdb910 0x55ec2afe4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43312/0 (socket says 172.21.15.131:43312) 2024-05-08T22:12:11.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.434+0000 7f988ca00700 1 -- 172.21.15.131:0/2622865536 learned_addr learned my addr 172.21.15.131:0/2622865536 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:11.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.434+0000 7f9887e00700 1 --2- 172.21.15.131:0/2622865536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ec2afe0e80 0x55ec2afe5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:11.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.435+0000 7f988ca00700 1 -- 172.21.15.131:0/2622865536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afe17a0 msgr2=0x55ec2afe5770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:12:11.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.436+0000 7f988d400700 1 --2- 172.21.15.131:0/2622865536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afe17a0 0x55ec2afe5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:11.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.436+0000 7f988ca00700 1 --2- 172.21.15.131:0/2622865536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afe17a0 0x55ec2afe5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.436+0000 7f988ca00700 1 -- 172.21.15.131:0/2622865536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ec2afe0e80 msgr2=0x55ec2afe5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:11.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.436+0000 7f988ca00700 1 --2- 172.21.15.131:0/2622865536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ec2afe0e80 0x55ec2afe5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.436+0000 7f988ca00700 1 -- 172.21.15.131:0/2622865536 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f987400e040 con 0x55ec2afdb910 2024-05-08T22:12:11.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.436+0000 7f9887e00700 1 --2- 172.21.15.131:0/2622865536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ec2afe0e80 0x55ec2afe5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:12:11.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.436+0000 7f988ca00700 1 --2- 172.21.15.131:0/2622865536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec2afdb910 0x55ec2afe4cf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f98800156f0 tx=0x7f9880004b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b54812adc8673770 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:11.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.436+0000 7f988d400700 1 --2- 172.21.15.131:0/2622865536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afe17a0 0x55ec2afe5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:12:11.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.442+0000 7f9872000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:12:11.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.442+0000 7f9872000700 1 -- 172.21.15.131:0/2622865536 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880020670 con 0x55ec2afdb910 2024-05-08T22:12:11.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.442+0000 7f9872000700 1 -- 172.21.15.131:0/2622865536 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f98800207d0 con 0x55ec2afdb910 2024-05-08T22:12:11.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.451+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/2622865536 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec2ae842f0 con 0x55ec2afdb910 2024-05-08T22:12:11.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.455+0000 7f9872000700 1 -- 172.21.15.131:0/2622865536 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9880020990 con 0x55ec2afdb910 2024-05-08T22:12:11.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.455+0000 7f98a2fbcb80 10 client.26251 fetch_fsmap learned FSMap version 26 2024-05-08T22:12:11.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.455+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/2622865536 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55ec2ad58be0 con 0x55ec2afdb910 2024-05-08T22:12:11.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.468+0000 7f9872000700 1 -- 172.21.15.131:0/2622865536 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ec2ae842f0 con 0x55ec2afdb910 2024-05-08T22:12:11.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.468+0000 7f9872000700 1 -- 172.21.15.131:0/2622865536 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f9880029cf0 con 0x55ec2afdb910 2024-05-08T22:12:11.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.474+0000 7f98a2fbcb80 10 client.26251 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:12:11.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.474+0000 7f98a2fbcb80 10 client.26251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:12:11.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.474+0000 7f98a2fbcb80 20 client.26251 populate_metadata read hostname 'smithi131' 2024-05-08T22:12:11.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.474+0000 7f98a2fbcb80 1 --2- 172.21.15.131:0/2622865536 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ec2aff5370 0x55ec2aff7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:11.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.474+0000 7f98a2fbcb80 4 client.26251 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:12:11.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.474+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/2622865536 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec2afaa890 con 0x55ec2aff5370 2024-05-08T22:12:11.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.478+0000 7f9887e00700 1 --2- 172.21.15.131:0/2622865536 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ec2aff5370 0x55ec2aff7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:11.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.489+0000 7f9887e00700 1 --2- 172.21.15.131:0/2622865536 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ec2aff5370 0x55ec2aff7790 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=355747e61506347f server_cookie=690d68e03a9bc7 in_seq=0 out_seq=0 2024-05-08T22:12:11.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.489+0000 7f9872000700 10 client.26251 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:12:11.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.491+0000 7f9872000700 1 -- 172.21.15.131:0/2622865536 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ec2afaa890 con 0x55ec2aff5370 2024-05-08T22:12:11.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.491+0000 7f9872000700 10 client.26251 handle_command_reply: tid=0 2024-05-08T22:12:11.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.500+0000 7f98a2fbcb80 1 client.26251 shutdown 2024-05-08T22:12:11.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.508+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/2622865536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec2afdb910 msgr2=0x55ec2afe4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:11.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.508+0000 7f98a2fbcb80 1 --2- 172.21.15.131:0/2622865536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec2afdb910 0x55ec2afe4cf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f98800156f0 tx=0x7f9880004b60 comp rx=0 tx=0).stop 2024-05-08T22:12:11.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.515+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/2622865536 shutdown_connections 2024-05-08T22:12:11.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.515+0000 7f98a2fbcb80 1 --2- 172.21.15.131:0/2622865536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afe17a0 0x55ec2afe5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.515+0000 7f98a2fbcb80 1 --2- 172.21.15.131:0/2622865536 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ec2aff5370 0x55ec2aff7790 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.515+0000 7f98a2fbcb80 1 --2- 172.21.15.131:0/2622865536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec2afdb910 0x55ec2afe4cf0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.515+0000 7f98a2fbcb80 1 --2- 172.21.15.131:0/2622865536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ec2afe0e80 0x55ec2afe5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.515+0000 7f9872000700 0 client.26251 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:12:11.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.515+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/2622865536 >> 172.21.15.131:0/2622865536 conn(0x55ec2af39290 msgr2=0x55ec2afd5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:11.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.518+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/2622865536 shutdown_connections 2024-05-08T22:12:11.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.522+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/2622865536 wait complete. 2024-05-08T22:12:11.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.526+0000 7f98a2fbcb80 20 client.26251 trim_cache size 0 max 16384 2024-05-08T22:12:11.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.701+0000 7f98a2fbcb80 1 Processor -- start 2024-05-08T22:12:11.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.702+0000 7f98a2fbcb80 1 -- start start 2024-05-08T22:12:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.706+0000 7f98a2fbcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ec2afdb910 0x55ec2afdb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.706+0000 7f98a2fbcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec2afe0e80 0x55ec2afe04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.706+0000 7f98a2fbcb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afe17a0 0x55ec2afd5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.706+0000 7f98a2fbcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2afaa890 con 0x55ec2afdb910 2024-05-08T22:12:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.706+0000 7f98a2fbcb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2ae842f0 con 0x55ec2afe17a0 2024-05-08T22:12:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.706+0000 7f98a2fbcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2aff2920 con 0x55ec2afe0e80 2024-05-08T22:12:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.706+0000 7f988d400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afe17a0 0x55ec2afd5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.706+0000 7f988d400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afe17a0 0x55ec2afd5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:41354/0 (socket says 172.21.15.131:41354) 2024-05-08T22:12:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.706+0000 7f988d400700 1 -- 172.21.15.131:0/2917755862 learned_addr learned my addr 172.21.15.131:0/2917755862 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.706+0000 7f988d400700 1 -- 172.21.15.131:0/2917755862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec2afe0e80 msgr2=0x55ec2afe04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.707+0000 7f988d400700 1 --2- 172.21.15.131:0/2917755862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec2afe0e80 0x55ec2afe04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.707+0000 7f988d400700 1 -- 172.21.15.131:0/2917755862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ec2afdb910 msgr2=0x55ec2afdb3d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:12:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.707+0000 7f988d400700 1 --2- 172.21.15.131:0/2917755862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ec2afdb910 0x55ec2afdb3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.707+0000 7f988d400700 1 -- 172.21.15.131:0/2917755862 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec2afd2560 con 0x55ec2afe17a0 2024-05-08T22:12:11.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.707+0000 7f988d400700 1 --2- 172.21.15.131:0/2917755862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afe17a0 0x55ec2afd5da0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9878005b40 tx=0x7f9878009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c9c1c80c7ba053 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:11.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.708+0000 7f987e600700 1 -- 172.21.15.131:0/2917755862 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9878016e80 con 0x55ec2afe17a0 2024-05-08T22:12:11.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.708+0000 7f987e600700 1 -- 172.21.15.131:0/2917755862 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f987809b670 con 0x55ec2afe17a0 2024-05-08T22:12:11.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.708+0000 7f987e600700 1 -- 172.21.15.131:0/2917755862 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987809b1d0 con 0x55ec2afe17a0 2024-05-08T22:12:11.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.709+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/2917755862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afe17a0 msgr2=0x55ec2afd5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:11.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.709+0000 7f98a2fbcb80 1 --2- 172.21.15.131:0/2917755862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afe17a0 0x55ec2afd5da0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9878005b40 tx=0x7f9878009ca0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.713+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/2917755862 shutdown_connections 2024-05-08T22:12:11.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.714+0000 7f98a2fbcb80 1 --2- 172.21.15.131:0/2917755862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afe17a0 0x55ec2afd5da0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.714+0000 7f98a2fbcb80 1 --2- 172.21.15.131:0/2917755862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ec2afdb910 0x55ec2afdb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.714+0000 7f98a2fbcb80 1 --2- 172.21.15.131:0/2917755862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec2afe0e80 0x55ec2afe04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.714+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/2917755862 >> 172.21.15.131:0/2917755862 conn(0x55ec2af39290 msgr2=0x55ec2afdfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:11.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.719+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/2917755862 shutdown_connections 2024-05-08T22:12:11.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.719+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/2917755862 wait complete. 2024-05-08T22:12:11.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.720+0000 7f98a2fbcb80 1 -- start start 2024-05-08T22:12:11.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.720+0000 7f98a2fbcb80 1 Processor -- start 2024-05-08T22:12:11.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.721+0000 7f98a2fbcb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afdb910 0x55ec2afeadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:11.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.725+0000 7f98a2fbcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec2afe0e80 0x55ec2afeb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:11.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.725+0000 7f988ca00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afdb910 0x55ec2afeadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:11.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.725+0000 7f988ca00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afdb910 0x55ec2afeadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:41370/0 (socket says 172.21.15.131:41370) 2024-05-08T22:12:11.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.725+0000 7f988ca00700 1 -- 172.21.15.131:0/1844424472 learned_addr learned my addr 172.21.15.131:0/1844424472 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:11.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.725+0000 7f98a2fbcb80 1 --2- 172.21.15.131:0/1844424472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ec2afe17a0 0x55ec2afeb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:11.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.725+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/1844424472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2b04b880 con 0x55ec2afe17a0 2024-05-08T22:12:11.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.725+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/1844424472 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2b04b9c0 con 0x55ec2afdb910 2024-05-08T22:12:11.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.725+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/1844424472 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2b04bb00 con 0x55ec2afe0e80 2024-05-08T22:12:11.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.725+0000 7f9887e00700 1 --2- 172.21.15.131:0/1844424472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec2afe0e80 0x55ec2afeb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:11.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.726+0000 7f988d400700 1 --2- 172.21.15.131:0/1844424472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ec2afe17a0 0x55ec2afeb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:11.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.726+0000 7f9887e00700 1 -- 172.21.15.131:0/1844424472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afdb910 msgr2=0x55ec2afeadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:11.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.726+0000 7f9887e00700 1 --2- 172.21.15.131:0/1844424472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afdb910 0x55ec2afeadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.726+0000 7f9887e00700 1 -- 172.21.15.131:0/1844424472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ec2afe17a0 msgr2=0x55ec2afeb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:11.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.726+0000 7f9887e00700 1 --2- 172.21.15.131:0/1844424472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ec2afe17a0 0x55ec2afeb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.726+0000 7f9887e00700 1 -- 172.21.15.131:0/1844424472 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98881167a0 con 0x55ec2afe0e80 2024-05-08T22:12:11.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.726+0000 7f98a2fbcb80 10 client.0 _ll_register_callbacks cb 0x55ec2aed0bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:12:11.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.726+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/1844424472 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9874012c10 con 0x55ec2afe0e80 2024-05-08T22:12:11.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.726+0000 7f988d400700 1 --2- 172.21.15.131:0/1844424472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ec2afe17a0 0x55ec2afeb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:12:11.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.726+0000 7f988ca00700 1 --2- 172.21.15.131:0/1844424472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afdb910 0x55ec2afeadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:12:11.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.727+0000 7f9887e00700 1 --2- 172.21.15.131:0/1844424472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec2afe0e80 0x55ec2afeb300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9874013040 tx=0x7f9874012ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34dc2e71f36a7c79 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:11.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.727+0000 7f9872000700 10 client.26254 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:12:11.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.733+0000 7f9872000700 1 -- 172.21.15.131:0/1844424472 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987402b070 con 0x55ec2afe0e80 2024-05-08T22:12:11.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.733+0000 7f9872000700 1 -- 172.21.15.131:0/1844424472 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f9874023040 con 0x55ec2afe0e80 2024-05-08T22:12:11.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.733+0000 7f9872000700 1 -- 172.21.15.131:0/1844424472 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9874026df0 con 0x55ec2afe0e80 2024-05-08T22:12:11.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.738+0000 7f98a2fbcb80 10 client.26254 fetch_fsmap learned FSMap version 26 2024-05-08T22:12:11.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.738+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/1844424472 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=26}) v3 -- 0x55ec2aff1640 con 0x55ec2afe0e80 2024-05-08T22:12:11.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.740+0000 7f9872000700 1 -- 172.21.15.131:0/1844424472 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f9874005a20 con 0x55ec2afe0e80 2024-05-08T22:12:11.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.741+0000 7f98a2fbcb80 10 client.26254 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:12:11.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.741+0000 7f98a2fbcb80 10 client.26254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:12:11.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.741+0000 7f98a2fbcb80 20 client.26254 populate_metadata read hostname 'smithi131' 2024-05-08T22:12:11.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.741+0000 7f98a2fbcb80 1 --2- 172.21.15.131:0/1844424472 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ec2b04d030 0x55ec2b04f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:11.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.741+0000 7f98a2fbcb80 4 client.26254 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T22:12:11.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.741+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/1844424472 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ec2af23f40 con 0x55ec2b04d030 2024-05-08T22:12:11.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.742+0000 7f988ca00700 1 --2- 172.21.15.131:0/1844424472 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ec2b04d030 0x55ec2b04f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:11.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.743+0000 7f988ca00700 1 --2- 172.21.15.131:0/1844424472 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ec2b04d030 0x55ec2b04f450 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cdfa0bc07e672ad server_cookie=33fd63f1b167032 in_seq=0 out_seq=0 2024-05-08T22:12:11.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.743+0000 7f9872000700 10 client.26254 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:12:11.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.844+0000 7f9872000700 1 -- 172.21.15.131:0/1844424472 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ec2af23f40 con 0x55ec2b04d030 2024-05-08T22:12:11.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.844+0000 7f9872000700 10 client.26254 handle_command_reply: tid=0 2024-05-08T22:12:11.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.849+0000 7f98a2fbcb80 1 client.26254 shutdown 2024-05-08T22:12:11.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.851+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/1844424472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec2afe0e80 msgr2=0x55ec2afeb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:11.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.851+0000 7f98a2fbcb80 1 --2- 172.21.15.131:0/1844424472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec2afe0e80 0x55ec2afeb300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9874013040 tx=0x7f9874012ea0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.851+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/1844424472 shutdown_connections 2024-05-08T22:12:11.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.851+0000 7f98a2fbcb80 1 --2- 172.21.15.131:0/1844424472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ec2afe17a0 0x55ec2afeb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.851+0000 7f98a2fbcb80 1 --2- 172.21.15.131:0/1844424472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec2afdb910 0x55ec2afeadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.851+0000 7f98a2fbcb80 1 --2- 172.21.15.131:0/1844424472 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ec2b04d030 0x55ec2b04f450 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.852+0000 7f98a2fbcb80 1 --2- 172.21.15.131:0/1844424472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec2afe0e80 0x55ec2afeb300 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.852+0000 7f9872000700 0 client.26254 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:12:11.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.852+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/1844424472 >> 172.21.15.131:0/1844424472 conn(0x55ec2af39290 msgr2=0x55ec2afe6410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:11.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.853+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/1844424472 shutdown_connections 2024-05-08T22:12:11.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.853+0000 7f98a2fbcb80 1 -- 172.21.15.131:0/1844424472 wait complete. 2024-05-08T22:12:11.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.854+0000 7f98a2fbcb80 20 client.26254 trim_cache size 0 max 16384 2024-05-08T22:12:11.856 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:12:11.856 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-08T22:12:11.856 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "ca36159f-7052-4d2f-921f-ee97cd0c21fb", 2024-05-08T22:12:11.856 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-08T22:12:11.856 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:12:11.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.857+0000 7f988f400700 1 -- 172.21.15.131:0/3018782188 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f986c0a1420 msgr2=0x7f986c0a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:11.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.857+0000 7f988f400700 1 --2- 172.21.15.131:0/3018782188 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f986c0a1420 0x7f986c0a38e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9880009fd0 tx=0x7f988000b040 comp rx=0 tx=0).stop 2024-05-08T22:12:11.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.858+0000 7f988f400700 1 -- 172.21.15.131:0/3018782188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f988811a780 msgr2=0x7f9888111b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:11.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.859+0000 7f988f400700 1 --2- 172.21.15.131:0/3018782188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f988811a780 0x7f9888111b00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f988806a300 tx=0x7f9878003e00 comp rx=0 tx=0).stop 2024-05-08T22:12:11.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.861+0000 7f988f400700 1 -- 172.21.15.131:0/3018782188 shutdown_connections 2024-05-08T22:12:11.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.861+0000 7f988f400700 1 --2- 172.21.15.131:0/3018782188 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f986c0a1420 0x7f986c0a38e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.861+0000 7f988f400700 1 --2- 172.21.15.131:0/3018782188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f988811a780 0x7f9888111b00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.861+0000 7f988f400700 1 --2- 172.21.15.131:0/3018782188 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f988810f6a0 0x7f9888118a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.861+0000 7f988f400700 1 --2- 172.21.15.131:0/3018782188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f988810fb10 0x7f9888118fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:11.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.861+0000 7f988f400700 1 -- 172.21.15.131:0/3018782188 >> 172.21.15.131:0/3018782188 conn(0x7f988806bff0 msgr2=0x7f988806f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:11.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.864+0000 7f988f400700 1 -- 172.21.15.131:0/3018782188 shutdown_connections 2024-05-08T22:12:11.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:11.864+0000 7f988f400700 1 -- 172.21.15.131:0/3018782188 wait complete. 2024-05-08T22:12:11.991 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:12:12.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.944+0000 7f8d96800700 1 -- 172.21.15.131:0/3896534921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d90103d20 msgr2=0x7f8d90107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:12.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.944+0000 7f8d96800700 1 --2- 172.21.15.131:0/3896534921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d90103d20 0x7f8d90107d60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c009990 tx=0x7f8d7c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:12:12.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.945+0000 7f8d96800700 1 -- 172.21.15.131:0/3896534921 shutdown_connections 2024-05-08T22:12:12.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.945+0000 7f8d96800700 1 --2- 172.21.15.131:0/3896534921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d90108490 0x7f8d90110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:12.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.945+0000 7f8d96800700 1 --2- 172.21.15.131:0/3896534921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d90103370 0x7f8d90103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:12.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.945+0000 7f8d96800700 1 --2- 172.21.15.131:0/3896534921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d90103d20 0x7f8d90107d60 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:12.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.945+0000 7f8d96800700 1 -- 172.21.15.131:0/3896534921 >> 172.21.15.131:0/3896534921 conn(0x7f8d900febb0 msgr2=0x7f8d90100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:12.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.945+0000 7f8d96800700 1 -- 172.21.15.131:0/3896534921 shutdown_connections 2024-05-08T22:12:12.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.947+0000 7f8d96800700 1 -- 172.21.15.131:0/3896534921 wait complete. 2024-05-08T22:12:12.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.949+0000 7f8d96800700 1 Processor -- start 2024-05-08T22:12:12.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.949+0000 7f8d96800700 1 -- start start 2024-05-08T22:12:12.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.950+0000 7f8d96800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d90103370 0x7f8d9010d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:12.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.950+0000 7f8d96800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d90108490 0x7f8d9010d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:12.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.951+0000 7f8d8f400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d90103370 0x7f8d9010d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:12.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.951+0000 7f8d8f400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d90103370 0x7f8d9010d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:41378/0 (socket says 172.21.15.131:41378) 2024-05-08T22:12:12.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.952+0000 7f8d8ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d90108490 0x7f8d9010d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:12.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.952+0000 7f8d8ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d90108490 0x7f8d9010d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43356/0 (socket says 172.21.15.131:43356) 2024-05-08T22:12:12.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.952+0000 7f8d8ea00700 1 -- 172.21.15.131:0/2490194941 learned_addr learned my addr 172.21.15.131:0/2490194941 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:12.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.952+0000 7f8d96800700 1 --2- 172.21.15.131:0/2490194941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d9010dba0 0x7f8d9019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:12.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.952+0000 7f8d96800700 1 -- 172.21.15.131:0/2490194941 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d90110300 con 0x7f8d9010dba0 2024-05-08T22:12:12.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.952+0000 7f8d96800700 1 -- 172.21.15.131:0/2490194941 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9019fd10 con 0x7f8d90103370 2024-05-08T22:12:12.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.952+0000 7f8d96800700 1 -- 172.21.15.131:0/2490194941 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9019fed0 con 0x7f8d90108490 2024-05-08T22:12:12.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.952+0000 7f8d8fe00700 1 --2- 172.21.15.131:0/2490194941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d9010dba0 0x7f8d9019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:12.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.953+0000 7f8d8ea00700 1 -- 172.21.15.131:0/2490194941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d90103370 msgr2=0x7f8d9010d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:12.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.953+0000 7f8d8ea00700 1 --2- 172.21.15.131:0/2490194941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d90103370 0x7f8d9010d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:12.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.953+0000 7f8d8ea00700 1 -- 172.21.15.131:0/2490194941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d9010dba0 msgr2=0x7f8d9019f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:12.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.953+0000 7f8d8ea00700 1 --2- 172.21.15.131:0/2490194941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d9010dba0 0x7f8d9019f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:12.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.953+0000 7f8d8ea00700 1 -- 172.21.15.131:0/2490194941 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7c0095f0 con 0x7f8d90108490 2024-05-08T22:12:12.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.953+0000 7f8d8fe00700 1 --2- 172.21.15.131:0/2490194941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d9010dba0 0x7f8d9019f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:12:12.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.955+0000 7f8d8ea00700 1 --2- 172.21.15.131:0/2490194941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d90108490 0x7f8d9010d660 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0098e0 tx=0x7f8d7c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:12.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.956+0000 7f8d87e00700 1 -- 172.21.15.131:0/2490194941 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c01d070 con 0x7f8d90108490 2024-05-08T22:12:12.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.956+0000 7f8d87e00700 1 -- 172.21.15.131:0/2490194941 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d7c00fb30 con 0x7f8d90108490 2024-05-08T22:12:12.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.956+0000 7f8d96800700 1 -- 172.21.15.131:0/2490194941 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d901a0150 con 0x7f8d90108490 2024-05-08T22:12:12.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.956+0000 7f8d87e00700 1 -- 172.21.15.131:0/2490194941 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c021a80 con 0x7f8d90108490 2024-05-08T22:12:12.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.956+0000 7f8d96800700 1 -- 172.21.15.131:0/2490194941 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d901a0600 con 0x7f8d90108490 2024-05-08T22:12:12.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.969+0000 7f8d87e00700 1 -- 172.21.15.131:0/2490194941 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f8d7c00fca0 con 0x7f8d90108490 2024-05-08T22:12:12.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.971+0000 7f8d87e00700 1 --2- 172.21.15.131:0/2490194941 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8d740a1460 0x7f8d740a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:12.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.971+0000 7f8d87e00700 1 -- 172.21.15.131:0/2490194941 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f8d7c0cdc00 con 0x7f8d90108490 2024-05-08T22:12:12.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.971+0000 7f8d8f400700 1 --2- 172.21.15.131:0/2490194941 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8d740a1460 0x7f8d740a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:12.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.979+0000 7f8d8f400700 1 --2- 172.21.15.131:0/2490194941 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8d740a1460 0x7f8d740a3920 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8d88009cf0 tx=0x7f8d88009040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:12.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.990+0000 7f8daa269b80 1 Processor -- start 2024-05-08T22:12:12.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.991+0000 7f8daa269b80 1 -- start start 2024-05-08T22:12:12.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.993+0000 7f8daa269b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df91853e00 0x55df918541e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:12.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.993+0000 7f8daa269b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df91854720 0x55df9184e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:12.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.993+0000 7f8daa269b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df9184e890 0x55df9184ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:12.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.993+0000 7f8d8fe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df9184e890 0x55df9184ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:12.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.994+0000 7f8d8fe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df9184e890 0x55df9184ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:41392/0 (socket says 172.21.15.131:41392) 2024-05-08T22:12:12.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.994+0000 7f8d8fe00700 1 -- 172.21.15.131:0/3051228619 learned_addr learned my addr 172.21.15.131:0/3051228619 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:12.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.995+0000 7f8daa269b80 1 -- 172.21.15.131:0/3051228619 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df917a5150 con 0x55df91854720 2024-05-08T22:12:12.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.995+0000 7f8daa269b80 1 -- 172.21.15.131:0/3051228619 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df917a2b10 con 0x55df9184e890 2024-05-08T22:12:12.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.995+0000 7f8d8f400700 1 --2- 172.21.15.131:0/3051228619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df91853e00 0x55df918541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:12.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.995+0000 7f8daa269b80 1 -- 172.21.15.131:0/3051228619 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9179e400 con 0x55df91853e00 2024-05-08T22:12:12.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.995+0000 7f8d8ea00700 1 --2- 172.21.15.131:0/3051228619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df91854720 0x55df9184e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:13.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.997+0000 7f8d8f400700 1 -- 172.21.15.131:0/3051228619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df9184e890 msgr2=0x55df9184ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:13.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.997+0000 7f8d8f400700 1 --2- 172.21.15.131:0/3051228619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df9184e890 0x55df9184ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.997+0000 7f8d8f400700 1 -- 172.21.15.131:0/3051228619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df91854720 msgr2=0x55df9184e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:13.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.997+0000 7f8d8f400700 1 --2- 172.21.15.131:0/3051228619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df91854720 0x55df9184e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.997+0000 7f8d8f400700 1 -- 172.21.15.131:0/3051228619 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df9161f3b0 con 0x55df91853e00 2024-05-08T22:12:13.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.998+0000 7f8d8fe00700 1 --2- 172.21.15.131:0/3051228619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df9184e890 0x55df9184ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:12:13.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.998+0000 7f8d8ea00700 1 --2- 172.21.15.131:0/3051228619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df91854720 0x55df9184e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:12:13.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:12.998+0000 7f8d8f400700 1 --2- 172.21.15.131:0/3051228619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df91853e00 0x55df918541e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8d88009f40 tx=0x7f8d88012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2bd2cb2189159b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:13.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.000+0000 7f8d84c00700 1 -- 172.21.15.131:0/3051228619 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88021400 con 0x55df91853e00 2024-05-08T22:12:13.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.000+0000 7f8d84c00700 1 -- 172.21.15.131:0/3051228619 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d88021a20 con 0x55df91853e00 2024-05-08T22:12:13.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.000+0000 7f8d84c00700 1 -- 172.21.15.131:0/3051228619 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88014a40 con 0x55df91853e00 2024-05-08T22:12:13.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.003+0000 7f8daa269b80 1 -- 172.21.15.131:0/3051228619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df91853e00 msgr2=0x55df918541e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:13.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.003+0000 7f8daa269b80 1 --2- 172.21.15.131:0/3051228619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df91853e00 0x55df918541e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8d88009f40 tx=0x7f8d88012000 comp rx=0 tx=0).stop 2024-05-08T22:12:13.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.004+0000 7f8daa269b80 1 -- 172.21.15.131:0/3051228619 shutdown_connections 2024-05-08T22:12:13.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.004+0000 7f8daa269b80 1 --2- 172.21.15.131:0/3051228619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df9184e890 0x55df9184ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.004+0000 7f8daa269b80 1 --2- 172.21.15.131:0/3051228619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df91853e00 0x55df918541e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.004+0000 7f8daa269b80 1 --2- 172.21.15.131:0/3051228619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df91854720 0x55df9184e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.004+0000 7f8daa269b80 1 -- 172.21.15.131:0/3051228619 >> 172.21.15.131:0/3051228619 conn(0x55df917ac210 msgr2=0x55df91848df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:13.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.004+0000 7f8daa269b80 1 -- 172.21.15.131:0/3051228619 shutdown_connections 2024-05-08T22:12:13.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.005+0000 7f8daa269b80 1 -- 172.21.15.131:0/3051228619 wait complete. 2024-05-08T22:12:13.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.005+0000 7f8daa269b80 1 -- start start 2024-05-08T22:12:13.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.006+0000 7f8daa269b80 1 Processor -- start 2024-05-08T22:12:13.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.008+0000 7f8daa269b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df9184e890 0x55df91857aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:13.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.008+0000 7f8daa269b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df91853e00 0x55df91857fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:13.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.008+0000 7f8daa269b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df91854720 0x55df91858520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:13.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.009+0000 7f8d8fe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df91854720 0x55df91858520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:13.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.009+0000 7f8d8fe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df91854720 0x55df91858520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:41402/0 (socket says 172.21.15.131:41402) 2024-05-08T22:12:13.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.009+0000 7f8d8fe00700 1 -- 172.21.15.131:0/465079139 learned_addr learned my addr 172.21.15.131:0/465079139 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:13.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.010+0000 7f8d8ea00700 1 --2- 172.21.15.131:0/465079139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df91853e00 0x55df91857fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:13.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.011+0000 7f8daa269b80 1 -- 172.21.15.131:0/465079139 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9173a560 con 0x55df91853e00 2024-05-08T22:12:13.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.011+0000 7f8daa269b80 1 -- 172.21.15.131:0/465079139 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9173a200 con 0x55df91854720 2024-05-08T22:12:13.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.011+0000 7f8daa269b80 1 -- 172.21.15.131:0/465079139 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df916216a0 con 0x55df9184e890 2024-05-08T22:12:13.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.011+0000 7f8d8f400700 1 --2- 172.21.15.131:0/465079139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df9184e890 0x55df91857aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:13.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.011+0000 7f8daa269b80 10 client.0 _ll_register_callbacks cb 0x55df91743b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:12:13.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.012+0000 7f8d8f400700 1 -- 172.21.15.131:0/465079139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df91854720 msgr2=0x55df91858520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:13.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.012+0000 7f8d8f400700 1 --2- 172.21.15.131:0/465079139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df91854720 0x55df91858520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.012+0000 7f8d8f400700 1 -- 172.21.15.131:0/465079139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df91853e00 msgr2=0x55df91857fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:13.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.012+0000 7f8d8f400700 1 --2- 172.21.15.131:0/465079139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df91853e00 0x55df91857fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.012+0000 7f8d8f400700 1 -- 172.21.15.131:0/465079139 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d80009aa0 con 0x55df9184e890 2024-05-08T22:12:13.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.013+0000 7f8d8fe00700 1 --2- 172.21.15.131:0/465079139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df91854720 0x55df91858520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:12:13.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.015+0000 7f8d8f400700 1 --2- 172.21.15.131:0/465079139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df9184e890 0x55df91857aa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8d88007b70 tx=0x7f8d88021b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6cc9471871ccc99 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:13.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.015+0000 7f8d79600700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:12:13.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.017+0000 7f8d79600700 1 -- 172.21.15.131:0/465079139 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88004c90 con 0x55df9184e890 2024-05-08T22:12:13.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.017+0000 7f8d79600700 1 -- 172.21.15.131:0/465079139 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f8d88004df0 con 0x55df9184e890 2024-05-08T22:12:13.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.017+0000 7f8daa269b80 1 -- 172.21.15.131:0/465079139 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df916f7270 con 0x55df9184e890 2024-05-08T22:12:13.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.017+0000 7f8d79600700 1 -- 172.21.15.131:0/465079139 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d88021400 con 0x55df9184e890 2024-05-08T22:12:13.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.017+0000 7f8daa269b80 10 client.26266 fetch_fsmap learned FSMap version 26 2024-05-08T22:12:13.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.017+0000 7f8daa269b80 1 -- 172.21.15.131:0/465079139 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f8d88007d80 con 0x55df9184e890 2024-05-08T22:12:13.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.017+0000 7f8d79600700 1 -- 172.21.15.131:0/465079139 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55df916f7270 con 0x55df9184e890 2024-05-08T22:12:13.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.018+0000 7f8d79600700 1 -- 172.21.15.131:0/465079139 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8d88005040 con 0x55df9184e890 2024-05-08T22:12:13.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.018+0000 7f8daa269b80 10 client.26266 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:12:13.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.018+0000 7f8daa269b80 10 client.26266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:12:13.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.018+0000 7f8daa269b80 20 client.26266 populate_metadata read hostname 'smithi131' 2024-05-08T22:12:13.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.018+0000 7f8daa269b80 1 --2- 172.21.15.131:0/465079139 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55df918680e0 0x55df9186a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:13.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.018+0000 7f8daa269b80 4 client.26266 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:12:13.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.018+0000 7f8daa269b80 1 -- 172.21.15.131:0/465079139 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df9181d810 con 0x55df918680e0 2024-05-08T22:12:13.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.018+0000 7f8d8ea00700 1 --2- 172.21.15.131:0/465079139 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55df918680e0 0x55df9186a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:13.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.021+0000 7f8d79600700 10 client.26266 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:12:13.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.022+0000 7f8d8ea00700 1 --2- 172.21.15.131:0/465079139 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55df918680e0 0x55df9186a500 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19b0ead8f55149bb server_cookie=5ee85027d02d0b83 in_seq=0 out_seq=0 2024-05-08T22:12:13.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.023+0000 7f8d79600700 1 -- 172.21.15.131:0/465079139 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55df9181d810 con 0x55df918680e0 2024-05-08T22:12:13.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.023+0000 7f8d79600700 10 client.26266 handle_command_reply: tid=0 2024-05-08T22:12:13.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.024+0000 7f8daa269b80 1 client.26266 shutdown 2024-05-08T22:12:13.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.024+0000 7f8daa269b80 1 -- 172.21.15.131:0/465079139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df9184e890 msgr2=0x55df91857aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:13.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.024+0000 7f8daa269b80 1 --2- 172.21.15.131:0/465079139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df9184e890 0x55df91857aa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8d88007b70 tx=0x7f8d88021b90 comp rx=0 tx=0).stop 2024-05-08T22:12:13.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.024+0000 7f8daa269b80 1 -- 172.21.15.131:0/465079139 shutdown_connections 2024-05-08T22:12:13.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.024+0000 7f8daa269b80 1 --2- 172.21.15.131:0/465079139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df91854720 0x55df91858520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.024+0000 7f8daa269b80 1 --2- 172.21.15.131:0/465079139 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55df918680e0 0x55df9186a500 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.024+0000 7f8daa269b80 1 --2- 172.21.15.131:0/465079139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df9184e890 0x55df91857aa0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.024+0000 7f8d79600700 0 client.26266 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:12:13.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.024+0000 7f8daa269b80 1 --2- 172.21.15.131:0/465079139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df91853e00 0x55df91857fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.024+0000 7f8daa269b80 1 -- 172.21.15.131:0/465079139 >> 172.21.15.131:0/465079139 conn(0x55df917ac210 msgr2=0x55df918485a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:13.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.026+0000 7f8daa269b80 1 -- 172.21.15.131:0/465079139 shutdown_connections 2024-05-08T22:12:13.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.026+0000 7f8daa269b80 1 -- 172.21.15.131:0/465079139 wait complete. 2024-05-08T22:12:13.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.028+0000 7f8daa269b80 20 client.26266 trim_cache size 0 max 16384 2024-05-08T22:12:13.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.079+0000 7f8daa269b80 1 Processor -- start 2024-05-08T22:12:13.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.079+0000 7f8daa269b80 1 -- start start 2024-05-08T22:12:13.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.080+0000 7f8daa269b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df9184e890 0x55df91848d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:13.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.080+0000 7f8daa269b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df91853e00 0x55df91858340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:13.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.080+0000 7f8daa269b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df91854720 0x55df91852b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:13.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.080+0000 7f8daa269b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9186f6b0 con 0x55df91853e00 2024-05-08T22:12:13.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.080+0000 7f8daa269b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df91849260 con 0x55df91854720 2024-05-08T22:12:13.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.080+0000 7f8daa269b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df91858f30 con 0x55df9184e890 2024-05-08T22:12:13.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.081+0000 7f8d8f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df9184e890 0x55df91848d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:13.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.081+0000 7f8d8f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df9184e890 0x55df91848d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43386/0 (socket says 172.21.15.131:43386) 2024-05-08T22:12:13.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.081+0000 7f8d8f400700 1 -- 172.21.15.131:0/998061372 learned_addr learned my addr 172.21.15.131:0/998061372 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:13.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.081+0000 7f8d8fe00700 1 --2- 172.21.15.131:0/998061372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df91854720 0x55df91852b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.081+0000 7f8d8fe00700 1 -- 172.21.15.131:0/998061372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df9184e890 msgr2=0x55df91848d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.081+0000 7f8d8fe00700 1 --2- 172.21.15.131:0/998061372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df9184e890 0x55df91848d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.081+0000 7f8d8fe00700 1 -- 172.21.15.131:0/998061372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df91853e00 msgr2=0x55df91858340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.081+0000 7f8d8fe00700 1 --2- 172.21.15.131:0/998061372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df91853e00 0x55df91858340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.081+0000 7f8d8fe00700 1 -- 172.21.15.131:0/998061372 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df915cbb60 con 0x55df91854720 2024-05-08T22:12:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.083+0000 7f8d8fe00700 1 --2- 172.21.15.131:0/998061372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df91854720 0x55df91852b90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8d80014430 tx=0x7f8d80009110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfa1db5d780db768 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.083+0000 7f8d85600700 1 -- 172.21.15.131:0/998061372 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8001f480 con 0x55df91854720 2024-05-08T22:12:13.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.083+0000 7f8d85600700 1 -- 172.21.15.131:0/998061372 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d8001faa0 con 0x55df91854720 2024-05-08T22:12:13.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.083+0000 7f8d85600700 1 -- 172.21.15.131:0/998061372 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d80016b10 con 0x55df91854720 2024-05-08T22:12:13.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.087+0000 7f8daa269b80 1 -- 172.21.15.131:0/998061372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df91854720 msgr2=0x55df91852b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:13.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.087+0000 7f8daa269b80 1 --2- 172.21.15.131:0/998061372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df91854720 0x55df91852b90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8d80014430 tx=0x7f8d80009110 comp rx=0 tx=0).stop 2024-05-08T22:12:13.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.088+0000 7f8daa269b80 1 -- 172.21.15.131:0/998061372 shutdown_connections 2024-05-08T22:12:13.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.088+0000 7f8daa269b80 1 --2- 172.21.15.131:0/998061372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df91854720 0x55df91852b90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.088+0000 7f8daa269b80 1 --2- 172.21.15.131:0/998061372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df9184e890 0x55df91848d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.088+0000 7f8daa269b80 1 --2- 172.21.15.131:0/998061372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df91853e00 0x55df91858340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.088+0000 7f8daa269b80 1 -- 172.21.15.131:0/998061372 >> 172.21.15.131:0/998061372 conn(0x55df917ac210 msgr2=0x55df91851ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:13.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.088+0000 7f8daa269b80 1 -- 172.21.15.131:0/998061372 shutdown_connections 2024-05-08T22:12:13.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.088+0000 7f8daa269b80 1 -- 172.21.15.131:0/998061372 wait complete. 2024-05-08T22:12:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.089+0000 7f8daa269b80 1 -- start start 2024-05-08T22:12:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.090+0000 7f8daa269b80 1 Processor -- start 2024-05-08T22:12:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.091+0000 7f8daa269b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df9184e890 0x55df9185d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.091+0000 7f8daa269b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df91853e00 0x55df9185d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.091+0000 7f8daa269b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df91854720 0x55df918bd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.092+0000 7f8d8fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df91854720 0x55df918bd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.092+0000 7f8d8fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df91854720 0x55df918bd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59440/0 (socket says 172.21.15.131:59440) 2024-05-08T22:12:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.092+0000 7f8d8fe00700 1 -- 172.21.15.131:0/1008239326 learned_addr learned my addr 172.21.15.131:0/1008239326 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.092+0000 7f8daa269b80 1 -- 172.21.15.131:0/1008239326 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9185df30 con 0x55df91854720 2024-05-08T22:12:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.092+0000 7f8d8f400700 1 --2- 172.21.15.131:0/1008239326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df9184e890 0x55df9185d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.093+0000 7f8d8ea00700 1 --2- 172.21.15.131:0/1008239326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df91853e00 0x55df9185d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.095+0000 7f8daa269b80 1 -- 172.21.15.131:0/1008239326 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df918bdae0 con 0x55df91853e00 2024-05-08T22:12:13.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.095+0000 7f8daa269b80 1 -- 172.21.15.131:0/1008239326 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df918bdc20 con 0x55df9184e890 2024-05-08T22:12:13.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.095+0000 7f8daa269b80 10 client.0 _ll_register_callbacks cb 0x55df91743b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:12:13.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.095+0000 7f8d8fe00700 1 -- 172.21.15.131:0/1008239326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df9184e890 msgr2=0x55df9185d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:13.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.095+0000 7f8d8fe00700 1 --2- 172.21.15.131:0/1008239326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df9184e890 0x55df9185d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.095+0000 7f8d8fe00700 1 -- 172.21.15.131:0/1008239326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df91853e00 msgr2=0x55df9185d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:13.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.095+0000 7f8d8fe00700 1 --2- 172.21.15.131:0/1008239326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df91853e00 0x55df9185d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.096+0000 7f8d8fe00700 1 -- 172.21.15.131:0/1008239326 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d80009d90 con 0x55df91854720 2024-05-08T22:12:13.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.096+0000 7f8d8f400700 1 --2- 172.21.15.131:0/1008239326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df9184e890 0x55df9185d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:12:13.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.096+0000 7f8d8ea00700 1 --2- 172.21.15.131:0/1008239326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df91853e00 0x55df9185d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:12:13.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.096+0000 7f8d8fe00700 1 --2- 172.21.15.131:0/1008239326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df91854720 0x55df918bd5a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8d80004790 tx=0x7f8d8001f760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a68ba7c01b387586 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:13.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.096+0000 7f8d79600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:12:13.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.098+0000 7f8d79600700 1 -- 172.21.15.131:0/1008239326 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d80023070 con 0x55df91854720 2024-05-08T22:12:13.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.098+0000 7f8d79600700 1 -- 172.21.15.131:0/1008239326 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f8d8001a070 con 0x55df91854720 2024-05-08T22:12:13.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.098+0000 7f8daa269b80 1 -- 172.21.15.131:0/1008239326 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d88021a20 con 0x55df91854720 2024-05-08T22:12:13.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.100+0000 7f8d79600700 1 -- 172.21.15.131:0/1008239326 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d80016430 con 0x55df91854720 2024-05-08T22:12:13.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.100+0000 7f8daa269b80 10 client.16182 fetch_fsmap learned FSMap version 26 2024-05-08T22:12:13.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.100+0000 7f8daa269b80 1 -- 172.21.15.131:0/1008239326 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55df91764560 con 0x55df91854720 2024-05-08T22:12:13.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.101+0000 7f8d79600700 1 -- 172.21.15.131:0/1008239326 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d88021a20 con 0x55df91854720 2024-05-08T22:12:13.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.103+0000 7f8d79600700 1 -- 172.21.15.131:0/1008239326 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8d80011030 con 0x55df91854720 2024-05-08T22:12:13.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.103+0000 7f8daa269b80 10 client.16182 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:12:13.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.103+0000 7f8daa269b80 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:12:13.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.103+0000 7f8daa269b80 20 client.16182 populate_metadata read hostname 'smithi131' 2024-05-08T22:12:13.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.103+0000 7f8daa269b80 1 --2- 172.21.15.131:0/1008239326 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55df918680e0 0x55df918c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:13.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.104+0000 7f8daa269b80 4 client.16182 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:12:13.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.104+0000 7f8daa269b80 1 -- 172.21.15.131:0/1008239326 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55df91796ec0 con 0x55df918680e0 2024-05-08T22:12:13.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.104+0000 7f8d8f400700 1 --2- 172.21.15.131:0/1008239326 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55df918680e0 0x55df918c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:13.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.105+0000 7f8d8f400700 1 --2- 172.21.15.131:0/1008239326 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55df918680e0 0x55df918c12b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb154ad9f977fe1b server_cookie=de8497feac1ee173 in_seq=0 out_seq=0 2024-05-08T22:12:13.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.107+0000 7f8d79600700 10 client.16182 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:12:13.112 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:12:13.112 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "scrub active (20221 inodes in the stack)", 2024-05-08T22:12:13.112 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-08T22:12:13.113 INFO:teuthology.orchestra.run.smithi131.stdout: "ca36159f-7052-4d2f-921f-ee97cd0c21fb": { 2024-05-08T22:12:13.113 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-08T22:12:13.113 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "ca36159f-7052-4d2f-921f-ee97cd0c21fb", 2024-05-08T22:12:13.113 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-08T22:12:13.113 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:12:13.113 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:12:13.113 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:12:13.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.109+0000 7f8d79600700 1 -- 172.21.15.131:0/1008239326 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+269 (crc 0 0 0) 0x55df91796ec0 con 0x55df918680e0 2024-05-08T22:12:13.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.109+0000 7f8d79600700 10 client.16182 handle_command_reply: tid=0 2024-05-08T22:12:13.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.109+0000 7f8daa269b80 1 client.16182 shutdown 2024-05-08T22:12:13.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.110+0000 7f8daa269b80 1 -- 172.21.15.131:0/1008239326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df91854720 msgr2=0x55df918bd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:13.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.110+0000 7f8daa269b80 1 --2- 172.21.15.131:0/1008239326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df91854720 0x55df918bd5a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8d80004790 tx=0x7f8d8001f760 comp rx=0 tx=0).stop 2024-05-08T22:12:13.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.110+0000 7f8daa269b80 1 -- 172.21.15.131:0/1008239326 shutdown_connections 2024-05-08T22:12:13.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.110+0000 7f8daa269b80 1 --2- 172.21.15.131:0/1008239326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df91854720 0x55df918bd5a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.110+0000 7f8daa269b80 1 --2- 172.21.15.131:0/1008239326 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55df918680e0 0x55df918c12b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.110+0000 7f8daa269b80 1 --2- 172.21.15.131:0/1008239326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df9184e890 0x55df9185d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.110+0000 7f8daa269b80 1 --2- 172.21.15.131:0/1008239326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df91853e00 0x55df9185d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.110+0000 7f8daa269b80 1 -- 172.21.15.131:0/1008239326 >> 172.21.15.131:0/1008239326 conn(0x55df917ac210 msgr2=0x55df91858380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:13.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.110+0000 7f8d79600700 0 client.16182 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:12:13.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.111+0000 7f8daa269b80 1 -- 172.21.15.131:0/1008239326 shutdown_connections 2024-05-08T22:12:13.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.111+0000 7f8daa269b80 1 -- 172.21.15.131:0/1008239326 wait complete. 2024-05-08T22:12:13.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.111+0000 7f8daa269b80 20 client.16182 trim_cache size 0 max 16384 2024-05-08T22:12:13.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.112+0000 7f8d96800700 1 -- 172.21.15.131:0/2490194941 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8d740a1460 msgr2=0x7f8d740a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:13.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.112+0000 7f8d96800700 1 --2- 172.21.15.131:0/2490194941 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8d740a1460 0x7f8d740a3920 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8d88009cf0 tx=0x7f8d88009040 comp rx=0 tx=0).stop 2024-05-08T22:12:13.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.112+0000 7f8d96800700 1 -- 172.21.15.131:0/2490194941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d90108490 msgr2=0x7f8d9010d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:13.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.112+0000 7f8d96800700 1 --2- 172.21.15.131:0/2490194941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d90108490 0x7f8d9010d660 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0098e0 tx=0x7f8d7c005740 comp rx=0 tx=0).stop 2024-05-08T22:12:13.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.113+0000 7f8d96800700 1 -- 172.21.15.131:0/2490194941 shutdown_connections 2024-05-08T22:12:13.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.113+0000 7f8d96800700 1 --2- 172.21.15.131:0/2490194941 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8d740a1460 0x7f8d740a3920 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.113+0000 7f8d96800700 1 --2- 172.21.15.131:0/2490194941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d9010dba0 0x7f8d9019f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.113+0000 7f8d96800700 1 --2- 172.21.15.131:0/2490194941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d90103370 0x7f8d9010d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.113+0000 7f8d96800700 1 --2- 172.21.15.131:0/2490194941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d90108490 0x7f8d9010d660 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:13.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.113+0000 7f8d96800700 1 -- 172.21.15.131:0/2490194941 >> 172.21.15.131:0/2490194941 conn(0x7f8d900febb0 msgr2=0x7f8d900ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:13.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.115+0000 7f8d96800700 1 -- 172.21.15.131:0/2490194941 shutdown_connections 2024-05-08T22:12:13.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:13.115+0000 7f8d96800700 1 -- 172.21.15.131:0/2490194941 wait complete. 2024-05-08T22:12:13.167 INFO:tasks.cephfs.filesystem:scrub status for tag:ca36159f-7052-4d2f-921f-ee97cd0c21fb - {'path': '/', 'tag': 'ca36159f-7052-4d2f-921f-ee97cd0c21fb', 'options': 'recursive,force'} 2024-05-08T22:12:13.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:12 smithi157 ceph-mon[77746]: pgmap v150: 129 pgs: 129 active+clean; 702 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.3 MiB/s wr, 927 op/s 2024-05-08T22:12:13.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:12 smithi157 ceph-mon[77746]: scrub queued for path: / 2024-05-08T22:12:13.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:12 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:12:13.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:12 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:12:13.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:12 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:12:13.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:12 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:12:13.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:12 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:12:13.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:12 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:12:13.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:12 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:12:13.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:12 smithi132 ceph-mon[78180]: pgmap v150: 129 pgs: 129 active+clean; 702 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.3 MiB/s wr, 927 op/s 2024-05-08T22:12:13.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:12 smithi132 ceph-mon[78180]: scrub queued for path: / 2024-05-08T22:12:13.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:12 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:12:13.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:12 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:12:13.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:12 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:12:13.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:12 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:12:13.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:12 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:12:13.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:12 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:12:13.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:12 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:12:13.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:12 smithi131 ceph-mon[37260]: pgmap v150: 129 pgs: 129 active+clean; 702 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.3 MiB/s wr, 927 op/s 2024-05-08T22:12:13.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:12 smithi131 ceph-mon[37260]: scrub queued for path: / 2024-05-08T22:12:13.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:12 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:12:13.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:12 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:12:13.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:12 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:12:13.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:12 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:12:13.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:12 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:12:13.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:12 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:12:13.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:12 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:12:14.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:13 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:12:14.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:13 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:12:14.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:13 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:12:14.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:13 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:12:14.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:13 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:12:14.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:13 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:12:14.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:13 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:12:14.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:13 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:12:14.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:13 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:12:14.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:13 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:12:14.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:13 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:12:14.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:13 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:12:14.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:13 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:12:14.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:13 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:12:14.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:13 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:12:15.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:14 smithi157 ceph-mon[77746]: pgmap v151: 129 pgs: 129 active+clean; 702 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.0 MiB/s wr, 889 op/s 2024-05-08T22:12:15.317 INFO:tasks.workunit.client.0.smithi131.stdout: 130 367331 255 231521 303 211350 1333 2024-05-08T22:12:15.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:14 smithi132 ceph-mon[78180]: pgmap v151: 129 pgs: 129 active+clean; 702 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.0 MiB/s wr, 889 op/s 2024-05-08T22:12:15.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:14 smithi131 ceph-mon[37260]: pgmap v151: 129 pgs: 129 active+clean; 702 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.0 MiB/s wr, 889 op/s 2024-05-08T22:12:17.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:16 smithi157 ceph-mon[77746]: pgmap v152: 129 pgs: 129 active+clean; 682 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.1 MiB/s wr, 1.17k op/s 2024-05-08T22:12:17.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:16 smithi132 ceph-mon[78180]: pgmap v152: 129 pgs: 129 active+clean; 682 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.1 MiB/s wr, 1.17k op/s 2024-05-08T22:12:17.401 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:16 smithi131 ceph-mon[37260]: pgmap v152: 129 pgs: 129 active+clean; 682 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.1 MiB/s wr, 1.17k op/s 2024-05-08T22:12:19.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:18 smithi157 ceph-mon[77746]: pgmap v153: 129 pgs: 129 active+clean; 682 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 605 op/s 2024-05-08T22:12:19.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:18 smithi132 ceph-mon[78180]: pgmap v153: 129 pgs: 129 active+clean; 682 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 605 op/s 2024-05-08T22:12:19.401 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:18 smithi131 ceph-mon[37260]: pgmap v153: 129 pgs: 129 active+clean; 682 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 605 op/s 2024-05-08T22:12:21.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:20 smithi157 ceph-mon[77746]: pgmap v154: 129 pgs: 129 active+clean; 680 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 716 op/s 2024-05-08T22:12:21.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:20 smithi132 ceph-mon[78180]: pgmap v154: 129 pgs: 129 active+clean; 680 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 716 op/s 2024-05-08T22:12:21.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:20 smithi131 ceph-mon[37260]: pgmap v154: 129 pgs: 129 active+clean; 680 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 716 op/s 2024-05-08T22:12:23.275 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:23 smithi157 ceph-mon[77746]: pgmap v155: 129 pgs: 129 active+clean; 688 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 581 op/s 2024-05-08T22:12:23.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:23 smithi132 ceph-mon[78180]: pgmap v155: 129 pgs: 129 active+clean; 688 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 581 op/s 2024-05-08T22:12:23.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:23 smithi131 ceph-mon[37260]: pgmap v155: 129 pgs: 129 active+clean; 688 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 581 op/s 2024-05-08T22:12:25.310 INFO:tasks.workunit.client.0.smithi131.stdout: 137 601411 316 383120 378 353107 1388 2024-05-08T22:12:25.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:25 smithi132 ceph-mon[78180]: pgmap v156: 129 pgs: 129 active+clean; 688 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 436 op/s 2024-05-08T22:12:25.399 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:25 smithi131 ceph-mon[37260]: pgmap v156: 129 pgs: 129 active+clean; 688 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 436 op/s 2024-05-08T22:12:25.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:25 smithi157 ceph-mon[77746]: pgmap v156: 129 pgs: 129 active+clean; 688 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 436 op/s 2024-05-08T22:12:27.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:27 smithi132 ceph-mon[78180]: pgmap v157: 129 pgs: 129 active+clean; 687 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 544 op/s 2024-05-08T22:12:27.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:27 smithi131 ceph-mon[37260]: pgmap v157: 129 pgs: 129 active+clean; 687 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 544 op/s 2024-05-08T22:12:27.525 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:27 smithi157 ceph-mon[77746]: pgmap v157: 129 pgs: 129 active+clean; 687 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 544 op/s 2024-05-08T22:12:29.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:29 smithi132 ceph-mon[78180]: pgmap v158: 129 pgs: 129 active+clean; 693 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 306 op/s 2024-05-08T22:12:29.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:29 smithi131 ceph-mon[37260]: pgmap v158: 129 pgs: 129 active+clean; 693 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 306 op/s 2024-05-08T22:12:29.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:29 smithi157 ceph-mon[77746]: pgmap v158: 129 pgs: 129 active+clean; 693 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 306 op/s 2024-05-08T22:12:31.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:31 smithi132 ceph-mon[78180]: pgmap v159: 129 pgs: 129 active+clean; 733 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.6 MiB/s wr, 606 op/s 2024-05-08T22:12:31.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:31 smithi131 ceph-mon[37260]: pgmap v159: 129 pgs: 129 active+clean; 733 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.6 MiB/s wr, 606 op/s 2024-05-08T22:12:31.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:31 smithi157 ceph-mon[77746]: pgmap v159: 129 pgs: 129 active+clean; 733 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.6 MiB/s wr, 606 op/s 2024-05-08T22:12:33.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:33 smithi132 ceph-mon[78180]: pgmap v160: 129 pgs: 129 active+clean; 740 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.0 MiB/s wr, 596 op/s 2024-05-08T22:12:33.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:33 smithi131 ceph-mon[37260]: pgmap v160: 129 pgs: 129 active+clean; 740 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.0 MiB/s wr, 596 op/s 2024-05-08T22:12:33.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:33 smithi157 ceph-mon[77746]: pgmap v160: 129 pgs: 129 active+clean; 740 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.0 MiB/s wr, 596 op/s 2024-05-08T22:12:35.311 INFO:tasks.workunit.client.0.smithi131.stdout: 142 445331 328 284280 328 252845 1142 2024-05-08T22:12:35.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:35 smithi132 ceph-mon[78180]: pgmap v161: 129 pgs: 129 active+clean; 745 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.8 MiB/s wr, 603 op/s 2024-05-08T22:12:35.395 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:35 smithi131 ceph-mon[37260]: pgmap v161: 129 pgs: 129 active+clean; 745 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.8 MiB/s wr, 603 op/s 2024-05-08T22:12:35.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:35 smithi157 ceph-mon[77746]: pgmap v161: 129 pgs: 129 active+clean; 745 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.8 MiB/s wr, 603 op/s 2024-05-08T22:12:37.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:37 smithi132 ceph-mon[78180]: pgmap v162: 129 pgs: 129 active+clean; 769 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.4 MiB/s wr, 860 op/s 2024-05-08T22:12:37.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:37 smithi131 ceph-mon[37260]: pgmap v162: 129 pgs: 129 active+clean; 769 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.4 MiB/s wr, 860 op/s 2024-05-08T22:12:37.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:37 smithi157 ceph-mon[77746]: pgmap v162: 129 pgs: 129 active+clean; 769 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.4 MiB/s wr, 860 op/s 2024-05-08T22:12:39.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:39 smithi132 ceph-mon[78180]: pgmap v163: 129 pgs: 129 active+clean; 767 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.6 MiB/s wr, 799 op/s 2024-05-08T22:12:39.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:39 smithi131 ceph-mon[37260]: pgmap v163: 129 pgs: 129 active+clean; 767 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.6 MiB/s wr, 799 op/s 2024-05-08T22:12:39.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:39 smithi157 ceph-mon[77746]: pgmap v163: 129 pgs: 129 active+clean; 767 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.6 MiB/s wr, 799 op/s 2024-05-08T22:12:41.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:41 smithi131 ceph-mon[37260]: pgmap v164: 129 pgs: 129 active+clean; 781 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.3 MiB/s wr, 850 op/s 2024-05-08T22:12:41.525 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:41 smithi157 ceph-mon[77746]: pgmap v164: 129 pgs: 129 active+clean; 781 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.3 MiB/s wr, 850 op/s 2024-05-08T22:12:41.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:41 smithi132 ceph-mon[78180]: pgmap v164: 129 pgs: 129 active+clean; 781 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.3 MiB/s wr, 850 op/s 2024-05-08T22:12:43.168 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:12:43.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:43 smithi131 ceph-mon[37260]: pgmap v165: 129 pgs: 129 active+clean; 780 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.5 MiB/s wr, 608 op/s 2024-05-08T22:12:43.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:43 smithi157 ceph-mon[77746]: pgmap v165: 129 pgs: 129 active+clean; 780 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.5 MiB/s wr, 608 op/s 2024-05-08T22:12:43.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:43 smithi132 ceph-mon[78180]: pgmap v165: 129 pgs: 129 active+clean; 780 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.5 MiB/s wr, 608 op/s 2024-05-08T22:12:44.394 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:44 smithi131 ceph-mon[37260]: pgmap v166: 129 pgs: 129 active+clean; 777 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 536 op/s 2024-05-08T22:12:44.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:44 smithi157 ceph-mon[77746]: pgmap v166: 129 pgs: 129 active+clean; 777 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 536 op/s 2024-05-08T22:12:44.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:44 smithi132 ceph-mon[78180]: pgmap v166: 129 pgs: 129 active+clean; 777 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 536 op/s 2024-05-08T22:12:45.311 INFO:tasks.workunit.client.0.smithi131.stdout: 150 477728 360 310388 363 277460 1476 2024-05-08T22:12:46.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:46 smithi132 ceph-mon[78180]: pgmap v167: 129 pgs: 129 active+clean; 790 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.4 MiB/s wr, 815 op/s 2024-05-08T22:12:46.653 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:46 smithi131 ceph-mon[37260]: pgmap v167: 129 pgs: 129 active+clean; 790 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.4 MiB/s wr, 815 op/s 2024-05-08T22:12:46.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:46 smithi157 ceph-mon[77746]: pgmap v167: 129 pgs: 129 active+clean; 790 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.4 MiB/s wr, 815 op/s 2024-05-08T22:12:47.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.785+0000 7f2519e00700 1 -- 172.21.15.131:0/1048877307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f251410d080 msgr2=0x7f251410d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:47.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.785+0000 7f2519e00700 1 --2- 172.21.15.131:0/1048877307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f251410d080 0x7f251410d500 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2500009990 tx=0x7f2500009ca0 comp rx=0 tx=0).stop 2024-05-08T22:12:47.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.790+0000 7f2519e00700 1 -- 172.21.15.131:0/1048877307 shutdown_connections 2024-05-08T22:12:47.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.790+0000 7f2519e00700 1 --2- 172.21.15.131:0/1048877307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f251410dba0 0x7f251411c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:47.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.790+0000 7f2519e00700 1 --2- 172.21.15.131:0/1048877307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f251410f520 0x7f251410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:47.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.790+0000 7f2519e00700 1 --2- 172.21.15.131:0/1048877307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f251410d080 0x7f251410d500 secure :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2500009990 tx=0x7f2500009ca0 comp rx=0 tx=0).stop 2024-05-08T22:12:47.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.790+0000 7f2519e00700 1 -- 172.21.15.131:0/1048877307 >> 172.21.15.131:0/1048877307 conn(0x7f251406be80 msgr2=0x7f251406c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:47.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.845+0000 7f2519e00700 1 -- 172.21.15.131:0/1048877307 shutdown_connections 2024-05-08T22:12:47.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.853+0000 7f2519e00700 1 -- 172.21.15.131:0/1048877307 wait complete. 2024-05-08T22:12:47.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.876+0000 7f2519e00700 1 Processor -- start 2024-05-08T22:12:47.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.877+0000 7f2519e00700 1 -- start start 2024-05-08T22:12:47.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.897+0000 7f2519e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f251410dba0 0x7f25141a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:47.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.897+0000 7f2519e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f251410f520 0x7f25141a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:47.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.897+0000 7f2519e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25141aac00 0x7f25141ab080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:47.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.897+0000 7f2519e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25141ab680 con 0x7f251410dba0 2024-05-08T22:12:47.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.897+0000 7f2519e00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25141ab7c0 con 0x7f25141aac00 2024-05-08T22:12:47.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.897+0000 7f2519e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25141ab930 con 0x7f251410f520 2024-05-08T22:12:47.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.901+0000 7f2513400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25141aac00 0x7f25141ab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:47.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.904+0000 7f2513400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25141aac00 0x7f25141ab080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:51270/0 (socket says 172.21.15.131:51270) 2024-05-08T22:12:47.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.904+0000 7f2513400700 1 -- 172.21.15.131:0/522261890 learned_addr learned my addr 172.21.15.131:0/522261890 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:47.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.908+0000 7f2512a00700 1 --2- 172.21.15.131:0/522261890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f251410dba0 0x7f25141a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:47.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.918+0000 7f2513400700 1 -- 172.21.15.131:0/522261890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f251410f520 msgr2=0x7f25141a6860 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T22:12:47.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.918+0000 7f2513400700 1 --2- 172.21.15.131:0/522261890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f251410f520 0x7f25141a6860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:47.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.918+0000 7f2513400700 1 -- 172.21.15.131:0/522261890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f251410dba0 msgr2=0x7f25141a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:47.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.918+0000 7f2513400700 1 --2- 172.21.15.131:0/522261890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f251410dba0 0x7f25141a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:47.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.918+0000 7f2513400700 1 -- 172.21.15.131:0/522261890 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25000095f0 con 0x7f25141aac00 2024-05-08T22:12:47.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.928+0000 7f2513400700 1 --2- 172.21.15.131:0/522261890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25141aac00 0x7f25141ab080 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f250c00b780 tx=0x7f250c00bb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:47.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.939+0000 7f2518a00700 1 -- 172.21.15.131:0/522261890 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c00f840 con 0x7f25141aac00 2024-05-08T22:12:47.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.939+0000 7f2518a00700 1 -- 172.21.15.131:0/522261890 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f250c00fe60 con 0x7f25141aac00 2024-05-08T22:12:47.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.939+0000 7f2518a00700 1 -- 172.21.15.131:0/522261890 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c00d3d0 con 0x7f25141aac00 2024-05-08T22:12:47.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.945+0000 7f2519e00700 1 -- 172.21.15.131:0/522261890 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251410ff60 con 0x7f25141aac00 2024-05-08T22:12:47.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:47.945+0000 7f2519e00700 1 -- 172.21.15.131:0/522261890 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25141103f0 con 0x7f25141aac00 2024-05-08T22:12:48.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.000+0000 7f252d907b80 1 Processor -- start 2024-05-08T22:12:48.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.009+0000 7f252d907b80 1 -- start start 2024-05-08T22:12:48.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.009+0000 7f252d907b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573a0b16050 0x5573a0b16430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:48.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.009+0000 7f252d907b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573a0b16970 0x5573a0b10350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:48.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.009+0000 7f252d907b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573a0b10890 0x5573a0b10cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:48.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.009+0000 7f252d907b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a0a67150 con 0x5573a0b16050 2024-05-08T22:12:48.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.009+0000 7f252d907b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a0a64b10 con 0x5573a0b16970 2024-05-08T22:12:48.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.009+0000 7f252d907b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a0a60400 con 0x5573a0b10890 2024-05-08T22:12:48.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.011+0000 7f2512a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573a0b16970 0x5573a0b10350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:48.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.011+0000 7f2512a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573a0b16970 0x5573a0b10350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:51274/0 (socket says 172.21.15.131:51274) 2024-05-08T22:12:48.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.011+0000 7f2512a00700 1 -- 172.21.15.131:0/3991093471 learned_addr learned my addr 172.21.15.131:0/3991093471 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:48.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.014+0000 7f2512a00700 1 -- 172.21.15.131:0/3991093471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573a0b10890 msgr2=0x5573a0b10cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:12:48.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.014+0000 7f2512a00700 1 --2- 172.21.15.131:0/3991093471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573a0b10890 0x5573a0b10cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.014+0000 7f2512a00700 1 -- 172.21.15.131:0/3991093471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573a0b16050 msgr2=0x5573a0b16430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:48.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.014+0000 7f2512a00700 1 --2- 172.21.15.131:0/3991093471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573a0b16050 0x5573a0b16430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.014+0000 7f2512a00700 1 -- 172.21.15.131:0/3991093471 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573a08e13b0 con 0x5573a0b16970 2024-05-08T22:12:48.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.019+0000 7f2512a00700 1 --2- 172.21.15.131:0/3991093471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573a0b16970 0x5573a0b10350 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f250400bd70 tx=0x7f250400f640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31b4ae3dd365519 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:48.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.020+0000 7f2518a00700 1 -- 172.21.15.131:0/522261890 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f250c00f9a0 con 0x7f25141aac00 2024-05-08T22:12:48.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.024+0000 7f2518a00700 1 --2- 172.21.15.131:0/522261890 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f24f80a14a0 0x7f24f80a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:48.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.042+0000 7f24ffe00700 1 -- 172.21.15.131:0/3991093471 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250400fe50 con 0x5573a0b16970 2024-05-08T22:12:48.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.042+0000 7f24ffe00700 1 -- 172.21.15.131:0/3991093471 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f250401a920 con 0x5573a0b16970 2024-05-08T22:12:48.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.042+0000 7f24ffe00700 1 -- 172.21.15.131:0/3991093471 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504011840 con 0x5573a0b16970 2024-05-08T22:12:48.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.043+0000 7f2512000700 1 --2- 172.21.15.131:0/522261890 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f24f80a14a0 0x7f24f80a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:48.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.044+0000 7f2518a00700 1 -- 172.21.15.131:0/522261890 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f250c0cd300 con 0x7f25141aac00 2024-05-08T22:12:48.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.058+0000 7f252d907b80 1 -- 172.21.15.131:0/3991093471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573a0b16970 msgr2=0x5573a0b10350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:48.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.058+0000 7f252d907b80 1 --2- 172.21.15.131:0/3991093471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573a0b16970 0x5573a0b10350 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f250400bd70 tx=0x7f250400f640 comp rx=0 tx=0).stop 2024-05-08T22:12:48.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.060+0000 7f2512000700 1 --2- 172.21.15.131:0/522261890 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f24f80a14a0 0x7f24f80a3960 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f250000b5c0 tx=0x7f2500005fb0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:48.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.072+0000 7f252d907b80 1 -- 172.21.15.131:0/3991093471 shutdown_connections 2024-05-08T22:12:48.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.072+0000 7f252d907b80 1 --2- 172.21.15.131:0/3991093471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573a0b10890 0x5573a0b10cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.072+0000 7f252d907b80 1 --2- 172.21.15.131:0/3991093471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573a0b16050 0x5573a0b16430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.072+0000 7f252d907b80 1 --2- 172.21.15.131:0/3991093471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573a0b16970 0x5573a0b10350 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.072+0000 7f252d907b80 1 -- 172.21.15.131:0/3991093471 >> 172.21.15.131:0/3991093471 conn(0x5573a0a6e210 msgr2=0x5573a0b0b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:48.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.078+0000 7f252d907b80 1 -- 172.21.15.131:0/3991093471 shutdown_connections 2024-05-08T22:12:48.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.079+0000 7f252d907b80 1 -- 172.21.15.131:0/3991093471 wait complete. 2024-05-08T22:12:48.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.081+0000 7f252d907b80 1 -- start start 2024-05-08T22:12:48.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.081+0000 7f252d907b80 1 Processor -- start 2024-05-08T22:12:48.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.101+0000 7f252d907b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573a0b10890 0x5573a0b1c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:48.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.101+0000 7f252d907b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573a0b16050 0x5573a0b19ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:48.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.108+0000 7f2512a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573a0b10890 0x5573a0b1c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:48.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.109+0000 7f252d907b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573a0b16970 0x5573a0b1a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:48.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.109+0000 7f2512000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573a0b16050 0x5573a0b19ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:48.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.109+0000 7f2512000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573a0b16050 0x5573a0b19ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56026/0 (socket says 172.21.15.131:56026) 2024-05-08T22:12:48.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.110+0000 7f2513400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573a0b16970 0x5573a0b1a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:48.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.110+0000 7f2513400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573a0b16970 0x5573a0b1a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:51282/0 (socket says 172.21.15.131:51282) 2024-05-08T22:12:48.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.110+0000 7f2512000700 1 -- 172.21.15.131:0/1666900963 learned_addr learned my addr 172.21.15.131:0/1666900963 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:48.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.110+0000 7f252d907b80 1 -- 172.21.15.131:0/1666900963 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a09fc560 con 0x5573a0b10890 2024-05-08T22:12:48.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.110+0000 7f252d907b80 1 -- 172.21.15.131:0/1666900963 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a09fc200 con 0x5573a0b16970 2024-05-08T22:12:48.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.111+0000 7f252d907b80 1 -- 172.21.15.131:0/1666900963 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a08e36a0 con 0x5573a0b16050 2024-05-08T22:12:48.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.111+0000 7f252d907b80 10 client.0 _ll_register_callbacks cb 0x5573a0a05b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:12:48.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.111+0000 7f2513400700 1 -- 172.21.15.131:0/1666900963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573a0b16050 msgr2=0x5573a0b19ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:48.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.111+0000 7f2513400700 1 --2- 172.21.15.131:0/1666900963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573a0b16050 0x5573a0b19ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.111+0000 7f2513400700 1 -- 172.21.15.131:0/1666900963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573a0b10890 msgr2=0x5573a0b1c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:48.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.111+0000 7f2513400700 1 --2- 172.21.15.131:0/1666900963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573a0b10890 0x5573a0b1c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.111+0000 7f2513400700 1 -- 172.21.15.131:0/1666900963 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f250400b9a0 con 0x5573a0b16970 2024-05-08T22:12:48.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.111+0000 7f2513400700 1 --2- 172.21.15.131:0/1666900963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573a0b16970 0x5573a0b1a3e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f250c0ca660 tx=0x7f250c09a7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f068924b012a9b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:48.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.111+0000 7f24fcc00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:12:48.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.112+0000 7f24fcc00700 1 -- 172.21.15.131:0/1666900963 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c0a24a0 con 0x5573a0b16970 2024-05-08T22:12:48.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.112+0000 7f24fcc00700 1 -- 172.21.15.131:0/1666900963 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f250c017620 con 0x5573a0b16970 2024-05-08T22:12:48.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.118+0000 7f252d907b80 1 -- 172.21.15.131:0/1666900963 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573a09b9270 con 0x5573a0b16970 2024-05-08T22:12:48.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.128+0000 7f24fcc00700 1 -- 172.21.15.131:0/1666900963 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f250c0b82d0 con 0x5573a0b16970 2024-05-08T22:12:48.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.128+0000 7f252d907b80 10 client.26219 fetch_fsmap learned FSMap version 26 2024-05-08T22:12:48.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.128+0000 7f252d907b80 1 -- 172.21.15.131:0/1666900963 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5573a088db60 con 0x5573a0b16970 2024-05-08T22:12:48.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.128+0000 7f24fcc00700 1 -- 172.21.15.131:0/1666900963 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5573a09b9270 con 0x5573a0b16970 2024-05-08T22:12:48.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.138+0000 7f24fcc00700 1 -- 172.21.15.131:0/1666900963 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f250c099510 con 0x5573a0b16970 2024-05-08T22:12:48.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.148+0000 7f252d907b80 10 client.26219 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:12:48.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.148+0000 7f252d907b80 10 client.26219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:12:48.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.148+0000 7f252d907b80 20 client.26219 populate_metadata read hostname 'smithi131' 2024-05-08T22:12:48.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.148+0000 7f252d907b80 1 --2- 172.21.15.131:0/1666900963 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5573a0b2a2b0 0x5573a0b2c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:48.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.148+0000 7f252d907b80 4 client.26219 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:12:48.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.148+0000 7f252d907b80 1 -- 172.21.15.131:0/1666900963 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573a0adf810 con 0x5573a0b2a2b0 2024-05-08T22:12:48.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.150+0000 7f2512a00700 1 --2- 172.21.15.131:0/1666900963 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5573a0b2a2b0 0x5573a0b2c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:48.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.156+0000 7f2512a00700 1 --2- 172.21.15.131:0/1666900963 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5573a0b2a2b0 0x5573a0b2c6d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdb0b870437ab861 server_cookie=2750ec8fc6727b9b in_seq=0 out_seq=0 2024-05-08T22:12:48.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.161+0000 7f24fcc00700 10 client.26219 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:12:48.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.161+0000 7f24fcc00700 1 -- 172.21.15.131:0/1666900963 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5573a0adf810 con 0x5573a0b2a2b0 2024-05-08T22:12:48.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.161+0000 7f24fcc00700 10 client.26219 handle_command_reply: tid=0 2024-05-08T22:12:48.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.165+0000 7f252d907b80 1 client.26219 shutdown 2024-05-08T22:12:48.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.176+0000 7f252d907b80 1 -- 172.21.15.131:0/1666900963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573a0b16970 msgr2=0x5573a0b1a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:48.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.176+0000 7f252d907b80 1 --2- 172.21.15.131:0/1666900963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573a0b16970 0x5573a0b1a3e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f250c0ca660 tx=0x7f250c09a7d0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.185+0000 7f252d907b80 1 -- 172.21.15.131:0/1666900963 shutdown_connections 2024-05-08T22:12:48.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.185+0000 7f252d907b80 1 --2- 172.21.15.131:0/1666900963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573a0b16970 0x5573a0b1a3e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.185+0000 7f252d907b80 1 --2- 172.21.15.131:0/1666900963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573a0b10890 0x5573a0b1c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.185+0000 7f252d907b80 1 --2- 172.21.15.131:0/1666900963 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5573a0b2a2b0 0x5573a0b2c6d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.185+0000 7f252d907b80 1 --2- 172.21.15.131:0/1666900963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573a0b16050 0x5573a0b19ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.185+0000 7f252d907b80 1 -- 172.21.15.131:0/1666900963 >> 172.21.15.131:0/1666900963 conn(0x5573a0a6e210 msgr2=0x5573a0b0b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:48.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.185+0000 7f24fcc00700 0 client.26219 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:12:48.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.227+0000 7f252d907b80 1 -- 172.21.15.131:0/1666900963 shutdown_connections 2024-05-08T22:12:48.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.229+0000 7f252d907b80 1 -- 172.21.15.131:0/1666900963 wait complete. 2024-05-08T22:12:48.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.244+0000 7f252d907b80 20 client.26219 trim_cache size 0 max 16384 2024-05-08T22:12:48.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:48 smithi132 ceph-mon[78180]: pgmap v168: 129 pgs: 129 active+clean; 791 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 581 op/s 2024-05-08T22:12:48.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:48 smithi131 ceph-mon[37260]: pgmap v168: 129 pgs: 129 active+clean; 791 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 581 op/s 2024-05-08T22:12:48.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.725+0000 7f252d907b80 1 Processor -- start 2024-05-08T22:12:48.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.741+0000 7f252d907b80 1 -- start start 2024-05-08T22:12:48.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.741+0000 7f252d907b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573a0b10890 0x5573a0b10350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:48.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.741+0000 7f252d907b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573a0b16050 0x5573a0b1c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:48.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.741+0000 7f252d907b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573a0b16970 0x5573a0b31030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:48.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.741+0000 7f252d907b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a0b1ac20 con 0x5573a0b16050 2024-05-08T22:12:48.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.741+0000 7f252d907b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a0b16790 con 0x5573a0b16970 2024-05-08T22:12:48.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.741+0000 7f252d907b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a0b1a9e0 con 0x5573a0b10890 2024-05-08T22:12:48.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.742+0000 7f2513400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573a0b16970 0x5573a0b31030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:48.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.742+0000 7f2513400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573a0b16970 0x5573a0b31030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:51294/0 (socket says 172.21.15.131:51294) 2024-05-08T22:12:48.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.742+0000 7f2513400700 1 -- 172.21.15.131:0/938019900 learned_addr learned my addr 172.21.15.131:0/938019900 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:48.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.744+0000 7f2512000700 1 --2- 172.21.15.131:0/938019900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573a0b16050 0x5573a0b1c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:48.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.748+0000 7f2512000700 1 -- 172.21.15.131:0/938019900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573a0b10890 msgr2=0x5573a0b10350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:12:48.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.748+0000 7f2512000700 1 --2- 172.21.15.131:0/938019900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573a0b10890 0x5573a0b10350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.748+0000 7f2512000700 1 -- 172.21.15.131:0/938019900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573a0b16970 msgr2=0x5573a0b31030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:48.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.748+0000 7f2512000700 1 --2- 172.21.15.131:0/938019900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573a0b16970 0x5573a0b31030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.748+0000 7f2512000700 1 -- 172.21.15.131:0/938019900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573a0a26560 con 0x5573a0b16050 2024-05-08T22:12:48.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.753+0000 7f2512000700 1 --2- 172.21.15.131:0/938019900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573a0b16050 0x5573a0b1c4e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x5573a0b1b270 tx=0x7f250001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59a5130d62b072b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:48.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.761+0000 7f2509200700 1 -- 172.21.15.131:0/938019900 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250001d070 con 0x5573a0b16050 2024-05-08T22:12:48.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.761+0000 7f2509200700 1 -- 172.21.15.131:0/938019900 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f250002a040 con 0x5573a0b16050 2024-05-08T22:12:48.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.761+0000 7f2509200700 1 -- 172.21.15.131:0/938019900 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250000f460 con 0x5573a0b16050 2024-05-08T22:12:48.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.766+0000 7f252d907b80 1 -- 172.21.15.131:0/938019900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573a0b16050 msgr2=0x5573a0b1c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:48.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.766+0000 7f252d907b80 1 --2- 172.21.15.131:0/938019900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573a0b16050 0x5573a0b1c4e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x5573a0b1b270 tx=0x7f250001d000 comp rx=0 tx=0).stop 2024-05-08T22:12:48.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:48 smithi157 ceph-mon[77746]: pgmap v168: 129 pgs: 129 active+clean; 791 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 581 op/s 2024-05-08T22:12:48.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.782+0000 7f252d907b80 1 -- 172.21.15.131:0/938019900 shutdown_connections 2024-05-08T22:12:48.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.782+0000 7f252d907b80 1 --2- 172.21.15.131:0/938019900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573a0b16970 0x5573a0b31030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.782+0000 7f252d907b80 1 --2- 172.21.15.131:0/938019900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573a0b10890 0x5573a0b10350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.782+0000 7f252d907b80 1 --2- 172.21.15.131:0/938019900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573a0b16050 0x5573a0b1c4e0 secure :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0x5573a0b1b270 tx=0x7f250001d000 comp rx=0 tx=0).stop 2024-05-08T22:12:48.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.782+0000 7f252d907b80 1 -- 172.21.15.131:0/938019900 >> 172.21.15.131:0/938019900 conn(0x5573a0a6e210 msgr2=0x5573a0b2c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:48.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.791+0000 7f252d907b80 1 -- 172.21.15.131:0/938019900 shutdown_connections 2024-05-08T22:12:48.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.807+0000 7f252d907b80 1 -- 172.21.15.131:0/938019900 wait complete. 2024-05-08T22:12:48.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.807+0000 7f252d907b80 1 -- start start 2024-05-08T22:12:48.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.808+0000 7f252d907b80 1 Processor -- start 2024-05-08T22:12:48.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.812+0000 7f252d907b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573a0b10890 0x5573a0b21360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:48.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.814+0000 7f252d907b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573a0b16970 0x5573a0b218a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:48.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.814+0000 7f2512a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573a0b10890 0x5573a0b21360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:48.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.814+0000 7f2512a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573a0b10890 0x5573a0b21360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56040/0 (socket says 172.21.15.131:56040) 2024-05-08T22:12:48.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.816+0000 7f2512000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573a0b16970 0x5573a0b218a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:48.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.816+0000 7f2512a00700 1 -- 172.21.15.131:0/973227372 learned_addr learned my addr 172.21.15.131:0/973227372 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:12:48.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.816+0000 7f252d907b80 1 --2- 172.21.15.131:0/973227372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573a0b12050 0x5573a0b124d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:48.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.817+0000 7f2513400700 1 --2- 172.21.15.131:0/973227372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573a0b12050 0x5573a0b124d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:48.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.817+0000 7f252d907b80 1 -- 172.21.15.131:0/973227372 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a0b12bb0 con 0x5573a0b16970 2024-05-08T22:12:48.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.817+0000 7f252d907b80 1 -- 172.21.15.131:0/973227372 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a0b12d20 con 0x5573a0b12050 2024-05-08T22:12:48.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.817+0000 7f252d907b80 1 -- 172.21.15.131:0/973227372 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a0b807b0 con 0x5573a0b10890 2024-05-08T22:12:48.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.818+0000 7f2512000700 1 -- 172.21.15.131:0/973227372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573a0b10890 msgr2=0x5573a0b21360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:48.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.818+0000 7f2512000700 1 --2- 172.21.15.131:0/973227372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573a0b10890 0x5573a0b21360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.818+0000 7f2512000700 1 -- 172.21.15.131:0/973227372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573a0b12050 msgr2=0x5573a0b124d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:12:48.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.818+0000 7f2512000700 1 --2- 172.21.15.131:0/973227372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573a0b12050 0x5573a0b124d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.818+0000 7f2512000700 1 -- 172.21.15.131:0/973227372 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251411d300 con 0x5573a0b16970 2024-05-08T22:12:48.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.819+0000 7f252d907b80 10 client.0 _ll_register_callbacks cb 0x5573a0a05b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:12:48.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.820+0000 7f2512000700 1 --2- 172.21.15.131:0/973227372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573a0b16970 0x5573a0b218a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x5573a0b1aed0 tx=0x7f2500003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe2f22d7987fe12 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:12:48.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.819+0000 7f252d907b80 1 -- 172.21.15.131:0/973227372 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f250401a920 con 0x5573a0b16970 2024-05-08T22:12:48.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.826+0000 7f24fcc00700 10 client.16194 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:12:48.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.826+0000 7f24fcc00700 1 -- 172.21.15.131:0/973227372 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250000f660 con 0x5573a0b16970 2024-05-08T22:12:48.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.826+0000 7f24fcc00700 1 -- 172.21.15.131:0/973227372 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f250000bab0 con 0x5573a0b16970 2024-05-08T22:12:48.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.826+0000 7f24fcc00700 1 -- 172.21.15.131:0/973227372 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f250000f7c0 con 0x5573a0b16970 2024-05-08T22:12:48.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.848+0000 7f252d907b80 10 client.16194 fetch_fsmap learned FSMap version 26 2024-05-08T22:12:48.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.848+0000 7f252d907b80 1 -- 172.21.15.131:0/973227372 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=26}) v3 -- 0x5573a08e5cd0 con 0x5573a0b16970 2024-05-08T22:12:48.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.853+0000 7f24fcc00700 1 -- 172.21.15.131:0/973227372 <== mon.0 v2:172.21.15.131:3300/0 4 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f250000f920 con 0x5573a0b16970 2024-05-08T22:12:48.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.853+0000 7f252d907b80 10 client.16194 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:12:48.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.853+0000 7f252d907b80 10 client.16194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:12:48.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.853+0000 7f252d907b80 20 client.16194 populate_metadata read hostname 'smithi131' 2024-05-08T22:12:48.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.853+0000 7f252d907b80 1 --2- 172.21.15.131:0/973227372 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5573a0b81650 0x5573a0b83a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:12:48.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.853+0000 7f252d907b80 4 client.16194 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:12:48.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.853+0000 7f252d907b80 1 -- 172.21.15.131:0/973227372 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f250401a920 con 0x5573a0b81650 2024-05-08T22:12:48.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.858+0000 7f2512a00700 1 --2- 172.21.15.131:0/973227372 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5573a0b81650 0x5573a0b83a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:12:48.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.861+0000 7f2512a00700 1 --2- 172.21.15.131:0/973227372 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5573a0b81650 0x5573a0b83a70 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62c63797fbc08780 server_cookie=db8a38abb0f7d767 in_seq=0 out_seq=0 2024-05-08T22:12:48.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.862+0000 7f24fcc00700 10 client.16194 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:12:48.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.880+0000 7f24fcc00700 1 -- 172.21.15.131:0/973227372 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+269 (crc 0 0 0) 0x7f250401a920 con 0x5573a0b81650 2024-05-08T22:12:48.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.880+0000 7f24fcc00700 10 client.16194 handle_command_reply: tid=0 2024-05-08T22:12:48.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.885+0000 7f252d907b80 1 client.16194 shutdown 2024-05-08T22:12:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.894+0000 7f252d907b80 1 -- 172.21.15.131:0/973227372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573a0b16970 msgr2=0x5573a0b218a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.894+0000 7f252d907b80 1 --2- 172.21.15.131:0/973227372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573a0b16970 0x5573a0b218a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x5573a0b1aed0 tx=0x7f2500003aa0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.903+0000 7f252d907b80 1 -- 172.21.15.131:0/973227372 shutdown_connections 2024-05-08T22:12:48.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.903+0000 7f252d907b80 1 --2- 172.21.15.131:0/973227372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5573a0b12050 0x5573a0b124d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.903+0000 7f252d907b80 1 --2- 172.21.15.131:0/973227372 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5573a0b81650 0x5573a0b83a70 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.903+0000 7f252d907b80 1 --2- 172.21.15.131:0/973227372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573a0b10890 0x5573a0b21360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.903+0000 7f252d907b80 1 --2- 172.21.15.131:0/973227372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5573a0b16970 0x5573a0b218a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.903+0000 7f252d907b80 1 -- 172.21.15.131:0/973227372 >> 172.21.15.131:0/973227372 conn(0x5573a0a6e210 msgr2=0x5573a0b31030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:48.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.903+0000 7f24fcc00700 0 client.16194 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:12:48.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.919+0000 7f252d907b80 1 -- 172.21.15.131:0/973227372 shutdown_connections 2024-05-08T22:12:48.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.928+0000 7f252d907b80 1 -- 172.21.15.131:0/973227372 wait complete. 2024-05-08T22:12:48.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.937+0000 7f252d907b80 20 client.16194 trim_cache size 0 max 16384 2024-05-08T22:12:48.957 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:12:48.958 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "scrub active (13890 inodes in the stack)", 2024-05-08T22:12:48.958 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-08T22:12:48.958 INFO:teuthology.orchestra.run.smithi131.stdout: "ca36159f-7052-4d2f-921f-ee97cd0c21fb": { 2024-05-08T22:12:48.958 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-08T22:12:48.958 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "ca36159f-7052-4d2f-921f-ee97cd0c21fb", 2024-05-08T22:12:48.958 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-08T22:12:48.958 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:12:48.958 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:12:48.958 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:12:48.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.982+0000 7f2519e00700 1 -- 172.21.15.131:0/522261890 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f24f80a14a0 msgr2=0x7f24f80a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:48.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.982+0000 7f2519e00700 1 --2- 172.21.15.131:0/522261890 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f24f80a14a0 0x7f24f80a3960 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f250000b5c0 tx=0x7f2500005fb0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.982+0000 7f2519e00700 1 -- 172.21.15.131:0/522261890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25141aac00 msgr2=0x7f25141ab080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:12:48.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.982+0000 7f2519e00700 1 --2- 172.21.15.131:0/522261890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25141aac00 0x7f25141ab080 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f250c00b780 tx=0x7f250c00bb40 comp rx=0 tx=0).stop 2024-05-08T22:12:48.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.982+0000 7f2519e00700 1 -- 172.21.15.131:0/522261890 shutdown_connections 2024-05-08T22:12:48.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.982+0000 7f2519e00700 1 --2- 172.21.15.131:0/522261890 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f24f80a14a0 0x7f24f80a3960 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.982+0000 7f2519e00700 1 --2- 172.21.15.131:0/522261890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25141aac00 0x7f25141ab080 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.982+0000 7f2519e00700 1 --2- 172.21.15.131:0/522261890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f251410dba0 0x7f25141a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.982+0000 7f2519e00700 1 --2- 172.21.15.131:0/522261890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f251410f520 0x7f25141a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:12:48.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.982+0000 7f2519e00700 1 -- 172.21.15.131:0/522261890 >> 172.21.15.131:0/522261890 conn(0x7f251406be80 msgr2=0x7f2514070550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:12:48.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.983+0000 7f2519e00700 1 -- 172.21.15.131:0/522261890 shutdown_connections 2024-05-08T22:12:48.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:12:48.983+0000 7f2519e00700 1 -- 172.21.15.131:0/522261890 wait complete. 2024-05-08T22:12:49.243 INFO:tasks.cephfs.filesystem:scrub status for tag:ca36159f-7052-4d2f-921f-ee97cd0c21fb - {'path': '/', 'tag': 'ca36159f-7052-4d2f-921f-ee97cd0c21fb', 'options': 'recursive,force'} 2024-05-08T22:12:50.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:50 smithi132 ceph-mon[78180]: pgmap v169: 129 pgs: 129 active+clean; 803 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.9 MiB/s wr, 829 op/s 2024-05-08T22:12:50.645 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:50 smithi131 ceph-mon[37260]: pgmap v169: 129 pgs: 129 active+clean; 803 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.9 MiB/s wr, 829 op/s 2024-05-08T22:12:50.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:50 smithi157 ceph-mon[77746]: pgmap v169: 129 pgs: 129 active+clean; 803 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.9 MiB/s wr, 829 op/s 2024-05-08T22:12:52.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:52 smithi132 ceph-mon[78180]: pgmap v170: 129 pgs: 129 active+clean; 808 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.4 MiB/s wr, 809 op/s 2024-05-08T22:12:52.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:52 smithi131 ceph-mon[37260]: pgmap v170: 129 pgs: 129 active+clean; 808 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.4 MiB/s wr, 809 op/s 2024-05-08T22:12:52.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:52 smithi157 ceph-mon[77746]: pgmap v170: 129 pgs: 129 active+clean; 808 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.4 MiB/s wr, 809 op/s 2024-05-08T22:12:54.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:54 smithi132 ceph-mon[78180]: pgmap v171: 129 pgs: 129 active+clean; 815 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.1 MiB/s wr, 786 op/s 2024-05-08T22:12:54.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:54 smithi131 ceph-mon[37260]: pgmap v171: 129 pgs: 129 active+clean; 815 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.1 MiB/s wr, 786 op/s 2024-05-08T22:12:54.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:54 smithi157 ceph-mon[77746]: pgmap v171: 129 pgs: 129 active+clean; 815 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.1 MiB/s wr, 786 op/s 2024-05-08T22:12:55.314 INFO:tasks.workunit.client.0.smithi131.stdout: 155 457179 327 297038 391 272584 1348 2024-05-08T22:12:56.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:56 smithi132 ceph-mon[78180]: pgmap v172: 129 pgs: 129 active+clean; 814 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.1 MiB/s wr, 924 op/s 2024-05-08T22:12:56.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:56 smithi131 ceph-mon[37260]: pgmap v172: 129 pgs: 129 active+clean; 814 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.1 MiB/s wr, 924 op/s 2024-05-08T22:12:56.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:56 smithi157 ceph-mon[77746]: pgmap v172: 129 pgs: 129 active+clean; 814 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.1 MiB/s wr, 924 op/s 2024-05-08T22:12:58.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:12:58 smithi132 ceph-mon[78180]: pgmap v173: 129 pgs: 129 active+clean; 814 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 603 op/s 2024-05-08T22:12:58.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:12:58 smithi131 ceph-mon[37260]: pgmap v173: 129 pgs: 129 active+clean; 814 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 603 op/s 2024-05-08T22:12:58.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:12:58 smithi157 ceph-mon[77746]: pgmap v173: 129 pgs: 129 active+clean; 814 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 603 op/s 2024-05-08T22:13:00.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:00 smithi131 ceph-mon[37260]: pgmap v174: 129 pgs: 129 active+clean; 816 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.8 MiB/s wr, 752 op/s 2024-05-08T22:13:00.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:00 smithi157 ceph-mon[77746]: pgmap v174: 129 pgs: 129 active+clean; 816 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.8 MiB/s wr, 752 op/s 2024-05-08T22:13:00.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:00 smithi132 ceph-mon[78180]: pgmap v174: 129 pgs: 129 active+clean; 816 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.8 MiB/s wr, 752 op/s 2024-05-08T22:13:01.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:01 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:13:01.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:01 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:13:01.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:01 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:13:02.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:02 smithi131 ceph-mon[37260]: pgmap v175: 129 pgs: 129 active+clean; 820 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 502 op/s 2024-05-08T22:13:02.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:02 smithi157 ceph-mon[77746]: pgmap v175: 129 pgs: 129 active+clean; 820 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 502 op/s 2024-05-08T22:13:02.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:02 smithi132 ceph-mon[78180]: pgmap v175: 129 pgs: 129 active+clean; 820 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 502 op/s 2024-05-08T22:13:04.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:04 smithi131 ceph-mon[37260]: pgmap v176: 129 pgs: 129 active+clean; 820 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 476 op/s 2024-05-08T22:13:04.775 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:04 smithi157 ceph-mon[77746]: pgmap v176: 129 pgs: 129 active+clean; 820 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 476 op/s 2024-05-08T22:13:04.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:04 smithi132 ceph-mon[78180]: pgmap v176: 129 pgs: 129 active+clean; 820 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 476 op/s 2024-05-08T22:13:05.326 INFO:tasks.workunit.client.0.smithi131.stdout: 163 533884 459 346026 372 316068 1198 2024-05-08T22:13:06.651 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:06 smithi131 ceph-mon[37260]: pgmap v177: 129 pgs: 129 active+clean; 826 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 694 op/s 2024-05-08T22:13:06.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:06 smithi157 ceph-mon[77746]: pgmap v177: 129 pgs: 129 active+clean; 826 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 694 op/s 2024-05-08T22:13:06.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:06 smithi132 ceph-mon[78180]: pgmap v177: 129 pgs: 129 active+clean; 826 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 694 op/s 2024-05-08T22:13:08.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:08 smithi157 ceph-mon[77746]: pgmap v178: 129 pgs: 129 active+clean; 826 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 545 op/s 2024-05-08T22:13:08.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:08 smithi132 ceph-mon[78180]: pgmap v178: 129 pgs: 129 active+clean; 826 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 545 op/s 2024-05-08T22:13:08.903 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:08 smithi131 ceph-mon[37260]: pgmap v178: 129 pgs: 129 active+clean; 826 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 545 op/s 2024-05-08T22:13:10.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:10 smithi157 ceph-mon[77746]: pgmap v179: 129 pgs: 129 active+clean; 848 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.7 MiB/s wr, 774 op/s 2024-05-08T22:13:10.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:10 smithi132 ceph-mon[78180]: pgmap v179: 129 pgs: 129 active+clean; 848 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.7 MiB/s wr, 774 op/s 2024-05-08T22:13:10.894 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:10 smithi131 ceph-mon[37260]: pgmap v179: 129 pgs: 129 active+clean; 848 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.7 MiB/s wr, 774 op/s 2024-05-08T22:13:11.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:11 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:13:11.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:11 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:13:11.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:11 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:13:11.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:11 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:13:11.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:11 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:13:11.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:11 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:13:11.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:11 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:13:11.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:11 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:13:11.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:11 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:13:15.332 INFO:tasks.workunit.client.0.smithi131.stdout: 169 436333 502 287373 432 248120 938 2024-05-08T22:13:16.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:16 smithi132 ceph-mon[78180]: pgmap v180: 129 pgs: 129 active+clean; 853 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.2 MiB/s wr, 658 op/s 2024-05-08T22:13:16.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:16 smithi132 ceph-mon[78180]: pgmap v181: 129 pgs: 129 active+clean; 853 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 613 op/s 2024-05-08T22:13:16.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:16 smithi132 ceph-mon[78180]: pgmap v182: 129 pgs: 129 active+clean; 894 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.4 MiB/s wr, 789 op/s 2024-05-08T22:13:16.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:16 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:13:16.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:16 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:13:16.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:16 smithi132 ceph-mon[78180]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:13:16.644 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:16 smithi131 ceph-mon[37260]: pgmap v180: 129 pgs: 129 active+clean; 853 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.2 MiB/s wr, 658 op/s 2024-05-08T22:13:16.644 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:16 smithi131 ceph-mon[37260]: pgmap v181: 129 pgs: 129 active+clean; 853 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 613 op/s 2024-05-08T22:13:16.644 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:16 smithi131 ceph-mon[37260]: pgmap v182: 129 pgs: 129 active+clean; 894 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.4 MiB/s wr, 789 op/s 2024-05-08T22:13:16.644 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:16 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:13:16.644 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:16 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:13:16.644 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:16 smithi131 ceph-mon[37260]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:13:16.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:16 smithi157 ceph-mon[77746]: pgmap v180: 129 pgs: 129 active+clean; 853 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.2 MiB/s wr, 658 op/s 2024-05-08T22:13:16.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:16 smithi157 ceph-mon[77746]: pgmap v181: 129 pgs: 129 active+clean; 853 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 613 op/s 2024-05-08T22:13:16.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:16 smithi157 ceph-mon[77746]: pgmap v182: 129 pgs: 129 active+clean; 894 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.4 MiB/s wr, 789 op/s 2024-05-08T22:13:16.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:16 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:13:16.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:16 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:13:16.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:16 smithi157 ceph-mon[77746]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:13:18.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:18 smithi132 ceph-mon[78180]: pgmap v183: 129 pgs: 129 active+clean; 899 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.5 MiB/s wr, 586 op/s 2024-05-08T22:13:18.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:18 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:13:18.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:18 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:13:18.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:18 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:13:18.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:18 smithi132 ceph-mon[78180]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:13:18.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:18 smithi131 ceph-mon[37260]: pgmap v183: 129 pgs: 129 active+clean; 899 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.5 MiB/s wr, 586 op/s 2024-05-08T22:13:18.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:18 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:13:18.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:18 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:13:18.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:18 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:13:18.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:18 smithi131 ceph-mon[37260]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:13:18.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:18 smithi157 ceph-mon[77746]: pgmap v183: 129 pgs: 129 active+clean; 899 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.5 MiB/s wr, 586 op/s 2024-05-08T22:13:18.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:18 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:13:18.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:18 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:13:18.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:18 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:13:18.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:18 smithi157 ceph-mon[77746]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:13:19.245 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:13:20.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:20 smithi132 ceph-mon[78180]: pgmap v184: 129 pgs: 129 active+clean; 913 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.4 MiB/s wr, 773 op/s 2024-05-08T22:13:20.677 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:20 smithi131 ceph-mon[37260]: pgmap v184: 129 pgs: 129 active+clean; 913 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.4 MiB/s wr, 773 op/s 2024-05-08T22:13:20.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:20 smithi157 ceph-mon[77746]: pgmap v184: 129 pgs: 129 active+clean; 913 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.4 MiB/s wr, 773 op/s 2024-05-08T22:13:22.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:22 smithi132 ceph-mon[78180]: pgmap v185: 129 pgs: 129 active+clean; 920 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 7.1 MiB/s wr, 668 op/s 2024-05-08T22:13:22.665 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:22 smithi131 ceph-mon[37260]: pgmap v185: 129 pgs: 129 active+clean; 920 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 7.1 MiB/s wr, 668 op/s 2024-05-08T22:13:22.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:22 smithi157 ceph-mon[77746]: pgmap v185: 129 pgs: 129 active+clean; 920 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 7.1 MiB/s wr, 668 op/s 2024-05-08T22:13:24.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.133+0000 7f17b5c00700 1 -- 172.21.15.131:0/159962747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17b010fb10 msgr2=0x7f17b01084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:13:24.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.133+0000 7f17b5c00700 1 --2- 172.21.15.131:0/159962747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17b010fb10 0x7f17b01084e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f179c009990 tx=0x7f179c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:13:24.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.135+0000 7f17af400700 1 --2- 172.21.15.131:0/159962747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17b011a780 0x7f17b011cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:13:24.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.144+0000 7f17b5c00700 1 -- 172.21.15.131:0/159962747 shutdown_connections 2024-05-08T22:13:24.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.144+0000 7f17b5c00700 1 --2- 172.21.15.131:0/159962747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17b011a780 0x7f17b011cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:24.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.144+0000 7f17b5c00700 1 --2- 172.21.15.131:0/159962747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b010f6a0 0x7f17b0107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:24.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.144+0000 7f17b5c00700 1 --2- 172.21.15.131:0/159962747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17b010fb10 0x7f17b01084e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:24.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.144+0000 7f17b5c00700 1 -- 172.21.15.131:0/159962747 >> 172.21.15.131:0/159962747 conn(0x7f17b006bff0 msgr2=0x7f17b006c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:13:24.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.167+0000 7f17b5c00700 1 -- 172.21.15.131:0/159962747 shutdown_connections 2024-05-08T22:13:24.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.171+0000 7f17b5c00700 1 -- 172.21.15.131:0/159962747 wait complete. 2024-05-08T22:13:24.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.172+0000 7f17b5c00700 1 Processor -- start 2024-05-08T22:13:24.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.175+0000 7f17b5c00700 1 -- start start 2024-05-08T22:13:24.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.189+0000 7f17b5c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b010f6a0 0x7f17b0113ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:13:24.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.189+0000 7f17b5c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17b010fb10 0x7f17b0114430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:13:24.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.189+0000 7f17b5c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17b011a780 0x7f17b0119770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:13:24.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.189+0000 7f17b5c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b0119f40 con 0x7f17b010f6a0 2024-05-08T22:13:24.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.189+0000 7f17b5c00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b011a0b0 con 0x7f17b010fb10 2024-05-08T22:13:24.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.189+0000 7f17b5c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b011a270 con 0x7f17b011a780 2024-05-08T22:13:24.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.190+0000 7f17aea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b010f6a0 0x7f17b0113ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:13:24.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.190+0000 7f17aea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b010f6a0 0x7f17b0113ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46032/0 (socket says 172.21.15.131:46032) 2024-05-08T22:13:24.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.190+0000 7f17aea00700 1 -- 172.21.15.131:0/2546249223 learned_addr learned my addr 172.21.15.131:0/2546249223 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:13:24.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.190+0000 7f17ae000700 1 --2- 172.21.15.131:0/2546249223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17b010fb10 0x7f17b0114430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:13:24.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.191+0000 7f17af400700 1 --2- 172.21.15.131:0/2546249223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17b011a780 0x7f17b0119770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:13:24.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.191+0000 7f17ae000700 1 -- 172.21.15.131:0/2546249223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17b011a780 msgr2=0x7f17b0119770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:13:24.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.191+0000 7f17ae000700 1 --2- 172.21.15.131:0/2546249223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17b011a780 0x7f17b0119770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:24.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.191+0000 7f17ae000700 1 -- 172.21.15.131:0/2546249223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b010f6a0 msgr2=0x7f17b0113ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:13:24.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.191+0000 7f17ae000700 1 --2- 172.21.15.131:0/2546249223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b010f6a0 0x7f17b0113ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:24.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.191+0000 7f17ae000700 1 -- 172.21.15.131:0/2546249223 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f179c0095f0 con 0x7f17b010fb10 2024-05-08T22:13:24.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.191+0000 7f17af400700 1 --2- 172.21.15.131:0/2546249223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17b011a780 0x7f17b0119770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:13:24.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.191+0000 7f17aea00700 1 --2- 172.21.15.131:0/2546249223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b010f6a0 0x7f17b0113ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:13:24.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.197+0000 7f17ae000700 1 --2- 172.21.15.131:0/2546249223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17b010fb10 0x7f17b0114430 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f179c006010 tx=0x7f179c003b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:13:24.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.212+0000 7f17a7400700 1 -- 172.21.15.131:0/2546249223 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c01d070 con 0x7f17b010fb10 2024-05-08T22:13:24.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.212+0000 7f17a7400700 1 -- 172.21.15.131:0/2546249223 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f179c003e80 con 0x7f17b010fb10 2024-05-08T22:13:24.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.212+0000 7f17a7400700 1 -- 172.21.15.131:0/2546249223 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c00fcc0 con 0x7f17b010fb10 2024-05-08T22:13:24.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.220+0000 7f17b5c00700 1 -- 172.21.15.131:0/2546249223 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b011a4f0 con 0x7f17b010fb10 2024-05-08T22:13:24.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.220+0000 7f17b5c00700 1 -- 172.21.15.131:0/2546249223 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b00793e0 con 0x7f17b010fb10 2024-05-08T22:13:24.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.246+0000 7f17a7400700 1 -- 172.21.15.131:0/2546249223 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f179c00f460 con 0x7f17b010fb10 2024-05-08T22:13:24.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.254+0000 7f17a7400700 1 --2- 172.21.15.131:0/2546249223 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f17940a14a0 0x7f17940a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:13:24.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.255+0000 7f17a7400700 1 -- 172.21.15.131:0/2546249223 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f179c0cdf60 con 0x7f17b010fb10 2024-05-08T22:13:24.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.255+0000 7f17aea00700 1 --2- 172.21.15.131:0/2546249223 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f17940a14a0 0x7f17940a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:13:24.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.262+0000 7f17aea00700 1 --2- 172.21.15.131:0/2546249223 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f17940a14a0 0x7f17940a3960 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f17a000f4d0 tx=0x7f17a0005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:13:24.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.278+0000 7f17c97d5b80 1 Processor -- start 2024-05-08T22:13:24.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.287+0000 7f17c97d5b80 1 -- start start 2024-05-08T22:13:24.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.288+0000 7f17c97d5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56455afc3e00 0x56455afc41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:13:24.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.288+0000 7f17c97d5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56455afc4720 0x56455afbe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:13:24.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.288+0000 7f17c97d5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56455afbe890 0x56455afbecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:13:24.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.288+0000 7f17c97d5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455af15150 con 0x56455afbe890 2024-05-08T22:13:24.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.288+0000 7f17c97d5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455af12b10 con 0x56455afc3e00 2024-05-08T22:13:24.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.288+0000 7f17c97d5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455af0e400 con 0x56455afc4720 2024-05-08T22:13:24.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.288+0000 7f17af400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56455afbe890 0x56455afbecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:13:24.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.289+0000 7f17af400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56455afbe890 0x56455afbecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46036/0 (socket says 172.21.15.131:46036) 2024-05-08T22:13:24.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.289+0000 7f17af400700 1 -- 172.21.15.131:0/523890233 learned_addr learned my addr 172.21.15.131:0/523890233 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:13:24.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.289+0000 7f17af400700 1 -- 172.21.15.131:0/523890233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56455afc4720 msgr2=0x56455afbe350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:13:24.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.289+0000 7f17af400700 1 --2- 172.21.15.131:0/523890233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56455afc4720 0x56455afbe350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:24.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.289+0000 7f17af400700 1 -- 172.21.15.131:0/523890233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56455afc3e00 msgr2=0x56455afc41e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:13:24.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.289+0000 7f17af400700 1 --2- 172.21.15.131:0/523890233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56455afc3e00 0x56455afc41e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:24.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.289+0000 7f17af400700 1 -- 172.21.15.131:0/523890233 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56455ad8f3b0 con 0x56455afbe890 2024-05-08T22:13:24.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.290+0000 7f17af400700 1 --2- 172.21.15.131:0/523890233 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56455afbe890 0x56455afbecd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f17a800d970 tx=0x7f17a800dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9587a7d8a41cb301 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:13:24.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.304+0000 7f179be00700 1 -- 172.21.15.131:0/523890233 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a800dec0 con 0x56455afbe890 2024-05-08T22:13:24.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.304+0000 7f179be00700 1 -- 172.21.15.131:0/523890233 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17a8004bb0 con 0x56455afbe890 2024-05-08T22:13:24.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.304+0000 7f179be00700 1 -- 172.21.15.131:0/523890233 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a8014670 con 0x56455afbe890 2024-05-08T22:13:24.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.313+0000 7f17c97d5b80 1 -- 172.21.15.131:0/523890233 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56455afbe890 msgr2=0x56455afbecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:13:24.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.313+0000 7f17c97d5b80 1 --2- 172.21.15.131:0/523890233 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56455afbe890 0x56455afbecd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f17a800d970 tx=0x7f17a800dc80 comp rx=0 tx=0).stop 2024-05-08T22:13:24.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.347+0000 7f17c97d5b80 1 -- 172.21.15.131:0/523890233 shutdown_connections 2024-05-08T22:13:24.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.347+0000 7f17c97d5b80 1 --2- 172.21.15.131:0/523890233 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56455afbe890 0x56455afbecd0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:24.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.347+0000 7f17c97d5b80 1 --2- 172.21.15.131:0/523890233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56455afc3e00 0x56455afc41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:24.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.347+0000 7f17c97d5b80 1 --2- 172.21.15.131:0/523890233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56455afc4720 0x56455afbe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:24.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.347+0000 7f17c97d5b80 1 -- 172.21.15.131:0/523890233 >> 172.21.15.131:0/523890233 conn(0x56455af1c210 msgr2=0x56455afb8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:13:24.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.350+0000 7f17c97d5b80 1 -- 172.21.15.131:0/523890233 shutdown_connections 2024-05-08T22:13:24.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.350+0000 7f17c97d5b80 1 -- 172.21.15.131:0/523890233 wait complete. 2024-05-08T22:13:24.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.352+0000 7f17c97d5b80 1 -- start start 2024-05-08T22:13:24.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.352+0000 7f17c97d5b80 1 Processor -- start 2024-05-08T22:13:24.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.365+0000 7f17c97d5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56455afbe890 0x56455afc7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:13:24.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.365+0000 7f17c97d5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56455afc3e00 0x56455afc81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:13:24.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.365+0000 7f17aea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56455afbe890 0x56455afc7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:13:24.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.365+0000 7f17aea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56455afbe890 0x56455afc7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:44062/0 (socket says 172.21.15.131:44062) 2024-05-08T22:13:24.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.365+0000 7f17aea00700 1 -- 172.21.15.131:0/3794956004 learned_addr learned my addr 172.21.15.131:0/3794956004 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:13:24.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.365+0000 7f17c97d5b80 1 --2- 172.21.15.131:0/3794956004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56455afc4720 0x56455afc8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:13:24.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.365+0000 7f17ae000700 1 --2- 172.21.15.131:0/3794956004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56455afc3e00 0x56455afc81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:13:24.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.366+0000 7f17af400700 1 --2- 172.21.15.131:0/3794956004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56455afc4720 0x56455afc8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:13:24.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.366+0000 7f17c97d5b80 1 -- 172.21.15.131:0/3794956004 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455aeaa560 con 0x56455afc4720 2024-05-08T22:13:24.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.366+0000 7f17c97d5b80 1 -- 172.21.15.131:0/3794956004 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455aeaa200 con 0x56455afbe890 2024-05-08T22:13:24.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.366+0000 7f17c97d5b80 1 -- 172.21.15.131:0/3794956004 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455ad916a0 con 0x56455afc3e00 2024-05-08T22:13:24.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.367+0000 7f17af400700 1 -- 172.21.15.131:0/3794956004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56455afc3e00 msgr2=0x56455afc81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:13:24.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.367+0000 7f17af400700 1 --2- 172.21.15.131:0/3794956004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56455afc3e00 0x56455afc81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:24.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.367+0000 7f17af400700 1 -- 172.21.15.131:0/3794956004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56455afbe890 msgr2=0x56455afc7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:13:24.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.367+0000 7f17af400700 1 --2- 172.21.15.131:0/3794956004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56455afbe890 0x56455afc7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:24.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.367+0000 7f17af400700 1 -- 172.21.15.131:0/3794956004 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a800d340 con 0x56455afc4720 2024-05-08T22:13:24.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.368+0000 7f17af400700 1 --2- 172.21.15.131:0/3794956004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56455afc4720 0x56455afc8720 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f17a8000f80 tx=0x7f17a800df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d22a8764845d33 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:13:24.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.371+0000 7f17c97d5b80 10 client.0 _ll_register_callbacks cb 0x56455aeb3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:13:24.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.371+0000 7f17c97d5b80 1 -- 172.21.15.131:0/3794956004 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56455ae67270 con 0x56455afc4720 2024-05-08T22:13:24.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.389+0000 7f1798c00700 10 client.16203 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:13:24.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.389+0000 7f1798c00700 1 -- 172.21.15.131:0/3794956004 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a8003a00 con 0x56455afc4720 2024-05-08T22:13:24.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.389+0000 7f1798c00700 1 -- 172.21.15.131:0/3794956004 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f17a8003b60 con 0x56455afc4720 2024-05-08T22:13:24.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.407+0000 7f1798c00700 1 -- 172.21.15.131:0/3794956004 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17a8003d40 con 0x56455afc4720 2024-05-08T22:13:24.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.448+0000 7f17c97d5b80 10 client.16203 fetch_fsmap learned FSMap version 26 2024-05-08T22:13:24.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.448+0000 7f17c97d5b80 1 -- 172.21.15.131:0/3794956004 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=26}) v3 -- 0x56455ad3bb60 con 0x56455afc4720 2024-05-08T22:13:24.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.450+0000 7f1798c00700 1 -- 172.21.15.131:0/3794956004 <== mon.0 v2:172.21.15.131:3300/0 4 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f17a8012070 con 0x56455afc4720 2024-05-08T22:13:24.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.461+0000 7f17c97d5b80 10 client.16203 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:13:24.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.461+0000 7f17c97d5b80 10 client.16203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:13:24.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.461+0000 7f17c97d5b80 20 client.16203 populate_metadata read hostname 'smithi131' 2024-05-08T22:13:24.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.461+0000 7f17c97d5b80 1 --2- 172.21.15.131:0/3794956004 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x56455afd7e40 0x56455afda260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:13:24.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.461+0000 7f17c97d5b80 4 client.16203 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:13:24.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.461+0000 7f17c97d5b80 1 -- 172.21.15.131:0/3794956004 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56455af8d810 con 0x56455afd7e40 2024-05-08T22:13:24.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.465+0000 7f17aea00700 1 --2- 172.21.15.131:0/3794956004 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x56455afd7e40 0x56455afda260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:13:24.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.473+0000 7f17aea00700 1 --2- 172.21.15.131:0/3794956004 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x56455afd7e40 0x56455afda260 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2185787b7d3cabe2 server_cookie=42aaac9f9baccf96 in_seq=0 out_seq=0 2024-05-08T22:13:24.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.473+0000 7f1798c00700 10 client.16203 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:13:24.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.478+0000 7f1798c00700 1 -- 172.21.15.131:0/3794956004 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56455af8d810 con 0x56455afd7e40 2024-05-08T22:13:24.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.478+0000 7f1798c00700 10 client.16203 handle_command_reply: tid=0 2024-05-08T22:13:24.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.478+0000 7f17c97d5b80 1 client.16203 shutdown 2024-05-08T22:13:24.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.490+0000 7f17c97d5b80 1 -- 172.21.15.131:0/3794956004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56455afc4720 msgr2=0x56455afc8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:13:24.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.490+0000 7f17c97d5b80 1 --2- 172.21.15.131:0/3794956004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56455afc4720 0x56455afc8720 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f17a8000f80 tx=0x7f17a800df70 comp rx=0 tx=0).stop 2024-05-08T22:13:24.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.491+0000 7f17c97d5b80 1 -- 172.21.15.131:0/3794956004 shutdown_connections 2024-05-08T22:13:24.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.491+0000 7f17c97d5b80 1 --2- 172.21.15.131:0/3794956004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56455afc4720 0x56455afc8720 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:24.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.491+0000 7f17c97d5b80 1 --2- 172.21.15.131:0/3794956004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56455afbe890 0x56455afc7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:24.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.491+0000 7f17c97d5b80 1 --2- 172.21.15.131:0/3794956004 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x56455afd7e40 0x56455afda260 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:24.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.491+0000 7f17c97d5b80 1 --2- 172.21.15.131:0/3794956004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56455afc3e00 0x56455afc81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:24.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.491+0000 7f17c97d5b80 1 -- 172.21.15.131:0/3794956004 >> 172.21.15.131:0/3794956004 conn(0x56455af1c210 msgr2=0x56455afb85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:13:24.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.491+0000 7f1798c00700 0 client.16203 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:13:24.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.491+0000 7f17c97d5b80 1 -- 172.21.15.131:0/3794956004 shutdown_connections 2024-05-08T22:13:24.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.503+0000 7f17c97d5b80 1 -- 172.21.15.131:0/3794956004 wait complete. 2024-05-08T22:13:24.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:24.505+0000 7f17c97d5b80 20 client.16203 trim_cache size 0 max 16384 2024-05-08T22:13:24.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:24 smithi131 ceph-mon[37260]: pgmap v186: 129 pgs: 129 active+clean; 920 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 6.6 MiB/s wr, 613 op/s 2024-05-08T22:13:24.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:24 smithi157 ceph-mon[77746]: pgmap v186: 129 pgs: 129 active+clean; 920 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 6.6 MiB/s wr, 613 op/s 2024-05-08T22:13:24.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:24 smithi132 ceph-mon[78180]: pgmap v186: 129 pgs: 129 active+clean; 920 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 6.6 MiB/s wr, 613 op/s 2024-05-08T22:13:25.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.115+0000 7f17c97d5b80 1 Processor -- start 2024-05-08T22:13:25.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.116+0000 7f17c97d5b80 1 -- start start 2024-05-08T22:13:25.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.116+0000 7f17c97d5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56455afbe890 0x56455afbe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:13:25.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.116+0000 7f17c97d5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56455afc3e00 0x56455afc3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:13:25.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.116+0000 7f17c97d5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56455afc4720 0x56455afb8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:13:25.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.116+0000 7f17c97d5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455afb3800 con 0x56455afc4720 2024-05-08T22:13:25.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.116+0000 7f17c97d5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455afd4590 con 0x56455afc3e00 2024-05-08T22:13:25.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.116+0000 7f17c97d5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455afc61f0 con 0x56455afbe890 2024-05-08T22:13:25.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.117+0000 7f17ae000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56455afc3e00 0x56455afc3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:13:25.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.117+0000 7f17ae000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56455afc3e00 0x56455afc3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:44074/0 (socket says 172.21.15.131:44074) 2024-05-08T22:13:25.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.117+0000 7f17ae000700 1 -- 172.21.15.131:0/922832926 learned_addr learned my addr 172.21.15.131:0/922832926 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:13:25.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.117+0000 7f17af400700 1 --2- 172.21.15.131:0/922832926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56455afc4720 0x56455afb8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:13:25.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.118+0000 7f17af400700 1 -- 172.21.15.131:0/922832926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56455afbe890 msgr2=0x56455afbe350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:13:25.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.118+0000 7f17af400700 1 --2- 172.21.15.131:0/922832926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56455afbe890 0x56455afbe350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:25.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.118+0000 7f17af400700 1 -- 172.21.15.131:0/922832926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56455afc3e00 msgr2=0x56455afc3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:13:25.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.118+0000 7f17af400700 1 --2- 172.21.15.131:0/922832926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56455afc3e00 0x56455afc3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:25.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.118+0000 7f17af400700 1 -- 172.21.15.131:0/922832926 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56455aed4560 con 0x56455afc4720 2024-05-08T22:13:25.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.118+0000 7f17af400700 1 --2- 172.21.15.131:0/922832926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56455afc4720 0x56455afb8d20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f17a80050e0 tx=0x7f17a8021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459d607d27df2923 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:13:25.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.128+0000 7f17a4c00700 1 -- 172.21.15.131:0/922832926 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a8007690 con 0x56455afc4720 2024-05-08T22:13:25.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.128+0000 7f17a4c00700 1 -- 172.21.15.131:0/922832926 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17a800b030 con 0x56455afc4720 2024-05-08T22:13:25.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.128+0000 7f17a4c00700 1 -- 172.21.15.131:0/922832926 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a800f560 con 0x56455afc4720 2024-05-08T22:13:25.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.134+0000 7f17c97d5b80 1 -- 172.21.15.131:0/922832926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56455afc4720 msgr2=0x56455afb8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:13:25.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.134+0000 7f17c97d5b80 1 --2- 172.21.15.131:0/922832926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56455afc4720 0x56455afb8d20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f17a80050e0 tx=0x7f17a8021000 comp rx=0 tx=0).stop 2024-05-08T22:13:25.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.152+0000 7f17c97d5b80 1 -- 172.21.15.131:0/922832926 shutdown_connections 2024-05-08T22:13:25.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.152+0000 7f17c97d5b80 1 --2- 172.21.15.131:0/922832926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56455afc4720 0x56455afb8d20 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:25.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.152+0000 7f17c97d5b80 1 --2- 172.21.15.131:0/922832926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56455afbe890 0x56455afbe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:25.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.152+0000 7f17c97d5b80 1 --2- 172.21.15.131:0/922832926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56455afc3e00 0x56455afc3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:25.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.152+0000 7f17c97d5b80 1 -- 172.21.15.131:0/922832926 >> 172.21.15.131:0/922832926 conn(0x56455af1c210 msgr2=0x56455afc2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:13:25.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.162+0000 7f17c97d5b80 1 -- 172.21.15.131:0/922832926 shutdown_connections 2024-05-08T22:13:25.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.162+0000 7f17c97d5b80 1 -- 172.21.15.131:0/922832926 wait complete. 2024-05-08T22:13:25.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.162+0000 7f17c97d5b80 1 -- start start 2024-05-08T22:13:25.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.163+0000 7f17c97d5b80 1 Processor -- start 2024-05-08T22:13:25.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.165+0000 7f17c97d5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56455afbe890 0x56455afcf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:13:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.168+0000 7f17aea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56455afbe890 0x56455afcf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:13:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.168+0000 7f17aea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56455afbe890 0x56455afcf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:44088/0 (socket says 172.21.15.131:44088) 2024-05-08T22:13:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.172+0000 7f17c97d5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56455afc3e00 0x56455afcd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:13:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.172+0000 7f17c97d5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56455afc4720 0x56455afcdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:13:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.172+0000 7f17c97d5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455afce180 con 0x56455afc4720 2024-05-08T22:13:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.172+0000 7f17c97d5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455afce2f0 con 0x56455afbe890 2024-05-08T22:13:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.172+0000 7f17c97d5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455afce4b0 con 0x56455afc3e00 2024-05-08T22:13:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.172+0000 7f17c97d5b80 10 client.0 _ll_register_callbacks cb 0x56455aeb3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:13:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.174+0000 7f17aea00700 1 -- 172.21.15.131:0/1726449612 learned_addr learned my addr 172.21.15.131:0/1726449612 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:13:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.174+0000 7f17ae000700 1 --2- 172.21.15.131:0/1726449612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56455afc3e00 0x56455afcd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:13:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.175+0000 7f17af400700 1 --2- 172.21.15.131:0/1726449612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56455afc4720 0x56455afcdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:13:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.175+0000 7f17ae000700 1 -- 172.21.15.131:0/1726449612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56455afbe890 msgr2=0x56455afcf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:13:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.175+0000 7f17ae000700 1 --2- 172.21.15.131:0/1726449612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56455afbe890 0x56455afcf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.175+0000 7f17ae000700 1 -- 172.21.15.131:0/1726449612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56455afc4720 msgr2=0x56455afcdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:13:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.175+0000 7f17ae000700 1 --2- 172.21.15.131:0/1726449612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56455afc4720 0x56455afcdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.175+0000 7f17ae000700 1 -- 172.21.15.131:0/1726449612 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56455ad3bb60 con 0x56455afc3e00 2024-05-08T22:13:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.175+0000 7f17af400700 1 --2- 172.21.15.131:0/1726449612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56455afc4720 0x56455afcdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:13:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.176+0000 7f17ae000700 1 --2- 172.21.15.131:0/1726449612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56455afc3e00 0x56455afcd700 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f179c09aa10 tx=0x7f179c0992c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=baeb10c2d1896e4a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:13:25.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.183+0000 7f1798c00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:13:25.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.183+0000 7f1798c00700 1 -- 172.21.15.131:0/1726449612 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c09b410 con 0x56455afc3e00 2024-05-08T22:13:25.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.183+0000 7f1798c00700 1 -- 172.21.15.131:0/1726449612 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f179c09ad10 con 0x56455afc3e00 2024-05-08T22:13:25.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.194+0000 7f17c97d5b80 1 -- 172.21.15.131:0/1726449612 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17a8004bb0 con 0x56455afc3e00 2024-05-08T22:13:25.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.194+0000 7f1798c00700 1 -- 172.21.15.131:0/1726449612 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f179c09eb40 con 0x56455afc3e00 2024-05-08T22:13:25.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.194+0000 7f17c97d5b80 10 client.26290 fetch_fsmap learned FSMap version 26 2024-05-08T22:13:25.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.194+0000 7f17c97d5b80 1 -- 172.21.15.131:0/1726449612 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56455ad93cd0 con 0x56455afc3e00 2024-05-08T22:13:25.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.194+0000 7f1798c00700 1 -- 172.21.15.131:0/1726449612 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17a8004bb0 con 0x56455afc3e00 2024-05-08T22:13:25.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.195+0000 7f1798c00700 1 -- 172.21.15.131:0/1726449612 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f179c09bd20 con 0x56455afc3e00 2024-05-08T22:13:25.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.195+0000 7f17c97d5b80 10 client.26290 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:13:25.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.195+0000 7f17c97d5b80 10 client.26290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:13:25.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.195+0000 7f17c97d5b80 20 client.26290 populate_metadata read hostname 'smithi131' 2024-05-08T22:13:25.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.195+0000 7f17c97d5b80 1 --2- 172.21.15.131:0/1726449612 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x56455b02f410 0x56455b031830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:13:25.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.196+0000 7f17c97d5b80 4 client.26290 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:13:25.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.196+0000 7f17c97d5b80 1 -- 172.21.15.131:0/1726449612 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56455af06ec0 con 0x56455b02f410 2024-05-08T22:13:25.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.196+0000 7f17aea00700 1 --2- 172.21.15.131:0/1726449612 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x56455b02f410 0x56455b031830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:13:25.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.197+0000 7f17aea00700 1 --2- 172.21.15.131:0/1726449612 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x56455b02f410 0x56455b031830 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=416a7bfa663a0df6 server_cookie=987b026385ca0336 in_seq=0 out_seq=0 2024-05-08T22:13:25.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.197+0000 7f1798c00700 10 client.26290 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:13:25.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.200+0000 7f1798c00700 1 -- 172.21.15.131:0/1726449612 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x56455af06ec0 con 0x56455b02f410 2024-05-08T22:13:25.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.200+0000 7f1798c00700 10 client.26290 handle_command_reply: tid=0 2024-05-08T22:13:25.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.200+0000 7f17c97d5b80 1 client.26290 shutdown 2024-05-08T22:13:25.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.201+0000 7f17c97d5b80 1 -- 172.21.15.131:0/1726449612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56455afc3e00 msgr2=0x56455afcd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:13:25.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.201+0000 7f17c97d5b80 1 --2- 172.21.15.131:0/1726449612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56455afc3e00 0x56455afcd700 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f179c09aa10 tx=0x7f179c0992c0 comp rx=0 tx=0).stop 2024-05-08T22:13:25.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.201+0000 7f17c97d5b80 1 -- 172.21.15.131:0/1726449612 shutdown_connections 2024-05-08T22:13:25.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.201+0000 7f17c97d5b80 1 --2- 172.21.15.131:0/1726449612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56455afc4720 0x56455afcdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:25.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.201+0000 7f17c97d5b80 1 --2- 172.21.15.131:0/1726449612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56455afbe890 0x56455afcf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:25.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.201+0000 7f17c97d5b80 1 --2- 172.21.15.131:0/1726449612 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x56455b02f410 0x56455b031830 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:25.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.201+0000 7f17c97d5b80 1 --2- 172.21.15.131:0/1726449612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56455afc3e00 0x56455afcd700 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:25.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.201+0000 7f17c97d5b80 1 -- 172.21.15.131:0/1726449612 >> 172.21.15.131:0/1726449612 conn(0x56455af1c210 msgr2=0x56455afc4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:13:25.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.201+0000 7f1798c00700 0 client.26290 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:13:25.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.204+0000 7f17c97d5b80 1 -- 172.21.15.131:0/1726449612 shutdown_connections 2024-05-08T22:13:25.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.205+0000 7f17c97d5b80 1 -- 172.21.15.131:0/1726449612 wait complete. 2024-05-08T22:13:25.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.207+0000 7f17c97d5b80 20 client.26290 trim_cache size 0 max 16384 2024-05-08T22:13:25.210 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:13:25.210 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "scrub active (8540 inodes in the stack)", 2024-05-08T22:13:25.210 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-08T22:13:25.210 INFO:teuthology.orchestra.run.smithi131.stdout: "ca36159f-7052-4d2f-921f-ee97cd0c21fb": { 2024-05-08T22:13:25.210 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-08T22:13:25.210 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "ca36159f-7052-4d2f-921f-ee97cd0c21fb", 2024-05-08T22:13:25.210 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-08T22:13:25.210 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:13:25.210 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:13:25.211 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:13:25.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.245+0000 7f17b5c00700 1 -- 172.21.15.131:0/2546249223 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f17940a14a0 msgr2=0x7f17940a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:13:25.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.245+0000 7f17b5c00700 1 --2- 172.21.15.131:0/2546249223 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f17940a14a0 0x7f17940a3960 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f17a000f4d0 tx=0x7f17a0005f90 comp rx=0 tx=0).stop 2024-05-08T22:13:25.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.245+0000 7f17b5c00700 1 -- 172.21.15.131:0/2546249223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17b010fb10 msgr2=0x7f17b0114430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:13:25.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.245+0000 7f17b5c00700 1 --2- 172.21.15.131:0/2546249223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17b010fb10 0x7f17b0114430 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f179c006010 tx=0x7f179c003b00 comp rx=0 tx=0).stop 2024-05-08T22:13:25.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.245+0000 7f17b5c00700 1 -- 172.21.15.131:0/2546249223 shutdown_connections 2024-05-08T22:13:25.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.245+0000 7f17b5c00700 1 --2- 172.21.15.131:0/2546249223 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f17940a14a0 0x7f17940a3960 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f17a000f4d0 tx=0x7f17a0005f90 comp rx=0 tx=0).stop 2024-05-08T22:13:25.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.245+0000 7f17b5c00700 1 --2- 172.21.15.131:0/2546249223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17b011a780 0x7f17b0119770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:25.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.245+0000 7f17b5c00700 1 --2- 172.21.15.131:0/2546249223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b010f6a0 0x7f17b0113ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:25.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.245+0000 7f17b5c00700 1 --2- 172.21.15.131:0/2546249223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17b010fb10 0x7f17b0114430 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:13:25.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.245+0000 7f17b5c00700 1 -- 172.21.15.131:0/2546249223 >> 172.21.15.131:0/2546249223 conn(0x7f17b006bff0 msgr2=0x7f17b011ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:13:25.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.249+0000 7f17b5c00700 1 -- 172.21.15.131:0/2546249223 shutdown_connections 2024-05-08T22:13:25.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:13:25.249+0000 7f17b5c00700 1 -- 172.21.15.131:0/2546249223 wait complete. 2024-05-08T22:13:25.337 INFO:tasks.workunit.client.0.smithi131.stdout: 177 447461 562 296427 372 255167 1140 2024-05-08T22:13:25.806 INFO:tasks.cephfs.filesystem:scrub status for tag:ca36159f-7052-4d2f-921f-ee97cd0c21fb - {'path': '/', 'tag': 'ca36159f-7052-4d2f-921f-ee97cd0c21fb', 'options': 'recursive,force'} 2024-05-08T22:13:26.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:26 smithi131 ceph-mon[37260]: pgmap v187: 129 pgs: 129 active+clean; 917 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.8 MiB/s wr, 786 op/s 2024-05-08T22:13:26.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:26 smithi157 ceph-mon[77746]: pgmap v187: 129 pgs: 129 active+clean; 917 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.8 MiB/s wr, 786 op/s 2024-05-08T22:13:26.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:26 smithi132 ceph-mon[78180]: pgmap v187: 129 pgs: 129 active+clean; 917 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.8 MiB/s wr, 786 op/s 2024-05-08T22:13:28.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:28 smithi131 ceph-mon[37260]: pgmap v188: 129 pgs: 129 active+clean; 909 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 597 op/s 2024-05-08T22:13:28.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:28 smithi157 ceph-mon[77746]: pgmap v188: 129 pgs: 129 active+clean; 909 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 597 op/s 2024-05-08T22:13:28.846 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:28 smithi132 ceph-mon[78180]: pgmap v188: 129 pgs: 129 active+clean; 909 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 597 op/s 2024-05-08T22:13:30.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:30 smithi131 ceph-mon[37260]: pgmap v189: 129 pgs: 129 active+clean; 912 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.8 MiB/s wr, 739 op/s 2024-05-08T22:13:30.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:30 smithi157 ceph-mon[77746]: pgmap v189: 129 pgs: 129 active+clean; 912 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.8 MiB/s wr, 739 op/s 2024-05-08T22:13:30.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:30 smithi132 ceph-mon[78180]: pgmap v189: 129 pgs: 129 active+clean; 912 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.8 MiB/s wr, 739 op/s 2024-05-08T22:13:32.647 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:32 smithi131 ceph-mon[37260]: pgmap v190: 129 pgs: 129 active+clean; 923 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 665 op/s 2024-05-08T22:13:32.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:32 smithi157 ceph-mon[77746]: pgmap v190: 129 pgs: 129 active+clean; 923 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 665 op/s 2024-05-08T22:13:32.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:32 smithi132 ceph-mon[78180]: pgmap v190: 129 pgs: 129 active+clean; 923 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 665 op/s 2024-05-08T22:13:34.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:34 smithi157 ceph-mon[77746]: pgmap v191: 129 pgs: 129 active+clean; 923 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 525 op/s 2024-05-08T22:13:34.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:34 smithi132 ceph-mon[78180]: pgmap v191: 129 pgs: 129 active+clean; 923 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 525 op/s 2024-05-08T22:13:34.894 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:34 smithi131 ceph-mon[37260]: pgmap v191: 129 pgs: 129 active+clean; 923 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 525 op/s 2024-05-08T22:13:35.335 INFO:tasks.workunit.client.0.smithi131.stdout: 182 430496 455 281703 234 241274 1002 2024-05-08T22:13:36.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:36 smithi157 ceph-mon[77746]: pgmap v192: 129 pgs: 129 active+clean; 942 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.9 MiB/s wr, 740 op/s 2024-05-08T22:13:36.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:36 smithi132 ceph-mon[78180]: pgmap v192: 129 pgs: 129 active+clean; 942 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.9 MiB/s wr, 740 op/s 2024-05-08T22:13:36.897 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:36 smithi131 ceph-mon[37260]: pgmap v192: 129 pgs: 129 active+clean; 942 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.9 MiB/s wr, 740 op/s 2024-05-08T22:13:38.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:38 smithi157 ceph-mon[77746]: pgmap v193: 129 pgs: 129 active+clean; 957 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.0 MiB/s wr, 692 op/s 2024-05-08T22:13:38.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:38 smithi132 ceph-mon[78180]: pgmap v193: 129 pgs: 129 active+clean; 957 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.0 MiB/s wr, 692 op/s 2024-05-08T22:13:38.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:38 smithi131 ceph-mon[37260]: pgmap v193: 129 pgs: 129 active+clean; 957 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.0 MiB/s wr, 692 op/s 2024-05-08T22:13:40.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:40 smithi157 ceph-mon[77746]: pgmap v194: 129 pgs: 129 active+clean; 974 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.6 MiB/s wr, 844 op/s 2024-05-08T22:13:40.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:40 smithi132 ceph-mon[78180]: pgmap v194: 129 pgs: 129 active+clean; 974 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.6 MiB/s wr, 844 op/s 2024-05-08T22:13:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:40 smithi131 ceph-mon[37260]: pgmap v194: 129 pgs: 129 active+clean; 974 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.6 MiB/s wr, 844 op/s 2024-05-08T22:13:42.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:42 smithi157 ceph-mon[77746]: pgmap v195: 129 pgs: 129 active+clean; 977 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.9 MiB/s wr, 732 op/s 2024-05-08T22:13:42.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:42 smithi132 ceph-mon[78180]: pgmap v195: 129 pgs: 129 active+clean; 977 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.9 MiB/s wr, 732 op/s 2024-05-08T22:13:42.897 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:42 smithi131 ceph-mon[37260]: pgmap v195: 129 pgs: 129 active+clean; 977 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.9 MiB/s wr, 732 op/s 2024-05-08T22:13:44.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:44 smithi157 ceph-mon[77746]: pgmap v196: 129 pgs: 129 active+clean; 977 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.9 MiB/s wr, 602 op/s 2024-05-08T22:13:44.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:44 smithi132 ceph-mon[78180]: pgmap v196: 129 pgs: 129 active+clean; 977 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.9 MiB/s wr, 602 op/s 2024-05-08T22:13:44.894 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:44 smithi131 ceph-mon[37260]: pgmap v196: 129 pgs: 129 active+clean; 977 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.9 MiB/s wr, 602 op/s 2024-05-08T22:13:45.342 INFO:tasks.workunit.client.0.smithi131.stdout: 188 494778 477 320494 361 273516 1270 2024-05-08T22:13:46.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:46 smithi157 ceph-mon[77746]: pgmap v197: 129 pgs: 129 active+clean; 982 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.8 MiB/s wr, 756 op/s 2024-05-08T22:13:46.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:46 smithi132 ceph-mon[78180]: pgmap v197: 129 pgs: 129 active+clean; 982 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.8 MiB/s wr, 756 op/s 2024-05-08T22:13:46.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:46 smithi131 ceph-mon[37260]: pgmap v197: 129 pgs: 129 active+clean; 982 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.8 MiB/s wr, 756 op/s 2024-05-08T22:13:48.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:48 smithi157 ceph-mon[77746]: pgmap v198: 129 pgs: 129 active+clean; 984 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.3 MiB/s wr, 609 op/s 2024-05-08T22:13:48.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:48 smithi132 ceph-mon[78180]: pgmap v198: 129 pgs: 129 active+clean; 984 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.3 MiB/s wr, 609 op/s 2024-05-08T22:13:48.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:48 smithi131 ceph-mon[37260]: pgmap v198: 129 pgs: 129 active+clean; 984 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.3 MiB/s wr, 609 op/s 2024-05-08T22:13:50.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:50 smithi132 ceph-mon[78180]: pgmap v199: 129 pgs: 129 active+clean; 994 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.9 MiB/s wr, 593 op/s 2024-05-08T22:13:50.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:50 smithi131 ceph-mon[37260]: pgmap v199: 129 pgs: 129 active+clean; 994 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.9 MiB/s wr, 593 op/s 2024-05-08T22:13:51.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:50 smithi157 ceph-mon[77746]: pgmap v199: 129 pgs: 129 active+clean; 994 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.9 MiB/s wr, 593 op/s 2024-05-08T22:13:52.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:52 smithi132 ceph-mon[78180]: pgmap v200: 129 pgs: 129 active+clean; 994 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 432 op/s 2024-05-08T22:13:52.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:52 smithi131 ceph-mon[37260]: pgmap v200: 129 pgs: 129 active+clean; 994 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 432 op/s 2024-05-08T22:13:53.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:52 smithi157 ceph-mon[77746]: pgmap v200: 129 pgs: 129 active+clean; 994 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 432 op/s 2024-05-08T22:13:54.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:54 smithi132 ceph-mon[78180]: pgmap v201: 129 pgs: 129 active+clean; 998 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 439 op/s 2024-05-08T22:13:54.897 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:54 smithi131 ceph-mon[37260]: pgmap v201: 129 pgs: 129 active+clean; 998 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 439 op/s 2024-05-08T22:13:55.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:54 smithi157 ceph-mon[77746]: pgmap v201: 129 pgs: 129 active+clean; 998 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 439 op/s 2024-05-08T22:13:55.345 INFO:tasks.workunit.client.0.smithi131.stdout: 195 449921 422 293941 317 252447 1186 2024-05-08T22:13:55.808 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:13:56.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:56 smithi132 ceph-mon[78180]: pgmap v202: 129 pgs: 129 active+clean; 1023 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.1 MiB/s wr, 787 op/s 2024-05-08T22:13:56.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:56 smithi131 ceph-mon[37260]: pgmap v202: 129 pgs: 129 active+clean; 1023 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.1 MiB/s wr, 787 op/s 2024-05-08T22:13:57.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:56 smithi157 ceph-mon[77746]: pgmap v202: 129 pgs: 129 active+clean; 1023 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.1 MiB/s wr, 787 op/s 2024-05-08T22:13:58.895 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:13:58 smithi131 ceph-mon[37260]: pgmap v203: 129 pgs: 129 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 723 op/s 2024-05-08T22:13:59.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:13:58 smithi157 ceph-mon[77746]: pgmap v203: 129 pgs: 129 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 723 op/s 2024-05-08T22:13:59.092 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:13:58 smithi132 ceph-mon[78180]: pgmap v203: 129 pgs: 129 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 723 op/s 2024-05-08T22:14:00.938 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:00 smithi131 ceph-mon[37260]: pgmap v204: 129 pgs: 129 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.3 MiB/s wr, 809 op/s 2024-05-08T22:14:01.025 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:00 smithi157 ceph-mon[77746]: pgmap v204: 129 pgs: 129 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.3 MiB/s wr, 809 op/s 2024-05-08T22:14:01.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:00 smithi132 ceph-mon[78180]: pgmap v204: 129 pgs: 129 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.3 MiB/s wr, 809 op/s 2024-05-08T22:14:02.894 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:02 smithi131 ceph-mon[37260]: pgmap v205: 129 pgs: 129 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.4 MiB/s wr, 717 op/s 2024-05-08T22:14:03.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:02 smithi157 ceph-mon[77746]: pgmap v205: 129 pgs: 129 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.4 MiB/s wr, 717 op/s 2024-05-08T22:14:03.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:02 smithi132 ceph-mon[78180]: pgmap v205: 129 pgs: 129 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.4 MiB/s wr, 717 op/s 2024-05-08T22:14:03.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.662+0000 7f89d3200700 1 -- 172.21.15.131:0/1789922541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89d41082c0 msgr2=0x7f89d4108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:03.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.662+0000 7f89d3200700 1 --2- 172.21.15.131:0/1789922541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89d41082c0 0x7f89d4108740 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f89c001a9a0 tx=0x7f89c001acb0 comp rx=0 tx=0).stop 2024-05-08T22:14:03.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.696+0000 7f89d3200700 1 -- 172.21.15.131:0/1789922541 shutdown_connections 2024-05-08T22:14:03.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.696+0000 7f89d3200700 1 --2- 172.21.15.131:0/1789922541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89d411a560 0x7f89d411c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:03.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.696+0000 7f89d3200700 1 --2- 172.21.15.131:0/1789922541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89d410f570 0x7f89d4107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:03.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.696+0000 7f89d3200700 1 --2- 172.21.15.131:0/1789922541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89d41082c0 0x7f89d4108740 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:03.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.696+0000 7f89d3200700 1 -- 172.21.15.131:0/1789922541 >> 172.21.15.131:0/1789922541 conn(0x7f89d406bee0 msgr2=0x7f89d406c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:03.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.696+0000 7f89d3200700 1 -- 172.21.15.131:0/1789922541 shutdown_connections 2024-05-08T22:14:03.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.705+0000 7f89d3200700 1 -- 172.21.15.131:0/1789922541 wait complete. 2024-05-08T22:14:03.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.710+0000 7f89d3200700 1 Processor -- start 2024-05-08T22:14:03.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.729+0000 7f89d3200700 1 -- start start 2024-05-08T22:14:03.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.730+0000 7f89d3200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89d41082c0 0x7f89d41a3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:03.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.730+0000 7f89d3200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89d410f570 0x7f89d41a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:03.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.731+0000 7f89d1400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89d410f570 0x7f89d41a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:03.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.732+0000 7f89d3200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89d411a560 0x7f89d41159f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:03.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.732+0000 7f89d1400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89d410f570 0x7f89d41a3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53338/0 (socket says 172.21.15.131:53338) 2024-05-08T22:14:03.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.733+0000 7f89d3200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d4116080 con 0x7f89d410f570 2024-05-08T22:14:03.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.733+0000 7f89d3200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d41161f0 con 0x7f89d41082c0 2024-05-08T22:14:03.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.734+0000 7f89d2800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89d411a560 0x7f89d41159f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:03.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.734+0000 7f89d2800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89d411a560 0x7f89d41159f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37294/0 (socket says 172.21.15.131:37294) 2024-05-08T22:14:03.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.734+0000 7f89d2800700 1 -- 172.21.15.131:0/4262323462 learned_addr learned my addr 172.21.15.131:0/4262323462 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:03.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.733+0000 7f89d3200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d41163b0 con 0x7f89d411a560 2024-05-08T22:14:03.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.742+0000 7f89d1400700 1 -- 172.21.15.131:0/4262323462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89d411a560 msgr2=0x7f89d41159f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:03.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.742+0000 7f89d1400700 1 --2- 172.21.15.131:0/4262323462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89d411a560 0x7f89d41159f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:03.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.742+0000 7f89d1400700 1 -- 172.21.15.131:0/4262323462 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89d41082c0 msgr2=0x7f89d41a3580 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T22:14:03.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.742+0000 7f89d1400700 1 --2- 172.21.15.131:0/4262323462 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89d41082c0 0x7f89d41a3580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:03.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.742+0000 7f89d1400700 1 -- 172.21.15.131:0/4262323462 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c001a600 con 0x7f89d410f570 2024-05-08T22:14:03.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.759+0000 7f89d1400700 1 --2- 172.21.15.131:0/4262323462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89d410f570 0x7f89d41a3ac0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f89c0000c00 tx=0x7f89c001a590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:03.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.771+0000 7f89caa00700 1 -- 172.21.15.131:0/4262323462 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c002d070 con 0x7f89d410f570 2024-05-08T22:14:03.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.771+0000 7f89caa00700 1 -- 172.21.15.131:0/4262323462 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89c0003680 con 0x7f89d410f570 2024-05-08T22:14:03.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.771+0000 7f89caa00700 1 -- 172.21.15.131:0/4262323462 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c0031be0 con 0x7f89d410f570 2024-05-08T22:14:03.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.794+0000 7f89d3200700 1 -- 172.21.15.131:0/4262323462 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d4116630 con 0x7f89d410f570 2024-05-08T22:14:03.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.794+0000 7f89d3200700 1 -- 172.21.15.131:0/4262323462 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d41aa600 con 0x7f89d410f570 2024-05-08T22:14:03.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.858+0000 7f89caa00700 1 -- 172.21.15.131:0/4262323462 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f89c003a3f0 con 0x7f89d410f570 2024-05-08T22:14:03.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.890+0000 7f89caa00700 1 --2- 172.21.15.131:0/4262323462 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f89b80a14a0 0x7f89b80a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:03.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.898+0000 7f89caa00700 1 -- 172.21.15.131:0/4262323462 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f89c00ded10 con 0x7f89d410f570 2024-05-08T22:14:03.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.900+0000 7f89d1e00700 1 --2- 172.21.15.131:0/4262323462 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f89b80a14a0 0x7f89b80a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:03.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.905+0000 7f89d1e00700 1 --2- 172.21.15.131:0/4262323462 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f89b80a14a0 0x7f89b80a3960 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f89c400ba80 tx=0x7f89c400b3f0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:03.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.923+0000 7f89ec480b80 1 Processor -- start 2024-05-08T22:14:03.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.926+0000 7f89ec480b80 1 -- start start 2024-05-08T22:14:03.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.927+0000 7f89ec480b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561ecbf48050 0x561ecbf48430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:03.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.927+0000 7f89ec480b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561ecbf48970 0x561ecbf42350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:03.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.927+0000 7f89ec480b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561ecbf42890 0x561ecbf42cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:03.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.928+0000 7f89d1e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561ecbf48050 0x561ecbf48430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:03.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.928+0000 7f89d1e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561ecbf48050 0x561ecbf48430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51738/0 (socket says 172.21.15.131:51738) 2024-05-08T22:14:03.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.930+0000 7f89d1400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561ecbf48970 0x561ecbf42350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:03.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.930+0000 7f89d1400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561ecbf48970 0x561ecbf42350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:49240/0 (socket says 172.21.15.131:49240) 2024-05-08T22:14:03.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.931+0000 7f89d2800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561ecbf42890 0x561ecbf42cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:03.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.932+0000 7f89d2800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561ecbf42890 0x561ecbf42cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51712/0 (socket says 172.21.15.131:51712) 2024-05-08T22:14:03.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.935+0000 7f89ec480b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ecbe99150 con 0x561ecbf48050 2024-05-08T22:14:03.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.936+0000 7f89ec480b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ecbe96b10 con 0x561ecbf48970 2024-05-08T22:14:03.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.940+0000 7f89ec480b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ecbe92400 con 0x561ecbf42890 2024-05-08T22:14:03.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.950+0000 7f89d1e00700 1 -- 172.21.15.131:0/2492627839 learned_addr learned my addr 172.21.15.131:0/2492627839 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:03.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.950+0000 7f89d1e00700 1 -- 172.21.15.131:0/2492627839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561ecbf42890 msgr2=0x561ecbf42cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:03.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.950+0000 7f89d1e00700 1 --2- 172.21.15.131:0/2492627839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561ecbf42890 0x561ecbf42cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:03.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.950+0000 7f89d1e00700 1 -- 172.21.15.131:0/2492627839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561ecbf48970 msgr2=0x561ecbf42350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:03.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.950+0000 7f89d1e00700 1 --2- 172.21.15.131:0/2492627839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561ecbf48970 0x561ecbf42350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:03.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.950+0000 7f89d1e00700 1 -- 172.21.15.131:0/2492627839 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ecbd133b0 con 0x561ecbf48050 2024-05-08T22:14:03.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.951+0000 7f89d1e00700 1 --2- 172.21.15.131:0/2492627839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561ecbf48050 0x561ecbf48430 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f89c400dc20 tx=0x7f89c4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e132543a59a9bfe5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:03.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.972+0000 7f89bf000700 1 -- 172.21.15.131:0/2492627839 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c4021070 con 0x561ecbf48050 2024-05-08T22:14:03.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.972+0000 7f89bf000700 1 -- 172.21.15.131:0/2492627839 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89c4010db0 con 0x561ecbf48050 2024-05-08T22:14:03.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.972+0000 7f89bf000700 1 -- 172.21.15.131:0/2492627839 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c4005260 con 0x561ecbf48050 2024-05-08T22:14:03.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.981+0000 7f89d1400700 1 --2- 172.21.15.131:0/2492627839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561ecbf48970 0x561ecbf42350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-08T22:14:03.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.982+0000 7f89d2800700 1 --2- 172.21.15.131:0/2492627839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561ecbf42890 0x561ecbf42cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-08T22:14:03.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.995+0000 7f89ec480b80 1 -- 172.21.15.131:0/2492627839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561ecbf48050 msgr2=0x561ecbf48430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:03.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:03.995+0000 7f89ec480b80 1 --2- 172.21.15.131:0/2492627839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561ecbf48050 0x561ecbf48430 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f89c400dc20 tx=0x7f89c4010000 comp rx=0 tx=0).stop 2024-05-08T22:14:04.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.035+0000 7f89ec480b80 1 -- 172.21.15.131:0/2492627839 shutdown_connections 2024-05-08T22:14:04.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.035+0000 7f89ec480b80 1 --2- 172.21.15.131:0/2492627839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561ecbf42890 0x561ecbf42cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:04.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.035+0000 7f89ec480b80 1 --2- 172.21.15.131:0/2492627839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561ecbf48050 0x561ecbf48430 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:04.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.035+0000 7f89ec480b80 1 --2- 172.21.15.131:0/2492627839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561ecbf48970 0x561ecbf42350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:04.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.035+0000 7f89ec480b80 1 -- 172.21.15.131:0/2492627839 >> 172.21.15.131:0/2492627839 conn(0x561ecbea0210 msgr2=0x561ecbf3d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:04.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.058+0000 7f89ec480b80 1 -- 172.21.15.131:0/2492627839 shutdown_connections 2024-05-08T22:14:04.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.058+0000 7f89ec480b80 1 -- 172.21.15.131:0/2492627839 wait complete. 2024-05-08T22:14:04.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.060+0000 7f89ec480b80 1 -- start start 2024-05-08T22:14:04.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.061+0000 7f89ec480b80 1 Processor -- start 2024-05-08T22:14:04.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.063+0000 7f89ec480b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561ecbf42890 0x561ecbf4bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:04.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.081+0000 7f89d1e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561ecbf42890 0x561ecbf4bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:04.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.081+0000 7f89d1e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561ecbf42890 0x561ecbf4bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51718/0 (socket says 172.21.15.131:51718) 2024-05-08T22:14:04.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.083+0000 7f89d1e00700 1 -- 172.21.15.131:0/141357207 learned_addr learned my addr 172.21.15.131:0/141357207 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:04.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.083+0000 7f89ec480b80 1 --2- 172.21.15.131:0/141357207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561ecbf48050 0x561ecbf4c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:04.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.083+0000 7f89ec480b80 1 --2- 172.21.15.131:0/141357207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561ecbf48970 0x561ecbf4c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:04.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.084+0000 7f89ec480b80 1 -- 172.21.15.131:0/141357207 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ecbe2e560 con 0x561ecbf48970 2024-05-08T22:14:04.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.084+0000 7f89ec480b80 1 -- 172.21.15.131:0/141357207 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ecbe2e200 con 0x561ecbf48050 2024-05-08T22:14:04.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.084+0000 7f89ec480b80 1 -- 172.21.15.131:0/141357207 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ecbd156a0 con 0x561ecbf42890 2024-05-08T22:14:04.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.084+0000 7f89d1400700 1 --2- 172.21.15.131:0/141357207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561ecbf48050 0x561ecbf4c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:04.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.084+0000 7f89d2800700 1 --2- 172.21.15.131:0/141357207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561ecbf48970 0x561ecbf4c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:04.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.085+0000 7f89d1400700 1 -- 172.21.15.131:0/141357207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561ecbf42890 msgr2=0x561ecbf4bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:04.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.085+0000 7f89d1400700 1 --2- 172.21.15.131:0/141357207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561ecbf42890 0x561ecbf4bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:04.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.085+0000 7f89d1400700 1 -- 172.21.15.131:0/141357207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561ecbf48970 msgr2=0x561ecbf4c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:04.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.085+0000 7f89d1400700 1 --2- 172.21.15.131:0/141357207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561ecbf48970 0x561ecbf4c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:04.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.085+0000 7f89d1400700 1 -- 172.21.15.131:0/141357207 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c400d8b0 con 0x561ecbf48050 2024-05-08T22:14:04.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.085+0000 7f89d2800700 1 --2- 172.21.15.131:0/141357207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561ecbf48970 0x561ecbf4c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:14:04.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.088+0000 7f89ec480b80 10 client.0 _ll_register_callbacks cb 0x561ecbe37b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:14:04.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.088+0000 7f89ec480b80 1 -- 172.21.15.131:0/141357207 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ecbdeb270 con 0x561ecbf48050 2024-05-08T22:14:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.090+0000 7f89d1400700 1 --2- 172.21.15.131:0/141357207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561ecbf48050 0x561ecbf4c200 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f89c00dbfc0 tx=0x7f89c002d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54cf07b4f4cb36e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.092+0000 7f89b3400700 10 client.26243 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:14:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.094+0000 7f89b3400700 1 -- 172.21.15.131:0/141357207 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c00c4070 con 0x561ecbf48050 2024-05-08T22:14:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.094+0000 7f89b3400700 1 -- 172.21.15.131:0/141357207 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f89c00ac5d0 con 0x561ecbf48050 2024-05-08T22:14:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.094+0000 7f89b3400700 1 -- 172.21.15.131:0/141357207 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89c00b0400 con 0x561ecbf48050 2024-05-08T22:14:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.094+0000 7f89ec480b80 10 client.26243 fetch_fsmap learned FSMap version 26 2024-05-08T22:14:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.094+0000 7f89ec480b80 1 -- 172.21.15.131:0/141357207 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({fsmap=26}) v3 -- 0x561ecbcbfb60 con 0x561ecbf48050 2024-05-08T22:14:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.095+0000 7f89b3400700 1 -- 172.21.15.131:0/141357207 <== mon.2 v2:172.21.15.132:3300/0 4 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f89c00dc240 con 0x561ecbf48050 2024-05-08T22:14:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.095+0000 7f89ec480b80 10 client.26243 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:14:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.095+0000 7f89ec480b80 10 client.26243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:14:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.095+0000 7f89ec480b80 20 client.26243 populate_metadata read hostname 'smithi131' 2024-05-08T22:14:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.095+0000 7f89ec480b80 1 --2- 172.21.15.131:0/141357207 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x561ecbf5bc00 0x561ecbf5e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.095+0000 7f89ec480b80 4 client.26243 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:14:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.095+0000 7f89ec480b80 1 -- 172.21.15.131:0/141357207 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ecbf11810 con 0x561ecbf5bc00 2024-05-08T22:14:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.096+0000 7f89d1e00700 1 --2- 172.21.15.131:0/141357207 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x561ecbf5bc00 0x561ecbf5e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:04.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.096+0000 7f89d1e00700 1 --2- 172.21.15.131:0/141357207 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x561ecbf5bc00 0x561ecbf5e0c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7d1790b33815e1a server_cookie=f0ba4cdaeb32115e in_seq=0 out_seq=0 2024-05-08T22:14:04.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.097+0000 7f89b3400700 10 client.26243 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:14:04.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.099+0000 7f89b3400700 1 -- 172.21.15.131:0/141357207 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561ecbf11810 con 0x561ecbf5bc00 2024-05-08T22:14:04.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.099+0000 7f89b3400700 10 client.26243 handle_command_reply: tid=0 2024-05-08T22:14:04.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.099+0000 7f89ec480b80 1 client.26243 shutdown 2024-05-08T22:14:04.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.100+0000 7f89ec480b80 1 -- 172.21.15.131:0/141357207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561ecbf48050 msgr2=0x561ecbf4c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:04.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.100+0000 7f89ec480b80 1 --2- 172.21.15.131:0/141357207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561ecbf48050 0x561ecbf4c200 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f89c00dbfc0 tx=0x7f89c002d000 comp rx=0 tx=0).stop 2024-05-08T22:14:04.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.101+0000 7f89ec480b80 1 -- 172.21.15.131:0/141357207 shutdown_connections 2024-05-08T22:14:04.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.101+0000 7f89ec480b80 1 --2- 172.21.15.131:0/141357207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561ecbf48970 0x561ecbf4c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:04.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.101+0000 7f89ec480b80 1 --2- 172.21.15.131:0/141357207 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x561ecbf5bc00 0x561ecbf5e0c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:04.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.101+0000 7f89ec480b80 1 --2- 172.21.15.131:0/141357207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561ecbf42890 0x561ecbf4bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:04.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.101+0000 7f89ec480b80 1 --2- 172.21.15.131:0/141357207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561ecbf48050 0x561ecbf4c200 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:04.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.101+0000 7f89ec480b80 1 -- 172.21.15.131:0/141357207 >> 172.21.15.131:0/141357207 conn(0x561ecbea0210 msgr2=0x561ecbf3d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:04.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.101+0000 7f89b3400700 0 client.26243 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:14:04.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.101+0000 7f89ec480b80 1 -- 172.21.15.131:0/141357207 shutdown_connections 2024-05-08T22:14:04.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.101+0000 7f89ec480b80 1 -- 172.21.15.131:0/141357207 wait complete. 2024-05-08T22:14:04.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.102+0000 7f89ec480b80 20 client.26243 trim_cache size 0 max 16384 2024-05-08T22:14:04.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.725+0000 7f89ec480b80 1 Processor -- start 2024-05-08T22:14:04.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.725+0000 7f89ec480b80 1 -- start start 2024-05-08T22:14:04.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.726+0000 7f89ec480b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561ecbf42890 0x561ecbf42350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:04.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.727+0000 7f89d1e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561ecbf42890 0x561ecbf42350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:04.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.727+0000 7f89d1e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561ecbf42890 0x561ecbf42350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51758/0 (socket says 172.21.15.131:51758) 2024-05-08T22:14:04.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.727+0000 7f89d1e00700 1 -- 172.21.15.131:0/2494943071 learned_addr learned my addr 172.21.15.131:0/2494943071 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:04.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.728+0000 7f89ec480b80 1 --2- 172.21.15.131:0/2494943071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561ecbf48050 0x561ecbf4c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:04.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.731+0000 7f89d1400700 1 --2- 172.21.15.131:0/2494943071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561ecbf48050 0x561ecbf4c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:04.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.738+0000 7f89ec480b80 1 --2- 172.21.15.131:0/2494943071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561ecbf48970 0x561ecbf49460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:04.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.738+0000 7f89ec480b80 1 -- 172.21.15.131:0/2494943071 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ecbf3c410 con 0x561ecbf42890 2024-05-08T22:14:04.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.738+0000 7f89ec480b80 1 -- 172.21.15.131:0/2494943071 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ecbf11840 con 0x561ecbf48050 2024-05-08T22:14:04.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.738+0000 7f89ec480b80 1 -- 172.21.15.131:0/2494943071 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ecbf4d2c0 con 0x561ecbf48970 2024-05-08T22:14:04.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.738+0000 7f89d2800700 1 --2- 172.21.15.131:0/2494943071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561ecbf48970 0x561ecbf49460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:04.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.742+0000 7f89d2800700 1 -- 172.21.15.131:0/2494943071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561ecbf48050 msgr2=0x561ecbf4c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:04.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.742+0000 7f89d2800700 1 --2- 172.21.15.131:0/2494943071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561ecbf48050 0x561ecbf4c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:04.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.742+0000 7f89d2800700 1 -- 172.21.15.131:0/2494943071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561ecbf42890 msgr2=0x561ecbf42350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:04.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.742+0000 7f89d2800700 1 --2- 172.21.15.131:0/2494943071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561ecbf42890 0x561ecbf42350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:04.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.742+0000 7f89d2800700 1 -- 172.21.15.131:0/2494943071 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ecbf3bfe0 con 0x561ecbf48970 2024-05-08T22:14:04.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.742+0000 7f89d1400700 1 --2- 172.21.15.131:0/2494943071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561ecbf48050 0x561ecbf4c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:14:04.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.743+0000 7f89d2800700 1 --2- 172.21.15.131:0/2494943071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561ecbf48970 0x561ecbf49460 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f89cc009790 tx=0x7f89cc009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82489d88d19cb423 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:04.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.743+0000 7f89d1e00700 1 --2- 172.21.15.131:0/2494943071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561ecbf42890 0x561ecbf42350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:14:04.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.747+0000 7f89bfa00700 1 -- 172.21.15.131:0/2494943071 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc003a00 con 0x561ecbf48970 2024-05-08T22:14:04.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.747+0000 7f89bfa00700 1 -- 172.21.15.131:0/2494943071 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89cc003b60 con 0x561ecbf48970 2024-05-08T22:14:04.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.747+0000 7f89bfa00700 1 -- 172.21.15.131:0/2494943071 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc017470 con 0x561ecbf48970 2024-05-08T22:14:04.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.752+0000 7f89ec480b80 1 -- 172.21.15.131:0/2494943071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561ecbf48970 msgr2=0x561ecbf49460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:04.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.752+0000 7f89ec480b80 1 --2- 172.21.15.131:0/2494943071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561ecbf48970 0x561ecbf49460 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f89cc009790 tx=0x7f89cc009040 comp rx=0 tx=0).stop 2024-05-08T22:14:04.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.765+0000 7f89ec480b80 1 -- 172.21.15.131:0/2494943071 shutdown_connections 2024-05-08T22:14:04.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.766+0000 7f89ec480b80 1 --2- 172.21.15.131:0/2494943071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561ecbf48970 0x561ecbf49460 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:04.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.766+0000 7f89ec480b80 1 --2- 172.21.15.131:0/2494943071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561ecbf42890 0x561ecbf42350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:04.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.766+0000 7f89ec480b80 1 --2- 172.21.15.131:0/2494943071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561ecbf48050 0x561ecbf4c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:04.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.767+0000 7f89ec480b80 1 -- 172.21.15.131:0/2494943071 >> 172.21.15.131:0/2494943071 conn(0x561ecbea0210 msgr2=0x561ecbf48d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:04.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.810+0000 7f89ec480b80 1 -- 172.21.15.131:0/2494943071 shutdown_connections 2024-05-08T22:14:04.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.812+0000 7f89ec480b80 1 -- 172.21.15.131:0/2494943071 wait complete. 2024-05-08T22:14:04.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.826+0000 7f89ec480b80 1 -- start start 2024-05-08T22:14:04.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.826+0000 7f89ec480b80 1 Processor -- start 2024-05-08T22:14:04.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.860+0000 7f89ec480b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561ecbf42890 0x561ecbf537a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:04.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.860+0000 7f89ec480b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561ecbf48050 0x561ecbf51c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:04.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.860+0000 7f89ec480b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561ecbf48970 0x561ecbf522a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:04.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.860+0000 7f89ec480b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ecbf52970 con 0x561ecbf48970 2024-05-08T22:14:04.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.860+0000 7f89ec480b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ecbf52ae0 con 0x561ecbf48050 2024-05-08T22:14:04.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.860+0000 7f89ec480b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ecbfb2240 con 0x561ecbf42890 2024-05-08T22:14:04.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.860+0000 7f89ec480b80 10 client.0 _ll_register_callbacks cb 0x561ecbe37b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:14:04.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.870+0000 7f89d2800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561ecbf48970 0x561ecbf522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:04.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.872+0000 7f89d1e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561ecbf42890 0x561ecbf537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:04.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.872+0000 7f89d1e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561ecbf42890 0x561ecbf537a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51734/0 (socket says 172.21.15.131:51734) 2024-05-08T22:14:04.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.872+0000 7f89d1e00700 1 -- 172.21.15.131:0/4147940924 learned_addr learned my addr 172.21.15.131:0/4147940924 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:04.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.875+0000 7f89d1400700 1 --2- 172.21.15.131:0/4147940924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561ecbf48050 0x561ecbf51c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:04.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.882+0000 7f89d1400700 1 -- 172.21.15.131:0/4147940924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561ecbf42890 msgr2=0x561ecbf537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:04.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.882+0000 7f89d1400700 1 --2- 172.21.15.131:0/4147940924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561ecbf42890 0x561ecbf537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:04.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.882+0000 7f89d1400700 1 -- 172.21.15.131:0/4147940924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561ecbf48970 msgr2=0x561ecbf522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:04.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.882+0000 7f89d1400700 1 --2- 172.21.15.131:0/4147940924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561ecbf48970 0x561ecbf522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:04.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.882+0000 7f89d1400700 1 -- 172.21.15.131:0/4147940924 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ecbd133b0 con 0x561ecbf48050 2024-05-08T22:14:04.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.883+0000 7f89d1400700 1 --2- 172.21.15.131:0/4147940924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561ecbf48050 0x561ecbf51c40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f89c001a9a0 tx=0x7f89c0005ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aead76bb41f48f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:04.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.883+0000 7f89b3400700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:14:04.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.883+0000 7f89b3400700 1 -- 172.21.15.131:0/4147940924 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c00c4040 con 0x561ecbf48050 2024-05-08T22:14:04.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.883+0000 7f89b3400700 1 -- 172.21.15.131:0/4147940924 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f89c00ac5d0 con 0x561ecbf48050 2024-05-08T22:14:04.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.884+0000 7f89ec480b80 1 -- 172.21.15.131:0/4147940924 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89c4010db0 con 0x561ecbf48050 2024-05-08T22:14:04.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.884+0000 7f89b3400700 1 -- 172.21.15.131:0/4147940924 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89c00b0400 con 0x561ecbf48050 2024-05-08T22:14:04.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.884+0000 7f89ec480b80 10 client.26249 fetch_fsmap learned FSMap version 26 2024-05-08T22:14:04.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.885+0000 7f89ec480b80 1 -- 172.21.15.131:0/4147940924 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561ecbcbfb60 con 0x561ecbf48050 2024-05-08T22:14:04.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.885+0000 7f89b3400700 1 -- 172.21.15.131:0/4147940924 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89c4010db0 con 0x561ecbf48050 2024-05-08T22:14:04.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.886+0000 7f89b3400700 1 -- 172.21.15.131:0/4147940924 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f89c00abac0 con 0x561ecbf48050 2024-05-08T22:14:04.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.886+0000 7f89ec480b80 10 client.26249 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:14:04.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.886+0000 7f89ec480b80 10 client.26249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:14:04.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.886+0000 7f89ec480b80 20 client.26249 populate_metadata read hostname 'smithi131' 2024-05-08T22:14:04.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.886+0000 7f89ec480b80 1 --2- 172.21.15.131:0/4147940924 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x561ecbfb3610 0x561ecbfb5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:04.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.886+0000 7f89ec480b80 4 client.26249 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:14:04.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.886+0000 7f89ec480b80 1 -- 172.21.15.131:0/4147940924 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561ecbe8aec0 con 0x561ecbfb3610 2024-05-08T22:14:04.897 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:04 smithi131 ceph-mon[37260]: pgmap v206: 129 pgs: 129 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.8 MiB/s wr, 788 op/s 2024-05-08T22:14:04.897 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:04 smithi131 ceph-mon[37260]: osdmap e89: 12 total, 12 up, 12 in 2024-05-08T22:14:04.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.918+0000 7f89d1e00700 1 --2- 172.21.15.131:0/4147940924 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x561ecbfb3610 0x561ecbfb5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:04.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.956+0000 7f89d1e00700 1 --2- 172.21.15.131:0/4147940924 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x561ecbfb3610 0x561ecbfb5a30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d13ca7ea858fc32 server_cookie=13f2a961f0d27271 in_seq=0 out_seq=0 2024-05-08T22:14:04.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.987+0000 7f89b3400700 10 client.26249 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:14:04.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.987+0000 7f89b3400700 1 -- 172.21.15.131:0/4147940924 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x561ecbe8aec0 con 0x561ecbfb3610 2024-05-08T22:14:04.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:04.987+0000 7f89b3400700 10 client.26249 handle_command_reply: tid=0 2024-05-08T22:14:05.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:05.019+0000 7f89ec480b80 1 client.26249 shutdown 2024-05-08T22:14:05.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:05.020+0000 7f89ec480b80 1 -- 172.21.15.131:0/4147940924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561ecbf48050 msgr2=0x561ecbf51c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:05.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:05.020+0000 7f89ec480b80 1 --2- 172.21.15.131:0/4147940924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561ecbf48050 0x561ecbf51c40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f89c001a9a0 tx=0x7f89c0005ab0 comp rx=0 tx=0).stop 2024-05-08T22:14:05.025 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:04 smithi157 ceph-mon[77746]: pgmap v206: 129 pgs: 129 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.8 MiB/s wr, 788 op/s 2024-05-08T22:14:05.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:04 smithi157 ceph-mon[77746]: osdmap e89: 12 total, 12 up, 12 in 2024-05-08T22:14:05.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:05.055+0000 7f89ec480b80 1 -- 172.21.15.131:0/4147940924 shutdown_connections 2024-05-08T22:14:05.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:05.055+0000 7f89ec480b80 1 --2- 172.21.15.131:0/4147940924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561ecbf48970 0x561ecbf522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:05.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:05.055+0000 7f89ec480b80 1 --2- 172.21.15.131:0/4147940924 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x561ecbfb3610 0x561ecbfb5a30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:05.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:05.055+0000 7f89ec480b80 1 --2- 172.21.15.131:0/4147940924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561ecbf42890 0x561ecbf537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:05.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:05.055+0000 7f89ec480b80 1 --2- 172.21.15.131:0/4147940924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561ecbf48050 0x561ecbf51c40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:05.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:05.055+0000 7f89ec480b80 1 -- 172.21.15.131:0/4147940924 >> 172.21.15.131:0/4147940924 conn(0x561ecbea0210 msgr2=0x561ecbf475a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:05.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:05.059+0000 7f89b3400700 0 client.26249 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:14:05.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:05.088+0000 7f89ec480b80 1 -- 172.21.15.131:0/4147940924 shutdown_connections 2024-05-08T22:14:05.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:05.088+0000 7f89ec480b80 1 -- 172.21.15.131:0/4147940924 wait complete. 2024-05-08T22:14:05.089 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:04 smithi132 ceph-mon[78180]: pgmap v206: 129 pgs: 129 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.8 MiB/s wr, 788 op/s 2024-05-08T22:14:05.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:04 smithi132 ceph-mon[78180]: osdmap e89: 12 total, 12 up, 12 in 2024-05-08T22:14:05.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:05.135+0000 7f89ec480b80 20 client.26249 trim_cache size 0 max 16384 2024-05-08T22:14:05.152 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:14:05.152 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "scrub active (3324 inodes in the stack)", 2024-05-08T22:14:05.152 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-08T22:14:05.152 INFO:teuthology.orchestra.run.smithi131.stdout: "ca36159f-7052-4d2f-921f-ee97cd0c21fb": { 2024-05-08T22:14:05.152 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-08T22:14:05.152 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "ca36159f-7052-4d2f-921f-ee97cd0c21fb", 2024-05-08T22:14:05.152 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-08T22:14:05.152 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:14:05.152 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:14:05.152 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:14:05.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:05.204+0000 7f89d3200700 1 -- 172.21.15.131:0/4262323462 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f89b80a14a0 msgr2=0x7f89b80a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:05.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:05.204+0000 7f89d3200700 1 --2- 172.21.15.131:0/4262323462 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f89b80a14a0 0x7f89b80a3960 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f89c400ba80 tx=0x7f89c400b3f0 comp rx=0 tx=0).stop 2024-05-08T22:14:05.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:05.204+0000 7f89d3200700 1 -- 172.21.15.131:0/4262323462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89d410f570 msgr2=0x7f89d41a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:05.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:05.204+0000 7f89d3200700 1 --2- 172.21.15.131:0/4262323462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89d410f570 0x7f89d41a3ac0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f89c0000c00 tx=0x7f89c001a590 comp rx=0 tx=0).stop 2024-05-08T22:14:05.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:05.204+0000 7f89d3200700 1 -- 172.21.15.131:0/4262323462 shutdown_connections 2024-05-08T22:14:05.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:05.204+0000 7f89d3200700 1 --2- 172.21.15.131:0/4262323462 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f89b80a14a0 0x7f89b80a3960 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:05.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:05.204+0000 7f89d3200700 1 --2- 172.21.15.131:0/4262323462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89d411a560 0x7f89d41159f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:05.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:05.204+0000 7f89d3200700 1 --2- 172.21.15.131:0/4262323462 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89d41082c0 0x7f89d41a3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:05.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:05.204+0000 7f89d3200700 1 --2- 172.21.15.131:0/4262323462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89d410f570 0x7f89d41a3ac0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:05.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:05.204+0000 7f89d3200700 1 -- 172.21.15.131:0/4262323462 >> 172.21.15.131:0/4262323462 conn(0x7f89d406bee0 msgr2=0x7f89d411a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:05.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:05.230+0000 7f89d3200700 1 -- 172.21.15.131:0/4262323462 shutdown_connections 2024-05-08T22:14:05.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:05.235+0000 7f89d3200700 1 -- 172.21.15.131:0/4262323462 wait complete. 2024-05-08T22:14:05.345 INFO:tasks.workunit.client.0.smithi131.stdout: 203 424758 491 279276 305 234420 937 2024-05-08T22:14:05.696 INFO:tasks.cephfs.filesystem:scrub status for tag:ca36159f-7052-4d2f-921f-ee97cd0c21fb - {'path': '/', 'tag': 'ca36159f-7052-4d2f-921f-ee97cd0c21fb', 'options': 'recursive,force'} 2024-05-08T22:14:06.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:06 smithi131 ceph-mon[37260]: pgmap v208: 129 pgs: 129 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.8 MiB/s wr, 775 op/s 2024-05-08T22:14:07.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:06 smithi157 ceph-mon[77746]: pgmap v208: 129 pgs: 129 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.8 MiB/s wr, 775 op/s 2024-05-08T22:14:07.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:06 smithi132 ceph-mon[78180]: pgmap v208: 129 pgs: 129 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.8 MiB/s wr, 775 op/s 2024-05-08T22:14:08.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:08 smithi131 ceph-mon[37260]: pgmap v209: 129 pgs: 129 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 758 op/s 2024-05-08T22:14:09.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:08 smithi157 ceph-mon[77746]: pgmap v209: 129 pgs: 129 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 758 op/s 2024-05-08T22:14:09.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:08 smithi132 ceph-mon[78180]: pgmap v209: 129 pgs: 129 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 758 op/s 2024-05-08T22:14:10.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:09 smithi157 ceph-mon[77746]: pgmap v210: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 742 op/s 2024-05-08T22:14:10.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:09 smithi132 ceph-mon[78180]: pgmap v210: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 742 op/s 2024-05-08T22:14:10.145 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:09 smithi131 ceph-mon[37260]: pgmap v210: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 742 op/s 2024-05-08T22:14:12.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:12 smithi132 ceph-mon[78180]: pgmap v211: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 736 op/s 2024-05-08T22:14:12.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:12 smithi131 ceph-mon[37260]: pgmap v211: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 736 op/s 2024-05-08T22:14:12.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:12 smithi157 ceph-mon[77746]: pgmap v211: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 736 op/s 2024-05-08T22:14:14.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:13 smithi157 ceph-mon[77746]: osdmap e90: 12 total, 12 up, 12 in 2024-05-08T22:14:14.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:13 smithi157 ceph-mon[77746]: pgmap v213: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 479 op/s 2024-05-08T22:14:14.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:13 smithi132 ceph-mon[78180]: osdmap e90: 12 total, 12 up, 12 in 2024-05-08T22:14:14.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:13 smithi132 ceph-mon[78180]: pgmap v213: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 479 op/s 2024-05-08T22:14:14.166 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:13 smithi131 ceph-mon[37260]: osdmap e90: 12 total, 12 up, 12 in 2024-05-08T22:14:14.166 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:13 smithi131 ceph-mon[37260]: pgmap v213: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 479 op/s 2024-05-08T22:14:15.352 INFO:tasks.workunit.client.0.smithi131.stdout: 208 500518 408 324525 415 274255 1083 2024-05-08T22:14:16.368 INFO:tasks.workunit.client.0.smithi131.stdout: 2024-05-08T22:14:16.368 INFO:tasks.workunit.client.0.smithi131.stdout:Final score for writes: 208 2024-05-08T22:14:16.368 INFO:tasks.workunit.client.0.smithi131.stdout:Final score for reads : 300802 2024-05-08T22:14:16.368 INFO:tasks.workunit.client.0.smithi131.stdout: 2024-05-08T22:14:16.369 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-08T22:14:16.369 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-08T22:14:16.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:16 smithi132 ceph-mon[78180]: pgmap v214: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 563 op/s 2024-05-08T22:14:16.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:16 smithi131 ceph-mon[37260]: pgmap v214: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 563 op/s 2024-05-08T22:14:16.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:16 smithi157 ceph-mon[77746]: pgmap v214: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 563 op/s 2024-05-08T22:14:17.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:17 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:17.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:17 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:17.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:17 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:18.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:18 smithi131 ceph-mon[37260]: pgmap v215: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 560 op/s 2024-05-08T22:14:18.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:18 smithi157 ceph-mon[77746]: pgmap v215: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 560 op/s 2024-05-08T22:14:18.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:18 smithi132 ceph-mon[78180]: pgmap v215: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 560 op/s 2024-05-08T22:14:19.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:19 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:14:19.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:19 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:14:19.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:19 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:14:19.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:19 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:14:19.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:19 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:14:19.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:19 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:14:19.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:19 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:14:19.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:19 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:14:19.843 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:19 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:14:19.843 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:19 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:14:19.843 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:19 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:14:19.843 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:19 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:14:20.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:20 smithi131 ceph-mon[37260]: pgmap v216: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 716 op/s 2024-05-08T22:14:20.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:20 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:14:20.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:20 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:14:20.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:20 smithi131 ceph-mon[37260]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:14:20.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:20 smithi157 ceph-mon[77746]: pgmap v216: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 716 op/s 2024-05-08T22:14:20.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:20 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:14:20.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:20 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:14:20.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:20 smithi157 ceph-mon[77746]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:14:20.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:20 smithi132 ceph-mon[78180]: pgmap v216: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 716 op/s 2024-05-08T22:14:20.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:20 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:14:20.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:20 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:14:20.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:20 smithi132 ceph-mon[78180]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:14:22.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:22 smithi131 ceph-mon[37260]: pgmap v217: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 737 op/s 2024-05-08T22:14:22.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:22 smithi157 ceph-mon[77746]: pgmap v217: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 737 op/s 2024-05-08T22:14:22.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:22 smithi132 ceph-mon[78180]: pgmap v217: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 737 op/s 2024-05-08T22:14:24.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:24 smithi131 ceph-mon[37260]: pgmap v218: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 759 op/s 2024-05-08T22:14:24.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:24 smithi157 ceph-mon[77746]: pgmap v218: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 759 op/s 2024-05-08T22:14:24.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:24 smithi132 ceph-mon[78180]: pgmap v218: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 759 op/s 2024-05-08T22:14:26.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:26 smithi131 ceph-mon[37260]: pgmap v219: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.0 MiB/s wr, 899 op/s 2024-05-08T22:14:26.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:26 smithi157 ceph-mon[77746]: pgmap v219: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.0 MiB/s wr, 899 op/s 2024-05-08T22:14:26.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:26 smithi132 ceph-mon[78180]: pgmap v219: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.0 MiB/s wr, 899 op/s 2024-05-08T22:14:27.781 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:27 smithi157 ceph-mon[77746]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T22:14:27.848 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:27 smithi132 ceph-mon[78180]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T22:14:27.900 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:27 smithi131 ceph-mon[37260]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T22:14:28.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:28 smithi157 ceph-mon[77746]: pgmap v220: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.6 MiB/s wr, 818 op/s 2024-05-08T22:14:28.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:28 smithi132 ceph-mon[78180]: pgmap v220: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.6 MiB/s wr, 818 op/s 2024-05-08T22:14:28.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:28 smithi131 ceph-mon[37260]: pgmap v220: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.6 MiB/s wr, 818 op/s 2024-05-08T22:14:30.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:30 smithi157 ceph-mon[77746]: pgmap v221: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.9 MiB/s wr, 1.19k op/s 2024-05-08T22:14:30.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:30 smithi132 ceph-mon[78180]: pgmap v221: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.9 MiB/s wr, 1.19k op/s 2024-05-08T22:14:30.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:30 smithi131 ceph-mon[37260]: pgmap v221: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.9 MiB/s wr, 1.19k op/s 2024-05-08T22:14:32.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:32 smithi157 ceph-mon[77746]: pgmap v222: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.8 MiB/s wr, 946 op/s 2024-05-08T22:14:32.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:32 smithi132 ceph-mon[78180]: pgmap v222: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.8 MiB/s wr, 946 op/s 2024-05-08T22:14:32.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:32 smithi131 ceph-mon[37260]: pgmap v222: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.8 MiB/s wr, 946 op/s 2024-05-08T22:14:34.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:34 smithi157 ceph-mon[77746]: pgmap v223: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.3 MiB/s wr, 996 op/s 2024-05-08T22:14:34.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:34 smithi132 ceph-mon[78180]: pgmap v223: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.3 MiB/s wr, 996 op/s 2024-05-08T22:14:34.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:34 smithi131 ceph-mon[37260]: pgmap v223: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.3 MiB/s wr, 996 op/s 2024-05-08T22:14:35.697 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:14:35.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.899+0000 7f96adc00700 1 -- 172.21.15.131:0/3137653838 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96a8101180 msgr2=0x7f96a8101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:35.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.899+0000 7f96adc00700 1 --2- 172.21.15.131:0/3137653838 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96a8101180 0x7f96a8101560 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9698009990 tx=0x7f9698009ca0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.899+0000 7f96adc00700 1 -- 172.21.15.131:0/3137653838 shutdown_connections 2024-05-08T22:14:35.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.899+0000 7f96adc00700 1 --2- 172.21.15.131:0/3137653838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96a81062a0 0x7f96a810de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.899+0000 7f96adc00700 1 --2- 172.21.15.131:0/3137653838 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96a8101180 0x7f96a8101560 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.899+0000 7f96adc00700 1 --2- 172.21.15.131:0/3137653838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96a8101b30 0x7f96a8105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.899+0000 7f96adc00700 1 -- 172.21.15.131:0/3137653838 >> 172.21.15.131:0/3137653838 conn(0x7f96a80fc9e0 msgr2=0x7f96a80fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:35.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.899+0000 7f96adc00700 1 -- 172.21.15.131:0/3137653838 shutdown_connections 2024-05-08T22:14:35.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.899+0000 7f96adc00700 1 -- 172.21.15.131:0/3137653838 wait complete. 2024-05-08T22:14:35.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.900+0000 7f96adc00700 1 Processor -- start 2024-05-08T22:14:35.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.901+0000 7f96adc00700 1 -- start start 2024-05-08T22:14:35.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.901+0000 7f96adc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96a8101180 0x7f96a8199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:35.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.901+0000 7f96adc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96a81062a0 0x7f96a8199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:35.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.901+0000 7f96adc00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96a819deb0 0x7f96a819e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:35.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.901+0000 7f96adc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a819e930 con 0x7f96a81062a0 2024-05-08T22:14:35.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.901+0000 7f96adc00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a819eaa0 con 0x7f96a819deb0 2024-05-08T22:14:35.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.901+0000 7f96adc00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a819ec60 con 0x7f96a8101180 2024-05-08T22:14:35.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.902+0000 7f96a7400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96a819deb0 0x7f96a819e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:35.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.902+0000 7f96a6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96a8101180 0x7f96a8199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:35.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.902+0000 7f96a7400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96a819deb0 0x7f96a819e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:57530/0 (socket says 172.21.15.131:57530) 2024-05-08T22:14:35.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.902+0000 7f96a7400700 1 -- 172.21.15.131:0/1856269311 learned_addr learned my addr 172.21.15.131:0/1856269311 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:35.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.902+0000 7f969fe00700 1 --2- 172.21.15.131:0/1856269311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96a81062a0 0x7f96a8199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:35.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.902+0000 7f96a6a00700 1 -- 172.21.15.131:0/1856269311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96a819deb0 msgr2=0x7f96a819e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:35.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.902+0000 7f96a6a00700 1 --2- 172.21.15.131:0/1856269311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96a819deb0 0x7f96a819e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.902+0000 7f96a6a00700 1 -- 172.21.15.131:0/1856269311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96a81062a0 msgr2=0x7f96a8199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:35.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.902+0000 7f96a6a00700 1 --2- 172.21.15.131:0/1856269311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96a81062a0 0x7f96a8199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.902+0000 7f96a6a00700 1 -- 172.21.15.131:0/1856269311 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96980095f0 con 0x7f96a8101180 2024-05-08T22:14:35.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.902+0000 7f96a7400700 1 --2- 172.21.15.131:0/1856269311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96a819deb0 0x7f96a819e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:14:35.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.902+0000 7f96a6a00700 1 --2- 172.21.15.131:0/1856269311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96a8101180 0x7f96a8199620 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f96980098e0 tx=0x7f9698005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:35.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.902+0000 7f969f400700 1 -- 172.21.15.131:0/1856269311 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969801d070 con 0x7f96a8101180 2024-05-08T22:14:35.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.902+0000 7f96adc00700 1 -- 172.21.15.131:0/1856269311 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a81a6160 con 0x7f96a8101180 2024-05-08T22:14:35.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.902+0000 7f969f400700 1 -- 172.21.15.131:0/1856269311 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9698022470 con 0x7f96a8101180 2024-05-08T22:14:35.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.903+0000 7f969f400700 1 -- 172.21.15.131:0/1856269311 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9698017650 con 0x7f96a8101180 2024-05-08T22:14:35.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.903+0000 7f96adc00700 1 -- 172.21.15.131:0/1856269311 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a81a65e0 con 0x7f96a8101180 2024-05-08T22:14:35.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.907+0000 7f969f400700 1 -- 172.21.15.131:0/1856269311 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f96980225e0 con 0x7f96a8101180 2024-05-08T22:14:35.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.908+0000 7f96c1778b80 1 Processor -- start 2024-05-08T22:14:35.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.908+0000 7f96c1778b80 1 -- start start 2024-05-08T22:14:35.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.909+0000 7f96c1778b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614d2f69e00 0x5614d2f6a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:35.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.909+0000 7f96c1778b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f6a720 0x5614d2f64350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:35.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.909+0000 7f96c1778b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614d2f64890 0x5614d2f64cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:35.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.909+0000 7f96c1778b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d2ebb150 con 0x5614d2f69e00 2024-05-08T22:14:35.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.909+0000 7f96c1778b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d2eb8b10 con 0x5614d2f6a720 2024-05-08T22:14:35.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.909+0000 7f96c1778b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d2eb4400 con 0x5614d2f64890 2024-05-08T22:14:35.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.909+0000 7f969f400700 1 --2- 172.21.15.131:0/1856269311 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f968c0a1290 0x7f968c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:35.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.909+0000 7f96a6a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f6a720 0x5614d2f64350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:35.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.909+0000 7f96a6a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f6a720 0x5614d2f64350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:57546/0 (socket says 172.21.15.131:57546) 2024-05-08T22:14:35.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.909+0000 7f96a6a00700 1 -- 172.21.15.131:0/4058519828 learned_addr learned my addr 172.21.15.131:0/4058519828 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:35.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.909+0000 7f96a7400700 1 --2- 172.21.15.131:0/4058519828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614d2f69e00 0x5614d2f6a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:35.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.909+0000 7f969f400700 1 -- 172.21.15.131:0/1856269311 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f96980ce410 con 0x7f96a8101180 2024-05-08T22:14:35.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.909+0000 7f96a7400700 1 --2- 172.21.15.131:0/1856269311 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f968c0a1290 0x7f968c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:35.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.909+0000 7f969fe00700 1 --2- 172.21.15.131:0/4058519828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614d2f64890 0x5614d2f64cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:35.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.909+0000 7f96a6a00700 1 -- 172.21.15.131:0/4058519828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614d2f64890 msgr2=0x5614d2f64cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:35.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.909+0000 7f96a6a00700 1 --2- 172.21.15.131:0/4058519828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614d2f64890 0x5614d2f64cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.909+0000 7f96a6a00700 1 -- 172.21.15.131:0/4058519828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614d2f69e00 msgr2=0x5614d2f6a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:35.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.909+0000 7f96a6a00700 1 --2- 172.21.15.131:0/4058519828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614d2f69e00 0x5614d2f6a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.909+0000 7f96a6a00700 1 -- 172.21.15.131:0/4058519828 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614d2d353b0 con 0x5614d2f6a720 2024-05-08T22:14:35.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.909+0000 7f96a7400700 1 --2- 172.21.15.131:0/4058519828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614d2f69e00 0x5614d2f6a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:14:35.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.910+0000 7f96a6a00700 1 --2- 172.21.15.131:0/4058519828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f6a720 0x5614d2f64350 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f969809b210 tx=0x7f96980909d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3add0f0e229b855f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:35.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.910+0000 7f9697e00700 1 -- 172.21.15.131:0/4058519828 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969809b050 con 0x5614d2f6a720 2024-05-08T22:14:35.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.910+0000 7f9697e00700 1 -- 172.21.15.131:0/4058519828 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f969809adc0 con 0x5614d2f6a720 2024-05-08T22:14:35.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.910+0000 7f9697e00700 1 -- 172.21.15.131:0/4058519828 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96980a3670 con 0x5614d2f6a720 2024-05-08T22:14:35.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.910+0000 7f96c1778b80 1 -- 172.21.15.131:0/4058519828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f6a720 msgr2=0x5614d2f64350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:35.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.910+0000 7f96c1778b80 1 --2- 172.21.15.131:0/4058519828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f6a720 0x5614d2f64350 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f969809b210 tx=0x7f96980909d0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.910+0000 7f96a7400700 1 --2- 172.21.15.131:0/1856269311 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f968c0a1290 0x7f968c0a3750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009d30 tx=0x7f96a000b040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:35.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.911+0000 7f96c1778b80 1 -- 172.21.15.131:0/4058519828 shutdown_connections 2024-05-08T22:14:35.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.911+0000 7f96c1778b80 1 --2- 172.21.15.131:0/4058519828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614d2f64890 0x5614d2f64cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.911+0000 7f96c1778b80 1 --2- 172.21.15.131:0/4058519828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614d2f69e00 0x5614d2f6a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.911+0000 7f96c1778b80 1 --2- 172.21.15.131:0/4058519828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f6a720 0x5614d2f64350 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.911+0000 7f96c1778b80 1 -- 172.21.15.131:0/4058519828 >> 172.21.15.131:0/4058519828 conn(0x5614d2ec2210 msgr2=0x5614d2f5edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:35.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.911+0000 7f96c1778b80 1 -- 172.21.15.131:0/4058519828 shutdown_connections 2024-05-08T22:14:35.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.911+0000 7f96c1778b80 1 -- 172.21.15.131:0/4058519828 wait complete. 2024-05-08T22:14:35.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.911+0000 7f96c1778b80 1 -- start start 2024-05-08T22:14:35.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.911+0000 7f96c1778b80 1 Processor -- start 2024-05-08T22:14:35.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.912+0000 7f96c1778b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614d2f64890 0x5614d2f70680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:35.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.912+0000 7f96c1778b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614d2f69e00 0x5614d2f6d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:35.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.912+0000 7f96c1778b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f6a720 0x5614d2f6df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:35.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.912+0000 7f96c1778b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d2e50560 con 0x5614d2f64890 2024-05-08T22:14:35.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.912+0000 7f96c1778b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d2e50200 con 0x5614d2f6a720 2024-05-08T22:14:35.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.912+0000 7f96c1778b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d2d376a0 con 0x5614d2f69e00 2024-05-08T22:14:35.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.912+0000 7f96c1778b80 10 client.0 _ll_register_callbacks cb 0x5614d2e59b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:14:35.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.912+0000 7f96a6a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614d2f64890 0x5614d2f70680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:35.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.912+0000 7f96a7400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f6a720 0x5614d2f6df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:35.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.912+0000 7f96a6a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614d2f64890 0x5614d2f70680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37528/0 (socket says 172.21.15.131:37528) 2024-05-08T22:14:35.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.912+0000 7f96a6a00700 1 -- 172.21.15.131:0/2536576940 learned_addr learned my addr 172.21.15.131:0/2536576940 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:35.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.912+0000 7f969fe00700 1 --2- 172.21.15.131:0/2536576940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614d2f69e00 0x5614d2f6d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:35.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.913+0000 7f96a6a00700 1 -- 172.21.15.131:0/2536576940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614d2f69e00 msgr2=0x5614d2f6d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:35.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.913+0000 7f96a6a00700 1 --2- 172.21.15.131:0/2536576940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614d2f69e00 0x5614d2f6d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.913+0000 7f96a6a00700 1 -- 172.21.15.131:0/2536576940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f6a720 msgr2=0x5614d2f6df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:35.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.913+0000 7f96a6a00700 1 --2- 172.21.15.131:0/2536576940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f6a720 0x5614d2f6df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.913+0000 7f96a6a00700 1 -- 172.21.15.131:0/2536576940 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a81a65e0 con 0x5614d2f64890 2024-05-08T22:14:35.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.913+0000 7f96a7400700 1 --2- 172.21.15.131:0/2536576940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f6a720 0x5614d2f6df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:14:35.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.913+0000 7f96a6a00700 1 --2- 172.21.15.131:0/2536576940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614d2f64890 0x5614d2f70680 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9698097780 tx=0x7f9698009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c7796014c58cb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:35.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.913+0000 7f9694c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:14:35.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.913+0000 7f9694c00700 1 -- 172.21.15.131:0/2536576940 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969809abe0 con 0x5614d2f64890 2024-05-08T22:14:35.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.913+0000 7f9694c00700 1 -- 172.21.15.131:0/2536576940 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f969809b980 con 0x5614d2f64890 2024-05-08T22:14:35.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.913+0000 7f96c1778b80 1 -- 172.21.15.131:0/2536576940 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614d2e0d270 con 0x5614d2f64890 2024-05-08T22:14:35.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.913+0000 7f9694c00700 1 -- 172.21.15.131:0/2536576940 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96980cbba0 con 0x5614d2f64890 2024-05-08T22:14:35.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.913+0000 7f96c1778b80 10 client.16239 fetch_fsmap learned FSMap version 27 2024-05-08T22:14:35.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.913+0000 7f96c1778b80 1 -- 172.21.15.131:0/2536576940 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5614d2ce1b60 con 0x5614d2f64890 2024-05-08T22:14:35.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.914+0000 7f9694c00700 1 -- 172.21.15.131:0/2536576940 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5614d2e0d270 con 0x5614d2f64890 2024-05-08T22:14:35.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.914+0000 7f9694c00700 1 -- 172.21.15.131:0/2536576940 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f9698063070 con 0x5614d2f64890 2024-05-08T22:14:35.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.914+0000 7f96c1778b80 10 client.16239 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T22:14:35.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.914+0000 7f96c1778b80 10 client.16239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:14:35.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.914+0000 7f96c1778b80 20 client.16239 populate_metadata read hostname 'smithi131' 2024-05-08T22:14:35.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.914+0000 7f96c1778b80 1 --2- 172.21.15.131:0/2536576940 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5614d2f7d9e0 0x5614d2f7fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:35.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.914+0000 7f96c1778b80 4 client.16239 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:14:35.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.914+0000 7f96c1778b80 1 -- 172.21.15.131:0/2536576940 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614d2f33810 con 0x5614d2f7d9e0 2024-05-08T22:14:35.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.914+0000 7f969fe00700 1 --2- 172.21.15.131:0/2536576940 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5614d2f7d9e0 0x5614d2f7fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:35.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.915+0000 7f969fe00700 1 --2- 172.21.15.131:0/2536576940 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5614d2f7d9e0 0x5614d2f7fe00 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cc34b119fdefce5 server_cookie=6fbe165e926a24d in_seq=0 out_seq=0 2024-05-08T22:14:35.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.915+0000 7f9694c00700 10 client.16239 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:14:35.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.916+0000 7f9694c00700 1 -- 172.21.15.131:0/2536576940 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5614d2f33810 con 0x5614d2f7d9e0 2024-05-08T22:14:35.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.916+0000 7f9694c00700 10 client.16239 handle_command_reply: tid=0 2024-05-08T22:14:35.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.916+0000 7f96c1778b80 1 client.16239 shutdown 2024-05-08T22:14:35.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.917+0000 7f96c1778b80 1 -- 172.21.15.131:0/2536576940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614d2f64890 msgr2=0x5614d2f70680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:35.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.917+0000 7f96c1778b80 1 --2- 172.21.15.131:0/2536576940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614d2f64890 0x5614d2f70680 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9698097780 tx=0x7f9698009290 comp rx=0 tx=0).stop 2024-05-08T22:14:35.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.917+0000 7f96c1778b80 1 -- 172.21.15.131:0/2536576940 shutdown_connections 2024-05-08T22:14:35.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.917+0000 7f96c1778b80 1 --2- 172.21.15.131:0/2536576940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f6a720 0x5614d2f6df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.917+0000 7f96c1778b80 1 --2- 172.21.15.131:0/2536576940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614d2f64890 0x5614d2f70680 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.917+0000 7f96c1778b80 1 --2- 172.21.15.131:0/2536576940 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5614d2f7d9e0 0x5614d2f7fe00 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.917+0000 7f96c1778b80 1 --2- 172.21.15.131:0/2536576940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614d2f69e00 0x5614d2f6d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.917+0000 7f96c1778b80 1 -- 172.21.15.131:0/2536576940 >> 172.21.15.131:0/2536576940 conn(0x5614d2ec2210 msgr2=0x5614d2f5e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:35.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.917+0000 7f9694c00700 0 client.16239 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:14:35.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.917+0000 7f96c1778b80 1 -- 172.21.15.131:0/2536576940 shutdown_connections 2024-05-08T22:14:35.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.917+0000 7f96c1778b80 1 -- 172.21.15.131:0/2536576940 wait complete. 2024-05-08T22:14:35.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.917+0000 7f96c1778b80 20 client.16239 trim_cache size 0 max 16384 2024-05-08T22:14:35.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.935+0000 7f96c1778b80 1 Processor -- start 2024-05-08T22:14:35.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.935+0000 7f96c1778b80 1 -- start start 2024-05-08T22:14:35.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.935+0000 7f96c1778b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614d2f64890 0x5614d2f79e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:35.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.935+0000 7f96c1778b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614d2f69e00 0x5614d2f5ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:35.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.935+0000 7f96c1778b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f6a720 0x5614d2f701b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:35.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.935+0000 7f96c1778b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d2f85050 con 0x5614d2f69e00 2024-05-08T22:14:35.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.935+0000 7f96c1778b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d2f608c0 con 0x5614d2f6a720 2024-05-08T22:14:35.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.935+0000 7f96c1778b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d2f59830 con 0x5614d2f64890 2024-05-08T22:14:35.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.935+0000 7f96a7400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f6a720 0x5614d2f701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:35.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.935+0000 7f969fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614d2f69e00 0x5614d2f5ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:35.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.935+0000 7f96a7400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f6a720 0x5614d2f701b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:57562/0 (socket says 172.21.15.131:57562) 2024-05-08T22:14:35.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.936+0000 7f96a7400700 1 -- 172.21.15.131:0/1571859701 learned_addr learned my addr 172.21.15.131:0/1571859701 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:35.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.936+0000 7f96a6a00700 1 --2- 172.21.15.131:0/1571859701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614d2f64890 0x5614d2f79e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:35.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.936+0000 7f96a7400700 1 -- 172.21.15.131:0/1571859701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614d2f64890 msgr2=0x5614d2f79e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:35.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.936+0000 7f96a7400700 1 --2- 172.21.15.131:0/1571859701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614d2f64890 0x5614d2f79e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.936+0000 7f96a7400700 1 -- 172.21.15.131:0/1571859701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614d2f69e00 msgr2=0x5614d2f5ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:35.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.936+0000 7f96a7400700 1 --2- 172.21.15.131:0/1571859701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614d2f69e00 0x5614d2f5ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.936+0000 7f96a7400700 1 -- 172.21.15.131:0/1571859701 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614d2f64f40 con 0x5614d2f6a720 2024-05-08T22:14:35.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.936+0000 7f96a6a00700 1 --2- 172.21.15.131:0/1571859701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614d2f64890 0x5614d2f79e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:14:35.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.936+0000 7f969fe00700 1 --2- 172.21.15.131:0/1571859701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614d2f69e00 0x5614d2f5ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:14:35.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.936+0000 7f96a7400700 1 --2- 172.21.15.131:0/1571859701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f6a720 0x5614d2f701b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f96a0008120 tx=0x7f96a0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f15c23356aaa57ad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:35.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.936+0000 7f969cc00700 1 -- 172.21.15.131:0/1571859701 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a0004bb0 con 0x5614d2f6a720 2024-05-08T22:14:35.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.936+0000 7f969cc00700 1 -- 172.21.15.131:0/1571859701 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96a000f040 con 0x5614d2f6a720 2024-05-08T22:14:35.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.936+0000 7f969cc00700 1 -- 172.21.15.131:0/1571859701 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a00044c0 con 0x5614d2f6a720 2024-05-08T22:14:35.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.937+0000 7f96c1778b80 1 -- 172.21.15.131:0/1571859701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f6a720 msgr2=0x5614d2f701b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:35.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.937+0000 7f96c1778b80 1 --2- 172.21.15.131:0/1571859701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f6a720 0x5614d2f701b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f96a0008120 tx=0x7f96a0009920 comp rx=0 tx=0).stop 2024-05-08T22:14:35.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.937+0000 7f96c1778b80 1 -- 172.21.15.131:0/1571859701 shutdown_connections 2024-05-08T22:14:35.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.937+0000 7f96c1778b80 1 --2- 172.21.15.131:0/1571859701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f6a720 0x5614d2f701b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.937+0000 7f96c1778b80 1 --2- 172.21.15.131:0/1571859701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614d2f64890 0x5614d2f79e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.937+0000 7f96c1778b80 1 --2- 172.21.15.131:0/1571859701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614d2f69e00 0x5614d2f5ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.937+0000 7f96c1778b80 1 -- 172.21.15.131:0/1571859701 >> 172.21.15.131:0/1571859701 conn(0x5614d2ec2210 msgr2=0x5614d2f79770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:35.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.937+0000 7f96c1778b80 1 -- 172.21.15.131:0/1571859701 shutdown_connections 2024-05-08T22:14:35.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.937+0000 7f96c1778b80 1 -- 172.21.15.131:0/1571859701 wait complete. 2024-05-08T22:14:35.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.937+0000 7f96c1778b80 1 -- start start 2024-05-08T22:14:35.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.938+0000 7f96c1778b80 1 Processor -- start 2024-05-08T22:14:35.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.938+0000 7f96c1778b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f64890 0x5614d2f74cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:35.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.939+0000 7f96c1778b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614d2f69e00 0x5614d2f75200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:35.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.939+0000 7f96c1778b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614d2f6a720 0x5614d2f65ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:35.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.939+0000 7f96c1778b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d2f661b0 con 0x5614d2f69e00 2024-05-08T22:14:35.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.939+0000 7f96c1778b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d2f66320 con 0x5614d2f64890 2024-05-08T22:14:35.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.939+0000 7f96c1778b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d2f664e0 con 0x5614d2f6a720 2024-05-08T22:14:35.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.939+0000 7f96c1778b80 10 client.0 _ll_register_callbacks cb 0x5614d2e59b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:14:35.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.939+0000 7f96a6a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f64890 0x5614d2f74cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:35.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.939+0000 7f96a6a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f64890 0x5614d2f74cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:57572/0 (socket says 172.21.15.131:57572) 2024-05-08T22:14:35.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.939+0000 7f969fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614d2f69e00 0x5614d2f75200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:35.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.939+0000 7f96a6a00700 1 -- 172.21.15.131:0/662032603 learned_addr learned my addr 172.21.15.131:0/662032603 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:35.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.939+0000 7f96a7400700 1 --2- 172.21.15.131:0/662032603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614d2f6a720 0x5614d2f65ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:35.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.939+0000 7f96a6a00700 1 -- 172.21.15.131:0/662032603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614d2f6a720 msgr2=0x5614d2f65ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:35.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.939+0000 7f96a6a00700 1 --2- 172.21.15.131:0/662032603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614d2f6a720 0x5614d2f65ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.939+0000 7f96a6a00700 1 -- 172.21.15.131:0/662032603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614d2f69e00 msgr2=0x5614d2f75200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:35.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.939+0000 7f96a6a00700 1 --2- 172.21.15.131:0/662032603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614d2f69e00 0x5614d2f75200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.939+0000 7f96a6a00700 1 -- 172.21.15.131:0/662032603 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614d2ce1b60 con 0x5614d2f64890 2024-05-08T22:14:35.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.939+0000 7f96a6a00700 1 --2- 172.21.15.131:0/662032603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f64890 0x5614d2f74cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9698006010 tx=0x7f9698009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff347ba320743fad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:35.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.939+0000 7f9694c00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:14:35.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.940+0000 7f9694c00700 1 -- 172.21.15.131:0/662032603 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969807f070 con 0x5614d2f64890 2024-05-08T22:14:35.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.940+0000 7f96c1778b80 1 -- 172.21.15.131:0/662032603 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f969809adc0 con 0x5614d2f64890 2024-05-08T22:14:35.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.940+0000 7f9694c00700 1 -- 172.21.15.131:0/662032603 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f969809b980 con 0x5614d2f64890 2024-05-08T22:14:35.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.940+0000 7f9694c00700 1 -- 172.21.15.131:0/662032603 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f969809b7a0 con 0x5614d2f64890 2024-05-08T22:14:35.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.940+0000 7f96c1778b80 10 client.26267 fetch_fsmap learned FSMap version 27 2024-05-08T22:14:35.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.940+0000 7f96c1778b80 1 -- 172.21.15.131:0/662032603 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f9690009df0 con 0x5614d2f64890 2024-05-08T22:14:35.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.940+0000 7f9694c00700 1 -- 172.21.15.131:0/662032603 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f969809adc0 con 0x5614d2f64890 2024-05-08T22:14:35.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.940+0000 7f9694c00700 1 -- 172.21.15.131:0/662032603 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 27) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f9698024080 con 0x5614d2f64890 2024-05-08T22:14:35.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.940+0000 7f96c1778b80 10 client.26267 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T22:14:35.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.940+0000 7f96c1778b80 10 client.26267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:14:35.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.940+0000 7f96c1778b80 20 client.26267 populate_metadata read hostname 'smithi131' 2024-05-08T22:14:35.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.940+0000 7f96c1778b80 1 --2- 172.21.15.131:0/662032603 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5614d2fd4cc0 0x5614d2fd70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:35.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.940+0000 7f96c1778b80 4 client.26267 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:14:35.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.940+0000 7f96c1778b80 1 -- 172.21.15.131:0/662032603 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5614d2eacec0 con 0x5614d2fd4cc0 2024-05-08T22:14:35.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.941+0000 7f969fe00700 1 --2- 172.21.15.131:0/662032603 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5614d2fd4cc0 0x5614d2fd70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:35.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.941+0000 7f9694c00700 10 client.26267 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:14:35.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.941+0000 7f969fe00700 1 --2- 172.21.15.131:0/662032603 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5614d2fd4cc0 0x5614d2fd70b0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f68cc81ccf1c7e6b server_cookie=7c5fc1b773d0ad5d in_seq=0 out_seq=0 2024-05-08T22:14:35.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.941+0000 7f9694c00700 1 -- 172.21.15.131:0/662032603 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5614d2eacec0 con 0x5614d2fd4cc0 2024-05-08T22:14:35.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.941+0000 7f9694c00700 10 client.26267 handle_command_reply: tid=0 2024-05-08T22:14:35.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.942+0000 7f96c1778b80 1 client.26267 shutdown 2024-05-08T22:14:35.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.942+0000 7f96c1778b80 1 -- 172.21.15.131:0/662032603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f64890 msgr2=0x5614d2f74cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:35.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.942+0000 7f96c1778b80 1 --2- 172.21.15.131:0/662032603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f64890 0x5614d2f74cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9698006010 tx=0x7f9698009ca0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.942+0000 7f96c1778b80 1 -- 172.21.15.131:0/662032603 shutdown_connections 2024-05-08T22:14:35.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.942+0000 7f96c1778b80 1 --2- 172.21.15.131:0/662032603 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5614d2fd4cc0 0x5614d2fd70b0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.942+0000 7f96c1778b80 1 --2- 172.21.15.131:0/662032603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614d2f6a720 0x5614d2f65ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.942+0000 7f96c1778b80 1 --2- 172.21.15.131:0/662032603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614d2f64890 0x5614d2f74cc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.942+0000 7f9694c00700 0 client.26267 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:14:35.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.942+0000 7f96c1778b80 1 --2- 172.21.15.131:0/662032603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5614d2f69e00 0x5614d2f75200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.942+0000 7f96c1778b80 1 -- 172.21.15.131:0/662032603 >> 172.21.15.131:0/662032603 conn(0x5614d2ec2210 msgr2=0x5614d2f79770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:35.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.942+0000 7f96c1778b80 1 -- 172.21.15.131:0/662032603 shutdown_connections 2024-05-08T22:14:35.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.942+0000 7f96c1778b80 1 -- 172.21.15.131:0/662032603 wait complete. 2024-05-08T22:14:35.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.942+0000 7f96c1778b80 20 client.26267 trim_cache size 0 max 16384 2024-05-08T22:14:35.948 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:14:35.948 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-08T22:14:35.948 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-08T22:14:35.948 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:14:35.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.943+0000 7f96adc00700 1 -- 172.21.15.131:0/1856269311 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f968c0a1290 msgr2=0x7f968c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:35.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.943+0000 7f96adc00700 1 --2- 172.21.15.131:0/1856269311 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f968c0a1290 0x7f968c0a3750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009d30 tx=0x7f96a000b040 comp rx=0 tx=0).stop 2024-05-08T22:14:35.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.943+0000 7f96adc00700 1 -- 172.21.15.131:0/1856269311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96a8101180 msgr2=0x7f96a8199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:35.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.943+0000 7f96adc00700 1 --2- 172.21.15.131:0/1856269311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96a8101180 0x7f96a8199620 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f96980098e0 tx=0x7f9698005740 comp rx=0 tx=0).stop 2024-05-08T22:14:35.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.943+0000 7f96adc00700 1 -- 172.21.15.131:0/1856269311 shutdown_connections 2024-05-08T22:14:35.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.943+0000 7f96adc00700 1 --2- 172.21.15.131:0/1856269311 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f968c0a1290 0x7f968c0a3750 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.943+0000 7f96adc00700 1 --2- 172.21.15.131:0/1856269311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96a819deb0 0x7f96a819e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.943+0000 7f96adc00700 1 --2- 172.21.15.131:0/1856269311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96a8101180 0x7f96a8199620 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.943+0000 7f96adc00700 1 --2- 172.21.15.131:0/1856269311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96a81062a0 0x7f96a8199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:35.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.943+0000 7f96adc00700 1 -- 172.21.15.131:0/1856269311 >> 172.21.15.131:0/1856269311 conn(0x7f96a80fc9e0 msgr2=0x7f96a80fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:35.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.943+0000 7f96adc00700 1 -- 172.21.15.131:0/1856269311 shutdown_connections 2024-05-08T22:14:35.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:35.943+0000 7f96adc00700 1 -- 172.21.15.131:0/1856269311 wait complete. 2024-05-08T22:14:35.962 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T22:14:35.962 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:14:36.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.164+0000 7f8dfe000700 1 -- 172.21.15.131:0/2476019112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df8101a00 msgr2=0x7f8df8105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:36.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.164+0000 7f8dfe000700 1 --2- 172.21.15.131:0/2476019112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df8101a00 0x7f8df8105a40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009990 tx=0x7f8de4009ca0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.164+0000 7f8dfe000700 1 -- 172.21.15.131:0/2476019112 shutdown_connections 2024-05-08T22:14:36.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.164+0000 7f8dfe000700 1 --2- 172.21.15.131:0/2476019112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8df8106170 0x7f8df810dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.164+0000 7f8dfe000700 1 --2- 172.21.15.131:0/2476019112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8df8101050 0x7f8df8101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.164+0000 7f8dfe000700 1 --2- 172.21.15.131:0/2476019112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df8101a00 0x7f8df8105a40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.164+0000 7f8dfe000700 1 -- 172.21.15.131:0/2476019112 >> 172.21.15.131:0/2476019112 conn(0x7f8df80fc8f0 msgr2=0x7f8df80fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:36.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.164+0000 7f8dfe000700 1 -- 172.21.15.131:0/2476019112 shutdown_connections 2024-05-08T22:14:36.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.164+0000 7f8dfe000700 1 -- 172.21.15.131:0/2476019112 wait complete. 2024-05-08T22:14:36.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.165+0000 7f8dfe000700 1 Processor -- start 2024-05-08T22:14:36.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.165+0000 7f8dfe000700 1 -- start start 2024-05-08T22:14:36.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.166+0000 7f8dfe000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8df8101050 0x7f8df8197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:36.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.166+0000 7f8dfe000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df8106170 0x7f8df8197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:36.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.166+0000 7f8dfe000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8df819bbc0 0x7f8df819c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:36.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.166+0000 7f8dfe000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df819c640 con 0x7f8df819bbc0 2024-05-08T22:14:36.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.166+0000 7f8dfe000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df819c7b0 con 0x7f8df8106170 2024-05-08T22:14:36.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.166+0000 7f8dfe000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df819c970 con 0x7f8df8101050 2024-05-08T22:14:36.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.166+0000 7f8df6a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df8106170 0x7f8df8197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:36.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.166+0000 7f8df6a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df8106170 0x7f8df8197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:57602/0 (socket says 172.21.15.131:57602) 2024-05-08T22:14:36.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.166+0000 7f8df6a00700 1 -- 172.21.15.131:0/1180558167 learned_addr learned my addr 172.21.15.131:0/1180558167 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:36.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.166+0000 7f8df7e00700 1 --2- 172.21.15.131:0/1180558167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8df819bbc0 0x7f8df819c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:36.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.166+0000 7f8df7400700 1 --2- 172.21.15.131:0/1180558167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8df8101050 0x7f8df8197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:36.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.166+0000 7f8df6a00700 1 -- 172.21.15.131:0/1180558167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8df8101050 msgr2=0x7f8df8197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:36.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.166+0000 7f8df6a00700 1 --2- 172.21.15.131:0/1180558167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8df8101050 0x7f8df8197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.166+0000 7f8df6a00700 1 -- 172.21.15.131:0/1180558167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8df819bbc0 msgr2=0x7f8df819c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:36.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.166+0000 7f8df6a00700 1 --2- 172.21.15.131:0/1180558167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8df819bbc0 0x7f8df819c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.166+0000 7f8df6a00700 1 -- 172.21.15.131:0/1180558167 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de8009520 con 0x7f8df8106170 2024-05-08T22:14:36.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.166+0000 7f8df7e00700 1 --2- 172.21.15.131:0/1180558167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8df819bbc0 0x7f8df819c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:14:36.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.166+0000 7f8df6a00700 1 --2- 172.21.15.131:0/1180558167 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df8106170 0x7f8df8197870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8de4000c00 tx=0x7f8de4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:36.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.167+0000 7f8defe00700 1 -- 172.21.15.131:0/1180558167 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de401d070 con 0x7f8df8106170 2024-05-08T22:14:36.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.167+0000 7f8defe00700 1 -- 172.21.15.131:0/1180558167 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8de4021410 con 0x7f8df8106170 2024-05-08T22:14:36.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.167+0000 7f8dfe000700 1 -- 172.21.15.131:0/1180558167 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de40095f0 con 0x7f8df8106170 2024-05-08T22:14:36.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.167+0000 7f8defe00700 1 -- 172.21.15.131:0/1180558167 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de4017650 con 0x7f8df8106170 2024-05-08T22:14:36.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.167+0000 7f8dfe000700 1 -- 172.21.15.131:0/1180558167 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df81a41b0 con 0x7f8df8106170 2024-05-08T22:14:36.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.168+0000 7f8dfe000700 1 -- 172.21.15.131:0/1180558167 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df8109390 con 0x7f8df8106170 2024-05-08T22:14:36.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.170+0000 7f8defe00700 1 -- 172.21.15.131:0/1180558167 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f8de4021580 con 0x7f8df8106170 2024-05-08T22:14:36.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.171+0000 7f8defe00700 1 --2- 172.21.15.131:0/1180558167 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8ddc0a1310 0x7f8ddc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:36.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.171+0000 7f8defe00700 1 -- 172.21.15.131:0/1180558167 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f8de40ce250 con 0x7f8df8106170 2024-05-08T22:14:36.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.172+0000 7f8df7400700 1 --2- 172.21.15.131:0/1180558167 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8ddc0a1310 0x7f8ddc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:36.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.172+0000 7f8df7400700 1 --2- 172.21.15.131:0/1180558167 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8ddc0a1310 0x7f8ddc0a37d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8de8009df0 tx=0x7f8de8009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:36.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.177+0000 7f8defe00700 1 -- 172.21.15.131:0/1180558167 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8de4097340 con 0x7f8df8106170 2024-05-08T22:14:36.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.599+0000 7f8dfe000700 1 -- 172.21.15.131:0/1180558167 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8df8102520 con 0x7f8df8106170 2024-05-08T22:14:36.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.600+0000 7f8defe00700 1 -- 172.21.15.131:0/1180558167 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10893 (secure 0 0 0) 0x7f8de409a960 con 0x7f8df8106170 2024-05-08T22:14:36.601 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:14:36.601 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:05:14.034908+0000","modified":"2024-05-08T22:14:26.383343+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:14:36.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.603+0000 7f8dfe000700 1 -- 172.21.15.131:0/1180558167 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8ddc0a1310 msgr2=0x7f8ddc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:36.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.603+0000 7f8dfe000700 1 --2- 172.21.15.131:0/1180558167 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8ddc0a1310 0x7f8ddc0a37d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8de8009df0 tx=0x7f8de8009260 comp rx=0 tx=0).stop 2024-05-08T22:14:36.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.603+0000 7f8dfe000700 1 -- 172.21.15.131:0/1180558167 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df8106170 msgr2=0x7f8df8197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:36.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.603+0000 7f8dfe000700 1 --2- 172.21.15.131:0/1180558167 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df8106170 0x7f8df8197870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8de4000c00 tx=0x7f8de4004730 comp rx=0 tx=0).stop 2024-05-08T22:14:36.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.603+0000 7f8dfe000700 1 -- 172.21.15.131:0/1180558167 shutdown_connections 2024-05-08T22:14:36.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.603+0000 7f8dfe000700 1 --2- 172.21.15.131:0/1180558167 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8ddc0a1310 0x7f8ddc0a37d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.603+0000 7f8dfe000700 1 --2- 172.21.15.131:0/1180558167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8df819bbc0 0x7f8df819c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.603+0000 7f8dfe000700 1 --2- 172.21.15.131:0/1180558167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8df8101050 0x7f8df8197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.603+0000 7f8dfe000700 1 --2- 172.21.15.131:0/1180558167 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df8106170 0x7f8df8197870 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.603+0000 7f8dfe000700 1 -- 172.21.15.131:0/1180558167 >> 172.21.15.131:0/1180558167 conn(0x7f8df80fc8f0 msgr2=0x7f8df80fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:36.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.603+0000 7f8dfe000700 1 -- 172.21.15.131:0/1180558167 shutdown_connections 2024-05-08T22:14:36.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.603+0000 7f8dfe000700 1 -- 172.21.15.131:0/1180558167 wait complete. 2024-05-08T22:14:36.605 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 27 2024-05-08T22:14:36.623 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T22:14:36.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:36 smithi157 ceph-mon[77746]: pgmap v224: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.7 MiB/s wr, 1.19k op/s 2024-05-08T22:14:36.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.831+0000 7f7ac9e00700 1 -- 172.21.15.131:0/3966587618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ac41060b0 msgr2=0x7f7ac410dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:36.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.831+0000 7f7ac9e00700 1 --2- 172.21.15.131:0/3966587618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ac41060b0 0x7f7ac410dc30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00b0a0 tx=0x7f7abc00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.831+0000 7f7ac9e00700 1 -- 172.21.15.131:0/3966587618 shutdown_connections 2024-05-08T22:14:36.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.831+0000 7f7ac9e00700 1 --2- 172.21.15.131:0/3966587618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ac41060b0 0x7f7ac410dc30 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.831+0000 7f7ac9e00700 1 --2- 172.21.15.131:0/3966587618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ac4100f90 0x7f7ac4101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.831+0000 7f7ac9e00700 1 --2- 172.21.15.131:0/3966587618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ac4101940 0x7f7ac4105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.831+0000 7f7ac9e00700 1 -- 172.21.15.131:0/3966587618 >> 172.21.15.131:0/3966587618 conn(0x7f7ac40fc810 msgr2=0x7f7ac40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:36.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.831+0000 7f7ac9e00700 1 -- 172.21.15.131:0/3966587618 shutdown_connections 2024-05-08T22:14:36.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.831+0000 7f7ac9e00700 1 -- 172.21.15.131:0/3966587618 wait complete. 2024-05-08T22:14:36.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.832+0000 7f7ac9e00700 1 Processor -- start 2024-05-08T22:14:36.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.833+0000 7f7ac9e00700 1 -- start start 2024-05-08T22:14:36.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.833+0000 7f7ac9e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ac4100f90 0x7f7ac41995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:36.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.833+0000 7f7ac9e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ac41060b0 0x7f7ac4199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:36.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.833+0000 7f7ac9e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ac419de30 0x7f7ac419e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:36.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.833+0000 7f7ac9e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac419e8b0 con 0x7f7ac41060b0 2024-05-08T22:14:36.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.833+0000 7f7ac9e00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac419ea20 con 0x7f7ac419de30 2024-05-08T22:14:36.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.833+0000 7f7ac9e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac419ebe0 con 0x7f7ac4100f90 2024-05-08T22:14:36.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.833+0000 7f7ac3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ac41060b0 0x7f7ac4199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:36.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.833+0000 7f7ac9400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ac419de30 0x7f7ac419e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:36.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.833+0000 7f7ac8a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ac4100f90 0x7f7ac41995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:36.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.833+0000 7f7ac3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ac41060b0 0x7f7ac4199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37594/0 (socket says 172.21.15.131:37594) 2024-05-08T22:14:36.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.833+0000 7f7ac3e00700 1 -- 172.21.15.131:0/3821746832 learned_addr learned my addr 172.21.15.131:0/3821746832 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:36.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.834+0000 7f7ac8a00700 1 -- 172.21.15.131:0/3821746832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ac419de30 msgr2=0x7f7ac419e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:36.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.834+0000 7f7ac8a00700 1 --2- 172.21.15.131:0/3821746832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ac419de30 0x7f7ac419e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.834+0000 7f7ac8a00700 1 -- 172.21.15.131:0/3821746832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ac41060b0 msgr2=0x7f7ac4199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:36.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.834+0000 7f7ac8a00700 1 --2- 172.21.15.131:0/3821746832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ac41060b0 0x7f7ac4199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.834+0000 7f7ac8a00700 1 -- 172.21.15.131:0/3821746832 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab8009520 con 0x7f7ac4100f90 2024-05-08T22:14:36.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.834+0000 7f7ac8a00700 1 --2- 172.21.15.131:0/3821746832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ac4100f90 0x7f7ac41995a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800e9f0 tx=0x7f7ab800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:36.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.834+0000 7f7ac1600700 1 -- 172.21.15.131:0/3821746832 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab800cb70 con 0x7f7ac4100f90 2024-05-08T22:14:36.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.834+0000 7f7ac9e00700 1 -- 172.21.15.131:0/3821746832 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7abc009d00 con 0x7f7ac4100f90 2024-05-08T22:14:36.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.834+0000 7f7ac1600700 1 -- 172.21.15.131:0/3821746832 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ab800ee90 con 0x7f7ac4100f90 2024-05-08T22:14:36.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.834+0000 7f7ac1600700 1 -- 172.21.15.131:0/3821746832 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8003cf0 con 0x7f7ac4100f90 2024-05-08T22:14:36.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.834+0000 7f7ac9e00700 1 -- 172.21.15.131:0/3821746832 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac41a64e0 con 0x7f7ac4100f90 2024-05-08T22:14:36.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.837+0000 7f7ac1600700 1 -- 172.21.15.131:0/3821746832 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f7ab800ccd0 con 0x7f7ac4100f90 2024-05-08T22:14:36.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.838+0000 7f7ac1600700 1 --2- 172.21.15.131:0/3821746832 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7aac0a1290 0x7f7aac0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:36.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.839+0000 7f7ac1600700 1 -- 172.21.15.131:0/3821746832 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f7ab8014070 con 0x7f7ac4100f90 2024-05-08T22:14:36.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.839+0000 7f7adeeaab80 1 Processor -- start 2024-05-08T22:14:36.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.839+0000 7f7ac3e00700 1 --2- 172.21.15.131:0/3821746832 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7aac0a1290 0x7f7aac0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:36.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.839+0000 7f7adeeaab80 1 -- start start 2024-05-08T22:14:36.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:36 smithi132 ceph-mon[78180]: pgmap v224: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.7 MiB/s wr, 1.19k op/s 2024-05-08T22:14:36.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.839+0000 7f7adeeaab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c88a3050 0x5574c88a3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:36.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.839+0000 7f7adeeaab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574c88a3970 0x5574c889d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:36.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.839+0000 7f7adeeaab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574c889d890 0x5574c889dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:36.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.839+0000 7f7adeeaab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c87f4150 con 0x5574c889d890 2024-05-08T22:14:36.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.839+0000 7f7adeeaab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c87f1b10 con 0x5574c88a3050 2024-05-08T22:14:36.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.839+0000 7f7adeeaab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c87ed400 con 0x5574c88a3970 2024-05-08T22:14:36.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.840+0000 7f7ac3e00700 1 --2- 172.21.15.131:0/3821746832 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7aac0a1290 0x7f7aac0a3750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7ac419aaa0 tx=0x7f7ab0009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:36.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.840+0000 7f7ac8a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c88a3050 0x5574c88a3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:36.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.840+0000 7f7ac8a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c88a3050 0x5574c88a3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:57618/0 (socket says 172.21.15.131:57618) 2024-05-08T22:14:36.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.840+0000 7f7ac8a00700 1 -- 172.21.15.131:0/4122330378 learned_addr learned my addr 172.21.15.131:0/4122330378 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:36.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.840+0000 7f7ac3e00700 1 --2- 172.21.15.131:0/4122330378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574c88a3970 0x5574c889d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:36.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.840+0000 7f7ac9400700 1 --2- 172.21.15.131:0/4122330378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574c889d890 0x5574c889dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:36.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.840+0000 7f7ac8a00700 1 -- 172.21.15.131:0/4122330378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574c88a3970 msgr2=0x5574c889d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:36.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.840+0000 7f7ac8a00700 1 --2- 172.21.15.131:0/4122330378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574c88a3970 0x5574c889d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.840+0000 7f7ac8a00700 1 -- 172.21.15.131:0/4122330378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574c889d890 msgr2=0x5574c889dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:36.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.840+0000 7f7ac8a00700 1 --2- 172.21.15.131:0/4122330378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574c889d890 0x5574c889dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.840+0000 7f7ac8a00700 1 -- 172.21.15.131:0/4122330378 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574c866e3b0 con 0x5574c88a3050 2024-05-08T22:14:36.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.840+0000 7f7ac8a00700 1 --2- 172.21.15.131:0/4122330378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c88a3050 0x5574c88a3430 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8005cb0 tx=0x7f7ab8099a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d38ad2b930002c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:36.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.840+0000 7f7ab5c00700 1 -- 172.21.15.131:0/4122330378 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab80b1070 con 0x5574c88a3050 2024-05-08T22:14:36.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.840+0000 7f7ab5c00700 1 -- 172.21.15.131:0/4122330378 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ab809deb0 con 0x5574c88a3050 2024-05-08T22:14:36.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.840+0000 7f7ab5c00700 1 -- 172.21.15.131:0/4122330378 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab809d070 con 0x5574c88a3050 2024-05-08T22:14:36.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.841+0000 7f7adeeaab80 1 -- 172.21.15.131:0/4122330378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c88a3050 msgr2=0x5574c88a3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:36.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.841+0000 7f7adeeaab80 1 --2- 172.21.15.131:0/4122330378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c88a3050 0x5574c88a3430 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8005cb0 tx=0x7f7ab8099a60 comp rx=0 tx=0).stop 2024-05-08T22:14:36.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.841+0000 7f7adeeaab80 1 -- 172.21.15.131:0/4122330378 shutdown_connections 2024-05-08T22:14:36.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.841+0000 7f7adeeaab80 1 --2- 172.21.15.131:0/4122330378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574c889d890 0x5574c889dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.841+0000 7f7adeeaab80 1 --2- 172.21.15.131:0/4122330378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c88a3050 0x5574c88a3430 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.841+0000 7f7adeeaab80 1 --2- 172.21.15.131:0/4122330378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574c88a3970 0x5574c889d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.841+0000 7f7adeeaab80 1 -- 172.21.15.131:0/4122330378 >> 172.21.15.131:0/4122330378 conn(0x5574c87fb210 msgr2=0x5574c88982d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:36.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.841+0000 7f7adeeaab80 1 -- 172.21.15.131:0/4122330378 shutdown_connections 2024-05-08T22:14:36.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.841+0000 7f7adeeaab80 1 -- 172.21.15.131:0/4122330378 wait complete. 2024-05-08T22:14:36.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.842+0000 7f7adeeaab80 1 -- start start 2024-05-08T22:14:36.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.842+0000 7f7adeeaab80 1 Processor -- start 2024-05-08T22:14:36.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.842+0000 7f7adeeaab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574c889d890 0x5574c88a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:36.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.842+0000 7f7adeeaab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574c88a3050 0x5574c88a6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:36.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.842+0000 7f7adeeaab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c88a3970 0x5574c88a73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:36.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.842+0000 7f7adeeaab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c8789560 con 0x5574c889d890 2024-05-08T22:14:36.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.843+0000 7f7adeeaab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c8789200 con 0x5574c88a3970 2024-05-08T22:14:36.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.843+0000 7f7adeeaab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c86706a0 con 0x5574c88a3050 2024-05-08T22:14:36.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.843+0000 7f7adeeaab80 10 client.0 _ll_register_callbacks cb 0x5574c8792b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:14:36.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.843+0000 7f7ac9400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c88a3970 0x5574c88a73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:36.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.843+0000 7f7ac8a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574c889d890 0x5574c88a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:36.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.843+0000 7f7ac9400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c88a3970 0x5574c88a73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:57628/0 (socket says 172.21.15.131:57628) 2024-05-08T22:14:36.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.843+0000 7f7ac9400700 1 -- 172.21.15.131:0/4134304141 learned_addr learned my addr 172.21.15.131:0/4134304141 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:36.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.843+0000 7f7ac3e00700 1 --2- 172.21.15.131:0/4134304141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574c88a3050 0x5574c88a6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:36.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.843+0000 7f7ac3e00700 1 -- 172.21.15.131:0/4134304141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c88a3970 msgr2=0x5574c88a73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:36.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.843+0000 7f7ac3e00700 1 --2- 172.21.15.131:0/4134304141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c88a3970 0x5574c88a73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.843+0000 7f7ac3e00700 1 -- 172.21.15.131:0/4134304141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574c889d890 msgr2=0x5574c88a99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:36.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.843+0000 7f7ac3e00700 1 --2- 172.21.15.131:0/4134304141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574c889d890 0x5574c88a99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.843+0000 7f7ac3e00700 1 -- 172.21.15.131:0/4134304141 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac41a64e0 con 0x5574c88a3050 2024-05-08T22:14:36.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.843+0000 7f7ac9400700 1 --2- 172.21.15.131:0/4134304141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c88a3970 0x5574c88a73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:14:36.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.843+0000 7f7ac8a00700 1 --2- 172.21.15.131:0/4134304141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574c889d890 0x5574c88a99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:14:36.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.843+0000 7f7ac3e00700 1 --2- 172.21.15.131:0/4134304141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574c88a3050 0x5574c88a6ca0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5574c889e5b0 tx=0x7f7ab00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebe7edd65fc3479f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:36.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.843+0000 7f7aaa000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:14:36.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.844+0000 7f7aaa000700 1 -- 172.21.15.131:0/4134304141 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab001e800 con 0x5574c88a3050 2024-05-08T22:14:36.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.844+0000 7f7adeeaab80 1 -- 172.21.15.131:0/4134304141 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574c8746270 con 0x5574c88a3050 2024-05-08T22:14:36.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.844+0000 7f7aaa000700 1 -- 172.21.15.131:0/4134304141 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f7ab001e960 con 0x5574c88a3050 2024-05-08T22:14:36.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.844+0000 7f7aaa000700 1 -- 172.21.15.131:0/4134304141 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ab001eb20 con 0x5574c88a3050 2024-05-08T22:14:36.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.844+0000 7f7adeeaab80 10 client.26341 fetch_fsmap learned FSMap version 27 2024-05-08T22:14:36.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.844+0000 7f7adeeaab80 1 -- 172.21.15.131:0/4134304141 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5574c861ab60 con 0x5574c88a3050 2024-05-08T22:14:36.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.844+0000 7f7aaa000700 1 -- 172.21.15.131:0/4134304141 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5574c8746270 con 0x5574c88a3050 2024-05-08T22:14:36.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.845+0000 7f7aaa000700 1 -- 172.21.15.131:0/4134304141 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7ab0028ce0 con 0x5574c88a3050 2024-05-08T22:14:36.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.845+0000 7f7adeeaab80 10 client.26341 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T22:14:36.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.845+0000 7f7adeeaab80 10 client.26341 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:14:36.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.845+0000 7f7adeeaab80 20 client.26341 populate_metadata read hostname 'smithi131' 2024-05-08T22:14:36.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.845+0000 7f7adeeaab80 1 --2- 172.21.15.131:0/4134304141 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x5574c88b7330 0x5574c88b9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:36.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.845+0000 7f7adeeaab80 4 client.26341 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:14:36.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.845+0000 7f7adeeaab80 1 -- 172.21.15.131:0/4134304141 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574c886c810 con 0x5574c88b7330 2024-05-08T22:14:36.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.845+0000 7f7ac8a00700 1 --2- 172.21.15.131:0/4134304141 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x5574c88b7330 0x5574c88b9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:36.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.845+0000 7f7aaa000700 10 client.26341 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:14:36.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.845+0000 7f7ac8a00700 1 --2- 172.21.15.131:0/4134304141 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x5574c88b7330 0x5574c88b9750 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=16d3e1ee05dc71dd server_cookie=224882bd03de2ec in_seq=0 out_seq=0 2024-05-08T22:14:36.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.846+0000 7f7aaa000700 1 -- 172.21.15.131:0/4134304141 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5574c886c810 con 0x5574c88b7330 2024-05-08T22:14:36.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.846+0000 7f7aaa000700 10 client.26341 handle_command_reply: tid=0 2024-05-08T22:14:36.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.846+0000 7f7adeeaab80 1 client.26341 shutdown 2024-05-08T22:14:36.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.846+0000 7f7adeeaab80 1 -- 172.21.15.131:0/4134304141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574c88a3050 msgr2=0x5574c88a6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:36.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.846+0000 7f7adeeaab80 1 --2- 172.21.15.131:0/4134304141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574c88a3050 0x5574c88a6ca0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5574c889e5b0 tx=0x7f7ab00047c0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.846+0000 7f7adeeaab80 1 -- 172.21.15.131:0/4134304141 shutdown_connections 2024-05-08T22:14:36.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.846+0000 7f7adeeaab80 1 --2- 172.21.15.131:0/4134304141 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x5574c88b7330 0x5574c88b9750 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.846+0000 7f7adeeaab80 1 --2- 172.21.15.131:0/4134304141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c88a3970 0x5574c88a73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.846+0000 7f7aaa000700 0 client.26341 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:14:36.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.846+0000 7f7adeeaab80 1 --2- 172.21.15.131:0/4134304141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574c889d890 0x5574c88a99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.846+0000 7f7adeeaab80 1 --2- 172.21.15.131:0/4134304141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574c88a3050 0x5574c88a6ca0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.846+0000 7f7adeeaab80 1 -- 172.21.15.131:0/4134304141 >> 172.21.15.131:0/4134304141 conn(0x5574c87fb210 msgr2=0x5574c88980b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:36.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.847+0000 7f7adeeaab80 1 -- 172.21.15.131:0/4134304141 shutdown_connections 2024-05-08T22:14:36.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.847+0000 7f7adeeaab80 1 -- 172.21.15.131:0/4134304141 wait complete. 2024-05-08T22:14:36.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.847+0000 7f7adeeaab80 20 client.26341 trim_cache size 0 max 16384 2024-05-08T22:14:36.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.860+0000 7f7adeeaab80 1 Processor -- start 2024-05-08T22:14:36.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.860+0000 7f7adeeaab80 1 -- start start 2024-05-08T22:14:36.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.861+0000 7f7adeeaab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c889d890 0x5574c889d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:36.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.861+0000 7f7adeeaab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574c88a3050 0x5574c88b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:36.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.861+0000 7f7adeeaab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574c88a3970 0x5574c88977b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:36.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.861+0000 7f7adeeaab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c8897cf0 con 0x5574c88a3970 2024-05-08T22:14:36.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.861+0000 7f7adeeaab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c8897e60 con 0x5574c889d890 2024-05-08T22:14:36.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.861+0000 7f7adeeaab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c88980b0 con 0x5574c88a3050 2024-05-08T22:14:36.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.861+0000 7f7ac8a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c889d890 0x5574c889d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:36.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.861+0000 7f7ac9400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574c88a3970 0x5574c88977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:36.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.861+0000 7f7ac8a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c889d890 0x5574c889d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:57640/0 (socket says 172.21.15.131:57640) 2024-05-08T22:14:36.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.861+0000 7f7ac8a00700 1 -- 172.21.15.131:0/1004426436 learned_addr learned my addr 172.21.15.131:0/1004426436 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:36.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.861+0000 7f7ac3e00700 1 --2- 172.21.15.131:0/1004426436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574c88a3050 0x5574c88b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:36.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.861+0000 7f7ac8a00700 1 -- 172.21.15.131:0/1004426436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574c88a3050 msgr2=0x5574c88b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:36.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.861+0000 7f7ac8a00700 1 --2- 172.21.15.131:0/1004426436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574c88a3050 0x5574c88b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.861+0000 7f7ac8a00700 1 -- 172.21.15.131:0/1004426436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574c88a3970 msgr2=0x5574c88977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:36.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.861+0000 7f7ac8a00700 1 --2- 172.21.15.131:0/1004426436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574c88a3970 0x5574c88977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.861+0000 7f7ac8a00700 1 -- 172.21.15.131:0/1004426436 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574c87b3560 con 0x5574c889d890 2024-05-08T22:14:36.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.861+0000 7f7ac3e00700 1 --2- 172.21.15.131:0/1004426436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574c88a3050 0x5574c88b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:14:36.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.862+0000 7f7ac8a00700 1 --2- 172.21.15.131:0/1004426436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c889d890 0x5574c889d350 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7ab809a010 tx=0x7f7ab8005c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea842dabed97fa69 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:36.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.862+0000 7f7ab6600700 1 -- 172.21.15.131:0/1004426436 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab809d070 con 0x5574c889d890 2024-05-08T22:14:36.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.862+0000 7f7ab6600700 1 -- 172.21.15.131:0/1004426436 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ab80cd1e0 con 0x5574c889d890 2024-05-08T22:14:36.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.862+0000 7f7ab6600700 1 -- 172.21.15.131:0/1004426436 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab80b1040 con 0x5574c889d890 2024-05-08T22:14:36.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.862+0000 7f7adeeaab80 1 -- 172.21.15.131:0/1004426436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c889d890 msgr2=0x5574c889d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:36.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.862+0000 7f7adeeaab80 1 --2- 172.21.15.131:0/1004426436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c889d890 0x5574c889d350 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7ab809a010 tx=0x7f7ab8005c40 comp rx=0 tx=0).stop 2024-05-08T22:14:36.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.862+0000 7f7adeeaab80 1 -- 172.21.15.131:0/1004426436 shutdown_connections 2024-05-08T22:14:36.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.862+0000 7f7adeeaab80 1 --2- 172.21.15.131:0/1004426436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574c88a3970 0x5574c88977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.862+0000 7f7adeeaab80 1 --2- 172.21.15.131:0/1004426436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c889d890 0x5574c889d350 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.862+0000 7f7adeeaab80 1 --2- 172.21.15.131:0/1004426436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574c88a3050 0x5574c88b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.862+0000 7f7adeeaab80 1 -- 172.21.15.131:0/1004426436 >> 172.21.15.131:0/1004426436 conn(0x5574c87fb210 msgr2=0x5574c8899820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:36.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.862+0000 7f7adeeaab80 1 -- 172.21.15.131:0/1004426436 shutdown_connections 2024-05-08T22:14:36.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.863+0000 7f7adeeaab80 1 -- 172.21.15.131:0/1004426436 wait complete. 2024-05-08T22:14:36.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.863+0000 7f7adeeaab80 1 -- start start 2024-05-08T22:14:36.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.863+0000 7f7adeeaab80 1 Processor -- start 2024-05-08T22:14:36.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.864+0000 7f7adeeaab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574c889d890 0x5574c88ae810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:36.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.864+0000 7f7adeeaab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c88a3050 0x5574c88ac310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:36.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.864+0000 7f7adeeaab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574c88a3970 0x5574c88aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:36.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.864+0000 7f7adeeaab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c88ad160 con 0x5574c88a3970 2024-05-08T22:14:36.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.864+0000 7f7adeeaab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c890c830 con 0x5574c88a3050 2024-05-08T22:14:36.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.864+0000 7f7adeeaab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c890c970 con 0x5574c889d890 2024-05-08T22:14:36.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.864+0000 7f7adeeaab80 10 client.0 _ll_register_callbacks cb 0x5574c8792b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:14:36.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.864+0000 7f7ac3e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c88a3050 0x5574c88ac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:36.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.864+0000 7f7ac9400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574c88a3970 0x5574c88aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:36.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.864+0000 7f7ac3e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c88a3050 0x5574c88ac310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:57656/0 (socket says 172.21.15.131:57656) 2024-05-08T22:14:36.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.864+0000 7f7ac3e00700 1 -- 172.21.15.131:0/3840581913 learned_addr learned my addr 172.21.15.131:0/3840581913 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:36.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.864+0000 7f7ac8a00700 1 --2- 172.21.15.131:0/3840581913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574c889d890 0x5574c88ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:36.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.864+0000 7f7ac3e00700 1 -- 172.21.15.131:0/3840581913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574c889d890 msgr2=0x5574c88ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:36.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.864+0000 7f7ac3e00700 1 --2- 172.21.15.131:0/3840581913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574c889d890 0x5574c88ae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.864+0000 7f7ac3e00700 1 -- 172.21.15.131:0/3840581913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574c88a3970 msgr2=0x5574c88aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:36.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.864+0000 7f7ac3e00700 1 --2- 172.21.15.131:0/3840581913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574c88a3970 0x5574c88aca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.864+0000 7f7ac3e00700 1 -- 172.21.15.131:0/3840581913 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574c866e3b0 con 0x5574c88a3050 2024-05-08T22:14:36.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.865+0000 7f7ac3e00700 1 --2- 172.21.15.131:0/3840581913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c88a3050 0x5574c88ac310 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x5574c889e5b0 tx=0x7f7ab0003ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=248e89fe01d18638 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:36.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.865+0000 7f7aaa000700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:14:36.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.865+0000 7f7aaa000700 1 -- 172.21.15.131:0/3840581913 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab000e640 con 0x5574c88a3050 2024-05-08T22:14:36.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.865+0000 7f7aaa000700 1 -- 172.21.15.131:0/3840581913 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f7ab000ebf0 con 0x5574c88a3050 2024-05-08T22:14:36.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.865+0000 7f7adeeaab80 1 -- 172.21.15.131:0/3840581913 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574c87e5ec0 con 0x5574c88a3050 2024-05-08T22:14:36.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.865+0000 7f7aaa000700 1 -- 172.21.15.131:0/3840581913 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ab000edb0 con 0x5574c88a3050 2024-05-08T22:14:36.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.865+0000 7f7adeeaab80 10 client.26291 fetch_fsmap learned FSMap version 27 2024-05-08T22:14:36.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.865+0000 7f7adeeaab80 1 -- 172.21.15.131:0/3840581913 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5574c8672cd0 con 0x5574c88a3050 2024-05-08T22:14:36.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.865+0000 7f7aaa000700 1 -- 172.21.15.131:0/3840581913 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5574c87e5ec0 con 0x5574c88a3050 2024-05-08T22:14:36.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.866+0000 7f7aaa000700 1 -- 172.21.15.131:0/3840581913 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 27) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7ab001a5e0 con 0x5574c88a3050 2024-05-08T22:14:36.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.866+0000 7f7adeeaab80 10 client.26291 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T22:14:36.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.866+0000 7f7adeeaab80 10 client.26291 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:14:36.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.866+0000 7f7adeeaab80 20 client.26291 populate_metadata read hostname 'smithi131' 2024-05-08T22:14:36.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.866+0000 7f7adeeaab80 1 --2- 172.21.15.131:0/3840581913 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x5574c890dfd0 0x5574c89103f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:36.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.866+0000 7f7adeeaab80 4 client.26291 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:14:36.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.866+0000 7f7adeeaab80 1 -- 172.21.15.131:0/3840581913 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5574c863bcf0 con 0x5574c890dfd0 2024-05-08T22:14:36.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.866+0000 7f7ac8a00700 1 --2- 172.21.15.131:0/3840581913 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x5574c890dfd0 0x5574c89103f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:36.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.866+0000 7f7ac8a00700 1 --2- 172.21.15.131:0/3840581913 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x5574c890dfd0 0x5574c89103f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=34bc8518369abbfa server_cookie=b5877686cad21f6b in_seq=0 out_seq=0 2024-05-08T22:14:36.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.866+0000 7f7aaa000700 10 client.26291 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:14:36.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.867+0000 7f7aaa000700 1 -- 172.21.15.131:0/3840581913 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5574c863bcf0 con 0x5574c890dfd0 2024-05-08T22:14:36.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.867+0000 7f7aaa000700 10 client.26291 handle_command_reply: tid=0 2024-05-08T22:14:36.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.867+0000 7f7adeeaab80 1 client.26291 shutdown 2024-05-08T22:14:36.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.867+0000 7f7adeeaab80 1 -- 172.21.15.131:0/3840581913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c88a3050 msgr2=0x5574c88ac310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:36.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.867+0000 7f7adeeaab80 1 --2- 172.21.15.131:0/3840581913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c88a3050 0x5574c88ac310 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x5574c889e5b0 tx=0x7f7ab0003ec0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.867+0000 7f7adeeaab80 1 -- 172.21.15.131:0/3840581913 shutdown_connections 2024-05-08T22:14:36.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.867+0000 7f7adeeaab80 1 --2- 172.21.15.131:0/3840581913 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x5574c890dfd0 0x5574c89103f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.867+0000 7f7adeeaab80 1 --2- 172.21.15.131:0/3840581913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574c88a3970 0x5574c88aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.867+0000 7f7adeeaab80 1 --2- 172.21.15.131:0/3840581913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574c889d890 0x5574c88ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.867+0000 7f7aaa000700 0 client.26291 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:14:36.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.867+0000 7f7adeeaab80 1 --2- 172.21.15.131:0/3840581913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574c88a3050 0x5574c88ac310 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.867+0000 7f7adeeaab80 1 -- 172.21.15.131:0/3840581913 >> 172.21.15.131:0/3840581913 conn(0x5574c87fb210 msgr2=0x5574c88982d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:36.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.867+0000 7f7adeeaab80 1 -- 172.21.15.131:0/3840581913 shutdown_connections 2024-05-08T22:14:36.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.867+0000 7f7adeeaab80 1 -- 172.21.15.131:0/3840581913 wait complete. 2024-05-08T22:14:36.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.867+0000 7f7adeeaab80 20 client.26291 trim_cache size 0 max 16384 2024-05-08T22:14:36.879 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:14:36.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.868+0000 7f7ac9e00700 1 -- 172.21.15.131:0/3821746832 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7aac0a1290 msgr2=0x7f7aac0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:36.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.868+0000 7f7ac9e00700 1 --2- 172.21.15.131:0/3821746832 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7aac0a1290 0x7f7aac0a3750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7ac419aaa0 tx=0x7f7ab0009260 comp rx=0 tx=0).stop 2024-05-08T22:14:36.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.868+0000 7f7ac9e00700 1 -- 172.21.15.131:0/3821746832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ac4100f90 msgr2=0x7f7ac41995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:36.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.868+0000 7f7ac9e00700 1 --2- 172.21.15.131:0/3821746832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ac4100f90 0x7f7ac41995a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800e9f0 tx=0x7f7ab800ed00 comp rx=0 tx=0).stop 2024-05-08T22:14:36.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.868+0000 7f7ac9e00700 1 -- 172.21.15.131:0/3821746832 shutdown_connections 2024-05-08T22:14:36.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.868+0000 7f7ac9e00700 1 --2- 172.21.15.131:0/3821746832 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7aac0a1290 0x7f7aac0a3750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.868+0000 7f7ac9e00700 1 --2- 172.21.15.131:0/3821746832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ac419de30 0x7f7ac419e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.868+0000 7f7ac9e00700 1 --2- 172.21.15.131:0/3821746832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ac4100f90 0x7f7ac41995a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.868+0000 7f7ac9e00700 1 --2- 172.21.15.131:0/3821746832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ac41060b0 0x7f7ac4199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:36.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.868+0000 7f7ac9e00700 1 -- 172.21.15.131:0/3821746832 >> 172.21.15.131:0/3821746832 conn(0x7f7ac40fc810 msgr2=0x7f7ac40feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:36.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.868+0000 7f7ac9e00700 1 -- 172.21.15.131:0/3821746832 shutdown_connections 2024-05-08T22:14:36.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:36.868+0000 7f7ac9e00700 1 -- 172.21.15.131:0/3821746832 wait complete. 2024-05-08T22:14:36.886 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:36 smithi131 ceph-mon[37260]: pgmap v224: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.7 MiB/s wr, 1.19k op/s 2024-05-08T22:14:36.888 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T22:14:37.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.188+0000 7fadde400700 1 -- 172.21.15.131:0/3396647413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadd810fb10 msgr2=0x7fadd81084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:37.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.188+0000 7fadde400700 1 --2- 172.21.15.131:0/3396647413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadd810fb10 0x7fadd81084e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fadc00098f0 tx=0x7fadc0009c00 comp rx=0 tx=0).stop 2024-05-08T22:14:37.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.189+0000 7fadde400700 1 -- 172.21.15.131:0/3396647413 shutdown_connections 2024-05-08T22:14:37.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.189+0000 7fadde400700 1 --2- 172.21.15.131:0/3396647413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadd811a780 0x7fadd811cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.189+0000 7fadde400700 1 --2- 172.21.15.131:0/3396647413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadd810f6a0 0x7fadd8107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.189+0000 7fadde400700 1 --2- 172.21.15.131:0/3396647413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadd810fb10 0x7fadd81084e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.189+0000 7fadde400700 1 -- 172.21.15.131:0/3396647413 >> 172.21.15.131:0/3396647413 conn(0x7fadd806bff0 msgr2=0x7fadd806c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:37.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.191+0000 7fadde400700 1 -- 172.21.15.131:0/3396647413 shutdown_connections 2024-05-08T22:14:37.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.191+0000 7fadde400700 1 -- 172.21.15.131:0/3396647413 wait complete. 2024-05-08T22:14:37.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.193+0000 7fadde400700 1 Processor -- start 2024-05-08T22:14:37.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.193+0000 7fadde400700 1 -- start start 2024-05-08T22:14:37.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.194+0000 7fadde400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadd810f6a0 0x7fadd81a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.194+0000 7fadde400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadd810fb10 0x7fadd81a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.194+0000 7fadde400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadd811a780 0x7fadd81aab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.194+0000 7fadde400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd81ab180 con 0x7fadd811a780 2024-05-08T22:14:37.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.194+0000 7fadde400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd81ab2f0 con 0x7fadd810fb10 2024-05-08T22:14:37.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.194+0000 7fadde400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd81ab4b0 con 0x7fadd810f6a0 2024-05-08T22:14:37.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.195+0000 7fadd7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadd810f6a0 0x7fadd81a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.195+0000 7fadd7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadd810f6a0 0x7fadd81a6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34354/0 (socket says 172.21.15.131:34354) 2024-05-08T22:14:37.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.195+0000 7fadd7400700 1 -- 172.21.15.131:0/1799480736 learned_addr learned my addr 172.21.15.131:0/1799480736 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:37.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.195+0000 7fadd7e00700 1 --2- 172.21.15.131:0/1799480736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadd811a780 0x7fadd81aab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.195+0000 7fadd6a00700 1 --2- 172.21.15.131:0/1799480736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadd810fb10 0x7fadd81a67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.195+0000 7fadd7e00700 1 -- 172.21.15.131:0/1799480736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadd810f6a0 msgr2=0x7fadd81a6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:37.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.195+0000 7fadd7e00700 1 --2- 172.21.15.131:0/1799480736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadd810f6a0 0x7fadd81a6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.195+0000 7fadd7e00700 1 -- 172.21.15.131:0/1799480736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadd810fb10 msgr2=0x7fadd81a67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:37.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.195+0000 7fadd7e00700 1 --2- 172.21.15.131:0/1799480736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadd810fb10 0x7fadd81a67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.195+0000 7fadd7e00700 1 -- 172.21.15.131:0/1799480736 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc0009520 con 0x7fadd811a780 2024-05-08T22:14:37.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.195+0000 7fadd7400700 1 --2- 172.21.15.131:0/1799480736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadd810f6a0 0x7fadd81a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:14:37.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.196+0000 7fadd6a00700 1 --2- 172.21.15.131:0/1799480736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadd810fb10 0x7fadd81a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:14:37.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.196+0000 7fadd7e00700 1 --2- 172.21.15.131:0/1799480736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadd811a780 0x7fadd81aab40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fadc8005fd0 tx=0x7fadc800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:37.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.199+0000 7fadcfe00700 1 -- 172.21.15.131:0/1799480736 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc8004220 con 0x7fadd811a780 2024-05-08T22:14:37.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.199+0000 7fadcfe00700 1 -- 172.21.15.131:0/1799480736 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadc8004cf0 con 0x7fadd811a780 2024-05-08T22:14:37.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.199+0000 7fadde400700 1 -- 172.21.15.131:0/1799480736 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd81ab790 con 0x7fadd811a780 2024-05-08T22:14:37.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.200+0000 7fadde400700 1 -- 172.21.15.131:0/1799480736 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd81b2fd0 con 0x7fadd811a780 2024-05-08T22:14:37.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.200+0000 7fadcfe00700 1 -- 172.21.15.131:0/1799480736 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc80056e0 con 0x7fadd811a780 2024-05-08T22:14:37.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.213+0000 7fadcfe00700 1 -- 172.21.15.131:0/1799480736 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fadc8004380 con 0x7fadd811a780 2024-05-08T22:14:37.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.216+0000 7fadcfe00700 1 --2- 172.21.15.131:0/1799480736 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fadbc0a13d0 0x7fadbc0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.217+0000 7fadcfe00700 1 -- 172.21.15.131:0/1799480736 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fadc8014070 con 0x7fadd811a780 2024-05-08T22:14:37.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.217+0000 7fadd7400700 1 --2- 172.21.15.131:0/1799480736 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fadbc0a13d0 0x7fadbc0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.220+0000 7fadd7400700 1 --2- 172.21.15.131:0/1799480736 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fadbc0a13d0 0x7fadbc0a3890 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fadd000c8c0 tx=0x7fadd0009040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:37.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.221+0000 7fadf1eb7b80 1 Processor -- start 2024-05-08T22:14:37.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.221+0000 7fadf1eb7b80 1 -- start start 2024-05-08T22:14:37.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.223+0000 7fadf1eb7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a25ec35df0 0x55a25ec361d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.223+0000 7fadf1eb7b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a25ec36710 0x55a25ec30360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.223+0000 7fadf1eb7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec308a0 0x55a25ec30d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.223+0000 7fadf1eb7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a25eb76150 con 0x55a25ec35df0 2024-05-08T22:14:37.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.223+0000 7fadf1eb7b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a25eb73b10 con 0x55a25ec36710 2024-05-08T22:14:37.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.223+0000 7fadf1eb7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a25eb6f400 con 0x55a25ec308a0 2024-05-08T22:14:37.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.224+0000 7fadd7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec308a0 0x55a25ec30d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.224+0000 7fadd7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec308a0 0x55a25ec30d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34362/0 (socket says 172.21.15.131:34362) 2024-05-08T22:14:37.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.224+0000 7fadd7e00700 1 -- 172.21.15.131:0/3353645780 learned_addr learned my addr 172.21.15.131:0/3353645780 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:37.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.224+0000 7fadd6a00700 1 --2- 172.21.15.131:0/3353645780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a25ec36710 0x55a25ec30360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.224+0000 7fadd7400700 1 --2- 172.21.15.131:0/3353645780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a25ec35df0 0x55a25ec361d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.225+0000 7fadd7e00700 1 -- 172.21.15.131:0/3353645780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a25ec36710 msgr2=0x55a25ec30360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:37.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.225+0000 7fadd7e00700 1 --2- 172.21.15.131:0/3353645780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a25ec36710 0x55a25ec30360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.225+0000 7fadd7e00700 1 -- 172.21.15.131:0/3353645780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a25ec35df0 msgr2=0x55a25ec361d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:37.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.225+0000 7fadd7e00700 1 --2- 172.21.15.131:0/3353645780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a25ec35df0 0x55a25ec361d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.225+0000 7fadd7e00700 1 -- 172.21.15.131:0/3353645780 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a25e9f03b0 con 0x55a25ec308a0 2024-05-08T22:14:37.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.225+0000 7fadd7400700 1 --2- 172.21.15.131:0/3353645780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a25ec35df0 0x55a25ec361d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:14:37.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.225+0000 7fadd7e00700 1 --2- 172.21.15.131:0/3353645780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec308a0 0x55a25ec30d00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fadc8096340 tx=0x7fadc8099230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bfb74aae363ead8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:37.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.226+0000 7fadccc00700 1 -- 172.21.15.131:0/3353645780 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc80998d0 con 0x55a25ec308a0 2024-05-08T22:14:37.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.226+0000 7fadccc00700 1 -- 172.21.15.131:0/3353645780 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadc809d700 con 0x55a25ec308a0 2024-05-08T22:14:37.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.226+0000 7fadccc00700 1 -- 172.21.15.131:0/3353645780 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc8099b30 con 0x55a25ec308a0 2024-05-08T22:14:37.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.227+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/3353645780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec308a0 msgr2=0x55a25ec30d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:37.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.227+0000 7fadf1eb7b80 1 --2- 172.21.15.131:0/3353645780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec308a0 0x55a25ec30d00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fadc8096340 tx=0x7fadc8099230 comp rx=0 tx=0).stop 2024-05-08T22:14:37.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.228+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/3353645780 shutdown_connections 2024-05-08T22:14:37.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.228+0000 7fadf1eb7b80 1 --2- 172.21.15.131:0/3353645780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec308a0 0x55a25ec30d00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.228+0000 7fadf1eb7b80 1 --2- 172.21.15.131:0/3353645780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a25ec35df0 0x55a25ec361d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.228+0000 7fadf1eb7b80 1 --2- 172.21.15.131:0/3353645780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a25ec36710 0x55a25ec30360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.228+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/3353645780 >> 172.21.15.131:0/3353645780 conn(0x55a25eb7d210 msgr2=0x55a25ec19df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:37.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.228+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/3353645780 shutdown_connections 2024-05-08T22:14:37.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.229+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/3353645780 wait complete. 2024-05-08T22:14:37.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.229+0000 7fadf1eb7b80 1 -- start start 2024-05-08T22:14:37.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.230+0000 7fadf1eb7b80 1 Processor -- start 2024-05-08T22:14:37.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.232+0000 7fadf1eb7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec308a0 0x55a25ec39a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.232+0000 7fadf1eb7b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a25ec35df0 0x55a25ec39fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.232+0000 7fadf1eb7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a25ec36710 0x55a25ec3a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.232+0000 7fadf1eb7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a25eb0b560 con 0x55a25ec36710 2024-05-08T22:14:37.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.232+0000 7fadf1eb7b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a25eb0b200 con 0x55a25ec35df0 2024-05-08T22:14:37.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.232+0000 7fadf1eb7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a25e9f26a0 con 0x55a25ec308a0 2024-05-08T22:14:37.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.232+0000 7fadf1eb7b80 10 client.0 _ll_register_callbacks cb 0x55a25eb14b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:14:37.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.233+0000 7fadd7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec308a0 0x55a25ec39a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.233+0000 7fadd7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec308a0 0x55a25ec39a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34368/0 (socket says 172.21.15.131:34368) 2024-05-08T22:14:37.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.233+0000 7fadd7400700 1 -- 172.21.15.131:0/3942084773 learned_addr learned my addr 172.21.15.131:0/3942084773 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:37.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.233+0000 7fadd6a00700 1 --2- 172.21.15.131:0/3942084773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a25ec35df0 0x55a25ec39fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.234+0000 7fadd7e00700 1 --2- 172.21.15.131:0/3942084773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a25ec36710 0x55a25ec3a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.235+0000 7fadd7e00700 1 -- 172.21.15.131:0/3942084773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec308a0 msgr2=0x55a25ec39a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:37.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.235+0000 7fadd7e00700 1 --2- 172.21.15.131:0/3942084773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec308a0 0x55a25ec39a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.235+0000 7fadd7e00700 1 -- 172.21.15.131:0/3942084773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a25ec35df0 msgr2=0x55a25ec39fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:37.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.235+0000 7fadd7e00700 1 --2- 172.21.15.131:0/3942084773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a25ec35df0 0x55a25ec39fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.235+0000 7fadd7e00700 1 -- 172.21.15.131:0/3942084773 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd81b2fd0 con 0x55a25ec36710 2024-05-08T22:14:37.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.237+0000 7fadd7e00700 1 --2- 172.21.15.131:0/3942084773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a25ec36710 0x55a25ec3a510 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fadc809c000 tx=0x7fadc809a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d376fad582c32b9d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:37.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.237+0000 7fadc5600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:14:37.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.238+0000 7fadc5600700 1 -- 172.21.15.131:0/3942084773 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc8099d30 con 0x55a25ec36710 2024-05-08T22:14:37.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.238+0000 7fadc5600700 1 -- 172.21.15.131:0/3942084773 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fadc80998d0 con 0x55a25ec36710 2024-05-08T22:14:37.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.238+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/3942084773 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a25eac8270 con 0x55a25ec36710 2024-05-08T22:14:37.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.238+0000 7fadc5600700 1 -- 172.21.15.131:0/3942084773 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadc80a18c0 con 0x55a25ec36710 2024-05-08T22:14:37.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.238+0000 7fadf1eb7b80 10 client.16275 fetch_fsmap learned FSMap version 27 2024-05-08T22:14:37.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.238+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/3942084773 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a25e99cb60 con 0x55a25ec36710 2024-05-08T22:14:37.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.239+0000 7fadc5600700 1 -- 172.21.15.131:0/3942084773 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a25eac8270 con 0x55a25ec36710 2024-05-08T22:14:37.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.239+0000 7fadc5600700 1 -- 172.21.15.131:0/3942084773 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fadc809d170 con 0x55a25ec36710 2024-05-08T22:14:37.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.239+0000 7fadf1eb7b80 10 client.16275 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T22:14:37.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.239+0000 7fadf1eb7b80 10 client.16275 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:14:37.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.239+0000 7fadf1eb7b80 20 client.16275 populate_metadata read hostname 'smithi131' 2024-05-08T22:14:37.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.239+0000 7fadf1eb7b80 1 --2- 172.21.15.131:0/3942084773 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55a25ec49b50 0x55a25ec4bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.239+0000 7fadf1eb7b80 4 client.16275 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:14:37.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.239+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/3942084773 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a25ebee810 con 0x55a25ec49b50 2024-05-08T22:14:37.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.240+0000 7fadd7400700 1 --2- 172.21.15.131:0/3942084773 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55a25ec49b50 0x55a25ec4bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.241+0000 7fadd7400700 1 --2- 172.21.15.131:0/3942084773 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55a25ec49b50 0x55a25ec4bf40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3248bf5742f1ccac server_cookie=e7c1bbd0a11fbba1 in_seq=0 out_seq=0 2024-05-08T22:14:37.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.242+0000 7fadc5600700 10 client.16275 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:14:37.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.243+0000 7fadc5600700 1 -- 172.21.15.131:0/3942084773 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a25ebee810 con 0x55a25ec49b50 2024-05-08T22:14:37.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.243+0000 7fadc5600700 10 client.16275 handle_command_reply: tid=0 2024-05-08T22:14:37.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.244+0000 7fadf1eb7b80 1 client.16275 shutdown 2024-05-08T22:14:37.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.244+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/3942084773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a25ec36710 msgr2=0x55a25ec3a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:37.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.244+0000 7fadf1eb7b80 1 --2- 172.21.15.131:0/3942084773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a25ec36710 0x55a25ec3a510 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fadc809c000 tx=0x7fadc809a0d0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.245+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/3942084773 shutdown_connections 2024-05-08T22:14:37.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.245+0000 7fadf1eb7b80 1 --2- 172.21.15.131:0/3942084773 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55a25ec49b50 0x55a25ec4bf40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.246+0000 7fadf1eb7b80 1 --2- 172.21.15.131:0/3942084773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a25ec36710 0x55a25ec3a510 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.246+0000 7fadf1eb7b80 1 --2- 172.21.15.131:0/3942084773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec308a0 0x55a25ec39a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.246+0000 7fadf1eb7b80 1 --2- 172.21.15.131:0/3942084773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a25ec35df0 0x55a25ec39fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.246+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/3942084773 >> 172.21.15.131:0/3942084773 conn(0x55a25eb7d210 msgr2=0x55a25ec195a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:37.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.246+0000 7fadc5600700 0 client.16275 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:14:37.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.246+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/3942084773 shutdown_connections 2024-05-08T22:14:37.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.246+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/3942084773 wait complete. 2024-05-08T22:14:37.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.246+0000 7fadf1eb7b80 20 client.16275 trim_cache size 0 max 16384 2024-05-08T22:14:37.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.273+0000 7fadf1eb7b80 1 Processor -- start 2024-05-08T22:14:37.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.274+0000 7fadf1eb7b80 1 -- start start 2024-05-08T22:14:37.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.274+0000 7fadf1eb7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a25ec308a0 0x55a25ec35440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.274+0000 7fadf1eb7b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a25ec35df0 0x55a25ec19d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.274+0000 7fadf1eb7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec36710 0x55a25ec38a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.274+0000 7fadf1eb7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a25ec3ae20 con 0x55a25ec308a0 2024-05-08T22:14:37.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.274+0000 7fadf1eb7b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a25ec1b870 con 0x55a25ec35df0 2024-05-08T22:14:37.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.274+0000 7fadf1eb7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a25ec16df0 con 0x55a25ec36710 2024-05-08T22:14:37.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.274+0000 7fadd7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec36710 0x55a25ec38a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.274+0000 7fadd7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec36710 0x55a25ec38a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34370/0 (socket says 172.21.15.131:34370) 2024-05-08T22:14:37.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.274+0000 7fadd7e00700 1 -- 172.21.15.131:0/2500271520 learned_addr learned my addr 172.21.15.131:0/2500271520 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:37.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.275+0000 7fadd6a00700 1 --2- 172.21.15.131:0/2500271520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a25ec35df0 0x55a25ec19d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.275+0000 7fadd7e00700 1 -- 172.21.15.131:0/2500271520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a25ec35df0 msgr2=0x55a25ec19d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:37.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.275+0000 7fadd7e00700 1 --2- 172.21.15.131:0/2500271520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a25ec35df0 0x55a25ec19d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.275+0000 7fadd7e00700 1 -- 172.21.15.131:0/2500271520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a25ec308a0 msgr2=0x55a25ec35440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:37.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.275+0000 7fadd7e00700 1 --2- 172.21.15.131:0/2500271520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a25ec308a0 0x55a25ec35440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.275+0000 7fadd7e00700 1 -- 172.21.15.131:0/2500271520 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a25eb35560 con 0x55a25ec36710 2024-05-08T22:14:37.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.276+0000 7fadd7e00700 1 --2- 172.21.15.131:0/2500271520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec36710 0x55a25ec38a70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fadc8097c20 tx=0x7fadc8097e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d6727d449352ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:37.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.276+0000 7fadcd600700 1 -- 172.21.15.131:0/2500271520 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc8098830 con 0x55a25ec36710 2024-05-08T22:14:37.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.276+0000 7fadcd600700 1 -- 172.21.15.131:0/2500271520 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadc8099230 con 0x55a25ec36710 2024-05-08T22:14:37.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.276+0000 7fadcd600700 1 -- 172.21.15.131:0/2500271520 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc80a1b20 con 0x55a25ec36710 2024-05-08T22:14:37.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.278+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/2500271520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec36710 msgr2=0x55a25ec38a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:37.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.278+0000 7fadf1eb7b80 1 --2- 172.21.15.131:0/2500271520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec36710 0x55a25ec38a70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fadc8097c20 tx=0x7fadc8097e80 comp rx=0 tx=0).stop 2024-05-08T22:14:37.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.278+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/2500271520 shutdown_connections 2024-05-08T22:14:37.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.278+0000 7fadf1eb7b80 1 --2- 172.21.15.131:0/2500271520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec36710 0x55a25ec38a70 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.278+0000 7fadf1eb7b80 1 --2- 172.21.15.131:0/2500271520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a25ec308a0 0x55a25ec35440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.278+0000 7fadf1eb7b80 1 --2- 172.21.15.131:0/2500271520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a25ec35df0 0x55a25ec19d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.278+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/2500271520 >> 172.21.15.131:0/2500271520 conn(0x55a25eb7d210 msgr2=0x55a25ec38360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:37.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.278+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/2500271520 shutdown_connections 2024-05-08T22:14:37.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.278+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/2500271520 wait complete. 2024-05-08T22:14:37.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.279+0000 7fadf1eb7b80 1 -- start start 2024-05-08T22:14:37.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.279+0000 7fadf1eb7b80 1 Processor -- start 2024-05-08T22:14:37.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.280+0000 7fadf1eb7b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a25ec308a0 0x55a25ec3f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.280+0000 7fadf1eb7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a25ec35df0 0x55a25ec3f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.280+0000 7fadf1eb7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec36710 0x55a25ec9ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.281+0000 7fadf1eb7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a25ec9f4c0 con 0x55a25ec35df0 2024-05-08T22:14:37.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.281+0000 7fadf1eb7b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a25ec9f600 con 0x55a25ec308a0 2024-05-08T22:14:37.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.281+0000 7fadf1eb7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a25ec9f740 con 0x55a25ec36710 2024-05-08T22:14:37.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.281+0000 7fadd6a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a25ec35df0 0x55a25ec3f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.281+0000 7fadf1eb7b80 10 client.0 _ll_register_callbacks cb 0x55a25eb14b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:14:37.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.281+0000 7fadd7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec36710 0x55a25ec9ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.281+0000 7fadd7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec36710 0x55a25ec9ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34372/0 (socket says 172.21.15.131:34372) 2024-05-08T22:14:37.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.281+0000 7fadd7e00700 1 -- 172.21.15.131:0/2384964491 learned_addr learned my addr 172.21.15.131:0/2384964491 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:37.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.282+0000 7fadd6a00700 1 -- 172.21.15.131:0/2384964491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec36710 msgr2=0x55a25ec9ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:37.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.282+0000 7fadd6a00700 1 --2- 172.21.15.131:0/2384964491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec36710 0x55a25ec9ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.282+0000 7fadd6a00700 1 -- 172.21.15.131:0/2384964491 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a25ec308a0 msgr2=0x55a25ec3f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:37.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.282+0000 7fadd6a00700 1 --2- 172.21.15.131:0/2384964491 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a25ec308a0 0x55a25ec3f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.282+0000 7fadd6a00700 1 -- 172.21.15.131:0/2384964491 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a25e99cb60 con 0x55a25ec35df0 2024-05-08T22:14:37.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.282+0000 7fadc5600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:14:37.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.282+0000 7fadd6a00700 1 --2- 172.21.15.131:0/2384964491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a25ec35df0 0x55a25ec3f840 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fadc0000c00 tx=0x7fadc000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3f33f741e080ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:37.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.283+0000 7fadc5600700 1 -- 172.21.15.131:0/2384964491 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc001c040 con 0x55a25ec35df0 2024-05-08T22:14:37.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.283+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/2384964491 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fadc809d700 con 0x55a25ec35df0 2024-05-08T22:14:37.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.283+0000 7fadc5600700 1 -- 172.21.15.131:0/2384964491 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fadc0005040 con 0x55a25ec35df0 2024-05-08T22:14:37.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.283+0000 7fadc5600700 1 -- 172.21.15.131:0/2384964491 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadc001c040 con 0x55a25ec35df0 2024-05-08T22:14:37.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.283+0000 7fadf1eb7b80 10 client.16278 fetch_fsmap learned FSMap version 27 2024-05-08T22:14:37.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.283+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/2384964491 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a25e9f4cd0 con 0x55a25ec35df0 2024-05-08T22:14:37.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.285+0000 7fadc5600700 1 -- 172.21.15.131:0/2384964491 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadc809d700 con 0x55a25ec35df0 2024-05-08T22:14:37.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.285+0000 7fadc5600700 1 -- 172.21.15.131:0/2384964491 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fadc0022040 con 0x55a25ec35df0 2024-05-08T22:14:37.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.285+0000 7fadf1eb7b80 10 client.16278 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T22:14:37.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.285+0000 7fadf1eb7b80 10 client.16278 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:14:37.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.285+0000 7fadf1eb7b80 20 client.16278 populate_metadata read hostname 'smithi131' 2024-05-08T22:14:37.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.285+0000 7fadf1eb7b80 1 --2- 172.21.15.131:0/2384964491 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55a25eca0df0 0x55a25eca3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.285+0000 7fadf1eb7b80 4 client.16278 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:14:37.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.285+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/2384964491 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a25eb67ec0 con 0x55a25eca0df0 2024-05-08T22:14:37.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.287+0000 7fadd7400700 1 --2- 172.21.15.131:0/2384964491 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55a25eca0df0 0x55a25eca3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.288+0000 7fadd7400700 1 --2- 172.21.15.131:0/2384964491 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55a25eca0df0 0x55a25eca3210 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5e7bdf6cb324c406 server_cookie=5283dd34d4516b8f in_seq=0 out_seq=0 2024-05-08T22:14:37.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.288+0000 7fadc5600700 10 client.16278 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:14:37.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.289+0000 7fadc5600700 1 -- 172.21.15.131:0/2384964491 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a25eb67ec0 con 0x55a25eca0df0 2024-05-08T22:14:37.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.289+0000 7fadc5600700 10 client.16278 handle_command_reply: tid=0 2024-05-08T22:14:37.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.289+0000 7fadf1eb7b80 1 client.16278 shutdown 2024-05-08T22:14:37.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.290+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/2384964491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a25ec35df0 msgr2=0x55a25ec3f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:37.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.290+0000 7fadf1eb7b80 1 --2- 172.21.15.131:0/2384964491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a25ec35df0 0x55a25ec3f840 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fadc0000c00 tx=0x7fadc000b790 comp rx=0 tx=0).stop 2024-05-08T22:14:37.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.290+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/2384964491 shutdown_connections 2024-05-08T22:14:37.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.290+0000 7fadf1eb7b80 1 --2- 172.21.15.131:0/2384964491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a25ec36710 0x55a25ec9ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.290+0000 7fadf1eb7b80 1 --2- 172.21.15.131:0/2384964491 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a25ec308a0 0x55a25ec3f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.290+0000 7fadf1eb7b80 1 --2- 172.21.15.131:0/2384964491 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55a25eca0df0 0x55a25eca3210 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.290+0000 7fadf1eb7b80 1 --2- 172.21.15.131:0/2384964491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a25ec35df0 0x55a25ec3f840 secure :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fadc0000c00 tx=0x7fadc000b790 comp rx=0 tx=0).stop 2024-05-08T22:14:37.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.290+0000 7fadc5600700 0 client.16278 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:14:37.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.290+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/2384964491 >> 172.21.15.131:0/2384964491 conn(0x55a25eb7d210 msgr2=0x55a25ec36af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:37.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.290+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/2384964491 shutdown_connections 2024-05-08T22:14:37.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.291+0000 7fadf1eb7b80 1 -- 172.21.15.131:0/2384964491 wait complete. 2024-05-08T22:14:37.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.291+0000 7fadf1eb7b80 20 client.16278 trim_cache size 0 max 16384 2024-05-08T22:14:37.292 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:14:37.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.292+0000 7fadde400700 1 -- 172.21.15.131:0/1799480736 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fadbc0a13d0 msgr2=0x7fadbc0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:37.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.292+0000 7fadde400700 1 --2- 172.21.15.131:0/1799480736 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fadbc0a13d0 0x7fadbc0a3890 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fadd000c8c0 tx=0x7fadd0009040 comp rx=0 tx=0).stop 2024-05-08T22:14:37.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.292+0000 7fadde400700 1 -- 172.21.15.131:0/1799480736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadd811a780 msgr2=0x7fadd81aab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:37.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.292+0000 7fadde400700 1 --2- 172.21.15.131:0/1799480736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadd811a780 0x7fadd81aab40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fadc8005fd0 tx=0x7fadc800ec20 comp rx=0 tx=0).stop 2024-05-08T22:14:37.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.294+0000 7fadde400700 1 -- 172.21.15.131:0/1799480736 shutdown_connections 2024-05-08T22:14:37.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.294+0000 7fadde400700 1 --2- 172.21.15.131:0/1799480736 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fadbc0a13d0 0x7fadbc0a3890 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.294+0000 7fadde400700 1 --2- 172.21.15.131:0/1799480736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadd811a780 0x7fadd81aab40 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.294+0000 7fadde400700 1 --2- 172.21.15.131:0/1799480736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadd810f6a0 0x7fadd81a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.294+0000 7fadde400700 1 --2- 172.21.15.131:0/1799480736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadd810fb10 0x7fadd81a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.294+0000 7fadde400700 1 -- 172.21.15.131:0/1799480736 >> 172.21.15.131:0/1799480736 conn(0x7fadd806bff0 msgr2=0x7fadd8118770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:37.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.296+0000 7fadde400700 1 -- 172.21.15.131:0/1799480736 shutdown_connections 2024-05-08T22:14:37.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.296+0000 7fadde400700 1 -- 172.21.15.131:0/1799480736 wait complete. 2024-05-08T22:14:37.333 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T22:14:37.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.635+0000 7fc809400700 1 -- 172.21.15.131:0/3918280026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc804100ab0 msgr2=0x7fc80410ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:37.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.635+0000 7fc809400700 1 --2- 172.21.15.131:0/3918280026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc804100ab0 0x7fc80410ccc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00b0a0 tx=0x7fc7fc00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.636+0000 7fc809400700 1 -- 172.21.15.131:0/3918280026 shutdown_connections 2024-05-08T22:14:37.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.636+0000 7fc809400700 1 --2- 172.21.15.131:0/3918280026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc804100ab0 0x7fc80410ccc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.636+0000 7fc809400700 1 --2- 172.21.15.131:0/3918280026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc804105f90 0x7fc804106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.636+0000 7fc809400700 1 --2- 172.21.15.131:0/3918280026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8041000f0 0x7fc804100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.636+0000 7fc809400700 1 -- 172.21.15.131:0/3918280026 >> 172.21.15.131:0/3918280026 conn(0x7fc804076ba0 msgr2=0x7fc804076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:37.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.636+0000 7fc809400700 1 -- 172.21.15.131:0/3918280026 shutdown_connections 2024-05-08T22:14:37.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.636+0000 7fc809400700 1 -- 172.21.15.131:0/3918280026 wait complete. 2024-05-08T22:14:37.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.637+0000 7fc809400700 1 Processor -- start 2024-05-08T22:14:37.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.637+0000 7fc809400700 1 -- start start 2024-05-08T22:14:37.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.637+0000 7fc809400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8041000f0 0x7fc8041996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.637+0000 7fc809400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc804100ab0 0x7fc804199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.637+0000 7fc809400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc804105f90 0x7fc80419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.637+0000 7fc809400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80419e600 con 0x7fc804100ab0 2024-05-08T22:14:37.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.637+0000 7fc809400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80419e770 con 0x7fc804105f90 2024-05-08T22:14:37.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.637+0000 7fc809400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80419e930 con 0x7fc8041000f0 2024-05-08T22:14:37.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.638+0000 7fc801e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc804100ab0 0x7fc804199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.638+0000 7fc802800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8041000f0 0x7fc8041996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.638+0000 7fc802800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8041000f0 0x7fc8041996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34398/0 (socket says 172.21.15.131:34398) 2024-05-08T22:14:37.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.638+0000 7fc802800700 1 -- 172.21.15.131:0/872754570 learned_addr learned my addr 172.21.15.131:0/872754570 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:37.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.638+0000 7fc803200700 1 --2- 172.21.15.131:0/872754570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc804105f90 0x7fc80419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.638+0000 7fc802800700 1 -- 172.21.15.131:0/872754570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc804105f90 msgr2=0x7fc80419dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:37.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.638+0000 7fc802800700 1 --2- 172.21.15.131:0/872754570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc804105f90 0x7fc80419dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.638+0000 7fc802800700 1 -- 172.21.15.131:0/872754570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc804100ab0 msgr2=0x7fc804199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:37.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.638+0000 7fc802800700 1 --2- 172.21.15.131:0/872754570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc804100ab0 0x7fc804199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.638+0000 7fc802800700 1 -- 172.21.15.131:0/872754570 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f4009520 con 0x7fc8041000f0 2024-05-08T22:14:37.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.638+0000 7fc803200700 1 --2- 172.21.15.131:0/872754570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc804105f90 0x7fc80419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:14:37.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.638+0000 7fc801e00700 1 --2- 172.21.15.131:0/872754570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc804100ab0 0x7fc804199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:14:37.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.638+0000 7fc802800700 1 --2- 172.21.15.131:0/872754570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8041000f0 0x7fc8041996e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400ea50 tx=0x7fc7f400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:37.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.638+0000 7fc7fb400700 1 -- 172.21.15.131:0/872754570 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f400cbd0 con 0x7fc8041000f0 2024-05-08T22:14:37.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.639+0000 7fc809400700 1 -- 172.21.15.131:0/872754570 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7fc009d00 con 0x7fc8041000f0 2024-05-08T22:14:37.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.639+0000 7fc7fb400700 1 -- 172.21.15.131:0/872754570 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7f400cd30 con 0x7fc8041000f0 2024-05-08T22:14:37.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.639+0000 7fc7fb400700 1 -- 172.21.15.131:0/872754570 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4018680 con 0x7fc8041000f0 2024-05-08T22:14:37.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.639+0000 7fc809400700 1 -- 172.21.15.131:0/872754570 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8041a6270 con 0x7fc8041000f0 2024-05-08T22:14:37.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:37 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/1180558167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:14:37.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.642+0000 7fc7fb400700 1 -- 172.21.15.131:0/872754570 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fc7f4010430 con 0x7fc8041000f0 2024-05-08T22:14:37.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.644+0000 7fc7fb400700 1 --2- 172.21.15.131:0/872754570 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc7e80a1290 0x7fc7e80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.644+0000 7fc81cfabb80 1 Processor -- start 2024-05-08T22:14:37.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.644+0000 7fc7fb400700 1 -- 172.21.15.131:0/872754570 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fc7f4014070 con 0x7fc8041000f0 2024-05-08T22:14:37.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.644+0000 7fc801e00700 1 --2- 172.21.15.131:0/872754570 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc7e80a1290 0x7fc7e80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.644+0000 7fc81cfabb80 1 -- start start 2024-05-08T22:14:37.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.644+0000 7fc81cfabb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecfee00 0x55e8fecff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.644+0000 7fc81cfabb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8fecff720 0x55e8fecf9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.644+0000 7fc81cfabb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fecf9890 0x55e8fecf9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.644+0000 7fc81cfabb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8fec50150 con 0x55e8fecf9890 2024-05-08T22:14:37.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.644+0000 7fc81cfabb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8fec4db10 con 0x55e8fecfee00 2024-05-08T22:14:37.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.644+0000 7fc81cfabb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8fec49400 con 0x55e8fecff720 2024-05-08T22:14:37.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.645+0000 7fc802800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecfee00 0x55e8fecff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.645+0000 7fc801e00700 1 --2- 172.21.15.131:0/872754570 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc7e80a1290 0x7fc7e80a3750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc80419ac30 tx=0x7fc7f0009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:37.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.645+0000 7fc802800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecfee00 0x55e8fecff1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:57724/0 (socket says 172.21.15.131:57724) 2024-05-08T22:14:37.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.645+0000 7fc802800700 1 -- 172.21.15.131:0/903705958 learned_addr learned my addr 172.21.15.131:0/903705958 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:37.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.645+0000 7fc801e00700 1 --2- 172.21.15.131:0/903705958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8fecff720 0x55e8fecf9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.645+0000 7fc803200700 1 --2- 172.21.15.131:0/903705958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fecf9890 0x55e8fecf9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.645+0000 7fc801e00700 1 -- 172.21.15.131:0/903705958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecfee00 msgr2=0x55e8fecff1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:37.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.645+0000 7fc801e00700 1 --2- 172.21.15.131:0/903705958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecfee00 0x55e8fecff1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.645+0000 7fc801e00700 1 -- 172.21.15.131:0/903705958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fecf9890 msgr2=0x55e8fecf9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:37.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.645+0000 7fc801e00700 1 --2- 172.21.15.131:0/903705958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fecf9890 0x55e8fecf9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.645+0000 7fc801e00700 1 -- 172.21.15.131:0/903705958 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8feaca3b0 con 0x55e8fecff720 2024-05-08T22:14:37.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.645+0000 7fc801e00700 1 --2- 172.21.15.131:0/903705958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8fecff720 0x55e8fecf9350 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc7f00042b0 tx=0x7fc7f0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b527432474b7dd0f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:37.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.646+0000 7fc7ef400700 1 -- 172.21.15.131:0/903705958 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0003d10 con 0x55e8fecff720 2024-05-08T22:14:37.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.646+0000 7fc7ef400700 1 -- 172.21.15.131:0/903705958 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7f0003e70 con 0x55e8fecff720 2024-05-08T22:14:37.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.646+0000 7fc7ef400700 1 -- 172.21.15.131:0/903705958 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f000e5d0 con 0x55e8fecff720 2024-05-08T22:14:37.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.646+0000 7fc81cfabb80 1 -- 172.21.15.131:0/903705958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8fecff720 msgr2=0x55e8fecf9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:37.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.646+0000 7fc81cfabb80 1 --2- 172.21.15.131:0/903705958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8fecff720 0x55e8fecf9350 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc7f00042b0 tx=0x7fc7f0014000 comp rx=0 tx=0).stop 2024-05-08T22:14:37.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.646+0000 7fc81cfabb80 1 -- 172.21.15.131:0/903705958 shutdown_connections 2024-05-08T22:14:37.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.646+0000 7fc81cfabb80 1 --2- 172.21.15.131:0/903705958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fecf9890 0x55e8fecf9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.646+0000 7fc81cfabb80 1 --2- 172.21.15.131:0/903705958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecfee00 0x55e8fecff1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.646+0000 7fc81cfabb80 1 --2- 172.21.15.131:0/903705958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8fecff720 0x55e8fecf9350 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.646+0000 7fc81cfabb80 1 -- 172.21.15.131:0/903705958 >> 172.21.15.131:0/903705958 conn(0x55e8fec57210 msgr2=0x55e8fecf3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:37.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.647+0000 7fc81cfabb80 1 -- 172.21.15.131:0/903705958 shutdown_connections 2024-05-08T22:14:37.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.647+0000 7fc81cfabb80 1 -- 172.21.15.131:0/903705958 wait complete. 2024-05-08T22:14:37.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.647+0000 7fc81cfabb80 1 -- start start 2024-05-08T22:14:37.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.647+0000 7fc81cfabb80 1 Processor -- start 2024-05-08T22:14:37.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.648+0000 7fc81cfabb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fecf9890 0x55e8fed05760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.648+0000 7fc81cfabb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8fecfee00 0x55e8fed02a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.648+0000 7fc81cfabb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecff720 0x55e8fed03140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.648+0000 7fc81cfabb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8febe5560 con 0x55e8fecf9890 2024-05-08T22:14:37.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.648+0000 7fc81cfabb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8febe5200 con 0x55e8fecff720 2024-05-08T22:14:37.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.648+0000 7fc81cfabb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8feacc6a0 con 0x55e8fecfee00 2024-05-08T22:14:37.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.648+0000 7fc81cfabb80 10 client.0 _ll_register_callbacks cb 0x55e8febeeb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:14:37.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.648+0000 7fc802800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fecf9890 0x55e8fed05760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.648+0000 7fc803200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecff720 0x55e8fed03140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.648+0000 7fc802800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fecf9890 0x55e8fed05760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37742/0 (socket says 172.21.15.131:37742) 2024-05-08T22:14:37.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.648+0000 7fc802800700 1 -- 172.21.15.131:0/3835694750 learned_addr learned my addr 172.21.15.131:0/3835694750 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:37.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.648+0000 7fc801e00700 1 --2- 172.21.15.131:0/3835694750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8fecfee00 0x55e8fed02a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.648+0000 7fc802800700 1 -- 172.21.15.131:0/3835694750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8fecfee00 msgr2=0x55e8fed02a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:37.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.648+0000 7fc802800700 1 --2- 172.21.15.131:0/3835694750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8fecfee00 0x55e8fed02a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.648+0000 7fc802800700 1 -- 172.21.15.131:0/3835694750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecff720 msgr2=0x55e8fed03140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:37.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.648+0000 7fc802800700 1 --2- 172.21.15.131:0/3835694750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecff720 0x55e8fed03140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.649+0000 7fc802800700 1 -- 172.21.15.131:0/3835694750 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f0017070 con 0x55e8fecf9890 2024-05-08T22:14:37.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.649+0000 7fc803200700 1 --2- 172.21.15.131:0/3835694750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecff720 0x55e8fed03140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:14:37.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.649+0000 7fc802800700 1 --2- 172.21.15.131:0/3835694750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fecf9890 0x55e8fed05760 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc7f40967a0 tx=0x7fc7f409aa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79cd01299cdc9953 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:37.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.649+0000 7fc7e3e00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:14:37.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.649+0000 7fc7e3e00700 1 -- 172.21.15.131:0/3835694750 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f40cd360 con 0x55e8fecf9890 2024-05-08T22:14:37.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.649+0000 7fc7e3e00700 1 -- 172.21.15.131:0/3835694750 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fc7f409e0d0 con 0x55e8fecf9890 2024-05-08T22:14:37.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.649+0000 7fc81cfabb80 1 -- 172.21.15.131:0/3835694750 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8feba2270 con 0x55e8fecf9890 2024-05-08T22:14:37.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.649+0000 7fc7e3e00700 1 -- 172.21.15.131:0/3835694750 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7f40a2720 con 0x55e8fecf9890 2024-05-08T22:14:37.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.649+0000 7fc81cfabb80 10 client.16290 fetch_fsmap learned FSMap version 27 2024-05-08T22:14:37.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.649+0000 7fc81cfabb80 1 -- 172.21.15.131:0/3835694750 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e8fea76b60 con 0x55e8fecf9890 2024-05-08T22:14:37.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.649+0000 7fc7e3e00700 1 -- 172.21.15.131:0/3835694750 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e8feba2270 con 0x55e8fecf9890 2024-05-08T22:14:37.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.650+0000 7fc7e3e00700 1 -- 172.21.15.131:0/3835694750 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fc7f409c070 con 0x55e8fecf9890 2024-05-08T22:14:37.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.650+0000 7fc81cfabb80 10 client.16290 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T22:14:37.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.650+0000 7fc81cfabb80 10 client.16290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:14:37.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.650+0000 7fc81cfabb80 20 client.16290 populate_metadata read hostname 'smithi131' 2024-05-08T22:14:37.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.650+0000 7fc81cfabb80 1 --2- 172.21.15.131:0/3835694750 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55e8fed13260 0x55e8fed15680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.650+0000 7fc81cfabb80 4 client.16290 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:14:37.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.650+0000 7fc81cfabb80 1 -- 172.21.15.131:0/3835694750 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8fecc8810 con 0x55e8fed13260 2024-05-08T22:14:37.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.650+0000 7fc801e00700 1 --2- 172.21.15.131:0/3835694750 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55e8fed13260 0x55e8fed15680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.651+0000 7fc7e3e00700 10 client.16290 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:14:37.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.651+0000 7fc801e00700 1 --2- 172.21.15.131:0/3835694750 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55e8fed13260 0x55e8fed15680 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e7a3ecd5d79b0d1 server_cookie=dd4478c9ff9dbbd8 in_seq=0 out_seq=0 2024-05-08T22:14:37.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.652+0000 7fc7e3e00700 1 -- 172.21.15.131:0/3835694750 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e8fecc8810 con 0x55e8fed13260 2024-05-08T22:14:37.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.652+0000 7fc7e3e00700 10 client.16290 handle_command_reply: tid=0 2024-05-08T22:14:37.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.652+0000 7fc81cfabb80 1 client.16290 shutdown 2024-05-08T22:14:37.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.652+0000 7fc81cfabb80 1 -- 172.21.15.131:0/3835694750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fecf9890 msgr2=0x55e8fed05760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:37.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.652+0000 7fc81cfabb80 1 --2- 172.21.15.131:0/3835694750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fecf9890 0x55e8fed05760 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc7f40967a0 tx=0x7fc7f409aa20 comp rx=0 tx=0).stop 2024-05-08T22:14:37.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.652+0000 7fc81cfabb80 1 -- 172.21.15.131:0/3835694750 shutdown_connections 2024-05-08T22:14:37.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.652+0000 7fc81cfabb80 1 --2- 172.21.15.131:0/3835694750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecff720 0x55e8fed03140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.652+0000 7fc81cfabb80 1 --2- 172.21.15.131:0/3835694750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fecf9890 0x55e8fed05760 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.652+0000 7fc81cfabb80 1 --2- 172.21.15.131:0/3835694750 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55e8fed13260 0x55e8fed15680 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.652+0000 7fc81cfabb80 1 --2- 172.21.15.131:0/3835694750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8fecfee00 0x55e8fed02a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.652+0000 7fc81cfabb80 1 -- 172.21.15.131:0/3835694750 >> 172.21.15.131:0/3835694750 conn(0x55e8fec57210 msgr2=0x55e8fecf35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:37.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.652+0000 7fc7e3e00700 0 client.16290 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:14:37.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.652+0000 7fc81cfabb80 1 -- 172.21.15.131:0/3835694750 shutdown_connections 2024-05-08T22:14:37.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.653+0000 7fc81cfabb80 1 -- 172.21.15.131:0/3835694750 wait complete. 2024-05-08T22:14:37.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.653+0000 7fc81cfabb80 20 client.16290 trim_cache size 0 max 16384 2024-05-08T22:14:37.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.668+0000 7fc81cfabb80 1 Processor -- start 2024-05-08T22:14:37.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.668+0000 7fc81cfabb80 1 -- start start 2024-05-08T22:14:37.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.668+0000 7fc81cfabb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8fecf9890 0x55e8fecf9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.668+0000 7fc81cfabb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecfee00 0x55e8fecf3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.668+0000 7fc81cfabb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fecff720 0x55e8fed0a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.668+0000 7fc81cfabb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8fecec040 con 0x55e8fecff720 2024-05-08T22:14:37.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.668+0000 7fc81cfabb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8fecfd450 con 0x55e8fecfee00 2024-05-08T22:14:37.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.668+0000 7fc81cfabb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8fecf4260 con 0x55e8fecf9890 2024-05-08T22:14:37.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.668+0000 7fc801e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecfee00 0x55e8fecf3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.668+0000 7fc801e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecfee00 0x55e8fecf3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:57736/0 (socket says 172.21.15.131:57736) 2024-05-08T22:14:37.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.668+0000 7fc801e00700 1 -- 172.21.15.131:0/2466882025 learned_addr learned my addr 172.21.15.131:0/2466882025 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:37.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.668+0000 7fc803200700 1 --2- 172.21.15.131:0/2466882025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fecff720 0x55e8fed0a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.669+0000 7fc802800700 1 --2- 172.21.15.131:0/2466882025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8fecf9890 0x55e8fecf9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.669+0000 7fc801e00700 1 -- 172.21.15.131:0/2466882025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8fecf9890 msgr2=0x55e8fecf9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:37.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.669+0000 7fc801e00700 1 --2- 172.21.15.131:0/2466882025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8fecf9890 0x55e8fecf9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.669+0000 7fc801e00700 1 -- 172.21.15.131:0/2466882025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fecff720 msgr2=0x55e8fed0a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:37.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.669+0000 7fc801e00700 1 --2- 172.21.15.131:0/2466882025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fecff720 0x55e8fed0a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.669+0000 7fc801e00700 1 -- 172.21.15.131:0/2466882025 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8fecf9f40 con 0x55e8fecfee00 2024-05-08T22:14:37.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.669+0000 7fc803200700 1 --2- 172.21.15.131:0/2466882025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fecff720 0x55e8fed0a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:14:37.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.669+0000 7fc801e00700 1 --2- 172.21.15.131:0/2466882025 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecfee00 0x55e8fecf3d20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc7f00042b0 tx=0x7fc7f0017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f07cdc7868ed2d26 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:37.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.669+0000 7fc7efe00700 1 -- 172.21.15.131:0/2466882025 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f002a040 con 0x55e8fecfee00 2024-05-08T22:14:37.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.669+0000 7fc7efe00700 1 -- 172.21.15.131:0/2466882025 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7f0003b10 con 0x55e8fecfee00 2024-05-08T22:14:37.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.669+0000 7fc7efe00700 1 -- 172.21.15.131:0/2466882025 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f001e3f0 con 0x55e8fecfee00 2024-05-08T22:14:37.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.670+0000 7fc81cfabb80 1 -- 172.21.15.131:0/2466882025 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecfee00 msgr2=0x55e8fecf3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:37.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.670+0000 7fc81cfabb80 1 --2- 172.21.15.131:0/2466882025 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecfee00 0x55e8fecf3d20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc7f00042b0 tx=0x7fc7f0017000 comp rx=0 tx=0).stop 2024-05-08T22:14:37.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.670+0000 7fc81cfabb80 1 -- 172.21.15.131:0/2466882025 shutdown_connections 2024-05-08T22:14:37.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.670+0000 7fc81cfabb80 1 --2- 172.21.15.131:0/2466882025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fecff720 0x55e8fed0a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.670+0000 7fc81cfabb80 1 --2- 172.21.15.131:0/2466882025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8fecf9890 0x55e8fecf9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.670+0000 7fc81cfabb80 1 --2- 172.21.15.131:0/2466882025 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecfee00 0x55e8fecf3d20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.670+0000 7fc81cfabb80 1 -- 172.21.15.131:0/2466882025 >> 172.21.15.131:0/2466882025 conn(0x55e8fec57210 msgr2=0x55e8fed0ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:37.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.670+0000 7fc81cfabb80 1 -- 172.21.15.131:0/2466882025 shutdown_connections 2024-05-08T22:14:37.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.670+0000 7fc81cfabb80 1 -- 172.21.15.131:0/2466882025 wait complete. 2024-05-08T22:14:37.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.670+0000 7fc81cfabb80 1 -- start start 2024-05-08T22:14:37.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.671+0000 7fc81cfabb80 1 Processor -- start 2024-05-08T22:14:37.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.671+0000 7fc81cfabb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8fecf9890 0x55e8fed0a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.671+0000 7fc81cfabb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecfee00 0x55e8fed08780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.671+0000 7fc81cfabb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fecff720 0x55e8fed08cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.671+0000 7fc81cfabb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8fed09200 con 0x55e8fecff720 2024-05-08T22:14:37.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.671+0000 7fc81cfabb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8fed09370 con 0x55e8fecfee00 2024-05-08T22:14:37.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.671+0000 7fc81cfabb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8fed09530 con 0x55e8fecf9890 2024-05-08T22:14:37.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.671+0000 7fc81cfabb80 10 client.0 _ll_register_callbacks cb 0x55e8febeeb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:14:37.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.671+0000 7fc801e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecfee00 0x55e8fed08780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.671+0000 7fc801e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecfee00 0x55e8fed08780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:57740/0 (socket says 172.21.15.131:57740) 2024-05-08T22:14:37.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.671+0000 7fc801e00700 1 -- 172.21.15.131:0/3039662800 learned_addr learned my addr 172.21.15.131:0/3039662800 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:37.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.672+0000 7fc803200700 1 --2- 172.21.15.131:0/3039662800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fecff720 0x55e8fed08cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.672+0000 7fc802800700 1 --2- 172.21.15.131:0/3039662800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8fecf9890 0x55e8fed0a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.672+0000 7fc801e00700 1 -- 172.21.15.131:0/3039662800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8fecf9890 msgr2=0x55e8fed0a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:37.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.672+0000 7fc801e00700 1 --2- 172.21.15.131:0/3039662800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8fecf9890 0x55e8fed0a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.672+0000 7fc801e00700 1 -- 172.21.15.131:0/3039662800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fecff720 msgr2=0x55e8fed08cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:37.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.672+0000 7fc801e00700 1 --2- 172.21.15.131:0/3039662800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fecff720 0x55e8fed08cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.672+0000 7fc801e00700 1 -- 172.21.15.131:0/3039662800 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8feaca3b0 con 0x55e8fecfee00 2024-05-08T22:14:37.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.672+0000 7fc803200700 1 --2- 172.21.15.131:0/3039662800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fecff720 0x55e8fed08cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:14:37.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.672+0000 7fc801e00700 1 --2- 172.21.15.131:0/3039662800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecfee00 0x55e8fed08780 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0004280 tx=0x7fc7f0004d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cebbf8fff2d7589 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:37.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.672+0000 7fc7e3e00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:14:37.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.672+0000 7fc7e3e00700 1 -- 172.21.15.131:0/3039662800 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f002a040 con 0x55e8fecfee00 2024-05-08T22:14:37.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.672+0000 7fc7e3e00700 1 -- 172.21.15.131:0/3039662800 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fc7f00050c0 con 0x55e8fecfee00 2024-05-08T22:14:37.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.672+0000 7fc81cfabb80 1 -- 172.21.15.131:0/3039662800 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7f0003e70 con 0x55e8fecfee00 2024-05-08T22:14:37.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.673+0000 7fc7e3e00700 1 -- 172.21.15.131:0/3039662800 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7f0005280 con 0x55e8fecfee00 2024-05-08T22:14:37.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.673+0000 7fc81cfabb80 10 client.26321 fetch_fsmap learned FSMap version 27 2024-05-08T22:14:37.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.673+0000 7fc81cfabb80 1 -- 172.21.15.131:0/3039662800 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e8fec0f560 con 0x55e8fecfee00 2024-05-08T22:14:37.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.673+0000 7fc7e3e00700 1 -- 172.21.15.131:0/3039662800 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7f0003e70 con 0x55e8fecfee00 2024-05-08T22:14:37.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.673+0000 7fc7e3e00700 1 -- 172.21.15.131:0/3039662800 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 27) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fc7f003b030 con 0x55e8fecfee00 2024-05-08T22:14:37.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.673+0000 7fc81cfabb80 10 client.26321 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T22:14:37.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.673+0000 7fc81cfabb80 10 client.26321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:14:37.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.673+0000 7fc81cfabb80 20 client.26321 populate_metadata read hostname 'smithi131' 2024-05-08T22:14:37.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.673+0000 7fc81cfabb80 1 --2- 172.21.15.131:0/3039662800 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55e8fed6a7f0 0x55e8fed6cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:37.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.673+0000 7fc81cfabb80 4 client.26321 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:14:37.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.673+0000 7fc81cfabb80 1 -- 172.21.15.131:0/3039662800 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e8fec41ec0 con 0x55e8fed6a7f0 2024-05-08T22:14:37.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.673+0000 7fc802800700 1 --2- 172.21.15.131:0/3039662800 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55e8fed6a7f0 0x55e8fed6cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:37.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.674+0000 7fc802800700 1 --2- 172.21.15.131:0/3039662800 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55e8fed6a7f0 0x55e8fed6cc10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fadf2dc9b8f56ddc server_cookie=b35f58be23fba61a in_seq=0 out_seq=0 2024-05-08T22:14:37.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.674+0000 7fc7e3e00700 10 client.26321 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:14:37.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.674+0000 7fc7e3e00700 1 -- 172.21.15.131:0/3039662800 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e8fec41ec0 con 0x55e8fed6a7f0 2024-05-08T22:14:37.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.674+0000 7fc7e3e00700 10 client.26321 handle_command_reply: tid=0 2024-05-08T22:14:37.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.674+0000 7fc81cfabb80 1 client.26321 shutdown 2024-05-08T22:14:37.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.674+0000 7fc81cfabb80 1 -- 172.21.15.131:0/3039662800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecfee00 msgr2=0x55e8fed08780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:37.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.674+0000 7fc81cfabb80 1 --2- 172.21.15.131:0/3039662800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecfee00 0x55e8fed08780 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0004280 tx=0x7fc7f0004d20 comp rx=0 tx=0).stop 2024-05-08T22:14:37.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.674+0000 7fc81cfabb80 1 -- 172.21.15.131:0/3039662800 shutdown_connections 2024-05-08T22:14:37.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.674+0000 7fc81cfabb80 1 --2- 172.21.15.131:0/3039662800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8fecff720 0x55e8fed08cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.674+0000 7fc81cfabb80 1 --2- 172.21.15.131:0/3039662800 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55e8fed6a7f0 0x55e8fed6cc10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.674+0000 7fc81cfabb80 1 --2- 172.21.15.131:0/3039662800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8fecf9890 0x55e8fed0a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.674+0000 7fc7e3e00700 0 client.26321 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:14:37.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.674+0000 7fc81cfabb80 1 --2- 172.21.15.131:0/3039662800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8fecfee00 0x55e8fed08780 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.674+0000 7fc81cfabb80 1 -- 172.21.15.131:0/3039662800 >> 172.21.15.131:0/3039662800 conn(0x55e8fec57210 msgr2=0x55e8fed0ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:37.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.675+0000 7fc81cfabb80 1 -- 172.21.15.131:0/3039662800 shutdown_connections 2024-05-08T22:14:37.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.675+0000 7fc81cfabb80 1 -- 172.21.15.131:0/3039662800 wait complete. 2024-05-08T22:14:37.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.675+0000 7fc81cfabb80 20 client.26321 trim_cache size 0 max 16384 2024-05-08T22:14:37.681 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:14:37.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.675+0000 7fc809400700 1 -- 172.21.15.131:0/872754570 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc7e80a1290 msgr2=0x7fc7e80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:37.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.675+0000 7fc809400700 1 --2- 172.21.15.131:0/872754570 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc7e80a1290 0x7fc7e80a3750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc80419ac30 tx=0x7fc7f0009260 comp rx=0 tx=0).stop 2024-05-08T22:14:37.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.675+0000 7fc809400700 1 -- 172.21.15.131:0/872754570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8041000f0 msgr2=0x7fc8041996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:37.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.675+0000 7fc809400700 1 --2- 172.21.15.131:0/872754570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8041000f0 0x7fc8041996e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400ea50 tx=0x7fc7f400ed60 comp rx=0 tx=0).stop 2024-05-08T22:14:37.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.675+0000 7fc809400700 1 -- 172.21.15.131:0/872754570 shutdown_connections 2024-05-08T22:14:37.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.675+0000 7fc809400700 1 --2- 172.21.15.131:0/872754570 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc7e80a1290 0x7fc7e80a3750 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.675+0000 7fc809400700 1 --2- 172.21.15.131:0/872754570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc804105f90 0x7fc80419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.675+0000 7fc809400700 1 --2- 172.21.15.131:0/872754570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8041000f0 0x7fc8041996e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.675+0000 7fc809400700 1 --2- 172.21.15.131:0/872754570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc804100ab0 0x7fc804199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:37.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.675+0000 7fc809400700 1 -- 172.21.15.131:0/872754570 >> 172.21.15.131:0/872754570 conn(0x7fc804076ba0 msgr2=0x7fc804069000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:37.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.676+0000 7fc809400700 1 -- 172.21.15.131:0/872754570 shutdown_connections 2024-05-08T22:14:37.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:37.676+0000 7fc809400700 1 -- 172.21.15.131:0/872754570 wait complete. 2024-05-08T22:14:37.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:37 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/1180558167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:14:37.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:37 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/1180558167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:14:38.699 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T22:14:38.699 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T22:14:38.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:38 smithi157 ceph-mon[77746]: pgmap v225: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.4 MiB/s wr, 1.07k op/s 2024-05-08T22:14:38.779 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:38 smithi131 ceph-mon[37260]: pgmap v225: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.4 MiB/s wr, 1.07k op/s 2024-05-08T22:14:38.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:38 smithi132 ceph-mon[78180]: pgmap v225: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.4 MiB/s wr, 1.07k op/s 2024-05-08T22:14:38.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.927+0000 7fd902a00700 1 -- 172.21.15.131:0/3323957472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8fc101b30 msgr2=0x7fd8fc105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:38.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.927+0000 7fd902a00700 1 --2- 172.21.15.131:0/3323957472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8fc101b30 0x7fd8fc105b70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009990 tx=0x7fd8e8009ca0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.927+0000 7fd902a00700 1 -- 172.21.15.131:0/3323957472 shutdown_connections 2024-05-08T22:14:38.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.927+0000 7fd902a00700 1 --2- 172.21.15.131:0/3323957472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8fc1062a0 0x7fd8fc10de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.927+0000 7fd902a00700 1 --2- 172.21.15.131:0/3323957472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8fc101180 0x7fd8fc101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.927+0000 7fd902a00700 1 --2- 172.21.15.131:0/3323957472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8fc101b30 0x7fd8fc105b70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.927+0000 7fd902a00700 1 -- 172.21.15.131:0/3323957472 >> 172.21.15.131:0/3323957472 conn(0x7fd8fc0fc9e0 msgr2=0x7fd8fc0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:38.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.927+0000 7fd902a00700 1 -- 172.21.15.131:0/3323957472 shutdown_connections 2024-05-08T22:14:38.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.927+0000 7fd902a00700 1 -- 172.21.15.131:0/3323957472 wait complete. 2024-05-08T22:14:38.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.928+0000 7fd902a00700 1 Processor -- start 2024-05-08T22:14:38.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.928+0000 7fd902a00700 1 -- start start 2024-05-08T22:14:38.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.928+0000 7fd902a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8fc101180 0x7fd8fc199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:38.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.929+0000 7fd902a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8fc1062a0 0x7fd8fc199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:38.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.929+0000 7fd902a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8fc19dee0 0x7fd8fc19e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:38.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.929+0000 7fd902a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc19e960 con 0x7fd8fc19dee0 2024-05-08T22:14:38.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.929+0000 7fd902a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc19ead0 con 0x7fd8fc101180 2024-05-08T22:14:38.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.929+0000 7fd902a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc19ec90 con 0x7fd8fc1062a0 2024-05-08T22:14:38.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.929+0000 7fd8fbe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8fc101180 0x7fd8fc199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:38.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.929+0000 7fd8fbe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8fc101180 0x7fd8fc199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:57760/0 (socket says 172.21.15.131:57760) 2024-05-08T22:14:38.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.929+0000 7fd8fbe00700 1 -- 172.21.15.131:0/652643654 learned_addr learned my addr 172.21.15.131:0/652643654 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:38.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.929+0000 7fd8fb400700 1 --2- 172.21.15.131:0/652643654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8fc1062a0 0x7fd8fc199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:38.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.929+0000 7fd900a00700 1 --2- 172.21.15.131:0/652643654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8fc19dee0 0x7fd8fc19e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:38.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.929+0000 7fd8fbe00700 1 -- 172.21.15.131:0/652643654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8fc1062a0 msgr2=0x7fd8fc199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:38.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.929+0000 7fd8fbe00700 1 --2- 172.21.15.131:0/652643654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8fc1062a0 0x7fd8fc199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.929+0000 7fd8fbe00700 1 -- 172.21.15.131:0/652643654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8fc19dee0 msgr2=0x7fd8fc19e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:38.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.929+0000 7fd8fbe00700 1 --2- 172.21.15.131:0/652643654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8fc19dee0 0x7fd8fc19e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.929+0000 7fd8fbe00700 1 -- 172.21.15.131:0/652643654 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e80095f0 con 0x7fd8fc101180 2024-05-08T22:14:38.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.929+0000 7fd8fbe00700 1 --2- 172.21.15.131:0/652643654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8fc101180 0x7fd8fc199600 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00e9a0 tx=0x7fd8ec00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:38.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.930+0000 7fd8f8c00700 1 -- 172.21.15.131:0/652643654 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec009790 con 0x7fd8fc101180 2024-05-08T22:14:38.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.930+0000 7fd8f8c00700 1 -- 172.21.15.131:0/652643654 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8ec00cbd0 con 0x7fd8fc101180 2024-05-08T22:14:38.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.930+0000 7fd902a00700 1 -- 172.21.15.131:0/652643654 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8fc1a61f0 con 0x7fd8fc101180 2024-05-08T22:14:38.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.930+0000 7fd8f8c00700 1 -- 172.21.15.131:0/652643654 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec018680 con 0x7fd8fc101180 2024-05-08T22:14:38.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.930+0000 7fd902a00700 1 -- 172.21.15.131:0/652643654 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8fc1a6720 con 0x7fd8fc101180 2024-05-08T22:14:38.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.933+0000 7fd8f8c00700 1 -- 172.21.15.131:0/652643654 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fd8ec00cd40 con 0x7fd8fc101180 2024-05-08T22:14:38.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.934+0000 7fd8f8c00700 1 --2- 172.21.15.131:0/652643654 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fd8e00a1310 0x7fd8e00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:38.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.934+0000 7fd916569b80 1 Processor -- start 2024-05-08T22:14:38.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.934+0000 7fd8f8c00700 1 -- 172.21.15.131:0/652643654 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fd8ec014070 con 0x7fd8fc101180 2024-05-08T22:14:38.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.934+0000 7fd916569b80 1 -- start start 2024-05-08T22:14:38.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.934+0000 7fd8fb400700 1 --2- 172.21.15.131:0/652643654 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fd8e00a1310 0x7fd8e00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:38.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.935+0000 7fd916569b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560953a8ee80 0x560953a8f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:38.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.935+0000 7fd916569b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560953a8f7a0 0x560953a893d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:38.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.935+0000 7fd916569b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560953a89910 0x560953a89d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:38.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.935+0000 7fd916569b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609539e01d0 con 0x560953a8ee80 2024-05-08T22:14:38.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.935+0000 7fd916569b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609539ddb90 con 0x560953a8f7a0 2024-05-08T22:14:38.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.935+0000 7fd916569b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609539d9480 con 0x560953a89910 2024-05-08T22:14:38.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.935+0000 7fd8fbe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560953a8ee80 0x560953a8f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:38.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.935+0000 7fd8fbe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560953a8ee80 0x560953a8f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37798/0 (socket says 172.21.15.131:37798) 2024-05-08T22:14:38.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.935+0000 7fd8fbe00700 1 -- 172.21.15.131:0/2891579041 learned_addr learned my addr 172.21.15.131:0/2891579041 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:38.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.935+0000 7fd900a00700 1 --2- 172.21.15.131:0/2891579041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560953a89910 0x560953a89d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:38.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.935+0000 7fd8fb400700 1 --2- 172.21.15.131:0/652643654 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fd8e00a1310 0x7fd8e00a37d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc19ab50 tx=0x7fd8e800b260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:38.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.935+0000 7fd8fb400700 1 --2- 172.21.15.131:0/2891579041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560953a8f7a0 0x560953a893d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:38.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.935+0000 7fd8fbe00700 1 -- 172.21.15.131:0/2891579041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560953a89910 msgr2=0x560953a89d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:38.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.935+0000 7fd8fbe00700 1 --2- 172.21.15.131:0/2891579041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560953a89910 0x560953a89d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.935+0000 7fd8fbe00700 1 -- 172.21.15.131:0/2891579041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560953a8f7a0 msgr2=0x560953a893d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:38.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.935+0000 7fd8fbe00700 1 --2- 172.21.15.131:0/2891579041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560953a8f7a0 0x560953a893d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.935+0000 7fd8fbe00700 1 -- 172.21.15.131:0/2891579041 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56095385a430 con 0x560953a8ee80 2024-05-08T22:14:38.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.935+0000 7fd900a00700 1 --2- 172.21.15.131:0/2891579041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560953a89910 0x560953a89d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:14:38.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.936+0000 7fd8fbe00700 1 --2- 172.21.15.131:0/2891579041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560953a8ee80 0x560953a8f260 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec096a70 tx=0x7fd8ec0999e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ed351816bf0958 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:38.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.936+0000 7fd8f1200700 1 -- 172.21.15.131:0/2891579041 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec09a000 con 0x560953a8ee80 2024-05-08T22:14:38.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.936+0000 7fd8f1200700 1 -- 172.21.15.131:0/2891579041 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8ec09de30 con 0x560953a8ee80 2024-05-08T22:14:38.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.936+0000 7fd8f1200700 1 -- 172.21.15.131:0/2891579041 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec0a1ff0 con 0x560953a8ee80 2024-05-08T22:14:38.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.936+0000 7fd916569b80 1 -- 172.21.15.131:0/2891579041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560953a8ee80 msgr2=0x560953a8f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:38.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.936+0000 7fd916569b80 1 --2- 172.21.15.131:0/2891579041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560953a8ee80 0x560953a8f260 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec096a70 tx=0x7fd8ec0999e0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.937+0000 7fd916569b80 1 -- 172.21.15.131:0/2891579041 shutdown_connections 2024-05-08T22:14:38.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.937+0000 7fd916569b80 1 --2- 172.21.15.131:0/2891579041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560953a89910 0x560953a89d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.937+0000 7fd916569b80 1 --2- 172.21.15.131:0/2891579041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560953a8ee80 0x560953a8f260 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.937+0000 7fd916569b80 1 --2- 172.21.15.131:0/2891579041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560953a8f7a0 0x560953a893d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.937+0000 7fd916569b80 1 -- 172.21.15.131:0/2891579041 >> 172.21.15.131:0/2891579041 conn(0x5609539e7290 msgr2=0x560953a83e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:38.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.937+0000 7fd916569b80 1 -- 172.21.15.131:0/2891579041 shutdown_connections 2024-05-08T22:14:38.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.937+0000 7fd916569b80 1 -- 172.21.15.131:0/2891579041 wait complete. 2024-05-08T22:14:38.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.937+0000 7fd916569b80 1 -- start start 2024-05-08T22:14:38.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.938+0000 7fd916569b80 1 Processor -- start 2024-05-08T22:14:38.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.938+0000 7fd916569b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560953a89910 0x560953a92b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:38.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.938+0000 7fd916569b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560953a8ee80 0x560953a930c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:38.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.938+0000 7fd916569b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560953a8f7a0 0x560953a93600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:38.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.938+0000 7fd916569b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609539755e0 con 0x560953a8ee80 2024-05-08T22:14:38.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.938+0000 7fd916569b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560953975280 con 0x560953a8f7a0 2024-05-08T22:14:38.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.938+0000 7fd916569b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56095385c720 con 0x560953a89910 2024-05-08T22:14:38.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.938+0000 7fd916569b80 10 client.0 _ll_register_callbacks cb 0x56095397ebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:14:38.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.939+0000 7fd8fbe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560953a89910 0x560953a92b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:38.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.939+0000 7fd900a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560953a8f7a0 0x560953a93600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:38.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.939+0000 7fd8fbe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560953a89910 0x560953a92b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34490/0 (socket says 172.21.15.131:34490) 2024-05-08T22:14:38.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.939+0000 7fd8fb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560953a8ee80 0x560953a930c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:38.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.939+0000 7fd8fbe00700 1 -- 172.21.15.131:0/3517520111 learned_addr learned my addr 172.21.15.131:0/3517520111 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:38.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.939+0000 7fd900a00700 1 -- 172.21.15.131:0/3517520111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560953a89910 msgr2=0x560953a92b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:38.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.939+0000 7fd900a00700 1 --2- 172.21.15.131:0/3517520111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560953a89910 0x560953a92b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.939+0000 7fd900a00700 1 -- 172.21.15.131:0/3517520111 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560953a8ee80 msgr2=0x560953a930c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:38.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.939+0000 7fd900a00700 1 --2- 172.21.15.131:0/3517520111 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560953a8ee80 0x560953a930c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.939+0000 7fd900a00700 1 -- 172.21.15.131:0/3517520111 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f4009050 con 0x560953a8f7a0 2024-05-08T22:14:38.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.939+0000 7fd8fbe00700 1 --2- 172.21.15.131:0/3517520111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560953a89910 0x560953a92b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:14:38.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.939+0000 7fd900a00700 1 --2- 172.21.15.131:0/3517520111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560953a8f7a0 0x560953a93600 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4007c00 tx=0x7fd8f4007c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94b673edda12b0e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:38.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.939+0000 7fd8e5600700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:14:38.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.939+0000 7fd8e5600700 1 -- 172.21.15.131:0/3517520111 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f400e050 con 0x560953a8f7a0 2024-05-08T22:14:38.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.939+0000 7fd916569b80 1 -- 172.21.15.131:0/3517520111 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609539322f0 con 0x560953a8f7a0 2024-05-08T22:14:38.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.939+0000 7fd8e5600700 1 -- 172.21.15.131:0/3517520111 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fd8f4004db0 con 0x560953a8f7a0 2024-05-08T22:14:38.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.940+0000 7fd8e5600700 1 -- 172.21.15.131:0/3517520111 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8f4003a00 con 0x560953a8f7a0 2024-05-08T22:14:38.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.940+0000 7fd916569b80 10 client.26333 fetch_fsmap learned FSMap version 27 2024-05-08T22:14:38.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.940+0000 7fd916569b80 1 -- 172.21.15.131:0/3517520111 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fd8fc1a6720 con 0x560953a8f7a0 2024-05-08T22:14:38.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.940+0000 7fd8e5600700 1 -- 172.21.15.131:0/3517520111 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5609539322f0 con 0x560953a8f7a0 2024-05-08T22:14:38.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.940+0000 7fd8e5600700 1 -- 172.21.15.131:0/3517520111 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 27) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fd8f401e060 con 0x560953a8f7a0 2024-05-08T22:14:38.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.940+0000 7fd916569b80 10 client.26333 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T22:14:38.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.940+0000 7fd916569b80 10 client.26333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:14:38.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.940+0000 7fd916569b80 20 client.26333 populate_metadata read hostname 'smithi131' 2024-05-08T22:14:38.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.940+0000 7fd916569b80 1 --2- 172.21.15.131:0/3517520111 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560953aa3300 0x560953aa5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:38.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.940+0000 7fd916569b80 4 client.26333 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:14:38.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.940+0000 7fd916569b80 1 -- 172.21.15.131:0/3517520111 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560953a58890 con 0x560953aa3300 2024-05-08T22:14:38.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.940+0000 7fd8fbe00700 1 --2- 172.21.15.131:0/3517520111 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560953aa3300 0x560953aa5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:38.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.941+0000 7fd8e5600700 10 client.26333 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:14:38.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.941+0000 7fd8fbe00700 1 --2- 172.21.15.131:0/3517520111 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560953aa3300 0x560953aa5720 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aabfc9ab7fab2d1 server_cookie=95ab7b8903cb35ab in_seq=0 out_seq=0 2024-05-08T22:14:38.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.944+0000 7fd8e5600700 1 -- 172.21.15.131:0/3517520111 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560953a58890 con 0x560953aa3300 2024-05-08T22:14:38.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.944+0000 7fd8e5600700 10 client.26333 handle_command_reply: tid=0 2024-05-08T22:14:38.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.944+0000 7fd916569b80 1 client.26333 shutdown 2024-05-08T22:14:38.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.944+0000 7fd916569b80 1 -- 172.21.15.131:0/3517520111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560953a8f7a0 msgr2=0x560953a93600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:38.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.944+0000 7fd916569b80 1 --2- 172.21.15.131:0/3517520111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560953a8f7a0 0x560953a93600 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4007c00 tx=0x7fd8f4007c30 comp rx=0 tx=0).stop 2024-05-08T22:14:38.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.944+0000 7fd916569b80 1 -- 172.21.15.131:0/3517520111 shutdown_connections 2024-05-08T22:14:38.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.944+0000 7fd916569b80 1 --2- 172.21.15.131:0/3517520111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560953a8f7a0 0x560953a93600 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.944+0000 7fd916569b80 1 --2- 172.21.15.131:0/3517520111 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560953aa3300 0x560953aa5720 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.944+0000 7fd916569b80 1 --2- 172.21.15.131:0/3517520111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560953a89910 0x560953a92b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.944+0000 7fd916569b80 1 --2- 172.21.15.131:0/3517520111 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560953a8ee80 0x560953a930c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.944+0000 7fd8e5600700 0 client.26333 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:14:38.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.944+0000 7fd916569b80 1 -- 172.21.15.131:0/3517520111 >> 172.21.15.131:0/3517520111 conn(0x5609539e7290 msgr2=0x560953a83620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:38.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.944+0000 7fd916569b80 1 -- 172.21.15.131:0/3517520111 shutdown_connections 2024-05-08T22:14:38.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.944+0000 7fd916569b80 1 -- 172.21.15.131:0/3517520111 wait complete. 2024-05-08T22:14:38.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.944+0000 7fd916569b80 20 client.26333 trim_cache size 0 max 16384 2024-05-08T22:14:38.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.963+0000 7fd916569b80 1 Processor -- start 2024-05-08T22:14:38.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.963+0000 7fd916569b80 1 -- start start 2024-05-08T22:14:38.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.963+0000 7fd916569b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560953a89910 0x560953a8e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:38.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.963+0000 7fd916569b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560953a8ee80 0x560953a83da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:38.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.963+0000 7fd916569b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560953a8f7a0 0x560953a93420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:38.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.963+0000 7fd916569b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560953a8e8b0 con 0x560953a8ee80 2024-05-08T22:14:38.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.963+0000 7fd916569b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560953aaa8f0 con 0x560953a89910 2024-05-08T22:14:38.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.963+0000 7fd916569b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560953a7e880 con 0x560953a8f7a0 2024-05-08T22:14:38.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.964+0000 7fd8fbe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560953a89910 0x560953a8e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:38.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.964+0000 7fd8fb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560953a8ee80 0x560953a83da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:38.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.964+0000 7fd900a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560953a8f7a0 0x560953a93420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:38.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.964+0000 7fd8fbe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560953a89910 0x560953a8e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:57772/0 (socket says 172.21.15.131:57772) 2024-05-08T22:14:38.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.964+0000 7fd8fbe00700 1 -- 172.21.15.131:0/1047983834 learned_addr learned my addr 172.21.15.131:0/1047983834 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:38.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.964+0000 7fd900a00700 1 -- 172.21.15.131:0/1047983834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560953a89910 msgr2=0x560953a8e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:38.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.964+0000 7fd900a00700 1 --2- 172.21.15.131:0/1047983834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560953a89910 0x560953a8e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.964+0000 7fd900a00700 1 -- 172.21.15.131:0/1047983834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560953a8ee80 msgr2=0x560953a83da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:38.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.964+0000 7fd900a00700 1 --2- 172.21.15.131:0/1047983834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560953a8ee80 0x560953a83da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.964+0000 7fd900a00700 1 -- 172.21.15.131:0/1047983834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560953a80560 con 0x560953a8f7a0 2024-05-08T22:14:38.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.964+0000 7fd900a00700 1 --2- 172.21.15.131:0/1047983834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560953a8f7a0 0x560953a93420 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4000f80 tx=0x7fd8f4009990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f7f1bc305f42782 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:38.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.964+0000 7fd8f1c00700 1 -- 172.21.15.131:0/1047983834 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f400e040 con 0x560953a8f7a0 2024-05-08T22:14:38.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.964+0000 7fd8f1c00700 1 -- 172.21.15.131:0/1047983834 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8f400f040 con 0x560953a8f7a0 2024-05-08T22:14:38.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.965+0000 7fd8f1c00700 1 -- 172.21.15.131:0/1047983834 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4013780 con 0x560953a8f7a0 2024-05-08T22:14:38.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.965+0000 7fd916569b80 1 -- 172.21.15.131:0/1047983834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560953a8f7a0 msgr2=0x560953a93420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:38.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.965+0000 7fd916569b80 1 --2- 172.21.15.131:0/1047983834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560953a8f7a0 0x560953a93420 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4000f80 tx=0x7fd8f4009990 comp rx=0 tx=0).stop 2024-05-08T22:14:38.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.965+0000 7fd916569b80 1 -- 172.21.15.131:0/1047983834 shutdown_connections 2024-05-08T22:14:38.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.965+0000 7fd916569b80 1 --2- 172.21.15.131:0/1047983834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560953a8f7a0 0x560953a93420 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.965+0000 7fd916569b80 1 --2- 172.21.15.131:0/1047983834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560953a89910 0x560953a8e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.965+0000 7fd916569b80 1 --2- 172.21.15.131:0/1047983834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560953a8ee80 0x560953a83da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.965+0000 7fd916569b80 1 -- 172.21.15.131:0/1047983834 >> 172.21.15.131:0/1047983834 conn(0x5609539e7290 msgr2=0x560953a8dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:38.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.965+0000 7fd916569b80 1 -- 172.21.15.131:0/1047983834 shutdown_connections 2024-05-08T22:14:38.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.965+0000 7fd916569b80 1 -- 172.21.15.131:0/1047983834 wait complete. 2024-05-08T22:14:38.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.966+0000 7fd916569b80 1 -- start start 2024-05-08T22:14:38.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.966+0000 7fd916569b80 1 Processor -- start 2024-05-08T22:14:38.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.966+0000 7fd916569b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560953a89910 0x560953a98ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:38.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.966+0000 7fd916569b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560953a8ee80 0x560953a99420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:38.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.966+0000 7fd916569b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560953a8f7a0 0x560953a99960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:38.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.966+0000 7fd916569b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560953af9870 con 0x560953a8ee80 2024-05-08T22:14:38.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.966+0000 7fd916569b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560953af99e0 con 0x560953a89910 2024-05-08T22:14:38.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.966+0000 7fd916569b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560953af9ba0 con 0x560953a8f7a0 2024-05-08T22:14:38.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.967+0000 7fd916569b80 10 client.0 _ll_register_callbacks cb 0x56095397ebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:14:38.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.967+0000 7fd8fbe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560953a89910 0x560953a98ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:38.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.967+0000 7fd8fbe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560953a89910 0x560953a98ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:57788/0 (socket says 172.21.15.131:57788) 2024-05-08T22:14:38.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.967+0000 7fd8fbe00700 1 -- 172.21.15.131:0/4270707169 learned_addr learned my addr 172.21.15.131:0/4270707169 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:38.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.967+0000 7fd900a00700 1 --2- 172.21.15.131:0/4270707169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560953a8f7a0 0x560953a99960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:38.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.967+0000 7fd8fb400700 1 --2- 172.21.15.131:0/4270707169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560953a8ee80 0x560953a99420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:38.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.967+0000 7fd8fbe00700 1 -- 172.21.15.131:0/4270707169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560953a8f7a0 msgr2=0x560953a99960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:38.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.967+0000 7fd8fbe00700 1 --2- 172.21.15.131:0/4270707169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560953a8f7a0 0x560953a99960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.967+0000 7fd8fbe00700 1 -- 172.21.15.131:0/4270707169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560953a8ee80 msgr2=0x560953a99420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:38.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.967+0000 7fd8fbe00700 1 --2- 172.21.15.131:0/4270707169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560953a8ee80 0x560953a99420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.967+0000 7fd8fbe00700 1 -- 172.21.15.131:0/4270707169 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8fc1a6720 con 0x560953a89910 2024-05-08T22:14:38.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.967+0000 7fd8fb400700 1 --2- 172.21.15.131:0/4270707169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560953a8ee80 0x560953a99420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:14:38.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.967+0000 7fd8fbe00700 1 --2- 172.21.15.131:0/4270707169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560953a89910 0x560953a98ee0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec099f90 tx=0x7fd8ec0caba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb786606ad06950c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:38.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.967+0000 7fd8e5600700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:14:38.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.968+0000 7fd8e5600700 1 -- 172.21.15.131:0/4270707169 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec0994f0 con 0x560953a89910 2024-05-08T22:14:38.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.968+0000 7fd8e5600700 1 -- 172.21.15.131:0/4270707169 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fd8ec082020 con 0x560953a89910 2024-05-08T22:14:38.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.968+0000 7fd916569b80 1 -- 172.21.15.131:0/4270707169 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8ec09de30 con 0x560953a89910 2024-05-08T22:14:38.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.968+0000 7fd8e5600700 1 -- 172.21.15.131:0/4270707169 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8ec099650 con 0x560953a89910 2024-05-08T22:14:38.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.968+0000 7fd916569b80 10 client.26339 fetch_fsmap learned FSMap version 27 2024-05-08T22:14:38.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.968+0000 7fd916569b80 1 -- 172.21.15.131:0/4270707169 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560953a89fc0 con 0x560953a89910 2024-05-08T22:14:38.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.968+0000 7fd8e5600700 1 -- 172.21.15.131:0/4270707169 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8ec09de30 con 0x560953a89910 2024-05-08T22:14:38.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.968+0000 7fd8e5600700 1 -- 172.21.15.131:0/4270707169 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 27) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fd8ec0a1ff0 con 0x560953a89910 2024-05-08T22:14:38.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.968+0000 7fd916569b80 10 client.26339 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T22:14:38.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.968+0000 7fd916569b80 10 client.26339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:14:38.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.968+0000 7fd916569b80 20 client.26339 populate_metadata read hostname 'smithi131' 2024-05-08T22:14:38.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.968+0000 7fd916569b80 1 --2- 172.21.15.131:0/4270707169 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560953afb030 0x560953afd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:38.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.968+0000 7fd916569b80 4 client.26339 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T22:14:38.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.968+0000 7fd916569b80 1 -- 172.21.15.131:0/4270707169 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5609539d1f40 con 0x560953afb030 2024-05-08T22:14:38.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.969+0000 7fd8fb400700 1 --2- 172.21.15.131:0/4270707169 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560953afb030 0x560953afd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:38.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.969+0000 7fd8e5600700 10 client.26339 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:14:38.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.969+0000 7fd8fb400700 1 --2- 172.21.15.131:0/4270707169 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560953afb030 0x560953afd450 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4de31071350dea27 server_cookie=87e50204f83d7b62 in_seq=0 out_seq=0 2024-05-08T22:14:38.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.987+0000 7fd8e5600700 1 -- 172.21.15.131:0/4270707169 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5609539d1f40 con 0x560953afb030 2024-05-08T22:14:38.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.987+0000 7fd8e5600700 10 client.26339 handle_command_reply: tid=0 2024-05-08T22:14:38.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.987+0000 7fd916569b80 1 client.26339 shutdown 2024-05-08T22:14:38.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.987+0000 7fd916569b80 1 -- 172.21.15.131:0/4270707169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560953a89910 msgr2=0x560953a98ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:38.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.987+0000 7fd916569b80 1 --2- 172.21.15.131:0/4270707169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560953a89910 0x560953a98ee0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec099f90 tx=0x7fd8ec0caba0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.987+0000 7fd916569b80 1 -- 172.21.15.131:0/4270707169 shutdown_connections 2024-05-08T22:14:38.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.987+0000 7fd916569b80 1 --2- 172.21.15.131:0/4270707169 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560953afb030 0x560953afd450 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.987+0000 7fd916569b80 1 --2- 172.21.15.131:0/4270707169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560953a8f7a0 0x560953a99960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.987+0000 7fd916569b80 1 --2- 172.21.15.131:0/4270707169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560953a89910 0x560953a98ee0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.987+0000 7fd8e5600700 0 client.26339 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:14:38.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.987+0000 7fd916569b80 1 --2- 172.21.15.131:0/4270707169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560953a8ee80 0x560953a99420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.987+0000 7fd916569b80 1 -- 172.21.15.131:0/4270707169 >> 172.21.15.131:0/4270707169 conn(0x5609539e7290 msgr2=0x560953a8fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:38.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.987+0000 7fd916569b80 1 -- 172.21.15.131:0/4270707169 shutdown_connections 2024-05-08T22:14:38.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.988+0000 7fd916569b80 1 -- 172.21.15.131:0/4270707169 wait complete. 2024-05-08T22:14:38.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.988+0000 7fd916569b80 20 client.26339 trim_cache size 0 max 16384 2024-05-08T22:14:38.990 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:14:38.990 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-08T22:14:38.990 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "162f8615-a983-4630-9a0e-7942b9cf2632", 2024-05-08T22:14:38.990 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-08T22:14:38.990 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:14:38.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.988+0000 7fd902a00700 1 -- 172.21.15.131:0/652643654 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fd8e00a1310 msgr2=0x7fd8e00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:38.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.988+0000 7fd902a00700 1 --2- 172.21.15.131:0/652643654 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fd8e00a1310 0x7fd8e00a37d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc19ab50 tx=0x7fd8e800b260 comp rx=0 tx=0).stop 2024-05-08T22:14:38.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.988+0000 7fd902a00700 1 -- 172.21.15.131:0/652643654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8fc101180 msgr2=0x7fd8fc199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:38.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.988+0000 7fd902a00700 1 --2- 172.21.15.131:0/652643654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8fc101180 0x7fd8fc199600 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00e9a0 tx=0x7fd8ec00ecb0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.988+0000 7fd902a00700 1 -- 172.21.15.131:0/652643654 shutdown_connections 2024-05-08T22:14:38.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.988+0000 7fd902a00700 1 --2- 172.21.15.131:0/652643654 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fd8e00a1310 0x7fd8e00a37d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.988+0000 7fd902a00700 1 --2- 172.21.15.131:0/652643654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8fc19dee0 0x7fd8fc19e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.988+0000 7fd902a00700 1 --2- 172.21.15.131:0/652643654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8fc101180 0x7fd8fc199600 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.988+0000 7fd902a00700 1 --2- 172.21.15.131:0/652643654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8fc1062a0 0x7fd8fc199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:38.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.988+0000 7fd902a00700 1 -- 172.21.15.131:0/652643654 >> 172.21.15.131:0/652643654 conn(0x7fd8fc0fc9e0 msgr2=0x7fd8fc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:38.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.988+0000 7fd902a00700 1 -- 172.21.15.131:0/652643654 shutdown_connections 2024-05-08T22:14:38.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:38.988+0000 7fd902a00700 1 -- 172.21.15.131:0/652643654 wait complete. 2024-05-08T22:14:39.011 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:14:39.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.214+0000 7fb705600700 1 -- 172.21.15.131:0/2932121079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7000733d0 msgr2=0x7fb7000737b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:39.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.214+0000 7fb705600700 1 --2- 172.21.15.131:0/2932121079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7000733d0 0x7fb7000737b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0009990 tx=0x7fb6f0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.214+0000 7fb705600700 1 -- 172.21.15.131:0/2932121079 shutdown_connections 2024-05-08T22:14:39.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.214+0000 7fb705600700 1 --2- 172.21.15.131:0/2932121079 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb70010d730 0x7fb70010fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.214+0000 7fb705600700 1 --2- 172.21.15.131:0/2932121079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7000733d0 0x7fb7000737b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.214+0000 7fb705600700 1 --2- 172.21.15.131:0/2932121079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb700073cf0 0x7fb70010d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.214+0000 7fb705600700 1 -- 172.21.15.131:0/2932121079 >> 172.21.15.131:0/2932121079 conn(0x7fb7000fc9c0 msgr2=0x7fb7000fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:39.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.214+0000 7fb705600700 1 -- 172.21.15.131:0/2932121079 shutdown_connections 2024-05-08T22:14:39.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.214+0000 7fb705600700 1 -- 172.21.15.131:0/2932121079 wait complete. 2024-05-08T22:14:39.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.215+0000 7fb705600700 1 Processor -- start 2024-05-08T22:14:39.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.215+0000 7fb705600700 1 -- start start 2024-05-08T22:14:39.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.215+0000 7fb705600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7000733d0 0x7fb700199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:39.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.215+0000 7fb705600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb700073cf0 0x7fb700199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:39.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.215+0000 7fb705600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb70010d730 0x7fb70019dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:39.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.215+0000 7fb705600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70019e4e0 con 0x7fb700073cf0 2024-05-08T22:14:39.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.215+0000 7fb705600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70019e650 con 0x7fb70010d730 2024-05-08T22:14:39.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.215+0000 7fb705600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70019e810 con 0x7fb7000733d0 2024-05-08T22:14:39.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.216+0000 7fb6fea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7000733d0 0x7fb700199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:39.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.216+0000 7fb6f7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb700073cf0 0x7fb700199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:39.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.216+0000 7fb6fea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7000733d0 0x7fb700199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34514/0 (socket says 172.21.15.131:34514) 2024-05-08T22:14:39.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.216+0000 7fb6fea00700 1 -- 172.21.15.131:0/694509582 learned_addr learned my addr 172.21.15.131:0/694509582 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:39.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.216+0000 7fb6ff400700 1 --2- 172.21.15.131:0/694509582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb70010d730 0x7fb70019dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:39.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.216+0000 7fb6fea00700 1 -- 172.21.15.131:0/694509582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb70010d730 msgr2=0x7fb70019dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:39.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.216+0000 7fb6fea00700 1 --2- 172.21.15.131:0/694509582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb70010d730 0x7fb70019dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.216+0000 7fb6fea00700 1 -- 172.21.15.131:0/694509582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb700073cf0 msgr2=0x7fb700199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:39.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.216+0000 7fb6fea00700 1 --2- 172.21.15.131:0/694509582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb700073cf0 0x7fb700199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.216+0000 7fb6fea00700 1 -- 172.21.15.131:0/694509582 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f00095f0 con 0x7fb7000733d0 2024-05-08T22:14:39.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.216+0000 7fb6fea00700 1 --2- 172.21.15.131:0/694509582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7000733d0 0x7fb700199610 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0000c00 tx=0x7fb6f0005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:39.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.216+0000 7fb6f7400700 1 -- 172.21.15.131:0/694509582 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f001d070 con 0x7fb7000733d0 2024-05-08T22:14:39.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.216+0000 7fb6f7400700 1 -- 172.21.15.131:0/694509582 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6f0022470 con 0x7fb7000733d0 2024-05-08T22:14:39.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.216+0000 7fb705600700 1 -- 172.21.15.131:0/694509582 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb70019ea90 con 0x7fb7000733d0 2024-05-08T22:14:39.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.216+0000 7fb6f7400700 1 -- 172.21.15.131:0/694509582 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0017650 con 0x7fb7000733d0 2024-05-08T22:14:39.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.217+0000 7fb705600700 1 -- 172.21.15.131:0/694509582 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7001a6330 con 0x7fb7000733d0 2024-05-08T22:14:39.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.219+0000 7fb6f7400700 1 -- 172.21.15.131:0/694509582 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fb6f00225e0 con 0x7fb7000733d0 2024-05-08T22:14:39.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.221+0000 7fb6f7400700 1 --2- 172.21.15.131:0/694509582 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb6e40a1290 0x7fb6e40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:39.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.221+0000 7fb6f7400700 1 -- 172.21.15.131:0/694509582 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fb6f00ce4e0 con 0x7fb7000733d0 2024-05-08T22:14:39.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.221+0000 7fb71907bb80 1 Processor -- start 2024-05-08T22:14:39.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.221+0000 7fb6f7e00700 1 --2- 172.21.15.131:0/694509582 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb6e40a1290 0x7fb6e40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:39.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.221+0000 7fb71907bb80 1 -- start start 2024-05-08T22:14:39.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.221+0000 7fb71907bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac255cce00 0x55ac255cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:39.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.221+0000 7fb71907bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255cd720 0x55ac255c7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:39.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.221+0000 7fb6f7e00700 1 --2- 172.21.15.131:0/694509582 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb6e40a1290 0x7fb6e40a3750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb70019ab10 tx=0x7fb6e8009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:39.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.222+0000 7fb71907bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac255c7890 0x55ac255c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:39.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.222+0000 7fb71907bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2551e150 con 0x55ac255c7890 2024-05-08T22:14:39.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.222+0000 7fb71907bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2551bb10 con 0x55ac255cd720 2024-05-08T22:14:39.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.222+0000 7fb71907bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac25517400 con 0x55ac255cce00 2024-05-08T22:14:39.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.222+0000 7fb6f7e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255cd720 0x55ac255c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:39.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.222+0000 7fb6f7e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255cd720 0x55ac255c7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:57810/0 (socket says 172.21.15.131:57810) 2024-05-08T22:14:39.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.222+0000 7fb6f7e00700 1 -- 172.21.15.131:0/4048685674 learned_addr learned my addr 172.21.15.131:0/4048685674 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:39.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.222+0000 7fb6fea00700 1 --2- 172.21.15.131:0/4048685674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac255cce00 0x55ac255cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:39.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.222+0000 7fb6ff400700 1 --2- 172.21.15.131:0/4048685674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac255c7890 0x55ac255c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:39.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.222+0000 7fb6f7e00700 1 -- 172.21.15.131:0/4048685674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac255cce00 msgr2=0x55ac255cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:39.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.222+0000 7fb6f7e00700 1 --2- 172.21.15.131:0/4048685674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac255cce00 0x55ac255cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.222+0000 7fb6f7e00700 1 -- 172.21.15.131:0/4048685674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac255c7890 msgr2=0x55ac255c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:39.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.222+0000 7fb6f7e00700 1 --2- 172.21.15.131:0/4048685674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac255c7890 0x55ac255c7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.222+0000 7fb6f7e00700 1 -- 172.21.15.131:0/4048685674 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac253983b0 con 0x55ac255cd720 2024-05-08T22:14:39.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.222+0000 7fb6ff400700 1 --2- 172.21.15.131:0/4048685674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac255c7890 0x55ac255c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:14:39.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.222+0000 7fb6f7e00700 1 --2- 172.21.15.131:0/4048685674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255cd720 0x55ac255c7350 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8011d50 tx=0x7fb6e8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc9eaee0b95f854d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:39.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.223+0000 7fb6ef400700 1 -- 172.21.15.131:0/4048685674 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e800ebd0 con 0x55ac255cd720 2024-05-08T22:14:39.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.223+0000 7fb6ef400700 1 -- 172.21.15.131:0/4048685674 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6e8021070 con 0x55ac255cd720 2024-05-08T22:14:39.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.223+0000 7fb6ef400700 1 -- 172.21.15.131:0/4048685674 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e801d670 con 0x55ac255cd720 2024-05-08T22:14:39.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.223+0000 7fb71907bb80 1 -- 172.21.15.131:0/4048685674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255cd720 msgr2=0x55ac255c7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:39.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.223+0000 7fb71907bb80 1 --2- 172.21.15.131:0/4048685674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255cd720 0x55ac255c7350 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8011d50 tx=0x7fb6e8014000 comp rx=0 tx=0).stop 2024-05-08T22:14:39.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.223+0000 7fb71907bb80 1 -- 172.21.15.131:0/4048685674 shutdown_connections 2024-05-08T22:14:39.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.223+0000 7fb71907bb80 1 --2- 172.21.15.131:0/4048685674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac255c7890 0x55ac255c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.223+0000 7fb71907bb80 1 --2- 172.21.15.131:0/4048685674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac255cce00 0x55ac255cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.223+0000 7fb71907bb80 1 --2- 172.21.15.131:0/4048685674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255cd720 0x55ac255c7350 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.223+0000 7fb71907bb80 1 -- 172.21.15.131:0/4048685674 >> 172.21.15.131:0/4048685674 conn(0x55ac25525210 msgr2=0x55ac255c1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:39.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.224+0000 7fb71907bb80 1 -- 172.21.15.131:0/4048685674 shutdown_connections 2024-05-08T22:14:39.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.224+0000 7fb71907bb80 1 -- 172.21.15.131:0/4048685674 wait complete. 2024-05-08T22:14:39.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.224+0000 7fb71907bb80 1 -- start start 2024-05-08T22:14:39.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.224+0000 7fb71907bb80 1 Processor -- start 2024-05-08T22:14:39.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.225+0000 7fb71907bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac255c7890 0x55ac255d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:39.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.225+0000 7fb71907bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255cce00 0x55ac255d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:39.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.225+0000 7fb71907bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac255cd720 0x55ac255d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:39.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.225+0000 7fb71907bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac254b3560 con 0x55ac255cd720 2024-05-08T22:14:39.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.225+0000 7fb71907bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac254b3200 con 0x55ac255cce00 2024-05-08T22:14:39.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.225+0000 7fb71907bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2539a6a0 con 0x55ac255c7890 2024-05-08T22:14:39.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.225+0000 7fb71907bb80 10 client.0 _ll_register_callbacks cb 0x55ac254bcb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:14:39.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.225+0000 7fb6f7e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255cce00 0x55ac255d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:39.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.225+0000 7fb6fea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac255c7890 0x55ac255d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:39.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.225+0000 7fb6f7e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255cce00 0x55ac255d10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:57824/0 (socket says 172.21.15.131:57824) 2024-05-08T22:14:39.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.225+0000 7fb6fea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac255c7890 0x55ac255d0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34540/0 (socket says 172.21.15.131:34540) 2024-05-08T22:14:39.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.225+0000 7fb6f7e00700 1 -- 172.21.15.131:0/1793375253 learned_addr learned my addr 172.21.15.131:0/1793375253 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:39.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.225+0000 7fb6ff400700 1 --2- 172.21.15.131:0/1793375253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac255cd720 0x55ac255d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:39.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.225+0000 7fb6f7e00700 1 -- 172.21.15.131:0/1793375253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac255c7890 msgr2=0x55ac255d0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:39.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.225+0000 7fb6f7e00700 1 --2- 172.21.15.131:0/1793375253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac255c7890 0x55ac255d0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.225+0000 7fb6f7e00700 1 -- 172.21.15.131:0/1793375253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac255cd720 msgr2=0x55ac255d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:39.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.225+0000 7fb6f7e00700 1 --2- 172.21.15.131:0/1793375253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac255cd720 0x55ac255d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.225+0000 7fb6f7e00700 1 -- 172.21.15.131:0/1793375253 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e8017070 con 0x55ac255cce00 2024-05-08T22:14:39.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.226+0000 7fb6f7e00700 1 --2- 172.21.15.131:0/1793375253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255cce00 0x55ac255d10f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800ea20 tx=0x7fb6e801cd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3a02a8982140166 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:39.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.226+0000 7fb6e3e00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:14:39.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.226+0000 7fb6e3e00700 1 -- 172.21.15.131:0/1793375253 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e800ede0 con 0x55ac255cce00 2024-05-08T22:14:39.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.226+0000 7fb71907bb80 1 -- 172.21.15.131:0/1793375253 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac25470270 con 0x55ac255cce00 2024-05-08T22:14:39.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.226+0000 7fb6e3e00700 1 -- 172.21.15.131:0/1793375253 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fb6e801c410 con 0x55ac255cce00 2024-05-08T22:14:39.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.226+0000 7fb6e3e00700 1 -- 172.21.15.131:0/1793375253 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6e801c5d0 con 0x55ac255cce00 2024-05-08T22:14:39.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.226+0000 7fb71907bb80 10 client.26351 fetch_fsmap learned FSMap version 27 2024-05-08T22:14:39.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.226+0000 7fb71907bb80 1 -- 172.21.15.131:0/1793375253 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ac25344b60 con 0x55ac255cce00 2024-05-08T22:14:39.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.226+0000 7fb6e3e00700 1 -- 172.21.15.131:0/1793375253 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ac25470270 con 0x55ac255cce00 2024-05-08T22:14:39.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.227+0000 7fb6e3e00700 1 -- 172.21.15.131:0/1793375253 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 27) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fb6e80036f0 con 0x55ac255cce00 2024-05-08T22:14:39.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.227+0000 7fb71907bb80 10 client.26351 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T22:14:39.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.227+0000 7fb71907bb80 10 client.26351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:14:39.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.227+0000 7fb71907bb80 20 client.26351 populate_metadata read hostname 'smithi131' 2024-05-08T22:14:39.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.227+0000 7fb71907bb80 1 --2- 172.21.15.131:0/1793375253 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ac255e1240 0x55ac255e3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:39.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.227+0000 7fb71907bb80 4 client.26351 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:14:39.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.227+0000 7fb71907bb80 1 -- 172.21.15.131:0/1793375253 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac25596810 con 0x55ac255e1240 2024-05-08T22:14:39.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.227+0000 7fb6fea00700 1 --2- 172.21.15.131:0/1793375253 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ac255e1240 0x55ac255e3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:39.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.227+0000 7fb6e3e00700 10 client.26351 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:14:39.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.227+0000 7fb6fea00700 1 --2- 172.21.15.131:0/1793375253 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ac255e1240 0x55ac255e3660 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70bffd7cd637f908 server_cookie=993eb1a1e107a790 in_seq=0 out_seq=0 2024-05-08T22:14:39.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.228+0000 7fb6e3e00700 1 -- 172.21.15.131:0/1793375253 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ac25596810 con 0x55ac255e1240 2024-05-08T22:14:39.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.228+0000 7fb6e3e00700 10 client.26351 handle_command_reply: tid=0 2024-05-08T22:14:39.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.228+0000 7fb71907bb80 1 client.26351 shutdown 2024-05-08T22:14:39.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.228+0000 7fb71907bb80 1 -- 172.21.15.131:0/1793375253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255cce00 msgr2=0x55ac255d10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:39.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.228+0000 7fb71907bb80 1 --2- 172.21.15.131:0/1793375253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255cce00 0x55ac255d10f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800ea20 tx=0x7fb6e801cd80 comp rx=0 tx=0).stop 2024-05-08T22:14:39.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.229+0000 7fb71907bb80 1 -- 172.21.15.131:0/1793375253 shutdown_connections 2024-05-08T22:14:39.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.229+0000 7fb71907bb80 1 --2- 172.21.15.131:0/1793375253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac255cd720 0x55ac255d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.229+0000 7fb71907bb80 1 --2- 172.21.15.131:0/1793375253 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ac255e1240 0x55ac255e3660 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.229+0000 7fb71907bb80 1 --2- 172.21.15.131:0/1793375253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac255c7890 0x55ac255d0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.229+0000 7fb6e3e00700 0 client.26351 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:14:39.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.229+0000 7fb71907bb80 1 --2- 172.21.15.131:0/1793375253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255cce00 0x55ac255d10f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.229+0000 7fb71907bb80 1 -- 172.21.15.131:0/1793375253 >> 172.21.15.131:0/1793375253 conn(0x55ac25525210 msgr2=0x55ac255c15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:39.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.229+0000 7fb71907bb80 1 -- 172.21.15.131:0/1793375253 shutdown_connections 2024-05-08T22:14:39.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.229+0000 7fb71907bb80 1 -- 172.21.15.131:0/1793375253 wait complete. 2024-05-08T22:14:39.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.229+0000 7fb71907bb80 20 client.26351 trim_cache size 0 max 16384 2024-05-08T22:14:39.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.246+0000 7fb71907bb80 1 Processor -- start 2024-05-08T22:14:39.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.246+0000 7fb71907bb80 1 -- start start 2024-05-08T22:14:39.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.247+0000 7fb71907bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac255c7890 0x55ac255dcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:39.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.247+0000 7fb71907bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac255cce00 0x55ac255c1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:39.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.247+0000 7fb71907bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255cd720 0x55ac255d8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:39.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.247+0000 7fb71907bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac25596810 con 0x55ac255cce00 2024-05-08T22:14:39.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.247+0000 7fb71907bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac25470270 con 0x55ac255cd720 2024-05-08T22:14:39.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.247+0000 7fb71907bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac255d3740 con 0x55ac255c7890 2024-05-08T22:14:39.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.247+0000 7fb6fea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac255c7890 0x55ac255dcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:39.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.247+0000 7fb6ff400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255cd720 0x55ac255d8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:39.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.247+0000 7fb6fea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac255c7890 0x55ac255dcc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34548/0 (socket says 172.21.15.131:34548) 2024-05-08T22:14:39.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.247+0000 7fb6fea00700 1 -- 172.21.15.131:0/1273131962 learned_addr learned my addr 172.21.15.131:0/1273131962 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:39.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.247+0000 7fb6f7e00700 1 --2- 172.21.15.131:0/1273131962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac255cce00 0x55ac255c1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:39.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.247+0000 7fb6fea00700 1 -- 172.21.15.131:0/1273131962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255cd720 msgr2=0x55ac255d8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:39.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.247+0000 7fb6fea00700 1 --2- 172.21.15.131:0/1273131962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255cd720 0x55ac255d8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.247+0000 7fb6fea00700 1 -- 172.21.15.131:0/1273131962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac255cce00 msgr2=0x55ac255c1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:39.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.247+0000 7fb6fea00700 1 --2- 172.21.15.131:0/1273131962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac255cce00 0x55ac255c1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.247+0000 7fb6fea00700 1 -- 172.21.15.131:0/1273131962 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac254dd560 con 0x55ac255c7890 2024-05-08T22:14:39.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.247+0000 7fb6ff400700 1 --2- 172.21.15.131:0/1273131962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255cd720 0x55ac255d8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:14:39.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.248+0000 7fb6fea00700 1 --2- 172.21.15.131:0/1273131962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac255c7890 0x55ac255dcc20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00cb840 tx=0x7fb6f00ce040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f89a7c26ea56480 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:39.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.248+0000 7fb6efe00700 1 -- 172.21.15.131:0/1273131962 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f00b7070 con 0x55ac255c7890 2024-05-08T22:14:39.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.248+0000 7fb6efe00700 1 -- 172.21.15.131:0/1273131962 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6f009ae90 con 0x55ac255c7890 2024-05-08T22:14:39.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.248+0000 7fb6efe00700 1 -- 172.21.15.131:0/1273131962 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f00a3740 con 0x55ac255c7890 2024-05-08T22:14:39.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.249+0000 7fb71907bb80 1 -- 172.21.15.131:0/1273131962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac255c7890 msgr2=0x55ac255dcc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:39.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.249+0000 7fb71907bb80 1 --2- 172.21.15.131:0/1273131962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac255c7890 0x55ac255dcc20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00cb840 tx=0x7fb6f00ce040 comp rx=0 tx=0).stop 2024-05-08T22:14:39.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.249+0000 7fb71907bb80 1 -- 172.21.15.131:0/1273131962 shutdown_connections 2024-05-08T22:14:39.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.249+0000 7fb71907bb80 1 --2- 172.21.15.131:0/1273131962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255cd720 0x55ac255d8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.249+0000 7fb71907bb80 1 --2- 172.21.15.131:0/1273131962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac255c7890 0x55ac255dcc20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.249+0000 7fb71907bb80 1 --2- 172.21.15.131:0/1273131962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac255cce00 0x55ac255c1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.249+0000 7fb71907bb80 1 -- 172.21.15.131:0/1273131962 >> 172.21.15.131:0/1273131962 conn(0x55ac25525210 msgr2=0x55ac255be210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:39.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.249+0000 7fb71907bb80 1 -- 172.21.15.131:0/1273131962 shutdown_connections 2024-05-08T22:14:39.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.249+0000 7fb71907bb80 1 -- 172.21.15.131:0/1273131962 wait complete. 2024-05-08T22:14:39.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.249+0000 7fb71907bb80 1 -- start start 2024-05-08T22:14:39.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.250+0000 7fb71907bb80 1 Processor -- start 2024-05-08T22:14:39.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.250+0000 7fb71907bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255c7890 0x55ac2562b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:39.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.250+0000 7fb71907bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac255cce00 0x55ac2562bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:39.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.250+0000 7fb71907bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac255cd720 0x55ac2562c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:39.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.250+0000 7fb71907bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac25636710 con 0x55ac255cce00 2024-05-08T22:14:39.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.250+0000 7fb71907bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac25636850 con 0x55ac255c7890 2024-05-08T22:14:39.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.250+0000 7fb71907bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac25636990 con 0x55ac255cd720 2024-05-08T22:14:39.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.250+0000 7fb71907bb80 10 client.0 _ll_register_callbacks cb 0x55ac254bcb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:14:39.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.250+0000 7fb6fea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255c7890 0x55ac2562b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:39.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.251+0000 7fb6fea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255c7890 0x55ac2562b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:57834/0 (socket says 172.21.15.131:57834) 2024-05-08T22:14:39.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.251+0000 7fb6f7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac255cce00 0x55ac2562bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:39.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.251+0000 7fb6ff400700 1 --2- 172.21.15.131:0/3266184368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac255cd720 0x55ac2562c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:39.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.251+0000 7fb6fea00700 1 -- 172.21.15.131:0/3266184368 learned_addr learned my addr 172.21.15.131:0/3266184368 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:14:39.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.251+0000 7fb6fea00700 1 -- 172.21.15.131:0/3266184368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac255cd720 msgr2=0x55ac2562c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:39.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.251+0000 7fb6fea00700 1 --2- 172.21.15.131:0/3266184368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac255cd720 0x55ac2562c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.251+0000 7fb6fea00700 1 -- 172.21.15.131:0/3266184368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac255cce00 msgr2=0x55ac2562bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:14:39.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.251+0000 7fb6fea00700 1 --2- 172.21.15.131:0/3266184368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac255cce00 0x55ac2562bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.251+0000 7fb6fea00700 1 -- 172.21.15.131:0/3266184368 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f8009160 con 0x55ac255c7890 2024-05-08T22:14:39.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.251+0000 7fb6f7e00700 1 --2- 172.21.15.131:0/3266184368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac255cce00 0x55ac2562bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:14:39.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.251+0000 7fb6ff400700 1 --2- 172.21.15.131:0/3266184368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac255cd720 0x55ac2562c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:14:39.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.251+0000 7fb6fea00700 1 --2- 172.21.15.131:0/3266184368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255c7890 0x55ac2562b690 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb6f009b2e0 tx=0x7fb6f009acb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=127b7b2d87ba3843 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:14:39.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.251+0000 7fb6e3e00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:14:39.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.251+0000 7fb6e3e00700 1 -- 172.21.15.131:0/3266184368 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f007e070 con 0x55ac255c7890 2024-05-08T22:14:39.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.251+0000 7fb6e3e00700 1 -- 172.21.15.131:0/3266184368 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fb6f00b0040 con 0x55ac255c7890 2024-05-08T22:14:39.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.251+0000 7fb71907bb80 1 -- 172.21.15.131:0/3266184368 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6e8021070 con 0x55ac255c7890 2024-05-08T22:14:39.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.252+0000 7fb6e3e00700 1 -- 172.21.15.131:0/3266184368 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6f00b7070 con 0x55ac255c7890 2024-05-08T22:14:39.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.252+0000 7fb71907bb80 10 client.26357 fetch_fsmap learned FSMap version 27 2024-05-08T22:14:39.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.252+0000 7fb71907bb80 1 -- 172.21.15.131:0/3266184368 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fb7001a6330 con 0x55ac255c7890 2024-05-08T22:14:39.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.252+0000 7fb6e3e00700 1 -- 172.21.15.131:0/3266184368 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6f00a7020 con 0x55ac255c7890 2024-05-08T22:14:39.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.252+0000 7fb6e3e00700 1 -- 172.21.15.131:0/3266184368 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 27) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fb6f0081030 con 0x55ac255c7890 2024-05-08T22:14:39.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.252+0000 7fb71907bb80 10 client.26357 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T22:14:39.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.252+0000 7fb71907bb80 10 client.26357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:14:39.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.252+0000 7fb71907bb80 20 client.26357 populate_metadata read hostname 'smithi131' 2024-05-08T22:14:39.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.252+0000 7fb71907bb80 1 --2- 172.21.15.131:0/3266184368 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ac25637ca0 0x55ac2563a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:14:39.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.252+0000 7fb71907bb80 4 client.26357 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:14:39.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.252+0000 7fb71907bb80 1 -- 172.21.15.131:0/3266184368 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ac2550fec0 con 0x55ac25637ca0 2024-05-08T22:14:39.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.252+0000 7fb6f7e00700 1 --2- 172.21.15.131:0/3266184368 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ac25637ca0 0x55ac2563a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:14:39.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.253+0000 7fb6f7e00700 1 --2- 172.21.15.131:0/3266184368 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ac25637ca0 0x55ac2563a0c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a22b9a1930912b5 server_cookie=869933144d6e5563 in_seq=0 out_seq=0 2024-05-08T22:14:39.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.253+0000 7fb6e3e00700 10 client.26357 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:14:39.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.258+0000 7fb6e3e00700 1 -- 172.21.15.131:0/3266184368 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x55ac2550fec0 con 0x55ac25637ca0 2024-05-08T22:14:39.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.258+0000 7fb6e3e00700 10 client.26357 handle_command_reply: tid=0 2024-05-08T22:14:39.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.258+0000 7fb71907bb80 1 client.26357 shutdown 2024-05-08T22:14:39.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.258+0000 7fb71907bb80 1 -- 172.21.15.131:0/3266184368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255c7890 msgr2=0x55ac2562b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:39.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.258+0000 7fb71907bb80 1 --2- 172.21.15.131:0/3266184368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255c7890 0x55ac2562b690 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb6f009b2e0 tx=0x7fb6f009acb0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.258+0000 7fb71907bb80 1 -- 172.21.15.131:0/3266184368 shutdown_connections 2024-05-08T22:14:39.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.258+0000 7fb71907bb80 1 --2- 172.21.15.131:0/3266184368 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55ac25637ca0 0x55ac2563a0c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.258+0000 7fb71907bb80 1 --2- 172.21.15.131:0/3266184368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac255cd720 0x55ac2562c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.258+0000 7fb6e3e00700 0 client.26357 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:14:39.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.258+0000 7fb71907bb80 1 --2- 172.21.15.131:0/3266184368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac255c7890 0x55ac2562b690 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.258+0000 7fb71907bb80 1 --2- 172.21.15.131:0/3266184368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac255cce00 0x55ac2562bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.258+0000 7fb71907bb80 1 -- 172.21.15.131:0/3266184368 >> 172.21.15.131:0/3266184368 conn(0x55ac25525210 msgr2=0x55ac255cad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:39.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.258+0000 7fb71907bb80 1 -- 172.21.15.131:0/3266184368 shutdown_connections 2024-05-08T22:14:39.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.258+0000 7fb71907bb80 1 -- 172.21.15.131:0/3266184368 wait complete. 2024-05-08T22:14:39.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.258+0000 7fb71907bb80 20 client.26357 trim_cache size 0 max 16384 2024-05-08T22:14:39.260 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:14:39.260 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "scrub active (3204 inodes in the stack)", 2024-05-08T22:14:39.261 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-08T22:14:39.261 INFO:teuthology.orchestra.run.smithi131.stdout: "162f8615-a983-4630-9a0e-7942b9cf2632": { 2024-05-08T22:14:39.261 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-08T22:14:39.261 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "162f8615-a983-4630-9a0e-7942b9cf2632", 2024-05-08T22:14:39.261 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-08T22:14:39.261 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:14:39.261 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:14:39.261 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:14:39.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.259+0000 7fb705600700 1 -- 172.21.15.131:0/694509582 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb6e40a1290 msgr2=0x7fb6e40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:39.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.259+0000 7fb705600700 1 --2- 172.21.15.131:0/694509582 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb6e40a1290 0x7fb6e40a3750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb70019ab10 tx=0x7fb6e8009260 comp rx=0 tx=0).stop 2024-05-08T22:14:39.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.259+0000 7fb705600700 1 -- 172.21.15.131:0/694509582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7000733d0 msgr2=0x7fb700199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:14:39.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.259+0000 7fb705600700 1 --2- 172.21.15.131:0/694509582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7000733d0 0x7fb700199610 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0000c00 tx=0x7fb6f0005740 comp rx=0 tx=0).stop 2024-05-08T22:14:39.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.259+0000 7fb705600700 1 -- 172.21.15.131:0/694509582 shutdown_connections 2024-05-08T22:14:39.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.259+0000 7fb705600700 1 --2- 172.21.15.131:0/694509582 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb6e40a1290 0x7fb6e40a3750 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.259+0000 7fb705600700 1 --2- 172.21.15.131:0/694509582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb70010d730 0x7fb70019dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.259+0000 7fb705600700 1 --2- 172.21.15.131:0/694509582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7000733d0 0x7fb700199610 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.259+0000 7fb705600700 1 --2- 172.21.15.131:0/694509582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb700073cf0 0x7fb700199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:14:39.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.259+0000 7fb705600700 1 -- 172.21.15.131:0/694509582 >> 172.21.15.131:0/694509582 conn(0x7fb7000fc9c0 msgr2=0x7fb7000fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:14:39.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.259+0000 7fb705600700 1 -- 172.21.15.131:0/694509582 shutdown_connections 2024-05-08T22:14:39.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:14:39.259+0000 7fb705600700 1 -- 172.21.15.131:0/694509582 wait complete. 2024-05-08T22:14:39.293 INFO:tasks.cephfs.filesystem:scrub status for tag:162f8615-a983-4630-9a0e-7942b9cf2632 - {'path': '/', 'tag': '162f8615-a983-4630-9a0e-7942b9cf2632', 'options': 'recursive,force'} 2024-05-08T22:14:40.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:40 smithi157 ceph-mon[77746]: scrub queued for path: / 2024-05-08T22:14:40.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:40 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:40.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:40 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:14:40.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:40 smithi157 ceph-mon[77746]: pgmap v226: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.8 MiB/s wr, 1.16k op/s 2024-05-08T22:14:40.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:40 smithi157 ceph-mon[77746]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T22:14:40.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:40 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:14:40.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:40 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:14:40.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:40 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:40.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:40 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:14:40.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:40 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:40.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:40 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:14:40.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:40 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:40.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:40 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:14:40.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:40 smithi132 ceph-mon[78180]: scrub queued for path: / 2024-05-08T22:14:40.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:40 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:40.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:40 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:14:40.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:40 smithi132 ceph-mon[78180]: pgmap v226: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.8 MiB/s wr, 1.16k op/s 2024-05-08T22:14:40.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:40 smithi132 ceph-mon[78180]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T22:14:40.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:40 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:14:40.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:40 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:14:40.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:40 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:40.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:40 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:14:40.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:40 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:40.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:40 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:14:40.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:40 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:40.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:40 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:14:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:40 smithi131 ceph-mon[37260]: scrub queued for path: / 2024-05-08T22:14:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:40 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:40 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:14:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:40 smithi131 ceph-mon[37260]: pgmap v226: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.8 MiB/s wr, 1.16k op/s 2024-05-08T22:14:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:40 smithi131 ceph-mon[37260]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T22:14:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:40 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:14:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:40 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:14:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:40 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:40 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:14:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:40 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:40 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:14:40.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:40 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:40.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:40 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:14:41.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:41 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:41.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:41 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:14:41.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:41 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:41.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:41 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:14:41.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:41 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:41.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:41 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:41.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:41 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:41.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:41 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:14:41.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:41 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:41.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:41 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:14:41.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:41 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:41.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:41 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:41.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:41 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:41.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:41 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:14:41.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:41 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:41.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:41 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:14:41.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:41 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:41.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:41 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:14:42.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:42 smithi157 ceph-mon[77746]: pgmap v227: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 733 op/s 2024-05-08T22:14:42.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:42 smithi132 ceph-mon[78180]: pgmap v227: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 733 op/s 2024-05-08T22:14:42.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:42 smithi131 ceph-mon[37260]: pgmap v227: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 733 op/s 2024-05-08T22:14:44.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:44 smithi157 ceph-mon[77746]: pgmap v228: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 759 op/s 2024-05-08T22:14:44.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:44 smithi132 ceph-mon[78180]: pgmap v228: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 759 op/s 2024-05-08T22:14:44.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:44 smithi131 ceph-mon[37260]: pgmap v228: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 759 op/s 2024-05-08T22:14:46.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:46 smithi157 ceph-mon[77746]: pgmap v229: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 971 op/s 2024-05-08T22:14:46.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:46 smithi132 ceph-mon[78180]: pgmap v229: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 971 op/s 2024-05-08T22:14:46.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:46 smithi131 ceph-mon[37260]: pgmap v229: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 971 op/s 2024-05-08T22:14:48.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:48 smithi157 ceph-mon[77746]: pgmap v230: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 768 op/s 2024-05-08T22:14:48.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:48 smithi132 ceph-mon[78180]: pgmap v230: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 768 op/s 2024-05-08T22:14:48.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:48 smithi131 ceph-mon[37260]: pgmap v230: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 768 op/s 2024-05-08T22:14:50.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:50 smithi157 ceph-mon[77746]: pgmap v231: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 1.11k op/s 2024-05-08T22:14:50.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:50 smithi132 ceph-mon[78180]: pgmap v231: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 1.11k op/s 2024-05-08T22:14:50.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:50 smithi131 ceph-mon[37260]: pgmap v231: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 1.11k op/s 2024-05-08T22:14:52.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:52 smithi157 ceph-mon[77746]: pgmap v232: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 953 op/s 2024-05-08T22:14:52.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:52 smithi132 ceph-mon[78180]: pgmap v232: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 953 op/s 2024-05-08T22:14:52.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:52 smithi131 ceph-mon[37260]: pgmap v232: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 953 op/s 2024-05-08T22:14:54.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:54 smithi157 ceph-mon[77746]: pgmap v233: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 1.02k op/s 2024-05-08T22:14:54.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:54 smithi132 ceph-mon[78180]: pgmap v233: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 1.02k op/s 2024-05-08T22:14:54.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:54 smithi131 ceph-mon[37260]: pgmap v233: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 1.02k op/s 2024-05-08T22:14:56.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:56 smithi132 ceph-mon[78180]: pgmap v234: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 1.31k op/s 2024-05-08T22:14:56.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:56 smithi131 ceph-mon[37260]: pgmap v234: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 1.31k op/s 2024-05-08T22:14:57.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:56 smithi157 ceph-mon[77746]: pgmap v234: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 1.31k op/s 2024-05-08T22:14:58.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:14:58 smithi132 ceph-mon[78180]: pgmap v235: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 1.15k op/s 2024-05-08T22:14:58.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:14:58 smithi131 ceph-mon[37260]: pgmap v235: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 1.15k op/s 2024-05-08T22:14:59.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:14:58 smithi157 ceph-mon[77746]: pgmap v235: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 1.15k op/s 2024-05-08T22:15:00.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:00 smithi132 ceph-mon[78180]: pgmap v236: 129 pgs: 129 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 1.43k op/s 2024-05-08T22:15:00.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:00 smithi131 ceph-mon[37260]: pgmap v236: 129 pgs: 129 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 1.43k op/s 2024-05-08T22:15:01.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:00 smithi157 ceph-mon[77746]: pgmap v236: 129 pgs: 129 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 1.43k op/s 2024-05-08T22:15:02.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:02 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:02.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:02 smithi132 ceph-mon[78180]: pgmap v237: 129 pgs: 129 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 1.04k op/s 2024-05-08T22:15:02.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:02 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:02.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:02 smithi131 ceph-mon[37260]: pgmap v237: 129 pgs: 129 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 1.04k op/s 2024-05-08T22:15:03.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:02 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:03.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:02 smithi157 ceph-mon[77746]: pgmap v237: 129 pgs: 129 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 1.04k op/s 2024-05-08T22:15:03.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:03 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:15:03.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:03 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:15:04.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:03 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:15:04.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:04 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:15:04.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:04 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:15:04.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:04 smithi132 ceph-mon[78180]: pgmap v238: 129 pgs: 129 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 1.13k op/s 2024-05-08T22:15:04.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:04 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:15:04.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:04 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:15:04.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:04 smithi131 ceph-mon[37260]: pgmap v238: 129 pgs: 129 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 1.13k op/s 2024-05-08T22:15:05.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:04 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:15:05.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:04 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:15:05.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:04 smithi157 ceph-mon[77746]: pgmap v238: 129 pgs: 129 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 1.13k op/s 2024-05-08T22:15:06.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:06 smithi132 ceph-mon[78180]: pgmap v239: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.9 MiB/s wr, 1.52k op/s 2024-05-08T22:15:06.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:06 smithi131 ceph-mon[37260]: pgmap v239: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.9 MiB/s wr, 1.52k op/s 2024-05-08T22:15:07.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:06 smithi157 ceph-mon[77746]: pgmap v239: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.9 MiB/s wr, 1.52k op/s 2024-05-08T22:15:08.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:08 smithi132 ceph-mon[78180]: pgmap v240: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 1.19k op/s 2024-05-08T22:15:08.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:08 smithi131 ceph-mon[37260]: pgmap v240: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 1.19k op/s 2024-05-08T22:15:09.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:08 smithi157 ceph-mon[77746]: pgmap v240: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 1.19k op/s 2024-05-08T22:15:09.293 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:15:09.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.608+0000 7f8011a00700 1 -- 172.21.15.131:0/3153584756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f800c100680 msgr2=0x7f800c100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:09.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.608+0000 7f8011a00700 1 --2- 172.21.15.131:0/3153584756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f800c100680 0x7f800c100a60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009990 tx=0x7f7ffc009ca0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.608+0000 7f8011a00700 1 -- 172.21.15.131:0/3153584756 shutdown_connections 2024-05-08T22:15:09.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.608+0000 7f8011a00700 1 --2- 172.21.15.131:0/3153584756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f800c1050f0 0x7f800c10cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.608+0000 7f8011a00700 1 --2- 172.21.15.131:0/3153584756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f800c100680 0x7f800c100a60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.608+0000 7f8011a00700 1 --2- 172.21.15.131:0/3153584756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f800c101030 0x7f800c1049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.608+0000 7f8011a00700 1 -- 172.21.15.131:0/3153584756 >> 172.21.15.131:0/3153584756 conn(0x7f800c076290 msgr2=0x7f800c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:09.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.608+0000 7f8011a00700 1 -- 172.21.15.131:0/3153584756 shutdown_connections 2024-05-08T22:15:09.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.610+0000 7f8011a00700 1 -- 172.21.15.131:0/3153584756 wait complete. 2024-05-08T22:15:09.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.611+0000 7f8011a00700 1 Processor -- start 2024-05-08T22:15:09.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.612+0000 7f8011a00700 1 -- start start 2024-05-08T22:15:09.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.612+0000 7f8011a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f800c100680 0x7f800c1a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:09.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.612+0000 7f8011a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f800c1050f0 0x7f800c1a3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:09.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.612+0000 7f8011a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f800c19d790 0x7f800c19dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:09.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.612+0000 7f8011a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c19e210 con 0x7f800c1050f0 2024-05-08T22:15:09.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.612+0000 7f8011a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c19e350 con 0x7f800c19d790 2024-05-08T22:15:09.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.612+0000 7f8011a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c19e510 con 0x7f800c100680 2024-05-08T22:15:09.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.612+0000 7f8011000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f800c19d790 0x7f800c19dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:09.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.613+0000 7f8011000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f800c19d790 0x7f800c19dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:41404/0 (socket says 172.21.15.131:41404) 2024-05-08T22:15:09.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.613+0000 7f8011000700 1 -- 172.21.15.131:0/1249583516 learned_addr learned my addr 172.21.15.131:0/1249583516 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:09.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.613+0000 7f800b400700 1 --2- 172.21.15.131:0/1249583516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f800c1050f0 0x7f800c1a3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:09.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.613+0000 7f800be00700 1 --2- 172.21.15.131:0/1249583516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f800c100680 0x7f800c1a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:09.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.613+0000 7f8011000700 1 -- 172.21.15.131:0/1249583516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f800c100680 msgr2=0x7f800c1a3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:09.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.613+0000 7f8011000700 1 --2- 172.21.15.131:0/1249583516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f800c100680 0x7f800c1a3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.613+0000 7f8011000700 1 -- 172.21.15.131:0/1249583516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f800c1050f0 msgr2=0x7f800c1a3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:09.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.613+0000 7f8011000700 1 --2- 172.21.15.131:0/1249583516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f800c1050f0 0x7f800c1a3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.613+0000 7f8011000700 1 -- 172.21.15.131:0/1249583516 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ffc0095f0 con 0x7f800c19d790 2024-05-08T22:15:09.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.613+0000 7f800b400700 1 --2- 172.21.15.131:0/1249583516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f800c1050f0 0x7f800c1a3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:15:09.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.616+0000 7f8011000700 1 --2- 172.21.15.131:0/1249583516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f800c19d790 0x7f800c19dc40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f800400cca0 tx=0x7f800400a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:09.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.617+0000 7f8008c00700 1 -- 172.21.15.131:0/1249583516 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800400aaf0 con 0x7f800c19d790 2024-05-08T22:15:09.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.617+0000 7f8008c00700 1 -- 172.21.15.131:0/1249583516 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f800400ac50 con 0x7f800c19d790 2024-05-08T22:15:09.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.618+0000 7f8008c00700 1 -- 172.21.15.131:0/1249583516 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80040115e0 con 0x7f800c19d790 2024-05-08T22:15:09.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.618+0000 7f8011a00700 1 -- 172.21.15.131:0/1249583516 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800c1aa4d0 con 0x7f800c19d790 2024-05-08T22:15:09.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.618+0000 7f8011a00700 1 -- 172.21.15.131:0/1249583516 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800c1aa9e0 con 0x7f800c19d790 2024-05-08T22:15:09.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.627+0000 7f8008c00700 1 -- 172.21.15.131:0/1249583516 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f8004012420 con 0x7f800c19d790 2024-05-08T22:15:09.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.629+0000 7f8026afbb80 1 Processor -- start 2024-05-08T22:15:09.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.630+0000 7f8008c00700 1 --2- 172.21.15.131:0/1249583516 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7ff00a1380 0x7f7ff00a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:09.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.630+0000 7f8008c00700 1 -- 172.21.15.131:0/1249583516 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f80040cd0b0 con 0x7f800c19d790 2024-05-08T22:15:09.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.630+0000 7f800b400700 1 --2- 172.21.15.131:0/1249583516 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7ff00a1380 0x7f7ff00a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:09.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.631+0000 7f8026afbb80 1 -- start start 2024-05-08T22:15:09.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.631+0000 7f8026afbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560018d04050 0x560018d04430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:09.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.632+0000 7f8026afbb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018d04970 0x560018cfe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:09.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.632+0000 7f8026afbb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560018cfe890 0x560018cfecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:09.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.632+0000 7f800be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560018d04050 0x560018d04430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:09.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.632+0000 7f800be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560018d04050 0x560018d04430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55850/0 (socket says 172.21.15.131:55850) 2024-05-08T22:15:09.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.632+0000 7f8026afbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560018c55150 con 0x560018d04050 2024-05-08T22:15:09.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.632+0000 7f8026afbb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560018c52b10 con 0x560018cfe890 2024-05-08T22:15:09.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.632+0000 7f8026afbb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560018c4e400 con 0x560018d04970 2024-05-08T22:15:09.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.632+0000 7f800be00700 1 -- 172.21.15.131:0/3016350456 learned_addr learned my addr 172.21.15.131:0/3016350456 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:09.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.633+0000 7f800b400700 1 --2- 172.21.15.131:0/3016350456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018d04970 0x560018cfe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:09.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.633+0000 7f8011000700 1 --2- 172.21.15.131:0/3016350456 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560018cfe890 0x560018cfecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:09.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.634+0000 7f800b400700 1 -- 172.21.15.131:0/3016350456 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560018cfe890 msgr2=0x560018cfecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:09.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.634+0000 7f800b400700 1 --2- 172.21.15.131:0/3016350456 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560018cfe890 0x560018cfecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.634+0000 7f800b400700 1 -- 172.21.15.131:0/3016350456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560018d04050 msgr2=0x560018d04430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:09.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.634+0000 7f800b400700 1 --2- 172.21.15.131:0/3016350456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560018d04050 0x560018d04430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.634+0000 7f800b400700 1 -- 172.21.15.131:0/3016350456 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560018acf3b0 con 0x560018d04970 2024-05-08T22:15:09.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.635+0000 7f800b400700 1 --2- 172.21.15.131:0/1249583516 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7ff00a1380 0x7f7ff00a3840 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f800c19ebb0 tx=0x7f7ff800b410 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:09.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.637+0000 7f800b400700 1 --2- 172.21.15.131:0/3016350456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018d04970 0x560018cfe350 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8005950 tx=0x7f7ff80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ee9a99221516d9f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:09.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.637+0000 7f8001600700 1 -- 172.21.15.131:0/3016350456 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8004020 con 0x560018d04970 2024-05-08T22:15:09.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.637+0000 7f8001600700 1 -- 172.21.15.131:0/3016350456 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ff8003680 con 0x560018d04970 2024-05-08T22:15:09.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.637+0000 7f8001600700 1 -- 172.21.15.131:0/3016350456 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff801d680 con 0x560018d04970 2024-05-08T22:15:09.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.638+0000 7f8026afbb80 1 -- 172.21.15.131:0/3016350456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018d04970 msgr2=0x560018cfe350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:09.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.638+0000 7f8026afbb80 1 --2- 172.21.15.131:0/3016350456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018d04970 0x560018cfe350 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8005950 tx=0x7f7ff80047c0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.638+0000 7f8026afbb80 1 -- 172.21.15.131:0/3016350456 shutdown_connections 2024-05-08T22:15:09.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.638+0000 7f8026afbb80 1 --2- 172.21.15.131:0/3016350456 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560018cfe890 0x560018cfecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.638+0000 7f8026afbb80 1 --2- 172.21.15.131:0/3016350456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560018d04050 0x560018d04430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.638+0000 7f8026afbb80 1 --2- 172.21.15.131:0/3016350456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018d04970 0x560018cfe350 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.638+0000 7f8026afbb80 1 -- 172.21.15.131:0/3016350456 >> 172.21.15.131:0/3016350456 conn(0x560018c5c210 msgr2=0x560018cf92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:09.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.640+0000 7f8026afbb80 1 -- 172.21.15.131:0/3016350456 shutdown_connections 2024-05-08T22:15:09.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.640+0000 7f8026afbb80 1 -- 172.21.15.131:0/3016350456 wait complete. 2024-05-08T22:15:09.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.640+0000 7f8026afbb80 1 -- start start 2024-05-08T22:15:09.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.641+0000 7f8026afbb80 1 Processor -- start 2024-05-08T22:15:09.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.642+0000 7f8026afbb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560018cfe890 0x560018d0a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:09.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.642+0000 7f8026afbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560018d04050 0x560018d07c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:09.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.642+0000 7f8026afbb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018d04970 0x560018d08360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:09.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.642+0000 7f8026afbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560018bea560 con 0x560018d04050 2024-05-08T22:15:09.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.642+0000 7f8026afbb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560018bea200 con 0x560018cfe890 2024-05-08T22:15:09.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.642+0000 7f8026afbb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560018ad16a0 con 0x560018d04970 2024-05-08T22:15:09.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.643+0000 7f8026afbb80 10 client.0 _ll_register_callbacks cb 0x560018bf3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:15:09.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.643+0000 7f8011000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018d04970 0x560018d08360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:09.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.643+0000 7f8011000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018d04970 0x560018d08360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:58860/0 (socket says 172.21.15.131:58860) 2024-05-08T22:15:09.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.643+0000 7f8011000700 1 -- 172.21.15.131:0/1867834347 learned_addr learned my addr 172.21.15.131:0/1867834347 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:09.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.643+0000 7f800be00700 1 --2- 172.21.15.131:0/1867834347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560018cfe890 0x560018d0a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:09.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.643+0000 7f800b400700 1 --2- 172.21.15.131:0/1867834347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560018d04050 0x560018d07c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:09.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.644+0000 7f8011000700 1 -- 172.21.15.131:0/1867834347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560018cfe890 msgr2=0x560018d0a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:09.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.644+0000 7f8011000700 1 --2- 172.21.15.131:0/1867834347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560018cfe890 0x560018d0a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.644+0000 7f8011000700 1 -- 172.21.15.131:0/1867834347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560018d04050 msgr2=0x560018d07c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:09.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.644+0000 7f8011000700 1 --2- 172.21.15.131:0/1867834347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560018d04050 0x560018d07c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.644+0000 7f8011000700 1 -- 172.21.15.131:0/1867834347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff8010dc0 con 0x560018d04970 2024-05-08T22:15:09.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.644+0000 7f800be00700 1 --2- 172.21.15.131:0/1867834347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560018cfe890 0x560018d0a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:15:09.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.644+0000 7f800b400700 1 --2- 172.21.15.131:0/1867834347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560018d04050 0x560018d07c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:15:09.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.644+0000 7f8011000700 1 --2- 172.21.15.131:0/1867834347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018d04970 0x560018d08360 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8004099e10 tx=0x7f800409a580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e595e37c10bff37a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:09.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.644+0000 7f7ff6000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:15:09.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.645+0000 7f7ff6000700 1 -- 172.21.15.131:0/1867834347 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800409dcb0 con 0x560018d04970 2024-05-08T22:15:09.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.645+0000 7f8026afbb80 1 -- 172.21.15.131:0/1867834347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560018ba7270 con 0x560018d04970 2024-05-08T22:15:09.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.645+0000 7f7ff6000700 1 -- 172.21.15.131:0/1867834347 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f80040a2300 con 0x560018d04970 2024-05-08T22:15:09.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.645+0000 7f7ff6000700 1 -- 172.21.15.131:0/1867834347 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80040b8170 con 0x560018d04970 2024-05-08T22:15:09.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.645+0000 7f8026afbb80 10 client.26431 fetch_fsmap learned FSMap version 28 2024-05-08T22:15:09.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.645+0000 7f8026afbb80 1 -- 172.21.15.131:0/1867834347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560018a7bb60 con 0x560018d04970 2024-05-08T22:15:09.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.645+0000 7f7ff6000700 1 -- 172.21.15.131:0/1867834347 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f800407c080 con 0x560018d04970 2024-05-08T22:15:09.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.646+0000 7f7ff6000700 1 -- 172.21.15.131:0/1867834347 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8004099370 con 0x560018d04970 2024-05-08T22:15:09.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.646+0000 7f8026afbb80 10 client.26431 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T22:15:09.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.646+0000 7f8026afbb80 10 client.26431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:15:09.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.646+0000 7f8026afbb80 20 client.26431 populate_metadata read hostname 'smithi131' 2024-05-08T22:15:09.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.646+0000 7f8026afbb80 1 --2- 172.21.15.131:0/1867834347 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560018d18370 0x560018d1a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:09.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.646+0000 7f8026afbb80 4 client.26431 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:15:09.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.646+0000 7f8026afbb80 1 -- 172.21.15.131:0/1867834347 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560018ccd810 con 0x560018d18370 2024-05-08T22:15:09.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.646+0000 7f800be00700 1 --2- 172.21.15.131:0/1867834347 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560018d18370 0x560018d1a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:09.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.647+0000 7f7ff6000700 10 client.26431 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:09.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.647+0000 7f800be00700 1 --2- 172.21.15.131:0/1867834347 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560018d18370 0x560018d1a790 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=427e70f8ec5d1c22 server_cookie=c1b0a400e0ff00cf in_seq=0 out_seq=0 2024-05-08T22:15:09.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.649+0000 7f7ff6000700 1 -- 172.21.15.131:0/1867834347 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560018ccd810 con 0x560018d18370 2024-05-08T22:15:09.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.649+0000 7f7ff6000700 10 client.26431 handle_command_reply: tid=0 2024-05-08T22:15:09.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.649+0000 7f8026afbb80 1 client.26431 shutdown 2024-05-08T22:15:09.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.650+0000 7f8026afbb80 1 -- 172.21.15.131:0/1867834347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018d04970 msgr2=0x560018d08360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:09.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.650+0000 7f8026afbb80 1 --2- 172.21.15.131:0/1867834347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018d04970 0x560018d08360 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8004099e10 tx=0x7f800409a580 comp rx=0 tx=0).stop 2024-05-08T22:15:09.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.650+0000 7f8026afbb80 1 -- 172.21.15.131:0/1867834347 shutdown_connections 2024-05-08T22:15:09.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.651+0000 7f8026afbb80 1 --2- 172.21.15.131:0/1867834347 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560018d18370 0x560018d1a790 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.651+0000 7f8026afbb80 1 --2- 172.21.15.131:0/1867834347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018d04970 0x560018d08360 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.651+0000 7f8026afbb80 1 --2- 172.21.15.131:0/1867834347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560018cfe890 0x560018d0a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.651+0000 7f8026afbb80 1 --2- 172.21.15.131:0/1867834347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560018d04050 0x560018d07c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.651+0000 7f8026afbb80 1 -- 172.21.15.131:0/1867834347 >> 172.21.15.131:0/1867834347 conn(0x560018c5c210 msgr2=0x560018cf90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:09.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.651+0000 7f7ff6000700 0 client.26431 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:09.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.651+0000 7f8026afbb80 1 -- 172.21.15.131:0/1867834347 shutdown_connections 2024-05-08T22:15:09.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.651+0000 7f8026afbb80 1 -- 172.21.15.131:0/1867834347 wait complete. 2024-05-08T22:15:09.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.651+0000 7f8026afbb80 20 client.26431 trim_cache size 0 max 16384 2024-05-08T22:15:09.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.688+0000 7f8026afbb80 1 Processor -- start 2024-05-08T22:15:09.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.688+0000 7f8026afbb80 1 -- start start 2024-05-08T22:15:09.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.688+0000 7f8026afbb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018cfe890 0x560018cf7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:09.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.688+0000 7f8026afbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560018d04050 0x560018d0a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:09.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.688+0000 7f8026afbb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560018d04970 0x560018d1f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:09.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.688+0000 7f8026afbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560018cf5e30 con 0x560018d04050 2024-05-08T22:15:09.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.688+0000 7f8026afbb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560018d14d40 con 0x560018d04970 2024-05-08T22:15:09.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.688+0000 7f8026afbb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560018cf1040 con 0x560018cfe890 2024-05-08T22:15:09.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.689+0000 7f800be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018cfe890 0x560018cf7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:09.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.689+0000 7f800be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018cfe890 0x560018cf7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:58862/0 (socket says 172.21.15.131:58862) 2024-05-08T22:15:09.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.689+0000 7f800be00700 1 -- 172.21.15.131:0/2133331701 learned_addr learned my addr 172.21.15.131:0/2133331701 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:09.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.689+0000 7f8011000700 1 --2- 172.21.15.131:0/2133331701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560018d04970 0x560018d1f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:09.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.689+0000 7f800b400700 1 --2- 172.21.15.131:0/2133331701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560018d04050 0x560018d0a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:09.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.689+0000 7f800be00700 1 -- 172.21.15.131:0/2133331701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560018d04970 msgr2=0x560018d1f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:09.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.689+0000 7f800be00700 1 --2- 172.21.15.131:0/2133331701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560018d04970 0x560018d1f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.689+0000 7f800be00700 1 -- 172.21.15.131:0/2133331701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560018d04050 msgr2=0x560018d0a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:09.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.689+0000 7f800be00700 1 --2- 172.21.15.131:0/2133331701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560018d04050 0x560018d0a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.689+0000 7f800be00700 1 -- 172.21.15.131:0/2133331701 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560018c14560 con 0x560018cfe890 2024-05-08T22:15:09.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.689+0000 7f8011000700 1 --2- 172.21.15.131:0/2133331701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560018d04970 0x560018d1f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:15:09.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.689+0000 7f800be00700 1 --2- 172.21.15.131:0/2133331701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018cfe890 0x560018cf7d10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00b5c0 tx=0x7f7ffc009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fa5565820fb55bb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:09.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.689+0000 7f8002000700 1 -- 172.21.15.131:0/2133331701 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc01d070 con 0x560018cfe890 2024-05-08T22:15:09.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.690+0000 7f8002000700 1 -- 172.21.15.131:0/2133331701 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ffc01e040 con 0x560018cfe890 2024-05-08T22:15:09.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.690+0000 7f8002000700 1 -- 172.21.15.131:0/2133331701 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc004cb0 con 0x560018cfe890 2024-05-08T22:15:09.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.691+0000 7f8026afbb80 1 -- 172.21.15.131:0/2133331701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018cfe890 msgr2=0x560018cf7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:09.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.691+0000 7f8026afbb80 1 --2- 172.21.15.131:0/2133331701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018cfe890 0x560018cf7d10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00b5c0 tx=0x7f7ffc009310 comp rx=0 tx=0).stop 2024-05-08T22:15:09.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.691+0000 7f8026afbb80 1 -- 172.21.15.131:0/2133331701 shutdown_connections 2024-05-08T22:15:09.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.691+0000 7f8026afbb80 1 --2- 172.21.15.131:0/2133331701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560018d04970 0x560018d1f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.691+0000 7f8026afbb80 1 --2- 172.21.15.131:0/2133331701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018cfe890 0x560018cf7d10 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.691+0000 7f8026afbb80 1 --2- 172.21.15.131:0/2133331701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560018d04050 0x560018d0a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.691+0000 7f8026afbb80 1 -- 172.21.15.131:0/2133331701 >> 172.21.15.131:0/2133331701 conn(0x560018c5c210 msgr2=0x560018cfa820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:09.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.691+0000 7f8026afbb80 1 -- 172.21.15.131:0/2133331701 shutdown_connections 2024-05-08T22:15:09.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.691+0000 7f8026afbb80 1 -- 172.21.15.131:0/2133331701 wait complete. 2024-05-08T22:15:09.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.692+0000 7f8026afbb80 1 -- start start 2024-05-08T22:15:09.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.692+0000 7f8026afbb80 1 Processor -- start 2024-05-08T22:15:09.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.693+0000 7f8026afbb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560018cfe890 0x560018d0f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:09.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.693+0000 7f8026afbb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018d04050 0x560018d0f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:09.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.693+0000 7f8026afbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560018d04970 0x560018cffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:09.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.694+0000 7f800be00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560018cfe890 0x560018d0f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:09.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.694+0000 7f800be00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560018cfe890 0x560018d0f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:41436/0 (socket says 172.21.15.131:41436) 2024-05-08T22:15:09.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.694+0000 7f8026afbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560018d00370 con 0x560018d04970 2024-05-08T22:15:09.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.694+0000 7f8026afbb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560018d004e0 con 0x560018cfe890 2024-05-08T22:15:09.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.694+0000 7f8026afbb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560018d006a0 con 0x560018d04050 2024-05-08T22:15:09.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.694+0000 7f800be00700 1 -- 172.21.15.131:0/3498187252 learned_addr learned my addr 172.21.15.131:0/3498187252 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.694+0000 7f8026afbb80 10 client.0 _ll_register_callbacks cb 0x560018bf3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:15:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.694+0000 7f8011000700 1 --2- 172.21.15.131:0/3498187252 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560018d04970 0x560018cffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.694+0000 7f800b400700 1 --2- 172.21.15.131:0/3498187252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018d04050 0x560018d0f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.694+0000 7f800be00700 1 -- 172.21.15.131:0/3498187252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018d04050 msgr2=0x560018d0f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.694+0000 7f800be00700 1 --2- 172.21.15.131:0/3498187252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018d04050 0x560018d0f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.694+0000 7f800be00700 1 -- 172.21.15.131:0/3498187252 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560018d04970 msgr2=0x560018cffcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.694+0000 7f800be00700 1 --2- 172.21.15.131:0/3498187252 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560018d04970 0x560018cffcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.694+0000 7f800be00700 1 -- 172.21.15.131:0/3498187252 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800c10d4e0 con 0x560018cfe890 2024-05-08T22:15:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.695+0000 7f800b400700 1 --2- 172.21.15.131:0/3498187252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018d04050 0x560018d0f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:15:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.695+0000 7f8011000700 1 --2- 172.21.15.131:0/3498187252 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560018d04970 0x560018cffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:15:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.695+0000 7f800be00700 1 --2- 172.21.15.131:0/3498187252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560018cfe890 0x560018d0f360 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0048d0 tx=0x7f7ffc004930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc2e25d882b7884c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.695+0000 7f7ff6000700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:15:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.695+0000 7f7ff6000700 1 -- 172.21.15.131:0/3498187252 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc01d050 con 0x560018cfe890 2024-05-08T22:15:09.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.695+0000 7f7ff6000700 1 -- 172.21.15.131:0/3498187252 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f7ffc00b8f0 con 0x560018cfe890 2024-05-08T22:15:09.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.695+0000 7f8026afbb80 1 -- 172.21.15.131:0/3498187252 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ff8003680 con 0x560018cfe890 2024-05-08T22:15:09.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.696+0000 7f7ff6000700 1 -- 172.21.15.131:0/3498187252 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ffc004ee0 con 0x560018cfe890 2024-05-08T22:15:09.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.696+0000 7f8026afbb80 10 client.26372 fetch_fsmap learned FSMap version 28 2024-05-08T22:15:09.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.696+0000 7f8026afbb80 1 -- 172.21.15.131:0/3498187252 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560018ad3cd0 con 0x560018cfe890 2024-05-08T22:15:09.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.696+0000 7f7ff6000700 1 -- 172.21.15.131:0/3498187252 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ff8003680 con 0x560018cfe890 2024-05-08T22:15:09.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.696+0000 7f7ff6000700 1 -- 172.21.15.131:0/3498187252 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7ffc03e070 con 0x560018cfe890 2024-05-08T22:15:09.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.699+0000 7f8026afbb80 10 client.26372 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T22:15:09.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.699+0000 7f8026afbb80 10 client.26372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:15:09.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.699+0000 7f8026afbb80 20 client.26372 populate_metadata read hostname 'smithi131' 2024-05-08T22:15:09.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.699+0000 7f8026afbb80 1 --2- 172.21.15.131:0/3498187252 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560018d80660 0x560018d82a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:09.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.699+0000 7f8026afbb80 4 client.26372 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:15:09.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.699+0000 7f8026afbb80 1 -- 172.21.15.131:0/3498187252 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560018c46ec0 con 0x560018d80660 2024-05-08T22:15:09.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.699+0000 7f800b400700 1 --2- 172.21.15.131:0/3498187252 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560018d80660 0x560018d82a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:09.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.700+0000 7f800b400700 1 --2- 172.21.15.131:0/3498187252 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560018d80660 0x560018d82a80 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d87122fe75b9ae1 server_cookie=ff37a5bf3feabfc8 in_seq=0 out_seq=0 2024-05-08T22:15:09.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.700+0000 7f7ff6000700 10 client.26372 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:09.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.700+0000 7f7ff6000700 1 -- 172.21.15.131:0/3498187252 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560018c46ec0 con 0x560018d80660 2024-05-08T22:15:09.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.700+0000 7f7ff6000700 10 client.26372 handle_command_reply: tid=0 2024-05-08T22:15:09.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.700+0000 7f8026afbb80 1 client.26372 shutdown 2024-05-08T22:15:09.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.703+0000 7f8026afbb80 1 -- 172.21.15.131:0/3498187252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560018cfe890 msgr2=0x560018d0f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:09.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.703+0000 7f8026afbb80 1 --2- 172.21.15.131:0/3498187252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560018cfe890 0x560018d0f360 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0048d0 tx=0x7f7ffc004930 comp rx=0 tx=0).stop 2024-05-08T22:15:09.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.704+0000 7f8026afbb80 1 -- 172.21.15.131:0/3498187252 shutdown_connections 2024-05-08T22:15:09.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.704+0000 7f8026afbb80 1 --2- 172.21.15.131:0/3498187252 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560018d04970 0x560018cffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.704+0000 7f8026afbb80 1 --2- 172.21.15.131:0/3498187252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560018cfe890 0x560018d0f360 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.704+0000 7f8026afbb80 1 --2- 172.21.15.131:0/3498187252 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560018d80660 0x560018d82a80 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.704+0000 7f8026afbb80 1 --2- 172.21.15.131:0/3498187252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560018d04050 0x560018d0f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.704+0000 7f8026afbb80 1 -- 172.21.15.131:0/3498187252 >> 172.21.15.131:0/3498187252 conn(0x560018c5c210 msgr2=0x560018cf9060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:09.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.704+0000 7f7ff6000700 0 client.26372 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:09.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.705+0000 7f8026afbb80 1 -- 172.21.15.131:0/3498187252 shutdown_connections 2024-05-08T22:15:09.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.705+0000 7f8026afbb80 1 -- 172.21.15.131:0/3498187252 wait complete. 2024-05-08T22:15:09.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.705+0000 7f8026afbb80 20 client.26372 trim_cache size 0 max 16384 2024-05-08T22:15:09.707 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:15:09.707 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-08T22:15:09.707 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-08T22:15:09.707 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:15:09.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.706+0000 7f8011a00700 1 -- 172.21.15.131:0/1249583516 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7ff00a1380 msgr2=0x7f7ff00a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:09.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.706+0000 7f8011a00700 1 --2- 172.21.15.131:0/1249583516 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7ff00a1380 0x7f7ff00a3840 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f800c19ebb0 tx=0x7f7ff800b410 comp rx=0 tx=0).stop 2024-05-08T22:15:09.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.706+0000 7f8011a00700 1 -- 172.21.15.131:0/1249583516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f800c19d790 msgr2=0x7f800c19dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:09.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.706+0000 7f8011a00700 1 --2- 172.21.15.131:0/1249583516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f800c19d790 0x7f800c19dc40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f800400cca0 tx=0x7f800400a060 comp rx=0 tx=0).stop 2024-05-08T22:15:09.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.706+0000 7f8011a00700 1 -- 172.21.15.131:0/1249583516 shutdown_connections 2024-05-08T22:15:09.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.706+0000 7f8011a00700 1 --2- 172.21.15.131:0/1249583516 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7ff00a1380 0x7f7ff00a3840 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.706+0000 7f8011a00700 1 --2- 172.21.15.131:0/1249583516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f800c19d790 0x7f800c19dc40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.706+0000 7f8011a00700 1 --2- 172.21.15.131:0/1249583516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f800c100680 0x7f800c1a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.706+0000 7f8011a00700 1 --2- 172.21.15.131:0/1249583516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f800c1050f0 0x7f800c1a3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:09.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.706+0000 7f8011a00700 1 -- 172.21.15.131:0/1249583516 >> 172.21.15.131:0/1249583516 conn(0x7f800c076290 msgr2=0x7f800c103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:09.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.708+0000 7f8011a00700 1 -- 172.21.15.131:0/1249583516 shutdown_connections 2024-05-08T22:15:09.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:09.708+0000 7f8011a00700 1 -- 172.21.15.131:0/1249583516 wait complete. 2024-05-08T22:15:09.739 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T22:15:09.740 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:15:10.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.037+0000 7f45aa400700 1 -- 172.21.15.131:0/548179763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45a4101020 msgr2=0x7f45a4101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:10.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.037+0000 7f45aa400700 1 --2- 172.21.15.131:0/548179763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45a4101020 0x7f45a4101400 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4598009990 tx=0x7f4598009ca0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.037+0000 7f45aa400700 1 -- 172.21.15.131:0/548179763 shutdown_connections 2024-05-08T22:15:10.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.037+0000 7f45aa400700 1 --2- 172.21.15.131:0/548179763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a4106140 0x7f45a410dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.037+0000 7f45aa400700 1 --2- 172.21.15.131:0/548179763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45a4101020 0x7f45a4101400 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.037+0000 7f45aa400700 1 --2- 172.21.15.131:0/548179763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45a41019d0 0x7f45a4105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.037+0000 7f45aa400700 1 -- 172.21.15.131:0/548179763 >> 172.21.15.131:0/548179763 conn(0x7f45a40fc8a0 msgr2=0x7f45a40fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:10.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.037+0000 7f45aa400700 1 -- 172.21.15.131:0/548179763 shutdown_connections 2024-05-08T22:15:10.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.037+0000 7f45aa400700 1 -- 172.21.15.131:0/548179763 wait complete. 2024-05-08T22:15:10.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.038+0000 7f45aa400700 1 Processor -- start 2024-05-08T22:15:10.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.039+0000 7f45aa400700 1 -- start start 2024-05-08T22:15:10.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.039+0000 7f45aa400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45a4101020 0x7f45a4199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:10.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.039+0000 7f45aa400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a4106140 0x7f45a4199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:10.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.039+0000 7f45aa400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45a419de00 0x7f45a419e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:10.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.039+0000 7f45aa400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a419e880 con 0x7f45a4101020 2024-05-08T22:15:10.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.039+0000 7f45aa400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a419e9f0 con 0x7f45a419de00 2024-05-08T22:15:10.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.039+0000 7f45aa400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a419ebb0 con 0x7f45a4106140 2024-05-08T22:15:10.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.039+0000 7f45a9a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45a419de00 0x7f45a419e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:10.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.039+0000 7f45a9000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45a4101020 0x7f45a4199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:10.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.039+0000 7f45a9a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45a419de00 0x7f45a419e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:41462/0 (socket says 172.21.15.131:41462) 2024-05-08T22:15:10.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.039+0000 7f45a9a00700 1 -- 172.21.15.131:0/3574240314 learned_addr learned my addr 172.21.15.131:0/3574240314 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:10.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.039+0000 7f45a3e00700 1 --2- 172.21.15.131:0/3574240314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a4106140 0x7f45a4199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:10.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.040+0000 7f45a9000700 1 -- 172.21.15.131:0/3574240314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a4106140 msgr2=0x7f45a4199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:10.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.040+0000 7f45a9000700 1 --2- 172.21.15.131:0/3574240314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a4106140 0x7f45a4199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.040+0000 7f45a9000700 1 -- 172.21.15.131:0/3574240314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45a419de00 msgr2=0x7f45a419e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:10.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.040+0000 7f45a9000700 1 --2- 172.21.15.131:0/3574240314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45a419de00 0x7f45a419e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.040+0000 7f45a9000700 1 -- 172.21.15.131:0/3574240314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45980095f0 con 0x7f45a4101020 2024-05-08T22:15:10.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.040+0000 7f45a9000700 1 --2- 172.21.15.131:0/3574240314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45a4101020 0x7f45a4199520 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f45980098e0 tx=0x7f4598005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:10.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.040+0000 7f45a1600700 1 -- 172.21.15.131:0/3574240314 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459801d070 con 0x7f45a4101020 2024-05-08T22:15:10.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.040+0000 7f45aa400700 1 -- 172.21.15.131:0/3574240314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a41a60b0 con 0x7f45a4101020 2024-05-08T22:15:10.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.040+0000 7f45a1600700 1 -- 172.21.15.131:0/3574240314 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4598022470 con 0x7f45a4101020 2024-05-08T22:15:10.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.040+0000 7f45a1600700 1 -- 172.21.15.131:0/3574240314 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4598017650 con 0x7f45a4101020 2024-05-08T22:15:10.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.040+0000 7f45aa400700 1 -- 172.21.15.131:0/3574240314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a41a6580 con 0x7f45a4101020 2024-05-08T22:15:10.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.042+0000 7f45aa400700 1 -- 172.21.15.131:0/3574240314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a41024f0 con 0x7f45a4101020 2024-05-08T22:15:10.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.044+0000 7f45a1600700 1 -- 172.21.15.131:0/3574240314 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f45980225e0 con 0x7f45a4101020 2024-05-08T22:15:10.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.047+0000 7f45a1600700 1 --2- 172.21.15.131:0/3574240314 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f458c0a1290 0x7f458c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:10.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.047+0000 7f45a1600700 1 -- 172.21.15.131:0/3574240314 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f45980cea50 con 0x7f45a4101020 2024-05-08T22:15:10.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.047+0000 7f45a3e00700 1 --2- 172.21.15.131:0/3574240314 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f458c0a1290 0x7f458c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:10.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.048+0000 7f45a3e00700 1 --2- 172.21.15.131:0/3574240314 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f458c0a1290 0x7f458c0a3750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f45a419aa70 tx=0x7f4590009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:10.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.051+0000 7f45a1600700 1 -- 172.21.15.131:0/3574240314 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4598097b40 con 0x7f45a4101020 2024-05-08T22:15:10.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.324+0000 7f45aa400700 1 -- 172.21.15.131:0/3574240314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45a4067ff0 con 0x7f45a4101020 2024-05-08T22:15:10.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.325+0000 7f45a1600700 1 -- 172.21.15.131:0/3574240314 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10894 (secure 0 0 0) 0x7f45a4067ff0 con 0x7f45a4101020 2024-05-08T22:15:10.327 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:15:10.327 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:05:14.034908+0000","modified":"2024-05-08T22:14:39.460105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:15:10.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.328+0000 7f45aa400700 1 -- 172.21.15.131:0/3574240314 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f458c0a1290 msgr2=0x7f458c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:10.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.328+0000 7f45aa400700 1 --2- 172.21.15.131:0/3574240314 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f458c0a1290 0x7f458c0a3750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f45a419aa70 tx=0x7f4590009260 comp rx=0 tx=0).stop 2024-05-08T22:15:10.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.328+0000 7f45aa400700 1 -- 172.21.15.131:0/3574240314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45a4101020 msgr2=0x7f45a4199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:10.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.328+0000 7f45aa400700 1 --2- 172.21.15.131:0/3574240314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45a4101020 0x7f45a4199520 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f45980098e0 tx=0x7f4598005740 comp rx=0 tx=0).stop 2024-05-08T22:15:10.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.328+0000 7f45aa400700 1 -- 172.21.15.131:0/3574240314 shutdown_connections 2024-05-08T22:15:10.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.328+0000 7f45aa400700 1 --2- 172.21.15.131:0/3574240314 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f458c0a1290 0x7f458c0a3750 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.328+0000 7f45aa400700 1 --2- 172.21.15.131:0/3574240314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45a419de00 0x7f45a419e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.328+0000 7f45aa400700 1 --2- 172.21.15.131:0/3574240314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45a4101020 0x7f45a4199520 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.328+0000 7f45aa400700 1 --2- 172.21.15.131:0/3574240314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a4106140 0x7f45a4199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.328+0000 7f45aa400700 1 -- 172.21.15.131:0/3574240314 >> 172.21.15.131:0/3574240314 conn(0x7f45a40fc8a0 msgr2=0x7f45a40fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:10.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.328+0000 7f45aa400700 1 -- 172.21.15.131:0/3574240314 shutdown_connections 2024-05-08T22:15:10.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.328+0000 7f45aa400700 1 -- 172.21.15.131:0/3574240314 wait complete. 2024-05-08T22:15:10.330 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 28 2024-05-08T22:15:10.349 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T22:15:10.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.617+0000 7feb63800700 1 -- 172.21.15.131:0/2535662583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb5c0ffdd0 msgr2=0x7feb5c10dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:10.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.617+0000 7feb63800700 1 --2- 172.21.15.131:0/2535662583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb5c0ffdd0 0x7feb5c10dda0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7feb5400b0a0 tx=0x7feb5400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.618+0000 7feb63800700 1 -- 172.21.15.131:0/2535662583 shutdown_connections 2024-05-08T22:15:10.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.618+0000 7feb63800700 1 --2- 172.21.15.131:0/2535662583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb5c0ffdd0 0x7feb5c10dda0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.618+0000 7feb63800700 1 --2- 172.21.15.131:0/2535662583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb5c105ef0 0x7feb5c1062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.618+0000 7feb63800700 1 --2- 172.21.15.131:0/2535662583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb5c0ff410 0x7feb5c0ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.618+0000 7feb63800700 1 -- 172.21.15.131:0/2535662583 >> 172.21.15.131:0/2535662583 conn(0x7feb5c0763a0 msgr2=0x7feb5c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:10.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.618+0000 7feb63800700 1 -- 172.21.15.131:0/2535662583 shutdown_connections 2024-05-08T22:15:10.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.618+0000 7feb63800700 1 -- 172.21.15.131:0/2535662583 wait complete. 2024-05-08T22:15:10.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.620+0000 7feb63800700 1 Processor -- start 2024-05-08T22:15:10.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.620+0000 7feb63800700 1 -- start start 2024-05-08T22:15:10.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.621+0000 7feb63800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb5c0ff410 0x7feb5c19f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:10.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.621+0000 7feb63800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb5c0ffdd0 0x7feb5c19f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:10.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.621+0000 7feb63800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb5c105ef0 0x7feb5c199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:10.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.621+0000 7feb63800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c19a480 con 0x7feb5c105ef0 2024-05-08T22:15:10.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.621+0000 7feb63800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c19a5f0 con 0x7feb5c0ffdd0 2024-05-08T22:15:10.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.621+0000 7feb63800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c19a7b0 con 0x7feb5c0ff410 2024-05-08T22:15:10.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.621+0000 7feb60e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb5c0ff410 0x7feb5c19f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:10.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.621+0000 7feb60e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb5c0ff410 0x7feb5c19f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:58908/0 (socket says 172.21.15.131:58908) 2024-05-08T22:15:10.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.621+0000 7feb5be00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb5c0ffdd0 0x7feb5c19f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:10.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.621+0000 7feb60e00700 1 -- 172.21.15.131:0/917256147 learned_addr learned my addr 172.21.15.131:0/917256147 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:10.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.622+0000 7feb60e00700 1 -- 172.21.15.131:0/917256147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb5c0ffdd0 msgr2=0x7feb5c19f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:10.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.622+0000 7feb60e00700 1 --2- 172.21.15.131:0/917256147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb5c0ffdd0 0x7feb5c19f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.622+0000 7feb60e00700 1 -- 172.21.15.131:0/917256147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb5c105ef0 msgr2=0x7feb5c199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:10.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.622+0000 7feb60e00700 1 --2- 172.21.15.131:0/917256147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb5c105ef0 0x7feb5c199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.622+0000 7feb60e00700 1 -- 172.21.15.131:0/917256147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb54009d00 con 0x7feb5c0ff410 2024-05-08T22:15:10.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.622+0000 7feb5be00700 1 --2- 172.21.15.131:0/917256147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb5c0ffdd0 0x7feb5c19f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:15:10.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.622+0000 7feb60e00700 1 --2- 172.21.15.131:0/917256147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb5c0ff410 0x7feb5c19f480 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7feb4800d950 tx=0x7feb4800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:10.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.622+0000 7feb59600700 1 -- 172.21.15.131:0/917256147 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb48009940 con 0x7feb5c0ff410 2024-05-08T22:15:10.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.622+0000 7feb59600700 1 -- 172.21.15.131:0/917256147 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb48010460 con 0x7feb5c0ff410 2024-05-08T22:15:10.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.622+0000 7feb59600700 1 -- 172.21.15.131:0/917256147 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4800f620 con 0x7feb5c0ff410 2024-05-08T22:15:10.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.623+0000 7feb63800700 1 -- 172.21.15.131:0/917256147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb5c19aa30 con 0x7feb5c0ff410 2024-05-08T22:15:10.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.623+0000 7feb63800700 1 -- 172.21.15.131:0/917256147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb5c110d70 con 0x7feb5c0ff410 2024-05-08T22:15:10.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.629+0000 7feb59600700 1 -- 172.21.15.131:0/917256147 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7feb48010890 con 0x7feb5c0ff410 2024-05-08T22:15:10.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.632+0000 7feb59600700 1 --2- 172.21.15.131:0/917256147 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7feb440a1310 0x7feb440a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:10.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.632+0000 7feb59600700 1 -- 172.21.15.131:0/917256147 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7feb480cd2b0 con 0x7feb5c0ff410 2024-05-08T22:15:10.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.633+0000 7feb5be00700 1 --2- 172.21.15.131:0/917256147 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7feb440a1310 0x7feb440a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:10.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.634+0000 7feb5be00700 1 --2- 172.21.15.131:0/917256147 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7feb440a1310 0x7feb440a37d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7feb5c19ad50 tx=0x7feb50009190 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:10.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.635+0000 7feb773d0b80 1 Processor -- start 2024-05-08T22:15:10.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.636+0000 7feb773d0b80 1 -- start start 2024-05-08T22:15:10.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.636+0000 7feb773d0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c30209e00 0x556c3020a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:10.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.636+0000 7feb773d0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c3020a720 0x556c30204350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:10.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.636+0000 7feb773d0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c30204890 0x556c30204cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:10.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.636+0000 7feb773d0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c3015b150 con 0x556c30209e00 2024-05-08T22:15:10.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.636+0000 7feb773d0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c30158b10 con 0x556c30204890 2024-05-08T22:15:10.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.636+0000 7feb773d0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c30154400 con 0x556c3020a720 2024-05-08T22:15:10.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.636+0000 7feb5be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c3020a720 0x556c30204350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:10.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.637+0000 7feb5be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c3020a720 0x556c30204350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:58918/0 (socket says 172.21.15.131:58918) 2024-05-08T22:15:10.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.637+0000 7feb5be00700 1 -- 172.21.15.131:0/4177184511 learned_addr learned my addr 172.21.15.131:0/4177184511 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:10.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.637+0000 7feb61800700 1 --2- 172.21.15.131:0/4177184511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c30204890 0x556c30204cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:10.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.637+0000 7feb60e00700 1 --2- 172.21.15.131:0/4177184511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c30209e00 0x556c3020a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:10.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.637+0000 7feb5be00700 1 -- 172.21.15.131:0/4177184511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c30204890 msgr2=0x556c30204cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:10.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.637+0000 7feb5be00700 1 --2- 172.21.15.131:0/4177184511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c30204890 0x556c30204cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.637+0000 7feb5be00700 1 -- 172.21.15.131:0/4177184511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c30209e00 msgr2=0x556c3020a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:10.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.637+0000 7feb5be00700 1 --2- 172.21.15.131:0/4177184511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c30209e00 0x556c3020a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.637+0000 7feb5be00700 1 -- 172.21.15.131:0/4177184511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c2ffd53b0 con 0x556c3020a720 2024-05-08T22:15:10.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.637+0000 7feb60e00700 1 --2- 172.21.15.131:0/4177184511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c30209e00 0x556c3020a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:15:10.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.638+0000 7feb5be00700 1 --2- 172.21.15.131:0/4177184511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c3020a720 0x556c30204350 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7feb5000edd0 tx=0x7feb50011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ca4db79a3f94b7d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:10.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.638+0000 7feb4e000700 1 -- 172.21.15.131:0/4177184511 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5000ee00 con 0x556c3020a720 2024-05-08T22:15:10.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.638+0000 7feb4e000700 1 -- 172.21.15.131:0/4177184511 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb50015dd0 con 0x556c3020a720 2024-05-08T22:15:10.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.638+0000 7feb4e000700 1 -- 172.21.15.131:0/4177184511 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb50013e90 con 0x556c3020a720 2024-05-08T22:15:10.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.639+0000 7feb773d0b80 1 -- 172.21.15.131:0/4177184511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c3020a720 msgr2=0x556c30204350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:10.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.639+0000 7feb773d0b80 1 --2- 172.21.15.131:0/4177184511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c3020a720 0x556c30204350 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7feb5000edd0 tx=0x7feb50011000 comp rx=0 tx=0).stop 2024-05-08T22:15:10.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.639+0000 7feb773d0b80 1 -- 172.21.15.131:0/4177184511 shutdown_connections 2024-05-08T22:15:10.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.639+0000 7feb773d0b80 1 --2- 172.21.15.131:0/4177184511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c30204890 0x556c30204cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.639+0000 7feb773d0b80 1 --2- 172.21.15.131:0/4177184511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c30209e00 0x556c3020a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.639+0000 7feb773d0b80 1 --2- 172.21.15.131:0/4177184511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c3020a720 0x556c30204350 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.639+0000 7feb773d0b80 1 -- 172.21.15.131:0/4177184511 >> 172.21.15.131:0/4177184511 conn(0x556c30162210 msgr2=0x556c301fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:10.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.639+0000 7feb773d0b80 1 -- 172.21.15.131:0/4177184511 shutdown_connections 2024-05-08T22:15:10.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.640+0000 7feb773d0b80 1 -- 172.21.15.131:0/4177184511 wait complete. 2024-05-08T22:15:10.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.641+0000 7feb773d0b80 1 -- start start 2024-05-08T22:15:10.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.641+0000 7feb773d0b80 1 Processor -- start 2024-05-08T22:15:10.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:10 smithi131 ceph-mon[37260]: pgmap v241: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 1.06k op/s 2024-05-08T22:15:10.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:10 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3574240314' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:15:10.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.643+0000 7feb773d0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c30204890 0x556c3020da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:10.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.643+0000 7feb773d0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c30209e00 0x556c3020dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:10.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.643+0000 7feb773d0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3020a720 0x556c3020e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:10.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.643+0000 7feb773d0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c300f0560 con 0x556c30204890 2024-05-08T22:15:10.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.643+0000 7feb773d0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c300f0200 con 0x556c3020a720 2024-05-08T22:15:10.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.643+0000 7feb773d0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c2ffd76a0 con 0x556c30209e00 2024-05-08T22:15:10.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.643+0000 7feb773d0b80 10 client.0 _ll_register_callbacks cb 0x556c300f9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:15:10.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.643+0000 7feb5be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c30209e00 0x556c3020dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:10.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.643+0000 7feb5be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c30209e00 0x556c3020dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:58932/0 (socket says 172.21.15.131:58932) 2024-05-08T22:15:10.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.643+0000 7feb5be00700 1 -- 172.21.15.131:0/2223593379 learned_addr learned my addr 172.21.15.131:0/2223593379 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:10.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.644+0000 7feb60e00700 1 --2- 172.21.15.131:0/2223593379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c30204890 0x556c3020da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:10.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.644+0000 7feb61800700 1 --2- 172.21.15.131:0/2223593379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3020a720 0x556c3020e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:10.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.644+0000 7feb61800700 1 -- 172.21.15.131:0/2223593379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c30209e00 msgr2=0x556c3020dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:10.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.644+0000 7feb61800700 1 --2- 172.21.15.131:0/2223593379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c30209e00 0x556c3020dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.644+0000 7feb61800700 1 -- 172.21.15.131:0/2223593379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c30204890 msgr2=0x556c3020da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:10.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.644+0000 7feb61800700 1 --2- 172.21.15.131:0/2223593379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c30204890 0x556c3020da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.644+0000 7feb61800700 1 -- 172.21.15.131:0/2223593379 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb5000ea80 con 0x556c3020a720 2024-05-08T22:15:10.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.645+0000 7feb61800700 1 --2- 172.21.15.131:0/2223593379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3020a720 0x556c3020e4f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7feb54006070 tx=0x7feb54004d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23ef0dc6b7273709 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:10.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.645+0000 7feb42a00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:15:10.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.645+0000 7feb42a00700 1 -- 172.21.15.131:0/2223593379 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5400e040 con 0x556c3020a720 2024-05-08T22:15:10.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.645+0000 7feb42a00700 1 -- 172.21.15.131:0/2223593379 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7feb540078f0 con 0x556c3020a720 2024-05-08T22:15:10.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.645+0000 7feb773d0b80 1 -- 172.21.15.131:0/2223593379 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c300ad270 con 0x556c3020a720 2024-05-08T22:15:10.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.646+0000 7feb42a00700 1 -- 172.21.15.131:0/2223593379 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb54009b80 con 0x556c3020a720 2024-05-08T22:15:10.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.646+0000 7feb773d0b80 10 client.26387 fetch_fsmap learned FSMap version 28 2024-05-08T22:15:10.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.646+0000 7feb773d0b80 1 -- 172.21.15.131:0/2223593379 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556c2ff81b60 con 0x556c3020a720 2024-05-08T22:15:10.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.647+0000 7feb42a00700 1 -- 172.21.15.131:0/2223593379 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556c300ad270 con 0x556c3020a720 2024-05-08T22:15:10.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.647+0000 7feb42a00700 1 -- 172.21.15.131:0/2223593379 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7feb54019050 con 0x556c3020a720 2024-05-08T22:15:10.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.647+0000 7feb773d0b80 10 client.26387 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T22:15:10.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.647+0000 7feb773d0b80 10 client.26387 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:15:10.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.647+0000 7feb773d0b80 20 client.26387 populate_metadata read hostname 'smithi131' 2024-05-08T22:15:10.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.648+0000 7feb773d0b80 1 --2- 172.21.15.131:0/2223593379 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x556c3021e1e0 0x556c30220600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:10.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.648+0000 7feb773d0b80 4 client.26387 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:15:10.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.648+0000 7feb773d0b80 1 -- 172.21.15.131:0/2223593379 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c301d3810 con 0x556c3021e1e0 2024-05-08T22:15:10.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.648+0000 7feb60e00700 1 --2- 172.21.15.131:0/2223593379 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x556c3021e1e0 0x556c30220600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:10.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.652+0000 7feb60e00700 1 --2- 172.21.15.131:0/2223593379 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x556c3021e1e0 0x556c30220600 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bca8f840ffe55e7 server_cookie=1d908ab982c879f in_seq=0 out_seq=0 2024-05-08T22:15:10.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.652+0000 7feb42a00700 10 client.26387 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:15:10.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.653+0000 7feb42a00700 1 -- 172.21.15.131:0/2223593379 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556c301d3810 con 0x556c3021e1e0 2024-05-08T22:15:10.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.653+0000 7feb42a00700 10 client.26387 handle_command_reply: tid=0 2024-05-08T22:15:10.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.653+0000 7feb773d0b80 1 client.26387 shutdown 2024-05-08T22:15:10.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.653+0000 7feb773d0b80 1 -- 172.21.15.131:0/2223593379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3020a720 msgr2=0x556c3020e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:10.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.654+0000 7feb773d0b80 1 --2- 172.21.15.131:0/2223593379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3020a720 0x556c3020e4f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7feb54006070 tx=0x7feb54004d80 comp rx=0 tx=0).stop 2024-05-08T22:15:10.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.654+0000 7feb773d0b80 1 -- 172.21.15.131:0/2223593379 shutdown_connections 2024-05-08T22:15:10.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.654+0000 7feb773d0b80 1 --2- 172.21.15.131:0/2223593379 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x556c3021e1e0 0x556c30220600 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.654+0000 7feb773d0b80 1 --2- 172.21.15.131:0/2223593379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3020a720 0x556c3020e4f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.654+0000 7feb773d0b80 1 --2- 172.21.15.131:0/2223593379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c30204890 0x556c3020da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.654+0000 7feb773d0b80 1 --2- 172.21.15.131:0/2223593379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c30209e00 0x556c3020dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.654+0000 7feb773d0b80 1 -- 172.21.15.131:0/2223593379 >> 172.21.15.131:0/2223593379 conn(0x556c30162210 msgr2=0x556c301fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:10.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.654+0000 7feb42a00700 0 client.26387 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:15:10.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.654+0000 7feb773d0b80 1 -- 172.21.15.131:0/2223593379 shutdown_connections 2024-05-08T22:15:10.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.654+0000 7feb773d0b80 1 -- 172.21.15.131:0/2223593379 wait complete. 2024-05-08T22:15:10.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.655+0000 7feb773d0b80 20 client.26387 trim_cache size 0 max 16384 2024-05-08T22:15:10.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.681+0000 7feb773d0b80 1 Processor -- start 2024-05-08T22:15:10.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.682+0000 7feb773d0b80 1 -- start start 2024-05-08T22:15:10.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.682+0000 7feb773d0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c30204890 0x556c30204350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:10.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.682+0000 7feb773d0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c30209e00 0x556c30209280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:10.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.682+0000 7feb773d0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c3020a720 0x556c301fed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:10.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.682+0000 7feb773d0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c301f7080 con 0x556c30204890 2024-05-08T22:15:10.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.682+0000 7feb773d0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c301fbdc0 con 0x556c30209e00 2024-05-08T22:15:10.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.682+0000 7feb773d0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c3020a560 con 0x556c3020a720 2024-05-08T22:15:10.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.683+0000 7feb60e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c30204890 0x556c30204350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:10.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.683+0000 7feb60e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c30204890 0x556c30204350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55942/0 (socket says 172.21.15.131:55942) 2024-05-08T22:15:10.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.683+0000 7feb60e00700 1 -- 172.21.15.131:0/2177514951 learned_addr learned my addr 172.21.15.131:0/2177514951 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:10.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.684+0000 7feb60e00700 1 -- 172.21.15.131:0/2177514951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c3020a720 msgr2=0x556c301fed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:10.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.684+0000 7feb60e00700 1 --2- 172.21.15.131:0/2177514951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c3020a720 0x556c301fed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.684+0000 7feb5be00700 1 --2- 172.21.15.131:0/2177514951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c30209e00 0x556c30209280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:10.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.684+0000 7feb60e00700 1 -- 172.21.15.131:0/2177514951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c30209e00 msgr2=0x556c30209280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:10.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.684+0000 7feb60e00700 1 --2- 172.21.15.131:0/2177514951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c30209e00 0x556c30209280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.684+0000 7feb60e00700 1 -- 172.21.15.131:0/2177514951 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c3011a560 con 0x556c30204890 2024-05-08T22:15:10.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.684+0000 7feb5be00700 1 --2- 172.21.15.131:0/2177514951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c30209e00 0x556c30209280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:15:10.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.684+0000 7feb60e00700 1 --2- 172.21.15.131:0/2177514951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c30204890 0x556c30204350 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7feb48096a40 tx=0x7feb48010c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992aa34d17e5c25 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:10.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.685+0000 7feb4ea00700 1 -- 172.21.15.131:0/2177514951 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb48082070 con 0x556c30204890 2024-05-08T22:15:10.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.685+0000 7feb4ea00700 1 -- 172.21.15.131:0/2177514951 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb480a2450 con 0x556c30204890 2024-05-08T22:15:10.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.685+0000 7feb4ea00700 1 -- 172.21.15.131:0/2177514951 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb480966e0 con 0x556c30204890 2024-05-08T22:15:10.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.686+0000 7feb773d0b80 1 -- 172.21.15.131:0/2177514951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c30204890 msgr2=0x556c30204350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:10.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.686+0000 7feb773d0b80 1 --2- 172.21.15.131:0/2177514951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c30204890 0x556c30204350 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7feb48096a40 tx=0x7feb48010c90 comp rx=0 tx=0).stop 2024-05-08T22:15:10.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.686+0000 7feb773d0b80 1 -- 172.21.15.131:0/2177514951 shutdown_connections 2024-05-08T22:15:10.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.686+0000 7feb773d0b80 1 --2- 172.21.15.131:0/2177514951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c3020a720 0x556c301fed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.686+0000 7feb773d0b80 1 --2- 172.21.15.131:0/2177514951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c30204890 0x556c30204350 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.686+0000 7feb773d0b80 1 --2- 172.21.15.131:0/2177514951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c30209e00 0x556c30209280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.686+0000 7feb773d0b80 1 -- 172.21.15.131:0/2177514951 >> 172.21.15.131:0/2177514951 conn(0x556c30162210 msgr2=0x556c30208b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:10.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.686+0000 7feb773d0b80 1 -- 172.21.15.131:0/2177514951 shutdown_connections 2024-05-08T22:15:10.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.687+0000 7feb773d0b80 1 -- 172.21.15.131:0/2177514951 wait complete. 2024-05-08T22:15:10.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.687+0000 7feb773d0b80 1 -- start start 2024-05-08T22:15:10.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.687+0000 7feb773d0b80 1 Processor -- start 2024-05-08T22:15:10.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.689+0000 7feb773d0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c30204890 0x556c30215360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:10.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.689+0000 7feb773d0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c30209e00 0x556c30205fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:10.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.689+0000 7feb773d0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3020a720 0x556c302064e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:10.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.689+0000 7feb773d0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c30215a70 con 0x556c30204890 2024-05-08T22:15:10.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.689+0000 7feb773d0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c30206a20 con 0x556c3020a720 2024-05-08T22:15:10.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.689+0000 7feb773d0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c30206be0 con 0x556c30209e00 2024-05-08T22:15:10.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.689+0000 7feb773d0b80 10 client.0 _ll_register_callbacks cb 0x556c300f9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:15:10.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.690+0000 7feb60e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c30204890 0x556c30215360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:10.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.690+0000 7feb60e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c30204890 0x556c30215360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55954/0 (socket says 172.21.15.131:55954) 2024-05-08T22:15:10.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.690+0000 7feb60e00700 1 -- 172.21.15.131:0/1882175929 learned_addr learned my addr 172.21.15.131:0/1882175929 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:10.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.690+0000 7feb60e00700 1 -- 172.21.15.131:0/1882175929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c30209e00 msgr2=0x556c30205fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:10.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.690+0000 7feb60e00700 1 --2- 172.21.15.131:0/1882175929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c30209e00 0x556c30205fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.690+0000 7feb60e00700 1 -- 172.21.15.131:0/1882175929 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3020a720 msgr2=0x556c302064e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:10.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.690+0000 7feb60e00700 1 --2- 172.21.15.131:0/1882175929 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3020a720 0x556c302064e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.690+0000 7feb60e00700 1 -- 172.21.15.131:0/1882175929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb5c110d70 con 0x556c30204890 2024-05-08T22:15:10.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.691+0000 7feb60e00700 1 --2- 172.21.15.131:0/1882175929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c30204890 0x556c30215360 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7feb480964d0 tx=0x7feb480cd160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aadcb3db53d7cac1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:10.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.691+0000 7feb42a00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:15:10.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.692+0000 7feb42a00700 1 -- 172.21.15.131:0/1882175929 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb480adea0 con 0x556c30204890 2024-05-08T22:15:10.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.692+0000 7feb773d0b80 1 -- 172.21.15.131:0/1882175929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb50015dd0 con 0x556c30204890 2024-05-08T22:15:10.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.692+0000 7feb42a00700 1 -- 172.21.15.131:0/1882175929 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7feb48094be0 con 0x556c30204890 2024-05-08T22:15:10.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.693+0000 7feb42a00700 1 -- 172.21.15.131:0/1882175929 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb48014020 con 0x556c30204890 2024-05-08T22:15:10.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.693+0000 7feb773d0b80 10 client.16368 fetch_fsmap learned FSMap version 28 2024-05-08T22:15:10.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.693+0000 7feb773d0b80 1 -- 172.21.15.131:0/1882175929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556c2ffd9cd0 con 0x556c30204890 2024-05-08T22:15:10.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.693+0000 7feb42a00700 1 -- 172.21.15.131:0/1882175929 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb50015dd0 con 0x556c30204890 2024-05-08T22:15:10.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.693+0000 7feb42a00700 1 -- 172.21.15.131:0/1882175929 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7feb4809a230 con 0x556c30204890 2024-05-08T22:15:10.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.694+0000 7feb773d0b80 10 client.16368 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T22:15:10.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.694+0000 7feb773d0b80 10 client.16368 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:15:10.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.694+0000 7feb773d0b80 20 client.16368 populate_metadata read hostname 'smithi131' 2024-05-08T22:15:10.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.694+0000 7feb773d0b80 1 --2- 172.21.15.131:0/1882175929 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x556c30286750 0x556c30288b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:10.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.694+0000 7feb773d0b80 4 client.16368 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:15:10.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.694+0000 7feb773d0b80 1 -- 172.21.15.131:0/1882175929 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556c3014cec0 con 0x556c30286750 2024-05-08T22:15:10.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.696+0000 7feb5be00700 1 --2- 172.21.15.131:0/1882175929 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x556c30286750 0x556c30288b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:10.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.697+0000 7feb42a00700 10 client.16368 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:15:10.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.697+0000 7feb5be00700 1 --2- 172.21.15.131:0/1882175929 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x556c30286750 0x556c30288b70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a1e3bea45665baa7 server_cookie=d725570f4c5b8628 in_seq=0 out_seq=0 2024-05-08T22:15:10.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.697+0000 7feb42a00700 1 -- 172.21.15.131:0/1882175929 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556c3014cec0 con 0x556c30286750 2024-05-08T22:15:10.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.697+0000 7feb42a00700 10 client.16368 handle_command_reply: tid=0 2024-05-08T22:15:10.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.697+0000 7feb773d0b80 1 client.16368 shutdown 2024-05-08T22:15:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.698+0000 7feb773d0b80 1 -- 172.21.15.131:0/1882175929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c30204890 msgr2=0x556c30215360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.698+0000 7feb773d0b80 1 --2- 172.21.15.131:0/1882175929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c30204890 0x556c30215360 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7feb480964d0 tx=0x7feb480cd160 comp rx=0 tx=0).stop 2024-05-08T22:15:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.698+0000 7feb773d0b80 1 -- 172.21.15.131:0/1882175929 shutdown_connections 2024-05-08T22:15:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.698+0000 7feb773d0b80 1 --2- 172.21.15.131:0/1882175929 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x556c30286750 0x556c30288b70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.698+0000 7feb773d0b80 1 --2- 172.21.15.131:0/1882175929 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3020a720 0x556c302064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.698+0000 7feb773d0b80 1 --2- 172.21.15.131:0/1882175929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c30204890 0x556c30215360 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.698+0000 7feb42a00700 0 client.16368 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:15:10.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.698+0000 7feb773d0b80 1 --2- 172.21.15.131:0/1882175929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c30209e00 0x556c30205fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.698+0000 7feb773d0b80 1 -- 172.21.15.131:0/1882175929 >> 172.21.15.131:0/1882175929 conn(0x556c30162210 msgr2=0x556c3020b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:10.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.698+0000 7feb773d0b80 1 -- 172.21.15.131:0/1882175929 shutdown_connections 2024-05-08T22:15:10.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.699+0000 7feb773d0b80 1 -- 172.21.15.131:0/1882175929 wait complete. 2024-05-08T22:15:10.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.699+0000 7feb773d0b80 20 client.16368 trim_cache size 0 max 16384 2024-05-08T22:15:10.701 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:15:10.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.701+0000 7feb63800700 1 -- 172.21.15.131:0/917256147 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7feb440a1310 msgr2=0x7feb440a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:10.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.701+0000 7feb63800700 1 --2- 172.21.15.131:0/917256147 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7feb440a1310 0x7feb440a37d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7feb5c19ad50 tx=0x7feb50009190 comp rx=0 tx=0).stop 2024-05-08T22:15:10.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.701+0000 7feb63800700 1 -- 172.21.15.131:0/917256147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb5c0ff410 msgr2=0x7feb5c19f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:10.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.701+0000 7feb63800700 1 --2- 172.21.15.131:0/917256147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb5c0ff410 0x7feb5c19f480 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7feb4800d950 tx=0x7feb4800dc60 comp rx=0 tx=0).stop 2024-05-08T22:15:10.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.704+0000 7feb63800700 1 -- 172.21.15.131:0/917256147 shutdown_connections 2024-05-08T22:15:10.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.704+0000 7feb63800700 1 --2- 172.21.15.131:0/917256147 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7feb440a1310 0x7feb440a37d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.704+0000 7feb63800700 1 --2- 172.21.15.131:0/917256147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb5c105ef0 0x7feb5c199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.704+0000 7feb63800700 1 --2- 172.21.15.131:0/917256147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb5c0ff410 0x7feb5c19f480 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.704+0000 7feb63800700 1 --2- 172.21.15.131:0/917256147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb5c0ffdd0 0x7feb5c19f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:10.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.704+0000 7feb63800700 1 -- 172.21.15.131:0/917256147 >> 172.21.15.131:0/917256147 conn(0x7feb5c0763a0 msgr2=0x7feb5c102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:10.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.704+0000 7feb63800700 1 -- 172.21.15.131:0/917256147 shutdown_connections 2024-05-08T22:15:10.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:10.704+0000 7feb63800700 1 -- 172.21.15.131:0/917256147 wait complete. 2024-05-08T22:15:10.744 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T22:15:10.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:10 smithi132 ceph-mon[78180]: pgmap v241: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 1.06k op/s 2024-05-08T22:15:10.843 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:10 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3574240314' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:15:11.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:10 smithi157 ceph-mon[77746]: pgmap v241: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 1.06k op/s 2024-05-08T22:15:11.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:10 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3574240314' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:15:11.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.065+0000 7f8058000700 1 -- 172.21.15.131:0/2561579611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80501088c0 msgr2=0x7f8050113bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:11.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.065+0000 7f8058000700 1 --2- 172.21.15.131:0/2561579611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80501088c0 0x7f8050113bb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f804000b0d0 tx=0x7f804000b3e0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.066+0000 7f8058000700 1 -- 172.21.15.131:0/2561579611 shutdown_connections 2024-05-08T22:15:11.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.066+0000 7f8058000700 1 --2- 172.21.15.131:0/2561579611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80501140f0 0x7f80501164e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.066+0000 7f8058000700 1 --2- 172.21.15.131:0/2561579611 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8050107fa0 0x7f8050108380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.066+0000 7f8058000700 1 --2- 172.21.15.131:0/2561579611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80501088c0 0x7f8050113bb0 secure :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f804000b0d0 tx=0x7f804000b3e0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.066+0000 7f8058000700 1 -- 172.21.15.131:0/2561579611 >> 172.21.15.131:0/2561579611 conn(0x7f805006bff0 msgr2=0x7f805006c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:11.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.066+0000 7f8058000700 1 -- 172.21.15.131:0/2561579611 shutdown_connections 2024-05-08T22:15:11.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.066+0000 7f8058000700 1 -- 172.21.15.131:0/2561579611 wait complete. 2024-05-08T22:15:11.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.068+0000 7f8058000700 1 Processor -- start 2024-05-08T22:15:11.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.068+0000 7f8058000700 1 -- start start 2024-05-08T22:15:11.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.069+0000 7f8058000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8050107fa0 0x7f8050109a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.070+0000 7f8058000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80501140f0 0x7f8050109f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.070+0000 7f8058000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f805010a490 0x7f80501ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.070+0000 7f8058000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80501ae610 con 0x7f805010a490 2024-05-08T22:15:11.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.070+0000 7f8058000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80501ae780 con 0x7f80501140f0 2024-05-08T22:15:11.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.070+0000 7f8058000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80501ae940 con 0x7f8050107fa0 2024-05-08T22:15:11.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.070+0000 7f8055600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8050107fa0 0x7f8050109a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.070+0000 7f8054c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80501140f0 0x7f8050109f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.070+0000 7f8055600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8050107fa0 0x7f8050109a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:58954/0 (socket says 172.21.15.131:58954) 2024-05-08T22:15:11.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.070+0000 7f8055600700 1 -- 172.21.15.131:0/1593067066 learned_addr learned my addr 172.21.15.131:0/1593067066 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:11.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.070+0000 7f8056000700 1 --2- 172.21.15.131:0/1593067066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f805010a490 0x7f80501ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.071+0000 7f8055600700 1 -- 172.21.15.131:0/1593067066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80501140f0 msgr2=0x7f8050109f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:11.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.071+0000 7f8055600700 1 --2- 172.21.15.131:0/1593067066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80501140f0 0x7f8050109f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.071+0000 7f8055600700 1 -- 172.21.15.131:0/1593067066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f805010a490 msgr2=0x7f80501ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:11.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.071+0000 7f8055600700 1 --2- 172.21.15.131:0/1593067066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f805010a490 0x7f80501ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.071+0000 7f8055600700 1 -- 172.21.15.131:0/1593067066 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8040009d00 con 0x7f8050107fa0 2024-05-08T22:15:11.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.071+0000 7f8054c00700 1 --2- 172.21.15.131:0/1593067066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80501140f0 0x7f8050109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:15:11.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.073+0000 7f8055600700 1 --2- 172.21.15.131:0/1593067066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8050107fa0 0x7f8050109a10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f804800e9a0 tx=0x7f804800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:11.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.074+0000 7f804e000700 1 -- 172.21.15.131:0/1593067066 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8048009790 con 0x7f8050107fa0 2024-05-08T22:15:11.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.074+0000 7f804e000700 1 -- 172.21.15.131:0/1593067066 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8048010430 con 0x7f8050107fa0 2024-05-08T22:15:11.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.074+0000 7f804e000700 1 -- 172.21.15.131:0/1593067066 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8048018680 con 0x7f8050107fa0 2024-05-08T22:15:11.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.074+0000 7f8058000700 1 -- 172.21.15.131:0/1593067066 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80501aec20 con 0x7f8050107fa0 2024-05-08T22:15:11.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.074+0000 7f8058000700 1 -- 172.21.15.131:0/1593067066 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80501af0b0 con 0x7f8050107fa0 2024-05-08T22:15:11.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.082+0000 7f804e000700 1 -- 172.21.15.131:0/1593067066 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f80480105a0 con 0x7f8050107fa0 2024-05-08T22:15:11.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.083+0000 7f806bbfab80 1 Processor -- start 2024-05-08T22:15:11.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.084+0000 7f806bbfab80 1 -- start start 2024-05-08T22:15:11.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.085+0000 7f804e000700 1 --2- 172.21.15.131:0/1593067066 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f80340a14a0 0x7f80340a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.085+0000 7f804e000700 1 -- 172.21.15.131:0/1593067066 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f8048014070 con 0x7f8050107fa0 2024-05-08T22:15:11.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.085+0000 7f806bbfab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644c2591660 0x5644c2591a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.085+0000 7f806bbfab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5644c2591f80 0x5644c258bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.085+0000 7f806bbfab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644c258c0f0 0x5644c258c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.085+0000 7f806bbfab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644c24da150 con 0x5644c2591660 2024-05-08T22:15:11.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.086+0000 7f806bbfab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644c24d7b10 con 0x5644c2591f80 2024-05-08T22:15:11.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.086+0000 7f806bbfab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644c24d3400 con 0x5644c258c0f0 2024-05-08T22:15:11.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.086+0000 7f8056000700 1 --2- 172.21.15.131:0/1593067066 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f80340a14a0 0x7f80340a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.086+0000 7f8055600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644c2591660 0x5644c2591a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.087+0000 7f8056000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644c258c0f0 0x5644c258c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.087+0000 7f8055600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644c2591660 0x5644c2591a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55988/0 (socket says 172.21.15.131:55988) 2024-05-08T22:15:11.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.087+0000 7f8056000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644c258c0f0 0x5644c258c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:58964/0 (socket says 172.21.15.131:58964) 2024-05-08T22:15:11.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.087+0000 7f8056000700 1 -- 172.21.15.131:0/763920691 learned_addr learned my addr 172.21.15.131:0/763920691 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:11.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.087+0000 7f8055600700 1 -- 172.21.15.131:0/763920691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644c258c0f0 msgr2=0x5644c258c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:11.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.087+0000 7f8055600700 1 --2- 172.21.15.131:0/763920691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644c258c0f0 0x5644c258c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.087+0000 7f8055600700 1 -- 172.21.15.131:0/763920691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5644c2591f80 msgr2=0x5644c258bb60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:15:11.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.087+0000 7f8055600700 1 --2- 172.21.15.131:0/763920691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5644c2591f80 0x5644c258bb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.087+0000 7f8055600700 1 -- 172.21.15.131:0/763920691 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644c23543b0 con 0x5644c2591660 2024-05-08T22:15:11.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.087+0000 7f8056000700 1 --2- 172.21.15.131:0/763920691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644c258c0f0 0x5644c258c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:15:11.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.088+0000 7f8056000700 1 --2- 172.21.15.131:0/1593067066 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f80340a14a0 0x7f80340a3960 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f803c015410 tx=0x7f803c0099c0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:11.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.088+0000 7f8055600700 1 --2- 172.21.15.131:0/763920691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644c2591660 0x5644c2591a40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f80480ca500 tx=0x7f8048099910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1242b6016a6bb551 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:11.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.088+0000 7f8046600700 1 -- 172.21.15.131:0/763920691 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804809dcf0 con 0x5644c2591660 2024-05-08T22:15:11.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.089+0000 7f8046600700 1 -- 172.21.15.131:0/763920691 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80480a2340 con 0x5644c2591660 2024-05-08T22:15:11.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.089+0000 7f8046600700 1 -- 172.21.15.131:0/763920691 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8048099610 con 0x5644c2591660 2024-05-08T22:15:11.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.089+0000 7f806bbfab80 1 -- 172.21.15.131:0/763920691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644c2591660 msgr2=0x5644c2591a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:11.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.089+0000 7f806bbfab80 1 --2- 172.21.15.131:0/763920691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644c2591660 0x5644c2591a40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f80480ca500 tx=0x7f8048099910 comp rx=0 tx=0).stop 2024-05-08T22:15:11.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.090+0000 7f806bbfab80 1 -- 172.21.15.131:0/763920691 shutdown_connections 2024-05-08T22:15:11.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.090+0000 7f806bbfab80 1 --2- 172.21.15.131:0/763920691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644c258c0f0 0x5644c258c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.090+0000 7f806bbfab80 1 --2- 172.21.15.131:0/763920691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644c2591660 0x5644c2591a40 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.090+0000 7f806bbfab80 1 --2- 172.21.15.131:0/763920691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5644c2591f80 0x5644c258bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.090+0000 7f806bbfab80 1 -- 172.21.15.131:0/763920691 >> 172.21.15.131:0/763920691 conn(0x5644c24e1210 msgr2=0x5644c257ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:11.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.092+0000 7f806bbfab80 1 -- 172.21.15.131:0/763920691 shutdown_connections 2024-05-08T22:15:11.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.092+0000 7f806bbfab80 1 -- 172.21.15.131:0/763920691 wait complete. 2024-05-08T22:15:11.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.093+0000 7f806bbfab80 1 -- start start 2024-05-08T22:15:11.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.093+0000 7f806bbfab80 1 Processor -- start 2024-05-08T22:15:11.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.094+0000 7f806bbfab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5644c258c0f0 0x5644c25952b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.095+0000 7f806bbfab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644c2591660 0x5644c25957f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.095+0000 7f806bbfab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644c2591f80 0x5644c2595d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.095+0000 7f8055600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5644c258c0f0 0x5644c25952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.095+0000 7f8055600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5644c258c0f0 0x5644c25952b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:41558/0 (socket says 172.21.15.131:41558) 2024-05-08T22:15:11.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.095+0000 7f8055600700 1 -- 172.21.15.131:0/212390646 learned_addr learned my addr 172.21.15.131:0/212390646 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:11.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.095+0000 7f8054c00700 1 --2- 172.21.15.131:0/212390646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644c2591660 0x5644c25957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.095+0000 7f806bbfab80 1 -- 172.21.15.131:0/212390646 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644c246f560 con 0x5644c2591f80 2024-05-08T22:15:11.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.095+0000 7f806bbfab80 1 -- 172.21.15.131:0/212390646 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644c246f200 con 0x5644c258c0f0 2024-05-08T22:15:11.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.096+0000 7f806bbfab80 1 -- 172.21.15.131:0/212390646 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644c23566a0 con 0x5644c2591660 2024-05-08T22:15:11.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.096+0000 7f806bbfab80 10 client.0 _ll_register_callbacks cb 0x5644c2478b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:15:11.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.096+0000 7f8056000700 1 --2- 172.21.15.131:0/212390646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644c2591f80 0x5644c2595d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.096+0000 7f8054c00700 1 -- 172.21.15.131:0/212390646 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5644c258c0f0 msgr2=0x5644c25952b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:11.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.097+0000 7f8054c00700 1 --2- 172.21.15.131:0/212390646 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5644c258c0f0 0x5644c25952b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.097+0000 7f8054c00700 1 -- 172.21.15.131:0/212390646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644c2591f80 msgr2=0x5644c2595d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:11.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.097+0000 7f8054c00700 1 --2- 172.21.15.131:0/212390646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644c2591f80 0x5644c2595d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.097+0000 7f8054c00700 1 -- 172.21.15.131:0/212390646 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80501af0b0 con 0x5644c2591660 2024-05-08T22:15:11.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.097+0000 7f8054c00700 1 --2- 172.21.15.131:0/212390646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644c2591660 0x5644c25957f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8040008000 tx=0x7f8040004c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f9b7fa1d5437ef6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:11.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.097+0000 7f803aa00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:15:11.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.098+0000 7f803aa00700 1 -- 172.21.15.131:0/212390646 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804000e040 con 0x5644c2591660 2024-05-08T22:15:11.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.098+0000 7f803aa00700 1 -- 172.21.15.131:0/212390646 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f80400078f0 con 0x5644c2591660 2024-05-08T22:15:11.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.098+0000 7f806bbfab80 1 -- 172.21.15.131:0/212390646 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5644c242c270 con 0x5644c2591660 2024-05-08T22:15:11.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.098+0000 7f803aa00700 1 -- 172.21.15.131:0/212390646 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80400094c0 con 0x5644c2591660 2024-05-08T22:15:11.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.098+0000 7f806bbfab80 10 client.26464 fetch_fsmap learned FSMap version 28 2024-05-08T22:15:11.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.098+0000 7f806bbfab80 1 -- 172.21.15.131:0/212390646 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5644c2300b60 con 0x5644c2591660 2024-05-08T22:15:11.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.101+0000 7f803aa00700 1 -- 172.21.15.131:0/212390646 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5644c242c270 con 0x5644c2591660 2024-05-08T22:15:11.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.101+0000 7f803aa00700 1 -- 172.21.15.131:0/212390646 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8040022020 con 0x5644c2591660 2024-05-08T22:15:11.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.101+0000 7f806bbfab80 10 client.26464 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T22:15:11.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.101+0000 7f806bbfab80 10 client.26464 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:15:11.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.101+0000 7f806bbfab80 20 client.26464 populate_metadata read hostname 'smithi131' 2024-05-08T22:15:11.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.101+0000 7f806bbfab80 1 --2- 172.21.15.131:0/212390646 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x5644c25a59e0 0x5644c25a7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.101+0000 7f806bbfab80 4 client.26464 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:15:11.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.101+0000 7f806bbfab80 1 -- 172.21.15.131:0/212390646 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644c2552810 con 0x5644c25a59e0 2024-05-08T22:15:11.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.103+0000 7f8055600700 1 --2- 172.21.15.131:0/212390646 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x5644c25a59e0 0x5644c25a7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.104+0000 7f8055600700 1 --2- 172.21.15.131:0/212390646 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x5644c25a59e0 0x5644c25a7e00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=932777b23bab5396 server_cookie=23ebec019bf7a514 in_seq=0 out_seq=0 2024-05-08T22:15:11.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.104+0000 7f803aa00700 10 client.26464 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:15:11.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.107+0000 7f803aa00700 1 -- 172.21.15.131:0/212390646 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5644c2552810 con 0x5644c25a59e0 2024-05-08T22:15:11.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.107+0000 7f803aa00700 10 client.26464 handle_command_reply: tid=0 2024-05-08T22:15:11.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.107+0000 7f806bbfab80 1 client.26464 shutdown 2024-05-08T22:15:11.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.107+0000 7f806bbfab80 1 -- 172.21.15.131:0/212390646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644c2591660 msgr2=0x5644c25957f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:11.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.107+0000 7f806bbfab80 1 --2- 172.21.15.131:0/212390646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644c2591660 0x5644c25957f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8040008000 tx=0x7f8040004c60 comp rx=0 tx=0).stop 2024-05-08T22:15:11.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.108+0000 7f806bbfab80 1 -- 172.21.15.131:0/212390646 shutdown_connections 2024-05-08T22:15:11.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.108+0000 7f806bbfab80 1 --2- 172.21.15.131:0/212390646 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x5644c25a59e0 0x5644c25a7e00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.108+0000 7f806bbfab80 1 --2- 172.21.15.131:0/212390646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644c2591f80 0x5644c2595d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.108+0000 7f806bbfab80 1 --2- 172.21.15.131:0/212390646 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5644c258c0f0 0x5644c25952b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.108+0000 7f806bbfab80 1 --2- 172.21.15.131:0/212390646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644c2591660 0x5644c25957f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.108+0000 7f806bbfab80 1 -- 172.21.15.131:0/212390646 >> 172.21.15.131:0/212390646 conn(0x5644c24e1210 msgr2=0x5644c257d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:11.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.110+0000 7f803aa00700 0 client.26464 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:15:11.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.110+0000 7f806bbfab80 1 -- 172.21.15.131:0/212390646 shutdown_connections 2024-05-08T22:15:11.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.110+0000 7f806bbfab80 1 -- 172.21.15.131:0/212390646 wait complete. 2024-05-08T22:15:11.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.111+0000 7f806bbfab80 20 client.26464 trim_cache size 0 max 16384 2024-05-08T22:15:11.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.141+0000 7f806bbfab80 1 Processor -- start 2024-05-08T22:15:11.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.141+0000 7f806bbfab80 1 -- start start 2024-05-08T22:15:11.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.141+0000 7f806bbfab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644c258c0f0 0x5644c258bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.141+0000 7f806bbfab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644c2591660 0x5644c257dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.141+0000 7f806bbfab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5644c2591f80 0x5644c25ac760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.141+0000 7f806bbfab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644c258cad0 con 0x5644c2591660 2024-05-08T22:15:11.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.141+0000 7f806bbfab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644c2576040 con 0x5644c2591f80 2024-05-08T22:15:11.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.141+0000 7f806bbfab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644c2578800 con 0x5644c258c0f0 2024-05-08T22:15:11.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.142+0000 7f8055600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644c258c0f0 0x5644c258bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.142+0000 7f8055600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644c258c0f0 0x5644c258bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:58978/0 (socket says 172.21.15.131:58978) 2024-05-08T22:15:11.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.142+0000 7f8055600700 1 -- 172.21.15.131:0/3654143899 learned_addr learned my addr 172.21.15.131:0/3654143899 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:11.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.142+0000 7f8054c00700 1 --2- 172.21.15.131:0/3654143899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644c2591660 0x5644c257dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.142+0000 7f8056000700 1 --2- 172.21.15.131:0/3654143899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5644c2591f80 0x5644c25ac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.143+0000 7f8056000700 1 -- 172.21.15.131:0/3654143899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644c258c0f0 msgr2=0x5644c258bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:11.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.143+0000 7f8056000700 1 --2- 172.21.15.131:0/3654143899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644c258c0f0 0x5644c258bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.143+0000 7f8056000700 1 -- 172.21.15.131:0/3654143899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644c2591660 msgr2=0x5644c257dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:11.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.143+0000 7f8056000700 1 --2- 172.21.15.131:0/3654143899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644c2591660 0x5644c257dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.143+0000 7f8056000700 1 -- 172.21.15.131:0/3654143899 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644c2499560 con 0x5644c2591f80 2024-05-08T22:15:11.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.143+0000 7f8054c00700 1 --2- 172.21.15.131:0/3654143899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644c2591660 0x5644c257dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:15:11.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.143+0000 7f8056000700 1 --2- 172.21.15.131:0/3654143899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5644c2591f80 0x5644c25ac760 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f803c015b60 tx=0x7f803c01c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94ff331ba8ee3cd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:11.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.146+0000 7f8047000700 1 -- 172.21.15.131:0/3654143899 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c009d10 con 0x5644c2591f80 2024-05-08T22:15:11.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.146+0000 7f8047000700 1 -- 172.21.15.131:0/3654143899 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f803c018e00 con 0x5644c2591f80 2024-05-08T22:15:11.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.146+0000 7f8047000700 1 -- 172.21.15.131:0/3654143899 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c0173e0 con 0x5644c2591f80 2024-05-08T22:15:11.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.147+0000 7f806bbfab80 1 -- 172.21.15.131:0/3654143899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5644c2591f80 msgr2=0x5644c25ac760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:11.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.147+0000 7f806bbfab80 1 --2- 172.21.15.131:0/3654143899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5644c2591f80 0x5644c25ac760 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f803c015b60 tx=0x7f803c01c000 comp rx=0 tx=0).stop 2024-05-08T22:15:11.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.147+0000 7f806bbfab80 1 -- 172.21.15.131:0/3654143899 shutdown_connections 2024-05-08T22:15:11.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.147+0000 7f806bbfab80 1 --2- 172.21.15.131:0/3654143899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5644c2591f80 0x5644c25ac760 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.147+0000 7f806bbfab80 1 --2- 172.21.15.131:0/3654143899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644c258c0f0 0x5644c258bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.147+0000 7f806bbfab80 1 --2- 172.21.15.131:0/3654143899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644c2591660 0x5644c257dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.147+0000 7f806bbfab80 1 -- 172.21.15.131:0/3654143899 >> 172.21.15.131:0/3654143899 conn(0x5644c24e1210 msgr2=0x5644c257a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:11.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.148+0000 7f806bbfab80 1 -- 172.21.15.131:0/3654143899 shutdown_connections 2024-05-08T22:15:11.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.148+0000 7f806bbfab80 1 -- 172.21.15.131:0/3654143899 wait complete. 2024-05-08T22:15:11.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.149+0000 7f806bbfab80 1 -- start start 2024-05-08T22:15:11.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.149+0000 7f806bbfab80 1 Processor -- start 2024-05-08T22:15:11.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.150+0000 7f806bbfab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5644c258c0f0 0x5644c259c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.150+0000 7f806bbfab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644c2591660 0x5644c259cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.150+0000 7f806bbfab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644c2591f80 0x5644c258d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.150+0000 7f806bbfab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644c258d810 con 0x5644c2591660 2024-05-08T22:15:11.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.150+0000 7f806bbfab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644c258d980 con 0x5644c258c0f0 2024-05-08T22:15:11.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.150+0000 7f806bbfab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644c258db40 con 0x5644c2591f80 2024-05-08T22:15:11.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.150+0000 7f806bbfab80 10 client.0 _ll_register_callbacks cb 0x5644c2478b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:15:11.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.150+0000 7f8055600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5644c258c0f0 0x5644c259c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.150+0000 7f8055600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5644c258c0f0 0x5644c259c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:41564/0 (socket says 172.21.15.131:41564) 2024-05-08T22:15:11.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.151+0000 7f8054c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644c2591660 0x5644c259cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.151+0000 7f8055600700 1 -- 172.21.15.131:0/2278962915 learned_addr learned my addr 172.21.15.131:0/2278962915 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:11.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.151+0000 7f8056000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644c2591f80 0x5644c258d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.151+0000 7f8055600700 1 -- 172.21.15.131:0/2278962915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644c2591f80 msgr2=0x5644c258d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:11.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.151+0000 7f8055600700 1 --2- 172.21.15.131:0/2278962915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644c2591f80 0x5644c258d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.151+0000 7f8055600700 1 -- 172.21.15.131:0/2278962915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644c2591660 msgr2=0x5644c259cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:11.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.151+0000 7f8055600700 1 --2- 172.21.15.131:0/2278962915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644c2591660 0x5644c259cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.151+0000 7f8055600700 1 -- 172.21.15.131:0/2278962915 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644c2300b60 con 0x5644c258c0f0 2024-05-08T22:15:11.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.152+0000 7f8055600700 1 --2- 172.21.15.131:0/2278962915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5644c258c0f0 0x5644c259c9c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f804801a010 tx=0x7f8048099dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5365f2a846a5ba5a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:11.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.152+0000 7f803aa00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:15:11.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.152+0000 7f803aa00700 1 -- 172.21.15.131:0/2278962915 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8048099820 con 0x5644c258c0f0 2024-05-08T22:15:11.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.152+0000 7f803aa00700 1 -- 172.21.15.131:0/2278962915 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f80480995b0 con 0x5644c258c0f0 2024-05-08T22:15:11.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.152+0000 7f806bbfab80 1 -- 172.21.15.131:0/2278962915 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80480a2340 con 0x5644c258c0f0 2024-05-08T22:15:11.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.153+0000 7f803aa00700 1 -- 172.21.15.131:0/2278962915 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80480b8250 con 0x5644c258c0f0 2024-05-08T22:15:11.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.153+0000 7f806bbfab80 10 client.26402 fetch_fsmap learned FSMap version 28 2024-05-08T22:15:11.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.153+0000 7f806bbfab80 1 -- 172.21.15.131:0/2278962915 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f803c015db0 con 0x5644c258c0f0 2024-05-08T22:15:11.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.154+0000 7f803aa00700 1 -- 172.21.15.131:0/2278962915 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80480a2340 con 0x5644c258c0f0 2024-05-08T22:15:11.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.154+0000 7f803aa00700 1 -- 172.21.15.131:0/2278962915 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8048094ad0 con 0x5644c258c0f0 2024-05-08T22:15:11.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.154+0000 7f806bbfab80 10 client.26402 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T22:15:11.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.154+0000 7f806bbfab80 10 client.26402 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:15:11.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.154+0000 7f806bbfab80 20 client.26402 populate_metadata read hostname 'smithi131' 2024-05-08T22:15:11.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.154+0000 7f806bbfab80 1 --2- 172.21.15.131:0/2278962915 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x5644c25fbf60 0x5644c25fe380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.154+0000 7f806bbfab80 4 client.26402 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:15:11.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.154+0000 7f806bbfab80 1 -- 172.21.15.131:0/2278962915 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5644c24cbec0 con 0x5644c25fbf60 2024-05-08T22:15:11.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.155+0000 7f8054c00700 1 --2- 172.21.15.131:0/2278962915 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x5644c25fbf60 0x5644c25fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.156+0000 7f8054c00700 1 --2- 172.21.15.131:0/2278962915 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x5644c25fbf60 0x5644c25fe380 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fbc9a6b0789a54ed server_cookie=cd216befe5e86647 in_seq=0 out_seq=0 2024-05-08T22:15:11.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.156+0000 7f803aa00700 10 client.26402 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:15:11.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.156+0000 7f803aa00700 1 -- 172.21.15.131:0/2278962915 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5644c24cbec0 con 0x5644c25fbf60 2024-05-08T22:15:11.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.156+0000 7f803aa00700 10 client.26402 handle_command_reply: tid=0 2024-05-08T22:15:11.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.156+0000 7f806bbfab80 1 client.26402 shutdown 2024-05-08T22:15:11.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.157+0000 7f806bbfab80 1 -- 172.21.15.131:0/2278962915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5644c258c0f0 msgr2=0x5644c259c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:11.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.157+0000 7f806bbfab80 1 --2- 172.21.15.131:0/2278962915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5644c258c0f0 0x5644c259c9c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f804801a010 tx=0x7f8048099dc0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.157+0000 7f806bbfab80 1 -- 172.21.15.131:0/2278962915 shutdown_connections 2024-05-08T22:15:11.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.157+0000 7f806bbfab80 1 --2- 172.21.15.131:0/2278962915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644c2591f80 0x5644c258d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.157+0000 7f806bbfab80 1 --2- 172.21.15.131:0/2278962915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5644c258c0f0 0x5644c259c9c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.157+0000 7f806bbfab80 1 --2- 172.21.15.131:0/2278962915 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x5644c25fbf60 0x5644c25fe380 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.157+0000 7f806bbfab80 1 --2- 172.21.15.131:0/2278962915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644c2591660 0x5644c259cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.157+0000 7f803aa00700 0 client.26402 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:15:11.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.157+0000 7f806bbfab80 1 -- 172.21.15.131:0/2278962915 >> 172.21.15.131:0/2278962915 conn(0x5644c24e1210 msgr2=0x5644c25ac760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:11.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.157+0000 7f806bbfab80 1 -- 172.21.15.131:0/2278962915 shutdown_connections 2024-05-08T22:15:11.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.158+0000 7f806bbfab80 1 -- 172.21.15.131:0/2278962915 wait complete. 2024-05-08T22:15:11.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.158+0000 7f806bbfab80 20 client.26402 trim_cache size 0 max 16384 2024-05-08T22:15:11.160 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:15:11.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.158+0000 7f8058000700 1 -- 172.21.15.131:0/1593067066 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f80340a14a0 msgr2=0x7f80340a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:11.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.158+0000 7f8058000700 1 --2- 172.21.15.131:0/1593067066 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f80340a14a0 0x7f80340a3960 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f803c015410 tx=0x7f803c0099c0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.158+0000 7f8058000700 1 -- 172.21.15.131:0/1593067066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8050107fa0 msgr2=0x7f8050109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:11.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.158+0000 7f8058000700 1 --2- 172.21.15.131:0/1593067066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8050107fa0 0x7f8050109a10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f804800e9a0 tx=0x7f804800ecb0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.159+0000 7f8058000700 1 -- 172.21.15.131:0/1593067066 shutdown_connections 2024-05-08T22:15:11.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.159+0000 7f8058000700 1 --2- 172.21.15.131:0/1593067066 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f80340a14a0 0x7f80340a3960 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.159+0000 7f8058000700 1 --2- 172.21.15.131:0/1593067066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f805010a490 0x7f80501ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.159+0000 7f8058000700 1 --2- 172.21.15.131:0/1593067066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8050107fa0 0x7f8050109a10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.159+0000 7f8058000700 1 --2- 172.21.15.131:0/1593067066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80501140f0 0x7f8050109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.159+0000 7f8058000700 1 -- 172.21.15.131:0/1593067066 >> 172.21.15.131:0/1593067066 conn(0x7f805006bff0 msgr2=0x7f805006fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:11.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.159+0000 7f8058000700 1 -- 172.21.15.131:0/1593067066 shutdown_connections 2024-05-08T22:15:11.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.159+0000 7f8058000700 1 -- 172.21.15.131:0/1593067066 wait complete. 2024-05-08T22:15:11.179 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T22:15:11.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.404+0000 7f377fc00700 1 -- 172.21.15.131:0/3665868170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3778101180 msgr2=0x7f3778101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:11.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.404+0000 7f377fc00700 1 --2- 172.21.15.131:0/3665868170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3778101180 0x7f3778101560 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f377000b0a0 tx=0x7f377000b3b0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.404+0000 7f377fc00700 1 -- 172.21.15.131:0/3665868170 shutdown_connections 2024-05-08T22:15:11.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.404+0000 7f377fc00700 1 --2- 172.21.15.131:0/3665868170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37781062a0 0x7f377810de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.404+0000 7f377fc00700 1 --2- 172.21.15.131:0/3665868170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3778101180 0x7f3778101560 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.404+0000 7f377fc00700 1 --2- 172.21.15.131:0/3665868170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3778101b30 0x7f3778105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.404+0000 7f377fc00700 1 -- 172.21.15.131:0/3665868170 >> 172.21.15.131:0/3665868170 conn(0x7f37780fc9e0 msgr2=0x7f37780fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:11.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.404+0000 7f377fc00700 1 -- 172.21.15.131:0/3665868170 shutdown_connections 2024-05-08T22:15:11.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.405+0000 7f377fc00700 1 -- 172.21.15.131:0/3665868170 wait complete. 2024-05-08T22:15:11.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.406+0000 7f377fc00700 1 Processor -- start 2024-05-08T22:15:11.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.407+0000 7f377fc00700 1 -- start start 2024-05-08T22:15:11.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.407+0000 7f377fc00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3778101180 0x7f377819f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.407+0000 7f377fc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37781062a0 0x7f377819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.407+0000 7f377fc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3778199450 0x7f3778199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.407+0000 7f377fc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3778199ed0 con 0x7f3778199450 2024-05-08T22:15:11.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.407+0000 7f377fc00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377819a040 con 0x7f3778101180 2024-05-08T22:15:11.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.407+0000 7f377fc00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377819a200 con 0x7f37781062a0 2024-05-08T22:15:11.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.407+0000 7f377d200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3778101180 0x7f377819f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.407+0000 7f377d200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3778101180 0x7f377819f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:41586/0 (socket says 172.21.15.131:41586) 2024-05-08T22:15:11.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.407+0000 7f377d200700 1 -- 172.21.15.131:0/2493902013 learned_addr learned my addr 172.21.15.131:0/2493902013 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:11.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.408+0000 7f377dc00700 1 --2- 172.21.15.131:0/2493902013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3778199450 0x7f3778199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.408+0000 7f377d200700 1 -- 172.21.15.131:0/2493902013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37781062a0 msgr2=0x7f377819f890 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T22:15:11.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.408+0000 7f377d200700 1 --2- 172.21.15.131:0/2493902013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37781062a0 0x7f377819f890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.408+0000 7f377d200700 1 -- 172.21.15.131:0/2493902013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3778199450 msgr2=0x7f3778199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:11.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.408+0000 7f377d200700 1 --2- 172.21.15.131:0/2493902013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3778199450 0x7f3778199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.408+0000 7f377d200700 1 -- 172.21.15.131:0/2493902013 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3770009d00 con 0x7f3778101180 2024-05-08T22:15:11.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.409+0000 7f377dc00700 1 --2- 172.21.15.131:0/2493902013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3778199450 0x7f3778199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:15:11.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.409+0000 7f377d200700 1 --2- 172.21.15.131:0/2493902013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3778101180 0x7f377819f350 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3770006070 tx=0x7f3770007b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:11.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.410+0000 7f3775600700 1 -- 172.21.15.131:0/2493902013 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377000e040 con 0x7f3778101180 2024-05-08T22:15:11.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.410+0000 7f3775600700 1 -- 172.21.15.131:0/2493902013 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3770003e10 con 0x7f3778101180 2024-05-08T22:15:11.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.410+0000 7f3775600700 1 -- 172.21.15.131:0/2493902013 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377000e040 con 0x7f3778101180 2024-05-08T22:15:11.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.410+0000 7f377fc00700 1 -- 172.21.15.131:0/2493902013 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37781a6160 con 0x7f3778101180 2024-05-08T22:15:11.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.412+0000 7f377fc00700 1 -- 172.21.15.131:0/2493902013 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37781a65f0 con 0x7f3778101180 2024-05-08T22:15:11.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.417+0000 7f3775600700 1 -- 172.21.15.131:0/2493902013 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f37700056c0 con 0x7f3778101180 2024-05-08T22:15:11.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.421+0000 7f3775600700 1 --2- 172.21.15.131:0/2493902013 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f375c0a1370 0x7f375c0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.421+0000 7f379370db80 1 Processor -- start 2024-05-08T22:15:11.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.421+0000 7f3775600700 1 -- 172.21.15.131:0/2493902013 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f37700cd2a0 con 0x7f3778101180 2024-05-08T22:15:11.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.423+0000 7f379370db80 1 -- start start 2024-05-08T22:15:11.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.423+0000 7f3777e00700 1 --2- 172.21.15.131:0/2493902013 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f375c0a1370 0x7f375c0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.423+0000 7f379370db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e65dbbc660 0x55e65dbbca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.423+0000 7f379370db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e65dbbcf80 0x55e65dbb6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.423+0000 7f379370db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65dbb70f0 0x55e65dbb7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.423+0000 7f379370db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65db05150 con 0x55e65dbbc660 2024-05-08T22:15:11.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.423+0000 7f379370db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65db02b10 con 0x55e65dbbcf80 2024-05-08T22:15:11.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.423+0000 7f379370db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65dafe400 con 0x55e65dbb70f0 2024-05-08T22:15:11.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.424+0000 7f377dc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65dbb70f0 0x55e65dbb7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.424+0000 7f377dc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65dbb70f0 0x55e65dbb7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59012/0 (socket says 172.21.15.131:59012) 2024-05-08T22:15:11.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.424+0000 7f377dc00700 1 -- 172.21.15.131:0/214220353 learned_addr learned my addr 172.21.15.131:0/214220353 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:11.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.424+0000 7f3777e00700 1 --2- 172.21.15.131:0/214220353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e65dbbcf80 0x55e65dbb6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.424+0000 7f3777e00700 1 --2- 172.21.15.131:0/2493902013 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f375c0a1370 0x7f375c0a3830 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f377819ab60 tx=0x7f376400b3f0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:11.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.424+0000 7f377dc00700 1 -- 172.21.15.131:0/214220353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e65dbbcf80 msgr2=0x55e65dbb6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:11.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.424+0000 7f377dc00700 1 --2- 172.21.15.131:0/214220353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e65dbbcf80 0x55e65dbb6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.424+0000 7f377dc00700 1 -- 172.21.15.131:0/214220353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e65dbbc660 msgr2=0x55e65dbbca40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:15:11.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.425+0000 7f377d200700 1 --2- 172.21.15.131:0/214220353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e65dbbc660 0x55e65dbbca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.425+0000 7f377dc00700 1 --2- 172.21.15.131:0/214220353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e65dbbc660 0x55e65dbbca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.425+0000 7f377dc00700 1 -- 172.21.15.131:0/214220353 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e65d97f3b0 con 0x55e65dbb70f0 2024-05-08T22:15:11.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.425+0000 7f3777e00700 1 --2- 172.21.15.131:0/214220353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e65dbbcf80 0x55e65dbb6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:15:11.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.425+0000 7f377d200700 1 --2- 172.21.15.131:0/214220353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e65dbbc660 0x55e65dbbca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:15:11.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.426+0000 7f377dc00700 1 --2- 172.21.15.131:0/214220353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65dbb70f0 0x55e65dbb7570 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f376800afa0 tx=0x7f376800f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dd7b45ea1a73123 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:11.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.426+0000 7f376e000700 1 -- 172.21.15.131:0/214220353 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3768004220 con 0x55e65dbb70f0 2024-05-08T22:15:11.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.426+0000 7f376e000700 1 -- 172.21.15.131:0/214220353 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3768004380 con 0x55e65dbb70f0 2024-05-08T22:15:11.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.426+0000 7f376e000700 1 -- 172.21.15.131:0/214220353 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3768018890 con 0x55e65dbb70f0 2024-05-08T22:15:11.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.427+0000 7f379370db80 1 -- 172.21.15.131:0/214220353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65dbb70f0 msgr2=0x55e65dbb7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:11.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.427+0000 7f379370db80 1 --2- 172.21.15.131:0/214220353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65dbb70f0 0x55e65dbb7570 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f376800afa0 tx=0x7f376800f6a0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.427+0000 7f379370db80 1 -- 172.21.15.131:0/214220353 shutdown_connections 2024-05-08T22:15:11.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.427+0000 7f379370db80 1 --2- 172.21.15.131:0/214220353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65dbb70f0 0x55e65dbb7570 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.427+0000 7f379370db80 1 --2- 172.21.15.131:0/214220353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e65dbbc660 0x55e65dbbca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.427+0000 7f379370db80 1 --2- 172.21.15.131:0/214220353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e65dbbcf80 0x55e65dbb6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.427+0000 7f379370db80 1 -- 172.21.15.131:0/214220353 >> 172.21.15.131:0/214220353 conn(0x55e65db0c210 msgr2=0x55e65dba8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:11.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.428+0000 7f379370db80 1 -- 172.21.15.131:0/214220353 shutdown_connections 2024-05-08T22:15:11.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.428+0000 7f379370db80 1 -- 172.21.15.131:0/214220353 wait complete. 2024-05-08T22:15:11.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.428+0000 7f379370db80 1 -- start start 2024-05-08T22:15:11.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.429+0000 7f379370db80 1 Processor -- start 2024-05-08T22:15:11.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.429+0000 7f379370db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e65dbb70f0 0x55e65dbc0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.430+0000 7f379370db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e65dbbc660 0x55e65dbc0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.430+0000 7f379370db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65dbbcf80 0x55e65dbc0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.430+0000 7f379370db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65da9a560 con 0x55e65dbb70f0 2024-05-08T22:15:11.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.430+0000 7f379370db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65da9a200 con 0x55e65dbbc660 2024-05-08T22:15:11.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.430+0000 7f379370db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65d9816a0 con 0x55e65dbbcf80 2024-05-08T22:15:11.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.430+0000 7f379370db80 10 client.0 _ll_register_callbacks cb 0x55e65daa3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:15:11.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.430+0000 7f3777e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e65dbbc660 0x55e65dbc0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.430+0000 7f377d200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e65dbb70f0 0x55e65dbc0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.430+0000 7f3777e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e65dbbc660 0x55e65dbc0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:41600/0 (socket says 172.21.15.131:41600) 2024-05-08T22:15:11.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.430+0000 7f3777e00700 1 -- 172.21.15.131:0/2825372996 learned_addr learned my addr 172.21.15.131:0/2825372996 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:11.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.431+0000 7f377d200700 1 -- 172.21.15.131:0/2825372996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65dbbcf80 msgr2=0x55e65dbc0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:11.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.431+0000 7f377d200700 1 --2- 172.21.15.131:0/2825372996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65dbbcf80 0x55e65dbc0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.431+0000 7f377d200700 1 -- 172.21.15.131:0/2825372996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e65dbbc660 msgr2=0x55e65dbc0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:11.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.431+0000 7f377d200700 1 --2- 172.21.15.131:0/2825372996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e65dbbc660 0x55e65dbc0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.431+0000 7f377d200700 1 -- 172.21.15.131:0/2825372996 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f376800ac90 con 0x55e65dbb70f0 2024-05-08T22:15:11.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.431+0000 7f377d200700 1 --2- 172.21.15.131:0/2825372996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e65dbb70f0 0x55e65dbc0440 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f37700ca600 tx=0x7f3770099ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2956f1a270c8ae2d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:11.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.431+0000 7f3762a00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:15:11.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.432+0000 7f3762a00700 1 -- 172.21.15.131:0/2825372996 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377009a4b0 con 0x55e65dbb70f0 2024-05-08T22:15:11.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.432+0000 7f3762a00700 1 -- 172.21.15.131:0/2825372996 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f377009de80 con 0x55e65dbb70f0 2024-05-08T22:15:11.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.432+0000 7f379370db80 1 -- 172.21.15.131:0/2825372996 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e65da57270 con 0x55e65dbb70f0 2024-05-08T22:15:11.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.433+0000 7f3762a00700 1 -- 172.21.15.131:0/2825372996 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37700a2440 con 0x55e65dbb70f0 2024-05-08T22:15:11.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.433+0000 7f379370db80 10 client.16389 fetch_fsmap learned FSMap version 28 2024-05-08T22:15:11.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.433+0000 7f379370db80 1 -- 172.21.15.131:0/2825372996 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e65d92bb60 con 0x55e65dbb70f0 2024-05-08T22:15:11.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.434+0000 7f3762a00700 1 -- 172.21.15.131:0/2825372996 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e65da57270 con 0x55e65dbb70f0 2024-05-08T22:15:11.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.434+0000 7f3762a00700 1 -- 172.21.15.131:0/2825372996 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f37700a1fb0 con 0x55e65dbb70f0 2024-05-08T22:15:11.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.434+0000 7f379370db80 10 client.16389 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T22:15:11.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.434+0000 7f379370db80 10 client.16389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:15:11.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.434+0000 7f379370db80 20 client.16389 populate_metadata read hostname 'smithi131' 2024-05-08T22:15:11.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.434+0000 7f379370db80 1 --2- 172.21.15.131:0/2825372996 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55e65dbd07a0 0x55e65dbd2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.434+0000 7f379370db80 4 client.16389 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:15:11.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.434+0000 7f379370db80 1 -- 172.21.15.131:0/2825372996 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e65db7d810 con 0x55e65dbd07a0 2024-05-08T22:15:11.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.434+0000 7f3777e00700 1 --2- 172.21.15.131:0/2825372996 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55e65dbd07a0 0x55e65dbd2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.437+0000 7f3762a00700 10 client.16389 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:11.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.437+0000 7f3777e00700 1 --2- 172.21.15.131:0/2825372996 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55e65dbd07a0 0x55e65dbd2bc0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc09a0e88824373 server_cookie=9a36d67fa39feabb in_seq=0 out_seq=0 2024-05-08T22:15:11.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.440+0000 7f3762a00700 1 -- 172.21.15.131:0/2825372996 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e65db7d810 con 0x55e65dbd07a0 2024-05-08T22:15:11.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.440+0000 7f3762a00700 10 client.16389 handle_command_reply: tid=0 2024-05-08T22:15:11.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.441+0000 7f379370db80 1 client.16389 shutdown 2024-05-08T22:15:11.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.441+0000 7f379370db80 1 -- 172.21.15.131:0/2825372996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e65dbb70f0 msgr2=0x55e65dbc0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:11.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.441+0000 7f379370db80 1 --2- 172.21.15.131:0/2825372996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e65dbb70f0 0x55e65dbc0440 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f37700ca600 tx=0x7f3770099ae0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.442+0000 7f379370db80 1 -- 172.21.15.131:0/2825372996 shutdown_connections 2024-05-08T22:15:11.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.442+0000 7f379370db80 1 --2- 172.21.15.131:0/2825372996 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55e65dbd07a0 0x55e65dbd2bc0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.442+0000 7f379370db80 1 --2- 172.21.15.131:0/2825372996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65dbbcf80 0x55e65dbc0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.442+0000 7f379370db80 1 --2- 172.21.15.131:0/2825372996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e65dbb70f0 0x55e65dbc0440 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.442+0000 7f379370db80 1 --2- 172.21.15.131:0/2825372996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e65dbbc660 0x55e65dbc0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.442+0000 7f3762a00700 0 client.16389 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:11.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.442+0000 7f379370db80 1 -- 172.21.15.131:0/2825372996 >> 172.21.15.131:0/2825372996 conn(0x55e65db0c210 msgr2=0x55e65dba85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:11.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.442+0000 7f379370db80 1 -- 172.21.15.131:0/2825372996 shutdown_connections 2024-05-08T22:15:11.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.443+0000 7f379370db80 1 -- 172.21.15.131:0/2825372996 wait complete. 2024-05-08T22:15:11.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.445+0000 7f379370db80 20 client.16389 trim_cache size 0 max 16384 2024-05-08T22:15:11.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.471+0000 7f379370db80 1 Processor -- start 2024-05-08T22:15:11.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.471+0000 7f379370db80 1 -- start start 2024-05-08T22:15:11.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.472+0000 7f379370db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65dbb70f0 0x55e65dbaaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.472+0000 7f379370db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e65dbbc660 0x55e65dbcc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.472+0000 7f379370db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e65dbbcf80 0x55e65dbb74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.472+0000 7f379370db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65dba9cd0 con 0x55e65dbbc660 2024-05-08T22:15:11.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.472+0000 7f379370db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65dbc1900 con 0x55e65dbbcf80 2024-05-08T22:15:11.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.472+0000 7f379370db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65dbd7df0 con 0x55e65dbb70f0 2024-05-08T22:15:11.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.472+0000 7f377d200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65dbb70f0 0x55e65dbaaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.473+0000 7f3777e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e65dbbc660 0x55e65dbcc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.473+0000 7f3777e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e65dbbc660 0x55e65dbcc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56058/0 (socket says 172.21.15.131:56058) 2024-05-08T22:15:11.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.473+0000 7f3777e00700 1 -- 172.21.15.131:0/1876989122 learned_addr learned my addr 172.21.15.131:0/1876989122 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:11.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.473+0000 7f3777e00700 1 -- 172.21.15.131:0/1876989122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65dbb70f0 msgr2=0x55e65dbaaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:11.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.473+0000 7f3777e00700 1 --2- 172.21.15.131:0/1876989122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65dbb70f0 0x55e65dbaaa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.473+0000 7f3777e00700 1 -- 172.21.15.131:0/1876989122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e65dbbcf80 msgr2=0x55e65dbb74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:11.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.473+0000 7f3777e00700 1 --2- 172.21.15.131:0/1876989122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e65dbbcf80 0x55e65dbb74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.473+0000 7f3777e00700 1 -- 172.21.15.131:0/1876989122 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e65dac4560 con 0x55e65dbbc660 2024-05-08T22:15:11.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.474+0000 7f3777e00700 1 --2- 172.21.15.131:0/1876989122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e65dbbc660 0x55e65dbcc420 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x55e65dba10a0 tx=0x7f3764014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c505ffed85b5ddbd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:11.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.474+0000 7f376ea00700 1 -- 172.21.15.131:0/1876989122 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3764024040 con 0x55e65dbbc660 2024-05-08T22:15:11.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.474+0000 7f376ea00700 1 -- 172.21.15.131:0/1876989122 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f376400beb0 con 0x55e65dbbc660 2024-05-08T22:15:11.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.475+0000 7f379370db80 1 -- 172.21.15.131:0/1876989122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e65dbbc660 msgr2=0x55e65dbcc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:11.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.475+0000 7f376ea00700 1 -- 172.21.15.131:0/1876989122 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376401f400 con 0x55e65dbbc660 2024-05-08T22:15:11.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.475+0000 7f379370db80 1 --2- 172.21.15.131:0/1876989122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e65dbbc660 0x55e65dbcc420 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x55e65dba10a0 tx=0x7f3764014000 comp rx=0 tx=0).stop 2024-05-08T22:15:11.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.476+0000 7f379370db80 1 -- 172.21.15.131:0/1876989122 shutdown_connections 2024-05-08T22:15:11.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.476+0000 7f379370db80 1 --2- 172.21.15.131:0/1876989122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e65dbbcf80 0x55e65dbb74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.476+0000 7f379370db80 1 --2- 172.21.15.131:0/1876989122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65dbb70f0 0x55e65dbaaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.476+0000 7f379370db80 1 --2- 172.21.15.131:0/1876989122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e65dbbc660 0x55e65dbcc420 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.476+0000 7f379370db80 1 -- 172.21.15.131:0/1876989122 >> 172.21.15.131:0/1876989122 conn(0x55e65db0c210 msgr2=0x55e65dbbb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:11.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.476+0000 7f379370db80 1 -- 172.21.15.131:0/1876989122 shutdown_connections 2024-05-08T22:15:11.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.476+0000 7f379370db80 1 -- 172.21.15.131:0/1876989122 wait complete. 2024-05-08T22:15:11.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.477+0000 7f379370db80 1 -- start start 2024-05-08T22:15:11.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.477+0000 7f379370db80 1 Processor -- start 2024-05-08T22:15:11.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.479+0000 7f379370db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e65dbb70f0 0x55e65dbc6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.479+0000 7f379370db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e65dbbc660 0x55e65dbc71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.479+0000 7f379370db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65dbbcf80 0x55e65dbc7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.479+0000 7f379370db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65dc25d80 con 0x55e65dbb70f0 2024-05-08T22:15:11.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.479+0000 7f379370db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65dc25ef0 con 0x55e65dbbc660 2024-05-08T22:15:11.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.479+0000 7f379370db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65dc260b0 con 0x55e65dbbcf80 2024-05-08T22:15:11.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.479+0000 7f379370db80 10 client.0 _ll_register_callbacks cb 0x55e65daa3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:15:11.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.479+0000 7f377dc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65dbbcf80 0x55e65dbc7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.479+0000 7f377dc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65dbbcf80 0x55e65dbc7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59036/0 (socket says 172.21.15.131:59036) 2024-05-08T22:15:11.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.480+0000 7f377dc00700 1 -- 172.21.15.131:0/1662521478 learned_addr learned my addr 172.21.15.131:0/1662521478 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:11.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.480+0000 7f377d200700 1 --2- 172.21.15.131:0/1662521478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e65dbb70f0 0x55e65dbc6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.480+0000 7f377dc00700 1 -- 172.21.15.131:0/1662521478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e65dbbc660 msgr2=0x55e65dbc71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:11.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.480+0000 7f377dc00700 1 --2- 172.21.15.131:0/1662521478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e65dbbc660 0x55e65dbc71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.480+0000 7f377dc00700 1 -- 172.21.15.131:0/1662521478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e65dbb70f0 msgr2=0x55e65dbc6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:11.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.480+0000 7f377dc00700 1 --2- 172.21.15.131:0/1662521478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e65dbb70f0 0x55e65dbc6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.480+0000 7f377dc00700 1 -- 172.21.15.131:0/1662521478 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f376400dc20 con 0x55e65dbbcf80 2024-05-08T22:15:11.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.480+0000 7f377d200700 1 --2- 172.21.15.131:0/1662521478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e65dbb70f0 0x55e65dbc6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:15:11.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.481+0000 7f377dc00700 1 --2- 172.21.15.131:0/1662521478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65dbbcf80 0x55e65dbc7730 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3768004f60 tx=0x7f3768004f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ee8c6eb0d156a87 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:11.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.481+0000 7f3762a00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:15:11.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.482+0000 7f3762a00700 1 -- 172.21.15.131:0/1662521478 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376800aab0 con 0x55e65dbbcf80 2024-05-08T22:15:11.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.482+0000 7f3762a00700 1 -- 172.21.15.131:0/1662521478 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f3768005250 con 0x55e65dbbcf80 2024-05-08T22:15:11.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.482+0000 7f379370db80 1 -- 172.21.15.131:0/1662521478 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3768004380 con 0x55e65dbbcf80 2024-05-08T22:15:11.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.483+0000 7f3762a00700 1 -- 172.21.15.131:0/1662521478 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3768005410 con 0x55e65dbbcf80 2024-05-08T22:15:11.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.483+0000 7f379370db80 10 client.26479 fetch_fsmap learned FSMap version 28 2024-05-08T22:15:11.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.483+0000 7f379370db80 1 -- 172.21.15.131:0/1662521478 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e65d983cd0 con 0x55e65dbbcf80 2024-05-08T22:15:11.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.483+0000 7f3762a00700 1 -- 172.21.15.131:0/1662521478 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3768004380 con 0x55e65dbbcf80 2024-05-08T22:15:11.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.483+0000 7f3762a00700 1 -- 172.21.15.131:0/1662521478 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3768020070 con 0x55e65dbbcf80 2024-05-08T22:15:11.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.483+0000 7f379370db80 10 client.26479 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T22:15:11.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.483+0000 7f379370db80 10 client.26479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:15:11.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.483+0000 7f379370db80 20 client.26479 populate_metadata read hostname 'smithi131' 2024-05-08T22:15:11.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.483+0000 7f379370db80 1 --2- 172.21.15.131:0/1662521478 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55e65dc27830 0x55e65dc29c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:11.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.484+0000 7f379370db80 4 client.26479 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:15:11.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.484+0000 7f379370db80 1 -- 172.21.15.131:0/1662521478 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e65daf6ec0 con 0x55e65dc27830 2024-05-08T22:15:11.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.484+0000 7f377d200700 1 --2- 172.21.15.131:0/1662521478 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55e65dc27830 0x55e65dc29c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:11.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.486+0000 7f3762a00700 10 client.26479 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:11.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.486+0000 7f377d200700 1 --2- 172.21.15.131:0/1662521478 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55e65dc27830 0x55e65dc29c50 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea943e4caabeea5e server_cookie=9285774e26a3c862 in_seq=0 out_seq=0 2024-05-08T22:15:11.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.490+0000 7f3762a00700 1 -- 172.21.15.131:0/1662521478 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e65daf6ec0 con 0x55e65dc27830 2024-05-08T22:15:11.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.490+0000 7f3762a00700 10 client.26479 handle_command_reply: tid=0 2024-05-08T22:15:11.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.490+0000 7f379370db80 1 client.26479 shutdown 2024-05-08T22:15:11.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.491+0000 7f379370db80 1 -- 172.21.15.131:0/1662521478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65dbbcf80 msgr2=0x55e65dbc7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:11.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.491+0000 7f379370db80 1 --2- 172.21.15.131:0/1662521478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65dbbcf80 0x55e65dbc7730 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3768004f60 tx=0x7f3768004f90 comp rx=0 tx=0).stop 2024-05-08T22:15:11.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.491+0000 7f379370db80 1 -- 172.21.15.131:0/1662521478 shutdown_connections 2024-05-08T22:15:11.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.491+0000 7f379370db80 1 --2- 172.21.15.131:0/1662521478 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55e65dc27830 0x55e65dc29c50 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.491+0000 7f379370db80 1 --2- 172.21.15.131:0/1662521478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e65dbbcf80 0x55e65dbc7730 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.491+0000 7f379370db80 1 --2- 172.21.15.131:0/1662521478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e65dbb70f0 0x55e65dbc6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.491+0000 7f379370db80 1 --2- 172.21.15.131:0/1662521478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e65dbbc660 0x55e65dbc71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.491+0000 7f379370db80 1 -- 172.21.15.131:0/1662521478 >> 172.21.15.131:0/1662521478 conn(0x55e65db0c210 msgr2=0x55e65dbbed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:11.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.492+0000 7f3762a00700 0 client.26479 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:11.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.492+0000 7f379370db80 1 -- 172.21.15.131:0/1662521478 shutdown_connections 2024-05-08T22:15:11.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.492+0000 7f379370db80 1 -- 172.21.15.131:0/1662521478 wait complete. 2024-05-08T22:15:11.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.492+0000 7f379370db80 20 client.26479 trim_cache size 0 max 16384 2024-05-08T22:15:11.494 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:15:11.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.493+0000 7f377fc00700 1 -- 172.21.15.131:0/2493902013 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f375c0a1370 msgr2=0x7f375c0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:11.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.493+0000 7f377fc00700 1 --2- 172.21.15.131:0/2493902013 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f375c0a1370 0x7f375c0a3830 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f377819ab60 tx=0x7f376400b3f0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.493+0000 7f377fc00700 1 -- 172.21.15.131:0/2493902013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3778101180 msgr2=0x7f377819f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:11.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.493+0000 7f377fc00700 1 --2- 172.21.15.131:0/2493902013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3778101180 0x7f377819f350 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3770006070 tx=0x7f3770007b60 comp rx=0 tx=0).stop 2024-05-08T22:15:11.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.494+0000 7f377fc00700 1 -- 172.21.15.131:0/2493902013 shutdown_connections 2024-05-08T22:15:11.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.494+0000 7f377fc00700 1 --2- 172.21.15.131:0/2493902013 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f375c0a1370 0x7f375c0a3830 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.494+0000 7f377fc00700 1 --2- 172.21.15.131:0/2493902013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3778199450 0x7f3778199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.494+0000 7f377fc00700 1 --2- 172.21.15.131:0/2493902013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3778101180 0x7f377819f350 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.494+0000 7f377fc00700 1 --2- 172.21.15.131:0/2493902013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37781062a0 0x7f377819f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:11.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.494+0000 7f377fc00700 1 -- 172.21.15.131:0/2493902013 >> 172.21.15.131:0/2493902013 conn(0x7f37780fc9e0 msgr2=0x7f37780fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:11.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.494+0000 7f377fc00700 1 -- 172.21.15.131:0/2493902013 shutdown_connections 2024-05-08T22:15:11.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:11.494+0000 7f377fc00700 1 -- 172.21.15.131:0/2493902013 wait complete. 2024-05-08T22:15:12.529 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T22:15:12.529 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T22:15:12.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:12 smithi132 ceph-mon[78180]: pgmap v242: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 772 KiB/s wr, 677 op/s 2024-05-08T22:15:12.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.870+0000 7f25a1c00700 1 -- 172.21.15.131:0/3147351769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f259c073c60 msgr2=0x7f259c1119c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:12.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.871+0000 7f25a1c00700 1 --2- 172.21.15.131:0/3147351769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f259c073c60 0x7f259c1119c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f258c00b0a0 tx=0x7f258c00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:15:12.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.871+0000 7f25a1c00700 1 -- 172.21.15.131:0/3147351769 shutdown_connections 2024-05-08T22:15:12.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.871+0000 7f25a1c00700 1 --2- 172.21.15.131:0/3147351769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f259c111f00 0x7f259c1142f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:12.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.871+0000 7f25a1c00700 1 --2- 172.21.15.131:0/3147351769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f259c073340 0x7f259c073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:12.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.871+0000 7f25a1c00700 1 --2- 172.21.15.131:0/3147351769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f259c073c60 0x7f259c1119c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:12.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.871+0000 7f25a1c00700 1 -- 172.21.15.131:0/3147351769 >> 172.21.15.131:0/3147351769 conn(0x7f259c0fc9c0 msgr2=0x7f259c0fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:12.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.871+0000 7f25a1c00700 1 -- 172.21.15.131:0/3147351769 shutdown_connections 2024-05-08T22:15:12.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.871+0000 7f25a1c00700 1 -- 172.21.15.131:0/3147351769 wait complete. 2024-05-08T22:15:12.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.872+0000 7f25a1c00700 1 Processor -- start 2024-05-08T22:15:12.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.872+0000 7f25a1c00700 1 -- start start 2024-05-08T22:15:12.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.873+0000 7f25a1c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f259c073340 0x7f259c19dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:12.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.873+0000 7f25a1c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f259c073c60 0x7f259c19e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:12.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.873+0000 7f25a1c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f259c111f00 0x7f259c1a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:12.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.873+0000 7f25a1c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c1a2950 con 0x7f259c073340 2024-05-08T22:15:12.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.873+0000 7f25a1c00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c1a2ac0 con 0x7f259c073c60 2024-05-08T22:15:12.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.873+0000 7f25a1c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c1a2c80 con 0x7f259c111f00 2024-05-08T22:15:12.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.873+0000 7f259b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f259c111f00 0x7f259c1a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:12.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.873+0000 7f259b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f259c111f00 0x7f259c1a2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59046/0 (socket says 172.21.15.131:59046) 2024-05-08T22:15:12.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.873+0000 7f259b400700 1 -- 172.21.15.131:0/1451038500 learned_addr learned my addr 172.21.15.131:0/1451038500 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:12.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.874+0000 7f259b400700 1 -- 172.21.15.131:0/1451038500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f259c073c60 msgr2=0x7f259c19e000 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T22:15:12.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.874+0000 7f259b400700 1 --2- 172.21.15.131:0/1451038500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f259c073c60 0x7f259c19e000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:12.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.874+0000 7f259b400700 1 -- 172.21.15.131:0/1451038500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f259c073340 msgr2=0x7f259c19dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:12.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.874+0000 7f259b400700 1 --2- 172.21.15.131:0/1451038500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f259c073340 0x7f259c19dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:12.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.874+0000 7f259b400700 1 -- 172.21.15.131:0/1451038500 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258c009d00 con 0x7f259c111f00 2024-05-08T22:15:12.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.875+0000 7f259b400700 1 --2- 172.21.15.131:0/1451038500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f259c111f00 0x7f259c1a2380 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f259400b450 tx=0x7f259400b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:12.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.877+0000 7f2593e00700 1 -- 172.21.15.131:0/1451038500 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2594011830 con 0x7f259c111f00 2024-05-08T22:15:12.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.877+0000 7f2593e00700 1 -- 172.21.15.131:0/1451038500 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2594011e50 con 0x7f259c111f00 2024-05-08T22:15:12.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.877+0000 7f2593e00700 1 -- 172.21.15.131:0/1451038500 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259400f2e0 con 0x7f259c111f00 2024-05-08T22:15:12.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.878+0000 7f25a1c00700 1 -- 172.21.15.131:0/1451038500 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f259c1a2f60 con 0x7f259c111f00 2024-05-08T22:15:12.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.878+0000 7f25a1c00700 1 -- 172.21.15.131:0/1451038500 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259c1aa810 con 0x7f259c111f00 2024-05-08T22:15:12.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.883+0000 7f2593e00700 1 -- 172.21.15.131:0/1451038500 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f25940105f0 con 0x7f259c111f00 2024-05-08T22:15:12.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.885+0000 7f25b5733b80 1 Processor -- start 2024-05-08T22:15:12.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.885+0000 7f25b5733b80 1 -- start start 2024-05-08T22:15:12.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.885+0000 7f25b5733b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e4e705e80 0x560e4e706260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:12.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.885+0000 7f25b5733b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4e7067a0 0x560e4e7003d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:12.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.885+0000 7f25b5733b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560e4e700910 0x560e4e700d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:12.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.885+0000 7f25b5733b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4e6571d0 con 0x560e4e705e80 2024-05-08T22:15:12.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.885+0000 7f25b5733b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4e654b90 con 0x560e4e700910 2024-05-08T22:15:12.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.885+0000 7f25b5733b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4e650480 con 0x560e4e7067a0 2024-05-08T22:15:12.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.886+0000 7f259aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4e7067a0 0x560e4e7003d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:12.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.886+0000 7f2593e00700 1 --2- 172.21.15.131:0/1451038500 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f25800a1460 0x7f25800a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:12.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.886+0000 7f259aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4e7067a0 0x560e4e7003d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59054/0 (socket says 172.21.15.131:59054) 2024-05-08T22:15:12.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.886+0000 7f259aa00700 1 -- 172.21.15.131:0/2873914446 learned_addr learned my addr 172.21.15.131:0/2873914446 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:12.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.886+0000 7f2593e00700 1 -- 172.21.15.131:0/1451038500 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f25940cca10 con 0x7f259c111f00 2024-05-08T22:15:12.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.886+0000 7f2592000700 1 --2- 172.21.15.131:0/2873914446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560e4e700910 0x560e4e700d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:12.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.886+0000 7f2592000700 1 --2- 172.21.15.131:0/2873914446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e4e705e80 0x560e4e706260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:12.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.887+0000 7f259aa00700 1 -- 172.21.15.131:0/2873914446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560e4e700910 msgr2=0x560e4e700d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:12.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.887+0000 7f259aa00700 1 --2- 172.21.15.131:0/2873914446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560e4e700910 0x560e4e700d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:12.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.887+0000 7f259b400700 1 --2- 172.21.15.131:0/1451038500 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f25800a1460 0x7f25800a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:12.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.887+0000 7f259aa00700 1 -- 172.21.15.131:0/2873914446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e4e705e80 msgr2=0x560e4e706260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:12.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.887+0000 7f259aa00700 1 --2- 172.21.15.131:0/2873914446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e4e705e80 0x560e4e706260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:12.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.887+0000 7f259aa00700 1 -- 172.21.15.131:0/2873914446 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e4e4d1430 con 0x560e4e7067a0 2024-05-08T22:15:12.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.887+0000 7f259aa00700 1 --2- 172.21.15.131:0/2873914446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4e7067a0 0x560e4e7003d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f258400aa80 tx=0x7f258400ad90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a4c511aa792ac9f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:12.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.888+0000 7f258be00700 1 -- 172.21.15.131:0/2873914446 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258400cc10 con 0x560e4e7067a0 2024-05-08T22:15:12.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.888+0000 7f259b400700 1 --2- 172.21.15.131:0/1451038500 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f25800a1460 0x7f25800a3920 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f25940a1730 tx=0x7f259400be40 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:12.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.889+0000 7f258be00700 1 -- 172.21.15.131:0/2873914446 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f258400cd70 con 0x560e4e7067a0 2024-05-08T22:15:12.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.889+0000 7f258be00700 1 -- 172.21.15.131:0/2873914446 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2584018660 con 0x560e4e7067a0 2024-05-08T22:15:12.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.890+0000 7f25b5733b80 1 -- 172.21.15.131:0/2873914446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4e7067a0 msgr2=0x560e4e7003d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:12.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.890+0000 7f25b5733b80 1 --2- 172.21.15.131:0/2873914446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4e7067a0 0x560e4e7003d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f258400aa80 tx=0x7f258400ad90 comp rx=0 tx=0).stop 2024-05-08T22:15:12.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:12 smithi131 ceph-mon[37260]: pgmap v242: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 772 KiB/s wr, 677 op/s 2024-05-08T22:15:12.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.890+0000 7f25b5733b80 1 -- 172.21.15.131:0/2873914446 shutdown_connections 2024-05-08T22:15:12.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.890+0000 7f25b5733b80 1 --2- 172.21.15.131:0/2873914446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560e4e700910 0x560e4e700d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:12.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.890+0000 7f25b5733b80 1 --2- 172.21.15.131:0/2873914446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e4e705e80 0x560e4e706260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:12.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.890+0000 7f25b5733b80 1 --2- 172.21.15.131:0/2873914446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4e7067a0 0x560e4e7003d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:12.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.890+0000 7f25b5733b80 1 -- 172.21.15.131:0/2873914446 >> 172.21.15.131:0/2873914446 conn(0x560e4e65e290 msgr2=0x560e4e6fae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:12.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.892+0000 7f25b5733b80 1 -- 172.21.15.131:0/2873914446 shutdown_connections 2024-05-08T22:15:12.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.892+0000 7f25b5733b80 1 -- 172.21.15.131:0/2873914446 wait complete. 2024-05-08T22:15:12.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.893+0000 7f25b5733b80 1 -- start start 2024-05-08T22:15:12.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.893+0000 7f25b5733b80 1 Processor -- start 2024-05-08T22:15:12.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.898+0000 7f25b5733b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e4e700910 0x560e4e709ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:12.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.899+0000 7f25b5733b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560e4e705e80 0x560e4e70a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:12.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.899+0000 7f25b5733b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4e7067a0 0x560e4e70a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:12.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.899+0000 7f25b5733b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4e5ec5e0 con 0x560e4e700910 2024-05-08T22:15:12.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.899+0000 7f25b5733b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4e5ec280 con 0x560e4e705e80 2024-05-08T22:15:12.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.899+0000 7f25b5733b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4e4d3720 con 0x560e4e7067a0 2024-05-08T22:15:12.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.899+0000 7f25b5733b80 10 client.0 _ll_register_callbacks cb 0x560e4e5f5bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:15:12.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.899+0000 7f2592000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4e7067a0 0x560e4e70a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:12.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.899+0000 7f259aa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560e4e705e80 0x560e4e70a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:12.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.899+0000 7f259aa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560e4e705e80 0x560e4e70a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:41638/0 (socket says 172.21.15.131:41638) 2024-05-08T22:15:12.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.899+0000 7f259aa00700 1 -- 172.21.15.131:0/2904076306 learned_addr learned my addr 172.21.15.131:0/2904076306 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:12.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.900+0000 7f2592000700 1 --2- 172.21.15.131:0/2904076306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e4e700910 0x560e4e709ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:12.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.901+0000 7f2592000700 1 -- 172.21.15.131:0/2904076306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4e7067a0 msgr2=0x560e4e70a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:12.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.901+0000 7f2592000700 1 --2- 172.21.15.131:0/2904076306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4e7067a0 0x560e4e70a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:12.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.901+0000 7f2592000700 1 -- 172.21.15.131:0/2904076306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560e4e705e80 msgr2=0x560e4e70a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:12.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.901+0000 7f2592000700 1 --2- 172.21.15.131:0/2904076306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560e4e705e80 0x560e4e70a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:12.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.901+0000 7f2592000700 1 -- 172.21.15.131:0/2904076306 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258400a6b0 con 0x560e4e700910 2024-05-08T22:15:12.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.901+0000 7f2592000700 1 --2- 172.21.15.131:0/2904076306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e4e700910 0x560e4e709ba0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f259c19efc0 tx=0x7f258c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7c00f66e58c97a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:12.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.901+0000 7f2588c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:15:12.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.902+0000 7f2588c00700 1 -- 172.21.15.131:0/2904076306 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c00e040 con 0x560e4e700910 2024-05-08T22:15:12.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.902+0000 7f2588c00700 1 -- 172.21.15.131:0/2904076306 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f258c005040 con 0x560e4e700910 2024-05-08T22:15:12.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.902+0000 7f25b5733b80 1 -- 172.21.15.131:0/2904076306 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e4e5a92f0 con 0x560e4e700910 2024-05-08T22:15:12.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.903+0000 7f2588c00700 1 -- 172.21.15.131:0/2904076306 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f258c007ae0 con 0x560e4e700910 2024-05-08T22:15:12.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.904+0000 7f25b5733b80 10 client.16401 fetch_fsmap learned FSMap version 28 2024-05-08T22:15:12.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.904+0000 7f25b5733b80 1 -- 172.21.15.131:0/2904076306 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560e4e47dbe0 con 0x560e4e700910 2024-05-08T22:15:12.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.904+0000 7f2588c00700 1 -- 172.21.15.131:0/2904076306 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560e4e5a92f0 con 0x560e4e700910 2024-05-08T22:15:12.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.904+0000 7f2588c00700 1 -- 172.21.15.131:0/2904076306 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f258c019070 con 0x560e4e700910 2024-05-08T22:15:12.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.904+0000 7f25b5733b80 10 client.16401 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T22:15:12.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.904+0000 7f25b5733b80 10 client.16401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:15:12.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.904+0000 7f25b5733b80 20 client.16401 populate_metadata read hostname 'smithi131' 2024-05-08T22:15:12.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.904+0000 7f25b5733b80 1 --2- 172.21.15.131:0/2904076306 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560e4e71a2d0 0x560e4e71c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:12.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.904+0000 7f25b5733b80 4 client.16401 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:15:12.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.904+0000 7f25b5733b80 1 -- 172.21.15.131:0/2904076306 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e4e6cf890 con 0x560e4e71a2d0 2024-05-08T22:15:12.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.905+0000 7f259aa00700 1 --2- 172.21.15.131:0/2904076306 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560e4e71a2d0 0x560e4e71c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:12.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.908+0000 7f2588c00700 10 client.16401 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:12.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:12.908+0000 7f259aa00700 1 --2- 172.21.15.131:0/2904076306 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560e4e71a2d0 0x560e4e71c6f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7334ddc95af0fef server_cookie=6945c03c9f8a3b37 in_seq=0 out_seq=0 2024-05-08T22:15:13.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.007+0000 7f2588c00700 1 -- 172.21.15.131:0/2904076306 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560e4e6cf890 con 0x560e4e71a2d0 2024-05-08T22:15:13.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.007+0000 7f2588c00700 10 client.16401 handle_command_reply: tid=0 2024-05-08T22:15:13.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.007+0000 7f25b5733b80 1 client.16401 shutdown 2024-05-08T22:15:13.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.009+0000 7f25b5733b80 1 -- 172.21.15.131:0/2904076306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e4e700910 msgr2=0x560e4e709ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:13.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.009+0000 7f25b5733b80 1 --2- 172.21.15.131:0/2904076306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e4e700910 0x560e4e709ba0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f259c19efc0 tx=0x7f258c000f40 comp rx=0 tx=0).stop 2024-05-08T22:15:13.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.009+0000 7f25b5733b80 1 -- 172.21.15.131:0/2904076306 shutdown_connections 2024-05-08T22:15:13.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.009+0000 7f25b5733b80 1 --2- 172.21.15.131:0/2904076306 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560e4e71a2d0 0x560e4e71c6f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.009+0000 7f25b5733b80 1 --2- 172.21.15.131:0/2904076306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4e7067a0 0x560e4e70a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.009+0000 7f25b5733b80 1 --2- 172.21.15.131:0/2904076306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e4e700910 0x560e4e709ba0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.009+0000 7f25b5733b80 1 --2- 172.21.15.131:0/2904076306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560e4e705e80 0x560e4e70a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.009+0000 7f2588c00700 0 client.16401 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:13.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.009+0000 7f25b5733b80 1 -- 172.21.15.131:0/2904076306 >> 172.21.15.131:0/2904076306 conn(0x560e4e65e290 msgr2=0x560e4e6fa620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:13.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.011+0000 7f25b5733b80 1 -- 172.21.15.131:0/2904076306 shutdown_connections 2024-05-08T22:15:13.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.011+0000 7f25b5733b80 1 -- 172.21.15.131:0/2904076306 wait complete. 2024-05-08T22:15:13.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.011+0000 7f25b5733b80 20 client.16401 trim_cache size 0 max 16384 2024-05-08T22:15:13.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:12 smithi157 ceph-mon[77746]: pgmap v242: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 772 KiB/s wr, 677 op/s 2024-05-08T22:15:13.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.045+0000 7f25b5733b80 1 Processor -- start 2024-05-08T22:15:13.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.045+0000 7f25b5733b80 1 -- start start 2024-05-08T22:15:13.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.046+0000 7f25b5733b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4e700910 0x560e4e7053a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:13.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.046+0000 7f25b5733b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e4e705e80 0x560e4e6fada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:13.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.046+0000 7f25b5733b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560e4e7067a0 0x560e4e70a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:13.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.046+0000 7f2592000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4e700910 0x560e4e7053a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:13.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.047+0000 7f25b5733b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4e70b1a0 con 0x560e4e705e80 2024-05-08T22:15:13.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.047+0000 7f25b5733b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4e6f7e40 con 0x560e4e7067a0 2024-05-08T22:15:13.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.047+0000 7f25b5733b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4e7058e0 con 0x560e4e700910 2024-05-08T22:15:13.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.047+0000 7f259aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e4e705e80 0x560e4e6fada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:13.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.047+0000 7f259aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e4e705e80 0x560e4e6fada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56104/0 (socket says 172.21.15.131:56104) 2024-05-08T22:15:13.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.047+0000 7f259aa00700 1 -- 172.21.15.131:0/3451598800 learned_addr learned my addr 172.21.15.131:0/3451598800 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:13.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.048+0000 7f259aa00700 1 -- 172.21.15.131:0/3451598800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4e700910 msgr2=0x560e4e7053a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:13.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.048+0000 7f259aa00700 1 --2- 172.21.15.131:0/3451598800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4e700910 0x560e4e7053a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.048+0000 7f259aa00700 1 -- 172.21.15.131:0/3451598800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560e4e7067a0 msgr2=0x560e4e70a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:13.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.048+0000 7f259aa00700 1 --2- 172.21.15.131:0/3451598800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560e4e7067a0 0x560e4e70a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.048+0000 7f259aa00700 1 -- 172.21.15.131:0/3451598800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e4e700fc0 con 0x560e4e705e80 2024-05-08T22:15:13.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.048+0000 7f259aa00700 1 --2- 172.21.15.131:0/3451598800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e4e705e80 0x560e4e6fada0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f258400c5f0 tx=0x7f258400af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f8506608ed33d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:13.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.049+0000 7f2590c00700 1 -- 172.21.15.131:0/3451598800 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258401c040 con 0x560e4e705e80 2024-05-08T22:15:13.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.049+0000 7f2590c00700 1 -- 172.21.15.131:0/3451598800 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f258402c070 con 0x560e4e705e80 2024-05-08T22:15:13.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.049+0000 7f2590c00700 1 -- 172.21.15.131:0/3451598800 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25840283f0 con 0x560e4e705e80 2024-05-08T22:15:13.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.050+0000 7f25b5733b80 1 -- 172.21.15.131:0/3451598800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e4e705e80 msgr2=0x560e4e6fada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:13.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.050+0000 7f25b5733b80 1 --2- 172.21.15.131:0/3451598800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e4e705e80 0x560e4e6fada0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f258400c5f0 tx=0x7f258400af40 comp rx=0 tx=0).stop 2024-05-08T22:15:13.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.050+0000 7f25b5733b80 1 -- 172.21.15.131:0/3451598800 shutdown_connections 2024-05-08T22:15:13.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.050+0000 7f25b5733b80 1 --2- 172.21.15.131:0/3451598800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560e4e7067a0 0x560e4e70a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.051+0000 7f25b5733b80 1 --2- 172.21.15.131:0/3451598800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4e700910 0x560e4e7053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.051+0000 7f25b5733b80 1 --2- 172.21.15.131:0/3451598800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e4e705e80 0x560e4e6fada0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.051+0000 7f25b5733b80 1 -- 172.21.15.131:0/3451598800 >> 172.21.15.131:0/3451598800 conn(0x560e4e65e290 msgr2=0x560e4e704c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:13.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.051+0000 7f25b5733b80 1 -- 172.21.15.131:0/3451598800 shutdown_connections 2024-05-08T22:15:13.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.051+0000 7f25b5733b80 1 -- 172.21.15.131:0/3451598800 wait complete. 2024-05-08T22:15:13.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.052+0000 7f25b5733b80 1 -- start start 2024-05-08T22:15:13.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.052+0000 7f25b5733b80 1 Processor -- start 2024-05-08T22:15:13.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.053+0000 7f25b5733b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4e700910 0x560e4e70fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:13.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.053+0000 7f25b5733b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560e4e705e80 0x560e4e710380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:13.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.053+0000 7f25b5733b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e4e7067a0 0x560e4e7108c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:13.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.053+0000 7f25b5733b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4e7707e0 con 0x560e4e7067a0 2024-05-08T22:15:13.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.053+0000 7f25b5733b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4e770920 con 0x560e4e705e80 2024-05-08T22:15:13.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.053+0000 7f25b5733b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4e770ae0 con 0x560e4e700910 2024-05-08T22:15:13.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.054+0000 7f25b5733b80 10 client.0 _ll_register_callbacks cb 0x560e4e5f5bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:15:13.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.054+0000 7f2592000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4e700910 0x560e4e70fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:13.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.054+0000 7f2592000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4e700910 0x560e4e70fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59078/0 (socket says 172.21.15.131:59078) 2024-05-08T22:15:13.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.054+0000 7f2592000700 1 -- 172.21.15.131:0/2876702022 learned_addr learned my addr 172.21.15.131:0/2876702022 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:13.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.054+0000 7f2592000700 1 --2- 172.21.15.131:0/2876702022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e4e7067a0 0x560e4e7108c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:13.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.056+0000 7f259aa00700 1 --2- 172.21.15.131:0/2876702022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560e4e705e80 0x560e4e710380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:13.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.056+0000 7f2592000700 1 -- 172.21.15.131:0/2876702022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4e700910 msgr2=0x560e4e70fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:13.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.056+0000 7f2592000700 1 --2- 172.21.15.131:0/2876702022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4e700910 0x560e4e70fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.056+0000 7f2592000700 1 -- 172.21.15.131:0/2876702022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560e4e705e80 msgr2=0x560e4e710380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:13.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.056+0000 7f2592000700 1 --2- 172.21.15.131:0/2876702022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560e4e705e80 0x560e4e710380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.057+0000 7f2592000700 1 -- 172.21.15.131:0/2876702022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e4e4d1430 con 0x560e4e7067a0 2024-05-08T22:15:13.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.057+0000 7f259aa00700 1 --2- 172.21.15.131:0/2876702022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560e4e705e80 0x560e4e710380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:15:13.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.058+0000 7f2592000700 1 --2- 172.21.15.131:0/2876702022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e4e7067a0 0x560e4e7108c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f258c004ec0 tx=0x7f258c009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ef226495bcba12 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:13.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.059+0000 7f2588c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:15:13.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.059+0000 7f2588c00700 1 -- 172.21.15.131:0/2876702022 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c00e040 con 0x560e4e7067a0 2024-05-08T22:15:13.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.059+0000 7f2588c00700 1 -- 172.21.15.131:0/2876702022 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f258c01fc60 con 0x560e4e7067a0 2024-05-08T22:15:13.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.059+0000 7f25b5733b80 1 -- 172.21.15.131:0/2876702022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f258400cd70 con 0x560e4e7067a0 2024-05-08T22:15:13.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.060+0000 7f2588c00700 1 -- 172.21.15.131:0/2876702022 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f258c00b830 con 0x560e4e7067a0 2024-05-08T22:15:13.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.060+0000 7f25b5733b80 10 client.16407 fetch_fsmap learned FSMap version 28 2024-05-08T22:15:13.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.060+0000 7f25b5733b80 1 -- 172.21.15.131:0/2876702022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560e4e6165e0 con 0x560e4e7067a0 2024-05-08T22:15:13.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.061+0000 7f2588c00700 1 -- 172.21.15.131:0/2876702022 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f258400cd70 con 0x560e4e7067a0 2024-05-08T22:15:13.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.061+0000 7f2588c00700 1 -- 172.21.15.131:0/2876702022 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f258c019070 con 0x560e4e7067a0 2024-05-08T22:15:13.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.061+0000 7f25b5733b80 10 client.16407 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T22:15:13.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.061+0000 7f25b5733b80 10 client.16407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:15:13.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.061+0000 7f25b5733b80 20 client.16407 populate_metadata read hostname 'smithi131' 2024-05-08T22:15:13.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.061+0000 7f25b5733b80 1 --2- 172.21.15.131:0/2876702022 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560e4e772000 0x560e4e774420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:13.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.061+0000 7f25b5733b80 4 client.16407 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T22:15:13.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.061+0000 7f25b5733b80 1 -- 172.21.15.131:0/2876702022 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560e4e648f40 con 0x560e4e772000 2024-05-08T22:15:13.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.069+0000 7f259aa00700 1 --2- 172.21.15.131:0/2876702022 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560e4e772000 0x560e4e774420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:13.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.075+0000 7f2588c00700 10 client.16407 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:13.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.075+0000 7f259aa00700 1 --2- 172.21.15.131:0/2876702022 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560e4e772000 0x560e4e774420 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b75ae678843add3 server_cookie=b5dea97283155a57 in_seq=0 out_seq=0 2024-05-08T22:15:13.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.079+0000 7f2588c00700 1 -- 172.21.15.131:0/2876702022 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560e4e648f40 con 0x560e4e772000 2024-05-08T22:15:13.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.079+0000 7f2588c00700 10 client.16407 handle_command_reply: tid=0 2024-05-08T22:15:13.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.079+0000 7f25b5733b80 1 client.16407 shutdown 2024-05-08T22:15:13.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.080+0000 7f25b5733b80 1 -- 172.21.15.131:0/2876702022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e4e7067a0 msgr2=0x560e4e7108c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:13.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.080+0000 7f25b5733b80 1 --2- 172.21.15.131:0/2876702022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e4e7067a0 0x560e4e7108c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f258c004ec0 tx=0x7f258c009e90 comp rx=0 tx=0).stop 2024-05-08T22:15:13.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.080+0000 7f25b5733b80 1 -- 172.21.15.131:0/2876702022 shutdown_connections 2024-05-08T22:15:13.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.080+0000 7f25b5733b80 1 --2- 172.21.15.131:0/2876702022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e4e7067a0 0x560e4e7108c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.080+0000 7f25b5733b80 1 --2- 172.21.15.131:0/2876702022 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560e4e772000 0x560e4e774420 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.080+0000 7f25b5733b80 1 --2- 172.21.15.131:0/2876702022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4e700910 0x560e4e70fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.080+0000 7f25b5733b80 1 --2- 172.21.15.131:0/2876702022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560e4e705e80 0x560e4e710380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.080+0000 7f25b5733b80 1 -- 172.21.15.131:0/2876702022 >> 172.21.15.131:0/2876702022 conn(0x560e4e65e290 msgr2=0x560e4e706b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:13.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.080+0000 7f2588c00700 0 client.16407 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:13.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.080+0000 7f25b5733b80 1 -- 172.21.15.131:0/2876702022 shutdown_connections 2024-05-08T22:15:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.080+0000 7f25b5733b80 1 -- 172.21.15.131:0/2876702022 wait complete. 2024-05-08T22:15:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.081+0000 7f25b5733b80 20 client.16407 trim_cache size 0 max 16384 2024-05-08T22:15:13.083 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:15:13.083 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-08T22:15:13.083 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "4d699c6d-e487-4b00-9d60-e0aadfef7157", 2024-05-08T22:15:13.083 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-08T22:15:13.083 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:15:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.082+0000 7f25a1c00700 1 -- 172.21.15.131:0/1451038500 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f25800a1460 msgr2=0x7f25800a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.082+0000 7f25a1c00700 1 --2- 172.21.15.131:0/1451038500 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f25800a1460 0x7f25800a3920 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f25940a1730 tx=0x7f259400be40 comp rx=0 tx=0).stop 2024-05-08T22:15:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.082+0000 7f25a1c00700 1 -- 172.21.15.131:0/1451038500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f259c111f00 msgr2=0x7f259c1a2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.082+0000 7f25a1c00700 1 --2- 172.21.15.131:0/1451038500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f259c111f00 0x7f259c1a2380 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f259400b450 tx=0x7f259400b810 comp rx=0 tx=0).stop 2024-05-08T22:15:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.082+0000 7f25a1c00700 1 -- 172.21.15.131:0/1451038500 shutdown_connections 2024-05-08T22:15:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.082+0000 7f25a1c00700 1 --2- 172.21.15.131:0/1451038500 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f25800a1460 0x7f25800a3920 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.082+0000 7f25a1c00700 1 --2- 172.21.15.131:0/1451038500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f259c111f00 0x7f259c1a2380 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.082+0000 7f25a1c00700 1 --2- 172.21.15.131:0/1451038500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f259c073340 0x7f259c19dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.082+0000 7f25a1c00700 1 --2- 172.21.15.131:0/1451038500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f259c073c60 0x7f259c19e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.082+0000 7f25a1c00700 1 -- 172.21.15.131:0/1451038500 >> 172.21.15.131:0/1451038500 conn(0x7f259c0fc9c0 msgr2=0x7f259c0fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:13.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.082+0000 7f25a1c00700 1 -- 172.21.15.131:0/1451038500 shutdown_connections 2024-05-08T22:15:13.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.083+0000 7f25a1c00700 1 -- 172.21.15.131:0/1451038500 wait complete. 2024-05-08T22:15:13.121 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:15:13.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.510+0000 7f5d89800700 1 -- 172.21.15.131:0/2358399206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d84068c90 msgr2=0x7f5d84105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:13.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.510+0000 7f5d89800700 1 --2- 172.21.15.131:0/2358399206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d84068c90 0x7f5d84105b60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5d700099c0 tx=0x7f5d70009cd0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.510+0000 7f5d89800700 1 -- 172.21.15.131:0/2358399206 shutdown_connections 2024-05-08T22:15:13.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.510+0000 7f5d89800700 1 --2- 172.21.15.131:0/2358399206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d84106290 0x7f5d8410de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.510+0000 7f5d89800700 1 --2- 172.21.15.131:0/2358399206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d840682e0 0x7f5d840686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.510+0000 7f5d89800700 1 --2- 172.21.15.131:0/2358399206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d84068c90 0x7f5d84105b60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.510+0000 7f5d89800700 1 -- 172.21.15.131:0/2358399206 >> 172.21.15.131:0/2358399206 conn(0x7f5d84076f40 msgr2=0x7f5d84077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:13.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.511+0000 7f5d89800700 1 -- 172.21.15.131:0/2358399206 shutdown_connections 2024-05-08T22:15:13.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.513+0000 7f5d89800700 1 -- 172.21.15.131:0/2358399206 wait complete. 2024-05-08T22:15:13.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.515+0000 7f5d89800700 1 Processor -- start 2024-05-08T22:15:13.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.515+0000 7f5d89800700 1 -- start start 2024-05-08T22:15:13.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.516+0000 7f5d89800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d840682e0 0x7f5d841994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:13.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.516+0000 7f5d89800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d84106290 0x7f5d84199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:13.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.516+0000 7f5d89800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8419dd50 0x7f5d8419e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:13.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.516+0000 7f5d89800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8419e7d0 con 0x7f5d84106290 2024-05-08T22:15:13.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.516+0000 7f5d89800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8419e940 con 0x7f5d840682e0 2024-05-08T22:15:13.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.516+0000 7f5d89800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8419eb00 con 0x7f5d8419dd50 2024-05-08T22:15:13.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.516+0000 7f5d82000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d84106290 0x7f5d84199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:13.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.516+0000 7f5d82000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d84106290 0x7f5d84199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56122/0 (socket says 172.21.15.131:56122) 2024-05-08T22:15:13.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.516+0000 7f5d82000700 1 -- 172.21.15.131:0/188852911 learned_addr learned my addr 172.21.15.131:0/188852911 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:13.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.517+0000 7f5d83400700 1 --2- 172.21.15.131:0/188852911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8419dd50 0x7f5d8419e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:13.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.517+0000 7f5d82000700 1 -- 172.21.15.131:0/188852911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8419dd50 msgr2=0x7f5d8419e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:13.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.517+0000 7f5d82000700 1 --2- 172.21.15.131:0/188852911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8419dd50 0x7f5d8419e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.517+0000 7f5d82000700 1 -- 172.21.15.131:0/188852911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d840682e0 msgr2=0x7f5d841994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:13.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.517+0000 7f5d82000700 1 --2- 172.21.15.131:0/188852911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d840682e0 0x7f5d841994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.517+0000 7f5d82000700 1 -- 172.21.15.131:0/188852911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d700095f0 con 0x7f5d84106290 2024-05-08T22:15:13.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.518+0000 7f5d82000700 1 --2- 172.21.15.131:0/188852911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d84106290 0x7f5d84199a00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5d70006010 tx=0x7f5d7000bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:13.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.518+0000 7f5d7b400700 1 -- 172.21.15.131:0/188852911 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7001d070 con 0x7f5d84106290 2024-05-08T22:15:13.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.518+0000 7f5d89800700 1 -- 172.21.15.131:0/188852911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d841a6000 con 0x7f5d84106290 2024-05-08T22:15:13.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.518+0000 7f5d7b400700 1 -- 172.21.15.131:0/188852911 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d70003df0 con 0x7f5d84106290 2024-05-08T22:15:13.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.519+0000 7f5d89800700 1 -- 172.21.15.131:0/188852911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d841a6420 con 0x7f5d84106290 2024-05-08T22:15:13.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.519+0000 7f5d7b400700 1 -- 172.21.15.131:0/188852911 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7001d070 con 0x7f5d84106290 2024-05-08T22:15:13.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.524+0000 7f5d7b400700 1 -- 172.21.15.131:0/188852911 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f5d7000f460 con 0x7f5d84106290 2024-05-08T22:15:13.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.527+0000 7f5d7b400700 1 --2- 172.21.15.131:0/188852911 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f5d680a12f0 0x7f5d680a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:13.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.527+0000 7f5d7b400700 1 -- 172.21.15.131:0/188852911 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f5d700cdd20 con 0x7f5d84106290 2024-05-08T22:15:13.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.527+0000 7f5d82a00700 1 --2- 172.21.15.131:0/188852911 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f5d680a12f0 0x7f5d680a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:13.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.528+0000 7f5d9d3c4b80 1 Processor -- start 2024-05-08T22:15:13.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.529+0000 7f5d82a00700 1 --2- 172.21.15.131:0/188852911 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f5d680a12f0 0x7f5d680a37b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5d74005950 tx=0x7f5d740058e0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:13.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.529+0000 7f5d9d3c4b80 1 -- start start 2024-05-08T22:15:13.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.529+0000 7f5d9d3c4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642b6355660 0x5642b6355a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:13.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.529+0000 7f5d9d3c4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b6355f80 0x5642b634fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:13.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.530+0000 7f5d82a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642b6355660 0x5642b6355a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:13.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.530+0000 7f5d82a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642b6355660 0x5642b6355a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:41688/0 (socket says 172.21.15.131:41688) 2024-05-08T22:15:13.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.530+0000 7f5d82a00700 1 -- 172.21.15.131:0/3045284702 learned_addr learned my addr 172.21.15.131:0/3045284702 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:13.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.530+0000 7f5d82000700 1 --2- 172.21.15.131:0/3045284702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b6355f80 0x5642b634fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:13.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.530+0000 7f5d9d3c4b80 1 --2- 172.21.15.131:0/3045284702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b63500f0 0x5642b6350570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:13.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.530+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/3045284702 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b629e150 con 0x5642b63500f0 2024-05-08T22:15:13.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.531+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/3045284702 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b629bb10 con 0x5642b6355660 2024-05-08T22:15:13.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.531+0000 7f5d83400700 1 --2- 172.21.15.131:0/3045284702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b63500f0 0x5642b6350570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:13.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.531+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/3045284702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b6297400 con 0x5642b6355f80 2024-05-08T22:15:13.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.531+0000 7f5d82000700 1 -- 172.21.15.131:0/3045284702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642b6355660 msgr2=0x5642b6355a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:13.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.531+0000 7f5d82000700 1 --2- 172.21.15.131:0/3045284702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642b6355660 0x5642b6355a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.531+0000 7f5d82000700 1 -- 172.21.15.131:0/3045284702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b63500f0 msgr2=0x5642b6350570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:13.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.532+0000 7f5d82000700 1 --2- 172.21.15.131:0/3045284702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b63500f0 0x5642b6350570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.533+0000 7f5d82000700 1 -- 172.21.15.131:0/3045284702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642b61183b0 con 0x5642b6355f80 2024-05-08T22:15:13.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.533+0000 7f5d83400700 1 --2- 172.21.15.131:0/3045284702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b63500f0 0x5642b6350570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:15:13.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.533+0000 7f5d82000700 1 --2- 172.21.15.131:0/3045284702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b6355f80 0x5642b634fb60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5d700cb080 tx=0x7f5d70009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98a50db495bbbbcb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:13.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.533+0000 7f5d6fe00700 1 -- 172.21.15.131:0/3045284702 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7009aad0 con 0x5642b6355f80 2024-05-08T22:15:13.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.534+0000 7f5d6fe00700 1 -- 172.21.15.131:0/3045284702 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d7009e900 con 0x5642b6355f80 2024-05-08T22:15:13.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.534+0000 7f5d6fe00700 1 -- 172.21.15.131:0/3045284702 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7009a290 con 0x5642b6355f80 2024-05-08T22:15:13.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.535+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/3045284702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b6355f80 msgr2=0x5642b634fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:13.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.535+0000 7f5d9d3c4b80 1 --2- 172.21.15.131:0/3045284702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b6355f80 0x5642b634fb60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5d700cb080 tx=0x7f5d70009310 comp rx=0 tx=0).stop 2024-05-08T22:15:13.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.535+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/3045284702 shutdown_connections 2024-05-08T22:15:13.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.536+0000 7f5d9d3c4b80 1 --2- 172.21.15.131:0/3045284702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b63500f0 0x5642b6350570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.536+0000 7f5d9d3c4b80 1 --2- 172.21.15.131:0/3045284702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642b6355660 0x5642b6355a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.536+0000 7f5d9d3c4b80 1 --2- 172.21.15.131:0/3045284702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b6355f80 0x5642b634fb60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.536+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/3045284702 >> 172.21.15.131:0/3045284702 conn(0x5642b62a5210 msgr2=0x5642b6341df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:13.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.536+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/3045284702 shutdown_connections 2024-05-08T22:15:13.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.536+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/3045284702 wait complete. 2024-05-08T22:15:13.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.538+0000 7f5d9d3c4b80 1 -- start start 2024-05-08T22:15:13.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.539+0000 7f5d9d3c4b80 1 Processor -- start 2024-05-08T22:15:13.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.540+0000 7f5d9d3c4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642b63500f0 0x5642b635bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:13.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.540+0000 7f5d9d3c4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b6355660 0x5642b6359270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:13.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.540+0000 7f5d9d3c4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b6355f80 0x5642b63599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:13.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.540+0000 7f5d9d3c4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b6233560 con 0x5642b6355f80 2024-05-08T22:15:13.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.540+0000 7f5d9d3c4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b6233200 con 0x5642b63500f0 2024-05-08T22:15:13.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.540+0000 7f5d9d3c4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b611a6a0 con 0x5642b6355660 2024-05-08T22:15:13.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.540+0000 7f5d9d3c4b80 10 client.0 _ll_register_callbacks cb 0x5642b623cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:15:13.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.540+0000 7f5d82000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b6355660 0x5642b6359270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:13.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.540+0000 7f5d82a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642b63500f0 0x5642b635bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:13.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.540+0000 7f5d82000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b6355660 0x5642b6359270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59102/0 (socket says 172.21.15.131:59102) 2024-05-08T22:15:13.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.540+0000 7f5d82000700 1 -- 172.21.15.131:0/254814413 learned_addr learned my addr 172.21.15.131:0/254814413 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:13.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.541+0000 7f5d83400700 1 --2- 172.21.15.131:0/254814413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b6355f80 0x5642b63599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:13.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.541+0000 7f5d82000700 1 -- 172.21.15.131:0/254814413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642b63500f0 msgr2=0x5642b635bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:13.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.541+0000 7f5d82000700 1 --2- 172.21.15.131:0/254814413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642b63500f0 0x5642b635bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.541+0000 7f5d82000700 1 -- 172.21.15.131:0/254814413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b6355f80 msgr2=0x5642b63599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:13.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.541+0000 7f5d82000700 1 --2- 172.21.15.131:0/254814413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b6355f80 0x5642b63599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.541+0000 7f5d82000700 1 -- 172.21.15.131:0/254814413 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d7c009b10 con 0x5642b6355660 2024-05-08T22:15:13.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.541+0000 7f5d82000700 1 --2- 172.21.15.131:0/254814413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b6355660 0x5642b6359270 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5d700cb130 tx=0x7f5d7009b0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=625daf61dfd543d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:13.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.542+0000 7f5d6cc00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:15:13.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.542+0000 7f5d6cc00700 1 -- 172.21.15.131:0/254814413 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7009aad0 con 0x5642b6355660 2024-05-08T22:15:13.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.542+0000 7f5d6cc00700 1 -- 172.21.15.131:0/254814413 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f5d700a2ec0 con 0x5642b6355660 2024-05-08T22:15:13.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.542+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/254814413 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5642b61f0270 con 0x5642b6355660 2024-05-08T22:15:13.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.543+0000 7f5d6cc00700 1 -- 172.21.15.131:0/254814413 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d7009aad0 con 0x5642b6355660 2024-05-08T22:15:13.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.543+0000 7f5d9d3c4b80 10 client.26506 fetch_fsmap learned FSMap version 28 2024-05-08T22:15:13.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.543+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/254814413 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f5d841a6420 con 0x5642b6355660 2024-05-08T22:15:13.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.543+0000 7f5d6cc00700 1 -- 172.21.15.131:0/254814413 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d700a2c40 con 0x5642b6355660 2024-05-08T22:15:13.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.543+0000 7f5d6cc00700 1 -- 172.21.15.131:0/254814413 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5d7009bd20 con 0x5642b6355660 2024-05-08T22:15:13.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.544+0000 7f5d9d3c4b80 10 client.26506 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T22:15:13.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.544+0000 7f5d9d3c4b80 10 client.26506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:15:13.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.544+0000 7f5d9d3c4b80 20 client.26506 populate_metadata read hostname 'smithi131' 2024-05-08T22:15:13.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.544+0000 7f5d9d3c4b80 1 --2- 172.21.15.131:0/254814413 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5642b6369160 0x5642b636b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:13.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.544+0000 7f5d9d3c4b80 4 client.26506 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:15:13.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.544+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/254814413 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5642b61f0270 con 0x5642b6369160 2024-05-08T22:15:13.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.545+0000 7f5d82a00700 1 --2- 172.21.15.131:0/254814413 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5642b6369160 0x5642b636b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:13.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.550+0000 7f5d6cc00700 10 client.26506 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:13.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.550+0000 7f5d82a00700 1 --2- 172.21.15.131:0/254814413 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5642b6369160 0x5642b636b550 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6854bb31847d6dd server_cookie=60babc77046d0add in_seq=0 out_seq=0 2024-05-08T22:15:13.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.587+0000 7f5d6cc00700 1 -- 172.21.15.131:0/254814413 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5642b61f0270 con 0x5642b6369160 2024-05-08T22:15:13.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.587+0000 7f5d6cc00700 10 client.26506 handle_command_reply: tid=0 2024-05-08T22:15:13.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.588+0000 7f5d9d3c4b80 1 client.26506 shutdown 2024-05-08T22:15:13.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.588+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/254814413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b6355660 msgr2=0x5642b6359270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:13.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.588+0000 7f5d9d3c4b80 1 --2- 172.21.15.131:0/254814413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b6355660 0x5642b6359270 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5d700cb130 tx=0x7f5d7009b0f0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.589+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/254814413 shutdown_connections 2024-05-08T22:15:13.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.589+0000 7f5d9d3c4b80 1 --2- 172.21.15.131:0/254814413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b6355f80 0x5642b63599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.589+0000 7f5d9d3c4b80 1 --2- 172.21.15.131:0/254814413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642b63500f0 0x5642b635bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.589+0000 7f5d9d3c4b80 1 --2- 172.21.15.131:0/254814413 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5642b6369160 0x5642b636b550 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.589+0000 7f5d9d3c4b80 1 --2- 172.21.15.131:0/254814413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b6355660 0x5642b6359270 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.589+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/254814413 >> 172.21.15.131:0/254814413 conn(0x5642b62a5210 msgr2=0x5642b63415a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:13.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.589+0000 7f5d6cc00700 0 client.26506 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:13.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.590+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/254814413 shutdown_connections 2024-05-08T22:15:13.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.592+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/254814413 wait complete. 2024-05-08T22:15:13.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.592+0000 7f5d9d3c4b80 20 client.26506 trim_cache size 0 max 16384 2024-05-08T22:15:13.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.630+0000 7f5d9d3c4b80 1 Processor -- start 2024-05-08T22:15:13.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.630+0000 7f5d9d3c4b80 1 -- start start 2024-05-08T22:15:13.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.631+0000 7f5d9d3c4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b63500f0 0x5642b6359960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:13.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.631+0000 7f5d9d3c4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642b6355660 0x5642b634fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:13.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.631+0000 7f5d9d3c4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b6355f80 0x5642b6365130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:13.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.631+0000 7f5d9d3c4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b633edc0 con 0x5642b6355f80 2024-05-08T22:15:13.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.631+0000 7f5d9d3c4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b633c800 con 0x5642b6355660 2024-05-08T22:15:13.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.631+0000 7f5d9d3c4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b6370740 con 0x5642b63500f0 2024-05-08T22:15:13.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.631+0000 7f5d82000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642b6355660 0x5642b634fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:13.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.631+0000 7f5d82000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642b6355660 0x5642b634fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:41708/0 (socket says 172.21.15.131:41708) 2024-05-08T22:15:13.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.631+0000 7f5d82000700 1 -- 172.21.15.131:0/169114432 learned_addr learned my addr 172.21.15.131:0/169114432 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:13.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.631+0000 7f5d82a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b63500f0 0x5642b6359960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:13.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.632+0000 7f5d83400700 1 --2- 172.21.15.131:0/169114432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b6355f80 0x5642b6365130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:13.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.632+0000 7f5d82a00700 1 -- 172.21.15.131:0/169114432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642b6355660 msgr2=0x5642b634fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:13.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.632+0000 7f5d82a00700 1 --2- 172.21.15.131:0/169114432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642b6355660 0x5642b634fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.632+0000 7f5d82a00700 1 -- 172.21.15.131:0/169114432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b6355f80 msgr2=0x5642b6365130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:13.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.632+0000 7f5d82a00700 1 --2- 172.21.15.131:0/169114432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b6355f80 0x5642b6365130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.632+0000 7f5d82a00700 1 -- 172.21.15.131:0/169114432 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642b60c4b60 con 0x5642b63500f0 2024-05-08T22:15:13.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.632+0000 7f5d82000700 1 --2- 172.21.15.131:0/169114432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642b6355660 0x5642b634fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:15:13.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.633+0000 7f5d82a00700 1 --2- 172.21.15.131:0/169114432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b63500f0 0x5642b6359960 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x5642b636be60 tx=0x7f5d74011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a476167df7b1f24 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:13.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.634+0000 7f5d78c00700 1 -- 172.21.15.131:0/169114432 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7402c070 con 0x5642b63500f0 2024-05-08T22:15:13.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.634+0000 7f5d78c00700 1 -- 172.21.15.131:0/169114432 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d740278e0 con 0x5642b63500f0 2024-05-08T22:15:13.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.634+0000 7f5d78c00700 1 -- 172.21.15.131:0/169114432 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d74022040 con 0x5642b63500f0 2024-05-08T22:15:13.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.635+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/169114432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b63500f0 msgr2=0x5642b6359960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:13.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.635+0000 7f5d9d3c4b80 1 --2- 172.21.15.131:0/169114432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b63500f0 0x5642b6359960 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x5642b636be60 tx=0x7f5d74011000 comp rx=0 tx=0).stop 2024-05-08T22:15:13.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.635+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/169114432 shutdown_connections 2024-05-08T22:15:13.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.635+0000 7f5d9d3c4b80 1 --2- 172.21.15.131:0/169114432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b6355f80 0x5642b6365130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.635+0000 7f5d9d3c4b80 1 --2- 172.21.15.131:0/169114432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b63500f0 0x5642b6359960 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.635+0000 7f5d9d3c4b80 1 --2- 172.21.15.131:0/169114432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642b6355660 0x5642b634fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.635+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/169114432 >> 172.21.15.131:0/169114432 conn(0x5642b62a5210 msgr2=0x5642b63532b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:13.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.635+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/169114432 shutdown_connections 2024-05-08T22:15:13.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.636+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/169114432 wait complete. 2024-05-08T22:15:13.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.637+0000 7f5d9d3c4b80 1 -- start start 2024-05-08T22:15:13.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.637+0000 7f5d9d3c4b80 1 Processor -- start 2024-05-08T22:15:13.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.640+0000 7f5d9d3c4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b63500f0 0x5642b6360bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:13.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.640+0000 7f5d9d3c4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642b6355660 0x5642b6352150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:13.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.640+0000 7f5d9d3c4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b6355f80 0x5642b6352890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:13.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.641+0000 7f5d9d3c4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b6351060 con 0x5642b63500f0 2024-05-08T22:15:13.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.641+0000 7f5d9d3c4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b63511d0 con 0x5642b6355660 2024-05-08T22:15:13.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.641+0000 7f5d9d3c4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b6351390 con 0x5642b6355f80 2024-05-08T22:15:13.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.641+0000 7f5d9d3c4b80 10 client.0 _ll_register_callbacks cb 0x5642b623cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:15:13.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.641+0000 7f5d83400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b6355f80 0x5642b6352890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:13.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.641+0000 7f5d82a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b63500f0 0x5642b6360bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:13.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.641+0000 7f5d83400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b6355f80 0x5642b6352890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59124/0 (socket says 172.21.15.131:59124) 2024-05-08T22:15:13.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.641+0000 7f5d83400700 1 -- 172.21.15.131:0/2889564031 learned_addr learned my addr 172.21.15.131:0/2889564031 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:13.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.641+0000 7f5d83400700 1 -- 172.21.15.131:0/2889564031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642b6355660 msgr2=0x5642b6352150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:15:13.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.641+0000 7f5d82000700 1 --2- 172.21.15.131:0/2889564031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642b6355660 0x5642b6352150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:13.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.642+0000 7f5d83400700 1 --2- 172.21.15.131:0/2889564031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642b6355660 0x5642b6352150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.642+0000 7f5d83400700 1 -- 172.21.15.131:0/2889564031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b63500f0 msgr2=0x5642b6360bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:13.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.642+0000 7f5d83400700 1 --2- 172.21.15.131:0/2889564031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b63500f0 0x5642b6360bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.642+0000 7f5d83400700 1 -- 172.21.15.131:0/2889564031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d841a6420 con 0x5642b6355f80 2024-05-08T22:15:13.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.642+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/2889564031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d7009e900 con 0x5642b6355f80 2024-05-08T22:15:13.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.642+0000 7f5d82a00700 1 --2- 172.21.15.131:0/2889564031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b63500f0 0x5642b6360bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:15:13.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.642+0000 7f5d82000700 1 --2- 172.21.15.131:0/2889564031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642b6355660 0x5642b6352150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:15:13.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.643+0000 7f5d83400700 1 --2- 172.21.15.131:0/2889564031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b6355f80 0x5642b6352890 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00add0 tx=0x7f5d7c00ae00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb39d3ae560cdce4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:13.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.643+0000 7f5d6cc00700 10 client.26512 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:15:13.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.643+0000 7f5d6cc00700 1 -- 172.21.15.131:0/2889564031 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c007a50 con 0x5642b6355f80 2024-05-08T22:15:13.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.643+0000 7f5d6cc00700 1 -- 172.21.15.131:0/2889564031 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f5d7c006e40 con 0x5642b6355f80 2024-05-08T22:15:13.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.644+0000 7f5d6cc00700 1 -- 172.21.15.131:0/2889564031 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d7c009e80 con 0x5642b6355f80 2024-05-08T22:15:13.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.644+0000 7f5d9d3c4b80 10 client.26512 fetch_fsmap learned FSMap version 28 2024-05-08T22:15:13.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.644+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/2889564031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=28}) v3 -- 0x7f5d7400e9a0 con 0x5642b6355f80 2024-05-08T22:15:13.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.645+0000 7f5d6cc00700 1 -- 172.21.15.131:0/2889564031 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5d7c007bb0 con 0x5642b6355f80 2024-05-08T22:15:13.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.645+0000 7f5d9d3c4b80 10 client.26512 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T22:15:13.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.645+0000 7f5d9d3c4b80 10 client.26512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:15:13.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.645+0000 7f5d9d3c4b80 20 client.26512 populate_metadata read hostname 'smithi131' 2024-05-08T22:15:13.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.645+0000 7f5d9d3c4b80 1 --2- 172.21.15.131:0/2889564031 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5642b63bff30 0x5642b63c2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:13.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.645+0000 7f5d9d3c4b80 4 client.26512 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:15:13.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.645+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/2889564031 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5642b6316810 con 0x5642b63bff30 2024-05-08T22:15:13.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.646+0000 7f5d82a00700 1 --2- 172.21.15.131:0/2889564031 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5642b63bff30 0x5642b63c2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:13.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.649+0000 7f5d6cc00700 10 client.26512 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:13.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.649+0000 7f5d82a00700 1 --2- 172.21.15.131:0/2889564031 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5642b63bff30 0x5642b63c2350 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f0404b40599cca server_cookie=48dcbfef0608196c in_seq=0 out_seq=0 2024-05-08T22:15:13.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.675+0000 7f5d6cc00700 1 -- 172.21.15.131:0/2889564031 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x5642b6316810 con 0x5642b63bff30 2024-05-08T22:15:13.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.675+0000 7f5d6cc00700 10 client.26512 handle_command_reply: tid=0 2024-05-08T22:15:13.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.676+0000 7f5d9d3c4b80 1 client.26512 shutdown 2024-05-08T22:15:13.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.677+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/2889564031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b6355f80 msgr2=0x5642b6352890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:13.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.677+0000 7f5d9d3c4b80 1 --2- 172.21.15.131:0/2889564031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b6355f80 0x5642b6352890 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00add0 tx=0x7f5d7c00ae00 comp rx=0 tx=0).stop 2024-05-08T22:15:13.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.677+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/2889564031 shutdown_connections 2024-05-08T22:15:13.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.677+0000 7f5d9d3c4b80 1 --2- 172.21.15.131:0/2889564031 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5642b63bff30 0x5642b63c2350 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.678+0000 7f5d9d3c4b80 1 --2- 172.21.15.131:0/2889564031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642b6355f80 0x5642b6352890 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.678+0000 7f5d9d3c4b80 1 --2- 172.21.15.131:0/2889564031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642b63500f0 0x5642b6360bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.678+0000 7f5d9d3c4b80 1 --2- 172.21.15.131:0/2889564031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642b6355660 0x5642b6352150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.678+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/2889564031 >> 172.21.15.131:0/2889564031 conn(0x5642b62a5210 msgr2=0x5642b63541a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:13.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.678+0000 7f5d6cc00700 0 client.26512 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:13.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.678+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/2889564031 shutdown_connections 2024-05-08T22:15:13.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.680+0000 7f5d9d3c4b80 1 -- 172.21.15.131:0/2889564031 wait complete. 2024-05-08T22:15:13.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.680+0000 7f5d9d3c4b80 20 client.26512 trim_cache size 0 max 16384 2024-05-08T22:15:13.681 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:15:13.681 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "scrub active (4620 inodes in the stack)", 2024-05-08T22:15:13.681 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-08T22:15:13.681 INFO:teuthology.orchestra.run.smithi131.stdout: "4d699c6d-e487-4b00-9d60-e0aadfef7157": { 2024-05-08T22:15:13.681 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-08T22:15:13.681 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "4d699c6d-e487-4b00-9d60-e0aadfef7157", 2024-05-08T22:15:13.681 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-08T22:15:13.682 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:15:13.682 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:15:13.682 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:15:13.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.681+0000 7f5d89800700 1 -- 172.21.15.131:0/188852911 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f5d680a12f0 msgr2=0x7f5d680a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:13.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.681+0000 7f5d89800700 1 --2- 172.21.15.131:0/188852911 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f5d680a12f0 0x7f5d680a37b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5d74005950 tx=0x7f5d740058e0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.681+0000 7f5d89800700 1 -- 172.21.15.131:0/188852911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d84106290 msgr2=0x7f5d84199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:13.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.681+0000 7f5d89800700 1 --2- 172.21.15.131:0/188852911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d84106290 0x7f5d84199a00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5d70006010 tx=0x7f5d7000bde0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.681+0000 7f5d89800700 1 -- 172.21.15.131:0/188852911 shutdown_connections 2024-05-08T22:15:13.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.681+0000 7f5d89800700 1 --2- 172.21.15.131:0/188852911 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f5d680a12f0 0x7f5d680a37b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.681+0000 7f5d89800700 1 --2- 172.21.15.131:0/188852911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8419dd50 0x7f5d8419e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.681+0000 7f5d89800700 1 --2- 172.21.15.131:0/188852911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d840682e0 0x7f5d841994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.681+0000 7f5d89800700 1 --2- 172.21.15.131:0/188852911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d84106290 0x7f5d84199a00 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:13.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.681+0000 7f5d89800700 1 -- 172.21.15.131:0/188852911 >> 172.21.15.131:0/188852911 conn(0x7f5d84076f40 msgr2=0x7f5d840fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:13.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.681+0000 7f5d89800700 1 -- 172.21.15.131:0/188852911 shutdown_connections 2024-05-08T22:15:13.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:13.682+0000 7f5d89800700 1 -- 172.21.15.131:0/188852911 wait complete. 2024-05-08T22:15:13.718 INFO:tasks.cephfs.filesystem:scrub status for tag:4d699c6d-e487-4b00-9d60-e0aadfef7157 - {'path': '/', 'tag': '4d699c6d-e487-4b00-9d60-e0aadfef7157', 'options': 'recursive,force'} 2024-05-08T22:15:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:14 smithi131 ceph-mon[37260]: scrub queued for path: / 2024-05-08T22:15:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:14 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:14 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:15:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:14 smithi131 ceph-mon[37260]: pgmap v243: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 858 KiB/s wr, 655 op/s 2024-05-08T22:15:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:14 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:15:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:14 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:15:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:14 smithi131 ceph-mon[37260]: osdmap e91: 12 total, 12 up, 12 in 2024-05-08T22:15:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:14 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:14 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:14 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:15:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:14 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:14.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:14 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:15:14.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:14 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:14.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:14 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:15:14.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:14 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:15.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:14 smithi157 ceph-mon[77746]: scrub queued for path: / 2024-05-08T22:15:15.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:14 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:15.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:14 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:15:15.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:14 smithi157 ceph-mon[77746]: pgmap v243: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 858 KiB/s wr, 655 op/s 2024-05-08T22:15:15.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:14 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:15:15.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:14 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:15:15.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:14 smithi157 ceph-mon[77746]: osdmap e91: 12 total, 12 up, 12 in 2024-05-08T22:15:15.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:14 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:15.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:14 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:15.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:14 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:15:15.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:14 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:15.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:14 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:15:15.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:14 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:15.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:14 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:15:15.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:14 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:15.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:14 smithi132 ceph-mon[78180]: scrub queued for path: / 2024-05-08T22:15:15.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:14 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:15.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:14 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:15:15.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:14 smithi132 ceph-mon[78180]: pgmap v243: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 858 KiB/s wr, 655 op/s 2024-05-08T22:15:15.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:14 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:15:15.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:14 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:15:15.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:14 smithi132 ceph-mon[78180]: osdmap e91: 12 total, 12 up, 12 in 2024-05-08T22:15:15.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:14 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:15.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:14 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:15.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:14 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:15:15.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:14 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:15.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:14 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:15:15.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:14 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:15.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:14 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:15:15.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:14 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:15.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:15 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:15:15.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:15 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:15.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:15 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:15:15.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:15 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:16.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:15 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:15:16.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:15 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:16.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:15 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:15:16.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:15 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:16.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:15 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:15:16.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:15 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:16.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:15 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:15:16.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:15 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:16.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:16 smithi131 ceph-mon[37260]: pgmap v245: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 395 op/s 2024-05-08T22:15:17.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:16 smithi157 ceph-mon[77746]: pgmap v245: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 395 op/s 2024-05-08T22:15:17.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:16 smithi132 ceph-mon[78180]: pgmap v245: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 395 op/s 2024-05-08T22:15:18.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:18 smithi131 ceph-mon[37260]: pgmap v246: 129 pgs: 129 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 519 op/s 2024-05-08T22:15:19.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:18 smithi157 ceph-mon[77746]: pgmap v246: 129 pgs: 129 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 519 op/s 2024-05-08T22:15:19.092 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:18 smithi132 ceph-mon[78180]: pgmap v246: 129 pgs: 129 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 519 op/s 2024-05-08T22:15:20.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:20 smithi131 ceph-mon[37260]: pgmap v247: 129 pgs: 129 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 974 op/s 2024-05-08T22:15:20.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:20 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:15:20.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:20 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:15:20.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:20 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:15:20.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:20 smithi131 ceph-mon[37260]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:15:21.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:20 smithi157 ceph-mon[77746]: pgmap v247: 129 pgs: 129 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 974 op/s 2024-05-08T22:15:21.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:20 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:15:21.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:20 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:15:21.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:20 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:15:21.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:20 smithi157 ceph-mon[77746]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:15:21.123 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:20 smithi132 ceph-mon[78180]: pgmap v247: 129 pgs: 129 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 974 op/s 2024-05-08T22:15:21.123 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:20 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:15:21.123 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:20 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:15:21.123 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:20 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:15:21.123 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:20 smithi132 ceph-mon[78180]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:15:22.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:22 smithi131 ceph-mon[37260]: pgmap v248: 129 pgs: 129 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 1.00k op/s 2024-05-08T22:15:23.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:22 smithi157 ceph-mon[77746]: pgmap v248: 129 pgs: 129 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 1.00k op/s 2024-05-08T22:15:23.294 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:22 smithi132 ceph-mon[78180]: pgmap v248: 129 pgs: 129 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 1.00k op/s 2024-05-08T22:15:24.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:23 smithi157 ceph-mon[77746]: osdmap e92: 12 total, 12 up, 12 in 2024-05-08T22:15:24.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:23 smithi157 ceph-mon[77746]: pgmap v250: 129 pgs: 129 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 812 op/s 2024-05-08T22:15:24.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:23 smithi132 ceph-mon[78180]: osdmap e92: 12 total, 12 up, 12 in 2024-05-08T22:15:24.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:23 smithi132 ceph-mon[78180]: pgmap v250: 129 pgs: 129 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 812 op/s 2024-05-08T22:15:24.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:23 smithi131 ceph-mon[37260]: osdmap e92: 12 total, 12 up, 12 in 2024-05-08T22:15:24.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:23 smithi131 ceph-mon[37260]: pgmap v250: 129 pgs: 129 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 812 op/s 2024-05-08T22:15:25.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:24 smithi157 ceph-mon[77746]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T22:15:25.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:24 smithi132 ceph-mon[78180]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T22:15:25.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:24 smithi131 ceph-mon[37260]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T22:15:26.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:25 smithi157 ceph-mon[77746]: pgmap v251: 129 pgs: 129 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.0 MiB/s wr, 898 op/s 2024-05-08T22:15:26.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:25 smithi132 ceph-mon[78180]: pgmap v251: 129 pgs: 129 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.0 MiB/s wr, 898 op/s 2024-05-08T22:15:26.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:25 smithi131 ceph-mon[37260]: pgmap v251: 129 pgs: 129 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.0 MiB/s wr, 898 op/s 2024-05-08T22:15:28.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:28 smithi131 ceph-mon[37260]: pgmap v252: 129 pgs: 129 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.6 MiB/s wr, 813 op/s 2024-05-08T22:15:28.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:28 smithi157 ceph-mon[77746]: pgmap v252: 129 pgs: 129 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.6 MiB/s wr, 813 op/s 2024-05-08T22:15:28.843 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:28 smithi132 ceph-mon[78180]: pgmap v252: 129 pgs: 129 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.6 MiB/s wr, 813 op/s 2024-05-08T22:15:30.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:30 smithi131 ceph-mon[37260]: pgmap v253: 129 pgs: 129 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.2 MiB/s wr, 584 op/s 2024-05-08T22:15:30.779 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:30 smithi157 ceph-mon[77746]: pgmap v253: 129 pgs: 129 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.2 MiB/s wr, 584 op/s 2024-05-08T22:15:30.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:30 smithi132 ceph-mon[78180]: pgmap v253: 129 pgs: 129 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.2 MiB/s wr, 584 op/s 2024-05-08T22:15:32.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:32 smithi131 ceph-mon[37260]: pgmap v254: 129 pgs: 129 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.2 MiB/s wr, 575 op/s 2024-05-08T22:15:32.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:32 smithi157 ceph-mon[77746]: pgmap v254: 129 pgs: 129 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.2 MiB/s wr, 575 op/s 2024-05-08T22:15:32.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:32 smithi132 ceph-mon[78180]: pgmap v254: 129 pgs: 129 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.2 MiB/s wr, 575 op/s 2024-05-08T22:15:34.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:34 smithi157 ceph-mon[77746]: pgmap v255: 129 pgs: 129 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.7 MiB/s wr, 592 op/s 2024-05-08T22:15:34.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:34 smithi132 ceph-mon[78180]: pgmap v255: 129 pgs: 129 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.7 MiB/s wr, 592 op/s 2024-05-08T22:15:34.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:34 smithi131 ceph-mon[37260]: pgmap v255: 129 pgs: 129 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.7 MiB/s wr, 592 op/s 2024-05-08T22:15:36.780 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:36 smithi157 ceph-mon[77746]: pgmap v256: 129 pgs: 129 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.2 MiB/s wr, 1.01k op/s 2024-05-08T22:15:36.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:36 smithi132 ceph-mon[78180]: pgmap v256: 129 pgs: 129 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.2 MiB/s wr, 1.01k op/s 2024-05-08T22:15:36.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:36 smithi131 ceph-mon[37260]: pgmap v256: 129 pgs: 129 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.2 MiB/s wr, 1.01k op/s 2024-05-08T22:15:38.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:38 smithi157 ceph-mon[77746]: pgmap v257: 129 pgs: 129 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.2 MiB/s wr, 1.02k op/s 2024-05-08T22:15:38.843 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:38 smithi132 ceph-mon[78180]: pgmap v257: 129 pgs: 129 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.2 MiB/s wr, 1.02k op/s 2024-05-08T22:15:38.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:38 smithi131 ceph-mon[37260]: pgmap v257: 129 pgs: 129 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.2 MiB/s wr, 1.02k op/s 2024-05-08T22:15:40.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:40 smithi157 ceph-mon[77746]: pgmap v258: 129 pgs: 129 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.9 MiB/s wr, 1.54k op/s 2024-05-08T22:15:40.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:40 smithi132 ceph-mon[78180]: pgmap v258: 129 pgs: 129 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.9 MiB/s wr, 1.54k op/s 2024-05-08T22:15:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:40 smithi131 ceph-mon[37260]: pgmap v258: 129 pgs: 129 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.9 MiB/s wr, 1.54k op/s 2024-05-08T22:15:41.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:41 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:41.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:41 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:41.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:41 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:42.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:42 smithi157 ceph-mon[77746]: pgmap v259: 129 pgs: 129 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 1.40k op/s 2024-05-08T22:15:42.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:42 smithi132 ceph-mon[78180]: pgmap v259: 129 pgs: 129 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 1.40k op/s 2024-05-08T22:15:42.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:42 smithi131 ceph-mon[37260]: pgmap v259: 129 pgs: 129 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 1.40k op/s 2024-05-08T22:15:43.720 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:15:43.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:43 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:15:43.825 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:43 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:15:43.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:43 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:15:43.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.978+0000 7fe723000700 1 -- 172.21.15.131:0/3628638878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe71c0733d0 msgr2=0x7fe71c0737b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:43.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.978+0000 7fe723000700 1 --2- 172.21.15.131:0/3628638878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe71c0733d0 0x7fe71c0737b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe71000b0a0 tx=0x7fe71000b3b0 comp rx=0 tx=0).stop 2024-05-08T22:15:43.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.978+0000 7fe723000700 1 -- 172.21.15.131:0/3628638878 shutdown_connections 2024-05-08T22:15:43.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.978+0000 7fe723000700 1 --2- 172.21.15.131:0/3628638878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71c10d560 0x7fe71c10f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:43.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.978+0000 7fe723000700 1 --2- 172.21.15.131:0/3628638878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe71c0733d0 0x7fe71c0737b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:43.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.978+0000 7fe723000700 1 --2- 172.21.15.131:0/3628638878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe71c073cf0 0x7fe71c10d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:43.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.978+0000 7fe723000700 1 -- 172.21.15.131:0/3628638878 >> 172.21.15.131:0/3628638878 conn(0x7fe71c0fc870 msgr2=0x7fe71c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:43.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.978+0000 7fe723000700 1 -- 172.21.15.131:0/3628638878 shutdown_connections 2024-05-08T22:15:43.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.978+0000 7fe723000700 1 -- 172.21.15.131:0/3628638878 wait complete. 2024-05-08T22:15:43.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.979+0000 7fe723000700 1 Processor -- start 2024-05-08T22:15:43.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.979+0000 7fe723000700 1 -- start start 2024-05-08T22:15:43.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.980+0000 7fe723000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe71c0733d0 0x7fe71c1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:43.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.980+0000 7fe723000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe71c073cf0 0x7fe71c199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:43.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.980+0000 7fe723000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71c10d560 0x7fe71c19de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:43.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.980+0000 7fe723000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c19e4c0 con 0x7fe71c073cf0 2024-05-08T22:15:43.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.980+0000 7fe723000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c19e630 con 0x7fe71c0733d0 2024-05-08T22:15:43.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.980+0000 7fe723000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c19e7f0 con 0x7fe71c10d560 2024-05-08T22:15:43.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.980+0000 7fe721c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe71c0733d0 0x7fe71c1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:43.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.980+0000 7fe721c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe71c0733d0 0x7fe71c1995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:54276/0 (socket says 172.21.15.131:54276) 2024-05-08T22:15:43.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.980+0000 7fe721c00700 1 -- 172.21.15.131:0/3291918207 learned_addr learned my addr 172.21.15.131:0/3291918207 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:43.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.980+0000 7fe721200700 1 --2- 172.21.15.131:0/3291918207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe71c073cf0 0x7fe71c199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:43.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.980+0000 7fe722600700 1 --2- 172.21.15.131:0/3291918207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71c10d560 0x7fe71c19de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:43.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.980+0000 7fe721c00700 1 -- 172.21.15.131:0/3291918207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71c10d560 msgr2=0x7fe71c19de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:43.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.980+0000 7fe721c00700 1 --2- 172.21.15.131:0/3291918207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71c10d560 0x7fe71c19de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:43.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.980+0000 7fe721c00700 1 -- 172.21.15.131:0/3291918207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe71c073cf0 msgr2=0x7fe71c199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:43.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.980+0000 7fe721c00700 1 --2- 172.21.15.131:0/3291918207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe71c073cf0 0x7fe71c199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:43.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.980+0000 7fe721c00700 1 -- 172.21.15.131:0/3291918207 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe708009520 con 0x7fe71c0733d0 2024-05-08T22:15:43.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.980+0000 7fe722600700 1 --2- 172.21.15.131:0/3291918207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71c10d560 0x7fe71c19de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:15:43.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.981+0000 7fe721c00700 1 --2- 172.21.15.131:0/3291918207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe71c0733d0 0x7fe71c1995a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe71000b070 tx=0x7fe710007a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:43.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.981+0000 7fe71a000700 1 -- 172.21.15.131:0/3291918207 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71000e040 con 0x7fe71c0733d0 2024-05-08T22:15:43.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.981+0000 7fe71a000700 1 -- 172.21.15.131:0/3291918207 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe710003c00 con 0x7fe71c0733d0 2024-05-08T22:15:43.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.981+0000 7fe723000700 1 -- 172.21.15.131:0/3291918207 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe710009d00 con 0x7fe71c0733d0 2024-05-08T22:15:43.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.981+0000 7fe71a000700 1 -- 172.21.15.131:0/3291918207 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71001b6e0 con 0x7fe71c0733d0 2024-05-08T22:15:43.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.981+0000 7fe723000700 1 -- 172.21.15.131:0/3291918207 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71c1a6130 con 0x7fe71c0733d0 2024-05-08T22:15:43.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.984+0000 7fe71a000700 1 -- 172.21.15.131:0/3291918207 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fe71001d470 con 0x7fe71c0733d0 2024-05-08T22:15:43.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.985+0000 7fe71a000700 1 --2- 172.21.15.131:0/3291918207 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe7040a1290 0x7fe7040a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:43.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.985+0000 7fe71a000700 1 -- 172.21.15.131:0/3291918207 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fe7100ce0e0 con 0x7fe71c0733d0 2024-05-08T22:15:43.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.985+0000 7fe738046b80 1 Processor -- start 2024-05-08T22:15:43.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.985+0000 7fe721200700 1 --2- 172.21.15.131:0/3291918207 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe7040a1290 0x7fe7040a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:43.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.986+0000 7fe738046b80 1 -- start start 2024-05-08T22:15:43.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.986+0000 7fe738046b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc3985a050 0x55bc3985a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:43.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.986+0000 7fe738046b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc3985a970 0x55bc39854350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:43.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.986+0000 7fe738046b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc39854890 0x55bc39854cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:43.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.986+0000 7fe738046b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc397ab150 con 0x55bc3985a050 2024-05-08T22:15:43.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.986+0000 7fe738046b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc397a8b10 con 0x55bc39854890 2024-05-08T22:15:43.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.986+0000 7fe738046b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc397a4400 con 0x55bc3985a970 2024-05-08T22:15:43.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.986+0000 7fe721c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc3985a050 0x55bc3985a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:43.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.986+0000 7fe722600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc39854890 0x55bc39854cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:43.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.986+0000 7fe722600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc39854890 0x55bc39854cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:54290/0 (socket says 172.21.15.131:54290) 2024-05-08T22:15:43.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.986+0000 7fe722600700 1 -- 172.21.15.131:0/3868690704 learned_addr learned my addr 172.21.15.131:0/3868690704 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:43.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.986+0000 7fe721c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc3985a050 0x55bc3985a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41606/0 (socket says 172.21.15.131:41606) 2024-05-08T22:15:43.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.986+0000 7fe721200700 1 --2- 172.21.15.131:0/3291918207 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe7040a1290 0x7fe7040a3750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe71c19aaf0 tx=0x7fe708009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:43.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.986+0000 7fe721200700 1 --2- 172.21.15.131:0/3868690704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc3985a970 0x55bc39854350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:43.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.987+0000 7fe722600700 1 -- 172.21.15.131:0/3868690704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc3985a970 msgr2=0x55bc39854350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:43.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.987+0000 7fe722600700 1 --2- 172.21.15.131:0/3868690704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc3985a970 0x55bc39854350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:43.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.987+0000 7fe722600700 1 -- 172.21.15.131:0/3868690704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc3985a050 msgr2=0x55bc3985a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:43.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.987+0000 7fe722600700 1 --2- 172.21.15.131:0/3868690704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc3985a050 0x55bc3985a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:43.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.987+0000 7fe722600700 1 -- 172.21.15.131:0/3868690704 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc396253b0 con 0x55bc39854890 2024-05-08T22:15:43.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.987+0000 7fe721c00700 1 --2- 172.21.15.131:0/3868690704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc3985a050 0x55bc3985a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:15:43.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.987+0000 7fe722600700 1 --2- 172.21.15.131:0/3868690704 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc39854890 0x55bc39854cd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe714009d30 tx=0x7fe7140106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=859927a5fd91bc91 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:43.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.987+0000 7fe70ea00700 1 -- 172.21.15.131:0/3868690704 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe714009800 con 0x55bc39854890 2024-05-08T22:15:43.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.987+0000 7fe70ea00700 1 -- 172.21.15.131:0/3868690704 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe714015070 con 0x55bc39854890 2024-05-08T22:15:43.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.987+0000 7fe70ea00700 1 -- 172.21.15.131:0/3868690704 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7140116a0 con 0x55bc39854890 2024-05-08T22:15:43.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.988+0000 7fe738046b80 1 -- 172.21.15.131:0/3868690704 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc39854890 msgr2=0x55bc39854cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:43.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.988+0000 7fe738046b80 1 --2- 172.21.15.131:0/3868690704 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc39854890 0x55bc39854cd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe714009d30 tx=0x7fe7140106b0 comp rx=0 tx=0).stop 2024-05-08T22:15:43.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.988+0000 7fe738046b80 1 -- 172.21.15.131:0/3868690704 shutdown_connections 2024-05-08T22:15:43.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.988+0000 7fe738046b80 1 --2- 172.21.15.131:0/3868690704 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc39854890 0x55bc39854cd0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:43.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.988+0000 7fe738046b80 1 --2- 172.21.15.131:0/3868690704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc3985a050 0x55bc3985a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:43.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.988+0000 7fe738046b80 1 --2- 172.21.15.131:0/3868690704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc3985a970 0x55bc39854350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:43.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.988+0000 7fe738046b80 1 -- 172.21.15.131:0/3868690704 >> 172.21.15.131:0/3868690704 conn(0x55bc397b2210 msgr2=0x55bc3984f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:43.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.988+0000 7fe738046b80 1 -- 172.21.15.131:0/3868690704 shutdown_connections 2024-05-08T22:15:43.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.988+0000 7fe738046b80 1 -- 172.21.15.131:0/3868690704 wait complete. 2024-05-08T22:15:43.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.988+0000 7fe738046b80 1 -- start start 2024-05-08T22:15:43.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.989+0000 7fe738046b80 1 Processor -- start 2024-05-08T22:15:43.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.989+0000 7fe738046b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc39854890 0x55bc39860890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:43.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.989+0000 7fe738046b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc3985a050 0x55bc3985db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:43.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.989+0000 7fe738046b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc3985a970 0x55bc3985e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:43.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.989+0000 7fe738046b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc39740560 con 0x55bc3985a970 2024-05-08T22:15:43.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.989+0000 7fe738046b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc39740200 con 0x55bc39854890 2024-05-08T22:15:43.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.989+0000 7fe738046b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc396276a0 con 0x55bc3985a050 2024-05-08T22:15:43.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.989+0000 7fe738046b80 10 client.0 _ll_register_callbacks cb 0x55bc39749b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:15:43.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.989+0000 7fe721c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc39854890 0x55bc39860890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:43.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.989+0000 7fe722600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc3985a970 0x55bc3985e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:43.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.989+0000 7fe721200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc3985a050 0x55bc3985db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:43.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.989+0000 7fe721c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc39854890 0x55bc39860890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:54306/0 (socket says 172.21.15.131:54306) 2024-05-08T22:15:43.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.989+0000 7fe721c00700 1 -- 172.21.15.131:0/1769542302 learned_addr learned my addr 172.21.15.131:0/1769542302 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:43.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.990+0000 7fe721c00700 1 -- 172.21.15.131:0/1769542302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc3985a050 msgr2=0x55bc3985db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:43.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.990+0000 7fe721c00700 1 --2- 172.21.15.131:0/1769542302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc3985a050 0x55bc3985db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:43.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.990+0000 7fe721c00700 1 -- 172.21.15.131:0/1769542302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc3985a970 msgr2=0x55bc3985e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:43.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.990+0000 7fe721c00700 1 --2- 172.21.15.131:0/1769542302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc3985a970 0x55bc3985e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:43.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.990+0000 7fe721c00700 1 -- 172.21.15.131:0/1769542302 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71c1a6130 con 0x55bc39854890 2024-05-08T22:15:43.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.990+0000 7fe721c00700 1 --2- 172.21.15.131:0/1769542302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc39854890 0x55bc39860890 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe71009adc0 tx=0x7fe71000b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eeddfe80a58cf37 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:43.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.990+0000 7fe703400700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:15:43.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.990+0000 7fe703400700 1 -- 172.21.15.131:0/1769542302 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7100b2070 con 0x55bc39854890 2024-05-08T22:15:43.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.990+0000 7fe703400700 1 -- 172.21.15.131:0/1769542302 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fe71009be40 con 0x55bc39854890 2024-05-08T22:15:43.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.990+0000 7fe738046b80 1 -- 172.21.15.131:0/1769542302 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc396fd270 con 0x55bc39854890 2024-05-08T22:15:43.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.990+0000 7fe703400700 1 -- 172.21.15.131:0/1769542302 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7100b5ea0 con 0x55bc39854890 2024-05-08T22:15:43.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.990+0000 7fe738046b80 10 client.26450 fetch_fsmap learned FSMap version 29 2024-05-08T22:15:43.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.990+0000 7fe738046b80 1 -- 172.21.15.131:0/1769542302 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fe714009960 con 0x55bc39854890 2024-05-08T22:15:43.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.991+0000 7fe703400700 1 -- 172.21.15.131:0/1769542302 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe710064080 con 0x55bc39854890 2024-05-08T22:15:43.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.991+0000 7fe703400700 1 -- 172.21.15.131:0/1769542302 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fe71009a320 con 0x55bc39854890 2024-05-08T22:15:43.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.991+0000 7fe738046b80 10 client.26450 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:15:43.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.991+0000 7fe738046b80 10 client.26450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:15:43.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.991+0000 7fe738046b80 20 client.26450 populate_metadata read hostname 'smithi131' 2024-05-08T22:15:43.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.991+0000 7fe738046b80 1 --2- 172.21.15.131:0/1769542302 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55bc3986e110 0x55bc39870530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:43.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.991+0000 7fe738046b80 4 client.26450 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:15:43.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.991+0000 7fe738046b80 1 -- 172.21.15.131:0/1769542302 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc39823810 con 0x55bc3986e110 2024-05-08T22:15:43.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.991+0000 7fe721200700 1 --2- 172.21.15.131:0/1769542302 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55bc3986e110 0x55bc39870530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:43.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.992+0000 7fe721200700 1 --2- 172.21.15.131:0/1769542302 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55bc3986e110 0x55bc39870530 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e78210d07a374d60 server_cookie=e069bd35fa66ca4a in_seq=0 out_seq=0 2024-05-08T22:15:43.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.992+0000 7fe703400700 10 client.26450 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:43.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.992+0000 7fe703400700 1 -- 172.21.15.131:0/1769542302 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bc39823810 con 0x55bc3986e110 2024-05-08T22:15:43.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.992+0000 7fe703400700 10 client.26450 handle_command_reply: tid=0 2024-05-08T22:15:43.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.992+0000 7fe738046b80 1 client.26450 shutdown 2024-05-08T22:15:43.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.993+0000 7fe738046b80 1 -- 172.21.15.131:0/1769542302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc39854890 msgr2=0x55bc39860890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:43.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.993+0000 7fe738046b80 1 --2- 172.21.15.131:0/1769542302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc39854890 0x55bc39860890 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe71009adc0 tx=0x7fe71000b3b0 comp rx=0 tx=0).stop 2024-05-08T22:15:43.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.993+0000 7fe738046b80 1 -- 172.21.15.131:0/1769542302 shutdown_connections 2024-05-08T22:15:43.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.993+0000 7fe738046b80 1 --2- 172.21.15.131:0/1769542302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc3985a970 0x55bc3985e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:43.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.993+0000 7fe738046b80 1 --2- 172.21.15.131:0/1769542302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc39854890 0x55bc39860890 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:43.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.993+0000 7fe738046b80 1 --2- 172.21.15.131:0/1769542302 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55bc3986e110 0x55bc39870530 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:43.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.993+0000 7fe738046b80 1 --2- 172.21.15.131:0/1769542302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc3985a050 0x55bc3985db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:43.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.993+0000 7fe738046b80 1 -- 172.21.15.131:0/1769542302 >> 172.21.15.131:0/1769542302 conn(0x55bc397b2210 msgr2=0x55bc3984f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:43.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.993+0000 7fe703400700 0 client.26450 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:43.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.993+0000 7fe738046b80 1 -- 172.21.15.131:0/1769542302 shutdown_connections 2024-05-08T22:15:43.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.993+0000 7fe738046b80 1 -- 172.21.15.131:0/1769542302 wait complete. 2024-05-08T22:15:44.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:43.993+0000 7fe738046b80 20 client.26450 trim_cache size 0 max 16384 2024-05-08T22:15:44.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.013+0000 7fe738046b80 1 Processor -- start 2024-05-08T22:15:44.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.013+0000 7fe738046b80 1 -- start start 2024-05-08T22:15:44.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.013+0000 7fe738046b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc39854890 0x55bc39854350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.013+0000 7fe738046b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc3985a050 0x55bc39865320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.013+0000 7fe738046b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc3985a970 0x55bc39865860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.014+0000 7fe738046b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc39875750 con 0x55bc39854890 2024-05-08T22:15:44.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.014+0000 7fe738046b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc398707a0 con 0x55bc3985a970 2024-05-08T22:15:44.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.014+0000 7fe738046b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc3984dd10 con 0x55bc3985a050 2024-05-08T22:15:44.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.014+0000 7fe721200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc3985a050 0x55bc39865320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.014+0000 7fe721200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc3985a050 0x55bc39865320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34740/0 (socket says 172.21.15.131:34740) 2024-05-08T22:15:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.014+0000 7fe721200700 1 -- 172.21.15.131:0/4190199165 learned_addr learned my addr 172.21.15.131:0/4190199165 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.014+0000 7fe721c00700 1 --2- 172.21.15.131:0/4190199165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc39854890 0x55bc39854350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.014+0000 7fe722600700 1 --2- 172.21.15.131:0/4190199165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc3985a970 0x55bc39865860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.014+0000 7fe721200700 1 -- 172.21.15.131:0/4190199165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc3985a970 msgr2=0x55bc39865860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.014+0000 7fe721200700 1 --2- 172.21.15.131:0/4190199165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc3985a970 0x55bc39865860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.014+0000 7fe721200700 1 -- 172.21.15.131:0/4190199165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc39854890 msgr2=0x55bc39854350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.014+0000 7fe721200700 1 --2- 172.21.15.131:0/4190199165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc39854890 0x55bc39854350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.014+0000 7fe721200700 1 -- 172.21.15.131:0/4190199165 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc395d1b60 con 0x55bc3985a050 2024-05-08T22:15:44.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.015+0000 7fe721200700 1 --2- 172.21.15.131:0/4190199165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc3985a050 0x55bc39865320 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55bc3985f0c0 tx=0x7fe708014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=963bde65101e5dd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:44.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.015+0000 7fe70f400700 1 -- 172.21.15.131:0/4190199165 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708025040 con 0x55bc3985a050 2024-05-08T22:15:44.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.015+0000 7fe70f400700 1 -- 172.21.15.131:0/4190199165 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe708016070 con 0x55bc3985a050 2024-05-08T22:15:44.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.015+0000 7fe70f400700 1 -- 172.21.15.131:0/4190199165 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708020a20 con 0x55bc3985a050 2024-05-08T22:15:44.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.016+0000 7fe738046b80 1 -- 172.21.15.131:0/4190199165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc3985a050 msgr2=0x55bc39865320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:44.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.016+0000 7fe738046b80 1 --2- 172.21.15.131:0/4190199165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc3985a050 0x55bc39865320 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55bc3985f0c0 tx=0x7fe708014000 comp rx=0 tx=0).stop 2024-05-08T22:15:44.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.016+0000 7fe738046b80 1 -- 172.21.15.131:0/4190199165 shutdown_connections 2024-05-08T22:15:44.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.016+0000 7fe738046b80 1 --2- 172.21.15.131:0/4190199165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc3985a970 0x55bc39865860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.016+0000 7fe738046b80 1 --2- 172.21.15.131:0/4190199165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc39854890 0x55bc39854350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.016+0000 7fe738046b80 1 --2- 172.21.15.131:0/4190199165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc3985a050 0x55bc39865320 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.016+0000 7fe738046b80 1 -- 172.21.15.131:0/4190199165 >> 172.21.15.131:0/4190199165 conn(0x55bc397b2210 msgr2=0x55bc398604a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:44.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.016+0000 7fe738046b80 1 -- 172.21.15.131:0/4190199165 shutdown_connections 2024-05-08T22:15:44.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.016+0000 7fe738046b80 1 -- 172.21.15.131:0/4190199165 wait complete. 2024-05-08T22:15:44.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.016+0000 7fe738046b80 1 -- start start 2024-05-08T22:15:44.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.017+0000 7fe738046b80 1 Processor -- start 2024-05-08T22:15:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.017+0000 7fe738046b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc39854890 0x55bc398656f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.017+0000 7fe738046b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc3985a050 0x55bc398561f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.017+0000 7fe738046b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc3985a970 0x55bc398567b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.017+0000 7fe738046b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc39856f10 con 0x55bc3985a970 2024-05-08T22:15:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.017+0000 7fe738046b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc39857080 con 0x55bc3985a050 2024-05-08T22:15:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.017+0000 7fe738046b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc398c4620 con 0x55bc39854890 2024-05-08T22:15:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.017+0000 7fe738046b80 10 client.0 _ll_register_callbacks cb 0x55bc39749b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:15:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.017+0000 7fe721200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc3985a050 0x55bc398561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.017+0000 7fe721c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc39854890 0x55bc398656f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.017+0000 7fe721200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc3985a050 0x55bc398561f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:54328/0 (socket says 172.21.15.131:54328) 2024-05-08T22:15:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.017+0000 7fe721200700 1 -- 172.21.15.131:0/185417226 learned_addr learned my addr 172.21.15.131:0/185417226 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.018+0000 7fe722600700 1 --2- 172.21.15.131:0/185417226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc3985a970 0x55bc398567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.018+0000 7fe721200700 1 -- 172.21.15.131:0/185417226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc39854890 msgr2=0x55bc398656f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.018+0000 7fe721200700 1 --2- 172.21.15.131:0/185417226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc39854890 0x55bc398656f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.018+0000 7fe721200700 1 -- 172.21.15.131:0/185417226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc3985a970 msgr2=0x55bc398567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.018+0000 7fe721200700 1 --2- 172.21.15.131:0/185417226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc3985a970 0x55bc398567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.018+0000 7fe721200700 1 -- 172.21.15.131:0/185417226 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe708011e70 con 0x55bc3985a050 2024-05-08T22:15:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.018+0000 7fe722600700 1 --2- 172.21.15.131:0/185417226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc3985a970 0x55bc398567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:15:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.018+0000 7fe721200700 1 --2- 172.21.15.131:0/185417226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc3985a050 0x55bc398561f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe708009b60 tx=0x7fe7080204b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1eafe47360bf8504 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.018+0000 7fe703400700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:15:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.018+0000 7fe703400700 1 -- 172.21.15.131:0/185417226 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708025050 con 0x55bc3985a050 2024-05-08T22:15:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.018+0000 7fe738046b80 1 -- 172.21.15.131:0/185417226 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe714015070 con 0x55bc3985a050 2024-05-08T22:15:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.018+0000 7fe703400700 1 -- 172.21.15.131:0/185417226 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fe708018070 con 0x55bc3985a050 2024-05-08T22:15:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.019+0000 7fe703400700 1 -- 172.21.15.131:0/185417226 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe70802b620 con 0x55bc3985a050 2024-05-08T22:15:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.019+0000 7fe738046b80 10 client.26456 fetch_fsmap learned FSMap version 29 2024-05-08T22:15:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.019+0000 7fe738046b80 1 -- 172.21.15.131:0/185417226 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bc3976a560 con 0x55bc3985a050 2024-05-08T22:15:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.019+0000 7fe703400700 1 -- 172.21.15.131:0/185417226 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe714015070 con 0x55bc3985a050 2024-05-08T22:15:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.019+0000 7fe703400700 1 -- 172.21.15.131:0/185417226 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fe708004950 con 0x55bc3985a050 2024-05-08T22:15:44.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.019+0000 7fe738046b80 10 client.26456 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:15:44.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.019+0000 7fe738046b80 10 client.26456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:15:44.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.019+0000 7fe738046b80 20 client.26456 populate_metadata read hostname 'smithi131' 2024-05-08T22:15:44.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.019+0000 7fe738046b80 1 --2- 172.21.15.131:0/185417226 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55bc398c5d70 0x55bc398c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.019+0000 7fe738046b80 4 client.26456 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:15:44.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.019+0000 7fe738046b80 1 -- 172.21.15.131:0/185417226 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bc3979cec0 con 0x55bc398c5d70 2024-05-08T22:15:44.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.019+0000 7fe721c00700 1 --2- 172.21.15.131:0/185417226 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55bc398c5d70 0x55bc398c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.020+0000 7fe703400700 10 client.26456 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:44.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.020+0000 7fe721c00700 1 --2- 172.21.15.131:0/185417226 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55bc398c5d70 0x55bc398c8190 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2be12ce4e938dab server_cookie=67d19bb0c5440541 in_seq=0 out_seq=0 2024-05-08T22:15:44.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.022+0000 7fe703400700 1 -- 172.21.15.131:0/185417226 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55bc3979cec0 con 0x55bc398c5d70 2024-05-08T22:15:44.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.022+0000 7fe703400700 10 client.26456 handle_command_reply: tid=0 2024-05-08T22:15:44.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.022+0000 7fe738046b80 1 client.26456 shutdown 2024-05-08T22:15:44.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.022+0000 7fe738046b80 1 -- 172.21.15.131:0/185417226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc3985a050 msgr2=0x55bc398561f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:44.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.022+0000 7fe738046b80 1 --2- 172.21.15.131:0/185417226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc3985a050 0x55bc398561f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe708009b60 tx=0x7fe7080204b0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.022+0000 7fe738046b80 1 -- 172.21.15.131:0/185417226 shutdown_connections 2024-05-08T22:15:44.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.022+0000 7fe738046b80 1 --2- 172.21.15.131:0/185417226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc3985a970 0x55bc398567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.022+0000 7fe738046b80 1 --2- 172.21.15.131:0/185417226 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55bc398c5d70 0x55bc398c8190 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.022+0000 7fe738046b80 1 --2- 172.21.15.131:0/185417226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc39854890 0x55bc398656f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.022+0000 7fe738046b80 1 --2- 172.21.15.131:0/185417226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bc3985a050 0x55bc398561f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.022+0000 7fe738046b80 1 -- 172.21.15.131:0/185417226 >> 172.21.15.131:0/185417226 conn(0x55bc397b2210 msgr2=0x55bc39858ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:44.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.022+0000 7fe703400700 0 client.26456 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:44.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.022+0000 7fe738046b80 1 -- 172.21.15.131:0/185417226 shutdown_connections 2024-05-08T22:15:44.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.022+0000 7fe738046b80 1 -- 172.21.15.131:0/185417226 wait complete. 2024-05-08T22:15:44.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.022+0000 7fe738046b80 20 client.26456 trim_cache size 0 max 16384 2024-05-08T22:15:44.025 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:15:44.025 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-08T22:15:44.025 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-08T22:15:44.026 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:15:44.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.023+0000 7fe723000700 1 -- 172.21.15.131:0/3291918207 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe7040a1290 msgr2=0x7fe7040a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:44.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.023+0000 7fe723000700 1 --2- 172.21.15.131:0/3291918207 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe7040a1290 0x7fe7040a3750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe71c19aaf0 tx=0x7fe708009260 comp rx=0 tx=0).stop 2024-05-08T22:15:44.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.023+0000 7fe723000700 1 -- 172.21.15.131:0/3291918207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe71c0733d0 msgr2=0x7fe71c1995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:44.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.023+0000 7fe723000700 1 --2- 172.21.15.131:0/3291918207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe71c0733d0 0x7fe71c1995a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe71000b070 tx=0x7fe710007a60 comp rx=0 tx=0).stop 2024-05-08T22:15:44.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.023+0000 7fe723000700 1 -- 172.21.15.131:0/3291918207 shutdown_connections 2024-05-08T22:15:44.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.023+0000 7fe723000700 1 --2- 172.21.15.131:0/3291918207 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe7040a1290 0x7fe7040a3750 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.023+0000 7fe723000700 1 --2- 172.21.15.131:0/3291918207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71c10d560 0x7fe71c19de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.023+0000 7fe723000700 1 --2- 172.21.15.131:0/3291918207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe71c0733d0 0x7fe71c1995a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.023+0000 7fe723000700 1 --2- 172.21.15.131:0/3291918207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe71c073cf0 0x7fe71c199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.023+0000 7fe723000700 1 -- 172.21.15.131:0/3291918207 >> 172.21.15.131:0/3291918207 conn(0x7fe71c0fc870 msgr2=0x7fe71c0fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:44.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.023+0000 7fe723000700 1 -- 172.21.15.131:0/3291918207 shutdown_connections 2024-05-08T22:15:44.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.023+0000 7fe723000700 1 -- 172.21.15.131:0/3291918207 wait complete. 2024-05-08T22:15:44.043 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T22:15:44.043 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:15:44.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.251+0000 7f9090400700 1 -- 172.21.15.131:0/2079805814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9088073c60 msgr2=0x7f90881118b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:44.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.251+0000 7f9090400700 1 --2- 172.21.15.131:0/2079805814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9088073c60 0x7f90881118b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9074009990 tx=0x7f9074009ca0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.251+0000 7f9090400700 1 -- 172.21.15.131:0/2079805814 shutdown_connections 2024-05-08T22:15:44.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.251+0000 7f9090400700 1 --2- 172.21.15.131:0/2079805814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9088111df0 0x7f90881141e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.251+0000 7f9090400700 1 --2- 172.21.15.131:0/2079805814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9088073340 0x7f9088073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.251+0000 7f9090400700 1 --2- 172.21.15.131:0/2079805814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9088073c60 0x7f90881118b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.251+0000 7f9090400700 1 -- 172.21.15.131:0/2079805814 >> 172.21.15.131:0/2079805814 conn(0x7f90880fc8f0 msgr2=0x7f90880fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:44.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.252+0000 7f9090400700 1 -- 172.21.15.131:0/2079805814 shutdown_connections 2024-05-08T22:15:44.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.252+0000 7f9090400700 1 -- 172.21.15.131:0/2079805814 wait complete. 2024-05-08T22:15:44.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.253+0000 7f9090400700 1 Processor -- start 2024-05-08T22:15:44.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.253+0000 7f9090400700 1 -- start start 2024-05-08T22:15:44.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.253+0000 7f9090400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9088073340 0x7f90881a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.253+0000 7f9090400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9088073c60 0x7f90881a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.253+0000 7f9090400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9088111df0 0x7f908819d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.253+0000 7f9090400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908819ddd0 con 0x7f9088073c60 2024-05-08T22:15:44.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.253+0000 7f9090400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908819df40 con 0x7f9088111df0 2024-05-08T22:15:44.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.253+0000 7f9090400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908819e100 con 0x7f9088073340 2024-05-08T22:15:44.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.253+0000 7f908e400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9088111df0 0x7f908819d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.253+0000 7f908da00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9088073340 0x7f90881a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.253+0000 7f908e400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9088111df0 0x7f908819d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:54344/0 (socket says 172.21.15.131:54344) 2024-05-08T22:15:44.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.253+0000 7f908da00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9088073340 0x7f90881a3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34756/0 (socket says 172.21.15.131:34756) 2024-05-08T22:15:44.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.253+0000 7f908d000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9088073c60 0x7f90881a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.253+0000 7f908e400700 1 -- 172.21.15.131:0/4034938823 learned_addr learned my addr 172.21.15.131:0/4034938823 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:44.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.254+0000 7f908da00700 1 -- 172.21.15.131:0/4034938823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9088111df0 msgr2=0x7f908819d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:44.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.254+0000 7f908da00700 1 --2- 172.21.15.131:0/4034938823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9088111df0 0x7f908819d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.254+0000 7f908da00700 1 -- 172.21.15.131:0/4034938823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9088073c60 msgr2=0x7f90881a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:44.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.254+0000 7f908da00700 1 --2- 172.21.15.131:0/4034938823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9088073c60 0x7f90881a3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.254+0000 7f908da00700 1 -- 172.21.15.131:0/4034938823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907c009520 con 0x7f9088073340 2024-05-08T22:15:44.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.254+0000 7f908d000700 1 --2- 172.21.15.131:0/4034938823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9088073c60 0x7f90881a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:15:44.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.254+0000 7f908da00700 1 --2- 172.21.15.131:0/4034938823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9088073340 0x7f90881a3640 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f907c00e9f0 tx=0x7f907c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:44.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.254+0000 7f9086000700 1 -- 172.21.15.131:0/4034938823 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c00cb70 con 0x7f9088073340 2024-05-08T22:15:44.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.254+0000 7f9086000700 1 -- 172.21.15.131:0/4034938823 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f907c00ee90 con 0x7f9088073340 2024-05-08T22:15:44.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.254+0000 7f9090400700 1 -- 172.21.15.131:0/4034938823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90740095f0 con 0x7f9088073340 2024-05-08T22:15:44.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.254+0000 7f9086000700 1 -- 172.21.15.131:0/4034938823 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c003cf0 con 0x7f9088073340 2024-05-08T22:15:44.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.255+0000 7f9090400700 1 -- 172.21.15.131:0/4034938823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90881aa450 con 0x7f9088073340 2024-05-08T22:15:44.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.255+0000 7f9090400700 1 -- 172.21.15.131:0/4034938823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90881042f0 con 0x7f9088073340 2024-05-08T22:15:44.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.257+0000 7f9086000700 1 -- 172.21.15.131:0/4034938823 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f907c00ccd0 con 0x7f9088073340 2024-05-08T22:15:44.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.259+0000 7f9086000700 1 --2- 172.21.15.131:0/4034938823 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f90700a1290 0x7f90700a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.259+0000 7f9086000700 1 -- 172.21.15.131:0/4034938823 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f907c014070 con 0x7f9088073340 2024-05-08T22:15:44.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.259+0000 7f908d000700 1 --2- 172.21.15.131:0/4034938823 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f90700a1290 0x7f90700a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.260+0000 7f908d000700 1 --2- 172.21.15.131:0/4034938823 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f90700a1290 0x7f90700a3750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f908819ee80 tx=0x7f9074005fd0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:44.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.263+0000 7f9086000700 1 -- 172.21.15.131:0/4034938823 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f907c0963f0 con 0x7f9088073340 2024-05-08T22:15:44.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.542+0000 7f9090400700 1 -- 172.21.15.131:0/4034938823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9088074a80 con 0x7f9088073340 2024-05-08T22:15:44.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.543+0000 7f9086000700 1 -- 172.21.15.131:0/4034938823 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10892 (secure 0 0 0) 0x7f907c099a10 con 0x7f9088073340 2024-05-08T22:15:44.544 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:15:44.544 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:05:14.034908+0000","modified":"2024-05-08T22:15:23.658983+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:15:44.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.546+0000 7f9090400700 1 -- 172.21.15.131:0/4034938823 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f90700a1290 msgr2=0x7f90700a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:44.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.546+0000 7f9090400700 1 --2- 172.21.15.131:0/4034938823 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f90700a1290 0x7f90700a3750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f908819ee80 tx=0x7f9074005fd0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.546+0000 7f9090400700 1 -- 172.21.15.131:0/4034938823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9088073340 msgr2=0x7f90881a3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:44.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.546+0000 7f9090400700 1 --2- 172.21.15.131:0/4034938823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9088073340 0x7f90881a3640 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f907c00e9f0 tx=0x7f907c00ed00 comp rx=0 tx=0).stop 2024-05-08T22:15:44.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.546+0000 7f9090400700 1 -- 172.21.15.131:0/4034938823 shutdown_connections 2024-05-08T22:15:44.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.546+0000 7f9090400700 1 --2- 172.21.15.131:0/4034938823 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f90700a1290 0x7f90700a3750 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.546+0000 7f9090400700 1 --2- 172.21.15.131:0/4034938823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9088111df0 0x7f908819d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.546+0000 7f9090400700 1 --2- 172.21.15.131:0/4034938823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9088073340 0x7f90881a3640 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.546+0000 7f9090400700 1 --2- 172.21.15.131:0/4034938823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9088073c60 0x7f90881a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.546+0000 7f9090400700 1 -- 172.21.15.131:0/4034938823 >> 172.21.15.131:0/4034938823 conn(0x7f90880fc8f0 msgr2=0x7f90880fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:44.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.546+0000 7f9090400700 1 -- 172.21.15.131:0/4034938823 shutdown_connections 2024-05-08T22:15:44.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.546+0000 7f9090400700 1 -- 172.21.15.131:0/4034938823 wait complete. 2024-05-08T22:15:44.548 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 29 2024-05-08T22:15:44.566 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T22:15:44.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:44 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:15:44.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:44 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:15:44.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:44 smithi157 ceph-mon[77746]: pgmap v260: 129 pgs: 129 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 1.47k op/s 2024-05-08T22:15:44.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:44 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:15:44.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:44 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:15:44.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:44 smithi132 ceph-mon[78180]: pgmap v260: 129 pgs: 129 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 1.47k op/s 2024-05-08T22:15:44.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.851+0000 7f6980a00700 1 -- 172.21.15.131:0/1211004230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f697c103d20 msgr2=0x7f697c107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:44.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.851+0000 7f6980a00700 1 --2- 172.21.15.131:0/1211004230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f697c103d20 0x7f697c107d60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f696c0130d0 tx=0x7f696c0133e0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.851+0000 7f6980a00700 1 -- 172.21.15.131:0/1211004230 shutdown_connections 2024-05-08T22:15:44.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.851+0000 7f6980a00700 1 --2- 172.21.15.131:0/1211004230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f697c108490 0x7f697c110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.851+0000 7f6980a00700 1 --2- 172.21.15.131:0/1211004230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f697c103370 0x7f697c103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.851+0000 7f6980a00700 1 --2- 172.21.15.131:0/1211004230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f697c103d20 0x7f697c107d60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.851+0000 7f6980a00700 1 -- 172.21.15.131:0/1211004230 >> 172.21.15.131:0/1211004230 conn(0x7f697c0febb0 msgr2=0x7f697c100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:44.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.851+0000 7f6980a00700 1 -- 172.21.15.131:0/1211004230 shutdown_connections 2024-05-08T22:15:44.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.851+0000 7f6980a00700 1 -- 172.21.15.131:0/1211004230 wait complete. 2024-05-08T22:15:44.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.853+0000 7f6980a00700 1 Processor -- start 2024-05-08T22:15:44.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.853+0000 7f6980a00700 1 -- start start 2024-05-08T22:15:44.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.854+0000 7f6980a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f697c103370 0x7f697c199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.854+0000 7f6980a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f697c108490 0x7f697c19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.854+0000 7f6980a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f697c1957e0 0x7f697c195c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.854+0000 7f6979e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f697c103370 0x7f697c199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.854+0000 7f6980a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c196370 con 0x7f697c108490 2024-05-08T22:15:44.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.854+0000 7f6980a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c1964e0 con 0x7f697c1957e0 2024-05-08T22:15:44.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.854+0000 7f6980a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c1966a0 con 0x7f697c103370 2024-05-08T22:15:44.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.854+0000 7f6979e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f697c103370 0x7f697c199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34776/0 (socket says 172.21.15.131:34776) 2024-05-08T22:15:44.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.854+0000 7f6979e00700 1 -- 172.21.15.131:0/1884390014 learned_addr learned my addr 172.21.15.131:0/1884390014 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:44.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.854+0000 7f6979400700 1 --2- 172.21.15.131:0/1884390014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f697c108490 0x7f697c19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.855+0000 7f697a800700 1 --2- 172.21.15.131:0/1884390014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f697c1957e0 0x7f697c195c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.855+0000 7f697a800700 1 -- 172.21.15.131:0/1884390014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f697c103370 msgr2=0x7f697c199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:44.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.855+0000 7f697a800700 1 --2- 172.21.15.131:0/1884390014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f697c103370 0x7f697c199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.855+0000 7f697a800700 1 -- 172.21.15.131:0/1884390014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f697c108490 msgr2=0x7f697c19a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:44.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.855+0000 7f697a800700 1 --2- 172.21.15.131:0/1884390014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f697c108490 0x7f697c19a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.855+0000 7f697a800700 1 -- 172.21.15.131:0/1884390014 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f696c011d30 con 0x7f697c1957e0 2024-05-08T22:15:44.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.856+0000 7f697a800700 1 --2- 172.21.15.131:0/1884390014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f697c1957e0 0x7f697c195c90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f697400efc0 tx=0x7f697400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:44.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.857+0000 7f6972a00700 1 -- 172.21.15.131:0/1884390014 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974018070 con 0x7f697c1957e0 2024-05-08T22:15:44.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.857+0000 7f6972a00700 1 -- 172.21.15.131:0/1884390014 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f697400f040 con 0x7f697c1957e0 2024-05-08T22:15:44.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.857+0000 7f6972a00700 1 -- 172.21.15.131:0/1884390014 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697400cc30 con 0x7f697c1957e0 2024-05-08T22:15:44.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.857+0000 7f6980a00700 1 -- 172.21.15.131:0/1884390014 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697c1a1e30 con 0x7f697c1957e0 2024-05-08T22:15:44.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.858+0000 7f6980a00700 1 -- 172.21.15.131:0/1884390014 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697c1a2260 con 0x7f697c1957e0 2024-05-08T22:15:44.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.865+0000 7f6972a00700 1 -- 172.21.15.131:0/1884390014 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f6974003a00 con 0x7f697c1957e0 2024-05-08T22:15:44.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.868+0000 7f6972a00700 1 --2- 172.21.15.131:0/1884390014 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f69600a13e0 0x7f69600a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.868+0000 7f6972a00700 1 -- 172.21.15.131:0/1884390014 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f69740cc940 con 0x7f697c1957e0 2024-05-08T22:15:44.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.869+0000 7f6979e00700 1 --2- 172.21.15.131:0/1884390014 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f69600a13e0 0x7f69600a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.870+0000 7f6979e00700 1 --2- 172.21.15.131:0/1884390014 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f69600a13e0 0x7f69600a38a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f696400ad20 tx=0x7f6964005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:44.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.873+0000 7f699447cb80 1 Processor -- start 2024-05-08T22:15:44.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.874+0000 7f699447cb80 1 -- start start 2024-05-08T22:15:44.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.874+0000 7f699447cb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94767ce00 0x55e94767d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.874+0000 7f699447cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e94767d720 0x55e947677350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.874+0000 7f699447cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e947677890 0x55e947677cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.874+0000 7f699447cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9475ce150 con 0x55e94767d720 2024-05-08T22:15:44.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.874+0000 7f699447cb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9475cbb10 con 0x55e94767ce00 2024-05-08T22:15:44.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.874+0000 7f699447cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9475c7400 con 0x55e947677890 2024-05-08T22:15:44.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.875+0000 7f697a800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e947677890 0x55e947677cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.875+0000 7f697a800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e947677890 0x55e947677cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34778/0 (socket says 172.21.15.131:34778) 2024-05-08T22:15:44.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.875+0000 7f697a800700 1 -- 172.21.15.131:0/3284927463 learned_addr learned my addr 172.21.15.131:0/3284927463 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:44.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.875+0000 7f6979e00700 1 --2- 172.21.15.131:0/3284927463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94767ce00 0x55e94767d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.875+0000 7f697a800700 1 -- 172.21.15.131:0/3284927463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94767ce00 msgr2=0x55e94767d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:44.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.875+0000 7f697a800700 1 --2- 172.21.15.131:0/3284927463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94767ce00 0x55e94767d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.875+0000 7f697a800700 1 -- 172.21.15.131:0/3284927463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e94767d720 msgr2=0x55e947677350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:44.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.875+0000 7f697a800700 1 --2- 172.21.15.131:0/3284927463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e94767d720 0x55e947677350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.875+0000 7f697a800700 1 -- 172.21.15.131:0/3284927463 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9474483b0 con 0x55e947677890 2024-05-08T22:15:44.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.875+0000 7f697a800700 1 --2- 172.21.15.131:0/3284927463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e947677890 0x55e947677cd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6974096100 tx=0x7f69740994e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d0a13534ccf1924 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:44.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.876+0000 7f696b000700 1 -- 172.21.15.131:0/3284927463 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974013e90 con 0x55e947677890 2024-05-08T22:15:44.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.876+0000 7f696b000700 1 -- 172.21.15.131:0/3284927463 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6974099d90 con 0x55e947677890 2024-05-08T22:15:44.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.876+0000 7f696b000700 1 -- 172.21.15.131:0/3284927463 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697409d160 con 0x55e947677890 2024-05-08T22:15:44.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.877+0000 7f699447cb80 1 -- 172.21.15.131:0/3284927463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e947677890 msgr2=0x55e947677cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:44.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.877+0000 7f699447cb80 1 --2- 172.21.15.131:0/3284927463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e947677890 0x55e947677cd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6974096100 tx=0x7f69740994e0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.877+0000 7f699447cb80 1 -- 172.21.15.131:0/3284927463 shutdown_connections 2024-05-08T22:15:44.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.877+0000 7f699447cb80 1 --2- 172.21.15.131:0/3284927463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e947677890 0x55e947677cd0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.877+0000 7f699447cb80 1 --2- 172.21.15.131:0/3284927463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94767ce00 0x55e94767d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.877+0000 7f699447cb80 1 --2- 172.21.15.131:0/3284927463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e94767d720 0x55e947677350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.877+0000 7f699447cb80 1 -- 172.21.15.131:0/3284927463 >> 172.21.15.131:0/3284927463 conn(0x55e9475d5210 msgr2=0x55e947671df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:44.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.877+0000 7f699447cb80 1 -- 172.21.15.131:0/3284927463 shutdown_connections 2024-05-08T22:15:44.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.877+0000 7f699447cb80 1 -- 172.21.15.131:0/3284927463 wait complete. 2024-05-08T22:15:44.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.878+0000 7f699447cb80 1 -- start start 2024-05-08T22:15:44.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.878+0000 7f699447cb80 1 Processor -- start 2024-05-08T22:15:44.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.879+0000 7f699447cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e947677890 0x55e947680b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.879+0000 7f699447cb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94767ce00 0x55e947681080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.879+0000 7f699447cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e94767d720 0x55e9476815c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.879+0000 7f699447cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e947563560 con 0x55e947677890 2024-05-08T22:15:44.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.879+0000 7f699447cb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e947563200 con 0x55e94767ce00 2024-05-08T22:15:44.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.879+0000 7f699447cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94744a6a0 con 0x55e94767d720 2024-05-08T22:15:44.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.880+0000 7f699447cb80 10 client.0 _ll_register_callbacks cb 0x55e94756cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:15:44.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.880+0000 7f6979400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94767ce00 0x55e947681080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.880+0000 7f697a800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e94767d720 0x55e9476815c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.881+0000 7f697a800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e94767d720 0x55e9476815c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34782/0 (socket says 172.21.15.131:34782) 2024-05-08T22:15:44.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.881+0000 7f697a800700 1 -- 172.21.15.131:0/643682077 learned_addr learned my addr 172.21.15.131:0/643682077 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:44.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.881+0000 7f6979400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94767ce00 0x55e947681080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:54380/0 (socket says 172.21.15.131:54380) 2024-05-08T22:15:44.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.881+0000 7f6979e00700 1 --2- 172.21.15.131:0/643682077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e947677890 0x55e947680b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.881+0000 7f6979400700 1 -- 172.21.15.131:0/643682077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e94767d720 msgr2=0x55e9476815c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:44.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.881+0000 7f6979400700 1 --2- 172.21.15.131:0/643682077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e94767d720 0x55e9476815c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.881+0000 7f6979400700 1 -- 172.21.15.131:0/643682077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e947677890 msgr2=0x55e947680b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:44.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.881+0000 7f6979400700 1 --2- 172.21.15.131:0/643682077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e947677890 0x55e947680b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.881+0000 7f6979400700 1 -- 172.21.15.131:0/643682077 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697c1a2260 con 0x55e94767ce00 2024-05-08T22:15:44.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.886+0000 7f6979400700 1 --2- 172.21.15.131:0/643682077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94767ce00 0x55e947681080 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f696c005e90 tx=0x7f696c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=528abbd3da39228c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:44.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.886+0000 7f695f400700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:15:44.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.887+0000 7f695f400700 1 -- 172.21.15.131:0/643682077 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c016040 con 0x55e94767ce00 2024-05-08T22:15:44.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.887+0000 7f695f400700 1 -- 172.21.15.131:0/643682077 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f696c0052d0 con 0x55e94767ce00 2024-05-08T22:15:44.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.887+0000 7f699447cb80 1 -- 172.21.15.131:0/643682077 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e947520270 con 0x55e94767ce00 2024-05-08T22:15:44.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.887+0000 7f695f400700 1 -- 172.21.15.131:0/643682077 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f696c005490 con 0x55e94767ce00 2024-05-08T22:15:44.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.887+0000 7f699447cb80 10 client.26474 fetch_fsmap learned FSMap version 29 2024-05-08T22:15:44.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.887+0000 7f699447cb80 1 -- 172.21.15.131:0/643682077 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e9473f4b60 con 0x55e94767ce00 2024-05-08T22:15:44.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.887+0000 7f695f400700 1 -- 172.21.15.131:0/643682077 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f696c02b020 con 0x55e94767ce00 2024-05-08T22:15:44.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.888+0000 7f695f400700 1 -- 172.21.15.131:0/643682077 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f696c021070 con 0x55e94767ce00 2024-05-08T22:15:44.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.888+0000 7f699447cb80 10 client.26474 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:15:44.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.888+0000 7f699447cb80 10 client.26474 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:15:44.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.888+0000 7f699447cb80 20 client.26474 populate_metadata read hostname 'smithi131' 2024-05-08T22:15:44.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.888+0000 7f699447cb80 1 --2- 172.21.15.131:0/643682077 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55e9476a21c0 0x55e9476a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.888+0000 7f699447cb80 4 client.26474 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:15:44.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.888+0000 7f699447cb80 1 -- 172.21.15.131:0/643682077 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e947646810 con 0x55e9476a21c0 2024-05-08T22:15:44.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.889+0000 7f6979e00700 1 --2- 172.21.15.131:0/643682077 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55e9476a21c0 0x55e9476a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:44 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:15:44.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:44 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:15:44.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:44 smithi131 ceph-mon[37260]: pgmap v260: 129 pgs: 129 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 1.47k op/s 2024-05-08T22:15:44.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.890+0000 7f695f400700 10 client.26474 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:15:44.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.890+0000 7f6979e00700 1 --2- 172.21.15.131:0/643682077 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55e9476a21c0 0x55e9476a45e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=283534dd596f0b6c server_cookie=c760d1942d3eadc8 in_seq=0 out_seq=0 2024-05-08T22:15:44.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.892+0000 7f695f400700 1 -- 172.21.15.131:0/643682077 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e947646810 con 0x55e9476a21c0 2024-05-08T22:15:44.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.892+0000 7f695f400700 10 client.26474 handle_command_reply: tid=0 2024-05-08T22:15:44.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.892+0000 7f699447cb80 1 client.26474 shutdown 2024-05-08T22:15:44.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.892+0000 7f699447cb80 1 -- 172.21.15.131:0/643682077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94767ce00 msgr2=0x55e947681080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:44.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.892+0000 7f699447cb80 1 --2- 172.21.15.131:0/643682077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94767ce00 0x55e947681080 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f696c005e90 tx=0x7f696c0118d0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.892+0000 7f699447cb80 1 -- 172.21.15.131:0/643682077 shutdown_connections 2024-05-08T22:15:44.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.892+0000 7f699447cb80 1 --2- 172.21.15.131:0/643682077 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55e9476a21c0 0x55e9476a45e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.893+0000 7f695f400700 0 client.26474 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:15:44.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.893+0000 7f699447cb80 1 --2- 172.21.15.131:0/643682077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e94767d720 0x55e9476815c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.893+0000 7f699447cb80 1 --2- 172.21.15.131:0/643682077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e947677890 0x55e947680b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.893+0000 7f699447cb80 1 --2- 172.21.15.131:0/643682077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94767ce00 0x55e947681080 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.893+0000 7f699447cb80 1 -- 172.21.15.131:0/643682077 >> 172.21.15.131:0/643682077 conn(0x55e9475d5210 msgr2=0x55e9476715a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:44.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.895+0000 7f699447cb80 1 -- 172.21.15.131:0/643682077 shutdown_connections 2024-05-08T22:15:44.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.895+0000 7f699447cb80 1 -- 172.21.15.131:0/643682077 wait complete. 2024-05-08T22:15:44.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.895+0000 7f699447cb80 20 client.26474 trim_cache size 0 max 16384 2024-05-08T22:15:44.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.918+0000 7f699447cb80 1 Processor -- start 2024-05-08T22:15:44.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.918+0000 7f699447cb80 1 -- start start 2024-05-08T22:15:44.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.918+0000 7f699447cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e947677890 0x55e947677350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.918+0000 7f699447cb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94767ce00 0x55e94769e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.918+0000 7f699447cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e94767d720 0x55e94768cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.918+0000 7f699447cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9476a97f0 con 0x55e947677890 2024-05-08T22:15:44.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.918+0000 7f699447cb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94767d430 con 0x55e94767ce00 2024-05-08T22:15:44.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.918+0000 7f699447cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94767f090 con 0x55e94767d720 2024-05-08T22:15:44.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.919+0000 7f6979400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94767ce00 0x55e94769e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.919+0000 7f697a800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e94767d720 0x55e94768cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.919+0000 7f6979400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94767ce00 0x55e94769e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:54396/0 (socket says 172.21.15.131:54396) 2024-05-08T22:15:44.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.919+0000 7f6979400700 1 -- 172.21.15.131:0/3589610281 learned_addr learned my addr 172.21.15.131:0/3589610281 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:44.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.919+0000 7f6979e00700 1 --2- 172.21.15.131:0/3589610281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e947677890 0x55e947677350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.920+0000 7f6979400700 1 -- 172.21.15.131:0/3589610281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e94767d720 msgr2=0x55e94768cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:44.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.920+0000 7f6979400700 1 --2- 172.21.15.131:0/3589610281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e94767d720 0x55e94768cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.920+0000 7f6979400700 1 -- 172.21.15.131:0/3589610281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e947677890 msgr2=0x55e947677350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:44.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.920+0000 7f6979400700 1 --2- 172.21.15.131:0/3589610281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e947677890 0x55e947677350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.920+0000 7f6979400700 1 -- 172.21.15.131:0/3589610281 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e94758d560 con 0x55e94767ce00 2024-05-08T22:15:44.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.920+0000 7f6979e00700 1 --2- 172.21.15.131:0/3589610281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e947677890 0x55e947677350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:15:44.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.920+0000 7f6979400700 1 --2- 172.21.15.131:0/3589610281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94767ce00 0x55e94769e300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f696c005a80 tx=0x7f696c011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=416dbfda92e78e5f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:44.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.920+0000 7f696ba00700 1 -- 172.21.15.131:0/3589610281 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c016040 con 0x55e94767ce00 2024-05-08T22:15:44.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.921+0000 7f696ba00700 1 -- 172.21.15.131:0/3589610281 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f696c02b020 con 0x55e94767ce00 2024-05-08T22:15:44.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.921+0000 7f696ba00700 1 -- 172.21.15.131:0/3589610281 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c00c610 con 0x55e94767ce00 2024-05-08T22:15:44.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.922+0000 7f699447cb80 1 -- 172.21.15.131:0/3589610281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94767ce00 msgr2=0x55e94769e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:44.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.922+0000 7f699447cb80 1 --2- 172.21.15.131:0/3589610281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94767ce00 0x55e94769e300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f696c005a80 tx=0x7f696c011980 comp rx=0 tx=0).stop 2024-05-08T22:15:44.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.923+0000 7f699447cb80 1 -- 172.21.15.131:0/3589610281 shutdown_connections 2024-05-08T22:15:44.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.923+0000 7f699447cb80 1 --2- 172.21.15.131:0/3589610281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e94767d720 0x55e94768cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.923+0000 7f699447cb80 1 --2- 172.21.15.131:0/3589610281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e947677890 0x55e947677350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.923+0000 7f699447cb80 1 --2- 172.21.15.131:0/3589610281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94767ce00 0x55e94769e300 secure :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f696c005a80 tx=0x7f696c011980 comp rx=0 tx=0).stop 2024-05-08T22:15:44.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.923+0000 7f699447cb80 1 -- 172.21.15.131:0/3589610281 >> 172.21.15.131:0/3589610281 conn(0x55e9475d5210 msgr2=0x55e94767f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:44.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.923+0000 7f699447cb80 1 -- 172.21.15.131:0/3589610281 shutdown_connections 2024-05-08T22:15:44.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.923+0000 7f699447cb80 1 -- 172.21.15.131:0/3589610281 wait complete. 2024-05-08T22:15:44.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.924+0000 7f699447cb80 1 -- start start 2024-05-08T22:15:44.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.924+0000 7f699447cb80 1 Processor -- start 2024-05-08T22:15:44.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.925+0000 7f699447cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e947677890 0x55e947686150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.925+0000 7f699447cb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94767d720 0x55e947686690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.925+0000 7f699447cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e947686bd0 0x55e9476f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.925+0000 7f699447cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9476f7c00 con 0x55e947677890 2024-05-08T22:15:44.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.925+0000 7f699447cb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9476f7d40 con 0x55e94767d720 2024-05-08T22:15:44.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.925+0000 7f699447cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9476f7f00 con 0x55e947686bd0 2024-05-08T22:15:44.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.925+0000 7f699447cb80 10 client.0 _ll_register_callbacks cb 0x55e94756cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:15:44.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.926+0000 7f6979400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94767d720 0x55e947686690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.927+0000 7f697a800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e947686bd0 0x55e9476f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.927+0000 7f697a800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e947686bd0 0x55e9476f76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34798/0 (socket says 172.21.15.131:34798) 2024-05-08T22:15:44.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.927+0000 7f697a800700 1 -- 172.21.15.131:0/850353760 learned_addr learned my addr 172.21.15.131:0/850353760 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:44.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.927+0000 7f6979e00700 1 --2- 172.21.15.131:0/850353760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e947677890 0x55e947686150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.927+0000 7f697a800700 1 -- 172.21.15.131:0/850353760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94767d720 msgr2=0x55e947686690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:44.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.927+0000 7f697a800700 1 --2- 172.21.15.131:0/850353760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94767d720 0x55e947686690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.927+0000 7f697a800700 1 -- 172.21.15.131:0/850353760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e947677890 msgr2=0x55e947686150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:44.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.927+0000 7f697a800700 1 --2- 172.21.15.131:0/850353760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e947677890 0x55e947686150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.927+0000 7f697a800700 1 -- 172.21.15.131:0/850353760 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9473f4b60 con 0x55e947686bd0 2024-05-08T22:15:44.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.927+0000 7f6979e00700 1 --2- 172.21.15.131:0/850353760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e947677890 0x55e947686150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:15:44.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.930+0000 7f697a800700 1 --2- 172.21.15.131:0/850353760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e947686bd0 0x55e9476f76c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f697400c6f0 tx=0x7f69740a1b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b7b2238141b3c16 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:44.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.930+0000 7f695f400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:15:44.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.931+0000 7f695f400700 1 -- 172.21.15.131:0/850353760 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974099690 con 0x55e947686bd0 2024-05-08T22:15:44.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.931+0000 7f695f400700 1 -- 172.21.15.131:0/850353760 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f697409d160 con 0x55e947686bd0 2024-05-08T22:15:44.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.931+0000 7f699447cb80 1 -- 172.21.15.131:0/850353760 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6974099d90 con 0x55e947686bd0 2024-05-08T22:15:44.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.932+0000 7f695f400700 1 -- 172.21.15.131:0/850353760 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69740998f0 con 0x55e947686bd0 2024-05-08T22:15:44.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.932+0000 7f699447cb80 10 client.26554 fetch_fsmap learned FSMap version 29 2024-05-08T22:15:44.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.932+0000 7f699447cb80 1 -- 172.21.15.131:0/850353760 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e94744ccd0 con 0x55e947686bd0 2024-05-08T22:15:44.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.932+0000 7f695f400700 1 -- 172.21.15.131:0/850353760 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6974099d90 con 0x55e947686bd0 2024-05-08T22:15:44.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.933+0000 7f695f400700 1 -- 172.21.15.131:0/850353760 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f69740a15f0 con 0x55e947686bd0 2024-05-08T22:15:44.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.933+0000 7f699447cb80 10 client.26554 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:15:44.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.933+0000 7f699447cb80 10 client.26554 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:15:44.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.933+0000 7f699447cb80 20 client.26554 populate_metadata read hostname 'smithi131' 2024-05-08T22:15:44.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.933+0000 7f699447cb80 1 --2- 172.21.15.131:0/850353760 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55e9476f91f0 0x55e9476fb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:44.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.933+0000 7f699447cb80 4 client.26554 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:15:44.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.933+0000 7f699447cb80 1 -- 172.21.15.131:0/850353760 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e9475bfec0 con 0x55e9476f91f0 2024-05-08T22:15:44.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.933+0000 7f6979e00700 1 --2- 172.21.15.131:0/850353760 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55e9476f91f0 0x55e9476fb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:44.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.935+0000 7f695f400700 10 client.26554 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:15:44.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.935+0000 7f6979e00700 1 --2- 172.21.15.131:0/850353760 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55e9476f91f0 0x55e9476fb610 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5fb96cd20e4a9523 server_cookie=18ea2d8c090029ad in_seq=0 out_seq=0 2024-05-08T22:15:44.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.935+0000 7f695f400700 1 -- 172.21.15.131:0/850353760 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e9475bfec0 con 0x55e9476f91f0 2024-05-08T22:15:44.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.936+0000 7f695f400700 10 client.26554 handle_command_reply: tid=0 2024-05-08T22:15:44.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.936+0000 7f699447cb80 1 client.26554 shutdown 2024-05-08T22:15:44.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.937+0000 7f699447cb80 1 -- 172.21.15.131:0/850353760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e947686bd0 msgr2=0x55e9476f76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:44.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.937+0000 7f699447cb80 1 --2- 172.21.15.131:0/850353760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e947686bd0 0x55e9476f76c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f697400c6f0 tx=0x7f69740a1b30 comp rx=0 tx=0).stop 2024-05-08T22:15:44.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.937+0000 7f699447cb80 1 -- 172.21.15.131:0/850353760 shutdown_connections 2024-05-08T22:15:44.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.937+0000 7f699447cb80 1 --2- 172.21.15.131:0/850353760 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55e9476f91f0 0x55e9476fb610 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.937+0000 7f699447cb80 1 --2- 172.21.15.131:0/850353760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e947686bd0 0x55e9476f76c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.938+0000 7f699447cb80 1 --2- 172.21.15.131:0/850353760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e947677890 0x55e947686150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.938+0000 7f699447cb80 1 --2- 172.21.15.131:0/850353760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94767d720 0x55e947686690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.938+0000 7f695f400700 0 client.26554 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:15:44.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.938+0000 7f699447cb80 1 -- 172.21.15.131:0/850353760 >> 172.21.15.131:0/850353760 conn(0x55e9475d5210 msgr2=0x55e94767f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:44.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.938+0000 7f699447cb80 1 -- 172.21.15.131:0/850353760 shutdown_connections 2024-05-08T22:15:44.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.938+0000 7f699447cb80 1 -- 172.21.15.131:0/850353760 wait complete. 2024-05-08T22:15:44.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.940+0000 7f699447cb80 20 client.26554 trim_cache size 0 max 16384 2024-05-08T22:15:44.941 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:15:44.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.941+0000 7f6980a00700 1 -- 172.21.15.131:0/1884390014 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f69600a13e0 msgr2=0x7f69600a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:44.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.941+0000 7f6980a00700 1 --2- 172.21.15.131:0/1884390014 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f69600a13e0 0x7f69600a38a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f696400ad20 tx=0x7f6964005f90 comp rx=0 tx=0).stop 2024-05-08T22:15:44.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.941+0000 7f6980a00700 1 -- 172.21.15.131:0/1884390014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f697c1957e0 msgr2=0x7f697c195c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:44.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.941+0000 7f6980a00700 1 --2- 172.21.15.131:0/1884390014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f697c1957e0 0x7f697c195c90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f697400efc0 tx=0x7f697400c3a0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.942+0000 7f6980a00700 1 -- 172.21.15.131:0/1884390014 shutdown_connections 2024-05-08T22:15:44.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.942+0000 7f6980a00700 1 --2- 172.21.15.131:0/1884390014 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f69600a13e0 0x7f69600a38a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.942+0000 7f6980a00700 1 --2- 172.21.15.131:0/1884390014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f697c1957e0 0x7f697c195c90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.942+0000 7f6980a00700 1 --2- 172.21.15.131:0/1884390014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f697c103370 0x7f697c199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.942+0000 7f6980a00700 1 --2- 172.21.15.131:0/1884390014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f697c108490 0x7f697c19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:44.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.942+0000 7f6980a00700 1 -- 172.21.15.131:0/1884390014 >> 172.21.15.131:0/1884390014 conn(0x7f697c0febb0 msgr2=0x7f697c106d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:44.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.943+0000 7f6980a00700 1 -- 172.21.15.131:0/1884390014 shutdown_connections 2024-05-08T22:15:44.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:44.943+0000 7f6980a00700 1 -- 172.21.15.131:0/1884390014 wait complete. 2024-05-08T22:15:44.982 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T22:15:45.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.319+0000 7f1fcb600700 1 -- 172.21.15.131:0/1660060509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fc40754d0 msgr2=0x7f1fc40758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:45.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.319+0000 7f1fcb600700 1 --2- 172.21.15.131:0/1660060509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fc40754d0 0x7f1fc40758b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009990 tx=0x7f1fb8009ca0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.320+0000 7f1fcb600700 1 -- 172.21.15.131:0/1660060509 shutdown_connections 2024-05-08T22:15:45.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.320+0000 7f1fcb600700 1 --2- 172.21.15.131:0/1660060509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fc4111da0 0x7f1fc4114190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.320+0000 7f1fcb600700 1 --2- 172.21.15.131:0/1660060509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fc40754d0 0x7f1fc40758b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.320+0000 7f1fcb600700 1 --2- 172.21.15.131:0/1660060509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fc4075df0 0x7f1fc4111860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.320+0000 7f1fcb600700 1 -- 172.21.15.131:0/1660060509 >> 172.21.15.131:0/1660060509 conn(0x7f1fc40febb0 msgr2=0x7f1fc4100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:45.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.321+0000 7f1fcb600700 1 -- 172.21.15.131:0/1660060509 shutdown_connections 2024-05-08T22:15:45.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.321+0000 7f1fcb600700 1 -- 172.21.15.131:0/1660060509 wait complete. 2024-05-08T22:15:45.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.322+0000 7f1fcb600700 1 Processor -- start 2024-05-08T22:15:45.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.322+0000 7f1fcb600700 1 -- start start 2024-05-08T22:15:45.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.323+0000 7f1fcb600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fc40754d0 0x7f1fc410b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.323+0000 7f1fcb600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fc4075df0 0x7f1fc4103100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.323+0000 7f1fcb600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fc4111da0 0x7f1fc4103640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.323+0000 7f1fcb600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc4103dd0 con 0x7f1fc4111da0 2024-05-08T22:15:45.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.323+0000 7f1fcb600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc4103f40 con 0x7f1fc40754d0 2024-05-08T22:15:45.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.323+0000 7f1fcb600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc41b2600 con 0x7f1fc4075df0 2024-05-08T22:15:45.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.323+0000 7f1fc3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fc4075df0 0x7f1fc4103100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.323+0000 7f1fc3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fc4075df0 0x7f1fc4103100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34804/0 (socket says 172.21.15.131:34804) 2024-05-08T22:15:45.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.323+0000 7f1fc3e00700 1 -- 172.21.15.131:0/1072410161 learned_addr learned my addr 172.21.15.131:0/1072410161 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:45.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.324+0000 7f1fc8c00700 1 --2- 172.21.15.131:0/1072410161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fc40754d0 0x7f1fc410b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.324+0000 7f1fc3e00700 1 -- 172.21.15.131:0/1072410161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fc40754d0 msgr2=0x7f1fc410b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:45.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.324+0000 7f1fc3e00700 1 --2- 172.21.15.131:0/1072410161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fc40754d0 0x7f1fc410b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.324+0000 7f1fc3e00700 1 -- 172.21.15.131:0/1072410161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fc4111da0 msgr2=0x7f1fc4103640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:45.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.324+0000 7f1fc3e00700 1 --2- 172.21.15.131:0/1072410161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fc4111da0 0x7f1fc4103640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.324+0000 7f1fc3e00700 1 -- 172.21.15.131:0/1072410161 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb80095f0 con 0x7f1fc4075df0 2024-05-08T22:15:45.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.324+0000 7f1fc3e00700 1 --2- 172.21.15.131:0/1072410161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fc4075df0 0x7f1fc4103100 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1fb002f990 tx=0x7f1fb002fca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:45.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.327+0000 7f1fc1600700 1 -- 172.21.15.131:0/1072410161 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0009b40 con 0x7f1fc4075df0 2024-05-08T22:15:45.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.327+0000 7f1fc1600700 1 -- 172.21.15.131:0/1072410161 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fb002fe80 con 0x7f1fc4075df0 2024-05-08T22:15:45.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.327+0000 7f1fc1600700 1 -- 172.21.15.131:0/1072410161 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0031630 con 0x7f1fc4075df0 2024-05-08T22:15:45.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.327+0000 7f1fcb600700 1 -- 172.21.15.131:0/1072410161 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc419d630 con 0x7f1fc4075df0 2024-05-08T22:15:45.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.327+0000 7f1fcb600700 1 -- 172.21.15.131:0/1072410161 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc419daf0 con 0x7f1fc4075df0 2024-05-08T22:15:45.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.333+0000 7f1fc1600700 1 -- 172.21.15.131:0/1072410161 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f1fb0031790 con 0x7f1fc4075df0 2024-05-08T22:15:45.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.333+0000 7f1fdf165b80 1 Processor -- start 2024-05-08T22:15:45.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.335+0000 7f1fdf165b80 1 -- start start 2024-05-08T22:15:45.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.336+0000 7f1fc1600700 1 --2- 172.21.15.131:0/1072410161 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1fac0a1460 0x7f1fac0a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.336+0000 7f1fc1600700 1 -- 172.21.15.131:0/1072410161 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f1fb00eeff0 con 0x7f1fc4075df0 2024-05-08T22:15:45.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.337+0000 7f1fdf165b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a3113ce00 0x562a3113d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.337+0000 7f1fdf165b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562a3113d720 0x562a31137350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.337+0000 7f1fdf165b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a31137890 0x562a31137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.337+0000 7f1fdf165b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a3108e150 con 0x562a31137890 2024-05-08T22:15:45.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.337+0000 7f1fdf165b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a3108bb10 con 0x562a3113d720 2024-05-08T22:15:45.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.337+0000 7f1fdf165b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a31087400 con 0x562a3113ce00 2024-05-08T22:15:45.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.337+0000 7f1fc8c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a3113ce00 0x562a3113d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.338+0000 7f1fc8c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a3113ce00 0x562a3113d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34810/0 (socket says 172.21.15.131:34810) 2024-05-08T22:15:45.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.337+0000 7f1fc9600700 1 --2- 172.21.15.131:0/1072410161 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1fac0a1460 0x7f1fac0a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.338+0000 7f1fc8c00700 1 -- 172.21.15.131:0/1160281761 learned_addr learned my addr 172.21.15.131:0/1160281761 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:45.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.338+0000 7f1fc9600700 1 --2- 172.21.15.131:0/1160281761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a31137890 0x562a31137cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.338+0000 7f1fc8c00700 1 -- 172.21.15.131:0/1160281761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562a3113d720 msgr2=0x562a31137350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:45.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.338+0000 7f1fc8c00700 1 --2- 172.21.15.131:0/1160281761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562a3113d720 0x562a31137350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.338+0000 7f1fc8c00700 1 -- 172.21.15.131:0/1160281761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a31137890 msgr2=0x562a31137cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:45.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.338+0000 7f1fc8c00700 1 --2- 172.21.15.131:0/1160281761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a31137890 0x562a31137cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.338+0000 7f1fc8c00700 1 -- 172.21.15.131:0/1160281761 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a30f083b0 con 0x562a3113ce00 2024-05-08T22:15:45.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.338+0000 7f1fc9600700 1 --2- 172.21.15.131:0/1160281761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a31137890 0x562a31137cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:15:45.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.339+0000 7f1fc8c00700 1 --2- 172.21.15.131:0/1160281761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a3113ce00 0x562a3113d1e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8005eb0 tx=0x7f1fb8005ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a87e80f1708f9455 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:45.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.339+0000 7f1fb5c00700 1 -- 172.21.15.131:0/1160281761 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb800be80 con 0x562a3113ce00 2024-05-08T22:15:45.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.339+0000 7f1fc9600700 1 --2- 172.21.15.131:0/1072410161 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1fac0a1460 0x7f1fac0a3920 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009710 tx=0x7f1fbc009350 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:45.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.339+0000 7f1fb5c00700 1 -- 172.21.15.131:0/1160281761 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fb801e040 con 0x562a3113ce00 2024-05-08T22:15:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.339+0000 7f1fb5c00700 1 -- 172.21.15.131:0/1160281761 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb800f460 con 0x562a3113ce00 2024-05-08T22:15:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.343+0000 7f1fdf165b80 1 -- 172.21.15.131:0/1160281761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a3113ce00 msgr2=0x562a3113d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.343+0000 7f1fdf165b80 1 --2- 172.21.15.131:0/1160281761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a3113ce00 0x562a3113d1e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8005eb0 tx=0x7f1fb8005ee0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.343+0000 7f1fdf165b80 1 -- 172.21.15.131:0/1160281761 shutdown_connections 2024-05-08T22:15:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.343+0000 7f1fdf165b80 1 --2- 172.21.15.131:0/1160281761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a31137890 0x562a31137cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.343+0000 7f1fdf165b80 1 --2- 172.21.15.131:0/1160281761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a3113ce00 0x562a3113d1e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.343+0000 7f1fdf165b80 1 --2- 172.21.15.131:0/1160281761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562a3113d720 0x562a31137350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.343+0000 7f1fdf165b80 1 -- 172.21.15.131:0/1160281761 >> 172.21.15.131:0/1160281761 conn(0x562a31095210 msgr2=0x562a31131df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.344+0000 7f1fdf165b80 1 -- 172.21.15.131:0/1160281761 shutdown_connections 2024-05-08T22:15:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.344+0000 7f1fdf165b80 1 -- 172.21.15.131:0/1160281761 wait complete. 2024-05-08T22:15:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.345+0000 7f1fdf165b80 1 -- start start 2024-05-08T22:15:45.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.346+0000 7f1fdf165b80 1 Processor -- start 2024-05-08T22:15:45.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.347+0000 7f1fdf165b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a31137890 0x562a31140c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.347+0000 7f1fdf165b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562a3113ce00 0x562a311411a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.347+0000 7f1fdf165b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a3113d720 0x562a311416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.347+0000 7f1fdf165b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a31023560 con 0x562a31137890 2024-05-08T22:15:45.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.347+0000 7f1fdf165b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a31023200 con 0x562a3113ce00 2024-05-08T22:15:45.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.347+0000 7f1fc9600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a3113d720 0x562a311416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.347+0000 7f1fc9600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a3113d720 0x562a311416e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34824/0 (socket says 172.21.15.131:34824) 2024-05-08T22:15:45.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.347+0000 7f1fc9600700 1 -- 172.21.15.131:0/74209201 learned_addr learned my addr 172.21.15.131:0/74209201 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:45.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.347+0000 7f1fdf165b80 1 -- 172.21.15.131:0/74209201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a30f0a6a0 con 0x562a3113d720 2024-05-08T22:15:45.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.348+0000 7f1fdf165b80 10 client.0 _ll_register_callbacks cb 0x562a3102cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:15:45.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.348+0000 7f1fc8c00700 1 --2- 172.21.15.131:0/74209201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a31137890 0x562a31140c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.348+0000 7f1fc8c00700 1 -- 172.21.15.131:0/74209201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a3113d720 msgr2=0x562a311416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:45.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.348+0000 7f1fc8c00700 1 --2- 172.21.15.131:0/74209201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a3113d720 0x562a311416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.348+0000 7f1fc8c00700 1 -- 172.21.15.131:0/74209201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562a3113ce00 msgr2=0x562a311411a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:45.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.348+0000 7f1fc8c00700 1 --2- 172.21.15.131:0/74209201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562a3113ce00 0x562a311411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.349+0000 7f1fc8c00700 1 -- 172.21.15.131:0/74209201 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc4114c80 con 0x562a31137890 2024-05-08T22:15:45.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.349+0000 7f1fc8c00700 1 --2- 172.21.15.131:0/74209201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a31137890 0x562a31140c60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8005e80 tx=0x7f1fb800bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70a932fb26292c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:45.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.349+0000 7f1faa000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:15:45.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.350+0000 7f1faa000700 1 -- 172.21.15.131:0/74209201 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8005480 con 0x562a31137890 2024-05-08T22:15:45.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.350+0000 7f1faa000700 1 -- 172.21.15.131:0/74209201 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f1fb80055e0 con 0x562a31137890 2024-05-08T22:15:45.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.350+0000 7f1fdf165b80 1 -- 172.21.15.131:0/74209201 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a30fe0270 con 0x562a31137890 2024-05-08T22:15:45.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.350+0000 7f1faa000700 1 -- 172.21.15.131:0/74209201 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fb802c470 con 0x562a31137890 2024-05-08T22:15:45.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.353+0000 7f1fdf165b80 10 client.16455 fetch_fsmap learned FSMap version 29 2024-05-08T22:15:45.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.353+0000 7f1fdf165b80 1 -- 172.21.15.131:0/74209201 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562a30eb4b60 con 0x562a31137890 2024-05-08T22:15:45.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.354+0000 7f1faa000700 1 -- 172.21.15.131:0/74209201 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562a30fe0270 con 0x562a31137890 2024-05-08T22:15:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.355+0000 7f1faa000700 1 -- 172.21.15.131:0/74209201 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f1fb80223f0 con 0x562a31137890 2024-05-08T22:15:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.355+0000 7f1fdf165b80 10 client.16455 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:15:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.355+0000 7f1fdf165b80 10 client.16455 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:15:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.355+0000 7f1fdf165b80 20 client.16455 populate_metadata read hostname 'smithi131' 2024-05-08T22:15:45.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.355+0000 7f1fdf165b80 1 --2- 172.21.15.131:0/74209201 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x562a31150ce0 0x562a311530d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.355+0000 7f1fdf165b80 4 client.16455 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:15:45.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.355+0000 7f1fdf165b80 1 -- 172.21.15.131:0/74209201 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a31106810 con 0x562a31150ce0 2024-05-08T22:15:45.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.357+0000 7f1fc3e00700 1 --2- 172.21.15.131:0/74209201 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x562a31150ce0 0x562a311530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.358+0000 7f1faa000700 10 client.16455 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:15:45.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.358+0000 7f1fc3e00700 1 --2- 172.21.15.131:0/74209201 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x562a31150ce0 0x562a311530d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=26f3f8c89ad52eb6 server_cookie=7cff8a6b65b4ce09 in_seq=0 out_seq=0 2024-05-08T22:15:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.361+0000 7f1faa000700 1 -- 172.21.15.131:0/74209201 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562a31106810 con 0x562a31150ce0 2024-05-08T22:15:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.361+0000 7f1faa000700 10 client.16455 handle_command_reply: tid=0 2024-05-08T22:15:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.361+0000 7f1fdf165b80 1 client.16455 shutdown 2024-05-08T22:15:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.361+0000 7f1fdf165b80 1 -- 172.21.15.131:0/74209201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a31137890 msgr2=0x562a31140c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:45.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.361+0000 7f1fdf165b80 1 --2- 172.21.15.131:0/74209201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a31137890 0x562a31140c60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8005e80 tx=0x7f1fb800bc10 comp rx=0 tx=0).stop 2024-05-08T22:15:45.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.362+0000 7f1fdf165b80 1 -- 172.21.15.131:0/74209201 shutdown_connections 2024-05-08T22:15:45.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.362+0000 7f1fdf165b80 1 --2- 172.21.15.131:0/74209201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a3113d720 0x562a311416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.362+0000 7f1fdf165b80 1 --2- 172.21.15.131:0/74209201 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x562a31150ce0 0x562a311530d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.362+0000 7f1fdf165b80 1 --2- 172.21.15.131:0/74209201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a31137890 0x562a31140c60 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.362+0000 7f1fdf165b80 1 --2- 172.21.15.131:0/74209201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562a3113ce00 0x562a311411a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.362+0000 7f1fdf165b80 1 -- 172.21.15.131:0/74209201 >> 172.21.15.131:0/74209201 conn(0x562a31095210 msgr2=0x562a311315a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:45.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.362+0000 7f1faa000700 0 client.16455 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:15:45.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.362+0000 7f1fdf165b80 1 -- 172.21.15.131:0/74209201 shutdown_connections 2024-05-08T22:15:45.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.362+0000 7f1fdf165b80 1 -- 172.21.15.131:0/74209201 wait complete. 2024-05-08T22:15:45.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.362+0000 7f1fdf165b80 20 client.16455 trim_cache size 0 max 16384 2024-05-08T22:15:45.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.396+0000 7f1fdf165b80 1 Processor -- start 2024-05-08T22:15:45.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.396+0000 7f1fdf165b80 1 -- start start 2024-05-08T22:15:45.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.396+0000 7f1fdf165b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a31137890 0x562a31137350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.396+0000 7f1fdf165b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562a3113ce00 0x562a31131d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.396+0000 7f1fdf165b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a3113d720 0x562a31141500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.396+0000 7f1fdf165b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a3112a040 con 0x562a3113d720 2024-05-08T22:15:45.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.396+0000 7f1fdf165b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a31158200 con 0x562a3113ce00 2024-05-08T22:15:45.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.396+0000 7f1fdf165b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a3113d430 con 0x562a31137890 2024-05-08T22:15:45.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.397+0000 7f1fc8c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a31137890 0x562a31137350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.397+0000 7f1fc3e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562a3113ce00 0x562a31131d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.397+0000 7f1fc8c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a31137890 0x562a31137350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34834/0 (socket says 172.21.15.131:34834) 2024-05-08T22:15:45.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.397+0000 7f1fc8c00700 1 -- 172.21.15.131:0/973699954 learned_addr learned my addr 172.21.15.131:0/973699954 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:45.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.397+0000 7f1fc9600700 1 --2- 172.21.15.131:0/973699954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a3113d720 0x562a31141500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.397+0000 7f1fc8c00700 1 -- 172.21.15.131:0/973699954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562a3113ce00 msgr2=0x562a31131d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:45.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.397+0000 7f1fc8c00700 1 --2- 172.21.15.131:0/973699954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562a3113ce00 0x562a31131d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.397+0000 7f1fc8c00700 1 -- 172.21.15.131:0/973699954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a3113d720 msgr2=0x562a31141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:45.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.397+0000 7f1fc8c00700 1 --2- 172.21.15.131:0/973699954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a3113d720 0x562a31141500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.397+0000 7f1fc8c00700 1 -- 172.21.15.131:0/973699954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a31137f40 con 0x562a31137890 2024-05-08T22:15:45.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.398+0000 7f1fc8c00700 1 --2- 172.21.15.131:0/973699954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a31137890 0x562a31137350 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009990 tx=0x7f1fb80094b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b651d6de94b929d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.398+0000 7f1fc9600700 1 --2- 172.21.15.131:0/973699954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a3113d720 0x562a31141500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:15:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.398+0000 7f1fb6600700 1 -- 172.21.15.131:0/973699954 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb80229d0 con 0x562a31137890 2024-05-08T22:15:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.398+0000 7f1fb6600700 1 -- 172.21.15.131:0/973699954 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fb80223f0 con 0x562a31137890 2024-05-08T22:15:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.398+0000 7f1fb6600700 1 -- 172.21.15.131:0/973699954 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb802b640 con 0x562a31137890 2024-05-08T22:15:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.399+0000 7f1fdf165b80 1 -- 172.21.15.131:0/973699954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a31137890 msgr2=0x562a31137350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.399+0000 7f1fdf165b80 1 --2- 172.21.15.131:0/973699954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a31137890 0x562a31137350 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009990 tx=0x7f1fb80094b0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.399+0000 7f1fdf165b80 1 -- 172.21.15.131:0/973699954 shutdown_connections 2024-05-08T22:15:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.399+0000 7f1fdf165b80 1 --2- 172.21.15.131:0/973699954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a3113d720 0x562a31141500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.399+0000 7f1fdf165b80 1 --2- 172.21.15.131:0/973699954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a31137890 0x562a31137350 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.399+0000 7f1fdf165b80 1 --2- 172.21.15.131:0/973699954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562a3113ce00 0x562a31131d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.399+0000 7f1fdf165b80 1 -- 172.21.15.131:0/973699954 >> 172.21.15.131:0/973699954 conn(0x562a31095210 msgr2=0x562a3113ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:45.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.400+0000 7f1fdf165b80 1 -- 172.21.15.131:0/973699954 shutdown_connections 2024-05-08T22:15:45.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.400+0000 7f1fdf165b80 1 -- 172.21.15.131:0/973699954 wait complete. 2024-05-08T22:15:45.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.400+0000 7f1fdf165b80 1 -- start start 2024-05-08T22:15:45.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.402+0000 7f1fdf165b80 1 Processor -- start 2024-05-08T22:15:45.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.403+0000 7f1fdf165b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562a31137890 0x562a31148570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.403+0000 7f1fdf165b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a3113ce00 0x562a311a20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.404+0000 7f1fdf165b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a3113d720 0x562a311a26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.404+0000 7f1fdf165b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a311a2dc0 con 0x562a3113d720 2024-05-08T22:15:45.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.404+0000 7f1fdf165b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a311a2f00 con 0x562a31137890 2024-05-08T22:15:45.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.404+0000 7f1fdf165b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a311460f0 con 0x562a3113ce00 2024-05-08T22:15:45.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.404+0000 7f1fdf165b80 10 client.0 _ll_register_callbacks cb 0x562a3102cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:15:45.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.404+0000 7f1fc9600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a3113d720 0x562a311a26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.404+0000 7f1fc3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a3113ce00 0x562a311a20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.404+0000 7f1fc3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a3113ce00 0x562a311a20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34840/0 (socket says 172.21.15.131:34840) 2024-05-08T22:15:45.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.404+0000 7f1fc3e00700 1 -- 172.21.15.131:0/374138419 learned_addr learned my addr 172.21.15.131:0/374138419 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:45.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.404+0000 7f1fc3e00700 1 -- 172.21.15.131:0/374138419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562a31137890 msgr2=0x562a31148570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:45.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.404+0000 7f1fc3e00700 1 --2- 172.21.15.131:0/374138419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562a31137890 0x562a31148570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.404+0000 7f1fc3e00700 1 -- 172.21.15.131:0/374138419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a3113d720 msgr2=0x562a311a26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:45.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.404+0000 7f1fc3e00700 1 --2- 172.21.15.131:0/374138419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a3113d720 0x562a311a26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.404+0000 7f1fc3e00700 1 -- 172.21.15.131:0/374138419 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a30eb4b60 con 0x562a3113ce00 2024-05-08T22:15:45.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.405+0000 7f1fc3e00700 1 --2- 172.21.15.131:0/374138419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a3113ce00 0x562a311a20d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009520 tx=0x7f1fb00bbcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f4187ed31814bdf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:45.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.405+0000 7f1faa000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:15:45.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.405+0000 7f1faa000700 1 -- 172.21.15.131:0/374138419 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0036070 con 0x562a3113ce00 2024-05-08T22:15:45.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.405+0000 7f1faa000700 1 -- 172.21.15.131:0/374138419 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f1fb00bbf10 con 0x562a3113ce00 2024-05-08T22:15:45.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.405+0000 7f1fdf165b80 1 -- 172.21.15.131:0/374138419 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1fb801e040 con 0x562a3113ce00 2024-05-08T22:15:45.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.406+0000 7f1faa000700 1 -- 172.21.15.131:0/374138419 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fb00bc0d0 con 0x562a3113ce00 2024-05-08T22:15:45.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.406+0000 7f1fdf165b80 10 client.26572 fetch_fsmap learned FSMap version 29 2024-05-08T22:15:45.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.406+0000 7f1fdf165b80 1 -- 172.21.15.131:0/374138419 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562a3114d0e0 con 0x562a3113ce00 2024-05-08T22:15:45.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.406+0000 7f1faa000700 1 -- 172.21.15.131:0/374138419 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fb801e040 con 0x562a3113ce00 2024-05-08T22:15:45.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.406+0000 7f1faa000700 1 -- 172.21.15.131:0/374138419 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f1fb00c3dc0 con 0x562a3113ce00 2024-05-08T22:15:45.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.406+0000 7f1fdf165b80 10 client.26572 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:15:45.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.406+0000 7f1fdf165b80 10 client.26572 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:15:45.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.406+0000 7f1fdf165b80 20 client.26572 populate_metadata read hostname 'smithi131' 2024-05-08T22:15:45.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.407+0000 7f1fdf165b80 1 --2- 172.21.15.131:0/374138419 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x562a311b87e0 0x562a311bac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.407+0000 7f1fdf165b80 4 client.26572 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:15:45.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.407+0000 7f1fdf165b80 1 -- 172.21.15.131:0/374138419 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562a3107fec0 con 0x562a311b87e0 2024-05-08T22:15:45.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.407+0000 7f1fc8c00700 1 --2- 172.21.15.131:0/374138419 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x562a311b87e0 0x562a311bac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.408+0000 7f1faa000700 10 client.26572 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:15:45.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.408+0000 7f1fc8c00700 1 --2- 172.21.15.131:0/374138419 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x562a311b87e0 0x562a311bac00 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2125f1a17f9e227c server_cookie=1f5f3c6d4d73e1fc in_seq=0 out_seq=0 2024-05-08T22:15:45.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.408+0000 7f1faa000700 1 -- 172.21.15.131:0/374138419 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562a3107fec0 con 0x562a311b87e0 2024-05-08T22:15:45.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.408+0000 7f1faa000700 10 client.26572 handle_command_reply: tid=0 2024-05-08T22:15:45.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.408+0000 7f1fdf165b80 1 client.26572 shutdown 2024-05-08T22:15:45.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.409+0000 7f1fdf165b80 1 -- 172.21.15.131:0/374138419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a3113ce00 msgr2=0x562a311a20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:45.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.409+0000 7f1fdf165b80 1 --2- 172.21.15.131:0/374138419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a3113ce00 0x562a311a20d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009520 tx=0x7f1fb00bbcb0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.409+0000 7f1fdf165b80 1 -- 172.21.15.131:0/374138419 shutdown_connections 2024-05-08T22:15:45.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.409+0000 7f1fdf165b80 1 --2- 172.21.15.131:0/374138419 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x562a311b87e0 0x562a311bac00 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.410+0000 7f1fdf165b80 1 --2- 172.21.15.131:0/374138419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a3113d720 0x562a311a26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.410+0000 7f1faa000700 0 client.26572 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:15:45.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.410+0000 7f1fdf165b80 1 --2- 172.21.15.131:0/374138419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562a31137890 0x562a31148570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.410+0000 7f1fdf165b80 1 --2- 172.21.15.131:0/374138419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a3113ce00 0x562a311a20d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.410+0000 7f1fdf165b80 1 -- 172.21.15.131:0/374138419 >> 172.21.15.131:0/374138419 conn(0x562a31095210 msgr2=0x562a3113f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:45.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.410+0000 7f1fdf165b80 1 -- 172.21.15.131:0/374138419 shutdown_connections 2024-05-08T22:15:45.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.410+0000 7f1fdf165b80 1 -- 172.21.15.131:0/374138419 wait complete. 2024-05-08T22:15:45.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.410+0000 7f1fdf165b80 20 client.26572 trim_cache size 0 max 16384 2024-05-08T22:15:45.412 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:15:45.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.412+0000 7f1fcb600700 1 -- 172.21.15.131:0/1072410161 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1fac0a1460 msgr2=0x7f1fac0a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:45.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.412+0000 7f1fcb600700 1 --2- 172.21.15.131:0/1072410161 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1fac0a1460 0x7f1fac0a3920 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009710 tx=0x7f1fbc009350 comp rx=0 tx=0).stop 2024-05-08T22:15:45.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.412+0000 7f1fcb600700 1 -- 172.21.15.131:0/1072410161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fc4075df0 msgr2=0x7f1fc4103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:45.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.412+0000 7f1fcb600700 1 --2- 172.21.15.131:0/1072410161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fc4075df0 0x7f1fc4103100 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1fb002f990 tx=0x7f1fb002fca0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.412+0000 7f1fcb600700 1 -- 172.21.15.131:0/1072410161 shutdown_connections 2024-05-08T22:15:45.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.412+0000 7f1fcb600700 1 --2- 172.21.15.131:0/1072410161 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1fac0a1460 0x7f1fac0a3920 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.412+0000 7f1fcb600700 1 --2- 172.21.15.131:0/1072410161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fc4111da0 0x7f1fc4103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.412+0000 7f1fcb600700 1 --2- 172.21.15.131:0/1072410161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fc40754d0 0x7f1fc410b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.412+0000 7f1fcb600700 1 --2- 172.21.15.131:0/1072410161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fc4075df0 0x7f1fc4103100 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.412+0000 7f1fcb600700 1 -- 172.21.15.131:0/1072410161 >> 172.21.15.131:0/1072410161 conn(0x7f1fc40febb0 msgr2=0x7f1fc4100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:45.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.412+0000 7f1fcb600700 1 -- 172.21.15.131:0/1072410161 shutdown_connections 2024-05-08T22:15:45.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.412+0000 7f1fcb600700 1 -- 172.21.15.131:0/1072410161 wait complete. 2024-05-08T22:15:45.444 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T22:15:45.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:45 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/4034938823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:15:45.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.785+0000 7fbd8a400700 1 -- 172.21.15.131:0/2907501620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd84108490 msgr2=0x7fbd84110010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:45.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.785+0000 7fbd8a400700 1 --2- 172.21.15.131:0/2907501620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd84108490 0x7fbd84110010 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c009990 tx=0x7fbd7c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.785+0000 7fbd8a400700 1 -- 172.21.15.131:0/2907501620 shutdown_connections 2024-05-08T22:15:45.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.785+0000 7fbd8a400700 1 --2- 172.21.15.131:0/2907501620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd84108490 0x7fbd84110010 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.785+0000 7fbd8a400700 1 --2- 172.21.15.131:0/2907501620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd84103370 0x7fbd84103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.785+0000 7fbd8a400700 1 --2- 172.21.15.131:0/2907501620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd84103d20 0x7fbd84107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.785+0000 7fbd8a400700 1 -- 172.21.15.131:0/2907501620 >> 172.21.15.131:0/2907501620 conn(0x7fbd840febb0 msgr2=0x7fbd84100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:45.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.786+0000 7fbd8a400700 1 -- 172.21.15.131:0/2907501620 shutdown_connections 2024-05-08T22:15:45.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.786+0000 7fbd8a400700 1 -- 172.21.15.131:0/2907501620 wait complete. 2024-05-08T22:15:45.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.787+0000 7fbd8a400700 1 Processor -- start 2024-05-08T22:15:45.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.787+0000 7fbd8a400700 1 -- start start 2024-05-08T22:15:45.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.788+0000 7fbd8a400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd84103370 0x7fbd8410fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.788+0000 7fbd8a400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd84108490 0x7fbd8410ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.788+0000 7fbd8a400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd8410d390 0x7fbd8410d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.788+0000 7fbd8a400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8419f780 con 0x7fbd84108490 2024-05-08T22:15:45.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.788+0000 7fbd8a400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8419f8f0 con 0x7fbd84103370 2024-05-08T22:15:45.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.788+0000 7fbd8a400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8419fab0 con 0x7fbd8410d390 2024-05-08T22:15:45.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.788+0000 7fbd83e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd8410d390 0x7fbd8410d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.788+0000 7fbd83400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd84103370 0x7fbd8410fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.788+0000 7fbd83e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd8410d390 0x7fbd8410d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34862/0 (socket says 172.21.15.131:34862) 2024-05-08T22:15:45.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.788+0000 7fbd83400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd84103370 0x7fbd8410fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:54482/0 (socket says 172.21.15.131:54482) 2024-05-08T22:15:45.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.788+0000 7fbd83e00700 1 -- 172.21.15.131:0/2305203033 learned_addr learned my addr 172.21.15.131:0/2305203033 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:45.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.788+0000 7fbd7aa00700 1 --2- 172.21.15.131:0/2305203033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd84108490 0x7fbd8410ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.789+0000 7fbd83e00700 1 -- 172.21.15.131:0/2305203033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd84103370 msgr2=0x7fbd8410fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:45.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.789+0000 7fbd83e00700 1 --2- 172.21.15.131:0/2305203033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd84103370 0x7fbd8410fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.789+0000 7fbd83e00700 1 -- 172.21.15.131:0/2305203033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd84108490 msgr2=0x7fbd8410ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:45.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.789+0000 7fbd83e00700 1 --2- 172.21.15.131:0/2305203033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd84108490 0x7fbd8410ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.789+0000 7fbd83e00700 1 -- 172.21.15.131:0/2305203033 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7c0095f0 con 0x7fbd8410d390 2024-05-08T22:15:45.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.789+0000 7fbd83400700 1 --2- 172.21.15.131:0/2305203033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd84103370 0x7fbd8410fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:15:45.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.789+0000 7fbd83e00700 1 --2- 172.21.15.131:0/2305203033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd8410d390 0x7fbd8410d840 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c005b40 tx=0x7fbd7c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:45.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.790+0000 7fbd80c00700 1 -- 172.21.15.131:0/2305203033 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c03f070 con 0x7fbd8410d390 2024-05-08T22:15:45.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.790+0000 7fbd80c00700 1 -- 172.21.15.131:0/2305203033 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd7c043410 con 0x7fbd8410d390 2024-05-08T22:15:45.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.790+0000 7fbd80c00700 1 -- 172.21.15.131:0/2305203033 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c039660 con 0x7fbd8410d390 2024-05-08T22:15:45.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.790+0000 7fbd8a400700 1 -- 172.21.15.131:0/2305203033 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd8419fd30 con 0x7fbd8410d390 2024-05-08T22:15:45.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.790+0000 7fbd8a400700 1 -- 172.21.15.131:0/2305203033 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd841a0190 con 0x7fbd8410d390 2024-05-08T22:15:45.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.795+0000 7fbd9de8cb80 1 Processor -- start 2024-05-08T22:15:45.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.795+0000 7fbd80c00700 1 -- 172.21.15.131:0/2305203033 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fbd7c0397c0 con 0x7fbd8410d390 2024-05-08T22:15:45.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.798+0000 7fbd80c00700 1 --2- 172.21.15.131:0/2305203033 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fbd680a1310 0x7fbd680a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.798+0000 7fbd80c00700 1 -- 172.21.15.131:0/2305203033 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fbd7c0f0350 con 0x7fbd8410d390 2024-05-08T22:15:45.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.799+0000 7fbd7aa00700 1 --2- 172.21.15.131:0/2305203033 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fbd680a1310 0x7fbd680a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.800+0000 7fbd9de8cb80 1 -- start start 2024-05-08T22:15:45.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.800+0000 7fbd7aa00700 1 --2- 172.21.15.131:0/2305203033 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fbd680a1310 0x7fbd680a37d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbd8410e530 tx=0x7fbd6c009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:45.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.801+0000 7fbd9de8cb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390667e00 0x55b3906681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.801+0000 7fbd9de8cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b390668720 0x55b390662350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.801+0000 7fbd9de8cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b390662890 0x55b390662cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.801+0000 7fbd9de8cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3905b9150 con 0x55b390662890 2024-05-08T22:15:45.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.801+0000 7fbd9de8cb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3905b6b10 con 0x55b390667e00 2024-05-08T22:15:45.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.801+0000 7fbd9de8cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3905b2400 con 0x55b390668720 2024-05-08T22:15:45.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.801+0000 7fbd83400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390667e00 0x55b3906681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.801+0000 7fbd83400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390667e00 0x55b3906681e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:54486/0 (socket says 172.21.15.131:54486) 2024-05-08T22:15:45.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.801+0000 7fbd83400700 1 -- 172.21.15.131:0/2407095476 learned_addr learned my addr 172.21.15.131:0/2407095476 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:45.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.801+0000 7fbd7aa00700 1 --2- 172.21.15.131:0/2407095476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b390668720 0x55b390662350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.802+0000 7fbd7aa00700 1 -- 172.21.15.131:0/2407095476 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390667e00 msgr2=0x55b3906681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:45.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.802+0000 7fbd7aa00700 1 --2- 172.21.15.131:0/2407095476 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390667e00 0x55b3906681e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.802+0000 7fbd7aa00700 1 -- 172.21.15.131:0/2407095476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b390662890 msgr2=0x55b390662cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:45.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.802+0000 7fbd83e00700 1 --2- 172.21.15.131:0/2407095476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b390662890 0x55b390662cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.802+0000 7fbd7aa00700 1 --2- 172.21.15.131:0/2407095476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b390662890 0x55b390662cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.802+0000 7fbd7aa00700 1 -- 172.21.15.131:0/2407095476 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3904333b0 con 0x55b390668720 2024-05-08T22:15:45.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.802+0000 7fbd83400700 1 --2- 172.21.15.131:0/2407095476 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390667e00 0x55b3906681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:15:45.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.803+0000 7fbd7aa00700 1 --2- 172.21.15.131:0/2407095476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b390668720 0x55b390662350 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009fd0 tx=0x7fbd6c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17875aae9087dbba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:45.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.803+0000 7fbd83e00700 1 --2- 172.21.15.131:0/2407095476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b390662890 0x55b390662cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:15:45.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.803+0000 7fbd78c00700 1 -- 172.21.15.131:0/2407095476 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c0119b0 con 0x55b390668720 2024-05-08T22:15:45.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.803+0000 7fbd78c00700 1 -- 172.21.15.131:0/2407095476 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd6c0044e0 con 0x55b390668720 2024-05-08T22:15:45.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.803+0000 7fbd78c00700 1 -- 172.21.15.131:0/2407095476 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c003a10 con 0x55b390668720 2024-05-08T22:15:45.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.804+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/2407095476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b390668720 msgr2=0x55b390662350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:45.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.804+0000 7fbd9de8cb80 1 --2- 172.21.15.131:0/2407095476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b390668720 0x55b390662350 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009fd0 tx=0x7fbd6c014000 comp rx=0 tx=0).stop 2024-05-08T22:15:45.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.805+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/2407095476 shutdown_connections 2024-05-08T22:15:45.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.805+0000 7fbd9de8cb80 1 --2- 172.21.15.131:0/2407095476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b390662890 0x55b390662cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.805+0000 7fbd9de8cb80 1 --2- 172.21.15.131:0/2407095476 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390667e00 0x55b3906681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.805+0000 7fbd9de8cb80 1 --2- 172.21.15.131:0/2407095476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b390668720 0x55b390662350 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.805+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/2407095476 >> 172.21.15.131:0/2407095476 conn(0x55b3905c0210 msgr2=0x55b39065cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:45.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.806+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/2407095476 shutdown_connections 2024-05-08T22:15:45.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.806+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/2407095476 wait complete. 2024-05-08T22:15:45.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.807+0000 7fbd9de8cb80 1 -- start start 2024-05-08T22:15:45.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.808+0000 7fbd9de8cb80 1 Processor -- start 2024-05-08T22:15:45.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.808+0000 7fbd9de8cb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390662890 0x55b39066bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.808+0000 7fbd9de8cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b390667e00 0x55b39066c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.808+0000 7fbd9de8cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b390668720 0x55b39066c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.808+0000 7fbd9de8cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b39054e560 con 0x55b390668720 2024-05-08T22:15:45.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.808+0000 7fbd9de8cb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b39054e200 con 0x55b390662890 2024-05-08T22:15:45.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.808+0000 7fbd9de8cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3904356a0 con 0x55b390667e00 2024-05-08T22:15:45.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.808+0000 7fbd9de8cb80 10 client.0 _ll_register_callbacks cb 0x55b390557b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:15:45.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.809+0000 7fbd83400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390662890 0x55b39066bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.809+0000 7fbd83400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390662890 0x55b39066bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:54488/0 (socket says 172.21.15.131:54488) 2024-05-08T22:15:45.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.809+0000 7fbd83400700 1 -- 172.21.15.131:0/303082213 learned_addr learned my addr 172.21.15.131:0/303082213 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:45.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.809+0000 7fbd7aa00700 1 --2- 172.21.15.131:0/303082213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b390667e00 0x55b39066c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.810+0000 7fbd83e00700 1 --2- 172.21.15.131:0/303082213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b390668720 0x55b39066c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.810+0000 7fbd83400700 1 -- 172.21.15.131:0/303082213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b390667e00 msgr2=0x55b39066c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:45.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.810+0000 7fbd83400700 1 --2- 172.21.15.131:0/303082213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b390667e00 0x55b39066c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.810+0000 7fbd83400700 1 -- 172.21.15.131:0/303082213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b390668720 msgr2=0x55b39066c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:45.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.810+0000 7fbd83400700 1 --2- 172.21.15.131:0/303082213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b390668720 0x55b39066c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.810+0000 7fbd83400700 1 -- 172.21.15.131:0/303082213 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd74007500 con 0x55b390662890 2024-05-08T22:15:45.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.810+0000 7fbd83e00700 1 --2- 172.21.15.131:0/303082213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b390668720 0x55b39066c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:15:45.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.811+0000 7fbd83400700 1 --2- 172.21.15.131:0/303082213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390662890 0x55b39066bb60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbd74013560 tx=0x7fbd74013590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d628882ba5690b3c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:45.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.812+0000 7fbd71600700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:15:45.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.812+0000 7fbd71600700 1 -- 172.21.15.131:0/303082213 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd74016630 con 0x55b390662890 2024-05-08T22:15:45.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.812+0000 7fbd71600700 1 -- 172.21.15.131:0/303082213 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fbd74013e70 con 0x55b390662890 2024-05-08T22:15:45.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.812+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/303082213 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b39050b270 con 0x55b390662890 2024-05-08T22:15:45.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.812+0000 7fbd71600700 1 -- 172.21.15.131:0/303082213 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd74016790 con 0x55b390662890 2024-05-08T22:15:45.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.812+0000 7fbd9de8cb80 10 client.26498 fetch_fsmap learned FSMap version 29 2024-05-08T22:15:45.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.812+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/303082213 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fbd6c017070 con 0x55b390662890 2024-05-08T22:15:45.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.813+0000 7fbd71600700 1 -- 172.21.15.131:0/303082213 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b39050b270 con 0x55b390662890 2024-05-08T22:15:45.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.814+0000 7fbd71600700 1 -- 172.21.15.131:0/303082213 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fbd74015ab0 con 0x55b390662890 2024-05-08T22:15:45.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.814+0000 7fbd9de8cb80 10 client.26498 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:15:45.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.814+0000 7fbd9de8cb80 10 client.26498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:15:45.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.814+0000 7fbd9de8cb80 20 client.26498 populate_metadata read hostname 'smithi131' 2024-05-08T22:15:45.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.814+0000 7fbd9de8cb80 1 --2- 172.21.15.131:0/303082213 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b39067c0e0 0x55b39067e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.814+0000 7fbd9de8cb80 4 client.26498 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:15:45.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.814+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/303082213 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b390631810 con 0x55b39067c0e0 2024-05-08T22:15:45.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.815+0000 7fbd7aa00700 1 --2- 172.21.15.131:0/303082213 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b39067c0e0 0x55b39067e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.823+0000 7fbd7aa00700 1 --2- 172.21.15.131:0/303082213 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b39067c0e0 0x55b39067e500 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e38bd1a652cf277 server_cookie=90b06d5b9af7cee8 in_seq=0 out_seq=0 2024-05-08T22:15:45.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.823+0000 7fbd71600700 10 client.26498 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:45.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.827+0000 7fbd71600700 1 -- 172.21.15.131:0/303082213 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b390631810 con 0x55b39067c0e0 2024-05-08T22:15:45.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.827+0000 7fbd71600700 10 client.26498 handle_command_reply: tid=0 2024-05-08T22:15:45.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.827+0000 7fbd9de8cb80 1 client.26498 shutdown 2024-05-08T22:15:45.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.828+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/303082213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390662890 msgr2=0x55b39066bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:45.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.828+0000 7fbd9de8cb80 1 --2- 172.21.15.131:0/303082213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390662890 0x55b39066bb60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbd74013560 tx=0x7fbd74013590 comp rx=0 tx=0).stop 2024-05-08T22:15:45.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.828+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/303082213 shutdown_connections 2024-05-08T22:15:45.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.828+0000 7fbd9de8cb80 1 --2- 172.21.15.131:0/303082213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b390668720 0x55b39066c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.828+0000 7fbd9de8cb80 1 --2- 172.21.15.131:0/303082213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390662890 0x55b39066bb60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.828+0000 7fbd9de8cb80 1 --2- 172.21.15.131:0/303082213 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b39067c0e0 0x55b39067e500 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.828+0000 7fbd9de8cb80 1 --2- 172.21.15.131:0/303082213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b390667e00 0x55b39066c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.828+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/303082213 >> 172.21.15.131:0/303082213 conn(0x55b3905c0210 msgr2=0x55b39065c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:45.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.828+0000 7fbd71600700 0 client.26498 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:45.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.829+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/303082213 shutdown_connections 2024-05-08T22:15:45.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.829+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/303082213 wait complete. 2024-05-08T22:15:45.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.829+0000 7fbd9de8cb80 20 client.26498 trim_cache size 0 max 16384 2024-05-08T22:15:45.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:45 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/4034938823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:15:45.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.857+0000 7fbd9de8cb80 1 Processor -- start 2024-05-08T22:15:45.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.857+0000 7fbd9de8cb80 1 -- start start 2024-05-08T22:15:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.858+0000 7fbd9de8cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b390662890 0x55b390662350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.858+0000 7fbd9de8cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b390667e00 0x55b39067e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.858+0000 7fbd9de8cb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390668720 0x55b390677c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.858+0000 7fbd83e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390668720 0x55b390677c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.858+0000 7fbd83e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390668720 0x55b390677c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:54498/0 (socket says 172.21.15.131:54498) 2024-05-08T22:15:45.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.858+0000 7fbd83e00700 1 -- 172.21.15.131:0/3542049803 learned_addr learned my addr 172.21.15.131:0/3542049803 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:45.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.858+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/3542049803 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b390659dc0 con 0x55b390662890 2024-05-08T22:15:45.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.858+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/3542049803 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b39065ee50 con 0x55b390668720 2024-05-08T22:15:45.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.858+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/3542049803 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b39065ea00 con 0x55b390667e00 2024-05-08T22:15:45.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.859+0000 7fbd7aa00700 1 --2- 172.21.15.131:0/3542049803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b390667e00 0x55b39067e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.859+0000 7fbd83e00700 1 -- 172.21.15.131:0/3542049803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b390667e00 msgr2=0x55b39067e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:45.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.859+0000 7fbd83e00700 1 --2- 172.21.15.131:0/3542049803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b390667e00 0x55b39067e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.859+0000 7fbd83400700 1 --2- 172.21.15.131:0/3542049803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b390662890 0x55b390662350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.859+0000 7fbd83e00700 1 -- 172.21.15.131:0/3542049803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b390662890 msgr2=0x55b390662350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:45.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.859+0000 7fbd83e00700 1 --2- 172.21.15.131:0/3542049803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b390662890 0x55b390662350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.859+0000 7fbd83e00700 1 -- 172.21.15.131:0/3542049803 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3903dfb60 con 0x55b390668720 2024-05-08T22:15:45.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.859+0000 7fbd83400700 1 --2- 172.21.15.131:0/3542049803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b390662890 0x55b390662350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:15:45.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.861+0000 7fbd83e00700 1 --2- 172.21.15.131:0/3542049803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390668720 0x55b390677c20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0ed870 tx=0x7fbd7c03f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24ba4d4c73692696 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.861+0000 7fbd79600700 1 -- 172.21.15.131:0/3542049803 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c0bd910 con 0x55b390668720 2024-05-08T22:15:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.862+0000 7fbd79600700 1 -- 172.21.15.131:0/3542049803 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd7c0f0170 con 0x55b390668720 2024-05-08T22:15:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.862+0000 7fbd79600700 1 -- 172.21.15.131:0/3542049803 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c0c3310 con 0x55b390668720 2024-05-08T22:15:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.862+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/3542049803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390668720 msgr2=0x55b390677c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.862+0000 7fbd9de8cb80 1 --2- 172.21.15.131:0/3542049803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390668720 0x55b390677c20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0ed870 tx=0x7fbd7c03f000 comp rx=0 tx=0).stop 2024-05-08T22:15:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.863+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/3542049803 shutdown_connections 2024-05-08T22:15:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.863+0000 7fbd9de8cb80 1 --2- 172.21.15.131:0/3542049803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390668720 0x55b390677c20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.863+0000 7fbd9de8cb80 1 --2- 172.21.15.131:0/3542049803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b390662890 0x55b390662350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.863+0000 7fbd9de8cb80 1 --2- 172.21.15.131:0/3542049803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b390667e00 0x55b39067e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.863+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/3542049803 >> 172.21.15.131:0/3542049803 conn(0x55b3905c0210 msgr2=0x55b39066a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:45.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.863+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/3542049803 shutdown_connections 2024-05-08T22:15:45.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.864+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/3542049803 wait complete. 2024-05-08T22:15:45.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.865+0000 7fbd9de8cb80 1 -- start start 2024-05-08T22:15:45.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.865+0000 7fbd9de8cb80 1 Processor -- start 2024-05-08T22:15:45.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.867+0000 7fbd9de8cb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390662890 0x55b390673360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.867+0000 7fbd9de8cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b390667e00 0x55b390671150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.867+0000 7fbd9de8cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b390668720 0x55b390671690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.867+0000 7fbd9de8cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b390671bd0 con 0x55b390668720 2024-05-08T22:15:45.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.867+0000 7fbd9de8cb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b390671d40 con 0x55b390662890 2024-05-08T22:15:45.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.867+0000 7fbd9de8cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b390671f00 con 0x55b390667e00 2024-05-08T22:15:45.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.868+0000 7fbd9de8cb80 10 client.0 _ll_register_callbacks cb 0x55b390557b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:15:45.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.868+0000 7fbd7aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b390667e00 0x55b390671150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.868+0000 7fbd83400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390662890 0x55b390673360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.868+0000 7fbd7aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b390667e00 0x55b390671150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34918/0 (socket says 172.21.15.131:34918) 2024-05-08T22:15:45.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.868+0000 7fbd7aa00700 1 -- 172.21.15.131:0/3348796974 learned_addr learned my addr 172.21.15.131:0/3348796974 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:45.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.868+0000 7fbd83400700 1 --2- 172.21.15.131:0/3348796974 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390662890 0x55b390673360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:54514/0 (socket says 172.21.15.131:54514) 2024-05-08T22:15:45.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.869+0000 7fbd83400700 1 -- 172.21.15.131:0/3348796974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b390667e00 msgr2=0x55b390671150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:45.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.869+0000 7fbd83400700 1 --2- 172.21.15.131:0/3348796974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b390667e00 0x55b390671150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.869+0000 7fbd83400700 1 -- 172.21.15.131:0/3348796974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b390668720 msgr2=0x55b390671690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:45.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.869+0000 7fbd83e00700 1 --2- 172.21.15.131:0/3348796974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b390668720 0x55b390671690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.869+0000 7fbd83400700 1 --2- 172.21.15.131:0/3348796974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b390668720 0x55b390671690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.869+0000 7fbd83400700 1 -- 172.21.15.131:0/3348796974 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd841a0190 con 0x55b390662890 2024-05-08T22:15:45.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.870+0000 7fbd83400700 1 --2- 172.21.15.131:0/3348796974 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390662890 0x55b390673360 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbd74000f80 tx=0x7fbd74009850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81ab089c80dae34a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:45.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.870+0000 7fbd71600700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:15:45.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.872+0000 7fbd71600700 1 -- 172.21.15.131:0/3348796974 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd74025070 con 0x55b390662890 2024-05-08T22:15:45.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.872+0000 7fbd71600700 1 -- 172.21.15.131:0/3348796974 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fbd74013e70 con 0x55b390662890 2024-05-08T22:15:45.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.872+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/3348796974 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd6c0044e0 con 0x55b390662890 2024-05-08T22:15:45.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.873+0000 7fbd71600700 1 -- 172.21.15.131:0/3348796974 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd74006000 con 0x55b390662890 2024-05-08T22:15:45.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.873+0000 7fbd9de8cb80 10 client.26504 fetch_fsmap learned FSMap version 29 2024-05-08T22:15:45.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.873+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/3348796974 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b390578560 con 0x55b390662890 2024-05-08T22:15:45.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.874+0000 7fbd71600700 1 -- 172.21.15.131:0/3348796974 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd6c0044e0 con 0x55b390662890 2024-05-08T22:15:45.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.874+0000 7fbd71600700 1 -- 172.21.15.131:0/3348796974 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fbd7401f710 con 0x55b390662890 2024-05-08T22:15:45.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.874+0000 7fbd9de8cb80 10 client.26504 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:15:45.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.874+0000 7fbd9de8cb80 10 client.26504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:15:45.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.874+0000 7fbd9de8cb80 20 client.26504 populate_metadata read hostname 'smithi131' 2024-05-08T22:15:45.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.874+0000 7fbd9de8cb80 1 --2- 172.21.15.131:0/3348796974 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b3906d29d0 0x55b3906d4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:45.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.874+0000 7fbd9de8cb80 4 client.26504 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:15:45.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.874+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/3348796974 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b3905aaec0 con 0x55b3906d29d0 2024-05-08T22:15:45.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.875+0000 7fbd7aa00700 1 --2- 172.21.15.131:0/3348796974 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b3906d29d0 0x55b3906d4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:45.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.878+0000 7fbd7aa00700 1 --2- 172.21.15.131:0/3348796974 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b3906d29d0 0x55b3906d4df0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e02ec10d3a512cd9 server_cookie=ab457b28901e1014 in_seq=0 out_seq=0 2024-05-08T22:15:45.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.878+0000 7fbd71600700 10 client.26504 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:45.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.880+0000 7fbd71600700 1 -- 172.21.15.131:0/3348796974 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b3905aaec0 con 0x55b3906d29d0 2024-05-08T22:15:45.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.882+0000 7fbd71600700 10 client.26504 handle_command_reply: tid=0 2024-05-08T22:15:45.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.882+0000 7fbd9de8cb80 1 client.26504 shutdown 2024-05-08T22:15:45.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.883+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/3348796974 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390662890 msgr2=0x55b390673360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:45.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.883+0000 7fbd9de8cb80 1 --2- 172.21.15.131:0/3348796974 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390662890 0x55b390673360 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbd74000f80 tx=0x7fbd74009850 comp rx=0 tx=0).stop 2024-05-08T22:15:45.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.884+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/3348796974 shutdown_connections 2024-05-08T22:15:45.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.884+0000 7fbd9de8cb80 1 --2- 172.21.15.131:0/3348796974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b390668720 0x55b390671690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.884+0000 7fbd9de8cb80 1 --2- 172.21.15.131:0/3348796974 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b390662890 0x55b390673360 secure :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbd74000f80 tx=0x7fbd74009850 comp rx=0 tx=0).stop 2024-05-08T22:15:45.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.884+0000 7fbd9de8cb80 1 --2- 172.21.15.131:0/3348796974 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b3906d29d0 0x55b3906d4df0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.884+0000 7fbd9de8cb80 1 --2- 172.21.15.131:0/3348796974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b390667e00 0x55b390671150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.884+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/3348796974 >> 172.21.15.131:0/3348796974 conn(0x55b3905c0210 msgr2=0x55b390666c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:45.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.884+0000 7fbd71600700 0 client.26504 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:45.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.885+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/3348796974 shutdown_connections 2024-05-08T22:15:45.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.885+0000 7fbd9de8cb80 1 -- 172.21.15.131:0/3348796974 wait complete. 2024-05-08T22:15:45.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.886+0000 7fbd9de8cb80 20 client.26504 trim_cache size 0 max 16384 2024-05-08T22:15:45.887 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:15:45.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.887+0000 7fbd8a400700 1 -- 172.21.15.131:0/2305203033 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fbd680a1310 msgr2=0x7fbd680a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:45.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.887+0000 7fbd8a400700 1 --2- 172.21.15.131:0/2305203033 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fbd680a1310 0x7fbd680a37d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbd8410e530 tx=0x7fbd6c009260 comp rx=0 tx=0).stop 2024-05-08T22:15:45.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.887+0000 7fbd8a400700 1 -- 172.21.15.131:0/2305203033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd8410d390 msgr2=0x7fbd8410d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:45.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.887+0000 7fbd8a400700 1 --2- 172.21.15.131:0/2305203033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd8410d390 0x7fbd8410d840 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c005b40 tx=0x7fbd7c005740 comp rx=0 tx=0).stop 2024-05-08T22:15:45.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.888+0000 7fbd8a400700 1 -- 172.21.15.131:0/2305203033 shutdown_connections 2024-05-08T22:15:45.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.888+0000 7fbd8a400700 1 --2- 172.21.15.131:0/2305203033 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fbd680a1310 0x7fbd680a37d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.888+0000 7fbd8a400700 1 --2- 172.21.15.131:0/2305203033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd8410d390 0x7fbd8410d840 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.888+0000 7fbd8a400700 1 --2- 172.21.15.131:0/2305203033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd84103370 0x7fbd8410fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.888+0000 7fbd8a400700 1 --2- 172.21.15.131:0/2305203033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd84108490 0x7fbd8410ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:45.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.888+0000 7fbd8a400700 1 -- 172.21.15.131:0/2305203033 >> 172.21.15.131:0/2305203033 conn(0x7fbd840febb0 msgr2=0x7fbd84100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:45.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.889+0000 7fbd8a400700 1 -- 172.21.15.131:0/2305203033 shutdown_connections 2024-05-08T22:15:45.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:45.890+0000 7fbd8a400700 1 -- 172.21.15.131:0/2305203033 wait complete. 2024-05-08T22:15:45.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:45 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/4034938823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:15:46.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:46 smithi157 ceph-mon[77746]: pgmap v261: 129 pgs: 129 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.9 MiB/s wr, 1.79k op/s 2024-05-08T22:15:46.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:46 smithi132 ceph-mon[78180]: pgmap v261: 129 pgs: 129 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.9 MiB/s wr, 1.79k op/s 2024-05-08T22:15:46.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:46 smithi131 ceph-mon[37260]: pgmap v261: 129 pgs: 129 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.9 MiB/s wr, 1.79k op/s 2024-05-08T22:15:46.922 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T22:15:46.922 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T22:15:47.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.242+0000 7f4df8800700 1 -- 172.21.15.131:0/2228954325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4df0106280 msgr2=0x7f4df010de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:47.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.242+0000 7f4df8800700 1 --2- 172.21.15.131:0/2228954325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4df0106280 0x7f4df010de00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00b0a0 tx=0x7f4dec00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.242+0000 7f4df8800700 1 -- 172.21.15.131:0/2228954325 shutdown_connections 2024-05-08T22:15:47.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.242+0000 7f4df8800700 1 --2- 172.21.15.131:0/2228954325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4df0106280 0x7f4df010de00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.242+0000 7f4df8800700 1 --2- 172.21.15.131:0/2228954325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4df0101160 0x7f4df0101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.242+0000 7f4df8800700 1 --2- 172.21.15.131:0/2228954325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4df0101b10 0x7f4df0105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.242+0000 7f4df8800700 1 -- 172.21.15.131:0/2228954325 >> 172.21.15.131:0/2228954325 conn(0x7f4df00fc9c0 msgr2=0x7f4df00fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:47.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.242+0000 7f4df8800700 1 -- 172.21.15.131:0/2228954325 shutdown_connections 2024-05-08T22:15:47.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.243+0000 7f4df8800700 1 -- 172.21.15.131:0/2228954325 wait complete. 2024-05-08T22:15:47.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.244+0000 7f4df8800700 1 Processor -- start 2024-05-08T22:15:47.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.244+0000 7f4df8800700 1 -- start start 2024-05-08T22:15:47.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.245+0000 7f4df8800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4df0101160 0x7f4df0199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.245+0000 7f4df8800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4df0106280 0x7f4df0199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.245+0000 7f4df8800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4df019de10 0x7f4df019e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.245+0000 7f4df8800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df019e890 con 0x7f4df0106280 2024-05-08T22:15:47.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.245+0000 7f4df8800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df019ea00 con 0x7f4df0101160 2024-05-08T22:15:47.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.245+0000 7f4df8800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df019ebc0 con 0x7f4df019de10 2024-05-08T22:15:47.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.245+0000 7f4df5e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4df0101160 0x7f4df0199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.245+0000 7f4df5400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4df0106280 0x7f4df0199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.245+0000 7f4df5e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4df0101160 0x7f4df0199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:54532/0 (socket says 172.21.15.131:54532) 2024-05-08T22:15:47.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.245+0000 7f4df5e00700 1 -- 172.21.15.131:0/2981624634 learned_addr learned my addr 172.21.15.131:0/2981624634 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:47.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.245+0000 7f4df6800700 1 --2- 172.21.15.131:0/2981624634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4df019de10 0x7f4df019e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.245+0000 7f4df5e00700 1 -- 172.21.15.131:0/2981624634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4df019de10 msgr2=0x7f4df019e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:47.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.245+0000 7f4df5e00700 1 --2- 172.21.15.131:0/2981624634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4df019de10 0x7f4df019e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.245+0000 7f4df5e00700 1 -- 172.21.15.131:0/2981624634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4df0106280 msgr2=0x7f4df0199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:47.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.245+0000 7f4df5e00700 1 --2- 172.21.15.131:0/2981624634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4df0106280 0x7f4df0199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.245+0000 7f4df5e00700 1 -- 172.21.15.131:0/2981624634 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de0009520 con 0x7f4df0101160 2024-05-08T22:15:47.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.245+0000 7f4df5400700 1 --2- 172.21.15.131:0/2981624634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4df0106280 0x7f4df0199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:15:47.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.245+0000 7f4df5e00700 1 --2- 172.21.15.131:0/2981624634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4df0101160 0x7f4df0199580 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4de400e9c0 tx=0x7f4de400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:47.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.246+0000 7f4deaa00700 1 -- 172.21.15.131:0/2981624634 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de40097b0 con 0x7f4df0101160 2024-05-08T22:15:47.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.246+0000 7f4deaa00700 1 -- 172.21.15.131:0/2981624634 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4de4004cf0 con 0x7f4df0101160 2024-05-08T22:15:47.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.246+0000 7f4df8800700 1 -- 172.21.15.131:0/2981624634 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dec009d00 con 0x7f4df0101160 2024-05-08T22:15:47.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.246+0000 7f4deaa00700 1 -- 172.21.15.131:0/2981624634 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de4005600 con 0x7f4df0101160 2024-05-08T22:15:47.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.246+0000 7f4df8800700 1 -- 172.21.15.131:0/2981624634 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df01a6420 con 0x7f4df0101160 2024-05-08T22:15:47.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.249+0000 7f4deaa00700 1 -- 172.21.15.131:0/2981624634 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f4de400cc60 con 0x7f4df0101160 2024-05-08T22:15:47.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.250+0000 7f4e0c2cfb80 1 Processor -- start 2024-05-08T22:15:47.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.250+0000 7f4deaa00700 1 --2- 172.21.15.131:0/2981624634 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f4dd80a1290 0x7f4dd80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.250+0000 7f4e0c2cfb80 1 -- start start 2024-05-08T22:15:47.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.251+0000 7f4deaa00700 1 -- 172.21.15.131:0/2981624634 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f4de4014070 con 0x7f4df0101160 2024-05-08T22:15:47.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.251+0000 7f4df6800700 1 --2- 172.21.15.131:0/2981624634 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f4dd80a1290 0x7f4dd80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.251+0000 7f4e0c2cfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f1410e80 0x5567f1411260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.251+0000 7f4e0c2cfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567f14117a0 0x5567f140b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.251+0000 7f4e0c2cfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f140b910 0x5567f140bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.251+0000 7f4e0c2cfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567f13621d0 con 0x5567f140b910 2024-05-08T22:15:47.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.251+0000 7f4e0c2cfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567f135fb90 con 0x5567f14117a0 2024-05-08T22:15:47.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.251+0000 7f4e0c2cfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567f135b480 con 0x5567f1410e80 2024-05-08T22:15:47.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.251+0000 7f4df5e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f1410e80 0x5567f1411260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.251+0000 7f4df5e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f1410e80 0x5567f1411260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34946/0 (socket says 172.21.15.131:34946) 2024-05-08T22:15:47.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.251+0000 7f4df5e00700 1 -- 172.21.15.131:0/3773727568 learned_addr learned my addr 172.21.15.131:0/3773727568 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:47.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.251+0000 7f4df6800700 1 --2- 172.21.15.131:0/3773727568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f140b910 0x5567f140bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.252+0000 7f4df5400700 1 --2- 172.21.15.131:0/3773727568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567f14117a0 0x5567f140b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.252+0000 7f4df5e00700 1 -- 172.21.15.131:0/3773727568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567f14117a0 msgr2=0x5567f140b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:47.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.252+0000 7f4df5e00700 1 --2- 172.21.15.131:0/3773727568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567f14117a0 0x5567f140b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.252+0000 7f4df5e00700 1 -- 172.21.15.131:0/3773727568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f140b910 msgr2=0x5567f140bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:47.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.252+0000 7f4df5e00700 1 --2- 172.21.15.131:0/3773727568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f140b910 0x5567f140bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.252+0000 7f4df5e00700 1 -- 172.21.15.131:0/3773727568 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567f11dc430 con 0x5567f1410e80 2024-05-08T22:15:47.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.252+0000 7f4df6800700 1 --2- 172.21.15.131:0/2981624634 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f4dd80a1290 0x7f4dd80a3750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4dec009610 tx=0x7f4dec007ca0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:47.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.252+0000 7f4df5e00700 1 --2- 172.21.15.131:0/3773727568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f1410e80 0x5567f1411260 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4de40a2000 tx=0x7f4de4099a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36b734b790efdef7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:47.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.252+0000 7f4ddf000700 1 -- 172.21.15.131:0/3773727568 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de40b1070 con 0x5567f1410e80 2024-05-08T22:15:47.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.252+0000 7f4ddf000700 1 -- 172.21.15.131:0/3773727568 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4de409a0b0 con 0x5567f1410e80 2024-05-08T22:15:47.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.252+0000 7f4ddf000700 1 -- 172.21.15.131:0/3773727568 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de40967c0 con 0x5567f1410e80 2024-05-08T22:15:47.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.253+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/3773727568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f1410e80 msgr2=0x5567f1411260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:47.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.253+0000 7f4e0c2cfb80 1 --2- 172.21.15.131:0/3773727568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f1410e80 0x5567f1411260 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4de40a2000 tx=0x7f4de4099a90 comp rx=0 tx=0).stop 2024-05-08T22:15:47.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.253+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/3773727568 shutdown_connections 2024-05-08T22:15:47.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.253+0000 7f4e0c2cfb80 1 --2- 172.21.15.131:0/3773727568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f140b910 0x5567f140bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.253+0000 7f4e0c2cfb80 1 --2- 172.21.15.131:0/3773727568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f1410e80 0x5567f1411260 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.253+0000 7f4e0c2cfb80 1 --2- 172.21.15.131:0/3773727568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567f14117a0 0x5567f140b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.253+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/3773727568 >> 172.21.15.131:0/3773727568 conn(0x5567f1369290 msgr2=0x5567f1405e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:47.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.253+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/3773727568 shutdown_connections 2024-05-08T22:15:47.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.253+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/3773727568 wait complete. 2024-05-08T22:15:47.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.253+0000 7f4e0c2cfb80 1 -- start start 2024-05-08T22:15:47.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.254+0000 7f4e0c2cfb80 1 Processor -- start 2024-05-08T22:15:47.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.254+0000 7f4e0c2cfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f140b910 0x5567f1414c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.254+0000 7f4e0c2cfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567f1410e80 0x5567f1415190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.254+0000 7f4e0c2cfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f14117a0 0x5567f14156d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.254+0000 7f4e0c2cfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567f12f75e0 con 0x5567f14117a0 2024-05-08T22:15:47.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.254+0000 7f4e0c2cfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567f12f7280 con 0x5567f1410e80 2024-05-08T22:15:47.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.254+0000 7f4e0c2cfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567f11de720 con 0x5567f140b910 2024-05-08T22:15:47.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.254+0000 7f4e0c2cfb80 10 client.0 _ll_register_callbacks cb 0x5567f1300bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:15:47.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.255+0000 7f4df5400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567f1410e80 0x5567f1415190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.255+0000 7f4df5400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567f1410e80 0x5567f1415190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:54560/0 (socket says 172.21.15.131:54560) 2024-05-08T22:15:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.255+0000 7f4df5400700 1 -- 172.21.15.131:0/2187072555 learned_addr learned my addr 172.21.15.131:0/2187072555 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.255+0000 7f4df5e00700 1 --2- 172.21.15.131:0/2187072555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f140b910 0x5567f1414c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.255+0000 7f4df6800700 1 --2- 172.21.15.131:0/2187072555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f14117a0 0x5567f14156d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.255+0000 7f4df5e00700 1 -- 172.21.15.131:0/2187072555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567f1410e80 msgr2=0x5567f1415190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.255+0000 7f4df5e00700 1 --2- 172.21.15.131:0/2187072555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567f1410e80 0x5567f1415190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.255+0000 7f4df5e00700 1 -- 172.21.15.131:0/2187072555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f14117a0 msgr2=0x5567f14156d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.255+0000 7f4df5e00700 1 --2- 172.21.15.131:0/2187072555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f14117a0 0x5567f14156d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.255+0000 7f4df5e00700 1 -- 172.21.15.131:0/2187072555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df01a6420 con 0x5567f140b910 2024-05-08T22:15:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.255+0000 7f4df5400700 1 --2- 172.21.15.131:0/2187072555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567f1410e80 0x5567f1415190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:15:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.255+0000 7f4df5e00700 1 --2- 172.21.15.131:0/2187072555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f140b910 0x5567f1414c50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4de4009fd0 tx=0x7f4de40ca6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3d04afeb0994269 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.255+0000 7f4dd3400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:15:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.255+0000 7f4dd3400700 1 -- 172.21.15.131:0/2187072555 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de40b1070 con 0x5567f140b910 2024-05-08T22:15:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.255+0000 7f4dd3400700 1 -- 172.21.15.131:0/2187072555 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f4de40cae40 con 0x5567f140b910 2024-05-08T22:15:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.255+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/2187072555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5567f12b42f0 con 0x5567f140b910 2024-05-08T22:15:47.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.256+0000 7f4dd3400700 1 -- 172.21.15.131:0/2187072555 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4de40981f0 con 0x5567f140b910 2024-05-08T22:15:47.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.256+0000 7f4e0c2cfb80 10 client.26602 fetch_fsmap learned FSMap version 29 2024-05-08T22:15:47.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.256+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/2187072555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5567f1188be0 con 0x5567f140b910 2024-05-08T22:15:47.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.256+0000 7f4dd3400700 1 -- 172.21.15.131:0/2187072555 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5567f12b42f0 con 0x5567f140b910 2024-05-08T22:15:47.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.256+0000 7f4dd3400700 1 -- 172.21.15.131:0/2187072555 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f4de40995a0 con 0x5567f140b910 2024-05-08T22:15:47.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.256+0000 7f4e0c2cfb80 10 client.26602 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:15:47.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.256+0000 7f4e0c2cfb80 10 client.26602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:15:47.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.256+0000 7f4e0c2cfb80 20 client.26602 populate_metadata read hostname 'smithi131' 2024-05-08T22:15:47.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.256+0000 7f4e0c2cfb80 1 --2- 172.21.15.131:0/2187072555 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5567f1425000 0x5567f1427420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.256+0000 7f4e0c2cfb80 4 client.26602 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:15:47.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.256+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/2187072555 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567f13da890 con 0x5567f1425000 2024-05-08T22:15:47.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.256+0000 7f4df5400700 1 --2- 172.21.15.131:0/2187072555 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5567f1425000 0x5567f1427420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.257+0000 7f4df5400700 1 --2- 172.21.15.131:0/2187072555 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5567f1425000 0x5567f1427420 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c86e30d052f10bc server_cookie=5ae0afb5bede506d in_seq=0 out_seq=0 2024-05-08T22:15:47.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.257+0000 7f4dd3400700 10 client.26602 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:47.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.258+0000 7f4dd3400700 1 -- 172.21.15.131:0/2187072555 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5567f13da890 con 0x5567f1425000 2024-05-08T22:15:47.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.258+0000 7f4dd3400700 10 client.26602 handle_command_reply: tid=0 2024-05-08T22:15:47.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.258+0000 7f4e0c2cfb80 1 client.26602 shutdown 2024-05-08T22:15:47.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.258+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/2187072555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f140b910 msgr2=0x5567f1414c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:47.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.258+0000 7f4e0c2cfb80 1 --2- 172.21.15.131:0/2187072555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f140b910 0x5567f1414c50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4de4009fd0 tx=0x7f4de40ca6e0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.258+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/2187072555 shutdown_connections 2024-05-08T22:15:47.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.258+0000 7f4e0c2cfb80 1 --2- 172.21.15.131:0/2187072555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f14117a0 0x5567f14156d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.258+0000 7f4e0c2cfb80 1 --2- 172.21.15.131:0/2187072555 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5567f1425000 0x5567f1427420 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.258+0000 7f4e0c2cfb80 1 --2- 172.21.15.131:0/2187072555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f140b910 0x5567f1414c50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.258+0000 7f4e0c2cfb80 1 --2- 172.21.15.131:0/2187072555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567f1410e80 0x5567f1415190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.258+0000 7f4dd3400700 0 client.26602 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:47.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.258+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/2187072555 >> 172.21.15.131:0/2187072555 conn(0x5567f1369290 msgr2=0x5567f1405620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:47.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.258+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/2187072555 shutdown_connections 2024-05-08T22:15:47.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.258+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/2187072555 wait complete. 2024-05-08T22:15:47.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.258+0000 7f4e0c2cfb80 20 client.26602 trim_cache size 0 max 16384 2024-05-08T22:15:47.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.277+0000 7f4e0c2cfb80 1 Processor -- start 2024-05-08T22:15:47.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.277+0000 7f4e0c2cfb80 1 -- start start 2024-05-08T22:15:47.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.277+0000 7f4e0c2cfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f140b910 0x5567f140b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.277+0000 7f4e0c2cfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f1410e80 0x5567f1410430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.277+0000 7f4e0c2cfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567f14117a0 0x5567f1405da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.277+0000 7f4e0c2cfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567f1421e40 con 0x5567f140b910 2024-05-08T22:15:47.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.277+0000 7f4e0c2cfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567f140e8f0 con 0x5567f14117a0 2024-05-08T22:15:47.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.277+0000 7f4e0c2cfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567f1417750 con 0x5567f1410e80 2024-05-08T22:15:47.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.277+0000 7f4df5e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f140b910 0x5567f140b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.277+0000 7f4df5400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f1410e80 0x5567f1410430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.277+0000 7f4df5400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f1410e80 0x5567f1410430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34964/0 (socket says 172.21.15.131:34964) 2024-05-08T22:15:47.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.277+0000 7f4df5400700 1 -- 172.21.15.131:0/586329588 learned_addr learned my addr 172.21.15.131:0/586329588 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:47.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.277+0000 7f4df6800700 1 --2- 172.21.15.131:0/586329588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567f14117a0 0x5567f1405da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.278+0000 7f4df5400700 1 -- 172.21.15.131:0/586329588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567f14117a0 msgr2=0x5567f1405da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:47.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.278+0000 7f4df5400700 1 --2- 172.21.15.131:0/586329588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567f14117a0 0x5567f1405da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.278+0000 7f4df5400700 1 -- 172.21.15.131:0/586329588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f140b910 msgr2=0x5567f140b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:47.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.278+0000 7f4df5400700 1 --2- 172.21.15.131:0/586329588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f140b910 0x5567f140b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.278+0000 7f4df5400700 1 -- 172.21.15.131:0/586329588 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567f140bfc0 con 0x5567f1410e80 2024-05-08T22:15:47.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.278+0000 7f4df5e00700 1 --2- 172.21.15.131:0/586329588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f140b910 0x5567f140b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:15:47.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.278+0000 7f4df5400700 1 --2- 172.21.15.131:0/586329588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f1410e80 0x5567f1410430 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x5567f1427cc0 tx=0x7f4de0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eef181bda7374082 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:47.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.278+0000 7f4ddfa00700 1 -- 172.21.15.131:0/586329588 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de0016040 con 0x5567f1410e80 2024-05-08T22:15:47.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.278+0000 7f4ddfa00700 1 -- 172.21.15.131:0/586329588 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4de0017040 con 0x5567f1410e80 2024-05-08T22:15:47.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.278+0000 7f4ddfa00700 1 -- 172.21.15.131:0/586329588 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de0003eb0 con 0x5567f1410e80 2024-05-08T22:15:47.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.279+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/586329588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f1410e80 msgr2=0x5567f1410430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:47.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.279+0000 7f4e0c2cfb80 1 --2- 172.21.15.131:0/586329588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f1410e80 0x5567f1410430 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x5567f1427cc0 tx=0x7f4de0009260 comp rx=0 tx=0).stop 2024-05-08T22:15:47.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.279+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/586329588 shutdown_connections 2024-05-08T22:15:47.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.279+0000 7f4e0c2cfb80 1 --2- 172.21.15.131:0/586329588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567f14117a0 0x5567f1405da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.279+0000 7f4e0c2cfb80 1 --2- 172.21.15.131:0/586329588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f140b910 0x5567f140b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.279+0000 7f4e0c2cfb80 1 --2- 172.21.15.131:0/586329588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f1410e80 0x5567f1410430 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.279+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/586329588 >> 172.21.15.131:0/586329588 conn(0x5567f1369290 msgr2=0x5567f140fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:47.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.279+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/586329588 shutdown_connections 2024-05-08T22:15:47.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.279+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/586329588 wait complete. 2024-05-08T22:15:47.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.279+0000 7f4e0c2cfb80 1 -- start start 2024-05-08T22:15:47.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.280+0000 7f4e0c2cfb80 1 Processor -- start 2024-05-08T22:15:47.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.280+0000 7f4e0c2cfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f140b910 0x5567f141c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.280+0000 7f4e0c2cfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567f1410e80 0x5567f141a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.280+0000 7f4e0c2cfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f14117a0 0x5567f141ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.280+0000 7f4e0c2cfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567f141b530 con 0x5567f140b910 2024-05-08T22:15:47.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.280+0000 7f4e0c2cfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567f141b6a0 con 0x5567f1410e80 2024-05-08T22:15:47.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.280+0000 7f4e0c2cfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567f147b4d0 con 0x5567f14117a0 2024-05-08T22:15:47.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.280+0000 7f4e0c2cfb80 10 client.0 _ll_register_callbacks cb 0x5567f1300bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:15:47.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.281+0000 7f4df5e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f140b910 0x5567f141c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.281+0000 7f4df5400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567f1410e80 0x5567f141a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.281+0000 7f4df5e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f140b910 0x5567f141c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41876/0 (socket says 172.21.15.131:41876) 2024-05-08T22:15:47.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.281+0000 7f4df5e00700 1 -- 172.21.15.131:0/3596490890 learned_addr learned my addr 172.21.15.131:0/3596490890 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:47.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.281+0000 7f4df6800700 1 --2- 172.21.15.131:0/3596490890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f14117a0 0x5567f141ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.281+0000 7f4df5400700 1 -- 172.21.15.131:0/3596490890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f14117a0 msgr2=0x5567f141ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:47.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.281+0000 7f4df5400700 1 --2- 172.21.15.131:0/3596490890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f14117a0 0x5567f141ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.281+0000 7f4df5400700 1 -- 172.21.15.131:0/3596490890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f140b910 msgr2=0x5567f141c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:47.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.281+0000 7f4df5400700 1 --2- 172.21.15.131:0/3596490890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f140b910 0x5567f141c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.281+0000 7f4df5400700 1 -- 172.21.15.131:0/3596490890 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567f1188be0 con 0x5567f1410e80 2024-05-08T22:15:47.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.281+0000 7f4df5e00700 1 --2- 172.21.15.131:0/3596490890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f140b910 0x5567f141c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:15:47.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.281+0000 7f4df6800700 1 --2- 172.21.15.131:0/3596490890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f14117a0 0x5567f141ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:15:47.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.281+0000 7f4df5400700 1 --2- 172.21.15.131:0/3596490890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567f1410e80 0x5567f141a910 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4de00042c0 tx=0x7f4de00037e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8110e70cf54779a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:47.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.281+0000 7f4dd3400700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:15:47.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.282+0000 7f4dd3400700 1 -- 172.21.15.131:0/3596490890 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de0016070 con 0x5567f1410e80 2024-05-08T22:15:47.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.282+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/3596490890 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4de409a0b0 con 0x5567f1410e80 2024-05-08T22:15:47.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.282+0000 7f4dd3400700 1 -- 172.21.15.131:0/3596490890 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f4de0005040 con 0x5567f1410e80 2024-05-08T22:15:47.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.282+0000 7f4dd3400700 1 -- 172.21.15.131:0/3596490890 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4de0005200 con 0x5567f1410e80 2024-05-08T22:15:47.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.282+0000 7f4e0c2cfb80 10 client.26522 fetch_fsmap learned FSMap version 29 2024-05-08T22:15:47.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.282+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/3596490890 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f4de00043d0 con 0x5567f1410e80 2024-05-08T22:15:47.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.282+0000 7f4dd3400700 1 -- 172.21.15.131:0/3596490890 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4de409a0b0 con 0x5567f1410e80 2024-05-08T22:15:47.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.282+0000 7f4dd3400700 1 -- 172.21.15.131:0/3596490890 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f4de001f210 con 0x5567f1410e80 2024-05-08T22:15:47.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.282+0000 7f4e0c2cfb80 10 client.26522 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:15:47.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.282+0000 7f4e0c2cfb80 10 client.26522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:15:47.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.282+0000 7f4e0c2cfb80 20 client.26522 populate_metadata read hostname 'smithi131' 2024-05-08T22:15:47.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.283+0000 7f4e0c2cfb80 1 --2- 172.21.15.131:0/3596490890 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5567f147cae0 0x5567f147ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.283+0000 7f4e0c2cfb80 4 client.26522 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T22:15:47.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.283+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/3596490890 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5567f1353f40 con 0x5567f147cae0 2024-05-08T22:15:47.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.283+0000 7f4df5e00700 1 --2- 172.21.15.131:0/3596490890 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5567f147cae0 0x5567f147ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.283+0000 7f4dd3400700 10 client.26522 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:47.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.283+0000 7f4df5e00700 1 --2- 172.21.15.131:0/3596490890 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5567f147cae0 0x5567f147ef00 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e39fbe860aaf605b server_cookie=9fb77eb090e1a18a in_seq=0 out_seq=0 2024-05-08T22:15:47.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.361+0000 7f4dd3400700 1 -- 172.21.15.131:0/3596490890 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5567f1353f40 con 0x5567f147cae0 2024-05-08T22:15:47.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.361+0000 7f4dd3400700 10 client.26522 handle_command_reply: tid=0 2024-05-08T22:15:47.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.361+0000 7f4e0c2cfb80 1 client.26522 shutdown 2024-05-08T22:15:47.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.361+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/3596490890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567f1410e80 msgr2=0x5567f141a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:47.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.361+0000 7f4e0c2cfb80 1 --2- 172.21.15.131:0/3596490890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567f1410e80 0x5567f141a910 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4de00042c0 tx=0x7f4de00037e0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.361+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/3596490890 shutdown_connections 2024-05-08T22:15:47.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.361+0000 7f4e0c2cfb80 1 --2- 172.21.15.131:0/3596490890 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5567f147cae0 0x5567f147ef00 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.361+0000 7f4e0c2cfb80 1 --2- 172.21.15.131:0/3596490890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567f14117a0 0x5567f141ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.361+0000 7f4e0c2cfb80 1 --2- 172.21.15.131:0/3596490890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567f140b910 0x5567f141c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.361+0000 7f4e0c2cfb80 1 --2- 172.21.15.131:0/3596490890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567f1410e80 0x5567f141a910 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.361+0000 7f4dd3400700 0 client.26522 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:47.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.361+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/3596490890 >> 172.21.15.131:0/3596490890 conn(0x5567f1369290 msgr2=0x5567f1416480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:47.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.361+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/3596490890 shutdown_connections 2024-05-08T22:15:47.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.361+0000 7f4e0c2cfb80 1 -- 172.21.15.131:0/3596490890 wait complete. 2024-05-08T22:15:47.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.362+0000 7f4e0c2cfb80 20 client.26522 trim_cache size 0 max 16384 2024-05-08T22:15:47.365 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:15:47.365 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-08T22:15:47.365 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "17afbd7c-1f59-4c74-83b8-c44e567b9276", 2024-05-08T22:15:47.365 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-08T22:15:47.365 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:15:47.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.362+0000 7f4df8800700 1 -- 172.21.15.131:0/2981624634 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f4dd80a1290 msgr2=0x7f4dd80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:47.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.362+0000 7f4df8800700 1 --2- 172.21.15.131:0/2981624634 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f4dd80a1290 0x7f4dd80a3750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4dec009610 tx=0x7f4dec007ca0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.362+0000 7f4df8800700 1 -- 172.21.15.131:0/2981624634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4df0101160 msgr2=0x7f4df0199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:47.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.362+0000 7f4df8800700 1 --2- 172.21.15.131:0/2981624634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4df0101160 0x7f4df0199580 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4de400e9c0 tx=0x7f4de400ecd0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.362+0000 7f4df8800700 1 -- 172.21.15.131:0/2981624634 shutdown_connections 2024-05-08T22:15:47.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.362+0000 7f4df8800700 1 --2- 172.21.15.131:0/2981624634 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f4dd80a1290 0x7f4dd80a3750 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.362+0000 7f4df8800700 1 --2- 172.21.15.131:0/2981624634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4df019de10 0x7f4df019e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.362+0000 7f4df8800700 1 --2- 172.21.15.131:0/2981624634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4df0101160 0x7f4df0199580 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.362+0000 7f4df8800700 1 --2- 172.21.15.131:0/2981624634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4df0106280 0x7f4df0199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.362+0000 7f4df8800700 1 -- 172.21.15.131:0/2981624634 >> 172.21.15.131:0/2981624634 conn(0x7f4df00fc9c0 msgr2=0x7f4df00feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:47.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.362+0000 7f4df8800700 1 -- 172.21.15.131:0/2981624634 shutdown_connections 2024-05-08T22:15:47.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.362+0000 7f4df8800700 1 -- 172.21.15.131:0/2981624634 wait complete. 2024-05-08T22:15:47.383 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:15:47.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.581+0000 7fe520200700 1 -- 172.21.15.131:0/1828861538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe518101180 msgr2=0x7fe518101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:47.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.581+0000 7fe520200700 1 --2- 172.21.15.131:0/1828861538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe518101180 0x7fe518101560 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009990 tx=0x7fe50c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.581+0000 7fe520200700 1 -- 172.21.15.131:0/1828861538 shutdown_connections 2024-05-08T22:15:47.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.581+0000 7fe520200700 1 --2- 172.21.15.131:0/1828861538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5181062a0 0x7fe51810de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.581+0000 7fe520200700 1 --2- 172.21.15.131:0/1828861538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe518101180 0x7fe518101560 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.581+0000 7fe520200700 1 --2- 172.21.15.131:0/1828861538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe518101b30 0x7fe518105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.581+0000 7fe520200700 1 -- 172.21.15.131:0/1828861538 >> 172.21.15.131:0/1828861538 conn(0x7fe5180fc9e0 msgr2=0x7fe5180fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:47.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.582+0000 7fe520200700 1 -- 172.21.15.131:0/1828861538 shutdown_connections 2024-05-08T22:15:47.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.582+0000 7fe520200700 1 -- 172.21.15.131:0/1828861538 wait complete. 2024-05-08T22:15:47.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.583+0000 7fe520200700 1 Processor -- start 2024-05-08T22:15:47.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.583+0000 7fe520200700 1 -- start start 2024-05-08T22:15:47.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.584+0000 7fe520200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe518101180 0x7fe5181996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.584+0000 7fe520200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5181062a0 0x7fe518199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.584+0000 7fe520200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe51819dfa0 0x7fe51819e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.584+0000 7fe520200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51819ea20 con 0x7fe51819dfa0 2024-05-08T22:15:47.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.584+0000 7fe520200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51819eb90 con 0x7fe5181062a0 2024-05-08T22:15:47.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.584+0000 7fe520200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51819ed50 con 0x7fe518101180 2024-05-08T22:15:47.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.584+0000 7fe51ce00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5181062a0 0x7fe518199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.584+0000 7fe51ce00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5181062a0 0x7fe518199c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:54602/0 (socket says 172.21.15.131:54602) 2024-05-08T22:15:47.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.584+0000 7fe51ce00700 1 -- 172.21.15.131:0/342273159 learned_addr learned my addr 172.21.15.131:0/342273159 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:47.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.584+0000 7fe51e200700 1 --2- 172.21.15.131:0/342273159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe51819dfa0 0x7fe51819e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.584+0000 7fe51d800700 1 --2- 172.21.15.131:0/342273159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe518101180 0x7fe5181996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.584+0000 7fe51ce00700 1 -- 172.21.15.131:0/342273159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe518101180 msgr2=0x7fe5181996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:47.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.584+0000 7fe51ce00700 1 --2- 172.21.15.131:0/342273159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe518101180 0x7fe5181996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.584+0000 7fe51ce00700 1 -- 172.21.15.131:0/342273159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe51819dfa0 msgr2=0x7fe51819e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:47.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.584+0000 7fe51ce00700 1 --2- 172.21.15.131:0/342273159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe51819dfa0 0x7fe51819e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.584+0000 7fe51ce00700 1 -- 172.21.15.131:0/342273159 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50c0095f0 con 0x7fe5181062a0 2024-05-08T22:15:47.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.584+0000 7fe51d800700 1 --2- 172.21.15.131:0/342273159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe518101180 0x7fe5181996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:15:47.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.585+0000 7fe51ce00700 1 --2- 172.21.15.131:0/342273159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5181062a0 0x7fe518199c00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe50400e970 tx=0x7fe50400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:47.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.585+0000 7fe516000700 1 -- 172.21.15.131:0/342273159 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe504009790 con 0x7fe5181062a0 2024-05-08T22:15:47.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.585+0000 7fe516000700 1 -- 172.21.15.131:0/342273159 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe50400eec0 con 0x7fe5181062a0 2024-05-08T22:15:47.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.585+0000 7fe520200700 1 -- 172.21.15.131:0/342273159 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5181a62b0 con 0x7fe5181062a0 2024-05-08T22:15:47.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.585+0000 7fe516000700 1 -- 172.21.15.131:0/342273159 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe504005590 con 0x7fe5181062a0 2024-05-08T22:15:47.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.585+0000 7fe520200700 1 -- 172.21.15.131:0/342273159 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5181a6790 con 0x7fe5181062a0 2024-05-08T22:15:47.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.588+0000 7fe516000700 1 -- 172.21.15.131:0/342273159 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fe50400cbd0 con 0x7fe5181062a0 2024-05-08T22:15:47.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.589+0000 7fe516000700 1 --2- 172.21.15.131:0/342273159 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe5000a1290 0x7fe5000a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.589+0000 7fe516000700 1 -- 172.21.15.131:0/342273159 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fe504014070 con 0x7fe5181062a0 2024-05-08T22:15:47.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.589+0000 7fe533dcab80 1 Processor -- start 2024-05-08T22:15:47.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.589+0000 7fe51d800700 1 --2- 172.21.15.131:0/342273159 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe5000a1290 0x7fe5000a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.590+0000 7fe533dcab80 1 -- start start 2024-05-08T22:15:47.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.590+0000 7fe533dcab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1de00 0x562ce2d1e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.590+0000 7fe533dcab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ce2d1e720 0x562ce2d18350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.590+0000 7fe533dcab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ce2d18890 0x562ce2d18cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.590+0000 7fe533dcab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce2c6f150 con 0x562ce2d1e720 2024-05-08T22:15:47.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.590+0000 7fe533dcab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce2c6cb10 con 0x562ce2d1de00 2024-05-08T22:15:47.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.590+0000 7fe533dcab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce2c68400 con 0x562ce2d18890 2024-05-08T22:15:47.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.590+0000 7fe51d800700 1 --2- 172.21.15.131:0/342273159 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe5000a1290 0x7fe5000a3750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe50c006010 tx=0x7fe50c00b260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:47.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.590+0000 7fe51d800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1de00 0x562ce2d1e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.590+0000 7fe51ce00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ce2d1e720 0x562ce2d18350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.590+0000 7fe51d800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1de00 0x562ce2d1e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:54608/0 (socket says 172.21.15.131:54608) 2024-05-08T22:15:47.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.590+0000 7fe51d800700 1 -- 172.21.15.131:0/590446911 learned_addr learned my addr 172.21.15.131:0/590446911 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:47.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.591+0000 7fe51e200700 1 --2- 172.21.15.131:0/590446911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ce2d18890 0x562ce2d18cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.591+0000 7fe51ce00700 1 -- 172.21.15.131:0/590446911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ce2d18890 msgr2=0x562ce2d18cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:47.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.591+0000 7fe51ce00700 1 --2- 172.21.15.131:0/590446911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ce2d18890 0x562ce2d18cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.591+0000 7fe51ce00700 1 -- 172.21.15.131:0/590446911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1de00 msgr2=0x562ce2d1e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:47.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.591+0000 7fe51ce00700 1 --2- 172.21.15.131:0/590446911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1de00 0x562ce2d1e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.591+0000 7fe51ce00700 1 -- 172.21.15.131:0/590446911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ce2ae93b0 con 0x562ce2d1e720 2024-05-08T22:15:47.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.591+0000 7fe51d800700 1 --2- 172.21.15.131:0/590446911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1de00 0x562ce2d1e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:15:47.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.591+0000 7fe51ce00700 1 --2- 172.21.15.131:0/590446911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ce2d1e720 0x562ce2d18350 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe5040ca6c0 tx=0x7fe50409a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43260f36e2ca94ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:47.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.591+0000 7fe50aa00700 1 -- 172.21.15.131:0/590446911 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5040b1070 con 0x562ce2d1e720 2024-05-08T22:15:47.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.591+0000 7fe50aa00700 1 -- 172.21.15.131:0/590446911 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe50409deb0 con 0x562ce2d1e720 2024-05-08T22:15:47.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.591+0000 7fe50aa00700 1 -- 172.21.15.131:0/590446911 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50409d070 con 0x562ce2d1e720 2024-05-08T22:15:47.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.592+0000 7fe533dcab80 1 -- 172.21.15.131:0/590446911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ce2d1e720 msgr2=0x562ce2d18350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:47.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.592+0000 7fe533dcab80 1 --2- 172.21.15.131:0/590446911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ce2d1e720 0x562ce2d18350 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe5040ca6c0 tx=0x7fe50409a780 comp rx=0 tx=0).stop 2024-05-08T22:15:47.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.592+0000 7fe533dcab80 1 -- 172.21.15.131:0/590446911 shutdown_connections 2024-05-08T22:15:47.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.592+0000 7fe533dcab80 1 --2- 172.21.15.131:0/590446911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ce2d18890 0x562ce2d18cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.592+0000 7fe533dcab80 1 --2- 172.21.15.131:0/590446911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1de00 0x562ce2d1e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.592+0000 7fe533dcab80 1 --2- 172.21.15.131:0/590446911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ce2d1e720 0x562ce2d18350 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.592+0000 7fe533dcab80 1 -- 172.21.15.131:0/590446911 >> 172.21.15.131:0/590446911 conn(0x562ce2c76210 msgr2=0x562ce2d12df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:47.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.592+0000 7fe533dcab80 1 -- 172.21.15.131:0/590446911 shutdown_connections 2024-05-08T22:15:47.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.592+0000 7fe533dcab80 1 -- 172.21.15.131:0/590446911 wait complete. 2024-05-08T22:15:47.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.593+0000 7fe533dcab80 1 -- start start 2024-05-08T22:15:47.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.593+0000 7fe533dcab80 1 Processor -- start 2024-05-08T22:15:47.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.593+0000 7fe533dcab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ce2d18890 0x562ce2d21b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.593+0000 7fe533dcab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1de00 0x562ce2d22040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.593+0000 7fe533dcab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ce2d1e720 0x562ce2d22580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.593+0000 7fe533dcab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce2c04560 con 0x562ce2d1e720 2024-05-08T22:15:47.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.593+0000 7fe533dcab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce2c04200 con 0x562ce2d1de00 2024-05-08T22:15:47.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.593+0000 7fe533dcab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce2aeb6a0 con 0x562ce2d18890 2024-05-08T22:15:47.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.593+0000 7fe533dcab80 10 client.0 _ll_register_callbacks cb 0x562ce2c0db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:15:47.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.593+0000 7fe51ce00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1de00 0x562ce2d22040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.594+0000 7fe51d800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ce2d18890 0x562ce2d21b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.594+0000 7fe51ce00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1de00 0x562ce2d22040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:54622/0 (socket says 172.21.15.131:54622) 2024-05-08T22:15:47.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.594+0000 7fe51ce00700 1 -- 172.21.15.131:0/1818365888 learned_addr learned my addr 172.21.15.131:0/1818365888 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:47.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.594+0000 7fe51e200700 1 --2- 172.21.15.131:0/1818365888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ce2d1e720 0x562ce2d22580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.594+0000 7fe51ce00700 1 -- 172.21.15.131:0/1818365888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ce2d18890 msgr2=0x562ce2d21b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:47.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.594+0000 7fe51ce00700 1 --2- 172.21.15.131:0/1818365888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ce2d18890 0x562ce2d21b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.594+0000 7fe51ce00700 1 -- 172.21.15.131:0/1818365888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ce2d1e720 msgr2=0x562ce2d22580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:47.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.594+0000 7fe51ce00700 1 --2- 172.21.15.131:0/1818365888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ce2d1e720 0x562ce2d22580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.594+0000 7fe51ce00700 1 -- 172.21.15.131:0/1818365888 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe51810e690 con 0x562ce2d1de00 2024-05-08T22:15:47.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.594+0000 7fe51ce00700 1 --2- 172.21.15.131:0/1818365888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1de00 0x562ce2d22040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe504099d80 tx=0x7fe5040a25b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6b8ed0172bf0f8d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:47.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.594+0000 7fe4ff400700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:15:47.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.594+0000 7fe4ff400700 1 -- 172.21.15.131:0/1818365888 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5040b1070 con 0x562ce2d1de00 2024-05-08T22:15:47.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.594+0000 7fe4ff400700 1 -- 172.21.15.131:0/1818365888 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fe504016540 con 0x562ce2d1de00 2024-05-08T22:15:47.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.594+0000 7fe533dcab80 1 -- 172.21.15.131:0/1818365888 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ce2bc1270 con 0x562ce2d1de00 2024-05-08T22:15:47.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.595+0000 7fe4ff400700 1 -- 172.21.15.131:0/1818365888 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe50409d070 con 0x562ce2d1de00 2024-05-08T22:15:47.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.595+0000 7fe533dcab80 10 client.26534 fetch_fsmap learned FSMap version 29 2024-05-08T22:15:47.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.595+0000 7fe533dcab80 1 -- 172.21.15.131:0/1818365888 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fe5181a6790 con 0x562ce2d1de00 2024-05-08T22:15:47.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.595+0000 7fe4ff400700 1 -- 172.21.15.131:0/1818365888 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562ce2bc1270 con 0x562ce2d1de00 2024-05-08T22:15:47.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.595+0000 7fe4ff400700 1 -- 172.21.15.131:0/1818365888 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fe504099670 con 0x562ce2d1de00 2024-05-08T22:15:47.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.595+0000 7fe533dcab80 10 client.26534 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:15:47.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.595+0000 7fe533dcab80 10 client.26534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:15:47.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.595+0000 7fe533dcab80 20 client.26534 populate_metadata read hostname 'smithi131' 2024-05-08T22:15:47.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.595+0000 7fe533dcab80 1 --2- 172.21.15.131:0/1818365888 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562ce2d31ee0 0x562ce2d34300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.595+0000 7fe533dcab80 4 client.26534 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:15:47.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.595+0000 7fe533dcab80 1 -- 172.21.15.131:0/1818365888 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ce2ce7810 con 0x562ce2d31ee0 2024-05-08T22:15:47.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.596+0000 7fe51d800700 1 --2- 172.21.15.131:0/1818365888 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562ce2d31ee0 0x562ce2d34300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.596+0000 7fe51d800700 1 --2- 172.21.15.131:0/1818365888 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562ce2d31ee0 0x562ce2d34300 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da8f997b291000e1 server_cookie=4e6563b00b5337e3 in_seq=0 out_seq=0 2024-05-08T22:15:47.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.596+0000 7fe4ff400700 10 client.26534 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:47.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.605+0000 7fe4ff400700 1 -- 172.21.15.131:0/1818365888 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562ce2ce7810 con 0x562ce2d31ee0 2024-05-08T22:15:47.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.606+0000 7fe4ff400700 10 client.26534 handle_command_reply: tid=0 2024-05-08T22:15:47.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.606+0000 7fe533dcab80 1 client.26534 shutdown 2024-05-08T22:15:47.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.606+0000 7fe533dcab80 1 -- 172.21.15.131:0/1818365888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1de00 msgr2=0x562ce2d22040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:47.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.606+0000 7fe533dcab80 1 --2- 172.21.15.131:0/1818365888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1de00 0x562ce2d22040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe504099d80 tx=0x7fe5040a25b0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.606+0000 7fe533dcab80 1 -- 172.21.15.131:0/1818365888 shutdown_connections 2024-05-08T22:15:47.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.606+0000 7fe533dcab80 1 --2- 172.21.15.131:0/1818365888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ce2d1e720 0x562ce2d22580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.606+0000 7fe533dcab80 1 --2- 172.21.15.131:0/1818365888 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562ce2d31ee0 0x562ce2d34300 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.606+0000 7fe533dcab80 1 --2- 172.21.15.131:0/1818365888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ce2d18890 0x562ce2d21b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.606+0000 7fe533dcab80 1 --2- 172.21.15.131:0/1818365888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1de00 0x562ce2d22040 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.606+0000 7fe4ff400700 0 client.26534 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:47.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.606+0000 7fe533dcab80 1 -- 172.21.15.131:0/1818365888 >> 172.21.15.131:0/1818365888 conn(0x562ce2c76210 msgr2=0x562ce2d125a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:47.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.606+0000 7fe533dcab80 1 -- 172.21.15.131:0/1818365888 shutdown_connections 2024-05-08T22:15:47.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.606+0000 7fe533dcab80 1 -- 172.21.15.131:0/1818365888 wait complete. 2024-05-08T22:15:47.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.607+0000 7fe533dcab80 20 client.26534 trim_cache size 0 max 16384 2024-05-08T22:15:47.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.633+0000 7fe533dcab80 1 Processor -- start 2024-05-08T22:15:47.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.633+0000 7fe533dcab80 1 -- start start 2024-05-08T22:15:47.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.633+0000 7fe533dcab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ce2d18890 0x562ce2d18350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.633+0000 7fe533dcab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ce2d1de00 0x562ce2d1d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.633+0000 7fe533dcab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1e720 0x562ce2d12d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.633+0000 7fe533dcab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce2d13b30 con 0x562ce2d18890 2024-05-08T22:15:47.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.633+0000 7fe533dcab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce2d24670 con 0x562ce2d1e720 2024-05-08T22:15:47.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.633+0000 7fe533dcab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce2d0fdc0 con 0x562ce2d1de00 2024-05-08T22:15:47.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.633+0000 7fe51d800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ce2d18890 0x562ce2d18350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.633+0000 7fe51e200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1e720 0x562ce2d12d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.633+0000 7fe51d800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ce2d18890 0x562ce2d18350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41904/0 (socket says 172.21.15.131:41904) 2024-05-08T22:15:47.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.633+0000 7fe51e200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1e720 0x562ce2d12d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:54628/0 (socket says 172.21.15.131:54628) 2024-05-08T22:15:47.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.633+0000 7fe51d800700 1 -- 172.21.15.131:0/2441830162 learned_addr learned my addr 172.21.15.131:0/2441830162 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:47.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.633+0000 7fe51ce00700 1 --2- 172.21.15.131:0/2441830162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ce2d1de00 0x562ce2d1d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.634+0000 7fe51d800700 1 -- 172.21.15.131:0/2441830162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ce2d1de00 msgr2=0x562ce2d1d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:47.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.634+0000 7fe51d800700 1 --2- 172.21.15.131:0/2441830162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ce2d1de00 0x562ce2d1d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.634+0000 7fe51d800700 1 -- 172.21.15.131:0/2441830162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1e720 msgr2=0x562ce2d12d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:47.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.634+0000 7fe51d800700 1 --2- 172.21.15.131:0/2441830162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1e720 0x562ce2d12d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.634+0000 7fe51d800700 1 -- 172.21.15.131:0/2441830162 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ce2d18f40 con 0x562ce2d18890 2024-05-08T22:15:47.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.634+0000 7fe51e200700 1 --2- 172.21.15.131:0/2441830162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1e720 0x562ce2d12d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:15:47.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.634+0000 7fe51d800700 1 --2- 172.21.15.131:0/2441830162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ce2d18890 0x562ce2d18350 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe50c000c00 tx=0x7fe50c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc57740bb1d2b38 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:47.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.634+0000 7fe50b400700 1 -- 172.21.15.131:0/2441830162 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c02b040 con 0x562ce2d18890 2024-05-08T22:15:47.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.634+0000 7fe50b400700 1 -- 172.21.15.131:0/2441830162 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe50c02c040 con 0x562ce2d18890 2024-05-08T22:15:47.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.634+0000 7fe50b400700 1 -- 172.21.15.131:0/2441830162 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c004ba0 con 0x562ce2d18890 2024-05-08T22:15:47.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.635+0000 7fe533dcab80 1 -- 172.21.15.131:0/2441830162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ce2d18890 msgr2=0x562ce2d18350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:47.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.635+0000 7fe533dcab80 1 --2- 172.21.15.131:0/2441830162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ce2d18890 0x562ce2d18350 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe50c000c00 tx=0x7fe50c01f000 comp rx=0 tx=0).stop 2024-05-08T22:15:47.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.635+0000 7fe533dcab80 1 -- 172.21.15.131:0/2441830162 shutdown_connections 2024-05-08T22:15:47.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.635+0000 7fe533dcab80 1 --2- 172.21.15.131:0/2441830162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1e720 0x562ce2d12d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.635+0000 7fe533dcab80 1 --2- 172.21.15.131:0/2441830162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ce2d18890 0x562ce2d18350 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.635+0000 7fe533dcab80 1 --2- 172.21.15.131:0/2441830162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ce2d1de00 0x562ce2d1d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.635+0000 7fe533dcab80 1 -- 172.21.15.131:0/2441830162 >> 172.21.15.131:0/2441830162 conn(0x562ce2c76210 msgr2=0x562ce2d1cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:47.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.635+0000 7fe533dcab80 1 -- 172.21.15.131:0/2441830162 shutdown_connections 2024-05-08T22:15:47.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.635+0000 7fe533dcab80 1 -- 172.21.15.131:0/2441830162 wait complete. 2024-05-08T22:15:47.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.636+0000 7fe533dcab80 1 -- start start 2024-05-08T22:15:47.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.636+0000 7fe533dcab80 1 Processor -- start 2024-05-08T22:15:47.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.637+0000 7fe533dcab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ce2d18890 0x562ce2d29360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.637+0000 7fe533dcab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ce2d1de00 0x562ce2d19d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.637+0000 7fe533dcab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1e720 0x562ce2d1a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.637+0000 7fe533dcab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce2d29a70 con 0x562ce2d18890 2024-05-08T22:15:47.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.637+0000 7fe533dcab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce2d1a810 con 0x562ce2d1e720 2024-05-08T22:15:47.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.637+0000 7fe533dcab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce2d1a9d0 con 0x562ce2d1de00 2024-05-08T22:15:47.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.637+0000 7fe533dcab80 10 client.0 _ll_register_callbacks cb 0x562ce2c0db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:15:47.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.637+0000 7fe51d800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ce2d18890 0x562ce2d29360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.637+0000 7fe51d800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ce2d18890 0x562ce2d29360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41920/0 (socket says 172.21.15.131:41920) 2024-05-08T22:15:47.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.637+0000 7fe51d800700 1 -- 172.21.15.131:0/4229426856 learned_addr learned my addr 172.21.15.131:0/4229426856 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:15:47.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.637+0000 7fe51e200700 1 --2- 172.21.15.131:0/4229426856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1e720 0x562ce2d1a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.637+0000 7fe51ce00700 1 --2- 172.21.15.131:0/4229426856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ce2d1de00 0x562ce2d19d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.637+0000 7fe51d800700 1 -- 172.21.15.131:0/4229426856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ce2d1de00 msgr2=0x562ce2d19d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:47.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.637+0000 7fe51d800700 1 --2- 172.21.15.131:0/4229426856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ce2d1de00 0x562ce2d19d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.637+0000 7fe51d800700 1 -- 172.21.15.131:0/4229426856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1e720 msgr2=0x562ce2d1a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:15:47.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.637+0000 7fe51d800700 1 --2- 172.21.15.131:0/4229426856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1e720 0x562ce2d1a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.637+0000 7fe51d800700 1 -- 172.21.15.131:0/4229426856 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe510009110 con 0x562ce2d18890 2024-05-08T22:15:47.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.638+0000 7fe51ce00700 1 --2- 172.21.15.131:0/4229426856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ce2d1de00 0x562ce2d19d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:15:47.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.638+0000 7fe51e200700 1 --2- 172.21.15.131:0/4229426856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1e720 0x562ce2d1a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:15:47.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.638+0000 7fe51d800700 1 --2- 172.21.15.131:0/4229426856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ce2d18890 0x562ce2d29360 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009350 tx=0x7fe50c005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56dace5792d7cb7b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:15:47.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.638+0000 7fe4ff400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:15:47.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.638+0000 7fe4ff400700 1 -- 172.21.15.131:0/4229426856 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c02b040 con 0x562ce2d18890 2024-05-08T22:15:47.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.638+0000 7fe533dcab80 1 -- 172.21.15.131:0/4229426856 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe50409deb0 con 0x562ce2d18890 2024-05-08T22:15:47.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.638+0000 7fe4ff400700 1 -- 172.21.15.131:0/4229426856 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fe50c004c30 con 0x562ce2d18890 2024-05-08T22:15:47.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.639+0000 7fe4ff400700 1 -- 172.21.15.131:0/4229426856 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe50c004df0 con 0x562ce2d18890 2024-05-08T22:15:47.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.639+0000 7fe533dcab80 10 client.16503 fetch_fsmap learned FSMap version 29 2024-05-08T22:15:47.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.639+0000 7fe533dcab80 1 -- 172.21.15.131:0/4229426856 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fe50c00ba50 con 0x562ce2d18890 2024-05-08T22:15:47.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.639+0000 7fe4ff400700 1 -- 172.21.15.131:0/4229426856 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe50409deb0 con 0x562ce2d18890 2024-05-08T22:15:47.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.639+0000 7fe4ff400700 1 -- 172.21.15.131:0/4229426856 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fe50c03e5a0 con 0x562ce2d18890 2024-05-08T22:15:47.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.639+0000 7fe533dcab80 10 client.16503 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:15:47.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.639+0000 7fe533dcab80 10 client.16503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:15:47.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.639+0000 7fe533dcab80 20 client.16503 populate_metadata read hostname 'smithi131' 2024-05-08T22:15:47.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.639+0000 7fe533dcab80 1 --2- 172.21.15.131:0/4229426856 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562ce2d89380 0x562ce2d8b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:15:47.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.639+0000 7fe533dcab80 4 client.16503 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:15:47.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.639+0000 7fe533dcab80 1 -- 172.21.15.131:0/4229426856 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562ce2c60ec0 con 0x562ce2d89380 2024-05-08T22:15:47.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.640+0000 7fe51ce00700 1 --2- 172.21.15.131:0/4229426856 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562ce2d89380 0x562ce2d8b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:15:47.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.640+0000 7fe4ff400700 10 client.16503 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:47.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.640+0000 7fe51ce00700 1 --2- 172.21.15.131:0/4229426856 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562ce2d89380 0x562ce2d8b7a0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc940b9644bd3239 server_cookie=1267a378da9c8082 in_seq=0 out_seq=0 2024-05-08T22:15:47.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.650+0000 7fe4ff400700 1 -- 172.21.15.131:0/4229426856 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x562ce2c60ec0 con 0x562ce2d89380 2024-05-08T22:15:47.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.650+0000 7fe4ff400700 10 client.16503 handle_command_reply: tid=0 2024-05-08T22:15:47.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.650+0000 7fe533dcab80 1 client.16503 shutdown 2024-05-08T22:15:47.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.650+0000 7fe533dcab80 1 -- 172.21.15.131:0/4229426856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ce2d18890 msgr2=0x562ce2d29360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:47.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.650+0000 7fe533dcab80 1 --2- 172.21.15.131:0/4229426856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ce2d18890 0x562ce2d29360 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009350 tx=0x7fe50c005810 comp rx=0 tx=0).stop 2024-05-08T22:15:47.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.650+0000 7fe533dcab80 1 -- 172.21.15.131:0/4229426856 shutdown_connections 2024-05-08T22:15:47.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.650+0000 7fe533dcab80 1 --2- 172.21.15.131:0/4229426856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ce2d1e720 0x562ce2d1a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.650+0000 7fe533dcab80 1 --2- 172.21.15.131:0/4229426856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ce2d18890 0x562ce2d29360 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.650+0000 7fe533dcab80 1 --2- 172.21.15.131:0/4229426856 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562ce2d89380 0x562ce2d8b7a0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.651+0000 7fe533dcab80 1 --2- 172.21.15.131:0/4229426856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ce2d1de00 0x562ce2d19d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.651+0000 7fe533dcab80 1 -- 172.21.15.131:0/4229426856 >> 172.21.15.131:0/4229426856 conn(0x562ce2c76210 msgr2=0x562ce2d233a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:47.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.651+0000 7fe4ff400700 0 client.16503 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:15:47.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.651+0000 7fe533dcab80 1 -- 172.21.15.131:0/4229426856 shutdown_connections 2024-05-08T22:15:47.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.651+0000 7fe533dcab80 1 -- 172.21.15.131:0/4229426856 wait complete. 2024-05-08T22:15:47.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.652+0000 7fe533dcab80 20 client.16503 trim_cache size 0 max 16384 2024-05-08T22:15:47.653 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:15:47.653 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "scrub active (8659 inodes in the stack)", 2024-05-08T22:15:47.653 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-08T22:15:47.653 INFO:teuthology.orchestra.run.smithi131.stdout: "17afbd7c-1f59-4c74-83b8-c44e567b9276": { 2024-05-08T22:15:47.654 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-08T22:15:47.654 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "17afbd7c-1f59-4c74-83b8-c44e567b9276", 2024-05-08T22:15:47.654 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-08T22:15:47.654 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:15:47.654 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:15:47.654 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:15:47.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.652+0000 7fe520200700 1 -- 172.21.15.131:0/342273159 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe5000a1290 msgr2=0x7fe5000a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:47.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.652+0000 7fe520200700 1 --2- 172.21.15.131:0/342273159 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe5000a1290 0x7fe5000a3750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe50c006010 tx=0x7fe50c00b260 comp rx=0 tx=0).stop 2024-05-08T22:15:47.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.652+0000 7fe520200700 1 -- 172.21.15.131:0/342273159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5181062a0 msgr2=0x7fe518199c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:15:47.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.652+0000 7fe520200700 1 --2- 172.21.15.131:0/342273159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5181062a0 0x7fe518199c00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe50400e970 tx=0x7fe50400ec80 comp rx=0 tx=0).stop 2024-05-08T22:15:47.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.653+0000 7fe520200700 1 -- 172.21.15.131:0/342273159 shutdown_connections 2024-05-08T22:15:47.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.653+0000 7fe520200700 1 --2- 172.21.15.131:0/342273159 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fe5000a1290 0x7fe5000a3750 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.653+0000 7fe520200700 1 --2- 172.21.15.131:0/342273159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe51819dfa0 0x7fe51819e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.653+0000 7fe520200700 1 --2- 172.21.15.131:0/342273159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe518101180 0x7fe5181996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.653+0000 7fe520200700 1 --2- 172.21.15.131:0/342273159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5181062a0 0x7fe518199c00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:15:47.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.653+0000 7fe520200700 1 -- 172.21.15.131:0/342273159 >> 172.21.15.131:0/342273159 conn(0x7fe5180fc9e0 msgr2=0x7fe5180fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:15:47.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.653+0000 7fe520200700 1 -- 172.21.15.131:0/342273159 shutdown_connections 2024-05-08T22:15:47.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:15:47.653+0000 7fe520200700 1 -- 172.21.15.131:0/342273159 wait complete. 2024-05-08T22:15:47.683 INFO:tasks.cephfs.filesystem:scrub status for tag:17afbd7c-1f59-4c74-83b8-c44e567b9276 - {'path': '/', 'tag': '17afbd7c-1f59-4c74-83b8-c44e567b9276', 'options': 'recursive,force'} 2024-05-08T22:15:48.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:48 smithi157 ceph-mon[77746]: pgmap v262: 129 pgs: 129 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 1.40k op/s 2024-05-08T22:15:48.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:48 smithi157 ceph-mon[77746]: scrub queued for path: / 2024-05-08T22:15:48.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:48 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:48.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:48 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:15:48.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:48 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:15:48.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:48 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:48.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:48 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:15:48.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:48 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:15:48.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:48 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:48.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:48 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:48.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:48 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:15:48.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:48 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:48.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:48 smithi132 ceph-mon[78180]: pgmap v262: 129 pgs: 129 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 1.40k op/s 2024-05-08T22:15:48.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:48 smithi132 ceph-mon[78180]: scrub queued for path: / 2024-05-08T22:15:48.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:48 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:48.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:48 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:15:48.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:48 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:15:48.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:48 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:48.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:48 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:15:48.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:48 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:15:48.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:48 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:48.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:48 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:48.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:48 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:15:48.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:48 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:48.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:48 smithi131 ceph-mon[37260]: pgmap v262: 129 pgs: 129 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 1.40k op/s 2024-05-08T22:15:48.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:48 smithi131 ceph-mon[37260]: scrub queued for path: / 2024-05-08T22:15:48.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:48 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:48.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:48 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:15:48.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:48 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:15:48.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:48 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:48.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:48 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:15:48.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:48 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:15:48.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:48 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:48.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:48 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:48.894 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:48 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:15:48.894 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:48 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:49.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:49 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:15:49.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:49 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:49.843 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:49 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:15:49.843 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:49 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:49.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:49 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:15:49.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:49 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:15:50.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:50 smithi157 ceph-mon[77746]: pgmap v263: 129 pgs: 129 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 1.47k op/s 2024-05-08T22:15:50.843 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:50 smithi132 ceph-mon[78180]: pgmap v263: 129 pgs: 129 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 1.47k op/s 2024-05-08T22:15:50.894 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:50 smithi131 ceph-mon[37260]: pgmap v263: 129 pgs: 129 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 1.47k op/s 2024-05-08T22:15:52.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:52 smithi157 ceph-mon[77746]: pgmap v264: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 940 op/s 2024-05-08T22:15:52.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:52 smithi132 ceph-mon[78180]: pgmap v264: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 940 op/s 2024-05-08T22:15:52.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:52 smithi131 ceph-mon[37260]: pgmap v264: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 940 op/s 2024-05-08T22:15:54.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:54 smithi157 ceph-mon[77746]: pgmap v265: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 987 op/s 2024-05-08T22:15:54.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:54 smithi132 ceph-mon[78180]: pgmap v265: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 987 op/s 2024-05-08T22:15:54.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:54 smithi131 ceph-mon[37260]: pgmap v265: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 987 op/s 2024-05-08T22:15:56.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:56 smithi157 ceph-mon[77746]: pgmap v266: 129 pgs: 129 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 1.40k op/s 2024-05-08T22:15:56.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:56 smithi132 ceph-mon[78180]: pgmap v266: 129 pgs: 129 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 1.40k op/s 2024-05-08T22:15:56.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:56 smithi131 ceph-mon[37260]: pgmap v266: 129 pgs: 129 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 1.40k op/s 2024-05-08T22:15:58.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:15:58 smithi157 ceph-mon[77746]: pgmap v267: 129 pgs: 129 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 1.13k op/s 2024-05-08T22:15:58.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:15:58 smithi132 ceph-mon[78180]: pgmap v267: 129 pgs: 129 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 1.13k op/s 2024-05-08T22:15:58.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:15:58 smithi131 ceph-mon[37260]: pgmap v267: 129 pgs: 129 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 1.13k op/s 2024-05-08T22:16:00.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:00 smithi157 ceph-mon[77746]: pgmap v268: 129 pgs: 129 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 1.54k op/s 2024-05-08T22:16:00.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:00 smithi132 ceph-mon[78180]: pgmap v268: 129 pgs: 129 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 1.54k op/s 2024-05-08T22:16:00.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:00 smithi131 ceph-mon[37260]: pgmap v268: 129 pgs: 129 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 1.54k op/s 2024-05-08T22:16:02.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:02 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:16:02.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:02 smithi157 ceph-mon[77746]: pgmap v269: 129 pgs: 129 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 1.33k op/s 2024-05-08T22:16:02.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:02 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:16:02.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:02 smithi132 ceph-mon[78180]: pgmap v269: 129 pgs: 129 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 1.33k op/s 2024-05-08T22:16:02.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:02 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:16:02.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:02 smithi131 ceph-mon[37260]: pgmap v269: 129 pgs: 129 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 1.33k op/s 2024-05-08T22:16:03.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:03 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:16:03.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:03 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:16:03.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:03 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:16:04.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:04 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:16:04.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:04 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:16:04.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:04 smithi157 ceph-mon[77746]: pgmap v270: 129 pgs: 129 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 1.41k op/s 2024-05-08T22:16:04.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:04 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:16:04.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:04 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:16:04.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:04 smithi132 ceph-mon[78180]: pgmap v270: 129 pgs: 129 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 1.41k op/s 2024-05-08T22:16:04.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:04 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:16:04.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:04 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:16:04.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:04 smithi131 ceph-mon[37260]: pgmap v270: 129 pgs: 129 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 1.41k op/s 2024-05-08T22:16:06.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:06 smithi157 ceph-mon[77746]: pgmap v271: 129 pgs: 129 active+clean; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 1.69k op/s 2024-05-08T22:16:06.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:06 smithi132 ceph-mon[78180]: pgmap v271: 129 pgs: 129 active+clean; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 1.69k op/s 2024-05-08T22:16:06.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:06 smithi131 ceph-mon[37260]: pgmap v271: 129 pgs: 129 active+clean; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 1.69k op/s 2024-05-08T22:16:08.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:08 smithi157 ceph-mon[77746]: pgmap v272: 129 pgs: 129 active+clean; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 1.20k op/s 2024-05-08T22:16:08.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:08 smithi132 ceph-mon[78180]: pgmap v272: 129 pgs: 129 active+clean; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 1.20k op/s 2024-05-08T22:16:08.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:08 smithi131 ceph-mon[37260]: pgmap v272: 129 pgs: 129 active+clean; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 1.20k op/s 2024-05-08T22:16:10.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:10 smithi132 ceph-mon[78180]: pgmap v273: 129 pgs: 129 active+clean; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 1.06k op/s 2024-05-08T22:16:10.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:10 smithi131 ceph-mon[37260]: pgmap v273: 129 pgs: 129 active+clean; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 1.06k op/s 2024-05-08T22:16:11.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:10 smithi157 ceph-mon[77746]: pgmap v273: 129 pgs: 129 active+clean; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 1.06k op/s 2024-05-08T22:16:12.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:12 smithi132 ceph-mon[78180]: pgmap v274: 129 pgs: 129 active+clean; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 903 KiB/s wr, 579 op/s 2024-05-08T22:16:12.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:12 smithi131 ceph-mon[37260]: pgmap v274: 129 pgs: 129 active+clean; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 903 KiB/s wr, 579 op/s 2024-05-08T22:16:13.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:12 smithi157 ceph-mon[77746]: pgmap v274: 129 pgs: 129 active+clean; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 903 KiB/s wr, 579 op/s 2024-05-08T22:16:14.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:14 smithi132 ceph-mon[78180]: pgmap v275: 129 pgs: 129 active+clean; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 556 op/s 2024-05-08T22:16:14.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:14 smithi132 ceph-mon[78180]: osdmap e93: 12 total, 12 up, 12 in 2024-05-08T22:16:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:14 smithi131 ceph-mon[37260]: pgmap v275: 129 pgs: 129 active+clean; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 556 op/s 2024-05-08T22:16:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:14 smithi131 ceph-mon[37260]: osdmap e93: 12 total, 12 up, 12 in 2024-05-08T22:16:15.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:14 smithi157 ceph-mon[77746]: pgmap v275: 129 pgs: 129 active+clean; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 556 op/s 2024-05-08T22:16:15.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:14 smithi157 ceph-mon[77746]: osdmap e93: 12 total, 12 up, 12 in 2024-05-08T22:16:16.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:16 smithi132 ceph-mon[78180]: pgmap v277: 129 pgs: 10 active+clean+snaptrim, 84 active+clean, 35 active+clean+snaptrim_wait; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 403 op/s 2024-05-08T22:16:16.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:16 smithi131 ceph-mon[37260]: pgmap v277: 129 pgs: 10 active+clean+snaptrim, 84 active+clean, 35 active+clean+snaptrim_wait; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 403 op/s 2024-05-08T22:16:17.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:16 smithi157 ceph-mon[77746]: pgmap v277: 129 pgs: 10 active+clean+snaptrim, 84 active+clean, 35 active+clean+snaptrim_wait; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 403 op/s 2024-05-08T22:16:17.684 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:16:18.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.062+0000 7f2956e00700 1 -- 172.21.15.131:0/1140950645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2948098b80 msgr2=0x7f294809cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:18.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.062+0000 7f2956e00700 1 --2- 172.21.15.131:0/1140950645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2948098b80 0x7f294809cbc0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f29440098f0 tx=0x7f2944009c00 comp rx=0 tx=0).stop 2024-05-08T22:16:18.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.063+0000 7f2956e00700 1 -- 172.21.15.131:0/1140950645 shutdown_connections 2024-05-08T22:16:18.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.063+0000 7f2956e00700 1 --2- 172.21.15.131:0/1140950645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f294809d2f0 0x7f29480a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.063+0000 7f2956e00700 1 --2- 172.21.15.131:0/1140950645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29480981d0 0x7f29480985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.063+0000 7f2956e00700 1 --2- 172.21.15.131:0/1140950645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2948098b80 0x7f294809cbc0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.063+0000 7f2956e00700 1 -- 172.21.15.131:0/1140950645 >> 172.21.15.131:0/1140950645 conn(0x7f2948093a50 msgr2=0x7f2948095e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:18.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.063+0000 7f2956e00700 1 -- 172.21.15.131:0/1140950645 shutdown_connections 2024-05-08T22:16:18.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.063+0000 7f2956e00700 1 -- 172.21.15.131:0/1140950645 wait complete. 2024-05-08T22:16:18.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.066+0000 7f2956e00700 1 Processor -- start 2024-05-08T22:16:18.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.066+0000 7f2956e00700 1 -- start start 2024-05-08T22:16:18.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.067+0000 7f2956e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29480981d0 0x7f294813f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:18.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.067+0000 7f2956e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f294809d2f0 0x7f294813fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:18.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.067+0000 7f2956e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2948143ec0 0x7f2948144340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:18.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.068+0000 7f2955a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29480981d0 0x7f294813f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:18.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.068+0000 7f2955a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29480981d0 0x7f294813f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43404/0 (socket says 172.21.15.131:43404) 2024-05-08T22:16:18.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.068+0000 7f2955a00700 1 -- 172.21.15.131:0/362439509 learned_addr learned my addr 172.21.15.131:0/362439509 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:18.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.068+0000 7f2956e00700 1 -- 172.21.15.131:0/362439509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2948144940 con 0x7f294809d2f0 2024-05-08T22:16:18.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.068+0000 7f2956e00700 1 -- 172.21.15.131:0/362439509 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2948144ab0 con 0x7f2948143ec0 2024-05-08T22:16:18.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.068+0000 7f2956e00700 1 -- 172.21.15.131:0/362439509 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2948144c70 con 0x7f29480981d0 2024-05-08T22:16:18.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.068+0000 7f2955a00700 1 -- 172.21.15.131:0/362439509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2948143ec0 msgr2=0x7f2948144340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:18.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.068+0000 7f2955a00700 1 --2- 172.21.15.131:0/362439509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2948143ec0 0x7f2948144340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.068+0000 7f2955a00700 1 -- 172.21.15.131:0/362439509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f294809d2f0 msgr2=0x7f294813fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:18.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.068+0000 7f2955a00700 1 --2- 172.21.15.131:0/362439509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f294809d2f0 0x7f294813fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.068+0000 7f2955a00700 1 -- 172.21.15.131:0/362439509 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2944009520 con 0x7f29480981d0 2024-05-08T22:16:18.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.072+0000 7f2955a00700 1 --2- 172.21.15.131:0/362439509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29480981d0 0x7f294813f5e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f293c00b530 tx=0x7f293c00b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:18.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.073+0000 7f294e000700 1 -- 172.21.15.131:0/362439509 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c011830 con 0x7f29480981d0 2024-05-08T22:16:18.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.073+0000 7f294e000700 1 -- 172.21.15.131:0/362439509 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f293c011e50 con 0x7f29480981d0 2024-05-08T22:16:18.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.073+0000 7f294e000700 1 -- 172.21.15.131:0/362439509 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c00f2e0 con 0x7f29480981d0 2024-05-08T22:16:18.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.076+0000 7f2956e00700 1 -- 172.21.15.131:0/362439509 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294814c1d0 con 0x7f29480981d0 2024-05-08T22:16:18.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.077+0000 7f2956e00700 1 -- 172.21.15.131:0/362439509 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f294814c660 con 0x7f29480981d0 2024-05-08T22:16:18.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.084+0000 7f296bfa6b80 1 Processor -- start 2024-05-08T22:16:18.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.084+0000 7f296bfa6b80 1 -- start start 2024-05-08T22:16:18.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.085+0000 7f296bfa6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d929e7050 0x558d929e7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:18.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.085+0000 7f294e000700 1 -- 172.21.15.131:0/362439509 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f293c010580 con 0x7f29480981d0 2024-05-08T22:16:18.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.087+0000 7f296bfa6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558d929e7970 0x558d929e1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:18.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.087+0000 7f296bfa6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d929e1890 0x558d929e1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:18.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.087+0000 7f296bfa6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d92938150 con 0x558d929e7050 2024-05-08T22:16:18.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.087+0000 7f296bfa6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d92935b10 con 0x558d929e7970 2024-05-08T22:16:18.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.087+0000 7f296bfa6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d92931400 con 0x558d929e1890 2024-05-08T22:16:18.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.088+0000 7f2956400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d929e7050 0x558d929e7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:18.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.088+0000 7f2956400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d929e7050 0x558d929e7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44572/0 (socket says 172.21.15.131:44572) 2024-05-08T22:16:18.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.088+0000 7f2956400700 1 -- 172.21.15.131:0/2582633660 learned_addr learned my addr 172.21.15.131:0/2582633660 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:18.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.088+0000 7f2956400700 1 -- 172.21.15.131:0/2582633660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d929e1890 msgr2=0x558d929e1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:18.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.088+0000 7f2956400700 1 --2- 172.21.15.131:0/2582633660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d929e1890 0x558d929e1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.088+0000 7f2956400700 1 -- 172.21.15.131:0/2582633660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558d929e7970 msgr2=0x558d929e1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:18.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.088+0000 7f2956400700 1 --2- 172.21.15.131:0/2582633660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558d929e7970 0x558d929e1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.088+0000 7f2956400700 1 -- 172.21.15.131:0/2582633660 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d927b23b0 con 0x558d929e7050 2024-05-08T22:16:18.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.090+0000 7f2956400700 1 --2- 172.21.15.131:0/2582633660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d929e7050 0x558d929e7430 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f29500504e0 tx=0x7f295006cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16748a242060b317 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:18.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.090+0000 7f2942a00700 1 -- 172.21.15.131:0/2582633660 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2950078710 con 0x558d929e7050 2024-05-08T22:16:18.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.090+0000 7f2942a00700 1 -- 172.21.15.131:0/2582633660 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f295006c300 con 0x558d929e7050 2024-05-08T22:16:18.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.090+0000 7f2942a00700 1 -- 172.21.15.131:0/2582633660 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2950069d50 con 0x558d929e7050 2024-05-08T22:16:18.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.092+0000 7f296bfa6b80 1 -- 172.21.15.131:0/2582633660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d929e7050 msgr2=0x558d929e7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:18.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.092+0000 7f296bfa6b80 1 --2- 172.21.15.131:0/2582633660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d929e7050 0x558d929e7430 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f29500504e0 tx=0x7f295006cb20 comp rx=0 tx=0).stop 2024-05-08T22:16:18.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.092+0000 7f296bfa6b80 1 -- 172.21.15.131:0/2582633660 shutdown_connections 2024-05-08T22:16:18.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.092+0000 7f296bfa6b80 1 --2- 172.21.15.131:0/2582633660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d929e1890 0x558d929e1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.093+0000 7f294e000700 1 --2- 172.21.15.131:0/362439509 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f29380a1390 0x7f29380a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.093+0000 7f294e000700 1 -- 172.21.15.131:0/362439509 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f293c0ccb90 con 0x7f29480981d0 2024-05-08T22:16:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.093+0000 7f2956400700 1 --2- 172.21.15.131:0/362439509 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f29380a1390 0x7f29380a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.094+0000 7f296bfa6b80 1 --2- 172.21.15.131:0/2582633660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d929e7050 0x558d929e7430 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.095+0000 7f296bfa6b80 1 --2- 172.21.15.131:0/2582633660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558d929e7970 0x558d929e1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.095+0000 7f296bfa6b80 1 -- 172.21.15.131:0/2582633660 >> 172.21.15.131:0/2582633660 conn(0x558d9293f210 msgr2=0x558d929dc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.095+0000 7f2956400700 1 --2- 172.21.15.131:0/362439509 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f29380a1390 0x7f29380a3850 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2950069fd0 tx=0x7f295007a040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.095+0000 7f296bfa6b80 1 -- 172.21.15.131:0/2582633660 shutdown_connections 2024-05-08T22:16:18.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.096+0000 7f296bfa6b80 1 -- 172.21.15.131:0/2582633660 wait complete. 2024-05-08T22:16:18.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.096+0000 7f296bfa6b80 1 -- start start 2024-05-08T22:16:18.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.097+0000 7f296bfa6b80 1 Processor -- start 2024-05-08T22:16:18.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.100+0000 7f296bfa6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d929e1890 0x558d929ed910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:18.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.100+0000 7f296bfa6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d929e7050 0x558d929eac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:18.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.101+0000 7f296bfa6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558d929e7970 0x558d929eb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:18.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.101+0000 7f296bfa6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d928cd560 con 0x558d929e1890 2024-05-08T22:16:18.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.101+0000 7f296bfa6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d928cd200 con 0x558d929e7970 2024-05-08T22:16:18.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.101+0000 7f296bfa6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d927b46a0 con 0x558d929e7050 2024-05-08T22:16:18.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.101+0000 7f296bfa6b80 10 client.0 _ll_register_callbacks cb 0x558d928d6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:16:18.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.102+0000 7f2955000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d929e7050 0x558d929eac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:18.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.102+0000 7f2955a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d929e1890 0x558d929ed910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:18.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.102+0000 7f2955a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d929e1890 0x558d929ed910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44576/0 (socket says 172.21.15.131:44576) 2024-05-08T22:16:18.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.102+0000 7f2955a00700 1 -- 172.21.15.131:0/568712196 learned_addr learned my addr 172.21.15.131:0/568712196 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:18.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.103+0000 7f2955a00700 1 -- 172.21.15.131:0/568712196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d929e7050 msgr2=0x558d929eac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:18.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.103+0000 7f2955a00700 1 --2- 172.21.15.131:0/568712196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d929e7050 0x558d929eac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.103+0000 7f2955a00700 1 -- 172.21.15.131:0/568712196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558d929e7970 msgr2=0x558d929eb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:18.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.103+0000 7f2955a00700 1 --2- 172.21.15.131:0/568712196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558d929e7970 0x558d929eb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.103+0000 7f2955a00700 1 -- 172.21.15.131:0/568712196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f295006c4c0 con 0x558d929e1890 2024-05-08T22:16:18.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.105+0000 7f2955a00700 1 --2- 172.21.15.131:0/568712196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d929e1890 0x558d929ed910 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f293c099610 tx=0x7f293c010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef4548cb9e55ba2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:18.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.105+0000 7f2937400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:16:18.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.105+0000 7f2937400700 1 -- 172.21.15.131:0/568712196 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c09d740 con 0x558d929e1890 2024-05-08T22:16:18.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.105+0000 7f2937400700 1 -- 172.21.15.131:0/568712196 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f293c0a1d90 con 0x558d929e1890 2024-05-08T22:16:18.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.106+0000 7f296bfa6b80 1 -- 172.21.15.131:0/568712196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d9288a270 con 0x558d929e1890 2024-05-08T22:16:18.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.107+0000 7f2937400700 1 -- 172.21.15.131:0/568712196 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f293c098e00 con 0x558d929e1890 2024-05-08T22:16:18.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.107+0000 7f296bfa6b80 10 client.16512 fetch_fsmap learned FSMap version 29 2024-05-08T22:16:18.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.107+0000 7f296bfa6b80 1 -- 172.21.15.131:0/568712196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558d9275eb60 con 0x558d929e1890 2024-05-08T22:16:18.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.108+0000 7f2937400700 1 -- 172.21.15.131:0/568712196 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558d9288a270 con 0x558d929e1890 2024-05-08T22:16:18.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.108+0000 7f2937400700 1 -- 172.21.15.131:0/568712196 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f293c0b9280 con 0x558d929e1890 2024-05-08T22:16:18.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.108+0000 7f296bfa6b80 10 client.16512 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:16:18.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.109+0000 7f296bfa6b80 10 client.16512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:16:18.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.109+0000 7f296bfa6b80 20 client.16512 populate_metadata read hostname 'smithi131' 2024-05-08T22:16:18.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.109+0000 7f296bfa6b80 1 --2- 172.21.15.131:0/568712196 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x558d929fb2f0 0x558d929fd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:18.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.109+0000 7f296bfa6b80 4 client.16512 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:16:18.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.109+0000 7f296bfa6b80 1 -- 172.21.15.131:0/568712196 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d929b0810 con 0x558d929fb2f0 2024-05-08T22:16:18.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.117+0000 7f2955000700 1 --2- 172.21.15.131:0/568712196 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x558d929fb2f0 0x558d929fd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:18.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.125+0000 7f2955000700 1 --2- 172.21.15.131:0/568712196 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x558d929fb2f0 0x558d929fd710 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87b9759822cb1cb3 server_cookie=7571d79b343ea62a in_seq=0 out_seq=0 2024-05-08T22:16:18.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.125+0000 7f2937400700 10 client.16512 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:18.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.132+0000 7f2937400700 1 -- 172.21.15.131:0/568712196 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558d929b0810 con 0x558d929fb2f0 2024-05-08T22:16:18.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.132+0000 7f2937400700 10 client.16512 handle_command_reply: tid=0 2024-05-08T22:16:18.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.132+0000 7f296bfa6b80 1 client.16512 shutdown 2024-05-08T22:16:18.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.135+0000 7f296bfa6b80 1 -- 172.21.15.131:0/568712196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d929e1890 msgr2=0x558d929ed910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:18.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.135+0000 7f296bfa6b80 1 --2- 172.21.15.131:0/568712196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d929e1890 0x558d929ed910 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f293c099610 tx=0x7f293c010430 comp rx=0 tx=0).stop 2024-05-08T22:16:18.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.135+0000 7f296bfa6b80 1 -- 172.21.15.131:0/568712196 shutdown_connections 2024-05-08T22:16:18.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.135+0000 7f296bfa6b80 1 --2- 172.21.15.131:0/568712196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558d929e7970 0x558d929eb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.135+0000 7f296bfa6b80 1 --2- 172.21.15.131:0/568712196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d929e1890 0x558d929ed910 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.135+0000 7f296bfa6b80 1 --2- 172.21.15.131:0/568712196 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x558d929fb2f0 0x558d929fd710 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.135+0000 7f296bfa6b80 1 --2- 172.21.15.131:0/568712196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d929e7050 0x558d929eac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.135+0000 7f296bfa6b80 1 -- 172.21.15.131:0/568712196 >> 172.21.15.131:0/568712196 conn(0x558d9293f210 msgr2=0x558d929dc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:18.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.135+0000 7f2937400700 0 client.16512 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:18.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.135+0000 7f296bfa6b80 1 -- 172.21.15.131:0/568712196 shutdown_connections 2024-05-08T22:16:18.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.135+0000 7f296bfa6b80 1 -- 172.21.15.131:0/568712196 wait complete. 2024-05-08T22:16:18.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.136+0000 7f296bfa6b80 20 client.16512 trim_cache size 0 max 16384 2024-05-08T22:16:18.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.180+0000 7f296bfa6b80 1 Processor -- start 2024-05-08T22:16:18.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.180+0000 7f296bfa6b80 1 -- start start 2024-05-08T22:16:18.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.181+0000 7f296bfa6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d929e1890 0x558d929f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:18.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.181+0000 7f296bfa6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d929e7050 0x558d929ed440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:18.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.181+0000 7f296bfa6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558d929e7970 0x558d929e4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:18.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.181+0000 7f296bfa6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d929f8a70 con 0x558d929e1890 2024-05-08T22:16:18.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.181+0000 7f296bfa6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d929d6800 con 0x558d929e7970 2024-05-08T22:16:18.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.181+0000 7f296bfa6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d929e9150 con 0x558d929e7050 2024-05-08T22:16:18.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.181+0000 7f2955000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d929e7050 0x558d929ed440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:18.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.181+0000 7f2955000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d929e7050 0x558d929ed440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43448/0 (socket says 172.21.15.131:43448) 2024-05-08T22:16:18.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.181+0000 7f2955000700 1 -- 172.21.15.131:0/728468482 learned_addr learned my addr 172.21.15.131:0/728468482 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:18.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.182+0000 7f2955000700 1 -- 172.21.15.131:0/728468482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558d929e7970 msgr2=0x558d929e4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:18.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.182+0000 7f2955000700 1 --2- 172.21.15.131:0/728468482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558d929e7970 0x558d929e4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.182+0000 7f2955000700 1 -- 172.21.15.131:0/728468482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d929e1890 msgr2=0x558d929f73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:18.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.182+0000 7f2955000700 1 --2- 172.21.15.131:0/728468482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d929e1890 0x558d929f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.182+0000 7f2955000700 1 -- 172.21.15.131:0/728468482 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d929e1f40 con 0x558d929e7050 2024-05-08T22:16:18.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.182+0000 7f2955000700 1 --2- 172.21.15.131:0/728468482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d929e7050 0x558d929ed440 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2944016000 tx=0x7f2944009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=586722e0d4aa9951 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:18.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.183+0000 7f2943400700 1 -- 172.21.15.131:0/728468482 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294401d040 con 0x558d929e7050 2024-05-08T22:16:18.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.183+0000 7f2943400700 1 -- 172.21.15.131:0/728468482 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2944010cf0 con 0x558d929e7050 2024-05-08T22:16:18.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.183+0000 7f2943400700 1 -- 172.21.15.131:0/728468482 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2944021a90 con 0x558d929e7050 2024-05-08T22:16:18.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.184+0000 7f296bfa6b80 1 -- 172.21.15.131:0/728468482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d929e7050 msgr2=0x558d929ed440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:18.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.184+0000 7f296bfa6b80 1 --2- 172.21.15.131:0/728468482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d929e7050 0x558d929ed440 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2944016000 tx=0x7f2944009400 comp rx=0 tx=0).stop 2024-05-08T22:16:18.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.185+0000 7f296bfa6b80 1 -- 172.21.15.131:0/728468482 shutdown_connections 2024-05-08T22:16:18.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.185+0000 7f296bfa6b80 1 --2- 172.21.15.131:0/728468482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558d929e7970 0x558d929e4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.185+0000 7f296bfa6b80 1 --2- 172.21.15.131:0/728468482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d929e1890 0x558d929f73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.185+0000 7f296bfa6b80 1 --2- 172.21.15.131:0/728468482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d929e7050 0x558d929ed440 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.185+0000 7f296bfa6b80 1 -- 172.21.15.131:0/728468482 >> 172.21.15.131:0/728468482 conn(0x558d9293f210 msgr2=0x558d929fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:18.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.185+0000 7f296bfa6b80 1 -- 172.21.15.131:0/728468482 shutdown_connections 2024-05-08T22:16:18.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.185+0000 7f296bfa6b80 1 -- 172.21.15.131:0/728468482 wait complete. 2024-05-08T22:16:18.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.186+0000 7f296bfa6b80 1 -- start start 2024-05-08T22:16:18.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.186+0000 7f296bfa6b80 1 Processor -- start 2024-05-08T22:16:18.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.188+0000 7f296bfa6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558d929e1890 0x558d92a01d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:18.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.188+0000 7f296bfa6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d929e7050 0x558d92a02260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:18.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.189+0000 7f296bfa6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d929e7970 0x558d929e2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:18.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.189+0000 7f296bfa6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d929e3080 con 0x558d929e7050 2024-05-08T22:16:18.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.189+0000 7f296bfa6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d929e31f0 con 0x558d929e1890 2024-05-08T22:16:18.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.189+0000 7f296bfa6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d929e3360 con 0x558d929e7970 2024-05-08T22:16:18.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.189+0000 7f296bfa6b80 10 client.0 _ll_register_callbacks cb 0x558d928d6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:16:18.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.189+0000 7f2955a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558d929e1890 0x558d92a01d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:18.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.190+0000 7f2955000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d929e7050 0x558d92a02260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:18.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.190+0000 7f2955000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d929e7050 0x558d92a02260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44594/0 (socket says 172.21.15.131:44594) 2024-05-08T22:16:18.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.190+0000 7f2955000700 1 -- 172.21.15.131:0/3161229621 learned_addr learned my addr 172.21.15.131:0/3161229621 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:18.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.190+0000 7f2955000700 1 -- 172.21.15.131:0/3161229621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d929e7970 msgr2=0x558d929e2b40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:16:18.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.190+0000 7f2955000700 1 --2- 172.21.15.131:0/3161229621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d929e7970 0x558d929e2b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.190+0000 7f2955000700 1 -- 172.21.15.131:0/3161229621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558d929e1890 msgr2=0x558d92a01d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:18.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.190+0000 7f2955000700 1 --2- 172.21.15.131:0/3161229621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558d929e1890 0x558d92a01d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.190+0000 7f2955000700 1 -- 172.21.15.131:0/3161229621 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29480a56e0 con 0x558d929e7050 2024-05-08T22:16:18.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.191+0000 7f2955000700 1 --2- 172.21.15.131:0/3161229621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d929e7050 0x558d92a02260 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2944000c00 tx=0x7f2944010af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a11313b553d042 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:18.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.191+0000 7f2937400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:16:18.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.193+0000 7f2937400700 1 -- 172.21.15.131:0/3161229621 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294401d040 con 0x558d929e7050 2024-05-08T22:16:18.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.193+0000 7f2937400700 1 -- 172.21.15.131:0/3161229621 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f2944021a20 con 0x558d929e7050 2024-05-08T22:16:18.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.193+0000 7f296bfa6b80 1 -- 172.21.15.131:0/3161229621 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f295006c300 con 0x558d929e7050 2024-05-08T22:16:18.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.195+0000 7f2937400700 1 -- 172.21.15.131:0/3161229621 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f294400f660 con 0x558d929e7050 2024-05-08T22:16:18.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.197+0000 7f296bfa6b80 10 client.16515 fetch_fsmap learned FSMap version 29 2024-05-08T22:16:18.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.197+0000 7f296bfa6b80 1 -- 172.21.15.131:0/3161229621 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558d928f7560 con 0x558d929e7050 2024-05-08T22:16:18.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.199+0000 7f2937400700 1 -- 172.21.15.131:0/3161229621 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f295006c300 con 0x558d929e7050 2024-05-08T22:16:18.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.199+0000 7f2937400700 1 -- 172.21.15.131:0/3161229621 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f2944028050 con 0x558d929e7050 2024-05-08T22:16:18.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.199+0000 7f296bfa6b80 10 client.16515 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:16:18.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.199+0000 7f296bfa6b80 10 client.16515 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:16:18.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.199+0000 7f296bfa6b80 20 client.16515 populate_metadata read hostname 'smithi131' 2024-05-08T22:16:18.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.199+0000 7f296bfa6b80 1 --2- 172.21.15.131:0/3161229621 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x558d92a522a0 0x558d92a54690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:18.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.199+0000 7f296bfa6b80 4 client.16515 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:16:18.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.199+0000 7f296bfa6b80 1 -- 172.21.15.131:0/3161229621 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558d92929ec0 con 0x558d92a522a0 2024-05-08T22:16:18.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.205+0000 7f2955a00700 1 --2- 172.21.15.131:0/3161229621 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x558d92a522a0 0x558d92a54690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:18.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.211+0000 7f2955a00700 1 --2- 172.21.15.131:0/3161229621 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x558d92a522a0 0x558d92a54690 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=693cb2cdfb0ccdbd server_cookie=50dd5ff0a819f8c in_seq=0 out_seq=0 2024-05-08T22:16:18.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.211+0000 7f2937400700 10 client.16515 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:18.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.219+0000 7f2937400700 1 -- 172.21.15.131:0/3161229621 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558d92929ec0 con 0x558d92a522a0 2024-05-08T22:16:18.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.219+0000 7f2937400700 10 client.16515 handle_command_reply: tid=0 2024-05-08T22:16:18.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.221+0000 7f296bfa6b80 1 client.16515 shutdown 2024-05-08T22:16:18.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.222+0000 7f296bfa6b80 1 -- 172.21.15.131:0/3161229621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d929e7050 msgr2=0x558d92a02260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:18.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.222+0000 7f296bfa6b80 1 --2- 172.21.15.131:0/3161229621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d929e7050 0x558d92a02260 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2944000c00 tx=0x7f2944010af0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.222+0000 7f296bfa6b80 1 -- 172.21.15.131:0/3161229621 shutdown_connections 2024-05-08T22:16:18.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.222+0000 7f296bfa6b80 1 --2- 172.21.15.131:0/3161229621 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x558d92a522a0 0x558d92a54690 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.222+0000 7f296bfa6b80 1 --2- 172.21.15.131:0/3161229621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d929e7970 0x558d929e2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.222+0000 7f296bfa6b80 1 --2- 172.21.15.131:0/3161229621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558d929e1890 0x558d92a01d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.222+0000 7f296bfa6b80 1 --2- 172.21.15.131:0/3161229621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d929e7050 0x558d92a02260 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.222+0000 7f296bfa6b80 1 -- 172.21.15.131:0/3161229621 >> 172.21.15.131:0/3161229621 conn(0x558d9293f210 msgr2=0x558d929e4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:18.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.222+0000 7f2937400700 0 client.16515 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:18.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.222+0000 7f296bfa6b80 1 -- 172.21.15.131:0/3161229621 shutdown_connections 2024-05-08T22:16:18.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.222+0000 7f296bfa6b80 1 -- 172.21.15.131:0/3161229621 wait complete. 2024-05-08T22:16:18.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.223+0000 7f296bfa6b80 20 client.16515 trim_cache size 0 max 16384 2024-05-08T22:16:18.225 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:16:18.225 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-08T22:16:18.225 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-08T22:16:18.225 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:16:18.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.223+0000 7f2956e00700 1 -- 172.21.15.131:0/362439509 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f29380a1390 msgr2=0x7f29380a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:18.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.223+0000 7f2956e00700 1 --2- 172.21.15.131:0/362439509 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f29380a1390 0x7f29380a3850 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2950069fd0 tx=0x7f295007a040 comp rx=0 tx=0).stop 2024-05-08T22:16:18.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.223+0000 7f2956e00700 1 -- 172.21.15.131:0/362439509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29480981d0 msgr2=0x7f294813f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:18.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.223+0000 7f2956e00700 1 --2- 172.21.15.131:0/362439509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29480981d0 0x7f294813f5e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f293c00b530 tx=0x7f293c00b8f0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.224+0000 7f2956e00700 1 -- 172.21.15.131:0/362439509 shutdown_connections 2024-05-08T22:16:18.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.224+0000 7f2956e00700 1 --2- 172.21.15.131:0/362439509 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f29380a1390 0x7f29380a3850 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.224+0000 7f2956e00700 1 --2- 172.21.15.131:0/362439509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2948143ec0 0x7f2948144340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.224+0000 7f2956e00700 1 --2- 172.21.15.131:0/362439509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29480981d0 0x7f294813f5e0 secure :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f293c00b530 tx=0x7f293c00b8f0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.224+0000 7f2956e00700 1 --2- 172.21.15.131:0/362439509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f294809d2f0 0x7f294813fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.224+0000 7f2956e00700 1 -- 172.21.15.131:0/362439509 >> 172.21.15.131:0/362439509 conn(0x7f2948093a50 msgr2=0x7f2948094e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:18.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.224+0000 7f2956e00700 1 -- 172.21.15.131:0/362439509 shutdown_connections 2024-05-08T22:16:18.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.224+0000 7f2956e00700 1 -- 172.21.15.131:0/362439509 wait complete. 2024-05-08T22:16:18.259 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T22:16:18.259 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:16:18.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.604+0000 7f3cca000700 1 -- 172.21.15.131:0/1498972205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc4106160 msgr2=0x7f3cc410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:18.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.604+0000 7f3cca000700 1 --2- 172.21.15.131:0/1498972205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc4106160 0x7f3cc410dd30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009f90 tx=0x7f3cbc00b330 comp rx=0 tx=0).stop 2024-05-08T22:16:18.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.605+0000 7f3cca000700 1 -- 172.21.15.131:0/1498972205 shutdown_connections 2024-05-08T22:16:18.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.605+0000 7f3cca000700 1 --2- 172.21.15.131:0/1498972205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc4106160 0x7f3cc410dd30 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.605+0000 7f3cca000700 1 --2- 172.21.15.131:0/1498972205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cc40ff5a0 0x7f3cc40ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.605+0000 7f3cca000700 1 --2- 172.21.15.131:0/1498972205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cc40fff50 0x7f3cc4105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.605+0000 7f3cca000700 1 -- 172.21.15.131:0/1498972205 >> 172.21.15.131:0/1498972205 conn(0x7f3cc4076270 msgr2=0x7f3cc40fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:18.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.607+0000 7f3cca000700 1 -- 172.21.15.131:0/1498972205 shutdown_connections 2024-05-08T22:16:18.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.608+0000 7f3cca000700 1 -- 172.21.15.131:0/1498972205 wait complete. 2024-05-08T22:16:18.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.609+0000 7f3cca000700 1 Processor -- start 2024-05-08T22:16:18.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.609+0000 7f3cca000700 1 -- start start 2024-05-08T22:16:18.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.610+0000 7f3cca000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc40ff5a0 0x7f3cc410ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:18.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.610+0000 7f3cca000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cc4106160 0x7f3cc410d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:18.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.610+0000 7f3cca000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cc410da00 0x7f3cc4199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:18.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.610+0000 7f3cca000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc419a380 con 0x7f3cc40ff5a0 2024-05-08T22:16:18.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.610+0000 7f3cca000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc419a4f0 con 0x7f3cc410da00 2024-05-08T22:16:18.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.610+0000 7f3cca000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc419a6b0 con 0x7f3cc4106160 2024-05-08T22:16:18.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.610+0000 7f3cc3e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cc410da00 0x7f3cc4199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:18.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.610+0000 7f3cc2a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cc4106160 0x7f3cc410d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:18.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.610+0000 7f3cc2a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cc4106160 0x7f3cc410d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43480/0 (socket says 172.21.15.131:43480) 2024-05-08T22:16:18.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.610+0000 7f3cc3e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cc410da00 0x7f3cc4199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:34618/0 (socket says 172.21.15.131:34618) 2024-05-08T22:16:18.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.610+0000 7f3cc2a00700 1 -- 172.21.15.131:0/2881815441 learned_addr learned my addr 172.21.15.131:0/2881815441 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:18.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.611+0000 7f3cc3400700 1 --2- 172.21.15.131:0/2881815441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc40ff5a0 0x7f3cc410ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:18.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.611+0000 7f3cc2a00700 1 -- 172.21.15.131:0/2881815441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cc410da00 msgr2=0x7f3cc4199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:18.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.611+0000 7f3cc2a00700 1 --2- 172.21.15.131:0/2881815441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cc410da00 0x7f3cc4199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.611+0000 7f3cc2a00700 1 -- 172.21.15.131:0/2881815441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc40ff5a0 msgr2=0x7f3cc410ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:18.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.611+0000 7f3cc2a00700 1 --2- 172.21.15.131:0/2881815441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc40ff5a0 0x7f3cc410ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:18.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.611+0000 7f3cc2a00700 1 -- 172.21.15.131:0/2881815441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cbc009c40 con 0x7f3cc4106160 2024-05-08T22:16:18.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.611+0000 7f3cc3400700 1 --2- 172.21.15.131:0/2881815441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc40ff5a0 0x7f3cc410ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:16:18.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.611+0000 7f3cc2a00700 1 --2- 172.21.15.131:0/2881815441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cc4106160 0x7f3cc410d360 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000e9c0 tx=0x7f3cb000ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:18.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.612+0000 7f3cbbe00700 1 -- 172.21.15.131:0/2881815441 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0009690 con 0x7f3cc4106160 2024-05-08T22:16:18.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.612+0000 7f3cbbe00700 1 -- 172.21.15.131:0/2881815441 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cb000eea0 con 0x7f3cc4106160 2024-05-08T22:16:18.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.612+0000 7f3cbbe00700 1 -- 172.21.15.131:0/2881815441 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0009690 con 0x7f3cc4106160 2024-05-08T22:16:18.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.612+0000 7f3cca000700 1 -- 172.21.15.131:0/2881815441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc419a990 con 0x7f3cc4106160 2024-05-08T22:16:18.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.612+0000 7f3cca000700 1 -- 172.21.15.131:0/2881815441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc41a2340 con 0x7f3cc4106160 2024-05-08T22:16:18.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.613+0000 7f3cca000700 1 -- 172.21.15.131:0/2881815441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc4105d60 con 0x7f3cc4106160 2024-05-08T22:16:18.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.616+0000 7f3cbbe00700 1 -- 172.21.15.131:0/2881815441 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f3cb000cbd0 con 0x7f3cc4106160 2024-05-08T22:16:18.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.617+0000 7f3cbbe00700 1 --2- 172.21.15.131:0/2881815441 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3ca80a13a0 0x7f3ca80a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:18.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.618+0000 7f3cbbe00700 1 -- 172.21.15.131:0/2881815441 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f3cb0014070 con 0x7f3cc4106160 2024-05-08T22:16:18.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.619+0000 7f3cc3400700 1 --2- 172.21.15.131:0/2881815441 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3ca80a13a0 0x7f3ca80a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:18.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.620+0000 7f3cc3400700 1 --2- 172.21.15.131:0/2881815441 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3ca80a13a0 0x7f3ca80a3860 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3cb40099d0 tx=0x7f3cb4008040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:18.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:18.623+0000 7f3cbbe00700 1 -- 172.21.15.131:0/2881815441 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3cb0052d60 con 0x7f3cc4106160 2024-05-08T22:16:18.843 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:18 smithi132 ceph-mon[78180]: pgmap v278: 129 pgs: 16 active+clean+snaptrim, 57 active+clean, 56 active+clean+snaptrim_wait; 1.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 479 op/s 2024-05-08T22:16:18.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:18 smithi131 ceph-mon[37260]: pgmap v278: 129 pgs: 16 active+clean+snaptrim, 57 active+clean, 56 active+clean+snaptrim_wait; 1.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 479 op/s 2024-05-08T22:16:19.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:18 smithi157 ceph-mon[77746]: pgmap v278: 129 pgs: 16 active+clean+snaptrim, 57 active+clean, 56 active+clean+snaptrim_wait; 1.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 479 op/s 2024-05-08T22:16:19.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.138+0000 7f3cca000700 1 -- 172.21.15.131:0/2881815441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3cc4102650 con 0x7f3cc4106160 2024-05-08T22:16:19.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.140+0000 7f3cbbe00700 1 -- 172.21.15.131:0/2881815441 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10892 (secure 0 0 0) 0x7f3cb00d0030 con 0x7f3cc4106160 2024-05-08T22:16:19.141 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:16:19.142 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:05:14.034908+0000","modified":"2024-05-08T22:15:23.658983+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:16:19.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.144+0000 7f3cca000700 1 -- 172.21.15.131:0/2881815441 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3ca80a13a0 msgr2=0x7f3ca80a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:19.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.144+0000 7f3cca000700 1 --2- 172.21.15.131:0/2881815441 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3ca80a13a0 0x7f3ca80a3860 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3cb40099d0 tx=0x7f3cb4008040 comp rx=0 tx=0).stop 2024-05-08T22:16:19.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.144+0000 7f3cca000700 1 -- 172.21.15.131:0/2881815441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cc4106160 msgr2=0x7f3cc410d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:19.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.144+0000 7f3cca000700 1 --2- 172.21.15.131:0/2881815441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cc4106160 0x7f3cc410d360 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000e9c0 tx=0x7f3cb000ed80 comp rx=0 tx=0).stop 2024-05-08T22:16:19.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.144+0000 7f3cca000700 1 -- 172.21.15.131:0/2881815441 shutdown_connections 2024-05-08T22:16:19.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.144+0000 7f3cca000700 1 --2- 172.21.15.131:0/2881815441 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3ca80a13a0 0x7f3ca80a3860 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.144+0000 7f3cca000700 1 --2- 172.21.15.131:0/2881815441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cc410da00 0x7f3cc4199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.144+0000 7f3cca000700 1 --2- 172.21.15.131:0/2881815441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc40ff5a0 0x7f3cc410ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.144+0000 7f3cca000700 1 --2- 172.21.15.131:0/2881815441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cc4106160 0x7f3cc410d360 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.144+0000 7f3cca000700 1 -- 172.21.15.131:0/2881815441 >> 172.21.15.131:0/2881815441 conn(0x7f3cc4076270 msgr2=0x7f3cc41040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:19.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.145+0000 7f3cca000700 1 -- 172.21.15.131:0/2881815441 shutdown_connections 2024-05-08T22:16:19.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.145+0000 7f3cca000700 1 -- 172.21.15.131:0/2881815441 wait complete. 2024-05-08T22:16:19.150 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 29 2024-05-08T22:16:19.180 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T22:16:19.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.548+0000 7f7af8800700 1 -- 172.21.15.131:0/3860475649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7af0068590 msgr2=0x7f7af0068a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:19.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.548+0000 7f7af8800700 1 --2- 172.21.15.131:0/3860475649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7af0068590 0x7f7af0068a10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7ae00098f0 tx=0x7f7ae0009c00 comp rx=0 tx=0).stop 2024-05-08T22:16:19.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.549+0000 7f7af8800700 1 -- 172.21.15.131:0/3860475649 shutdown_connections 2024-05-08T22:16:19.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.549+0000 7f7af8800700 1 --2- 172.21.15.131:0/3860475649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7af0103930 0x7f7af0103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.549+0000 7f7af8800700 1 --2- 172.21.15.131:0/3860475649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af0105e30 0x7f7af0068050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.549+0000 7f7af8800700 1 --2- 172.21.15.131:0/3860475649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7af0068590 0x7f7af0068a10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.549+0000 7f7af8800700 1 -- 172.21.15.131:0/3860475649 >> 172.21.15.131:0/3860475649 conn(0x7f7af0076f40 msgr2=0x7f7af0077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:19.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.549+0000 7f7af8800700 1 -- 172.21.15.131:0/3860475649 shutdown_connections 2024-05-08T22:16:19.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.549+0000 7f7af8800700 1 -- 172.21.15.131:0/3860475649 wait complete. 2024-05-08T22:16:19.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.551+0000 7f7af8800700 1 Processor -- start 2024-05-08T22:16:19.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.551+0000 7f7af8800700 1 -- start start 2024-05-08T22:16:19.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.552+0000 7f7af8800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af0068590 0x7f7af019da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:19.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.552+0000 7f7af8800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7af0103930 0x7f7af019df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:19.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.552+0000 7f7af8800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7af0105e30 0x7f7af01a2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:19.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.552+0000 7f7af8800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af01a2920 con 0x7f7af0068590 2024-05-08T22:16:19.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.552+0000 7f7af8800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af01a2a90 con 0x7f7af0105e30 2024-05-08T22:16:19.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.552+0000 7f7af8800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af01a2c50 con 0x7f7af0103930 2024-05-08T22:16:19.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.552+0000 7f7af5400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7af0103930 0x7f7af019df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:19.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.552+0000 7f7af5400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7af0103930 0x7f7af019df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43496/0 (socket says 172.21.15.131:43496) 2024-05-08T22:16:19.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.552+0000 7f7af5400700 1 -- 172.21.15.131:0/3267312925 learned_addr learned my addr 172.21.15.131:0/3267312925 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:19.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.553+0000 7f7af6800700 1 --2- 172.21.15.131:0/3267312925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7af0105e30 0x7f7af01a2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:19.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.553+0000 7f7af5e00700 1 --2- 172.21.15.131:0/3267312925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af0068590 0x7f7af019da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:19.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.555+0000 7f7af5400700 1 -- 172.21.15.131:0/3267312925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7af0105e30 msgr2=0x7f7af01a2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:19.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.555+0000 7f7af5400700 1 --2- 172.21.15.131:0/3267312925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7af0105e30 0x7f7af01a2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.555+0000 7f7af5400700 1 -- 172.21.15.131:0/3267312925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af0068590 msgr2=0x7f7af019da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:19.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.555+0000 7f7af5400700 1 --2- 172.21.15.131:0/3267312925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af0068590 0x7f7af019da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.555+0000 7f7af5400700 1 -- 172.21.15.131:0/3267312925 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae0009520 con 0x7f7af0103930 2024-05-08T22:16:19.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.556+0000 7f7af5400700 1 --2- 172.21.15.131:0/3267312925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7af0103930 0x7f7af019df80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0005950 tx=0x7f7ae0010ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:19.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.556+0000 7f7aeaa00700 1 -- 172.21.15.131:0/3267312925 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae001c070 con 0x7f7af0103930 2024-05-08T22:16:19.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.557+0000 7f7aeaa00700 1 -- 172.21.15.131:0/3267312925 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ae000bba0 con 0x7f7af0103930 2024-05-08T22:16:19.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.557+0000 7f7aeaa00700 1 -- 172.21.15.131:0/3267312925 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae00039c0 con 0x7f7af0103930 2024-05-08T22:16:19.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.557+0000 7f7af8800700 1 -- 172.21.15.131:0/3267312925 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af01a2ed0 con 0x7f7af0103930 2024-05-08T22:16:19.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.557+0000 7f7af8800700 1 -- 172.21.15.131:0/3267312925 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af01aa600 con 0x7f7af0103930 2024-05-08T22:16:19.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.562+0000 7f7aeaa00700 1 -- 172.21.15.131:0/3267312925 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f7ae0020030 con 0x7f7af0103930 2024-05-08T22:16:19.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.564+0000 7f7b0c2eab80 1 Processor -- start 2024-05-08T22:16:19.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.564+0000 7f7b0c2eab80 1 -- start start 2024-05-08T22:16:19.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.565+0000 7f7b0c2eab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc9c3fae00 0x55fc9c3fb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:19.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.565+0000 7f7b0c2eab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc9c3fb720 0x55fc9c3f5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:19.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.565+0000 7f7b0c2eab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fc9c3f5890 0x55fc9c3f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:19.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.565+0000 7f7b0c2eab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc9c34c150 con 0x55fc9c3fae00 2024-05-08T22:16:19.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.565+0000 7f7b0c2eab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc9c349b10 con 0x55fc9c3f5890 2024-05-08T22:16:19.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.565+0000 7f7b0c2eab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc9c345400 con 0x55fc9c3fb720 2024-05-08T22:16:19.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.566+0000 7f7af6800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc9c3fae00 0x55fc9c3fb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:19.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.566+0000 7f7af6800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc9c3fae00 0x55fc9c3fb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44652/0 (socket says 172.21.15.131:44652) 2024-05-08T22:16:19.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.566+0000 7f7af6800700 1 -- 172.21.15.131:0/3623969830 learned_addr learned my addr 172.21.15.131:0/3623969830 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:19.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.567+0000 7f7af5e00700 1 --2- 172.21.15.131:0/3623969830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc9c3fb720 0x55fc9c3f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:19.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.567+0000 7f7aeaa00700 1 --2- 172.21.15.131:0/3267312925 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7ad80a1310 0x7f7ad80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:19.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.567+0000 7f7af5e00700 1 -- 172.21.15.131:0/3623969830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fc9c3f5890 msgr2=0x55fc9c3f5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:19.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.567+0000 7f7af5e00700 1 --2- 172.21.15.131:0/3623969830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fc9c3f5890 0x55fc9c3f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.567+0000 7f7af5e00700 1 -- 172.21.15.131:0/3623969830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc9c3fae00 msgr2=0x55fc9c3fb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:19.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.567+0000 7f7af5e00700 1 --2- 172.21.15.131:0/3623969830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc9c3fae00 0x55fc9c3fb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.567+0000 7f7af5e00700 1 -- 172.21.15.131:0/3623969830 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc9c1c63b0 con 0x55fc9c3fb720 2024-05-08T22:16:19.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.568+0000 7f7af5e00700 1 --2- 172.21.15.131:0/3623969830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc9c3fb720 0x55fc9c3f5350 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009be0 tx=0x7f7ae4009ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a8e058159e03f6b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:19.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.568+0000 7f7aeaa00700 1 -- 172.21.15.131:0/3267312925 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f7ae00cdf90 con 0x7f7af0103930 2024-05-08T22:16:19.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.568+0000 7f7adf000700 1 -- 172.21.15.131:0/3623969830 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4010b80 con 0x55fc9c3fb720 2024-05-08T22:16:19.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.569+0000 7f7af6800700 1 --2- 172.21.15.131:0/3267312925 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7ad80a1310 0x7f7ad80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:19.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.569+0000 7f7adf000700 1 -- 172.21.15.131:0/3623969830 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ae40044f0 con 0x55fc9c3fb720 2024-05-08T22:16:19.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.569+0000 7f7adf000700 1 -- 172.21.15.131:0/3623969830 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4003e60 con 0x55fc9c3fb720 2024-05-08T22:16:19.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.570+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/3623969830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc9c3fb720 msgr2=0x55fc9c3f5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:19.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.570+0000 7f7b0c2eab80 1 --2- 172.21.15.131:0/3623969830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc9c3fb720 0x55fc9c3f5350 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009be0 tx=0x7f7ae4009ef0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.570+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/3623969830 shutdown_connections 2024-05-08T22:16:19.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.570+0000 7f7b0c2eab80 1 --2- 172.21.15.131:0/3623969830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fc9c3f5890 0x55fc9c3f5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.570+0000 7f7b0c2eab80 1 --2- 172.21.15.131:0/3623969830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc9c3fae00 0x55fc9c3fb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.570+0000 7f7b0c2eab80 1 --2- 172.21.15.131:0/3623969830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc9c3fb720 0x55fc9c3f5350 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.570+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/3623969830 >> 172.21.15.131:0/3623969830 conn(0x55fc9c353210 msgr2=0x55fc9c3efdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:19.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.570+0000 7f7af6800700 1 --2- 172.21.15.131:0/3267312925 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7ad80a1310 0x7f7ad80a37d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009fd0 tx=0x7f7aec009040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:19.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.572+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/3623969830 shutdown_connections 2024-05-08T22:16:19.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.573+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/3623969830 wait complete. 2024-05-08T22:16:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.574+0000 7f7b0c2eab80 1 -- start start 2024-05-08T22:16:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.574+0000 7f7b0c2eab80 1 Processor -- start 2024-05-08T22:16:19.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.576+0000 7f7b0c2eab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc9c3f5890 0x55fc9c3feb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:19.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.576+0000 7f7b0c2eab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc9c3fae00 0x55fc9c3ff060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:19.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.576+0000 7f7af5e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc9c3f5890 0x55fc9c3feb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:19.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.576+0000 7f7b0c2eab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fc9c3fb720 0x55fc9c3ff5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:19.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.577+0000 7f7b0c2eab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc9c2e1560 con 0x55fc9c3f5890 2024-05-08T22:16:19.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.577+0000 7f7b0c2eab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc9c2e1200 con 0x55fc9c3fb720 2024-05-08T22:16:19.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.577+0000 7f7b0c2eab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc9c1c86a0 con 0x55fc9c3fae00 2024-05-08T22:16:19.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.577+0000 7f7b0c2eab80 10 client.0 _ll_register_callbacks cb 0x55fc9c2eab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:16:19.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.577+0000 7f7af5e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc9c3f5890 0x55fc9c3feb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44660/0 (socket says 172.21.15.131:44660) 2024-05-08T22:16:19.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.577+0000 7f7af5e00700 1 -- 172.21.15.131:0/1049255911 learned_addr learned my addr 172.21.15.131:0/1049255911 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:19.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.577+0000 7f7af5400700 1 --2- 172.21.15.131:0/1049255911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc9c3fae00 0x55fc9c3ff060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:19.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.578+0000 7f7af5400700 1 -- 172.21.15.131:0/1049255911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fc9c3fb720 msgr2=0x55fc9c3ff5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:19.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.578+0000 7f7af5400700 1 --2- 172.21.15.131:0/1049255911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fc9c3fb720 0x55fc9c3ff5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.578+0000 7f7af5400700 1 -- 172.21.15.131:0/1049255911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc9c3f5890 msgr2=0x55fc9c3feb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:19.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.578+0000 7f7af5400700 1 --2- 172.21.15.131:0/1049255911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc9c3f5890 0x55fc9c3feb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.578+0000 7f7af5400700 1 -- 172.21.15.131:0/1049255911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae4009810 con 0x55fc9c3fae00 2024-05-08T22:16:19.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.578+0000 7f7af5400700 1 --2- 172.21.15.131:0/1049255911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc9c3fae00 0x55fc9c3ff060 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7ae00cb2a0 tx=0x7f7ae009a700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=450074c8ade5a8d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:19.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.578+0000 7f7ad3400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:16:19.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.578+0000 7f7ad3400700 1 -- 172.21.15.131:0/1049255911 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae009eb20 con 0x55fc9c3fae00 2024-05-08T22:16:19.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.578+0000 7f7ad3400700 1 -- 172.21.15.131:0/1049255911 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f7ae00a30e0 con 0x55fc9c3fae00 2024-05-08T22:16:19.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.579+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/1049255911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc9c29e270 con 0x55fc9c3fae00 2024-05-08T22:16:19.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.579+0000 7f7ad3400700 1 -- 172.21.15.131:0/1049255911 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ae00d00f0 con 0x55fc9c3fae00 2024-05-08T22:16:19.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.579+0000 7f7b0c2eab80 10 client.26647 fetch_fsmap learned FSMap version 29 2024-05-08T22:16:19.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.579+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/1049255911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fc9c172b60 con 0x55fc9c3fae00 2024-05-08T22:16:19.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.579+0000 7f7ad3400700 1 -- 172.21.15.131:0/1049255911 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fc9c29e270 con 0x55fc9c3fae00 2024-05-08T22:16:19.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.580+0000 7f7ad3400700 1 -- 172.21.15.131:0/1049255911 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7ae00053d0 con 0x55fc9c3fae00 2024-05-08T22:16:19.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.580+0000 7f7b0c2eab80 10 client.26647 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:16:19.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.580+0000 7f7b0c2eab80 10 client.26647 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:16:19.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.580+0000 7f7b0c2eab80 20 client.26647 populate_metadata read hostname 'smithi131' 2024-05-08T22:16:19.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.580+0000 7f7b0c2eab80 1 --2- 172.21.15.131:0/1049255911 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55fc9c40ee40 0x55fc9c411230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:19.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.580+0000 7f7b0c2eab80 4 client.26647 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:16:19.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.580+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/1049255911 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc9c3c4810 con 0x55fc9c40ee40 2024-05-08T22:16:19.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.581+0000 7f7af5e00700 1 --2- 172.21.15.131:0/1049255911 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55fc9c40ee40 0x55fc9c411230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:19.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.583+0000 7f7af5e00700 1 --2- 172.21.15.131:0/1049255911 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55fc9c40ee40 0x55fc9c411230 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3383fbed6e12134b server_cookie=875de4afad2c3fa0 in_seq=0 out_seq=0 2024-05-08T22:16:19.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.583+0000 7f7ad3400700 10 client.26647 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:16:19.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.584+0000 7f7ad3400700 1 -- 172.21.15.131:0/1049255911 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fc9c3c4810 con 0x55fc9c40ee40 2024-05-08T22:16:19.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.584+0000 7f7ad3400700 10 client.26647 handle_command_reply: tid=0 2024-05-08T22:16:19.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.585+0000 7f7b0c2eab80 1 client.26647 shutdown 2024-05-08T22:16:19.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.586+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/1049255911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc9c3fae00 msgr2=0x55fc9c3ff060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:19.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.586+0000 7f7b0c2eab80 1 --2- 172.21.15.131:0/1049255911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc9c3fae00 0x55fc9c3ff060 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7ae00cb2a0 tx=0x7f7ae009a700 comp rx=0 tx=0).stop 2024-05-08T22:16:19.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.587+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/1049255911 shutdown_connections 2024-05-08T22:16:19.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.587+0000 7f7b0c2eab80 1 --2- 172.21.15.131:0/1049255911 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55fc9c40ee40 0x55fc9c411230 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.587+0000 7f7b0c2eab80 1 --2- 172.21.15.131:0/1049255911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fc9c3fb720 0x55fc9c3ff5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.587+0000 7f7b0c2eab80 1 --2- 172.21.15.131:0/1049255911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc9c3f5890 0x55fc9c3feb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.587+0000 7f7b0c2eab80 1 --2- 172.21.15.131:0/1049255911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc9c3fae00 0x55fc9c3ff060 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.587+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/1049255911 >> 172.21.15.131:0/1049255911 conn(0x55fc9c353210 msgr2=0x55fc9c3ef5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:19.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.587+0000 7f7ad3400700 0 client.26647 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:16:19.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.587+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/1049255911 shutdown_connections 2024-05-08T22:16:19.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.587+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/1049255911 wait complete. 2024-05-08T22:16:19.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.588+0000 7f7b0c2eab80 20 client.26647 trim_cache size 0 max 16384 2024-05-08T22:16:19.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.615+0000 7f7b0c2eab80 1 Processor -- start 2024-05-08T22:16:19.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.618+0000 7f7b0c2eab80 1 -- start start 2024-05-08T22:16:19.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.619+0000 7f7b0c2eab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fc9c3f5890 0x55fc9c3f1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:19.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.619+0000 7f7b0c2eab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc9c3fae00 0x55fc9c40ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:19.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.619+0000 7f7b0c2eab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc9c3fb720 0x55fc9c3efd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:19.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.619+0000 7f7b0c2eab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc9c3ea830 con 0x55fc9c3fb720 2024-05-08T22:16:19.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.619+0000 7f7b0c2eab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc9c40be50 con 0x55fc9c3f5890 2024-05-08T22:16:19.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.619+0000 7f7b0c2eab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc9c3f1640 con 0x55fc9c3fae00 2024-05-08T22:16:19.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.620+0000 7f7af5400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc9c3fae00 0x55fc9c40ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:19.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.620+0000 7f7af5400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc9c3fae00 0x55fc9c40ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43526/0 (socket says 172.21.15.131:43526) 2024-05-08T22:16:19.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.620+0000 7f7af5400700 1 -- 172.21.15.131:0/2019907630 learned_addr learned my addr 172.21.15.131:0/2019907630 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:19.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.620+0000 7f7af6800700 1 --2- 172.21.15.131:0/2019907630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc9c3fb720 0x55fc9c3efd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:19.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.620+0000 7f7af5e00700 1 --2- 172.21.15.131:0/2019907630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fc9c3f5890 0x55fc9c3f1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:19.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.621+0000 7f7af6800700 1 -- 172.21.15.131:0/2019907630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc9c3fae00 msgr2=0x55fc9c40ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:19.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.621+0000 7f7af6800700 1 --2- 172.21.15.131:0/2019907630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc9c3fae00 0x55fc9c40ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.621+0000 7f7af6800700 1 -- 172.21.15.131:0/2019907630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fc9c3f5890 msgr2=0x55fc9c3f1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:19.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.621+0000 7f7af6800700 1 --2- 172.21.15.131:0/2019907630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fc9c3f5890 0x55fc9c3f1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.621+0000 7f7af6800700 1 -- 172.21.15.131:0/2019907630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc9c30b560 con 0x55fc9c3fb720 2024-05-08T22:16:19.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.622+0000 7f7af6800700 1 --2- 172.21.15.131:0/2019907630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc9c3fb720 0x55fc9c3efd20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0175e0 tx=0x7f7aec00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af4f82bc5cfb507 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:19.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.624+0000 7f7adfa00700 1 -- 172.21.15.131:0/2019907630 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec023650 con 0x55fc9c3fb720 2024-05-08T22:16:19.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.624+0000 7f7adfa00700 1 -- 172.21.15.131:0/2019907630 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7aec00f040 con 0x55fc9c3fb720 2024-05-08T22:16:19.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.624+0000 7f7adfa00700 1 -- 172.21.15.131:0/2019907630 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec016a50 con 0x55fc9c3fb720 2024-05-08T22:16:19.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.625+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/2019907630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc9c3fb720 msgr2=0x55fc9c3efd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:19.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.625+0000 7f7b0c2eab80 1 --2- 172.21.15.131:0/2019907630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc9c3fb720 0x55fc9c3efd20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0175e0 tx=0x7f7aec00b040 comp rx=0 tx=0).stop 2024-05-08T22:16:19.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.625+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/2019907630 shutdown_connections 2024-05-08T22:16:19.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.625+0000 7f7b0c2eab80 1 --2- 172.21.15.131:0/2019907630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc9c3fb720 0x55fc9c3efd20 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.625+0000 7f7b0c2eab80 1 --2- 172.21.15.131:0/2019907630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fc9c3f5890 0x55fc9c3f1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.625+0000 7f7b0c2eab80 1 --2- 172.21.15.131:0/2019907630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc9c3fae00 0x55fc9c40ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.625+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/2019907630 >> 172.21.15.131:0/2019907630 conn(0x55fc9c353210 msgr2=0x55fc9c3f8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:19.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.625+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/2019907630 shutdown_connections 2024-05-08T22:16:19.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.625+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/2019907630 wait complete. 2024-05-08T22:16:19.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.626+0000 7f7b0c2eab80 1 -- start start 2024-05-08T22:16:19.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.627+0000 7f7b0c2eab80 1 Processor -- start 2024-05-08T22:16:19.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.629+0000 7f7b0c2eab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fc9c3f5890 0x55fc9c4061a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:19.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.631+0000 7f7b0c2eab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc9c3fae00 0x55fc9c4066e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:19.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.631+0000 7f7b0c2eab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc9c3fb720 0x55fc9c404180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:19.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.631+0000 7f7b0c2eab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc9c404850 con 0x55fc9c3fb720 2024-05-08T22:16:19.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.631+0000 7f7b0c2eab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc9c4049c0 con 0x55fc9c3f5890 2024-05-08T22:16:19.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.631+0000 7f7b0c2eab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc9c404b80 con 0x55fc9c3fae00 2024-05-08T22:16:19.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.631+0000 7f7b0c2eab80 10 client.0 _ll_register_callbacks cb 0x55fc9c2eab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:16:19.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.632+0000 7f7af5e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fc9c3f5890 0x55fc9c4061a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:19.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.632+0000 7f7af5e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fc9c3f5890 0x55fc9c4061a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:34702/0 (socket says 172.21.15.131:34702) 2024-05-08T22:16:19.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.632+0000 7f7af5e00700 1 -- 172.21.15.131:0/651847182 learned_addr learned my addr 172.21.15.131:0/651847182 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:19.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.633+0000 7f7af5e00700 1 -- 172.21.15.131:0/651847182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc9c3fae00 msgr2=0x55fc9c4066e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:19.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.633+0000 7f7af5e00700 1 --2- 172.21.15.131:0/651847182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc9c3fae00 0x55fc9c4066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.633+0000 7f7af5e00700 1 -- 172.21.15.131:0/651847182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc9c3fb720 msgr2=0x55fc9c404180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:19.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.633+0000 7f7af5e00700 1 --2- 172.21.15.131:0/651847182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc9c3fb720 0x55fc9c404180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.633+0000 7f7af5e00700 1 -- 172.21.15.131:0/651847182 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc9c1cacd0 con 0x55fc9c3f5890 2024-05-08T22:16:19.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.633+0000 7f7af5e00700 1 --2- 172.21.15.131:0/651847182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fc9c3f5890 0x55fc9c4061a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40097b0 tx=0x7f7ae4010bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec848ca6324d5eba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:19.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.633+0000 7f7ad3400700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:16:19.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.634+0000 7f7ad3400700 1 -- 172.21.15.131:0/651847182 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4021040 con 0x55fc9c3f5890 2024-05-08T22:16:19.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.634+0000 7f7ad3400700 1 -- 172.21.15.131:0/651847182 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f7ae4017030 con 0x55fc9c3f5890 2024-05-08T22:16:19.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.634+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/651847182 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ae40044f0 con 0x55fc9c3f5890 2024-05-08T22:16:19.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.635+0000 7f7ad3400700 1 -- 172.21.15.131:0/651847182 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ae4010ea0 con 0x55fc9c3f5890 2024-05-08T22:16:19.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.635+0000 7f7b0c2eab80 10 client.26561 fetch_fsmap learned FSMap version 29 2024-05-08T22:16:19.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.635+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/651847182 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fc9c171030 con 0x55fc9c3f5890 2024-05-08T22:16:19.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.636+0000 7f7ad3400700 1 -- 172.21.15.131:0/651847182 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ae40044f0 con 0x55fc9c3f5890 2024-05-08T22:16:19.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.636+0000 7f7ad3400700 1 -- 172.21.15.131:0/651847182 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7ae4004890 con 0x55fc9c3f5890 2024-05-08T22:16:19.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.636+0000 7f7b0c2eab80 10 client.26561 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:16:19.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.636+0000 7f7b0c2eab80 10 client.26561 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:16:19.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.636+0000 7f7b0c2eab80 20 client.26561 populate_metadata read hostname 'smithi131' 2024-05-08T22:16:19.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.636+0000 7f7b0c2eab80 1 --2- 172.21.15.131:0/651847182 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55fc9c4654a0 0x55fc9c467900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:19.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.636+0000 7f7b0c2eab80 4 client.26561 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:16:19.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.636+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/651847182 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fc9c33dec0 con 0x55fc9c4654a0 2024-05-08T22:16:19.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.638+0000 7f7af5400700 1 --2- 172.21.15.131:0/651847182 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55fc9c4654a0 0x55fc9c467900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:19.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.639+0000 7f7af5400700 1 --2- 172.21.15.131:0/651847182 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55fc9c4654a0 0x55fc9c467900 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fdaa7aae9012d9ec server_cookie=967cca45db96b024 in_seq=0 out_seq=0 2024-05-08T22:16:19.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.640+0000 7f7ad3400700 10 client.26561 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:16:19.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.640+0000 7f7ad3400700 1 -- 172.21.15.131:0/651847182 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fc9c33dec0 con 0x55fc9c4654a0 2024-05-08T22:16:19.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.640+0000 7f7ad3400700 10 client.26561 handle_command_reply: tid=0 2024-05-08T22:16:19.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.640+0000 7f7b0c2eab80 1 client.26561 shutdown 2024-05-08T22:16:19.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.641+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/651847182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fc9c3f5890 msgr2=0x55fc9c4061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:19.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.641+0000 7f7b0c2eab80 1 --2- 172.21.15.131:0/651847182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fc9c3f5890 0x55fc9c4061a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40097b0 tx=0x7f7ae4010bc0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.641+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/651847182 shutdown_connections 2024-05-08T22:16:19.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.641+0000 7f7b0c2eab80 1 --2- 172.21.15.131:0/651847182 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55fc9c4654a0 0x55fc9c467900 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.641+0000 7f7b0c2eab80 1 --2- 172.21.15.131:0/651847182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc9c3fb720 0x55fc9c404180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.641+0000 7f7ad3400700 0 client.26561 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:16:19.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.641+0000 7f7b0c2eab80 1 --2- 172.21.15.131:0/651847182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fc9c3f5890 0x55fc9c4061a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.641+0000 7f7b0c2eab80 1 --2- 172.21.15.131:0/651847182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc9c3fae00 0x55fc9c4066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.641+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/651847182 >> 172.21.15.131:0/651847182 conn(0x55fc9c353210 msgr2=0x55fc9c3fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:19.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.641+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/651847182 shutdown_connections 2024-05-08T22:16:19.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.642+0000 7f7b0c2eab80 1 -- 172.21.15.131:0/651847182 wait complete. 2024-05-08T22:16:19.644 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:16:19.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.642+0000 7f7b0c2eab80 20 client.26561 trim_cache size 0 max 16384 2024-05-08T22:16:19.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.646+0000 7f7af8800700 1 -- 172.21.15.131:0/3267312925 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7ad80a1310 msgr2=0x7f7ad80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:19.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.646+0000 7f7af8800700 1 --2- 172.21.15.131:0/3267312925 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7ad80a1310 0x7f7ad80a37d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009fd0 tx=0x7f7aec009040 comp rx=0 tx=0).stop 2024-05-08T22:16:19.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.646+0000 7f7af8800700 1 -- 172.21.15.131:0/3267312925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7af0103930 msgr2=0x7f7af019df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:19.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.646+0000 7f7af8800700 1 --2- 172.21.15.131:0/3267312925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7af0103930 0x7f7af019df80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0005950 tx=0x7f7ae0010ff0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.646+0000 7f7af8800700 1 -- 172.21.15.131:0/3267312925 shutdown_connections 2024-05-08T22:16:19.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.646+0000 7f7af8800700 1 --2- 172.21.15.131:0/3267312925 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7ad80a1310 0x7f7ad80a37d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.646+0000 7f7af8800700 1 --2- 172.21.15.131:0/3267312925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7af0105e30 0x7f7af01a2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.646+0000 7f7af8800700 1 --2- 172.21.15.131:0/3267312925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af0068590 0x7f7af019da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.646+0000 7f7af8800700 1 --2- 172.21.15.131:0/3267312925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7af0103930 0x7f7af019df80 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:19.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.646+0000 7f7af8800700 1 -- 172.21.15.131:0/3267312925 >> 172.21.15.131:0/3267312925 conn(0x7f7af0076f40 msgr2=0x7f7af00fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:19.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.648+0000 7f7af8800700 1 -- 172.21.15.131:0/3267312925 shutdown_connections 2024-05-08T22:16:19.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:19.648+0000 7f7af8800700 1 -- 172.21.15.131:0/3267312925 wait complete. 2024-05-08T22:16:19.683 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T22:16:19.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:19 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2881815441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:16:20.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:19 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2881815441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:16:20.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.064+0000 7f3cb1600700 1 -- 172.21.15.131:0/1095229133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cac106290 msgr2=0x7f3cac10de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:20.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.064+0000 7f3cb1600700 1 --2- 172.21.15.131:0/1095229133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cac106290 0x7f3cac10de10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3ca400b0d0 tx=0x7f3ca400b3e0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.065+0000 7f3cb1600700 1 -- 172.21.15.131:0/1095229133 shutdown_connections 2024-05-08T22:16:20.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.065+0000 7f3cb1600700 1 --2- 172.21.15.131:0/1095229133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cac106290 0x7f3cac10de10 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.065+0000 7f3cb1600700 1 --2- 172.21.15.131:0/1095229133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cac0682e0 0x7f3cac0686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.065+0000 7f3cb1600700 1 --2- 172.21.15.131:0/1095229133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cac068c90 0x7f3cac105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.065+0000 7f3cb1600700 1 -- 172.21.15.131:0/1095229133 >> 172.21.15.131:0/1095229133 conn(0x7f3cac076f40 msgr2=0x7f3cac077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:20.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.068+0000 7f3cb1600700 1 -- 172.21.15.131:0/1095229133 shutdown_connections 2024-05-08T22:16:20.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.068+0000 7f3cb1600700 1 -- 172.21.15.131:0/1095229133 wait complete. 2024-05-08T22:16:20.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.069+0000 7f3cb1600700 1 Processor -- start 2024-05-08T22:16:20.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.071+0000 7f3cb1600700 1 -- start start 2024-05-08T22:16:20.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.071+0000 7f3cb1600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cac0682e0 0x7f3cac0ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.071+0000 7f3cb1600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cac106290 0x7f3cac1003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.071+0000 7f3cb1600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cac101b20 0x7f3cac100920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.071+0000 7f3cb1600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac1aa1a0 con 0x7f3cac106290 2024-05-08T22:16:20.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.071+0000 7f3cb1600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac1aa310 con 0x7f3cac0682e0 2024-05-08T22:16:20.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.071+0000 7f3cb1600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac1aa4d0 con 0x7f3cac101b20 2024-05-08T22:16:20.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.072+0000 7f3caaa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cac0682e0 0x7f3cac0ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.072+0000 7f3caaa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cac0682e0 0x7f3cac0ffe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:34718/0 (socket says 172.21.15.131:34718) 2024-05-08T22:16:20.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.072+0000 7f3caaa00700 1 -- 172.21.15.131:0/2972589751 learned_addr learned my addr 172.21.15.131:0/2972589751 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:20.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.072+0000 7f3cab400700 1 --2- 172.21.15.131:0/2972589751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cac101b20 0x7f3cac100920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.072+0000 7f3caaa00700 1 -- 172.21.15.131:0/2972589751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cac101b20 msgr2=0x7f3cac100920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:20.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.073+0000 7f3caaa00700 1 --2- 172.21.15.131:0/2972589751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cac101b20 0x7f3cac100920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.073+0000 7f3caaa00700 1 -- 172.21.15.131:0/2972589751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cac106290 msgr2=0x7f3cac1003a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:16:20.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.073+0000 7f3caaa00700 1 --2- 172.21.15.131:0/2972589751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cac106290 0x7f3cac1003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.073+0000 7f3caaa00700 1 -- 172.21.15.131:0/2972589751 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca4009d00 con 0x7f3cac0682e0 2024-05-08T22:16:20.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.073+0000 7f3cab400700 1 --2- 172.21.15.131:0/2972589751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cac101b20 0x7f3cac100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:16:20.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.074+0000 7f3caaa00700 1 --2- 172.21.15.131:0/2972589751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cac0682e0 0x7f3cac0ffe60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00b530 tx=0x7f3c9c00b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:20.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.074+0000 7f3ca3400700 1 -- 172.21.15.131:0/2972589751 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c011830 con 0x7f3cac0682e0 2024-05-08T22:16:20.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.074+0000 7f3ca3400700 1 -- 172.21.15.131:0/2972589751 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c9c011e50 con 0x7f3cac0682e0 2024-05-08T22:16:20.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.074+0000 7f3ca3400700 1 -- 172.21.15.131:0/2972589751 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c00f2e0 con 0x7f3cac0682e0 2024-05-08T22:16:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.074+0000 7f3cb1600700 1 -- 172.21.15.131:0/2972589751 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cac1aa7b0 con 0x7f3cac0682e0 2024-05-08T22:16:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.075+0000 7f3cb1600700 1 -- 172.21.15.131:0/2972589751 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cac1aacc0 con 0x7f3cac0682e0 2024-05-08T22:16:20.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.079+0000 7f3ca3400700 1 -- 172.21.15.131:0/2972589751 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f3c9c011990 con 0x7f3cac0682e0 2024-05-08T22:16:20.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.081+0000 7f3ca3400700 1 --2- 172.21.15.131:0/2972589751 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3c900a1390 0x7f3c900a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.082+0000 7f3ca3400700 1 -- 172.21.15.131:0/2972589751 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f3c9c0ccb90 con 0x7f3cac0682e0 2024-05-08T22:16:20.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.082+0000 7f3caa000700 1 --2- 172.21.15.131:0/2972589751 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3c900a1390 0x7f3c900a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.085+0000 7f3caa000700 1 --2- 172.21.15.131:0/2972589751 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3c900a1390 0x7f3c900a3850 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3c98005950 tx=0x7f3c980058e0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:20.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.086+0000 7f3cc5170b80 1 Processor -- start 2024-05-08T22:16:20.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.086+0000 7f3cc5170b80 1 -- start start 2024-05-08T22:16:20.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.087+0000 7f3cc5170b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561661e39e00 0x561661e3a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.087+0000 7f3cc5170b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561661e3a720 0x561661e34350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.087+0000 7f3cc5170b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561661e34890 0x561661e34cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.087+0000 7f3cc5170b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561661d8b150 con 0x561661e34890 2024-05-08T22:16:20.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.087+0000 7f3cc5170b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561661d88b10 con 0x561661e3a720 2024-05-08T22:16:20.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.087+0000 7f3cc5170b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561661d84400 con 0x561661e39e00 2024-05-08T22:16:20.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.087+0000 7f3caaa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561661e39e00 0x561661e3a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.087+0000 7f3caaa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561661e39e00 0x561661e3a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43564/0 (socket says 172.21.15.131:43564) 2024-05-08T22:16:20.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.087+0000 7f3caaa00700 1 -- 172.21.15.131:0/33348661 learned_addr learned my addr 172.21.15.131:0/33348661 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:20.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.087+0000 7f3caa000700 1 --2- 172.21.15.131:0/33348661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561661e3a720 0x561661e34350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.088+0000 7f3cab400700 1 --2- 172.21.15.131:0/33348661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561661e34890 0x561661e34cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.088+0000 7f3caaa00700 1 -- 172.21.15.131:0/33348661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561661e3a720 msgr2=0x561661e34350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:20.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.088+0000 7f3caaa00700 1 --2- 172.21.15.131:0/33348661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561661e3a720 0x561661e34350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.088+0000 7f3caaa00700 1 -- 172.21.15.131:0/33348661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561661e34890 msgr2=0x561661e34cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:20.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.088+0000 7f3caaa00700 1 --2- 172.21.15.131:0/33348661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561661e34890 0x561661e34cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.088+0000 7f3caaa00700 1 -- 172.21.15.131:0/33348661 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561661c053b0 con 0x561661e39e00 2024-05-08T22:16:20.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.088+0000 7f3caa000700 1 --2- 172.21.15.131:0/33348661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561661e3a720 0x561661e34350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:16:20.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.088+0000 7f3caaa00700 1 --2- 172.21.15.131:0/33348661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561661e39e00 0x561661e3a1e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c096380 tx=0x7f3c9c099340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2073fa1d3a83c7f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:20.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.089+0000 7f3c97400700 1 -- 172.21.15.131:0/33348661 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c0a1d90 con 0x561661e39e00 2024-05-08T22:16:20.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.089+0000 7f3c97400700 1 -- 172.21.15.131:0/33348661 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c9c097a50 con 0x561661e39e00 2024-05-08T22:16:20.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.089+0000 7f3c97400700 1 -- 172.21.15.131:0/33348661 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c0a1d90 con 0x561661e39e00 2024-05-08T22:16:20.092 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:19 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2881815441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:16:20.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.090+0000 7f3cc5170b80 1 -- 172.21.15.131:0/33348661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561661e39e00 msgr2=0x561661e3a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:20.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.090+0000 7f3cc5170b80 1 --2- 172.21.15.131:0/33348661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561661e39e00 0x561661e3a1e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c096380 tx=0x7f3c9c099340 comp rx=0 tx=0).stop 2024-05-08T22:16:20.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.090+0000 7f3cc5170b80 1 -- 172.21.15.131:0/33348661 shutdown_connections 2024-05-08T22:16:20.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.090+0000 7f3cc5170b80 1 --2- 172.21.15.131:0/33348661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561661e34890 0x561661e34cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.090+0000 7f3cc5170b80 1 --2- 172.21.15.131:0/33348661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561661e39e00 0x561661e3a1e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.090+0000 7f3cc5170b80 1 --2- 172.21.15.131:0/33348661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561661e3a720 0x561661e34350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.090+0000 7f3cc5170b80 1 -- 172.21.15.131:0/33348661 >> 172.21.15.131:0/33348661 conn(0x561661d92210 msgr2=0x561661e2edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:20.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.090+0000 7f3cc5170b80 1 -- 172.21.15.131:0/33348661 shutdown_connections 2024-05-08T22:16:20.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.090+0000 7f3cc5170b80 1 -- 172.21.15.131:0/33348661 wait complete. 2024-05-08T22:16:20.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.091+0000 7f3cc5170b80 1 -- start start 2024-05-08T22:16:20.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.091+0000 7f3cc5170b80 1 Processor -- start 2024-05-08T22:16:20.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.092+0000 7f3cc5170b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561661e34890 0x561661e3dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.092+0000 7f3cc5170b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561661e39e00 0x561661e3e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.092+0000 7f3cc5170b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561661e3a720 0x561661e3e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.092+0000 7f3cc5170b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561661d20560 con 0x561661e39e00 2024-05-08T22:16:20.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.092+0000 7f3cc5170b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561661d20200 con 0x561661e34890 2024-05-08T22:16:20.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.092+0000 7f3cc5170b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561661c076a0 con 0x561661e3a720 2024-05-08T22:16:20.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.092+0000 7f3cc5170b80 10 client.0 _ll_register_callbacks cb 0x561661d29b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:16:20.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.092+0000 7f3caaa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561661e34890 0x561661e3dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.092+0000 7f3caaa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561661e34890 0x561661e3dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:34732/0 (socket says 172.21.15.131:34732) 2024-05-08T22:16:20.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.092+0000 7f3caaa00700 1 -- 172.21.15.131:0/4164464812 learned_addr learned my addr 172.21.15.131:0/4164464812 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:20.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.093+0000 7f3caa000700 1 --2- 172.21.15.131:0/4164464812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561661e39e00 0x561661e3e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.093+0000 7f3cab400700 1 --2- 172.21.15.131:0/4164464812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561661e3a720 0x561661e3e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.093+0000 7f3caaa00700 1 -- 172.21.15.131:0/4164464812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561661e3a720 msgr2=0x561661e3e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:20.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.093+0000 7f3caaa00700 1 --2- 172.21.15.131:0/4164464812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561661e3a720 0x561661e3e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.093+0000 7f3caaa00700 1 -- 172.21.15.131:0/4164464812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561661e39e00 msgr2=0x561661e3e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:20.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.093+0000 7f3caaa00700 1 --2- 172.21.15.131:0/4164464812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561661e39e00 0x561661e3e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.093+0000 7f3caaa00700 1 -- 172.21.15.131:0/4164464812 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cac1aacc0 con 0x561661e34890 2024-05-08T22:16:20.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.096+0000 7f3caaa00700 1 --2- 172.21.15.131:0/4164464812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561661e34890 0x561661e3dbb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0a1860 tx=0x7f3c9c004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9680f0df7c8cf59 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:20.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.096+0000 7f3c8be00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:16:20.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.096+0000 7f3c8be00700 1 -- 172.21.15.131:0/4164464812 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c010f00 con 0x561661e34890 2024-05-08T22:16:20.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.096+0000 7f3c8be00700 1 -- 172.21.15.131:0/4164464812 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f3c9c0b22a0 con 0x561661e34890 2024-05-08T22:16:20.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.096+0000 7f3cc5170b80 1 -- 172.21.15.131:0/4164464812 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561661cdd270 con 0x561661e34890 2024-05-08T22:16:20.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.097+0000 7f3c8be00700 1 -- 172.21.15.131:0/4164464812 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c9c017d40 con 0x561661e34890 2024-05-08T22:16:20.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.097+0000 7f3cc5170b80 10 client.26570 fetch_fsmap learned FSMap version 29 2024-05-08T22:16:20.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.097+0000 7f3cc5170b80 1 -- 172.21.15.131:0/4164464812 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561661bb1b60 con 0x561661e34890 2024-05-08T22:16:20.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.098+0000 7f3c8be00700 1 -- 172.21.15.131:0/4164464812 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561661cdd270 con 0x561661e34890 2024-05-08T22:16:20.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.098+0000 7f3c8be00700 1 -- 172.21.15.131:0/4164464812 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f3c9c0a1900 con 0x561661e34890 2024-05-08T22:16:20.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.098+0000 7f3cc5170b80 10 client.26570 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:16:20.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.098+0000 7f3cc5170b80 10 client.26570 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:16:20.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.098+0000 7f3cc5170b80 20 client.26570 populate_metadata read hostname 'smithi131' 2024-05-08T22:16:20.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.098+0000 7f3cc5170b80 1 --2- 172.21.15.131:0/4164464812 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x561661e4dbe0 0x561661e4ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.098+0000 7f3cc5170b80 4 client.26570 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:16:20.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.099+0000 7f3caa000700 1 --2- 172.21.15.131:0/4164464812 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x561661e4dbe0 0x561661e4ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.099+0000 7f3cc5170b80 1 -- 172.21.15.131:0/4164464812 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561661e03810 con 0x561661e4dbe0 2024-05-08T22:16:20.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.100+0000 7f3c8be00700 10 client.26570 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:16:20.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.100+0000 7f3caa000700 1 --2- 172.21.15.131:0/4164464812 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x561661e4dbe0 0x561661e4ffd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=525d69391bd9f08f server_cookie=1eaa93152692e5d3 in_seq=0 out_seq=0 2024-05-08T22:16:20.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.103+0000 7f3c8be00700 1 -- 172.21.15.131:0/4164464812 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561661e03810 con 0x561661e4dbe0 2024-05-08T22:16:20.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.103+0000 7f3c8be00700 10 client.26570 handle_command_reply: tid=0 2024-05-08T22:16:20.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.103+0000 7f3cc5170b80 1 client.26570 shutdown 2024-05-08T22:16:20.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.103+0000 7f3cc5170b80 1 -- 172.21.15.131:0/4164464812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561661e34890 msgr2=0x561661e3dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:20.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.103+0000 7f3cc5170b80 1 --2- 172.21.15.131:0/4164464812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561661e34890 0x561661e3dbb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0a1860 tx=0x7f3c9c004860 comp rx=0 tx=0).stop 2024-05-08T22:16:20.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.105+0000 7f3cc5170b80 1 -- 172.21.15.131:0/4164464812 shutdown_connections 2024-05-08T22:16:20.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.105+0000 7f3cc5170b80 1 --2- 172.21.15.131:0/4164464812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561661e3a720 0x561661e3e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.105+0000 7f3cc5170b80 1 --2- 172.21.15.131:0/4164464812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561661e34890 0x561661e3dbb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.105+0000 7f3cc5170b80 1 --2- 172.21.15.131:0/4164464812 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x561661e4dbe0 0x561661e4ffd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.105+0000 7f3cc5170b80 1 --2- 172.21.15.131:0/4164464812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561661e39e00 0x561661e3e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.105+0000 7f3c8be00700 0 client.26570 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:16:20.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.105+0000 7f3cc5170b80 1 -- 172.21.15.131:0/4164464812 >> 172.21.15.131:0/4164464812 conn(0x561661d92210 msgr2=0x561661e2e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:20.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.106+0000 7f3cc5170b80 1 -- 172.21.15.131:0/4164464812 shutdown_connections 2024-05-08T22:16:20.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.106+0000 7f3cc5170b80 1 -- 172.21.15.131:0/4164464812 wait complete. 2024-05-08T22:16:20.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.106+0000 7f3cc5170b80 20 client.26570 trim_cache size 0 max 16384 2024-05-08T22:16:20.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.135+0000 7f3cc5170b80 1 Processor -- start 2024-05-08T22:16:20.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.136+0000 7f3cc5170b80 1 -- start start 2024-05-08T22:16:20.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.137+0000 7f3cc5170b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561661e34890 0x561661e34350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.137+0000 7f3cc5170b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561661e39e00 0x561661e4ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.137+0000 7f3cc5170b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561661e3a720 0x561661e49c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.137+0000 7f3cc5170b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561661e4ae20 con 0x561661e3a720 2024-05-08T22:16:20.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.137+0000 7f3cc5170b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561661e37870 con 0x561661e39e00 2024-05-08T22:16:20.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.137+0000 7f3cc5170b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561661e40720 con 0x561661e34890 2024-05-08T22:16:20.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.138+0000 7f3cab400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561661e3a720 0x561661e49c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.139+0000 7f3cab400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561661e3a720 0x561661e49c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44744/0 (socket says 172.21.15.131:44744) 2024-05-08T22:16:20.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.139+0000 7f3cab400700 1 -- 172.21.15.131:0/1518396880 learned_addr learned my addr 172.21.15.131:0/1518396880 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:20.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.140+0000 7f3cab400700 1 -- 172.21.15.131:0/1518396880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561661e34890 msgr2=0x561661e34350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:20.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.140+0000 7f3cab400700 1 --2- 172.21.15.131:0/1518396880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561661e34890 0x561661e34350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.140+0000 7f3cab400700 1 -- 172.21.15.131:0/1518396880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561661e39e00 msgr2=0x561661e4ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:20.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.140+0000 7f3cab400700 1 --2- 172.21.15.131:0/1518396880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561661e39e00 0x561661e4ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.140+0000 7f3cab400700 1 -- 172.21.15.131:0/1518396880 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561661d4a560 con 0x561661e3a720 2024-05-08T22:16:20.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.140+0000 7f3cab400700 1 --2- 172.21.15.131:0/1518396880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561661e3a720 0x561661e49c20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4009310 tx=0x7f3ca4009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e82f9723782c815 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:20.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.141+0000 7f3c97e00700 1 -- 172.21.15.131:0/1518396880 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca400e030 con 0x561661e3a720 2024-05-08T22:16:20.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.141+0000 7f3c97e00700 1 -- 172.21.15.131:0/1518396880 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ca4007d00 con 0x561661e3a720 2024-05-08T22:16:20.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.141+0000 7f3c97e00700 1 -- 172.21.15.131:0/1518396880 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4007740 con 0x561661e3a720 2024-05-08T22:16:20.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.142+0000 7f3cc5170b80 1 -- 172.21.15.131:0/1518396880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561661e3a720 msgr2=0x561661e49c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:20.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.142+0000 7f3cc5170b80 1 --2- 172.21.15.131:0/1518396880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561661e3a720 0x561661e49c20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4009310 tx=0x7f3ca4009340 comp rx=0 tx=0).stop 2024-05-08T22:16:20.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.143+0000 7f3cc5170b80 1 -- 172.21.15.131:0/1518396880 shutdown_connections 2024-05-08T22:16:20.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.143+0000 7f3cc5170b80 1 --2- 172.21.15.131:0/1518396880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561661e3a720 0x561661e49c20 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.143+0000 7f3cc5170b80 1 --2- 172.21.15.131:0/1518396880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561661e34890 0x561661e34350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.143+0000 7f3cc5170b80 1 --2- 172.21.15.131:0/1518396880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561661e39e00 0x561661e4ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.143+0000 7f3cc5170b80 1 -- 172.21.15.131:0/1518396880 >> 172.21.15.131:0/1518396880 conn(0x561661d92210 msgr2=0x561661e30870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:20.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.143+0000 7f3cc5170b80 1 -- 172.21.15.131:0/1518396880 shutdown_connections 2024-05-08T22:16:20.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.146+0000 7f3cc5170b80 1 -- 172.21.15.131:0/1518396880 wait complete. 2024-05-08T22:16:20.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.147+0000 7f3cc5170b80 1 -- start start 2024-05-08T22:16:20.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.147+0000 7f3cc5170b80 1 Processor -- start 2024-05-08T22:16:20.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.148+0000 7f3cc5170b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561661e34890 0x561661e454c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.148+0000 7f3cc5170b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561661e39e00 0x561661e43150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.148+0000 7f3cc5170b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561661e3a720 0x561661e43690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.148+0000 7f3cc5170b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561661e43d80 con 0x561661e3a720 2024-05-08T22:16:20.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.148+0000 7f3cc5170b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561661e43ef0 con 0x561661e39e00 2024-05-08T22:16:20.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.148+0000 7f3cc5170b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561661eb4050 con 0x561661e34890 2024-05-08T22:16:20.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.148+0000 7f3cc5170b80 10 client.0 _ll_register_callbacks cb 0x561661d29b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:16:20.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.149+0000 7f3caa000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561661e39e00 0x561661e43150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.149+0000 7f3caa000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561661e39e00 0x561661e43150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:34748/0 (socket says 172.21.15.131:34748) 2024-05-08T22:16:20.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.149+0000 7f3caa000700 1 -- 172.21.15.131:0/951541313 learned_addr learned my addr 172.21.15.131:0/951541313 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:20.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.149+0000 7f3caaa00700 1 --2- 172.21.15.131:0/951541313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561661e34890 0x561661e454c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.149+0000 7f3caa000700 1 -- 172.21.15.131:0/951541313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561661e34890 msgr2=0x561661e454c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:20.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.149+0000 7f3cab400700 1 --2- 172.21.15.131:0/951541313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561661e3a720 0x561661e43690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.149+0000 7f3caa000700 1 --2- 172.21.15.131:0/951541313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561661e34890 0x561661e454c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.150+0000 7f3caa000700 1 -- 172.21.15.131:0/951541313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561661e3a720 msgr2=0x561661e43690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:20.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.150+0000 7f3caa000700 1 --2- 172.21.15.131:0/951541313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561661e3a720 0x561661e43690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.150+0000 7f3caa000700 1 -- 172.21.15.131:0/951541313 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cac10e650 con 0x561661e39e00 2024-05-08T22:16:20.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.150+0000 7f3caa000700 1 --2- 172.21.15.131:0/951541313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561661e39e00 0x561661e43150 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3cac101590 tx=0x7f3c98009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62014c427e67bf7f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:20.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.150+0000 7f3c8be00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:16:20.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.150+0000 7f3c8be00700 1 -- 172.21.15.131:0/951541313 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c98023040 con 0x561661e39e00 2024-05-08T22:16:20.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.151+0000 7f3cc5170b80 1 -- 172.21.15.131:0/951541313 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c9c097a50 con 0x561661e39e00 2024-05-08T22:16:20.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.151+0000 7f3c8be00700 1 -- 172.21.15.131:0/951541313 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f3c98004a30 con 0x561661e39e00 2024-05-08T22:16:20.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.151+0000 7f3c8be00700 1 -- 172.21.15.131:0/951541313 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c98004bf0 con 0x561661e39e00 2024-05-08T22:16:20.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.151+0000 7f3cc5170b80 10 client.26573 fetch_fsmap learned FSMap version 29 2024-05-08T22:16:20.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.151+0000 7f3cc5170b80 1 -- 172.21.15.131:0/951541313 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561661c09cd0 con 0x561661e39e00 2024-05-08T22:16:20.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.155+0000 7f3c8be00700 1 -- 172.21.15.131:0/951541313 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c9c097a50 con 0x561661e39e00 2024-05-08T22:16:20.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.155+0000 7f3c8be00700 1 -- 172.21.15.131:0/951541313 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f3c98005380 con 0x561661e39e00 2024-05-08T22:16:20.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.156+0000 7f3cc5170b80 10 client.26573 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:16:20.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.156+0000 7f3cc5170b80 10 client.26573 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:16:20.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.156+0000 7f3cc5170b80 20 client.26573 populate_metadata read hostname 'smithi131' 2024-05-08T22:16:20.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.156+0000 7f3cc5170b80 1 --2- 172.21.15.131:0/951541313 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x561661eb57b0 0x561661eb7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.156+0000 7f3cc5170b80 4 client.26573 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:16:20.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.156+0000 7f3cc5170b80 1 -- 172.21.15.131:0/951541313 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561661d7cec0 con 0x561661eb57b0 2024-05-08T22:16:20.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.157+0000 7f3caaa00700 1 --2- 172.21.15.131:0/951541313 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x561661eb57b0 0x561661eb7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.159+0000 7f3caaa00700 1 --2- 172.21.15.131:0/951541313 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x561661eb57b0 0x561661eb7bd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e884608c73af8ec1 server_cookie=954f653949c213d2 in_seq=0 out_seq=0 2024-05-08T22:16:20.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.159+0000 7f3c8be00700 10 client.26573 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:16:20.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.160+0000 7f3c8be00700 1 -- 172.21.15.131:0/951541313 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561661d7cec0 con 0x561661eb57b0 2024-05-08T22:16:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.161+0000 7f3c8be00700 10 client.26573 handle_command_reply: tid=0 2024-05-08T22:16:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.161+0000 7f3cc5170b80 1 client.26573 shutdown 2024-05-08T22:16:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.162+0000 7f3cc5170b80 1 -- 172.21.15.131:0/951541313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561661e39e00 msgr2=0x561661e43150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.162+0000 7f3cc5170b80 1 --2- 172.21.15.131:0/951541313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561661e39e00 0x561661e43150 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3cac101590 tx=0x7f3c98009260 comp rx=0 tx=0).stop 2024-05-08T22:16:20.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.164+0000 7f3cc5170b80 1 -- 172.21.15.131:0/951541313 shutdown_connections 2024-05-08T22:16:20.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.164+0000 7f3cc5170b80 1 --2- 172.21.15.131:0/951541313 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x561661eb57b0 0x561661eb7bd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.164+0000 7f3cc5170b80 1 --2- 172.21.15.131:0/951541313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561661e3a720 0x561661e43690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.164+0000 7f3cc5170b80 1 --2- 172.21.15.131:0/951541313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561661e34890 0x561661e454c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.164+0000 7f3cc5170b80 1 --2- 172.21.15.131:0/951541313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561661e39e00 0x561661e43150 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.164+0000 7f3cc5170b80 1 -- 172.21.15.131:0/951541313 >> 172.21.15.131:0/951541313 conn(0x561661d92210 msgr2=0x561661e3c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:20.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.164+0000 7f3c8be00700 0 client.26573 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:16:20.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.165+0000 7f3cc5170b80 1 -- 172.21.15.131:0/951541313 shutdown_connections 2024-05-08T22:16:20.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.165+0000 7f3cc5170b80 1 -- 172.21.15.131:0/951541313 wait complete. 2024-05-08T22:16:20.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.166+0000 7f3cc5170b80 20 client.26573 trim_cache size 0 max 16384 2024-05-08T22:16:20.167 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:16:20.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.168+0000 7f3cb1600700 1 -- 172.21.15.131:0/2972589751 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3c900a1390 msgr2=0x7f3c900a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:20.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.168+0000 7f3cb1600700 1 --2- 172.21.15.131:0/2972589751 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3c900a1390 0x7f3c900a3850 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3c98005950 tx=0x7f3c980058e0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.168+0000 7f3cb1600700 1 -- 172.21.15.131:0/2972589751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cac0682e0 msgr2=0x7f3cac0ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:20.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.168+0000 7f3cb1600700 1 --2- 172.21.15.131:0/2972589751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cac0682e0 0x7f3cac0ffe60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00b530 tx=0x7f3c9c00b8f0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.169+0000 7f3cb1600700 1 -- 172.21.15.131:0/2972589751 shutdown_connections 2024-05-08T22:16:20.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.169+0000 7f3cb1600700 1 --2- 172.21.15.131:0/2972589751 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3c900a1390 0x7f3c900a3850 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.169+0000 7f3cb1600700 1 --2- 172.21.15.131:0/2972589751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cac101b20 0x7f3cac100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.169+0000 7f3cb1600700 1 --2- 172.21.15.131:0/2972589751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cac0682e0 0x7f3cac0ffe60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.169+0000 7f3cb1600700 1 --2- 172.21.15.131:0/2972589751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cac106290 0x7f3cac1003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.169+0000 7f3cb1600700 1 -- 172.21.15.131:0/2972589751 >> 172.21.15.131:0/2972589751 conn(0x7f3cac076f40 msgr2=0x7f3cac0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:20.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.169+0000 7f3cb1600700 1 -- 172.21.15.131:0/2972589751 shutdown_connections 2024-05-08T22:16:20.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.169+0000 7f3cb1600700 1 -- 172.21.15.131:0/2972589751 wait complete. 2024-05-08T22:16:20.204 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T22:16:20.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.586+0000 7fad3b000700 1 -- 172.21.15.131:0/4173545183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad34105e30 msgr2=0x7fad34068050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:20.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.586+0000 7fad3b000700 1 --2- 172.21.15.131:0/4173545183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad34105e30 0x7fad34068050 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00b0a0 tx=0x7fad2c00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.587+0000 7fad3b000700 1 -- 172.21.15.131:0/4173545183 shutdown_connections 2024-05-08T22:16:20.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.587+0000 7fad3b000700 1 --2- 172.21.15.131:0/4173545183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad34103930 0x7fad34103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.587+0000 7fad3b000700 1 --2- 172.21.15.131:0/4173545183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad34105e30 0x7fad34068050 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.587+0000 7fad3b000700 1 --2- 172.21.15.131:0/4173545183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad34068590 0x7fad34068a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.587+0000 7fad3b000700 1 -- 172.21.15.131:0/4173545183 >> 172.21.15.131:0/4173545183 conn(0x7fad34076f40 msgr2=0x7fad34077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:20.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.587+0000 7fad3b000700 1 -- 172.21.15.131:0/4173545183 shutdown_connections 2024-05-08T22:16:20.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.587+0000 7fad3b000700 1 -- 172.21.15.131:0/4173545183 wait complete. 2024-05-08T22:16:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.589+0000 7fad3b000700 1 Processor -- start 2024-05-08T22:16:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.589+0000 7fad3b000700 1 -- start start 2024-05-08T22:16:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.590+0000 7fad3b000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad34068590 0x7fad341a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.590+0000 7fad3b000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad34103930 0x7fad341a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.590+0000 7fad3b000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad34105e30 0x7fad3419d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.590+0000 7fad3b000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3419dfc0 con 0x7fad34103930 2024-05-08T22:16:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.590+0000 7fad3b000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3419e130 con 0x7fad34068590 2024-05-08T22:16:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.590+0000 7fad3b000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3419e2f0 con 0x7fad34105e30 2024-05-08T22:16:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.592+0000 7fad33e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad34068590 0x7fad341a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.592+0000 7fad39000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad34105e30 0x7fad3419d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.592+0000 7fad33e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad34068590 0x7fad341a3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:34768/0 (socket says 172.21.15.131:34768) 2024-05-08T22:16:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.592+0000 7fad33e00700 1 -- 172.21.15.131:0/2642923779 learned_addr learned my addr 172.21.15.131:0/2642923779 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:20.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.592+0000 7fad39000700 1 -- 172.21.15.131:0/2642923779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad34068590 msgr2=0x7fad341a3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:20.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.592+0000 7fad39000700 1 --2- 172.21.15.131:0/2642923779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad34068590 0x7fad341a3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.592+0000 7fad39000700 1 -- 172.21.15.131:0/2642923779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad34103930 msgr2=0x7fad341a3d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:16:20.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.592+0000 7fad39000700 1 --2- 172.21.15.131:0/2642923779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad34103930 0x7fad341a3d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.592+0000 7fad39000700 1 -- 172.21.15.131:0/2642923779 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2c009d00 con 0x7fad34105e30 2024-05-08T22:16:20.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.592+0000 7fad39000700 1 --2- 172.21.15.131:0/2642923779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad34105e30 0x7fad3419d960 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fad2400b3f0 tx=0x7fad2400b700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:20.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.593+0000 7fad30c00700 1 -- 172.21.15.131:0/2642923779 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad24011830 con 0x7fad34105e30 2024-05-08T22:16:20.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.593+0000 7fad30c00700 1 -- 172.21.15.131:0/2642923779 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad24011e50 con 0x7fad34105e30 2024-05-08T22:16:20.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.593+0000 7fad30c00700 1 -- 172.21.15.131:0/2642923779 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad24010de0 con 0x7fad34105e30 2024-05-08T22:16:20.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.594+0000 7fad3b000700 1 -- 172.21.15.131:0/2642923779 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad3419e5d0 con 0x7fad34105e30 2024-05-08T22:16:20.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.594+0000 7fad3b000700 1 -- 172.21.15.131:0/2642923779 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad341aa820 con 0x7fad34105e30 2024-05-08T22:16:20.599 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:20 smithi131 ceph-mon[37260]: pgmap v279: 129 pgs: 19 active+clean+snaptrim, 48 active+clean, 62 active+clean+snaptrim_wait; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 598 op/s 2024-05-08T22:16:20.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.607+0000 7fad30c00700 1 -- 172.21.15.131:0/2642923779 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fad24010480 con 0x7fad34105e30 2024-05-08T22:16:20.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.610+0000 7fad30c00700 1 --2- 172.21.15.131:0/2642923779 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fad180a1310 0x7fad180a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.610+0000 7fad30c00700 1 -- 172.21.15.131:0/2642923779 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fad240cc970 con 0x7fad34105e30 2024-05-08T22:16:20.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.611+0000 7fad33e00700 1 --2- 172.21.15.131:0/2642923779 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fad180a1310 0x7fad180a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.613+0000 7fad4ead5b80 1 Processor -- start 2024-05-08T22:16:20.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.613+0000 7fad33e00700 1 --2- 172.21.15.131:0/2642923779 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fad180a1310 0x7fad180a37d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00b830 tx=0x7fad2c006230 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:20.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.614+0000 7fad4ead5b80 1 -- start start 2024-05-08T22:16:20.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.614+0000 7fad4ead5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560174183e00 0x5601741841e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.614+0000 7fad4ead5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560174184720 0x56017417e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.614+0000 7fad4ead5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017417e890 0x56017417ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.614+0000 7fad4ead5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601740d5150 con 0x560174184720 2024-05-08T22:16:20.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.614+0000 7fad4ead5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601740d2b10 con 0x560174183e00 2024-05-08T22:16:20.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.614+0000 7fad4ead5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601740ce400 con 0x56017417e890 2024-05-08T22:16:20.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.615+0000 7fad33400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560174184720 0x56017417e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.615+0000 7fad33e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560174183e00 0x5601741841e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.615+0000 7fad33e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560174183e00 0x5601741841e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:34778/0 (socket says 172.21.15.131:34778) 2024-05-08T22:16:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.615+0000 7fad33400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560174184720 0x56017417e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44784/0 (socket says 172.21.15.131:44784) 2024-05-08T22:16:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.615+0000 7fad33e00700 1 -- 172.21.15.131:0/3625793930 learned_addr learned my addr 172.21.15.131:0/3625793930 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.615+0000 7fad39000700 1 --2- 172.21.15.131:0/3625793930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017417e890 0x56017417ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.615+0000 7fad39000700 1 -- 172.21.15.131:0/3625793930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560174183e00 msgr2=0x5601741841e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.615+0000 7fad39000700 1 --2- 172.21.15.131:0/3625793930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560174183e00 0x5601741841e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.615+0000 7fad39000700 1 -- 172.21.15.131:0/3625793930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560174184720 msgr2=0x56017417e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.615+0000 7fad39000700 1 --2- 172.21.15.131:0/3625793930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560174184720 0x56017417e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.615+0000 7fad39000700 1 -- 172.21.15.131:0/3625793930 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560173f4f3b0 con 0x56017417e890 2024-05-08T22:16:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.616+0000 7fad39000700 1 --2- 172.21.15.131:0/3625793930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017417e890 0x56017417ecd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fad240c9d90 tx=0x7fad24099520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=565ee211851dacc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.616+0000 7fad33400700 1 --2- 172.21.15.131:0/3625793930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560174184720 0x56017417e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-08T22:16:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.616+0000 7fad29600700 1 -- 172.21.15.131:0/3625793930 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad24099e50 con 0x56017417e890 2024-05-08T22:16:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.616+0000 7fad29600700 1 -- 172.21.15.131:0/3625793930 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad24099750 con 0x56017417e890 2024-05-08T22:16:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.616+0000 7fad29600700 1 -- 172.21.15.131:0/3625793930 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2409d380 con 0x56017417e890 2024-05-08T22:16:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.619+0000 7fad4ead5b80 1 -- 172.21.15.131:0/3625793930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017417e890 msgr2=0x56017417ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.619+0000 7fad4ead5b80 1 --2- 172.21.15.131:0/3625793930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017417e890 0x56017417ecd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fad240c9d90 tx=0x7fad24099520 comp rx=0 tx=0).stop 2024-05-08T22:16:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.619+0000 7fad4ead5b80 1 -- 172.21.15.131:0/3625793930 shutdown_connections 2024-05-08T22:16:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.619+0000 7fad4ead5b80 1 --2- 172.21.15.131:0/3625793930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017417e890 0x56017417ecd0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.619+0000 7fad4ead5b80 1 --2- 172.21.15.131:0/3625793930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560174183e00 0x5601741841e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.619+0000 7fad4ead5b80 1 --2- 172.21.15.131:0/3625793930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560174184720 0x56017417e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.619+0000 7fad4ead5b80 1 -- 172.21.15.131:0/3625793930 >> 172.21.15.131:0/3625793930 conn(0x5601740dc210 msgr2=0x560174178df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.619+0000 7fad4ead5b80 1 -- 172.21.15.131:0/3625793930 shutdown_connections 2024-05-08T22:16:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.619+0000 7fad4ead5b80 1 -- 172.21.15.131:0/3625793930 wait complete. 2024-05-08T22:16:20.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.620+0000 7fad4ead5b80 1 -- start start 2024-05-08T22:16:20.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.620+0000 7fad4ead5b80 1 Processor -- start 2024-05-08T22:16:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.622+0000 7fad4ead5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017417e890 0x560174187b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.622+0000 7fad4ead5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560174183e00 0x5601741880d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.622+0000 7fad4ead5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560174184720 0x560174188610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.623+0000 7fad33e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017417e890 0x560174187b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.623+0000 7fad33e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017417e890 0x560174187b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43654/0 (socket says 172.21.15.131:43654) 2024-05-08T22:16:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.623+0000 7fad33400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560174183e00 0x5601741880d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.623+0000 7fad33400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560174183e00 0x5601741880d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:34780/0 (socket says 172.21.15.131:34780) 2024-05-08T22:16:20.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.623+0000 7fad33400700 1 -- 172.21.15.131:0/3437607219 learned_addr learned my addr 172.21.15.131:0/3437607219 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:20.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.624+0000 7fad4ead5b80 1 -- 172.21.15.131:0/3437607219 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017406a560 con 0x560174184720 2024-05-08T22:16:20.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.624+0000 7fad4ead5b80 1 -- 172.21.15.131:0/3437607219 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017406a200 con 0x560174183e00 2024-05-08T22:16:20.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.624+0000 7fad4ead5b80 1 -- 172.21.15.131:0/3437607219 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560173f516a0 con 0x56017417e890 2024-05-08T22:16:20.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.624+0000 7fad4ead5b80 10 client.0 _ll_register_callbacks cb 0x560174073b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:16:20.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.624+0000 7fad39000700 1 --2- 172.21.15.131:0/3437607219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560174184720 0x560174188610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.625+0000 7fad39000700 1 -- 172.21.15.131:0/3437607219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017417e890 msgr2=0x560174187b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:20.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.625+0000 7fad39000700 1 --2- 172.21.15.131:0/3437607219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017417e890 0x560174187b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.625+0000 7fad39000700 1 -- 172.21.15.131:0/3437607219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560174183e00 msgr2=0x5601741880d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:20.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.625+0000 7fad39000700 1 --2- 172.21.15.131:0/3437607219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560174183e00 0x5601741880d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.626+0000 7fad39000700 1 -- 172.21.15.131:0/3437607219 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad341aa820 con 0x560174184720 2024-05-08T22:16:20.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.626+0000 7fad39000700 1 --2- 172.21.15.131:0/3437607219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560174184720 0x560174188610 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fad240c9dc0 tx=0x7fad240b92f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48039bc129c01941 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:20.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.628+0000 7fad1e000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:16:20.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.628+0000 7fad1e000700 1 -- 172.21.15.131:0/3437607219 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2409d580 con 0x560174184720 2024-05-08T22:16:20.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.628+0000 7fad1e000700 1 -- 172.21.15.131:0/3437607219 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fad240a1bd0 con 0x560174184720 2024-05-08T22:16:20.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.628+0000 7fad4ead5b80 1 -- 172.21.15.131:0/3437607219 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560174027270 con 0x560174184720 2024-05-08T22:16:20.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.629+0000 7fad1e000700 1 -- 172.21.15.131:0/3437607219 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad2409d330 con 0x560174184720 2024-05-08T22:16:20.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.629+0000 7fad4ead5b80 10 client.16548 fetch_fsmap learned FSMap version 29 2024-05-08T22:16:20.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.629+0000 7fad4ead5b80 1 -- 172.21.15.131:0/3437607219 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560173efbb60 con 0x560174184720 2024-05-08T22:16:20.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.629+0000 7fad1e000700 1 -- 172.21.15.131:0/3437607219 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560174027270 con 0x560174184720 2024-05-08T22:16:20.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.630+0000 7fad1e000700 1 -- 172.21.15.131:0/3437607219 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fad240a1740 con 0x560174184720 2024-05-08T22:16:20.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.630+0000 7fad4ead5b80 10 client.16548 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:16:20.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.630+0000 7fad4ead5b80 10 client.16548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:16:20.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.630+0000 7fad4ead5b80 20 client.16548 populate_metadata read hostname 'smithi131' 2024-05-08T22:16:20.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.630+0000 7fad4ead5b80 1 --2- 172.21.15.131:0/3437607219 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560174197e70 0x56017419a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.630+0000 7fad4ead5b80 4 client.16548 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:16:20.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.630+0000 7fad4ead5b80 1 -- 172.21.15.131:0/3437607219 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56017414d810 con 0x560174197e70 2024-05-08T22:16:20.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.631+0000 7fad33e00700 1 --2- 172.21.15.131:0/3437607219 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560174197e70 0x56017419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.633+0000 7fad33e00700 1 --2- 172.21.15.131:0/3437607219 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560174197e70 0x56017419a290 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46053f07ecc69ebc server_cookie=21f830ad52a73f26 in_seq=0 out_seq=0 2024-05-08T22:16:20.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.633+0000 7fad1e000700 10 client.16548 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:20.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.634+0000 7fad1e000700 1 -- 172.21.15.131:0/3437607219 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56017414d810 con 0x560174197e70 2024-05-08T22:16:20.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.634+0000 7fad1e000700 10 client.16548 handle_command_reply: tid=0 2024-05-08T22:16:20.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.634+0000 7fad4ead5b80 1 client.16548 shutdown 2024-05-08T22:16:20.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.634+0000 7fad4ead5b80 1 -- 172.21.15.131:0/3437607219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560174184720 msgr2=0x560174188610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:20.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.634+0000 7fad4ead5b80 1 --2- 172.21.15.131:0/3437607219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560174184720 0x560174188610 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fad240c9dc0 tx=0x7fad240b92f0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.635+0000 7fad4ead5b80 1 -- 172.21.15.131:0/3437607219 shutdown_connections 2024-05-08T22:16:20.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.635+0000 7fad4ead5b80 1 --2- 172.21.15.131:0/3437607219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560174184720 0x560174188610 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.635+0000 7fad4ead5b80 1 --2- 172.21.15.131:0/3437607219 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560174197e70 0x56017419a290 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.635+0000 7fad4ead5b80 1 --2- 172.21.15.131:0/3437607219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017417e890 0x560174187b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.635+0000 7fad4ead5b80 1 --2- 172.21.15.131:0/3437607219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560174183e00 0x5601741880d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.635+0000 7fad4ead5b80 1 -- 172.21.15.131:0/3437607219 >> 172.21.15.131:0/3437607219 conn(0x5601740dc210 msgr2=0x5601741785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:20.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.635+0000 7fad1e000700 0 client.16548 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:20.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.635+0000 7fad4ead5b80 1 -- 172.21.15.131:0/3437607219 shutdown_connections 2024-05-08T22:16:20.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.635+0000 7fad4ead5b80 1 -- 172.21.15.131:0/3437607219 wait complete. 2024-05-08T22:16:20.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.636+0000 7fad4ead5b80 20 client.16548 trim_cache size 0 max 16384 2024-05-08T22:16:20.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.664+0000 7fad4ead5b80 1 Processor -- start 2024-05-08T22:16:20.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.664+0000 7fad4ead5b80 1 -- start start 2024-05-08T22:16:20.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.665+0000 7fad4ead5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017417e890 0x56017417e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.665+0000 7fad4ead5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560174183e00 0x560174178d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.665+0000 7fad4ead5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560174184720 0x5601741880e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.665+0000 7fad4ead5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017418a690 con 0x56017417e890 2024-05-08T22:16:20.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.665+0000 7fad4ead5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560174175dc0 con 0x560174184720 2024-05-08T22:16:20.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.665+0000 7fad4ead5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017419f4a0 con 0x560174183e00 2024-05-08T22:16:20.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.666+0000 7fad39000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560174184720 0x5601741880e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.666+0000 7fad39000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560174184720 0x5601741880e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:34790/0 (socket says 172.21.15.131:34790) 2024-05-08T22:16:20.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.666+0000 7fad39000700 1 -- 172.21.15.131:0/1205357024 learned_addr learned my addr 172.21.15.131:0/1205357024 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:20.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.666+0000 7fad33e00700 1 --2- 172.21.15.131:0/1205357024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017417e890 0x56017417e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.666+0000 7fad33400700 1 --2- 172.21.15.131:0/1205357024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560174183e00 0x560174178d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.666+0000 7fad39000700 1 -- 172.21.15.131:0/1205357024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560174183e00 msgr2=0x560174178d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:20.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.666+0000 7fad39000700 1 --2- 172.21.15.131:0/1205357024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560174183e00 0x560174178d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.666+0000 7fad39000700 1 -- 172.21.15.131:0/1205357024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017417e890 msgr2=0x56017417e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:20.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.666+0000 7fad39000700 1 --2- 172.21.15.131:0/1205357024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017417e890 0x56017417e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.666+0000 7fad39000700 1 -- 172.21.15.131:0/1205357024 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560174094560 con 0x560174184720 2024-05-08T22:16:20.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.667+0000 7fad39000700 1 --2- 172.21.15.131:0/1205357024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560174184720 0x5601741880e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fad2400bb00 tx=0x7fad24099520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfcc63a5c8163609 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:20.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.670+0000 7fad2a000700 1 -- 172.21.15.131:0/1205357024 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2407aea0 con 0x560174184720 2024-05-08T22:16:20.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.670+0000 7fad2a000700 1 -- 172.21.15.131:0/1205357024 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad2407f070 con 0x560174184720 2024-05-08T22:16:20.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.671+0000 7fad4ead5b80 1 -- 172.21.15.131:0/1205357024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560174184720 msgr2=0x5601741880e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:20.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.671+0000 7fad4ead5b80 1 --2- 172.21.15.131:0/1205357024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560174184720 0x5601741880e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fad2400bb00 tx=0x7fad24099520 comp rx=0 tx=0).stop 2024-05-08T22:16:20.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.673+0000 7fad4ead5b80 1 -- 172.21.15.131:0/1205357024 shutdown_connections 2024-05-08T22:16:20.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.673+0000 7fad4ead5b80 1 --2- 172.21.15.131:0/1205357024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560174184720 0x5601741880e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.673+0000 7fad4ead5b80 1 --2- 172.21.15.131:0/1205357024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017417e890 0x56017417e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.673+0000 7fad4ead5b80 1 --2- 172.21.15.131:0/1205357024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560174183e00 0x560174178d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.673+0000 7fad4ead5b80 1 -- 172.21.15.131:0/1205357024 >> 172.21.15.131:0/1205357024 conn(0x5601740dc210 msgr2=0x56017417ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:20.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.674+0000 7fad4ead5b80 1 -- 172.21.15.131:0/1205357024 shutdown_connections 2024-05-08T22:16:20.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.674+0000 7fad4ead5b80 1 -- 172.21.15.131:0/1205357024 wait complete. 2024-05-08T22:16:20.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.675+0000 7fad4ead5b80 1 -- start start 2024-05-08T22:16:20.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.675+0000 7fad4ead5b80 1 Processor -- start 2024-05-08T22:16:20.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.676+0000 7fad4ead5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017417e890 0x56017418f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.676+0000 7fad4ead5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560174183e00 0x56017418d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.677+0000 7fad4ead5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560174184720 0x56017418d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.677+0000 7fad4ead5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017418ddb0 con 0x56017417e890 2024-05-08T22:16:20.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.677+0000 7fad4ead5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017418df20 con 0x560174184720 2024-05-08T22:16:20.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.677+0000 7fad4ead5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601741fe380 con 0x560174183e00 2024-05-08T22:16:20.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.677+0000 7fad4ead5b80 10 client.0 _ll_register_callbacks cb 0x560174073b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:16:20.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.677+0000 7fad33400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560174183e00 0x56017418d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.677+0000 7fad33400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560174183e00 0x56017418d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43658/0 (socket says 172.21.15.131:43658) 2024-05-08T22:16:20.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.677+0000 7fad33400700 1 -- 172.21.15.131:0/1458395875 learned_addr learned my addr 172.21.15.131:0/1458395875 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:20.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.678+0000 7fad39000700 1 --2- 172.21.15.131:0/1458395875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560174184720 0x56017418d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.678+0000 7fad33e00700 1 --2- 172.21.15.131:0/1458395875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017417e890 0x56017418f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.679+0000 7fad39000700 1 -- 172.21.15.131:0/1458395875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560174183e00 msgr2=0x56017418d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:20.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.679+0000 7fad39000700 1 --2- 172.21.15.131:0/1458395875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560174183e00 0x56017418d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.679+0000 7fad39000700 1 -- 172.21.15.131:0/1458395875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017417e890 msgr2=0x56017418f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:20.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.679+0000 7fad39000700 1 --2- 172.21.15.131:0/1458395875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017417e890 0x56017418f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.679+0000 7fad39000700 1 -- 172.21.15.131:0/1458395875 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560173efbb60 con 0x560174184720 2024-05-08T22:16:20.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.679+0000 7fad33e00700 1 --2- 172.21.15.131:0/1458395875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017417e890 0x56017418f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:16:20.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.681+0000 7fad39000700 1 --2- 172.21.15.131:0/1458395875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560174184720 0x56017418d650 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fad240ca280 tx=0x7fad24095490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee30aebdc38141bc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:20.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.681+0000 7fad1e000700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:16:20.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.683+0000 7fad1e000700 1 -- 172.21.15.131:0/1458395875 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2407aea0 con 0x560174184720 2024-05-08T22:16:20.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.683+0000 7fad1e000700 1 -- 172.21.15.131:0/1458395875 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fad24011c80 con 0x560174184720 2024-05-08T22:16:20.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.683+0000 7fad4ead5b80 1 -- 172.21.15.131:0/1458395875 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601740c6ec0 con 0x560174184720 2024-05-08T22:16:20.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.683+0000 7fad1e000700 1 -- 172.21.15.131:0/1458395875 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad2409d580 con 0x560174184720 2024-05-08T22:16:20.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.683+0000 7fad4ead5b80 10 client.26591 fetch_fsmap learned FSMap version 29 2024-05-08T22:16:20.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.683+0000 7fad4ead5b80 1 -- 172.21.15.131:0/1458395875 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560173f53cd0 con 0x560174184720 2024-05-08T22:16:20.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.684+0000 7fad1e000700 1 -- 172.21.15.131:0/1458395875 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5601740c6ec0 con 0x560174184720 2024-05-08T22:16:20.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.684+0000 7fad1e000700 1 -- 172.21.15.131:0/1458395875 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fad24095540 con 0x560174184720 2024-05-08T22:16:20.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.684+0000 7fad4ead5b80 10 client.26591 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:16:20.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.684+0000 7fad4ead5b80 10 client.26591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:16:20.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.684+0000 7fad4ead5b80 20 client.26591 populate_metadata read hostname 'smithi131' 2024-05-08T22:16:20.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.684+0000 7fad4ead5b80 1 --2- 172.21.15.131:0/1458395875 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5601741ff680 0x560174201a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:20.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.684+0000 7fad4ead5b80 4 client.26591 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:16:20.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.684+0000 7fad4ead5b80 1 -- 172.21.15.131:0/1458395875 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560173f1ccf0 con 0x5601741ff680 2024-05-08T22:16:20.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.684+0000 7fad33e00700 1 --2- 172.21.15.131:0/1458395875 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5601741ff680 0x560174201a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:20.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.687+0000 7fad1e000700 10 client.26591 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:20.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.687+0000 7fad33e00700 1 --2- 172.21.15.131:0/1458395875 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5601741ff680 0x560174201a70 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee6ab7a986d89a30 server_cookie=32230eb3eea912c2 in_seq=0 out_seq=0 2024-05-08T22:16:20.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.690+0000 7fad1e000700 1 -- 172.21.15.131:0/1458395875 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560173f1ccf0 con 0x5601741ff680 2024-05-08T22:16:20.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.690+0000 7fad1e000700 10 client.26591 handle_command_reply: tid=0 2024-05-08T22:16:20.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.690+0000 7fad4ead5b80 1 client.26591 shutdown 2024-05-08T22:16:20.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.690+0000 7fad4ead5b80 1 -- 172.21.15.131:0/1458395875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560174184720 msgr2=0x56017418d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:20.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.690+0000 7fad4ead5b80 1 --2- 172.21.15.131:0/1458395875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560174184720 0x56017418d650 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fad240ca280 tx=0x7fad24095490 comp rx=0 tx=0).stop 2024-05-08T22:16:20.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.690+0000 7fad4ead5b80 1 -- 172.21.15.131:0/1458395875 shutdown_connections 2024-05-08T22:16:20.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.690+0000 7fad4ead5b80 1 --2- 172.21.15.131:0/1458395875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560174184720 0x56017418d650 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.690+0000 7fad4ead5b80 1 --2- 172.21.15.131:0/1458395875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56017417e890 0x56017418f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.690+0000 7fad4ead5b80 1 --2- 172.21.15.131:0/1458395875 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5601741ff680 0x560174201a70 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.690+0000 7fad4ead5b80 1 --2- 172.21.15.131:0/1458395875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560174183e00 0x56017418d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.690+0000 7fad4ead5b80 1 -- 172.21.15.131:0/1458395875 >> 172.21.15.131:0/1458395875 conn(0x5601740dc210 msgr2=0x5601741863d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:20.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.690+0000 7fad1e000700 0 client.26591 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:20.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.691+0000 7fad4ead5b80 1 -- 172.21.15.131:0/1458395875 shutdown_connections 2024-05-08T22:16:20.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.691+0000 7fad4ead5b80 1 -- 172.21.15.131:0/1458395875 wait complete. 2024-05-08T22:16:20.693 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:16:20.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.691+0000 7fad4ead5b80 20 client.26591 trim_cache size 0 max 16384 2024-05-08T22:16:20.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.692+0000 7fad3b000700 1 -- 172.21.15.131:0/2642923779 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fad180a1310 msgr2=0x7fad180a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:20.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.692+0000 7fad3b000700 1 --2- 172.21.15.131:0/2642923779 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fad180a1310 0x7fad180a37d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00b830 tx=0x7fad2c006230 comp rx=0 tx=0).stop 2024-05-08T22:16:20.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.692+0000 7fad3b000700 1 -- 172.21.15.131:0/2642923779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad34105e30 msgr2=0x7fad3419d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:20.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.692+0000 7fad3b000700 1 --2- 172.21.15.131:0/2642923779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad34105e30 0x7fad3419d960 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fad2400b3f0 tx=0x7fad2400b700 comp rx=0 tx=0).stop 2024-05-08T22:16:20.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.692+0000 7fad3b000700 1 -- 172.21.15.131:0/2642923779 shutdown_connections 2024-05-08T22:16:20.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.692+0000 7fad3b000700 1 --2- 172.21.15.131:0/2642923779 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fad180a1310 0x7fad180a37d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.692+0000 7fad3b000700 1 --2- 172.21.15.131:0/2642923779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad34105e30 0x7fad3419d960 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.692+0000 7fad3b000700 1 --2- 172.21.15.131:0/2642923779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad34068590 0x7fad341a3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.692+0000 7fad3b000700 1 --2- 172.21.15.131:0/2642923779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad34103930 0x7fad341a3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:20.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.692+0000 7fad3b000700 1 -- 172.21.15.131:0/2642923779 >> 172.21.15.131:0/2642923779 conn(0x7fad34076f40 msgr2=0x7fad340fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:20.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.693+0000 7fad3b000700 1 -- 172.21.15.131:0/2642923779 shutdown_connections 2024-05-08T22:16:20.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:20.694+0000 7fad3b000700 1 -- 172.21.15.131:0/2642923779 wait complete. 2024-05-08T22:16:21.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:20 smithi157 ceph-mon[77746]: pgmap v279: 129 pgs: 19 active+clean+snaptrim, 48 active+clean, 62 active+clean+snaptrim_wait; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 598 op/s 2024-05-08T22:16:21.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:20 smithi132 ceph-mon[78180]: pgmap v279: 129 pgs: 19 active+clean+snaptrim, 48 active+clean, 62 active+clean+snaptrim_wait; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 598 op/s 2024-05-08T22:16:21.734 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T22:16:21.734 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T22:16:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:21 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:16:22.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:21 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:16:22.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.074+0000 7f50fd600700 1 -- 172.21.15.131:0/2876169823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50f8106290 msgr2=0x7f50f810de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:22.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.074+0000 7f50fd600700 1 --2- 172.21.15.131:0/2876169823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50f8106290 0x7f50f810de10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009f90 tx=0x7f50f000b330 comp rx=0 tx=0).stop 2024-05-08T22:16:22.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.074+0000 7f50fd600700 1 -- 172.21.15.131:0/2876169823 shutdown_connections 2024-05-08T22:16:22.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.074+0000 7f50fd600700 1 --2- 172.21.15.131:0/2876169823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50f8106290 0x7f50f810de10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.074+0000 7f50fd600700 1 --2- 172.21.15.131:0/2876169823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50f80682e0 0x7f50f80686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.074+0000 7f50fd600700 1 --2- 172.21.15.131:0/2876169823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50f8068c90 0x7f50f8105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.074+0000 7f50fd600700 1 -- 172.21.15.131:0/2876169823 >> 172.21.15.131:0/2876169823 conn(0x7f50f8076f40 msgr2=0x7f50f8077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:22.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.074+0000 7f50fd600700 1 -- 172.21.15.131:0/2876169823 shutdown_connections 2024-05-08T22:16:22.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.074+0000 7f50fd600700 1 -- 172.21.15.131:0/2876169823 wait complete. 2024-05-08T22:16:22.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.076+0000 7f50fd600700 1 Processor -- start 2024-05-08T22:16:22.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.076+0000 7f50fd600700 1 -- start start 2024-05-08T22:16:22.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.076+0000 7f50fd600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50f80682e0 0x7f50f8199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.076+0000 7f50fd600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50f8106290 0x7f50f8199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.077+0000 7f50fd600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50f819df20 0x7f50f819e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.077+0000 7f50fd600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f819e9a0 con 0x7f50f819df20 2024-05-08T22:16:22.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.077+0000 7f50fd600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f819eb10 con 0x7f50f80682e0 2024-05-08T22:16:22.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.077+0000 7f50fd600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f819ecd0 con 0x7f50f8106290 2024-05-08T22:16:22.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.077+0000 7f50f6a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50f80682e0 0x7f50f8199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.077+0000 7f50f6a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50f80682e0 0x7f50f8199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:34826/0 (socket says 172.21.15.131:34826) 2024-05-08T22:16:22.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.077+0000 7f50f6a00700 1 -- 172.21.15.131:0/3999523489 learned_addr learned my addr 172.21.15.131:0/3999523489 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:22.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.077+0000 7f50f6000700 1 --2- 172.21.15.131:0/3999523489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50f8106290 0x7f50f8199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.077+0000 7f50f7400700 1 --2- 172.21.15.131:0/3999523489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50f819df20 0x7f50f819e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.077+0000 7f50f6a00700 1 -- 172.21.15.131:0/3999523489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50f8106290 msgr2=0x7f50f8199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:22.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.077+0000 7f50f6a00700 1 --2- 172.21.15.131:0/3999523489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50f8106290 0x7f50f8199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.077+0000 7f50f6a00700 1 -- 172.21.15.131:0/3999523489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50f819df20 msgr2=0x7f50f819e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:22.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.077+0000 7f50f6a00700 1 --2- 172.21.15.131:0/3999523489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50f819df20 0x7f50f819e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.077+0000 7f50f6a00700 1 -- 172.21.15.131:0/3999523489 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e80095f0 con 0x7f50f80682e0 2024-05-08T22:16:22.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.078+0000 7f50f6a00700 1 --2- 172.21.15.131:0/3999523489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50f80682e0 0x7f50f8199640 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f50e800ec10 tx=0x7f50e800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:22.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.078+0000 7f50ef400700 1 -- 172.21.15.131:0/3999523489 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e800cd40 con 0x7f50f80682e0 2024-05-08T22:16:22.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.078+0000 7f50ef400700 1 -- 172.21.15.131:0/3999523489 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50e80044e0 con 0x7f50f80682e0 2024-05-08T22:16:22.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.078+0000 7f50ef400700 1 -- 172.21.15.131:0/3999523489 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e8010690 con 0x7f50f80682e0 2024-05-08T22:16:22.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.078+0000 7f50fd600700 1 -- 172.21.15.131:0/3999523489 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f0009c40 con 0x7f50f80682e0 2024-05-08T22:16:22.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.080+0000 7f50fd600700 1 -- 172.21.15.131:0/3999523489 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f81a6530 con 0x7f50f80682e0 2024-05-08T22:16:22.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.084+0000 7f50ef400700 1 -- 172.21.15.131:0/3999523489 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f50e8004020 con 0x7f50f80682e0 2024-05-08T22:16:22.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.086+0000 7f50ef400700 1 --2- 172.21.15.131:0/3999523489 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f50dc0a1390 0x7f50dc0a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.087+0000 7f50ef400700 1 -- 172.21.15.131:0/3999523489 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f50e8014070 con 0x7f50f80682e0 2024-05-08T22:16:22.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.087+0000 7f51110bcb80 1 Processor -- start 2024-05-08T22:16:22.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.087+0000 7f50f6000700 1 --2- 172.21.15.131:0/3999523489 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f50dc0a1390 0x7f50dc0a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.087+0000 7f51110bcb80 1 -- start start 2024-05-08T22:16:22.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:21 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:16:22.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.088+0000 7f51110bcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567efe7ee80 0x5567efe7f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.088+0000 7f50f6000700 1 --2- 172.21.15.131:0/3999523489 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f50dc0a1390 0x7f50dc0a3850 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f50f819ab90 tx=0x7f50e4009190 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:22.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.088+0000 7f51110bcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567efe7f7a0 0x5567efe793d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.088+0000 7f51110bcb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567efe79910 0x5567efe79d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.088+0000 7f51110bcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567efdd01d0 con 0x5567efe7f7a0 2024-05-08T22:16:22.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.088+0000 7f51110bcb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567efdcdb90 con 0x5567efe79910 2024-05-08T22:16:22.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.088+0000 7f51110bcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567efdc9480 con 0x5567efe7ee80 2024-05-08T22:16:22.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.088+0000 7f50f6000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567efe7f7a0 0x5567efe793d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.089+0000 7f50f6000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567efe7f7a0 0x5567efe793d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44834/0 (socket says 172.21.15.131:44834) 2024-05-08T22:16:22.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.089+0000 7f50f6000700 1 -- 172.21.15.131:0/3942340104 learned_addr learned my addr 172.21.15.131:0/3942340104 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:22.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.089+0000 7f50f6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567efe7ee80 0x5567efe7f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.089+0000 7f50f7400700 1 --2- 172.21.15.131:0/3942340104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567efe79910 0x5567efe79d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.089+0000 7f50f7400700 1 -- 172.21.15.131:0/3942340104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567efe7ee80 msgr2=0x5567efe7f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:22.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.089+0000 7f50f7400700 1 --2- 172.21.15.131:0/3942340104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567efe7ee80 0x5567efe7f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.089+0000 7f50f7400700 1 -- 172.21.15.131:0/3942340104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567efe7f7a0 msgr2=0x5567efe793d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:22.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.089+0000 7f50f7400700 1 --2- 172.21.15.131:0/3942340104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567efe7f7a0 0x5567efe793d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.089+0000 7f50f7400700 1 -- 172.21.15.131:0/3942340104 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567efc4a430 con 0x5567efe79910 2024-05-08T22:16:22.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.090+0000 7f50f7400700 1 --2- 172.21.15.131:0/3942340104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567efe79910 0x5567efe79d50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f50f000b6c0 tx=0x7f50f0007b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c72c3c20b5915faf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:22.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.090+0000 7f50e3400700 1 -- 172.21.15.131:0/3942340104 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f000e070 con 0x5567efe79910 2024-05-08T22:16:22.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.091+0000 7f50e3400700 1 -- 172.21.15.131:0/3942340104 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50f0007d00 con 0x5567efe79910 2024-05-08T22:16:22.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.091+0000 7f50e3400700 1 -- 172.21.15.131:0/3942340104 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f0007680 con 0x5567efe79910 2024-05-08T22:16:22.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.092+0000 7f51110bcb80 1 -- 172.21.15.131:0/3942340104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567efe79910 msgr2=0x5567efe79d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:22.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.092+0000 7f51110bcb80 1 --2- 172.21.15.131:0/3942340104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567efe79910 0x5567efe79d50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f50f000b6c0 tx=0x7f50f0007b10 comp rx=0 tx=0).stop 2024-05-08T22:16:22.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.093+0000 7f51110bcb80 1 -- 172.21.15.131:0/3942340104 shutdown_connections 2024-05-08T22:16:22.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.093+0000 7f51110bcb80 1 --2- 172.21.15.131:0/3942340104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567efe79910 0x5567efe79d50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.093+0000 7f51110bcb80 1 --2- 172.21.15.131:0/3942340104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567efe7ee80 0x5567efe7f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.093+0000 7f51110bcb80 1 --2- 172.21.15.131:0/3942340104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567efe7f7a0 0x5567efe793d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.093+0000 7f51110bcb80 1 -- 172.21.15.131:0/3942340104 >> 172.21.15.131:0/3942340104 conn(0x5567efdd7290 msgr2=0x5567efe73e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:22.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.093+0000 7f51110bcb80 1 -- 172.21.15.131:0/3942340104 shutdown_connections 2024-05-08T22:16:22.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.093+0000 7f51110bcb80 1 -- 172.21.15.131:0/3942340104 wait complete. 2024-05-08T22:16:22.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.094+0000 7f51110bcb80 1 -- start start 2024-05-08T22:16:22.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.094+0000 7f51110bcb80 1 Processor -- start 2024-05-08T22:16:22.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.095+0000 7f51110bcb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567efe79910 0x5567efe857d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.095+0000 7f51110bcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567efe7ee80 0x5567efe82a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.095+0000 7f51110bcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567efe7f7a0 0x5567efe83190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.095+0000 7f51110bcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567efd655e0 con 0x5567efe7ee80 2024-05-08T22:16:22.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.095+0000 7f51110bcb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567efd65280 con 0x5567efe79910 2024-05-08T22:16:22.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.095+0000 7f51110bcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567efc4c720 con 0x5567efe7f7a0 2024-05-08T22:16:22.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.095+0000 7f51110bcb80 10 client.0 _ll_register_callbacks cb 0x5567efd6ebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:16:22.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.096+0000 7f50f6000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567efe7ee80 0x5567efe82a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.096+0000 7f50f6000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567efe7ee80 0x5567efe82a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44842/0 (socket says 172.21.15.131:44842) 2024-05-08T22:16:22.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.096+0000 7f50f6000700 1 -- 172.21.15.131:0/3045254090 learned_addr learned my addr 172.21.15.131:0/3045254090 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:22.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.096+0000 7f50f7400700 1 --2- 172.21.15.131:0/3045254090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567efe7f7a0 0x5567efe83190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.097+0000 7f50f6a00700 1 --2- 172.21.15.131:0/3045254090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567efe79910 0x5567efe857d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.097+0000 7f50f6a00700 1 -- 172.21.15.131:0/3045254090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567efe7f7a0 msgr2=0x5567efe83190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:22.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.097+0000 7f50f6a00700 1 --2- 172.21.15.131:0/3045254090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567efe7f7a0 0x5567efe83190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.097+0000 7f50f6a00700 1 -- 172.21.15.131:0/3045254090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567efe7ee80 msgr2=0x5567efe82a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:22.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.097+0000 7f50f6a00700 1 --2- 172.21.15.131:0/3045254090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567efe7ee80 0x5567efe82a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.097+0000 7f50f6a00700 1 -- 172.21.15.131:0/3045254090 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f810e650 con 0x5567efe79910 2024-05-08T22:16:22.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.098+0000 7f50f6a00700 1 --2- 172.21.15.131:0/3045254090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567efe79910 0x5567efe857d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f50e80ca9a0 tx=0x7f50e8003e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71ed46595997df7a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:22.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.098+0000 7f50d7e00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:16:22.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.098+0000 7f50d7e00700 1 -- 172.21.15.131:0/3045254090 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e809a360 con 0x5567efe79910 2024-05-08T22:16:22.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.098+0000 7f50d7e00700 1 -- 172.21.15.131:0/3045254090 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f50e809e190 con 0x5567efe79910 2024-05-08T22:16:22.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.098+0000 7f51110bcb80 1 -- 172.21.15.131:0/3045254090 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5567efd222f0 con 0x5567efe79910 2024-05-08T22:16:22.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.098+0000 7f50d7e00700 1 -- 172.21.15.131:0/3045254090 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50e80a27e0 con 0x5567efe79910 2024-05-08T22:16:22.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.100+0000 7f51110bcb80 10 client.26603 fetch_fsmap learned FSMap version 29 2024-05-08T22:16:22.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.100+0000 7f51110bcb80 1 -- 172.21.15.131:0/3045254090 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5567efbf6be0 con 0x5567efe79910 2024-05-08T22:16:22.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.101+0000 7f50d7e00700 1 -- 172.21.15.131:0/3045254090 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5567efd222f0 con 0x5567efe79910 2024-05-08T22:16:22.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.101+0000 7f50d7e00700 1 -- 172.21.15.131:0/3045254090 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f50e8099850 con 0x5567efe79910 2024-05-08T22:16:22.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.101+0000 7f51110bcb80 10 client.26603 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:16:22.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.101+0000 7f51110bcb80 10 client.26603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:16:22.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.101+0000 7f51110bcb80 20 client.26603 populate_metadata read hostname 'smithi131' 2024-05-08T22:16:22.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.101+0000 7f51110bcb80 1 --2- 172.21.15.131:0/3045254090 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5567efea4250 0x5567efea6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.101+0000 7f51110bcb80 4 client.26603 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:16:22.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.101+0000 7f51110bcb80 1 -- 172.21.15.131:0/3045254090 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567efe48890 con 0x5567efea4250 2024-05-08T22:16:22.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.101+0000 7f50f6000700 1 --2- 172.21.15.131:0/3045254090 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5567efea4250 0x5567efea6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.102+0000 7f50f6000700 1 --2- 172.21.15.131:0/3045254090 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5567efea4250 0x5567efea6670 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7c98d30c386b378 server_cookie=16de50475f582447 in_seq=0 out_seq=0 2024-05-08T22:16:22.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.102+0000 7f50d7e00700 10 client.26603 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:22.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.105+0000 7f50d7e00700 1 -- 172.21.15.131:0/3045254090 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5567efe48890 con 0x5567efea4250 2024-05-08T22:16:22.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.105+0000 7f50d7e00700 10 client.26603 handle_command_reply: tid=0 2024-05-08T22:16:22.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.105+0000 7f51110bcb80 1 client.26603 shutdown 2024-05-08T22:16:22.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.107+0000 7f51110bcb80 1 -- 172.21.15.131:0/3045254090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567efe79910 msgr2=0x5567efe857d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:22.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.107+0000 7f51110bcb80 1 --2- 172.21.15.131:0/3045254090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567efe79910 0x5567efe857d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f50e80ca9a0 tx=0x7f50e8003e80 comp rx=0 tx=0).stop 2024-05-08T22:16:22.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.108+0000 7f51110bcb80 1 -- 172.21.15.131:0/3045254090 shutdown_connections 2024-05-08T22:16:22.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.108+0000 7f51110bcb80 1 --2- 172.21.15.131:0/3045254090 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5567efea4250 0x5567efea6670 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.108+0000 7f51110bcb80 1 --2- 172.21.15.131:0/3045254090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567efe7f7a0 0x5567efe83190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.108+0000 7f51110bcb80 1 --2- 172.21.15.131:0/3045254090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567efe79910 0x5567efe857d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.108+0000 7f51110bcb80 1 --2- 172.21.15.131:0/3045254090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567efe7ee80 0x5567efe82a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.108+0000 7f51110bcb80 1 -- 172.21.15.131:0/3045254090 >> 172.21.15.131:0/3045254090 conn(0x5567efdd7290 msgr2=0x5567efe73620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:22.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.108+0000 7f50d7e00700 0 client.26603 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:22.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.108+0000 7f51110bcb80 1 -- 172.21.15.131:0/3045254090 shutdown_connections 2024-05-08T22:16:22.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.108+0000 7f51110bcb80 1 -- 172.21.15.131:0/3045254090 wait complete. 2024-05-08T22:16:22.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.111+0000 7f51110bcb80 20 client.26603 trim_cache size 0 max 16384 2024-05-08T22:16:22.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.173+0000 7f51110bcb80 1 Processor -- start 2024-05-08T22:16:22.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.173+0000 7f51110bcb80 1 -- start start 2024-05-08T22:16:22.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.173+0000 7f51110bcb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567efe79910 0x5567efe7e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.173+0000 7f51110bcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567efe7ee80 0x5567efe73da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.173+0000 7f51110bcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567efe7f7a0 0x5567efe8a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.173+0000 7f51110bcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567efe757a0 con 0x5567efe7ee80 2024-05-08T22:16:22.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.173+0000 7f51110bcb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567efea0c20 con 0x5567efe79910 2024-05-08T22:16:22.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.173+0000 7f51110bcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567efe6eb10 con 0x5567efe7f7a0 2024-05-08T22:16:22.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.174+0000 7f50f6a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567efe79910 0x5567efe7e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.174+0000 7f50f6a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567efe79910 0x5567efe7e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:34868/0 (socket says 172.21.15.131:34868) 2024-05-08T22:16:22.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.174+0000 7f50f6a00700 1 -- 172.21.15.131:0/1659852061 learned_addr learned my addr 172.21.15.131:0/1659852061 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:22.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.174+0000 7f50f6000700 1 --2- 172.21.15.131:0/1659852061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567efe7ee80 0x5567efe73da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.174+0000 7f50f7400700 1 --2- 172.21.15.131:0/1659852061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567efe7f7a0 0x5567efe8a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.175+0000 7f50f6a00700 1 -- 172.21.15.131:0/1659852061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567efe7f7a0 msgr2=0x5567efe8a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.175+0000 7f50f6a00700 1 --2- 172.21.15.131:0/1659852061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567efe7f7a0 0x5567efe8a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.175+0000 7f50f6a00700 1 -- 172.21.15.131:0/1659852061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567efe7ee80 msgr2=0x5567efe73da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.175+0000 7f50f6a00700 1 --2- 172.21.15.131:0/1659852061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567efe7ee80 0x5567efe73da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.175+0000 7f50f6a00700 1 -- 172.21.15.131:0/1659852061 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567efe79fc0 con 0x5567efe79910 2024-05-08T22:16:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.175+0000 7f50f7400700 1 --2- 172.21.15.131:0/1659852061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567efe7f7a0 0x5567efe8a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:16:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.175+0000 7f50f6000700 1 --2- 172.21.15.131:0/1659852061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567efe7ee80 0x5567efe73da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:16:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.176+0000 7f50f6a00700 1 --2- 172.21.15.131:0/1659852061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567efe79910 0x5567efe7e260 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f50e8096dd0 tx=0x7f50e8098910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e6351189b37aa3d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.176+0000 7f50e3e00700 1 -- 172.21.15.131:0/1659852061 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e80b8950 con 0x5567efe79910 2024-05-08T22:16:22.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.176+0000 7f50e3e00700 1 -- 172.21.15.131:0/1659852061 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50e809a7c0 con 0x5567efe79910 2024-05-08T22:16:22.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.176+0000 7f50e3e00700 1 -- 172.21.15.131:0/1659852061 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e80a27e0 con 0x5567efe79910 2024-05-08T22:16:22.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.179+0000 7f51110bcb80 1 -- 172.21.15.131:0/1659852061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567efe79910 msgr2=0x5567efe7e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:22.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.179+0000 7f51110bcb80 1 --2- 172.21.15.131:0/1659852061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567efe79910 0x5567efe7e260 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f50e8096dd0 tx=0x7f50e8098910 comp rx=0 tx=0).stop 2024-05-08T22:16:22.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.181+0000 7f51110bcb80 1 -- 172.21.15.131:0/1659852061 shutdown_connections 2024-05-08T22:16:22.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.181+0000 7f51110bcb80 1 --2- 172.21.15.131:0/1659852061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567efe7f7a0 0x5567efe8a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.181+0000 7f51110bcb80 1 --2- 172.21.15.131:0/1659852061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567efe79910 0x5567efe7e260 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.181+0000 7f51110bcb80 1 --2- 172.21.15.131:0/1659852061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567efe7ee80 0x5567efe73da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.181+0000 7f51110bcb80 1 -- 172.21.15.131:0/1659852061 >> 172.21.15.131:0/1659852061 conn(0x5567efdd7290 msgr2=0x5567efe7db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:22.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.181+0000 7f51110bcb80 1 -- 172.21.15.131:0/1659852061 shutdown_connections 2024-05-08T22:16:22.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.181+0000 7f51110bcb80 1 -- 172.21.15.131:0/1659852061 wait complete. 2024-05-08T22:16:22.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.182+0000 7f51110bcb80 1 -- start start 2024-05-08T22:16:22.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.182+0000 7f51110bcb80 1 Processor -- start 2024-05-08T22:16:22.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.183+0000 7f51110bcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567efe79910 0x5567efe89480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.183+0000 7f51110bcb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567efe7ee80 0x5567efe899c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.183+0000 7f51110bcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567efe7f7a0 0x5567efefa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.184+0000 7f51110bcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567efefaca0 con 0x5567efe79910 2024-05-08T22:16:22.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.184+0000 7f51110bcb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567efefade0 con 0x5567efe7ee80 2024-05-08T22:16:22.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.184+0000 7f51110bcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567efefafa0 con 0x5567efe7f7a0 2024-05-08T22:16:22.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.184+0000 7f51110bcb80 10 client.0 _ll_register_callbacks cb 0x5567efd6ebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:16:22.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.184+0000 7f50f7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567efe7f7a0 0x5567efefa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.184+0000 7f50f7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567efe7f7a0 0x5567efefa760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43708/0 (socket says 172.21.15.131:43708) 2024-05-08T22:16:22.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.184+0000 7f50f7400700 1 -- 172.21.15.131:0/2321814193 learned_addr learned my addr 172.21.15.131:0/2321814193 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:22.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.185+0000 7f50f7400700 1 -- 172.21.15.131:0/2321814193 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567efe7ee80 msgr2=0x5567efe899c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:16:22.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.185+0000 7f50f7400700 1 --2- 172.21.15.131:0/2321814193 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567efe7ee80 0x5567efe899c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.185+0000 7f50f7400700 1 -- 172.21.15.131:0/2321814193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567efe79910 msgr2=0x5567efe89480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:22.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.185+0000 7f50f7400700 1 --2- 172.21.15.131:0/2321814193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567efe79910 0x5567efe89480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.185+0000 7f50f7400700 1 -- 172.21.15.131:0/2321814193 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f0007e70 con 0x5567efe7f7a0 2024-05-08T22:16:22.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.185+0000 7f50f7400700 1 --2- 172.21.15.131:0/2321814193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567efe7f7a0 0x5567efefa760 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f50f0000f80 tx=0x7f50f0004b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b1512553c797b0f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:22.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.185+0000 7f50d7e00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:16:22.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.186+0000 7f50d7e00700 1 -- 172.21.15.131:0/2321814193 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f000e040 con 0x5567efe7f7a0 2024-05-08T22:16:22.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.186+0000 7f50d7e00700 1 -- 172.21.15.131:0/2321814193 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f50f00073e0 con 0x5567efe7f7a0 2024-05-08T22:16:22.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.186+0000 7f51110bcb80 1 -- 172.21.15.131:0/2321814193 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50f0007d00 con 0x5567efe7f7a0 2024-05-08T22:16:22.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.187+0000 7f50d7e00700 1 -- 172.21.15.131:0/2321814193 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50f00075a0 con 0x5567efe7f7a0 2024-05-08T22:16:22.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.188+0000 7f51110bcb80 10 client.26692 fetch_fsmap learned FSMap version 29 2024-05-08T22:16:22.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.188+0000 7f51110bcb80 1 -- 172.21.15.131:0/2321814193 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5567efbf6be0 con 0x5567efe7f7a0 2024-05-08T22:16:22.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.188+0000 7f50d7e00700 1 -- 172.21.15.131:0/2321814193 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50f0007d00 con 0x5567efe7f7a0 2024-05-08T22:16:22.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.188+0000 7f50d7e00700 1 -- 172.21.15.131:0/2321814193 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f50f000b050 con 0x5567efe7f7a0 2024-05-08T22:16:22.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.188+0000 7f51110bcb80 10 client.26692 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:16:22.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.188+0000 7f51110bcb80 10 client.26692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:16:22.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.188+0000 7f51110bcb80 20 client.26692 populate_metadata read hostname 'smithi131' 2024-05-08T22:16:22.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.188+0000 7f51110bcb80 1 --2- 172.21.15.131:0/2321814193 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5567efefc420 0x5567efefe840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.188+0000 7f51110bcb80 4 client.26692 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T22:16:22.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.188+0000 7f51110bcb80 1 -- 172.21.15.131:0/2321814193 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5567efdc1f40 con 0x5567efefc420 2024-05-08T22:16:22.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.191+0000 7f50f6a00700 1 --2- 172.21.15.131:0/2321814193 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5567efefc420 0x5567efefe840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.192+0000 7f50f6a00700 1 --2- 172.21.15.131:0/2321814193 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5567efefc420 0x5567efefe840 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fbc2bddf6d7aa60 server_cookie=3521f9f4bebb599f in_seq=0 out_seq=0 2024-05-08T22:16:22.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.192+0000 7f50d7e00700 10 client.26692 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:22.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.193+0000 7f50d7e00700 1 -- 172.21.15.131:0/2321814193 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5567efdc1f40 con 0x5567efefc420 2024-05-08T22:16:22.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.193+0000 7f50d7e00700 10 client.26692 handle_command_reply: tid=0 2024-05-08T22:16:22.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.193+0000 7f51110bcb80 1 client.26692 shutdown 2024-05-08T22:16:22.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.193+0000 7f51110bcb80 1 -- 172.21.15.131:0/2321814193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567efe7f7a0 msgr2=0x5567efefa760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:22.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.193+0000 7f51110bcb80 1 --2- 172.21.15.131:0/2321814193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567efe7f7a0 0x5567efefa760 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f50f0000f80 tx=0x7f50f0004b00 comp rx=0 tx=0).stop 2024-05-08T22:16:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.193+0000 7f51110bcb80 1 -- 172.21.15.131:0/2321814193 shutdown_connections 2024-05-08T22:16:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.193+0000 7f51110bcb80 1 --2- 172.21.15.131:0/2321814193 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5567efefc420 0x5567efefe840 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.193+0000 7f51110bcb80 1 --2- 172.21.15.131:0/2321814193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567efe7f7a0 0x5567efefa760 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.193+0000 7f51110bcb80 1 --2- 172.21.15.131:0/2321814193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567efe79910 0x5567efe89480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.193+0000 7f50d7e00700 0 client.26692 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.193+0000 7f51110bcb80 1 --2- 172.21.15.131:0/2321814193 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567efe7ee80 0x5567efe899c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.193+0000 7f51110bcb80 1 -- 172.21.15.131:0/2321814193 >> 172.21.15.131:0/2321814193 conn(0x5567efdd7290 msgr2=0x5567efe7fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.194+0000 7f51110bcb80 1 -- 172.21.15.131:0/2321814193 shutdown_connections 2024-05-08T22:16:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.194+0000 7f51110bcb80 1 -- 172.21.15.131:0/2321814193 wait complete. 2024-05-08T22:16:22.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.196+0000 7f51110bcb80 20 client.26692 trim_cache size 0 max 16384 2024-05-08T22:16:22.197 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:16:22.197 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-08T22:16:22.197 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "de01f542-81ad-48e9-933e-74e5ea6cd93d", 2024-05-08T22:16:22.197 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-08T22:16:22.197 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:16:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.197+0000 7f50fd600700 1 -- 172.21.15.131:0/3999523489 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f50dc0a1390 msgr2=0x7f50dc0a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.197+0000 7f50fd600700 1 --2- 172.21.15.131:0/3999523489 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f50dc0a1390 0x7f50dc0a3850 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f50f819ab90 tx=0x7f50e4009190 comp rx=0 tx=0).stop 2024-05-08T22:16:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.197+0000 7f50fd600700 1 -- 172.21.15.131:0/3999523489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50f80682e0 msgr2=0x7f50f8199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.197+0000 7f50fd600700 1 --2- 172.21.15.131:0/3999523489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50f80682e0 0x7f50f8199640 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f50e800ec10 tx=0x7f50e800ef20 comp rx=0 tx=0).stop 2024-05-08T22:16:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.197+0000 7f50fd600700 1 -- 172.21.15.131:0/3999523489 shutdown_connections 2024-05-08T22:16:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.197+0000 7f50fd600700 1 --2- 172.21.15.131:0/3999523489 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f50dc0a1390 0x7f50dc0a3850 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.197+0000 7f50fd600700 1 --2- 172.21.15.131:0/3999523489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50f819df20 0x7f50f819e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.197+0000 7f50fd600700 1 --2- 172.21.15.131:0/3999523489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50f80682e0 0x7f50f8199640 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.197+0000 7f50fd600700 1 --2- 172.21.15.131:0/3999523489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50f8106290 0x7f50f8199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.197+0000 7f50fd600700 1 -- 172.21.15.131:0/3999523489 >> 172.21.15.131:0/3999523489 conn(0x7f50f8076f40 msgr2=0x7f50f80fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:22.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.197+0000 7f50fd600700 1 -- 172.21.15.131:0/3999523489 shutdown_connections 2024-05-08T22:16:22.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.197+0000 7f50fd600700 1 -- 172.21.15.131:0/3999523489 wait complete. 2024-05-08T22:16:22.241 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:16:22.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.608+0000 7fda05e00700 1 -- 172.21.15.131:0/324206712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda00108490 msgr2=0x7fda00110010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:22.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.608+0000 7fda05e00700 1 --2- 172.21.15.131:0/324206712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda00108490 0x7fda00110010 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800b0a0 tx=0x7fd9f800b3b0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.608+0000 7fda05e00700 1 -- 172.21.15.131:0/324206712 shutdown_connections 2024-05-08T22:16:22.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.608+0000 7fda05e00700 1 --2- 172.21.15.131:0/324206712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda00108490 0x7fda00110010 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.608+0000 7fda05e00700 1 --2- 172.21.15.131:0/324206712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda00103370 0x7fda00103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.608+0000 7fda05e00700 1 --2- 172.21.15.131:0/324206712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda00103d20 0x7fda00107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.608+0000 7fda05e00700 1 -- 172.21.15.131:0/324206712 >> 172.21.15.131:0/324206712 conn(0x7fda000febb0 msgr2=0x7fda00100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:22.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.608+0000 7fda05e00700 1 -- 172.21.15.131:0/324206712 shutdown_connections 2024-05-08T22:16:22.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.608+0000 7fda05e00700 1 -- 172.21.15.131:0/324206712 wait complete. 2024-05-08T22:16:22.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.610+0000 7fda05e00700 1 Processor -- start 2024-05-08T22:16:22.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.610+0000 7fda05e00700 1 -- start start 2024-05-08T22:16:22.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.611+0000 7fda05e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda00103370 0x7fda000754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.611+0000 7fda05e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda00108490 0x7fda000759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.611+0000 7fd9fea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda00103370 0x7fda000754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.611+0000 7fd9fea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda00103370 0x7fda000754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:34886/0 (socket says 172.21.15.131:34886) 2024-05-08T22:16:22.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.611+0000 7fda05e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda00075f30 0x7fda001a9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.611+0000 7fd9fea00700 1 -- 172.21.15.131:0/3365840684 learned_addr learned my addr 172.21.15.131:0/3365840684 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:22.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.611+0000 7fda05e00700 1 -- 172.21.15.131:0/3365840684 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda000796a0 con 0x7fda00108490 2024-05-08T22:16:22.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.611+0000 7fda05e00700 1 -- 172.21.15.131:0/3365840684 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda001aa280 con 0x7fda00103370 2024-05-08T22:16:22.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.611+0000 7fda05e00700 1 -- 172.21.15.131:0/3365840684 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda001aa440 con 0x7fda00075f30 2024-05-08T22:16:22.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.611+0000 7fd9fe000700 1 --2- 172.21.15.131:0/3365840684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda00108490 0x7fda000759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.612+0000 7fd9ff400700 1 --2- 172.21.15.131:0/3365840684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda00075f30 0x7fda001a9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.612+0000 7fd9fea00700 1 -- 172.21.15.131:0/3365840684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda00075f30 msgr2=0x7fda001a9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:22.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.612+0000 7fd9fea00700 1 --2- 172.21.15.131:0/3365840684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda00075f30 0x7fda001a9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.612+0000 7fd9fea00700 1 -- 172.21.15.131:0/3365840684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda00108490 msgr2=0x7fda000759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:22.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.612+0000 7fd9fea00700 1 --2- 172.21.15.131:0/3365840684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda00108490 0x7fda000759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.612+0000 7fd9fea00700 1 -- 172.21.15.131:0/3365840684 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f8009d00 con 0x7fda00103370 2024-05-08T22:16:22.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.612+0000 7fd9fe000700 1 --2- 172.21.15.131:0/3365840684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda00108490 0x7fda000759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:16:22.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.612+0000 7fd9fea00700 1 --2- 172.21.15.131:0/3365840684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda00103370 0x7fda000754b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd9f000d950 tx=0x7fd9f000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:22.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.612+0000 7fda04a00700 1 -- 172.21.15.131:0/3365840684 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0009940 con 0x7fda00103370 2024-05-08T22:16:22.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.612+0000 7fda05e00700 1 -- 172.21.15.131:0/3365840684 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda001aa720 con 0x7fda00103370 2024-05-08T22:16:22.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.612+0000 7fda04a00700 1 -- 172.21.15.131:0/3365840684 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9f0010460 con 0x7fda00103370 2024-05-08T22:16:22.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.612+0000 7fda04a00700 1 -- 172.21.15.131:0/3365840684 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f000f620 con 0x7fda00103370 2024-05-08T22:16:22.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.613+0000 7fda05e00700 1 -- 172.21.15.131:0/3365840684 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda001aac30 con 0x7fda00103370 2024-05-08T22:16:22.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.616+0000 7fda04a00700 1 -- 172.21.15.131:0/3365840684 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fd9f00108d0 con 0x7fda00103370 2024-05-08T22:16:22.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.618+0000 7fda04a00700 1 --2- 172.21.15.131:0/3365840684 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fd9e40a1290 0x7fd9e40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.619+0000 7fda04a00700 1 -- 172.21.15.131:0/3365840684 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fd9f00ccff0 con 0x7fda00103370 2024-05-08T22:16:22.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.619+0000 7fd9fe000700 1 --2- 172.21.15.131:0/3365840684 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fd9e40a1290 0x7fd9e40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.620+0000 7fda19a02b80 1 Processor -- start 2024-05-08T22:16:22.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.620+0000 7fda19a02b80 1 -- start start 2024-05-08T22:16:22.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.620+0000 7fda19a02b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256789e00 0x56225678a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.620+0000 7fd9fe000700 1 --2- 172.21.15.131:0/3365840684 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fd9e40a1290 0x7fd9e40a3750 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fda00076be0 tx=0x7fd9ec009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:22.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.620+0000 7fda19a02b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56225678a720 0x562256784350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.620+0000 7fda19a02b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562256784890 0x562256784cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.620+0000 7fda19a02b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622566db150 con 0x56225678a720 2024-05-08T22:16:22.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.620+0000 7fda19a02b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622566d8b10 con 0x562256784890 2024-05-08T22:16:22.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.620+0000 7fda19a02b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622566d4400 con 0x562256789e00 2024-05-08T22:16:22.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.621+0000 7fd9fea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256789e00 0x56225678a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.621+0000 7fd9fea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256789e00 0x56225678a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43742/0 (socket says 172.21.15.131:43742) 2024-05-08T22:16:22.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.621+0000 7fd9fea00700 1 -- 172.21.15.131:0/1936623894 learned_addr learned my addr 172.21.15.131:0/1936623894 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:22.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.621+0000 7fd9ff400700 1 --2- 172.21.15.131:0/1936623894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562256784890 0x562256784cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.621+0000 7fd9fea00700 1 -- 172.21.15.131:0/1936623894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562256784890 msgr2=0x562256784cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:22.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.621+0000 7fd9fea00700 1 --2- 172.21.15.131:0/1936623894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562256784890 0x562256784cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.621+0000 7fd9fea00700 1 -- 172.21.15.131:0/1936623894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56225678a720 msgr2=0x562256784350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:22.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.621+0000 7fd9fea00700 1 --2- 172.21.15.131:0/1936623894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56225678a720 0x562256784350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.621+0000 7fd9fea00700 1 -- 172.21.15.131:0/1936623894 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622565553b0 con 0x562256789e00 2024-05-08T22:16:22.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.621+0000 7fd9fea00700 1 --2- 172.21.15.131:0/1936623894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256789e00 0x56225678a1e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd9f00961a0 tx=0x7fd9f009a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c7338f9ef772792 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:22.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.621+0000 7fd9ebe00700 1 -- 172.21.15.131:0/1936623894 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f00a2120 con 0x562256789e00 2024-05-08T22:16:22.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.621+0000 7fd9ebe00700 1 -- 172.21.15.131:0/1936623894 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9f00963b0 con 0x562256789e00 2024-05-08T22:16:22.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.622+0000 7fd9ebe00700 1 -- 172.21.15.131:0/1936623894 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f00a1c90 con 0x562256789e00 2024-05-08T22:16:22.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.622+0000 7fda19a02b80 1 -- 172.21.15.131:0/1936623894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256789e00 msgr2=0x56225678a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:22.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.622+0000 7fda19a02b80 1 --2- 172.21.15.131:0/1936623894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256789e00 0x56225678a1e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd9f00961a0 tx=0x7fd9f009a420 comp rx=0 tx=0).stop 2024-05-08T22:16:22.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.622+0000 7fda19a02b80 1 -- 172.21.15.131:0/1936623894 shutdown_connections 2024-05-08T22:16:22.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.623+0000 7fda19a02b80 1 --2- 172.21.15.131:0/1936623894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562256784890 0x562256784cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.623+0000 7fda19a02b80 1 --2- 172.21.15.131:0/1936623894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256789e00 0x56225678a1e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.623+0000 7fda19a02b80 1 --2- 172.21.15.131:0/1936623894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56225678a720 0x562256784350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.623+0000 7fda19a02b80 1 -- 172.21.15.131:0/1936623894 >> 172.21.15.131:0/1936623894 conn(0x5622566e2210 msgr2=0x56225677edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:22.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.623+0000 7fda19a02b80 1 -- 172.21.15.131:0/1936623894 shutdown_connections 2024-05-08T22:16:22.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.623+0000 7fda19a02b80 1 -- 172.21.15.131:0/1936623894 wait complete. 2024-05-08T22:16:22.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.623+0000 7fda19a02b80 1 -- start start 2024-05-08T22:16:22.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.624+0000 7fda19a02b80 1 Processor -- start 2024-05-08T22:16:22.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.625+0000 7fda19a02b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256784890 0x56225678db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.625+0000 7fda19a02b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562256789e00 0x56225678e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.625+0000 7fda19a02b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56225678a720 0x56225678e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.625+0000 7fda19a02b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562256670560 con 0x56225678a720 2024-05-08T22:16:22.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.625+0000 7fda19a02b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562256670200 con 0x562256789e00 2024-05-08T22:16:22.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.625+0000 7fda19a02b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622565576a0 con 0x562256784890 2024-05-08T22:16:22.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.625+0000 7fda19a02b80 10 client.0 _ll_register_callbacks cb 0x562256679b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:16:22.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.625+0000 7fd9fe000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562256789e00 0x56225678e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.625+0000 7fd9fe000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562256789e00 0x56225678e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:34890/0 (socket says 172.21.15.131:34890) 2024-05-08T22:16:22.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.625+0000 7fd9fe000700 1 -- 172.21.15.131:0/4000414023 learned_addr learned my addr 172.21.15.131:0/4000414023 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:22.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.625+0000 7fd9ff400700 1 --2- 172.21.15.131:0/4000414023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56225678a720 0x56225678e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.625+0000 7fd9fea00700 1 --2- 172.21.15.131:0/4000414023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256784890 0x56225678db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.626+0000 7fd9fea00700 1 -- 172.21.15.131:0/4000414023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562256789e00 msgr2=0x56225678e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:22.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.626+0000 7fd9fea00700 1 --2- 172.21.15.131:0/4000414023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562256789e00 0x56225678e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.626+0000 7fd9fea00700 1 -- 172.21.15.131:0/4000414023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56225678a720 msgr2=0x56225678e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:22.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.626+0000 7fd9fea00700 1 --2- 172.21.15.131:0/4000414023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56225678a720 0x56225678e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.626+0000 7fd9fea00700 1 -- 172.21.15.131:0/4000414023 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda001aac30 con 0x562256784890 2024-05-08T22:16:22.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.626+0000 7fd9fe000700 1 --2- 172.21.15.131:0/4000414023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562256789e00 0x56225678e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:16:22.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.626+0000 7fd9fea00700 1 --2- 172.21.15.131:0/4000414023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256784890 0x56225678db50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd9f00a1bf0 tx=0x7fd9f009a240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fa8b3471a81dbb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:22.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.626+0000 7fd9e8c00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:16:22.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.626+0000 7fd9e8c00700 1 -- 172.21.15.131:0/4000414023 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f00a2120 con 0x562256784890 2024-05-08T22:16:22.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.626+0000 7fda19a02b80 1 -- 172.21.15.131:0/4000414023 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56225662d270 con 0x562256784890 2024-05-08T22:16:22.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.627+0000 7fd9e8c00700 1 -- 172.21.15.131:0/4000414023 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fd9f00993a0 con 0x562256784890 2024-05-08T22:16:22.630 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:22 smithi131 ceph-mon[37260]: pgmap v280: 129 pgs: 18 active+clean+snaptrim, 55 active+clean, 56 active+clean+snaptrim_wait; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 600 op/s 2024-05-08T22:16:22.630 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:22 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:16:22.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.632+0000 7fd9e8c00700 1 -- 172.21.15.131:0/4000414023 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9f00a1c90 con 0x562256784890 2024-05-08T22:16:22.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.632+0000 7fda19a02b80 10 client.26704 fetch_fsmap learned FSMap version 29 2024-05-08T22:16:22.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.632+0000 7fda19a02b80 1 -- 172.21.15.131:0/4000414023 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562256501b60 con 0x562256784890 2024-05-08T22:16:22.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.632+0000 7fd9e8c00700 1 -- 172.21.15.131:0/4000414023 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56225662d270 con 0x562256784890 2024-05-08T22:16:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.632+0000 7fd9e8c00700 1 -- 172.21.15.131:0/4000414023 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fd9f0098c00 con 0x562256784890 2024-05-08T22:16:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.632+0000 7fda19a02b80 10 client.26704 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:16:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.632+0000 7fda19a02b80 10 client.26704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:16:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.632+0000 7fda19a02b80 20 client.26704 populate_metadata read hostname 'smithi131' 2024-05-08T22:16:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.632+0000 7fda19a02b80 1 --2- 172.21.15.131:0/4000414023 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x56225679dee0 0x5622567a0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.633+0000 7fda19a02b80 4 client.26704 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:16:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.633+0000 7fda19a02b80 1 -- 172.21.15.131:0/4000414023 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562256753810 con 0x56225679dee0 2024-05-08T22:16:22.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.633+0000 7fd9fe000700 1 --2- 172.21.15.131:0/4000414023 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x56225679dee0 0x5622567a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.634+0000 7fd9e8c00700 10 client.26704 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.634+0000 7fd9fe000700 1 --2- 172.21.15.131:0/4000414023 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x56225679dee0 0x5622567a0300 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3aff22259c618eb server_cookie=74bde34e6a36a02f in_seq=0 out_seq=0 2024-05-08T22:16:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.634+0000 7fd9e8c00700 1 -- 172.21.15.131:0/4000414023 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562256753810 con 0x56225679dee0 2024-05-08T22:16:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.634+0000 7fd9e8c00700 10 client.26704 handle_command_reply: tid=0 2024-05-08T22:16:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.634+0000 7fda19a02b80 1 client.26704 shutdown 2024-05-08T22:16:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.634+0000 7fda19a02b80 1 -- 172.21.15.131:0/4000414023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256784890 msgr2=0x56225678db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.634+0000 7fda19a02b80 1 --2- 172.21.15.131:0/4000414023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256784890 0x56225678db50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd9f00a1bf0 tx=0x7fd9f009a240 comp rx=0 tx=0).stop 2024-05-08T22:16:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.635+0000 7fda19a02b80 1 -- 172.21.15.131:0/4000414023 shutdown_connections 2024-05-08T22:16:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.635+0000 7fda19a02b80 1 --2- 172.21.15.131:0/4000414023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56225678a720 0x56225678e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.635+0000 7fda19a02b80 1 --2- 172.21.15.131:0/4000414023 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x56225679dee0 0x5622567a0300 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.635+0000 7fda19a02b80 1 --2- 172.21.15.131:0/4000414023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256784890 0x56225678db50 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.635+0000 7fd9e8c00700 0 client.26704 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:22.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.635+0000 7fda19a02b80 1 --2- 172.21.15.131:0/4000414023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562256789e00 0x56225678e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.635+0000 7fda19a02b80 1 -- 172.21.15.131:0/4000414023 >> 172.21.15.131:0/4000414023 conn(0x5622566e2210 msgr2=0x56225677e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:22.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.635+0000 7fda19a02b80 1 -- 172.21.15.131:0/4000414023 shutdown_connections 2024-05-08T22:16:22.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.636+0000 7fda19a02b80 1 -- 172.21.15.131:0/4000414023 wait complete. 2024-05-08T22:16:22.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.636+0000 7fda19a02b80 20 client.26704 trim_cache size 0 max 16384 2024-05-08T22:16:22.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.656+0000 7fda19a02b80 1 Processor -- start 2024-05-08T22:16:22.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.656+0000 7fda19a02b80 1 -- start start 2024-05-08T22:16:22.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.657+0000 7fda19a02b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256784890 0x562256784350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.657+0000 7fda19a02b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562256789e00 0x562256799c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.657+0000 7fda19a02b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56225678a720 0x56225677ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.657+0000 7fda19a02b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622567a5510 con 0x562256789e00 2024-05-08T22:16:22.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.657+0000 7fda19a02b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562256779800 con 0x56225678a720 2024-05-08T22:16:22.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.657+0000 7fda19a02b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56225678bb10 con 0x562256784890 2024-05-08T22:16:22.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.657+0000 7fd9fea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256784890 0x562256784350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.657+0000 7fd9ff400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56225678a720 0x56225677ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.657+0000 7fd9ff400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56225678a720 0x56225677ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:34894/0 (socket says 172.21.15.131:34894) 2024-05-08T22:16:22.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.657+0000 7fd9fea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256784890 0x562256784350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43752/0 (socket says 172.21.15.131:43752) 2024-05-08T22:16:22.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.657+0000 7fd9ff400700 1 -- 172.21.15.131:0/3736670464 learned_addr learned my addr 172.21.15.131:0/3736670464 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:22.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.657+0000 7fd9fe000700 1 --2- 172.21.15.131:0/3736670464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562256789e00 0x562256799c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.657+0000 7fd9fea00700 1 -- 172.21.15.131:0/3736670464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56225678a720 msgr2=0x56225677ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:22.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.657+0000 7fd9fea00700 1 --2- 172.21.15.131:0/3736670464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56225678a720 0x56225677ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.657+0000 7fd9fea00700 1 -- 172.21.15.131:0/3736670464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562256789e00 msgr2=0x562256799c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:22.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.657+0000 7fd9fea00700 1 --2- 172.21.15.131:0/3736670464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562256789e00 0x562256799c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.657+0000 7fd9fea00700 1 -- 172.21.15.131:0/3736670464 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56225669a560 con 0x562256784890 2024-05-08T22:16:22.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.657+0000 7fd9ff400700 1 --2- 172.21.15.131:0/3736670464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56225678a720 0x56225677ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:16:22.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.658+0000 7fd9fea00700 1 --2- 172.21.15.131:0/3736670464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256784890 0x562256784350 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd9f009dc90 tx=0x7fd9f0005910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f736f50d954c8e1e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:22.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.658+0000 7fd9f5200700 1 -- 172.21.15.131:0/3736670464 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f00a2120 con 0x562256784890 2024-05-08T22:16:22.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.658+0000 7fd9f5200700 1 -- 172.21.15.131:0/3736670464 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9f0099ca0 con 0x562256784890 2024-05-08T22:16:22.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.658+0000 7fd9f5200700 1 -- 172.21.15.131:0/3736670464 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f00a1c90 con 0x562256784890 2024-05-08T22:16:22.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.659+0000 7fda19a02b80 1 -- 172.21.15.131:0/3736670464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256784890 msgr2=0x562256784350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:22.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.659+0000 7fda19a02b80 1 --2- 172.21.15.131:0/3736670464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256784890 0x562256784350 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd9f009dc90 tx=0x7fd9f0005910 comp rx=0 tx=0).stop 2024-05-08T22:16:22.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.659+0000 7fda19a02b80 1 -- 172.21.15.131:0/3736670464 shutdown_connections 2024-05-08T22:16:22.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.659+0000 7fda19a02b80 1 --2- 172.21.15.131:0/3736670464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56225678a720 0x56225677ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.659+0000 7fda19a02b80 1 --2- 172.21.15.131:0/3736670464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256784890 0x562256784350 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.659+0000 7fda19a02b80 1 --2- 172.21.15.131:0/3736670464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562256789e00 0x562256799c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.659+0000 7fda19a02b80 1 -- 172.21.15.131:0/3736670464 >> 172.21.15.131:0/3736670464 conn(0x5622566e2210 msgr2=0x562256784c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.659+0000 7fda19a02b80 1 -- 172.21.15.131:0/3736670464 shutdown_connections 2024-05-08T22:16:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.659+0000 7fda19a02b80 1 -- 172.21.15.131:0/3736670464 wait complete. 2024-05-08T22:16:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.659+0000 7fda19a02b80 1 -- start start 2024-05-08T22:16:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.660+0000 7fda19a02b80 1 Processor -- start 2024-05-08T22:16:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.660+0000 7fda19a02b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256784890 0x562256795360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.660+0000 7fda19a02b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562256789e00 0x5622567859a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.660+0000 7fda19a02b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56225678a720 0x562256785ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.660+0000 7fda19a02b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562256786420 con 0x56225678a720 2024-05-08T22:16:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.660+0000 7fda19a02b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562256786590 con 0x562256789e00 2024-05-08T22:16:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.660+0000 7fda19a02b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562256786750 con 0x562256784890 2024-05-08T22:16:22.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.660+0000 7fda19a02b80 10 client.0 _ll_register_callbacks cb 0x562256679b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:16:22.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.661+0000 7fd9fea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256784890 0x562256795360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.661+0000 7fd9fea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256784890 0x562256795360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43754/0 (socket says 172.21.15.131:43754) 2024-05-08T22:16:22.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.661+0000 7fd9fea00700 1 -- 172.21.15.131:0/1236361669 learned_addr learned my addr 172.21.15.131:0/1236361669 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:22.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.661+0000 7fd9fe000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562256789e00 0x5622567859a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.661+0000 7fd9ff400700 1 --2- 172.21.15.131:0/1236361669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56225678a720 0x562256785ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.661+0000 7fd9fea00700 1 -- 172.21.15.131:0/1236361669 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562256789e00 msgr2=0x5622567859a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:22.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.661+0000 7fd9fea00700 1 --2- 172.21.15.131:0/1236361669 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562256789e00 0x5622567859a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.661+0000 7fd9fea00700 1 -- 172.21.15.131:0/1236361669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56225678a720 msgr2=0x562256785ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:22.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.661+0000 7fd9fea00700 1 --2- 172.21.15.131:0/1236361669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56225678a720 0x562256785ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.661+0000 7fd9fea00700 1 -- 172.21.15.131:0/1236361669 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda00110880 con 0x562256784890 2024-05-08T22:16:22.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.661+0000 7fd9fea00700 1 --2- 172.21.15.131:0/1236361669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256784890 0x562256795360 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd9f00961a0 tx=0x7fd9f00adf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3834e79251cba4cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:22.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.661+0000 7fd9e8c00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:16:22.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.661+0000 7fd9e8c00700 1 -- 172.21.15.131:0/1236361669 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f00add70 con 0x562256784890 2024-05-08T22:16:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.661+0000 7fd9e8c00700 1 -- 172.21.15.131:0/1236361669 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fd9f0064050 con 0x562256784890 2024-05-08T22:16:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.661+0000 7fda19a02b80 1 -- 172.21.15.131:0/1236361669 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9f00963b0 con 0x562256784890 2024-05-08T22:16:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.662+0000 7fd9e8c00700 1 -- 172.21.15.131:0/1236361669 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9f00a2120 con 0x562256784890 2024-05-08T22:16:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.662+0000 7fda19a02b80 10 client.26710 fetch_fsmap learned FSMap version 29 2024-05-08T22:16:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.662+0000 7fda19a02b80 1 -- 172.21.15.131:0/1236361669 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562256501b60 con 0x562256784890 2024-05-08T22:16:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.662+0000 7fd9e8c00700 1 -- 172.21.15.131:0/1236361669 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9f00a8020 con 0x562256784890 2024-05-08T22:16:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.662+0000 7fd9e8c00700 1 -- 172.21.15.131:0/1236361669 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fd9f007e070 con 0x562256784890 2024-05-08T22:16:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.662+0000 7fda19a02b80 10 client.26710 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:16:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.662+0000 7fda19a02b80 10 client.26710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:16:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.662+0000 7fda19a02b80 20 client.26710 populate_metadata read hostname 'smithi131' 2024-05-08T22:16:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.662+0000 7fda19a02b80 1 --2- 172.21.15.131:0/1236361669 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5622567f44a0 0x5622567f6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.662+0000 7fda19a02b80 4 client.26710 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:16:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.662+0000 7fda19a02b80 1 -- 172.21.15.131:0/1236361669 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5622566ccec0 con 0x5622567f44a0 2024-05-08T22:16:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.662+0000 7fd9fe000700 1 --2- 172.21.15.131:0/1236361669 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5622567f44a0 0x5622567f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.663+0000 7fd9fe000700 1 --2- 172.21.15.131:0/1236361669 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5622567f44a0 0x5622567f6890 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1c48d12f7054a9f server_cookie=40cd0b44280a56b9 in_seq=0 out_seq=0 2024-05-08T22:16:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.663+0000 7fd9e8c00700 10 client.26710 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.663+0000 7fd9e8c00700 1 -- 172.21.15.131:0/1236361669 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x5622566ccec0 con 0x5622567f44a0 2024-05-08T22:16:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.663+0000 7fd9e8c00700 10 client.26710 handle_command_reply: tid=0 2024-05-08T22:16:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.663+0000 7fda19a02b80 1 client.26710 shutdown 2024-05-08T22:16:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.663+0000 7fda19a02b80 1 -- 172.21.15.131:0/1236361669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256784890 msgr2=0x562256795360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.663+0000 7fda19a02b80 1 --2- 172.21.15.131:0/1236361669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256784890 0x562256795360 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd9f00961a0 tx=0x7fd9f00adf90 comp rx=0 tx=0).stop 2024-05-08T22:16:22.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.663+0000 7fda19a02b80 1 -- 172.21.15.131:0/1236361669 shutdown_connections 2024-05-08T22:16:22.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.663+0000 7fda19a02b80 1 --2- 172.21.15.131:0/1236361669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56225678a720 0x562256785ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.663+0000 7fda19a02b80 1 --2- 172.21.15.131:0/1236361669 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x5622567f44a0 0x5622567f6890 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.663+0000 7fda19a02b80 1 --2- 172.21.15.131:0/1236361669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562256784890 0x562256795360 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.663+0000 7fda19a02b80 1 --2- 172.21.15.131:0/1236361669 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562256789e00 0x5622567859a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.663+0000 7fda19a02b80 1 -- 172.21.15.131:0/1236361669 >> 172.21.15.131:0/1236361669 conn(0x5622566e2210 msgr2=0x56225678c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:22.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.663+0000 7fd9e8c00700 0 client.26710 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:22.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.664+0000 7fda19a02b80 1 -- 172.21.15.131:0/1236361669 shutdown_connections 2024-05-08T22:16:22.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.664+0000 7fda19a02b80 1 -- 172.21.15.131:0/1236361669 wait complete. 2024-05-08T22:16:22.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.664+0000 7fda19a02b80 20 client.26710 trim_cache size 0 max 16384 2024-05-08T22:16:22.670 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:16:22.670 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "scrub active (8025 inodes in the stack)", 2024-05-08T22:16:22.670 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-08T22:16:22.670 INFO:teuthology.orchestra.run.smithi131.stdout: "de01f542-81ad-48e9-933e-74e5ea6cd93d": { 2024-05-08T22:16:22.670 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-08T22:16:22.671 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "de01f542-81ad-48e9-933e-74e5ea6cd93d", 2024-05-08T22:16:22.671 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-08T22:16:22.671 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:16:22.671 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:16:22.671 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:16:22.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.664+0000 7fda05e00700 1 -- 172.21.15.131:0/3365840684 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fd9e40a1290 msgr2=0x7fd9e40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:22.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.664+0000 7fda05e00700 1 --2- 172.21.15.131:0/3365840684 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fd9e40a1290 0x7fd9e40a3750 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fda00076be0 tx=0x7fd9ec009260 comp rx=0 tx=0).stop 2024-05-08T22:16:22.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.664+0000 7fda05e00700 1 -- 172.21.15.131:0/3365840684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda00103370 msgr2=0x7fda000754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:22.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.664+0000 7fda05e00700 1 --2- 172.21.15.131:0/3365840684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda00103370 0x7fda000754b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd9f000d950 tx=0x7fd9f000dc60 comp rx=0 tx=0).stop 2024-05-08T22:16:22.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.665+0000 7fda05e00700 1 -- 172.21.15.131:0/3365840684 shutdown_connections 2024-05-08T22:16:22.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.665+0000 7fda05e00700 1 --2- 172.21.15.131:0/3365840684 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fd9e40a1290 0x7fd9e40a3750 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.665+0000 7fda05e00700 1 --2- 172.21.15.131:0/3365840684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda00075f30 0x7fda001a9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.665+0000 7fda05e00700 1 --2- 172.21.15.131:0/3365840684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda00103370 0x7fda000754b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.665+0000 7fda05e00700 1 --2- 172.21.15.131:0/3365840684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda00108490 0x7fda000759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:22.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.665+0000 7fda05e00700 1 -- 172.21.15.131:0/3365840684 >> 172.21.15.131:0/3365840684 conn(0x7fda000febb0 msgr2=0x7fda00100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:22.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.665+0000 7fda05e00700 1 -- 172.21.15.131:0/3365840684 shutdown_connections 2024-05-08T22:16:22.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:22.665+0000 7fda05e00700 1 -- 172.21.15.131:0/3365840684 wait complete. 2024-05-08T22:16:22.684 INFO:tasks.cephfs.filesystem:scrub status for tag:de01f542-81ad-48e9-933e-74e5ea6cd93d - {'path': '/', 'tag': 'de01f542-81ad-48e9-933e-74e5ea6cd93d', 'options': 'recursive,force'} 2024-05-08T22:16:23.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:22 smithi157 ceph-mon[77746]: pgmap v280: 129 pgs: 18 active+clean+snaptrim, 55 active+clean, 56 active+clean+snaptrim_wait; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 600 op/s 2024-05-08T22:16:23.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:22 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:16:23.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:22 smithi132 ceph-mon[78180]: pgmap v280: 129 pgs: 18 active+clean+snaptrim, 55 active+clean, 56 active+clean+snaptrim_wait; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 600 op/s 2024-05-08T22:16:23.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:22 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:16:23.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:23 smithi131 ceph-mon[37260]: scrub queued for path: / 2024-05-08T22:16:23.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:23 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:16:23.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:23 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:16:23.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:23 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:16:23.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:23 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:16:23.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:23 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:16:23.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:23 smithi131 ceph-mon[37260]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:16:24.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:23 smithi157 ceph-mon[77746]: scrub queued for path: / 2024-05-08T22:16:24.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:23 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:16:24.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:23 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:16:24.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:23 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:16:24.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:23 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:16:24.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:23 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:16:24.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:23 smithi157 ceph-mon[77746]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:16:24.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:23 smithi132 ceph-mon[78180]: scrub queued for path: / 2024-05-08T22:16:24.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:23 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:16:24.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:23 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:16:24.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:23 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:16:24.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:23 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:16:24.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:23 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:16:24.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:23 smithi132 ceph-mon[78180]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:16:24.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:24 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:16:24.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:24 smithi131 ceph-mon[37260]: pgmap v281: 129 pgs: 18 active+clean+snaptrim, 55 active+clean, 56 active+clean+snaptrim_wait; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 594 op/s 2024-05-08T22:16:24.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:24 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:16:25.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:24 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:16:25.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:24 smithi157 ceph-mon[77746]: pgmap v281: 129 pgs: 18 active+clean+snaptrim, 55 active+clean, 56 active+clean+snaptrim_wait; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 594 op/s 2024-05-08T22:16:25.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:24 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:16:25.094 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:24 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:16:25.094 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:24 smithi132 ceph-mon[78180]: pgmap v281: 129 pgs: 18 active+clean+snaptrim, 55 active+clean, 56 active+clean+snaptrim_wait; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 594 op/s 2024-05-08T22:16:25.094 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:24 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:16:27.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:26 smithi157 ceph-mon[77746]: pgmap v282: 129 pgs: 13 active+clean+snaptrim, 81 active+clean, 35 active+clean+snaptrim_wait; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 604 op/s 2024-05-08T22:16:27.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:26 smithi132 ceph-mon[78180]: pgmap v282: 129 pgs: 13 active+clean+snaptrim, 81 active+clean, 35 active+clean+snaptrim_wait; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 604 op/s 2024-05-08T22:16:27.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:26 smithi131 ceph-mon[37260]: pgmap v282: 129 pgs: 13 active+clean+snaptrim, 81 active+clean, 35 active+clean+snaptrim_wait; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 604 op/s 2024-05-08T22:16:29.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:28 smithi157 ceph-mon[77746]: pgmap v283: 129 pgs: 8 active+clean+snaptrim, 101 active+clean, 20 active+clean+snaptrim_wait; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 484 op/s 2024-05-08T22:16:29.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:28 smithi132 ceph-mon[78180]: pgmap v283: 129 pgs: 8 active+clean+snaptrim, 101 active+clean, 20 active+clean+snaptrim_wait; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 484 op/s 2024-05-08T22:16:29.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:28 smithi131 ceph-mon[37260]: pgmap v283: 129 pgs: 8 active+clean+snaptrim, 101 active+clean, 20 active+clean+snaptrim_wait; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 484 op/s 2024-05-08T22:16:31.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:30 smithi157 ceph-mon[77746]: pgmap v284: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 846 op/s 2024-05-08T22:16:31.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:30 smithi132 ceph-mon[78180]: pgmap v284: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 846 op/s 2024-05-08T22:16:31.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:30 smithi131 ceph-mon[37260]: pgmap v284: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 846 op/s 2024-05-08T22:16:32.025 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:31 smithi157 ceph-mon[77746]: pgmap v285: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 785 op/s 2024-05-08T22:16:32.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:31 smithi132 ceph-mon[78180]: pgmap v285: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 785 op/s 2024-05-08T22:16:32.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:31 smithi131 ceph-mon[37260]: pgmap v285: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 785 op/s 2024-05-08T22:16:33.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:32 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:16:33.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:32 smithi157 ceph-mon[77746]: osdmap e94: 12 total, 12 up, 12 in 2024-05-08T22:16:33.093 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:32 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:16:33.093 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:32 smithi132 ceph-mon[78180]: osdmap e94: 12 total, 12 up, 12 in 2024-05-08T22:16:33.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:32 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:16:33.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:32 smithi131 ceph-mon[37260]: osdmap e94: 12 total, 12 up, 12 in 2024-05-08T22:16:34.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:33 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:16:34.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:33 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:16:34.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:33 smithi157 ceph-mon[77746]: pgmap v287: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 1.02k op/s 2024-05-08T22:16:34.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:33 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:16:34.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:33 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:16:34.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:33 smithi132 ceph-mon[78180]: pgmap v287: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 1.02k op/s 2024-05-08T22:16:34.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:33 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:16:34.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:33 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:16:34.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:33 smithi131 ceph-mon[37260]: pgmap v287: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 1.02k op/s 2024-05-08T22:16:35.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:34 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:16:35.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:34 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:16:35.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:34 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:16:36.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:35 smithi157 ceph-mon[77746]: pgmap v288: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 1.30k op/s 2024-05-08T22:16:36.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:35 smithi132 ceph-mon[78180]: pgmap v288: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 1.30k op/s 2024-05-08T22:16:36.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:35 smithi131 ceph-mon[37260]: pgmap v288: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 1.30k op/s 2024-05-08T22:16:38.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:38 smithi157 ceph-mon[77746]: pgmap v289: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 1.22k op/s 2024-05-08T22:16:38.847 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:38 smithi132 ceph-mon[78180]: pgmap v289: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 1.22k op/s 2024-05-08T22:16:38.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:38 smithi131 ceph-mon[37260]: pgmap v289: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 1.22k op/s 2024-05-08T22:16:39.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:39 smithi157 ceph-mon[77746]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T22:16:39.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:39 smithi132 ceph-mon[78180]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T22:16:39.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:39 smithi131 ceph-mon[37260]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T22:16:40.779 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:40 smithi157 ceph-mon[77746]: pgmap v290: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 924 op/s 2024-05-08T22:16:40.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:40 smithi132 ceph-mon[78180]: pgmap v290: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 924 op/s 2024-05-08T22:16:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:40 smithi131 ceph-mon[37260]: pgmap v290: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 924 op/s 2024-05-08T22:16:42.779 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:42 smithi157 ceph-mon[77746]: pgmap v291: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 890 op/s 2024-05-08T22:16:42.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:42 smithi132 ceph-mon[78180]: pgmap v291: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 890 op/s 2024-05-08T22:16:42.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:42 smithi131 ceph-mon[37260]: pgmap v291: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 890 op/s 2024-05-08T22:16:44.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:44 smithi157 ceph-mon[77746]: pgmap v292: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 773 op/s 2024-05-08T22:16:44.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:44 smithi132 ceph-mon[78180]: pgmap v292: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 773 op/s 2024-05-08T22:16:44.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:44 smithi131 ceph-mon[37260]: pgmap v292: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 773 op/s 2024-05-08T22:16:46.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:46 smithi157 ceph-mon[77746]: pgmap v293: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 882 op/s 2024-05-08T22:16:46.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:46 smithi132 ceph-mon[78180]: pgmap v293: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 882 op/s 2024-05-08T22:16:46.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:46 smithi131 ceph-mon[37260]: pgmap v293: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 882 op/s 2024-05-08T22:16:48.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:48 smithi157 ceph-mon[77746]: pgmap v294: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 540 op/s 2024-05-08T22:16:48.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:48 smithi132 ceph-mon[78180]: pgmap v294: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 540 op/s 2024-05-08T22:16:48.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:48 smithi131 ceph-mon[37260]: pgmap v294: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 540 op/s 2024-05-08T22:16:50.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:50 smithi157 ceph-mon[77746]: pgmap v295: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.4 MiB/s wr, 736 op/s 2024-05-08T22:16:50.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:50 smithi132 ceph-mon[78180]: pgmap v295: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.4 MiB/s wr, 736 op/s 2024-05-08T22:16:50.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:50 smithi131 ceph-mon[37260]: pgmap v295: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.4 MiB/s wr, 736 op/s 2024-05-08T22:16:52.686 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:16:52.743 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:52 smithi131 ceph-mon[37260]: pgmap v296: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 561 op/s 2024-05-08T22:16:52.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:52 smithi157 ceph-mon[77746]: pgmap v296: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 561 op/s 2024-05-08T22:16:52.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:52 smithi132 ceph-mon[78180]: pgmap v296: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 561 op/s 2024-05-08T22:16:52.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.940+0000 7fba14a00700 1 -- 172.21.15.131:0/3868910871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba080a59d0 msgr2=0x7fba080aba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:52.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.940+0000 7fba14a00700 1 --2- 172.21.15.131:0/3868910871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba080a59d0 0x7fba080aba30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0099c0 tx=0x7fb9fc009cd0 comp rx=0 tx=0).stop 2024-05-08T22:16:52.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.941+0000 7fba14a00700 1 -- 172.21.15.131:0/3868910871 shutdown_connections 2024-05-08T22:16:52.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.941+0000 7fba14a00700 1 --2- 172.21.15.131:0/3868910871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba080ac160 0x7fba080b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:52.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.941+0000 7fba14a00700 1 --2- 172.21.15.131:0/3868910871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba080a5020 0x7fba080a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:52.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.941+0000 7fba14a00700 1 --2- 172.21.15.131:0/3868910871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba080a59d0 0x7fba080aba30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:52.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.941+0000 7fba14a00700 1 -- 172.21.15.131:0/3868910871 >> 172.21.15.131:0/3868910871 conn(0x7fba0801a6a0 msgr2=0x7fba0801aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:52.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.941+0000 7fba14a00700 1 -- 172.21.15.131:0/3868910871 shutdown_connections 2024-05-08T22:16:52.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.941+0000 7fba14a00700 1 -- 172.21.15.131:0/3868910871 wait complete. 2024-05-08T22:16:52.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.943+0000 7fba14a00700 1 Processor -- start 2024-05-08T22:16:52.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.943+0000 7fba14a00700 1 -- start start 2024-05-08T22:16:52.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.944+0000 7fba14a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba080a5020 0x7fba0813f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:52.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.944+0000 7fba14a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba080ac160 0x7fba0813fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:52.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.944+0000 7fba14a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba08143da0 0x7fba08144220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:52.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.944+0000 7fba14a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba08144820 con 0x7fba08143da0 2024-05-08T22:16:52.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.944+0000 7fba14a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba08144990 con 0x7fba080a5020 2024-05-08T22:16:52.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.944+0000 7fba14a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba08144b50 con 0x7fba080ac160 2024-05-08T22:16:52.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.944+0000 7fba0ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba080ac160 0x7fba0813fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:52.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.945+0000 7fba0ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba080ac160 0x7fba0813fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54464/0 (socket says 172.21.15.131:54464) 2024-05-08T22:16:52.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.945+0000 7fba0ea00700 1 -- 172.21.15.131:0/2011299645 learned_addr learned my addr 172.21.15.131:0/2011299645 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:52.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.945+0000 7fba0fe00700 1 --2- 172.21.15.131:0/2011299645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba08143da0 0x7fba08144220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:52.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.945+0000 7fba0ea00700 1 -- 172.21.15.131:0/2011299645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba080a5020 msgr2=0x7fba0813f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:52.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.945+0000 7fba0ea00700 1 --2- 172.21.15.131:0/2011299645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba080a5020 0x7fba0813f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:52.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.945+0000 7fba0ea00700 1 -- 172.21.15.131:0/2011299645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba08143da0 msgr2=0x7fba08144220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:52.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.945+0000 7fba0ea00700 1 --2- 172.21.15.131:0/2011299645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba08143da0 0x7fba08144220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:52.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.945+0000 7fba0ea00700 1 -- 172.21.15.131:0/2011299645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9fc0095f0 con 0x7fba080ac160 2024-05-08T22:16:52.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.947+0000 7fba0ea00700 1 --2- 172.21.15.131:0/2011299645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba080ac160 0x7fba0813fa00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc006010 tx=0x7fb9fc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:52.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.951+0000 7fba03e00700 1 -- 172.21.15.131:0/2011299645 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc023ad0 con 0x7fba080ac160 2024-05-08T22:16:52.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.951+0000 7fba03e00700 1 -- 172.21.15.131:0/2011299645 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9fc027070 con 0x7fba080ac160 2024-05-08T22:16:52.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.951+0000 7fba03e00700 1 -- 172.21.15.131:0/2011299645 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc003a20 con 0x7fba080ac160 2024-05-08T22:16:52.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.952+0000 7fba14a00700 1 -- 172.21.15.131:0/2011299645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba080adbc0 con 0x7fba080ac160 2024-05-08T22:16:52.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.952+0000 7fba14a00700 1 -- 172.21.15.131:0/2011299645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba080ae080 con 0x7fba080ac160 2024-05-08T22:16:52.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.958+0000 7fba03e00700 1 -- 172.21.15.131:0/2011299645 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fb9fc029030 con 0x7fba080ac160 2024-05-08T22:16:52.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.959+0000 7fba29adcb80 1 Processor -- start 2024-05-08T22:16:52.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.959+0000 7fba29adcb80 1 -- start start 2024-05-08T22:16:52.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.960+0000 7fba29adcb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf1f400050 0x55cf1f400430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:52.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.960+0000 7fba29adcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf1f400970 0x55cf1f3fa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:52.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.960+0000 7fba29adcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf1f3fa890 0x55cf1f3facd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:52.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.960+0000 7fba29adcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf1f351150 con 0x55cf1f400970 2024-05-08T22:16:52.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.960+0000 7fba29adcb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf1f34eb10 con 0x55cf1f400050 2024-05-08T22:16:52.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.960+0000 7fba29adcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf1f34a400 con 0x55cf1f3fa890 2024-05-08T22:16:52.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.960+0000 7fba0ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf1f3fa890 0x55cf1f3facd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:52.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.960+0000 7fba0ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf1f3fa890 0x55cf1f3facd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54468/0 (socket says 172.21.15.131:54468) 2024-05-08T22:16:52.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.960+0000 7fba0ea00700 1 -- 172.21.15.131:0/1480790124 learned_addr learned my addr 172.21.15.131:0/1480790124 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:52.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.961+0000 7fba0f400700 1 --2- 172.21.15.131:0/1480790124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf1f400970 0x55cf1f3fa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:52.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.962+0000 7fba0f400700 1 -- 172.21.15.131:0/1480790124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf1f3fa890 msgr2=0x55cf1f3facd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:52.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.962+0000 7fba0f400700 1 --2- 172.21.15.131:0/1480790124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf1f3fa890 0x55cf1f3facd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:52.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.962+0000 7fba0f400700 1 -- 172.21.15.131:0/1480790124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf1f400050 msgr2=0x55cf1f400430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:16:52.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.962+0000 7fba0f400700 1 --2- 172.21.15.131:0/1480790124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf1f400050 0x55cf1f400430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:52.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.962+0000 7fba0f400700 1 -- 172.21.15.131:0/1480790124 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf1f1cb3b0 con 0x55cf1f400970 2024-05-08T22:16:52.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.962+0000 7fba0f400700 1 --2- 172.21.15.131:0/1480790124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf1f400970 0x55cf1f3fa350 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fba040097f0 tx=0x7fba04009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6c42f49d9125d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:52.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.963+0000 7fb9f7e00700 1 -- 172.21.15.131:0/1480790124 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba04033640 con 0x55cf1f400970 2024-05-08T22:16:52.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.963+0000 7fb9f7e00700 1 -- 172.21.15.131:0/1480790124 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba04033c60 con 0x55cf1f400970 2024-05-08T22:16:52.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.963+0000 7fb9f7e00700 1 -- 172.21.15.131:0/1480790124 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba04032d20 con 0x55cf1f400970 2024-05-08T22:16:52.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.961+0000 7fba03e00700 1 --2- 172.21.15.131:0/2011299645 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb9f80a13e0 0x7fb9f80a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:52.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.964+0000 7fba03e00700 1 -- 172.21.15.131:0/2011299645 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fb9fc0d68b0 con 0x7fba080ac160 2024-05-08T22:16:52.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.965+0000 7fba0fe00700 1 --2- 172.21.15.131:0/2011299645 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb9f80a13e0 0x7fb9f80a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:52.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.966+0000 7fba29adcb80 1 -- 172.21.15.131:0/1480790124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf1f400970 msgr2=0x55cf1f3fa350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:52.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.966+0000 7fba29adcb80 1 --2- 172.21.15.131:0/1480790124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf1f400970 0x55cf1f3fa350 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fba040097f0 tx=0x7fba04009b00 comp rx=0 tx=0).stop 2024-05-08T22:16:52.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.966+0000 7fba29adcb80 1 -- 172.21.15.131:0/1480790124 shutdown_connections 2024-05-08T22:16:52.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.966+0000 7fba29adcb80 1 --2- 172.21.15.131:0/1480790124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf1f3fa890 0x55cf1f3facd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:52.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.966+0000 7fba29adcb80 1 --2- 172.21.15.131:0/1480790124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf1f400050 0x55cf1f400430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:52.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.966+0000 7fba29adcb80 1 --2- 172.21.15.131:0/1480790124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf1f400970 0x55cf1f3fa350 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:52.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.966+0000 7fba29adcb80 1 -- 172.21.15.131:0/1480790124 >> 172.21.15.131:0/1480790124 conn(0x55cf1f358210 msgr2=0x55cf1f3f52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:52.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.967+0000 7fba29adcb80 1 -- 172.21.15.131:0/1480790124 shutdown_connections 2024-05-08T22:16:52.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.967+0000 7fba29adcb80 1 -- 172.21.15.131:0/1480790124 wait complete. 2024-05-08T22:16:52.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.967+0000 7fba0fe00700 1 --2- 172.21.15.131:0/2011299645 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb9f80a13e0 0x7fb9f80a38a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fba10064340 tx=0x7fba1004fd00 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:52.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.967+0000 7fba29adcb80 1 -- start start 2024-05-08T22:16:52.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.968+0000 7fba29adcb80 1 Processor -- start 2024-05-08T22:16:52.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.969+0000 7fba29adcb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf1f3fa890 0x55cf1f4068c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:52.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.969+0000 7fba29adcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf1f400050 0x55cf1f403bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:52.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.969+0000 7fba29adcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf1f400970 0x55cf1f4041e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:52.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.969+0000 7fba29adcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf1f2e6560 con 0x55cf1f400050 2024-05-08T22:16:52.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.969+0000 7fba29adcb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf1f2e6200 con 0x55cf1f3fa890 2024-05-08T22:16:52.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.969+0000 7fba29adcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf1f1cd6a0 con 0x55cf1f400970 2024-05-08T22:16:52.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.969+0000 7fba29adcb80 10 client.0 _ll_register_callbacks cb 0x55cf1f2efb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:16:52.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.969+0000 7fba0f400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf1f3fa890 0x55cf1f4068c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:52.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.969+0000 7fba0f400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf1f3fa890 0x55cf1f4068c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:35746/0 (socket says 172.21.15.131:35746) 2024-05-08T22:16:52.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.969+0000 7fba0f400700 1 -- 172.21.15.131:0/3181340776 learned_addr learned my addr 172.21.15.131:0/3181340776 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:52.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.970+0000 7fba0fe00700 1 --2- 172.21.15.131:0/3181340776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf1f400970 0x55cf1f4041e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:52.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.970+0000 7fba0ea00700 1 --2- 172.21.15.131:0/3181340776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf1f400050 0x55cf1f403bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:52.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.971+0000 7fba0f400700 1 -- 172.21.15.131:0/3181340776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf1f400970 msgr2=0x55cf1f4041e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:52.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.971+0000 7fba0f400700 1 --2- 172.21.15.131:0/3181340776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf1f400970 0x55cf1f4041e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:52.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.971+0000 7fba0f400700 1 -- 172.21.15.131:0/3181340776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf1f400050 msgr2=0x55cf1f403bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:52.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.971+0000 7fba0f400700 1 --2- 172.21.15.131:0/3181340776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf1f400050 0x55cf1f403bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:52.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.971+0000 7fba0f400700 1 -- 172.21.15.131:0/3181340776 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba04009420 con 0x55cf1f3fa890 2024-05-08T22:16:52.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.971+0000 7fba0ea00700 1 --2- 172.21.15.131:0/3181340776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf1f400050 0x55cf1f403bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:16:52.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.974+0000 7fba0f400700 1 --2- 172.21.15.131:0/3181340776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf1f3fa890 0x55cf1f4068c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fba04032800 tx=0x7fba04033920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf4051edd287dde0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:52.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.974+0000 7fb9f4c00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:16:52.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.977+0000 7fb9f4c00700 1 -- 172.21.15.131:0/3181340776 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba04033dd0 con 0x55cf1f3fa890 2024-05-08T22:16:52.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.977+0000 7fb9f4c00700 1 -- 172.21.15.131:0/3181340776 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fba04032bb0 con 0x55cf1f3fa890 2024-05-08T22:16:52.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.978+0000 7fba29adcb80 1 -- 172.21.15.131:0/3181340776 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cf1f2a3270 con 0x55cf1f3fa890 2024-05-08T22:16:52.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.978+0000 7fb9f4c00700 1 -- 172.21.15.131:0/3181340776 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba04032d70 con 0x55cf1f3fa890 2024-05-08T22:16:52.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.980+0000 7fba29adcb80 10 client.26630 fetch_fsmap learned FSMap version 30 2024-05-08T22:16:52.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.980+0000 7fba29adcb80 1 -- 172.21.15.131:0/3181340776 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55cf1f177b60 con 0x55cf1f3fa890 2024-05-08T22:16:52.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.980+0000 7fb9f4c00700 1 -- 172.21.15.131:0/3181340776 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cf1f2a3270 con 0x55cf1f3fa890 2024-05-08T22:16:52.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.980+0000 7fb9f4c00700 1 -- 172.21.15.131:0/3181340776 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fba0404c070 con 0x55cf1f3fa890 2024-05-08T22:16:52.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.980+0000 7fba29adcb80 10 client.26630 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:16:52.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.980+0000 7fba29adcb80 10 client.26630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:16:52.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.981+0000 7fba29adcb80 20 client.26630 populate_metadata read hostname 'smithi131' 2024-05-08T22:16:52.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.981+0000 7fba29adcb80 1 --2- 172.21.15.131:0/3181340776 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55cf1f413f50 0x55cf1f416370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:52.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.981+0000 7fba29adcb80 4 client.26630 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:16:52.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.981+0000 7fba29adcb80 1 -- 172.21.15.131:0/3181340776 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf1f3c9810 con 0x55cf1f413f50 2024-05-08T22:16:52.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.985+0000 7fba0ea00700 1 --2- 172.21.15.131:0/3181340776 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55cf1f413f50 0x55cf1f416370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:52.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.989+0000 7fb9f4c00700 10 client.26630 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:52.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.989+0000 7fba0ea00700 1 --2- 172.21.15.131:0/3181340776 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55cf1f413f50 0x55cf1f416370 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4743e161d5f55a61 server_cookie=31625b711b39f79e in_seq=0 out_seq=0 2024-05-08T22:16:53.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.998+0000 7fb9f4c00700 1 -- 172.21.15.131:0/3181340776 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cf1f3c9810 con 0x55cf1f413f50 2024-05-08T22:16:53.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.998+0000 7fb9f4c00700 10 client.26630 handle_command_reply: tid=0 2024-05-08T22:16:53.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.999+0000 7fba29adcb80 1 client.26630 shutdown 2024-05-08T22:16:53.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.999+0000 7fba29adcb80 1 -- 172.21.15.131:0/3181340776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf1f3fa890 msgr2=0x55cf1f4068c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:53.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.999+0000 7fba29adcb80 1 --2- 172.21.15.131:0/3181340776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf1f3fa890 0x55cf1f4068c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fba04032800 tx=0x7fba04033920 comp rx=0 tx=0).stop 2024-05-08T22:16:53.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.999+0000 7fba29adcb80 1 -- 172.21.15.131:0/3181340776 shutdown_connections 2024-05-08T22:16:53.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:52.999+0000 7fba29adcb80 1 --2- 172.21.15.131:0/3181340776 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55cf1f413f50 0x55cf1f416370 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.000+0000 7fba29adcb80 1 --2- 172.21.15.131:0/3181340776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf1f400970 0x55cf1f4041e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.000+0000 7fba29adcb80 1 --2- 172.21.15.131:0/3181340776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf1f3fa890 0x55cf1f4068c0 secure :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fba04032800 tx=0x7fba04033920 comp rx=0 tx=0).stop 2024-05-08T22:16:53.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.000+0000 7fba29adcb80 1 --2- 172.21.15.131:0/3181340776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf1f400050 0x55cf1f403bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.000+0000 7fba29adcb80 1 -- 172.21.15.131:0/3181340776 >> 172.21.15.131:0/3181340776 conn(0x55cf1f358210 msgr2=0x55cf1f3f50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:53.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.000+0000 7fb9f4c00700 0 client.26630 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:53.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.000+0000 7fba29adcb80 1 -- 172.21.15.131:0/3181340776 shutdown_connections 2024-05-08T22:16:53.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.000+0000 7fba29adcb80 1 -- 172.21.15.131:0/3181340776 wait complete. 2024-05-08T22:16:53.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.000+0000 7fba29adcb80 20 client.26630 trim_cache size 0 max 16384 2024-05-08T22:16:53.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.037+0000 7fba29adcb80 1 Processor -- start 2024-05-08T22:16:53.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.037+0000 7fba29adcb80 1 -- start start 2024-05-08T22:16:53.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.038+0000 7fba29adcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf1f358210 0x55cf1f4063f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:53.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.038+0000 7fba29adcb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf1f400050 0x55cf1f40b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:53.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.038+0000 7fba29adcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf1f400970 0x55cf1f40b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:53.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.038+0000 7fba29adcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf1f3ef800 con 0x55cf1f400970 2024-05-08T22:16:53.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.038+0000 7fba29adcb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf1f41c930 con 0x55cf1f400050 2024-05-08T22:16:53.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.038+0000 7fba29adcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf1f3fb3d0 con 0x55cf1f358210 2024-05-08T22:16:53.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.039+0000 7fba0ea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf1f400050 0x55cf1f40b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:53.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.039+0000 7fba0f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf1f358210 0x55cf1f4063f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:53.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.039+0000 7fba0ea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf1f400050 0x55cf1f40b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:35754/0 (socket says 172.21.15.131:35754) 2024-05-08T22:16:53.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.039+0000 7fba0ea00700 1 -- 172.21.15.131:0/36151722 learned_addr learned my addr 172.21.15.131:0/36151722 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:53.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.039+0000 7fba0fe00700 1 --2- 172.21.15.131:0/36151722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf1f400970 0x55cf1f40b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:53.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.039+0000 7fba0ea00700 1 -- 172.21.15.131:0/36151722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf1f358210 msgr2=0x55cf1f4063f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:53.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.040+0000 7fba0ea00700 1 --2- 172.21.15.131:0/36151722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf1f358210 0x55cf1f4063f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.040+0000 7fba0ea00700 1 -- 172.21.15.131:0/36151722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf1f400970 msgr2=0x55cf1f40b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.040+0000 7fba0ea00700 1 --2- 172.21.15.131:0/36151722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf1f400970 0x55cf1f40b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.040+0000 7fba0ea00700 1 -- 172.21.15.131:0/36151722 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf1f310560 con 0x55cf1f400050 2024-05-08T22:16:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.040+0000 7fba0f400700 1 --2- 172.21.15.131:0/36151722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf1f358210 0x55cf1f4063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:16:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.040+0000 7fba0ea00700 1 --2- 172.21.15.131:0/36151722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf1f400050 0x55cf1f40b1f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0a3700 tx=0x7fb9fc0c3040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4e67affbb945b58 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.040+0000 7fba01200700 1 -- 172.21.15.131:0/36151722 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc09fab0 con 0x55cf1f400050 2024-05-08T22:16:53.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.041+0000 7fba01200700 1 -- 172.21.15.131:0/36151722 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9fc0a3540 con 0x55cf1f400050 2024-05-08T22:16:53.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.041+0000 7fba29adcb80 1 -- 172.21.15.131:0/36151722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf1f400050 msgr2=0x55cf1f40b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:53.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.041+0000 7fba29adcb80 1 --2- 172.21.15.131:0/36151722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf1f400050 0x55cf1f40b1f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0a3700 tx=0x7fb9fc0c3040 comp rx=0 tx=0).stop 2024-05-08T22:16:53.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.043+0000 7fba29adcb80 1 -- 172.21.15.131:0/36151722 shutdown_connections 2024-05-08T22:16:53.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.043+0000 7fba29adcb80 1 --2- 172.21.15.131:0/36151722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf1f400970 0x55cf1f40b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.043+0000 7fba29adcb80 1 --2- 172.21.15.131:0/36151722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf1f358210 0x55cf1f4063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.043+0000 7fba29adcb80 1 --2- 172.21.15.131:0/36151722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf1f400050 0x55cf1f40b1f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.043+0000 7fba29adcb80 1 -- 172.21.15.131:0/36151722 >> 172.21.15.131:0/36151722 conn(0x55cf1f3faf40 msgr2=0x55cf1f3fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:53.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.044+0000 7fba29adcb80 1 -- 172.21.15.131:0/36151722 shutdown_connections 2024-05-08T22:16:53.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.044+0000 7fba29adcb80 1 -- 172.21.15.131:0/36151722 wait complete. 2024-05-08T22:16:53.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.045+0000 7fba29adcb80 1 -- start start 2024-05-08T22:16:53.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.045+0000 7fba29adcb80 1 Processor -- start 2024-05-08T22:16:53.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.046+0000 7fba29adcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf1f358210 0x55cf1f40b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:53.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.046+0000 7fba29adcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf1f400050 0x55cf1f3fca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:53.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.046+0000 7fba29adcb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf1f400970 0x55cf1f3fcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:53.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.047+0000 7fba29adcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf1f3fd490 con 0x55cf1f358210 2024-05-08T22:16:53.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.047+0000 7fba29adcb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf1f3fd600 con 0x55cf1f400970 2024-05-08T22:16:53.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.047+0000 7fba29adcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf1f3fd7c0 con 0x55cf1f400050 2024-05-08T22:16:53.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.047+0000 7fba29adcb80 10 client.0 _ll_register_callbacks cb 0x55cf1f2efb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:16:53.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.047+0000 7fba0fe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf1f400970 0x55cf1f3fcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:53.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.047+0000 7fba0fe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf1f400970 0x55cf1f3fcf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:35760/0 (socket says 172.21.15.131:35760) 2024-05-08T22:16:53.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.047+0000 7fba0fe00700 1 -- 172.21.15.131:0/2133873016 learned_addr learned my addr 172.21.15.131:0/2133873016 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:53.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.047+0000 7fba0ea00700 1 --2- 172.21.15.131:0/2133873016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf1f400050 0x55cf1f3fca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:53.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.048+0000 7fba0f400700 1 --2- 172.21.15.131:0/2133873016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf1f358210 0x55cf1f40b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:53.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.048+0000 7fba0fe00700 1 -- 172.21.15.131:0/2133873016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf1f400050 msgr2=0x55cf1f3fca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.048+0000 7fba0fe00700 1 --2- 172.21.15.131:0/2133873016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf1f400050 0x55cf1f3fca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.048+0000 7fba0fe00700 1 -- 172.21.15.131:0/2133873016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf1f358210 msgr2=0x55cf1f40b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.048+0000 7fba0fe00700 1 --2- 172.21.15.131:0/2133873016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf1f358210 0x55cf1f40b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.048+0000 7fba0fe00700 1 -- 172.21.15.131:0/2133873016 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba080ae080 con 0x55cf1f400970 2024-05-08T22:16:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.049+0000 7fba0fe00700 1 --2- 172.21.15.131:0/2133873016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf1f400970 0x55cf1f3fcf50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fba10050600 tx=0x7fba10066eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d85b7314d656717d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:53.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.049+0000 7fb9f4c00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:16:53.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.055+0000 7fb9f4c00700 1 -- 172.21.15.131:0/2133873016 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba10069bd0 con 0x55cf1f400970 2024-05-08T22:16:53.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.055+0000 7fb9f4c00700 1 -- 172.21.15.131:0/2133873016 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fba10069d30 con 0x55cf1f400970 2024-05-08T22:16:53.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.056+0000 7fba29adcb80 1 -- 172.21.15.131:0/2133873016 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba04033c60 con 0x55cf1f400970 2024-05-08T22:16:53.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.057+0000 7fb9f4c00700 1 -- 172.21.15.131:0/2133873016 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba10087400 con 0x55cf1f400970 2024-05-08T22:16:53.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.057+0000 7fba29adcb80 10 client.26636 fetch_fsmap learned FSMap version 30 2024-05-08T22:16:53.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.057+0000 7fba29adcb80 1 -- 172.21.15.131:0/2133873016 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55cf1f1cfcd0 con 0x55cf1f400970 2024-05-08T22:16:53.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.058+0000 7fb9f4c00700 1 -- 172.21.15.131:0/2133873016 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba04033c60 con 0x55cf1f400970 2024-05-08T22:16:53.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.058+0000 7fb9f4c00700 1 -- 172.21.15.131:0/2133873016 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fba1008f540 con 0x55cf1f400970 2024-05-08T22:16:53.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.058+0000 7fba29adcb80 10 client.26636 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:16:53.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.058+0000 7fba29adcb80 10 client.26636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:16:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.058+0000 7fba29adcb80 20 client.26636 populate_metadata read hostname 'smithi131' 2024-05-08T22:16:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.058+0000 7fba29adcb80 1 --2- 172.21.15.131:0/2133873016 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55cf1f46ae10 0x55cf1f46d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.058+0000 7fba29adcb80 4 client.26636 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:16:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.058+0000 7fba29adcb80 1 -- 172.21.15.131:0/2133873016 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cf1f342ec0 con 0x55cf1f46ae10 2024-05-08T22:16:53.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.063+0000 7fba0f400700 1 --2- 172.21.15.131:0/2133873016 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55cf1f46ae10 0x55cf1f46d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:53.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.072+0000 7fb9f4c00700 10 client.26636 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:53.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.072+0000 7fba0f400700 1 --2- 172.21.15.131:0/2133873016 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55cf1f46ae10 0x55cf1f46d230 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a32bfdaaad884759 server_cookie=b59ebd80bade91c7 in_seq=0 out_seq=0 2024-05-08T22:16:53.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.082+0000 7fb9f4c00700 1 -- 172.21.15.131:0/2133873016 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55cf1f342ec0 con 0x55cf1f46ae10 2024-05-08T22:16:53.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.082+0000 7fb9f4c00700 10 client.26636 handle_command_reply: tid=0 2024-05-08T22:16:53.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.082+0000 7fba29adcb80 1 client.26636 shutdown 2024-05-08T22:16:53.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.083+0000 7fba29adcb80 1 -- 172.21.15.131:0/2133873016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf1f400970 msgr2=0x55cf1f3fcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:53.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.083+0000 7fba29adcb80 1 --2- 172.21.15.131:0/2133873016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf1f400970 0x55cf1f3fcf50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fba10050600 tx=0x7fba10066eb0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.083+0000 7fba29adcb80 1 -- 172.21.15.131:0/2133873016 shutdown_connections 2024-05-08T22:16:53.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.083+0000 7fba29adcb80 1 --2- 172.21.15.131:0/2133873016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf1f400970 0x55cf1f3fcf50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.083+0000 7fba29adcb80 1 --2- 172.21.15.131:0/2133873016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf1f358210 0x55cf1f40b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.083+0000 7fba29adcb80 1 --2- 172.21.15.131:0/2133873016 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55cf1f46ae10 0x55cf1f46d230 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.083+0000 7fba29adcb80 1 --2- 172.21.15.131:0/2133873016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf1f400050 0x55cf1f3fca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.083+0000 7fba29adcb80 1 -- 172.21.15.131:0/2133873016 >> 172.21.15.131:0/2133873016 conn(0x55cf1f3faf40 msgr2=0x55cf1f40fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:53.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.083+0000 7fb9f4c00700 0 client.26636 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:53.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.083+0000 7fba29adcb80 1 -- 172.21.15.131:0/2133873016 shutdown_connections 2024-05-08T22:16:53.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.084+0000 7fba29adcb80 1 -- 172.21.15.131:0/2133873016 wait complete. 2024-05-08T22:16:53.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.084+0000 7fba29adcb80 20 client.26636 trim_cache size 0 max 16384 2024-05-08T22:16:53.086 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:16:53.086 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-08T22:16:53.086 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-08T22:16:53.086 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:16:53.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.085+0000 7fba14a00700 1 -- 172.21.15.131:0/2011299645 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb9f80a13e0 msgr2=0x7fb9f80a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:53.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.085+0000 7fba14a00700 1 --2- 172.21.15.131:0/2011299645 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb9f80a13e0 0x7fb9f80a38a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fba10064340 tx=0x7fba1004fd00 comp rx=0 tx=0).stop 2024-05-08T22:16:53.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.085+0000 7fba14a00700 1 -- 172.21.15.131:0/2011299645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba080ac160 msgr2=0x7fba0813fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:53.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.085+0000 7fba14a00700 1 --2- 172.21.15.131:0/2011299645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba080ac160 0x7fba0813fa00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc006010 tx=0x7fb9fc004730 comp rx=0 tx=0).stop 2024-05-08T22:16:53.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.085+0000 7fba14a00700 1 -- 172.21.15.131:0/2011299645 shutdown_connections 2024-05-08T22:16:53.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.085+0000 7fba14a00700 1 --2- 172.21.15.131:0/2011299645 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb9f80a13e0 0x7fb9f80a38a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.085+0000 7fba14a00700 1 --2- 172.21.15.131:0/2011299645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba08143da0 0x7fba08144220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.085+0000 7fba14a00700 1 --2- 172.21.15.131:0/2011299645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba080a5020 0x7fba0813f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.085+0000 7fba14a00700 1 --2- 172.21.15.131:0/2011299645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba080ac160 0x7fba0813fa00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.085+0000 7fba14a00700 1 -- 172.21.15.131:0/2011299645 >> 172.21.15.131:0/2011299645 conn(0x7fba0801a6a0 msgr2=0x7fba080a3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:53.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.086+0000 7fba14a00700 1 -- 172.21.15.131:0/2011299645 shutdown_connections 2024-05-08T22:16:53.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.086+0000 7fba14a00700 1 -- 172.21.15.131:0/2011299645 wait complete. 2024-05-08T22:16:53.120 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T22:16:53.120 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:16:53.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.453+0000 7ff253200700 1 -- 172.21.15.131:0/1027284059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff25410d680 msgr2=0x7ff25410fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:53.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.453+0000 7ff253200700 1 --2- 172.21.15.131:0/1027284059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff25410d680 0x7ff25410fa70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff24c0098f0 tx=0x7ff24c009c00 comp rx=0 tx=0).stop 2024-05-08T22:16:53.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.453+0000 7ff253200700 1 -- 172.21.15.131:0/1027284059 shutdown_connections 2024-05-08T22:16:53.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.453+0000 7ff253200700 1 --2- 172.21.15.131:0/1027284059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff25410d680 0x7ff25410fa70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.453+0000 7ff253200700 1 --2- 172.21.15.131:0/1027284059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff254073350 0x7ff254073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.453+0000 7ff253200700 1 --2- 172.21.15.131:0/1027284059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff254073c70 0x7ff25410d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.453+0000 7ff253200700 1 -- 172.21.15.131:0/1027284059 >> 172.21.15.131:0/1027284059 conn(0x7ff2540fc910 msgr2=0x7ff2540fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:53.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.454+0000 7ff253200700 1 -- 172.21.15.131:0/1027284059 shutdown_connections 2024-05-08T22:16:53.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.454+0000 7ff253200700 1 -- 172.21.15.131:0/1027284059 wait complete. 2024-05-08T22:16:53.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.455+0000 7ff253200700 1 Processor -- start 2024-05-08T22:16:53.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.456+0000 7ff253200700 1 -- start start 2024-05-08T22:16:53.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.456+0000 7ff253200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff254073350 0x7ff25410ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:53.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.456+0000 7ff253200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff254073c70 0x7ff254109f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:53.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.456+0000 7ff253200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff25410d680 0x7ff25410a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:53.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.456+0000 7ff253200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25410ac70 con 0x7ff254073350 2024-05-08T22:16:53.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.456+0000 7ff253200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25410ade0 con 0x7ff25410d680 2024-05-08T22:16:53.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.456+0000 7ff253200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2541a9e20 con 0x7ff254073c70 2024-05-08T22:16:53.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.456+0000 7ff249400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff254073c70 0x7ff254109f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:53.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.457+0000 7ff249400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff254073c70 0x7ff254109f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54512/0 (socket says 172.21.15.131:54512) 2024-05-08T22:16:53.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.457+0000 7ff249400700 1 -- 172.21.15.131:0/4197012899 learned_addr learned my addr 172.21.15.131:0/4197012899 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:53.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.457+0000 7ff252800700 1 --2- 172.21.15.131:0/4197012899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff25410d680 0x7ff25410a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:53.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.457+0000 7ff252800700 1 -- 172.21.15.131:0/4197012899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff254073c70 msgr2=0x7ff254109f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:53.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.457+0000 7ff252800700 1 --2- 172.21.15.131:0/4197012899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff254073c70 0x7ff254109f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.457+0000 7ff252800700 1 -- 172.21.15.131:0/4197012899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff254073350 msgr2=0x7ff25410ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:53.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.457+0000 7ff252800700 1 --2- 172.21.15.131:0/4197012899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff254073350 0x7ff25410ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.457+0000 7ff252800700 1 -- 172.21.15.131:0/4197012899 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24c009520 con 0x7ff25410d680 2024-05-08T22:16:53.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.457+0000 7ff249400700 1 --2- 172.21.15.131:0/4197012899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff254073c70 0x7ff254109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:16:53.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.458+0000 7ff252800700 1 --2- 172.21.15.131:0/4197012899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff25410d680 0x7ff25410a4a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009810 tx=0x7ff24c00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:53.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.461+0000 7ff24b400700 1 -- 172.21.15.131:0/4197012899 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c01c070 con 0x7ff25410d680 2024-05-08T22:16:53.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.461+0000 7ff24b400700 1 -- 172.21.15.131:0/4197012899 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff24c00baa0 con 0x7ff25410d680 2024-05-08T22:16:53.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.461+0000 7ff24b400700 1 -- 172.21.15.131:0/4197012899 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c004930 con 0x7ff25410d680 2024-05-08T22:16:53.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.461+0000 7ff253200700 1 -- 172.21.15.131:0/4197012899 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2541aa020 con 0x7ff25410d680 2024-05-08T22:16:53.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.461+0000 7ff253200700 1 -- 172.21.15.131:0/4197012899 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2541aa4a0 con 0x7ff25410d680 2024-05-08T22:16:53.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.463+0000 7ff253200700 1 -- 172.21.15.131:0/4197012899 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff254104480 con 0x7ff25410d680 2024-05-08T22:16:53.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.471+0000 7ff24b400700 1 -- 172.21.15.131:0/4197012899 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7ff24c020070 con 0x7ff25410d680 2024-05-08T22:16:53.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.473+0000 7ff24b400700 1 --2- 172.21.15.131:0/4197012899 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff2380a1310 0x7ff2380a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:53.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.473+0000 7ff24b400700 1 -- 172.21.15.131:0/4197012899 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7ff24c0cdfc0 con 0x7ff25410d680 2024-05-08T22:16:53.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.474+0000 7ff251e00700 1 --2- 172.21.15.131:0/4197012899 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff2380a1310 0x7ff2380a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:53.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.475+0000 7ff251e00700 1 --2- 172.21.15.131:0/4197012899 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff2380a1310 0x7ff2380a37d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff24400e350 tx=0x7ff24400b040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:53.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.478+0000 7ff24b400700 1 -- 172.21.15.131:0/4197012899 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff24c0970b0 con 0x7ff25410d680 2024-05-08T22:16:53.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.955+0000 7ff253200700 1 -- 172.21.15.131:0/4197012899 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff254067ba0 con 0x7ff25410d680 2024-05-08T22:16:53.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.956+0000 7ff24b400700 1 -- 172.21.15.131:0/4197012899 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10891 (secure 0 0 0) 0x7ff24c027020 con 0x7ff25410d680 2024-05-08T22:16:53.959 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:16:53.959 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:05:14.034908+0000","modified":"2024-05-08T22:16:38.400949+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:16:53.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.960+0000 7ff253200700 1 -- 172.21.15.131:0/4197012899 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff2380a1310 msgr2=0x7ff2380a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:53.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.960+0000 7ff253200700 1 --2- 172.21.15.131:0/4197012899 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff2380a1310 0x7ff2380a37d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff24400e350 tx=0x7ff24400b040 comp rx=0 tx=0).stop 2024-05-08T22:16:53.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.960+0000 7ff253200700 1 -- 172.21.15.131:0/4197012899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff25410d680 msgr2=0x7ff25410a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:53.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.960+0000 7ff253200700 1 --2- 172.21.15.131:0/4197012899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff25410d680 0x7ff25410a4a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009810 tx=0x7ff24c00fa00 comp rx=0 tx=0).stop 2024-05-08T22:16:53.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.960+0000 7ff253200700 1 -- 172.21.15.131:0/4197012899 shutdown_connections 2024-05-08T22:16:53.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.960+0000 7ff253200700 1 --2- 172.21.15.131:0/4197012899 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff2380a1310 0x7ff2380a37d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.960+0000 7ff253200700 1 --2- 172.21.15.131:0/4197012899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff25410d680 0x7ff25410a4a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.960+0000 7ff253200700 1 --2- 172.21.15.131:0/4197012899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff254073350 0x7ff25410ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.960+0000 7ff253200700 1 --2- 172.21.15.131:0/4197012899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff254073c70 0x7ff254109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:53.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.960+0000 7ff253200700 1 -- 172.21.15.131:0/4197012899 >> 172.21.15.131:0/4197012899 conn(0x7ff2540fc910 msgr2=0x7ff2540fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:53.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.961+0000 7ff253200700 1 -- 172.21.15.131:0/4197012899 shutdown_connections 2024-05-08T22:16:53.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:53.961+0000 7ff253200700 1 -- 172.21.15.131:0/4197012899 wait complete. 2024-05-08T22:16:53.968 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 30 2024-05-08T22:16:53.998 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T22:16:54.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.354+0000 7fdd51400700 1 -- 172.21.15.131:0/739204064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd4c103930 msgr2=0x7fdd4c103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:54.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.354+0000 7fdd51400700 1 --2- 172.21.15.131:0/739204064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd4c103930 0x7fdd4c103db0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdd4400b0a0 tx=0x7fdd4400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.355+0000 7fdd51400700 1 -- 172.21.15.131:0/739204064 shutdown_connections 2024-05-08T22:16:54.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.355+0000 7fdd51400700 1 --2- 172.21.15.131:0/739204064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd4c103930 0x7fdd4c103db0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.355+0000 7fdd51400700 1 --2- 172.21.15.131:0/739204064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd4c105e30 0x7fdd4c068050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.355+0000 7fdd51400700 1 --2- 172.21.15.131:0/739204064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd4c068590 0x7fdd4c068a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.355+0000 7fdd51400700 1 -- 172.21.15.131:0/739204064 >> 172.21.15.131:0/739204064 conn(0x7fdd4c076f40 msgr2=0x7fdd4c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:54.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.355+0000 7fdd51400700 1 -- 172.21.15.131:0/739204064 shutdown_connections 2024-05-08T22:16:54.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.355+0000 7fdd51400700 1 -- 172.21.15.131:0/739204064 wait complete. 2024-05-08T22:16:54.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.356+0000 7fdd51400700 1 Processor -- start 2024-05-08T22:16:54.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.357+0000 7fdd51400700 1 -- start start 2024-05-08T22:16:54.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.357+0000 7fdd51400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd4c068590 0x7fdd4c1a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.357+0000 7fdd51400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd4c103930 0x7fdd4c1a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.357+0000 7fdd51400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd4c105e30 0x7fdd4c19d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.357+0000 7fdd51400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c19dfa0 con 0x7fdd4c068590 2024-05-08T22:16:54.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.357+0000 7fdd51400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c19e110 con 0x7fdd4c105e30 2024-05-08T22:16:54.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.357+0000 7fdd51400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c19e2d0 con 0x7fdd4c103930 2024-05-08T22:16:54.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.358+0000 7fdd4a800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd4c068590 0x7fdd4c1a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.358+0000 7fdd4b200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd4c105e30 0x7fdd4c19d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.358+0000 7fdd4b200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd4c105e30 0x7fdd4c19d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:44636/0 (socket says 172.21.15.131:44636) 2024-05-08T22:16:54.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.358+0000 7fdd4b200700 1 -- 172.21.15.131:0/1754768649 learned_addr learned my addr 172.21.15.131:0/1754768649 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:54.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.358+0000 7fdd49e00700 1 --2- 172.21.15.131:0/1754768649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd4c103930 0x7fdd4c1a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.358+0000 7fdd4b200700 1 -- 172.21.15.131:0/1754768649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd4c103930 msgr2=0x7fdd4c1a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:54.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.358+0000 7fdd4b200700 1 --2- 172.21.15.131:0/1754768649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd4c103930 0x7fdd4c1a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.358+0000 7fdd4b200700 1 -- 172.21.15.131:0/1754768649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd4c068590 msgr2=0x7fdd4c1a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:54.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.358+0000 7fdd4b200700 1 --2- 172.21.15.131:0/1754768649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd4c068590 0x7fdd4c1a37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.358+0000 7fdd4b200700 1 -- 172.21.15.131:0/1754768649 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd44009d00 con 0x7fdd4c105e30 2024-05-08T22:16:54.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.358+0000 7fdd4a800700 1 --2- 172.21.15.131:0/1754768649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd4c068590 0x7fdd4c1a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:16:54.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.359+0000 7fdd4b200700 1 --2- 172.21.15.131:0/1754768649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd4c105e30 0x7fdd4c19d940 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdd440077f0 tx=0x7fdd44007820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:54.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.359+0000 7fdd43400700 1 -- 172.21.15.131:0/1754768649 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4400e040 con 0x7fdd4c105e30 2024-05-08T22:16:54.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.359+0000 7fdd43400700 1 -- 172.21.15.131:0/1754768649 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd44004c60 con 0x7fdd4c105e30 2024-05-08T22:16:54.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.359+0000 7fdd51400700 1 -- 172.21.15.131:0/1754768649 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd4c19e550 con 0x7fdd4c105e30 2024-05-08T22:16:54.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.360+0000 7fdd43400700 1 -- 172.21.15.131:0/1754768649 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd44005280 con 0x7fdd4c105e30 2024-05-08T22:16:54.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.360+0000 7fdd51400700 1 -- 172.21.15.131:0/1754768649 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd4c1aa7b0 con 0x7fdd4c105e30 2024-05-08T22:16:54.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.369+0000 7fdd43400700 1 -- 172.21.15.131:0/1754768649 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fdd4401b450 con 0x7fdd4c105e30 2024-05-08T22:16:54.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.373+0000 7fdd43400700 1 --2- 172.21.15.131:0/1754768649 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fdd300a1310 0x7fdd300a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.373+0000 7fdd43400700 1 -- 172.21.15.131:0/1754768649 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fdd440ce880 con 0x7fdd4c105e30 2024-05-08T22:16:54.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.374+0000 7fdd4a800700 1 --2- 172.21.15.131:0/1754768649 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fdd300a1310 0x7fdd300a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.379+0000 7fdd64febb80 1 Processor -- start 2024-05-08T22:16:54.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.379+0000 7fdd64febb80 1 -- start start 2024-05-08T22:16:54.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.380+0000 7fdd64febb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564849de3e00 0x564849de41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.380+0000 7fdd64febb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564849de4720 0x564849dde350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.380+0000 7fdd64febb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564849dde890 0x564849ddecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.380+0000 7fdd64febb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564849d35150 con 0x564849de3e00 2024-05-08T22:16:54.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.380+0000 7fdd64febb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564849d32b10 con 0x564849de4720 2024-05-08T22:16:54.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.380+0000 7fdd64febb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564849d2e400 con 0x564849dde890 2024-05-08T22:16:54.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.380+0000 7fdd4b200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564849dde890 0x564849ddecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.380+0000 7fdd4b200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564849dde890 0x564849ddecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45930/0 (socket says 172.21.15.131:45930) 2024-05-08T22:16:54.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.380+0000 7fdd4b200700 1 -- 172.21.15.131:0/3718050495 learned_addr learned my addr 172.21.15.131:0/3718050495 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:54.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.381+0000 7fdd4b200700 1 -- 172.21.15.131:0/3718050495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564849de4720 msgr2=0x564849dde350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:16:54.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.381+0000 7fdd4b200700 1 --2- 172.21.15.131:0/3718050495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564849de4720 0x564849dde350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.381+0000 7fdd4b200700 1 -- 172.21.15.131:0/3718050495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564849de3e00 msgr2=0x564849de41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:54.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.381+0000 7fdd4b200700 1 --2- 172.21.15.131:0/3718050495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564849de3e00 0x564849de41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.381+0000 7fdd4b200700 1 -- 172.21.15.131:0/3718050495 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564849baf3b0 con 0x564849dde890 2024-05-08T22:16:54.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.381+0000 7fdd4b200700 1 --2- 172.21.15.131:0/3718050495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564849dde890 0x564849ddecd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdd44097b30 tx=0x7fdd44009c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c5efb502e0fea0f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:54.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.381+0000 7fdd4a800700 1 --2- 172.21.15.131:0/1754768649 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fdd300a1310 0x7fdd300a37d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c005e10 tx=0x7fdd3c005d60 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:54.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.381+0000 7fdd37400700 1 -- 172.21.15.131:0/3718050495 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4401bb80 con 0x564849dde890 2024-05-08T22:16:54.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.381+0000 7fdd37400700 1 -- 172.21.15.131:0/3718050495 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd44004dd0 con 0x564849dde890 2024-05-08T22:16:54.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.382+0000 7fdd64febb80 1 -- 172.21.15.131:0/3718050495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564849dde890 msgr2=0x564849ddecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:54.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.382+0000 7fdd64febb80 1 --2- 172.21.15.131:0/3718050495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564849dde890 0x564849ddecd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdd44097b30 tx=0x7fdd44009c90 comp rx=0 tx=0).stop 2024-05-08T22:16:54.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.383+0000 7fdd64febb80 1 -- 172.21.15.131:0/3718050495 shutdown_connections 2024-05-08T22:16:54.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.383+0000 7fdd64febb80 1 --2- 172.21.15.131:0/3718050495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564849dde890 0x564849ddecd0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.383+0000 7fdd64febb80 1 --2- 172.21.15.131:0/3718050495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564849de3e00 0x564849de41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.383+0000 7fdd64febb80 1 --2- 172.21.15.131:0/3718050495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564849de4720 0x564849dde350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.383+0000 7fdd64febb80 1 -- 172.21.15.131:0/3718050495 >> 172.21.15.131:0/3718050495 conn(0x564849d3c210 msgr2=0x564849dd8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:54.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.383+0000 7fdd64febb80 1 -- 172.21.15.131:0/3718050495 shutdown_connections 2024-05-08T22:16:54.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.384+0000 7fdd64febb80 1 -- 172.21.15.131:0/3718050495 wait complete. 2024-05-08T22:16:54.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.384+0000 7fdd64febb80 1 -- start start 2024-05-08T22:16:54.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.385+0000 7fdd64febb80 1 Processor -- start 2024-05-08T22:16:54.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.386+0000 7fdd64febb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564849dde890 0x564849de7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.386+0000 7fdd64febb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564849de3e00 0x564849de7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.386+0000 7fdd64febb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564849de4720 0x564849de84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.386+0000 7fdd64febb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564849cca560 con 0x564849de4720 2024-05-08T22:16:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.387+0000 7fdd64febb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564849cca200 con 0x564849dde890 2024-05-08T22:16:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.387+0000 7fdd4a800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564849dde890 0x564849de7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.388+0000 7fdd4a800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564849dde890 0x564849de7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:44650/0 (socket says 172.21.15.131:44650) 2024-05-08T22:16:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.388+0000 7fdd4a800700 1 -- 172.21.15.131:0/1933206092 learned_addr learned my addr 172.21.15.131:0/1933206092 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:54.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.388+0000 7fdd4b200700 1 --2- 172.21.15.131:0/1933206092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564849de4720 0x564849de84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.388+0000 7fdd49e00700 1 --2- 172.21.15.131:0/1933206092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564849de3e00 0x564849de7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.388+0000 7fdd64febb80 1 -- 172.21.15.131:0/1933206092 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564849bb16a0 con 0x564849de3e00 2024-05-08T22:16:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.389+0000 7fdd64febb80 10 client.0 _ll_register_callbacks cb 0x564849cd3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:16:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.389+0000 7fdd49e00700 1 -- 172.21.15.131:0/1933206092 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564849dde890 msgr2=0x564849de7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.389+0000 7fdd49e00700 1 --2- 172.21.15.131:0/1933206092 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564849dde890 0x564849de7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.389+0000 7fdd49e00700 1 -- 172.21.15.131:0/1933206092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564849de4720 msgr2=0x564849de84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.389+0000 7fdd49e00700 1 --2- 172.21.15.131:0/1933206092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564849de4720 0x564849de84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.389+0000 7fdd49e00700 1 -- 172.21.15.131:0/1933206092 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd4c1aa7b0 con 0x564849de3e00 2024-05-08T22:16:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.389+0000 7fdd49e00700 1 --2- 172.21.15.131:0/1933206092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564849de3e00 0x564849de7f90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x564849de4b80 tx=0x7fdd380106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3ad174762f4c49a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.390+0000 7fdd2be00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:16:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.390+0000 7fdd4a800700 1 --2- 172.21.15.131:0/1933206092 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564849dde890 0x564849de7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:16:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.390+0000 7fdd2be00700 1 -- 172.21.15.131:0/1933206092 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd38010ed0 con 0x564849de3e00 2024-05-08T22:16:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.390+0000 7fdd4b200700 1 --2- 172.21.15.131:0/1933206092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564849de4720 0x564849de84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:16:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.390+0000 7fdd64febb80 1 -- 172.21.15.131:0/1933206092 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564849c87270 con 0x564849de3e00 2024-05-08T22:16:54.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.391+0000 7fdd2be00700 1 -- 172.21.15.131:0/1933206092 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fdd38004830 con 0x564849de3e00 2024-05-08T22:16:54.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.391+0000 7fdd2be00700 1 -- 172.21.15.131:0/1933206092 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd380049f0 con 0x564849de3e00 2024-05-08T22:16:54.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.391+0000 7fdd64febb80 10 client.26743 fetch_fsmap learned FSMap version 30 2024-05-08T22:16:54.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.391+0000 7fdd64febb80 1 -- 172.21.15.131:0/1933206092 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564849b5bb60 con 0x564849de3e00 2024-05-08T22:16:54.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.391+0000 7fdd2be00700 1 -- 172.21.15.131:0/1933206092 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564849c87270 con 0x564849de3e00 2024-05-08T22:16:54.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.392+0000 7fdd2be00700 1 -- 172.21.15.131:0/1933206092 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fdd3801a4a0 con 0x564849de3e00 2024-05-08T22:16:54.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.396+0000 7fdd64febb80 10 client.26743 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:16:54.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.396+0000 7fdd64febb80 10 client.26743 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:16:54.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.396+0000 7fdd64febb80 20 client.26743 populate_metadata read hostname 'smithi131' 2024-05-08T22:16:54.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.396+0000 7fdd64febb80 1 --2- 172.21.15.131:0/1933206092 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x564849df8070 0x564849dfa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.396+0000 7fdd64febb80 4 client.26743 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:16:54.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.396+0000 7fdd64febb80 1 -- 172.21.15.131:0/1933206092 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564849dad810 con 0x564849df8070 2024-05-08T22:16:54.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.396+0000 7fdd4a800700 1 --2- 172.21.15.131:0/1933206092 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x564849df8070 0x564849dfa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.397+0000 7fdd4a800700 1 --2- 172.21.15.131:0/1933206092 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x564849df8070 0x564849dfa490 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=800212200fe1ec19 server_cookie=1fbb15ed684d415a in_seq=0 out_seq=0 2024-05-08T22:16:54.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.400+0000 7fdd2be00700 10 client.26743 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:16:54.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.401+0000 7fdd2be00700 1 -- 172.21.15.131:0/1933206092 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564849dad810 con 0x564849df8070 2024-05-08T22:16:54.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.401+0000 7fdd2be00700 10 client.26743 handle_command_reply: tid=0 2024-05-08T22:16:54.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.401+0000 7fdd64febb80 1 client.26743 shutdown 2024-05-08T22:16:54.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.402+0000 7fdd64febb80 1 -- 172.21.15.131:0/1933206092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564849de3e00 msgr2=0x564849de7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:54.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.402+0000 7fdd64febb80 1 --2- 172.21.15.131:0/1933206092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564849de3e00 0x564849de7f90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x564849de4b80 tx=0x7fdd380106b0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.402+0000 7fdd64febb80 1 -- 172.21.15.131:0/1933206092 shutdown_connections 2024-05-08T22:16:54.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.402+0000 7fdd64febb80 1 --2- 172.21.15.131:0/1933206092 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x564849df8070 0x564849dfa490 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.402+0000 7fdd64febb80 1 --2- 172.21.15.131:0/1933206092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564849de4720 0x564849de84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.402+0000 7fdd64febb80 1 --2- 172.21.15.131:0/1933206092 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564849dde890 0x564849de7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.402+0000 7fdd64febb80 1 --2- 172.21.15.131:0/1933206092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564849de3e00 0x564849de7f90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.402+0000 7fdd64febb80 1 -- 172.21.15.131:0/1933206092 >> 172.21.15.131:0/1933206092 conn(0x564849d3c210 msgr2=0x564849dd85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:54.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.403+0000 7fdd2be00700 0 client.26743 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:16:54.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.403+0000 7fdd64febb80 1 -- 172.21.15.131:0/1933206092 shutdown_connections 2024-05-08T22:16:54.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.404+0000 7fdd64febb80 1 -- 172.21.15.131:0/1933206092 wait complete. 2024-05-08T22:16:54.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.405+0000 7fdd64febb80 20 client.26743 trim_cache size 0 max 16384 2024-05-08T22:16:54.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.424+0000 7fdd64febb80 1 Processor -- start 2024-05-08T22:16:54.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.424+0000 7fdd64febb80 1 -- start start 2024-05-08T22:16:54.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.424+0000 7fdd64febb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564849dde890 0x564849dde350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.424+0000 7fdd64febb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564849de3e00 0x564849df4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.424+0000 7fdd64febb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564849de4720 0x564849dd8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.424+0000 7fdd64febb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564849c87270 con 0x564849de4720 2024-05-08T22:16:54.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.424+0000 7fdd64febb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564849dea5e0 con 0x564849de3e00 2024-05-08T22:16:54.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.424+0000 7fdd64febb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564849ddf230 con 0x564849dde890 2024-05-08T22:16:54.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.424+0000 7fdd49e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564849de3e00 0x564849df4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.424+0000 7fdd49e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564849de3e00 0x564849df4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:44656/0 (socket says 172.21.15.131:44656) 2024-05-08T22:16:54.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.424+0000 7fdd49e00700 1 -- 172.21.15.131:0/1510545386 learned_addr learned my addr 172.21.15.131:0/1510545386 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:54.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.425+0000 7fdd4a800700 1 --2- 172.21.15.131:0/1510545386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564849dde890 0x564849dde350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.425+0000 7fdd4b200700 1 --2- 172.21.15.131:0/1510545386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564849de4720 0x564849dd8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.425+0000 7fdd49e00700 1 -- 172.21.15.131:0/1510545386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564849dde890 msgr2=0x564849dde350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:54.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.425+0000 7fdd49e00700 1 --2- 172.21.15.131:0/1510545386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564849dde890 0x564849dde350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.425+0000 7fdd49e00700 1 -- 172.21.15.131:0/1510545386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564849de4720 msgr2=0x564849dd8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:54.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.425+0000 7fdd49e00700 1 --2- 172.21.15.131:0/1510545386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564849de4720 0x564849dd8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.425+0000 7fdd49e00700 1 -- 172.21.15.131:0/1510545386 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564849cf4560 con 0x564849de3e00 2024-05-08T22:16:54.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.425+0000 7fdd4b200700 1 --2- 172.21.15.131:0/1510545386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564849de4720 0x564849dd8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:16:54.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.425+0000 7fdd49e00700 1 --2- 172.21.15.131:0/1510545386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564849de3e00 0x564849df4160 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdd38001eb0 tx=0x7fdd38005ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68a8b18d13a02d5b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:54.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.425+0000 7fdd37e00700 1 -- 172.21.15.131:0/1510545386 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3801a480 con 0x564849de3e00 2024-05-08T22:16:54.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.426+0000 7fdd37e00700 1 -- 172.21.15.131:0/1510545386 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd38010ed0 con 0x564849de3e00 2024-05-08T22:16:54.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.426+0000 7fdd37e00700 1 -- 172.21.15.131:0/1510545386 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd38003990 con 0x564849de3e00 2024-05-08T22:16:54.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.426+0000 7fdd64febb80 1 -- 172.21.15.131:0/1510545386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564849de3e00 msgr2=0x564849df4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:54.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.426+0000 7fdd64febb80 1 --2- 172.21.15.131:0/1510545386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564849de3e00 0x564849df4160 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdd38001eb0 tx=0x7fdd38005ca0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.426+0000 7fdd64febb80 1 -- 172.21.15.131:0/1510545386 shutdown_connections 2024-05-08T22:16:54.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.426+0000 7fdd64febb80 1 --2- 172.21.15.131:0/1510545386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564849de4720 0x564849dd8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.426+0000 7fdd64febb80 1 --2- 172.21.15.131:0/1510545386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564849dde890 0x564849dde350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.426+0000 7fdd64febb80 1 --2- 172.21.15.131:0/1510545386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564849de3e00 0x564849df4160 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.426+0000 7fdd64febb80 1 -- 172.21.15.131:0/1510545386 >> 172.21.15.131:0/1510545386 conn(0x564849d3c210 msgr2=0x564849de1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:54.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.427+0000 7fdd64febb80 1 -- 172.21.15.131:0/1510545386 shutdown_connections 2024-05-08T22:16:54.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.427+0000 7fdd64febb80 1 -- 172.21.15.131:0/1510545386 wait complete. 2024-05-08T22:16:54.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.427+0000 7fdd64febb80 1 -- start start 2024-05-08T22:16:54.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.428+0000 7fdd64febb80 1 Processor -- start 2024-05-08T22:16:54.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.428+0000 7fdd64febb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564849dde890 0x564849def010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.429+0000 7fdd64febb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564849de3e00 0x564849def550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.429+0000 7fdd64febb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564849de4720 0x564849ddf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.429+0000 7fdd64febb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564849ddfe70 con 0x564849de3e00 2024-05-08T22:16:54.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.429+0000 7fdd64febb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564849ddffb0 con 0x564849de4720 2024-05-08T22:16:54.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.429+0000 7fdd64febb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564849de0140 con 0x564849dde890 2024-05-08T22:16:54.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.429+0000 7fdd64febb80 10 client.0 _ll_register_callbacks cb 0x564849cd3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:16:54.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.429+0000 7fdd4a800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564849dde890 0x564849def010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.429+0000 7fdd4a800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564849dde890 0x564849def010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45962/0 (socket says 172.21.15.131:45962) 2024-05-08T22:16:54.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.429+0000 7fdd4a800700 1 -- 172.21.15.131:0/3221818879 learned_addr learned my addr 172.21.15.131:0/3221818879 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:54.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.430+0000 7fdd49e00700 1 --2- 172.21.15.131:0/3221818879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564849de3e00 0x564849def550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.430+0000 7fdd4a800700 1 -- 172.21.15.131:0/3221818879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564849de4720 msgr2=0x564849ddf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:54.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.430+0000 7fdd4a800700 1 --2- 172.21.15.131:0/3221818879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564849de4720 0x564849ddf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.430+0000 7fdd4a800700 1 -- 172.21.15.131:0/3221818879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564849de3e00 msgr2=0x564849def550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:54.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.431+0000 7fdd4a800700 1 --2- 172.21.15.131:0/3221818879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564849de3e00 0x564849def550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.431+0000 7fdd4a800700 1 -- 172.21.15.131:0/3221818879 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd3801f040 con 0x564849dde890 2024-05-08T22:16:54.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.431+0000 7fdd49e00700 1 --2- 172.21.15.131:0/3221818879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564849de3e00 0x564849def550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:16:54.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.431+0000 7fdd4a800700 1 --2- 172.21.15.131:0/3221818879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564849dde890 0x564849def010 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x564849dfad70 tx=0x7fdd3c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d865b2a497e8051 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:54.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.431+0000 7fdd2be00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:16:54.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.432+0000 7fdd2be00700 1 -- 172.21.15.131:0/3221818879 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c004440 con 0x564849dde890 2024-05-08T22:16:54.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.432+0000 7fdd2be00700 1 -- 172.21.15.131:0/3221818879 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fdd3c025250 con 0x564849dde890 2024-05-08T22:16:54.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.432+0000 7fdd64febb80 1 -- 172.21.15.131:0/3221818879 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd44004dd0 con 0x564849dde890 2024-05-08T22:16:54.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.433+0000 7fdd2be00700 1 -- 172.21.15.131:0/3221818879 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd3c025410 con 0x564849dde890 2024-05-08T22:16:54.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.433+0000 7fdd64febb80 10 client.26749 fetch_fsmap learned FSMap version 30 2024-05-08T22:16:54.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.433+0000 7fdd64febb80 1 -- 172.21.15.131:0/3221818879 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564849bb3cd0 con 0x564849dde890 2024-05-08T22:16:54.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.438+0000 7fdd2be00700 1 -- 172.21.15.131:0/3221818879 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd44004dd0 con 0x564849dde890 2024-05-08T22:16:54.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.438+0000 7fdd2be00700 1 -- 172.21.15.131:0/3221818879 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fdd3c011440 con 0x564849dde890 2024-05-08T22:16:54.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.438+0000 7fdd64febb80 10 client.26749 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:16:54.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.438+0000 7fdd64febb80 10 client.26749 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:16:54.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.438+0000 7fdd64febb80 20 client.26749 populate_metadata read hostname 'smithi131' 2024-05-08T22:16:54.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.438+0000 7fdd64febb80 1 --2- 172.21.15.131:0/3221818879 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x564849e4e6e0 0x564849e50b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.438+0000 7fdd64febb80 4 client.26749 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:16:54.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.438+0000 7fdd64febb80 1 -- 172.21.15.131:0/3221818879 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564849d26ec0 con 0x564849e4e6e0 2024-05-08T22:16:54.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.440+0000 7fdd49e00700 1 --2- 172.21.15.131:0/3221818879 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x564849e4e6e0 0x564849e50b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.442+0000 7fdd49e00700 1 --2- 172.21.15.131:0/3221818879 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x564849e4e6e0 0x564849e50b00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=daf4204ab71b27c3 server_cookie=cf28aa88b10889f0 in_seq=0 out_seq=0 2024-05-08T22:16:54.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.442+0000 7fdd2be00700 10 client.26749 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:16:54.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.442+0000 7fdd2be00700 1 -- 172.21.15.131:0/3221818879 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564849d26ec0 con 0x564849e4e6e0 2024-05-08T22:16:54.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.443+0000 7fdd2be00700 10 client.26749 handle_command_reply: tid=0 2024-05-08T22:16:54.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.443+0000 7fdd64febb80 1 client.26749 shutdown 2024-05-08T22:16:54.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.444+0000 7fdd64febb80 1 -- 172.21.15.131:0/3221818879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564849dde890 msgr2=0x564849def010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:54.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.444+0000 7fdd64febb80 1 --2- 172.21.15.131:0/3221818879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564849dde890 0x564849def010 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x564849dfad70 tx=0x7fdd3c004750 comp rx=0 tx=0).stop 2024-05-08T22:16:54.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.444+0000 7fdd64febb80 1 -- 172.21.15.131:0/3221818879 shutdown_connections 2024-05-08T22:16:54.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.444+0000 7fdd64febb80 1 --2- 172.21.15.131:0/3221818879 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x564849e4e6e0 0x564849e50b00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.444+0000 7fdd64febb80 1 --2- 172.21.15.131:0/3221818879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564849de4720 0x564849ddf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.444+0000 7fdd64febb80 1 --2- 172.21.15.131:0/3221818879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564849dde890 0x564849def010 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.444+0000 7fdd2be00700 0 client.26749 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:16:54.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.444+0000 7fdd64febb80 1 --2- 172.21.15.131:0/3221818879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564849de3e00 0x564849def550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.445+0000 7fdd64febb80 1 -- 172.21.15.131:0/3221818879 >> 172.21.15.131:0/3221818879 conn(0x564849d3c210 msgr2=0x564849de6290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:54.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.445+0000 7fdd64febb80 1 -- 172.21.15.131:0/3221818879 shutdown_connections 2024-05-08T22:16:54.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.445+0000 7fdd64febb80 1 -- 172.21.15.131:0/3221818879 wait complete. 2024-05-08T22:16:54.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.446+0000 7fdd64febb80 20 client.26749 trim_cache size 0 max 16384 2024-05-08T22:16:54.450 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:16:54.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.447+0000 7fdd51400700 1 -- 172.21.15.131:0/1754768649 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fdd300a1310 msgr2=0x7fdd300a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:54.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.447+0000 7fdd51400700 1 --2- 172.21.15.131:0/1754768649 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fdd300a1310 0x7fdd300a37d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c005e10 tx=0x7fdd3c005d60 comp rx=0 tx=0).stop 2024-05-08T22:16:54.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.447+0000 7fdd51400700 1 -- 172.21.15.131:0/1754768649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd4c105e30 msgr2=0x7fdd4c19d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:54.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.447+0000 7fdd51400700 1 --2- 172.21.15.131:0/1754768649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd4c105e30 0x7fdd4c19d940 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdd440077f0 tx=0x7fdd44007820 comp rx=0 tx=0).stop 2024-05-08T22:16:54.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.447+0000 7fdd51400700 1 -- 172.21.15.131:0/1754768649 shutdown_connections 2024-05-08T22:16:54.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.447+0000 7fdd51400700 1 --2- 172.21.15.131:0/1754768649 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fdd300a1310 0x7fdd300a37d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.447+0000 7fdd51400700 1 --2- 172.21.15.131:0/1754768649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd4c105e30 0x7fdd4c19d940 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.447+0000 7fdd51400700 1 --2- 172.21.15.131:0/1754768649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd4c068590 0x7fdd4c1a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.447+0000 7fdd51400700 1 --2- 172.21.15.131:0/1754768649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd4c103930 0x7fdd4c1a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.447+0000 7fdd51400700 1 -- 172.21.15.131:0/1754768649 >> 172.21.15.131:0/1754768649 conn(0x7fdd4c076f40 msgr2=0x7fdd4c0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:54.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.448+0000 7fdd51400700 1 -- 172.21.15.131:0/1754768649 shutdown_connections 2024-05-08T22:16:54.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.448+0000 7fdd51400700 1 -- 172.21.15.131:0/1754768649 wait complete. 2024-05-08T22:16:54.480 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T22:16:54.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:54 smithi157 ceph-mon[77746]: pgmap v297: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 579 op/s 2024-05-08T22:16:54.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:54 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/4197012899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:16:54.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.827+0000 7fa1d5a00700 1 -- 172.21.15.131:0/2481525464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1d010fb10 msgr2=0x7fa1d01084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:54.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.827+0000 7fa1d5a00700 1 --2- 172.21.15.131:0/2481525464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1d010fb10 0x7fa1d01084e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc0098f0 tx=0x7fa1bc009c00 comp rx=0 tx=0).stop 2024-05-08T22:16:54.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.828+0000 7fa1d5a00700 1 -- 172.21.15.131:0/2481525464 shutdown_connections 2024-05-08T22:16:54.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.828+0000 7fa1d5a00700 1 --2- 172.21.15.131:0/2481525464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1d011a780 0x7fa1d011cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.828+0000 7fa1d5a00700 1 --2- 172.21.15.131:0/2481525464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1d010f6a0 0x7fa1d0107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.828+0000 7fa1d5a00700 1 --2- 172.21.15.131:0/2481525464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1d010fb10 0x7fa1d01084e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.828+0000 7fa1d5a00700 1 -- 172.21.15.131:0/2481525464 >> 172.21.15.131:0/2481525464 conn(0x7fa1d006bff0 msgr2=0x7fa1d006c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:54.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.828+0000 7fa1d5a00700 1 -- 172.21.15.131:0/2481525464 shutdown_connections 2024-05-08T22:16:54.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.828+0000 7fa1d5a00700 1 -- 172.21.15.131:0/2481525464 wait complete. 2024-05-08T22:16:54.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.830+0000 7fa1d5a00700 1 Processor -- start 2024-05-08T22:16:54.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.830+0000 7fa1d5a00700 1 -- start start 2024-05-08T22:16:54.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.831+0000 7fa1d5a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1d010f6a0 0x7fa1d01a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.831+0000 7fa1d5a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1d010fb10 0x7fa1d01a6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.831+0000 7fa1d5a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1d011a780 0x7fa1d01aab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.831+0000 7fa1d5a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d01ab150 con 0x7fa1d011a780 2024-05-08T22:16:54.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.831+0000 7fa1d5a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d01ab2c0 con 0x7fa1d010f6a0 2024-05-08T22:16:54.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.831+0000 7fa1d5a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d01ab480 con 0x7fa1d010fb10 2024-05-08T22:16:54.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.831+0000 7fa1cea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1d010f6a0 0x7fa1d01a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.831+0000 7fa1cea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1d010f6a0 0x7fa1d01a6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:44676/0 (socket says 172.21.15.131:44676) 2024-05-08T22:16:54.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.831+0000 7fa1cea00700 1 -- 172.21.15.131:0/1061035952 learned_addr learned my addr 172.21.15.131:0/1061035952 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:54.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.831+0000 7fa1ce000700 1 --2- 172.21.15.131:0/1061035952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1d010fb10 0x7fa1d01a6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.832+0000 7fa1cf400700 1 --2- 172.21.15.131:0/1061035952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1d011a780 0x7fa1d01aab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.832+0000 7fa1cf400700 1 -- 172.21.15.131:0/1061035952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1d010fb10 msgr2=0x7fa1d01a6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:54.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.832+0000 7fa1cf400700 1 --2- 172.21.15.131:0/1061035952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1d010fb10 0x7fa1d01a6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.832+0000 7fa1cf400700 1 -- 172.21.15.131:0/1061035952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1d010f6a0 msgr2=0x7fa1d01a6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:54.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.832+0000 7fa1cf400700 1 --2- 172.21.15.131:0/1061035952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1d010f6a0 0x7fa1d01a6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.832+0000 7fa1cf400700 1 -- 172.21.15.131:0/1061035952 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1bc009520 con 0x7fa1d011a780 2024-05-08T22:16:54.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.833+0000 7fa1cf400700 1 --2- 172.21.15.131:0/1061035952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1d011a780 0x7fa1d01aab10 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800cc20 tx=0x7fa1c800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:54.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.836+0000 7fa1c7400700 1 -- 172.21.15.131:0/1061035952 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c800abe0 con 0x7fa1d011a780 2024-05-08T22:16:54.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.836+0000 7fa1c7400700 1 -- 172.21.15.131:0/1061035952 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1c800ad40 con 0x7fa1d011a780 2024-05-08T22:16:54.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.836+0000 7fa1d5a00700 1 -- 172.21.15.131:0/1061035952 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1d01ab760 con 0x7fa1d011a780 2024-05-08T22:16:54.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.836+0000 7fa1c7400700 1 -- 172.21.15.131:0/1061035952 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8011660 con 0x7fa1d011a780 2024-05-08T22:16:54.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.837+0000 7fa1d5a00700 1 -- 172.21.15.131:0/1061035952 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d01b2fa0 con 0x7fa1d011a780 2024-05-08T22:16:54.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:54 smithi132 ceph-mon[78180]: pgmap v297: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 579 op/s 2024-05-08T22:16:54.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:54 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/4197012899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:16:54.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.845+0000 7fa1c7400700 1 -- 172.21.15.131:0/1061035952 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fa1c8012420 con 0x7fa1d011a780 2024-05-08T22:16:54.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.848+0000 7fa1c7400700 1 --2- 172.21.15.131:0/1061035952 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa1b40a13e0 0x7fa1b40a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.848+0000 7fa1c7400700 1 -- 172.21.15.131:0/1061035952 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fa1c80cd130 con 0x7fa1d011a780 2024-05-08T22:16:54.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.848+0000 7fa1cea00700 1 --2- 172.21.15.131:0/1061035952 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa1b40a13e0 0x7fa1b40a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.849+0000 7fa1e95dbb80 1 Processor -- start 2024-05-08T22:16:54.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.849+0000 7fa1e95dbb80 1 -- start start 2024-05-08T22:16:54.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.849+0000 7fa1cea00700 1 --2- 172.21.15.131:0/1061035952 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa1b40a13e0 0x7fa1b40a38a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000bcf0 tx=0x7fa1c000b2c0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:54.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.849+0000 7fa1e95dbb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a8d3bce00 0x556a8d3bd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.849+0000 7fa1e95dbb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a8d3bd720 0x556a8d3b7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.849+0000 7fa1e95dbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a8d3b7890 0x556a8d3b7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.849+0000 7fa1e95dbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a8d30e150 con 0x556a8d3b7890 2024-05-08T22:16:54.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.849+0000 7fa1e95dbb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a8d30bb10 con 0x556a8d3bce00 2024-05-08T22:16:54.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.849+0000 7fa1e95dbb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a8d307400 con 0x556a8d3bd720 2024-05-08T22:16:54.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.850+0000 7fa1cea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a8d3bce00 0x556a8d3bd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.850+0000 7fa1ce000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a8d3bd720 0x556a8d3b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.850+0000 7fa1cea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a8d3bce00 0x556a8d3bd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:44682/0 (socket says 172.21.15.131:44682) 2024-05-08T22:16:54.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.850+0000 7fa1cea00700 1 -- 172.21.15.131:0/1470557733 learned_addr learned my addr 172.21.15.131:0/1470557733 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:54.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.850+0000 7fa1cea00700 1 -- 172.21.15.131:0/1470557733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a8d3bd720 msgr2=0x556a8d3b7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:54.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.850+0000 7fa1cea00700 1 --2- 172.21.15.131:0/1470557733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a8d3bd720 0x556a8d3b7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.850+0000 7fa1cea00700 1 -- 172.21.15.131:0/1470557733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a8d3b7890 msgr2=0x556a8d3b7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:54.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.850+0000 7fa1cea00700 1 --2- 172.21.15.131:0/1470557733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a8d3b7890 0x556a8d3b7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.850+0000 7fa1cea00700 1 -- 172.21.15.131:0/1470557733 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a8d1883b0 con 0x556a8d3bce00 2024-05-08T22:16:54.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.851+0000 7fa1cea00700 1 --2- 172.21.15.131:0/1470557733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a8d3bce00 0x556a8d3bd1e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000ff00 tx=0x7fa1c0012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26f60f38aceafb18 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:54.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.852+0000 7fa1bbe00700 1 -- 172.21.15.131:0/1470557733 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0011bb0 con 0x556a8d3bce00 2024-05-08T22:16:54.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.852+0000 7fa1bbe00700 1 -- 172.21.15.131:0/1470557733 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1c0011d10 con 0x556a8d3bce00 2024-05-08T22:16:54.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.852+0000 7fa1bbe00700 1 -- 172.21.15.131:0/1470557733 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c001d650 con 0x556a8d3bce00 2024-05-08T22:16:54.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.853+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/1470557733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a8d3bce00 msgr2=0x556a8d3bd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:54.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.853+0000 7fa1e95dbb80 1 --2- 172.21.15.131:0/1470557733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a8d3bce00 0x556a8d3bd1e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000ff00 tx=0x7fa1c0012000 comp rx=0 tx=0).stop 2024-05-08T22:16:54.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.853+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/1470557733 shutdown_connections 2024-05-08T22:16:54.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.853+0000 7fa1e95dbb80 1 --2- 172.21.15.131:0/1470557733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a8d3b7890 0x556a8d3b7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.853+0000 7fa1e95dbb80 1 --2- 172.21.15.131:0/1470557733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a8d3bce00 0x556a8d3bd1e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.853+0000 7fa1e95dbb80 1 --2- 172.21.15.131:0/1470557733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a8d3bd720 0x556a8d3b7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.853+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/1470557733 >> 172.21.15.131:0/1470557733 conn(0x556a8d315210 msgr2=0x556a8d3b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:54.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.853+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/1470557733 shutdown_connections 2024-05-08T22:16:54.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.854+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/1470557733 wait complete. 2024-05-08T22:16:54.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.854+0000 7fa1e95dbb80 1 -- start start 2024-05-08T22:16:54.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.855+0000 7fa1e95dbb80 1 Processor -- start 2024-05-08T22:16:54.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.856+0000 7fa1e95dbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a8d3b7890 0x556a8d3c0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.856+0000 7fa1e95dbb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a8d3bce00 0x556a8d3c1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.856+0000 7fa1e95dbb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a8d3bd720 0x556a8d3c17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.856+0000 7fa1e95dbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a8d2a3560 con 0x556a8d3b7890 2024-05-08T22:16:54.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.856+0000 7fa1e95dbb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a8d2a3200 con 0x556a8d3bce00 2024-05-08T22:16:54.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.856+0000 7fa1e95dbb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a8d18a6a0 con 0x556a8d3bd720 2024-05-08T22:16:54.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.856+0000 7fa1e95dbb80 10 client.0 _ll_register_callbacks cb 0x556a8d2acb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:16:54.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.856+0000 7fa1cf400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a8d3bd720 0x556a8d3c17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.856+0000 7fa1cf400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a8d3bd720 0x556a8d3c17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46010/0 (socket says 172.21.15.131:46010) 2024-05-08T22:16:54.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.856+0000 7fa1cf400700 1 -- 172.21.15.131:0/2300788516 learned_addr learned my addr 172.21.15.131:0/2300788516 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:54.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.857+0000 7fa1cea00700 1 --2- 172.21.15.131:0/2300788516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a8d3b7890 0x556a8d3c0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.857+0000 7fa1ce000700 1 --2- 172.21.15.131:0/2300788516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a8d3bce00 0x556a8d3c1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.857+0000 7fa1cf400700 1 -- 172.21.15.131:0/2300788516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a8d3bce00 msgr2=0x556a8d3c1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:54.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.857+0000 7fa1cf400700 1 --2- 172.21.15.131:0/2300788516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a8d3bce00 0x556a8d3c1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.858+0000 7fa1cf400700 1 -- 172.21.15.131:0/2300788516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a8d3b7890 msgr2=0x556a8d3c0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:54.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.859+0000 7fa1cf400700 1 --2- 172.21.15.131:0/2300788516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a8d3b7890 0x556a8d3c0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.859+0000 7fa1cf400700 1 -- 172.21.15.131:0/2300788516 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c000fbe0 con 0x556a8d3bd720 2024-05-08T22:16:54.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.859+0000 7fa1cf400700 1 --2- 172.21.15.131:0/2300788516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a8d3bd720 0x556a8d3c17c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa1c80ca600 tx=0x7fa1c80999f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51e0e4793fed9235 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:54.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.859+0000 7fa1b8c00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:16:54.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.859+0000 7fa1b8c00700 1 -- 172.21.15.131:0/2300788516 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c80cade0 con 0x556a8d3bd720 2024-05-08T22:16:54.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.859+0000 7fa1b8c00700 1 -- 172.21.15.131:0/2300788516 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fa1c80a23e0 con 0x556a8d3bd720 2024-05-08T22:16:54.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.860+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/2300788516 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a8d260270 con 0x556a8d3bd720 2024-05-08T22:16:54.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.860+0000 7fa1ce000700 1 --2- 172.21.15.131:0/2300788516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a8d3bce00 0x556a8d3c1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:16:54.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.860+0000 7fa1cea00700 1 --2- 172.21.15.131:0/2300788516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a8d3b7890 0x556a8d3c0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:16:54.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.860+0000 7fa1b8c00700 1 -- 172.21.15.131:0/2300788516 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1c809a1c0 con 0x556a8d3bd720 2024-05-08T22:16:54.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.860+0000 7fa1e95dbb80 10 client.26761 fetch_fsmap learned FSMap version 30 2024-05-08T22:16:54.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.860+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/2300788516 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556a8d134b60 con 0x556a8d3bd720 2024-05-08T22:16:54.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.861+0000 7fa1b8c00700 1 -- 172.21.15.131:0/2300788516 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556a8d260270 con 0x556a8d3bd720 2024-05-08T22:16:54.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.862+0000 7fa1b8c00700 1 -- 172.21.15.131:0/2300788516 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa1c80a1f50 con 0x556a8d3bd720 2024-05-08T22:16:54.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.862+0000 7fa1e95dbb80 10 client.26761 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:16:54.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.862+0000 7fa1e95dbb80 10 client.26761 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:16:54.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.862+0000 7fa1e95dbb80 20 client.26761 populate_metadata read hostname 'smithi131' 2024-05-08T22:16:54.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.863+0000 7fa1e95dbb80 1 --2- 172.21.15.131:0/2300788516 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x556a8d3e2170 0x556a8d3e4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.864+0000 7fa1e95dbb80 4 client.26761 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:16:54.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.864+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/2300788516 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a8d386810 con 0x556a8d3e2170 2024-05-08T22:16:54.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.865+0000 7fa1cea00700 1 --2- 172.21.15.131:0/2300788516 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x556a8d3e2170 0x556a8d3e4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.866+0000 7fa1cea00700 1 --2- 172.21.15.131:0/2300788516 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x556a8d3e2170 0x556a8d3e4590 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28587d8df1d4a1bb server_cookie=57be99c365c8dd9d in_seq=0 out_seq=0 2024-05-08T22:16:54.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.866+0000 7fa1b8c00700 10 client.26761 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:16:54.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.867+0000 7fa1b8c00700 1 -- 172.21.15.131:0/2300788516 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556a8d386810 con 0x556a8d3e2170 2024-05-08T22:16:54.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.868+0000 7fa1b8c00700 10 client.26761 handle_command_reply: tid=0 2024-05-08T22:16:54.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.868+0000 7fa1e95dbb80 1 client.26761 shutdown 2024-05-08T22:16:54.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.868+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/2300788516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a8d3bd720 msgr2=0x556a8d3c17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:54.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.868+0000 7fa1e95dbb80 1 --2- 172.21.15.131:0/2300788516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a8d3bd720 0x556a8d3c17c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa1c80ca600 tx=0x7fa1c80999f0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.868+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/2300788516 shutdown_connections 2024-05-08T22:16:54.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.868+0000 7fa1e95dbb80 1 --2- 172.21.15.131:0/2300788516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a8d3bd720 0x556a8d3c17c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.868+0000 7fa1e95dbb80 1 --2- 172.21.15.131:0/2300788516 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x556a8d3e2170 0x556a8d3e4590 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.868+0000 7fa1e95dbb80 1 --2- 172.21.15.131:0/2300788516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a8d3b7890 0x556a8d3c0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.868+0000 7fa1e95dbb80 1 --2- 172.21.15.131:0/2300788516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a8d3bce00 0x556a8d3c1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.868+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/2300788516 >> 172.21.15.131:0/2300788516 conn(0x556a8d315210 msgr2=0x556a8d3b15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:54.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.869+0000 7fa1b8c00700 0 client.26761 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:16:54.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.869+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/2300788516 shutdown_connections 2024-05-08T22:16:54.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.869+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/2300788516 wait complete. 2024-05-08T22:16:54.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.871+0000 7fa1e95dbb80 20 client.26761 trim_cache size 0 max 16384 2024-05-08T22:16:54.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:54 smithi131 ceph-mon[37260]: pgmap v297: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 579 op/s 2024-05-08T22:16:54.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:54 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/4197012899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:16:54.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.897+0000 7fa1e95dbb80 1 Processor -- start 2024-05-08T22:16:54.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.897+0000 7fa1e95dbb80 1 -- start start 2024-05-08T22:16:54.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.898+0000 7fa1e95dbb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a8d3b7890 0x556a8d3b7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.899+0000 7fa1e95dbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a8d3bce00 0x556a8d3ddb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.899+0000 7fa1e95dbb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a8d3bd720 0x556a8d3b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.899+0000 7fa1e95dbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a8d3b3800 con 0x556a8d3bce00 2024-05-08T22:16:54.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.899+0000 7fa1e95dbb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a8d3e97a0 con 0x556a8d3bd720 2024-05-08T22:16:54.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.899+0000 7fa1e95dbb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a8d3c2200 con 0x556a8d3b7890 2024-05-08T22:16:54.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.899+0000 7fa1cf400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a8d3bd720 0x556a8d3b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.899+0000 7fa1cea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a8d3b7890 0x556a8d3b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.899+0000 7fa1cea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a8d3b7890 0x556a8d3b7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46022/0 (socket says 172.21.15.131:46022) 2024-05-08T22:16:54.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.899+0000 7fa1cea00700 1 -- 172.21.15.131:0/3437546944 learned_addr learned my addr 172.21.15.131:0/3437546944 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:54.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.899+0000 7fa1ce000700 1 --2- 172.21.15.131:0/3437546944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a8d3bce00 0x556a8d3ddb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.900+0000 7fa1cf400700 1 -- 172.21.15.131:0/3437546944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a8d3b7890 msgr2=0x556a8d3b7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:54.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.900+0000 7fa1cf400700 1 --2- 172.21.15.131:0/3437546944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a8d3b7890 0x556a8d3b7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.900+0000 7fa1cf400700 1 -- 172.21.15.131:0/3437546944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a8d3bce00 msgr2=0x556a8d3ddb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:54.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.900+0000 7fa1cf400700 1 --2- 172.21.15.131:0/3437546944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a8d3bce00 0x556a8d3ddb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.900+0000 7fa1cf400700 1 -- 172.21.15.131:0/3437546944 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a8d2cd560 con 0x556a8d3bd720 2024-05-08T22:16:54.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.900+0000 7fa1cf400700 1 --2- 172.21.15.131:0/3437546944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a8d3bd720 0x556a8d3b1d20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa1c80caa90 tx=0x7fa1c809a0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29163ff3c328c454 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:54.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.900+0000 7fa1c4c00700 1 -- 172.21.15.131:0/3437546944 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c80a23e0 con 0x556a8d3bd720 2024-05-08T22:16:54.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.900+0000 7fa1c4c00700 1 -- 172.21.15.131:0/3437546944 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1c8099450 con 0x556a8d3bd720 2024-05-08T22:16:54.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.900+0000 7fa1c4c00700 1 -- 172.21.15.131:0/3437546944 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c80a20c0 con 0x556a8d3bd720 2024-05-08T22:16:54.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.902+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/3437546944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a8d3bd720 msgr2=0x556a8d3b1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:54.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.902+0000 7fa1e95dbb80 1 --2- 172.21.15.131:0/3437546944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a8d3bd720 0x556a8d3b1d20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa1c80caa90 tx=0x7fa1c809a0a0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.904+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/3437546944 shutdown_connections 2024-05-08T22:16:54.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.904+0000 7fa1e95dbb80 1 --2- 172.21.15.131:0/3437546944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a8d3bd720 0x556a8d3b1d20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.904+0000 7fa1e95dbb80 1 --2- 172.21.15.131:0/3437546944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a8d3b7890 0x556a8d3b7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.904+0000 7fa1e95dbb80 1 --2- 172.21.15.131:0/3437546944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a8d3bce00 0x556a8d3ddb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.904+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/3437546944 >> 172.21.15.131:0/3437546944 conn(0x556a8d315210 msgr2=0x556a8d3bf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:54.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.904+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/3437546944 shutdown_connections 2024-05-08T22:16:54.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.904+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/3437546944 wait complete. 2024-05-08T22:16:54.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.905+0000 7fa1e95dbb80 1 -- start start 2024-05-08T22:16:54.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.906+0000 7fa1e95dbb80 1 Processor -- start 2024-05-08T22:16:54.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.907+0000 7fa1e95dbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a8d3b7890 0x556a8d3d7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.907+0000 7fa1e95dbb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a8d3bce00 0x556a8d3d75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.907+0000 7fa1e95dbb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a8d3bd720 0x556a8d3d7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.907+0000 7fa1e95dbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a8d437670 con 0x556a8d3b7890 2024-05-08T22:16:54.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.907+0000 7fa1e95dbb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a8d4377b0 con 0x556a8d3bce00 2024-05-08T22:16:54.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.907+0000 7fa1e95dbb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a8d4378f0 con 0x556a8d3bd720 2024-05-08T22:16:54.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.907+0000 7fa1e95dbb80 10 client.0 _ll_register_callbacks cb 0x556a8d2acb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:16:54.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.909+0000 7fa1cf400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a8d3bd720 0x556a8d3d7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.909+0000 7fa1cf400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a8d3bd720 0x556a8d3d7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46038/0 (socket says 172.21.15.131:46038) 2024-05-08T22:16:54.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.909+0000 7fa1cf400700 1 -- 172.21.15.131:0/2345023556 learned_addr learned my addr 172.21.15.131:0/2345023556 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:54.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.909+0000 7fa1ce000700 1 --2- 172.21.15.131:0/2345023556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a8d3bce00 0x556a8d3d75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.910+0000 7fa1cf400700 1 -- 172.21.15.131:0/2345023556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a8d3bce00 msgr2=0x556a8d3d75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:54.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.910+0000 7fa1cf400700 1 --2- 172.21.15.131:0/2345023556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a8d3bce00 0x556a8d3d75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.910+0000 7fa1cf400700 1 -- 172.21.15.131:0/2345023556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a8d3b7890 msgr2=0x556a8d3d7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:54.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.910+0000 7fa1cf400700 1 --2- 172.21.15.131:0/2345023556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a8d3b7890 0x556a8d3d7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.910+0000 7fa1cf400700 1 -- 172.21.15.131:0/2345023556 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a8d134b60 con 0x556a8d3bd720 2024-05-08T22:16:54.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.910+0000 7fa1ce000700 1 --2- 172.21.15.131:0/2345023556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a8d3bce00 0x556a8d3d75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:16:54.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.911+0000 7fa1cf400700 1 --2- 172.21.15.131:0/2345023556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a8d3bd720 0x556a8d3d7b00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8011980 tx=0x7fa1c8018f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b3a63e1fffe4561 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:54.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.911+0000 7fa1b8c00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:16:54.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.911+0000 7fa1b8c00700 1 -- 172.21.15.131:0/2345023556 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8012950 con 0x556a8d3bd720 2024-05-08T22:16:54.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.911+0000 7fa1b8c00700 1 -- 172.21.15.131:0/2345023556 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fa1c80a23e0 con 0x556a8d3bd720 2024-05-08T22:16:54.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.911+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/2345023556 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1c0011d10 con 0x556a8d3bd720 2024-05-08T22:16:54.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.912+0000 7fa1b8c00700 1 -- 172.21.15.131:0/2345023556 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1c809a1c0 con 0x556a8d3bd720 2024-05-08T22:16:54.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.912+0000 7fa1e95dbb80 10 client.26767 fetch_fsmap learned FSMap version 30 2024-05-08T22:16:54.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.912+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/2345023556 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556a8d18ccd0 con 0x556a8d3bd720 2024-05-08T22:16:54.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.912+0000 7fa1b8c00700 1 -- 172.21.15.131:0/2345023556 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1c0011d10 con 0x556a8d3bd720 2024-05-08T22:16:54.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.912+0000 7fa1b8c00700 1 -- 172.21.15.131:0/2345023556 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa1c8094c40 con 0x556a8d3bd720 2024-05-08T22:16:54.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.912+0000 7fa1e95dbb80 10 client.26767 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:16:54.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.912+0000 7fa1e95dbb80 10 client.26767 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:16:54.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.912+0000 7fa1e95dbb80 20 client.26767 populate_metadata read hostname 'smithi131' 2024-05-08T22:16:54.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.913+0000 7fa1e95dbb80 1 --2- 172.21.15.131:0/2345023556 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x556a8d438b90 0x556a8d43af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:54.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.913+0000 7fa1e95dbb80 4 client.26767 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:16:54.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.913+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/2345023556 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556a8d2ffec0 con 0x556a8d438b90 2024-05-08T22:16:54.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.915+0000 7fa1cea00700 1 --2- 172.21.15.131:0/2345023556 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x556a8d438b90 0x556a8d43af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:54.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.916+0000 7fa1cea00700 1 --2- 172.21.15.131:0/2345023556 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x556a8d438b90 0x556a8d43af80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e3d6247670ac8e7 server_cookie=a8198a31a91c006d in_seq=0 out_seq=0 2024-05-08T22:16:54.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.916+0000 7fa1b8c00700 10 client.26767 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:16:54.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.917+0000 7fa1b8c00700 1 -- 172.21.15.131:0/2345023556 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556a8d2ffec0 con 0x556a8d438b90 2024-05-08T22:16:54.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.917+0000 7fa1b8c00700 10 client.26767 handle_command_reply: tid=0 2024-05-08T22:16:54.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.917+0000 7fa1e95dbb80 1 client.26767 shutdown 2024-05-08T22:16:54.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.917+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/2345023556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a8d3bd720 msgr2=0x556a8d3d7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:54.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.917+0000 7fa1e95dbb80 1 --2- 172.21.15.131:0/2345023556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a8d3bd720 0x556a8d3d7b00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8011980 tx=0x7fa1c8018f70 comp rx=0 tx=0).stop 2024-05-08T22:16:54.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.918+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/2345023556 shutdown_connections 2024-05-08T22:16:54.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.918+0000 7fa1e95dbb80 1 --2- 172.21.15.131:0/2345023556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a8d3bd720 0x556a8d3d7b00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.918+0000 7fa1e95dbb80 1 --2- 172.21.15.131:0/2345023556 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x556a8d438b90 0x556a8d43af80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.918+0000 7fa1e95dbb80 1 --2- 172.21.15.131:0/2345023556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a8d3b7890 0x556a8d3d7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.918+0000 7fa1e95dbb80 1 --2- 172.21.15.131:0/2345023556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a8d3bce00 0x556a8d3d75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.918+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/2345023556 >> 172.21.15.131:0/2345023556 conn(0x556a8d315210 msgr2=0x556a8d3bbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:54.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.918+0000 7fa1b8c00700 0 client.26767 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:16:54.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.919+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/2345023556 shutdown_connections 2024-05-08T22:16:54.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.919+0000 7fa1e95dbb80 1 -- 172.21.15.131:0/2345023556 wait complete. 2024-05-08T22:16:54.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.919+0000 7fa1e95dbb80 20 client.26767 trim_cache size 0 max 16384 2024-05-08T22:16:54.922 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:16:54.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.921+0000 7fa1d5a00700 1 -- 172.21.15.131:0/1061035952 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa1b40a13e0 msgr2=0x7fa1b40a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:54.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.921+0000 7fa1d5a00700 1 --2- 172.21.15.131:0/1061035952 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa1b40a13e0 0x7fa1b40a38a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000bcf0 tx=0x7fa1c000b2c0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.921+0000 7fa1d5a00700 1 -- 172.21.15.131:0/1061035952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1d011a780 msgr2=0x7fa1d01aab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:54.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.921+0000 7fa1d5a00700 1 --2- 172.21.15.131:0/1061035952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1d011a780 0x7fa1d01aab10 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800cc20 tx=0x7fa1c800a060 comp rx=0 tx=0).stop 2024-05-08T22:16:54.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.924+0000 7fa1d5a00700 1 -- 172.21.15.131:0/1061035952 shutdown_connections 2024-05-08T22:16:54.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.924+0000 7fa1d5a00700 1 --2- 172.21.15.131:0/1061035952 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa1b40a13e0 0x7fa1b40a38a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.924+0000 7fa1d5a00700 1 --2- 172.21.15.131:0/1061035952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1d011a780 0x7fa1d01aab10 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.924+0000 7fa1d5a00700 1 --2- 172.21.15.131:0/1061035952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1d010f6a0 0x7fa1d01a6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.924+0000 7fa1d5a00700 1 --2- 172.21.15.131:0/1061035952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1d010fb10 0x7fa1d01a6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:54.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.924+0000 7fa1d5a00700 1 -- 172.21.15.131:0/1061035952 >> 172.21.15.131:0/1061035952 conn(0x7fa1d006bff0 msgr2=0x7fa1d0118770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:54.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.924+0000 7fa1d5a00700 1 -- 172.21.15.131:0/1061035952 shutdown_connections 2024-05-08T22:16:54.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:54.924+0000 7fa1d5a00700 1 -- 172.21.15.131:0/1061035952 wait complete. 2024-05-08T22:16:54.962 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T22:16:55.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.316+0000 7f1c15400700 1 -- 172.21.15.131:0/11228932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c10105030 msgr2=0x7f1c1010cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:55.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.316+0000 7f1c15400700 1 --2- 172.21.15.131:0/11228932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c10105030 0x7f1c1010cbb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1c0800b0a0 tx=0x7f1c0800b3b0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.317+0000 7f1c15400700 1 -- 172.21.15.131:0/11228932 shutdown_connections 2024-05-08T22:16:55.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.317+0000 7f1c15400700 1 --2- 172.21.15.131:0/11228932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c10105030 0x7f1c1010cbb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.317+0000 7f1c15400700 1 --2- 172.21.15.131:0/11228932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c100fff10 0x7f1c101002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.317+0000 7f1c15400700 1 --2- 172.21.15.131:0/11228932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c101008c0 0x7f1c10104900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.317+0000 7f1c15400700 1 -- 172.21.15.131:0/11228932 >> 172.21.15.131:0/11228932 conn(0x7f1c100fb770 msgr2=0x7f1c100fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:55.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.317+0000 7f1c15400700 1 -- 172.21.15.131:0/11228932 shutdown_connections 2024-05-08T22:16:55.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.317+0000 7f1c15400700 1 -- 172.21.15.131:0/11228932 wait complete. 2024-05-08T22:16:55.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.319+0000 7f1c15400700 1 Processor -- start 2024-05-08T22:16:55.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.320+0000 7f1c15400700 1 -- start start 2024-05-08T22:16:55.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.321+0000 7f1c15400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c100fff10 0x7f1c1019d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:55.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.321+0000 7f1c15400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c10105030 0x7f1c10195350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:55.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.321+0000 7f1c15400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c1019d670 0x7f1c10195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:55.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.321+0000 7f1c15400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c10195ff0 con 0x7f1c1019d670 2024-05-08T22:16:55.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.321+0000 7f1c15400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c10196160 con 0x7f1c10105030 2024-05-08T22:16:55.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.321+0000 7f1c15400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c101b2b90 con 0x7f1c100fff10 2024-05-08T22:16:55.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.322+0000 7f1c0f400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c10105030 0x7f1c10195350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:55.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.322+0000 7f1c0f400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c10105030 0x7f1c10195350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:44714/0 (socket says 172.21.15.131:44714) 2024-05-08T22:16:55.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.322+0000 7f1c0f400700 1 -- 172.21.15.131:0/2059215479 learned_addr learned my addr 172.21.15.131:0/2059215479 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:55.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.323+0000 7f1c0fe00700 1 --2- 172.21.15.131:0/2059215479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c100fff10 0x7f1c1019d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:55.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.324+0000 7f1c14a00700 1 --2- 172.21.15.131:0/2059215479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c1019d670 0x7f1c10195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:55.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.324+0000 7f1c0fe00700 1 -- 172.21.15.131:0/2059215479 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c10105030 msgr2=0x7f1c10195350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:55.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.324+0000 7f1c0fe00700 1 --2- 172.21.15.131:0/2059215479 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c10105030 0x7f1c10195350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.324+0000 7f1c0fe00700 1 -- 172.21.15.131:0/2059215479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c1019d670 msgr2=0x7f1c10195890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:55.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.324+0000 7f1c0fe00700 1 --2- 172.21.15.131:0/2059215479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c1019d670 0x7f1c10195890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.324+0000 7f1c0fe00700 1 -- 172.21.15.131:0/2059215479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c08009d00 con 0x7f1c100fff10 2024-05-08T22:16:55.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.325+0000 7f1c0fe00700 1 --2- 172.21.15.131:0/2059215479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c100fff10 0x7f1c1019d130 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000b450 tx=0x7f1c0000b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:55.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.326+0000 7f1c0cc00700 1 -- 172.21.15.131:0/2059215479 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00011830 con 0x7f1c100fff10 2024-05-08T22:16:55.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.326+0000 7f1c0cc00700 1 -- 172.21.15.131:0/2059215479 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c0000be60 con 0x7f1c100fff10 2024-05-08T22:16:55.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.326+0000 7f1c0cc00700 1 -- 172.21.15.131:0/2059215479 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00010d50 con 0x7f1c100fff10 2024-05-08T22:16:55.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.327+0000 7f1c15400700 1 -- 172.21.15.131:0/2059215479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c101b2e40 con 0x7f1c100fff10 2024-05-08T22:16:55.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.327+0000 7f1c15400700 1 -- 172.21.15.131:0/2059215479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c101b3270 con 0x7f1c100fff10 2024-05-08T22:16:55.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.332+0000 7f1c0cc00700 1 -- 172.21.15.131:0/2059215479 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f1c00010380 con 0x7f1c100fff10 2024-05-08T22:16:55.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.333+0000 7f1c2a427b80 1 Processor -- start 2024-05-08T22:16:55.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.334+0000 7f1c2a427b80 1 -- start start 2024-05-08T22:16:55.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.334+0000 7f1c2a427b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560216410050 0x560216410430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:55.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.334+0000 7f1c2a427b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560216410970 0x56021640a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:55.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.334+0000 7f1c2a427b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56021640a890 0x56021640acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:55.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.334+0000 7f1c2a427b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560216361150 con 0x560216410970 2024-05-08T22:16:55.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.335+0000 7f1c2a427b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56021635eb10 con 0x56021640a890 2024-05-08T22:16:55.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.335+0000 7f1c2a427b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56021635a400 con 0x560216410050 2024-05-08T22:16:55.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.335+0000 7f1c0f400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56021640a890 0x56021640acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:55.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.335+0000 7f1c0f400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56021640a890 0x56021640acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:44722/0 (socket says 172.21.15.131:44722) 2024-05-08T22:16:55.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.335+0000 7f1c0f400700 1 -- 172.21.15.131:0/243687450 learned_addr learned my addr 172.21.15.131:0/243687450 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:55.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.335+0000 7f1c14a00700 1 --2- 172.21.15.131:0/243687450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560216410050 0x560216410430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:55.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.336+0000 7f1c0f400700 1 -- 172.21.15.131:0/243687450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560216410050 msgr2=0x560216410430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:55.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.336+0000 7f1c0f400700 1 --2- 172.21.15.131:0/243687450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560216410050 0x560216410430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.336+0000 7f1c0f400700 1 -- 172.21.15.131:0/243687450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560216410970 msgr2=0x56021640a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:55.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.336+0000 7f1c0f400700 1 --2- 172.21.15.131:0/243687450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560216410970 0x56021640a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.336+0000 7f1c0f400700 1 -- 172.21.15.131:0/243687450 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602161db3b0 con 0x56021640a890 2024-05-08T22:16:55.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.336+0000 7f1c14a00700 1 --2- 172.21.15.131:0/243687450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560216410050 0x560216410430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:16:55.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.336+0000 7f1c0f400700 1 --2- 172.21.15.131:0/243687450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56021640a890 0x56021640acd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc009d70 tx=0x7f1bfc011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac30150f60afc2cb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:55.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.337+0000 7f1c0cc00700 1 --2- 172.21.15.131:0/2059215479 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1bf40a13e0 0x7f1bf40a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:55.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.337+0000 7f1c0cc00700 1 -- 172.21.15.131:0/2059215479 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f1c000cc870 con 0x7f1c100fff10 2024-05-08T22:16:55.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.337+0000 7f1c14a00700 1 --2- 172.21.15.131:0/2059215479 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1bf40a13e0 0x7f1bf40a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:55.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.338+0000 7f1c05200700 1 -- 172.21.15.131:0/243687450 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc004a30 con 0x56021640a890 2024-05-08T22:16:55.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.338+0000 7f1c05200700 1 -- 172.21.15.131:0/243687450 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bfc0054e0 con 0x56021640a890 2024-05-08T22:16:55.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.338+0000 7f1c05200700 1 -- 172.21.15.131:0/243687450 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc0198f0 con 0x56021640a890 2024-05-08T22:16:55.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.339+0000 7f1c2a427b80 1 -- 172.21.15.131:0/243687450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56021640a890 msgr2=0x56021640acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:55.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.339+0000 7f1c2a427b80 1 --2- 172.21.15.131:0/243687450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56021640a890 0x56021640acd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc009d70 tx=0x7f1bfc011720 comp rx=0 tx=0).stop 2024-05-08T22:16:55.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.340+0000 7f1c14a00700 1 --2- 172.21.15.131:0/2059215479 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1bf40a13e0 0x7f1bf40a38a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1c0800b780 tx=0x7f1c08006210 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:55.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.340+0000 7f1c2a427b80 1 -- 172.21.15.131:0/243687450 shutdown_connections 2024-05-08T22:16:55.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.340+0000 7f1c2a427b80 1 --2- 172.21.15.131:0/243687450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56021640a890 0x56021640acd0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.340+0000 7f1c2a427b80 1 --2- 172.21.15.131:0/243687450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560216410050 0x560216410430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.340+0000 7f1c2a427b80 1 --2- 172.21.15.131:0/243687450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560216410970 0x56021640a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.340+0000 7f1c2a427b80 1 -- 172.21.15.131:0/243687450 >> 172.21.15.131:0/243687450 conn(0x560216368210 msgr2=0x5602164052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:55.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.341+0000 7f1c2a427b80 1 -- 172.21.15.131:0/243687450 shutdown_connections 2024-05-08T22:16:55.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.341+0000 7f1c2a427b80 1 -- 172.21.15.131:0/243687450 wait complete. 2024-05-08T22:16:55.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.341+0000 7f1c2a427b80 1 -- start start 2024-05-08T22:16:55.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.342+0000 7f1c2a427b80 1 Processor -- start 2024-05-08T22:16:55.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.343+0000 7f1c2a427b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56021640a890 0x560216413c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:55.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.343+0000 7f1c2a427b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560216410050 0x560216414190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:55.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.343+0000 7f1c2a427b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560216410970 0x5602164146d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:55.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.343+0000 7f1c2a427b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602162f6560 con 0x560216410970 2024-05-08T22:16:55.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.343+0000 7f1c2a427b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602162f6200 con 0x560216410050 2024-05-08T22:16:55.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.343+0000 7f1c2a427b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602161dd6a0 con 0x56021640a890 2024-05-08T22:16:55.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.343+0000 7f1c2a427b80 10 client.0 _ll_register_callbacks cb 0x5602162ffb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:16:55.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.343+0000 7f1c0fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56021640a890 0x560216413c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:55.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.344+0000 7f1c0fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56021640a890 0x560216413c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46082/0 (socket says 172.21.15.131:46082) 2024-05-08T22:16:55.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.344+0000 7f1c0fe00700 1 -- 172.21.15.131:0/3422362109 learned_addr learned my addr 172.21.15.131:0/3422362109 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:55.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.344+0000 7f1c0fe00700 1 -- 172.21.15.131:0/3422362109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560216410050 msgr2=0x560216414190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:16:55.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.344+0000 7f1c14a00700 1 --2- 172.21.15.131:0/3422362109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560216410970 0x5602164146d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:55.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.344+0000 7f1c0fe00700 1 --2- 172.21.15.131:0/3422362109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560216410050 0x560216414190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.344+0000 7f1c0fe00700 1 -- 172.21.15.131:0/3422362109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560216410970 msgr2=0x5602164146d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:55.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.344+0000 7f1c0fe00700 1 --2- 172.21.15.131:0/3422362109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560216410970 0x5602164146d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.344+0000 7f1c0fe00700 1 -- 172.21.15.131:0/3422362109 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bfc0099d0 con 0x56021640a890 2024-05-08T22:16:55.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.345+0000 7f1c14a00700 1 --2- 172.21.15.131:0/3422362109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560216410970 0x5602164146d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:16:55.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.348+0000 7f1c0fe00700 1 --2- 172.21.15.131:0/3422362109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56021640a890 0x560216413c50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009fd0 tx=0x7f1c00095a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61a2ba99fe886ae9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:55.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.348+0000 7f1bf9600700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:16:55.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.351+0000 7f1bf9600700 1 -- 172.21.15.131:0/3422362109 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c000b90f0 con 0x56021640a890 2024-05-08T22:16:55.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.351+0000 7f1bf9600700 1 -- 172.21.15.131:0/3422362109 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f1c00099d40 con 0x56021640a890 2024-05-08T22:16:55.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.351+0000 7f1c2a427b80 1 -- 172.21.15.131:0/3422362109 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602162b3270 con 0x56021640a890 2024-05-08T22:16:55.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.351+0000 7f1bf9600700 1 -- 172.21.15.131:0/3422362109 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c00099640 con 0x56021640a890 2024-05-08T22:16:55.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.351+0000 7f1c2a427b80 10 client.26776 fetch_fsmap learned FSMap version 30 2024-05-08T22:16:55.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.351+0000 7f1c2a427b80 1 -- 172.21.15.131:0/3422362109 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560216187b60 con 0x56021640a890 2024-05-08T22:16:55.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.352+0000 7f1bf9600700 1 -- 172.21.15.131:0/3422362109 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5602162b3270 con 0x56021640a890 2024-05-08T22:16:55.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.352+0000 7f1bf9600700 1 -- 172.21.15.131:0/3422362109 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1c000a1630 con 0x56021640a890 2024-05-08T22:16:55.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.352+0000 7f1c2a427b80 10 client.26776 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:16:55.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.352+0000 7f1c2a427b80 10 client.26776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:16:55.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.352+0000 7f1c2a427b80 20 client.26776 populate_metadata read hostname 'smithi131' 2024-05-08T22:16:55.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.352+0000 7f1c2a427b80 1 --2- 172.21.15.131:0/3422362109 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560216424340 0x560216426760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:55.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.352+0000 7f1c2a427b80 4 client.26776 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:16:55.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.352+0000 7f1c2a427b80 1 -- 172.21.15.131:0/3422362109 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5602163d9810 con 0x560216424340 2024-05-08T22:16:55.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.353+0000 7f1c0f400700 1 --2- 172.21.15.131:0/3422362109 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560216424340 0x560216426760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:55.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.355+0000 7f1c0f400700 1 --2- 172.21.15.131:0/3422362109 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560216424340 0x560216426760 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5f69c92ab6bf7de server_cookie=3e32ae1469bb9286 in_seq=0 out_seq=0 2024-05-08T22:16:55.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.355+0000 7f1bf9600700 10 client.26776 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:55.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.356+0000 7f1bf9600700 1 -- 172.21.15.131:0/3422362109 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5602163d9810 con 0x560216424340 2024-05-08T22:16:55.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.356+0000 7f1bf9600700 10 client.26776 handle_command_reply: tid=0 2024-05-08T22:16:55.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.356+0000 7f1c2a427b80 1 client.26776 shutdown 2024-05-08T22:16:55.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.356+0000 7f1c2a427b80 1 -- 172.21.15.131:0/3422362109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56021640a890 msgr2=0x560216413c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:55.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.356+0000 7f1c2a427b80 1 --2- 172.21.15.131:0/3422362109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56021640a890 0x560216413c50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009fd0 tx=0x7f1c00095a90 comp rx=0 tx=0).stop 2024-05-08T22:16:55.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.356+0000 7f1c2a427b80 1 -- 172.21.15.131:0/3422362109 shutdown_connections 2024-05-08T22:16:55.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.356+0000 7f1c2a427b80 1 --2- 172.21.15.131:0/3422362109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560216410970 0x5602164146d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.356+0000 7f1c2a427b80 1 --2- 172.21.15.131:0/3422362109 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560216424340 0x560216426760 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.356+0000 7f1c2a427b80 1 --2- 172.21.15.131:0/3422362109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56021640a890 0x560216413c50 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.356+0000 7f1c2a427b80 1 --2- 172.21.15.131:0/3422362109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560216410050 0x560216414190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.356+0000 7f1c2a427b80 1 -- 172.21.15.131:0/3422362109 >> 172.21.15.131:0/3422362109 conn(0x560216368210 msgr2=0x5602164050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:55.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.358+0000 7f1bf9600700 0 client.26776 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:55.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.358+0000 7f1c2a427b80 1 -- 172.21.15.131:0/3422362109 shutdown_connections 2024-05-08T22:16:55.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.359+0000 7f1c2a427b80 1 -- 172.21.15.131:0/3422362109 wait complete. 2024-05-08T22:16:55.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.359+0000 7f1c2a427b80 20 client.26776 trim_cache size 0 max 16384 2024-05-08T22:16:55.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.385+0000 7f1c2a427b80 1 Processor -- start 2024-05-08T22:16:55.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.385+0000 7f1c2a427b80 1 -- start start 2024-05-08T22:16:55.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.386+0000 7f1c2a427b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56021640a890 0x56021640a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:55.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.386+0000 7f1c2a427b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560216410050 0x5602164144f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:55.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.386+0000 7f1c2a427b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560216410970 0x56021641b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:55.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.386+0000 7f1c2a427b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560216401dc0 con 0x560216410050 2024-05-08T22:16:55.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.386+0000 7f1c2a427b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56021642b910 con 0x560216410970 2024-05-08T22:16:55.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.386+0000 7f1c2a427b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560216404390 con 0x56021640a890 2024-05-08T22:16:55.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.386+0000 7f1c14a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560216410970 0x56021641b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:55.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.387+0000 7f1c14a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560216410970 0x56021641b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:44736/0 (socket says 172.21.15.131:44736) 2024-05-08T22:16:55.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.387+0000 7f1c14a00700 1 -- 172.21.15.131:0/3661327753 learned_addr learned my addr 172.21.15.131:0/3661327753 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:55.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.387+0000 7f1c0f400700 1 --2- 172.21.15.131:0/3661327753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560216410050 0x5602164144f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:55.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.387+0000 7f1c14a00700 1 -- 172.21.15.131:0/3661327753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56021640a890 msgr2=0x56021640a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:55.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.387+0000 7f1c14a00700 1 --2- 172.21.15.131:0/3661327753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56021640a890 0x56021640a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.387+0000 7f1c14a00700 1 -- 172.21.15.131:0/3661327753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560216410050 msgr2=0x5602164144f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:55.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.387+0000 7f1c14a00700 1 --2- 172.21.15.131:0/3661327753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560216410050 0x5602164144f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.387+0000 7f1c14a00700 1 -- 172.21.15.131:0/3661327753 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560216320560 con 0x560216410970 2024-05-08T22:16:55.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.387+0000 7f1c14a00700 1 --2- 172.21.15.131:0/3661327753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560216410970 0x56021641b3b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1c08015000 tx=0x7f1c0801c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61e7e99e79974ffd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:55.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.388+0000 7f1c05c00700 1 -- 172.21.15.131:0/3661327753 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0800e040 con 0x560216410970 2024-05-08T22:16:55.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.388+0000 7f1c05c00700 1 -- 172.21.15.131:0/3661327753 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c08007e20 con 0x560216410970 2024-05-08T22:16:55.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.388+0000 7f1c05c00700 1 -- 172.21.15.131:0/3661327753 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c08005a20 con 0x560216410970 2024-05-08T22:16:55.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.390+0000 7f1c2a427b80 1 -- 172.21.15.131:0/3661327753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560216410970 msgr2=0x56021641b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:55.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.390+0000 7f1c2a427b80 1 --2- 172.21.15.131:0/3661327753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560216410970 0x56021641b3b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1c08015000 tx=0x7f1c0801c000 comp rx=0 tx=0).stop 2024-05-08T22:16:55.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.390+0000 7f1c2a427b80 1 -- 172.21.15.131:0/3661327753 shutdown_connections 2024-05-08T22:16:55.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.392+0000 7f1c2a427b80 1 --2- 172.21.15.131:0/3661327753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560216410970 0x56021641b3b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.392+0000 7f1c2a427b80 1 --2- 172.21.15.131:0/3661327753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56021640a890 0x56021640a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.392+0000 7f1c2a427b80 1 --2- 172.21.15.131:0/3661327753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560216410050 0x5602164144f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.392+0000 7f1c2a427b80 1 -- 172.21.15.131:0/3661327753 >> 172.21.15.131:0/3661327753 conn(0x560216368210 msgr2=0x56021640dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:55.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.393+0000 7f1c2a427b80 1 -- 172.21.15.131:0/3661327753 shutdown_connections 2024-05-08T22:16:55.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.393+0000 7f1c2a427b80 1 -- 172.21.15.131:0/3661327753 wait complete. 2024-05-08T22:16:55.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.394+0000 7f1c2a427b80 1 -- start start 2024-05-08T22:16:55.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.394+0000 7f1c2a427b80 1 Processor -- start 2024-05-08T22:16:55.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.395+0000 7f1c2a427b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56021640a890 0x56021641b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:55.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.395+0000 7f1c2a427b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560216410050 0x56021641b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:55.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.395+0000 7f1c2a427b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560216410970 0x56021640ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:55.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.396+0000 7f1c2a427b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56021640d870 con 0x560216410970 2024-05-08T22:16:55.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.396+0000 7f1c2a427b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56021640bf40 con 0x56021640a890 2024-05-08T22:16:55.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.396+0000 7f1c2a427b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56021640c100 con 0x560216410050 2024-05-08T22:16:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.396+0000 7f1c2a427b80 10 client.0 _ll_register_callbacks cb 0x5602162ffb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:16:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.396+0000 7f1c0fe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56021640a890 0x56021641b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.396+0000 7f1c0fe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56021640a890 0x56021641b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:44738/0 (socket says 172.21.15.131:44738) 2024-05-08T22:16:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.396+0000 7f1c0fe00700 1 -- 172.21.15.131:0/906967084 learned_addr learned my addr 172.21.15.131:0/906967084 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.396+0000 7f1c0f400700 1 --2- 172.21.15.131:0/906967084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560216410050 0x56021641b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:55.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.397+0000 7f1c14a00700 1 --2- 172.21.15.131:0/906967084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560216410970 0x56021640ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:55.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.397+0000 7f1c0f400700 1 -- 172.21.15.131:0/906967084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56021640a890 msgr2=0x56021641b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:55.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.397+0000 7f1c0f400700 1 --2- 172.21.15.131:0/906967084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56021640a890 0x56021641b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.397+0000 7f1c0f400700 1 -- 172.21.15.131:0/906967084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560216410970 msgr2=0x56021640ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:55.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.397+0000 7f1c0f400700 1 --2- 172.21.15.131:0/906967084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560216410970 0x56021640ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.397+0000 7f1c0f400700 1 -- 172.21.15.131:0/906967084 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c1010d420 con 0x560216410050 2024-05-08T22:16:55.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.397+0000 7f1c14a00700 1 --2- 172.21.15.131:0/906967084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560216410970 0x56021640ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:16:55.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.397+0000 7f1c0f400700 1 --2- 172.21.15.131:0/906967084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560216410050 0x56021641b900 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc000c00 tx=0x7f1bfc009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7ee099b59bbd13f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:55.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.397+0000 7f1bf9600700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:16:55.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.397+0000 7f1bf9600700 1 -- 172.21.15.131:0/906967084 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc009ec0 con 0x560216410050 2024-05-08T22:16:55.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.398+0000 7f1bf9600700 1 -- 172.21.15.131:0/906967084 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f1bfc029150 con 0x560216410050 2024-05-08T22:16:55.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.398+0000 7f1c2a427b80 1 -- 172.21.15.131:0/906967084 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bfc0054e0 con 0x560216410050 2024-05-08T22:16:55.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.398+0000 7f1bf9600700 1 -- 172.21.15.131:0/906967084 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bfc029310 con 0x560216410050 2024-05-08T22:16:55.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.398+0000 7f1c2a427b80 10 client.26779 fetch_fsmap learned FSMap version 30 2024-05-08T22:16:55.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.398+0000 7f1c2a427b80 1 -- 172.21.15.131:0/906967084 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5602161dfcd0 con 0x560216410050 2024-05-08T22:16:55.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.398+0000 7f1bf9600700 1 -- 172.21.15.131:0/906967084 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bfc0054e0 con 0x560216410050 2024-05-08T22:16:55.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.399+0000 7f1bf9600700 1 -- 172.21.15.131:0/906967084 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1bfc011440 con 0x560216410050 2024-05-08T22:16:55.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.399+0000 7f1c2a427b80 10 client.26779 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:16:55.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.399+0000 7f1c2a427b80 10 client.26779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:16:55.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.399+0000 7f1c2a427b80 20 client.26779 populate_metadata read hostname 'smithi131' 2024-05-08T22:16:55.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.399+0000 7f1c2a427b80 1 --2- 172.21.15.131:0/906967084 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x56021647b750 0x56021647db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:55.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.399+0000 7f1c2a427b80 4 client.26779 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:16:55.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.399+0000 7f1c2a427b80 1 -- 172.21.15.131:0/906967084 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560216352ec0 con 0x56021647b750 2024-05-08T22:16:55.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.400+0000 7f1c0fe00700 1 --2- 172.21.15.131:0/906967084 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x56021647b750 0x56021647db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:55.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.400+0000 7f1c0fe00700 1 --2- 172.21.15.131:0/906967084 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x56021647b750 0x56021647db70 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad16e293be91b7df server_cookie=f649c534f140ef16 in_seq=0 out_seq=0 2024-05-08T22:16:55.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.400+0000 7f1bf9600700 10 client.26779 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:55.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.408+0000 7f1bf9600700 1 -- 172.21.15.131:0/906967084 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560216352ec0 con 0x56021647b750 2024-05-08T22:16:55.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.408+0000 7f1bf9600700 10 client.26779 handle_command_reply: tid=0 2024-05-08T22:16:55.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.408+0000 7f1c2a427b80 1 client.26779 shutdown 2024-05-08T22:16:55.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.408+0000 7f1c2a427b80 1 -- 172.21.15.131:0/906967084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560216410050 msgr2=0x56021641b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:55.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.408+0000 7f1c2a427b80 1 --2- 172.21.15.131:0/906967084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560216410050 0x56021641b900 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc000c00 tx=0x7f1bfc009ca0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.408+0000 7f1c2a427b80 1 -- 172.21.15.131:0/906967084 shutdown_connections 2024-05-08T22:16:55.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.408+0000 7f1c2a427b80 1 --2- 172.21.15.131:0/906967084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560216410970 0x56021640ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.408+0000 7f1c2a427b80 1 --2- 172.21.15.131:0/906967084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56021640a890 0x56021641b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.408+0000 7f1c2a427b80 1 --2- 172.21.15.131:0/906967084 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x56021647b750 0x56021647db70 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.408+0000 7f1c2a427b80 1 --2- 172.21.15.131:0/906967084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560216410050 0x56021641b900 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.408+0000 7f1c2a427b80 1 -- 172.21.15.131:0/906967084 >> 172.21.15.131:0/906967084 conn(0x560216368210 msgr2=0x5602164123b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:55.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.409+0000 7f1bf9600700 0 client.26779 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:55.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.409+0000 7f1c2a427b80 1 -- 172.21.15.131:0/906967084 shutdown_connections 2024-05-08T22:16:55.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.409+0000 7f1c2a427b80 1 -- 172.21.15.131:0/906967084 wait complete. 2024-05-08T22:16:55.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.409+0000 7f1c2a427b80 20 client.26779 trim_cache size 0 max 16384 2024-05-08T22:16:55.412 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:16:55.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.410+0000 7f1c15400700 1 -- 172.21.15.131:0/2059215479 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1bf40a13e0 msgr2=0x7f1bf40a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:55.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.410+0000 7f1c15400700 1 --2- 172.21.15.131:0/2059215479 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1bf40a13e0 0x7f1bf40a38a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1c0800b780 tx=0x7f1c08006210 comp rx=0 tx=0).stop 2024-05-08T22:16:55.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.410+0000 7f1c15400700 1 -- 172.21.15.131:0/2059215479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c100fff10 msgr2=0x7f1c1019d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:55.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.410+0000 7f1c15400700 1 --2- 172.21.15.131:0/2059215479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c100fff10 0x7f1c1019d130 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000b450 tx=0x7f1c0000b760 comp rx=0 tx=0).stop 2024-05-08T22:16:55.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.410+0000 7f1c15400700 1 -- 172.21.15.131:0/2059215479 shutdown_connections 2024-05-08T22:16:55.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.410+0000 7f1c15400700 1 --2- 172.21.15.131:0/2059215479 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1bf40a13e0 0x7f1bf40a38a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.410+0000 7f1c15400700 1 --2- 172.21.15.131:0/2059215479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c1019d670 0x7f1c10195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.410+0000 7f1c15400700 1 --2- 172.21.15.131:0/2059215479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c100fff10 0x7f1c1019d130 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.410+0000 7f1c15400700 1 --2- 172.21.15.131:0/2059215479 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c10105030 0x7f1c10195350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:55.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.410+0000 7f1c15400700 1 -- 172.21.15.131:0/2059215479 >> 172.21.15.131:0/2059215479 conn(0x7f1c100fb770 msgr2=0x7f1c1010b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:55.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.410+0000 7f1c15400700 1 -- 172.21.15.131:0/2059215479 shutdown_connections 2024-05-08T22:16:55.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:55.410+0000 7f1c15400700 1 -- 172.21.15.131:0/2059215479 wait complete. 2024-05-08T22:16:56.431 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T22:16:56.431 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T22:16:56.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.687+0000 7fed73800700 1 -- 172.21.15.131:0/740958241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c068c90 msgr2=0x7fed6c105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:56.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.687+0000 7fed73800700 1 --2- 172.21.15.131:0/740958241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c068c90 0x7fed6c105b60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fed54009920 tx=0x7fed54009c30 comp rx=0 tx=0).stop 2024-05-08T22:16:56.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.687+0000 7fed73800700 1 -- 172.21.15.131:0/740958241 shutdown_connections 2024-05-08T22:16:56.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.687+0000 7fed73800700 1 --2- 172.21.15.131:0/740958241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed6c106290 0x7fed6c10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.687+0000 7fed73800700 1 --2- 172.21.15.131:0/740958241 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed6c0682e0 0x7fed6c0686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.687+0000 7fed73800700 1 --2- 172.21.15.131:0/740958241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c068c90 0x7fed6c105b60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.687+0000 7fed73800700 1 -- 172.21.15.131:0/740958241 >> 172.21.15.131:0/740958241 conn(0x7fed6c076f40 msgr2=0x7fed6c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:56.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.687+0000 7fed73800700 1 -- 172.21.15.131:0/740958241 shutdown_connections 2024-05-08T22:16:56.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.688+0000 7fed73800700 1 -- 172.21.15.131:0/740958241 wait complete. 2024-05-08T22:16:56.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.689+0000 7fed73800700 1 Processor -- start 2024-05-08T22:16:56.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.689+0000 7fed73800700 1 -- start start 2024-05-08T22:16:56.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.690+0000 7fed73800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c0682e0 0x7fed6c115240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:56.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.690+0000 7fed73800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed6c106290 0x7fed6c1157a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:56.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.690+0000 7fed73800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed6c1103a0 0x7fed6c110850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:56.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.690+0000 7fed73800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c110f60 con 0x7fed6c1103a0 2024-05-08T22:16:56.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.690+0000 7fed73800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c1110d0 con 0x7fed6c106290 2024-05-08T22:16:56.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.690+0000 7fed73800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c11afc0 con 0x7fed6c0682e0 2024-05-08T22:16:56.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.690+0000 7fed70e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c0682e0 0x7fed6c115240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:56.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.690+0000 7fed70e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c0682e0 0x7fed6c115240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46112/0 (socket says 172.21.15.131:46112) 2024-05-08T22:16:56.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.690+0000 7fed70e00700 1 -- 172.21.15.131:0/2436605441 learned_addr learned my addr 172.21.15.131:0/2436605441 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:56.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.690+0000 7fed71800700 1 --2- 172.21.15.131:0/2436605441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed6c1103a0 0x7fed6c110850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:56.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.691+0000 7fed70e00700 1 -- 172.21.15.131:0/2436605441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed6c106290 msgr2=0x7fed6c1157a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:56.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.691+0000 7fed70e00700 1 --2- 172.21.15.131:0/2436605441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed6c106290 0x7fed6c1157a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.691+0000 7fed70e00700 1 -- 172.21.15.131:0/2436605441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed6c1103a0 msgr2=0x7fed6c110850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:56.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.691+0000 7fed70e00700 1 --2- 172.21.15.131:0/2436605441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed6c1103a0 0x7fed6c110850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.691+0000 7fed70e00700 1 -- 172.21.15.131:0/2436605441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed54009520 con 0x7fed6c0682e0 2024-05-08T22:16:56.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.692+0000 7fed70e00700 1 --2- 172.21.15.131:0/2436605441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c0682e0 0x7fed6c115240 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00ba50 tx=0x7fed5c00be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:56.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.694+0000 7fed69600700 1 -- 172.21.15.131:0/2436605441 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c00cab0 con 0x7fed6c0682e0 2024-05-08T22:16:56.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.694+0000 7fed69600700 1 -- 172.21.15.131:0/2436605441 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed5c00cc10 con 0x7fed6c0682e0 2024-05-08T22:16:56.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.694+0000 7fed69600700 1 -- 172.21.15.131:0/2436605441 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c0075d0 con 0x7fed6c0682e0 2024-05-08T22:16:56.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.695+0000 7fed73800700 1 -- 172.21.15.131:0/2436605441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6c11b2a0 con 0x7fed6c0682e0 2024-05-08T22:16:56.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.695+0000 7fed73800700 1 -- 172.21.15.131:0/2436605441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6c11b760 con 0x7fed6c0682e0 2024-05-08T22:16:56.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.705+0000 7fed69600700 1 -- 172.21.15.131:0/2436605441 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fed5c014440 con 0x7fed6c0682e0 2024-05-08T22:16:56.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.712+0000 7fed8727bb80 1 Processor -- start 2024-05-08T22:16:56.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.712+0000 7fed69600700 1 --2- 172.21.15.131:0/2436605441 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fed500a1460 0x7fed500a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:56.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.712+0000 7fed69600700 1 -- 172.21.15.131:0/2436605441 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fed5c006e40 con 0x7fed6c0682e0 2024-05-08T22:16:56.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.714+0000 7fed8727bb80 1 -- start start 2024-05-08T22:16:56.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.714+0000 7fed71800700 1 --2- 172.21.15.131:0/2436605441 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fed500a1460 0x7fed500a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:56.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.714+0000 7fed8727bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb5b73ae80 0x55fb5b73b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:56.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.714+0000 7fed8727bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b73b7a0 0x55fb5b7353d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:56.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.715+0000 7fed8727bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb5b735910 0x55fb5b735d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:56.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.715+0000 7fed8727bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5b68c1d0 con 0x55fb5b73ae80 2024-05-08T22:16:56.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.715+0000 7fed8727bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5b689b90 con 0x55fb5b73b7a0 2024-05-08T22:16:56.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.715+0000 7fed8727bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5b685480 con 0x55fb5b735910 2024-05-08T22:16:56.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.715+0000 7fed6be00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b73b7a0 0x55fb5b7353d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:56.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.715+0000 7fed6be00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b73b7a0 0x55fb5b7353d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:44752/0 (socket says 172.21.15.131:44752) 2024-05-08T22:16:56.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.715+0000 7fed6be00700 1 -- 172.21.15.131:0/3207035880 learned_addr learned my addr 172.21.15.131:0/3207035880 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:56.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.716+0000 7fed6be00700 1 -- 172.21.15.131:0/3207035880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb5b735910 msgr2=0x55fb5b735d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:16:56.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.716+0000 7fed6be00700 1 --2- 172.21.15.131:0/3207035880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb5b735910 0x55fb5b735d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.716+0000 7fed6be00700 1 -- 172.21.15.131:0/3207035880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb5b73ae80 msgr2=0x55fb5b73b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:56.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.716+0000 7fed6be00700 1 --2- 172.21.15.131:0/3207035880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb5b73ae80 0x55fb5b73b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.716+0000 7fed71800700 1 --2- 172.21.15.131:0/2436605441 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fed500a1460 0x7fed500a3920 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fed64005950 tx=0x7fed640058e0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:56.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.716+0000 7fed6be00700 1 -- 172.21.15.131:0/3207035880 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb5b506430 con 0x55fb5b73b7a0 2024-05-08T22:16:56.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.716+0000 7fed6be00700 1 --2- 172.21.15.131:0/3207035880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b73b7a0 0x55fb5b7353d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fed54016000 tx=0x7fed54009420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c03c2dec5d038ea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:56.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.717+0000 7fed62000700 1 -- 172.21.15.131:0/3207035880 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5401c070 con 0x55fb5b73b7a0 2024-05-08T22:16:56.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.717+0000 7fed62000700 1 -- 172.21.15.131:0/3207035880 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed54010e10 con 0x55fb5b73b7a0 2024-05-08T22:16:56.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.717+0000 7fed62000700 1 -- 172.21.15.131:0/3207035880 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5400b790 con 0x55fb5b73b7a0 2024-05-08T22:16:56.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.718+0000 7fed8727bb80 1 -- 172.21.15.131:0/3207035880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b73b7a0 msgr2=0x55fb5b7353d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:56.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.718+0000 7fed8727bb80 1 --2- 172.21.15.131:0/3207035880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b73b7a0 0x55fb5b7353d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fed54016000 tx=0x7fed54009420 comp rx=0 tx=0).stop 2024-05-08T22:16:56.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.718+0000 7fed8727bb80 1 -- 172.21.15.131:0/3207035880 shutdown_connections 2024-05-08T22:16:56.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.719+0000 7fed8727bb80 1 --2- 172.21.15.131:0/3207035880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb5b735910 0x55fb5b735d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.719+0000 7fed8727bb80 1 --2- 172.21.15.131:0/3207035880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb5b73ae80 0x55fb5b73b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.719+0000 7fed8727bb80 1 --2- 172.21.15.131:0/3207035880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b73b7a0 0x55fb5b7353d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.719+0000 7fed8727bb80 1 -- 172.21.15.131:0/3207035880 >> 172.21.15.131:0/3207035880 conn(0x55fb5b693290 msgr2=0x55fb5b72fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:56.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.719+0000 7fed8727bb80 1 -- 172.21.15.131:0/3207035880 shutdown_connections 2024-05-08T22:16:56.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.719+0000 7fed8727bb80 1 -- 172.21.15.131:0/3207035880 wait complete. 2024-05-08T22:16:56.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.720+0000 7fed8727bb80 1 -- start start 2024-05-08T22:16:56.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.721+0000 7fed8727bb80 1 Processor -- start 2024-05-08T22:16:56.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.722+0000 7fed8727bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b735910 0x55fb5b73ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:56.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.722+0000 7fed8727bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb5b73ae80 0x55fb5b73f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:56.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.722+0000 7fed8727bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb5b73b7a0 0x55fb5b73f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:56.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.723+0000 7fed8727bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5b6215e0 con 0x55fb5b73b7a0 2024-05-08T22:16:56.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.723+0000 7fed70e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b735910 0x55fb5b73ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:56.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.723+0000 7fed8727bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5b621280 con 0x55fb5b735910 2024-05-08T22:16:56.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.723+0000 7fed8727bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5b508720 con 0x55fb5b73ae80 2024-05-08T22:16:56.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.723+0000 7fed8727bb80 10 client.0 _ll_register_callbacks cb 0x55fb5b62abb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:16:56.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.723+0000 7fed71800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb5b73b7a0 0x55fb5b73f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:56.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.723+0000 7fed70e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b735910 0x55fb5b73ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:44762/0 (socket says 172.21.15.131:44762) 2024-05-08T22:16:56.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.723+0000 7fed70e00700 1 -- 172.21.15.131:0/2858661635 learned_addr learned my addr 172.21.15.131:0/2858661635 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:56.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.723+0000 7fed6be00700 1 --2- 172.21.15.131:0/2858661635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb5b73ae80 0x55fb5b73f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:56.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.723+0000 7fed70e00700 1 -- 172.21.15.131:0/2858661635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb5b73ae80 msgr2=0x55fb5b73f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:56.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.724+0000 7fed70e00700 1 --2- 172.21.15.131:0/2858661635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb5b73ae80 0x55fb5b73f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.724+0000 7fed70e00700 1 -- 172.21.15.131:0/2858661635 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb5b73b7a0 msgr2=0x55fb5b73f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:56.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.724+0000 7fed70e00700 1 --2- 172.21.15.131:0/2858661635 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb5b73b7a0 0x55fb5b73f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.724+0000 7fed70e00700 1 -- 172.21.15.131:0/2858661635 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6c10e650 con 0x55fb5b735910 2024-05-08T22:16:56.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.724+0000 7fed6be00700 1 --2- 172.21.15.131:0/2858661635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb5b73ae80 0x55fb5b73f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:16:56.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.725+0000 7fed70e00700 1 --2- 172.21.15.131:0/2858661635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b735910 0x55fb5b73ec70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fed5c095fd0 tx=0x7fed5c0c9ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a3f949a3f6d77be server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:56.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.725+0000 7fed5aa00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:16:56.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.725+0000 7fed5aa00700 1 -- 172.21.15.131:0/2858661635 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c099560 con 0x55fb5b735910 2024-05-08T22:16:56.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.725+0000 7fed5aa00700 1 -- 172.21.15.131:0/2858661635 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fed5c09d390 con 0x55fb5b735910 2024-05-08T22:16:56.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.726+0000 7fed8727bb80 1 -- 172.21.15.131:0/2858661635 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb5b5de2f0 con 0x55fb5b735910 2024-05-08T22:16:56.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.726+0000 7fed5aa00700 1 -- 172.21.15.131:0/2858661635 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed5c0a19e0 con 0x55fb5b735910 2024-05-08T22:16:56.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.726+0000 7fed8727bb80 10 client.26690 fetch_fsmap learned FSMap version 30 2024-05-08T22:16:56.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.726+0000 7fed8727bb80 1 -- 172.21.15.131:0/2858661635 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55fb5b4b2be0 con 0x55fb5b735910 2024-05-08T22:16:56.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.726+0000 7fed5aa00700 1 -- 172.21.15.131:0/2858661635 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fb5b5de2f0 con 0x55fb5b735910 2024-05-08T22:16:56.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.728+0000 7fed5aa00700 1 -- 172.21.15.131:0/2858661635 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fed5c0cc300 con 0x55fb5b735910 2024-05-08T22:16:56.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.728+0000 7fed8727bb80 10 client.26690 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:16:56.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.728+0000 7fed8727bb80 10 client.26690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:16:56.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.728+0000 7fed8727bb80 20 client.26690 populate_metadata read hostname 'smithi131' 2024-05-08T22:16:56.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.728+0000 7fed8727bb80 1 --2- 172.21.15.131:0/2858661635 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55fb5b74f290 0x55fb5b751680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:56.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.728+0000 7fed8727bb80 4 client.26690 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:16:56.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.728+0000 7fed8727bb80 1 -- 172.21.15.131:0/2858661635 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb5b704890 con 0x55fb5b74f290 2024-05-08T22:16:56.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.729+0000 7fed6be00700 1 --2- 172.21.15.131:0/2858661635 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55fb5b74f290 0x55fb5b751680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:56.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.730+0000 7fed6be00700 1 --2- 172.21.15.131:0/2858661635 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55fb5b74f290 0x55fb5b751680 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad54bded77b22e23 server_cookie=1adc535182532743 in_seq=0 out_seq=0 2024-05-08T22:16:56.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.730+0000 7fed5aa00700 10 client.26690 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:56.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.739+0000 7fed5aa00700 1 -- 172.21.15.131:0/2858661635 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fb5b704890 con 0x55fb5b74f290 2024-05-08T22:16:56.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.739+0000 7fed5aa00700 10 client.26690 handle_command_reply: tid=0 2024-05-08T22:16:56.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.741+0000 7fed8727bb80 1 client.26690 shutdown 2024-05-08T22:16:56.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.742+0000 7fed8727bb80 1 -- 172.21.15.131:0/2858661635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b735910 msgr2=0x55fb5b73ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:56.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.742+0000 7fed8727bb80 1 --2- 172.21.15.131:0/2858661635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b735910 0x55fb5b73ec70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fed5c095fd0 tx=0x7fed5c0c9ff0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.742+0000 7fed8727bb80 1 -- 172.21.15.131:0/2858661635 shutdown_connections 2024-05-08T22:16:56.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.742+0000 7fed8727bb80 1 --2- 172.21.15.131:0/2858661635 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb5b73b7a0 0x55fb5b73f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.743+0000 7fed8727bb80 1 --2- 172.21.15.131:0/2858661635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b735910 0x55fb5b73ec70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.743+0000 7fed8727bb80 1 --2- 172.21.15.131:0/2858661635 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55fb5b74f290 0x55fb5b751680 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.743+0000 7fed8727bb80 1 --2- 172.21.15.131:0/2858661635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb5b73ae80 0x55fb5b73f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.743+0000 7fed5aa00700 0 client.26690 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:56.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.743+0000 7fed8727bb80 1 -- 172.21.15.131:0/2858661635 >> 172.21.15.131:0/2858661635 conn(0x55fb5b693290 msgr2=0x55fb5b72f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:56.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.743+0000 7fed8727bb80 1 -- 172.21.15.131:0/2858661635 shutdown_connections 2024-05-08T22:16:56.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.743+0000 7fed8727bb80 1 -- 172.21.15.131:0/2858661635 wait complete. 2024-05-08T22:16:56.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.743+0000 7fed8727bb80 20 client.26690 trim_cache size 0 max 16384 2024-05-08T22:16:56.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:56 smithi157 ceph-mon[77746]: pgmap v298: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 688 op/s 2024-05-08T22:16:56.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.781+0000 7fed8727bb80 1 Processor -- start 2024-05-08T22:16:56.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.781+0000 7fed8727bb80 1 -- start start 2024-05-08T22:16:56.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.782+0000 7fed8727bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb5b735910 0x55fb5b73a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:56.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.782+0000 7fed8727bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb5b73ae80 0x55fb5b72fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:56.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.782+0000 7fed8727bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b73b7a0 0x55fb5b73f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:56.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.782+0000 7fed8727bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5b7388f0 con 0x55fb5b735910 2024-05-08T22:16:56.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.782+0000 7fed8727bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5b74f060 con 0x55fb5b73b7a0 2024-05-08T22:16:56.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.782+0000 7fed8727bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5b73a960 con 0x55fb5b73ae80 2024-05-08T22:16:56.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.783+0000 7fed70e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb5b735910 0x55fb5b73a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:56.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.783+0000 7fed71800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b73b7a0 0x55fb5b73f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:56.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.783+0000 7fed70e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb5b735910 0x55fb5b73a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54964/0 (socket says 172.21.15.131:54964) 2024-05-08T22:16:56.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.783+0000 7fed70e00700 1 -- 172.21.15.131:0/2538929041 learned_addr learned my addr 172.21.15.131:0/2538929041 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:56.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.783+0000 7fed70e00700 1 -- 172.21.15.131:0/2538929041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb5b73ae80 msgr2=0x55fb5b72fda0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:16:56.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.783+0000 7fed70e00700 1 --2- 172.21.15.131:0/2538929041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb5b73ae80 0x55fb5b72fda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.783+0000 7fed70e00700 1 -- 172.21.15.131:0/2538929041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b73b7a0 msgr2=0x55fb5b73f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:56.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.783+0000 7fed70e00700 1 --2- 172.21.15.131:0/2538929041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b73b7a0 0x55fb5b73f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.783+0000 7fed70e00700 1 -- 172.21.15.131:0/2538929041 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb5b72c560 con 0x55fb5b735910 2024-05-08T22:16:56.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.784+0000 7fed70e00700 1 --2- 172.21.15.131:0/2538929041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb5b735910 0x55fb5b73a420 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fed5c000f80 tx=0x7fed5c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326e49aa839441d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:56.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.784+0000 7fed62a00700 1 -- 172.21.15.131:0/2538929041 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c09d390 con 0x55fb5b735910 2024-05-08T22:16:56.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.785+0000 7fed62a00700 1 -- 172.21.15.131:0/2538929041 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed5c0a1550 con 0x55fb5b735910 2024-05-08T22:16:56.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.785+0000 7fed62a00700 1 -- 172.21.15.131:0/2538929041 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c0a1790 con 0x55fb5b735910 2024-05-08T22:16:56.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.785+0000 7fed8727bb80 1 -- 172.21.15.131:0/2538929041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb5b735910 msgr2=0x55fb5b73a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:56.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.785+0000 7fed8727bb80 1 --2- 172.21.15.131:0/2538929041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb5b735910 0x55fb5b73a420 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fed5c000f80 tx=0x7fed5c004ad0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.786+0000 7fed8727bb80 1 -- 172.21.15.131:0/2538929041 shutdown_connections 2024-05-08T22:16:56.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.786+0000 7fed8727bb80 1 --2- 172.21.15.131:0/2538929041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b73b7a0 0x55fb5b73f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.786+0000 7fed8727bb80 1 --2- 172.21.15.131:0/2538929041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb5b735910 0x55fb5b73a420 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.786+0000 7fed8727bb80 1 --2- 172.21.15.131:0/2538929041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb5b73ae80 0x55fb5b72fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.786+0000 7fed8727bb80 1 -- 172.21.15.131:0/2538929041 >> 172.21.15.131:0/2538929041 conn(0x55fb5b693290 msgr2=0x55fb5b739d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:56.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.786+0000 7fed8727bb80 1 -- 172.21.15.131:0/2538929041 shutdown_connections 2024-05-08T22:16:56.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.786+0000 7fed8727bb80 1 -- 172.21.15.131:0/2538929041 wait complete. 2024-05-08T22:16:56.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.786+0000 7fed8727bb80 1 -- start start 2024-05-08T22:16:56.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.787+0000 7fed8727bb80 1 Processor -- start 2024-05-08T22:16:56.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.788+0000 7fed8727bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b73ae80 0x55fb5b744c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:56.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.788+0000 7fed8727bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb5b73b7a0 0x55fb5b7451b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:56.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.788+0000 7fed8727bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb5b7456f0 0x55fb5b7a5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:56.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.788+0000 7fed6be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb5b73b7a0 0x55fb5b7451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:56.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.788+0000 7fed6be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb5b73b7a0 0x55fb5b7451b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46142/0 (socket says 172.21.15.131:46142) 2024-05-08T22:16:56.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.789+0000 7fed70e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b73ae80 0x55fb5b744c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:56.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.789+0000 7fed70e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b73ae80 0x55fb5b744c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:44786/0 (socket says 172.21.15.131:44786) 2024-05-08T22:16:56.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.789+0000 7fed8727bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5b7469c0 con 0x55fb5b7456f0 2024-05-08T22:16:56.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.789+0000 7fed8727bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5b7a5c40 con 0x55fb5b73ae80 2024-05-08T22:16:56.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.789+0000 7fed8727bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5b7a5d80 con 0x55fb5b73b7a0 2024-05-08T22:16:56.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.789+0000 7fed8727bb80 10 client.0 _ll_register_callbacks cb 0x55fb5b62abb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:16:56.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.789+0000 7fed71800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb5b7456f0 0x55fb5b7a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:56.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.789+0000 7fed71800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb5b7456f0 0x55fb5b7a5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54968/0 (socket says 172.21.15.131:54968) 2024-05-08T22:16:56.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.790+0000 7fed6be00700 1 -- 172.21.15.131:0/1119912293 learned_addr learned my addr 172.21.15.131:0/1119912293 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.790+0000 7fed71800700 1 -- 172.21.15.131:0/1119912293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb5b73b7a0 msgr2=0x55fb5b7451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.790+0000 7fed71800700 1 --2- 172.21.15.131:0/1119912293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb5b73b7a0 0x55fb5b7451b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.790+0000 7fed71800700 1 -- 172.21.15.131:0/1119912293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b73ae80 msgr2=0x55fb5b744c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:56.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.792+0000 7fed71800700 1 --2- 172.21.15.131:0/1119912293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b73ae80 0x55fb5b744c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.792+0000 7fed71800700 1 -- 172.21.15.131:0/1119912293 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb5b4b2be0 con 0x55fb5b7456f0 2024-05-08T22:16:56.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.792+0000 7fed70e00700 1 --2- 172.21.15.131:0/1119912293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b73ae80 0x55fb5b744c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:16:56.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.792+0000 7fed71800700 1 --2- 172.21.15.131:0/1119912293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb5b7456f0 0x55fb5b7a5700 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fed64005cd0 tx=0x7fed64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac564a7df92c654 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.794+0000 7fed5aa00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:16:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.794+0000 7fed5aa00700 1 -- 172.21.15.131:0/1119912293 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64004030 con 0x55fb5b7456f0 2024-05-08T22:16:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.794+0000 7fed5aa00700 1 -- 172.21.15.131:0/1119912293 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fed64004190 con 0x55fb5b7456f0 2024-05-08T22:16:56.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.794+0000 7fed8727bb80 1 -- 172.21.15.131:0/1119912293 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed54010e10 con 0x55fb5b7456f0 2024-05-08T22:16:56.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.795+0000 7fed5aa00700 1 -- 172.21.15.131:0/1119912293 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed64004350 con 0x55fb5b7456f0 2024-05-08T22:16:56.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.795+0000 7fed8727bb80 10 client.16635 fetch_fsmap learned FSMap version 30 2024-05-08T22:16:56.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.795+0000 7fed8727bb80 1 -- 172.21.15.131:0/1119912293 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55fb5b64b5e0 con 0x55fb5b7456f0 2024-05-08T22:16:56.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.795+0000 7fed5aa00700 1 -- 172.21.15.131:0/1119912293 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed54010e10 con 0x55fb5b7456f0 2024-05-08T22:16:56.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.796+0000 7fed5aa00700 1 -- 172.21.15.131:0/1119912293 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fed64029440 con 0x55fb5b7456f0 2024-05-08T22:16:56.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.797+0000 7fed8727bb80 10 client.16635 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:16:56.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.797+0000 7fed8727bb80 10 client.16635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:16:56.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.797+0000 7fed8727bb80 20 client.16635 populate_metadata read hostname 'smithi131' 2024-05-08T22:16:56.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.797+0000 7fed8727bb80 1 --2- 172.21.15.131:0/1119912293 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55fb5b7a7480 0x55fb5b7a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:56.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.797+0000 7fed8727bb80 4 client.16635 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T22:16:56.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.797+0000 7fed8727bb80 1 -- 172.21.15.131:0/1119912293 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55fb5b67df40 con 0x55fb5b7a7480 2024-05-08T22:16:56.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.798+0000 7fed70e00700 1 --2- 172.21.15.131:0/1119912293 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55fb5b7a7480 0x55fb5b7a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:56.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.803+0000 7fed70e00700 1 --2- 172.21.15.131:0/1119912293 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55fb5b7a7480 0x55fb5b7a98a0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfa9f6618038822f server_cookie=2c7368c067f77a0a in_seq=0 out_seq=0 2024-05-08T22:16:56.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.803+0000 7fed5aa00700 10 client.16635 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:56.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.807+0000 7fed5aa00700 1 -- 172.21.15.131:0/1119912293 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fb5b67df40 con 0x55fb5b7a7480 2024-05-08T22:16:56.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.808+0000 7fed5aa00700 10 client.16635 handle_command_reply: tid=0 2024-05-08T22:16:56.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.808+0000 7fed8727bb80 1 client.16635 shutdown 2024-05-08T22:16:56.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.808+0000 7fed8727bb80 1 -- 172.21.15.131:0/1119912293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb5b7456f0 msgr2=0x55fb5b7a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:56.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.808+0000 7fed8727bb80 1 --2- 172.21.15.131:0/1119912293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb5b7456f0 0x55fb5b7a5700 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fed64005cd0 tx=0x7fed64004750 comp rx=0 tx=0).stop 2024-05-08T22:16:56.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.808+0000 7fed8727bb80 1 -- 172.21.15.131:0/1119912293 shutdown_connections 2024-05-08T22:16:56.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.808+0000 7fed8727bb80 1 --2- 172.21.15.131:0/1119912293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb5b7456f0 0x55fb5b7a5700 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.808+0000 7fed8727bb80 1 --2- 172.21.15.131:0/1119912293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fb5b73ae80 0x55fb5b744c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.808+0000 7fed8727bb80 1 --2- 172.21.15.131:0/1119912293 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55fb5b7a7480 0x55fb5b7a98a0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.808+0000 7fed8727bb80 1 --2- 172.21.15.131:0/1119912293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb5b73b7a0 0x55fb5b7451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.808+0000 7fed8727bb80 1 -- 172.21.15.131:0/1119912293 >> 172.21.15.131:0/1119912293 conn(0x55fb5b693290 msgr2=0x55fb5b73bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:56.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.808+0000 7fed5aa00700 0 client.16635 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:56.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.809+0000 7fed8727bb80 1 -- 172.21.15.131:0/1119912293 shutdown_connections 2024-05-08T22:16:56.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.809+0000 7fed8727bb80 1 -- 172.21.15.131:0/1119912293 wait complete. 2024-05-08T22:16:56.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.810+0000 7fed8727bb80 20 client.16635 trim_cache size 0 max 16384 2024-05-08T22:16:56.811 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:16:56.811 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-08T22:16:56.812 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "77343017-da3f-48cf-87a8-13dad216b78f", 2024-05-08T22:16:56.812 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-08T22:16:56.812 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:16:56.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.811+0000 7fed73800700 1 -- 172.21.15.131:0/2436605441 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fed500a1460 msgr2=0x7fed500a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:56.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.811+0000 7fed73800700 1 --2- 172.21.15.131:0/2436605441 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fed500a1460 0x7fed500a3920 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fed64005950 tx=0x7fed640058e0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.811+0000 7fed73800700 1 -- 172.21.15.131:0/2436605441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c0682e0 msgr2=0x7fed6c115240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:56.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.811+0000 7fed73800700 1 --2- 172.21.15.131:0/2436605441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c0682e0 0x7fed6c115240 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00ba50 tx=0x7fed5c00be10 comp rx=0 tx=0).stop 2024-05-08T22:16:56.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.811+0000 7fed73800700 1 -- 172.21.15.131:0/2436605441 shutdown_connections 2024-05-08T22:16:56.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.811+0000 7fed73800700 1 --2- 172.21.15.131:0/2436605441 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fed500a1460 0x7fed500a3920 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.811+0000 7fed73800700 1 --2- 172.21.15.131:0/2436605441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed6c1103a0 0x7fed6c110850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.811+0000 7fed73800700 1 --2- 172.21.15.131:0/2436605441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c0682e0 0x7fed6c115240 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.811+0000 7fed73800700 1 --2- 172.21.15.131:0/2436605441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed6c106290 0x7fed6c1157a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:56.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.811+0000 7fed73800700 1 -- 172.21.15.131:0/2436605441 >> 172.21.15.131:0/2436605441 conn(0x7fed6c076f40 msgr2=0x7fed6c0fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:56.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.811+0000 7fed73800700 1 -- 172.21.15.131:0/2436605441 shutdown_connections 2024-05-08T22:16:56.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:56.812+0000 7fed73800700 1 -- 172.21.15.131:0/2436605441 wait complete. 2024-05-08T22:16:56.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:56 smithi132 ceph-mon[78180]: pgmap v298: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 688 op/s 2024-05-08T22:16:56.845 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:56 smithi131 ceph-mon[37260]: pgmap v298: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 688 op/s 2024-05-08T22:16:56.849 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:16:57.115 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2024-05-08T22:16:57.116 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-08T22:16:57.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.226+0000 7ff80e600700 1 -- 172.21.15.131:0/2649172676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff80009b0f0 msgr2=0x7ff8000a2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:57.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.226+0000 7ff80e600700 1 --2- 172.21.15.131:0/2649172676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff80009b0f0 0x7ff8000a2c80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff808066c30 tx=0x7ff80806a7b0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.226+0000 7ff80e600700 1 -- 172.21.15.131:0/2649172676 shutdown_connections 2024-05-08T22:16:57.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.226+0000 7ff80e600700 1 --2- 172.21.15.131:0/2649172676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff80009b0f0 0x7ff8000a2c80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.226+0000 7ff80e600700 1 --2- 172.21.15.131:0/2649172676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff800006860 0x7ff800006c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.226+0000 7ff80e600700 1 --2- 172.21.15.131:0/2649172676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff800007210 0x7ff80009aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.226+0000 7ff80e600700 1 -- 172.21.15.131:0/2649172676 >> 172.21.15.131:0/2649172676 conn(0x7ff80000a800 msgr2=0x7ff80000ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:57.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.228+0000 7ff80e600700 1 -- 172.21.15.131:0/2649172676 shutdown_connections 2024-05-08T22:16:57.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.228+0000 7ff80e600700 1 -- 172.21.15.131:0/2649172676 wait complete. 2024-05-08T22:16:57.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.229+0000 7ff80e600700 1 Processor -- start 2024-05-08T22:16:57.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.230+0000 7ff80e600700 1 -- start start 2024-05-08T22:16:57.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.230+0000 7ff80e600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff800006860 0x7ff80012c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:57.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.230+0000 7ff80e600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff80009b0f0 0x7ff80012c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:57.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.230+0000 7ff80e600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff800130ab0 0x7ff800130f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:57.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.230+0000 7ff80e600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff800131530 con 0x7ff80009b0f0 2024-05-08T22:16:57.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.230+0000 7ff80e600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff800131670 con 0x7ff800006860 2024-05-08T22:16:57.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.230+0000 7ff80e600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8001317e0 con 0x7ff800130ab0 2024-05-08T22:16:57.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.231+0000 7ff80d200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff800006860 0x7ff80012c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:57.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.231+0000 7ff80d200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff800006860 0x7ff80012c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:44808/0 (socket says 172.21.15.131:44808) 2024-05-08T22:16:57.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.231+0000 7ff80d200700 1 -- 172.21.15.131:0/3353472514 learned_addr learned my addr 172.21.15.131:0/3353472514 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:57.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.231+0000 7ff807e00700 1 --2- 172.21.15.131:0/3353472514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff80009b0f0 0x7ff80012c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:57.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.231+0000 7ff80dc00700 1 --2- 172.21.15.131:0/3353472514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff800130ab0 0x7ff800130f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:57.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.232+0000 7ff807e00700 1 -- 172.21.15.131:0/3353472514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff800130ab0 msgr2=0x7ff800130f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:57.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.232+0000 7ff807e00700 1 --2- 172.21.15.131:0/3353472514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff800130ab0 0x7ff800130f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.232+0000 7ff807e00700 1 -- 172.21.15.131:0/3353472514 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff800006860 msgr2=0x7ff80012c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:57.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.232+0000 7ff807e00700 1 --2- 172.21.15.131:0/3353472514 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff800006860 0x7ff80012c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.232+0000 7ff807e00700 1 -- 172.21.15.131:0/3353472514 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80804f4d0 con 0x7ff80009b0f0 2024-05-08T22:16:57.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.233+0000 7ff807e00700 1 --2- 172.21.15.131:0/3353472514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff80009b0f0 0x7ff80012c710 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff80000bd40 tx=0x7ff7fc00b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:57.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.233+0000 7ff805600700 1 -- 172.21.15.131:0/3353472514 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc019070 con 0x7ff80009b0f0 2024-05-08T22:16:57.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.233+0000 7ff805600700 1 -- 172.21.15.131:0/3353472514 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7fc01d8d0 con 0x7ff80009b0f0 2024-05-08T22:16:57.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.233+0000 7ff805600700 1 -- 172.21.15.131:0/3353472514 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc01c910 con 0x7ff80009b0f0 2024-05-08T22:16:57.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.233+0000 7ff80e600700 1 -- 172.21.15.131:0/3353472514 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff800094e80 con 0x7ff80009b0f0 2024-05-08T22:16:57.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.233+0000 7ff80e600700 1 -- 172.21.15.131:0/3353472514 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff800095340 con 0x7ff80009b0f0 2024-05-08T22:16:57.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.241+0000 7ff8237a8b80 1 Processor -- start 2024-05-08T22:16:57.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.241+0000 7ff805600700 1 -- 172.21.15.131:0/3353472514 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7ff7fc01da40 con 0x7ff80009b0f0 2024-05-08T22:16:57.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.242+0000 7ff8237a8b80 1 -- start start 2024-05-08T22:16:57.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.243+0000 7ff8237a8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1eda7f050 0x55b1eda7f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:57.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.243+0000 7ff805600700 1 --2- 172.21.15.131:0/3353472514 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff7f00a1310 0x7ff7f00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:57.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.244+0000 7ff805600700 1 -- 172.21.15.131:0/3353472514 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7ff7fc0d4b70 con 0x7ff80009b0f0 2024-05-08T22:16:57.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.244+0000 7ff8237a8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1eda7f970 0x55b1eda79350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:57.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.244+0000 7ff8237a8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b1eda79890 0x55b1eda79cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:57.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.244+0000 7ff8237a8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ed9d0150 con 0x55b1eda7f970 2024-05-08T22:16:57.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.244+0000 7ff8237a8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ed9cdb10 con 0x55b1eda79890 2024-05-08T22:16:57.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.244+0000 7ff80d200700 1 --2- 172.21.15.131:0/3353472514 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff7f00a1310 0x7ff7f00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:57.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.244+0000 7ff8237a8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ed9c9400 con 0x55b1eda7f050 2024-05-08T22:16:57.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.245+0000 7ff80dc00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b1eda79890 0x55b1eda79cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:57.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.245+0000 7ff80dc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1eda7f050 0x55b1eda7f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:57.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.245+0000 7ff80dc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1eda7f050 0x55b1eda7f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46162/0 (socket says 172.21.15.131:46162) 2024-05-08T22:16:57.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.245+0000 7ff80dc00700 1 -- 172.21.15.131:0/2893901057 learned_addr learned my addr 172.21.15.131:0/2893901057 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:57.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.245+0000 7ff807e00700 1 --2- 172.21.15.131:0/2893901057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1eda7f970 0x55b1eda79350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:57.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.246+0000 7ff80d200700 1 --2- 172.21.15.131:0/3353472514 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff7f00a1310 0x7ff7f00a37d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4005950 tx=0x7ff7f4009310 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:57.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.246+0000 7ff80dc00700 1 -- 172.21.15.131:0/2893901057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b1eda79890 msgr2=0x55b1eda79cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:57.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.246+0000 7ff80dc00700 1 --2- 172.21.15.131:0/2893901057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b1eda79890 0x55b1eda79cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.246+0000 7ff80dc00700 1 -- 172.21.15.131:0/2893901057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1eda7f970 msgr2=0x55b1eda79350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:57.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.246+0000 7ff80dc00700 1 --2- 172.21.15.131:0/2893901057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1eda7f970 0x55b1eda79350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.246+0000 7ff80dc00700 1 -- 172.21.15.131:0/2893901057 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1ed84a3b0 con 0x55b1eda7f050 2024-05-08T22:16:57.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.250+0000 7ff80dc00700 1 --2- 172.21.15.131:0/2893901057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1eda7f050 0x55b1eda7f430 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff80804ff70 tx=0x7ff808066d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13eec608f25c2fb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:57.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.252+0000 7ff7fa000700 1 -- 172.21.15.131:0/2893901057 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff808081440 con 0x55b1eda7f050 2024-05-08T22:16:57.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.252+0000 7ff7fa000700 1 -- 172.21.15.131:0/2893901057 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff808088070 con 0x55b1eda7f050 2024-05-08T22:16:57.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.253+0000 7ff7fa000700 1 -- 172.21.15.131:0/2893901057 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff808081440 con 0x55b1eda7f050 2024-05-08T22:16:57.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.254+0000 7ff8237a8b80 1 -- 172.21.15.131:0/2893901057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1eda7f050 msgr2=0x55b1eda7f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:57.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.254+0000 7ff8237a8b80 1 --2- 172.21.15.131:0/2893901057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1eda7f050 0x55b1eda7f430 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff80804ff70 tx=0x7ff808066d30 comp rx=0 tx=0).stop 2024-05-08T22:16:57.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.255+0000 7ff8237a8b80 1 -- 172.21.15.131:0/2893901057 shutdown_connections 2024-05-08T22:16:57.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.255+0000 7ff8237a8b80 1 --2- 172.21.15.131:0/2893901057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b1eda79890 0x55b1eda79cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.256+0000 7ff8237a8b80 1 --2- 172.21.15.131:0/2893901057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1eda7f050 0x55b1eda7f430 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.256+0000 7ff8237a8b80 1 --2- 172.21.15.131:0/2893901057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1eda7f970 0x55b1eda79350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.256+0000 7ff8237a8b80 1 -- 172.21.15.131:0/2893901057 >> 172.21.15.131:0/2893901057 conn(0x55b1ed9d7210 msgr2=0x55b1eda742d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:57.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.256+0000 7ff8237a8b80 1 -- 172.21.15.131:0/2893901057 shutdown_connections 2024-05-08T22:16:57.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.257+0000 7ff8237a8b80 1 -- 172.21.15.131:0/2893901057 wait complete. 2024-05-08T22:16:57.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.257+0000 7ff8237a8b80 1 -- start start 2024-05-08T22:16:57.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.258+0000 7ff8237a8b80 1 Processor -- start 2024-05-08T22:16:57.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.260+0000 7ff8237a8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1eda79890 0x55b1eda858d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:57.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.260+0000 7ff8237a8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b1eda7f050 0x55b1eda82c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:57.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.260+0000 7ff8237a8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1eda7f970 0x55b1eda83300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:57.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.260+0000 7ff8237a8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ed965560 con 0x55b1eda79890 2024-05-08T22:16:57.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.260+0000 7ff8237a8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ed965200 con 0x55b1eda7f050 2024-05-08T22:16:57.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.260+0000 7ff8237a8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ed84c6a0 con 0x55b1eda7f970 2024-05-08T22:16:57.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.261+0000 7ff8237a8b80 10 client.0 _ll_register_callbacks cb 0x55b1ed96eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:16:57.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.261+0000 7ff807e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b1eda7f050 0x55b1eda82c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:57.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.261+0000 7ff807e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b1eda7f050 0x55b1eda82c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:44828/0 (socket says 172.21.15.131:44828) 2024-05-08T22:16:57.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.261+0000 7ff807e00700 1 -- 172.21.15.131:0/661889359 learned_addr learned my addr 172.21.15.131:0/661889359 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:57.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.262+0000 7ff807e00700 1 -- 172.21.15.131:0/661889359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1eda7f970 msgr2=0x55b1eda83300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:16:57.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.262+0000 7ff807e00700 1 --2- 172.21.15.131:0/661889359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1eda7f970 0x55b1eda83300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.262+0000 7ff807e00700 1 -- 172.21.15.131:0/661889359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1eda79890 msgr2=0x55b1eda858d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:57.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.262+0000 7ff807e00700 1 --2- 172.21.15.131:0/661889359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1eda79890 0x55b1eda858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.262+0000 7ff807e00700 1 -- 172.21.15.131:0/661889359 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8000a3490 con 0x55b1eda7f050 2024-05-08T22:16:57.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.262+0000 7ff7eea00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:16:57.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.262+0000 7ff807e00700 1 --2- 172.21.15.131:0/661889359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b1eda7f050 0x55b1eda82c10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc09dd40 tx=0x7ff7fc0a20c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63e382074b4ed9ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:57.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.263+0000 7ff7eea00700 1 -- 172.21.15.131:0/661889359 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc0a5740 con 0x55b1eda7f050 2024-05-08T22:16:57.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.264+0000 7ff7eea00700 1 -- 172.21.15.131:0/661889359 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7ff7fc0a9d10 con 0x55b1eda7f050 2024-05-08T22:16:57.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.264+0000 7ff8237a8b80 1 -- 172.21.15.131:0/661889359 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1ed922270 con 0x55b1eda7f050 2024-05-08T22:16:57.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.264+0000 7ff7eea00700 1 -- 172.21.15.131:0/661889359 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7fc0ba260 con 0x55b1eda7f050 2024-05-08T22:16:57.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.266+0000 7ff8237a8b80 10 client.26708 fetch_fsmap learned FSMap version 30 2024-05-08T22:16:57.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.266+0000 7ff8237a8b80 1 -- 172.21.15.131:0/661889359 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b1ed7f6b60 con 0x55b1eda7f050 2024-05-08T22:16:57.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.267+0000 7ff7eea00700 1 -- 172.21.15.131:0/661889359 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b1ed922270 con 0x55b1eda7f050 2024-05-08T22:16:57.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.267+0000 7ff7eea00700 1 -- 172.21.15.131:0/661889359 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff7fc0a9880 con 0x55b1eda7f050 2024-05-08T22:16:57.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.268+0000 7ff8237a8b80 10 client.26708 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:16:57.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.268+0000 7ff8237a8b80 10 client.26708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:16:57.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.268+0000 7ff8237a8b80 20 client.26708 populate_metadata read hostname 'smithi131' 2024-05-08T22:16:57.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.268+0000 7ff8237a8b80 1 --2- 172.21.15.131:0/661889359 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b1edaa4300 0x55b1edaa6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:57.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.268+0000 7ff8237a8b80 4 client.26708 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:16:57.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.268+0000 7ff8237a8b80 1 -- 172.21.15.131:0/661889359 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b1eda48810 con 0x55b1edaa4300 2024-05-08T22:16:57.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.268+0000 7ff80d200700 1 --2- 172.21.15.131:0/661889359 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b1edaa4300 0x55b1edaa6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:57.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.270+0000 7ff80d200700 1 --2- 172.21.15.131:0/661889359 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b1edaa4300 0x55b1edaa6720 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca600ac1a548458 server_cookie=cc1ea596af9b42de in_seq=0 out_seq=0 2024-05-08T22:16:57.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.270+0000 7ff7eea00700 10 client.26708 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:57.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.270+0000 7ff7eea00700 1 -- 172.21.15.131:0/661889359 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b1eda48810 con 0x55b1edaa4300 2024-05-08T22:16:57.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.271+0000 7ff7eea00700 10 client.26708 handle_command_reply: tid=0 2024-05-08T22:16:57.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.271+0000 7ff8237a8b80 1 client.26708 shutdown 2024-05-08T22:16:57.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.271+0000 7ff8237a8b80 1 -- 172.21.15.131:0/661889359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b1eda7f050 msgr2=0x55b1eda82c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:57.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.271+0000 7ff8237a8b80 1 --2- 172.21.15.131:0/661889359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b1eda7f050 0x55b1eda82c10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc09dd40 tx=0x7ff7fc0a20c0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.272+0000 7ff8237a8b80 1 -- 172.21.15.131:0/661889359 shutdown_connections 2024-05-08T22:16:57.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.272+0000 7ff8237a8b80 1 --2- 172.21.15.131:0/661889359 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b1edaa4300 0x55b1edaa6720 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.273+0000 7ff7eea00700 0 client.26708 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:57.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.273+0000 7ff8237a8b80 1 --2- 172.21.15.131:0/661889359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1eda7f970 0x55b1eda83300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.273+0000 7ff8237a8b80 1 --2- 172.21.15.131:0/661889359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1eda79890 0x55b1eda858d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.273+0000 7ff8237a8b80 1 --2- 172.21.15.131:0/661889359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b1eda7f050 0x55b1eda82c10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.273+0000 7ff8237a8b80 1 -- 172.21.15.131:0/661889359 >> 172.21.15.131:0/661889359 conn(0x55b1ed9d7210 msgr2=0x55b1eda740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:57.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.273+0000 7ff8237a8b80 1 -- 172.21.15.131:0/661889359 shutdown_connections 2024-05-08T22:16:57.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.273+0000 7ff8237a8b80 1 -- 172.21.15.131:0/661889359 wait complete. 2024-05-08T22:16:57.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.274+0000 7ff8237a8b80 20 client.26708 trim_cache size 0 max 16384 2024-05-08T22:16:57.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.309+0000 7ff8237a8b80 1 Processor -- start 2024-05-08T22:16:57.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.310+0000 7ff8237a8b80 1 -- start start 2024-05-08T22:16:57.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.310+0000 7ff8237a8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1eda79890 0x55b1eda79350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:57.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.310+0000 7ff8237a8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1eda7f050 0x55b1eda83300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:57.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.310+0000 7ff8237a8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b1eda7f970 0x55b1eda74dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:57.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.310+0000 7ff8237a8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1eda6c040 con 0x55b1eda7f050 2024-05-08T22:16:57.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.310+0000 7ff8237a8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1eda7ec20 con 0x55b1eda7f970 2024-05-08T22:16:57.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.310+0000 7ff8237a8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1edaa6970 con 0x55b1eda79890 2024-05-08T22:16:57.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.311+0000 7ff80dc00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b1eda7f970 0x55b1eda74dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:57.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.311+0000 7ff80d200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1eda79890 0x55b1eda79350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:57.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.311+0000 7ff80d200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1eda79890 0x55b1eda79350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46180/0 (socket says 172.21.15.131:46180) 2024-05-08T22:16:57.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.311+0000 7ff80d200700 1 -- 172.21.15.131:0/210116358 learned_addr learned my addr 172.21.15.131:0/210116358 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:57.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.312+0000 7ff807e00700 1 --2- 172.21.15.131:0/210116358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1eda7f050 0x55b1eda83300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:57.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.313+0000 7ff807e00700 1 -- 172.21.15.131:0/210116358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1eda79890 msgr2=0x55b1eda79350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:57.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.313+0000 7ff807e00700 1 --2- 172.21.15.131:0/210116358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1eda79890 0x55b1eda79350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.313+0000 7ff807e00700 1 -- 172.21.15.131:0/210116358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b1eda7f970 msgr2=0x55b1eda74dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:57.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.313+0000 7ff807e00700 1 --2- 172.21.15.131:0/210116358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b1eda7f970 0x55b1eda74dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.313+0000 7ff807e00700 1 -- 172.21.15.131:0/210116358 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1ed98f560 con 0x55b1eda7f050 2024-05-08T22:16:57.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.314+0000 7ff807e00700 1 --2- 172.21.15.131:0/210116358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1eda7f050 0x55b1eda83300 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc09dd90 tx=0x7ff7fc0a1790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229c731d902205d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:57.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.314+0000 7ff7faa00700 1 -- 172.21.15.131:0/210116358 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc0ba260 con 0x55b1eda7f050 2024-05-08T22:16:57.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.314+0000 7ff7faa00700 1 -- 172.21.15.131:0/210116358 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7fc0a9d10 con 0x55b1eda7f050 2024-05-08T22:16:57.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.314+0000 7ff7faa00700 1 -- 172.21.15.131:0/210116358 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc0c10b0 con 0x55b1eda7f050 2024-05-08T22:16:57.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.315+0000 7ff8237a8b80 1 -- 172.21.15.131:0/210116358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1eda7f050 msgr2=0x55b1eda83300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:57.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.315+0000 7ff8237a8b80 1 --2- 172.21.15.131:0/210116358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1eda7f050 0x55b1eda83300 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc09dd90 tx=0x7ff7fc0a1790 comp rx=0 tx=0).stop 2024-05-08T22:16:57.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.315+0000 7ff8237a8b80 1 -- 172.21.15.131:0/210116358 shutdown_connections 2024-05-08T22:16:57.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.315+0000 7ff8237a8b80 1 --2- 172.21.15.131:0/210116358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b1eda7f970 0x55b1eda74dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.315+0000 7ff8237a8b80 1 --2- 172.21.15.131:0/210116358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1eda79890 0x55b1eda79350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.315+0000 7ff8237a8b80 1 --2- 172.21.15.131:0/210116358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1eda7f050 0x55b1eda83300 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.316+0000 7ff8237a8b80 1 -- 172.21.15.131:0/210116358 >> 172.21.15.131:0/210116358 conn(0x55b1ed9d7210 msgr2=0x55b1eda79c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:57.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.316+0000 7ff8237a8b80 1 -- 172.21.15.131:0/210116358 shutdown_connections 2024-05-08T22:16:57.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.316+0000 7ff8237a8b80 1 -- 172.21.15.131:0/210116358 wait complete. 2024-05-08T22:16:57.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.316+0000 7ff8237a8b80 1 -- start start 2024-05-08T22:16:57.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.317+0000 7ff8237a8b80 1 Processor -- start 2024-05-08T22:16:57.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.318+0000 7ff8237a8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1eda79890 0x55b1eda8a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:57.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.318+0000 7ff8237a8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b1eda7f050 0x55b1eda7aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:57.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.318+0000 7ff8237a8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1eda7f970 0x55b1eda7af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:57.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.318+0000 7ff8237a8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1eda7b480 con 0x55b1eda79890 2024-05-08T22:16:57.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.318+0000 7ff8237a8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1eda7b5f0 con 0x55b1eda7f050 2024-05-08T22:16:57.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.318+0000 7ff8237a8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1eda7b7b0 con 0x55b1eda7f970 2024-05-08T22:16:57.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.318+0000 7ff8237a8b80 10 client.0 _ll_register_callbacks cb 0x55b1ed96eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:16:57.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.318+0000 7ff807e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b1eda7f050 0x55b1eda7aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:57.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.319+0000 7ff807e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b1eda7f050 0x55b1eda7aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:44850/0 (socket says 172.21.15.131:44850) 2024-05-08T22:16:57.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.319+0000 7ff807e00700 1 -- 172.21.15.131:0/2703883712 learned_addr learned my addr 172.21.15.131:0/2703883712 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:16:57.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.319+0000 7ff80dc00700 1 --2- 172.21.15.131:0/2703883712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1eda7f970 0x55b1eda7af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:57.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.319+0000 7ff80d200700 1 --2- 172.21.15.131:0/2703883712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1eda79890 0x55b1eda8a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:57.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.320+0000 7ff80dc00700 1 -- 172.21.15.131:0/2703883712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b1eda7f050 msgr2=0x55b1eda7aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:57.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.320+0000 7ff80dc00700 1 --2- 172.21.15.131:0/2703883712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b1eda7f050 0x55b1eda7aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.320+0000 7ff80dc00700 1 -- 172.21.15.131:0/2703883712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1eda79890 msgr2=0x55b1eda8a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:16:57.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.320+0000 7ff80dc00700 1 --2- 172.21.15.131:0/2703883712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1eda79890 0x55b1eda8a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.320+0000 7ff80dc00700 1 -- 172.21.15.131:0/2703883712 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b1ed7f6b60 con 0x55b1eda7f970 2024-05-08T22:16:57.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.320+0000 7ff80d200700 1 --2- 172.21.15.131:0/2703883712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1eda79890 0x55b1eda8a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:16:57.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.321+0000 7ff80dc00700 1 --2- 172.21.15.131:0/2703883712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1eda7f970 0x55b1eda7af40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x55b1eda85f80 tx=0x7ff808062520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47685eb08ca1005d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:16:57.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.321+0000 7ff7eea00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:16:57.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.321+0000 7ff7eea00700 1 -- 172.21.15.131:0/2703883712 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff808081c50 con 0x55b1eda7f970 2024-05-08T22:16:57.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.321+0000 7ff7eea00700 1 -- 172.21.15.131:0/2703883712 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7ff808081db0 con 0x55b1eda7f970 2024-05-08T22:16:57.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.322+0000 7ff8237a8b80 1 -- 172.21.15.131:0/2703883712 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff808088070 con 0x55b1eda7f970 2024-05-08T22:16:57.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.324+0000 7ff7eea00700 1 -- 172.21.15.131:0/2703883712 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff808066ac0 con 0x55b1eda7f970 2024-05-08T22:16:57.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.324+0000 7ff8237a8b80 10 client.26806 fetch_fsmap learned FSMap version 30 2024-05-08T22:16:57.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.324+0000 7ff8237a8b80 1 -- 172.21.15.131:0/2703883712 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b1ed84ecd0 con 0x55b1eda7f970 2024-05-08T22:16:57.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.324+0000 7ff7eea00700 1 -- 172.21.15.131:0/2703883712 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff808088070 con 0x55b1eda7f970 2024-05-08T22:16:57.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.324+0000 7ff7eea00700 1 -- 172.21.15.131:0/2703883712 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff8080767f0 con 0x55b1eda7f970 2024-05-08T22:16:57.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.325+0000 7ff8237a8b80 10 client.26806 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:16:57.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.325+0000 7ff8237a8b80 10 client.26806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:16:57.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.325+0000 7ff8237a8b80 20 client.26806 populate_metadata read hostname 'smithi131' 2024-05-08T22:16:57.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.325+0000 7ff8237a8b80 1 --2- 172.21.15.131:0/2703883712 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b1edaa4300 0x55b1edafcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:16:57.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.325+0000 7ff8237a8b80 4 client.26806 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:16:57.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.325+0000 7ff8237a8b80 1 -- 172.21.15.131:0/2703883712 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b1ed9c1ec0 con 0x55b1edaa4300 2024-05-08T22:16:57.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.326+0000 7ff80d200700 1 --2- 172.21.15.131:0/2703883712 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b1edaa4300 0x55b1edafcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:16:57.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.326+0000 7ff80d200700 1 --2- 172.21.15.131:0/2703883712 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b1edaa4300 0x55b1edafcab0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6313694d12808521 server_cookie=a9a33e74122824ba in_seq=0 out_seq=0 2024-05-08T22:16:57.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.326+0000 7ff7eea00700 10 client.26806 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:57.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.327+0000 7ff7eea00700 1 -- 172.21.15.131:0/2703883712 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b1ed9c1ec0 con 0x55b1edaa4300 2024-05-08T22:16:57.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.327+0000 7ff7eea00700 10 client.26806 handle_command_reply: tid=0 2024-05-08T22:16:57.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.327+0000 7ff8237a8b80 1 client.26806 shutdown 2024-05-08T22:16:57.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.327+0000 7ff8237a8b80 1 -- 172.21.15.131:0/2703883712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1eda7f970 msgr2=0x55b1eda7af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:57.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.327+0000 7ff8237a8b80 1 --2- 172.21.15.131:0/2703883712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1eda7f970 0x55b1eda7af40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x55b1eda85f80 tx=0x7ff808062520 comp rx=0 tx=0).stop 2024-05-08T22:16:57.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.327+0000 7ff8237a8b80 1 -- 172.21.15.131:0/2703883712 shutdown_connections 2024-05-08T22:16:57.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.327+0000 7ff8237a8b80 1 --2- 172.21.15.131:0/2703883712 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b1edaa4300 0x55b1edafcab0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.327+0000 7ff7eea00700 0 client.26806 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:16:57.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.327+0000 7ff8237a8b80 1 --2- 172.21.15.131:0/2703883712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b1eda7f970 0x55b1eda7af40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.327+0000 7ff8237a8b80 1 --2- 172.21.15.131:0/2703883712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b1eda79890 0x55b1eda8a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.327+0000 7ff8237a8b80 1 --2- 172.21.15.131:0/2703883712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b1eda7f050 0x55b1eda7aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.327+0000 7ff8237a8b80 1 -- 172.21.15.131:0/2703883712 >> 172.21.15.131:0/2703883712 conn(0x55b1ed9d7210 msgr2=0x55b1eda7ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:57.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.328+0000 7ff8237a8b80 1 -- 172.21.15.131:0/2703883712 shutdown_connections 2024-05-08T22:16:57.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.328+0000 7ff8237a8b80 1 -- 172.21.15.131:0/2703883712 wait complete. 2024-05-08T22:16:57.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.328+0000 7ff8237a8b80 20 client.26806 trim_cache size 0 max 16384 2024-05-08T22:16:57.330 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:16:57.330 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T22:16:57.330 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-08T22:16:57.330 INFO:teuthology.orchestra.run.smithi131.stdout: "77343017-da3f-48cf-87a8-13dad216b78f": { 2024-05-08T22:16:57.330 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-08T22:16:57.330 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "77343017-da3f-48cf-87a8-13dad216b78f", 2024-05-08T22:16:57.330 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-08T22:16:57.330 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:16:57.331 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:16:57.331 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:16:57.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.329+0000 7ff80e600700 1 -- 172.21.15.131:0/3353472514 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff7f00a1310 msgr2=0x7ff7f00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:57.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.329+0000 7ff80e600700 1 --2- 172.21.15.131:0/3353472514 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff7f00a1310 0x7ff7f00a37d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4005950 tx=0x7ff7f4009310 comp rx=0 tx=0).stop 2024-05-08T22:16:57.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.329+0000 7ff80e600700 1 -- 172.21.15.131:0/3353472514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff80009b0f0 msgr2=0x7ff80012c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:16:57.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.329+0000 7ff80e600700 1 --2- 172.21.15.131:0/3353472514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff80009b0f0 0x7ff80012c710 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff80000bd40 tx=0x7ff7fc00b5a0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.329+0000 7ff80e600700 1 -- 172.21.15.131:0/3353472514 shutdown_connections 2024-05-08T22:16:57.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.329+0000 7ff80e600700 1 --2- 172.21.15.131:0/3353472514 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff7f00a1310 0x7ff7f00a37d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.329+0000 7ff80e600700 1 --2- 172.21.15.131:0/3353472514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff800130ab0 0x7ff800130f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.329+0000 7ff80e600700 1 --2- 172.21.15.131:0/3353472514 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff800006860 0x7ff80012c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.329+0000 7ff80e600700 1 --2- 172.21.15.131:0/3353472514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff80009b0f0 0x7ff80012c710 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:16:57.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.329+0000 7ff80e600700 1 -- 172.21.15.131:0/3353472514 >> 172.21.15.131:0/3353472514 conn(0x7ff80000a800 msgr2=0x7ff8000932b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:16:57.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.330+0000 7ff80e600700 1 -- 172.21.15.131:0/3353472514 shutdown_connections 2024-05-08T22:16:57.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:16:57.330+0000 7ff80e600700 1 -- 172.21.15.131:0/3353472514 wait complete. 2024-05-08T22:16:57.368 INFO:tasks.cephfs.filesystem:scrub status for tag:77343017-da3f-48cf-87a8-13dad216b78f - {'path': '/', 'tag': '77343017-da3f-48cf-87a8-13dad216b78f', 'options': 'recursive,force'} 2024-05-08T22:16:57.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:57 smithi131 ceph-mon[37260]: scrub queued for path: / 2024-05-08T22:16:57.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:57 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:16:57.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:57 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:16:57.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:57 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:16:57.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:57 smithi157 ceph-mon[77746]: scrub queued for path: / 2024-05-08T22:16:57.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:57 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:16:57.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:57 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:16:57.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:57 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:16:57.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:57 smithi132 ceph-mon[78180]: scrub queued for path: / 2024-05-08T22:16:57.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:57 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:16:57.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:57 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:16:57.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:57 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:16:58.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:58 smithi157 ceph-mon[77746]: pgmap v299: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.9 MiB/s wr, 539 op/s 2024-05-08T22:16:58.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:16:58 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:16:58.800 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:58 smithi131 ceph-mon[37260]: pgmap v299: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.9 MiB/s wr, 539 op/s 2024-05-08T22:16:58.800 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:16:58 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:16:58.802 DEBUG:teuthology.parallel:result is None 2024-05-08T22:16:58.803 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-08T22:16:58.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:58 smithi132 ceph-mon[78180]: pgmap v299: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.9 MiB/s wr, 539 op/s 2024-05-08T22:16:58.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:16:58 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:16:58.875 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-08T22:16:58.875 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-08T22:16:58.886 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-08T22:16:58.886 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T22:16:58.886 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi131.front.sepia.ceph.com 2024-05-08T22:16:58.886 DEBUG:teuthology.orchestra.run.smithi131:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-08T22:16:58.956 DEBUG:teuthology.orchestra.run.smithi131:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-08T22:17:00.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:00 smithi157 ceph-mon[77746]: pgmap v300: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.9 MiB/s wr, 701 op/s 2024-05-08T22:17:00.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:00 smithi132 ceph-mon[78180]: pgmap v300: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.9 MiB/s wr, 701 op/s 2024-05-08T22:17:00.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:00 smithi131 ceph-mon[37260]: pgmap v300: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.9 MiB/s wr, 701 op/s 2024-05-08T22:17:02.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:02 smithi157 ceph-mon[77746]: pgmap v301: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 466 op/s 2024-05-08T22:17:02.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:02 smithi132 ceph-mon[78180]: pgmap v301: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 466 op/s 2024-05-08T22:17:02.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:02 smithi131 ceph-mon[37260]: pgmap v301: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 466 op/s 2024-05-08T22:17:04.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:04 smithi157 ceph-mon[77746]: pgmap v302: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 458 op/s 2024-05-08T22:17:04.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:04 smithi132 ceph-mon[78180]: pgmap v302: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 458 op/s 2024-05-08T22:17:04.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:04 smithi131 ceph-mon[37260]: pgmap v302: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 458 op/s 2024-05-08T22:17:06.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:06 smithi157 ceph-mon[77746]: pgmap v303: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 439 op/s 2024-05-08T22:17:06.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:06 smithi132 ceph-mon[78180]: pgmap v303: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 439 op/s 2024-05-08T22:17:06.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:06 smithi131 ceph-mon[37260]: pgmap v303: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 439 op/s 2024-05-08T22:17:08.845 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:08 smithi132 ceph-mon[78180]: pgmap v304: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 970 KiB/s wr, 291 op/s 2024-05-08T22:17:08.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:08 smithi131 ceph-mon[37260]: pgmap v304: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 970 KiB/s wr, 291 op/s 2024-05-08T22:17:09.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:08 smithi157 ceph-mon[77746]: pgmap v304: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 970 KiB/s wr, 291 op/s 2024-05-08T22:17:09.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:09 smithi132 ceph-mon[78180]: osdmap e95: 12 total, 12 up, 12 in 2024-05-08T22:17:09.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:09 smithi131 ceph-mon[37260]: osdmap e95: 12 total, 12 up, 12 in 2024-05-08T22:17:10.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:09 smithi157 ceph-mon[77746]: osdmap e95: 12 total, 12 up, 12 in 2024-05-08T22:17:10.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:10 smithi132 ceph-mon[78180]: pgmap v306: 129 pgs: 16 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 65 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 88 KiB/s wr, 48 op/s 2024-05-08T22:17:10.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:10 smithi131 ceph-mon[37260]: pgmap v306: 129 pgs: 16 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 65 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 88 KiB/s wr, 48 op/s 2024-05-08T22:17:11.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:10 smithi157 ceph-mon[77746]: pgmap v306: 129 pgs: 16 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 65 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 88 KiB/s wr, 48 op/s 2024-05-08T22:17:12.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:12 smithi132 ceph-mon[78180]: pgmap v307: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 88 KiB/s wr, 40 op/s 2024-05-08T22:17:12.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:12 smithi131 ceph-mon[37260]: pgmap v307: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 88 KiB/s wr, 40 op/s 2024-05-08T22:17:13.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:12 smithi157 ceph-mon[77746]: pgmap v307: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 88 KiB/s wr, 40 op/s 2024-05-08T22:17:14.028 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:13 smithi157 ceph-mon[77746]: pgmap v308: 129 pgs: 22 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 38 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 67 KiB/s wr, 28 op/s 2024-05-08T22:17:14.091 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:13 smithi132 ceph-mon[78180]: pgmap v308: 129 pgs: 22 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 38 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 67 KiB/s wr, 28 op/s 2024-05-08T22:17:14.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:13 smithi131 ceph-mon[37260]: pgmap v308: 129 pgs: 22 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 38 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 67 KiB/s wr, 28 op/s 2024-05-08T22:17:16.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:16 smithi157 ceph-mon[77746]: pgmap v309: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-05-08T22:17:16.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:16 smithi132 ceph-mon[78180]: pgmap v309: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-05-08T22:17:16.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:16 smithi131 ceph-mon[37260]: pgmap v309: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-05-08T22:17:17.641 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:17:17 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:17:17.214+0000 7fecd6800700 -1 calc_pg_upmaps abort due to max <= 0 2024-05-08T22:17:18.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:18 smithi157 ceph-mon[77746]: pgmap v310: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-08T22:17:18.844 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:18 smithi132 ceph-mon[78180]: pgmap v310: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-08T22:17:18.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:18 smithi131 ceph-mon[37260]: pgmap v310: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-08T22:17:20.785 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:20 smithi157 ceph-mon[77746]: pgmap v311: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-08T22:17:20.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:20 smithi132 ceph-mon[78180]: pgmap v311: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-08T22:17:20.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:20 smithi131 ceph-mon[37260]: pgmap v311: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-08T22:17:22.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:22 smithi157 ceph-mon[77746]: pgmap v312: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-05-08T22:17:22.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:22 smithi132 ceph-mon[78180]: pgmap v312: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-05-08T22:17:22.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:22 smithi131 ceph-mon[37260]: pgmap v312: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-05-08T22:17:23.780 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:23 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:17:23.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:23 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:17:23.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:23 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:17:24.780 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:24 smithi157 ceph-mon[77746]: pgmap v313: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-08T22:17:24.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:24 smithi132 ceph-mon[78180]: pgmap v313: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-08T22:17:24.894 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:24 smithi131 ceph-mon[37260]: pgmap v313: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-08T22:17:26.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:26 smithi132 ceph-mon[78180]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:17:26.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:26 smithi132 ceph-mon[78180]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:17:26.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:26 smithi132 ceph-mon[78180]: pgmap v314: 129 pgs: 17 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 57 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-08T22:17:26.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:26 smithi132 ceph-mon[78180]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:17:26.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:26 smithi132 ceph-mon[78180]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:17:26.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:26 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:26.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:26 smithi132 ceph-mon[78180]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:26.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:26 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:26.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:26 smithi132 ceph-mon[78180]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:26.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:26 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:26.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:26 smithi132 ceph-mon[78180]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:26.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:26 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:26.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:26 smithi132 ceph-mon[78180]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:26.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:26 smithi131 ceph-mon[37260]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:17:26.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:26 smithi131 ceph-mon[37260]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:17:26.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:26 smithi131 ceph-mon[37260]: pgmap v314: 129 pgs: 17 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 57 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-08T22:17:26.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:26 smithi131 ceph-mon[37260]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:17:26.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:26 smithi131 ceph-mon[37260]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:17:26.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:26 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:26.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:26 smithi131 ceph-mon[37260]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:26.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:26 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:26.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:26 smithi131 ceph-mon[37260]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:26.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:26 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:26.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:26 smithi131 ceph-mon[37260]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:26.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:26 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:26.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:26 smithi131 ceph-mon[37260]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:26.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:26 smithi157 ceph-mon[77746]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:17:26.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:26 smithi157 ceph-mon[77746]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:17:26.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:26 smithi157 ceph-mon[77746]: pgmap v314: 129 pgs: 17 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 57 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-08T22:17:26.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:26 smithi157 ceph-mon[77746]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:17:26.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:26 smithi157 ceph-mon[77746]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:17:26.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:26 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:26.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:26 smithi157 ceph-mon[77746]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:26.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:26 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:26.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:26 smithi157 ceph-mon[77746]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:26.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:26 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:26.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:26 smithi157 ceph-mon[77746]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:26.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:26 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:26.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:26 smithi157 ceph-mon[77746]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:27.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:27 smithi132 ceph-mon[78180]: Adjusting osd_memory_target on smithi157 to 182.5M 2024-05-08T22:17:27.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:27 smithi132 ceph-mon[78180]: Unable to set osd_memory_target on smithi157 to 191374848: error parsing value: Value '191374848' is below minimum 939524096 2024-05-08T22:17:27.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:27 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:27.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:27 smithi132 ceph-mon[78180]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:27.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:27 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:27.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:27 smithi132 ceph-mon[78180]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:27.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:27 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:27.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:27 smithi132 ceph-mon[78180]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:27.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:27 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:27.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:27 smithi132 ceph-mon[78180]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:27.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:27 smithi132 ceph-mon[78180]: Adjusting osd_memory_target on smithi132 to 182.5M 2024-05-08T22:17:27.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:27 smithi132 ceph-mon[78180]: Unable to set osd_memory_target on smithi132 to 191378432: error parsing value: Value '191378432' is below minimum 939524096 2024-05-08T22:17:27.369 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:17:27.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:27 smithi131 ceph-mon[37260]: Adjusting osd_memory_target on smithi157 to 182.5M 2024-05-08T22:17:27.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:27 smithi131 ceph-mon[37260]: Unable to set osd_memory_target on smithi157 to 191374848: error parsing value: Value '191374848' is below minimum 939524096 2024-05-08T22:17:27.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:27 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:27.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:27 smithi131 ceph-mon[37260]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:27.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:27 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:27.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:27 smithi131 ceph-mon[37260]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:27.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:27 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:27.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:27 smithi131 ceph-mon[37260]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:27.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:27 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:27.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:27 smithi131 ceph-mon[37260]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:27.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:27 smithi131 ceph-mon[37260]: Adjusting osd_memory_target on smithi132 to 182.5M 2024-05-08T22:17:27.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:27 smithi131 ceph-mon[37260]: Unable to set osd_memory_target on smithi132 to 191378432: error parsing value: Value '191378432' is below minimum 939524096 2024-05-08T22:17:27.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:27 smithi157 ceph-mon[77746]: Adjusting osd_memory_target on smithi157 to 182.5M 2024-05-08T22:17:27.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:27 smithi157 ceph-mon[77746]: Unable to set osd_memory_target on smithi157 to 191374848: error parsing value: Value '191374848' is below minimum 939524096 2024-05-08T22:17:27.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:27 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:27.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:27 smithi157 ceph-mon[77746]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:27.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:27 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:27.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:27 smithi157 ceph-mon[77746]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:27.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:27 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:27.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:27 smithi157 ceph-mon[77746]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:27.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:27 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:27.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:27 smithi157 ceph-mon[77746]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:27.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:27 smithi157 ceph-mon[77746]: Adjusting osd_memory_target on smithi132 to 182.5M 2024-05-08T22:17:27.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:27 smithi157 ceph-mon[77746]: Unable to set osd_memory_target on smithi132 to 191378432: error parsing value: Value '191378432' is below minimum 939524096 2024-05-08T22:17:27.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.723+0000 7ff5b2200700 1 -- 172.21.15.131:0/3941224261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5ac10d7b0 msgr2=0x7ff5ac10fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:27.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.723+0000 7ff5b2200700 1 --2- 172.21.15.131:0/3941224261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5ac10d7b0 0x7ff5ac10fbe0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400b0a0 tx=0x7ff5a400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.723+0000 7ff5b2200700 1 -- 172.21.15.131:0/3941224261 shutdown_connections 2024-05-08T22:17:27.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.724+0000 7ff5b2200700 1 --2- 172.21.15.131:0/3941224261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5ac10d7b0 0x7ff5ac10fbe0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.724+0000 7ff5b2200700 1 --2- 172.21.15.131:0/3941224261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5ac073370 0x7ff5ac073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.724+0000 7ff5b2200700 1 --2- 172.21.15.131:0/3941224261 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5ac073c90 0x7ff5ac10d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.724+0000 7ff5b2200700 1 -- 172.21.15.131:0/3941224261 >> 172.21.15.131:0/3941224261 conn(0x7ff5ac0fc9e0 msgr2=0x7ff5ac0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:27.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.724+0000 7ff5b2200700 1 -- 172.21.15.131:0/3941224261 shutdown_connections 2024-05-08T22:17:27.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.724+0000 7ff5b2200700 1 -- 172.21.15.131:0/3941224261 wait complete. 2024-05-08T22:17:27.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.727+0000 7ff5b2200700 1 Processor -- start 2024-05-08T22:17:27.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.727+0000 7ff5b2200700 1 -- start start 2024-05-08T22:17:27.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.728+0000 7ff5b2200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5ac073370 0x7ff5ac19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:27.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.729+0000 7ff5ab400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5ac073370 0x7ff5ac19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:27.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.729+0000 7ff5ab400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5ac073370 0x7ff5ac19f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:52878/0 (socket says 172.21.15.131:52878) 2024-05-08T22:17:27.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.729+0000 7ff5b2200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5ac073c90 0x7ff5ac19fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:27.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.729+0000 7ff5b2200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5ac10d7b0 0x7ff5ac199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:27.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.729+0000 7ff5b2200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac199e30 con 0x7ff5ac073c90 2024-05-08T22:17:27.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.729+0000 7ff5b2200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac199fa0 con 0x7ff5ac10d7b0 2024-05-08T22:17:27.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.729+0000 7ff5b2200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac19a160 con 0x7ff5ac073370 2024-05-08T22:17:27.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.729+0000 7ff5ab400700 1 -- 172.21.15.131:0/2666545255 learned_addr learned my addr 172.21.15.131:0/2666545255 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:27.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.730+0000 7ff5abe00700 1 --2- 172.21.15.131:0/2666545255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5ac10d7b0 0x7ff5ac199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:27.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.730+0000 7ff5ab400700 1 -- 172.21.15.131:0/2666545255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5ac10d7b0 msgr2=0x7ff5ac199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:27.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.730+0000 7ff5ab400700 1 --2- 172.21.15.131:0/2666545255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5ac10d7b0 0x7ff5ac199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.730+0000 7ff5ab400700 1 -- 172.21.15.131:0/2666545255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5ac073c90 msgr2=0x7ff5ac19fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:27.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.730+0000 7ff5ab400700 1 --2- 172.21.15.131:0/2666545255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5ac073c90 0x7ff5ac19fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.730+0000 7ff5ab400700 1 -- 172.21.15.131:0/2666545255 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a4009d00 con 0x7ff5ac073370 2024-05-08T22:17:27.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.730+0000 7ff5abe00700 1 --2- 172.21.15.131:0/2666545255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5ac10d7b0 0x7ff5ac199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:17:27.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.732+0000 7ff5ab400700 1 --2- 172.21.15.131:0/2666545255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5ac073370 0x7ff5ac19f5c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00b530 tx=0x7ff59c00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:27.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.735+0000 7ff5a3e00700 1 -- 172.21.15.131:0/2666545255 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c011830 con 0x7ff5ac073370 2024-05-08T22:17:27.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.735+0000 7ff5a3e00700 1 -- 172.21.15.131:0/2666545255 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff59c011e50 con 0x7ff5ac073370 2024-05-08T22:17:27.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.735+0000 7ff5b2200700 1 -- 172.21.15.131:0/2666545255 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5ac19a440 con 0x7ff5ac073370 2024-05-08T22:17:27.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.736+0000 7ff5b2200700 1 -- 172.21.15.131:0/2666545255 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ac1a6540 con 0x7ff5ac073370 2024-05-08T22:17:27.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.736+0000 7ff5a3e00700 1 -- 172.21.15.131:0/2666545255 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c00f2e0 con 0x7ff5ac073370 2024-05-08T22:17:27.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.742+0000 7ff5a3e00700 1 -- 172.21.15.131:0/2666545255 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7ff59c010380 con 0x7ff5ac073370 2024-05-08T22:17:27.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.744+0000 7ff5a3e00700 1 --2- 172.21.15.131:0/2666545255 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff5900a1460 0x7ff5900a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:27.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.746+0000 7ff5a3e00700 1 -- 172.21.15.131:0/2666545255 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7ff59c0ccb30 con 0x7ff5ac073370 2024-05-08T22:17:27.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.746+0000 7ff5aaa00700 1 --2- 172.21.15.131:0/2666545255 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff5900a1460 0x7ff5900a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:27.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.749+0000 7ff5aaa00700 1 --2- 172.21.15.131:0/2666545255 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff5900a1460 0x7ff5900a3920 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac19ae40 tx=0x7ff598005c80 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:27.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.750+0000 7ff5c5dd6b80 1 Processor -- start 2024-05-08T22:17:27.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.750+0000 7ff5c5dd6b80 1 -- start start 2024-05-08T22:17:27.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.751+0000 7ff5c5dd6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e07e8be00 0x555e07e8c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:27.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.751+0000 7ff5c5dd6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e07e8c720 0x555e07e86350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:27.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.751+0000 7ff5c5dd6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e07e86890 0x555e07e86cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:27.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.751+0000 7ff5c5dd6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e07ddd150 con 0x555e07e8be00 2024-05-08T22:17:27.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.751+0000 7ff5c5dd6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e07ddab10 con 0x555e07e86890 2024-05-08T22:17:27.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.751+0000 7ff5c5dd6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e07dd6400 con 0x555e07e8c720 2024-05-08T22:17:27.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.751+0000 7ff5aaa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e07e8c720 0x555e07e86350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:27.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.751+0000 7ff5aaa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e07e8c720 0x555e07e86350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:52884/0 (socket says 172.21.15.131:52884) 2024-05-08T22:17:27.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.751+0000 7ff5aaa00700 1 -- 172.21.15.131:0/3888135559 learned_addr learned my addr 172.21.15.131:0/3888135559 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:27.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.751+0000 7ff5ab400700 1 --2- 172.21.15.131:0/3888135559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e07e8be00 0x555e07e8c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:27.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.752+0000 7ff5abe00700 1 --2- 172.21.15.131:0/3888135559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e07e86890 0x555e07e86cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:27.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.753+0000 7ff5abe00700 1 -- 172.21.15.131:0/3888135559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e07e8c720 msgr2=0x555e07e86350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:27.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.753+0000 7ff5abe00700 1 --2- 172.21.15.131:0/3888135559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e07e8c720 0x555e07e86350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.753+0000 7ff5abe00700 1 -- 172.21.15.131:0/3888135559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e07e8be00 msgr2=0x555e07e8c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:27.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.754+0000 7ff5abe00700 1 --2- 172.21.15.131:0/3888135559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e07e8be00 0x555e07e8c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.754+0000 7ff5abe00700 1 -- 172.21.15.131:0/3888135559 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e07c573b0 con 0x555e07e86890 2024-05-08T22:17:27.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.754+0000 7ff5ab400700 1 --2- 172.21.15.131:0/3888135559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e07e8be00 0x555e07e8c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:17:27.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.755+0000 7ff5abe00700 1 --2- 172.21.15.131:0/3888135559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e07e86890 0x555e07e86cd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4009850 tx=0x7ff5a4009c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bae94e2fea75f6ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:27.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.759+0000 7ff597e00700 1 -- 172.21.15.131:0/3888135559 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a400e040 con 0x555e07e86890 2024-05-08T22:17:27.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.759+0000 7ff597e00700 1 -- 172.21.15.131:0/3888135559 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5a4009450 con 0x555e07e86890 2024-05-08T22:17:27.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.759+0000 7ff597e00700 1 -- 172.21.15.131:0/3888135559 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a40052d0 con 0x555e07e86890 2024-05-08T22:17:27.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.764+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/3888135559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e07e86890 msgr2=0x555e07e86cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:27.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.764+0000 7ff5c5dd6b80 1 --2- 172.21.15.131:0/3888135559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e07e86890 0x555e07e86cd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4009850 tx=0x7ff5a4009c50 comp rx=0 tx=0).stop 2024-05-08T22:17:27.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.764+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/3888135559 shutdown_connections 2024-05-08T22:17:27.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.764+0000 7ff5c5dd6b80 1 --2- 172.21.15.131:0/3888135559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e07e86890 0x555e07e86cd0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.764+0000 7ff5c5dd6b80 1 --2- 172.21.15.131:0/3888135559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e07e8be00 0x555e07e8c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.764+0000 7ff5c5dd6b80 1 --2- 172.21.15.131:0/3888135559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e07e8c720 0x555e07e86350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.764+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/3888135559 >> 172.21.15.131:0/3888135559 conn(0x555e07de4210 msgr2=0x555e07e80df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:27.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.764+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/3888135559 shutdown_connections 2024-05-08T22:17:27.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.766+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/3888135559 wait complete. 2024-05-08T22:17:27.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.767+0000 7ff5c5dd6b80 1 -- start start 2024-05-08T22:17:27.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.767+0000 7ff5c5dd6b80 1 Processor -- start 2024-05-08T22:17:27.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.771+0000 7ff5c5dd6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e07e86890 0x555e07e8fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:27.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.772+0000 7ff5c5dd6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e07e8be00 0x555e07e901e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:27.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.772+0000 7ff5c5dd6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e07e8c720 0x555e07e90720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:27.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.772+0000 7ff5aaa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e07e8be00 0x555e07e901e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:27.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.772+0000 7ff5aaa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e07e8be00 0x555e07e901e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:43772/0 (socket says 172.21.15.131:43772) 2024-05-08T22:17:27.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.772+0000 7ff5c5dd6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e07d72560 con 0x555e07e86890 2024-05-08T22:17:27.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.772+0000 7ff5c5dd6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e07d72200 con 0x555e07e8be00 2024-05-08T22:17:27.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.772+0000 7ff5c5dd6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e07c596a0 con 0x555e07e8c720 2024-05-08T22:17:27.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.772+0000 7ff5aaa00700 1 -- 172.21.15.131:0/690161174 learned_addr learned my addr 172.21.15.131:0/690161174 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:27.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.773+0000 7ff5c5dd6b80 10 client.0 _ll_register_callbacks cb 0x555e07d7bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:17:27.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.773+0000 7ff5aaa00700 1 -- 172.21.15.131:0/690161174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e07e8c720 msgr2=0x555e07e90720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:27.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.773+0000 7ff5aaa00700 1 --2- 172.21.15.131:0/690161174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e07e8c720 0x555e07e90720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.773+0000 7ff5aaa00700 1 -- 172.21.15.131:0/690161174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e07e86890 msgr2=0x555e07e8fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:27.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.773+0000 7ff5aaa00700 1 --2- 172.21.15.131:0/690161174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e07e86890 0x555e07e8fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.773+0000 7ff5aaa00700 1 -- 172.21.15.131:0/690161174 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ac1a6540 con 0x555e07e8be00 2024-05-08T22:17:27.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.777+0000 7ff5aaa00700 1 --2- 172.21.15.131:0/690161174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e07e8be00 0x555e07e901e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff59800f700 tx=0x7ff59800fac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49cca63f8e3bdf27 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:27.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.777+0000 7ff594c00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:17:27.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.778+0000 7ff594c00700 1 -- 172.21.15.131:0/690161174 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598004220 con 0x555e07e8be00 2024-05-08T22:17:27.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.778+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/690161174 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e07d2f270 con 0x555e07e8be00 2024-05-08T22:17:27.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.779+0000 7ff594c00700 1 -- 172.21.15.131:0/690161174 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7ff598004380 con 0x555e07e8be00 2024-05-08T22:17:27.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.779+0000 7ff594c00700 1 -- 172.21.15.131:0/690161174 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff598004540 con 0x555e07e8be00 2024-05-08T22:17:27.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.779+0000 7ff5c5dd6b80 10 client.26723 fetch_fsmap learned FSMap version 30 2024-05-08T22:17:27.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.779+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/690161174 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7ff5ac1108f0 con 0x555e07e8be00 2024-05-08T22:17:27.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.780+0000 7ff594c00700 1 -- 172.21.15.131:0/690161174 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555e07d2f270 con 0x555e07e8be00 2024-05-08T22:17:27.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.780+0000 7ff594c00700 1 -- 172.21.15.131:0/690161174 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff598023030 con 0x555e07e8be00 2024-05-08T22:17:27.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.780+0000 7ff5c5dd6b80 10 client.26723 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:17:27.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.780+0000 7ff5c5dd6b80 10 client.26723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:17:27.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.780+0000 7ff5c5dd6b80 20 client.26723 populate_metadata read hostname 'smithi131' 2024-05-08T22:17:27.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.780+0000 7ff5c5dd6b80 1 --2- 172.21.15.131:0/690161174 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x555e07eb9ac0 0x555e07ebbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:27.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.780+0000 7ff5c5dd6b80 4 client.26723 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:17:27.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.780+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/690161174 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e07e55810 con 0x555e07eb9ac0 2024-05-08T22:17:27.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.781+0000 7ff5ab400700 1 --2- 172.21.15.131:0/690161174 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x555e07eb9ac0 0x555e07ebbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:27.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.785+0000 7ff5ab400700 1 --2- 172.21.15.131:0/690161174 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x555e07eb9ac0 0x555e07ebbee0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ceca8ad16b9fa63 server_cookie=85e61dcd75c71acb in_seq=0 out_seq=0 2024-05-08T22:17:27.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.785+0000 7ff594c00700 10 client.26723 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:17:27.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.785+0000 7ff594c00700 1 -- 172.21.15.131:0/690161174 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555e07e55810 con 0x555e07eb9ac0 2024-05-08T22:17:27.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.785+0000 7ff594c00700 10 client.26723 handle_command_reply: tid=0 2024-05-08T22:17:27.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.786+0000 7ff5c5dd6b80 1 client.26723 shutdown 2024-05-08T22:17:27.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.786+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/690161174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e07e8be00 msgr2=0x555e07e901e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:27.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.787+0000 7ff5c5dd6b80 1 --2- 172.21.15.131:0/690161174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e07e8be00 0x555e07e901e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff59800f700 tx=0x7ff59800fac0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.787+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/690161174 shutdown_connections 2024-05-08T22:17:27.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.787+0000 7ff5c5dd6b80 1 --2- 172.21.15.131:0/690161174 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x555e07eb9ac0 0x555e07ebbee0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.787+0000 7ff5c5dd6b80 1 --2- 172.21.15.131:0/690161174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e07e8c720 0x555e07e90720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.787+0000 7ff5c5dd6b80 1 --2- 172.21.15.131:0/690161174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e07e86890 0x555e07e8fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.787+0000 7ff5c5dd6b80 1 --2- 172.21.15.131:0/690161174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e07e8be00 0x555e07e901e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.787+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/690161174 >> 172.21.15.131:0/690161174 conn(0x555e07de4210 msgr2=0x555e07e805a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:27.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.787+0000 7ff594c00700 0 client.26723 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:17:27.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.788+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/690161174 shutdown_connections 2024-05-08T22:17:27.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.788+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/690161174 wait complete. 2024-05-08T22:17:27.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.789+0000 7ff5c5dd6b80 20 client.26723 trim_cache size 0 max 16384 2024-05-08T22:17:27.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.822+0000 7ff5c5dd6b80 1 Processor -- start 2024-05-08T22:17:27.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.822+0000 7ff5c5dd6b80 1 -- start start 2024-05-08T22:17:27.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.823+0000 7ff5c5dd6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e07e86890 0x555e07e86350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:27.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.823+0000 7ff5c5dd6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e07e8be00 0x555e07e80d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:27.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.823+0000 7ff5c5dd6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e07e8c720 0x555e07e90510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:27.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.823+0000 7ff5ab400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e07e86890 0x555e07e86350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:27.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.824+0000 7ff5ab400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e07e86890 0x555e07e86350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:52904/0 (socket says 172.21.15.131:52904) 2024-05-08T22:17:27.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.824+0000 7ff5aaa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e07e8be00 0x555e07e80d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:27.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.824+0000 7ff5aaa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e07e8be00 0x555e07e80d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43746/0 (socket says 172.21.15.131:43746) 2024-05-08T22:17:27.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.824+0000 7ff5c5dd6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e07e827e0 con 0x555e07e8be00 2024-05-08T22:17:27.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.824+0000 7ff5c5dd6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e07e7b800 con 0x555e07e8c720 2024-05-08T22:17:27.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.824+0000 7ff5c5dd6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e07ec1110 con 0x555e07e86890 2024-05-08T22:17:27.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.824+0000 7ff5ab400700 1 -- 172.21.15.131:0/1483308130 learned_addr learned my addr 172.21.15.131:0/1483308130 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:27.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.824+0000 7ff5abe00700 1 --2- 172.21.15.131:0/1483308130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e07e8c720 0x555e07e90510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:27.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.825+0000 7ff5ab400700 1 -- 172.21.15.131:0/1483308130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e07e8c720 msgr2=0x555e07e90510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:27.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.825+0000 7ff5ab400700 1 --2- 172.21.15.131:0/1483308130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e07e8c720 0x555e07e90510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.825+0000 7ff5ab400700 1 -- 172.21.15.131:0/1483308130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e07e8be00 msgr2=0x555e07e80d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:27.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.825+0000 7ff5ab400700 1 --2- 172.21.15.131:0/1483308130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e07e8be00 0x555e07e80d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.825+0000 7ff5ab400700 1 -- 172.21.15.131:0/1483308130 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e07c03b60 con 0x555e07e86890 2024-05-08T22:17:27.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.826+0000 7ff5ab400700 1 --2- 172.21.15.131:0/1483308130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e07e86890 0x555e07e86350 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff59c095db0 tx=0x7ff59c004860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=667220b403bb415c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:27.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.826+0000 7ff5a1200700 1 -- 172.21.15.131:0/1483308130 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0b7ea0 con 0x555e07e86890 2024-05-08T22:17:27.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.827+0000 7ff5a1200700 1 -- 172.21.15.131:0/1483308130 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff59c09d6e0 con 0x555e07e86890 2024-05-08T22:17:27.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.827+0000 7ff5a1200700 1 -- 172.21.15.131:0/1483308130 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c099d70 con 0x555e07e86890 2024-05-08T22:17:27.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.828+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/1483308130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e07e86890 msgr2=0x555e07e86350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:27.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.828+0000 7ff5c5dd6b80 1 --2- 172.21.15.131:0/1483308130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e07e86890 0x555e07e86350 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff59c095db0 tx=0x7ff59c004860 comp rx=0 tx=0).stop 2024-05-08T22:17:27.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.829+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/1483308130 shutdown_connections 2024-05-08T22:17:27.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.829+0000 7ff5c5dd6b80 1 --2- 172.21.15.131:0/1483308130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e07e8c720 0x555e07e90510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.829+0000 7ff5c5dd6b80 1 --2- 172.21.15.131:0/1483308130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e07e86890 0x555e07e86350 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.829+0000 7ff5c5dd6b80 1 --2- 172.21.15.131:0/1483308130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e07e8be00 0x555e07e80d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.829+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/1483308130 >> 172.21.15.131:0/1483308130 conn(0x555e07de4210 msgr2=0x555e07e89e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:27.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.830+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/1483308130 shutdown_connections 2024-05-08T22:17:27.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.830+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/1483308130 wait complete. 2024-05-08T22:17:27.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.830+0000 7ff5c5dd6b80 1 -- start start 2024-05-08T22:17:27.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.831+0000 7ff5c5dd6b80 1 Processor -- start 2024-05-08T22:17:27.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.832+0000 7ff5c5dd6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e07e86890 0x555e07eb0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:27.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.832+0000 7ff5c5dd6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e07e8be00 0x555e07e87a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:27.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.832+0000 7ff5c5dd6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e07e8c720 0x555e07e87f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:27.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.832+0000 7ff5ab400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e07e86890 0x555e07eb0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:27.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.832+0000 7ff5abe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e07e8c720 0x555e07e87f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:27.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.833+0000 7ff5ab400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e07e86890 0x555e07eb0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:52920/0 (socket says 172.21.15.131:52920) 2024-05-08T22:17:27.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.833+0000 7ff5c5dd6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e07e88680 con 0x555e07e8c720 2024-05-08T22:17:27.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.833+0000 7ff5c5dd6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e07e887f0 con 0x555e07e8be00 2024-05-08T22:17:27.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.833+0000 7ff5c5dd6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e07f10000 con 0x555e07e86890 2024-05-08T22:17:27.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.833+0000 7ff5c5dd6b80 10 client.0 _ll_register_callbacks cb 0x555e07d7bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:17:27.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.833+0000 7ff5aaa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e07e8be00 0x555e07e87a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:27.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.833+0000 7ff5aaa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e07e8be00 0x555e07e87a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:43792/0 (socket says 172.21.15.131:43792) 2024-05-08T22:17:27.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.833+0000 7ff5aaa00700 1 -- 172.21.15.131:0/4090285736 learned_addr learned my addr 172.21.15.131:0/4090285736 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:27.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.834+0000 7ff5aaa00700 1 -- 172.21.15.131:0/4090285736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e07e86890 msgr2=0x555e07eb0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:27.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.834+0000 7ff5aaa00700 1 --2- 172.21.15.131:0/4090285736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e07e86890 0x555e07eb0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.834+0000 7ff5aaa00700 1 -- 172.21.15.131:0/4090285736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e07e8c720 msgr2=0x555e07e87f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:27.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.834+0000 7ff5aaa00700 1 --2- 172.21.15.131:0/4090285736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e07e8c720 0x555e07e87f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.834+0000 7ff5aaa00700 1 -- 172.21.15.131:0/4090285736 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ac19a440 con 0x555e07e8be00 2024-05-08T22:17:27.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.834+0000 7ff5ab400700 1 --2- 172.21.15.131:0/4090285736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e07e86890 0x555e07eb0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-08T22:17:27.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.834+0000 7ff5aaa00700 1 --2- 172.21.15.131:0/4090285736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e07e8be00 0x555e07e87a30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x555e07e91270 tx=0x7ff59801db00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ba29e2c33f4665 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:27.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.834+0000 7ff594c00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:17:27.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.838+0000 7ff594c00700 1 -- 172.21.15.131:0/4090285736 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598004c80 con 0x555e07e8be00 2024-05-08T22:17:27.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.838+0000 7ff594c00700 1 -- 172.21.15.131:0/4090285736 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7ff598004de0 con 0x555e07e8be00 2024-05-08T22:17:27.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.838+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/4090285736 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff5a4009450 con 0x555e07e8be00 2024-05-08T22:17:27.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.838+0000 7ff594c00700 1 -- 172.21.15.131:0/4090285736 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff59801d8e0 con 0x555e07e8be00 2024-05-08T22:17:27.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.838+0000 7ff5c5dd6b80 10 client.26729 fetch_fsmap learned FSMap version 30 2024-05-08T22:17:27.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.838+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/4090285736 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555e07d9c560 con 0x555e07e8be00 2024-05-08T22:17:27.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.838+0000 7ff594c00700 1 -- 172.21.15.131:0/4090285736 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5a4009450 con 0x555e07e8be00 2024-05-08T22:17:27.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.839+0000 7ff594c00700 1 -- 172.21.15.131:0/4090285736 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff59800f7e0 con 0x555e07e8be00 2024-05-08T22:17:27.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.839+0000 7ff5c5dd6b80 10 client.26729 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:17:27.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.839+0000 7ff5c5dd6b80 10 client.26729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:17:27.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.839+0000 7ff5c5dd6b80 20 client.26729 populate_metadata read hostname 'smithi131' 2024-05-08T22:17:27.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.839+0000 7ff5c5dd6b80 1 --2- 172.21.15.131:0/4090285736 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x555e07ea4360 0x555e07ea6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:27.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.839+0000 7ff5c5dd6b80 4 client.26729 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:17:27.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.839+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/4090285736 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555e07dceec0 con 0x555e07ea4360 2024-05-08T22:17:27.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.841+0000 7ff5ab400700 1 --2- 172.21.15.131:0/4090285736 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x555e07ea4360 0x555e07ea6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:27.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.842+0000 7ff5ab400700 1 --2- 172.21.15.131:0/4090285736 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x555e07ea4360 0x555e07ea6780 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a49672b8903448d9 server_cookie=fe82cd0080ba5ad8 in_seq=0 out_seq=0 2024-05-08T22:17:27.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.842+0000 7ff594c00700 10 client.26729 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:17:27.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.843+0000 7ff594c00700 1 -- 172.21.15.131:0/4090285736 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555e07dceec0 con 0x555e07ea4360 2024-05-08T22:17:27.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.843+0000 7ff594c00700 10 client.26729 handle_command_reply: tid=0 2024-05-08T22:17:27.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.843+0000 7ff5c5dd6b80 1 client.26729 shutdown 2024-05-08T22:17:27.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.843+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/4090285736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e07e8be00 msgr2=0x555e07e87a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:27.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.843+0000 7ff5c5dd6b80 1 --2- 172.21.15.131:0/4090285736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e07e8be00 0x555e07e87a30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x555e07e91270 tx=0x7ff59801db00 comp rx=0 tx=0).stop 2024-05-08T22:17:27.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.844+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/4090285736 shutdown_connections 2024-05-08T22:17:27.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.844+0000 7ff5c5dd6b80 1 --2- 172.21.15.131:0/4090285736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e07e8c720 0x555e07e87f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.844+0000 7ff5c5dd6b80 1 --2- 172.21.15.131:0/4090285736 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x555e07ea4360 0x555e07ea6780 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.844+0000 7ff5c5dd6b80 1 --2- 172.21.15.131:0/4090285736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e07e86890 0x555e07eb0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.844+0000 7ff594c00700 0 client.26729 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:17:27.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.844+0000 7ff5c5dd6b80 1 --2- 172.21.15.131:0/4090285736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e07e8be00 0x555e07e87a30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.844+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/4090285736 >> 172.21.15.131:0/4090285736 conn(0x555e07de4210 msgr2=0x555e07e8e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:27.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.844+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/4090285736 shutdown_connections 2024-05-08T22:17:27.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.845+0000 7ff5c5dd6b80 1 -- 172.21.15.131:0/4090285736 wait complete. 2024-05-08T22:17:27.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.849+0000 7ff5c5dd6b80 20 client.26729 trim_cache size 0 max 16384 2024-05-08T22:17:27.851 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:17:27.851 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-08T22:17:27.851 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-08T22:17:27.851 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:17:27.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.851+0000 7ff5b2200700 1 -- 172.21.15.131:0/2666545255 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff5900a1460 msgr2=0x7ff5900a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:27.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.851+0000 7ff5b2200700 1 --2- 172.21.15.131:0/2666545255 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff5900a1460 0x7ff5900a3920 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac19ae40 tx=0x7ff598005c80 comp rx=0 tx=0).stop 2024-05-08T22:17:27.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.851+0000 7ff5b2200700 1 -- 172.21.15.131:0/2666545255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5ac073370 msgr2=0x7ff5ac19f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:27.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.851+0000 7ff5b2200700 1 --2- 172.21.15.131:0/2666545255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5ac073370 0x7ff5ac19f5c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00b530 tx=0x7ff59c00b840 comp rx=0 tx=0).stop 2024-05-08T22:17:27.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.851+0000 7ff5b2200700 1 -- 172.21.15.131:0/2666545255 shutdown_connections 2024-05-08T22:17:27.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.851+0000 7ff5b2200700 1 --2- 172.21.15.131:0/2666545255 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7ff5900a1460 0x7ff5900a3920 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.851+0000 7ff5b2200700 1 --2- 172.21.15.131:0/2666545255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5ac10d7b0 0x7ff5ac199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.851+0000 7ff5b2200700 1 --2- 172.21.15.131:0/2666545255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5ac073370 0x7ff5ac19f5c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.851+0000 7ff5b2200700 1 --2- 172.21.15.131:0/2666545255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5ac073c90 0x7ff5ac19fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:27.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.851+0000 7ff5b2200700 1 -- 172.21.15.131:0/2666545255 >> 172.21.15.131:0/2666545255 conn(0x7ff5ac0fc9e0 msgr2=0x7ff5ac0fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:27.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.851+0000 7ff5b2200700 1 -- 172.21.15.131:0/2666545255 shutdown_connections 2024-05-08T22:17:27.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:27.851+0000 7ff5b2200700 1 -- 172.21.15.131:0/2666545255 wait complete. 2024-05-08T22:17:27.884 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T22:17:27.884 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:17:28.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:28 smithi131 ceph-mon[37260]: pgmap v315: 129 pgs: 16 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 64 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-08T22:17:28.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.259+0000 7fac8f000700 1 -- 172.21.15.131:0/4097959414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac88106180 msgr2=0x7fac8810dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:28.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.259+0000 7fac8f000700 1 --2- 172.21.15.131:0/4097959414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac88106180 0x7fac8810dd00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fac8000b0a0 tx=0x7fac8000b3b0 comp rx=0 tx=0).stop 2024-05-08T22:17:28.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.260+0000 7fac8f000700 1 -- 172.21.15.131:0/4097959414 shutdown_connections 2024-05-08T22:17:28.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.260+0000 7fac8f000700 1 --2- 172.21.15.131:0/4097959414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac88106180 0x7fac8810dd00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:28.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.260+0000 7fac8f000700 1 --2- 172.21.15.131:0/4097959414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac88101060 0x7fac88101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:28.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.260+0000 7fac8f000700 1 --2- 172.21.15.131:0/4097959414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac88101a10 0x7fac88105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:28.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.260+0000 7fac8f000700 1 -- 172.21.15.131:0/4097959414 >> 172.21.15.131:0/4097959414 conn(0x7fac880fc920 msgr2=0x7fac880fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:28.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.260+0000 7fac8f000700 1 -- 172.21.15.131:0/4097959414 shutdown_connections 2024-05-08T22:17:28.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.261+0000 7fac8f000700 1 -- 172.21.15.131:0/4097959414 wait complete. 2024-05-08T22:17:28.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.262+0000 7fac8f000700 1 Processor -- start 2024-05-08T22:17:28.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.262+0000 7fac8f000700 1 -- start start 2024-05-08T22:17:28.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.263+0000 7fac8f000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac88101060 0x7fac881993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:28.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.263+0000 7fac8f000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac88106180 0x7fac88199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:28.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.263+0000 7fac8f000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac8819dcb0 0x7fac8819e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:28.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.263+0000 7fac8f000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8819e730 con 0x7fac88106180 2024-05-08T22:17:28.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.263+0000 7fac8f000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8819e8a0 con 0x7fac88101060 2024-05-08T22:17:28.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.263+0000 7fac8f000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8819ea60 con 0x7fac8819dcb0 2024-05-08T22:17:28.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.263+0000 7fac87e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac88101060 0x7fac881993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:28.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.263+0000 7fac87e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac88101060 0x7fac881993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:43816/0 (socket says 172.21.15.131:43816) 2024-05-08T22:17:28.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.263+0000 7fac87e00700 1 -- 172.21.15.131:0/2198077785 learned_addr learned my addr 172.21.15.131:0/2198077785 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:28.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.264+0000 7fac8d000700 1 --2- 172.21.15.131:0/2198077785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac8819dcb0 0x7fac8819e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:28.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.264+0000 7fac87400700 1 --2- 172.21.15.131:0/2198077785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac88106180 0x7fac88199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:28.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.264+0000 7fac87e00700 1 -- 172.21.15.131:0/2198077785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac8819dcb0 msgr2=0x7fac8819e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:28.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.264+0000 7fac87e00700 1 --2- 172.21.15.131:0/2198077785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac8819dcb0 0x7fac8819e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:28.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.264+0000 7fac87e00700 1 -- 172.21.15.131:0/2198077785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac88106180 msgr2=0x7fac88199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:28.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.264+0000 7fac87e00700 1 --2- 172.21.15.131:0/2198077785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac88106180 0x7fac88199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:28.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.264+0000 7fac87e00700 1 -- 172.21.15.131:0/2198077785 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac80009d00 con 0x7fac88101060 2024-05-08T22:17:28.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.264+0000 7fac87400700 1 --2- 172.21.15.131:0/2198077785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac88106180 0x7fac88199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:17:28.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.266+0000 7fac87e00700 1 --2- 172.21.15.131:0/2198077785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac88101060 0x7fac881993d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fac7800c920 tx=0x7fac7800cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:28.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.266+0000 7fac84c00700 1 -- 172.21.15.131:0/2198077785 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac78007b00 con 0x7fac88101060 2024-05-08T22:17:28.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.266+0000 7fac84c00700 1 -- 172.21.15.131:0/2198077785 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac7800ce70 con 0x7fac88101060 2024-05-08T22:17:28.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.266+0000 7fac84c00700 1 -- 172.21.15.131:0/2198077785 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac780054e0 con 0x7fac88101060 2024-05-08T22:17:28.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.267+0000 7fac8f000700 1 -- 172.21.15.131:0/2198077785 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac881a5fc0 con 0x7fac88101060 2024-05-08T22:17:28.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.267+0000 7fac8f000700 1 -- 172.21.15.131:0/2198077785 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac881a6440 con 0x7fac88101060 2024-05-08T22:17:28.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.268+0000 7fac8f000700 1 -- 172.21.15.131:0/2198077785 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac680052f0 con 0x7fac88101060 2024-05-08T22:17:28.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.273+0000 7fac84c00700 1 -- 172.21.15.131:0/2198077785 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fac78004830 con 0x7fac88101060 2024-05-08T22:17:28.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.276+0000 7fac84c00700 1 --2- 172.21.15.131:0/2198077785 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fac6c0a1310 0x7fac6c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:28.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.276+0000 7fac87400700 1 --2- 172.21.15.131:0/2198077785 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fac6c0a1310 0x7fac6c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:28.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.276+0000 7fac84c00700 1 -- 172.21.15.131:0/2198077785 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fac78099f20 con 0x7fac88101060 2024-05-08T22:17:28.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.277+0000 7fac87400700 1 --2- 172.21.15.131:0/2198077785 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fac6c0a1310 0x7fac6c0a37d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fac74005fd0 tx=0x7fac74005d40 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:28.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.282+0000 7fac84c00700 1 -- 172.21.15.131:0/2198077785 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fac780995f0 con 0x7fac88101060 2024-05-08T22:17:28.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:28 smithi157 ceph-mon[77746]: pgmap v315: 129 pgs: 16 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 64 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-08T22:17:28.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:28 smithi132 ceph-mon[78180]: pgmap v315: 129 pgs: 16 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 64 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-08T22:17:28.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.764+0000 7fac8f000700 1 -- 172.21.15.131:0/2198077785 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac680055a0 con 0x7fac88101060 2024-05-08T22:17:28.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.765+0000 7fac84c00700 1 -- 172.21.15.131:0/2198077785 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10891 (secure 0 0 0) 0x7fac7801d030 con 0x7fac88101060 2024-05-08T22:17:28.766 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:17:28.766 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:05:14.034908+0000","modified":"2024-05-08T22:16:38.400949+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:17:28.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.769+0000 7fac8f000700 1 -- 172.21.15.131:0/2198077785 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fac6c0a1310 msgr2=0x7fac6c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:28.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.769+0000 7fac8f000700 1 --2- 172.21.15.131:0/2198077785 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fac6c0a1310 0x7fac6c0a37d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fac74005fd0 tx=0x7fac74005d40 comp rx=0 tx=0).stop 2024-05-08T22:17:28.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.769+0000 7fac8f000700 1 -- 172.21.15.131:0/2198077785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac88101060 msgr2=0x7fac881993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:28.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.769+0000 7fac8f000700 1 --2- 172.21.15.131:0/2198077785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac88101060 0x7fac881993d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fac7800c920 tx=0x7fac7800cc30 comp rx=0 tx=0).stop 2024-05-08T22:17:28.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.770+0000 7fac8f000700 1 -- 172.21.15.131:0/2198077785 shutdown_connections 2024-05-08T22:17:28.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.770+0000 7fac8f000700 1 --2- 172.21.15.131:0/2198077785 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fac6c0a1310 0x7fac6c0a37d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:28.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.770+0000 7fac8f000700 1 --2- 172.21.15.131:0/2198077785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac8819dcb0 0x7fac8819e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:28.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.770+0000 7fac8f000700 1 --2- 172.21.15.131:0/2198077785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac88101060 0x7fac881993d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:28.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.770+0000 7fac8f000700 1 --2- 172.21.15.131:0/2198077785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac88106180 0x7fac88199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:28.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.770+0000 7fac8f000700 1 -- 172.21.15.131:0/2198077785 >> 172.21.15.131:0/2198077785 conn(0x7fac880fc920 msgr2=0x7fac880febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:28.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.770+0000 7fac8f000700 1 -- 172.21.15.131:0/2198077785 shutdown_connections 2024-05-08T22:17:28.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:28.770+0000 7fac8f000700 1 -- 172.21.15.131:0/2198077785 wait complete. 2024-05-08T22:17:28.774 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 30 2024-05-08T22:17:28.802 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T22:17:29.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.026+0000 7f12bb200700 1 -- 172.21.15.131:0/2786482378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12bc100f90 msgr2=0x7f12bc101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.026+0000 7f12bb200700 1 --2- 172.21.15.131:0/2786482378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12bc100f90 0x7f12bc101370 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f12ac009990 tx=0x7f12ac009ca0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.026+0000 7f12bb200700 1 -- 172.21.15.131:0/2786482378 shutdown_connections 2024-05-08T22:17:29.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.026+0000 7f12bb200700 1 --2- 172.21.15.131:0/2786482378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12bc1060b0 0x7f12bc10dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.026+0000 7f12bb200700 1 --2- 172.21.15.131:0/2786482378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12bc100f90 0x7f12bc101370 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.026+0000 7f12bb200700 1 --2- 172.21.15.131:0/2786482378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12bc101940 0x7f12bc105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.026+0000 7f12bb200700 1 -- 172.21.15.131:0/2786482378 >> 172.21.15.131:0/2786482378 conn(0x7f12bc0fc810 msgr2=0x7f12bc0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:29.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.026+0000 7f12bb200700 1 -- 172.21.15.131:0/2786482378 shutdown_connections 2024-05-08T22:17:29.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.026+0000 7f12bb200700 1 -- 172.21.15.131:0/2786482378 wait complete. 2024-05-08T22:17:29.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.027+0000 7f12bb200700 1 Processor -- start 2024-05-08T22:17:29.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.027+0000 7f12bb200700 1 -- start start 2024-05-08T22:17:29.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.028+0000 7f12bb200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12bc100f90 0x7f12bc197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.028+0000 7f12bb200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12bc1060b0 0x7f12bc197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.028+0000 7f12bb200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12bc19bb20 0x7f12bc19bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.028+0000 7f12bb200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12bc19c5a0 con 0x7f12bc1060b0 2024-05-08T22:17:29.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.028+0000 7f12bb200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12bc19c710 con 0x7f12bc19bb20 2024-05-08T22:17:29.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.028+0000 7f12bb200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12bc19c8d0 con 0x7f12bc100f90 2024-05-08T22:17:29.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.028+0000 7f12b9400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12bc1060b0 0x7f12bc197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.028+0000 7f12b9400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12bc1060b0 0x7f12bc197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43806/0 (socket says 172.21.15.131:43806) 2024-05-08T22:17:29.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.028+0000 7f12b9400700 1 -- 172.21.15.131:0/2093001536 learned_addr learned my addr 172.21.15.131:0/2093001536 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:29.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.028+0000 7f12b9e00700 1 --2- 172.21.15.131:0/2093001536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12bc100f90 0x7f12bc197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.028+0000 7f12ba800700 1 --2- 172.21.15.131:0/2093001536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12bc19bb20 0x7f12bc19bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.028+0000 7f12b9e00700 1 -- 172.21.15.131:0/2093001536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12bc19bb20 msgr2=0x7f12bc19bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.028+0000 7f12b9e00700 1 --2- 172.21.15.131:0/2093001536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12bc19bb20 0x7f12bc19bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.028+0000 7f12b9e00700 1 -- 172.21.15.131:0/2093001536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12bc1060b0 msgr2=0x7f12bc197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.028+0000 7f12b9e00700 1 --2- 172.21.15.131:0/2093001536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12bc1060b0 0x7f12bc197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.028+0000 7f12b9e00700 1 -- 172.21.15.131:0/2093001536 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a8009520 con 0x7f12bc100f90 2024-05-08T22:17:29.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.028+0000 7f12ba800700 1 --2- 172.21.15.131:0/2093001536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12bc19bb20 0x7f12bc19bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:17:29.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.028+0000 7f12b9400700 1 --2- 172.21.15.131:0/2093001536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12bc1060b0 0x7f12bc197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:17:29.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.029+0000 7f12b9e00700 1 --2- 172.21.15.131:0/2093001536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12bc100f90 0x7f12bc197240 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0098e0 tx=0x7f12ac005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:29.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.029+0000 7f12b2a00700 1 -- 172.21.15.131:0/2093001536 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac01d070 con 0x7f12bc100f90 2024-05-08T22:17:29.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.029+0000 7f12b2a00700 1 -- 172.21.15.131:0/2093001536 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12ac022470 con 0x7f12bc100f90 2024-05-08T22:17:29.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.029+0000 7f12bb200700 1 -- 172.21.15.131:0/2093001536 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12ac0095f0 con 0x7f12bc100f90 2024-05-08T22:17:29.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.029+0000 7f12b2a00700 1 -- 172.21.15.131:0/2093001536 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac017650 con 0x7f12bc100f90 2024-05-08T22:17:29.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.029+0000 7f12bb200700 1 -- 172.21.15.131:0/2093001536 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12bc1a40c0 con 0x7f12bc100f90 2024-05-08T22:17:29.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.032+0000 7f12b2a00700 1 -- 172.21.15.131:0/2093001536 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f12ac0225e0 con 0x7f12bc100f90 2024-05-08T22:17:29.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.033+0000 7f12b2a00700 1 --2- 172.21.15.131:0/2093001536 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f12a00a1310 0x7f12a00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.033+0000 7f12d4bc8b80 1 Processor -- start 2024-05-08T22:17:29.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.033+0000 7f12d4bc8b80 1 -- start start 2024-05-08T22:17:29.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.033+0000 7f12b2a00700 1 -- 172.21.15.131:0/2093001536 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f12ac0ce360 con 0x7f12bc100f90 2024-05-08T22:17:29.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.033+0000 7f12b9400700 1 --2- 172.21.15.131:0/2093001536 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f12a00a1310 0x7f12a00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.034+0000 7f12d4bc8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d721f3f050 0x55d721f3f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.034+0000 7f12d4bc8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d721f3f970 0x55d721f39350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.034+0000 7f12d4bc8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d721f39890 0x55d721f39cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.034+0000 7f12d4bc8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d721e90150 con 0x55d721f39890 2024-05-08T22:17:29.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.034+0000 7f12d4bc8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d721e8db10 con 0x55d721f3f970 2024-05-08T22:17:29.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.034+0000 7f12d4bc8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d721e89400 con 0x55d721f3f050 2024-05-08T22:17:29.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.034+0000 7f12ba800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d721f39890 0x55d721f39cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.034+0000 7f12b9e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d721f3f050 0x55d721f3f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.034+0000 7f12ba800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d721f39890 0x55d721f39cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43816/0 (socket says 172.21.15.131:43816) 2024-05-08T22:17:29.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.034+0000 7f12ba800700 1 -- 172.21.15.131:0/2291274324 learned_addr learned my addr 172.21.15.131:0/2291274324 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:29.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.034+0000 7f12b9400700 1 --2- 172.21.15.131:0/2291274324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d721f3f970 0x55d721f39350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.034+0000 7f12b9400700 1 --2- 172.21.15.131:0/2093001536 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f12a00a1310 0x7f12a00a37d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f12bc198790 tx=0x7f12a8009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.034+0000 7f12b9e00700 1 -- 172.21.15.131:0/2291274324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d721f3f970 msgr2=0x55d721f39350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.034+0000 7f12b9e00700 1 --2- 172.21.15.131:0/2291274324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d721f3f970 0x55d721f39350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.034+0000 7f12b9e00700 1 -- 172.21.15.131:0/2291274324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d721f39890 msgr2=0x55d721f39cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.034+0000 7f12b9e00700 1 --2- 172.21.15.131:0/2291274324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d721f39890 0x55d721f39cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.034+0000 7f12b9e00700 1 -- 172.21.15.131:0/2291274324 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d721d0a3b0 con 0x55d721f3f050 2024-05-08T22:17:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.034+0000 7f12b9400700 1 --2- 172.21.15.131:0/2291274324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d721f3f970 0x55d721f39350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:17:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.034+0000 7f12ba800700 1 --2- 172.21.15.131:0/2291274324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d721f39890 0x55d721f39cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:17:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.035+0000 7f12b9e00700 1 --2- 172.21.15.131:0/2291274324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d721f3f050 0x55d721f3f430 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0cb670 tx=0x7f12ac09b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c29756c5b12d567 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.035+0000 7f12a7400700 1 -- 172.21.15.131:0/2291274324 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac09b8e0 con 0x55d721f3f050 2024-05-08T22:17:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.035+0000 7f12a7400700 1 -- 172.21.15.131:0/2291274324 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12ac09b1e0 con 0x55d721f3f050 2024-05-08T22:17:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.035+0000 7f12a7400700 1 -- 172.21.15.131:0/2291274324 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac022ac0 con 0x55d721f3f050 2024-05-08T22:17:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.035+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/2291274324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d721f3f050 msgr2=0x55d721f3f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.035+0000 7f12d4bc8b80 1 --2- 172.21.15.131:0/2291274324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d721f3f050 0x55d721f3f430 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0cb670 tx=0x7f12ac09b000 comp rx=0 tx=0).stop 2024-05-08T22:17:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.036+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/2291274324 shutdown_connections 2024-05-08T22:17:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.036+0000 7f12d4bc8b80 1 --2- 172.21.15.131:0/2291274324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d721f39890 0x55d721f39cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.036+0000 7f12d4bc8b80 1 --2- 172.21.15.131:0/2291274324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d721f3f050 0x55d721f3f430 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.036+0000 7f12d4bc8b80 1 --2- 172.21.15.131:0/2291274324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d721f3f970 0x55d721f39350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.036+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/2291274324 >> 172.21.15.131:0/2291274324 conn(0x55d721e97210 msgr2=0x55d721f342d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:29.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.036+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/2291274324 shutdown_connections 2024-05-08T22:17:29.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.036+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/2291274324 wait complete. 2024-05-08T22:17:29.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.036+0000 7f12d4bc8b80 1 -- start start 2024-05-08T22:17:29.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.036+0000 7f12d4bc8b80 1 Processor -- start 2024-05-08T22:17:29.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.037+0000 7f12d4bc8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d721f39890 0x55d721f42cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.037+0000 7f12d4bc8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d721f3f050 0x55d721f43200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.037+0000 7f12d4bc8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d721f3f970 0x55d721f43740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.037+0000 7f12d4bc8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d721e25560 con 0x55d721f3f970 2024-05-08T22:17:29.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.037+0000 7f12d4bc8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d721e25200 con 0x55d721f39890 2024-05-08T22:17:29.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.037+0000 7f12d4bc8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d721d0c6a0 con 0x55d721f3f050 2024-05-08T22:17:29.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.037+0000 7f12d4bc8b80 10 client.0 _ll_register_callbacks cb 0x55d721e2eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:17:29.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.037+0000 7f12b9e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d721f39890 0x55d721f42cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.037+0000 7f12ba800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d721f3f970 0x55d721f43740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.037+0000 7f12b9e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d721f39890 0x55d721f42cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:43850/0 (socket says 172.21.15.131:43850) 2024-05-08T22:17:29.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.037+0000 7f12b9e00700 1 -- 172.21.15.131:0/3342367153 learned_addr learned my addr 172.21.15.131:0/3342367153 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:29.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.037+0000 7f12b9400700 1 --2- 172.21.15.131:0/3342367153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d721f3f050 0x55d721f43200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.037+0000 7f12b9e00700 1 -- 172.21.15.131:0/3342367153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d721f3f050 msgr2=0x55d721f43200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:29.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.037+0000 7f12b9e00700 1 --2- 172.21.15.131:0/3342367153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d721f3f050 0x55d721f43200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.037+0000 7f12b9e00700 1 -- 172.21.15.131:0/3342367153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d721f3f970 msgr2=0x55d721f43740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:29.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.037+0000 7f12b9e00700 1 --2- 172.21.15.131:0/3342367153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d721f3f970 0x55d721f43740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.037+0000 7f12b9e00700 1 -- 172.21.15.131:0/3342367153 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a8017050 con 0x55d721f39890 2024-05-08T22:17:29.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.038+0000 7f12b9e00700 1 --2- 172.21.15.131:0/3342367153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d721f39890 0x55d721f42cc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f12ac097c50 tx=0x7f12ac00b630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c11b22e5b4b6dd68 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:29.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.038+0000 7f129be00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:17:29.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.038+0000 7f129be00700 1 -- 172.21.15.131:0/3342367153 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac09f010 con 0x55d721f39890 2024-05-08T22:17:29.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.038+0000 7f129be00700 1 -- 172.21.15.131:0/3342367153 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f12ac0a35d0 con 0x55d721f39890 2024-05-08T22:17:29.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.038+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/3342367153 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d721de2270 con 0x55d721f39890 2024-05-08T22:17:29.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.038+0000 7f129be00700 1 -- 172.21.15.131:0/3342367153 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12ac0a10b0 con 0x55d721f39890 2024-05-08T22:17:29.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.038+0000 7f12d4bc8b80 10 client.26747 fetch_fsmap learned FSMap version 30 2024-05-08T22:17:29.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.038+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/3342367153 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f12bc1a40c0 con 0x55d721f39890 2024-05-08T22:17:29.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.038+0000 7f129be00700 1 -- 172.21.15.131:0/3342367153 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12ac082030 con 0x55d721f39890 2024-05-08T22:17:29.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.039+0000 7f129be00700 1 -- 172.21.15.131:0/3342367153 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f12ac09a760 con 0x55d721f39890 2024-05-08T22:17:29.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.039+0000 7f12d4bc8b80 10 client.26747 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:17:29.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.039+0000 7f12d4bc8b80 10 client.26747 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:17:29.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.039+0000 7f12d4bc8b80 20 client.26747 populate_metadata read hostname 'smithi131' 2024-05-08T22:17:29.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.039+0000 7f12d4bc8b80 1 --2- 172.21.15.131:0/3342367153 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d721f52cb0 0x55d721f550d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.039+0000 7f12d4bc8b80 4 client.26747 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:17:29.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.039+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/3342367153 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d721f08810 con 0x55d721f52cb0 2024-05-08T22:17:29.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.039+0000 7f12b9400700 1 --2- 172.21.15.131:0/3342367153 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d721f52cb0 0x55d721f550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.039+0000 7f129be00700 10 client.26747 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:17:29.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.039+0000 7f12b9400700 1 --2- 172.21.15.131:0/3342367153 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d721f52cb0 0x55d721f550d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f16e6b81caf2635d server_cookie=3bde7be427b7c69 in_seq=0 out_seq=0 2024-05-08T22:17:29.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.040+0000 7f129be00700 1 -- 172.21.15.131:0/3342367153 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d721f08810 con 0x55d721f52cb0 2024-05-08T22:17:29.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.040+0000 7f129be00700 10 client.26747 handle_command_reply: tid=0 2024-05-08T22:17:29.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.040+0000 7f12d4bc8b80 1 client.26747 shutdown 2024-05-08T22:17:29.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.040+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/3342367153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d721f39890 msgr2=0x55d721f42cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.040+0000 7f12d4bc8b80 1 --2- 172.21.15.131:0/3342367153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d721f39890 0x55d721f42cc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f12ac097c50 tx=0x7f12ac00b630 comp rx=0 tx=0).stop 2024-05-08T22:17:29.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.040+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/3342367153 shutdown_connections 2024-05-08T22:17:29.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.040+0000 7f12d4bc8b80 1 --2- 172.21.15.131:0/3342367153 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d721f52cb0 0x55d721f550d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.040+0000 7f12d4bc8b80 1 --2- 172.21.15.131:0/3342367153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d721f3f970 0x55d721f43740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.040+0000 7f129be00700 0 client.26747 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:17:29.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.040+0000 7f12d4bc8b80 1 --2- 172.21.15.131:0/3342367153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d721f39890 0x55d721f42cc0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.040+0000 7f12d4bc8b80 1 --2- 172.21.15.131:0/3342367153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d721f3f050 0x55d721f43200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.040+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/3342367153 >> 172.21.15.131:0/3342367153 conn(0x55d721e97210 msgr2=0x55d721f340b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:29.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.041+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/3342367153 shutdown_connections 2024-05-08T22:17:29.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.041+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/3342367153 wait complete. 2024-05-08T22:17:29.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.041+0000 7f12d4bc8b80 20 client.26747 trim_cache size 0 max 16384 2024-05-08T22:17:29.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.054+0000 7f12d4bc8b80 1 Processor -- start 2024-05-08T22:17:29.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.054+0000 7f12d4bc8b80 1 -- start start 2024-05-08T22:17:29.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.055+0000 7f12d4bc8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d721f39890 0x55d721f39350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.055+0000 7f12d4bc8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d721f3f050 0x55d721f3e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.055+0000 7f12d4bc8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d721f3f970 0x55d721f43560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.055+0000 7f12d4bc8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d721f44240 con 0x55d721f3f970 2024-05-08T22:17:29.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.055+0000 7f12d4bc8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d721f5a2e0 con 0x55d721f3f050 2024-05-08T22:17:29.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.055+0000 7f12d4bc8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d721f41210 con 0x55d721f39890 2024-05-08T22:17:29.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.055+0000 7f12ba800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d721f3f970 0x55d721f43560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.055+0000 7f12b9e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d721f39890 0x55d721f39350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.055+0000 7f12b9400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d721f3f050 0x55d721f3e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.055+0000 7f12ba800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d721f3f970 0x55d721f43560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43832/0 (socket says 172.21.15.131:43832) 2024-05-08T22:17:29.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.055+0000 7f12ba800700 1 -- 172.21.15.131:0/2562623924 learned_addr learned my addr 172.21.15.131:0/2562623924 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:29.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.055+0000 7f12b9e00700 1 --2- 172.21.15.131:0/2562623924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d721f39890 0x55d721f39350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:52976/0 (socket says 172.21.15.131:52976) 2024-05-08T22:17:29.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.055+0000 7f12b9400700 1 -- 172.21.15.131:0/2562623924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d721f39890 msgr2=0x55d721f39350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:29.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.055+0000 7f12b9400700 1 --2- 172.21.15.131:0/2562623924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d721f39890 0x55d721f39350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.055+0000 7f12b9400700 1 -- 172.21.15.131:0/2562623924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d721f3f970 msgr2=0x55d721f43560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:29.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.055+0000 7f12b9400700 1 --2- 172.21.15.131:0/2562623924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d721f3f970 0x55d721f43560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.055+0000 7f12b9400700 1 -- 172.21.15.131:0/2562623924 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d721cb6b60 con 0x55d721f3f050 2024-05-08T22:17:29.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.055+0000 7f12ba800700 1 --2- 172.21.15.131:0/2562623924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d721f3f970 0x55d721f43560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:17:29.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.056+0000 7f12b9400700 1 --2- 172.21.15.131:0/2562623924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d721f3f050 0x55d721f3e3d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55d721f3a5b0 tx=0x7f12a8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1df7a32c6242f63 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:29.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.056+0000 7f12a7e00700 1 -- 172.21.15.131:0/2562623924 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a8022030 con 0x55d721f3f050 2024-05-08T22:17:29.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.056+0000 7f12a7e00700 1 -- 172.21.15.131:0/2562623924 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12a8023040 con 0x55d721f3f050 2024-05-08T22:17:29.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.056+0000 7f12a7e00700 1 -- 172.21.15.131:0/2562623924 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a8004d10 con 0x55d721f3f050 2024-05-08T22:17:29.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.056+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/2562623924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d721f3f050 msgr2=0x55d721f3e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.056+0000 7f12d4bc8b80 1 --2- 172.21.15.131:0/2562623924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d721f3f050 0x55d721f3e3d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55d721f3a5b0 tx=0x7f12a8014000 comp rx=0 tx=0).stop 2024-05-08T22:17:29.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.056+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/2562623924 shutdown_connections 2024-05-08T22:17:29.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.056+0000 7f12d4bc8b80 1 --2- 172.21.15.131:0/2562623924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d721f3f970 0x55d721f43560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.056+0000 7f12d4bc8b80 1 --2- 172.21.15.131:0/2562623924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d721f39890 0x55d721f39350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.056+0000 7f12d4bc8b80 1 --2- 172.21.15.131:0/2562623924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d721f3f050 0x55d721f3e3d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.056+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/2562623924 >> 172.21.15.131:0/2562623924 conn(0x55d721e97210 msgr2=0x55d721f3dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:29.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.056+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/2562623924 shutdown_connections 2024-05-08T22:17:29.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.056+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/2562623924 wait complete. 2024-05-08T22:17:29.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.057+0000 7f12d4bc8b80 1 -- start start 2024-05-08T22:17:29.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.057+0000 7f12d4bc8b80 1 Processor -- start 2024-05-08T22:17:29.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.057+0000 7f12d4bc8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d721f39890 0x55d721f46260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.057+0000 7f12d4bc8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d721f3f050 0x55d721f467a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.057+0000 7f12d4bc8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d721f3f970 0x55d721fa81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.057+0000 7f12d4bc8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d721f46f10 con 0x55d721f3f050 2024-05-08T22:17:29.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.057+0000 7f12d4bc8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d721fa86e0 con 0x55d721f3f970 2024-05-08T22:17:29.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.057+0000 7f12d4bc8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d721fa88a0 con 0x55d721f39890 2024-05-08T22:17:29.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.058+0000 7f12d4bc8b80 10 client.0 _ll_register_callbacks cb 0x55d721e2eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:17:29.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.058+0000 7f12b9e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d721f39890 0x55d721f46260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.058+0000 7f12b9400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d721f3f050 0x55d721f467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.058+0000 7f12b9e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d721f39890 0x55d721f46260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:52984/0 (socket says 172.21.15.131:52984) 2024-05-08T22:17:29.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.058+0000 7f12b9e00700 1 -- 172.21.15.131:0/2582707983 learned_addr learned my addr 172.21.15.131:0/2582707983 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:29.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.058+0000 7f12ba800700 1 --2- 172.21.15.131:0/2582707983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d721f3f970 0x55d721fa81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.058+0000 7f12b9e00700 1 -- 172.21.15.131:0/2582707983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d721f3f970 msgr2=0x55d721fa81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:29.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.058+0000 7f12b9e00700 1 --2- 172.21.15.131:0/2582707983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d721f3f970 0x55d721fa81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.058+0000 7f12b9e00700 1 -- 172.21.15.131:0/2582707983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d721f3f050 msgr2=0x55d721f467a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:29.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.058+0000 7f12b9e00700 1 --2- 172.21.15.131:0/2582707983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d721f3f050 0x55d721f467a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.058+0000 7f12b9e00700 1 -- 172.21.15.131:0/2582707983 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b4009360 con 0x55d721f39890 2024-05-08T22:17:29.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.058+0000 7f12b9400700 1 --2- 172.21.15.131:0/2582707983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d721f3f050 0x55d721f467a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:17:29.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.058+0000 7f12b9e00700 1 --2- 172.21.15.131:0/2582707983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d721f39890 0x55d721f46260 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f12ac099840 tx=0x7f12ac009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7447cd23dabc77ed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:29.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.058+0000 7f129be00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:17:29.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.058+0000 7f129be00700 1 -- 172.21.15.131:0/2582707983 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac09f010 con 0x55d721f39890 2024-05-08T22:17:29.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.058+0000 7f129be00700 1 -- 172.21.15.131:0/2582707983 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f12ac0a35d0 con 0x55d721f39890 2024-05-08T22:17:29.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.058+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/2582707983 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12ac09b1e0 con 0x55d721f39890 2024-05-08T22:17:29.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.059+0000 7f129be00700 1 -- 172.21.15.131:0/2582707983 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12ac09ad20 con 0x55d721f39890 2024-05-08T22:17:29.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.059+0000 7f12d4bc8b80 10 client.26839 fetch_fsmap learned FSMap version 30 2024-05-08T22:17:29.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.059+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/2582707983 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f12a800fa20 con 0x55d721f39890 2024-05-08T22:17:29.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.059+0000 7f129be00700 1 -- 172.21.15.131:0/2582707983 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12ac09b1e0 con 0x55d721f39890 2024-05-08T22:17:29.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.059+0000 7f129be00700 1 -- 172.21.15.131:0/2582707983 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f12ac0b9260 con 0x55d721f39890 2024-05-08T22:17:29.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.059+0000 7f12d4bc8b80 10 client.26839 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:17:29.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.059+0000 7f12d4bc8b80 10 client.26839 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:17:29.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.059+0000 7f12d4bc8b80 20 client.26839 populate_metadata read hostname 'smithi131' 2024-05-08T22:17:29.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.059+0000 7f12d4bc8b80 1 --2- 172.21.15.131:0/2582707983 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d721fa9be0 0x55d721fabfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.059+0000 7f12d4bc8b80 4 client.26839 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:17:29.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.059+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/2582707983 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d721e81ec0 con 0x55d721fa9be0 2024-05-08T22:17:29.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.059+0000 7f12b9400700 1 --2- 172.21.15.131:0/2582707983 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d721fa9be0 0x55d721fabfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.060+0000 7f12b9400700 1 --2- 172.21.15.131:0/2582707983 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d721fa9be0 0x55d721fabfd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cce6ba2e4bfabdf6 server_cookie=dfaebad1121eaafe in_seq=0 out_seq=0 2024-05-08T22:17:29.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.060+0000 7f129be00700 10 client.26839 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:17:29.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.060+0000 7f129be00700 1 -- 172.21.15.131:0/2582707983 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d721e81ec0 con 0x55d721fa9be0 2024-05-08T22:17:29.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.060+0000 7f129be00700 10 client.26839 handle_command_reply: tid=0 2024-05-08T22:17:29.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.060+0000 7f12d4bc8b80 1 client.26839 shutdown 2024-05-08T22:17:29.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.060+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/2582707983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d721f39890 msgr2=0x55d721f46260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.060+0000 7f12d4bc8b80 1 --2- 172.21.15.131:0/2582707983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d721f39890 0x55d721f46260 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f12ac099840 tx=0x7f12ac009ca0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.060+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/2582707983 shutdown_connections 2024-05-08T22:17:29.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.060+0000 7f12d4bc8b80 1 --2- 172.21.15.131:0/2582707983 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d721fa9be0 0x55d721fabfd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.060+0000 7f12d4bc8b80 1 --2- 172.21.15.131:0/2582707983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d721f3f970 0x55d721fa81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.060+0000 7f12d4bc8b80 1 --2- 172.21.15.131:0/2582707983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d721f39890 0x55d721f46260 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.060+0000 7f12d4bc8b80 1 --2- 172.21.15.131:0/2582707983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d721f3f050 0x55d721f467a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.060+0000 7f129be00700 0 client.26839 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:17:29.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.060+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/2582707983 >> 172.21.15.131:0/2582707983 conn(0x55d721e97210 msgr2=0x55d721f435a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:29.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.060+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/2582707983 shutdown_connections 2024-05-08T22:17:29.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.060+0000 7f12d4bc8b80 1 -- 172.21.15.131:0/2582707983 wait complete. 2024-05-08T22:17:29.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.061+0000 7f12d4bc8b80 20 client.26839 trim_cache size 0 max 16384 2024-05-08T22:17:29.067 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:17:29.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.061+0000 7f12bb200700 1 -- 172.21.15.131:0/2093001536 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f12a00a1310 msgr2=0x7f12a00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.061+0000 7f12bb200700 1 --2- 172.21.15.131:0/2093001536 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f12a00a1310 0x7f12a00a37d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f12bc198790 tx=0x7f12a8009260 comp rx=0 tx=0).stop 2024-05-08T22:17:29.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.061+0000 7f12bb200700 1 -- 172.21.15.131:0/2093001536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12bc100f90 msgr2=0x7f12bc197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.061+0000 7f12bb200700 1 --2- 172.21.15.131:0/2093001536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12bc100f90 0x7f12bc197240 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0098e0 tx=0x7f12ac005740 comp rx=0 tx=0).stop 2024-05-08T22:17:29.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.061+0000 7f12bb200700 1 -- 172.21.15.131:0/2093001536 shutdown_connections 2024-05-08T22:17:29.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.061+0000 7f12bb200700 1 --2- 172.21.15.131:0/2093001536 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f12a00a1310 0x7f12a00a37d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.061+0000 7f12bb200700 1 --2- 172.21.15.131:0/2093001536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12bc19bb20 0x7f12bc19bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.061+0000 7f12bb200700 1 --2- 172.21.15.131:0/2093001536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12bc100f90 0x7f12bc197240 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.061+0000 7f12bb200700 1 --2- 172.21.15.131:0/2093001536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12bc1060b0 0x7f12bc197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.061+0000 7f12bb200700 1 -- 172.21.15.131:0/2093001536 >> 172.21.15.131:0/2093001536 conn(0x7f12bc0fc810 msgr2=0x7f12bc0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:29.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.061+0000 7f12bb200700 1 -- 172.21.15.131:0/2093001536 shutdown_connections 2024-05-08T22:17:29.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.061+0000 7f12bb200700 1 -- 172.21.15.131:0/2093001536 wait complete. 2024-05-08T22:17:29.088 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T22:17:29.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:29 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2198077785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:17:29.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.468+0000 7f3acfa00700 1 -- 172.21.15.131:0/2124373315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ac810d9c0 msgr2=0x7f3ac810dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.468+0000 7f3acfa00700 1 --2- 172.21.15.131:0/2124373315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ac810d9c0 0x7f3ac810dda0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009580 tx=0x7f3ac0009890 comp rx=0 tx=0).stop 2024-05-08T22:17:29.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.471+0000 7f3acfa00700 1 -- 172.21.15.131:0/2124373315 shutdown_connections 2024-05-08T22:17:29.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.471+0000 7f3acfa00700 1 --2- 172.21.15.131:0/2124373315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ac8108960 0x7f3ac8114880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.471+0000 7f3acfa00700 1 --2- 172.21.15.131:0/2124373315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ac810d9c0 0x7f3ac810dda0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.471+0000 7f3acfa00700 1 --2- 172.21.15.131:0/2124373315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ac8107fa0 0x7f3ac8108420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.471+0000 7f3acfa00700 1 -- 172.21.15.131:0/2124373315 >> 172.21.15.131:0/2124373315 conn(0x7f3ac806bff0 msgr2=0x7f3ac806c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:29.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.471+0000 7f3acfa00700 1 -- 172.21.15.131:0/2124373315 shutdown_connections 2024-05-08T22:17:29.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.471+0000 7f3acfa00700 1 -- 172.21.15.131:0/2124373315 wait complete. 2024-05-08T22:17:29.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.473+0000 7f3acfa00700 1 Processor -- start 2024-05-08T22:17:29.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.473+0000 7f3acfa00700 1 -- start start 2024-05-08T22:17:29.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.475+0000 7f3acfa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ac8107fa0 0x7f3ac81a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.475+0000 7f3acfa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ac8108960 0x7f3ac81a6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.475+0000 7f3acfa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ac810d9c0 0x7f3ac81aaac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.475+0000 7f3acfa00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac81ab100 con 0x7f3ac810d9c0 2024-05-08T22:17:29.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.475+0000 7f3acfa00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac81ab270 con 0x7f3ac8107fa0 2024-05-08T22:17:29.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.475+0000 7f3acfa00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac81ab430 con 0x7f3ac8108960 2024-05-08T22:17:29.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.475+0000 7f3acda00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ac810d9c0 0x7f3ac81aaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.475+0000 7f3ac7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ac8108960 0x7f3ac81a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.475+0000 7f3acda00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ac810d9c0 0x7f3ac81aaac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43870/0 (socket says 172.21.15.131:43870) 2024-05-08T22:17:29.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.475+0000 7f3acda00700 1 -- 172.21.15.131:0/3408169501 learned_addr learned my addr 172.21.15.131:0/3408169501 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:29.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.476+0000 7f3acda00700 1 -- 172.21.15.131:0/3408169501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ac8108960 msgr2=0x7f3ac81a6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.476+0000 7f3acda00700 1 --2- 172.21.15.131:0/3408169501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ac8108960 0x7f3ac81a6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.476+0000 7f3acda00700 1 -- 172.21.15.131:0/3408169501 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ac8107fa0 msgr2=0x7f3ac81a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T22:17:29.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.476+0000 7f3acda00700 1 --2- 172.21.15.131:0/3408169501 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ac8107fa0 0x7f3ac81a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.476+0000 7f3acda00700 1 -- 172.21.15.131:0/3408169501 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab000bc60 con 0x7f3ac810d9c0 2024-05-08T22:17:29.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.476+0000 7f3ac7e00700 1 --2- 172.21.15.131:0/3408169501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ac8108960 0x7f3ac81a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:17:29.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.477+0000 7f3acda00700 1 --2- 172.21.15.131:0/3408169501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ac810d9c0 0x7f3ac81aaac0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000f4d0 tx=0x7f3ab000f7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:29.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.477+0000 7f3ac5600700 1 -- 172.21.15.131:0/3408169501 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0019070 con 0x7f3ac810d9c0 2024-05-08T22:17:29.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.477+0000 7f3ac5600700 1 -- 172.21.15.131:0/3408169501 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ab0009bd0 con 0x7f3ac810d9c0 2024-05-08T22:17:29.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.477+0000 7f3ac5600700 1 -- 172.21.15.131:0/3408169501 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0010040 con 0x7f3ac810d9c0 2024-05-08T22:17:29.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.479+0000 7f3acfa00700 1 -- 172.21.15.131:0/3408169501 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac00091e0 con 0x7f3ac810d9c0 2024-05-08T22:17:29.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.479+0000 7f3acfa00700 1 -- 172.21.15.131:0/3408169501 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac81b2d70 con 0x7f3ac810d9c0 2024-05-08T22:17:29.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.490+0000 7f3ac5600700 1 -- 172.21.15.131:0/3408169501 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f3ab0009030 con 0x7f3ac810d9c0 2024-05-08T22:17:29.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.491+0000 7f3ae347fb80 1 Processor -- start 2024-05-08T22:17:29.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.491+0000 7f3ae347fb80 1 -- start start 2024-05-08T22:17:29.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.492+0000 7f3ac5600700 1 --2- 172.21.15.131:0/3408169501 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3aac0a1460 0x7f3aac0a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.492+0000 7f3ac5600700 1 -- 172.21.15.131:0/3408169501 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f3ab00cc890 con 0x7f3ac810d9c0 2024-05-08T22:17:29.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.492+0000 7f3ae347fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425f3e00 0x5579425f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.492+0000 7f3ae347fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579425f4720 0x5579425ee350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.492+0000 7f3ae347fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579425ee890 0x5579425eecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.492+0000 7f3ae347fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557942545150 con 0x5579425ee890 2024-05-08T22:17:29.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.492+0000 7f3ae347fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557942542b10 con 0x5579425f4720 2024-05-08T22:17:29.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.492+0000 7f3ae347fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55794253e400 con 0x5579425f3e00 2024-05-08T22:17:29.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.493+0000 7f3acd000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425f3e00 0x5579425f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.493+0000 7f3acd000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425f3e00 0x5579425f41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53020/0 (socket says 172.21.15.131:53020) 2024-05-08T22:17:29.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.493+0000 7f3acd000700 1 -- 172.21.15.131:0/2887906495 learned_addr learned my addr 172.21.15.131:0/2887906495 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:29.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.493+0000 7f3acda00700 1 --2- 172.21.15.131:0/2887906495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579425ee890 0x5579425eecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.493+0000 7f3acd000700 1 -- 172.21.15.131:0/2887906495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579425f4720 msgr2=0x5579425ee350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:17:29.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.493+0000 7f3acd000700 1 --2- 172.21.15.131:0/2887906495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579425f4720 0x5579425ee350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.493+0000 7f3acd000700 1 -- 172.21.15.131:0/2887906495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579425ee890 msgr2=0x5579425eecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:29.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.493+0000 7f3acd000700 1 --2- 172.21.15.131:0/2887906495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579425ee890 0x5579425eecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.494+0000 7f3acd000700 1 -- 172.21.15.131:0/2887906495 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579423bf3b0 con 0x5579425f3e00 2024-05-08T22:17:29.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.494+0000 7f3acd000700 1 --2- 172.21.15.131:0/2887906495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425f3e00 0x5579425f41e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0000c00 tx=0x7f3ac0005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a049c3d414ab9bf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:29.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.494+0000 7f3abe000700 1 -- 172.21.15.131:0/2887906495 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac001c070 con 0x5579425f3e00 2024-05-08T22:17:29.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.495+0000 7f3abe000700 1 -- 172.21.15.131:0/2887906495 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ac0003680 con 0x5579425f3e00 2024-05-08T22:17:29.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.495+0000 7f3abe000700 1 -- 172.21.15.131:0/2887906495 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0003830 con 0x5579425f3e00 2024-05-08T22:17:29.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.497+0000 7f3ac7e00700 1 --2- 172.21.15.131:0/3408169501 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3aac0a1460 0x7f3aac0a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.497+0000 7f3ae347fb80 1 -- 172.21.15.131:0/2887906495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425f3e00 msgr2=0x5579425f41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.497+0000 7f3ae347fb80 1 --2- 172.21.15.131:0/2887906495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425f3e00 0x5579425f41e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0000c00 tx=0x7f3ac0005a90 comp rx=0 tx=0).stop 2024-05-08T22:17:29.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.497+0000 7f3ae347fb80 1 -- 172.21.15.131:0/2887906495 shutdown_connections 2024-05-08T22:17:29.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.497+0000 7f3ae347fb80 1 --2- 172.21.15.131:0/2887906495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579425ee890 0x5579425eecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.497+0000 7f3ae347fb80 1 --2- 172.21.15.131:0/2887906495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425f3e00 0x5579425f41e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.497+0000 7f3ae347fb80 1 --2- 172.21.15.131:0/2887906495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579425f4720 0x5579425ee350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.497+0000 7f3ae347fb80 1 -- 172.21.15.131:0/2887906495 >> 172.21.15.131:0/2887906495 conn(0x55794254c210 msgr2=0x5579425e8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:29.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.498+0000 7f3ae347fb80 1 -- 172.21.15.131:0/2887906495 shutdown_connections 2024-05-08T22:17:29.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.498+0000 7f3ac7e00700 1 --2- 172.21.15.131:0/3408169501 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3aac0a1460 0x7f3aac0a3920 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x5579425eabd0 tx=0x7f3ab800f040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:29.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.498+0000 7f3ae347fb80 1 -- 172.21.15.131:0/2887906495 wait complete. 2024-05-08T22:17:29.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.499+0000 7f3ae347fb80 1 -- start start 2024-05-08T22:17:29.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.499+0000 7f3ae347fb80 1 Processor -- start 2024-05-08T22:17:29.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.501+0000 7f3ae347fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425ee890 0x5579425f7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.501+0000 7f3ae347fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579425f3e00 0x5579425f8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.501+0000 7f3ae347fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579425f4720 0x5579425f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.501+0000 7f3acd000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425ee890 0x5579425f7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.501+0000 7f3acd000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425ee890 0x5579425f7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53028/0 (socket says 172.21.15.131:53028) 2024-05-08T22:17:29.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.501+0000 7f3ac7e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579425f3e00 0x5579425f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.501+0000 7f3ac7e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579425f3e00 0x5579425f8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:43910/0 (socket says 172.21.15.131:43910) 2024-05-08T22:17:29.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.501+0000 7f3acda00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579425f4720 0x5579425f85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.501+0000 7f3acda00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579425f4720 0x5579425f85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43884/0 (socket says 172.21.15.131:43884) 2024-05-08T22:17:29.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.502+0000 7f3ae347fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579424da560 con 0x5579425f4720 2024-05-08T22:17:29.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.502+0000 7f3ae347fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579424da200 con 0x5579425f3e00 2024-05-08T22:17:29.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.502+0000 7f3ae347fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579423c16a0 con 0x5579425ee890 2024-05-08T22:17:29.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.502+0000 7f3ae347fb80 10 client.0 _ll_register_callbacks cb 0x5579424e3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:17:29.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.502+0000 7f3acd000700 1 -- 172.21.15.131:0/2292548093 learned_addr learned my addr 172.21.15.131:0/2292548093 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:29.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.502+0000 7f3acd000700 1 -- 172.21.15.131:0/2292548093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579425f3e00 msgr2=0x5579425f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:29.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.502+0000 7f3acd000700 1 --2- 172.21.15.131:0/2292548093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579425f3e00 0x5579425f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.502+0000 7f3acd000700 1 -- 172.21.15.131:0/2292548093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579425f4720 msgr2=0x5579425f85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:29.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.502+0000 7f3acd000700 1 --2- 172.21.15.131:0/2292548093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579425f4720 0x5579425f85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.502+0000 7f3acd000700 1 -- 172.21.15.131:0/2292548093 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac8115180 con 0x5579425ee890 2024-05-08T22:17:29.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.503+0000 7f3acd000700 1 --2- 172.21.15.131:0/2292548093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425ee890 0x5579425f7b20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00098d0 tx=0x7f3ac00057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6875dc7d81e7a82c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:29.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.503+0000 7f3ab6a00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:17:29.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.503+0000 7f3ab6a00700 1 -- 172.21.15.131:0/2292548093 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac001c040 con 0x5579425ee890 2024-05-08T22:17:29.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.503+0000 7f3ab6a00700 1 -- 172.21.15.131:0/2292548093 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f3ac0003830 con 0x5579425ee890 2024-05-08T22:17:29.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.503+0000 7f3ae347fb80 1 -- 172.21.15.131:0/2292548093 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5579425bd810 con 0x5579425ee890 2024-05-08T22:17:29.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.503+0000 7f3ab6a00700 1 -- 172.21.15.131:0/2292548093 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ac000be90 con 0x5579425ee890 2024-05-08T22:17:29.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.503+0000 7f3ae347fb80 10 client.26851 fetch_fsmap learned FSMap version 30 2024-05-08T22:17:29.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.503+0000 7f3ae347fb80 1 -- 172.21.15.131:0/2292548093 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55794236bb60 con 0x5579425ee890 2024-05-08T22:17:29.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.504+0000 7f3ab6a00700 1 -- 172.21.15.131:0/2292548093 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5579425bd810 con 0x5579425ee890 2024-05-08T22:17:29.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.504+0000 7f3ab6a00700 1 -- 172.21.15.131:0/2292548093 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3ac000ec50 con 0x5579425ee890 2024-05-08T22:17:29.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.504+0000 7f3ae347fb80 10 client.26851 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:17:29.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.504+0000 7f3ae347fb80 10 client.26851 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:17:29.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.504+0000 7f3ae347fb80 20 client.26851 populate_metadata read hostname 'smithi131' 2024-05-08T22:17:29.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.504+0000 7f3ae347fb80 1 --2- 172.21.15.131:0/2292548093 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x557942607c90 0x55794260a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.504+0000 7f3ae347fb80 4 client.26851 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:17:29.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.504+0000 7f3ae347fb80 1 -- 172.21.15.131:0/2292548093 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557942536ec0 con 0x557942607c90 2024-05-08T22:17:29.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.505+0000 7f3ac7e00700 1 --2- 172.21.15.131:0/2292548093 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x557942607c90 0x55794260a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.505+0000 7f3ab6a00700 10 client.26851 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:17:29.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.505+0000 7f3ac7e00700 1 --2- 172.21.15.131:0/2292548093 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x557942607c90 0x55794260a080 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fde86a8a53b7821e server_cookie=1c4f8854e1c28e9 in_seq=0 out_seq=0 2024-05-08T22:17:29.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.506+0000 7f3ab6a00700 1 -- 172.21.15.131:0/2292548093 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557942536ec0 con 0x557942607c90 2024-05-08T22:17:29.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.506+0000 7f3ab6a00700 10 client.26851 handle_command_reply: tid=0 2024-05-08T22:17:29.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.506+0000 7f3ae347fb80 1 client.26851 shutdown 2024-05-08T22:17:29.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.507+0000 7f3ae347fb80 1 -- 172.21.15.131:0/2292548093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425ee890 msgr2=0x5579425f7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.507+0000 7f3ae347fb80 1 --2- 172.21.15.131:0/2292548093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425ee890 0x5579425f7b20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00098d0 tx=0x7f3ac00057f0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.507+0000 7f3ae347fb80 1 -- 172.21.15.131:0/2292548093 shutdown_connections 2024-05-08T22:17:29.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.507+0000 7f3ae347fb80 1 --2- 172.21.15.131:0/2292548093 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x557942607c90 0x55794260a080 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.507+0000 7f3ae347fb80 1 --2- 172.21.15.131:0/2292548093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579425f4720 0x5579425f85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.507+0000 7f3ae347fb80 1 --2- 172.21.15.131:0/2292548093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425ee890 0x5579425f7b20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.507+0000 7f3ae347fb80 1 --2- 172.21.15.131:0/2292548093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579425f3e00 0x5579425f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.507+0000 7f3ae347fb80 1 -- 172.21.15.131:0/2292548093 >> 172.21.15.131:0/2292548093 conn(0x55794254c210 msgr2=0x5579425e85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:29.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.507+0000 7f3ab6a00700 0 client.26851 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:17:29.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.507+0000 7f3ae347fb80 1 -- 172.21.15.131:0/2292548093 shutdown_connections 2024-05-08T22:17:29.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.507+0000 7f3ae347fb80 1 -- 172.21.15.131:0/2292548093 wait complete. 2024-05-08T22:17:29.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.507+0000 7f3ae347fb80 20 client.26851 trim_cache size 0 max 16384 2024-05-08T22:17:29.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:29 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2198077785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:17:29.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.527+0000 7f3ae347fb80 1 Processor -- start 2024-05-08T22:17:29.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.527+0000 7f3ae347fb80 1 -- start start 2024-05-08T22:17:29.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.528+0000 7f3ae347fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579425ee890 0x5579425ee350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.528+0000 7f3ae347fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579425f3e00 0x557942603b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.528+0000 7f3ae347fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425f4720 0x5579425e8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.528+0000 7f3ae347fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579425f4560 con 0x5579425f3e00 2024-05-08T22:17:29.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.528+0000 7f3ae347fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579425f9160 con 0x5579425ee890 2024-05-08T22:17:29.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.528+0000 7f3ae347fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55794260f1f0 con 0x5579425f4720 2024-05-08T22:17:29.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.528+0000 7f3acd000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579425ee890 0x5579425ee350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.528+0000 7f3acd000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579425ee890 0x5579425ee350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:43926/0 (socket says 172.21.15.131:43926) 2024-05-08T22:17:29.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.528+0000 7f3acd000700 1 -- 172.21.15.131:0/316142795 learned_addr learned my addr 172.21.15.131:0/316142795 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:29.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.528+0000 7f3ac7e00700 1 --2- 172.21.15.131:0/316142795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579425f3e00 0x557942603b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.528+0000 7f3acda00700 1 --2- 172.21.15.131:0/316142795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425f4720 0x5579425e8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.529+0000 7f3ac7e00700 1 -- 172.21.15.131:0/316142795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425f4720 msgr2=0x5579425e8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:29.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.529+0000 7f3ac7e00700 1 --2- 172.21.15.131:0/316142795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425f4720 0x5579425e8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.529+0000 7f3ac7e00700 1 -- 172.21.15.131:0/316142795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579425ee890 msgr2=0x5579425ee350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:29.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.529+0000 7f3ac7e00700 1 --2- 172.21.15.131:0/316142795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579425ee890 0x5579425ee350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.529+0000 7f3ac7e00700 1 -- 172.21.15.131:0/316142795 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557942504560 con 0x5579425f3e00 2024-05-08T22:17:29.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.529+0000 7f3ac7e00700 1 --2- 172.21.15.131:0/316142795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579425f3e00 0x557942603b60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800abe0 tx=0x7f3ab8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4a50afd2c62718 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:29.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.530+0000 7f3abea00700 1 -- 172.21.15.131:0/316142795 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8012040 con 0x5579425f3e00 2024-05-08T22:17:29.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.530+0000 7f3abea00700 1 -- 172.21.15.131:0/316142795 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ab801e040 con 0x5579425f3e00 2024-05-08T22:17:29.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.530+0000 7f3abea00700 1 -- 172.21.15.131:0/316142795 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab80258a0 con 0x5579425f3e00 2024-05-08T22:17:29.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.531+0000 7f3ae347fb80 1 -- 172.21.15.131:0/316142795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579425f3e00 msgr2=0x557942603b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.531+0000 7f3ae347fb80 1 --2- 172.21.15.131:0/316142795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579425f3e00 0x557942603b60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800abe0 tx=0x7f3ab8013000 comp rx=0 tx=0).stop 2024-05-08T22:17:29.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.531+0000 7f3ae347fb80 1 -- 172.21.15.131:0/316142795 shutdown_connections 2024-05-08T22:17:29.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.531+0000 7f3ae347fb80 1 --2- 172.21.15.131:0/316142795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425f4720 0x5579425e8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.531+0000 7f3ae347fb80 1 --2- 172.21.15.131:0/316142795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579425ee890 0x5579425ee350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.531+0000 7f3ae347fb80 1 --2- 172.21.15.131:0/316142795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579425f3e00 0x557942603b60 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.531+0000 7f3ae347fb80 1 -- 172.21.15.131:0/316142795 >> 172.21.15.131:0/316142795 conn(0x55794254c210 msgr2=0x5579425e5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:29.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.531+0000 7f3ae347fb80 1 -- 172.21.15.131:0/316142795 shutdown_connections 2024-05-08T22:17:29.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.531+0000 7f3ae347fb80 1 -- 172.21.15.131:0/316142795 wait complete. 2024-05-08T22:17:29.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.531+0000 7f3ae347fb80 1 -- start start 2024-05-08T22:17:29.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.532+0000 7f3ae347fb80 1 Processor -- start 2024-05-08T22:17:29.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.533+0000 7f3ae347fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579425ee890 0x5579425ff590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.533+0000 7f3ae347fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425f3e00 0x5579425f0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.533+0000 7f3ae347fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579425f4720 0x5579425ef9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.533+0000 7f3ae347fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579425f1680 con 0x5579425ee890 2024-05-08T22:17:29.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.533+0000 7f3ae347fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579425f17f0 con 0x5579425f4720 2024-05-08T22:17:29.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.533+0000 7f3ac7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425f3e00 0x5579425f0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.533+0000 7f3ae347fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579425efee0 con 0x5579425f3e00 2024-05-08T22:17:29.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.533+0000 7f3ae347fb80 10 client.0 _ll_register_callbacks cb 0x5579424e3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:17:29.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.533+0000 7f3acda00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579425f4720 0x5579425ef9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.533+0000 7f3ac7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425f3e00 0x5579425f0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53038/0 (socket says 172.21.15.131:53038) 2024-05-08T22:17:29.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.533+0000 7f3ac7e00700 1 -- 172.21.15.131:0/3549241045 learned_addr learned my addr 172.21.15.131:0/3549241045 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:29.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.533+0000 7f3acd000700 1 --2- 172.21.15.131:0/3549241045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579425ee890 0x5579425ff590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.533+0000 7f3ac7e00700 1 -- 172.21.15.131:0/3549241045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579425f4720 msgr2=0x5579425ef9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:29.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.533+0000 7f3ac7e00700 1 --2- 172.21.15.131:0/3549241045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579425f4720 0x5579425ef9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.533+0000 7f3ac7e00700 1 -- 172.21.15.131:0/3549241045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579425ee890 msgr2=0x5579425ff590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:29.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.534+0000 7f3ac7e00700 1 --2- 172.21.15.131:0/3549241045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579425ee890 0x5579425ff590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.534+0000 7f3ac7e00700 1 -- 172.21.15.131:0/3549241045 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab800ddc0 con 0x5579425f3e00 2024-05-08T22:17:29.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.534+0000 7f3acda00700 1 --2- 172.21.15.131:0/3549241045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579425f4720 0x5579425ef9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:17:29.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.534+0000 7f3acd000700 1 --2- 172.21.15.131:0/3549241045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579425ee890 0x5579425ff590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:17:29.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.534+0000 7f3ac7e00700 1 --2- 172.21.15.131:0/3549241045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425f3e00 0x5579425f0da0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800f010 tx=0x7f3ab8028a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7de76b4fd0fb91d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:29.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.534+0000 7f3ab6a00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:17:29.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.534+0000 7f3ab6a00700 1 -- 172.21.15.131:0/3549241045 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8012040 con 0x5579425f3e00 2024-05-08T22:17:29.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.535+0000 7f3ae347fb80 1 -- 172.21.15.131:0/3549241045 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557942497270 con 0x5579425f3e00 2024-05-08T22:17:29.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.535+0000 7f3ab6a00700 1 -- 172.21.15.131:0/3549241045 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f3ab8028e40 con 0x5579425f3e00 2024-05-08T22:17:29.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.535+0000 7f3ab6a00700 1 -- 172.21.15.131:0/3549241045 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ab8004020 con 0x5579425f3e00 2024-05-08T22:17:29.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.535+0000 7f3ae347fb80 10 client.26857 fetch_fsmap learned FSMap version 30 2024-05-08T22:17:29.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.535+0000 7f3ae347fb80 1 -- 172.21.15.131:0/3549241045 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5579423c3cd0 con 0x5579425f3e00 2024-05-08T22:17:29.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.535+0000 7f3ab6a00700 1 -- 172.21.15.131:0/3549241045 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557942497270 con 0x5579425f3e00 2024-05-08T22:17:29.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.535+0000 7f3ab6a00700 1 -- 172.21.15.131:0/3549241045 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3ab8004750 con 0x5579425f3e00 2024-05-08T22:17:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.536+0000 7f3ae347fb80 10 client.26857 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:17:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.536+0000 7f3ae347fb80 10 client.26857 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:17:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.536+0000 7f3ae347fb80 20 client.26857 populate_metadata read hostname 'smithi131' 2024-05-08T22:17:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.536+0000 7f3ae347fb80 1 --2- 172.21.15.131:0/3549241045 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55794265ed20 0x557942661140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.536+0000 7f3ae347fb80 4 client.26857 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:17:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.536+0000 7f3ae347fb80 1 -- 172.21.15.131:0/3549241045 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55794238ccf0 con 0x55794265ed20 2024-05-08T22:17:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.536+0000 7f3acd000700 1 --2- 172.21.15.131:0/3549241045 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55794265ed20 0x557942661140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.537+0000 7f3acd000700 1 --2- 172.21.15.131:0/3549241045 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55794265ed20 0x557942661140 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f1b6227e1262f4e server_cookie=8fb51f8839801443 in_seq=0 out_seq=0 2024-05-08T22:17:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.537+0000 7f3ab6a00700 10 client.26857 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:17:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.537+0000 7f3ab6a00700 1 -- 172.21.15.131:0/3549241045 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55794238ccf0 con 0x55794265ed20 2024-05-08T22:17:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.537+0000 7f3ab6a00700 10 client.26857 handle_command_reply: tid=0 2024-05-08T22:17:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.537+0000 7f3ae347fb80 1 client.26857 shutdown 2024-05-08T22:17:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.538+0000 7f3ae347fb80 1 -- 172.21.15.131:0/3549241045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425f3e00 msgr2=0x5579425f0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.538+0000 7f3ae347fb80 1 --2- 172.21.15.131:0/3549241045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425f3e00 0x5579425f0da0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800f010 tx=0x7f3ab8028a10 comp rx=0 tx=0).stop 2024-05-08T22:17:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.538+0000 7f3ae347fb80 1 -- 172.21.15.131:0/3549241045 shutdown_connections 2024-05-08T22:17:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.538+0000 7f3ae347fb80 1 --2- 172.21.15.131:0/3549241045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579425f4720 0x5579425ef9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.538+0000 7f3ae347fb80 1 --2- 172.21.15.131:0/3549241045 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55794265ed20 0x557942661140 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.538+0000 7f3ae347fb80 1 --2- 172.21.15.131:0/3549241045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579425ee890 0x5579425ff590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.538+0000 7f3ae347fb80 1 --2- 172.21.15.131:0/3549241045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579425f3e00 0x5579425f0da0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.538+0000 7f3ab6a00700 0 client.26857 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:17:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.538+0000 7f3ae347fb80 1 -- 172.21.15.131:0/3549241045 >> 172.21.15.131:0/3549241045 conn(0x55794254c210 msgr2=0x5579425f6330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.538+0000 7f3ae347fb80 1 -- 172.21.15.131:0/3549241045 shutdown_connections 2024-05-08T22:17:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.538+0000 7f3ae347fb80 1 -- 172.21.15.131:0/3549241045 wait complete. 2024-05-08T22:17:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.538+0000 7f3ae347fb80 20 client.26857 trim_cache size 0 max 16384 2024-05-08T22:17:29.541 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:17:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.539+0000 7f3acfa00700 1 -- 172.21.15.131:0/3408169501 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3aac0a1460 msgr2=0x7f3aac0a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.539+0000 7f3acfa00700 1 --2- 172.21.15.131:0/3408169501 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3aac0a1460 0x7f3aac0a3920 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x5579425eabd0 tx=0x7f3ab800f040 comp rx=0 tx=0).stop 2024-05-08T22:17:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.539+0000 7f3acfa00700 1 -- 172.21.15.131:0/3408169501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ac810d9c0 msgr2=0x7f3ac81aaac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.539+0000 7f3acfa00700 1 --2- 172.21.15.131:0/3408169501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ac810d9c0 0x7f3ac81aaac0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000f4d0 tx=0x7f3ab000f7e0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.539+0000 7f3acfa00700 1 -- 172.21.15.131:0/3408169501 shutdown_connections 2024-05-08T22:17:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.539+0000 7f3acfa00700 1 --2- 172.21.15.131:0/3408169501 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3aac0a1460 0x7f3aac0a3920 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.539+0000 7f3acfa00700 1 --2- 172.21.15.131:0/3408169501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ac810d9c0 0x7f3ac81aaac0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.539+0000 7f3acfa00700 1 --2- 172.21.15.131:0/3408169501 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ac8107fa0 0x7f3ac81a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.539+0000 7f3acfa00700 1 --2- 172.21.15.131:0/3408169501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ac8108960 0x7f3ac81a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.539+0000 7f3acfa00700 1 -- 172.21.15.131:0/3408169501 >> 172.21.15.131:0/3408169501 conn(0x7f3ac806bff0 msgr2=0x7f3ac810a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.539+0000 7f3acfa00700 1 -- 172.21.15.131:0/3408169501 shutdown_connections 2024-05-08T22:17:29.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.539+0000 7f3acfa00700 1 -- 172.21.15.131:0/3408169501 wait complete. 2024-05-08T22:17:29.560 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T22:17:29.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:29 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2198077785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:17:29.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.802+0000 7effed000700 1 -- 172.21.15.131:0/2505190709 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7effe80754d0 msgr2=0x7effe80758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.802+0000 7effed000700 1 --2- 172.21.15.131:0/2505190709 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7effe80754d0 0x7effe80758b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7effd8009990 tx=0x7effd8009ca0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.805+0000 7effed000700 1 -- 172.21.15.131:0/2505190709 shutdown_connections 2024-05-08T22:17:29.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.805+0000 7effed000700 1 --2- 172.21.15.131:0/2505190709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe8111c10 0x7effe8114000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.805+0000 7effed000700 1 --2- 172.21.15.131:0/2505190709 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7effe80754d0 0x7effe80758b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.805+0000 7effed000700 1 --2- 172.21.15.131:0/2505190709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effe8075df0 0x7effe81116d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.805+0000 7effed000700 1 -- 172.21.15.131:0/2505190709 >> 172.21.15.131:0/2505190709 conn(0x7effe80fea00 msgr2=0x7effe8100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:29.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.805+0000 7effed000700 1 -- 172.21.15.131:0/2505190709 shutdown_connections 2024-05-08T22:17:29.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.805+0000 7effed000700 1 -- 172.21.15.131:0/2505190709 wait complete. 2024-05-08T22:17:29.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.808+0000 7effed000700 1 Processor -- start 2024-05-08T22:17:29.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.809+0000 7effed000700 1 -- start start 2024-05-08T22:17:29.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.811+0000 7effed000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe80754d0 0x7effe8110fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.811+0000 7effed000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effe8075df0 0x7effe8111510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.812+0000 7effed000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7effe8111c10 0x7effe810a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.812+0000 7effed000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe810a8b0 con 0x7effe8075df0 2024-05-08T22:17:29.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.812+0000 7effed000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe810aa20 con 0x7effe8111c10 2024-05-08T22:17:29.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.812+0000 7effed000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe810abe0 con 0x7effe80754d0 2024-05-08T22:17:29.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.812+0000 7effe7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe80754d0 0x7effe8110fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.812+0000 7effe7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe80754d0 0x7effe8110fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53044/0 (socket says 172.21.15.131:53044) 2024-05-08T22:17:29.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.812+0000 7effe7400700 1 -- 172.21.15.131:0/500481750 learned_addr learned my addr 172.21.15.131:0/500481750 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:29.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.812+0000 7effe6a00700 1 --2- 172.21.15.131:0/500481750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effe8075df0 0x7effe8111510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.812+0000 7effe7e00700 1 --2- 172.21.15.131:0/500481750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7effe8111c10 0x7effe810a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.812+0000 7effe7400700 1 -- 172.21.15.131:0/500481750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7effe8111c10 msgr2=0x7effe810a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.812+0000 7effe7400700 1 --2- 172.21.15.131:0/500481750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7effe8111c10 0x7effe810a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.812+0000 7effe7400700 1 -- 172.21.15.131:0/500481750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effe8075df0 msgr2=0x7effe8111510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.813+0000 7effe7400700 1 --2- 172.21.15.131:0/500481750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effe8075df0 0x7effe8111510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.813+0000 7effe7400700 1 -- 172.21.15.131:0/500481750 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd80095f0 con 0x7effe80754d0 2024-05-08T22:17:29.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.814+0000 7effe6a00700 1 --2- 172.21.15.131:0/500481750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effe8075df0 0x7effe8111510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:17:29.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.814+0000 7effe7400700 1 --2- 172.21.15.131:0/500481750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe80754d0 0x7effe8110fd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7effd8003730 tx=0x7effd8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:29.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.815+0000 7effdfe00700 1 -- 172.21.15.131:0/500481750 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd801d070 con 0x7effe80754d0 2024-05-08T22:17:29.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.815+0000 7effdfe00700 1 -- 172.21.15.131:0/500481750 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effd8004450 con 0x7effe80754d0 2024-05-08T22:17:29.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.815+0000 7effdfe00700 1 -- 172.21.15.131:0/500481750 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd8004dc0 con 0x7effe80754d0 2024-05-08T22:17:29.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.815+0000 7effed000700 1 -- 172.21.15.131:0/500481750 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe810ae60 con 0x7effe80754d0 2024-05-08T22:17:29.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.815+0000 7effed000700 1 -- 172.21.15.131:0/500481750 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe810ed40 con 0x7effe80754d0 2024-05-08T22:17:29.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.820+0000 7effdfe00700 1 -- 172.21.15.131:0/500481750 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7effd8005050 con 0x7effe80754d0 2024-05-08T22:17:29.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.823+0000 7effdfe00700 1 --2- 172.21.15.131:0/500481750 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7effcc0a1410 0x7effcc0a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.824+0000 7effdfe00700 1 -- 172.21.15.131:0/500481750 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7effd80ce030 con 0x7effe80754d0 2024-05-08T22:17:29.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.825+0000 7effe6a00700 1 --2- 172.21.15.131:0/500481750 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7effcc0a1410 0x7effcc0a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.827+0000 7f00021c8b80 1 Processor -- start 2024-05-08T22:17:29.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.827+0000 7effe6a00700 1 --2- 172.21.15.131:0/500481750 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7effcc0a1410 0x7effcc0a38d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7effd4005d90 tx=0x7effd4005d00 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:29.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.828+0000 7f00021c8b80 1 -- start start 2024-05-08T22:17:29.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.829+0000 7f00021c8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562f5fe3a050 0x562f5fe3a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.829+0000 7f00021c8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562f5fe3a970 0x562f5fe34350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.829+0000 7f00021c8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe34890 0x562f5fe34cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.829+0000 7f00021c8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5fd8b150 con 0x562f5fe3a970 2024-05-08T22:17:29.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.829+0000 7f00021c8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5fd88b10 con 0x562f5fe3a050 2024-05-08T22:17:29.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.829+0000 7f00021c8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5fd84400 con 0x562f5fe34890 2024-05-08T22:17:29.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.829+0000 7effe7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe34890 0x562f5fe34cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.829+0000 7effe7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe34890 0x562f5fe34cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53056/0 (socket says 172.21.15.131:53056) 2024-05-08T22:17:29.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.829+0000 7effe7e00700 1 -- 172.21.15.131:0/259717651 learned_addr learned my addr 172.21.15.131:0/259717651 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:29.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.830+0000 7effe6a00700 1 --2- 172.21.15.131:0/259717651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562f5fe3a970 0x562f5fe34350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.830+0000 7effe7e00700 1 -- 172.21.15.131:0/259717651 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562f5fe3a050 msgr2=0x562f5fe3a430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:17:29.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.830+0000 7effe7e00700 1 --2- 172.21.15.131:0/259717651 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562f5fe3a050 0x562f5fe3a430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.830+0000 7effe7e00700 1 -- 172.21.15.131:0/259717651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562f5fe3a970 msgr2=0x562f5fe34350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:29.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.830+0000 7effe7e00700 1 --2- 172.21.15.131:0/259717651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562f5fe3a970 0x562f5fe34350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.830+0000 7effe7e00700 1 -- 172.21.15.131:0/259717651 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f5fc053b0 con 0x562f5fe34890 2024-05-08T22:17:29.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.830+0000 7effe7e00700 1 --2- 172.21.15.131:0/259717651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe34890 0x562f5fe34cd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7effe0009cb0 tx=0x7effe0007500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48a06153902b2dbb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:29.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.830+0000 7effdcc00700 1 -- 172.21.15.131:0/259717651 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe0018070 con 0x562f5fe34890 2024-05-08T22:17:29.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.830+0000 7effdcc00700 1 -- 172.21.15.131:0/259717651 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effe000f050 con 0x562f5fe34890 2024-05-08T22:17:29.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.830+0000 7effdcc00700 1 -- 172.21.15.131:0/259717651 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe0014410 con 0x562f5fe34890 2024-05-08T22:17:29.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.831+0000 7f00021c8b80 1 -- 172.21.15.131:0/259717651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe34890 msgr2=0x562f5fe34cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.831+0000 7f00021c8b80 1 --2- 172.21.15.131:0/259717651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe34890 0x562f5fe34cd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7effe0009cb0 tx=0x7effe0007500 comp rx=0 tx=0).stop 2024-05-08T22:17:29.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.832+0000 7f00021c8b80 1 -- 172.21.15.131:0/259717651 shutdown_connections 2024-05-08T22:17:29.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.832+0000 7f00021c8b80 1 --2- 172.21.15.131:0/259717651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe34890 0x562f5fe34cd0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.832+0000 7f00021c8b80 1 --2- 172.21.15.131:0/259717651 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562f5fe3a050 0x562f5fe3a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.832+0000 7f00021c8b80 1 --2- 172.21.15.131:0/259717651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562f5fe3a970 0x562f5fe34350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.832+0000 7f00021c8b80 1 -- 172.21.15.131:0/259717651 >> 172.21.15.131:0/259717651 conn(0x562f5fd92210 msgr2=0x562f5fe2f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:29.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.832+0000 7f00021c8b80 1 -- 172.21.15.131:0/259717651 shutdown_connections 2024-05-08T22:17:29.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.832+0000 7f00021c8b80 1 -- 172.21.15.131:0/259717651 wait complete. 2024-05-08T22:17:29.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.833+0000 7f00021c8b80 1 -- start start 2024-05-08T22:17:29.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.833+0000 7f00021c8b80 1 Processor -- start 2024-05-08T22:17:29.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.834+0000 7f00021c8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe34890 0x562f5fe40910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.834+0000 7effe7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe34890 0x562f5fe40910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.834+0000 7effe7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe34890 0x562f5fe40910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53058/0 (socket says 172.21.15.131:53058) 2024-05-08T22:17:29.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.834+0000 7f00021c8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562f5fe3a050 0x562f5fe3dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.835+0000 7f00021c8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562f5fe3a970 0x562f5fe3e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.835+0000 7effe7400700 1 -- 172.21.15.131:0/1734176750 learned_addr learned my addr 172.21.15.131:0/1734176750 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:29.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.835+0000 7f00021c8b80 1 -- 172.21.15.131:0/1734176750 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5fd20560 con 0x562f5fe3a970 2024-05-08T22:17:29.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.835+0000 7f00021c8b80 1 -- 172.21.15.131:0/1734176750 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5fd20200 con 0x562f5fe3a050 2024-05-08T22:17:29.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.835+0000 7f00021c8b80 1 -- 172.21.15.131:0/1734176750 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5fc076a0 con 0x562f5fe34890 2024-05-08T22:17:29.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.835+0000 7f00021c8b80 10 client.0 _ll_register_callbacks cb 0x562f5fd29b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:17:29.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.835+0000 7effe6a00700 1 --2- 172.21.15.131:0/1734176750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562f5fe3a050 0x562f5fe3dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.835+0000 7effe7400700 1 -- 172.21.15.131:0/1734176750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562f5fe3a050 msgr2=0x562f5fe3dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:29.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.835+0000 7effe7400700 1 --2- 172.21.15.131:0/1734176750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562f5fe3a050 0x562f5fe3dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.835+0000 7effe7400700 1 -- 172.21.15.131:0/1734176750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562f5fe3a970 msgr2=0x562f5fe3e340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:17:29.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.835+0000 7effe7400700 1 --2- 172.21.15.131:0/1734176750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562f5fe3a970 0x562f5fe3e340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.835+0000 7effe7400700 1 -- 172.21.15.131:0/1734176750 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe00098e0 con 0x562f5fe34890 2024-05-08T22:17:29.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.835+0000 7effe7400700 1 --2- 172.21.15.131:0/1734176750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe34890 0x562f5fe40910 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7effd8097290 tx=0x7effd809a8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3ca958723171a3a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:29.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.836+0000 7effd1600700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:17:29.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.836+0000 7effd1600700 1 -- 172.21.15.131:0/1734176750 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd809ad90 con 0x562f5fe34890 2024-05-08T22:17:29.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.836+0000 7f00021c8b80 1 -- 172.21.15.131:0/1734176750 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f5fcdd270 con 0x562f5fe34890 2024-05-08T22:17:29.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.836+0000 7effd1600700 1 -- 172.21.15.131:0/1734176750 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7effd809ebc0 con 0x562f5fe34890 2024-05-08T22:17:29.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.836+0000 7effd1600700 1 -- 172.21.15.131:0/1734176750 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7effd809ad90 con 0x562f5fe34890 2024-05-08T22:17:29.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.836+0000 7f00021c8b80 10 client.26869 fetch_fsmap learned FSMap version 30 2024-05-08T22:17:29.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.836+0000 7f00021c8b80 1 -- 172.21.15.131:0/1734176750 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562f5fbb1b60 con 0x562f5fe34890 2024-05-08T22:17:29.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.837+0000 7effd1600700 1 -- 172.21.15.131:0/1734176750 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562f5fcdd270 con 0x562f5fe34890 2024-05-08T22:17:29.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.837+0000 7effd1600700 1 -- 172.21.15.131:0/1734176750 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7effd80aec60 con 0x562f5fe34890 2024-05-08T22:17:29.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.837+0000 7f00021c8b80 10 client.26869 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:17:29.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.837+0000 7f00021c8b80 10 client.26869 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:17:29.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.837+0000 7f00021c8b80 20 client.26869 populate_metadata read hostname 'smithi131' 2024-05-08T22:17:29.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.837+0000 7f00021c8b80 1 --2- 172.21.15.131:0/1734176750 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562f5fe4e030 0x562f5fe50420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.837+0000 7f00021c8b80 4 client.26869 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:17:29.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.837+0000 7f00021c8b80 1 -- 172.21.15.131:0/1734176750 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f5fe03810 con 0x562f5fe4e030 2024-05-08T22:17:29.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.838+0000 7effe6a00700 1 --2- 172.21.15.131:0/1734176750 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562f5fe4e030 0x562f5fe50420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.841+0000 7effe6a00700 1 --2- 172.21.15.131:0/1734176750 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562f5fe4e030 0x562f5fe50420 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7302f64dc31cf6a server_cookie=ae3922871931cfec in_seq=0 out_seq=0 2024-05-08T22:17:29.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.841+0000 7effd1600700 10 client.26869 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:17:29.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.842+0000 7effd1600700 1 -- 172.21.15.131:0/1734176750 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562f5fe03810 con 0x562f5fe4e030 2024-05-08T22:17:29.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.842+0000 7effd1600700 10 client.26869 handle_command_reply: tid=0 2024-05-08T22:17:29.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.842+0000 7f00021c8b80 1 client.26869 shutdown 2024-05-08T22:17:29.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.843+0000 7f00021c8b80 1 -- 172.21.15.131:0/1734176750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe34890 msgr2=0x562f5fe40910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.843+0000 7f00021c8b80 1 --2- 172.21.15.131:0/1734176750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe34890 0x562f5fe40910 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7effd8097290 tx=0x7effd809a8d0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.843+0000 7f00021c8b80 1 -- 172.21.15.131:0/1734176750 shutdown_connections 2024-05-08T22:17:29.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.843+0000 7f00021c8b80 1 --2- 172.21.15.131:0/1734176750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562f5fe3a970 0x562f5fe3e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.843+0000 7f00021c8b80 1 --2- 172.21.15.131:0/1734176750 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562f5fe4e030 0x562f5fe50420 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.843+0000 7f00021c8b80 1 --2- 172.21.15.131:0/1734176750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe34890 0x562f5fe40910 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.843+0000 7f00021c8b80 1 --2- 172.21.15.131:0/1734176750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562f5fe3a050 0x562f5fe3dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.843+0000 7f00021c8b80 1 -- 172.21.15.131:0/1734176750 >> 172.21.15.131:0/1734176750 conn(0x562f5fd92210 msgr2=0x562f5fe2f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:29.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.843+0000 7effd1600700 0 client.26869 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:17:29.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.843+0000 7f00021c8b80 1 -- 172.21.15.131:0/1734176750 shutdown_connections 2024-05-08T22:17:29.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.843+0000 7f00021c8b80 1 -- 172.21.15.131:0/1734176750 wait complete. 2024-05-08T22:17:29.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.844+0000 7f00021c8b80 20 client.26869 trim_cache size 0 max 16384 2024-05-08T22:17:29.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.877+0000 7f00021c8b80 1 Processor -- start 2024-05-08T22:17:29.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.878+0000 7f00021c8b80 1 -- start start 2024-05-08T22:17:29.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.878+0000 7f00021c8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562f5fe34890 0x562f5fe34350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.878+0000 7f00021c8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562f5fe3a050 0x562f5fe34c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.878+0000 7f00021c8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe3a970 0x562f5fe309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.878+0000 7f00021c8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5fe4aed0 con 0x562f5fe34890 2024-05-08T22:17:29.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.878+0000 7f00021c8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5fe3a430 con 0x562f5fe3a050 2024-05-08T22:17:29.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.878+0000 7f00021c8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5fe3c170 con 0x562f5fe3a970 2024-05-08T22:17:29.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.878+0000 7effe7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe3a970 0x562f5fe309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.879+0000 7effe7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe3a970 0x562f5fe309c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53070/0 (socket says 172.21.15.131:53070) 2024-05-08T22:17:29.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.879+0000 7effe7e00700 1 -- 172.21.15.131:0/1767749938 learned_addr learned my addr 172.21.15.131:0/1767749938 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:29.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.879+0000 7effe6a00700 1 --2- 172.21.15.131:0/1767749938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562f5fe3a050 0x562f5fe34c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.880+0000 7effe7400700 1 --2- 172.21.15.131:0/1767749938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562f5fe34890 0x562f5fe34350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.880+0000 7effe7e00700 1 -- 172.21.15.131:0/1767749938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562f5fe3a050 msgr2=0x562f5fe34c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:29.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.880+0000 7effe7e00700 1 --2- 172.21.15.131:0/1767749938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562f5fe3a050 0x562f5fe34c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.885+0000 7effe7e00700 1 -- 172.21.15.131:0/1767749938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562f5fe34890 msgr2=0x562f5fe34350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:29.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.885+0000 7effe7e00700 1 --2- 172.21.15.131:0/1767749938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562f5fe34890 0x562f5fe34350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.885+0000 7effe7e00700 1 -- 172.21.15.131:0/1767749938 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f5fd4a560 con 0x562f5fe3a970 2024-05-08T22:17:29.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.885+0000 7effe7400700 1 --2- 172.21.15.131:0/1767749938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562f5fe34890 0x562f5fe34350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:17:29.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.885+0000 7effe6a00700 1 --2- 172.21.15.131:0/1767749938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562f5fe3a050 0x562f5fe34c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:17:29.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.885+0000 7effe7e00700 1 --2- 172.21.15.131:0/1767749938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe3a970 0x562f5fe309c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7effe0009470 tx=0x7effe0009170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee55ca4ccb55a7de server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:29.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.886+0000 7effdd600700 1 -- 172.21.15.131:0/1767749938 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe0018030 con 0x562f5fe3a970 2024-05-08T22:17:29.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.886+0000 7effdd600700 1 -- 172.21.15.131:0/1767749938 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effe0019040 con 0x562f5fe3a970 2024-05-08T22:17:29.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.886+0000 7effdd600700 1 -- 172.21.15.131:0/1767749938 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe0014410 con 0x562f5fe3a970 2024-05-08T22:17:29.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.887+0000 7f00021c8b80 1 -- 172.21.15.131:0/1767749938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe3a970 msgr2=0x562f5fe309c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.887+0000 7f00021c8b80 1 --2- 172.21.15.131:0/1767749938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe3a970 0x562f5fe309c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7effe0009470 tx=0x7effe0009170 comp rx=0 tx=0).stop 2024-05-08T22:17:29.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.888+0000 7f00021c8b80 1 -- 172.21.15.131:0/1767749938 shutdown_connections 2024-05-08T22:17:29.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.888+0000 7f00021c8b80 1 --2- 172.21.15.131:0/1767749938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe3a970 0x562f5fe309c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.888+0000 7f00021c8b80 1 --2- 172.21.15.131:0/1767749938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562f5fe34890 0x562f5fe34350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.888+0000 7f00021c8b80 1 --2- 172.21.15.131:0/1767749938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562f5fe3a050 0x562f5fe34c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.888+0000 7f00021c8b80 1 -- 172.21.15.131:0/1767749938 >> 172.21.15.131:0/1767749938 conn(0x562f5fd92210 msgr2=0x562f5fe2dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:29.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.888+0000 7f00021c8b80 1 -- 172.21.15.131:0/1767749938 shutdown_connections 2024-05-08T22:17:29.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.888+0000 7f00021c8b80 1 -- 172.21.15.131:0/1767749938 wait complete. 2024-05-08T22:17:29.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.889+0000 7f00021c8b80 1 -- start start 2024-05-08T22:17:29.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.889+0000 7f00021c8b80 1 Processor -- start 2024-05-08T22:17:29.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.890+0000 7f00021c8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562f5fe34890 0x562f5fe45390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.890+0000 7f00021c8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe3a050 0x562f5fe458d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.890+0000 7f00021c8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562f5fe3a970 0x562f5fe35b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.891+0000 7effe6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe3a050 0x562f5fe458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.891+0000 7effe6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe3a050 0x562f5fe458d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53086/0 (socket says 172.21.15.131:53086) 2024-05-08T22:17:29.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.891+0000 7effe6a00700 1 -- 172.21.15.131:0/1898945716 learned_addr learned my addr 172.21.15.131:0/1898945716 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:29.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.891+0000 7f00021c8b80 1 -- 172.21.15.131:0/1898945716 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5fe36250 con 0x562f5fe34890 2024-05-08T22:17:29.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.892+0000 7effe7400700 1 --2- 172.21.15.131:0/1898945716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562f5fe34890 0x562f5fe45390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.892+0000 7f00021c8b80 1 -- 172.21.15.131:0/1898945716 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5fe363c0 con 0x562f5fe3a970 2024-05-08T22:17:29.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.892+0000 7f00021c8b80 1 -- 172.21.15.131:0/1898945716 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5fe36580 con 0x562f5fe3a050 2024-05-08T22:17:29.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.892+0000 7f00021c8b80 10 client.0 _ll_register_callbacks cb 0x562f5fd29b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:17:29.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.892+0000 7effe6a00700 1 -- 172.21.15.131:0/1898945716 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562f5fe3a970 msgr2=0x562f5fe35b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:29.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.892+0000 7effe6a00700 1 --2- 172.21.15.131:0/1898945716 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562f5fe3a970 0x562f5fe35b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.893+0000 7effe6a00700 1 -- 172.21.15.131:0/1898945716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562f5fe34890 msgr2=0x562f5fe45390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:29.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.893+0000 7effe6a00700 1 --2- 172.21.15.131:0/1898945716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562f5fe34890 0x562f5fe45390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.893+0000 7effe6a00700 1 -- 172.21.15.131:0/1898945716 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f5fbb1b60 con 0x562f5fe3a050 2024-05-08T22:17:29.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.893+0000 7effe7400700 1 --2- 172.21.15.131:0/1898945716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562f5fe34890 0x562f5fe45390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:17:29.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.894+0000 7effe6a00700 1 --2- 172.21.15.131:0/1898945716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe3a050 0x562f5fe458d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7effd400e7d0 tx=0x7effd40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a1c1387ea1c99bf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:29.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.894+0000 7effd1600700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:17:29.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.894+0000 7effd1600700 1 -- 172.21.15.131:0/1898945716 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd4024050 con 0x562f5fe3a050 2024-05-08T22:17:29.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.894+0000 7effd1600700 1 -- 172.21.15.131:0/1898945716 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7effd4026b00 con 0x562f5fe3a050 2024-05-08T22:17:29.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.895+0000 7f00021c8b80 1 -- 172.21.15.131:0/1898945716 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7effe000f050 con 0x562f5fe3a050 2024-05-08T22:17:29.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.895+0000 7effd1600700 1 -- 172.21.15.131:0/1898945716 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7effd4026cc0 con 0x562f5fe3a050 2024-05-08T22:17:29.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.895+0000 7f00021c8b80 10 client.26875 fetch_fsmap learned FSMap version 30 2024-05-08T22:17:29.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.895+0000 7f00021c8b80 1 -- 172.21.15.131:0/1898945716 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7effd400e9d0 con 0x562f5fe3a050 2024-05-08T22:17:29.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.895+0000 7effd1600700 1 -- 172.21.15.131:0/1898945716 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effe000f050 con 0x562f5fe3a050 2024-05-08T22:17:29.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.896+0000 7effd1600700 1 -- 172.21.15.131:0/1898945716 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7effd4026220 con 0x562f5fe3a050 2024-05-08T22:17:29.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.897+0000 7f00021c8b80 10 client.26875 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:17:29.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.897+0000 7f00021c8b80 10 client.26875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:17:29.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.897+0000 7f00021c8b80 20 client.26875 populate_metadata read hostname 'smithi131' 2024-05-08T22:17:29.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.897+0000 7f00021c8b80 1 --2- 172.21.15.131:0/1898945716 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562f5fea0900 0x562f5fea0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:29.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.897+0000 7f00021c8b80 4 client.26875 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:17:29.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.897+0000 7f00021c8b80 1 -- 172.21.15.131:0/1898945716 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562f5fd7cec0 con 0x562f5fea0900 2024-05-08T22:17:29.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.897+0000 7effe7400700 1 --2- 172.21.15.131:0/1898945716 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562f5fea0900 0x562f5fea0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:29.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.898+0000 7effe7400700 1 --2- 172.21.15.131:0/1898945716 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562f5fea0900 0x562f5fea0d10 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e4ad9a96f05fe67 server_cookie=a6a586aadfc4e4e9 in_seq=0 out_seq=0 2024-05-08T22:17:29.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.898+0000 7effd1600700 10 client.26875 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:17:29.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.899+0000 7effd1600700 1 -- 172.21.15.131:0/1898945716 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562f5fd7cec0 con 0x562f5fea0900 2024-05-08T22:17:29.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.899+0000 7effd1600700 10 client.26875 handle_command_reply: tid=0 2024-05-08T22:17:29.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.899+0000 7f00021c8b80 1 client.26875 shutdown 2024-05-08T22:17:29.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.901+0000 7f00021c8b80 1 -- 172.21.15.131:0/1898945716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe3a050 msgr2=0x562f5fe458d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.901+0000 7f00021c8b80 1 --2- 172.21.15.131:0/1898945716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe3a050 0x562f5fe458d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7effd400e7d0 tx=0x7effd40047c0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.901+0000 7f00021c8b80 1 -- 172.21.15.131:0/1898945716 shutdown_connections 2024-05-08T22:17:29.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.901+0000 7f00021c8b80 1 --2- 172.21.15.131:0/1898945716 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562f5fe3a970 0x562f5fe35b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.901+0000 7f00021c8b80 1 --2- 172.21.15.131:0/1898945716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562f5fe34890 0x562f5fe45390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.901+0000 7f00021c8b80 1 --2- 172.21.15.131:0/1898945716 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x562f5fea0900 0x562f5fea0d10 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.901+0000 7f00021c8b80 1 --2- 172.21.15.131:0/1898945716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f5fe3a050 0x562f5fe458d0 secure :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7effd400e7d0 tx=0x7effd40047c0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.901+0000 7f00021c8b80 1 -- 172.21.15.131:0/1898945716 >> 172.21.15.131:0/1898945716 conn(0x562f5fd92210 msgr2=0x562f5fe2f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:29.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.901+0000 7effd1600700 0 client.26875 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:17:29.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.901+0000 7f00021c8b80 1 -- 172.21.15.131:0/1898945716 shutdown_connections 2024-05-08T22:17:29.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.901+0000 7f00021c8b80 1 -- 172.21.15.131:0/1898945716 wait complete. 2024-05-08T22:17:29.905 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:17:29.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.903+0000 7f00021c8b80 20 client.26875 trim_cache size 0 max 16384 2024-05-08T22:17:29.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.904+0000 7effed000700 1 -- 172.21.15.131:0/500481750 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7effcc0a1410 msgr2=0x7effcc0a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.904+0000 7effed000700 1 --2- 172.21.15.131:0/500481750 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7effcc0a1410 0x7effcc0a38d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7effd4005d90 tx=0x7effd4005d00 comp rx=0 tx=0).stop 2024-05-08T22:17:29.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.904+0000 7effed000700 1 -- 172.21.15.131:0/500481750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe80754d0 msgr2=0x7effe8110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:29.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.904+0000 7effed000700 1 --2- 172.21.15.131:0/500481750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe80754d0 0x7effe8110fd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7effd8003730 tx=0x7effd8009f90 comp rx=0 tx=0).stop 2024-05-08T22:17:29.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.905+0000 7effed000700 1 -- 172.21.15.131:0/500481750 shutdown_connections 2024-05-08T22:17:29.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.905+0000 7effed000700 1 --2- 172.21.15.131:0/500481750 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7effcc0a1410 0x7effcc0a38d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.905+0000 7effed000700 1 --2- 172.21.15.131:0/500481750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7effe8111c10 0x7effe810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.905+0000 7effed000700 1 --2- 172.21.15.131:0/500481750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe80754d0 0x7effe8110fd0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.905+0000 7effed000700 1 --2- 172.21.15.131:0/500481750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effe8075df0 0x7effe8111510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:29.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.905+0000 7effed000700 1 -- 172.21.15.131:0/500481750 >> 172.21.15.131:0/500481750 conn(0x7effe80fea00 msgr2=0x7effe8112880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:29.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.905+0000 7effed000700 1 -- 172.21.15.131:0/500481750 shutdown_connections 2024-05-08T22:17:29.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:29.906+0000 7effed000700 1 -- 172.21.15.131:0/500481750 wait complete. 2024-05-08T22:17:30.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:30 smithi131 ceph-mon[37260]: pgmap v316: 129 pgs: 15 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 65 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:17:30.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:30 smithi157 ceph-mon[77746]: pgmap v316: 129 pgs: 15 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 65 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:17:30.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:30 smithi132 ceph-mon[78180]: pgmap v316: 129 pgs: 15 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 65 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:17:30.967 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T22:17:30.968 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T22:17:31.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.368+0000 7f3922400700 1 -- 172.21.15.131:0/2074421660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f391c103350 msgr2=0x7f391c103730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:31.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.368+0000 7f3922400700 1 --2- 172.21.15.131:0/2074421660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f391c103350 0x7f391c103730 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f390c009990 tx=0x7f390c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.373+0000 7f3922400700 1 -- 172.21.15.131:0/2074421660 shutdown_connections 2024-05-08T22:17:31.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.373+0000 7f3922400700 1 --2- 172.21.15.131:0/2074421660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f391c108470 0x7f391c10fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.373+0000 7f3922400700 1 --2- 172.21.15.131:0/2074421660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f391c103350 0x7f391c103730 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.373+0000 7f3922400700 1 --2- 172.21.15.131:0/2074421660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f391c103d00 0x7f391c107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.373+0000 7f3922400700 1 -- 172.21.15.131:0/2074421660 >> 172.21.15.131:0/2074421660 conn(0x7f391c0febb0 msgr2=0x7f391c100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:31.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.375+0000 7f3922400700 1 -- 172.21.15.131:0/2074421660 shutdown_connections 2024-05-08T22:17:31.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.375+0000 7f3922400700 1 -- 172.21.15.131:0/2074421660 wait complete. 2024-05-08T22:17:31.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.377+0000 7f3922400700 1 Processor -- start 2024-05-08T22:17:31.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.378+0000 7f3922400700 1 -- start start 2024-05-08T22:17:31.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.379+0000 7f3922400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f391c103350 0x7f391c1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.379+0000 7f3922400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f391c108470 0x7f391c199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.379+0000 7f3922400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f391c19df50 0x7f391c19e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.379+0000 7f3922400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c19e9d0 con 0x7f391c108470 2024-05-08T22:17:31.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.379+0000 7f3922400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c19eb40 con 0x7f391c103350 2024-05-08T22:17:31.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.379+0000 7f3922400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c19ed00 con 0x7f391c19df50 2024-05-08T22:17:31.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.379+0000 7f391aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f391c108470 0x7f391c199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.379+0000 7f391aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f391c108470 0x7f391c199c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43950/0 (socket says 172.21.15.131:43950) 2024-05-08T22:17:31.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.379+0000 7f391aa00700 1 -- 172.21.15.131:0/2660783217 learned_addr learned my addr 172.21.15.131:0/2660783217 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:31.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.379+0000 7f391b400700 1 --2- 172.21.15.131:0/2660783217 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f391c103350 0x7f391c1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.380+0000 7f391aa00700 1 -- 172.21.15.131:0/2660783217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f391c19df50 msgr2=0x7f391c19e3d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T22:17:31.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.380+0000 7f391aa00700 1 --2- 172.21.15.131:0/2660783217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f391c19df50 0x7f391c19e3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.380+0000 7f391aa00700 1 -- 172.21.15.131:0/2660783217 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f391c103350 msgr2=0x7f391c1996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:31.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.380+0000 7f391aa00700 1 --2- 172.21.15.131:0/2660783217 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f391c103350 0x7f391c1996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.380+0000 7f391aa00700 1 -- 172.21.15.131:0/2660783217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390c0095f0 con 0x7f391c108470 2024-05-08T22:17:31.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.381+0000 7f391b400700 1 --2- 172.21.15.131:0/2660783217 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f391c103350 0x7f391c1996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:17:31.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.381+0000 7f391aa00700 1 --2- 172.21.15.131:0/2660783217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f391c108470 0x7f391c199c00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f391400efc0 tx=0x7f391400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:31.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.382+0000 7f3913e00700 1 -- 172.21.15.131:0/2660783217 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391400cad0 con 0x7f391c108470 2024-05-08T22:17:31.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.382+0000 7f3913e00700 1 -- 172.21.15.131:0/2660783217 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f391400f040 con 0x7f391c108470 2024-05-08T22:17:31.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.382+0000 7f3922400700 1 -- 172.21.15.131:0/2660783217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f391c1a6200 con 0x7f391c108470 2024-05-08T22:17:31.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.382+0000 7f3922400700 1 -- 172.21.15.131:0/2660783217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f391c1a6730 con 0x7f391c108470 2024-05-08T22:17:31.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.382+0000 7f3913e00700 1 -- 172.21.15.131:0/2660783217 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3914007ea0 con 0x7f391c108470 2024-05-08T22:17:31.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.392+0000 7f3913e00700 1 -- 172.21.15.131:0/2660783217 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f391400cc30 con 0x7f391c108470 2024-05-08T22:17:31.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.394+0000 7f3935f07b80 1 Processor -- start 2024-05-08T22:17:31.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.394+0000 7f3935f07b80 1 -- start start 2024-05-08T22:17:31.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.395+0000 7f3935f07b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556997c1fe80 0x556997c20260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.395+0000 7f3935f07b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556997c207a0 0x556997c1a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.396+0000 7f3935f07b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556997c1a910 0x556997c1ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.396+0000 7f3935f07b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556997b711d0 con 0x556997c207a0 2024-05-08T22:17:31.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.396+0000 7f3935f07b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556997b6eb90 con 0x556997c1fe80 2024-05-08T22:17:31.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.396+0000 7f3935f07b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556997b6a480 con 0x556997c1a910 2024-05-08T22:17:31.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.396+0000 7f3913e00700 1 --2- 172.21.15.131:0/2660783217 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f39000a1390 0x7f39000a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.396+0000 7f3913e00700 1 -- 172.21.15.131:0/2660783217 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f39140cd3e0 con 0x7f391c108470 2024-05-08T22:17:31.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.396+0000 7f391be00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556997c1fe80 0x556997c20260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.396+0000 7f391be00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556997c1fe80 0x556997c20260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:44028/0 (socket says 172.21.15.131:44028) 2024-05-08T22:17:31.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.396+0000 7f391be00700 1 -- 172.21.15.131:0/1896363417 learned_addr learned my addr 172.21.15.131:0/1896363417 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:31.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.397+0000 7f391be00700 1 --2- 172.21.15.131:0/2660783217 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f39000a1390 0x7f39000a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.397+0000 7f391aa00700 1 --2- 172.21.15.131:0/1896363417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556997c1a910 0x556997c1ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.398+0000 7f391b400700 1 --2- 172.21.15.131:0/1896363417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556997c207a0 0x556997c1a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.398+0000 7f391be00700 1 -- 172.21.15.131:0/1896363417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556997c1a910 msgr2=0x556997c1ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:31.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.398+0000 7f391be00700 1 --2- 172.21.15.131:0/1896363417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556997c1a910 0x556997c1ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.398+0000 7f391be00700 1 -- 172.21.15.131:0/1896363417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556997c207a0 msgr2=0x556997c1a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:31.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.398+0000 7f391be00700 1 --2- 172.21.15.131:0/1896363417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556997c207a0 0x556997c1a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.398+0000 7f391be00700 1 -- 172.21.15.131:0/1896363417 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569979eb430 con 0x556997c1fe80 2024-05-08T22:17:31.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.398+0000 7f391b400700 1 --2- 172.21.15.131:0/1896363417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556997c207a0 0x556997c1a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:17:31.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.398+0000 7f391be00700 1 --2- 172.21.15.131:0/1896363417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556997c1fe80 0x556997c20260 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3908009f90 tx=0x7f39080124f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=371357c6d182f051 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:31.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.400+0000 7f3910c00700 1 -- 172.21.15.131:0/1896363417 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3908012e70 con 0x556997c1fe80 2024-05-08T22:17:31.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.400+0000 7f3910c00700 1 -- 172.21.15.131:0/1896363417 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39080168f0 con 0x556997c1fe80 2024-05-08T22:17:31.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.400+0000 7f3910c00700 1 -- 172.21.15.131:0/1896363417 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390801ea60 con 0x556997c1fe80 2024-05-08T22:17:31.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.401+0000 7f3935f07b80 1 -- 172.21.15.131:0/1896363417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556997c1fe80 msgr2=0x556997c20260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:31.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.401+0000 7f3935f07b80 1 --2- 172.21.15.131:0/1896363417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556997c1fe80 0x556997c20260 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3908009f90 tx=0x7f39080124f0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.401+0000 7f391be00700 1 --2- 172.21.15.131:0/2660783217 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f39000a1390 0x7f39000a3850 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3908012fd0 tx=0x7f3908015670 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:31.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.401+0000 7f3935f07b80 1 -- 172.21.15.131:0/1896363417 shutdown_connections 2024-05-08T22:17:31.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.401+0000 7f3935f07b80 1 --2- 172.21.15.131:0/1896363417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556997c1a910 0x556997c1ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.402+0000 7f3935f07b80 1 --2- 172.21.15.131:0/1896363417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556997c1fe80 0x556997c20260 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.402+0000 7f3935f07b80 1 --2- 172.21.15.131:0/1896363417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556997c207a0 0x556997c1a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.402+0000 7f3935f07b80 1 -- 172.21.15.131:0/1896363417 >> 172.21.15.131:0/1896363417 conn(0x556997b78290 msgr2=0x556997c14e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:31.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.402+0000 7f3935f07b80 1 -- 172.21.15.131:0/1896363417 shutdown_connections 2024-05-08T22:17:31.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.402+0000 7f3935f07b80 1 -- 172.21.15.131:0/1896363417 wait complete. 2024-05-08T22:17:31.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.403+0000 7f3935f07b80 1 -- start start 2024-05-08T22:17:31.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.403+0000 7f3935f07b80 1 Processor -- start 2024-05-08T22:17:31.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.404+0000 7f3935f07b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556997c1a910 0x556997c23b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.404+0000 7f3935f07b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556997c1fe80 0x556997c24070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.404+0000 7f3935f07b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556997c207a0 0x556997c245b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.404+0000 7f3935f07b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556997b065e0 con 0x556997c1fe80 2024-05-08T22:17:31.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.404+0000 7f3935f07b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556997b06280 con 0x556997c207a0 2024-05-08T22:17:31.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.404+0000 7f3935f07b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569979ed720 con 0x556997c1a910 2024-05-08T22:17:31.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.404+0000 7f3935f07b80 10 client.0 _ll_register_callbacks cb 0x556997b0fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:17:31.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.405+0000 7f391b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556997c1a910 0x556997c23b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.405+0000 7f391b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556997c1a910 0x556997c23b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53122/0 (socket says 172.21.15.131:53122) 2024-05-08T22:17:31.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.405+0000 7f391b400700 1 -- 172.21.15.131:0/2030561395 learned_addr learned my addr 172.21.15.131:0/2030561395 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:31.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.405+0000 7f391be00700 1 --2- 172.21.15.131:0/2030561395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556997c207a0 0x556997c245b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.406+0000 7f391b400700 1 -- 172.21.15.131:0/2030561395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556997c207a0 msgr2=0x556997c245b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:31.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.406+0000 7f391b400700 1 --2- 172.21.15.131:0/2030561395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556997c207a0 0x556997c245b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.406+0000 7f391aa00700 1 --2- 172.21.15.131:0/2030561395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556997c1fe80 0x556997c24070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.406+0000 7f391b400700 1 -- 172.21.15.131:0/2030561395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556997c1fe80 msgr2=0x556997c24070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:31.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.406+0000 7f391b400700 1 --2- 172.21.15.131:0/2030561395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556997c1fe80 0x556997c24070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.406+0000 7f391b400700 1 -- 172.21.15.131:0/2030561395 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3908009b40 con 0x556997c1a910 2024-05-08T22:17:31.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.407+0000 7f391aa00700 1 --2- 172.21.15.131:0/2030561395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556997c1fe80 0x556997c24070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:17:31.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.407+0000 7f391b400700 1 --2- 172.21.15.131:0/2030561395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556997c1a910 0x556997c23b30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f390c000c00 tx=0x7f390c00b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51d6e0e91d3598e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:31.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.407+0000 7f3905600700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:17:31.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.408+0000 7f3905600700 1 -- 172.21.15.131:0/2030561395 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c00f870 con 0x556997c1a910 2024-05-08T22:17:31.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.408+0000 7f3905600700 1 -- 172.21.15.131:0/2030561395 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f390c00f9d0 con 0x556997c1a910 2024-05-08T22:17:31.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.409+0000 7f3935f07b80 1 -- 172.21.15.131:0/2030561395 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556997ac32f0 con 0x556997c1a910 2024-05-08T22:17:31.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.416+0000 7f3905600700 1 -- 172.21.15.131:0/2030561395 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f390c00bea0 con 0x556997c1a910 2024-05-08T22:17:31.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.417+0000 7f3935f07b80 10 client.26884 fetch_fsmap learned FSMap version 30 2024-05-08T22:17:31.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.417+0000 7f3935f07b80 1 -- 172.21.15.131:0/2030561395 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556997997be0 con 0x556997c1a910 2024-05-08T22:17:31.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.418+0000 7f3905600700 1 -- 172.21.15.131:0/2030561395 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556997ac32f0 con 0x556997c1a910 2024-05-08T22:17:31.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.419+0000 7f3905600700 1 -- 172.21.15.131:0/2030561395 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f390c02a070 con 0x556997c1a910 2024-05-08T22:17:31.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.419+0000 7f3935f07b80 10 client.26884 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:17:31.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.419+0000 7f3935f07b80 10 client.26884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:17:31.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.419+0000 7f3935f07b80 20 client.26884 populate_metadata read hostname 'smithi131' 2024-05-08T22:17:31.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.419+0000 7f3935f07b80 1 --2- 172.21.15.131:0/2030561395 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x556997c45160 0x556997c47580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.419+0000 7f3935f07b80 4 client.26884 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:17:31.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.419+0000 7f3935f07b80 1 -- 172.21.15.131:0/2030561395 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556997be9890 con 0x556997c45160 2024-05-08T22:17:31.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.420+0000 7f391aa00700 1 --2- 172.21.15.131:0/2030561395 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x556997c45160 0x556997c47580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.422+0000 7f3905600700 10 client.26884 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:17:31.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.422+0000 7f391aa00700 1 --2- 172.21.15.131:0/2030561395 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x556997c45160 0x556997c47580 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa9480236c8307ee server_cookie=ed844bea5b1ab70e in_seq=0 out_seq=0 2024-05-08T22:17:31.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.423+0000 7f3905600700 1 -- 172.21.15.131:0/2030561395 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556997be9890 con 0x556997c45160 2024-05-08T22:17:31.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.423+0000 7f3905600700 10 client.26884 handle_command_reply: tid=0 2024-05-08T22:17:31.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.424+0000 7f3935f07b80 1 client.26884 shutdown 2024-05-08T22:17:31.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.426+0000 7f3935f07b80 1 -- 172.21.15.131:0/2030561395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556997c1a910 msgr2=0x556997c23b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:31.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.426+0000 7f3935f07b80 1 --2- 172.21.15.131:0/2030561395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556997c1a910 0x556997c23b30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f390c000c00 tx=0x7f390c00b920 comp rx=0 tx=0).stop 2024-05-08T22:17:31.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.428+0000 7f3935f07b80 1 -- 172.21.15.131:0/2030561395 shutdown_connections 2024-05-08T22:17:31.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.428+0000 7f3935f07b80 1 --2- 172.21.15.131:0/2030561395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556997c207a0 0x556997c245b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.428+0000 7f3935f07b80 1 --2- 172.21.15.131:0/2030561395 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x556997c45160 0x556997c47580 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.428+0000 7f3935f07b80 1 --2- 172.21.15.131:0/2030561395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556997c1a910 0x556997c23b30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.428+0000 7f3935f07b80 1 --2- 172.21.15.131:0/2030561395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556997c1fe80 0x556997c24070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.428+0000 7f3905600700 0 client.26884 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:17:31.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.428+0000 7f3935f07b80 1 -- 172.21.15.131:0/2030561395 >> 172.21.15.131:0/2030561395 conn(0x556997b78290 msgr2=0x556997c14620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:31.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.429+0000 7f3935f07b80 1 -- 172.21.15.131:0/2030561395 shutdown_connections 2024-05-08T22:17:31.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.429+0000 7f3935f07b80 1 -- 172.21.15.131:0/2030561395 wait complete. 2024-05-08T22:17:31.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.431+0000 7f3935f07b80 20 client.26884 trim_cache size 0 max 16384 2024-05-08T22:17:31.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.475+0000 7f3935f07b80 1 Processor -- start 2024-05-08T22:17:31.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.476+0000 7f3935f07b80 1 -- start start 2024-05-08T22:17:31.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.477+0000 7f3935f07b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556997c1a910 0x556997c14da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.477+0000 7f3935f07b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556997c1fe80 0x556997c2b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.477+0000 7f3935f07b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556997c207a0 0x556997c2b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.477+0000 7f3935f07b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556997c21b90 con 0x556997c1a910 2024-05-08T22:17:31.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.477+0000 7f3935f07b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556997c24f20 con 0x556997c1fe80 2024-05-08T22:17:31.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.477+0000 7f3935f07b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556997c1e510 con 0x556997c207a0 2024-05-08T22:17:31.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.478+0000 7f391aa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556997c1fe80 0x556997c2b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.478+0000 7f391be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556997c207a0 0x556997c2b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.478+0000 7f391aa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556997c1fe80 0x556997c2b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:44038/0 (socket says 172.21.15.131:44038) 2024-05-08T22:17:31.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.478+0000 7f391aa00700 1 -- 172.21.15.131:0/3327993877 learned_addr learned my addr 172.21.15.131:0/3327993877 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:31.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.478+0000 7f391b400700 1 --2- 172.21.15.131:0/3327993877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556997c1a910 0x556997c14da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.479+0000 7f391be00700 1 -- 172.21.15.131:0/3327993877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556997c1fe80 msgr2=0x556997c2b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:31.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.479+0000 7f391be00700 1 --2- 172.21.15.131:0/3327993877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556997c1fe80 0x556997c2b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.479+0000 7f391be00700 1 -- 172.21.15.131:0/3327993877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556997c1a910 msgr2=0x556997c14da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:31.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.479+0000 7f391be00700 1 --2- 172.21.15.131:0/3327993877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556997c1a910 0x556997c14da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.479+0000 7f391be00700 1 -- 172.21.15.131:0/3327993877 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556997b305e0 con 0x556997c207a0 2024-05-08T22:17:31.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.480+0000 7f391be00700 1 --2- 172.21.15.131:0/3327993877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556997c207a0 0x556997c2b7b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3908009f60 tx=0x7f3908021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df6f535a0dd8dd57 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:31.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.480+0000 7f3911600700 1 -- 172.21.15.131:0/3327993877 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390800c070 con 0x556997c207a0 2024-05-08T22:17:31.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.480+0000 7f3911600700 1 -- 172.21.15.131:0/3327993877 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3908009940 con 0x556997c207a0 2024-05-08T22:17:31.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.481+0000 7f3911600700 1 -- 172.21.15.131:0/3327993877 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390800c070 con 0x556997c207a0 2024-05-08T22:17:31.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.482+0000 7f3935f07b80 1 -- 172.21.15.131:0/3327993877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556997c207a0 msgr2=0x556997c2b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:31.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.482+0000 7f3935f07b80 1 --2- 172.21.15.131:0/3327993877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556997c207a0 0x556997c2b7b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3908009f60 tx=0x7f3908021000 comp rx=0 tx=0).stop 2024-05-08T22:17:31.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.482+0000 7f3935f07b80 1 -- 172.21.15.131:0/3327993877 shutdown_connections 2024-05-08T22:17:31.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.482+0000 7f3935f07b80 1 --2- 172.21.15.131:0/3327993877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556997c207a0 0x556997c2b7b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.482+0000 7f3935f07b80 1 --2- 172.21.15.131:0/3327993877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556997c1a910 0x556997c14da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.482+0000 7f3935f07b80 1 --2- 172.21.15.131:0/3327993877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556997c1fe80 0x556997c2b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.482+0000 7f3935f07b80 1 -- 172.21.15.131:0/3327993877 >> 172.21.15.131:0/3327993877 conn(0x556997b78290 msgr2=0x556997c22400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:31.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.483+0000 7f3935f07b80 1 -- 172.21.15.131:0/3327993877 shutdown_connections 2024-05-08T22:17:31.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.483+0000 7f3935f07b80 1 -- 172.21.15.131:0/3327993877 wait complete. 2024-05-08T22:17:31.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.483+0000 7f3935f07b80 1 -- start start 2024-05-08T22:17:31.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.484+0000 7f3935f07b80 1 Processor -- start 2024-05-08T22:17:31.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.485+0000 7f3935f07b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556997c1a910 0x556997c27db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.485+0000 7f3935f07b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556997c1fe80 0x556997c282f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.486+0000 7f3935f07b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556997c207a0 0x556997c28830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.486+0000 7f3935f07b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556997ca3e90 con 0x556997c1a910 2024-05-08T22:17:31.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.486+0000 7f3935f07b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556997ca3fd0 con 0x556997c1fe80 2024-05-08T22:17:31.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.486+0000 7f3935f07b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556997ca4110 con 0x556997c207a0 2024-05-08T22:17:31.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.486+0000 7f3935f07b80 10 client.0 _ll_register_callbacks cb 0x556997b0fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:17:31.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.486+0000 7f391b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556997c1a910 0x556997c27db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.486+0000 7f391aa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556997c1fe80 0x556997c282f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.486+0000 7f391b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556997c1a910 0x556997c27db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43986/0 (socket says 172.21.15.131:43986) 2024-05-08T22:17:31.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.486+0000 7f391b400700 1 -- 172.21.15.131:0/3540857377 learned_addr learned my addr 172.21.15.131:0/3540857377 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:31.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.487+0000 7f391be00700 1 --2- 172.21.15.131:0/3540857377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556997c207a0 0x556997c28830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.487+0000 7f391b400700 1 -- 172.21.15.131:0/3540857377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556997c207a0 msgr2=0x556997c28830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:31.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.487+0000 7f391b400700 1 --2- 172.21.15.131:0/3540857377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556997c207a0 0x556997c28830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.487+0000 7f391b400700 1 -- 172.21.15.131:0/3540857377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556997c1fe80 msgr2=0x556997c282f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:31.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.487+0000 7f391b400700 1 --2- 172.21.15.131:0/3540857377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556997c1fe80 0x556997c282f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.487+0000 7f391b400700 1 -- 172.21.15.131:0/3540857377 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569979eb430 con 0x556997c1a910 2024-05-08T22:17:31.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.487+0000 7f391aa00700 1 --2- 172.21.15.131:0/3540857377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556997c1fe80 0x556997c282f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:17:31.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.488+0000 7f391b400700 1 --2- 172.21.15.131:0/3540857377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556997c1a910 0x556997c27db0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f390c009ce0 tx=0x7f390c017ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce13c131adccb32 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:31.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.488+0000 7f3905600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:17:31.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.488+0000 7f3905600700 1 -- 172.21.15.131:0/3540857377 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c017ea0 con 0x556997c1a910 2024-05-08T22:17:31.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.489+0000 7f3905600700 1 -- 172.21.15.131:0/3540857377 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f390c00b2c0 con 0x556997c1a910 2024-05-08T22:17:31.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.489+0000 7f3935f07b80 1 -- 172.21.15.131:0/3540857377 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39080168f0 con 0x556997c1a910 2024-05-08T22:17:31.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.490+0000 7f3905600700 1 -- 172.21.15.131:0/3540857377 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f390c0163a0 con 0x556997c1a910 2024-05-08T22:17:31.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.491+0000 7f3935f07b80 10 client.16707 fetch_fsmap learned FSMap version 30 2024-05-08T22:17:31.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.491+0000 7f3935f07b80 1 -- 172.21.15.131:0/3540857377 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5569979efd50 con 0x556997c1a910 2024-05-08T22:17:31.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.494+0000 7f3905600700 1 -- 172.21.15.131:0/3540857377 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39080168f0 con 0x556997c1a910 2024-05-08T22:17:31.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.494+0000 7f3905600700 1 -- 172.21.15.131:0/3540857377 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f390c0099c0 con 0x556997c1a910 2024-05-08T22:17:31.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.494+0000 7f3935f07b80 10 client.16707 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:17:31.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.494+0000 7f3935f07b80 10 client.16707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:17:31.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.494+0000 7f3935f07b80 20 client.16707 populate_metadata read hostname 'smithi131' 2024-05-08T22:17:31.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.494+0000 7f3935f07b80 1 --2- 172.21.15.131:0/3540857377 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x556997ca54e0 0x556997ca7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.495+0000 7f3935f07b80 4 client.16707 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T22:17:31.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.495+0000 7f3935f07b80 1 -- 172.21.15.131:0/3540857377 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556997b62f40 con 0x556997ca54e0 2024-05-08T22:17:31.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.495+0000 7f391aa00700 1 --2- 172.21.15.131:0/3540857377 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x556997ca54e0 0x556997ca7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.496+0000 7f3905600700 10 client.16707 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:17:31.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.498+0000 7f391aa00700 1 --2- 172.21.15.131:0/3540857377 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x556997ca54e0 0x556997ca7900 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef87ea414a2d0345 server_cookie=f431ab3f010089d4 in_seq=0 out_seq=0 2024-05-08T22:17:31.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.500+0000 7f3905600700 1 -- 172.21.15.131:0/3540857377 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556997b62f40 con 0x556997ca54e0 2024-05-08T22:17:31.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.500+0000 7f3905600700 10 client.16707 handle_command_reply: tid=0 2024-05-08T22:17:31.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.500+0000 7f3935f07b80 1 client.16707 shutdown 2024-05-08T22:17:31.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.501+0000 7f3935f07b80 1 -- 172.21.15.131:0/3540857377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556997c1a910 msgr2=0x556997c27db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:31.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.501+0000 7f3935f07b80 1 --2- 172.21.15.131:0/3540857377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556997c1a910 0x556997c27db0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f390c009ce0 tx=0x7f390c017ad0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.501+0000 7f3935f07b80 1 -- 172.21.15.131:0/3540857377 shutdown_connections 2024-05-08T22:17:31.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.501+0000 7f3935f07b80 1 --2- 172.21.15.131:0/3540857377 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x556997ca54e0 0x556997ca7900 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.502+0000 7f3935f07b80 1 --2- 172.21.15.131:0/3540857377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556997c207a0 0x556997c28830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.502+0000 7f3905600700 0 client.16707 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:17:31.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.502+0000 7f3935f07b80 1 --2- 172.21.15.131:0/3540857377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556997c1a910 0x556997c27db0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.502+0000 7f3935f07b80 1 --2- 172.21.15.131:0/3540857377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556997c1fe80 0x556997c282f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.503+0000 7f3935f07b80 1 -- 172.21.15.131:0/3540857377 >> 172.21.15.131:0/3540857377 conn(0x556997b78290 msgr2=0x556997c1ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:31.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.503+0000 7f3935f07b80 1 -- 172.21.15.131:0/3540857377 shutdown_connections 2024-05-08T22:17:31.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.503+0000 7f3935f07b80 1 -- 172.21.15.131:0/3540857377 wait complete. 2024-05-08T22:17:31.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.504+0000 7f3935f07b80 20 client.16707 trim_cache size 0 max 16384 2024-05-08T22:17:31.505 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:17:31.505 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-08T22:17:31.505 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "09b0b710-8316-4d4d-9c6e-7daf17ae730b", 2024-05-08T22:17:31.505 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-08T22:17:31.506 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:17:31.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.505+0000 7f3922400700 1 -- 172.21.15.131:0/2660783217 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f39000a1390 msgr2=0x7f39000a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:31.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.505+0000 7f3922400700 1 --2- 172.21.15.131:0/2660783217 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f39000a1390 0x7f39000a3850 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3908012fd0 tx=0x7f3908015670 comp rx=0 tx=0).stop 2024-05-08T22:17:31.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.505+0000 7f3922400700 1 -- 172.21.15.131:0/2660783217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f391c108470 msgr2=0x7f391c199c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:31.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.505+0000 7f3922400700 1 --2- 172.21.15.131:0/2660783217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f391c108470 0x7f391c199c00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f391400efc0 tx=0x7f391400c3a0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.507+0000 7f3922400700 1 -- 172.21.15.131:0/2660783217 shutdown_connections 2024-05-08T22:17:31.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.507+0000 7f3922400700 1 --2- 172.21.15.131:0/2660783217 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f39000a1390 0x7f39000a3850 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.507+0000 7f3922400700 1 --2- 172.21.15.131:0/2660783217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f391c19df50 0x7f391c19e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.507+0000 7f3922400700 1 --2- 172.21.15.131:0/2660783217 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f391c103350 0x7f391c1996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.507+0000 7f3922400700 1 --2- 172.21.15.131:0/2660783217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f391c108470 0x7f391c199c00 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.507+0000 7f3922400700 1 -- 172.21.15.131:0/2660783217 >> 172.21.15.131:0/2660783217 conn(0x7f391c0febb0 msgr2=0x7f391c0fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:31.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.507+0000 7f3922400700 1 -- 172.21.15.131:0/2660783217 shutdown_connections 2024-05-08T22:17:31.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.507+0000 7f3922400700 1 -- 172.21.15.131:0/2660783217 wait complete. 2024-05-08T22:17:31.553 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:17:31.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.930+0000 7fb97c600700 1 -- 172.21.15.131:0/3255929025 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb97410d960 msgr2=0x7fb97410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:31.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.930+0000 7fb97c600700 1 --2- 172.21.15.131:0/3255929025 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb97410d960 0x7fb97410dd40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb96c00c310 tx=0x7fb96c00c620 comp rx=0 tx=0).stop 2024-05-08T22:17:31.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.931+0000 7fb97c600700 1 -- 172.21.15.131:0/3255929025 shutdown_connections 2024-05-08T22:17:31.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.931+0000 7fb97c600700 1 --2- 172.21.15.131:0/3255929025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb974108960 0x7fb9741147b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.931+0000 7fb97c600700 1 --2- 172.21.15.131:0/3255929025 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb97410d960 0x7fb97410dd40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.931+0000 7fb97c600700 1 --2- 172.21.15.131:0/3255929025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb974107fa0 0x7fb974108420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.931+0000 7fb97c600700 1 -- 172.21.15.131:0/3255929025 >> 172.21.15.131:0/3255929025 conn(0x7fb97406c440 msgr2=0x7fb97406c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:31.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.931+0000 7fb97c600700 1 -- 172.21.15.131:0/3255929025 shutdown_connections 2024-05-08T22:17:31.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.931+0000 7fb97c600700 1 -- 172.21.15.131:0/3255929025 wait complete. 2024-05-08T22:17:31.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.933+0000 7fb97c600700 1 Processor -- start 2024-05-08T22:17:31.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.933+0000 7fb97c600700 1 -- start start 2024-05-08T22:17:31.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.934+0000 7fb97c600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb974107fa0 0x7fb9741a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.935+0000 7fb97c600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb974108960 0x7fb9741a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.935+0000 7fb97c600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb97410d960 0x7fb97419db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.935+0000 7fb97c600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97419e2f0 con 0x7fb97410d960 2024-05-08T22:17:31.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.935+0000 7fb97c600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97419e460 con 0x7fb974108960 2024-05-08T22:17:31.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.935+0000 7fb97c600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97419e620 con 0x7fb974107fa0 2024-05-08T22:17:31.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.935+0000 7fb979c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb974107fa0 0x7fb9741a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.936+0000 7fb979c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb974107fa0 0x7fb9741a48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53146/0 (socket says 172.21.15.131:53146) 2024-05-08T22:17:31.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.936+0000 7fb979c00700 1 -- 172.21.15.131:0/1958551322 learned_addr learned my addr 172.21.15.131:0/1958551322 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:31.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.936+0000 7fb979200700 1 --2- 172.21.15.131:0/1958551322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb974108960 0x7fb9741a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.936+0000 7fb979c00700 1 -- 172.21.15.131:0/1958551322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb974108960 msgr2=0x7fb9741a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:31.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.936+0000 7fb979c00700 1 --2- 172.21.15.131:0/1958551322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb974108960 0x7fb9741a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.936+0000 7fb979c00700 1 -- 172.21.15.131:0/1958551322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb97410d960 msgr2=0x7fb97419db20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T22:17:31.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.937+0000 7fb97a600700 1 --2- 172.21.15.131:0/1958551322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb97410d960 0x7fb97419db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.937+0000 7fb979c00700 1 --2- 172.21.15.131:0/1958551322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb97410d960 0x7fb97419db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.937+0000 7fb979c00700 1 -- 172.21.15.131:0/1958551322 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96c00c050 con 0x7fb974107fa0 2024-05-08T22:17:31.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.937+0000 7fb979200700 1 --2- 172.21.15.131:0/1958551322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb974108960 0x7fb9741a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:17:31.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.938+0000 7fb97a600700 1 --2- 172.21.15.131:0/1958551322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb97410d960 0x7fb97419db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:17:31.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.938+0000 7fb979c00700 1 --2- 172.21.15.131:0/1958551322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb974107fa0 0x7fb9741a48e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0062a0 tx=0x7fb96c0127c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:31.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.938+0000 7fb972000700 1 -- 172.21.15.131:0/1958551322 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c00f040 con 0x7fb974107fa0 2024-05-08T22:17:31.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.938+0000 7fb972000700 1 -- 172.21.15.131:0/1958551322 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb96c012ea0 con 0x7fb974107fa0 2024-05-08T22:17:31.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.938+0000 7fb972000700 1 -- 172.21.15.131:0/1958551322 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c0057f0 con 0x7fb974107fa0 2024-05-08T22:17:31.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.939+0000 7fb97c600700 1 -- 172.21.15.131:0/1958551322 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9741a2400 con 0x7fb974107fa0 2024-05-08T22:17:31.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.939+0000 7fb97c600700 1 -- 172.21.15.131:0/1958551322 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9741a2860 con 0x7fb974107fa0 2024-05-08T22:17:31.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.950+0000 7fb972000700 1 -- 172.21.15.131:0/1958551322 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fb96c005950 con 0x7fb974107fa0 2024-05-08T22:17:31.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.951+0000 7fb99004db80 1 Processor -- start 2024-05-08T22:17:31.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.952+0000 7fb99004db80 1 -- start start 2024-05-08T22:17:31.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.953+0000 7fb99004db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7ee00 0x560409d7f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.953+0000 7fb99004db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560409d7f720 0x560409d79350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.953+0000 7fb99004db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560409d79890 0x560409d79cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.954+0000 7fb99004db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560409cd0150 con 0x560409d7f720 2024-05-08T22:17:31.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.954+0000 7fb99004db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560409ccdb10 con 0x560409d79890 2024-05-08T22:17:31.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.954+0000 7fb99004db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560409cc9400 con 0x560409d7ee00 2024-05-08T22:17:31.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.954+0000 7fb979c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560409d7f720 0x560409d79350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.954+0000 7fb979c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560409d7f720 0x560409d79350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44018/0 (socket says 172.21.15.131:44018) 2024-05-08T22:17:31.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.954+0000 7fb979c00700 1 -- 172.21.15.131:0/473995442 learned_addr learned my addr 172.21.15.131:0/473995442 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:31.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.955+0000 7fb97a600700 1 --2- 172.21.15.131:0/473995442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7ee00 0x560409d7f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.955+0000 7fb972000700 1 --2- 172.21.15.131:0/1958551322 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb95c0a1300 0x7fb95c0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.955+0000 7fb972000700 1 -- 172.21.15.131:0/1958551322 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fb96c0ce780 con 0x7fb974107fa0 2024-05-08T22:17:31.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.956+0000 7fb97a600700 1 -- 172.21.15.131:0/473995442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560409d79890 msgr2=0x560409d79cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:17:31.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.956+0000 7fb97a600700 1 --2- 172.21.15.131:0/473995442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560409d79890 0x560409d79cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.956+0000 7fb97a600700 1 -- 172.21.15.131:0/473995442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560409d7f720 msgr2=0x560409d79350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:31.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.956+0000 7fb97a600700 1 --2- 172.21.15.131:0/473995442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560409d7f720 0x560409d79350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.956+0000 7fb97a600700 1 -- 172.21.15.131:0/473995442 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560409b4a3b0 con 0x560409d7ee00 2024-05-08T22:17:31.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.956+0000 7fb97a600700 1 --2- 172.21.15.131:0/473995442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7ee00 0x560409d7f1e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb968009d00 tx=0x7fb96800e5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9951eea6bed0da16 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:31.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.957+0000 7fb97a600700 1 --2- 172.21.15.131:0/1958551322 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb95c0a1300 0x7fb95c0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.958+0000 7fb962a00700 1 -- 172.21.15.131:0/473995442 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96800ede0 con 0x560409d7ee00 2024-05-08T22:17:31.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.958+0000 7fb962a00700 1 -- 172.21.15.131:0/473995442 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb968004500 con 0x560409d7ee00 2024-05-08T22:17:31.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.958+0000 7fb962a00700 1 -- 172.21.15.131:0/473995442 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96801b6e0 con 0x560409d7ee00 2024-05-08T22:17:31.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.961+0000 7fb97a600700 1 --2- 172.21.15.131:0/1958551322 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb95c0a1300 0x7fb95c0a37c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb968000c00 tx=0x7fb968003f70 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:31.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.961+0000 7fb99004db80 1 -- 172.21.15.131:0/473995442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7ee00 msgr2=0x560409d7f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:31.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.961+0000 7fb99004db80 1 --2- 172.21.15.131:0/473995442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7ee00 0x560409d7f1e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb968009d00 tx=0x7fb96800e5d0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.962+0000 7fb99004db80 1 -- 172.21.15.131:0/473995442 shutdown_connections 2024-05-08T22:17:31.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.962+0000 7fb99004db80 1 --2- 172.21.15.131:0/473995442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560409d79890 0x560409d79cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.962+0000 7fb99004db80 1 --2- 172.21.15.131:0/473995442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7ee00 0x560409d7f1e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.962+0000 7fb99004db80 1 --2- 172.21.15.131:0/473995442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560409d7f720 0x560409d79350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.962+0000 7fb99004db80 1 -- 172.21.15.131:0/473995442 >> 172.21.15.131:0/473995442 conn(0x560409cd7210 msgr2=0x560409d73df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:31.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.963+0000 7fb99004db80 1 -- 172.21.15.131:0/473995442 shutdown_connections 2024-05-08T22:17:31.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.963+0000 7fb99004db80 1 -- 172.21.15.131:0/473995442 wait complete. 2024-05-08T22:17:31.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.963+0000 7fb99004db80 1 -- start start 2024-05-08T22:17:31.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.964+0000 7fb99004db80 1 Processor -- start 2024-05-08T22:17:31.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.965+0000 7fb99004db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560409d79890 0x560409d82b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.965+0000 7fb99004db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560409d7ee00 0x560409d830c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.966+0000 7fb99004db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7f720 0x560409d83600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.966+0000 7fb99004db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560409c65560 con 0x560409d7ee00 2024-05-08T22:17:31.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.966+0000 7fb99004db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560409c65200 con 0x560409d79890 2024-05-08T22:17:31.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.966+0000 7fb99004db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560409b4c6a0 con 0x560409d7f720 2024-05-08T22:17:31.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.966+0000 7fb97a600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7f720 0x560409d83600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.966+0000 7fb97a600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7f720 0x560409d83600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53172/0 (socket says 172.21.15.131:53172) 2024-05-08T22:17:31.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.966+0000 7fb99004db80 10 client.0 _ll_register_callbacks cb 0x560409c6eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:17:31.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.966+0000 7fb97a600700 1 -- 172.21.15.131:0/3697019295 learned_addr learned my addr 172.21.15.131:0/3697019295 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:31.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.966+0000 7fb979c00700 1 --2- 172.21.15.131:0/3697019295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560409d79890 0x560409d82b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.967+0000 7fb979200700 1 --2- 172.21.15.131:0/3697019295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560409d7ee00 0x560409d830c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.967+0000 7fb97a600700 1 -- 172.21.15.131:0/3697019295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560409d79890 msgr2=0x560409d82b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:31.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.968+0000 7fb97a600700 1 --2- 172.21.15.131:0/3697019295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560409d79890 0x560409d82b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.968+0000 7fb97a600700 1 -- 172.21.15.131:0/3697019295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560409d7ee00 msgr2=0x560409d830c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:31.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.968+0000 7fb97a600700 1 --2- 172.21.15.131:0/3697019295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560409d7ee00 0x560409d830c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.968+0000 7fb97a600700 1 -- 172.21.15.131:0/3697019295 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb968009800 con 0x560409d7f720 2024-05-08T22:17:31.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.968+0000 7fb97a600700 1 --2- 172.21.15.131:0/3697019295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7f720 0x560409d83600 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb968003db0 tx=0x7fb968018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b681547aefc69c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:31.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.971+0000 7fb957400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:17:31.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.971+0000 7fb957400700 1 -- 172.21.15.131:0/3697019295 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96801f040 con 0x560409d7f720 2024-05-08T22:17:31.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.971+0000 7fb957400700 1 -- 172.21.15.131:0/3697019295 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fb968016070 con 0x560409d7f720 2024-05-08T22:17:31.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.971+0000 7fb99004db80 1 -- 172.21.15.131:0/3697019295 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560409c22270 con 0x560409d7f720 2024-05-08T22:17:31.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.971+0000 7fb957400700 1 -- 172.21.15.131:0/3697019295 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb96801b6e0 con 0x560409d7f720 2024-05-08T22:17:31.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.971+0000 7fb99004db80 10 client.26902 fetch_fsmap learned FSMap version 30 2024-05-08T22:17:31.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.971+0000 7fb99004db80 1 -- 172.21.15.131:0/3697019295 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560409af6b60 con 0x560409d7f720 2024-05-08T22:17:31.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.972+0000 7fb957400700 1 -- 172.21.15.131:0/3697019295 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560409c22270 con 0x560409d7f720 2024-05-08T22:17:31.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.972+0000 7fb957400700 1 -- 172.21.15.131:0/3697019295 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb968022050 con 0x560409d7f720 2024-05-08T22:17:31.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.972+0000 7fb99004db80 10 client.26902 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:17:31.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.972+0000 7fb99004db80 10 client.26902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:17:31.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.972+0000 7fb99004db80 20 client.26902 populate_metadata read hostname 'smithi131' 2024-05-08T22:17:31.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.972+0000 7fb99004db80 1 --2- 172.21.15.131:0/3697019295 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560409d93050 0x560409d95470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:31.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.972+0000 7fb99004db80 4 client.26902 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:17:31.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.972+0000 7fb99004db80 1 -- 172.21.15.131:0/3697019295 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560409d48810 con 0x560409d93050 2024-05-08T22:17:31.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.972+0000 7fb979c00700 1 --2- 172.21.15.131:0/3697019295 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560409d93050 0x560409d95470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:31.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.973+0000 7fb957400700 10 client.26902 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:17:31.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.973+0000 7fb979c00700 1 --2- 172.21.15.131:0/3697019295 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560409d93050 0x560409d95470 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69d856d84cfb5740 server_cookie=fedcac5917ff75ef in_seq=0 out_seq=0 2024-05-08T22:17:31.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.974+0000 7fb957400700 1 -- 172.21.15.131:0/3697019295 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560409d48810 con 0x560409d93050 2024-05-08T22:17:31.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.974+0000 7fb957400700 10 client.26902 handle_command_reply: tid=0 2024-05-08T22:17:31.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.974+0000 7fb99004db80 1 client.26902 shutdown 2024-05-08T22:17:31.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.977+0000 7fb99004db80 1 -- 172.21.15.131:0/3697019295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7f720 msgr2=0x560409d83600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:31.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.977+0000 7fb99004db80 1 --2- 172.21.15.131:0/3697019295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7f720 0x560409d83600 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb968003db0 tx=0x7fb968018000 comp rx=0 tx=0).stop 2024-05-08T22:17:31.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.978+0000 7fb99004db80 1 -- 172.21.15.131:0/3697019295 shutdown_connections 2024-05-08T22:17:31.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.978+0000 7fb99004db80 1 --2- 172.21.15.131:0/3697019295 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560409d93050 0x560409d95470 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.978+0000 7fb99004db80 1 --2- 172.21.15.131:0/3697019295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7f720 0x560409d83600 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.978+0000 7fb99004db80 1 --2- 172.21.15.131:0/3697019295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560409d79890 0x560409d82b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.978+0000 7fb99004db80 1 --2- 172.21.15.131:0/3697019295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560409d7ee00 0x560409d830c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:31.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.978+0000 7fb99004db80 1 -- 172.21.15.131:0/3697019295 >> 172.21.15.131:0/3697019295 conn(0x560409cd7210 msgr2=0x560409d735a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:31.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.978+0000 7fb957400700 0 client.26902 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:17:31.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.978+0000 7fb99004db80 1 -- 172.21.15.131:0/3697019295 shutdown_connections 2024-05-08T22:17:31.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.979+0000 7fb99004db80 1 -- 172.21.15.131:0/3697019295 wait complete. 2024-05-08T22:17:31.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:31.979+0000 7fb99004db80 20 client.26902 trim_cache size 0 max 16384 2024-05-08T22:17:32.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.016+0000 7fb99004db80 1 Processor -- start 2024-05-08T22:17:32.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.016+0000 7fb99004db80 1 -- start start 2024-05-08T22:17:32.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.016+0000 7fb99004db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560409d79890 0x560409d79350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:32.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.016+0000 7fb99004db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560409d7ee00 0x560409d7e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:32.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.016+0000 7fb99004db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7f720 0x560409d73d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:32.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.016+0000 7fb99004db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560409d85e50 con 0x560409d79890 2024-05-08T22:17:32.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.016+0000 7fb99004db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560409d6c040 con 0x560409d7ee00 2024-05-08T22:17:32.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.016+0000 7fb99004db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560409d84180 con 0x560409d7f720 2024-05-08T22:17:32.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.017+0000 7fb97a600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7f720 0x560409d73d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:32.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.017+0000 7fb979200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560409d7ee00 0x560409d7e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:32.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.017+0000 7fb979200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560409d7ee00 0x560409d7e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:44088/0 (socket says 172.21.15.131:44088) 2024-05-08T22:17:32.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.017+0000 7fb979200700 1 -- 172.21.15.131:0/558414584 learned_addr learned my addr 172.21.15.131:0/558414584 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:32.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.017+0000 7fb97a600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7f720 0x560409d73d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53178/0 (socket says 172.21.15.131:53178) 2024-05-08T22:17:32.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.018+0000 7fb979200700 1 -- 172.21.15.131:0/558414584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7f720 msgr2=0x560409d73d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:32.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.018+0000 7fb979c00700 1 --2- 172.21.15.131:0/558414584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560409d79890 0x560409d79350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:32.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.018+0000 7fb979200700 1 --2- 172.21.15.131:0/558414584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7f720 0x560409d73d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.018+0000 7fb979200700 1 -- 172.21.15.131:0/558414584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560409d79890 msgr2=0x560409d79350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:32.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.018+0000 7fb979200700 1 --2- 172.21.15.131:0/558414584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560409d79890 0x560409d79350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.018+0000 7fb979200700 1 -- 172.21.15.131:0/558414584 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560409d79f40 con 0x560409d7ee00 2024-05-08T22:17:32.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.018+0000 7fb97a600700 1 --2- 172.21.15.131:0/558414584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7f720 0x560409d73d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:17:32.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.018+0000 7fb979c00700 1 --2- 172.21.15.131:0/558414584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560409d79890 0x560409d79350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:17:32.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.018+0000 7fb979200700 1 --2- 172.21.15.131:0/558414584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560409d7ee00 0x560409d7e4e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb964009e30 tx=0x7fb964009190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e035f07969a92618 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:32.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.019+0000 7fb963400700 1 -- 172.21.15.131:0/558414584 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb964011c60 con 0x560409d7ee00 2024-05-08T22:17:32.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.019+0000 7fb963400700 1 -- 172.21.15.131:0/558414584 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb964011dc0 con 0x560409d7ee00 2024-05-08T22:17:32.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.019+0000 7fb963400700 1 -- 172.21.15.131:0/558414584 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9640146d0 con 0x560409d7ee00 2024-05-08T22:17:32.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.022+0000 7fb99004db80 1 -- 172.21.15.131:0/558414584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560409d7ee00 msgr2=0x560409d7e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:32.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.022+0000 7fb99004db80 1 --2- 172.21.15.131:0/558414584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560409d7ee00 0x560409d7e4e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb964009e30 tx=0x7fb964009190 comp rx=0 tx=0).stop 2024-05-08T22:17:32.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.023+0000 7fb99004db80 1 -- 172.21.15.131:0/558414584 shutdown_connections 2024-05-08T22:17:32.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.023+0000 7fb99004db80 1 --2- 172.21.15.131:0/558414584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7f720 0x560409d73d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.023+0000 7fb99004db80 1 --2- 172.21.15.131:0/558414584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560409d79890 0x560409d79350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.023+0000 7fb99004db80 1 --2- 172.21.15.131:0/558414584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560409d7ee00 0x560409d7e4e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.023+0000 7fb99004db80 1 -- 172.21.15.131:0/558414584 >> 172.21.15.131:0/558414584 conn(0x560409cd7210 msgr2=0x560409d81450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:32.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.024+0000 7fb99004db80 1 -- 172.21.15.131:0/558414584 shutdown_connections 2024-05-08T22:17:32.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.024+0000 7fb99004db80 1 -- 172.21.15.131:0/558414584 wait complete. 2024-05-08T22:17:32.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.025+0000 7fb99004db80 1 -- start start 2024-05-08T22:17:32.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.025+0000 7fb99004db80 1 Processor -- start 2024-05-08T22:17:32.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.027+0000 7fb99004db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560409d79890 0x560409d8a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:32.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.027+0000 7fb99004db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7ee00 0x560409d88b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:32.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.027+0000 7fb99004db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560409d7f720 0x560409d89280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:32.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.028+0000 7fb99004db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560409d89960 con 0x560409d7f720 2024-05-08T22:17:32.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.028+0000 7fb99004db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560409de9540 con 0x560409d79890 2024-05-08T22:17:32.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.028+0000 7fb99004db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560409de9680 con 0x560409d7ee00 2024-05-08T22:17:32.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.028+0000 7fb99004db80 10 client.0 _ll_register_callbacks cb 0x560409c6eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:17:32.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.028+0000 7fb979c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560409d79890 0x560409d8a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:32.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.028+0000 7fb979200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7ee00 0x560409d88b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:32.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.029+0000 7fb979200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7ee00 0x560409d88b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53188/0 (socket says 172.21.15.131:53188) 2024-05-08T22:17:32.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.029+0000 7fb979200700 1 -- 172.21.15.131:0/4045212460 learned_addr learned my addr 172.21.15.131:0/4045212460 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:32.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.030+0000 7fb979200700 1 -- 172.21.15.131:0/4045212460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560409d79890 msgr2=0x560409d8a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:32.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.030+0000 7fb979200700 1 --2- 172.21.15.131:0/4045212460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560409d79890 0x560409d8a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.030+0000 7fb979200700 1 -- 172.21.15.131:0/4045212460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560409d7f720 msgr2=0x560409d89280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:17:32.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.030+0000 7fb979200700 1 --2- 172.21.15.131:0/4045212460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560409d7f720 0x560409d89280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.030+0000 7fb979200700 1 -- 172.21.15.131:0/4045212460 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb964009ae0 con 0x560409d7ee00 2024-05-08T22:17:32.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.032+0000 7fb957400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:17:32.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.032+0000 7fb979200700 1 --2- 172.21.15.131:0/4045212460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7ee00 0x560409d88b40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb96401b000 tx=0x7fb964015e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fe6a310de209e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:32.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.032+0000 7fb957400700 1 -- 172.21.15.131:0/4045212460 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96401e860 con 0x560409d7ee00 2024-05-08T22:17:32.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.033+0000 7fb957400700 1 -- 172.21.15.131:0/4045212460 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fb96401e9c0 con 0x560409d7ee00 2024-05-08T22:17:32.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.033+0000 7fb99004db80 1 -- 172.21.15.131:0/4045212460 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb968004500 con 0x560409d7ee00 2024-05-08T22:17:32.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.034+0000 7fb957400700 1 -- 172.21.15.131:0/4045212460 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb964015c80 con 0x560409d7ee00 2024-05-08T22:17:32.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.034+0000 7fb99004db80 10 client.26908 fetch_fsmap learned FSMap version 30 2024-05-08T22:17:32.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.034+0000 7fb99004db80 1 -- 172.21.15.131:0/4045212460 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560409c8f560 con 0x560409d7ee00 2024-05-08T22:17:32.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.034+0000 7fb957400700 1 -- 172.21.15.131:0/4045212460 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb968004500 con 0x560409d7ee00 2024-05-08T22:17:32.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.035+0000 7fb957400700 1 -- 172.21.15.131:0/4045212460 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb964029b40 con 0x560409d7ee00 2024-05-08T22:17:32.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.035+0000 7fb99004db80 10 client.26908 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:17:32.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.035+0000 7fb99004db80 10 client.26908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:17:32.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.035+0000 7fb99004db80 20 client.26908 populate_metadata read hostname 'smithi131' 2024-05-08T22:17:32.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.035+0000 7fb99004db80 1 --2- 172.21.15.131:0/4045212460 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560409deabc0 0x560409decfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:32.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.035+0000 7fb99004db80 4 client.26908 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:17:32.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.035+0000 7fb99004db80 1 -- 172.21.15.131:0/4045212460 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560409cc1ec0 con 0x560409deabc0 2024-05-08T22:17:32.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.036+0000 7fb979c00700 1 --2- 172.21.15.131:0/4045212460 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560409deabc0 0x560409decfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:32.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.036+0000 7fb957400700 10 client.26908 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:17:32.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.037+0000 7fb979c00700 1 --2- 172.21.15.131:0/4045212460 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560409deabc0 0x560409decfe0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80031af097b129df server_cookie=655a29736f47c618 in_seq=0 out_seq=0 2024-05-08T22:17:32.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.037+0000 7fb957400700 1 -- 172.21.15.131:0/4045212460 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560409cc1ec0 con 0x560409deabc0 2024-05-08T22:17:32.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.037+0000 7fb957400700 10 client.26908 handle_command_reply: tid=0 2024-05-08T22:17:32.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.037+0000 7fb99004db80 1 client.26908 shutdown 2024-05-08T22:17:32.042 DEBUG:teuthology.orchestra.run.smithi131:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-08T22:17:32.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.041+0000 7fb99004db80 1 -- 172.21.15.131:0/4045212460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7ee00 msgr2=0x560409d88b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:32.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.041+0000 7fb99004db80 1 --2- 172.21.15.131:0/4045212460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7ee00 0x560409d88b40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb96401b000 tx=0x7fb964015e20 comp rx=0 tx=0).stop 2024-05-08T22:17:32.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.041+0000 7fb99004db80 1 -- 172.21.15.131:0/4045212460 shutdown_connections 2024-05-08T22:17:32.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.041+0000 7fb99004db80 1 --2- 172.21.15.131:0/4045212460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560409d7f720 0x560409d89280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.041+0000 7fb99004db80 1 --2- 172.21.15.131:0/4045212460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560409d79890 0x560409d8a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.041+0000 7fb99004db80 1 --2- 172.21.15.131:0/4045212460 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x560409deabc0 0x560409decfe0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.041+0000 7fb99004db80 1 --2- 172.21.15.131:0/4045212460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560409d7ee00 0x560409d88b40 secure :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb96401b000 tx=0x7fb964015e20 comp rx=0 tx=0).stop 2024-05-08T22:17:32.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.041+0000 7fb99004db80 1 -- 172.21.15.131:0/4045212460 >> 172.21.15.131:0/4045212460 conn(0x560409cd7210 msgr2=0x560409d7e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:32.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.041+0000 7fb957400700 0 client.26908 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:17:32.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.041+0000 7fb99004db80 1 -- 172.21.15.131:0/4045212460 shutdown_connections 2024-05-08T22:17:32.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.042+0000 7fb99004db80 1 -- 172.21.15.131:0/4045212460 wait complete. 2024-05-08T22:17:32.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.043+0000 7fb99004db80 20 client.26908 trim_cache size 0 max 16384 2024-05-08T22:17:32.049 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:17:32.049 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T22:17:32.049 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-08T22:17:32.050 INFO:teuthology.orchestra.run.smithi131.stdout: "09b0b710-8316-4d4d-9c6e-7daf17ae730b": { 2024-05-08T22:17:32.050 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-08T22:17:32.050 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "09b0b710-8316-4d4d-9c6e-7daf17ae730b", 2024-05-08T22:17:32.050 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-08T22:17:32.050 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:17:32.050 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:17:32.050 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:17:32.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.046+0000 7fb97c600700 1 -- 172.21.15.131:0/1958551322 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb95c0a1300 msgr2=0x7fb95c0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:32.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.046+0000 7fb97c600700 1 --2- 172.21.15.131:0/1958551322 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb95c0a1300 0x7fb95c0a37c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb968000c00 tx=0x7fb968003f70 comp rx=0 tx=0).stop 2024-05-08T22:17:32.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.046+0000 7fb97c600700 1 -- 172.21.15.131:0/1958551322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb974107fa0 msgr2=0x7fb9741a48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:32.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.046+0000 7fb97c600700 1 --2- 172.21.15.131:0/1958551322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb974107fa0 0x7fb9741a48e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0062a0 tx=0x7fb96c0127c0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.048+0000 7fb97c600700 1 -- 172.21.15.131:0/1958551322 shutdown_connections 2024-05-08T22:17:32.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.048+0000 7fb97c600700 1 --2- 172.21.15.131:0/1958551322 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb95c0a1300 0x7fb95c0a37c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.048+0000 7fb97c600700 1 --2- 172.21.15.131:0/1958551322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb97410d960 0x7fb97419db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.048+0000 7fb97c600700 1 --2- 172.21.15.131:0/1958551322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb974107fa0 0x7fb9741a48e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.048+0000 7fb97c600700 1 --2- 172.21.15.131:0/1958551322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb974108960 0x7fb9741a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.048+0000 7fb97c600700 1 -- 172.21.15.131:0/1958551322 >> 172.21.15.131:0/1958551322 conn(0x7fb97406c440 msgr2=0x7fb974113070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:32.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.049+0000 7fb97c600700 1 -- 172.21.15.131:0/1958551322 shutdown_connections 2024-05-08T22:17:32.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.049+0000 7fb97c600700 1 -- 172.21.15.131:0/1958551322 wait complete. 2024-05-08T22:17:32.090 INFO:tasks.cephfs.filesystem:scrub status for tag:09b0b710-8316-4d4d-9c6e-7daf17ae730b - {'path': '/', 'tag': '09b0b710-8316-4d4d-9c6e-7daf17ae730b', 'options': 'recursive,force'} 2024-05-08T22:17:32.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.370+0000 7f63dbe00700 1 -- 172.21.15.131:0/4272490349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63dc06b010 msgr2=0x7f63dc06b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:32.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.370+0000 7f63dbe00700 1 --2- 172.21.15.131:0/4272490349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63dc06b010 0x7f63dc06b490 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f63c8009990 tx=0x7f63c8009ca0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.370+0000 7f63dbe00700 1 -- 172.21.15.131:0/4272490349 shutdown_connections 2024-05-08T22:17:32.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.370+0000 7f63dbe00700 1 --2- 172.21.15.131:0/4272490349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63dc112ae0 0x7f63dc114ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.370+0000 7f63dbe00700 1 --2- 172.21.15.131:0/4272490349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63dc072cf0 0x7f63dc06aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.370+0000 7f63dbe00700 1 --2- 172.21.15.131:0/4272490349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63dc06b010 0x7f63dc06b490 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.370+0000 7f63dbe00700 1 -- 172.21.15.131:0/4272490349 >> 172.21.15.131:0/4272490349 conn(0x7f63dc076a30 msgr2=0x7f63dc0fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:32.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.371+0000 7f63dbe00700 1 -- 172.21.15.131:0/4272490349 shutdown_connections 2024-05-08T22:17:32.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.371+0000 7f63dbe00700 1 -- 172.21.15.131:0/4272490349 wait complete. 2024-05-08T22:17:32.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.373+0000 7f63dbe00700 1 Processor -- start 2024-05-08T22:17:32.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.373+0000 7f63dbe00700 1 -- start start 2024-05-08T22:17:32.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.374+0000 7f63dbe00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63dc072cf0 0x7f63dc112f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:32.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.374+0000 7f63dbe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63dc112ae0 0x7f63dc1134d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:32.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.374+0000 7f63da000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63dc112ae0 0x7f63dc1134d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:32.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.374+0000 7f63da000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63dc112ae0 0x7f63dc1134d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44066/0 (socket says 172.21.15.131:44066) 2024-05-08T22:17:32.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.374+0000 7f63daa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63dc072cf0 0x7f63dc112f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:32.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.374+0000 7f63dbe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63dc114c50 0x7f63dc113a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:32.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.375+0000 7f63dbe00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc1b2890 con 0x7f63dc112ae0 2024-05-08T22:17:32.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.375+0000 7f63dbe00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc1b2a00 con 0x7f63dc072cf0 2024-05-08T22:17:32.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.375+0000 7f63dbe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc1b2bc0 con 0x7f63dc114c50 2024-05-08T22:17:32.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.375+0000 7f63daa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63dc072cf0 0x7f63dc112f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:44122/0 (socket says 172.21.15.131:44122) 2024-05-08T22:17:32.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.375+0000 7f63daa00700 1 -- 172.21.15.131:0/949354649 learned_addr learned my addr 172.21.15.131:0/949354649 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:32.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.375+0000 7f63db400700 1 --2- 172.21.15.131:0/949354649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63dc114c50 0x7f63dc113a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:32.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.375+0000 7f63daa00700 1 -- 172.21.15.131:0/949354649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63dc114c50 msgr2=0x7f63dc113a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:32.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.375+0000 7f63daa00700 1 --2- 172.21.15.131:0/949354649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63dc114c50 0x7f63dc113a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.375+0000 7f63daa00700 1 -- 172.21.15.131:0/949354649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63dc112ae0 msgr2=0x7f63dc1134d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:32.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.375+0000 7f63daa00700 1 --2- 172.21.15.131:0/949354649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63dc112ae0 0x7f63dc1134d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.375+0000 7f63daa00700 1 -- 172.21.15.131:0/949354649 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c80095f0 con 0x7f63dc072cf0 2024-05-08T22:17:32.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.375+0000 7f63da000700 1 --2- 172.21.15.131:0/949354649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63dc112ae0 0x7f63dc1134d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:17:32.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.376+0000 7f63daa00700 1 --2- 172.21.15.131:0/949354649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63dc072cf0 0x7f63dc112f90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f63d4007f10 tx=0x7f63d400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:32.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.376+0000 7f63e0a00700 1 -- 172.21.15.131:0/949354649 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d400ddd0 con 0x7f63dc072cf0 2024-05-08T22:17:32.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.376+0000 7f63e0a00700 1 -- 172.21.15.131:0/949354649 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63d400f040 con 0x7f63dc072cf0 2024-05-08T22:17:32.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.376+0000 7f63e0a00700 1 -- 172.21.15.131:0/949354649 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d4004530 con 0x7f63dc072cf0 2024-05-08T22:17:32.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.377+0000 7f63dbe00700 1 -- 172.21.15.131:0/949354649 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63dc1b2ea0 con 0x7f63dc072cf0 2024-05-08T22:17:32.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.377+0000 7f63dbe00700 1 -- 172.21.15.131:0/949354649 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63dc1b3330 con 0x7f63dc072cf0 2024-05-08T22:17:32.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.381+0000 7f63dbe00700 1 -- 172.21.15.131:0/949354649 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63bc005320 con 0x7f63dc072cf0 2024-05-08T22:17:32.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.386+0000 7f63e0a00700 1 -- 172.21.15.131:0/949354649 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f63d4004690 con 0x7f63dc072cf0 2024-05-08T22:17:32.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.388+0000 7f63e0a00700 1 --2- 172.21.15.131:0/949354649 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f63c00a13d0 0x7f63c00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:32.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.388+0000 7f63e0a00700 1 -- 172.21.15.131:0/949354649 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f63d40ccbc0 con 0x7f63dc072cf0 2024-05-08T22:17:32.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.390+0000 7f63da000700 1 --2- 172.21.15.131:0/949354649 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f63c00a13d0 0x7f63c00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:32.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.392+0000 7f63e0a00700 1 -- 172.21.15.131:0/949354649 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f63d4095b70 con 0x7f63dc072cf0 2024-05-08T22:17:32.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.393+0000 7f63da000700 1 --2- 172.21.15.131:0/949354649 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f63c00a13d0 0x7f63c00a3890 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f63c80098e0 tx=0x7f63c8005fb0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:32.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:32 smithi157 ceph-mon[77746]: pgmap v317: 129 pgs: 14 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 66 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:17:32.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:32 smithi157 ceph-mon[77746]: scrub queued for path: / 2024-05-08T22:17:32.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:32 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:17:32.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:32 smithi157 ceph-mon[77746]: scrub summary: active paths [/] 2024-05-08T22:17:32.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:32 smithi157 ceph-mon[77746]: scrub summary: idle+waiting paths [/] 2024-05-08T22:17:32.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:32 smithi157 ceph-mon[77746]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:17:32.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:32 smithi157 ceph-mon[77746]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:17:32.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:32 smithi132 ceph-mon[78180]: pgmap v317: 129 pgs: 14 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 66 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:17:32.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:32 smithi132 ceph-mon[78180]: scrub queued for path: / 2024-05-08T22:17:32.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:32 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:17:32.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:32 smithi132 ceph-mon[78180]: scrub summary: active paths [/] 2024-05-08T22:17:32.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:32 smithi132 ceph-mon[78180]: scrub summary: idle+waiting paths [/] 2024-05-08T22:17:32.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:32 smithi132 ceph-mon[78180]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:17:32.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:32 smithi132 ceph-mon[78180]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:17:32.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.853+0000 7f63dbe00700 1 -- 172.21.15.131:0/949354649 --> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f63bc000bf0 con 0x7f63c00a13d0 2024-05-08T22:17:32.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.876+0000 7f63e0a00700 1 -- 172.21.15.131:0/949354649 <== mgr.15366 v2:172.21.15.131:6800/1157588501 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+350 (secure 0 0 0) 0x7f63bc000bf0 con 0x7f63c00a13d0 2024-05-08T22:17:32.877 INFO:teuthology.orchestra.run.smithi131.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-08T00:00:00", "created": "2024-05-08T22:07:29", "first": "2024-05-08T22:08:00", "last": "2024-05-08T22:17:00", "last_pruned": "2024-05-08T22:17:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-05-08T22:17:32.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.880+0000 7f63dbe00700 1 -- 172.21.15.131:0/949354649 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f63c00a13d0 msgr2=0x7f63c00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:32.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.880+0000 7f63dbe00700 1 --2- 172.21.15.131:0/949354649 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f63c00a13d0 0x7f63c00a3890 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f63c80098e0 tx=0x7f63c8005fb0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.880+0000 7f63dbe00700 1 -- 172.21.15.131:0/949354649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63dc072cf0 msgr2=0x7f63dc112f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:32.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.880+0000 7f63dbe00700 1 --2- 172.21.15.131:0/949354649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63dc072cf0 0x7f63dc112f90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f63d4007f10 tx=0x7f63d400d3b0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.880+0000 7f63dbe00700 1 -- 172.21.15.131:0/949354649 shutdown_connections 2024-05-08T22:17:32.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.880+0000 7f63dbe00700 1 --2- 172.21.15.131:0/949354649 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f63c00a13d0 0x7f63c00a3890 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.880+0000 7f63dbe00700 1 --2- 172.21.15.131:0/949354649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63dc114c50 0x7f63dc113a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.880+0000 7f63dbe00700 1 --2- 172.21.15.131:0/949354649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63dc072cf0 0x7f63dc112f90 secure :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f63d4007f10 tx=0x7f63d400d3b0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.880+0000 7f63dbe00700 1 --2- 172.21.15.131:0/949354649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63dc112ae0 0x7f63dc1134d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:32.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.880+0000 7f63dbe00700 1 -- 172.21.15.131:0/949354649 >> 172.21.15.131:0/949354649 conn(0x7f63dc076a30 msgr2=0x7f63dc068280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:32.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.881+0000 7f63dbe00700 1 -- 172.21.15.131:0/949354649 shutdown_connections 2024-05-08T22:17:32.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:32.881+0000 7f63dbe00700 1 -- 172.21.15.131:0/949354649 wait complete. 2024-05-08T22:17:32.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:32 smithi131 ceph-mon[37260]: pgmap v317: 129 pgs: 14 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 66 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:17:32.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:32 smithi131 ceph-mon[37260]: scrub queued for path: / 2024-05-08T22:17:32.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:32 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:17:32.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:32 smithi131 ceph-mon[37260]: scrub summary: active paths [/] 2024-05-08T22:17:32.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:32 smithi131 ceph-mon[37260]: scrub summary: idle+waiting paths [/] 2024-05-08T22:17:32.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:32 smithi131 ceph-mon[37260]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:17:32.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:32 smithi131 ceph-mon[37260]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:17:32.924 DEBUG:teuthology.orchestra.run.smithi131:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-08T22:17:33.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.263+0000 7f3f5a600700 1 -- 172.21.15.131:0/252715355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f5410f630 msgr2=0x7f3f5410fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:33.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.263+0000 7f3f5a600700 1 --2- 172.21.15.131:0/252715355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f5410f630 0x7f3f5410fa10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009990 tx=0x7f3f44009ca0 comp rx=0 tx=0).stop 2024-05-08T22:17:33.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.264+0000 7f3f5a600700 1 -- 172.21.15.131:0/252715355 shutdown_connections 2024-05-08T22:17:33.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.264+0000 7f3f5a600700 1 --2- 172.21.15.131:0/252715355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f54107fa0 0x7f3f54108450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:33.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.264+0000 7f3f5a600700 1 --2- 172.21.15.131:0/252715355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f5410f630 0x7f3f5410fa10 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:33.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.264+0000 7f3f5a600700 1 --2- 172.21.15.131:0/252715355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f5410d440 0x7f3f5410d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:33.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.264+0000 7f3f5a600700 1 -- 172.21.15.131:0/252715355 >> 172.21.15.131:0/252715355 conn(0x7f3f5406bf80 msgr2=0x7f3f5406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:33.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.264+0000 7f3f5a600700 1 -- 172.21.15.131:0/252715355 shutdown_connections 2024-05-08T22:17:33.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.264+0000 7f3f5a600700 1 -- 172.21.15.131:0/252715355 wait complete. 2024-05-08T22:17:33.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.266+0000 7f3f5a600700 1 Processor -- start 2024-05-08T22:17:33.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.267+0000 7f3f5a600700 1 -- start start 2024-05-08T22:17:33.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.267+0000 7f3f5a600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f54107fa0 0x7f3f541a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:33.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.267+0000 7f3f5a600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f5410d440 0x7f3f541a6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:33.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.267+0000 7f3f5a600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f5410f630 0x7f3f541aabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:33.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.267+0000 7f3f5a600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f541ab210 con 0x7f3f5410d440 2024-05-08T22:17:33.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.267+0000 7f3f5a600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f541ab380 con 0x7f3f54107fa0 2024-05-08T22:17:33.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.267+0000 7f3f5a600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f541ab540 con 0x7f3f5410f630 2024-05-08T22:17:33.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.268+0000 7f3f53400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f54107fa0 0x7f3f541a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:33.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.268+0000 7f3f52a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f5410d440 0x7f3f541a6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:33.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.268+0000 7f3f52a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f5410d440 0x7f3f541a6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44076/0 (socket says 172.21.15.131:44076) 2024-05-08T22:17:33.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.268+0000 7f3f52a00700 1 -- 172.21.15.131:0/1465344014 learned_addr learned my addr 172.21.15.131:0/1465344014 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:17:33.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.268+0000 7f3f53e00700 1 --2- 172.21.15.131:0/1465344014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f5410f630 0x7f3f541aabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:33.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.269+0000 7f3f53400700 1 -- 172.21.15.131:0/1465344014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f5410f630 msgr2=0x7f3f541aabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:33.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.269+0000 7f3f53400700 1 --2- 172.21.15.131:0/1465344014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f5410f630 0x7f3f541aabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:33.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.269+0000 7f3f53400700 1 -- 172.21.15.131:0/1465344014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f5410d440 msgr2=0x7f3f541a6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:33.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.269+0000 7f3f53400700 1 --2- 172.21.15.131:0/1465344014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f5410d440 0x7f3f541a6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:33.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.269+0000 7f3f53400700 1 -- 172.21.15.131:0/1465344014 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f40009520 con 0x7f3f54107fa0 2024-05-08T22:17:33.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.269+0000 7f3f52a00700 1 --2- 172.21.15.131:0/1465344014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f5410d440 0x7f3f541a6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:17:33.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.270+0000 7f3f53400700 1 --2- 172.21.15.131:0/1465344014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f54107fa0 0x7f3f541a62f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3f44000c00 tx=0x7f3f44005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:33.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.270+0000 7f3f4be00700 1 -- 172.21.15.131:0/1465344014 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4401d070 con 0x7f3f54107fa0 2024-05-08T22:17:33.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.271+0000 7f3f4be00700 1 -- 172.21.15.131:0/1465344014 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f44021410 con 0x7f3f54107fa0 2024-05-08T22:17:33.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.271+0000 7f3f4be00700 1 -- 172.21.15.131:0/1465344014 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44017650 con 0x7f3f54107fa0 2024-05-08T22:17:33.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.271+0000 7f3f5a600700 1 -- 172.21.15.131:0/1465344014 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f440095f0 con 0x7f3f54107fa0 2024-05-08T22:17:33.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.271+0000 7f3f5a600700 1 -- 172.21.15.131:0/1465344014 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f541aba00 con 0x7f3f54107fa0 2024-05-08T22:17:33.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.273+0000 7f3f5a600700 1 -- 172.21.15.131:0/1465344014 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f54072860 con 0x7f3f54107fa0 2024-05-08T22:17:33.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.275+0000 7f3f4be00700 1 -- 172.21.15.131:0/1465344014 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f3f44021580 con 0x7f3f54107fa0 2024-05-08T22:17:33.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.277+0000 7f3f4be00700 1 --2- 172.21.15.131:0/1465344014 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3f380a1420 0x7f3f380a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:17:33.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.278+0000 7f3f4be00700 1 -- 172.21.15.131:0/1465344014 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f3f440cdd70 con 0x7f3f54107fa0 2024-05-08T22:17:33.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.278+0000 7f3f52a00700 1 --2- 172.21.15.131:0/1465344014 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3f380a1420 0x7f3f380a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:17:33.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.279+0000 7f3f52a00700 1 --2- 172.21.15.131:0/1465344014 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3f380a1420 0x7f3f380a38e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3f541a7840 tx=0x7f3f40009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:17:33.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.281+0000 7f3f4be00700 1 -- 172.21.15.131:0/1465344014 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3f44096db0 con 0x7f3f54107fa0 2024-05-08T22:17:33.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.731+0000 7f3f5a600700 1 -- 172.21.15.131:0/1465344014 --> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f3f54067ea0 con 0x7f3f380a1420 2024-05-08T22:17:33.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.752+0000 7f3f4be00700 1 -- 172.21.15.131:0/1465344014 <== mgr.15366 v2:172.21.15.131:6800/1157588501 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f3f54067ea0 con 0x7f3f380a1420 2024-05-08T22:17:33.754 INFO:teuthology.orchestra.run.smithi131.stdout:/ 1m 6m3h 2024-05-08T22:17:33.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.756+0000 7f3f5a600700 1 -- 172.21.15.131:0/1465344014 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3f380a1420 msgr2=0x7f3f380a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:33.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.756+0000 7f3f5a600700 1 --2- 172.21.15.131:0/1465344014 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3f380a1420 0x7f3f380a38e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3f541a7840 tx=0x7f3f40009260 comp rx=0 tx=0).stop 2024-05-08T22:17:33.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.756+0000 7f3f5a600700 1 -- 172.21.15.131:0/1465344014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f54107fa0 msgr2=0x7f3f541a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:17:33.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.756+0000 7f3f5a600700 1 --2- 172.21.15.131:0/1465344014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f54107fa0 0x7f3f541a62f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3f44000c00 tx=0x7f3f44005740 comp rx=0 tx=0).stop 2024-05-08T22:17:33.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.757+0000 7f3f5a600700 1 -- 172.21.15.131:0/1465344014 shutdown_connections 2024-05-08T22:17:33.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.757+0000 7f3f5a600700 1 --2- 172.21.15.131:0/1465344014 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3f380a1420 0x7f3f380a38e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:33.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.757+0000 7f3f5a600700 1 --2- 172.21.15.131:0/1465344014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f5410f630 0x7f3f541aabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:33.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.757+0000 7f3f5a600700 1 --2- 172.21.15.131:0/1465344014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f54107fa0 0x7f3f541a62f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:33.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.757+0000 7f3f5a600700 1 --2- 172.21.15.131:0/1465344014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f5410d440 0x7f3f541a6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:17:33.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.757+0000 7f3f5a600700 1 -- 172.21.15.131:0/1465344014 >> 172.21.15.131:0/1465344014 conn(0x7f3f5406bf80 msgr2=0x7f3f5406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:17:33.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.758+0000 7f3f5a600700 1 -- 172.21.15.131:0/1465344014 shutdown_connections 2024-05-08T22:17:33.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:17:33.758+0000 7f3f5a600700 1 -- 172.21.15.131:0/1465344014 wait complete. 2024-05-08T22:17:33.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:33 smithi157 ceph-mon[77746]: scrub summary: idle 2024-05-08T22:17:33.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:33 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:33.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:33 smithi157 ceph-mon[77746]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:33.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:33 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:33.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:33 smithi157 ceph-mon[77746]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:33.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:33 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:33.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:33 smithi157 ceph-mon[77746]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:33.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:33 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:33.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:33 smithi157 ceph-mon[77746]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:33.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:33 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:17:33.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:33 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:17:33.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:33 smithi157 ceph-mon[77746]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:17:33.789 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:33 smithi131 ceph-mon[37260]: scrub summary: idle 2024-05-08T22:17:33.789 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:33 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:33.789 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:33 smithi131 ceph-mon[37260]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:33.789 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:33 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:33.789 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:33 smithi131 ceph-mon[37260]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:33.789 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:33 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:33.789 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:33 smithi131 ceph-mon[37260]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:33.790 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:33 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:33.790 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:33 smithi131 ceph-mon[37260]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:33.790 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:33 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:17:33.790 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:33 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:17:33.790 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:33 smithi131 ceph-mon[37260]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:17:33.792 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-08T22:17:33.802 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-08T22:17:33.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:33 smithi132 ceph-mon[78180]: scrub summary: idle 2024-05-08T22:17:33.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:33 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:33.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:33 smithi132 ceph-mon[78180]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:33.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:33 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:33.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:33 smithi132 ceph-mon[78180]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:33.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:33 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:33.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:33 smithi132 ceph-mon[78180]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:33.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:33 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:33.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:33 smithi132 ceph-mon[78180]: from='mgr.15366 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:17:33.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:33 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:17:33.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:33 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:17:33.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:33 smithi132 ceph-mon[78180]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:17:34.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:34 smithi157 ceph-mon[77746]: from='client.26813 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:17:34.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:34 smithi157 ceph-mon[77746]: Adjusting osd_memory_target on smithi131 to 221.2M 2024-05-08T22:17:34.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:34 smithi157 ceph-mon[77746]: Unable to set osd_memory_target on smithi131 to 232010956: error parsing value: Value '232010956' is below minimum 939524096 2024-05-08T22:17:34.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:34 smithi157 ceph-mon[77746]: pgmap v318: 129 pgs: 10 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 85 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-08T22:17:34.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:34 smithi132 ceph-mon[78180]: from='client.26813 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:17:34.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:34 smithi132 ceph-mon[78180]: Adjusting osd_memory_target on smithi131 to 221.2M 2024-05-08T22:17:34.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:34 smithi132 ceph-mon[78180]: Unable to set osd_memory_target on smithi131 to 232010956: error parsing value: Value '232010956' is below minimum 939524096 2024-05-08T22:17:34.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:34 smithi132 ceph-mon[78180]: pgmap v318: 129 pgs: 10 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 85 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-08T22:17:34.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:34 smithi131 ceph-mon[37260]: from='client.26813 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:17:34.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:34 smithi131 ceph-mon[37260]: Adjusting osd_memory_target on smithi131 to 221.2M 2024-05-08T22:17:34.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:34 smithi131 ceph-mon[37260]: Unable to set osd_memory_target on smithi131 to 232010956: error parsing value: Value '232010956' is below minimum 939524096 2024-05-08T22:17:34.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:34 smithi131 ceph-mon[37260]: pgmap v318: 129 pgs: 10 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 85 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-08T22:17:35.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:35 smithi157 ceph-mon[77746]: from='client.26819 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:17:35.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:35 smithi132 ceph-mon[78180]: from='client.26819 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:17:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:35 smithi131 ceph-mon[37260]: from='client.26819 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:17:36.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:36 smithi157 ceph-mon[77746]: pgmap v319: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 9 op/s 2024-05-08T22:17:36.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:36 smithi132 ceph-mon[78180]: pgmap v319: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 9 op/s 2024-05-08T22:17:36.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:36 smithi131 ceph-mon[37260]: pgmap v319: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 9 op/s 2024-05-08T22:17:38.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:38 smithi157 ceph-mon[77746]: pgmap v320: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-08T22:17:38.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:38 smithi157 ceph-mon[77746]: osdmap e96: 12 total, 12 up, 12 in 2024-05-08T22:17:38.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:38 smithi132 ceph-mon[78180]: pgmap v320: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-08T22:17:38.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:38 smithi132 ceph-mon[78180]: osdmap e96: 12 total, 12 up, 12 in 2024-05-08T22:17:38.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:38 smithi131 ceph-mon[37260]: pgmap v320: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-08T22:17:38.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:38 smithi131 ceph-mon[37260]: osdmap e96: 12 total, 12 up, 12 in 2024-05-08T22:17:40.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:40 smithi157 ceph-mon[77746]: pgmap v322: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 614 B/s wr, 10 op/s 2024-05-08T22:17:40.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:40 smithi132 ceph-mon[78180]: pgmap v322: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 614 B/s wr, 10 op/s 2024-05-08T22:17:40.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:40 smithi131 ceph-mon[37260]: pgmap v322: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 614 B/s wr, 10 op/s 2024-05-08T22:17:42.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:42 smithi157 ceph-mon[77746]: pgmap v323: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 614 B/s wr, 10 op/s 2024-05-08T22:17:42.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:42 smithi132 ceph-mon[78180]: pgmap v323: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 614 B/s wr, 10 op/s 2024-05-08T22:17:42.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:42 smithi131 ceph-mon[37260]: pgmap v323: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 614 B/s wr, 10 op/s 2024-05-08T22:17:44.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:43 smithi157 ceph-mon[77746]: osdmap e97: 12 total, 12 up, 12 in 2024-05-08T22:17:44.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:43 smithi157 ceph-mon[77746]: pgmap v325: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T22:17:44.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:43 smithi132 ceph-mon[78180]: osdmap e97: 12 total, 12 up, 12 in 2024-05-08T22:17:44.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:43 smithi132 ceph-mon[78180]: pgmap v325: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T22:17:44.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:43 smithi131 ceph-mon[37260]: osdmap e97: 12 total, 12 up, 12 in 2024-05-08T22:17:44.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:43 smithi131 ceph-mon[37260]: pgmap v325: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T22:17:46.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:46 smithi157 ceph-mon[77746]: pgmap v326: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:17:46.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:46 smithi132 ceph-mon[78180]: pgmap v326: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:17:46.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:46 smithi131 ceph-mon[37260]: pgmap v326: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:17:48.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:48 smithi157 ceph-mon[77746]: pgmap v327: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-05-08T22:17:48.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:48 smithi132 ceph-mon[78180]: pgmap v327: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-05-08T22:17:48.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:48 smithi131 ceph-mon[37260]: pgmap v327: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-05-08T22:17:50.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:50 smithi157 ceph-mon[77746]: pgmap v328: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-08T22:17:50.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:50 smithi132 ceph-mon[78180]: pgmap v328: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-08T22:17:50.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:50 smithi131 ceph-mon[37260]: pgmap v328: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-08T22:17:52.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:52 smithi157 ceph-mon[77746]: pgmap v329: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-08T22:17:52.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:52 smithi132 ceph-mon[78180]: pgmap v329: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-08T22:17:52.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:52 smithi131 ceph-mon[37260]: pgmap v329: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-08T22:17:54.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:54 smithi157 ceph-mon[77746]: pgmap v330: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T22:17:54.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:54 smithi132 ceph-mon[78180]: pgmap v330: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T22:17:54.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:54 smithi131 ceph-mon[37260]: pgmap v330: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T22:17:56.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:56 smithi157 ceph-mon[77746]: pgmap v331: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:17:56.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:56 smithi132 ceph-mon[78180]: pgmap v331: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:17:56.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:56 smithi131 ceph-mon[37260]: pgmap v331: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:17:58.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:17:58 smithi157 ceph-mon[77746]: pgmap v332: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:17:58.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:17:58 smithi132 ceph-mon[78180]: pgmap v332: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:17:58.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:17:58 smithi131 ceph-mon[37260]: pgmap v332: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:18:00.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:00 smithi157 ceph-mon[77746]: pgmap v333: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:18:00.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:00 smithi132 ceph-mon[78180]: pgmap v333: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:18:00.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:00 smithi131 ceph-mon[37260]: pgmap v333: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:18:02.091 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T22:18:02.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.295+0000 7fc4a9000700 1 -- 172.21.15.131:0/3421340156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4a4101160 msgr2=0x7fc4a4101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:02.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.295+0000 7fc4a9000700 1 --2- 172.21.15.131:0/3421340156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4a4101160 0x7fc4a4101540 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc494009990 tx=0x7fc494009ca0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.296+0000 7fc4a9000700 1 -- 172.21.15.131:0/3421340156 shutdown_connections 2024-05-08T22:18:02.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.296+0000 7fc4a9000700 1 --2- 172.21.15.131:0/3421340156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4a4106280 0x7fc4a410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.296+0000 7fc4a9000700 1 --2- 172.21.15.131:0/3421340156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4a4101160 0x7fc4a4101540 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.296+0000 7fc4a9000700 1 --2- 172.21.15.131:0/3421340156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4a4101b10 0x7fc4a4105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.296+0000 7fc4a9000700 1 -- 172.21.15.131:0/3421340156 >> 172.21.15.131:0/3421340156 conn(0x7fc4a40fc9c0 msgr2=0x7fc4a40fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:02.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.296+0000 7fc4a9000700 1 -- 172.21.15.131:0/3421340156 shutdown_connections 2024-05-08T22:18:02.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.296+0000 7fc4a9000700 1 -- 172.21.15.131:0/3421340156 wait complete. 2024-05-08T22:18:02.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.297+0000 7fc4a9000700 1 Processor -- start 2024-05-08T22:18:02.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.297+0000 7fc4a9000700 1 -- start start 2024-05-08T22:18:02.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.297+0000 7fc4a9000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4a4101160 0x7fc4a41995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:02.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.297+0000 7fc4a9000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4a4106280 0x7fc4a4199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:02.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.297+0000 7fc4a9000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4a419ded0 0x7fc4a419e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:02.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.297+0000 7fc4a9000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a419e950 con 0x7fc4a4101160 2024-05-08T22:18:02.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.297+0000 7fc4a9000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a419eac0 con 0x7fc4a419ded0 2024-05-08T22:18:02.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.297+0000 7fc4a9000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a419ec80 con 0x7fc4a4106280 2024-05-08T22:18:02.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.297+0000 7fc4a1e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4a4101160 0x7fc4a41995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:02.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.298+0000 7fc4a1400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4a4106280 0x7fc4a4199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.298+0000 7fc4a1e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4a4101160 0x7fc4a41995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34882/0 (socket says 172.21.15.131:34882) 2024-05-08T22:18:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.298+0000 7fc4a1e00700 1 -- 172.21.15.131:0/3071050698 learned_addr learned my addr 172.21.15.131:0/3071050698 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.298+0000 7fc4a2800700 1 --2- 172.21.15.131:0/3071050698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4a419ded0 0x7fc4a419e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.298+0000 7fc4a1400700 1 -- 172.21.15.131:0/3071050698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4a419ded0 msgr2=0x7fc4a419e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.298+0000 7fc4a1400700 1 --2- 172.21.15.131:0/3071050698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4a419ded0 0x7fc4a419e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.298+0000 7fc4a1400700 1 -- 172.21.15.131:0/3071050698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4a4101160 msgr2=0x7fc4a41995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.298+0000 7fc4a1400700 1 --2- 172.21.15.131:0/3071050698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4a4101160 0x7fc4a41995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.298+0000 7fc4a1400700 1 -- 172.21.15.131:0/3071050698 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4940095f0 con 0x7fc4a4106280 2024-05-08T22:18:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.298+0000 7fc4a2800700 1 --2- 172.21.15.131:0/3071050698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4a419ded0 0x7fc4a419e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:18:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.298+0000 7fc4a1e00700 1 --2- 172.21.15.131:0/3071050698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4a4101160 0x7fc4a41995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:18:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.298+0000 7fc4a1400700 1 --2- 172.21.15.131:0/3071050698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4a4106280 0x7fc4a4199b30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc49000e9c0 tx=0x7fc49000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.298+0000 7fc49aa00700 1 -- 172.21.15.131:0/3071050698 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49000cbd0 con 0x7fc4a4106280 2024-05-08T22:18:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.299+0000 7fc49aa00700 1 -- 172.21.15.131:0/3071050698 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc49000eeb0 con 0x7fc4a4106280 2024-05-08T22:18:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.299+0000 7fc4a9000700 1 -- 172.21.15.131:0/3071050698 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a41a61e0 con 0x7fc4a4106280 2024-05-08T22:18:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.299+0000 7fc49aa00700 1 -- 172.21.15.131:0/3071050698 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc490005020 con 0x7fc4a4106280 2024-05-08T22:18:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.299+0000 7fc4a9000700 1 -- 172.21.15.131:0/3071050698 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a41a6670 con 0x7fc4a4106280 2024-05-08T22:18:02.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.302+0000 7fc49aa00700 1 -- 172.21.15.131:0/3071050698 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fc49000cd30 con 0x7fc4a4106280 2024-05-08T22:18:02.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.303+0000 7fc4bca77b80 1 Processor -- start 2024-05-08T22:18:02.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.303+0000 7fc4bca77b80 1 -- start start 2024-05-08T22:18:02.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.303+0000 7fc4bca77b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9db5f2e00 0x55a9db5f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:02.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.303+0000 7fc4bca77b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f3720 0x55a9db5ed350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:02.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.303+0000 7fc4bca77b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9db5ed890 0x55a9db5edcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:02.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.303+0000 7fc4bca77b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9db544150 con 0x55a9db5f3720 2024-05-08T22:18:02.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.303+0000 7fc4bca77b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9db541b10 con 0x55a9db5f2e00 2024-05-08T22:18:02.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.303+0000 7fc4bca77b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9db53d400 con 0x55a9db5ed890 2024-05-08T22:18:02.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.303+0000 7fc49aa00700 1 --2- 172.21.15.131:0/3071050698 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc4880a1290 0x7fc4880a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:02.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.303+0000 7fc49aa00700 1 -- 172.21.15.131:0/3071050698 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9509+0+0 (secure 0 0 0) 0x7fc490014070 con 0x7fc4a4106280 2024-05-08T22:18:02.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.303+0000 7fc4a1e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f3720 0x55a9db5ed350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:02.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.303+0000 7fc4a2800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9db5f2e00 0x55a9db5f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:02.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.303+0000 7fc4a1e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f3720 0x55a9db5ed350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34898/0 (socket says 172.21.15.131:34898) 2024-05-08T22:18:02.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.303+0000 7fc4a1e00700 1 -- 172.21.15.131:0/352311464 learned_addr learned my addr 172.21.15.131:0/352311464 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:02.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.304+0000 7fc4a2800700 1 --2- 172.21.15.131:0/3071050698 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc4880a1290 0x7fc4880a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:02.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.304+0000 7fc4a1400700 1 --2- 172.21.15.131:0/352311464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9db5ed890 0x55a9db5edcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:02.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.304+0000 7fc4a1e00700 1 -- 172.21.15.131:0/352311464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9db5ed890 msgr2=0x55a9db5edcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:02.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.304+0000 7fc4a1e00700 1 --2- 172.21.15.131:0/352311464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9db5ed890 0x55a9db5edcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.304+0000 7fc4a1e00700 1 -- 172.21.15.131:0/352311464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9db5f2e00 msgr2=0x55a9db5f31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:02.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.304+0000 7fc4a1e00700 1 --2- 172.21.15.131:0/352311464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9db5f2e00 0x55a9db5f31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.304+0000 7fc4a1e00700 1 -- 172.21.15.131:0/352311464 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9db3be3b0 con 0x55a9db5f3720 2024-05-08T22:18:02.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.304+0000 7fc4a2800700 1 --2- 172.21.15.131:0/3071050698 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc4880a1290 0x7fc4880a3750 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc49c012e90 tx=0x7fc49c00b040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:02.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.304+0000 7fc4a1e00700 1 --2- 172.21.15.131:0/352311464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f3720 0x55a9db5ed350 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc49400b630 tx=0x7fc494005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e4a3965bd0ac96 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:02.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.304+0000 7fc48f400700 1 -- 172.21.15.131:0/352311464 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49401d070 con 0x55a9db5f3720 2024-05-08T22:18:02.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.305+0000 7fc48f400700 1 -- 172.21.15.131:0/352311464 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc49401e040 con 0x55a9db5f3720 2024-05-08T22:18:02.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.305+0000 7fc48f400700 1 -- 172.21.15.131:0/352311464 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc494004b70 con 0x55a9db5f3720 2024-05-08T22:18:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.305+0000 7fc4bca77b80 1 -- 172.21.15.131:0/352311464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f3720 msgr2=0x55a9db5ed350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.305+0000 7fc4bca77b80 1 --2- 172.21.15.131:0/352311464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f3720 0x55a9db5ed350 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc49400b630 tx=0x7fc494005bc0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.305+0000 7fc4bca77b80 1 -- 172.21.15.131:0/352311464 shutdown_connections 2024-05-08T22:18:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.305+0000 7fc4bca77b80 1 --2- 172.21.15.131:0/352311464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9db5ed890 0x55a9db5edcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.305+0000 7fc4bca77b80 1 --2- 172.21.15.131:0/352311464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9db5f2e00 0x55a9db5f31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.305+0000 7fc4bca77b80 1 --2- 172.21.15.131:0/352311464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f3720 0x55a9db5ed350 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.305+0000 7fc4bca77b80 1 -- 172.21.15.131:0/352311464 >> 172.21.15.131:0/352311464 conn(0x55a9db54b210 msgr2=0x55a9db5e7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.305+0000 7fc4bca77b80 1 -- 172.21.15.131:0/352311464 shutdown_connections 2024-05-08T22:18:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.305+0000 7fc4bca77b80 1 -- 172.21.15.131:0/352311464 wait complete. 2024-05-08T22:18:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.306+0000 7fc4bca77b80 1 -- start start 2024-05-08T22:18:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.306+0000 7fc4bca77b80 1 Processor -- start 2024-05-08T22:18:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.307+0000 7fc4bca77b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9db5ed890 0x55a9db5f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.307+0000 7fc4bca77b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f2e00 0x55a9db5f7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.307+0000 7fc4bca77b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9db5f3720 0x55a9db5f7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.307+0000 7fc4bca77b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9db4d9560 con 0x55a9db5f2e00 2024-05-08T22:18:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.307+0000 7fc4bca77b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9db4d9200 con 0x55a9db5ed890 2024-05-08T22:18:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.307+0000 7fc4bca77b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9db3c06a0 con 0x55a9db5f3720 2024-05-08T22:18:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.307+0000 7fc4bca77b80 10 client.0 _ll_register_callbacks cb 0x55a9db4e2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:18:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.307+0000 7fc4a2800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9db5f3720 0x55a9db5f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.307+0000 7fc4a1400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f2e00 0x55a9db5f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:02.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.307+0000 7fc4a2800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9db5f3720 0x55a9db5f7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40528/0 (socket says 172.21.15.131:40528) 2024-05-08T22:18:02.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.307+0000 7fc4a2800700 1 -- 172.21.15.131:0/1101913650 learned_addr learned my addr 172.21.15.131:0/1101913650 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:02.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.307+0000 7fc4a1e00700 1 --2- 172.21.15.131:0/1101913650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9db5ed890 0x55a9db5f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:02.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.307+0000 7fc4a2800700 1 -- 172.21.15.131:0/1101913650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9db5ed890 msgr2=0x55a9db5f6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:02.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.307+0000 7fc4a2800700 1 --2- 172.21.15.131:0/1101913650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9db5ed890 0x55a9db5f6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.307+0000 7fc4a2800700 1 -- 172.21.15.131:0/1101913650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f2e00 msgr2=0x55a9db5f7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:02.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.307+0000 7fc4a2800700 1 --2- 172.21.15.131:0/1101913650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f2e00 0x55a9db5f7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.307+0000 7fc4a2800700 1 -- 172.21.15.131:0/1101913650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a410e670 con 0x55a9db5f3720 2024-05-08T22:18:02.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.307+0000 7fc4a1400700 1 --2- 172.21.15.131:0/1101913650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f2e00 0x55a9db5f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:18:02.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.308+0000 7fc4a2800700 1 --2- 172.21.15.131:0/1101913650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9db5f3720 0x55a9db5f7580 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0094b0 tx=0x7fc49c004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fae3f3bbaba17bd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:02.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.308+0000 7fc483e00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:18:02.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.308+0000 7fc483e00700 1 -- 172.21.15.131:0/1101913650 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c021070 con 0x55a9db5f3720 2024-05-08T22:18:02.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.308+0000 7fc4bca77b80 1 -- 172.21.15.131:0/1101913650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9db496270 con 0x55a9db5f3720 2024-05-08T22:18:02.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.308+0000 7fc483e00700 1 -- 172.21.15.131:0/1101913650 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9509+0+0 (secure 0 0 0) 0x7fc49c007e00 con 0x55a9db5f3720 2024-05-08T22:18:02.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.308+0000 7fc483e00700 1 -- 172.21.15.131:0/1101913650 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc49c00e040 con 0x55a9db5f3720 2024-05-08T22:18:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.308+0000 7fc4bca77b80 10 client.26932 fetch_fsmap learned FSMap version 30 2024-05-08T22:18:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.308+0000 7fc4bca77b80 1 -- 172.21.15.131:0/1101913650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a9db36ab60 con 0x55a9db5f3720 2024-05-08T22:18:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.308+0000 7fc483e00700 1 -- 172.21.15.131:0/1101913650 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a9db496270 con 0x55a9db5f3720 2024-05-08T22:18:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.309+0000 7fc483e00700 1 -- 172.21.15.131:0/1101913650 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc49c025410 con 0x55a9db5f3720 2024-05-08T22:18:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.309+0000 7fc4bca77b80 10 client.26932 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:18:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.309+0000 7fc4bca77b80 10 client.26932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:18:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.309+0000 7fc4bca77b80 20 client.26932 populate_metadata read hostname 'smithi131' 2024-05-08T22:18:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.309+0000 7fc4bca77b80 1 --2- 172.21.15.131:0/1101913650 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55a9db6071a0 0x55a9db6095c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.309+0000 7fc4bca77b80 4 client.26932 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:18:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.309+0000 7fc4bca77b80 1 -- 172.21.15.131:0/1101913650 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9db5bc810 con 0x55a9db6071a0 2024-05-08T22:18:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.309+0000 7fc4a1e00700 1 --2- 172.21.15.131:0/1101913650 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55a9db6071a0 0x55a9db6095c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:02.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.310+0000 7fc4a1e00700 1 --2- 172.21.15.131:0/1101913650 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55a9db6071a0 0x55a9db6095c0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ac32c845c336b60 server_cookie=ed377bdbf66f270e in_seq=0 out_seq=0 2024-05-08T22:18:02.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.310+0000 7fc483e00700 10 client.26932 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:18:02.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.310+0000 7fc483e00700 1 -- 172.21.15.131:0/1101913650 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a9db5bc810 con 0x55a9db6071a0 2024-05-08T22:18:02.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.310+0000 7fc483e00700 10 client.26932 handle_command_reply: tid=0 2024-05-08T22:18:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.310+0000 7fc4bca77b80 1 client.26932 shutdown 2024-05-08T22:18:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.311+0000 7fc4bca77b80 1 -- 172.21.15.131:0/1101913650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9db5f3720 msgr2=0x55a9db5f7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.311+0000 7fc4bca77b80 1 --2- 172.21.15.131:0/1101913650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9db5f3720 0x55a9db5f7580 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0094b0 tx=0x7fc49c004b40 comp rx=0 tx=0).stop 2024-05-08T22:18:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.311+0000 7fc4bca77b80 1 -- 172.21.15.131:0/1101913650 shutdown_connections 2024-05-08T22:18:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.311+0000 7fc4bca77b80 1 --2- 172.21.15.131:0/1101913650 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55a9db6071a0 0x55a9db6095c0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.311+0000 7fc4bca77b80 1 --2- 172.21.15.131:0/1101913650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9db5f3720 0x55a9db5f7580 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.311+0000 7fc483e00700 0 client.26932 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:18:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.311+0000 7fc4bca77b80 1 --2- 172.21.15.131:0/1101913650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9db5ed890 0x55a9db5f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.311+0000 7fc4bca77b80 1 --2- 172.21.15.131:0/1101913650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f2e00 0x55a9db5f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.311+0000 7fc4bca77b80 1 -- 172.21.15.131:0/1101913650 >> 172.21.15.131:0/1101913650 conn(0x55a9db54b210 msgr2=0x55a9db5e75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.311+0000 7fc4bca77b80 1 -- 172.21.15.131:0/1101913650 shutdown_connections 2024-05-08T22:18:02.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.311+0000 7fc4bca77b80 1 -- 172.21.15.131:0/1101913650 wait complete. 2024-05-08T22:18:02.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.311+0000 7fc4bca77b80 20 client.26932 trim_cache size 0 max 16384 2024-05-08T22:18:02.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.328+0000 7fc4bca77b80 1 Processor -- start 2024-05-08T22:18:02.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.328+0000 7fc4bca77b80 1 -- start start 2024-05-08T22:18:02.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.329+0000 7fc4bca77b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9db5ed890 0x55a9db5ed350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:02.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.329+0000 7fc4bca77b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9db5f2e00 0x55a9db5e4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:02.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.329+0000 7fc4bca77b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f3720 0x55a9db5e7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:02.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.329+0000 7fc4bca77b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9db5e4dc0 con 0x55a9db5f3720 2024-05-08T22:18:02.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.329+0000 7fc4bca77b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9db5e9e50 con 0x55a9db5f2e00 2024-05-08T22:18:02.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.329+0000 7fc4bca77b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9db5f3560 con 0x55a9db5ed890 2024-05-08T22:18:02.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.329+0000 7fc4a2800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f3720 0x55a9db5e7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:02.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.329+0000 7fc4a2800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f3720 0x55a9db5e7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34918/0 (socket says 172.21.15.131:34918) 2024-05-08T22:18:02.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.329+0000 7fc4a2800700 1 -- 172.21.15.131:0/2732756930 learned_addr learned my addr 172.21.15.131:0/2732756930 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:02.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.329+0000 7fc4a1e00700 1 --2- 172.21.15.131:0/2732756930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9db5ed890 0x55a9db5ed350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:02.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.329+0000 7fc4a1400700 1 --2- 172.21.15.131:0/2732756930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9db5f2e00 0x55a9db5e4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:02.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.329+0000 7fc4a2800700 1 -- 172.21.15.131:0/2732756930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9db5ed890 msgr2=0x55a9db5ed350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:02.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.329+0000 7fc4a2800700 1 --2- 172.21.15.131:0/2732756930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9db5ed890 0x55a9db5ed350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.329+0000 7fc4a2800700 1 -- 172.21.15.131:0/2732756930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9db5f2e00 msgr2=0x55a9db5e4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:02.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.329+0000 7fc4a2800700 1 --2- 172.21.15.131:0/2732756930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9db5f2e00 0x55a9db5e4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.329+0000 7fc4a2800700 1 -- 172.21.15.131:0/2732756930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9db503560 con 0x55a9db5f3720 2024-05-08T22:18:02.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.330+0000 7fc4a2800700 1 --2- 172.21.15.131:0/2732756930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f3720 0x55a9db5e7d20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc49c012fa0 tx=0x7fc49c012fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72182fb15ba1d2f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:02.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.330+0000 7fc48fe00700 1 -- 172.21.15.131:0/2732756930 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c021070 con 0x55a9db5f3720 2024-05-08T22:18:02.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.330+0000 7fc48fe00700 1 -- 172.21.15.131:0/2732756930 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc49c007e00 con 0x55a9db5f3720 2024-05-08T22:18:02.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.330+0000 7fc48fe00700 1 -- 172.21.15.131:0/2732756930 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c00e040 con 0x55a9db5f3720 2024-05-08T22:18:02.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.331+0000 7fc4bca77b80 1 -- 172.21.15.131:0/2732756930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f3720 msgr2=0x55a9db5e7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:02.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.331+0000 7fc4bca77b80 1 --2- 172.21.15.131:0/2732756930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f3720 0x55a9db5e7d20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc49c012fa0 tx=0x7fc49c012fd0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.331+0000 7fc4bca77b80 1 -- 172.21.15.131:0/2732756930 shutdown_connections 2024-05-08T22:18:02.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.331+0000 7fc4bca77b80 1 --2- 172.21.15.131:0/2732756930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f3720 0x55a9db5e7d20 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.331+0000 7fc4bca77b80 1 --2- 172.21.15.131:0/2732756930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9db5ed890 0x55a9db5ed350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.331+0000 7fc4bca77b80 1 --2- 172.21.15.131:0/2732756930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9db5f2e00 0x55a9db5e4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.331+0000 7fc4bca77b80 1 -- 172.21.15.131:0/2732756930 >> 172.21.15.131:0/2732756930 conn(0x55a9db54b210 msgr2=0x55a9db5f6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:02.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.331+0000 7fc4bca77b80 1 -- 172.21.15.131:0/2732756930 shutdown_connections 2024-05-08T22:18:02.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.331+0000 7fc4bca77b80 1 -- 172.21.15.131:0/2732756930 wait complete. 2024-05-08T22:18:02.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.331+0000 7fc4bca77b80 1 -- start start 2024-05-08T22:18:02.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.331+0000 7fc4bca77b80 1 Processor -- start 2024-05-08T22:18:02.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.332+0000 7fc4bca77b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9db5ed890 0x55a9db5fe510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:02.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.332+0000 7fc4bca77b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f2e00 0x55a9db5fa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:02.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.332+0000 7fc4bca77b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9db5f3720 0x55a9db5fa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:02.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.332+0000 7fc4bca77b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9db5fad70 con 0x55a9db5f2e00 2024-05-08T22:18:02.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.332+0000 7fc4bca77b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9db5faee0 con 0x55a9db5ed890 2024-05-08T22:18:02.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.332+0000 7fc4bca77b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9db65c690 con 0x55a9db5f3720 2024-05-08T22:18:02.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.332+0000 7fc4bca77b80 10 client.0 _ll_register_callbacks cb 0x55a9db4e2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:18:02.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.332+0000 7fc4a1400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f2e00 0x55a9db5fa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:02.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.332+0000 7fc4a1400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f2e00 0x55a9db5fa0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34926/0 (socket says 172.21.15.131:34926) 2024-05-08T22:18:02.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.332+0000 7fc4a1400700 1 -- 172.21.15.131:0/3330615229 learned_addr learned my addr 172.21.15.131:0/3330615229 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:02.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.332+0000 7fc4a1e00700 1 --2- 172.21.15.131:0/3330615229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9db5ed890 0x55a9db5fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:02.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.332+0000 7fc4a2800700 1 --2- 172.21.15.131:0/3330615229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9db5f3720 0x55a9db5fa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:02.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.332+0000 7fc4a1400700 1 -- 172.21.15.131:0/3330615229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9db5f3720 msgr2=0x55a9db5fa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:02.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.332+0000 7fc4a1400700 1 --2- 172.21.15.131:0/3330615229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9db5f3720 0x55a9db5fa660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.332+0000 7fc4a1400700 1 -- 172.21.15.131:0/3330615229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9db5ed890 msgr2=0x55a9db5fe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:02.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.332+0000 7fc4a1400700 1 --2- 172.21.15.131:0/3330615229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9db5ed890 0x55a9db5fe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.333+0000 7fc4a1400700 1 -- 172.21.15.131:0/3330615229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9db36ab60 con 0x55a9db5f2e00 2024-05-08T22:18:02.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.333+0000 7fc4a1400700 1 --2- 172.21.15.131:0/3330615229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f2e00 0x55a9db5fa0e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc490099ab0 tx=0x7fc490005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2304fe66ada0a459 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:02.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.333+0000 7fc483e00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:18:02.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.333+0000 7fc483e00700 1 -- 172.21.15.131:0/3330615229 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4900b1070 con 0x55a9db5f2e00 2024-05-08T22:18:02.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.333+0000 7fc483e00700 1 -- 172.21.15.131:0/3330615229 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9509+0+0 (secure 0 0 0) 0x7fc490099db0 con 0x55a9db5f2e00 2024-05-08T22:18:02.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.333+0000 7fc4bca77b80 1 -- 172.21.15.131:0/3330615229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc49401e040 con 0x55a9db5f2e00 2024-05-08T22:18:02.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.333+0000 7fc483e00700 1 -- 172.21.15.131:0/3330615229 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4900b4ea0 con 0x55a9db5f2e00 2024-05-08T22:18:02.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.333+0000 7fc4bca77b80 10 client.16743 fetch_fsmap learned FSMap version 30 2024-05-08T22:18:02.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.333+0000 7fc4bca77b80 1 -- 172.21.15.131:0/3330615229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a9db3c2cd0 con 0x55a9db5f2e00 2024-05-08T22:18:02.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.333+0000 7fc483e00700 1 -- 172.21.15.131:0/3330615229 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc490078e90 con 0x55a9db5f2e00 2024-05-08T22:18:02.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.334+0000 7fc483e00700 1 -- 172.21.15.131:0/3330615229 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc490095ba0 con 0x55a9db5f2e00 2024-05-08T22:18:02.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.334+0000 7fc4bca77b80 10 client.16743 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:18:02.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.334+0000 7fc4bca77b80 10 client.16743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:18:02.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.334+0000 7fc4bca77b80 20 client.16743 populate_metadata read hostname 'smithi131' 2024-05-08T22:18:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.334+0000 7fc4bca77b80 1 --2- 172.21.15.131:0/3330615229 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55a9db65de20 0x55a9db660240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.334+0000 7fc4bca77b80 4 client.16743 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:18:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.334+0000 7fc4bca77b80 1 -- 172.21.15.131:0/3330615229 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a9db535ec0 con 0x55a9db65de20 2024-05-08T22:18:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.334+0000 7fc4a1e00700 1 --2- 172.21.15.131:0/3330615229 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55a9db65de20 0x55a9db660240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.334+0000 7fc4a1e00700 1 --2- 172.21.15.131:0/3330615229 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55a9db65de20 0x55a9db660240 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dd3c9e6bf8a547e server_cookie=ecb406b08b7a0574 in_seq=0 out_seq=0 2024-05-08T22:18:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.334+0000 7fc483e00700 10 client.16743 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:18:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.335+0000 7fc483e00700 1 -- 172.21.15.131:0/3330615229 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a9db535ec0 con 0x55a9db65de20 2024-05-08T22:18:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.335+0000 7fc483e00700 10 client.16743 handle_command_reply: tid=0 2024-05-08T22:18:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.335+0000 7fc4bca77b80 1 client.16743 shutdown 2024-05-08T22:18:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.335+0000 7fc4bca77b80 1 -- 172.21.15.131:0/3330615229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f2e00 msgr2=0x55a9db5fa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.335+0000 7fc4bca77b80 1 --2- 172.21.15.131:0/3330615229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f2e00 0x55a9db5fa0e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc490099ab0 tx=0x7fc490005bd0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.335+0000 7fc4bca77b80 1 -- 172.21.15.131:0/3330615229 shutdown_connections 2024-05-08T22:18:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.335+0000 7fc4bca77b80 1 --2- 172.21.15.131:0/3330615229 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55a9db65de20 0x55a9db660240 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.335+0000 7fc4bca77b80 1 --2- 172.21.15.131:0/3330615229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9db5f3720 0x55a9db5fa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.335+0000 7fc4bca77b80 1 --2- 172.21.15.131:0/3330615229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9db5ed890 0x55a9db5fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.335+0000 7fc483e00700 0 client.16743 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:18:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.335+0000 7fc4bca77b80 1 --2- 172.21.15.131:0/3330615229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a9db5f2e00 0x55a9db5fa0e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.335+0000 7fc4bca77b80 1 -- 172.21.15.131:0/3330615229 >> 172.21.15.131:0/3330615229 conn(0x55a9db54b210 msgr2=0x55a9db5f73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.335+0000 7fc4bca77b80 1 -- 172.21.15.131:0/3330615229 shutdown_connections 2024-05-08T22:18:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.335+0000 7fc4bca77b80 1 -- 172.21.15.131:0/3330615229 wait complete. 2024-05-08T22:18:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.335+0000 7fc4bca77b80 20 client.16743 trim_cache size 0 max 16384 2024-05-08T22:18:02.341 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:18:02.341 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-08T22:18:02.341 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-08T22:18:02.341 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:18:02.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.336+0000 7fc4a9000700 1 -- 172.21.15.131:0/3071050698 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc4880a1290 msgr2=0x7fc4880a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:02.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.336+0000 7fc4a9000700 1 --2- 172.21.15.131:0/3071050698 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc4880a1290 0x7fc4880a3750 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc49c012e90 tx=0x7fc49c00b040 comp rx=0 tx=0).stop 2024-05-08T22:18:02.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.336+0000 7fc4a9000700 1 -- 172.21.15.131:0/3071050698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4a4106280 msgr2=0x7fc4a4199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:02.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.336+0000 7fc4a9000700 1 --2- 172.21.15.131:0/3071050698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4a4106280 0x7fc4a4199b30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc49000e9c0 tx=0x7fc49000ecd0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.336+0000 7fc4a9000700 1 -- 172.21.15.131:0/3071050698 shutdown_connections 2024-05-08T22:18:02.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.336+0000 7fc4a9000700 1 --2- 172.21.15.131:0/3071050698 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fc4880a1290 0x7fc4880a3750 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.336+0000 7fc4a9000700 1 --2- 172.21.15.131:0/3071050698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4a419ded0 0x7fc4a419e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.336+0000 7fc4a9000700 1 --2- 172.21.15.131:0/3071050698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4a4101160 0x7fc4a41995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.336+0000 7fc4a9000700 1 --2- 172.21.15.131:0/3071050698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4a4106280 0x7fc4a4199b30 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.336+0000 7fc4a9000700 1 -- 172.21.15.131:0/3071050698 >> 172.21.15.131:0/3071050698 conn(0x7fc4a40fc9c0 msgr2=0x7fc4a40fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:02.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.336+0000 7fc4a9000700 1 -- 172.21.15.131:0/3071050698 shutdown_connections 2024-05-08T22:18:02.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.336+0000 7fc4a9000700 1 -- 172.21.15.131:0/3071050698 wait complete. 2024-05-08T22:18:02.356 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T22:18:02.356 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:18:02.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.552+0000 7f18dd000700 1 -- 172.21.15.131:0/3948836564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18d8111dc0 msgr2=0x7f18d81141b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:02.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.552+0000 7f18dd000700 1 --2- 172.21.15.131:0/3948836564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18d8111dc0 0x7f18d81141b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f18d000b0a0 tx=0x7f18d000b3b0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.552+0000 7f18dd000700 1 -- 172.21.15.131:0/3948836564 shutdown_connections 2024-05-08T22:18:02.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.552+0000 7f18dd000700 1 --2- 172.21.15.131:0/3948836564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18d8111dc0 0x7f18d81141b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.552+0000 7f18dd000700 1 --2- 172.21.15.131:0/3948836564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18d8073340 0x7f18d8073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.552+0000 7f18dd000700 1 --2- 172.21.15.131:0/3948836564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d8073c60 0x7f18d8111880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.552+0000 7f18dd000700 1 -- 172.21.15.131:0/3948836564 >> 172.21.15.131:0/3948836564 conn(0x7f18d80fc8c0 msgr2=0x7f18d80fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:02.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.552+0000 7f18dd000700 1 -- 172.21.15.131:0/3948836564 shutdown_connections 2024-05-08T22:18:02.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.552+0000 7f18dd000700 1 -- 172.21.15.131:0/3948836564 wait complete. 2024-05-08T22:18:02.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.553+0000 7f18dd000700 1 Processor -- start 2024-05-08T22:18:02.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.553+0000 7f18dd000700 1 -- start start 2024-05-08T22:18:02.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.554+0000 7f18dd000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d8073340 0x7f18d819da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:02.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.554+0000 7f18dd000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18d8073c60 0x7f18d819dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:02.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.554+0000 7f18dd000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18d8111dc0 0x7f18d81a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:02.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.554+0000 7f18dd000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d81a2940 con 0x7f18d8111dc0 2024-05-08T22:18:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.554+0000 7f18dd000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d81a2ab0 con 0x7f18d8073c60 2024-05-08T22:18:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.554+0000 7f18dd000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d81a2c70 con 0x7f18d8073340 2024-05-08T22:18:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.554+0000 7f18d7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18d8111dc0 0x7f18d81a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.554+0000 7f18d7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d8073340 0x7f18d819da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.554+0000 7f18d6a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18d8073c60 0x7f18d819dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.554+0000 7f18d7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18d8111dc0 0x7f18d81a2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34940/0 (socket says 172.21.15.131:34940) 2024-05-08T22:18:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.554+0000 7f18d7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d8073340 0x7f18d819da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40552/0 (socket says 172.21.15.131:40552) 2024-05-08T22:18:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.554+0000 7f18d7e00700 1 -- 172.21.15.131:0/4085948671 learned_addr learned my addr 172.21.15.131:0/4085948671 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.555+0000 7f18d7400700 1 -- 172.21.15.131:0/4085948671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18d8073c60 msgr2=0x7f18d819dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.555+0000 7f18d7400700 1 --2- 172.21.15.131:0/4085948671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18d8073c60 0x7f18d819dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.555+0000 7f18d7400700 1 -- 172.21.15.131:0/4085948671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18d8111dc0 msgr2=0x7f18d81a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.555+0000 7f18d7400700 1 --2- 172.21.15.131:0/4085948671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18d8111dc0 0x7f18d81a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.555+0000 7f18d7400700 1 -- 172.21.15.131:0/4085948671 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d0009d00 con 0x7f18d8073340 2024-05-08T22:18:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.555+0000 7f18d7e00700 1 --2- 172.21.15.131:0/4085948671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18d8111dc0 0x7f18d81a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:18:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.555+0000 7f18d7400700 1 --2- 172.21.15.131:0/4085948671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d8073340 0x7f18d819da60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f18c800e9f0 tx=0x7f18c800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:02.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.555+0000 7f18cfe00700 1 -- 172.21.15.131:0/4085948671 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c800c9c0 con 0x7f18d8073340 2024-05-08T22:18:02.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.555+0000 7f18dd000700 1 -- 172.21.15.131:0/4085948671 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d81a2f50 con 0x7f18d8073340 2024-05-08T22:18:02.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.555+0000 7f18cfe00700 1 -- 172.21.15.131:0/4085948671 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18c8010430 con 0x7f18d8073340 2024-05-08T22:18:02.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.555+0000 7f18cfe00700 1 -- 172.21.15.131:0/4085948671 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c8018680 con 0x7f18d8073340 2024-05-08T22:18:02.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.555+0000 7f18dd000700 1 -- 172.21.15.131:0/4085948671 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d81aa800 con 0x7f18d8073340 2024-05-08T22:18:02.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.556+0000 7f18dd000700 1 -- 172.21.15.131:0/4085948671 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d81042c0 con 0x7f18d8073340 2024-05-08T22:18:02.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.558+0000 7f18cfe00700 1 -- 172.21.15.131:0/4085948671 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f18c8010860 con 0x7f18d8073340 2024-05-08T22:18:02.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.559+0000 7f18cfe00700 1 --2- 172.21.15.131:0/4085948671 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f18bc0a1290 0x7f18bc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:02.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.560+0000 7f18cfe00700 1 -- 172.21.15.131:0/4085948671 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9509+0+0 (secure 0 0 0) 0x7f18c8014070 con 0x7f18d8073340 2024-05-08T22:18:02.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.560+0000 7f18d6a00700 1 --2- 172.21.15.131:0/4085948671 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f18bc0a1290 0x7f18bc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:02.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.560+0000 7f18d6a00700 1 --2- 172.21.15.131:0/4085948671 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f18bc0a1290 0x7f18bc0a3750 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f18d819efb0 tx=0x7f18c4009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:02.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.562+0000 7f18cfe00700 1 -- 172.21.15.131:0/4085948671 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f18c8096230 con 0x7f18d8073340 2024-05-08T22:18:02.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:02 smithi157 ceph-mon[77746]: pgmap v334: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:18:02.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.816+0000 7f18dd000700 1 -- 172.21.15.131:0/4085948671 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18d8074a80 con 0x7f18d8073340 2024-05-08T22:18:02.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.818+0000 7f18cfe00700 1 -- 172.21.15.131:0/4085948671 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10891 (secure 0 0 0) 0x7f18c8099850 con 0x7f18d8073340 2024-05-08T22:18:02.819 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:18:02.819 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:05:14.034908+0000","modified":"2024-05-08T22:16:38.400949+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:18:02.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.820+0000 7f18dd000700 1 -- 172.21.15.131:0/4085948671 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f18bc0a1290 msgr2=0x7f18bc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:02.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.820+0000 7f18dd000700 1 --2- 172.21.15.131:0/4085948671 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f18bc0a1290 0x7f18bc0a3750 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f18d819efb0 tx=0x7f18c4009260 comp rx=0 tx=0).stop 2024-05-08T22:18:02.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.820+0000 7f18dd000700 1 -- 172.21.15.131:0/4085948671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d8073340 msgr2=0x7f18d819da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:02.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.820+0000 7f18dd000700 1 --2- 172.21.15.131:0/4085948671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d8073340 0x7f18d819da60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f18c800e9f0 tx=0x7f18c800ed00 comp rx=0 tx=0).stop 2024-05-08T22:18:02.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.820+0000 7f18dd000700 1 -- 172.21.15.131:0/4085948671 shutdown_connections 2024-05-08T22:18:02.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.820+0000 7f18dd000700 1 --2- 172.21.15.131:0/4085948671 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f18bc0a1290 0x7f18bc0a3750 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.820+0000 7f18dd000700 1 --2- 172.21.15.131:0/4085948671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18d8111dc0 0x7f18d81a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.820+0000 7f18dd000700 1 --2- 172.21.15.131:0/4085948671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d8073340 0x7f18d819da60 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.820+0000 7f18dd000700 1 --2- 172.21.15.131:0/4085948671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18d8073c60 0x7f18d819dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:02.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.820+0000 7f18dd000700 1 -- 172.21.15.131:0/4085948671 >> 172.21.15.131:0/4085948671 conn(0x7f18d80fc8c0 msgr2=0x7f18d80fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:02.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.820+0000 7f18dd000700 1 -- 172.21.15.131:0/4085948671 shutdown_connections 2024-05-08T22:18:02.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:02.820+0000 7f18dd000700 1 -- 172.21.15.131:0/4085948671 wait complete. 2024-05-08T22:18:02.823 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 30 2024-05-08T22:18:02.838 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:02 smithi131 ceph-mon[37260]: pgmap v334: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:18:02.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:02 smithi132 ceph-mon[78180]: pgmap v334: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:18:02.841 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T22:18:03.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.036+0000 7f3a93a00700 1 -- 172.21.15.131:0/3446319139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a8c101160 msgr2=0x7f3a8c101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.036+0000 7f3a93a00700 1 --2- 172.21.15.131:0/3446319139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a8c101160 0x7f3a8c101540 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3a80009990 tx=0x7f3a80009ca0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.036+0000 7f3a93a00700 1 -- 172.21.15.131:0/3446319139 shutdown_connections 2024-05-08T22:18:03.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.036+0000 7f3a93a00700 1 --2- 172.21.15.131:0/3446319139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a8c106280 0x7f3a8c10de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.036+0000 7f3a93a00700 1 --2- 172.21.15.131:0/3446319139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a8c101160 0x7f3a8c101540 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.036+0000 7f3a93a00700 1 --2- 172.21.15.131:0/3446319139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a8c101b10 0x7f3a8c105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.036+0000 7f3a93a00700 1 -- 172.21.15.131:0/3446319139 >> 172.21.15.131:0/3446319139 conn(0x7f3a8c0fc9c0 msgr2=0x7f3a8c0fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:03.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.036+0000 7f3a93a00700 1 -- 172.21.15.131:0/3446319139 shutdown_connections 2024-05-08T22:18:03.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.036+0000 7f3a93a00700 1 -- 172.21.15.131:0/3446319139 wait complete. 2024-05-08T22:18:03.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.037+0000 7f3a93a00700 1 Processor -- start 2024-05-08T22:18:03.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.037+0000 7f3a93a00700 1 -- start start 2024-05-08T22:18:03.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.038+0000 7f3a93a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a8c101160 0x7f3a8c197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.038+0000 7f3a93a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a8c106280 0x7f3a8c197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.038+0000 7f3a93a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a8c19bd30 0x7f3a8c19c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.038+0000 7f3a93a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c19c7b0 con 0x7f3a8c106280 2024-05-08T22:18:03.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.038+0000 7f3a93a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c19c920 con 0x7f3a8c19bd30 2024-05-08T22:18:03.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.038+0000 7f3a93a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c19cae0 con 0x7f3a8c101160 2024-05-08T22:18:03.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.038+0000 7f3a8be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a8c106280 0x7f3a8c197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.038+0000 7f3a8be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a8c106280 0x7f3a8c197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34958/0 (socket says 172.21.15.131:34958) 2024-05-08T22:18:03.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.038+0000 7f3a91000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a8c101160 0x7f3a8c197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.038+0000 7f3a8be00700 1 -- 172.21.15.131:0/3572505958 learned_addr learned my addr 172.21.15.131:0/3572505958 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:03.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.038+0000 7f3a91a00700 1 --2- 172.21.15.131:0/3572505958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a8c19bd30 0x7f3a8c19c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.038+0000 7f3a8be00700 1 -- 172.21.15.131:0/3572505958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a8c101160 msgr2=0x7f3a8c197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.038+0000 7f3a8be00700 1 --2- 172.21.15.131:0/3572505958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a8c101160 0x7f3a8c197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.038+0000 7f3a8be00700 1 -- 172.21.15.131:0/3572505958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a8c19bd30 msgr2=0x7f3a8c19c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.038+0000 7f3a8be00700 1 --2- 172.21.15.131:0/3572505958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a8c19bd30 0x7f3a8c19c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.038+0000 7f3a8be00700 1 -- 172.21.15.131:0/3572505958 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a800095f0 con 0x7f3a8c106280 2024-05-08T22:18:03.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.039+0000 7f3a8be00700 1 --2- 172.21.15.131:0/3572505958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a8c106280 0x7f3a8c197990 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3a7800e970 tx=0x7f3a7800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:03.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.039+0000 7f3a89600700 1 -- 172.21.15.131:0/3572505958 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a78009790 con 0x7f3a8c106280 2024-05-08T22:18:03.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.039+0000 7f3a89600700 1 -- 172.21.15.131:0/3572505958 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a7800c7e0 con 0x7f3a8c106280 2024-05-08T22:18:03.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.039+0000 7f3a93a00700 1 -- 172.21.15.131:0/3572505958 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a8c1a3fe0 con 0x7f3a8c106280 2024-05-08T22:18:03.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.039+0000 7f3a89600700 1 -- 172.21.15.131:0/3572505958 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a780054e0 con 0x7f3a8c106280 2024-05-08T22:18:03.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.039+0000 7f3a93a00700 1 -- 172.21.15.131:0/3572505958 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a8c1a44c0 con 0x7f3a8c106280 2024-05-08T22:18:03.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.042+0000 7f3a89600700 1 -- 172.21.15.131:0/3572505958 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f3a7800c950 con 0x7f3a8c106280 2024-05-08T22:18:03.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.043+0000 7f3aa7573b80 1 Processor -- start 2024-05-08T22:18:03.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.043+0000 7f3a89600700 1 --2- 172.21.15.131:0/3572505958 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3a740a1290 0x7f3a740a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.044+0000 7f3aa7573b80 1 -- start start 2024-05-08T22:18:03.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.044+0000 7f3a89600700 1 -- 172.21.15.131:0/3572505958 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9509+0+0 (secure 0 0 0) 0x7f3a78014070 con 0x7f3a8c106280 2024-05-08T22:18:03.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.044+0000 7f3a91a00700 1 --2- 172.21.15.131:0/3572505958 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3a740a1290 0x7f3a740a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.044+0000 7f3aa7573b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d24c9411f0 0x55d24c9415d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.044+0000 7f3aa7573b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d24c941b10 0x55d24c93b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.044+0000 7f3aa7573b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c93bca0 0x55d24c93c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.044+0000 7f3aa7573b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24c88e150 con 0x55d24c93bca0 2024-05-08T22:18:03.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.044+0000 7f3aa7573b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24c88bb10 con 0x55d24c941b10 2024-05-08T22:18:03.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.044+0000 7f3aa7573b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24c887400 con 0x55d24c9411f0 2024-05-08T22:18:03.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.044+0000 7f3a91000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d24c9411f0 0x55d24c9415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.044+0000 7f3a91a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c93bca0 0x55d24c93c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.044+0000 7f3a8be00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d24c941b10 0x55d24c93b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.044+0000 7f3a91000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d24c9411f0 0x55d24c9415d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40574/0 (socket says 172.21.15.131:40574) 2024-05-08T22:18:03.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.044+0000 7f3a91000700 1 -- 172.21.15.131:0/60451285 learned_addr learned my addr 172.21.15.131:0/60451285 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:03.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.045+0000 7f3a91000700 1 -- 172.21.15.131:0/60451285 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d24c941b10 msgr2=0x55d24c93b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:03.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.045+0000 7f3a91000700 1 --2- 172.21.15.131:0/60451285 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d24c941b10 0x55d24c93b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.045+0000 7f3a91000700 1 -- 172.21.15.131:0/60451285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c93bca0 msgr2=0x55d24c93c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:03.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.045+0000 7f3a91000700 1 --2- 172.21.15.131:0/60451285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c93bca0 0x55d24c93c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.045+0000 7f3a91000700 1 -- 172.21.15.131:0/60451285 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d24c7083b0 con 0x55d24c9411f0 2024-05-08T22:18:03.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.045+0000 7f3a91a00700 1 --2- 172.21.15.131:0/3572505958 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3a740a1290 0x7f3a740a3750 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400f870 tx=0x7f3a8400fc30 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:03.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.045+0000 7f3a91000700 1 --2- 172.21.15.131:0/60451285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d24c9411f0 0x55d24c9415d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3a80005ec0 tx=0x7f3a80005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32ae5da3e052b4b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:03.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.045+0000 7f3a7e000700 1 -- 172.21.15.131:0/60451285 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8001d070 con 0x55d24c9411f0 2024-05-08T22:18:03.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.045+0000 7f3a7e000700 1 -- 172.21.15.131:0/60451285 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a8001e040 con 0x55d24c9411f0 2024-05-08T22:18:03.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.045+0000 7f3a7e000700 1 -- 172.21.15.131:0/60451285 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a800047f0 con 0x55d24c9411f0 2024-05-08T22:18:03.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.046+0000 7f3aa7573b80 1 -- 172.21.15.131:0/60451285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d24c9411f0 msgr2=0x55d24c9415d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.046+0000 7f3aa7573b80 1 --2- 172.21.15.131:0/60451285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d24c9411f0 0x55d24c9415d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3a80005ec0 tx=0x7f3a80005ef0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.046+0000 7f3aa7573b80 1 -- 172.21.15.131:0/60451285 shutdown_connections 2024-05-08T22:18:03.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.046+0000 7f3aa7573b80 1 --2- 172.21.15.131:0/60451285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c93bca0 0x55d24c93c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.046+0000 7f3aa7573b80 1 --2- 172.21.15.131:0/60451285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d24c9411f0 0x55d24c9415d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.046+0000 7f3aa7573b80 1 --2- 172.21.15.131:0/60451285 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d24c941b10 0x55d24c93b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.046+0000 7f3aa7573b80 1 -- 172.21.15.131:0/60451285 >> 172.21.15.131:0/60451285 conn(0x55d24c895210 msgr2=0x55d24c931df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:03.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.046+0000 7f3aa7573b80 1 -- 172.21.15.131:0/60451285 shutdown_connections 2024-05-08T22:18:03.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.046+0000 7f3aa7573b80 1 -- 172.21.15.131:0/60451285 wait complete. 2024-05-08T22:18:03.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.047+0000 7f3aa7573b80 1 -- start start 2024-05-08T22:18:03.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.047+0000 7f3aa7573b80 1 Processor -- start 2024-05-08T22:18:03.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.047+0000 7f3aa7573b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d24c93bca0 0x55d24c944ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.047+0000 7f3aa7573b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d24c9411f0 0x55d24c945430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.047+0000 7f3aa7573b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c941b10 0x55d24c945970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.047+0000 7f3aa7573b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24c823560 con 0x55d24c941b10 2024-05-08T22:18:03.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.047+0000 7f3aa7573b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24c823200 con 0x55d24c93bca0 2024-05-08T22:18:03.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.047+0000 7f3aa7573b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24c70a6a0 con 0x55d24c9411f0 2024-05-08T22:18:03.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.048+0000 7f3aa7573b80 10 client.0 _ll_register_callbacks cb 0x55d24c82cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:18:03.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.048+0000 7f3a91a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c941b10 0x55d24c945970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.048+0000 7f3a91000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d24c93bca0 0x55d24c944ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.048+0000 7f3a8be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d24c9411f0 0x55d24c945430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.048+0000 7f3a91000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d24c93bca0 0x55d24c944ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:53258/0 (socket says 172.21.15.131:53258) 2024-05-08T22:18:03.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.048+0000 7f3a8be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d24c9411f0 0x55d24c945430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40584/0 (socket says 172.21.15.131:40584) 2024-05-08T22:18:03.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.048+0000 7f3a91000700 1 -- 172.21.15.131:0/3822077074 learned_addr learned my addr 172.21.15.131:0/3822077074 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:03.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.048+0000 7f3a91a00700 1 --2- 172.21.15.131:0/3822077074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c941b10 0x55d24c945970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34986/0 (socket says 172.21.15.131:34986) 2024-05-08T22:18:03.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.048+0000 7f3a91000700 1 -- 172.21.15.131:0/3822077074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d24c9411f0 msgr2=0x55d24c945430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:03.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.048+0000 7f3a91000700 1 --2- 172.21.15.131:0/3822077074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d24c9411f0 0x55d24c945430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.048+0000 7f3a91000700 1 -- 172.21.15.131:0/3822077074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c941b10 msgr2=0x55d24c945970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:03.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.048+0000 7f3a91000700 1 --2- 172.21.15.131:0/3822077074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c941b10 0x55d24c945970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.048+0000 7f3a91000700 1 -- 172.21.15.131:0/3822077074 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a8c10e670 con 0x55d24c93bca0 2024-05-08T22:18:03.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.048+0000 7f3a91a00700 1 --2- 172.21.15.131:0/3822077074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c941b10 0x55d24c945970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:18:03.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.048+0000 7f3a8be00700 1 --2- 172.21.15.131:0/3822077074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d24c9411f0 0x55d24c945430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:18:03.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.048+0000 7f3a91000700 1 --2- 172.21.15.131:0/3822077074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d24c93bca0 0x55d24c944ef0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3a80005920 tx=0x7f3a8000b920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff4c8d81f88e4596 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:03.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.048+0000 7f3a72a00700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:18:03.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.049+0000 7f3a72a00700 1 -- 172.21.15.131:0/3822077074 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8001d030 con 0x55d24c93bca0 2024-05-08T22:18:03.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.049+0000 7f3aa7573b80 1 -- 172.21.15.131:0/3822077074 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d24c7e0270 con 0x55d24c93bca0 2024-05-08T22:18:03.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.049+0000 7f3a72a00700 1 -- 172.21.15.131:0/3822077074 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9509+0+0 (secure 0 0 0) 0x7f3a80004700 con 0x55d24c93bca0 2024-05-08T22:18:03.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.049+0000 7f3a72a00700 1 -- 172.21.15.131:0/3822077074 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a8000bed0 con 0x55d24c93bca0 2024-05-08T22:18:03.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.049+0000 7f3aa7573b80 10 client.26855 fetch_fsmap learned FSMap version 30 2024-05-08T22:18:03.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.049+0000 7f3aa7573b80 1 -- 172.21.15.131:0/3822077074 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d24c6b4b60 con 0x55d24c93bca0 2024-05-08T22:18:03.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.049+0000 7f3a72a00700 1 -- 172.21.15.131:0/3822077074 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d24c7e0270 con 0x55d24c93bca0 2024-05-08T22:18:03.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.049+0000 7f3a72a00700 1 -- 172.21.15.131:0/3822077074 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3a8000f770 con 0x55d24c93bca0 2024-05-08T22:18:03.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.049+0000 7f3aa7573b80 10 client.26855 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:18:03.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.049+0000 7f3aa7573b80 10 client.26855 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:18:03.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.050+0000 7f3aa7573b80 20 client.26855 populate_metadata read hostname 'smithi131' 2024-05-08T22:18:03.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.050+0000 7f3aa7573b80 1 --2- 172.21.15.131:0/3822077074 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d24c9550e0 0x55d24c9574d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.050+0000 7f3aa7573b80 4 client.26855 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:18:03.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.050+0000 7f3aa7573b80 1 -- 172.21.15.131:0/3822077074 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d24c906810 con 0x55d24c9550e0 2024-05-08T22:18:03.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.050+0000 7f3a8be00700 1 --2- 172.21.15.131:0/3822077074 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d24c9550e0 0x55d24c9574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.050+0000 7f3a72a00700 10 client.26855 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:18:03.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.050+0000 7f3a8be00700 1 --2- 172.21.15.131:0/3822077074 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d24c9550e0 0x55d24c9574d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=88f4053bf5398f5e server_cookie=a8fe03ccd5617925 in_seq=0 out_seq=0 2024-05-08T22:18:03.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.051+0000 7f3a72a00700 1 -- 172.21.15.131:0/3822077074 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d24c906810 con 0x55d24c9550e0 2024-05-08T22:18:03.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.051+0000 7f3a72a00700 10 client.26855 handle_command_reply: tid=0 2024-05-08T22:18:03.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.051+0000 7f3aa7573b80 1 client.26855 shutdown 2024-05-08T22:18:03.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.051+0000 7f3aa7573b80 1 -- 172.21.15.131:0/3822077074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d24c93bca0 msgr2=0x55d24c944ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.051+0000 7f3aa7573b80 1 --2- 172.21.15.131:0/3822077074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d24c93bca0 0x55d24c944ef0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3a80005920 tx=0x7f3a8000b920 comp rx=0 tx=0).stop 2024-05-08T22:18:03.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.051+0000 7f3aa7573b80 1 -- 172.21.15.131:0/3822077074 shutdown_connections 2024-05-08T22:18:03.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.051+0000 7f3aa7573b80 1 --2- 172.21.15.131:0/3822077074 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d24c9550e0 0x55d24c9574d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.051+0000 7f3aa7573b80 1 --2- 172.21.15.131:0/3822077074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c941b10 0x55d24c945970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.051+0000 7f3a72a00700 0 client.26855 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:18:03.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.051+0000 7f3aa7573b80 1 --2- 172.21.15.131:0/3822077074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d24c93bca0 0x55d24c944ef0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.051+0000 7f3aa7573b80 1 --2- 172.21.15.131:0/3822077074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d24c9411f0 0x55d24c945430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.051+0000 7f3aa7573b80 1 -- 172.21.15.131:0/3822077074 >> 172.21.15.131:0/3822077074 conn(0x55d24c895210 msgr2=0x55d24c9315a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:03.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.051+0000 7f3aa7573b80 1 -- 172.21.15.131:0/3822077074 shutdown_connections 2024-05-08T22:18:03.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.051+0000 7f3aa7573b80 1 -- 172.21.15.131:0/3822077074 wait complete. 2024-05-08T22:18:03.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.052+0000 7f3aa7573b80 20 client.26855 trim_cache size 0 max 16384 2024-05-08T22:18:03.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.065+0000 7f3aa7573b80 1 Processor -- start 2024-05-08T22:18:03.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.065+0000 7f3aa7573b80 1 -- start start 2024-05-08T22:18:03.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.065+0000 7f3aa7573b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c93bca0 0x55d24c951650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.065+0000 7f3aa7573b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d24c9411f0 0x55d24c9406c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.065+0000 7f3aa7573b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d24c941b10 0x55d24c931d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.065+0000 7f3aa7573b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24c92edc0 con 0x55d24c93bca0 2024-05-08T22:18:03.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.065+0000 7f3aa7573b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24c940c00 con 0x55d24c9411f0 2024-05-08T22:18:03.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.065+0000 7f3aa7573b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24c933670 con 0x55d24c941b10 2024-05-08T22:18:03.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.066+0000 7f3a91000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c93bca0 0x55d24c951650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.066+0000 7f3a91000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c93bca0 0x55d24c951650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34996/0 (socket says 172.21.15.131:34996) 2024-05-08T22:18:03.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.066+0000 7f3a91000700 1 -- 172.21.15.131:0/1475759121 learned_addr learned my addr 172.21.15.131:0/1475759121 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:03.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.066+0000 7f3a8be00700 1 --2- 172.21.15.131:0/1475759121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d24c9411f0 0x55d24c9406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.066+0000 7f3a91a00700 1 --2- 172.21.15.131:0/1475759121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d24c941b10 0x55d24c931d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.066+0000 7f3a91000700 1 -- 172.21.15.131:0/1475759121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d24c941b10 msgr2=0x55d24c931d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:03.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.066+0000 7f3a91000700 1 --2- 172.21.15.131:0/1475759121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d24c941b10 0x55d24c931d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.066+0000 7f3a91000700 1 -- 172.21.15.131:0/1475759121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d24c9411f0 msgr2=0x55d24c9406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:03.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.066+0000 7f3a91000700 1 --2- 172.21.15.131:0/1475759121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d24c9411f0 0x55d24c9406c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.066+0000 7f3a91000700 1 -- 172.21.15.131:0/1475759121 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d24c84d560 con 0x55d24c93bca0 2024-05-08T22:18:03.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.066+0000 7f3a8be00700 1 --2- 172.21.15.131:0/1475759121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d24c9411f0 0x55d24c9406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:18:03.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.066+0000 7f3a91000700 1 --2- 172.21.15.131:0/1475759121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c93bca0 0x55d24c951650 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000b2e0 tx=0x7f3a80005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b73f1e8244fffa6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:03.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.067+0000 7f3a7ea00700 1 -- 172.21.15.131:0/1475759121 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8001d040 con 0x55d24c93bca0 2024-05-08T22:18:03.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.067+0000 7f3a7ea00700 1 -- 172.21.15.131:0/1475759121 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a8000ba40 con 0x55d24c93bca0 2024-05-08T22:18:03.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.067+0000 7f3a7ea00700 1 -- 172.21.15.131:0/1475759121 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a800177c0 con 0x55d24c93bca0 2024-05-08T22:18:03.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.067+0000 7f3aa7573b80 1 -- 172.21.15.131:0/1475759121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c93bca0 msgr2=0x55d24c951650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.067+0000 7f3aa7573b80 1 --2- 172.21.15.131:0/1475759121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c93bca0 0x55d24c951650 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000b2e0 tx=0x7f3a80005fd0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.067+0000 7f3aa7573b80 1 -- 172.21.15.131:0/1475759121 shutdown_connections 2024-05-08T22:18:03.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.067+0000 7f3aa7573b80 1 --2- 172.21.15.131:0/1475759121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d24c941b10 0x55d24c931d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.067+0000 7f3aa7573b80 1 --2- 172.21.15.131:0/1475759121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c93bca0 0x55d24c951650 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.067+0000 7f3aa7573b80 1 --2- 172.21.15.131:0/1475759121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d24c9411f0 0x55d24c9406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.067+0000 7f3aa7573b80 1 -- 172.21.15.131:0/1475759121 >> 172.21.15.131:0/1475759121 conn(0x55d24c895210 msgr2=0x55d24c93ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:03.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.067+0000 7f3aa7573b80 1 -- 172.21.15.131:0/1475759121 shutdown_connections 2024-05-08T22:18:03.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.067+0000 7f3aa7573b80 1 -- 172.21.15.131:0/1475759121 wait complete. 2024-05-08T22:18:03.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.068+0000 7f3aa7573b80 1 -- start start 2024-05-08T22:18:03.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.068+0000 7f3aa7573b80 1 Processor -- start 2024-05-08T22:18:03.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.068+0000 7f3aa7573b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d24c93bca0 0x55d24c94c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.068+0000 7f3aa7573b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c9411f0 0x55d24c949ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.068+0000 7f3aa7573b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d24c941b10 0x55d24c94a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.068+0000 7f3aa7573b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24c94a560 con 0x55d24c9411f0 2024-05-08T22:18:03.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.068+0000 7f3aa7573b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24c94a6a0 con 0x55d24c93bca0 2024-05-08T22:18:03.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.068+0000 7f3aa7573b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24c94a830 con 0x55d24c941b10 2024-05-08T22:18:03.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.068+0000 7f3aa7573b80 10 client.0 _ll_register_callbacks cb 0x55d24c82cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:18:03.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.069+0000 7f3a8be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c9411f0 0x55d24c949ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.069+0000 7f3a8be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c9411f0 0x55d24c949ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35008/0 (socket says 172.21.15.131:35008) 2024-05-08T22:18:03.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.069+0000 7f3a8be00700 1 -- 172.21.15.131:0/4351307 learned_addr learned my addr 172.21.15.131:0/4351307 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:03.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.069+0000 7f3a91000700 1 --2- 172.21.15.131:0/4351307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d24c93bca0 0x55d24c94c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.069+0000 7f3a91a00700 1 --2- 172.21.15.131:0/4351307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d24c941b10 0x55d24c94a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.069+0000 7f3a8be00700 1 -- 172.21.15.131:0/4351307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d24c941b10 msgr2=0x55d24c94a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:03.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.069+0000 7f3a8be00700 1 --2- 172.21.15.131:0/4351307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d24c941b10 0x55d24c94a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.069+0000 7f3a8be00700 1 -- 172.21.15.131:0/4351307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d24c93bca0 msgr2=0x55d24c94c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:03.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.069+0000 7f3a8be00700 1 --2- 172.21.15.131:0/4351307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d24c93bca0 0x55d24c94c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.069+0000 7f3a8be00700 1 -- 172.21.15.131:0/4351307 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a8c1a44c0 con 0x55d24c9411f0 2024-05-08T22:18:03.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.069+0000 7f3a91a00700 1 --2- 172.21.15.131:0/4351307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d24c941b10 0x55d24c94a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:18:03.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.069+0000 7f3a8be00700 1 --2- 172.21.15.131:0/4351307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c9411f0 0x55d24c949ae0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3a78099d30 tx=0x7f3a780964a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9222ad747ccb229e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:03.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.069+0000 7f3a72a00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:18:03.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.069+0000 7f3a72a00700 1 -- 172.21.15.131:0/4351307 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7809de60 con 0x55d24c9411f0 2024-05-08T22:18:03.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.069+0000 7f3a72a00700 1 -- 172.21.15.131:0/4351307 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9509+0+0 (secure 0 0 0) 0x7f3a780a24b0 con 0x55d24c9411f0 2024-05-08T22:18:03.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.069+0000 7f3aa7573b80 1 -- 172.21.15.131:0/4351307 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d24c87fec0 con 0x55d24c9411f0 2024-05-08T22:18:03.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.070+0000 7f3a72a00700 1 -- 172.21.15.131:0/4351307 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a780b2be0 con 0x55d24c9411f0 2024-05-08T22:18:03.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.070+0000 7f3aa7573b80 10 client.16767 fetch_fsmap learned FSMap version 30 2024-05-08T22:18:03.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.070+0000 7f3aa7573b80 1 -- 172.21.15.131:0/4351307 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d24c6b4b60 con 0x55d24c9411f0 2024-05-08T22:18:03.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.070+0000 7f3a72a00700 1 -- 172.21.15.131:0/4351307 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d24c87fec0 con 0x55d24c9411f0 2024-05-08T22:18:03.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.070+0000 7f3a72a00700 1 -- 172.21.15.131:0/4351307 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3a78099520 con 0x55d24c9411f0 2024-05-08T22:18:03.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.070+0000 7f3aa7573b80 10 client.16767 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:18:03.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.070+0000 7f3aa7573b80 10 client.16767 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-05-08T22:18:03.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.070+0000 7f3aa7573b80 20 client.16767 populate_metadata read hostname 'smithi131' 2024-05-08T22:18:03.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.070+0000 7f3aa7573b80 1 --2- 172.21.15.131:0/4351307 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d24c9abaf0 0x55d24c9adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.070+0000 7f3aa7573b80 4 client.16767 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:18:03.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.070+0000 7f3aa7573b80 1 -- 172.21.15.131:0/4351307 --> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d24c6d5cf0 con 0x55d24c9abaf0 2024-05-08T22:18:03.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.070+0000 7f3a91000700 1 --2- 172.21.15.131:0/4351307 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d24c9abaf0 0x55d24c9adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.071+0000 7f3a91000700 1 --2- 172.21.15.131:0/4351307 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d24c9abaf0 0x55d24c9adf10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7aa55faf13c5fba2 server_cookie=d39dd3f06f16be93 in_seq=0 out_seq=0 2024-05-08T22:18:03.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.071+0000 7f3a72a00700 10 client.16767 ms_handle_connect on v2:172.21.15.131:6838/1622880274 2024-05-08T22:18:03.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.071+0000 7f3a72a00700 1 -- 172.21.15.131:0/4351307 <== mds.2 v2:172.21.15.131:6838/1622880274 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d24c6d5cf0 con 0x55d24c9abaf0 2024-05-08T22:18:03.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.071+0000 7f3a72a00700 10 client.16767 handle_command_reply: tid=0 2024-05-08T22:18:03.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.071+0000 7f3aa7573b80 1 client.16767 shutdown 2024-05-08T22:18:03.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.071+0000 7f3aa7573b80 1 -- 172.21.15.131:0/4351307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c9411f0 msgr2=0x55d24c949ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.071+0000 7f3aa7573b80 1 --2- 172.21.15.131:0/4351307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c9411f0 0x55d24c949ae0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3a78099d30 tx=0x7f3a780964a0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.071+0000 7f3aa7573b80 1 -- 172.21.15.131:0/4351307 shutdown_connections 2024-05-08T22:18:03.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.071+0000 7f3aa7573b80 1 --2- 172.21.15.131:0/4351307 >> [v2:172.21.15.131:6838/1622880274,v1:172.21.15.131:6839/1622880274] conn(0x55d24c9abaf0 0x55d24c9adf10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.071+0000 7f3aa7573b80 1 --2- 172.21.15.131:0/4351307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d24c941b10 0x55d24c94a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.071+0000 7f3aa7573b80 1 --2- 172.21.15.131:0/4351307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d24c93bca0 0x55d24c94c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.071+0000 7f3aa7573b80 1 --2- 172.21.15.131:0/4351307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d24c9411f0 0x55d24c949ae0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.071+0000 7f3aa7573b80 1 -- 172.21.15.131:0/4351307 >> 172.21.15.131:0/4351307 conn(0x55d24c895210 msgr2=0x55d24c9457d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:03.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.071+0000 7f3a72a00700 0 client.16767 ms_handle_reset on v2:172.21.15.131:6838/1622880274 2024-05-08T22:18:03.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.071+0000 7f3aa7573b80 1 -- 172.21.15.131:0/4351307 shutdown_connections 2024-05-08T22:18:03.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.071+0000 7f3aa7573b80 1 -- 172.21.15.131:0/4351307 wait complete. 2024-05-08T22:18:03.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.072+0000 7f3aa7573b80 20 client.16767 trim_cache size 0 max 16384 2024-05-08T22:18:03.077 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:18:03.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.072+0000 7f3a93a00700 1 -- 172.21.15.131:0/3572505958 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3a740a1290 msgr2=0x7f3a740a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.072+0000 7f3a93a00700 1 --2- 172.21.15.131:0/3572505958 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3a740a1290 0x7f3a740a3750 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400f870 tx=0x7f3a8400fc30 comp rx=0 tx=0).stop 2024-05-08T22:18:03.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.072+0000 7f3a93a00700 1 -- 172.21.15.131:0/3572505958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a8c106280 msgr2=0x7f3a8c197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.072+0000 7f3a93a00700 1 --2- 172.21.15.131:0/3572505958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a8c106280 0x7f3a8c197990 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3a7800e970 tx=0x7f3a7800ec80 comp rx=0 tx=0).stop 2024-05-08T22:18:03.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.072+0000 7f3a93a00700 1 -- 172.21.15.131:0/3572505958 shutdown_connections 2024-05-08T22:18:03.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.072+0000 7f3a93a00700 1 --2- 172.21.15.131:0/3572505958 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3a740a1290 0x7f3a740a3750 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.072+0000 7f3a93a00700 1 --2- 172.21.15.131:0/3572505958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a8c19bd30 0x7f3a8c19c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.072+0000 7f3a93a00700 1 --2- 172.21.15.131:0/3572505958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a8c101160 0x7f3a8c197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.072+0000 7f3a93a00700 1 --2- 172.21.15.131:0/3572505958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a8c106280 0x7f3a8c197990 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.072+0000 7f3a93a00700 1 -- 172.21.15.131:0/3572505958 >> 172.21.15.131:0/3572505958 conn(0x7f3a8c0fc9c0 msgr2=0x7f3a8c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:03.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.072+0000 7f3a93a00700 1 -- 172.21.15.131:0/3572505958 shutdown_connections 2024-05-08T22:18:03.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.072+0000 7f3a93a00700 1 -- 172.21.15.131:0/3572505958 wait complete. 2024-05-08T22:18:03.091 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T22:18:03.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.285+0000 7f8ff6600700 1 -- 172.21.15.131:0/9328210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff0108490 msgr2=0x7f8ff0110010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.285+0000 7f8ff6600700 1 --2- 172.21.15.131:0/9328210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff0108490 0x7f8ff0110010 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800b0a0 tx=0x7f8fe800b3b0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.286+0000 7f8ff6600700 1 -- 172.21.15.131:0/9328210 shutdown_connections 2024-05-08T22:18:03.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.286+0000 7f8ff6600700 1 --2- 172.21.15.131:0/9328210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff0108490 0x7f8ff0110010 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.286+0000 7f8ff6600700 1 --2- 172.21.15.131:0/9328210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ff0103370 0x7f8ff0103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.286+0000 7f8ff6600700 1 --2- 172.21.15.131:0/9328210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ff0103d20 0x7f8ff0107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.286+0000 7f8ff6600700 1 -- 172.21.15.131:0/9328210 >> 172.21.15.131:0/9328210 conn(0x7f8ff00febb0 msgr2=0x7f8ff0100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:03.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.286+0000 7f8ff6600700 1 -- 172.21.15.131:0/9328210 shutdown_connections 2024-05-08T22:18:03.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.286+0000 7f8ff6600700 1 -- 172.21.15.131:0/9328210 wait complete. 2024-05-08T22:18:03.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.287+0000 7f8ff6600700 1 Processor -- start 2024-05-08T22:18:03.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.287+0000 7f8ff6600700 1 -- start start 2024-05-08T22:18:03.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.287+0000 7f8ff6600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ff0103370 0x7f8ff0199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.287+0000 7f8ff6600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ff0108490 0x7f8ff0199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.287+0000 7f8ff6600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff019cf90 0x7f8ff019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.287+0000 7f8ff6600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff019d980 con 0x7f8ff019cf90 2024-05-08T22:18:03.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.287+0000 7f8ff6600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff019daf0 con 0x7f8ff0103370 2024-05-08T22:18:03.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.287+0000 7f8ff6600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff019dcb0 con 0x7f8ff0108490 2024-05-08T22:18:03.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.287+0000 7f8fefe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff019cf90 0x7f8ff019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.287+0000 7f8fef400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ff0103370 0x7f8ff0199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.288+0000 7f8feea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ff0108490 0x7f8ff0199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.288+0000 7f8fefe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff019cf90 0x7f8ff019d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35024/0 (socket says 172.21.15.131:35024) 2024-05-08T22:18:03.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.288+0000 7f8fefe00700 1 -- 172.21.15.131:0/739864653 learned_addr learned my addr 172.21.15.131:0/739864653 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:03.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.288+0000 7f8fefe00700 1 -- 172.21.15.131:0/739864653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ff0108490 msgr2=0x7f8ff0199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.288+0000 7f8fefe00700 1 --2- 172.21.15.131:0/739864653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ff0108490 0x7f8ff0199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.288+0000 7f8fefe00700 1 -- 172.21.15.131:0/739864653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ff0103370 msgr2=0x7f8ff0199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.288+0000 7f8fefe00700 1 --2- 172.21.15.131:0/739864653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ff0103370 0x7f8ff0199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.288+0000 7f8fefe00700 1 -- 172.21.15.131:0/739864653 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe0009520 con 0x7f8ff019cf90 2024-05-08T22:18:03.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.288+0000 7f8fefe00700 1 --2- 172.21.15.131:0/739864653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff019cf90 0x7f8ff019d440 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800b070 tx=0x7f8fe800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:03.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.288+0000 7f8fe7e00700 1 -- 172.21.15.131:0/739864653 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe800e040 con 0x7f8ff019cf90 2024-05-08T22:18:03.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.288+0000 7f8fe7e00700 1 -- 172.21.15.131:0/739864653 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fe801d480 con 0x7f8ff019cf90 2024-05-08T22:18:03.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.288+0000 7f8ff6600700 1 -- 172.21.15.131:0/739864653 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe8009d00 con 0x7f8ff019cf90 2024-05-08T22:18:03.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.288+0000 7f8fe7e00700 1 -- 172.21.15.131:0/739864653 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe80052a0 con 0x7f8ff019cf90 2024-05-08T22:18:03.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.289+0000 7f8ff6600700 1 -- 172.21.15.131:0/739864653 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff01a6560 con 0x7f8ff019cf90 2024-05-08T22:18:03.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.293+0000 7f8fe7e00700 1 -- 172.21.15.131:0/739864653 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f8fe801d5f0 con 0x7f8ff019cf90 2024-05-08T22:18:03.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.294+0000 7f900a0cfb80 1 Processor -- start 2024-05-08T22:18:03.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.294+0000 7f900a0cfb80 1 -- start start 2024-05-08T22:18:03.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.294+0000 7f8fe7e00700 1 --2- 172.21.15.131:0/739864653 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8fd40a12c0 0x7f8fd40a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.294+0000 7f900a0cfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275cbe00 0x55dd275cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.294+0000 7f900a0cfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd275cc720 0x55dd275c6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.294+0000 7f900a0cfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd275c6890 0x55dd275c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.294+0000 7f8fe7e00700 1 -- 172.21.15.131:0/739864653 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9509+0+0 (secure 0 0 0) 0x7f8fe80ce0e0 con 0x7f8ff019cf90 2024-05-08T22:18:03.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.294+0000 7f900a0cfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2751d150 con 0x55dd275cbe00 2024-05-08T22:18:03.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.294+0000 7f900a0cfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2751ab10 con 0x55dd275cc720 2024-05-08T22:18:03.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.294+0000 7f900a0cfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd27516400 con 0x55dd275c6890 2024-05-08T22:18:03.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.294+0000 7f8feea00700 1 --2- 172.21.15.131:0/739864653 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8fd40a12c0 0x7f8fd40a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.294+0000 7f8fef400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275cbe00 0x55dd275cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.295+0000 7f8feea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd275cc720 0x55dd275c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.295+0000 7f8fef400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275cbe00 0x55dd275cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35036/0 (socket says 172.21.15.131:35036) 2024-05-08T22:18:03.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.295+0000 7f8feea00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd275cc720 0x55dd275c6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:53302/0 (socket says 172.21.15.131:53302) 2024-05-08T22:18:03.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.295+0000 7f8fef400700 1 -- 172.21.15.131:0/7953841 learned_addr learned my addr 172.21.15.131:0/7953841 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:03.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.295+0000 7f8fefe00700 1 --2- 172.21.15.131:0/7953841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd275c6890 0x55dd275c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.295+0000 7f8fef400700 1 -- 172.21.15.131:0/7953841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd275c6890 msgr2=0x55dd275c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:03.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.295+0000 7f8fef400700 1 --2- 172.21.15.131:0/7953841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd275c6890 0x55dd275c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.295+0000 7f8fef400700 1 -- 172.21.15.131:0/7953841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd275cc720 msgr2=0x55dd275c6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:03.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.295+0000 7f8fef400700 1 --2- 172.21.15.131:0/7953841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd275cc720 0x55dd275c6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.295+0000 7f8fef400700 1 -- 172.21.15.131:0/7953841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd273973b0 con 0x55dd275cbe00 2024-05-08T22:18:03.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.295+0000 7f8fefe00700 1 --2- 172.21.15.131:0/7953841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd275c6890 0x55dd275c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:18:03.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.295+0000 7f8feea00700 1 --2- 172.21.15.131:0/739864653 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8fd40a12c0 0x7f8fd40a3780 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8ff019acb0 tx=0x7f8fdc009ac0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:03.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.295+0000 7f8fef400700 1 --2- 172.21.15.131:0/7953841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275cbe00 0x55dd275cc1e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0009f10 tx=0x7f8fe0011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b437b9991c17b746 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.296+0000 7f8fe4c00700 1 -- 172.21.15.131:0/7953841 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0019630 con 0x55dd275cbe00 2024-05-08T22:18:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.296+0000 7f8fe4c00700 1 -- 172.21.15.131:0/7953841 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fe0019c50 con 0x55dd275cbe00 2024-05-08T22:18:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.296+0000 7f8fe4c00700 1 -- 172.21.15.131:0/7953841 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe000ec10 con 0x55dd275cbe00 2024-05-08T22:18:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.296+0000 7f900a0cfb80 1 -- 172.21.15.131:0/7953841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275cbe00 msgr2=0x55dd275cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.296+0000 7f900a0cfb80 1 --2- 172.21.15.131:0/7953841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275cbe00 0x55dd275cc1e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0009f10 tx=0x7f8fe0011720 comp rx=0 tx=0).stop 2024-05-08T22:18:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.296+0000 7f900a0cfb80 1 -- 172.21.15.131:0/7953841 shutdown_connections 2024-05-08T22:18:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.296+0000 7f900a0cfb80 1 --2- 172.21.15.131:0/7953841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd275c6890 0x55dd275c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.296+0000 7f900a0cfb80 1 --2- 172.21.15.131:0/7953841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275cbe00 0x55dd275cc1e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.296+0000 7f900a0cfb80 1 --2- 172.21.15.131:0/7953841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd275cc720 0x55dd275c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.296+0000 7f900a0cfb80 1 -- 172.21.15.131:0/7953841 >> 172.21.15.131:0/7953841 conn(0x55dd27524210 msgr2=0x55dd275c0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.296+0000 7f900a0cfb80 1 -- 172.21.15.131:0/7953841 shutdown_connections 2024-05-08T22:18:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.296+0000 7f900a0cfb80 1 -- 172.21.15.131:0/7953841 wait complete. 2024-05-08T22:18:03.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.297+0000 7f900a0cfb80 1 -- start start 2024-05-08T22:18:03.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.297+0000 7f900a0cfb80 1 Processor -- start 2024-05-08T22:18:03.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.298+0000 7f900a0cfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd275c6890 0x55dd275cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.298+0000 7f900a0cfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd275cbe00 0x55dd275d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.298+0000 7f900a0cfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275cc720 0x55dd275d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.298+0000 7f900a0cfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd274b2560 con 0x55dd275cc720 2024-05-08T22:18:03.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.298+0000 7f900a0cfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd274b2200 con 0x55dd275c6890 2024-05-08T22:18:03.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.298+0000 7f900a0cfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd273996a0 con 0x55dd275cbe00 2024-05-08T22:18:03.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.298+0000 7f900a0cfb80 10 client.0 _ll_register_callbacks cb 0x55dd274bbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:18:03.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.298+0000 7f8fefe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275cc720 0x55dd275d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.298+0000 7f8fefe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275cc720 0x55dd275d0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35038/0 (socket says 172.21.15.131:35038) 2024-05-08T22:18:03.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.298+0000 7f8fefe00700 1 -- 172.21.15.131:0/2223590146 learned_addr learned my addr 172.21.15.131:0/2223590146 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:03.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.298+0000 7f8feea00700 1 --2- 172.21.15.131:0/2223590146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd275cbe00 0x55dd275d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.298+0000 7f8fef400700 1 --2- 172.21.15.131:0/2223590146 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd275c6890 0x55dd275cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.298+0000 7f8fefe00700 1 -- 172.21.15.131:0/2223590146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd275cbe00 msgr2=0x55dd275d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:03.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.298+0000 7f8fefe00700 1 --2- 172.21.15.131:0/2223590146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd275cbe00 0x55dd275d0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.298+0000 7f8fefe00700 1 -- 172.21.15.131:0/2223590146 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd275c6890 msgr2=0x55dd275cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:03.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.298+0000 7f8fefe00700 1 --2- 172.21.15.131:0/2223590146 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd275c6890 0x55dd275cfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.298+0000 7f8fefe00700 1 -- 172.21.15.131:0/2223590146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff01a6560 con 0x55dd275cc720 2024-05-08T22:18:03.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.298+0000 7f8fef400700 1 --2- 172.21.15.131:0/2223590146 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd275c6890 0x55dd275cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:18:03.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.299+0000 7f8fefe00700 1 --2- 172.21.15.131:0/2223590146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275cc720 0x55dd275d0560 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80cb500 tx=0x7f8fe8097310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8869c7ad8ff4916 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:03.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.299+0000 7f8fd9600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:18:03.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.299+0000 7f8fd9600700 1 -- 172.21.15.131:0/2223590146 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe809b300 con 0x55dd275cc720 2024-05-08T22:18:03.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.299+0000 7f8fd9600700 1 -- 172.21.15.131:0/2223590146 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9509+0+0 (secure 0 0 0) 0x7f8fe809aea0 con 0x55dd275cc720 2024-05-08T22:18:03.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.299+0000 7f900a0cfb80 1 -- 172.21.15.131:0/2223590146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd2746f270 con 0x55dd275cc720 2024-05-08T22:18:03.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.299+0000 7f8fd9600700 1 -- 172.21.15.131:0/2223590146 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fe801dae0 con 0x55dd275cc720 2024-05-08T22:18:03.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.299+0000 7f900a0cfb80 10 client.16779 fetch_fsmap learned FSMap version 30 2024-05-08T22:18:03.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.299+0000 7f900a0cfb80 1 -- 172.21.15.131:0/2223590146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f8fe0009b70 con 0x55dd275cc720 2024-05-08T22:18:03.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.299+0000 7f8fd9600700 1 -- 172.21.15.131:0/2223590146 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dd2746f270 con 0x55dd275cc720 2024-05-08T22:18:03.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.299+0000 7f8fd9600700 1 -- 172.21.15.131:0/2223590146 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8fe80a2e00 con 0x55dd275cc720 2024-05-08T22:18:03.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.300+0000 7f900a0cfb80 10 client.16779 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:18:03.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.300+0000 7f900a0cfb80 10 client.16779 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:18:03.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.300+0000 7f900a0cfb80 20 client.16779 populate_metadata read hostname 'smithi131' 2024-05-08T22:18:03.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.300+0000 7f900a0cfb80 1 --2- 172.21.15.131:0/2223590146 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55dd275dff20 0x55dd275e2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.300+0000 7f900a0cfb80 4 client.16779 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:18:03.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.300+0000 7f900a0cfb80 1 -- 172.21.15.131:0/2223590146 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd27595810 con 0x55dd275dff20 2024-05-08T22:18:03.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.300+0000 7f8fef400700 1 --2- 172.21.15.131:0/2223590146 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55dd275dff20 0x55dd275e2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.300+0000 7f8fef400700 1 --2- 172.21.15.131:0/2223590146 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55dd275dff20 0x55dd275e2310 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b593850ef9d0e4d server_cookie=a1ce399c4d1a2e41 in_seq=0 out_seq=0 2024-05-08T22:18:03.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.300+0000 7f8fd9600700 10 client.16779 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:18:03.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.301+0000 7f8fd9600700 1 -- 172.21.15.131:0/2223590146 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dd27595810 con 0x55dd275dff20 2024-05-08T22:18:03.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.301+0000 7f8fd9600700 10 client.16779 handle_command_reply: tid=0 2024-05-08T22:18:03.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.301+0000 7f900a0cfb80 1 client.16779 shutdown 2024-05-08T22:18:03.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.301+0000 7f900a0cfb80 1 -- 172.21.15.131:0/2223590146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275cc720 msgr2=0x55dd275d0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.301+0000 7f900a0cfb80 1 --2- 172.21.15.131:0/2223590146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275cc720 0x55dd275d0560 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80cb500 tx=0x7f8fe8097310 comp rx=0 tx=0).stop 2024-05-08T22:18:03.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.301+0000 7f900a0cfb80 1 -- 172.21.15.131:0/2223590146 shutdown_connections 2024-05-08T22:18:03.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.301+0000 7f900a0cfb80 1 --2- 172.21.15.131:0/2223590146 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55dd275dff20 0x55dd275e2310 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.301+0000 7f900a0cfb80 1 --2- 172.21.15.131:0/2223590146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275cc720 0x55dd275d0560 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.301+0000 7f900a0cfb80 1 --2- 172.21.15.131:0/2223590146 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd275c6890 0x55dd275cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.301+0000 7f900a0cfb80 1 --2- 172.21.15.131:0/2223590146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd275cbe00 0x55dd275d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.301+0000 7f900a0cfb80 1 -- 172.21.15.131:0/2223590146 >> 172.21.15.131:0/2223590146 conn(0x55dd27524210 msgr2=0x55dd275c05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:03.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.301+0000 7f8fd9600700 0 client.16779 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:18:03.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.301+0000 7f900a0cfb80 1 -- 172.21.15.131:0/2223590146 shutdown_connections 2024-05-08T22:18:03.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.301+0000 7f900a0cfb80 1 -- 172.21.15.131:0/2223590146 wait complete. 2024-05-08T22:18:03.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.302+0000 7f900a0cfb80 20 client.16779 trim_cache size 0 max 16384 2024-05-08T22:18:03.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.315+0000 7f900a0cfb80 1 Processor -- start 2024-05-08T22:18:03.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.315+0000 7f900a0cfb80 1 -- start start 2024-05-08T22:18:03.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.316+0000 7f900a0cfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275c6890 0x55dd275dbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.316+0000 7f900a0cfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd275cbe00 0x55dd275c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.316+0000 7f900a0cfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd275cc720 0x55dd275c0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.316+0000 7f900a0cfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd275b9110 con 0x55dd275c6890 2024-05-08T22:18:03.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.316+0000 7f900a0cfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd275c1b30 con 0x55dd275cbe00 2024-05-08T22:18:03.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.316+0000 7f900a0cfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd275d1150 con 0x55dd275cc720 2024-05-08T22:18:03.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.316+0000 7f8fef400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275c6890 0x55dd275dbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.316+0000 7f8fef400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275c6890 0x55dd275dbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35046/0 (socket says 172.21.15.131:35046) 2024-05-08T22:18:03.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.316+0000 7f8fef400700 1 -- 172.21.15.131:0/781579126 learned_addr learned my addr 172.21.15.131:0/781579126 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:03.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.316+0000 7f8fefe00700 1 --2- 172.21.15.131:0/781579126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd275cc720 0x55dd275c0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.316+0000 7f8feea00700 1 --2- 172.21.15.131:0/781579126 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd275cbe00 0x55dd275c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.316+0000 7f8fef400700 1 -- 172.21.15.131:0/781579126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd275cc720 msgr2=0x55dd275c0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:03.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.316+0000 7f8fef400700 1 --2- 172.21.15.131:0/781579126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd275cc720 0x55dd275c0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.316+0000 7f8fef400700 1 -- 172.21.15.131:0/781579126 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd275cbe00 msgr2=0x55dd275c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:03.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.316+0000 7f8fef400700 1 --2- 172.21.15.131:0/781579126 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd275cbe00 0x55dd275c6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.316+0000 7f8fef400700 1 -- 172.21.15.131:0/781579126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd27343b60 con 0x55dd275c6890 2024-05-08T22:18:03.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.316+0000 7f8feea00700 1 --2- 172.21.15.131:0/781579126 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd275cbe00 0x55dd275c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:18:03.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.316+0000 7f8fefe00700 1 --2- 172.21.15.131:0/781579126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd275cc720 0x55dd275c0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:18:03.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.316+0000 7f8fef400700 1 --2- 172.21.15.131:0/781579126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275c6890 0x55dd275dbc50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0009f10 tx=0x7f8fe000e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce692add7b7dcc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:03.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.317+0000 7f8fe5600700 1 -- 172.21.15.131:0/781579126 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe000ebf0 con 0x55dd275c6890 2024-05-08T22:18:03.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.317+0000 7f8fe5600700 1 -- 172.21.15.131:0/781579126 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fe0005ce0 con 0x55dd275c6890 2024-05-08T22:18:03.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.317+0000 7f8fe5600700 1 -- 172.21.15.131:0/781579126 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0005390 con 0x55dd275c6890 2024-05-08T22:18:03.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.317+0000 7f900a0cfb80 1 -- 172.21.15.131:0/781579126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275c6890 msgr2=0x55dd275dbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.317+0000 7f900a0cfb80 1 --2- 172.21.15.131:0/781579126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275c6890 0x55dd275dbc50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0009f10 tx=0x7f8fe000e730 comp rx=0 tx=0).stop 2024-05-08T22:18:03.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.317+0000 7f900a0cfb80 1 -- 172.21.15.131:0/781579126 shutdown_connections 2024-05-08T22:18:03.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.317+0000 7f900a0cfb80 1 --2- 172.21.15.131:0/781579126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd275cc720 0x55dd275c0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.317+0000 7f900a0cfb80 1 --2- 172.21.15.131:0/781579126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275c6890 0x55dd275dbc50 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.317+0000 7f900a0cfb80 1 --2- 172.21.15.131:0/781579126 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd275cbe00 0x55dd275c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.317+0000 7f900a0cfb80 1 -- 172.21.15.131:0/781579126 >> 172.21.15.131:0/781579126 conn(0x55dd27524210 msgr2=0x55dd275ce3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:03.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.317+0000 7f900a0cfb80 1 -- 172.21.15.131:0/781579126 shutdown_connections 2024-05-08T22:18:03.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.317+0000 7f900a0cfb80 1 -- 172.21.15.131:0/781579126 wait complete. 2024-05-08T22:18:03.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.318+0000 7f900a0cfb80 1 -- start start 2024-05-08T22:18:03.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.318+0000 7f900a0cfb80 1 Processor -- start 2024-05-08T22:18:03.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.318+0000 7f900a0cfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd275c6890 0x55dd275d3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.318+0000 7f900a0cfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275cbe00 0x55dd275d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.318+0000 7f900a0cfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd275cc720 0x55dd27635400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.318+0000 7f900a0cfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd275d3f70 con 0x55dd275cbe00 2024-05-08T22:18:03.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.318+0000 7f900a0cfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd27635940 con 0x55dd275c6890 2024-05-08T22:18:03.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.318+0000 7f900a0cfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd27635a80 con 0x55dd275cc720 2024-05-08T22:18:03.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.318+0000 7f900a0cfb80 10 client.0 _ll_register_callbacks cb 0x55dd274bbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:18:03.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.319+0000 7f8fefe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd275cc720 0x55dd27635400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.319+0000 7f8feea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275cbe00 0x55dd275d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.319+0000 7f8fefe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd275cc720 0x55dd27635400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40622/0 (socket says 172.21.15.131:40622) 2024-05-08T22:18:03.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.319+0000 7f8fefe00700 1 -- 172.21.15.131:0/3888946500 learned_addr learned my addr 172.21.15.131:0/3888946500 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:03.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.319+0000 7f8fef400700 1 --2- 172.21.15.131:0/3888946500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd275c6890 0x55dd275d3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.319+0000 7f8fefe00700 1 -- 172.21.15.131:0/3888946500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd275c6890 msgr2=0x55dd275d3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:03.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.319+0000 7f8fefe00700 1 --2- 172.21.15.131:0/3888946500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd275c6890 0x55dd275d3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.319+0000 7f8fefe00700 1 -- 172.21.15.131:0/3888946500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275cbe00 msgr2=0x55dd275d36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:03.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.319+0000 7f8fefe00700 1 --2- 172.21.15.131:0/3888946500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275cbe00 0x55dd275d36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.319+0000 7f8fefe00700 1 -- 172.21.15.131:0/3888946500 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd273973b0 con 0x55dd275cc720 2024-05-08T22:18:03.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.319+0000 7f8fefe00700 1 --2- 172.21.15.131:0/3888946500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd275cc720 0x55dd27635400 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8099790 tx=0x7f8fe80b9310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=135405e5954e7a51 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:03.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.319+0000 7f8fd9600700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-08T22:18:03.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.320+0000 7f8fd9600700 1 -- 172.21.15.131:0/3888946500 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8009b80 con 0x55dd275cc720 2024-05-08T22:18:03.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.320+0000 7f900a0cfb80 1 -- 172.21.15.131:0/3888946500 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fe0019c50 con 0x55dd275cc720 2024-05-08T22:18:03.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.320+0000 7f8fd9600700 1 -- 172.21.15.131:0/3888946500 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9509+0+0 (secure 0 0 0) 0x7f8fe809aea0 con 0x55dd275cc720 2024-05-08T22:18:03.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.320+0000 7f8fd9600700 1 -- 172.21.15.131:0/3888946500 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fe8009960 con 0x55dd275cc720 2024-05-08T22:18:03.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.320+0000 7f900a0cfb80 10 client.26980 fetch_fsmap learned FSMap version 30 2024-05-08T22:18:03.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.320+0000 7f900a0cfb80 1 -- 172.21.15.131:0/3888946500 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55dd274dc560 con 0x55dd275cc720 2024-05-08T22:18:03.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.320+0000 7f8fd9600700 1 -- 172.21.15.131:0/3888946500 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fe0019c50 con 0x55dd275cc720 2024-05-08T22:18:03.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.320+0000 7f8fd9600700 1 -- 172.21.15.131:0/3888946500 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8fe80cbba0 con 0x55dd275cc720 2024-05-08T22:18:03.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.320+0000 7f900a0cfb80 10 client.26980 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:18:03.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.320+0000 7f900a0cfb80 10 client.26980 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T22:18:03.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.320+0000 7f900a0cfb80 20 client.26980 populate_metadata read hostname 'smithi131' 2024-05-08T22:18:03.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.320+0000 7f900a0cfb80 1 --2- 172.21.15.131:0/3888946500 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55dd27636c50 0x55dd27639040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.320+0000 7f900a0cfb80 4 client.26980 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:18:03.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.320+0000 7f900a0cfb80 1 -- 172.21.15.131:0/3888946500 --> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dd2750eec0 con 0x55dd27636c50 2024-05-08T22:18:03.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.321+0000 7f8fef400700 1 --2- 172.21.15.131:0/3888946500 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55dd27636c50 0x55dd27639040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.321+0000 7f8fef400700 1 --2- 172.21.15.131:0/3888946500 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55dd27636c50 0x55dd27639040 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4c38f4ecadcd26a5 server_cookie=6229332bf519ba01 in_seq=0 out_seq=0 2024-05-08T22:18:03.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.321+0000 7f8fd9600700 10 client.26980 ms_handle_connect on v2:172.21.15.131:6840/1686013987 2024-05-08T22:18:03.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.321+0000 7f8fd9600700 1 -- 172.21.15.131:0/3888946500 <== mds.1 v2:172.21.15.131:6840/1686013987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dd2750eec0 con 0x55dd27636c50 2024-05-08T22:18:03.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.321+0000 7f8fd9600700 10 client.26980 handle_command_reply: tid=0 2024-05-08T22:18:03.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.321+0000 7f900a0cfb80 1 client.26980 shutdown 2024-05-08T22:18:03.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.321+0000 7f900a0cfb80 1 -- 172.21.15.131:0/3888946500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd275cc720 msgr2=0x55dd27635400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.321+0000 7f900a0cfb80 1 --2- 172.21.15.131:0/3888946500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd275cc720 0x55dd27635400 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8099790 tx=0x7f8fe80b9310 comp rx=0 tx=0).stop 2024-05-08T22:18:03.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.321+0000 7f900a0cfb80 1 -- 172.21.15.131:0/3888946500 shutdown_connections 2024-05-08T22:18:03.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.321+0000 7f900a0cfb80 1 --2- 172.21.15.131:0/3888946500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd275cc720 0x55dd27635400 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.321+0000 7f900a0cfb80 1 --2- 172.21.15.131:0/3888946500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd275c6890 0x55dd275d3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.322+0000 7f900a0cfb80 1 --2- 172.21.15.131:0/3888946500 >> [v2:172.21.15.131:6840/1686013987,v1:172.21.15.131:6841/1686013987] conn(0x55dd27636c50 0x55dd27639040 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.322+0000 7f900a0cfb80 1 --2- 172.21.15.131:0/3888946500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd275cbe00 0x55dd275d36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.322+0000 7f900a0cfb80 1 -- 172.21.15.131:0/3888946500 >> 172.21.15.131:0/3888946500 conn(0x55dd27524210 msgr2=0x55dd275cabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:03.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.322+0000 7f8fd9600700 0 client.26980 ms_handle_reset on v2:172.21.15.131:6840/1686013987 2024-05-08T22:18:03.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.322+0000 7f900a0cfb80 1 -- 172.21.15.131:0/3888946500 shutdown_connections 2024-05-08T22:18:03.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.322+0000 7f900a0cfb80 1 -- 172.21.15.131:0/3888946500 wait complete. 2024-05-08T22:18:03.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.322+0000 7f900a0cfb80 20 client.26980 trim_cache size 0 max 16384 2024-05-08T22:18:03.327 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:18:03.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.322+0000 7f8ff6600700 1 -- 172.21.15.131:0/739864653 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8fd40a12c0 msgr2=0x7f8fd40a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.322+0000 7f8ff6600700 1 --2- 172.21.15.131:0/739864653 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8fd40a12c0 0x7f8fd40a3780 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8ff019acb0 tx=0x7f8fdc009ac0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.322+0000 7f8ff6600700 1 -- 172.21.15.131:0/739864653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff019cf90 msgr2=0x7f8ff019d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.322+0000 7f8ff6600700 1 --2- 172.21.15.131:0/739864653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff019cf90 0x7f8ff019d440 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800b070 tx=0x7f8fe800bda0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.322+0000 7f8ff6600700 1 -- 172.21.15.131:0/739864653 shutdown_connections 2024-05-08T22:18:03.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.322+0000 7f8ff6600700 1 --2- 172.21.15.131:0/739864653 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f8fd40a12c0 0x7f8fd40a3780 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.322+0000 7f8ff6600700 1 --2- 172.21.15.131:0/739864653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff019cf90 0x7f8ff019d440 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.322+0000 7f8ff6600700 1 --2- 172.21.15.131:0/739864653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ff0103370 0x7f8ff0199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.322+0000 7f8ff6600700 1 --2- 172.21.15.131:0/739864653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ff0108490 0x7f8ff0199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.322+0000 7f8ff6600700 1 -- 172.21.15.131:0/739864653 >> 172.21.15.131:0/739864653 conn(0x7f8ff00febb0 msgr2=0x7f8ff0100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:03.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.323+0000 7f8ff6600700 1 -- 172.21.15.131:0/739864653 shutdown_connections 2024-05-08T22:18:03.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.323+0000 7f8ff6600700 1 -- 172.21.15.131:0/739864653 wait complete. 2024-05-08T22:18:03.342 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T22:18:03.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.538+0000 7f4134c00700 1 -- 172.21.15.131:0/2298236276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4130101160 msgr2=0x7f4130101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.538+0000 7f4134c00700 1 --2- 172.21.15.131:0/2298236276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4130101160 0x7f4130101540 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4120009990 tx=0x7f4120009ca0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.538+0000 7f4134c00700 1 -- 172.21.15.131:0/2298236276 shutdown_connections 2024-05-08T22:18:03.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.538+0000 7f4134c00700 1 --2- 172.21.15.131:0/2298236276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4130106280 0x7f413010de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.538+0000 7f4134c00700 1 --2- 172.21.15.131:0/2298236276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4130101160 0x7f4130101540 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.538+0000 7f4134c00700 1 --2- 172.21.15.131:0/2298236276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4130101b10 0x7f4130105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.538+0000 7f4134c00700 1 -- 172.21.15.131:0/2298236276 >> 172.21.15.131:0/2298236276 conn(0x7f41300fc9c0 msgr2=0x7f41300fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:03.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.538+0000 7f4134c00700 1 -- 172.21.15.131:0/2298236276 shutdown_connections 2024-05-08T22:18:03.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.538+0000 7f4134c00700 1 -- 172.21.15.131:0/2298236276 wait complete. 2024-05-08T22:18:03.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.539+0000 7f4134c00700 1 Processor -- start 2024-05-08T22:18:03.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.539+0000 7f4134c00700 1 -- start start 2024-05-08T22:18:03.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.539+0000 7f4134c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4130101160 0x7f4130199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.539+0000 7f4134c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4130106280 0x7f4130199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.539+0000 7f4134c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f413019df40 0x7f413019e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.539+0000 7f4134c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413019e9c0 con 0x7f4130101160 2024-05-08T22:18:03.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.539+0000 7f4134c00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413019eb30 con 0x7f4130106280 2024-05-08T22:18:03.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.539+0000 7f4134c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413019ecf0 con 0x7f413019df40 2024-05-08T22:18:03.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.540+0000 7f412de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4130101160 0x7f4130199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.540+0000 7f412e800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f413019df40 0x7f413019e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.540+0000 7f412de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4130101160 0x7f4130199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35068/0 (socket says 172.21.15.131:35068) 2024-05-08T22:18:03.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.540+0000 7f412de00700 1 -- 172.21.15.131:0/1048719866 learned_addr learned my addr 172.21.15.131:0/1048719866 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:03.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.540+0000 7f4127e00700 1 --2- 172.21.15.131:0/1048719866 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4130106280 0x7f4130199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.540+0000 7f412e800700 1 -- 172.21.15.131:0/1048719866 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4130106280 msgr2=0x7f4130199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.540+0000 7f412e800700 1 --2- 172.21.15.131:0/1048719866 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4130106280 0x7f4130199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.540+0000 7f412e800700 1 -- 172.21.15.131:0/1048719866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4130101160 msgr2=0x7f4130199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.540+0000 7f412e800700 1 --2- 172.21.15.131:0/1048719866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4130101160 0x7f4130199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.540+0000 7f412e800700 1 -- 172.21.15.131:0/1048719866 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41200095f0 con 0x7f413019df40 2024-05-08T22:18:03.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.540+0000 7f412e800700 1 --2- 172.21.15.131:0/1048719866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f413019df40 0x7f413019e3c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f412800efc0 tx=0x7f412800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:03.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.541+0000 7f4126a00700 1 -- 172.21.15.131:0/1048719866 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4128018070 con 0x7f413019df40 2024-05-08T22:18:03.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.541+0000 7f4126a00700 1 -- 172.21.15.131:0/1048719866 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f412800f040 con 0x7f413019df40 2024-05-08T22:18:03.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.541+0000 7f4134c00700 1 -- 172.21.15.131:0/1048719866 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41301a61f0 con 0x7f413019df40 2024-05-08T22:18:03.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.541+0000 7f4126a00700 1 -- 172.21.15.131:0/1048719866 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412800cd40 con 0x7f413019df40 2024-05-08T22:18:03.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.541+0000 7f4134c00700 1 -- 172.21.15.131:0/1048719866 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41301a6680 con 0x7f413019df40 2024-05-08T22:18:03.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.544+0000 7f4126a00700 1 -- 172.21.15.131:0/1048719866 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f4128003a00 con 0x7f413019df40 2024-05-08T22:18:03.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.546+0000 7f41486fab80 1 Processor -- start 2024-05-08T22:18:03.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.546+0000 7f41486fab80 1 -- start start 2024-05-08T22:18:03.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.546+0000 7f4126a00700 1 --2- 172.21.15.131:0/1048719866 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f41140a1290 0x7f41140a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.546+0000 7f4126a00700 1 -- 172.21.15.131:0/1048719866 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9509+0+0 (secure 0 0 0) 0x7f41280ccc80 con 0x7f413019df40 2024-05-08T22:18:03.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.546+0000 7f41486fab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c14e00 0x55b2a9c151e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.546+0000 7f41486fab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2a9c15720 0x55b2a9c0f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.546+0000 7f4127e00700 1 --2- 172.21.15.131:0/1048719866 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f41140a1290 0x7f41140a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.546+0000 7f41486fab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2a9c0f890 0x55b2a9c0fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.546+0000 7f41486fab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a9b66150 con 0x55b2a9c14e00 2024-05-08T22:18:03.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.546+0000 7f41486fab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a9b63b10 con 0x55b2a9c15720 2024-05-08T22:18:03.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.546+0000 7f41486fab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a9b5f400 con 0x55b2a9c0f890 2024-05-08T22:18:03.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.547+0000 7f412de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c14e00 0x55b2a9c151e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.547+0000 7f412de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c14e00 0x55b2a9c151e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35082/0 (socket says 172.21.15.131:35082) 2024-05-08T22:18:03.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.547+0000 7f412de00700 1 -- 172.21.15.131:0/2450797604 learned_addr learned my addr 172.21.15.131:0/2450797604 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:03.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.547+0000 7f412e800700 1 --2- 172.21.15.131:0/2450797604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2a9c0f890 0x55b2a9c0fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.547+0000 7f4127e00700 1 --2- 172.21.15.131:0/2450797604 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2a9c15720 0x55b2a9c0f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.547+0000 7f412de00700 1 -- 172.21.15.131:0/2450797604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2a9c0f890 msgr2=0x55b2a9c0fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:03.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.547+0000 7f412de00700 1 --2- 172.21.15.131:0/2450797604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2a9c0f890 0x55b2a9c0fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.547+0000 7f412de00700 1 -- 172.21.15.131:0/2450797604 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2a9c15720 msgr2=0x55b2a9c0f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:03.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.547+0000 7f412de00700 1 --2- 172.21.15.131:0/2450797604 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2a9c15720 0x55b2a9c0f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.547+0000 7f412de00700 1 -- 172.21.15.131:0/2450797604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2a99e03b0 con 0x55b2a9c14e00 2024-05-08T22:18:03.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.547+0000 7f4127e00700 1 --2- 172.21.15.131:0/1048719866 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f41140a1290 0x7f41140a3750 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f413019abb0 tx=0x7f41180098d0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:03.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.548+0000 7f412de00700 1 --2- 172.21.15.131:0/2450797604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c14e00 0x55b2a9c151e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f412000b5e0 tx=0x7f412000b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafb8a7424921e1b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:03.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.548+0000 7f411f000700 1 -- 172.21.15.131:0/2450797604 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412001d070 con 0x55b2a9c14e00 2024-05-08T22:18:03.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.548+0000 7f411f000700 1 -- 172.21.15.131:0/2450797604 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f412001e040 con 0x55b2a9c14e00 2024-05-08T22:18:03.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.548+0000 7f411f000700 1 -- 172.21.15.131:0/2450797604 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41200048f0 con 0x55b2a9c14e00 2024-05-08T22:18:03.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.549+0000 7f41486fab80 1 -- 172.21.15.131:0/2450797604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c14e00 msgr2=0x55b2a9c151e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.549+0000 7f41486fab80 1 --2- 172.21.15.131:0/2450797604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c14e00 0x55b2a9c151e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f412000b5e0 tx=0x7f412000b610 comp rx=0 tx=0).stop 2024-05-08T22:18:03.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.549+0000 7f41486fab80 1 -- 172.21.15.131:0/2450797604 shutdown_connections 2024-05-08T22:18:03.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.549+0000 7f41486fab80 1 --2- 172.21.15.131:0/2450797604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2a9c0f890 0x55b2a9c0fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.549+0000 7f41486fab80 1 --2- 172.21.15.131:0/2450797604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c14e00 0x55b2a9c151e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.549+0000 7f41486fab80 1 --2- 172.21.15.131:0/2450797604 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2a9c15720 0x55b2a9c0f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.549+0000 7f41486fab80 1 -- 172.21.15.131:0/2450797604 >> 172.21.15.131:0/2450797604 conn(0x55b2a9b6d210 msgr2=0x55b2a9c09df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:03.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.549+0000 7f41486fab80 1 -- 172.21.15.131:0/2450797604 shutdown_connections 2024-05-08T22:18:03.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.549+0000 7f41486fab80 1 -- 172.21.15.131:0/2450797604 wait complete. 2024-05-08T22:18:03.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.549+0000 7f41486fab80 1 -- start start 2024-05-08T22:18:03.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.550+0000 7f41486fab80 1 Processor -- start 2024-05-08T22:18:03.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.550+0000 7f41486fab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2a9c0f890 0x55b2a9c1b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.550+0000 7f41486fab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2a9c14e00 0x55b2a9c189f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.550+0000 7f41486fab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c15720 0x55b2a9c190e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.550+0000 7f41486fab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a9afb560 con 0x55b2a9c15720 2024-05-08T22:18:03.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.550+0000 7f41486fab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a9afb200 con 0x55b2a9c0f890 2024-05-08T22:18:03.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.550+0000 7f41486fab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a99e26a0 con 0x55b2a9c14e00 2024-05-08T22:18:03.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.551+0000 7f41486fab80 10 client.0 _ll_register_callbacks cb 0x55b2a9b04b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:18:03.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.551+0000 7f412e800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c15720 0x55b2a9c190e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.551+0000 7f412de00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2a9c0f890 0x55b2a9c1b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.551+0000 7f412e800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c15720 0x55b2a9c190e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35092/0 (socket says 172.21.15.131:35092) 2024-05-08T22:18:03.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.551+0000 7f412e800700 1 -- 172.21.15.131:0/1870792457 learned_addr learned my addr 172.21.15.131:0/1870792457 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:03.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.551+0000 7f4127e00700 1 --2- 172.21.15.131:0/1870792457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2a9c14e00 0x55b2a9c189f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.551+0000 7f412e800700 1 -- 172.21.15.131:0/1870792457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2a9c14e00 msgr2=0x55b2a9c189f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:03.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.551+0000 7f412e800700 1 --2- 172.21.15.131:0/1870792457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2a9c14e00 0x55b2a9c189f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.551+0000 7f412e800700 1 -- 172.21.15.131:0/1870792457 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2a9c0f890 msgr2=0x55b2a9c1b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:03.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.551+0000 7f412e800700 1 --2- 172.21.15.131:0/1870792457 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2a9c0f890 0x55b2a9c1b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.551+0000 7f412e800700 1 -- 172.21.15.131:0/1870792457 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f413010e670 con 0x55b2a9c15720 2024-05-08T22:18:03.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.551+0000 7f412e800700 1 --2- 172.21.15.131:0/1870792457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c15720 0x55b2a9c190e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f412800c6f0 tx=0x7f41280995a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d53e35667cd41a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:03.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.551+0000 7f4113400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-08T22:18:03.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.552+0000 7f4113400700 1 -- 172.21.15.131:0/1870792457 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4128004ad0 con 0x55b2a9c15720 2024-05-08T22:18:03.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.552+0000 7f4113400700 1 -- 172.21.15.131:0/1870792457 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9509+0+0 (secure 0 0 0) 0x7f412809a0e0 con 0x55b2a9c15720 2024-05-08T22:18:03.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.552+0000 7f41486fab80 1 -- 172.21.15.131:0/1870792457 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b2a9ab8270 con 0x55b2a9c15720 2024-05-08T22:18:03.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.552+0000 7f4113400700 1 -- 172.21.15.131:0/1870792457 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41280999e0 con 0x55b2a9c15720 2024-05-08T22:18:03.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.552+0000 7f41486fab80 10 client.16797 fetch_fsmap learned FSMap version 30 2024-05-08T22:18:03.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.552+0000 7f41486fab80 1 -- 172.21.15.131:0/1870792457 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b2a998cb60 con 0x55b2a9c15720 2024-05-08T22:18:03.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.552+0000 7f4113400700 1 -- 172.21.15.131:0/1870792457 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4128063020 con 0x55b2a9c15720 2024-05-08T22:18:03.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.552+0000 7f4113400700 1 -- 172.21.15.131:0/1870792457 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4128098ed0 con 0x55b2a9c15720 2024-05-08T22:18:03.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.552+0000 7f41486fab80 10 client.16797 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:18:03.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.552+0000 7f41486fab80 10 client.16797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:18:03.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.552+0000 7f41486fab80 20 client.16797 populate_metadata read hostname 'smithi131' 2024-05-08T22:18:03.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.552+0000 7f41486fab80 1 --2- 172.21.15.131:0/1870792457 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b2a9c29000 0x55b2a9c2b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.552+0000 7f41486fab80 4 client.16797 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:18:03.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.552+0000 7f41486fab80 1 -- 172.21.15.131:0/1870792457 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b2a9bde810 con 0x55b2a9c29000 2024-05-08T22:18:03.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.553+0000 7f412de00700 1 --2- 172.21.15.131:0/1870792457 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b2a9c29000 0x55b2a9c2b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.553+0000 7f412de00700 1 --2- 172.21.15.131:0/1870792457 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b2a9c29000 0x55b2a9c2b420 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cae8c0ffc230879 server_cookie=c329163bf044385f in_seq=0 out_seq=0 2024-05-08T22:18:03.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.553+0000 7f4113400700 10 client.16797 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:18:03.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.554+0000 7f4113400700 1 -- 172.21.15.131:0/1870792457 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b2a9bde810 con 0x55b2a9c29000 2024-05-08T22:18:03.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.554+0000 7f4113400700 10 client.16797 handle_command_reply: tid=0 2024-05-08T22:18:03.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.554+0000 7f41486fab80 1 client.16797 shutdown 2024-05-08T22:18:03.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.554+0000 7f41486fab80 1 -- 172.21.15.131:0/1870792457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c15720 msgr2=0x55b2a9c190e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.554+0000 7f41486fab80 1 --2- 172.21.15.131:0/1870792457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c15720 0x55b2a9c190e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f412800c6f0 tx=0x7f41280995a0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.554+0000 7f41486fab80 1 -- 172.21.15.131:0/1870792457 shutdown_connections 2024-05-08T22:18:03.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.554+0000 7f41486fab80 1 --2- 172.21.15.131:0/1870792457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c15720 0x55b2a9c190e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.554+0000 7f41486fab80 1 --2- 172.21.15.131:0/1870792457 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2a9c0f890 0x55b2a9c1b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.554+0000 7f41486fab80 1 --2- 172.21.15.131:0/1870792457 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b2a9c29000 0x55b2a9c2b420 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.554+0000 7f41486fab80 1 --2- 172.21.15.131:0/1870792457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2a9c14e00 0x55b2a9c189f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.554+0000 7f41486fab80 1 -- 172.21.15.131:0/1870792457 >> 172.21.15.131:0/1870792457 conn(0x55b2a9b6d210 msgr2=0x55b2a9c095a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:03.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.554+0000 7f4113400700 0 client.16797 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:18:03.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.555+0000 7f41486fab80 1 -- 172.21.15.131:0/1870792457 shutdown_connections 2024-05-08T22:18:03.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.555+0000 7f41486fab80 1 -- 172.21.15.131:0/1870792457 wait complete. 2024-05-08T22:18:03.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.555+0000 7f41486fab80 20 client.16797 trim_cache size 0 max 16384 2024-05-08T22:18:03.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.568+0000 7f41486fab80 1 Processor -- start 2024-05-08T22:18:03.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.569+0000 7f41486fab80 1 -- start start 2024-05-08T22:18:03.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.569+0000 7f41486fab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c0f890 0x55b2a9c18ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.569+0000 7f41486fab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2a9c14e00 0x55b2a9c09d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.569+0000 7f41486fab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2a9c15720 0x55b2a9c12b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.569+0000 7f41486fab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a9c0b0c0 con 0x55b2a9c0f890 2024-05-08T22:18:03.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.569+0000 7f41486fab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a9bde810 con 0x55b2a9c15720 2024-05-08T22:18:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.569+0000 7f41486fab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a9c0b890 con 0x55b2a9c14e00 2024-05-08T22:18:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.569+0000 7f412de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c0f890 0x55b2a9c18ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.569+0000 7f412de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c0f890 0x55b2a9c18ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35094/0 (socket says 172.21.15.131:35094) 2024-05-08T22:18:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.569+0000 7f412de00700 1 -- 172.21.15.131:0/2023542894 learned_addr learned my addr 172.21.15.131:0/2023542894 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.569+0000 7f412e800700 1 --2- 172.21.15.131:0/2023542894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2a9c15720 0x55b2a9c12b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.569+0000 7f4127e00700 1 --2- 172.21.15.131:0/2023542894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2a9c14e00 0x55b2a9c09d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.570+0000 7f412de00700 1 -- 172.21.15.131:0/2023542894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2a9c14e00 msgr2=0x55b2a9c09d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.570+0000 7f412de00700 1 --2- 172.21.15.131:0/2023542894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2a9c14e00 0x55b2a9c09d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.570+0000 7f412de00700 1 -- 172.21.15.131:0/2023542894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2a9c15720 msgr2=0x55b2a9c12b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.570+0000 7f412de00700 1 --2- 172.21.15.131:0/2023542894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2a9c15720 0x55b2a9c12b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.570+0000 7f412de00700 1 -- 172.21.15.131:0/2023542894 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2a9b25560 con 0x55b2a9c0f890 2024-05-08T22:18:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.570+0000 7f4127e00700 1 --2- 172.21.15.131:0/2023542894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2a9c14e00 0x55b2a9c09d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:18:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.570+0000 7f412e800700 1 --2- 172.21.15.131:0/2023542894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2a9c15720 0x55b2a9c12b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:18:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.570+0000 7f412de00700 1 --2- 172.21.15.131:0/2023542894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c0f890 0x55b2a9c18ff0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4120005320 tx=0x7f412001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2cfa755f2ae3fe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.570+0000 7f411fa00700 1 -- 172.21.15.131:0/2023542894 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412001d040 con 0x55b2a9c0f890 2024-05-08T22:18:03.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.570+0000 7f411fa00700 1 -- 172.21.15.131:0/2023542894 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4120005bc0 con 0x55b2a9c0f890 2024-05-08T22:18:03.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.570+0000 7f411fa00700 1 -- 172.21.15.131:0/2023542894 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4120034550 con 0x55b2a9c0f890 2024-05-08T22:18:03.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.571+0000 7f41486fab80 1 -- 172.21.15.131:0/2023542894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c0f890 msgr2=0x55b2a9c18ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.571+0000 7f41486fab80 1 --2- 172.21.15.131:0/2023542894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c0f890 0x55b2a9c18ff0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4120005320 tx=0x7f412001a040 comp rx=0 tx=0).stop 2024-05-08T22:18:03.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.571+0000 7f41486fab80 1 -- 172.21.15.131:0/2023542894 shutdown_connections 2024-05-08T22:18:03.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.571+0000 7f41486fab80 1 --2- 172.21.15.131:0/2023542894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2a9c15720 0x55b2a9c12b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.571+0000 7f41486fab80 1 --2- 172.21.15.131:0/2023542894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c0f890 0x55b2a9c18ff0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.571+0000 7f41486fab80 1 --2- 172.21.15.131:0/2023542894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2a9c14e00 0x55b2a9c09d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.571+0000 7f41486fab80 1 -- 172.21.15.131:0/2023542894 >> 172.21.15.131:0/2023542894 conn(0x55b2a9b6d210 msgr2=0x55b2a9c2fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:03.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.571+0000 7f41486fab80 1 -- 172.21.15.131:0/2023542894 shutdown_connections 2024-05-08T22:18:03.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.571+0000 7f41486fab80 1 -- 172.21.15.131:0/2023542894 wait complete. 2024-05-08T22:18:03.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.571+0000 7f41486fab80 1 -- start start 2024-05-08T22:18:03.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.572+0000 7f41486fab80 1 Processor -- start 2024-05-08T22:18:03.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.572+0000 7f41486fab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2a9c0f890 0x55b2a9c20120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.572+0000 7f41486fab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c14e00 0x55b2a9c20660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.572+0000 7f41486fab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2a9c15720 0x55b2a9c1c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.572+0000 7f41486fab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a9c1c700 con 0x55b2a9c14e00 2024-05-08T22:18:03.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.572+0000 7f41486fab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a9c1c870 con 0x55b2a9c0f890 2024-05-08T22:18:03.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.572+0000 7f41486fab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a9c1ca30 con 0x55b2a9c15720 2024-05-08T22:18:03.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.572+0000 7f41486fab80 10 client.0 _ll_register_callbacks cb 0x7f4128095e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:18:03.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.573+0000 7f4127e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c14e00 0x55b2a9c20660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.573+0000 7f412de00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2a9c0f890 0x55b2a9c20120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.573+0000 7f412de00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2a9c0f890 0x55b2a9c20120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:53356/0 (socket says 172.21.15.131:53356) 2024-05-08T22:18:03.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.573+0000 7f4127e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c14e00 0x55b2a9c20660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35100/0 (socket says 172.21.15.131:35100) 2024-05-08T22:18:03.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.573+0000 7f412de00700 1 -- 172.21.15.131:0/4055820812 learned_addr learned my addr 172.21.15.131:0/4055820812 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:03.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.573+0000 7f412e800700 1 --2- 172.21.15.131:0/4055820812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2a9c15720 0x55b2a9c1c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.573+0000 7f412de00700 1 -- 172.21.15.131:0/4055820812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2a9c15720 msgr2=0x55b2a9c1c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:03.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.573+0000 7f412de00700 1 --2- 172.21.15.131:0/4055820812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2a9c15720 0x55b2a9c1c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.573+0000 7f412de00700 1 -- 172.21.15.131:0/4055820812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c14e00 msgr2=0x55b2a9c20660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:18:03.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.573+0000 7f412de00700 1 --2- 172.21.15.131:0/4055820812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c14e00 0x55b2a9c20660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.573+0000 7f412de00700 1 -- 172.21.15.131:0/4055820812 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b2a998cb60 con 0x55b2a9c0f890 2024-05-08T22:18:03.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.573+0000 7f4127e00700 1 --2- 172.21.15.131:0/4055820812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c14e00 0x55b2a9c20660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:18:03.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.573+0000 7f412de00700 1 --2- 172.21.15.131:0/4055820812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2a9c0f890 0x55b2a9c20120 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4120009350 tx=0x7f4120005450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15237387c05e54a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:03.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.573+0000 7f4113400700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-05-08T22:18:03.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.573+0000 7f4113400700 1 -- 172.21.15.131:0/4055820812 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412001d040 con 0x55b2a9c0f890 2024-05-08T22:18:03.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.574+0000 7f41486fab80 1 -- 172.21.15.131:0/4055820812 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b2a9b57ec0 con 0x55b2a9c0f890 2024-05-08T22:18:03.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.574+0000 7f4113400700 1 -- 172.21.15.131:0/4055820812 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9509+0+0 (secure 0 0 0) 0x7f412000b8f0 con 0x55b2a9c0f890 2024-05-08T22:18:03.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.574+0000 7f4113400700 1 -- 172.21.15.131:0/4055820812 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f412000bab0 con 0x55b2a9c0f890 2024-05-08T22:18:03.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.574+0000 7f41486fab80 10 client.26894 fetch_fsmap learned FSMap version 30 2024-05-08T22:18:03.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.574+0000 7f41486fab80 1 -- 172.21.15.131:0/4055820812 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b2a99e03b0 con 0x55b2a9c0f890 2024-05-08T22:18:03.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.574+0000 7f4113400700 1 -- 172.21.15.131:0/4055820812 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b2a9b57ec0 con 0x55b2a9c0f890 2024-05-08T22:18:03.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.574+0000 7f4113400700 1 -- 172.21.15.131:0/4055820812 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f412003c070 con 0x55b2a9c0f890 2024-05-08T22:18:03.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.574+0000 7f41486fab80 10 client.26894 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:18:03.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.574+0000 7f41486fab80 10 client.26894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T22:18:03.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.574+0000 7f41486fab80 20 client.26894 populate_metadata read hostname 'smithi131' 2024-05-08T22:18:03.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.574+0000 7f41486fab80 1 --2- 172.21.15.131:0/4055820812 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b2a9c7f280 0x55b2a9c816a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:03.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.574+0000 7f41486fab80 4 client.26894 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T22:18:03.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.574+0000 7f41486fab80 1 -- 172.21.15.131:0/4055820812 --> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b2a99adcf0 con 0x55b2a9c7f280 2024-05-08T22:18:03.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.575+0000 7f4127e00700 1 --2- 172.21.15.131:0/4055820812 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b2a9c7f280 0x55b2a9c816a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:03.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.575+0000 7f4113400700 10 client.26894 ms_handle_connect on v2:172.21.15.132:6838/4205858754 2024-05-08T22:18:03.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.575+0000 7f4127e00700 1 --2- 172.21.15.131:0/4055820812 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b2a9c7f280 0x55b2a9c816a0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abc1e061dbdfbfb3 server_cookie=34598fa476656c0c in_seq=0 out_seq=0 2024-05-08T22:18:03.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.575+0000 7f4113400700 1 -- 172.21.15.131:0/4055820812 <== mds.0 v2:172.21.15.132:6838/4205858754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b2a99adcf0 con 0x55b2a9c7f280 2024-05-08T22:18:03.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.575+0000 7f4113400700 10 client.26894 handle_command_reply: tid=0 2024-05-08T22:18:03.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.575+0000 7f41486fab80 1 client.26894 shutdown 2024-05-08T22:18:03.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.575+0000 7f41486fab80 1 -- 172.21.15.131:0/4055820812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2a9c0f890 msgr2=0x55b2a9c20120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.575+0000 7f41486fab80 1 --2- 172.21.15.131:0/4055820812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2a9c0f890 0x55b2a9c20120 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4120009350 tx=0x7f4120005450 comp rx=0 tx=0).stop 2024-05-08T22:18:03.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.575+0000 7f41486fab80 1 -- 172.21.15.131:0/4055820812 shutdown_connections 2024-05-08T22:18:03.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.575+0000 7f41486fab80 1 --2- 172.21.15.131:0/4055820812 >> [v2:172.21.15.132:6838/4205858754,v1:172.21.15.132:6839/4205858754] conn(0x55b2a9c7f280 0x55b2a9c816a0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.575+0000 7f41486fab80 1 --2- 172.21.15.131:0/4055820812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2a9c15720 0x55b2a9c1c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.575+0000 7f41486fab80 1 --2- 172.21.15.131:0/4055820812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2a9c0f890 0x55b2a9c20120 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.575+0000 7f41486fab80 1 --2- 172.21.15.131:0/4055820812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2a9c14e00 0x55b2a9c20660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.575+0000 7f4113400700 0 client.26894 ms_handle_reset on v2:172.21.15.132:6838/4205858754 2024-05-08T22:18:03.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.575+0000 7f41486fab80 1 -- 172.21.15.131:0/4055820812 >> 172.21.15.131:0/4055820812 conn(0x55b2a9b6d210 msgr2=0x55b2a9c12b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:03.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.576+0000 7f41486fab80 1 -- 172.21.15.131:0/4055820812 shutdown_connections 2024-05-08T22:18:03.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.576+0000 7f41486fab80 1 -- 172.21.15.131:0/4055820812 wait complete. 2024-05-08T22:18:03.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.576+0000 7f41486fab80 20 client.26894 trim_cache size 0 max 16384 2024-05-08T22:18:03.582 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-08T22:18:03.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.576+0000 7f4134c00700 1 -- 172.21.15.131:0/1048719866 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f41140a1290 msgr2=0x7f41140a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.576+0000 7f4134c00700 1 --2- 172.21.15.131:0/1048719866 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f41140a1290 0x7f41140a3750 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f413019abb0 tx=0x7f41180098d0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.576+0000 7f4134c00700 1 -- 172.21.15.131:0/1048719866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f413019df40 msgr2=0x7f413019e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:03.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.576+0000 7f4134c00700 1 --2- 172.21.15.131:0/1048719866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f413019df40 0x7f413019e3c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f412800efc0 tx=0x7f412800c3a0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.576+0000 7f4134c00700 1 -- 172.21.15.131:0/1048719866 shutdown_connections 2024-05-08T22:18:03.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.576+0000 7f4134c00700 1 --2- 172.21.15.131:0/1048719866 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f41140a1290 0x7f41140a3750 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.576+0000 7f4134c00700 1 --2- 172.21.15.131:0/1048719866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f413019df40 0x7f413019e3c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.576+0000 7f4134c00700 1 --2- 172.21.15.131:0/1048719866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4130101160 0x7f4130199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.576+0000 7f4134c00700 1 --2- 172.21.15.131:0/1048719866 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4130106280 0x7f4130199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:03.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.576+0000 7f4134c00700 1 -- 172.21.15.131:0/1048719866 >> 172.21.15.131:0/1048719866 conn(0x7f41300fc9c0 msgr2=0x7f41300fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:03.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.577+0000 7f4134c00700 1 -- 172.21.15.131:0/1048719866 shutdown_connections 2024-05-08T22:18:03.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:03.577+0000 7f4134c00700 1 -- 172.21.15.131:0/1048719866 wait complete. 2024-05-08T22:18:03.593 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:03 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/4085948671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:18:03.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:03 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/4085948671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:18:03.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:03 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/4085948671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:18:04.596 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-05-08T22:18:04.596 INFO:tasks.fwd_scrub:done joining 2024-05-08T22:18:04.597 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-08T22:18:04.607 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:18:04.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:04 smithi157 ceph-mon[77746]: pgmap v335: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-08T22:18:04.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.809+0000 7fa755400700 1 -- 172.21.15.131:0/798133886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa750101a00 msgr2=0x7fa750105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:04.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.809+0000 7fa755400700 1 --2- 172.21.15.131:0/798133886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa750101a00 0x7fa750105a40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa73c0098f0 tx=0x7fa73c009c00 comp rx=0 tx=0).stop 2024-05-08T22:18:04.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.809+0000 7fa755400700 1 -- 172.21.15.131:0/798133886 shutdown_connections 2024-05-08T22:18:04.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.809+0000 7fa755400700 1 --2- 172.21.15.131:0/798133886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa750106170 0x7fa75010dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:04.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.809+0000 7fa755400700 1 --2- 172.21.15.131:0/798133886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa750101050 0x7fa750101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:04.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.809+0000 7fa755400700 1 --2- 172.21.15.131:0/798133886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa750101a00 0x7fa750105a40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:04.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.809+0000 7fa755400700 1 -- 172.21.15.131:0/798133886 >> 172.21.15.131:0/798133886 conn(0x7fa7500fc8f0 msgr2=0x7fa7500fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:04.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.809+0000 7fa755400700 1 -- 172.21.15.131:0/798133886 shutdown_connections 2024-05-08T22:18:04.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.809+0000 7fa755400700 1 -- 172.21.15.131:0/798133886 wait complete. 2024-05-08T22:18:04.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.811+0000 7fa755400700 1 Processor -- start 2024-05-08T22:18:04.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.811+0000 7fa755400700 1 -- start start 2024-05-08T22:18:04.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.811+0000 7fa755400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa750101050 0x7fa7501995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:04.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.811+0000 7fa755400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa750106170 0x7fa750199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:04.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.812+0000 7fa755400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa75019de40 0x7fa75019e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:04.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.812+0000 7fa755400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75019e8c0 con 0x7fa75019de40 2024-05-08T22:18:04.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.812+0000 7fa755400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75019ea30 con 0x7fa750101050 2024-05-08T22:18:04.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.812+0000 7fa755400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75019ebf0 con 0x7fa750106170 2024-05-08T22:18:04.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.812+0000 7fa74e800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa750101050 0x7fa7501995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:04.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.812+0000 7fa74e800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa750101050 0x7fa7501995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:40736/0 (socket says 172.21.15.131:40736) 2024-05-08T22:18:04.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.812+0000 7fa74e800700 1 -- 172.21.15.131:0/151436004 learned_addr learned my addr 172.21.15.131:0/151436004 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:04.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.812+0000 7fa74f200700 1 --2- 172.21.15.131:0/151436004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa75019de40 0x7fa75019e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:04.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.812+0000 7fa74de00700 1 --2- 172.21.15.131:0/151436004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa750106170 0x7fa750199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:04.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.812+0000 7fa74e800700 1 -- 172.21.15.131:0/151436004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa750106170 msgr2=0x7fa750199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:04.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.812+0000 7fa74e800700 1 --2- 172.21.15.131:0/151436004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa750106170 0x7fa750199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:04.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.812+0000 7fa74e800700 1 -- 172.21.15.131:0/151436004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa75019de40 msgr2=0x7fa75019e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:04.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.812+0000 7fa74e800700 1 --2- 172.21.15.131:0/151436004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa75019de40 0x7fa75019e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:04.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.812+0000 7fa74e800700 1 -- 172.21.15.131:0/151436004 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa740009520 con 0x7fa750101050 2024-05-08T22:18:04.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.813+0000 7fa74e800700 1 --2- 172.21.15.131:0/151436004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa750101050 0x7fa7501995b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa74000eb30 tx=0x7fa74000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:04.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.813+0000 7fa747400700 1 -- 172.21.15.131:0/151436004 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa740009880 con 0x7fa750101050 2024-05-08T22:18:04.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.813+0000 7fa755400700 1 -- 172.21.15.131:0/151436004 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa748009d00 con 0x7fa750101050 2024-05-08T22:18:04.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.813+0000 7fa747400700 1 -- 172.21.15.131:0/151436004 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7400044e0 con 0x7fa750101050 2024-05-08T22:18:04.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.813+0000 7fa747400700 1 -- 172.21.15.131:0/151436004 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa740003e90 con 0x7fa750101050 2024-05-08T22:18:04.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.813+0000 7fa755400700 1 -- 172.21.15.131:0/151436004 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73c009520 con 0x7fa750101050 2024-05-08T22:18:04.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.814+0000 7fa755400700 1 -- 172.21.15.131:0/151436004 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa750102520 con 0x7fa750101050 2024-05-08T22:18:04.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.816+0000 7fa747400700 1 -- 172.21.15.131:0/151436004 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fa740004000 con 0x7fa750101050 2024-05-08T22:18:04.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.817+0000 7fa747400700 1 --2- 172.21.15.131:0/151436004 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa7340a1290 0x7fa7340a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:04.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.817+0000 7fa747400700 1 -- 172.21.15.131:0/151436004 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9509+0+0 (secure 0 0 0) 0x7fa740014070 con 0x7fa750101050 2024-05-08T22:18:04.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.818+0000 7fa74de00700 1 --2- 172.21.15.131:0/151436004 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa7340a1290 0x7fa7340a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:04.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.818+0000 7fa74de00700 1 --2- 172.21.15.131:0/151436004 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa7340a1290 0x7fa7340a3750 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa73c009810 tx=0x7fa73c019040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:04.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:04.820+0000 7fa747400700 1 -- 172.21.15.131:0/151436004 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa740096710 con 0x7fa750101050 2024-05-08T22:18:04.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:04 smithi132 ceph-mon[78180]: pgmap v335: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-08T22:18:04.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:04 smithi131 ceph-mon[37260]: pgmap v335: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-08T22:18:05.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.029+0000 7fa755400700 1 -- 172.21.15.131:0/151436004 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa75019a900 con 0x7fa750101050 2024-05-08T22:18:05.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.031+0000 7fa747400700 1 -- 172.21.15.131:0/151436004 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+25323 (secure 0 0 0) 0x7fa740099d30 con 0x7fa750101050 2024-05-08T22:18:05.032 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:18:05.033 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":97,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:17:42.698406+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:05:12.469858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:05:13.484446+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:05:17.016186+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/2704707055":"2024-05-09T22:07:17.096274+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6801/2485665042":"2024-05-09T22:07:17.096274+0000","172.21.15.131:0/126236497":"2024-05-09T22:07:17.096274+0000","172.21.15.131:0/25489954":"2024-05-09T22:07:17.096274+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/1773942606":"2024-05-09T22:07:17.096274+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/2485665042":"2024-05-09T22:07:17.096274+0000","172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3120268590":"2024-05-09T22:07:17.096274+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:18:05.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.033+0000 7fa755400700 1 -- 172.21.15.131:0/151436004 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa7340a1290 msgr2=0x7fa7340a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:05.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.033+0000 7fa755400700 1 --2- 172.21.15.131:0/151436004 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa7340a1290 0x7fa7340a3750 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa73c009810 tx=0x7fa73c019040 comp rx=0 tx=0).stop 2024-05-08T22:18:05.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.034+0000 7fa755400700 1 -- 172.21.15.131:0/151436004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa750101050 msgr2=0x7fa7501995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:05.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.034+0000 7fa755400700 1 --2- 172.21.15.131:0/151436004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa750101050 0x7fa7501995b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa74000eb30 tx=0x7fa74000ee40 comp rx=0 tx=0).stop 2024-05-08T22:18:05.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.034+0000 7fa755400700 1 -- 172.21.15.131:0/151436004 shutdown_connections 2024-05-08T22:18:05.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.034+0000 7fa755400700 1 --2- 172.21.15.131:0/151436004 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa7340a1290 0x7fa7340a3750 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.034+0000 7fa755400700 1 --2- 172.21.15.131:0/151436004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa75019de40 0x7fa75019e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.034+0000 7fa755400700 1 --2- 172.21.15.131:0/151436004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa750101050 0x7fa7501995b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.034+0000 7fa755400700 1 --2- 172.21.15.131:0/151436004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa750106170 0x7fa750199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.034+0000 7fa755400700 1 -- 172.21.15.131:0/151436004 >> 172.21.15.131:0/151436004 conn(0x7fa7500fc8f0 msgr2=0x7fa7500fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:05.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.034+0000 7fa755400700 1 -- 172.21.15.131:0/151436004 shutdown_connections 2024-05-08T22:18:05.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.034+0000 7fa755400700 1 -- 172.21.15.131:0/151436004 wait complete. 2024-05-08T22:18:05.054 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:01:41.271078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:05:12.469858+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:05:13.484446+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 95, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:05:17.016186+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 95, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:18:05.055 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:18:05.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.251+0000 7f3152600700 1 -- 172.21.15.131:0/3980883719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f314c10d7b0 msgr2=0x7f314c10fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:05.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.251+0000 7f3152600700 1 --2- 172.21.15.131:0/3980883719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f314c10d7b0 0x7f314c10fbe0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f314400b0a0 tx=0x7f314400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.251+0000 7f3152600700 1 -- 172.21.15.131:0/3980883719 shutdown_connections 2024-05-08T22:18:05.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.251+0000 7f3152600700 1 --2- 172.21.15.131:0/3980883719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f314c10d7b0 0x7f314c10fbe0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.251+0000 7f3152600700 1 --2- 172.21.15.131:0/3980883719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f314c073370 0x7f314c073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.251+0000 7f3152600700 1 --2- 172.21.15.131:0/3980883719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f314c073c90 0x7f314c10d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.251+0000 7f3152600700 1 -- 172.21.15.131:0/3980883719 >> 172.21.15.131:0/3980883719 conn(0x7f314c0fc9e0 msgr2=0x7f314c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:05.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.251+0000 7f3152600700 1 -- 172.21.15.131:0/3980883719 shutdown_connections 2024-05-08T22:18:05.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.251+0000 7f3152600700 1 -- 172.21.15.131:0/3980883719 wait complete. 2024-05-08T22:18:05.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.252+0000 7f3152600700 1 Processor -- start 2024-05-08T22:18:05.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.252+0000 7f3152600700 1 -- start start 2024-05-08T22:18:05.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.252+0000 7f3152600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f314c073370 0x7f314c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:05.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.252+0000 7f3152600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f314c073c90 0x7f314c199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:05.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.252+0000 7f3152600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f314c10d7b0 0x7f314c19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:05.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.252+0000 7f3152600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c19d580 con 0x7f314c073c90 2024-05-08T22:18:05.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.252+0000 7f3152600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c19d6f0 con 0x7f314c10d7b0 2024-05-08T22:18:05.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.252+0000 7f3152600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c19d8b0 con 0x7f314c073370 2024-05-08T22:18:05.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.253+0000 7f314b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f314c073370 0x7f314c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:05.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.253+0000 7f314aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f314c073c90 0x7f314c199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:05.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.253+0000 7f314b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f314c073370 0x7f314c1997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43506/0 (socket says 172.21.15.131:43506) 2024-05-08T22:18:05.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.253+0000 7f314b400700 1 -- 172.21.15.131:0/3826552331 learned_addr learned my addr 172.21.15.131:0/3826552331 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:05.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.253+0000 7f314be00700 1 --2- 172.21.15.131:0/3826552331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f314c10d7b0 0x7f314c19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:05.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.253+0000 7f314b400700 1 -- 172.21.15.131:0/3826552331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f314c10d7b0 msgr2=0x7f314c19cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:05.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.253+0000 7f314b400700 1 --2- 172.21.15.131:0/3826552331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f314c10d7b0 0x7f314c19cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.253+0000 7f314b400700 1 -- 172.21.15.131:0/3826552331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f314c073c90 msgr2=0x7f314c199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:05.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.253+0000 7f314b400700 1 --2- 172.21.15.131:0/3826552331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f314c073c90 0x7f314c199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.253+0000 7f314b400700 1 -- 172.21.15.131:0/3826552331 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3144009d00 con 0x7f314c073370 2024-05-08T22:18:05.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.253+0000 7f314aa00700 1 --2- 172.21.15.131:0/3826552331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f314c073c90 0x7f314c199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:18:05.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.253+0000 7f314b400700 1 --2- 172.21.15.131:0/3826552331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f314c073370 0x7f314c1997a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f313c00e9c0 tx=0x7f313c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:05.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.253+0000 7f3143e00700 1 -- 172.21.15.131:0/3826552331 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c0097b0 con 0x7f314c073370 2024-05-08T22:18:05.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.254+0000 7f3143e00700 1 -- 172.21.15.131:0/3826552331 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f313c004cf0 con 0x7f314c073370 2024-05-08T22:18:05.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.254+0000 7f3152600700 1 -- 172.21.15.131:0/3826552331 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f314c19db90 con 0x7f314c073370 2024-05-08T22:18:05.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.254+0000 7f3143e00700 1 -- 172.21.15.131:0/3826552331 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c005600 con 0x7f314c073370 2024-05-08T22:18:05.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.254+0000 7f3152600700 1 -- 172.21.15.131:0/3826552331 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f314c1a64f0 con 0x7f314c073370 2024-05-08T22:18:05.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.255+0000 7f3152600700 1 -- 172.21.15.131:0/3826552331 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f314c1089c0 con 0x7f314c073370 2024-05-08T22:18:05.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.257+0000 7f3143e00700 1 -- 172.21.15.131:0/3826552331 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f313c00cc60 con 0x7f314c073370 2024-05-08T22:18:05.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.258+0000 7f3143e00700 1 --2- 172.21.15.131:0/3826552331 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f31300a1290 0x7f31300a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:05.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.258+0000 7f3143e00700 1 -- 172.21.15.131:0/3826552331 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9509+0+0 (secure 0 0 0) 0x7f313c014070 con 0x7f314c073370 2024-05-08T22:18:05.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.258+0000 7f314aa00700 1 --2- 172.21.15.131:0/3826552331 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f31300a1290 0x7f31300a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:05.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.259+0000 7f314aa00700 1 --2- 172.21.15.131:0/3826552331 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f31300a1290 0x7f31300a3750 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f314c19aca0 tx=0x7f3138009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:05.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.261+0000 7f3143e00700 1 -- 172.21.15.131:0/3826552331 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f313c096600 con 0x7f314c073370 2024-05-08T22:18:05.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.469+0000 7f3152600700 1 -- 172.21.15.131:0/3826552331 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f314c19aa50 con 0x7f314c073370 2024-05-08T22:18:05.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.470+0000 7f3143e00700 1 -- 172.21.15.131:0/3826552331 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) v1 ==== 93+0+10 (secure 0 0 0) 0x7f313c099c20 con 0x7f314c073370 2024-05-08T22:18:05.471 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-08T22:18:05.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.472+0000 7f3152600700 1 -- 172.21.15.131:0/3826552331 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f31300a1290 msgr2=0x7f31300a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:05.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.472+0000 7f3152600700 1 --2- 172.21.15.131:0/3826552331 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f31300a1290 0x7f31300a3750 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f314c19aca0 tx=0x7f3138009260 comp rx=0 tx=0).stop 2024-05-08T22:18:05.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.472+0000 7f3152600700 1 -- 172.21.15.131:0/3826552331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f314c073370 msgr2=0x7f314c1997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:05.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.472+0000 7f3152600700 1 --2- 172.21.15.131:0/3826552331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f314c073370 0x7f314c1997a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f313c00e9c0 tx=0x7f313c00ecd0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.472+0000 7f3152600700 1 -- 172.21.15.131:0/3826552331 shutdown_connections 2024-05-08T22:18:05.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.472+0000 7f3152600700 1 --2- 172.21.15.131:0/3826552331 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f31300a1290 0x7f31300a3750 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.472+0000 7f3152600700 1 --2- 172.21.15.131:0/3826552331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f314c10d7b0 0x7f314c19cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.472+0000 7f3152600700 1 --2- 172.21.15.131:0/3826552331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f314c073370 0x7f314c1997a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.472+0000 7f3152600700 1 --2- 172.21.15.131:0/3826552331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f314c073c90 0x7f314c199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.472+0000 7f3152600700 1 -- 172.21.15.131:0/3826552331 >> 172.21.15.131:0/3826552331 conn(0x7f314c0fc9e0 msgr2=0x7f314c0fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:05.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.472+0000 7f3152600700 1 -- 172.21.15.131:0/3826552331 shutdown_connections 2024-05-08T22:18:05.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.473+0000 7f3152600700 1 -- 172.21.15.131:0/3826552331 wait complete. 2024-05-08T22:18:05.492 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:18:05.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.688+0000 7f5476a00700 1 -- 172.21.15.131:0/1994285531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5470101b30 msgr2=0x7f5470105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:05.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.688+0000 7f5476a00700 1 --2- 172.21.15.131:0/1994285531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5470101b30 0x7f5470105b70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f545c009990 tx=0x7f545c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.688+0000 7f5476a00700 1 -- 172.21.15.131:0/1994285531 shutdown_connections 2024-05-08T22:18:05.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.688+0000 7f5476a00700 1 --2- 172.21.15.131:0/1994285531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54701062a0 0x7f547010de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.688+0000 7f5476a00700 1 --2- 172.21.15.131:0/1994285531 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5470101180 0x7f5470101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.688+0000 7f5476a00700 1 --2- 172.21.15.131:0/1994285531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5470101b30 0x7f5470105b70 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.688+0000 7f5476a00700 1 -- 172.21.15.131:0/1994285531 >> 172.21.15.131:0/1994285531 conn(0x7f54700fc9e0 msgr2=0x7f54700fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:05.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.688+0000 7f5476a00700 1 -- 172.21.15.131:0/1994285531 shutdown_connections 2024-05-08T22:18:05.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.688+0000 7f5476a00700 1 -- 172.21.15.131:0/1994285531 wait complete. 2024-05-08T22:18:05.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.689+0000 7f5476a00700 1 Processor -- start 2024-05-08T22:18:05.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.689+0000 7f5476a00700 1 -- start start 2024-05-08T22:18:05.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.689+0000 7f5476a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5470101180 0x7f5470199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:05.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.689+0000 7f5476a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54701062a0 0x7f5470199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:05.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.690+0000 7f5476a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f547019df30 0x7f547019e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:05.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.690+0000 7f5476a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547019e9b0 con 0x7f547019df30 2024-05-08T22:18:05.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.690+0000 7f5476a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547019eb20 con 0x7f5470101180 2024-05-08T22:18:05.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.690+0000 7f5476a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547019ece0 con 0x7f54701062a0 2024-05-08T22:18:05.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.690+0000 7f5474a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f547019df30 0x7f547019e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:05.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.690+0000 7f546f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54701062a0 0x7f5470199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:05.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.690+0000 7f5474a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f547019df30 0x7f547019e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60980/0 (socket says 172.21.15.131:60980) 2024-05-08T22:18:05.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.690+0000 7f5474a00700 1 -- 172.21.15.131:0/3482517149 learned_addr learned my addr 172.21.15.131:0/3482517149 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:05.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.690+0000 7f546fe00700 1 --2- 172.21.15.131:0/3482517149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5470101180 0x7f5470199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:05.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.690+0000 7f5474a00700 1 -- 172.21.15.131:0/3482517149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54701062a0 msgr2=0x7f5470199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:05.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.690+0000 7f5474a00700 1 --2- 172.21.15.131:0/3482517149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54701062a0 0x7f5470199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.690+0000 7f5474a00700 1 -- 172.21.15.131:0/3482517149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5470101180 msgr2=0x7f5470199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:05.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.690+0000 7f5474a00700 1 --2- 172.21.15.131:0/3482517149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5470101180 0x7f5470199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.690+0000 7f5474a00700 1 -- 172.21.15.131:0/3482517149 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c0095f0 con 0x7f547019df30 2024-05-08T22:18:05.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.691+0000 7f5474a00700 1 --2- 172.21.15.131:0/3482517149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f547019df30 0x7f547019e3b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5468007ee0 tx=0x7f546800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:05.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.691+0000 7f546cc00700 1 -- 172.21.15.131:0/3482517149 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5468017070 con 0x7f547019df30 2024-05-08T22:18:05.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.691+0000 7f546cc00700 1 -- 172.21.15.131:0/3482517149 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f546800f040 con 0x7f547019df30 2024-05-08T22:18:05.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.691+0000 7f546cc00700 1 -- 172.21.15.131:0/3482517149 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5468005580 con 0x7f547019df30 2024-05-08T22:18:05.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.691+0000 7f5476a00700 1 -- 172.21.15.131:0/3482517149 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54701a6240 con 0x7f547019df30 2024-05-08T22:18:05.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.691+0000 7f5476a00700 1 -- 172.21.15.131:0/3482517149 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54701a6720 con 0x7f547019df30 2024-05-08T22:18:05.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.692+0000 7f5476a00700 1 -- 172.21.15.131:0/3482517149 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5470102650 con 0x7f547019df30 2024-05-08T22:18:05.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.694+0000 7f546cc00700 1 -- 172.21.15.131:0/3482517149 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f546800db20 con 0x7f547019df30 2024-05-08T22:18:05.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.695+0000 7f546cc00700 1 --2- 172.21.15.131:0/3482517149 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f54540a1310 0x7f54540a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:05.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.695+0000 7f546cc00700 1 -- 172.21.15.131:0/3482517149 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9509+0+0 (secure 0 0 0) 0x7f54680cc710 con 0x7f547019df30 2024-05-08T22:18:05.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.695+0000 7f546fe00700 1 --2- 172.21.15.131:0/3482517149 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f54540a1310 0x7f54540a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:05.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.696+0000 7f546fe00700 1 --2- 172.21.15.131:0/3482517149 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f54540a1310 0x7f54540a37d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5460009c60 tx=0x7f5460009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:05.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.700+0000 7f546cc00700 1 -- 172.21.15.131:0/3482517149 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5468095780 con 0x7f547019df30 2024-05-08T22:18:05.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:05 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/151436004' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:18:05.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:05 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/151436004' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:18:05.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:05 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/151436004' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:18:05.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.907+0000 7f5476a00700 1 -- 172.21.15.131:0/3482517149 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5470101b30 con 0x7f547019df30 2024-05-08T22:18:05.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.907+0000 7f546cc00700 1 -- 172.21.15.131:0/3482517149 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5468098da0 con 0x7f547019df30 2024-05-08T22:18:05.908 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-08T22:18:05.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.910+0000 7f5476a00700 1 -- 172.21.15.131:0/3482517149 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f54540a1310 msgr2=0x7f54540a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:05.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.910+0000 7f5476a00700 1 --2- 172.21.15.131:0/3482517149 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f54540a1310 0x7f54540a37d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5460009c60 tx=0x7f5460009260 comp rx=0 tx=0).stop 2024-05-08T22:18:05.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.910+0000 7f5476a00700 1 -- 172.21.15.131:0/3482517149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f547019df30 msgr2=0x7f547019e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:05.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.910+0000 7f5476a00700 1 --2- 172.21.15.131:0/3482517149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f547019df30 0x7f547019e3b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5468007ee0 tx=0x7f546800d3b0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.910+0000 7f5476a00700 1 -- 172.21.15.131:0/3482517149 shutdown_connections 2024-05-08T22:18:05.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.910+0000 7f5476a00700 1 --2- 172.21.15.131:0/3482517149 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f54540a1310 0x7f54540a37d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.910+0000 7f5476a00700 1 --2- 172.21.15.131:0/3482517149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f547019df30 0x7f547019e3b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.910+0000 7f5476a00700 1 --2- 172.21.15.131:0/3482517149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5470101180 0x7f5470199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.910+0000 7f5476a00700 1 --2- 172.21.15.131:0/3482517149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54701062a0 0x7f5470199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:05.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.910+0000 7f5476a00700 1 -- 172.21.15.131:0/3482517149 >> 172.21.15.131:0/3482517149 conn(0x7f54700fc9e0 msgr2=0x7f54700fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:05.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.910+0000 7f5476a00700 1 -- 172.21.15.131:0/3482517149 shutdown_connections 2024-05-08T22:18:05.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:05.910+0000 7f5476a00700 1 -- 172.21.15.131:0/3482517149 wait complete. 2024-05-08T22:18:05.931 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:18:06.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.127+0000 7f4c3e800700 1 -- 172.21.15.131:0/939326418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c38101b30 msgr2=0x7f4c38105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:06.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.127+0000 7f4c3e800700 1 --2- 172.21.15.131:0/939326418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c38101b30 0x7f4c38105b70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4c24009990 tx=0x7f4c24009ca0 comp rx=0 tx=0).stop 2024-05-08T22:18:06.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.128+0000 7f4c3e800700 1 -- 172.21.15.131:0/939326418 shutdown_connections 2024-05-08T22:18:06.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.128+0000 7f4c3e800700 1 --2- 172.21.15.131:0/939326418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c381062a0 0x7f4c3810de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:06.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.128+0000 7f4c3e800700 1 --2- 172.21.15.131:0/939326418 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c38101180 0x7f4c38101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:06.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.128+0000 7f4c3e800700 1 --2- 172.21.15.131:0/939326418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c38101b30 0x7f4c38105b70 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:06.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.128+0000 7f4c3e800700 1 -- 172.21.15.131:0/939326418 >> 172.21.15.131:0/939326418 conn(0x7f4c380fc9e0 msgr2=0x7f4c380fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:06.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.128+0000 7f4c3e800700 1 -- 172.21.15.131:0/939326418 shutdown_connections 2024-05-08T22:18:06.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.128+0000 7f4c3e800700 1 -- 172.21.15.131:0/939326418 wait complete. 2024-05-08T22:18:06.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.129+0000 7f4c3e800700 1 Processor -- start 2024-05-08T22:18:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.129+0000 7f4c3e800700 1 -- start start 2024-05-08T22:18:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.129+0000 7f4c3e800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c38101180 0x7f4c380732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.129+0000 7f4c3e800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c381062a0 0x7f4c380737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.129+0000 7f4c3e800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c38073d20 0x7f4c381a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.129+0000 7f4c3e800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c381a5c70 con 0x7f4c38101180 2024-05-08T22:18:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.129+0000 7f4c3e800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c381a5de0 con 0x7f4c38073d20 2024-05-08T22:18:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.129+0000 7f4c3e800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c381a5fa0 con 0x7f4c381062a0 2024-05-08T22:18:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.129+0000 7f4c37400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c38101180 0x7f4c380732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.129+0000 7f4c37400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c38101180 0x7f4c380732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60998/0 (socket says 172.21.15.131:60998) 2024-05-08T22:18:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.129+0000 7f4c37400700 1 -- 172.21.15.131:0/2069296223 learned_addr learned my addr 172.21.15.131:0/2069296223 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.130+0000 7f4c36a00700 1 --2- 172.21.15.131:0/2069296223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c381062a0 0x7f4c380737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.130+0000 7f4c37e00700 1 --2- 172.21.15.131:0/2069296223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c38073d20 0x7f4c381a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.130+0000 7f4c37400700 1 -- 172.21.15.131:0/2069296223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c381062a0 msgr2=0x7f4c380737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.130+0000 7f4c37400700 1 --2- 172.21.15.131:0/2069296223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c381062a0 0x7f4c380737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:06.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.130+0000 7f4c37400700 1 -- 172.21.15.131:0/2069296223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c38073d20 msgr2=0x7f4c381a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:06.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.130+0000 7f4c37400700 1 --2- 172.21.15.131:0/2069296223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c38073d20 0x7f4c381a5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:06.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.130+0000 7f4c37400700 1 -- 172.21.15.131:0/2069296223 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c240095f0 con 0x7f4c38101180 2024-05-08T22:18:06.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.130+0000 7f4c37e00700 1 --2- 172.21.15.131:0/2069296223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c38073d20 0x7f4c381a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:18:06.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.130+0000 7f4c37400700 1 --2- 172.21.15.131:0/2069296223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c38101180 0x7f4c380732a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800e9a0 tx=0x7f4c2800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:06.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.130+0000 7f4c2fe00700 1 -- 172.21.15.131:0/2069296223 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c28009790 con 0x7f4c38101180 2024-05-08T22:18:06.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.130+0000 7f4c2fe00700 1 -- 172.21.15.131:0/2069296223 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c2800cbd0 con 0x7f4c38101180 2024-05-08T22:18:06.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.130+0000 7f4c2fe00700 1 -- 172.21.15.131:0/2069296223 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c28018680 con 0x7f4c38101180 2024-05-08T22:18:06.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.130+0000 7f4c3e800700 1 -- 172.21.15.131:0/2069296223 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c381a6280 con 0x7f4c38101180 2024-05-08T22:18:06.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.131+0000 7f4c3e800700 1 -- 172.21.15.131:0/2069296223 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c381a6710 con 0x7f4c38101180 2024-05-08T22:18:06.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.132+0000 7f4c3e800700 1 -- 172.21.15.131:0/2069296223 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c38109570 con 0x7f4c38101180 2024-05-08T22:18:06.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.133+0000 7f4c2fe00700 1 -- 172.21.15.131:0/2069296223 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f4c2800cd40 con 0x7f4c38101180 2024-05-08T22:18:06.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.135+0000 7f4c2fe00700 1 --2- 172.21.15.131:0/2069296223 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f4c1c0a1290 0x7f4c1c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:06.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.136+0000 7f4c2fe00700 1 -- 172.21.15.131:0/2069296223 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9509+0+0 (secure 0 0 0) 0x7f4c28014070 con 0x7f4c38101180 2024-05-08T22:18:06.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.136+0000 7f4c36a00700 1 --2- 172.21.15.131:0/2069296223 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f4c1c0a1290 0x7f4c1c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:06.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.136+0000 7f4c36a00700 1 --2- 172.21.15.131:0/2069296223 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f4c1c0a1290 0x7f4c1c0a3750 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4c38074d50 tx=0x7f4c24005fd0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:06.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.140+0000 7f4c2fe00700 1 -- 172.21.15.131:0/2069296223 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4c28052d60 con 0x7f4c38101180 2024-05-08T22:18:06.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.346+0000 7f4c3e800700 1 -- 172.21.15.131:0/2069296223 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4c38074780 con 0x7f4c38101180 2024-05-08T22:18:06.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.347+0000 7f4c2fe00700 1 -- 172.21.15.131:0/2069296223 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4c280963b0 con 0x7f4c38101180 2024-05-08T22:18:06.348 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-08T22:18:06.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.349+0000 7f4c3e800700 1 -- 172.21.15.131:0/2069296223 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f4c1c0a1290 msgr2=0x7f4c1c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:06.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.349+0000 7f4c3e800700 1 --2- 172.21.15.131:0/2069296223 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f4c1c0a1290 0x7f4c1c0a3750 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4c38074d50 tx=0x7f4c24005fd0 comp rx=0 tx=0).stop 2024-05-08T22:18:06.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.349+0000 7f4c3e800700 1 -- 172.21.15.131:0/2069296223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c38101180 msgr2=0x7f4c380732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:06.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.349+0000 7f4c3e800700 1 --2- 172.21.15.131:0/2069296223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c38101180 0x7f4c380732a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800e9a0 tx=0x7f4c2800ecb0 comp rx=0 tx=0).stop 2024-05-08T22:18:06.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.350+0000 7f4c3e800700 1 -- 172.21.15.131:0/2069296223 shutdown_connections 2024-05-08T22:18:06.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.350+0000 7f4c3e800700 1 --2- 172.21.15.131:0/2069296223 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f4c1c0a1290 0x7f4c1c0a3750 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:06.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.350+0000 7f4c3e800700 1 --2- 172.21.15.131:0/2069296223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c38073d20 0x7f4c381a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:06.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.350+0000 7f4c3e800700 1 --2- 172.21.15.131:0/2069296223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c38101180 0x7f4c380732a0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:06.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.350+0000 7f4c3e800700 1 --2- 172.21.15.131:0/2069296223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c381062a0 0x7f4c380737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:06.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.350+0000 7f4c3e800700 1 -- 172.21.15.131:0/2069296223 >> 172.21.15.131:0/2069296223 conn(0x7f4c380fc9e0 msgr2=0x7f4c380fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:06.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.350+0000 7f4c3e800700 1 -- 172.21.15.131:0/2069296223 shutdown_connections 2024-05-08T22:18:06.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.350+0000 7f4c3e800700 1 -- 172.21.15.131:0/2069296223 wait complete. 2024-05-08T22:18:06.369 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:18:06.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.565+0000 7fa181a00700 1 -- 172.21.15.131:0/922470357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa17c101b10 msgr2=0x7fa17c105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:06.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.565+0000 7fa181a00700 1 --2- 172.21.15.131:0/922470357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa17c101b10 0x7fa17c105b50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa168009990 tx=0x7fa168009ca0 comp rx=0 tx=0).stop 2024-05-08T22:18:06.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.565+0000 7fa181a00700 1 -- 172.21.15.131:0/922470357 shutdown_connections 2024-05-08T22:18:06.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.565+0000 7fa181a00700 1 --2- 172.21.15.131:0/922470357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa17c106280 0x7fa17c10de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:06.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.565+0000 7fa181a00700 1 --2- 172.21.15.131:0/922470357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa17c101160 0x7fa17c101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:06.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.565+0000 7fa181a00700 1 --2- 172.21.15.131:0/922470357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa17c101b10 0x7fa17c105b50 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:06.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.565+0000 7fa181a00700 1 -- 172.21.15.131:0/922470357 >> 172.21.15.131:0/922470357 conn(0x7fa17c0fc9c0 msgr2=0x7fa17c0fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:06.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.565+0000 7fa181a00700 1 -- 172.21.15.131:0/922470357 shutdown_connections 2024-05-08T22:18:06.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.565+0000 7fa181a00700 1 -- 172.21.15.131:0/922470357 wait complete. 2024-05-08T22:18:06.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.566+0000 7fa181a00700 1 Processor -- start 2024-05-08T22:18:06.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.566+0000 7fa181a00700 1 -- start start 2024-05-08T22:18:06.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.567+0000 7fa181a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa17c101160 0x7fa17c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:06.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.567+0000 7fa181a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa17c106280 0x7fa17c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:06.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.567+0000 7fa181a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa17c073d20 0x7fa17c1a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:06.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.567+0000 7fa181a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c1a5c00 con 0x7fa17c106280 2024-05-08T22:18:06.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.567+0000 7fa181a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c1a5d40 con 0x7fa17c101160 2024-05-08T22:18:06.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.567+0000 7fa181a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c1a5f00 con 0x7fa17c073d20 2024-05-08T22:18:06.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.567+0000 7fa17a000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa17c106280 0x7fa17c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:06.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.567+0000 7fa17aa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa17c101160 0x7fa17c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:06.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.567+0000 7fa17a000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa17c106280 0x7fa17c0737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:32790/0 (socket says 172.21.15.131:32790) 2024-05-08T22:18:06.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.567+0000 7fa17a000700 1 -- 172.21.15.131:0/3732064189 learned_addr learned my addr 172.21.15.131:0/3732064189 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:06.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.567+0000 7fa17b400700 1 --2- 172.21.15.131:0/3732064189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa17c073d20 0x7fa17c1a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:06.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.567+0000 7fa17a000700 1 -- 172.21.15.131:0/3732064189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa17c073d20 msgr2=0x7fa17c1a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:06.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.567+0000 7fa17a000700 1 --2- 172.21.15.131:0/3732064189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa17c073d20 0x7fa17c1a5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:06.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.567+0000 7fa17a000700 1 -- 172.21.15.131:0/3732064189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa17c101160 msgr2=0x7fa17c0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:06.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.567+0000 7fa17a000700 1 --2- 172.21.15.131:0/3732064189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa17c101160 0x7fa17c0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:06.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.567+0000 7fa17a000700 1 -- 172.21.15.131:0/3732064189 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1680095f0 con 0x7fa17c106280 2024-05-08T22:18:06.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.567+0000 7fa17aa00700 1 --2- 172.21.15.131:0/3732064189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa17c101160 0x7fa17c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:18:06.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.568+0000 7fa17a000700 1 --2- 172.21.15.131:0/3732064189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa17c106280 0x7fa17c0737e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa168000c00 tx=0x7fa168004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:06.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.568+0000 7fa173400700 1 -- 172.21.15.131:0/3732064189 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16801d070 con 0x7fa17c106280 2024-05-08T22:18:06.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.568+0000 7fa173400700 1 -- 172.21.15.131:0/3732064189 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa168021410 con 0x7fa17c106280 2024-05-08T22:18:06.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.568+0000 7fa181a00700 1 -- 172.21.15.131:0/3732064189 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa17c1a6180 con 0x7fa17c106280 2024-05-08T22:18:06.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.568+0000 7fa173400700 1 -- 172.21.15.131:0/3732064189 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa168017650 con 0x7fa17c106280 2024-05-08T22:18:06.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.568+0000 7fa181a00700 1 -- 172.21.15.131:0/3732064189 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa17c1a65a0 con 0x7fa17c106280 2024-05-08T22:18:06.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.569+0000 7fa181a00700 1 -- 172.21.15.131:0/3732064189 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa17c109550 con 0x7fa17c106280 2024-05-08T22:18:06.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.571+0000 7fa173400700 1 -- 172.21.15.131:0/3732064189 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fa168021580 con 0x7fa17c106280 2024-05-08T22:18:06.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.572+0000 7fa173400700 1 --2- 172.21.15.131:0/3732064189 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa1600a1290 0x7fa1600a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:06.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.572+0000 7fa173400700 1 -- 172.21.15.131:0/3732064189 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9509+0+0 (secure 0 0 0) 0x7fa1680ce320 con 0x7fa17c106280 2024-05-08T22:18:06.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.572+0000 7fa17aa00700 1 --2- 172.21.15.131:0/3732064189 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa1600a1290 0x7fa1600a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:06.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.573+0000 7fa17aa00700 1 --2- 172.21.15.131:0/3732064189 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa1600a1290 0x7fa1600a3750 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa16c011490 tx=0x7fa16c009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:06.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.576+0000 7fa173400700 1 -- 172.21.15.131:0/3732064189 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa168097420 con 0x7fa17c106280 2024-05-08T22:18:06.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:06 smithi131 ceph-mon[37260]: pgmap v336: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-05-08T22:18:06.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:06 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3826552331' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:18:06.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:06 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3482517149' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:18:06.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:06 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2069296223' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:18:06.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:06 smithi157 ceph-mon[77746]: pgmap v336: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-05-08T22:18:06.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:06 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3826552331' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:18:06.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:06 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3482517149' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:18:06.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:06 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2069296223' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:18:06.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.786+0000 7fa181a00700 1 -- 172.21.15.131:0/3732064189 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa17c074730 con 0x7fa17c106280 2024-05-08T22:18:06.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.787+0000 7fa173400700 1 -- 172.21.15.131:0/3732064189 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v97) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa16809aa40 con 0x7fa17c106280 2024-05-08T22:18:06.788 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-08T22:18:06.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.790+0000 7fa181a00700 1 -- 172.21.15.131:0/3732064189 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa1600a1290 msgr2=0x7fa1600a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:06.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.790+0000 7fa181a00700 1 --2- 172.21.15.131:0/3732064189 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa1600a1290 0x7fa1600a3750 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa16c011490 tx=0x7fa16c009260 comp rx=0 tx=0).stop 2024-05-08T22:18:06.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.790+0000 7fa181a00700 1 -- 172.21.15.131:0/3732064189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa17c106280 msgr2=0x7fa17c0737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:06.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.790+0000 7fa181a00700 1 --2- 172.21.15.131:0/3732064189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa17c106280 0x7fa17c0737e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa168000c00 tx=0x7fa168004730 comp rx=0 tx=0).stop 2024-05-08T22:18:06.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.790+0000 7fa181a00700 1 -- 172.21.15.131:0/3732064189 shutdown_connections 2024-05-08T22:18:06.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.790+0000 7fa181a00700 1 --2- 172.21.15.131:0/3732064189 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa1600a1290 0x7fa1600a3750 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:06.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.790+0000 7fa181a00700 1 --2- 172.21.15.131:0/3732064189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa17c073d20 0x7fa17c1a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:06.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.790+0000 7fa181a00700 1 --2- 172.21.15.131:0/3732064189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa17c101160 0x7fa17c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:06.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.790+0000 7fa181a00700 1 --2- 172.21.15.131:0/3732064189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa17c106280 0x7fa17c0737e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:06.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.790+0000 7fa181a00700 1 -- 172.21.15.131:0/3732064189 >> 172.21.15.131:0/3732064189 conn(0x7fa17c0fc9c0 msgr2=0x7fa17c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:06.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.790+0000 7fa181a00700 1 -- 172.21.15.131:0/3732064189 shutdown_connections 2024-05-08T22:18:06.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:06.790+0000 7fa181a00700 1 -- 172.21.15.131:0/3732064189 wait complete. 2024-05-08T22:18:06.810 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-08T22:18:06.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:06 smithi132 ceph-mon[78180]: pgmap v336: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-05-08T22:18:06.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:06 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3826552331' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:18:06.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:06 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3482517149' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:18:06.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:06 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2069296223' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:18:07.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.004+0000 7f40ee400700 1 -- 172.21.15.131:0/2460217948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40e8101050 msgr2=0x7f40e8101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:07.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.004+0000 7f40ee400700 1 --2- 172.21.15.131:0/2460217948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40e8101050 0x7f40e8101430 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f40d80098f0 tx=0x7f40d8009c00 comp rx=0 tx=0).stop 2024-05-08T22:18:07.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.004+0000 7f40ee400700 1 -- 172.21.15.131:0/2460217948 shutdown_connections 2024-05-08T22:18:07.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.004+0000 7f40ee400700 1 --2- 172.21.15.131:0/2460217948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40e8106170 0x7f40e810dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:07.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.004+0000 7f40ee400700 1 --2- 172.21.15.131:0/2460217948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40e8101050 0x7f40e8101430 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:07.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.004+0000 7f40ee400700 1 --2- 172.21.15.131:0/2460217948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40e8101a00 0x7f40e8105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:07.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.004+0000 7f40ee400700 1 -- 172.21.15.131:0/2460217948 >> 172.21.15.131:0/2460217948 conn(0x7f40e80fc8f0 msgr2=0x7f40e80fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:07.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.004+0000 7f40ee400700 1 -- 172.21.15.131:0/2460217948 shutdown_connections 2024-05-08T22:18:07.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.004+0000 7f40ee400700 1 -- 172.21.15.131:0/2460217948 wait complete. 2024-05-08T22:18:07.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.005+0000 7f40ee400700 1 Processor -- start 2024-05-08T22:18:07.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.005+0000 7f40ee400700 1 -- start start 2024-05-08T22:18:07.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.005+0000 7f40ee400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40e8101050 0x7f40e80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:07.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.005+0000 7f40ee400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40e8106170 0x7f40e80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:07.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.005+0000 7f40ee400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40e8073d20 0x7f40e81a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:07.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.005+0000 7f40ee400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e81a5a70 con 0x7f40e8073d20 2024-05-08T22:18:07.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.005+0000 7f40ee400700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e81a5be0 con 0x7f40e8101050 2024-05-08T22:18:07.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.005+0000 7f40ee400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e81a5da0 con 0x7f40e8106170 2024-05-08T22:18:07.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.006+0000 7f40e6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40e8106170 0x7f40e80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:07.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.006+0000 7f40e6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40e8106170 0x7f40e80737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43564/0 (socket says 172.21.15.131:43564) 2024-05-08T22:18:07.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.006+0000 7f40e7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40e8073d20 0x7f40e81a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:07.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.006+0000 7f40e6a00700 1 -- 172.21.15.131:0/2054894806 learned_addr learned my addr 172.21.15.131:0/2054894806 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:07.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.006+0000 7f40e7400700 1 --2- 172.21.15.131:0/2054894806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40e8101050 0x7f40e80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:07.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.006+0000 7f40e6a00700 1 -- 172.21.15.131:0/2054894806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40e8101050 msgr2=0x7f40e80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:07.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.006+0000 7f40e6a00700 1 --2- 172.21.15.131:0/2054894806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40e8101050 0x7f40e80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:07.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.006+0000 7f40e6a00700 1 -- 172.21.15.131:0/2054894806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40e8073d20 msgr2=0x7f40e81a54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:07.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.006+0000 7f40e6a00700 1 --2- 172.21.15.131:0/2054894806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40e8073d20 0x7f40e81a54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:07.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.006+0000 7f40e6a00700 1 -- 172.21.15.131:0/2054894806 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d40095f0 con 0x7f40e8106170 2024-05-08T22:18:07.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.006+0000 7f40e6a00700 1 --2- 172.21.15.131:0/2054894806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40e8106170 0x7f40e80737e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f40d400ec70 tx=0x7f40d400ef80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:07.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.007+0000 7f40dfe00700 1 -- 172.21.15.131:0/2054894806 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d40099d0 con 0x7f40e8106170 2024-05-08T22:18:07.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.007+0000 7f40dfe00700 1 -- 172.21.15.131:0/2054894806 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40d40044e0 con 0x7f40e8106170 2024-05-08T22:18:07.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.007+0000 7f40ee400700 1 -- 172.21.15.131:0/2054894806 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d8009520 con 0x7f40e8106170 2024-05-08T22:18:07.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.007+0000 7f40dfe00700 1 -- 172.21.15.131:0/2054894806 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d4010430 con 0x7f40e8106170 2024-05-08T22:18:07.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.007+0000 7f40ee400700 1 -- 172.21.15.131:0/2054894806 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e81a6400 con 0x7f40e8106170 2024-05-08T22:18:07.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.008+0000 7f40ee400700 1 -- 172.21.15.131:0/2054894806 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40e8109390 con 0x7f40e8106170 2024-05-08T22:18:07.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.010+0000 7f40dfe00700 1 -- 172.21.15.131:0/2054894806 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f40d4003ef0 con 0x7f40e8106170 2024-05-08T22:18:07.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.011+0000 7f40dfe00700 1 --2- 172.21.15.131:0/2054894806 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f40cc0a1290 0x7f40cc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:07.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.011+0000 7f40dfe00700 1 -- 172.21.15.131:0/2054894806 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9509+0+0 (secure 0 0 0) 0x7f40d4014070 con 0x7f40e8106170 2024-05-08T22:18:07.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.012+0000 7f40e7400700 1 --2- 172.21.15.131:0/2054894806 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f40cc0a1290 0x7f40cc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:07.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.012+0000 7f40e7400700 1 --2- 172.21.15.131:0/2054894806 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f40cc0a1290 0x7f40cc0a3750 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f40d8009fd0 tx=0x7f40d8019040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:07.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.014+0000 7f40dfe00700 1 -- 172.21.15.131:0/2054894806 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f40d4096730 con 0x7f40e8106170 2024-05-08T22:18:07.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.284+0000 7f40ee400700 1 -- 172.21.15.131:0/2054894806 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f40e8102520 con 0x7f40e8106170 2024-05-08T22:18:07.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.294+0000 7f40dfe00700 1 -- 172.21.15.131:0/2054894806 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) v1 ==== 81+0+733952 (secure 0 0 0) 0x7f40d4099d50 con 0x7f40e8106170 2024-05-08T22:18:07.306 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:18:07.306 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:18:07.306 INFO:teuthology.orchestra.run.smithi131.stdout: "epoch": 25, 2024-05-08T22:18:07.306 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.306 INFO:teuthology.orchestra.run.smithi131.stdout: "active_gid": 15366, 2024-05-08T22:18:07.306 INFO:teuthology.orchestra.run.smithi131.stdout: "active_name": "x", 2024-05-08T22:18:07.306 INFO:teuthology.orchestra.run.smithi131.stdout: "active_addrs": { 2024-05-08T22:18:07.306 INFO:teuthology.orchestra.run.smithi131.stdout: "addrvec": [ 2024-05-08T22:18:07.306 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.306 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v2", 2024-05-08T22:18:07.306 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "172.21.15.131:6800", 2024-05-08T22:18:07.306 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 1157588501 2024-05-08T22:18:07.307 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.307 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.307 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-05-08T22:18:07.307 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "172.21.15.131:6801", 2024-05-08T22:18:07.307 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 1157588501 2024-05-08T22:18:07.307 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.307 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-08T22:18:07.307 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.307 INFO:teuthology.orchestra.run.smithi131.stdout: "active_addr": "172.21.15.131:6801/1157588501", 2024-05-08T22:18:07.307 INFO:teuthology.orchestra.run.smithi131.stdout: "active_change": "2024-05-08T22:07:17.096512+0000", 2024-05-08T22:18:07.307 INFO:teuthology.orchestra.run.smithi131.stdout: "active_mgr_features": 4540138322906710015, 2024-05-08T22:18:07.307 INFO:teuthology.orchestra.run.smithi131.stdout: "available": true, 2024-05-08T22:18:07.307 INFO:teuthology.orchestra.run.smithi131.stdout: "standbys": [ 2024-05-08T22:18:07.307 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.308 INFO:teuthology.orchestra.run.smithi131.stdout: "gid": 25388, 2024-05-08T22:18:07.308 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "y", 2024-05-08T22:18:07.308 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr_features": 4540138322906710015, 2024-05-08T22:18:07.308 INFO:teuthology.orchestra.run.smithi131.stdout: "available_modules": [ 2024-05-08T22:18:07.308 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.308 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "alerts", 2024-05-08T22:18:07.308 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.308 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.308 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.308 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-08T22:18:07.308 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-08T22:18:07.308 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.308 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.308 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.308 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-08T22:18:07.309 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.309 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.309 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.309 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T22:18:07.309 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.309 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.309 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.309 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.309 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.309 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.309 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.309 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.309 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.309 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.309 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.310 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.310 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.310 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.310 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.310 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.310 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.310 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.310 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.310 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.310 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.310 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.310 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.310 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.310 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.310 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.310 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.311 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.311 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.311 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.311 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.311 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.311 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.311 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.311 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.311 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.311 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.311 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.311 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.311 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.311 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.311 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.311 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.312 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.312 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.312 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.312 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.312 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.312 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.312 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.312 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.312 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.312 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.312 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.312 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.312 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.312 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.312 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.312 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.313 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.313 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.313 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.313 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.313 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.313 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.313 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.313 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.313 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.313 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.313 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.313 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.313 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.313 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.313 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.313 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_destination": { 2024-05-08T22:18:07.314 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_destination", 2024-05-08T22:18:07.314 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.314 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.314 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.314 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.314 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.314 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.314 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.314 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Email address to send alerts to", 2024-05-08T22:18:07.314 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.314 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.314 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.314 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.314 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_from_name": { 2024-05-08T22:18:07.314 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_from_name", 2024-05-08T22:18:07.315 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.315 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.315 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.315 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "Ceph", 2024-05-08T22:18:07.315 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.315 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.315 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.315 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Email From: name", 2024-05-08T22:18:07.315 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.315 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.315 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.315 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.315 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_host": { 2024-05-08T22:18:07.315 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_host", 2024-05-08T22:18:07.316 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.316 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.316 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.316 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.316 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.316 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.316 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.316 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP server", 2024-05-08T22:18:07.316 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.316 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.316 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.316 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.316 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_password": { 2024-05-08T22:18:07.316 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_password", 2024-05-08T22:18:07.316 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.316 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.317 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.317 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.317 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.317 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.317 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.317 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Password to authenticate with", 2024-05-08T22:18:07.317 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.317 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.317 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.317 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.317 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_port": { 2024-05-08T22:18:07.317 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_port", 2024-05-08T22:18:07.317 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.317 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.317 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.317 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "465", 2024-05-08T22:18:07.317 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.318 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.318 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.318 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP port", 2024-05-08T22:18:07.318 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.318 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.318 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.318 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.318 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_sender": { 2024-05-08T22:18:07.318 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_sender", 2024-05-08T22:18:07.318 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.318 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.318 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.318 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.318 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.318 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.318 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.319 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP envelope sender", 2024-05-08T22:18:07.319 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.319 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.319 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.319 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.319 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_ssl": { 2024-05-08T22:18:07.319 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_ssl", 2024-05-08T22:18:07.319 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.319 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.319 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.319 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.319 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.319 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.319 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.319 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T22:18:07.320 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.320 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.320 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.320 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.320 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_user": { 2024-05-08T22:18:07.320 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_user", 2024-05-08T22:18:07.320 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.320 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.320 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.320 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.320 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.320 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.320 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.320 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "User to authenticate as", 2024-05-08T22:18:07.320 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.321 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.321 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.321 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.321 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.321 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.321 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.321 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "balancer", 2024-05-08T22:18:07.321 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.321 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.321 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.321 INFO:teuthology.orchestra.run.smithi131.stdout: "active": { 2024-05-08T22:18:07.321 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "active", 2024-05-08T22:18:07.321 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.321 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.321 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.321 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.322 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.322 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.322 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.322 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T22:18:07.322 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.322 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.322 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.322 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.322 INFO:teuthology.orchestra.run.smithi131.stdout: "begin_time": { 2024-05-08T22:18:07.322 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "begin_time", 2024-05-08T22:18:07.322 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.322 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.322 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.322 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0000", 2024-05-08T22:18:07.323 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.323 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.323 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.323 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T22:18:07.323 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T22:18:07.323 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.323 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.323 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.323 INFO:teuthology.orchestra.run.smithi131.stdout: "begin_weekday": { 2024-05-08T22:18:07.323 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "begin_weekday", 2024-05-08T22:18:07.323 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-08T22:18:07.323 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.323 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.323 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-08T22:18:07.323 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0", 2024-05-08T22:18:07.324 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "6", 2024-05-08T22:18:07.324 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.324 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T22:18:07.324 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T22:18:07.324 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.324 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.324 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.324 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_max_iterations": { 2024-05-08T22:18:07.324 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_max_iterations", 2024-05-08T22:18:07.324 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-08T22:18:07.324 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.324 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.324 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "25", 2024-05-08T22:18:07.324 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-05-08T22:18:07.324 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "250", 2024-05-08T22:18:07.325 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.325 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T22:18:07.325 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.325 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.325 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.325 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.325 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_metrics": { 2024-05-08T22:18:07.325 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_metrics", 2024-05-08T22:18:07.325 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.325 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.325 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.325 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T22:18:07.325 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.325 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.326 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.326 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T22:18:07.326 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T22:18:07.326 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.326 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.326 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.326 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_step": { 2024-05-08T22:18:07.326 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_step", 2024-05-08T22:18:07.326 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-08T22:18:07.326 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.326 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.326 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0.5", 2024-05-08T22:18:07.326 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0.001", 2024-05-08T22:18:07.326 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "0.999", 2024-05-08T22:18:07.327 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.327 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "aggressiveness of optimization", 2024-05-08T22:18:07.327 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T22:18:07.327 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.327 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.327 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.327 INFO:teuthology.orchestra.run.smithi131.stdout: "end_time": { 2024-05-08T22:18:07.327 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "end_time", 2024-05-08T22:18:07.327 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.327 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.327 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.327 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "2359", 2024-05-08T22:18:07.327 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.327 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.327 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.328 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T22:18:07.328 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T22:18:07.328 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.328 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.328 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.328 INFO:teuthology.orchestra.run.smithi131.stdout: "end_weekday": { 2024-05-08T22:18:07.328 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "end_weekday", 2024-05-08T22:18:07.328 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-08T22:18:07.328 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.328 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.328 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-08T22:18:07.328 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0", 2024-05-08T22:18:07.328 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "6", 2024-05-08T22:18:07.328 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.329 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T22:18:07.329 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T22:18:07.329 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.329 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.329 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.329 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.329 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.329 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.329 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.329 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.329 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.329 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.329 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.329 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.329 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.329 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.330 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.330 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.330 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.330 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.330 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.330 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.330 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.330 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.330 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.330 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.330 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.330 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.330 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.330 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.331 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.331 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.331 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.331 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.331 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.331 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.331 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.331 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.331 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.331 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.331 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.331 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.331 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.331 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.332 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.332 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.332 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.332 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.332 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.332 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.332 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.332 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.332 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.332 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.332 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.332 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.332 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.332 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.333 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.333 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.333 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.333 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.333 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.333 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.333 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.333 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.333 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.333 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.333 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.333 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.333 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.333 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.333 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.333 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.334 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.334 INFO:teuthology.orchestra.run.smithi131.stdout: "min_score": { 2024-05-08T22:18:07.334 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "min_score", 2024-05-08T22:18:07.334 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-08T22:18:07.334 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.334 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.334 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-08T22:18:07.334 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.334 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.334 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.334 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T22:18:07.334 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.334 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.335 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.335 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.335 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": { 2024-05-08T22:18:07.335 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mode", 2024-05-08T22:18:07.335 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.335 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.335 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.335 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "upmap", 2024-05-08T22:18:07.335 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.335 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.335 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.335 INFO:teuthology.orchestra.run.smithi131.stdout: "crush-compat", 2024-05-08T22:18:07.335 INFO:teuthology.orchestra.run.smithi131.stdout: "none", 2024-05-08T22:18:07.335 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap" 2024-05-08T22:18:07.335 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.335 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Balancer mode", 2024-05-08T22:18:07.336 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.336 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.336 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.336 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.336 INFO:teuthology.orchestra.run.smithi131.stdout: "pool_ids": { 2024-05-08T22:18:07.336 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pool_ids", 2024-05-08T22:18:07.336 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.336 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.336 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.336 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.336 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.336 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.336 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.336 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T22:18:07.336 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.337 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.337 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.337 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.337 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-08T22:18:07.337 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-08T22:18:07.337 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.337 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.337 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.337 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-08T22:18:07.337 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.337 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.337 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.337 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T22:18:07.337 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.337 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.337 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.338 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.338 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap_max_deviation": { 2024-05-08T22:18:07.338 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "upmap_max_deviation", 2024-05-08T22:18:07.338 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.338 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.338 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.338 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-05-08T22:18:07.338 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-05-08T22:18:07.338 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.338 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.338 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T22:18:07.338 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T22:18:07.338 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.338 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.338 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.338 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap_max_optimizations": { 2024-05-08T22:18:07.339 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "upmap_max_optimizations", 2024-05-08T22:18:07.339 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-08T22:18:07.339 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.339 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.339 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-08T22:18:07.339 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.339 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.339 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.339 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T22:18:07.339 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.339 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.339 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.339 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.339 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.339 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.339 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.340 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cephadm", 2024-05-08T22:18:07.340 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.340 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.340 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.340 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_down_multiplier": { 2024-05-08T22:18:07.340 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_down_multiplier", 2024-05-08T22:18:07.340 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-08T22:18:07.340 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.340 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.340 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3.0", 2024-05-08T22:18:07.340 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.340 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.340 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.340 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T22:18:07.340 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.340 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.340 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.341 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.341 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_refresh_rate": { 2024-05-08T22:18:07.341 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_refresh_rate", 2024-05-08T22:18:07.341 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.341 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.341 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.341 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "20", 2024-05-08T22:18:07.341 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.341 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.341 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.341 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T22:18:07.341 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.341 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.341 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.341 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.341 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_starting_port": { 2024-05-08T22:18:07.342 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_starting_port", 2024-05-08T22:18:07.342 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.342 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.342 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.342 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "4721", 2024-05-08T22:18:07.342 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.342 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.342 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.342 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T22:18:07.342 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.342 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.342 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.342 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.343 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_ptrace": { 2024-05-08T22:18:07.343 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_ptrace", 2024-05-08T22:18:07.343 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.343 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.343 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.343 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.343 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.343 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.343 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.343 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T22:18:07.343 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T22:18:07.343 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.343 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.343 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.343 INFO:teuthology.orchestra.run.smithi131.stdout: "autotune_interval": { 2024-05-08T22:18:07.344 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "autotune_interval", 2024-05-08T22:18:07.344 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.344 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.344 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.344 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-08T22:18:07.344 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.344 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.344 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.344 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T22:18:07.344 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.344 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.344 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.344 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.344 INFO:teuthology.orchestra.run.smithi131.stdout: "autotune_memory_target_ratio": { 2024-05-08T22:18:07.344 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T22:18:07.345 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-08T22:18:07.345 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.345 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.345 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0.7", 2024-05-08T22:18:07.345 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.345 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.345 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.345 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T22:18:07.345 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.345 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.345 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.345 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.345 INFO:teuthology.orchestra.run.smithi131.stdout: "cgroups_split": { 2024-05-08T22:18:07.346 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cgroups_split", 2024-05-08T22:18:07.346 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.346 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.346 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.346 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.346 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.346 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.346 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.346 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T22:18:07.346 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.346 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.346 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.346 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.347 INFO:teuthology.orchestra.run.smithi131.stdout: "config_checks_enabled": { 2024-05-08T22:18:07.347 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_checks_enabled", 2024-05-08T22:18:07.347 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.347 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.347 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.347 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.347 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.347 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.347 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.347 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T22:18:07.347 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.347 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.347 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.347 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.347 INFO:teuthology.orchestra.run.smithi131.stdout: "config_dashboard": { 2024-05-08T22:18:07.348 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_dashboard", 2024-05-08T22:18:07.348 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.348 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.348 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.348 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.348 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.348 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.348 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.348 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T22:18:07.348 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.348 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.348 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.348 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.348 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_alertmanager": { 2024-05-08T22:18:07.349 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_alertmanager", 2024-05-08T22:18:07.349 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.349 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.349 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.349 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T22:18:07.349 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.349 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.349 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.349 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-05-08T22:18:07.349 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.349 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.349 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.349 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.349 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_base": { 2024-05-08T22:18:07.349 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_base", 2024-05-08T22:18:07.350 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.350 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.350 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.350 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T22:18:07.350 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.350 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.350 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.350 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Container image name, without the tag", 2024-05-08T22:18:07.350 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.350 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.350 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.350 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.350 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_elasticsearch": { 2024-05-08T22:18:07.350 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_elasticsearch", 2024-05-08T22:18:07.351 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.351 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.351 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.351 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T22:18:07.351 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.351 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.351 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.351 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "elasticsearch container image", 2024-05-08T22:18:07.351 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.351 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.351 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.351 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.351 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_grafana": { 2024-05-08T22:18:07.351 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_grafana", 2024-05-08T22:18:07.352 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.352 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.352 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.352 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-08T22:18:07.352 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.352 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.352 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.352 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-05-08T22:18:07.352 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.352 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.352 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.352 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.352 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_haproxy": { 2024-05-08T22:18:07.352 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_haproxy", 2024-05-08T22:18:07.352 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.353 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.353 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.353 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T22:18:07.353 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.353 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.353 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.353 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "HAproxy container image", 2024-05-08T22:18:07.353 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.353 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.353 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.353 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.353 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_agent": { 2024-05-08T22:18:07.353 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_agent", 2024-05-08T22:18:07.353 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.353 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.354 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.354 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T22:18:07.354 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.354 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.354 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.354 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger agent container image", 2024-05-08T22:18:07.354 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.354 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.354 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.354 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.354 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_collector": { 2024-05-08T22:18:07.354 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_collector", 2024-05-08T22:18:07.355 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.355 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.355 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.355 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T22:18:07.355 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.355 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.355 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.355 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger collector container image", 2024-05-08T22:18:07.355 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.355 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.355 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.355 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.355 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_query": { 2024-05-08T22:18:07.355 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_query", 2024-05-08T22:18:07.355 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.356 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.356 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.356 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T22:18:07.356 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.356 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.356 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.356 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger query container image", 2024-05-08T22:18:07.356 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.356 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.356 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.356 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.356 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_keepalived": { 2024-05-08T22:18:07.356 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_keepalived", 2024-05-08T22:18:07.357 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.357 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.357 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.357 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T22:18:07.357 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.357 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.357 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.357 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Keepalived container image", 2024-05-08T22:18:07.357 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.357 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.357 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.357 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.357 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_loki": { 2024-05-08T22:18:07.357 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_loki", 2024-05-08T22:18:07.357 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.358 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.358 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.358 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T22:18:07.358 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.358 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.358 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.358 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Loki container image", 2024-05-08T22:18:07.358 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.358 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.358 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.358 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.358 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_node_exporter": { 2024-05-08T22:18:07.358 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_node_exporter", 2024-05-08T22:18:07.358 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.359 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.359 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.359 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T22:18:07.359 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.359 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.359 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.359 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-05-08T22:18:07.359 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.359 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.359 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.359 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.359 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_nvmeof": { 2024-05-08T22:18:07.359 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_nvmeof", 2024-05-08T22:18:07.359 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.359 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.359 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.360 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-08T22:18:07.360 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.360 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.360 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.360 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Nvme-of container image", 2024-05-08T22:18:07.360 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.360 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.360 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.360 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.360 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_prometheus": { 2024-05-08T22:18:07.360 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_prometheus", 2024-05-08T22:18:07.360 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.360 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.360 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.360 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T22:18:07.361 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.361 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.361 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.361 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-05-08T22:18:07.361 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.361 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.361 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.361 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.361 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_promtail": { 2024-05-08T22:18:07.361 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_promtail", 2024-05-08T22:18:07.361 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.361 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.361 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.361 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T22:18:07.361 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.362 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.362 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.362 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Promtail container image", 2024-05-08T22:18:07.362 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.362 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.362 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.362 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.362 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_snmp_gateway": { 2024-05-08T22:18:07.362 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_snmp_gateway", 2024-05-08T22:18:07.362 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.362 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.362 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.362 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T22:18:07.362 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.363 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.363 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.363 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SNMP Gateway container image", 2024-05-08T22:18:07.363 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.363 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.363 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.363 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.363 INFO:teuthology.orchestra.run.smithi131.stdout: "container_init": { 2024-05-08T22:18:07.363 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_init", 2024-05-08T22:18:07.363 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.363 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.363 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.363 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.363 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.363 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.363 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.364 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T22:18:07.364 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.364 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.364 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.364 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.364 INFO:teuthology.orchestra.run.smithi131.stdout: "daemon_cache_timeout": { 2024-05-08T22:18:07.364 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "daemon_cache_timeout", 2024-05-08T22:18:07.364 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.364 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.364 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.364 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-08T22:18:07.364 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.364 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.364 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.365 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T22:18:07.365 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.365 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.365 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.365 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.365 INFO:teuthology.orchestra.run.smithi131.stdout: "default_cephadm_command_timeout": { 2024-05-08T22:18:07.365 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T22:18:07.365 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.365 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.365 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.365 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "900", 2024-05-08T22:18:07.365 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.365 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.365 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.365 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T22:18:07.365 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.366 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.366 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.366 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.366 INFO:teuthology.orchestra.run.smithi131.stdout: "default_registry": { 2024-05-08T22:18:07.366 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "default_registry", 2024-05-08T22:18:07.366 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.366 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.366 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.366 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io", 2024-05-08T22:18:07.366 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.366 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.366 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.366 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T22:18:07.367 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.367 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.367 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.367 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.367 INFO:teuthology.orchestra.run.smithi131.stdout: "device_cache_timeout": { 2024-05-08T22:18:07.367 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_cache_timeout", 2024-05-08T22:18:07.367 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.367 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.367 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.367 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1800", 2024-05-08T22:18:07.367 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.367 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.367 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.367 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache device inventory", 2024-05-08T22:18:07.368 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.368 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.368 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.368 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.368 INFO:teuthology.orchestra.run.smithi131.stdout: "device_enhanced_scan": { 2024-05-08T22:18:07.368 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_enhanced_scan", 2024-05-08T22:18:07.368 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.368 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.368 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.368 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.368 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.368 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.368 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.368 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T22:18:07.369 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.369 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.369 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.369 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.369 INFO:teuthology.orchestra.run.smithi131.stdout: "facts_cache_timeout": { 2024-05-08T22:18:07.369 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "facts_cache_timeout", 2024-05-08T22:18:07.369 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.369 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.369 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.369 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-08T22:18:07.369 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.369 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.369 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.369 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache host facts data", 2024-05-08T22:18:07.369 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.370 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.370 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.370 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.370 INFO:teuthology.orchestra.run.smithi131.stdout: "host_check_interval": { 2024-05-08T22:18:07.370 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "host_check_interval", 2024-05-08T22:18:07.370 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.370 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.370 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.370 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-08T22:18:07.370 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.370 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.370 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.370 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to perform a host check", 2024-05-08T22:18:07.371 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.371 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.371 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.371 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.371 INFO:teuthology.orchestra.run.smithi131.stdout: "hw_monitoring": { 2024-05-08T22:18:07.371 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "hw_monitoring", 2024-05-08T22:18:07.371 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.371 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.371 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.371 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.371 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.371 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.371 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.371 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T22:18:07.371 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.372 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.372 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.372 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.372 INFO:teuthology.orchestra.run.smithi131.stdout: "inventory_list_all": { 2024-05-08T22:18:07.372 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "inventory_list_all", 2024-05-08T22:18:07.372 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.372 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.372 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.372 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.372 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.372 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.372 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.372 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T22:18:07.372 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.372 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.372 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.373 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.373 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.373 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.373 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.373 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.373 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.373 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.373 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.373 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.373 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.373 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.373 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.373 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.373 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.373 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.374 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.374 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.374 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.374 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.374 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.374 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.374 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.374 INFO:teuthology.orchestra.run.smithi131.stdout: "log_refresh_metadata": { 2024-05-08T22:18:07.374 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_refresh_metadata", 2024-05-08T22:18:07.374 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.374 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.374 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.374 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.374 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.375 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.375 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.375 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T22:18:07.375 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.375 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.375 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.375 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.375 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.375 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.375 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.375 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.375 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.375 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.375 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.376 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.376 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.376 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T22:18:07.376 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.376 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.376 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.376 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.376 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.376 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.376 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.376 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.376 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.376 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.376 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.376 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.376 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.376 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.377 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.377 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.377 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.377 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.377 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.377 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.377 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.377 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.377 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.377 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.377 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.377 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.377 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.377 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.378 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.378 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.378 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.378 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.378 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.378 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.378 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.378 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.378 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.378 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.378 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.378 INFO:teuthology.orchestra.run.smithi131.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T22:18:07.379 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T22:18:07.379 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.379 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.379 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.379 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.379 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.379 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.379 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.379 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T22:18:07.379 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.379 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.379 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.379 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.379 INFO:teuthology.orchestra.run.smithi131.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T22:18:07.380 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T22:18:07.380 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.380 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.380 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.380 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "*", 2024-05-08T22:18:07.380 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.380 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.380 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.380 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T22:18:07.380 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.380 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.380 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.380 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.381 INFO:teuthology.orchestra.run.smithi131.stdout: "max_count_per_host": { 2024-05-08T22:18:07.381 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_count_per_host", 2024-05-08T22:18:07.381 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.381 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.381 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.381 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-08T22:18:07.381 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.381 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.381 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.381 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "max number of daemons per service per host", 2024-05-08T22:18:07.381 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.381 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.381 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.381 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.381 INFO:teuthology.orchestra.run.smithi131.stdout: "max_osd_draining_count": { 2024-05-08T22:18:07.382 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_osd_draining_count", 2024-05-08T22:18:07.382 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.382 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.382 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.382 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-08T22:18:07.382 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.382 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.382 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.382 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T22:18:07.382 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.382 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.382 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.382 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.383 INFO:teuthology.orchestra.run.smithi131.stdout: "migration_current": { 2024-05-08T22:18:07.383 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "migration_current", 2024-05-08T22:18:07.383 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.383 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.383 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.383 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.383 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.383 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.383 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.383 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "internal - do not modify", 2024-05-08T22:18:07.383 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.383 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.383 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.383 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.383 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": { 2024-05-08T22:18:07.384 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mode", 2024-05-08T22:18:07.384 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.384 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.384 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.384 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "root", 2024-05-08T22:18:07.384 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.384 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.384 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.384 INFO:teuthology.orchestra.run.smithi131.stdout: "cephadm-package", 2024-05-08T22:18:07.384 INFO:teuthology.orchestra.run.smithi131.stdout: "root" 2024-05-08T22:18:07.384 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.384 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T22:18:07.384 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.384 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.385 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.385 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.385 INFO:teuthology.orchestra.run.smithi131.stdout: "oob_default_addr": { 2024-05-08T22:18:07.385 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "oob_default_addr", 2024-05-08T22:18:07.385 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.385 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.385 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.385 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "169.254.1.1", 2024-05-08T22:18:07.385 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.385 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.385 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.385 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T22:18:07.385 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.385 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.386 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.386 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.386 INFO:teuthology.orchestra.run.smithi131.stdout: "prometheus_alerts_path": { 2024-05-08T22:18:07.386 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prometheus_alerts_path", 2024-05-08T22:18:07.386 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.386 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.386 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.386 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T22:18:07.386 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.386 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.386 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.386 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T22:18:07.386 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.386 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.387 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.387 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.387 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_insecure": { 2024-05-08T22:18:07.387 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_insecure", 2024-05-08T22:18:07.387 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.387 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.387 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.387 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.387 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.387 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.387 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.387 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T22:18:07.387 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.387 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.388 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.388 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.388 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_password": { 2024-05-08T22:18:07.388 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_password", 2024-05-08T22:18:07.388 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.388 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.388 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.388 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.388 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.388 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.388 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.388 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T22:18:07.388 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.388 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.388 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.389 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.389 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_url": { 2024-05-08T22:18:07.389 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_url", 2024-05-08T22:18:07.389 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.389 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.389 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.389 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.389 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.389 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.389 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.389 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T22:18:07.389 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.389 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.389 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.389 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.389 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_username": { 2024-05-08T22:18:07.390 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_username", 2024-05-08T22:18:07.390 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.390 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.390 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.390 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.390 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.390 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.390 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.390 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T22:18:07.390 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.390 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.390 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.390 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.390 INFO:teuthology.orchestra.run.smithi131.stdout: "secure_monitoring_stack": { 2024-05-08T22:18:07.391 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "secure_monitoring_stack", 2024-05-08T22:18:07.391 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.391 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.391 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.391 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.391 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.391 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.391 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.391 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T22:18:07.391 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.391 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.391 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.391 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.391 INFO:teuthology.orchestra.run.smithi131.stdout: "service_discovery_port": { 2024-05-08T22:18:07.392 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "service_discovery_port", 2024-05-08T22:18:07.392 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.392 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.392 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.392 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8765", 2024-05-08T22:18:07.392 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.392 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.392 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.392 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "cephadm service discovery port", 2024-05-08T22:18:07.392 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.392 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.392 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.392 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.392 INFO:teuthology.orchestra.run.smithi131.stdout: "ssh_config_file": { 2024-05-08T22:18:07.393 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssh_config_file", 2024-05-08T22:18:07.393 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.393 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.393 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.393 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.393 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.393 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.393 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.393 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T22:18:07.393 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.393 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.393 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.393 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.393 INFO:teuthology.orchestra.run.smithi131.stdout: "use_agent": { 2024-05-08T22:18:07.393 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "use_agent", 2024-05-08T22:18:07.393 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.394 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.394 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.394 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.394 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.394 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.394 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.394 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T22:18:07.394 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.394 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.394 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.394 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.394 INFO:teuthology.orchestra.run.smithi131.stdout: "use_repo_digest": { 2024-05-08T22:18:07.394 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "use_repo_digest", 2024-05-08T22:18:07.394 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.394 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.395 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.395 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.395 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.395 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.395 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.395 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T22:18:07.395 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.395 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.395 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.395 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.395 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_failed_host_check": { 2024-05-08T22:18:07.395 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_failed_host_check", 2024-05-08T22:18:07.395 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.395 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.396 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.396 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.396 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.396 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.396 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.396 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T22:18:07.396 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.396 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.396 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.396 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.396 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_stray_daemons": { 2024-05-08T22:18:07.396 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_stray_daemons", 2024-05-08T22:18:07.396 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.396 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.397 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.397 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.397 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.397 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.397 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.397 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T22:18:07.397 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.397 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.397 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.397 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.397 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_stray_hosts": { 2024-05-08T22:18:07.397 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_stray_hosts", 2024-05-08T22:18:07.397 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.398 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.398 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.398 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.398 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.398 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.398 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.398 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T22:18:07.398 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.398 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.398 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.398 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.398 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.398 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.398 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.399 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crash", 2024-05-08T22:18:07.399 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.399 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.399 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.399 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.399 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.399 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.399 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.399 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.399 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.399 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.399 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.399 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.399 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.400 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.400 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.400 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.400 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.400 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.400 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.400 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.400 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.400 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.400 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.400 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.400 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.400 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.400 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.400 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.401 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.401 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.401 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.401 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.401 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.401 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.401 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.401 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.401 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.401 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.401 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.401 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.401 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.402 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.402 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.402 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.402 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.402 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.402 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.402 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.402 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.402 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.402 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.402 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.402 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.402 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.403 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.403 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.403 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.403 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.403 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.403 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.403 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.403 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.403 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.403 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.403 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.403 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.403 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.404 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.404 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.404 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.404 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.404 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.404 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.404 INFO:teuthology.orchestra.run.smithi131.stdout: "retain_interval": { 2024-05-08T22:18:07.404 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "retain_interval", 2024-05-08T22:18:07.404 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.404 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.404 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.404 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "31536000", 2024-05-08T22:18:07.404 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.404 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.405 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.405 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T22:18:07.405 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.405 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.405 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.405 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.405 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_recent_interval": { 2024-05-08T22:18:07.405 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_recent_interval", 2024-05-08T22:18:07.405 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.405 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.405 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.405 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1209600", 2024-05-08T22:18:07.405 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.406 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.406 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.406 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T22:18:07.406 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.406 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.406 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.406 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.406 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.406 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.406 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.406 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "dashboard", 2024-05-08T22:18:07.406 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.406 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.406 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.407 INFO:teuthology.orchestra.run.smithi131.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T22:18:07.407 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T22:18:07.407 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.407 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.407 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.407 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-08T22:18:07.407 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.407 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.407 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.407 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.407 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.407 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.407 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.407 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.407 INFO:teuthology.orchestra.run.smithi131.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T22:18:07.407 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T22:18:07.408 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.408 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.408 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.408 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.408 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.408 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.408 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.408 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.408 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.408 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.408 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.408 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.408 INFO:teuthology.orchestra.run.smithi131.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T22:18:07.408 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T22:18:07.408 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.409 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.409 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.409 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.409 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.409 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.409 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.409 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.409 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.409 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.409 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.409 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.409 INFO:teuthology.orchestra.run.smithi131.stdout: "AUDIT_API_ENABLED": { 2024-05-08T22:18:07.409 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T22:18:07.409 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.409 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.409 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.410 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.410 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.410 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.410 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.410 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.410 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.410 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.410 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.410 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.410 INFO:teuthology.orchestra.run.smithi131.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T22:18:07.410 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T22:18:07.410 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.410 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.410 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.410 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.411 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.411 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.411 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.411 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.411 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.411 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.411 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.411 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.411 INFO:teuthology.orchestra.run.smithi131.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T22:18:07.411 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T22:18:07.411 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.411 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.411 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.411 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.411 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.411 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.412 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.412 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.412 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.412 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.412 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.412 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.412 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T22:18:07.412 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T22:18:07.412 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.412 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.412 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.412 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.412 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.412 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.412 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.412 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.413 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.413 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.413 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.413 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.413 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T22:18:07.413 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T22:18:07.413 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.413 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.413 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.413 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.413 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.413 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.413 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.413 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.413 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.413 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.414 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.414 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.414 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T22:18:07.414 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T22:18:07.414 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.414 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.414 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.414 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.414 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.414 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.414 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.414 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.414 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.414 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.414 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.415 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.415 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T22:18:07.415 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T22:18:07.415 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.415 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.415 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.415 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.415 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.415 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.415 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.415 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.415 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.415 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.415 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.415 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.415 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T22:18:07.416 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T22:18:07.416 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.416 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.416 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.416 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.416 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.416 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.416 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.416 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.416 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.416 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.416 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.416 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.416 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T22:18:07.416 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T22:18:07.416 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.417 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.417 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.417 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.417 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.417 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.417 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.417 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.417 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.417 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.417 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.417 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.417 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T22:18:07.417 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T22:18:07.417 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.417 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.418 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.418 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.418 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.418 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.418 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.418 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.418 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.418 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.418 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.418 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.418 INFO:teuthology.orchestra.run.smithi131.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T22:18:07.418 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T22:18:07.418 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.418 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.418 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.419 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.419 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.419 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.419 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.419 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.419 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.419 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.419 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.419 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.419 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T22:18:07.419 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T22:18:07.419 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.419 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.419 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.419 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-05-08T22:18:07.419 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.419 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.420 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.420 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.420 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.420 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.420 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.420 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.420 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T22:18:07.420 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T22:18:07.420 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.420 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.420 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.420 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.420 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.420 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.420 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.420 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.421 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.421 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.421 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.421 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.421 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_URL": { 2024-05-08T22:18:07.421 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_URL", 2024-05-08T22:18:07.421 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.421 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.421 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.421 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.421 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.421 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.421 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.421 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.421 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.421 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.422 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.422 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.422 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T22:18:07.422 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T22:18:07.422 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.422 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.422 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.422 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-05-08T22:18:07.422 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.422 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.422 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.422 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.422 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.422 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.422 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.423 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.423 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T22:18:07.423 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T22:18:07.423 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.423 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.423 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.423 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.423 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.423 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.423 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.423 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.423 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.423 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.423 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.423 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.423 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T22:18:07.423 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T22:18:07.424 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.424 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.424 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.424 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.424 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.424 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.424 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.424 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.424 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.424 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.424 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.424 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.424 INFO:teuthology.orchestra.run.smithi131.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T22:18:07.424 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T22:18:07.424 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.424 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.425 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.425 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.425 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.425 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.425 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.425 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.425 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.425 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.425 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.425 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.425 INFO:teuthology.orchestra.run.smithi131.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T22:18:07.425 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T22:18:07.425 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.425 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.425 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.425 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.426 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.426 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.426 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.426 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.426 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.426 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.426 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.426 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.426 INFO:teuthology.orchestra.run.smithi131.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T22:18:07.426 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T22:18:07.426 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.426 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.426 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.426 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.427 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.427 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.427 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.427 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.427 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.427 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.427 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.427 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.427 INFO:teuthology.orchestra.run.smithi131.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T22:18:07.427 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T22:18:07.427 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.427 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.427 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.428 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.428 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.428 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.428 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.428 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.428 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.428 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.428 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.428 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.428 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T22:18:07.428 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T22:18:07.428 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.428 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.428 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.429 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.429 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.429 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.429 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.429 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.429 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.429 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.429 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.429 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.429 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T22:18:07.429 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T22:18:07.429 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.429 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.429 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.430 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.430 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.430 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.430 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.430 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.430 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.430 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.430 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.430 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.430 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T22:18:07.430 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T22:18:07.430 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.430 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.430 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.431 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.431 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.431 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.431 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.431 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.431 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.431 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.431 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.431 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.431 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T22:18:07.431 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T22:18:07.431 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.431 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.431 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.431 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.431 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.432 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.432 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.432 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.432 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.432 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.432 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.432 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.432 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T22:18:07.432 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T22:18:07.432 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.432 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.432 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.432 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.432 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.432 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.433 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.433 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.433 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.433 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.433 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.433 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.433 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T22:18:07.433 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T22:18:07.433 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.433 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.433 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.433 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.433 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.433 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.433 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.433 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.434 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.434 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.434 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.434 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.434 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T22:18:07.434 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T22:18:07.434 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.434 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.434 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.434 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.434 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.434 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.434 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.434 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.434 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.435 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.435 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.435 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.435 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T22:18:07.435 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T22:18:07.435 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.435 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.435 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.435 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.435 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.435 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.435 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.435 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.435 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.435 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.436 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.436 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.436 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T22:18:07.436 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T22:18:07.436 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.436 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.436 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.436 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T22:18:07.436 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.436 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.436 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.436 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.436 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.436 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.436 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.436 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.437 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T22:18:07.437 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T22:18:07.437 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.437 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.437 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.437 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-08T22:18:07.437 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.437 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.437 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.437 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.437 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.437 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.437 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.437 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.437 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T22:18:07.437 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T22:18:07.438 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.438 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.438 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.438 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8", 2024-05-08T22:18:07.438 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.438 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.438 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.438 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.438 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.438 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.438 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.438 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.438 INFO:teuthology.orchestra.run.smithi131.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T22:18:07.438 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T22:18:07.438 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.439 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.439 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.439 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "45", 2024-05-08T22:18:07.439 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.439 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.439 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.439 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.439 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.439 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.439 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.439 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.439 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T22:18:07.439 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T22:18:07.439 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.440 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.440 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.440 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.440 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.440 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.440 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.440 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.440 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.440 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.440 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.440 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.440 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T22:18:07.440 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T22:18:07.440 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.441 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.441 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.441 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-05-08T22:18:07.441 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.441 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.441 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.441 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.441 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.441 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.441 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.441 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.441 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T22:18:07.441 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T22:18:07.441 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.442 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.442 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.442 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.442 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.442 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.442 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.442 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.442 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.442 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.442 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.442 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.442 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T22:18:07.442 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T22:18:07.442 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.443 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.443 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.443 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.443 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.443 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.443 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.443 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.443 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.443 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.443 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.443 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.443 INFO:teuthology.orchestra.run.smithi131.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T22:18:07.443 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T22:18:07.443 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.443 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.443 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.444 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.444 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.444 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.444 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.444 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.444 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.444 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.444 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.444 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.444 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T22:18:07.444 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T22:18:07.444 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.444 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.445 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.445 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-08T22:18:07.445 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.445 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.445 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.445 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.445 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.445 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.445 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.445 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.445 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T22:18:07.445 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T22:18:07.445 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.445 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.445 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.446 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-08T22:18:07.446 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.446 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.446 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.446 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.446 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.446 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.446 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.446 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.446 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T22:18:07.446 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T22:18:07.446 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.446 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.446 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.447 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-05-08T22:18:07.447 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.447 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.447 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.447 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.447 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.447 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.447 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.447 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.447 INFO:teuthology.orchestra.run.smithi131.stdout: "cross_origin_url": { 2024-05-08T22:18:07.447 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cross_origin_url", 2024-05-08T22:18:07.447 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.447 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.448 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.448 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.448 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.448 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.448 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.448 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.448 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.448 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.448 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.448 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.448 INFO:teuthology.orchestra.run.smithi131.stdout: "crt_file": { 2024-05-08T22:18:07.448 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crt_file", 2024-05-08T22:18:07.448 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.448 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.448 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.449 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.449 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.449 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.449 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.449 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.449 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.449 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.449 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.449 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.449 INFO:teuthology.orchestra.run.smithi131.stdout: "debug": { 2024-05-08T22:18:07.449 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "debug", 2024-05-08T22:18:07.449 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.449 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.449 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.449 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.449 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.450 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.450 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.450 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable/disable debug options", 2024-05-08T22:18:07.450 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.450 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.450 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.450 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.450 INFO:teuthology.orchestra.run.smithi131.stdout: "jwt_token_ttl": { 2024-05-08T22:18:07.450 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "jwt_token_ttl", 2024-05-08T22:18:07.450 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.450 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.450 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.450 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "28800", 2024-05-08T22:18:07.450 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.451 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.451 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.451 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.451 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.451 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.451 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.451 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.451 INFO:teuthology.orchestra.run.smithi131.stdout: "key_file": { 2024-05-08T22:18:07.451 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "key_file", 2024-05-08T22:18:07.451 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.451 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.451 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.451 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.451 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.451 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.452 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.452 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.452 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.452 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.452 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.452 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.452 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.452 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.452 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.452 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.452 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.452 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.452 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.452 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.453 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.453 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.453 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.453 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.453 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.453 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.453 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.453 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.453 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.453 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.453 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.453 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.453 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.453 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.453 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.453 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.454 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.454 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.454 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.454 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.454 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.454 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.454 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.454 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.454 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.454 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.454 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.454 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.454 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.455 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.455 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.455 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.455 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.455 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.455 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.455 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.455 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.455 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.455 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.455 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.455 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.455 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.455 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.455 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.456 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.456 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.456 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.456 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.456 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.456 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.456 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.456 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.456 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.456 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.456 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.456 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.456 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.456 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.457 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.457 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.457 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.457 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.457 INFO:teuthology.orchestra.run.smithi131.stdout: "motd": { 2024-05-08T22:18:07.457 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "motd", 2024-05-08T22:18:07.457 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.457 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.457 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.457 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.457 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.457 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.457 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.457 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "The message of the day", 2024-05-08T22:18:07.458 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.458 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.458 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.458 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.458 INFO:teuthology.orchestra.run.smithi131.stdout: "redirect_resolve_ip_addr": { 2024-05-08T22:18:07.458 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T22:18:07.458 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.458 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.458 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.458 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.458 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.458 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.458 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.458 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.458 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.459 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.459 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.459 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.459 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-05-08T22:18:07.459 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-05-08T22:18:07.459 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.459 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.459 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.459 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "::", 2024-05-08T22:18:07.459 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.459 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.459 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.459 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.459 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.460 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.460 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.460 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.460 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-05-08T22:18:07.460 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-05-08T22:18:07.460 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.460 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.460 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.460 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8080", 2024-05-08T22:18:07.460 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.460 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.460 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.460 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.460 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.460 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.460 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.460 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.461 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl": { 2024-05-08T22:18:07.461 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl", 2024-05-08T22:18:07.461 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.461 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.461 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.461 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.461 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.461 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.461 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.461 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.461 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.461 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.461 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.461 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.461 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl_server_port": { 2024-05-08T22:18:07.461 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl_server_port", 2024-05-08T22:18:07.462 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.462 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.462 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.462 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8443", 2024-05-08T22:18:07.462 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.462 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.462 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.462 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.462 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.462 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.462 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.462 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.462 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_behaviour": { 2024-05-08T22:18:07.462 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_behaviour", 2024-05-08T22:18:07.462 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.463 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.463 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.463 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "redirect", 2024-05-08T22:18:07.463 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.463 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.463 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.463 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.463 INFO:teuthology.orchestra.run.smithi131.stdout: "redirect" 2024-05-08T22:18:07.463 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.463 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.463 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.463 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.463 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.463 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.463 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_error_status_code": { 2024-05-08T22:18:07.464 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_error_status_code", 2024-05-08T22:18:07.464 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.464 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.464 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.464 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "500", 2024-05-08T22:18:07.464 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "400", 2024-05-08T22:18:07.464 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "599", 2024-05-08T22:18:07.464 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.464 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.464 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.464 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.464 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.464 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.464 INFO:teuthology.orchestra.run.smithi131.stdout: "url_prefix": { 2024-05-08T22:18:07.464 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "url_prefix", 2024-05-08T22:18:07.464 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.464 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.465 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.465 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.465 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.465 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.465 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.465 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.465 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.465 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.465 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.465 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.465 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.465 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.465 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.465 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "devicehealth", 2024-05-08T22:18:07.466 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.466 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.466 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.466 INFO:teuthology.orchestra.run.smithi131.stdout: "enable_monitoring": { 2024-05-08T22:18:07.466 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enable_monitoring", 2024-05-08T22:18:07.466 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.466 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.466 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.466 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.466 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.466 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.466 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.466 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "monitor device health metrics", 2024-05-08T22:18:07.466 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.466 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.467 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.467 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.467 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.467 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.467 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.467 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.467 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.467 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.467 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.467 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.467 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.467 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.467 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.467 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.467 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.467 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.468 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.468 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.468 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.468 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.468 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.468 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.468 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.468 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.468 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.468 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.468 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.468 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.468 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.468 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.468 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.468 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.469 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.469 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.469 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.469 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.469 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.469 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.469 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.469 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.469 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.469 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.469 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.469 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.469 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.469 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.469 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.470 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.470 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.470 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.470 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.470 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.470 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.470 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.470 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.470 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.470 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.470 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.470 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.470 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.470 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.470 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.471 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.471 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.471 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.471 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.471 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.471 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.471 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.471 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.471 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.471 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.471 INFO:teuthology.orchestra.run.smithi131.stdout: "mark_out_threshold": { 2024-05-08T22:18:07.471 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mark_out_threshold", 2024-05-08T22:18:07.471 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.471 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.471 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.472 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "2419200", 2024-05-08T22:18:07.472 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.472 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.472 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.472 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T22:18:07.472 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.472 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.472 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.472 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.472 INFO:teuthology.orchestra.run.smithi131.stdout: "pool_name": { 2024-05-08T22:18:07.472 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pool_name", 2024-05-08T22:18:07.472 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.472 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.472 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.472 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "device_health_metrics", 2024-05-08T22:18:07.472 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.472 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.473 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.473 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T22:18:07.473 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.473 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.473 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.473 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.473 INFO:teuthology.orchestra.run.smithi131.stdout: "retention_period": { 2024-05-08T22:18:07.473 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "retention_period", 2024-05-08T22:18:07.473 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.473 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.473 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.473 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15552000", 2024-05-08T22:18:07.473 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.473 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.473 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.474 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long to retain device health metrics", 2024-05-08T22:18:07.474 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.474 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.474 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.474 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.474 INFO:teuthology.orchestra.run.smithi131.stdout: "scrape_frequency": { 2024-05-08T22:18:07.474 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "scrape_frequency", 2024-05-08T22:18:07.474 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.474 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.474 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.474 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "86400", 2024-05-08T22:18:07.474 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.474 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.474 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.475 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T22:18:07.475 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.475 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.475 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.475 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.475 INFO:teuthology.orchestra.run.smithi131.stdout: "self_heal": { 2024-05-08T22:18:07.475 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "self_heal", 2024-05-08T22:18:07.475 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.475 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.475 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.475 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.475 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.475 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.475 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.475 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T22:18:07.475 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.475 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.476 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.476 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.476 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-08T22:18:07.476 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-08T22:18:07.476 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.476 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.476 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.476 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-08T22:18:07.476 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.476 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.476 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.476 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T22:18:07.476 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.476 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.476 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.476 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.476 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_threshold": { 2024-05-08T22:18:07.477 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_threshold", 2024-05-08T22:18:07.477 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.477 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.477 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.477 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "7257600", 2024-05-08T22:18:07.477 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.477 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.477 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.477 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T22:18:07.477 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.477 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.477 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.477 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.477 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.477 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.478 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.478 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "diskprediction_local", 2024-05-08T22:18:07.478 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.478 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.478 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.478 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.478 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.478 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.478 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.478 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.478 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.478 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.478 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.478 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.478 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.479 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.479 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.479 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.479 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.479 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.479 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.479 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.479 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.479 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.479 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.479 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.479 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.479 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.479 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.479 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.479 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.480 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.480 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.480 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.480 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.480 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.480 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.480 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.480 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.480 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.480 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.480 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.480 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.480 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.480 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.480 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.480 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.481 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.481 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.481 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.481 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.481 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.481 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.481 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.481 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.481 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.481 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.481 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.481 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.481 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.481 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.481 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.482 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.482 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.482 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.482 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.482 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.482 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.482 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.482 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.482 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.482 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.482 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.482 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.482 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.482 INFO:teuthology.orchestra.run.smithi131.stdout: "predict_interval": { 2024-05-08T22:18:07.482 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "predict_interval", 2024-05-08T22:18:07.483 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.483 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.483 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.483 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "86400", 2024-05-08T22:18:07.483 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.483 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.483 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.483 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.483 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.483 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.483 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.483 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.483 INFO:teuthology.orchestra.run.smithi131.stdout: "predictor_model": { 2024-05-08T22:18:07.483 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "predictor_model", 2024-05-08T22:18:07.483 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.484 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.484 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.484 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "prophetstor", 2024-05-08T22:18:07.484 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.484 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.484 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.484 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.484 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.484 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.484 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.484 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.484 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-08T22:18:07.484 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-08T22:18:07.484 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.484 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.485 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.485 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-08T22:18:07.485 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.485 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.485 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.485 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.485 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.485 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.485 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.485 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.485 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.485 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.485 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.485 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "influx", 2024-05-08T22:18:07.485 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": false, 2024-05-08T22:18:07.485 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "influxdb python module not found", 2024-05-08T22:18:07.485 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.486 INFO:teuthology.orchestra.run.smithi131.stdout: "batch_size": { 2024-05-08T22:18:07.486 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "batch_size", 2024-05-08T22:18:07.486 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.486 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.486 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.486 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5000", 2024-05-08T22:18:07.486 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.486 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.486 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.486 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T22:18:07.486 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.486 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.486 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.486 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.486 INFO:teuthology.orchestra.run.smithi131.stdout: "database": { 2024-05-08T22:18:07.487 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "database", 2024-05-08T22:18:07.487 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.487 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.487 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.487 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "ceph", 2024-05-08T22:18:07.487 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.487 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.487 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.487 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T22:18:07.487 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.487 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.487 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.487 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.487 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": { 2024-05-08T22:18:07.487 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "hostname", 2024-05-08T22:18:07.487 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.488 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.488 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.488 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.488 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.488 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.488 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.488 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB server hostname", 2024-05-08T22:18:07.488 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.488 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.488 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.488 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.488 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-08T22:18:07.488 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-08T22:18:07.488 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.488 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.488 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.488 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "30", 2024-05-08T22:18:07.489 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "5", 2024-05-08T22:18:07.489 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.489 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.489 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T22:18:07.489 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.489 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.489 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.489 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.489 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.489 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.489 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.489 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.489 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.489 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.489 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.489 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.490 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.490 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.490 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.490 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.490 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.490 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.490 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.490 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.490 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.490 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.490 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.490 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.490 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.490 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.490 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.491 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.491 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.491 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.491 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.491 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.491 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.491 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.491 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.491 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.491 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.491 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.491 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.491 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.491 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.491 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.492 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.492 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.492 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.492 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.492 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.492 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.492 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.492 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.492 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.492 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.492 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.492 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.492 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.492 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.492 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.492 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.493 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.493 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.493 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.493 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.493 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.493 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.493 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.493 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.493 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.493 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.493 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.493 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.493 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.493 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.493 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.494 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.494 INFO:teuthology.orchestra.run.smithi131.stdout: "password": { 2024-05-08T22:18:07.494 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "password", 2024-05-08T22:18:07.494 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.494 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.494 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.494 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.494 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.494 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.494 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.494 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "password of InfluxDB server user", 2024-05-08T22:18:07.494 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.494 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.494 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.495 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.495 INFO:teuthology.orchestra.run.smithi131.stdout: "port": { 2024-05-08T22:18:07.495 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "port", 2024-05-08T22:18:07.495 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.495 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.495 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.495 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8086", 2024-05-08T22:18:07.495 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.495 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.495 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.495 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB server port", 2024-05-08T22:18:07.495 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.495 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.495 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.495 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.496 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl": { 2024-05-08T22:18:07.496 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl", 2024-05-08T22:18:07.496 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.496 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.496 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.496 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "false", 2024-05-08T22:18:07.496 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.496 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.496 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.496 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T22:18:07.496 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.496 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.496 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.496 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.496 INFO:teuthology.orchestra.run.smithi131.stdout: "threads": { 2024-05-08T22:18:07.496 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "threads", 2024-05-08T22:18:07.496 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.497 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.497 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.497 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-05-08T22:18:07.497 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-05-08T22:18:07.497 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "32", 2024-05-08T22:18:07.497 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.497 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T22:18:07.497 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.497 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.497 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.497 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.497 INFO:teuthology.orchestra.run.smithi131.stdout: "username": { 2024-05-08T22:18:07.497 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "username", 2024-05-08T22:18:07.497 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.497 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.498 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.498 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.498 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.498 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.498 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.498 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "username of InfluxDB server user", 2024-05-08T22:18:07.498 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.498 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.498 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.498 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.498 INFO:teuthology.orchestra.run.smithi131.stdout: "verify_ssl": { 2024-05-08T22:18:07.498 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "verify_ssl", 2024-05-08T22:18:07.498 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.498 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.499 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.499 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "true", 2024-05-08T22:18:07.499 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.499 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.499 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.499 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T22:18:07.499 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.499 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.499 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.499 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.499 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.499 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.499 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.499 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "insights", 2024-05-08T22:18:07.500 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.500 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.500 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.500 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.500 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.500 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.500 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.500 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.500 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.500 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.500 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.500 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.500 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.500 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.500 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.500 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.500 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.501 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.501 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.501 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.501 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.501 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.501 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.501 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.501 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.501 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.501 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.501 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.501 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.501 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.501 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.502 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.502 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.502 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.502 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.502 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.502 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.502 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.502 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.502 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.502 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.502 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.502 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.502 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.502 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.502 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.503 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.503 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.503 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.503 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.503 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.503 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.503 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.503 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.503 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.503 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.503 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.503 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.503 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.503 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.503 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.503 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.503 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.504 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.504 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.504 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.504 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.504 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.504 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.504 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.504 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.504 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.504 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.504 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.504 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.504 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.504 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "iostat", 2024-05-08T22:18:07.504 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.505 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.505 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.505 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.505 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.505 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.505 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.505 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.505 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.505 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.505 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.505 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.505 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.505 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.506 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.506 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.506 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.506 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.506 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.506 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.506 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.506 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.506 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.506 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.506 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.506 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.506 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.506 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.507 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.507 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.507 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.507 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.507 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.507 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.507 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.507 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.507 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.507 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.507 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.507 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.507 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.507 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.507 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.508 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.508 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.508 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.508 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.508 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.508 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.508 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.508 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.508 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.508 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.508 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.508 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.508 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.508 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.508 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.508 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.509 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.509 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.509 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.509 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.509 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.509 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.509 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.509 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.509 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.509 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.509 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.509 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.509 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.509 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.509 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.509 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.510 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.510 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "k8sevents", 2024-05-08T22:18:07.510 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.510 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.510 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.510 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_event_retention_days": { 2024-05-08T22:18:07.510 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ceph_event_retention_days", 2024-05-08T22:18:07.510 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.510 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.510 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.510 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "7", 2024-05-08T22:18:07.510 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.510 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.510 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.511 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T22:18:07.511 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.511 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.511 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.511 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.511 INFO:teuthology.orchestra.run.smithi131.stdout: "config_check_secs": { 2024-05-08T22:18:07.511 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_check_secs", 2024-05-08T22:18:07.511 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.511 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.511 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.511 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-08T22:18:07.511 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "10", 2024-05-08T22:18:07.511 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.511 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.511 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T22:18:07.512 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.512 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.512 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.512 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.512 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.512 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.512 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.512 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.512 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.512 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.512 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.512 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.512 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.512 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.512 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.513 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.513 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.513 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.513 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.513 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.513 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.513 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.513 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.513 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.513 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.513 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.513 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.513 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.513 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.513 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.514 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.514 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.514 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.514 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.514 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.514 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.514 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.514 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.514 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.514 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.514 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.514 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.514 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.514 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.515 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.515 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.515 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.515 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.515 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.515 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.515 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.515 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.515 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.515 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.515 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.515 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.515 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.515 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.515 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.515 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.516 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.516 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.516 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.516 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.516 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.516 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.516 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.516 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.516 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.516 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.516 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.516 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.516 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.516 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.516 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.516 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.517 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.517 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "localpool", 2024-05-08T22:18:07.517 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.517 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.517 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.517 INFO:teuthology.orchestra.run.smithi131.stdout: "failure_domain": { 2024-05-08T22:18:07.517 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "failure_domain", 2024-05-08T22:18:07.517 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.517 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.517 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.517 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "host", 2024-05-08T22:18:07.517 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.517 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.517 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.517 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "failure domain for any created local pool", 2024-05-08T22:18:07.517 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T22:18:07.517 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.518 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.518 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.518 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.518 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.518 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.518 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.518 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.518 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.518 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.518 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.518 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.518 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.518 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.518 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.519 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.519 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.519 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.519 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.519 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.519 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.519 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.519 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.519 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.519 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.519 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.519 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.519 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.519 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.519 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.519 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.519 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.520 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.520 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.520 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.520 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.520 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.520 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.520 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.520 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.520 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.520 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.520 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.520 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.520 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.520 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.520 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.521 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.521 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.521 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.521 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.521 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.521 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.521 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.521 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.521 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.521 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.521 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.521 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.521 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.521 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.521 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.521 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.521 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.522 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.522 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.522 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.522 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.522 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.522 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.522 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.522 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.522 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.522 INFO:teuthology.orchestra.run.smithi131.stdout: "min_size": { 2024-05-08T22:18:07.522 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "min_size", 2024-05-08T22:18:07.522 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.522 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.522 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.522 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.523 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.523 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.523 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.523 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default min_size for any created local pool", 2024-05-08T22:18:07.523 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T22:18:07.523 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.523 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.523 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.523 INFO:teuthology.orchestra.run.smithi131.stdout: "num_rep": { 2024-05-08T22:18:07.523 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "num_rep", 2024-05-08T22:18:07.523 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.523 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.523 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.523 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3", 2024-05-08T22:18:07.523 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.523 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.524 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.524 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default replica count for any created local pool", 2024-05-08T22:18:07.524 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.524 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.524 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.524 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.524 INFO:teuthology.orchestra.run.smithi131.stdout: "pg_num": { 2024-05-08T22:18:07.524 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pg_num", 2024-05-08T22:18:07.524 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.524 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.524 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.524 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "128", 2024-05-08T22:18:07.524 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.524 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.524 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.524 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T22:18:07.525 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.525 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.525 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.525 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.525 INFO:teuthology.orchestra.run.smithi131.stdout: "prefix": { 2024-05-08T22:18:07.525 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prefix", 2024-05-08T22:18:07.525 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.525 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.525 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.525 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.525 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.525 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.525 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.525 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "name prefix for any created local pool", 2024-05-08T22:18:07.526 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.526 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.526 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.526 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.526 INFO:teuthology.orchestra.run.smithi131.stdout: "subtree": { 2024-05-08T22:18:07.526 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "subtree", 2024-05-08T22:18:07.526 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.526 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.526 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.526 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "rack", 2024-05-08T22:18:07.526 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.526 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.526 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.527 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T22:18:07.527 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T22:18:07.527 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.527 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.527 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.527 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.527 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.527 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.527 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mds_autoscaler", 2024-05-08T22:18:07.527 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.527 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.527 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.527 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.527 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.528 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.528 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.528 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.528 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.528 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.528 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.528 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.528 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.528 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.528 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.528 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.528 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.528 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.529 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.529 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.529 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.529 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.529 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.529 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.529 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.529 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.529 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.529 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.529 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.529 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.529 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.529 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.530 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.530 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.530 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.530 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.530 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.530 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.530 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.530 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.530 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.530 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.530 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.530 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.530 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.530 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.530 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.530 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.531 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.531 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.531 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.531 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.531 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.531 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.531 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.531 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.531 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.531 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.531 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.531 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.531 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.531 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.531 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.531 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.532 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.532 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.532 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.532 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.532 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.532 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.532 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.532 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.532 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.532 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.532 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.532 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.532 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mirroring", 2024-05-08T22:18:07.532 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.532 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.532 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.533 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.533 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.533 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.533 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.533 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.533 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.533 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.533 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.533 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.533 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.533 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.533 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.533 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.533 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.533 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.534 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.534 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.534 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.534 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.534 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.534 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.534 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.534 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.534 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.534 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.534 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.534 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.534 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.534 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.534 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.535 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.535 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.535 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.535 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.535 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.535 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.535 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.535 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.535 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.535 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.535 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.535 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.535 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.535 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.535 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.535 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.535 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.537 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.537 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.537 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.537 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.537 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.537 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.537 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.537 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.537 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.537 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.537 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.537 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "nfs", 2024-05-08T22:18:07.537 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.537 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.537 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.538 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.538 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.538 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.538 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.538 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.538 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.538 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.538 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.538 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.538 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.538 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.538 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.538 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.538 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.539 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.539 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.539 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.539 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.539 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.539 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.539 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.539 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.539 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.539 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.539 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.539 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.540 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.540 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.540 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.540 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.540 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.540 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.540 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.540 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.540 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.540 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.540 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.540 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.540 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.541 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.541 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.541 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.541 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.541 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.541 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.541 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.541 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.541 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.541 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.541 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.541 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.541 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.541 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.541 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.542 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.542 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.542 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.542 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.542 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.542 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.542 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.542 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.542 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.542 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.542 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.542 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.542 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.542 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.543 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.543 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.543 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.543 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.543 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.543 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "orchestrator", 2024-05-08T22:18:07.543 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.543 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.543 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.543 INFO:teuthology.orchestra.run.smithi131.stdout: "fail_fs": { 2024-05-08T22:18:07.543 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "fail_fs", 2024-05-08T22:18:07.543 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.543 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.543 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.543 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.543 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.544 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.544 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.544 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T22:18:07.544 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.544 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.544 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.544 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.544 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.544 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.544 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.544 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.544 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.544 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.544 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.544 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.545 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.545 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.545 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.545 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.545 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.545 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.545 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.545 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.545 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.545 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.545 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.545 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.545 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.545 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.545 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.546 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.546 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.546 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.546 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.546 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.546 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.546 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.546 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.546 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.546 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.546 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.546 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.546 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.546 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.547 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.547 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.547 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.547 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.547 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.547 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.547 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.547 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.547 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.547 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.547 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.547 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.547 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.547 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.548 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.548 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.548 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.548 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.548 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.548 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.548 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.548 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.548 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.548 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.548 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.548 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.548 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.548 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.548 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.549 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.549 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.549 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.549 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.549 INFO:teuthology.orchestra.run.smithi131.stdout: "orchestrator": { 2024-05-08T22:18:07.549 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "orchestrator", 2024-05-08T22:18:07.549 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.549 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.549 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.549 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.549 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.549 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.549 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.549 INFO:teuthology.orchestra.run.smithi131.stdout: "cephadm", 2024-05-08T22:18:07.549 INFO:teuthology.orchestra.run.smithi131.stdout: "rook", 2024-05-08T22:18:07.550 INFO:teuthology.orchestra.run.smithi131.stdout: "test_orchestrator" 2024-05-08T22:18:07.550 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.550 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Orchestrator backend", 2024-05-08T22:18:07.550 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.550 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.550 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.550 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.550 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.550 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.550 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.550 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "osd_perf_query", 2024-05-08T22:18:07.550 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.550 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.550 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.551 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.551 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.551 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.551 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.551 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.551 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.551 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.551 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.551 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.551 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.551 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.551 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.551 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.551 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.552 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.552 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.552 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.552 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.552 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.552 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.552 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.552 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.552 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.552 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.552 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.552 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.552 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.552 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.552 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.553 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.553 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.553 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.553 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.553 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.553 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.553 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.553 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.553 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.553 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.553 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.553 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.553 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.553 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.554 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.554 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.554 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.554 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.554 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.554 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.554 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.554 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.554 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.554 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.554 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.554 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.554 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.554 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.555 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.555 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.555 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.555 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.555 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.555 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.555 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.555 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.555 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.555 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.555 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.555 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.555 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.555 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.555 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.555 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.556 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "osd_support", 2024-05-08T22:18:07.556 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.556 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.556 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.556 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.556 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.556 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.556 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.556 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.556 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.556 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.556 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.556 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.556 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.557 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.557 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.557 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.557 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.557 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.557 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.557 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.557 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.557 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.557 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.557 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.557 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.557 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.557 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.558 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.558 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.558 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.558 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.558 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.558 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.558 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.558 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.558 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.558 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.558 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.558 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.558 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.558 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.559 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.559 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.559 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.559 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.559 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.559 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.559 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.559 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.559 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.559 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.559 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.559 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.559 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.559 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.559 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.560 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.560 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.560 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.560 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.560 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.560 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.560 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.560 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.560 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.560 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.560 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.560 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.560 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.560 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.560 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.560 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.561 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.561 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.561 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.561 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.561 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pg_autoscaler", 2024-05-08T22:18:07.561 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.561 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.561 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.561 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.561 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.561 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.561 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.561 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.561 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.561 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.562 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.562 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.562 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.562 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.562 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.562 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.562 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.562 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.562 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.562 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.562 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.562 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.562 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.562 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.562 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.563 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.563 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.563 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.563 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.563 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.563 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.563 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.563 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.563 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.563 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.563 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.563 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.563 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.563 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.563 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.564 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.564 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.564 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.564 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.564 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.564 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.564 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.564 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.564 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.564 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.564 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.564 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.564 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.564 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.564 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.565 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.565 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.565 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.565 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.565 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.565 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.565 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.565 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.565 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.565 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.565 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.565 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.565 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.565 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.565 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.566 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.566 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.566 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.566 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-08T22:18:07.566 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-08T22:18:07.566 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.566 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.566 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.566 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-08T22:18:07.566 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.566 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.566 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.566 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.567 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.567 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.567 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.567 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.567 INFO:teuthology.orchestra.run.smithi131.stdout: "threshold": { 2024-05-08T22:18:07.567 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "threshold", 2024-05-08T22:18:07.567 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-08T22:18:07.567 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.567 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.567 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3.0", 2024-05-08T22:18:07.567 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1.0", 2024-05-08T22:18:07.567 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.567 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.567 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "scaling threshold", 2024-05-08T22:18:07.567 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T22:18:07.568 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.568 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.568 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.568 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.568 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.568 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.568 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "progress", 2024-05-08T22:18:07.568 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.568 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.568 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.568 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_pg_recovery_event": { 2024-05-08T22:18:07.568 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_pg_recovery_event", 2024-05-08T22:18:07.568 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.568 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.569 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.569 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.569 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.569 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.569 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.569 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T22:18:07.569 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.569 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.569 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.569 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.569 INFO:teuthology.orchestra.run.smithi131.stdout: "enabled": { 2024-05-08T22:18:07.569 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enabled", 2024-05-08T22:18:07.569 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.569 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.570 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.570 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.570 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.570 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.570 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.570 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.570 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.570 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.570 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.570 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.570 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.570 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.570 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.570 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.571 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.571 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.571 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.571 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.571 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.571 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.571 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.571 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.571 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.571 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.571 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.571 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.571 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.571 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.571 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.572 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.572 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.572 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.572 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.572 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.572 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.572 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.572 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.572 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.572 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.572 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.572 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.572 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.572 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.572 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.573 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.573 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.573 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.573 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.573 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.573 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.573 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.573 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.573 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.573 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.573 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.573 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.573 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.573 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.573 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.574 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.574 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.574 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.574 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.574 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.574 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.574 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.574 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.574 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.574 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.574 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.574 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.574 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.575 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.575 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.575 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.575 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.575 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.575 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.575 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.575 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.575 INFO:teuthology.orchestra.run.smithi131.stdout: "max_completed_events": { 2024-05-08T22:18:07.575 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_completed_events", 2024-05-08T22:18:07.575 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.575 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.575 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.575 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "50", 2024-05-08T22:18:07.576 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.576 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.576 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.576 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "number of past completed events to remember", 2024-05-08T22:18:07.576 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.576 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.576 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.576 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.576 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-08T22:18:07.576 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-08T22:18:07.576 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.576 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.576 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.576 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-05-08T22:18:07.576 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.577 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.577 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.577 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long the module is going to sleep", 2024-05-08T22:18:07.577 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.577 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.577 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.577 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.577 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.577 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.577 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.577 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prometheus", 2024-05-08T22:18:07.577 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.577 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.577 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.577 INFO:teuthology.orchestra.run.smithi131.stdout: "cache": { 2024-05-08T22:18:07.577 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cache", 2024-05-08T22:18:07.578 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.578 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.578 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.578 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.578 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.578 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.578 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.578 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.578 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.578 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.578 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.578 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.578 INFO:teuthology.orchestra.run.smithi131.stdout: "exclude_perf_counters": { 2024-05-08T22:18:07.578 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "exclude_perf_counters", 2024-05-08T22:18:07.578 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.579 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.579 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.579 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.579 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.579 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.579 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.579 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T22:18:07.579 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T22:18:07.579 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.579 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.579 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.579 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.579 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.579 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.579 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.580 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.580 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.580 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.580 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.580 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.580 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.580 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.580 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.580 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.580 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.580 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.580 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.580 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.580 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.580 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.581 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.581 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.581 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.581 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.581 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.581 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.581 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.581 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.581 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.581 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.581 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.581 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.581 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.581 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.582 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.582 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.582 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.582 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.582 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.582 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.582 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.582 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.582 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.582 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.582 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.582 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.582 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.582 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.583 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.583 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.583 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.583 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.583 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.583 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.583 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.583 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.583 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.583 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.583 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.583 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.583 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.583 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.584 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.584 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.584 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.584 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.584 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.584 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.584 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.584 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.584 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.584 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_stats_pools": { 2024-05-08T22:18:07.584 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_stats_pools", 2024-05-08T22:18:07.584 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.584 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.584 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.584 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.585 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.585 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.585 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.585 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.585 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.585 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.585 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.585 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.585 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T22:18:07.585 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T22:18:07.585 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.585 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.585 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.585 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "300", 2024-05-08T22:18:07.585 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.586 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.586 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.586 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.586 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.586 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.586 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.586 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.586 INFO:teuthology.orchestra.run.smithi131.stdout: "scrape_interval": { 2024-05-08T22:18:07.586 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "scrape_interval", 2024-05-08T22:18:07.586 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-08T22:18:07.586 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.586 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.586 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15.0", 2024-05-08T22:18:07.586 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.586 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.587 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.587 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.587 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.587 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.587 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.587 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.587 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-05-08T22:18:07.587 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-05-08T22:18:07.587 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.587 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.587 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.587 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "::", 2024-05-08T22:18:07.587 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.587 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.587 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.587 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T22:18:07.587 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.588 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.588 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.588 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.588 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-05-08T22:18:07.588 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-05-08T22:18:07.588 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.588 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.588 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.588 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "9283", 2024-05-08T22:18:07.588 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.588 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.588 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.588 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T22:18:07.588 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.588 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.588 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.589 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.589 INFO:teuthology.orchestra.run.smithi131.stdout: "stale_cache_strategy": { 2024-05-08T22:18:07.589 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "stale_cache_strategy", 2024-05-08T22:18:07.589 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.589 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.589 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.589 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "log", 2024-05-08T22:18:07.589 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.589 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.589 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.589 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.589 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.589 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.589 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.589 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.589 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_behaviour": { 2024-05-08T22:18:07.589 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_behaviour", 2024-05-08T22:18:07.590 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.590 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.590 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.590 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "default", 2024-05-08T22:18:07.590 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.590 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.590 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.590 INFO:teuthology.orchestra.run.smithi131.stdout: "default", 2024-05-08T22:18:07.590 INFO:teuthology.orchestra.run.smithi131.stdout: "error" 2024-05-08T22:18:07.590 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.590 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.590 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.590 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.590 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.591 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.591 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_error_status_code": { 2024-05-08T22:18:07.591 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_error_status_code", 2024-05-08T22:18:07.591 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.591 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.591 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.591 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "500", 2024-05-08T22:18:07.591 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "400", 2024-05-08T22:18:07.591 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "599", 2024-05-08T22:18:07.591 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.591 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.591 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.591 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.591 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.591 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.591 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.592 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.592 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.592 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_support", 2024-05-08T22:18:07.592 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.592 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.592 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.592 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.592 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.592 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.592 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.592 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.592 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.592 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.592 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.592 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.593 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.593 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.593 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.593 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.593 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.593 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.593 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.593 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.593 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.593 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.593 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.593 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.593 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.593 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.594 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.594 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.594 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.594 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.594 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.594 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.594 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.594 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.594 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.594 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.594 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.594 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.594 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.594 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.595 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.595 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.595 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.595 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.595 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.595 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.595 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.595 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.595 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.595 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.595 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.595 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.595 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.595 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.595 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.596 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.596 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.596 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.596 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.596 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.596 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.596 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.596 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.596 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.596 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.596 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.596 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.596 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.596 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.596 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.597 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.597 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.597 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.597 INFO:teuthology.orchestra.run.smithi131.stdout: "max_concurrent_snap_create": { 2024-05-08T22:18:07.597 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_concurrent_snap_create", 2024-05-08T22:18:07.597 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.597 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.597 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.597 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-08T22:18:07.597 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.597 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.597 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.597 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.597 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.597 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.598 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.598 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.598 INFO:teuthology.orchestra.run.smithi131.stdout: "mirror_snapshot_schedule": { 2024-05-08T22:18:07.598 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T22:18:07.598 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.598 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.598 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.598 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.598 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.598 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.598 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.598 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.598 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.598 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.599 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.599 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.599 INFO:teuthology.orchestra.run.smithi131.stdout: "trash_purge_schedule": { 2024-05-08T22:18:07.599 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "trash_purge_schedule", 2024-05-08T22:18:07.599 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.599 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.599 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.599 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.599 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.599 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.599 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.599 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.599 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.599 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.599 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.599 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.599 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.600 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.600 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.600 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "restful", 2024-05-08T22:18:07.600 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.600 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.600 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.600 INFO:teuthology.orchestra.run.smithi131.stdout: "enable_auth": { 2024-05-08T22:18:07.600 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enable_auth", 2024-05-08T22:18:07.600 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.600 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.600 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.600 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.600 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.600 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.600 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.601 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.601 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.601 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.601 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.601 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.601 INFO:teuthology.orchestra.run.smithi131.stdout: "key_file": { 2024-05-08T22:18:07.601 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "key_file", 2024-05-08T22:18:07.601 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.601 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.601 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.601 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.601 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.601 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.601 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.601 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.602 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.602 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.602 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.602 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.602 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.602 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.602 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.602 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.602 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.602 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.602 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.602 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.602 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.602 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.603 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.603 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.603 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.603 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.603 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.603 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.603 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.603 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.603 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.603 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.603 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.603 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.603 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.603 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.603 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.604 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.604 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.604 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.604 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.604 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.604 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.604 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.604 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.604 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.604 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.604 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.604 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.604 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.605 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.605 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.605 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.605 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.605 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.605 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.605 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.605 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.605 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.605 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.605 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.606 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.606 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.606 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.606 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.606 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.606 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.606 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.606 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.606 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.606 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.606 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.606 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.606 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.606 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.607 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.607 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.607 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.607 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.607 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.607 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.607 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.607 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-05-08T22:18:07.607 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-05-08T22:18:07.607 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.607 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.607 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.607 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.607 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.607 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.608 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.608 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.608 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.608 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.608 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.608 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.608 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-05-08T22:18:07.608 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-05-08T22:18:07.608 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.608 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.608 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.608 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.608 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.608 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.609 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.609 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.609 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.609 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.609 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.609 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.609 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.609 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.609 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.609 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rgw", 2024-05-08T22:18:07.609 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.609 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.609 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.609 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.609 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.610 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.610 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.610 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.610 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.610 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.610 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.610 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.610 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.610 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.610 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.610 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.610 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.610 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.610 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.611 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.611 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.611 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.611 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.611 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.611 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.611 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.611 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.611 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.611 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.611 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.611 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.611 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.611 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.611 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.612 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.612 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.612 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.612 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.612 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.612 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.612 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.612 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.612 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.612 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.612 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.612 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.612 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.612 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.612 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.612 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.613 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.613 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.613 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.613 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.613 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.613 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.613 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.613 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.613 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.613 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.613 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.613 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.613 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.613 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.613 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.614 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.614 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.614 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.614 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.614 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.614 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.614 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.614 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.614 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.614 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.614 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.614 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rook", 2024-05-08T22:18:07.614 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.614 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.614 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.615 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.615 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.615 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.615 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.615 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.615 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.615 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.615 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.615 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.615 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.615 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.615 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.615 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.615 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.615 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.616 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.616 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.616 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.616 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.616 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.616 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.616 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.616 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.616 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.616 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.616 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.616 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.616 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.616 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.617 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.617 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.617 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.617 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.617 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.617 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.617 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.617 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.617 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.617 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.617 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.617 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.617 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.617 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.617 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.618 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.618 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.618 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.618 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.618 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.618 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.618 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.618 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.618 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.618 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.618 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.618 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.618 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.619 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.619 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.619 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.619 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.619 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.619 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.619 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.619 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.619 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.619 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.619 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.619 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.619 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.619 INFO:teuthology.orchestra.run.smithi131.stdout: "storage_class": { 2024-05-08T22:18:07.619 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "storage_class", 2024-05-08T22:18:07.620 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.620 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.620 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.620 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "local", 2024-05-08T22:18:07.620 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.620 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.620 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.620 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T22:18:07.620 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.620 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.620 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.620 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.620 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.620 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.620 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.621 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "selftest", 2024-05-08T22:18:07.621 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.621 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.621 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.621 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.621 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.621 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.621 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.621 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.621 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.621 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.621 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.621 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.621 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.621 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.622 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.622 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.622 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.622 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.622 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.622 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.622 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.622 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.622 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.622 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.622 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.622 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.622 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.622 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.622 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.623 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.623 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.623 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.623 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.623 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.623 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.623 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.623 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.623 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.623 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.623 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.623 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.623 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.623 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.623 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.623 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.624 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.624 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.624 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.624 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.624 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.624 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.624 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.624 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.624 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.624 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.624 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.624 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.624 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.624 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.624 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.625 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.625 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.625 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.625 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.625 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.625 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.625 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.625 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.625 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.625 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.625 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.625 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.625 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.625 INFO:teuthology.orchestra.run.smithi131.stdout: "roption1": { 2024-05-08T22:18:07.625 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "roption1", 2024-05-08T22:18:07.626 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.626 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.626 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.626 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.626 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.626 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.626 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.626 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.626 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.626 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.626 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.626 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.626 INFO:teuthology.orchestra.run.smithi131.stdout: "roption2": { 2024-05-08T22:18:07.626 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "roption2", 2024-05-08T22:18:07.627 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.627 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.627 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.627 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "xyz", 2024-05-08T22:18:07.627 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.627 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.627 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.627 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.627 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.627 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.627 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.627 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.627 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption1": { 2024-05-08T22:18:07.627 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption1", 2024-05-08T22:18:07.627 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.627 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.628 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.628 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.628 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.628 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.628 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.628 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.628 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.628 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.628 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.628 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.628 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption2": { 2024-05-08T22:18:07.628 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption2", 2024-05-08T22:18:07.628 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.628 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.629 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.629 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.629 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.629 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.629 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.629 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.629 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.629 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.629 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.629 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.629 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption3": { 2024-05-08T22:18:07.629 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption3", 2024-05-08T22:18:07.629 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-08T22:18:07.629 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.629 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.629 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.630 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.630 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.630 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.630 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.630 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.630 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.630 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.630 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.630 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption4": { 2024-05-08T22:18:07.630 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption4", 2024-05-08T22:18:07.630 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.630 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.630 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.630 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.631 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.631 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.631 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.631 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.631 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.631 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.631 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.631 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.631 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption5": { 2024-05-08T22:18:07.631 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption5", 2024-05-08T22:18:07.631 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.631 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.631 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.631 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.631 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.632 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.632 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.632 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.632 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.632 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.632 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.632 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.632 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption6": { 2024-05-08T22:18:07.632 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption6", 2024-05-08T22:18:07.632 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.632 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.632 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.632 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.632 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.632 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.632 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.633 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.633 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.633 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.633 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.633 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.633 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption7": { 2024-05-08T22:18:07.633 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption7", 2024-05-08T22:18:07.633 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.633 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.633 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.633 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.633 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-05-08T22:18:07.633 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "42", 2024-05-08T22:18:07.633 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.633 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.633 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.634 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.634 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.634 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.634 INFO:teuthology.orchestra.run.smithi131.stdout: "testkey": { 2024-05-08T22:18:07.634 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testkey", 2024-05-08T22:18:07.634 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.634 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.634 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.634 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.634 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.634 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.634 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.634 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.634 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.635 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.635 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.635 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.635 INFO:teuthology.orchestra.run.smithi131.stdout: "testlkey": { 2024-05-08T22:18:07.635 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testlkey", 2024-05-08T22:18:07.635 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.635 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.635 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.635 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.635 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.635 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.635 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.635 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.635 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.635 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.636 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.636 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.636 INFO:teuthology.orchestra.run.smithi131.stdout: "testnewline": { 2024-05-08T22:18:07.636 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testnewline", 2024-05-08T22:18:07.636 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.636 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.636 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.636 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.636 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.636 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.636 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.636 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.636 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.636 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.636 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.637 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.637 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.637 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.637 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.637 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snap_schedule", 2024-05-08T22:18:07.637 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.637 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.637 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.637 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_m_granularity": { 2024-05-08T22:18:07.637 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_m_granularity", 2024-05-08T22:18:07.637 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.637 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.637 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.637 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.637 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.638 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.638 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.638 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T22:18:07.638 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.638 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.638 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.638 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.638 INFO:teuthology.orchestra.run.smithi131.stdout: "dump_on_update": { 2024-05-08T22:18:07.638 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "dump_on_update", 2024-05-08T22:18:07.638 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.638 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.638 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.638 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.638 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.638 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.639 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.639 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "dump database to debug log on update", 2024-05-08T22:18:07.639 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.639 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.639 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.639 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.639 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.639 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.639 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.639 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.639 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.639 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.639 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.639 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.639 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.639 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.640 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.640 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.640 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.640 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.640 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.640 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.640 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.640 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.640 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.640 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.640 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.640 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.640 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.640 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.640 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.640 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.642 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.642 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.642 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.642 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.642 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.642 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.642 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.642 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.642 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.642 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.642 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.642 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.642 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.642 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.642 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.643 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.643 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.643 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.643 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.643 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.643 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.643 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.643 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.643 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.643 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.643 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.643 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.643 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.643 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.643 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.643 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "stats", 2024-05-08T22:18:07.643 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.644 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.644 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.644 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.644 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.644 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.644 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.644 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.644 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.644 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.644 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.644 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.644 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.644 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.644 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.644 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.645 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.645 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.645 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.645 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.645 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.645 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.645 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.645 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.645 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.645 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.645 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.645 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.645 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.645 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.645 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.646 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.646 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.646 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.646 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.646 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.646 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.646 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.646 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.646 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.646 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.646 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.646 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.646 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.646 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.646 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.646 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.647 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.647 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.647 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.647 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.647 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.647 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.647 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.647 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.647 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.647 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.647 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.647 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.647 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.647 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.647 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.647 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.648 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.648 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.648 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.648 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.648 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.648 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.648 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.648 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.648 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.648 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.648 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.648 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.648 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.648 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "status", 2024-05-08T22:18:07.648 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.648 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.649 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.649 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.649 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.649 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.649 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.649 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.649 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.649 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.649 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.649 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.649 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.649 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.649 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.649 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.649 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.650 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.650 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.650 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.650 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.650 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.650 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.650 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.650 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.650 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.650 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.650 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.650 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.650 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.650 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.650 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.650 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.651 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.651 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.651 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.651 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.651 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.651 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.651 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.651 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.651 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.651 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.651 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.651 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.651 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.651 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.651 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.652 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.652 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.652 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.652 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.652 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.652 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.652 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.652 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.652 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.652 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.652 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.652 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.652 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.652 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.652 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.653 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.653 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.653 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.653 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.653 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.653 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.653 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.653 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.653 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.653 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.653 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.653 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.653 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.653 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "telegraf", 2024-05-08T22:18:07.653 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.653 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.653 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.654 INFO:teuthology.orchestra.run.smithi131.stdout: "address": { 2024-05-08T22:18:07.654 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "address", 2024-05-08T22:18:07.654 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.654 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.654 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.654 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T22:18:07.654 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.654 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.654 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.654 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.654 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.654 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.654 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.654 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.654 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-08T22:18:07.655 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-08T22:18:07.655 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.655 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.655 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.655 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15", 2024-05-08T22:18:07.655 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.655 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.655 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.655 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.655 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.655 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.655 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.655 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.655 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "telemetry", 2024-05-08T22:18:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_basic": { 2024-05-08T22:18:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_basic", 2024-05-08T22:18:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T22:18:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_crash": { 2024-05-08T22:18:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_crash", 2024-05-08T22:18:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T22:18:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_device": { 2024-05-08T22:18:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_device", 2024-05-08T22:18:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T22:18:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_ident": { 2024-05-08T22:18:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_ident", 2024-05-08T22:18:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T22:18:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_perf": { 2024-05-08T22:18:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_perf", 2024-05-08T22:18:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T22:18:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: "contact": { 2024-05-08T22:18:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "contact", 2024-05-08T22:18:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: "description": { 2024-05-08T22:18:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "description", 2024-05-08T22:18:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.667 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.667 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.667 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.667 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.667 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.667 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.667 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.667 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.667 INFO:teuthology.orchestra.run.smithi131.stdout: "device_url": { 2024-05-08T22:18:07.667 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_url", 2024-05-08T22:18:07.667 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.667 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.667 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.667 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T22:18:07.667 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.668 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.668 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.668 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.668 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.668 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.668 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.668 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.668 INFO:teuthology.orchestra.run.smithi131.stdout: "enabled": { 2024-05-08T22:18:07.668 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enabled", 2024-05-08T22:18:07.668 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.668 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.668 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.668 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.668 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.668 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.668 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.668 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.668 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.669 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.669 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.669 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.669 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-08T22:18:07.669 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-08T22:18:07.669 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.669 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.669 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.669 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "24", 2024-05-08T22:18:07.669 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "8", 2024-05-08T22:18:07.669 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.669 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.669 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.669 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.669 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.670 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.670 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.670 INFO:teuthology.orchestra.run.smithi131.stdout: "last_opt_revision": { 2024-05-08T22:18:07.670 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "last_opt_revision", 2024-05-08T22:18:07.670 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.670 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.670 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.670 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1", 2024-05-08T22:18:07.670 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.670 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.670 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.670 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.670 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.670 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.670 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.671 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.671 INFO:teuthology.orchestra.run.smithi131.stdout: "leaderboard": { 2024-05-08T22:18:07.671 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "leaderboard", 2024-05-08T22:18:07.671 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.671 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.671 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.671 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.671 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.671 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.671 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.671 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.671 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.671 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.671 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.671 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.671 INFO:teuthology.orchestra.run.smithi131.stdout: "leaderboard_description": { 2024-05-08T22:18:07.671 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "leaderboard_description", 2024-05-08T22:18:07.672 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.672 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.672 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.672 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.672 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.672 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.672 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.672 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.672 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.672 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.672 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.672 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.672 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.672 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.673 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.673 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.673 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.673 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.673 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.673 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.673 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.673 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.673 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.673 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.673 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.673 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.673 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.673 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.673 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.673 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.673 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.674 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.674 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.674 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.674 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.674 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.674 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.674 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.674 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.674 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.674 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.674 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.674 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.674 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.674 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.674 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.674 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.675 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.675 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.675 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.675 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.675 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.675 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.675 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.675 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.675 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.675 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.675 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.675 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.675 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.675 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.675 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.675 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.676 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.676 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.676 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.676 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.676 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.676 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.676 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.676 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.676 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.676 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.676 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.676 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.676 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.676 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.676 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.676 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.677 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.677 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.677 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.677 INFO:teuthology.orchestra.run.smithi131.stdout: "organization": { 2024-05-08T22:18:07.677 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "organization", 2024-05-08T22:18:07.677 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.677 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.677 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.677 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.677 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.677 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.677 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.677 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.677 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.677 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.677 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.678 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.678 INFO:teuthology.orchestra.run.smithi131.stdout: "proxy": { 2024-05-08T22:18:07.678 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "proxy", 2024-05-08T22:18:07.678 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.678 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.678 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.678 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.678 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.678 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.678 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.678 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.678 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.678 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.678 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.678 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.678 INFO:teuthology.orchestra.run.smithi131.stdout: "url": { 2024-05-08T22:18:07.678 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "url", 2024-05-08T22:18:07.679 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.679 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.679 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.679 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T22:18:07.679 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.679 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.679 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.679 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.679 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.679 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.679 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.679 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.679 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.679 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.679 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.679 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "test_orchestrator", 2024-05-08T22:18:07.679 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.680 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.680 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.680 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.680 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.680 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.680 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.680 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.680 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.680 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.680 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.680 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.680 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.680 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.680 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.680 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.680 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.681 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.681 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.681 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.681 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.681 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.681 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.681 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.681 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.681 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.681 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.681 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.681 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.681 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.681 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.682 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.682 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.682 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.682 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.682 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.682 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.682 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.682 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.682 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.682 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.682 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.682 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.682 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.682 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.683 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.683 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.683 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.683 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.683 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.683 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.683 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.683 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.683 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.683 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.683 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.683 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.683 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.683 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.683 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.683 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.684 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.684 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.684 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.684 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.684 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.684 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.684 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.684 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.684 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.684 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.684 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.684 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.684 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.684 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.684 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.684 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "volumes", 2024-05-08T22:18:07.684 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.685 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.685 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.685 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.685 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.685 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.685 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.685 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.685 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.685 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.685 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.685 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.685 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.685 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.685 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.685 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.685 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.686 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.686 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.686 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.686 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.686 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.686 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.686 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.686 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.686 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.686 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.686 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.686 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.686 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.686 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.686 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.687 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.687 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.687 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.687 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.687 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.687 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.687 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.687 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.687 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.687 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.687 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.687 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.687 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.687 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.687 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.687 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.687 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.688 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.688 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.688 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.688 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.688 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.688 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.688 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.688 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.688 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.688 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.688 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.688 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.688 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.688 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.688 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.689 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.689 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.689 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.689 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.689 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.689 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.689 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.689 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.689 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.689 INFO:teuthology.orchestra.run.smithi131.stdout: "max_concurrent_clones": { 2024-05-08T22:18:07.689 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_concurrent_clones", 2024-05-08T22:18:07.689 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.689 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.689 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.689 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "4", 2024-05-08T22:18:07.689 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.689 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.690 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.690 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T22:18:07.690 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.690 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.690 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.690 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.690 INFO:teuthology.orchestra.run.smithi131.stdout: "snapshot_clone_delay": { 2024-05-08T22:18:07.690 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snapshot_clone_delay", 2024-05-08T22:18:07.690 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.690 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.690 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.690 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-08T22:18:07.690 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.690 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.690 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.691 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T22:18:07.691 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.691 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.691 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.691 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.691 INFO:teuthology.orchestra.run.smithi131.stdout: "snapshot_clone_no_wait": { 2024-05-08T22:18:07.691 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T22:18:07.691 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.691 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.691 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.691 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.691 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.691 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.691 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.691 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T22:18:07.691 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.691 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.692 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.692 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.692 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.692 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.692 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.692 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix", 2024-05-08T22:18:07.692 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.692 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.692 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.692 INFO:teuthology.orchestra.run.smithi131.stdout: "discovery_interval": { 2024-05-08T22:18:07.692 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "discovery_interval", 2024-05-08T22:18:07.692 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-08T22:18:07.692 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.692 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.692 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "100", 2024-05-08T22:18:07.692 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.692 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.693 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.693 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.693 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.693 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.693 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.693 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.693 INFO:teuthology.orchestra.run.smithi131.stdout: "identifier": { 2024-05-08T22:18:07.693 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "identifier", 2024-05-08T22:18:07.693 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.693 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.693 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.693 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.693 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.693 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.693 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.693 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.694 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.694 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.694 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.694 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.694 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-08T22:18:07.694 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-08T22:18:07.694 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.694 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.694 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.694 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-08T22:18:07.694 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.694 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.694 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.694 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.694 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.695 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.695 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.695 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.695 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.695 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.695 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.695 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.695 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.695 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.695 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.695 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.695 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.695 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.695 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.695 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.696 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.696 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.696 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.696 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.696 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.696 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.696 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.696 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.696 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.696 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.696 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.696 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.696 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.696 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.696 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.696 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.697 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.697 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.697 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.697 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.697 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.697 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.697 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.697 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.697 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.697 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.697 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.697 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.697 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.697 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.697 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.697 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.697 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.698 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.698 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.698 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.698 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.698 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.698 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.698 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.698 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.698 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.698 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.698 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.698 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.698 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.698 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.698 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.699 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.699 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.699 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.699 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.699 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.699 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.699 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.699 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.699 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.699 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.699 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_host": { 2024-05-08T22:18:07.699 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_host", 2024-05-08T22:18:07.699 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.699 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.699 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.699 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.700 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.700 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.700 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.700 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.700 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.700 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.700 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.700 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.700 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_port": { 2024-05-08T22:18:07.700 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_port", 2024-05-08T22:18:07.700 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.700 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.700 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.700 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10051", 2024-05-08T22:18:07.700 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.700 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.701 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.701 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.701 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.701 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.701 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.701 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.701 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_sender": { 2024-05-08T22:18:07.701 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_sender", 2024-05-08T22:18:07.701 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.701 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.701 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.701 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T22:18:07.701 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.701 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.701 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.701 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.701 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.702 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.702 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.702 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.702 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.702 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.702 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-08T22:18:07.702 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.702 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.702 INFO:teuthology.orchestra.run.smithi131.stdout: "gid": 25402, 2024-05-08T22:18:07.702 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "z", 2024-05-08T22:18:07.702 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr_features": 4540138322906710015, 2024-05-08T22:18:07.702 INFO:teuthology.orchestra.run.smithi131.stdout: "available_modules": [ 2024-05-08T22:18:07.702 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.702 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "alerts", 2024-05-08T22:18:07.702 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.703 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.703 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.703 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-08T22:18:07.703 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-08T22:18:07.703 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.703 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.703 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.703 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-08T22:18:07.703 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.703 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.703 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.703 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T22:18:07.703 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.703 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.703 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.703 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.704 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.704 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.704 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.704 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.704 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.704 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.704 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.704 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.704 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.704 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.704 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.704 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.704 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.704 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.704 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.705 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.705 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.705 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.705 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.705 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.705 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.705 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.705 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.705 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.705 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.705 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.705 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.705 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.705 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.705 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.706 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.706 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.706 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.706 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.706 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.706 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.706 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.706 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.706 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.706 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.706 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.706 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.706 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.706 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.706 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.707 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.707 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.707 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.707 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.707 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.707 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.707 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.707 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.707 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.707 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.707 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.707 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.707 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.707 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.707 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.708 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.708 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.708 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.708 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.708 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.708 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.708 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.708 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.708 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.708 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.708 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_destination": { 2024-05-08T22:18:07.708 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_destination", 2024-05-08T22:18:07.708 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.708 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.708 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.708 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.708 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.709 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.709 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.709 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Email address to send alerts to", 2024-05-08T22:18:07.709 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.709 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.709 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.709 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.709 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_from_name": { 2024-05-08T22:18:07.709 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_from_name", 2024-05-08T22:18:07.709 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.709 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.709 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.709 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "Ceph", 2024-05-08T22:18:07.709 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.709 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.710 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.710 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Email From: name", 2024-05-08T22:18:07.710 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.710 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.710 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.710 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.710 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_host": { 2024-05-08T22:18:07.710 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_host", 2024-05-08T22:18:07.710 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.710 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.710 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.710 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.710 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.710 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.710 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.710 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP server", 2024-05-08T22:18:07.711 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.711 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.711 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.711 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.711 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_password": { 2024-05-08T22:18:07.711 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_password", 2024-05-08T22:18:07.711 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.711 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.711 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.711 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.711 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.711 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.711 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.711 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Password to authenticate with", 2024-05-08T22:18:07.711 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.711 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.712 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.712 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.712 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_port": { 2024-05-08T22:18:07.712 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_port", 2024-05-08T22:18:07.712 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.712 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.712 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.712 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "465", 2024-05-08T22:18:07.712 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.712 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.712 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.712 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP port", 2024-05-08T22:18:07.712 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.712 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.712 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.712 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.713 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_sender": { 2024-05-08T22:18:07.713 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_sender", 2024-05-08T22:18:07.713 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.713 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.713 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.713 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.713 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.713 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.713 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.713 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP envelope sender", 2024-05-08T22:18:07.713 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.713 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.713 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.713 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.713 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_ssl": { 2024-05-08T22:18:07.713 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_ssl", 2024-05-08T22:18:07.713 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.714 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.714 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.714 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.714 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.714 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.714 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.714 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T22:18:07.714 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.714 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.714 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.714 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.714 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_user": { 2024-05-08T22:18:07.714 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_user", 2024-05-08T22:18:07.714 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.714 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.715 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.715 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.715 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.715 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.715 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.715 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "User to authenticate as", 2024-05-08T22:18:07.715 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.715 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.715 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.715 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.715 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.715 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.715 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.715 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "balancer", 2024-05-08T22:18:07.715 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.715 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.715 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.716 INFO:teuthology.orchestra.run.smithi131.stdout: "active": { 2024-05-08T22:18:07.716 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "active", 2024-05-08T22:18:07.716 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.716 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.716 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.716 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.716 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.716 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.716 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.716 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T22:18:07.716 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.716 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.716 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.716 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.716 INFO:teuthology.orchestra.run.smithi131.stdout: "begin_time": { 2024-05-08T22:18:07.716 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "begin_time", 2024-05-08T22:18:07.717 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.717 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.717 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.717 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0000", 2024-05-08T22:18:07.717 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.717 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.717 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.717 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T22:18:07.717 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T22:18:07.717 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.717 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.717 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.717 INFO:teuthology.orchestra.run.smithi131.stdout: "begin_weekday": { 2024-05-08T22:18:07.717 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "begin_weekday", 2024-05-08T22:18:07.718 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-08T22:18:07.718 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.718 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.718 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-08T22:18:07.718 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0", 2024-05-08T22:18:07.718 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "6", 2024-05-08T22:18:07.718 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.718 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T22:18:07.718 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T22:18:07.718 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.718 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.718 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.718 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_max_iterations": { 2024-05-08T22:18:07.718 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_max_iterations", 2024-05-08T22:18:07.718 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-08T22:18:07.718 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.719 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.719 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "25", 2024-05-08T22:18:07.719 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-05-08T22:18:07.719 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "250", 2024-05-08T22:18:07.719 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.719 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T22:18:07.719 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.719 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.719 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.719 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.719 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_metrics": { 2024-05-08T22:18:07.719 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_metrics", 2024-05-08T22:18:07.719 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.719 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.719 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.720 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T22:18:07.720 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.720 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.720 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.720 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T22:18:07.720 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T22:18:07.720 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.720 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.720 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.720 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_step": { 2024-05-08T22:18:07.720 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_step", 2024-05-08T22:18:07.720 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-08T22:18:07.720 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.720 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.720 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0.5", 2024-05-08T22:18:07.720 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0.001", 2024-05-08T22:18:07.720 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "0.999", 2024-05-08T22:18:07.721 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.721 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "aggressiveness of optimization", 2024-05-08T22:18:07.721 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T22:18:07.721 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.721 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.721 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.721 INFO:teuthology.orchestra.run.smithi131.stdout: "end_time": { 2024-05-08T22:18:07.721 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "end_time", 2024-05-08T22:18:07.721 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.721 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.721 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.721 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "2359", 2024-05-08T22:18:07.721 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.721 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.721 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.721 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T22:18:07.722 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T22:18:07.722 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.722 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.722 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.722 INFO:teuthology.orchestra.run.smithi131.stdout: "end_weekday": { 2024-05-08T22:18:07.722 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "end_weekday", 2024-05-08T22:18:07.722 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-08T22:18:07.722 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.722 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.722 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-08T22:18:07.722 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0", 2024-05-08T22:18:07.722 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "6", 2024-05-08T22:18:07.722 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.722 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T22:18:07.722 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T22:18:07.722 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.723 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.723 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.723 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.723 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.723 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.723 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.723 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.723 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.723 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.723 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.723 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.723 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.723 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.723 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.723 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.724 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.724 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.724 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.724 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.724 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.724 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.724 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.724 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.724 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.724 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.724 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.724 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.724 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.724 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.724 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.725 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.725 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.725 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.725 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.725 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.725 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.725 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.725 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.725 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.725 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.725 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.725 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.725 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.725 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.725 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.725 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.725 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.726 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.726 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.726 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.726 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.726 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.726 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.726 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.726 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.726 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.726 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.726 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.726 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.726 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.726 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.726 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.726 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.727 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.727 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.727 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.727 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.727 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.727 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.727 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.727 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.727 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.727 INFO:teuthology.orchestra.run.smithi131.stdout: "min_score": { 2024-05-08T22:18:07.727 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "min_score", 2024-05-08T22:18:07.727 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-08T22:18:07.727 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.727 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.728 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-08T22:18:07.728 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.728 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.728 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.728 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T22:18:07.728 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.728 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.728 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.728 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.728 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": { 2024-05-08T22:18:07.728 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mode", 2024-05-08T22:18:07.728 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.728 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.728 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.728 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "upmap", 2024-05-08T22:18:07.729 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.729 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.729 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.729 INFO:teuthology.orchestra.run.smithi131.stdout: "crush-compat", 2024-05-08T22:18:07.729 INFO:teuthology.orchestra.run.smithi131.stdout: "none", 2024-05-08T22:18:07.729 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap" 2024-05-08T22:18:07.729 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.729 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Balancer mode", 2024-05-08T22:18:07.729 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.729 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.729 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.729 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.729 INFO:teuthology.orchestra.run.smithi131.stdout: "pool_ids": { 2024-05-08T22:18:07.730 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pool_ids", 2024-05-08T22:18:07.730 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.730 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.730 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.730 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.730 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.730 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.730 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.730 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T22:18:07.730 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.730 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.730 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.730 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.730 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-08T22:18:07.731 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-08T22:18:07.731 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.731 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.731 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.731 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-08T22:18:07.731 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.731 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.731 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.731 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T22:18:07.731 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.731 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.731 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.731 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.731 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap_max_deviation": { 2024-05-08T22:18:07.731 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "upmap_max_deviation", 2024-05-08T22:18:07.732 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.732 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.732 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.732 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-05-08T22:18:07.732 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-05-08T22:18:07.732 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.732 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.732 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T22:18:07.732 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T22:18:07.732 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.732 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.732 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.732 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap_max_optimizations": { 2024-05-08T22:18:07.732 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "upmap_max_optimizations", 2024-05-08T22:18:07.732 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-08T22:18:07.732 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.733 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.733 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-08T22:18:07.733 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.733 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.733 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.733 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T22:18:07.733 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.733 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.733 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.733 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.733 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.733 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.733 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.733 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cephadm", 2024-05-08T22:18:07.733 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.733 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.734 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.734 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_down_multiplier": { 2024-05-08T22:18:07.734 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_down_multiplier", 2024-05-08T22:18:07.734 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-08T22:18:07.734 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.734 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.734 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3.0", 2024-05-08T22:18:07.734 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.734 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.734 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.734 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T22:18:07.734 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.734 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.734 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.734 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.734 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_refresh_rate": { 2024-05-08T22:18:07.735 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_refresh_rate", 2024-05-08T22:18:07.735 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.735 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.735 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.735 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "20", 2024-05-08T22:18:07.735 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.735 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.735 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.735 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T22:18:07.735 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.735 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.735 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.735 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.735 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_starting_port": { 2024-05-08T22:18:07.735 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_starting_port", 2024-05-08T22:18:07.735 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.736 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.736 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.736 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "4721", 2024-05-08T22:18:07.736 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.736 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.736 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.736 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T22:18:07.736 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.736 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.736 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.736 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.736 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_ptrace": { 2024-05-08T22:18:07.736 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_ptrace", 2024-05-08T22:18:07.736 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.736 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.736 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.736 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.737 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.737 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.737 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.737 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T22:18:07.737 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T22:18:07.737 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.737 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.737 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.737 INFO:teuthology.orchestra.run.smithi131.stdout: "autotune_interval": { 2024-05-08T22:18:07.737 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "autotune_interval", 2024-05-08T22:18:07.737 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.737 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.737 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.737 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-08T22:18:07.737 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.737 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.738 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.738 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T22:18:07.738 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.738 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.738 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.738 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.738 INFO:teuthology.orchestra.run.smithi131.stdout: "autotune_memory_target_ratio": { 2024-05-08T22:18:07.738 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T22:18:07.738 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-08T22:18:07.738 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.738 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.738 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0.7", 2024-05-08T22:18:07.738 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.738 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.738 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.739 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T22:18:07.739 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.739 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.739 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.739 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.739 INFO:teuthology.orchestra.run.smithi131.stdout: "cgroups_split": { 2024-05-08T22:18:07.739 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cgroups_split", 2024-05-08T22:18:07.739 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.739 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.739 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.739 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.739 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.739 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.739 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.739 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T22:18:07.739 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.739 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.740 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.740 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.740 INFO:teuthology.orchestra.run.smithi131.stdout: "config_checks_enabled": { 2024-05-08T22:18:07.740 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_checks_enabled", 2024-05-08T22:18:07.740 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.740 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.740 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.740 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.740 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.740 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.740 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.740 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T22:18:07.740 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.740 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.740 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.741 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.741 INFO:teuthology.orchestra.run.smithi131.stdout: "config_dashboard": { 2024-05-08T22:18:07.741 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_dashboard", 2024-05-08T22:18:07.741 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.741 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.741 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.741 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.741 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.741 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.741 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.741 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T22:18:07.741 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.741 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.741 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.741 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.741 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_alertmanager": { 2024-05-08T22:18:07.742 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_alertmanager", 2024-05-08T22:18:07.742 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.742 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.742 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.742 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T22:18:07.742 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.742 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.742 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.742 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-05-08T22:18:07.742 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.742 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.742 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.742 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.742 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_base": { 2024-05-08T22:18:07.742 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_base", 2024-05-08T22:18:07.743 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.743 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.743 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.743 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T22:18:07.743 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.743 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.743 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.743 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Container image name, without the tag", 2024-05-08T22:18:07.743 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.743 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.743 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.743 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.743 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_elasticsearch": { 2024-05-08T22:18:07.743 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_elasticsearch", 2024-05-08T22:18:07.743 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.744 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.744 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.744 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T22:18:07.744 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.744 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.744 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.744 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "elasticsearch container image", 2024-05-08T22:18:07.744 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.744 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.744 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.744 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.744 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_grafana": { 2024-05-08T22:18:07.744 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_grafana", 2024-05-08T22:18:07.744 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.744 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.744 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.745 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-08T22:18:07.745 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.745 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.745 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.745 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-05-08T22:18:07.745 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.745 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.745 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.745 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.745 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_haproxy": { 2024-05-08T22:18:07.745 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_haproxy", 2024-05-08T22:18:07.745 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.745 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.745 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.745 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T22:18:07.745 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.746 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.746 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.746 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "HAproxy container image", 2024-05-08T22:18:07.746 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.746 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.746 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.746 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.746 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_agent": { 2024-05-08T22:18:07.746 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_agent", 2024-05-08T22:18:07.746 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.746 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.746 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.746 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T22:18:07.746 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.746 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.746 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.747 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger agent container image", 2024-05-08T22:18:07.747 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.747 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.747 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.747 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.747 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_collector": { 2024-05-08T22:18:07.747 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_collector", 2024-05-08T22:18:07.747 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.747 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.747 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.747 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T22:18:07.747 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.747 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.747 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.747 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger collector container image", 2024-05-08T22:18:07.747 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.747 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.748 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.748 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.748 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_query": { 2024-05-08T22:18:07.748 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_query", 2024-05-08T22:18:07.748 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.748 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.748 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.748 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T22:18:07.748 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.748 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.748 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.748 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger query container image", 2024-05-08T22:18:07.748 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.748 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.748 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.748 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.749 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_keepalived": { 2024-05-08T22:18:07.749 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_keepalived", 2024-05-08T22:18:07.749 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.749 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.749 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.749 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T22:18:07.749 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.749 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.749 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.749 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Keepalived container image", 2024-05-08T22:18:07.749 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.749 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.749 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.749 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.749 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_loki": { 2024-05-08T22:18:07.750 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_loki", 2024-05-08T22:18:07.750 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.750 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.750 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.750 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T22:18:07.750 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.750 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.750 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.750 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Loki container image", 2024-05-08T22:18:07.750 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.750 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.750 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.750 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.750 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_node_exporter": { 2024-05-08T22:18:07.751 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_node_exporter", 2024-05-08T22:18:07.751 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.751 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.751 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.751 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T22:18:07.751 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.751 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.751 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.751 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-05-08T22:18:07.751 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.751 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.751 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.751 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.751 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_nvmeof": { 2024-05-08T22:18:07.751 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_nvmeof", 2024-05-08T22:18:07.751 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.752 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.752 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.752 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-08T22:18:07.752 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.752 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.752 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.752 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Nvme-of container image", 2024-05-08T22:18:07.752 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.752 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.752 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.752 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.752 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_prometheus": { 2024-05-08T22:18:07.752 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_prometheus", 2024-05-08T22:18:07.752 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.752 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.752 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.753 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T22:18:07.753 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.753 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.753 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.753 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-05-08T22:18:07.753 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.753 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.753 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.753 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.753 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_promtail": { 2024-05-08T22:18:07.753 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_promtail", 2024-05-08T22:18:07.753 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.753 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.753 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.753 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T22:18:07.754 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.754 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.754 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.754 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Promtail container image", 2024-05-08T22:18:07.754 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.754 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.754 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.754 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.754 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_snmp_gateway": { 2024-05-08T22:18:07.754 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_snmp_gateway", 2024-05-08T22:18:07.754 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.754 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.754 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.755 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T22:18:07.755 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.755 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.755 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.755 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SNMP Gateway container image", 2024-05-08T22:18:07.755 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.755 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.755 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.755 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.755 INFO:teuthology.orchestra.run.smithi131.stdout: "container_init": { 2024-05-08T22:18:07.755 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_init", 2024-05-08T22:18:07.755 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.755 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.755 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.755 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.755 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.756 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.756 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.756 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T22:18:07.756 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.756 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.756 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.756 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.756 INFO:teuthology.orchestra.run.smithi131.stdout: "daemon_cache_timeout": { 2024-05-08T22:18:07.756 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "daemon_cache_timeout", 2024-05-08T22:18:07.756 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.756 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.756 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.756 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-08T22:18:07.756 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.757 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.757 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.757 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T22:18:07.757 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.757 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.757 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.757 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.757 INFO:teuthology.orchestra.run.smithi131.stdout: "default_cephadm_command_timeout": { 2024-05-08T22:18:07.757 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T22:18:07.757 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.757 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.757 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.757 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "900", 2024-05-08T22:18:07.757 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.757 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.758 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.758 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T22:18:07.758 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.758 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.758 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.758 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.758 INFO:teuthology.orchestra.run.smithi131.stdout: "default_registry": { 2024-05-08T22:18:07.758 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "default_registry", 2024-05-08T22:18:07.758 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.758 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.758 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.758 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io", 2024-05-08T22:18:07.758 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.759 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.759 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.759 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T22:18:07.759 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.759 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.759 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.759 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.759 INFO:teuthology.orchestra.run.smithi131.stdout: "device_cache_timeout": { 2024-05-08T22:18:07.759 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_cache_timeout", 2024-05-08T22:18:07.759 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.759 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.759 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.759 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1800", 2024-05-08T22:18:07.759 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.759 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.759 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.759 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache device inventory", 2024-05-08T22:18:07.760 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.760 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.760 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.760 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.760 INFO:teuthology.orchestra.run.smithi131.stdout: "device_enhanced_scan": { 2024-05-08T22:18:07.760 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_enhanced_scan", 2024-05-08T22:18:07.760 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.760 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.760 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.760 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.760 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.760 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.760 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.760 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T22:18:07.761 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.761 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.761 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.761 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.761 INFO:teuthology.orchestra.run.smithi131.stdout: "facts_cache_timeout": { 2024-05-08T22:18:07.761 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "facts_cache_timeout", 2024-05-08T22:18:07.761 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.761 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.761 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.761 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-08T22:18:07.761 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.761 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.761 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.761 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache host facts data", 2024-05-08T22:18:07.761 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.762 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.762 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.762 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.762 INFO:teuthology.orchestra.run.smithi131.stdout: "host_check_interval": { 2024-05-08T22:18:07.762 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "host_check_interval", 2024-05-08T22:18:07.762 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.762 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.762 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.762 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-08T22:18:07.762 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.762 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.762 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.762 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to perform a host check", 2024-05-08T22:18:07.763 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.763 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.763 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.763 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.763 INFO:teuthology.orchestra.run.smithi131.stdout: "hw_monitoring": { 2024-05-08T22:18:07.763 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "hw_monitoring", 2024-05-08T22:18:07.763 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.763 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.763 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.763 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.763 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.763 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.763 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.763 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T22:18:07.763 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.764 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.764 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.764 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.764 INFO:teuthology.orchestra.run.smithi131.stdout: "inventory_list_all": { 2024-05-08T22:18:07.764 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "inventory_list_all", 2024-05-08T22:18:07.764 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.764 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.764 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.764 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.764 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.764 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.764 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.764 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T22:18:07.764 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.764 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.765 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.765 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.765 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.765 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.765 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.765 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.765 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.765 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.765 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.765 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.765 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.765 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.765 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.765 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.765 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.766 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.766 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.766 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.766 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.766 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.766 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.766 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.766 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.766 INFO:teuthology.orchestra.run.smithi131.stdout: "log_refresh_metadata": { 2024-05-08T22:18:07.766 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_refresh_metadata", 2024-05-08T22:18:07.766 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.766 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.766 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.766 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.767 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.767 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.767 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.767 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T22:18:07.767 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.767 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.767 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.767 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.767 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.767 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.767 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.767 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.767 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.767 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.768 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.768 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.768 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.768 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T22:18:07.768 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.768 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.768 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.768 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.768 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.768 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.768 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.768 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.768 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.768 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.769 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.769 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.769 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.769 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.769 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.769 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.769 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.769 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.769 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.769 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.769 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.769 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.769 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.769 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.769 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.770 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.770 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.770 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.770 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.770 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.770 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.770 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.770 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.770 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.770 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.770 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.770 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.770 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.770 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.771 INFO:teuthology.orchestra.run.smithi131.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T22:18:07.771 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T22:18:07.771 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.771 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.771 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.771 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.771 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.771 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.771 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.771 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T22:18:07.771 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.771 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.771 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.771 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.771 INFO:teuthology.orchestra.run.smithi131.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T22:18:07.772 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T22:18:07.772 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.772 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.772 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.772 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "*", 2024-05-08T22:18:07.772 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.772 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.772 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.772 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T22:18:07.772 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.772 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.772 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.772 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.772 INFO:teuthology.orchestra.run.smithi131.stdout: "max_count_per_host": { 2024-05-08T22:18:07.772 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_count_per_host", 2024-05-08T22:18:07.773 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.773 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.773 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.773 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-08T22:18:07.773 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.773 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.773 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.773 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "max number of daemons per service per host", 2024-05-08T22:18:07.773 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.773 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.773 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.773 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.773 INFO:teuthology.orchestra.run.smithi131.stdout: "max_osd_draining_count": { 2024-05-08T22:18:07.773 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_osd_draining_count", 2024-05-08T22:18:07.774 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.774 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.774 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.774 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-08T22:18:07.774 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.774 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.774 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.774 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T22:18:07.774 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.774 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.774 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.774 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.774 INFO:teuthology.orchestra.run.smithi131.stdout: "migration_current": { 2024-05-08T22:18:07.775 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "migration_current", 2024-05-08T22:18:07.775 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.775 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.775 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.775 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.775 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.775 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.775 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.775 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "internal - do not modify", 2024-05-08T22:18:07.775 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.775 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.775 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.775 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.775 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": { 2024-05-08T22:18:07.776 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mode", 2024-05-08T22:18:07.776 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.776 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.776 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.776 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "root", 2024-05-08T22:18:07.776 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.776 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.776 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.776 INFO:teuthology.orchestra.run.smithi131.stdout: "cephadm-package", 2024-05-08T22:18:07.776 INFO:teuthology.orchestra.run.smithi131.stdout: "root" 2024-05-08T22:18:07.776 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.776 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T22:18:07.776 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.776 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.777 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.777 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.777 INFO:teuthology.orchestra.run.smithi131.stdout: "oob_default_addr": { 2024-05-08T22:18:07.777 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "oob_default_addr", 2024-05-08T22:18:07.777 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.777 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.777 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.777 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "169.254.1.1", 2024-05-08T22:18:07.777 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.777 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.777 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.777 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T22:18:07.777 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.777 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.778 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.778 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.778 INFO:teuthology.orchestra.run.smithi131.stdout: "prometheus_alerts_path": { 2024-05-08T22:18:07.778 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prometheus_alerts_path", 2024-05-08T22:18:07.778 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.778 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.778 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.778 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T22:18:07.778 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.778 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.778 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.778 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T22:18:07.778 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.778 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.778 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.779 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.779 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_insecure": { 2024-05-08T22:18:07.779 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_insecure", 2024-05-08T22:18:07.779 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.779 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.779 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.779 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.779 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.779 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.779 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.779 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T22:18:07.779 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.779 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.779 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.779 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.780 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_password": { 2024-05-08T22:18:07.780 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_password", 2024-05-08T22:18:07.780 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.780 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.780 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.780 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.780 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.780 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.780 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.780 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T22:18:07.780 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.780 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.780 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.780 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.780 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_url": { 2024-05-08T22:18:07.780 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_url", 2024-05-08T22:18:07.781 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.781 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.781 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.781 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.781 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.781 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.781 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.781 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T22:18:07.781 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.781 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.781 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.781 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.781 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_username": { 2024-05-08T22:18:07.781 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_username", 2024-05-08T22:18:07.781 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.782 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.782 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.782 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.782 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.782 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.782 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.782 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T22:18:07.782 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.782 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.782 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.782 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.782 INFO:teuthology.orchestra.run.smithi131.stdout: "secure_monitoring_stack": { 2024-05-08T22:18:07.782 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "secure_monitoring_stack", 2024-05-08T22:18:07.782 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.782 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.783 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.783 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.783 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.783 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.783 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.783 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T22:18:07.783 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.783 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.783 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.783 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.783 INFO:teuthology.orchestra.run.smithi131.stdout: "service_discovery_port": { 2024-05-08T22:18:07.783 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "service_discovery_port", 2024-05-08T22:18:07.783 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.783 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.783 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.784 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8765", 2024-05-08T22:18:07.784 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.784 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.784 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.784 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "cephadm service discovery port", 2024-05-08T22:18:07.784 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.784 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.784 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.784 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.784 INFO:teuthology.orchestra.run.smithi131.stdout: "ssh_config_file": { 2024-05-08T22:18:07.784 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssh_config_file", 2024-05-08T22:18:07.784 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.784 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.784 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.784 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.784 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.784 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.785 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.785 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T22:18:07.785 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.785 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.785 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.785 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.785 INFO:teuthology.orchestra.run.smithi131.stdout: "use_agent": { 2024-05-08T22:18:07.785 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "use_agent", 2024-05-08T22:18:07.785 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.785 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.785 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.785 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.785 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.785 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.785 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.785 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T22:18:07.786 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.786 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.786 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.786 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.786 INFO:teuthology.orchestra.run.smithi131.stdout: "use_repo_digest": { 2024-05-08T22:18:07.786 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "use_repo_digest", 2024-05-08T22:18:07.786 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.786 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.786 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.786 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.786 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.786 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.786 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.786 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T22:18:07.787 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.787 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.787 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.787 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.787 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_failed_host_check": { 2024-05-08T22:18:07.787 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_failed_host_check", 2024-05-08T22:18:07.787 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.787 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.787 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.787 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.787 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.787 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.787 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.787 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T22:18:07.787 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.787 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.788 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.788 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.788 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_stray_daemons": { 2024-05-08T22:18:07.788 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_stray_daemons", 2024-05-08T22:18:07.788 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.788 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.788 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.788 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.788 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.788 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.788 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.788 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T22:18:07.788 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.788 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.789 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.789 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.789 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_stray_hosts": { 2024-05-08T22:18:07.789 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_stray_hosts", 2024-05-08T22:18:07.789 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.789 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.789 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.789 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.789 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.789 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.789 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.789 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T22:18:07.789 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.789 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.789 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.789 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.790 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.790 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.790 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.790 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crash", 2024-05-08T22:18:07.790 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.790 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.790 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.790 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.790 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.790 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.790 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.790 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.790 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.790 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.790 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.791 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.791 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.791 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.791 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.791 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.791 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.791 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.791 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.791 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.791 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.791 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.791 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.791 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.791 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.791 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.791 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.791 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.792 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.792 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.792 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.792 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.792 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.792 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.792 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.792 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.792 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.792 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.792 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.792 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.792 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.792 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.792 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.792 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.793 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.793 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.793 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.793 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.793 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.793 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.793 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.793 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.793 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.793 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.793 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.793 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.793 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.793 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.793 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.793 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.794 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.794 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.794 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.794 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.794 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.794 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.794 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.794 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.794 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.794 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.794 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.794 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.794 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.794 INFO:teuthology.orchestra.run.smithi131.stdout: "retain_interval": { 2024-05-08T22:18:07.795 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "retain_interval", 2024-05-08T22:18:07.795 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.795 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.795 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.795 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "31536000", 2024-05-08T22:18:07.795 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.795 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.795 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.795 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T22:18:07.795 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.795 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.795 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.795 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.795 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_recent_interval": { 2024-05-08T22:18:07.795 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_recent_interval", 2024-05-08T22:18:07.795 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.795 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.796 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.796 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1209600", 2024-05-08T22:18:07.796 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.796 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.796 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.796 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T22:18:07.796 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.796 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.796 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.796 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.796 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.796 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.796 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.796 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "dashboard", 2024-05-08T22:18:07.796 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.797 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.797 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.797 INFO:teuthology.orchestra.run.smithi131.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T22:18:07.797 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T22:18:07.797 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.797 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.797 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.797 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-08T22:18:07.797 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.797 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.797 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.797 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.797 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.797 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.797 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.798 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.798 INFO:teuthology.orchestra.run.smithi131.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T22:18:07.798 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T22:18:07.798 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.798 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.798 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.798 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.798 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.798 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.798 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.798 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.798 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.798 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.799 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.799 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.799 INFO:teuthology.orchestra.run.smithi131.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T22:18:07.799 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T22:18:07.799 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.799 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.799 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.799 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.799 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.799 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.799 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.799 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.799 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.799 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.800 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.800 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.800 INFO:teuthology.orchestra.run.smithi131.stdout: "AUDIT_API_ENABLED": { 2024-05-08T22:18:07.800 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T22:18:07.800 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.800 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.800 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.800 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.800 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.800 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.800 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.800 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.800 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.800 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.801 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.801 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.801 INFO:teuthology.orchestra.run.smithi131.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T22:18:07.801 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T22:18:07.801 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.801 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.801 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.801 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.801 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.801 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.801 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.801 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.801 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.801 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.801 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.802 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.802 INFO:teuthology.orchestra.run.smithi131.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T22:18:07.802 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T22:18:07.802 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.802 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.802 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.802 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.802 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.802 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.802 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.802 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.802 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.802 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.802 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.802 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.803 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T22:18:07.803 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T22:18:07.803 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.803 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.803 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.803 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.803 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.803 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.803 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.803 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.803 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.803 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.803 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.804 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.804 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T22:18:07.804 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T22:18:07.804 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.804 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.804 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.804 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.804 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.804 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.804 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.804 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.804 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.804 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.804 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.804 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.805 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T22:18:07.805 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T22:18:07.805 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.805 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.805 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.805 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.805 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.805 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.805 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.805 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.805 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.805 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.805 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.805 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.806 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T22:18:07.806 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T22:18:07.806 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.806 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.806 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.806 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.806 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.806 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.806 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.806 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.806 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.806 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.806 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.806 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.807 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T22:18:07.807 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T22:18:07.807 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.807 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.807 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.807 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.807 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.807 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.807 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.807 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.807 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.807 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.807 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.807 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.808 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T22:18:07.808 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T22:18:07.808 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.808 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.808 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.808 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.808 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.808 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.808 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.808 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.808 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.808 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.808 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.808 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.808 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T22:18:07.809 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T22:18:07.809 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.809 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.809 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.809 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.809 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.809 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.809 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.809 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.809 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.809 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.809 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.809 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.809 INFO:teuthology.orchestra.run.smithi131.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T22:18:07.809 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T22:18:07.809 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.810 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.810 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.810 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.810 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.810 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.810 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.810 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.810 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.810 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.810 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.810 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.810 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T22:18:07.810 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T22:18:07.811 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.811 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.811 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.811 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-05-08T22:18:07.811 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.811 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.811 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.811 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.811 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.811 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.811 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.811 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.811 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T22:18:07.811 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T22:18:07.812 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.812 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.812 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.812 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.812 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.812 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.812 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.812 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.812 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.812 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.812 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.812 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.812 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_URL": { 2024-05-08T22:18:07.812 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_URL", 2024-05-08T22:18:07.812 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.812 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.813 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.813 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.813 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.813 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.813 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.813 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.813 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.813 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.813 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.813 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.813 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T22:18:07.813 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T22:18:07.813 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.813 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.813 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.813 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-05-08T22:18:07.814 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.814 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.814 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.814 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.814 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.814 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.814 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.814 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.814 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T22:18:07.814 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T22:18:07.814 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.814 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.814 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.814 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.814 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.815 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.815 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.815 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.815 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.815 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.815 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.815 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.815 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T22:18:07.815 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T22:18:07.815 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.815 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.815 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.815 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.815 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.815 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.816 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.816 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.816 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.816 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.816 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.816 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.816 INFO:teuthology.orchestra.run.smithi131.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T22:18:07.816 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T22:18:07.816 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.816 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.816 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.816 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.816 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.816 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.817 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.817 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.817 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.817 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.817 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.817 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.817 INFO:teuthology.orchestra.run.smithi131.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T22:18:07.817 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T22:18:07.817 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.817 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.817 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.817 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.817 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.817 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.817 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.817 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.818 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.818 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.818 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.818 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.818 INFO:teuthology.orchestra.run.smithi131.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T22:18:07.818 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T22:18:07.818 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.818 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.818 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.818 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.818 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.818 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.819 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.819 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.819 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.819 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.819 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.819 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.819 INFO:teuthology.orchestra.run.smithi131.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T22:18:07.819 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T22:18:07.819 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.819 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.819 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.819 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.819 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.819 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.819 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.820 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.820 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.820 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.820 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.820 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.820 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T22:18:07.820 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T22:18:07.820 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.820 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.820 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.820 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.820 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.820 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.820 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.821 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.821 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.821 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.821 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.821 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.821 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T22:18:07.821 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T22:18:07.821 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.821 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.821 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.821 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.821 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.821 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.821 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.821 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.821 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.822 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.822 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.822 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.822 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T22:18:07.822 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T22:18:07.822 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.822 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.822 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.822 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.822 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.822 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.822 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.822 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.822 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.823 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.823 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.823 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.823 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T22:18:07.823 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T22:18:07.823 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.823 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.823 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.823 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.823 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.823 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.823 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.823 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.823 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.824 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.824 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.824 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.824 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T22:18:07.824 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T22:18:07.824 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.824 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.824 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.824 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.824 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.824 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.824 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.824 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.824 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.824 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.825 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.825 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.825 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T22:18:07.825 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T22:18:07.825 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.825 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.825 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.825 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.825 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.825 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.825 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.825 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.825 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.825 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.825 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.826 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.826 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T22:18:07.826 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T22:18:07.826 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.826 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.826 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.826 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.826 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.826 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.826 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.826 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.826 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.826 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.826 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.826 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.827 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T22:18:07.827 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T22:18:07.827 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.827 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.827 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.827 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.827 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.827 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.827 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.827 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.827 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.827 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.827 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.827 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.827 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T22:18:07.828 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T22:18:07.828 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.828 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.828 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.828 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T22:18:07.828 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.828 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.828 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.828 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.828 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.828 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.828 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.828 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.828 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T22:18:07.828 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T22:18:07.829 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.829 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.829 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.829 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-08T22:18:07.829 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.829 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.829 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.829 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.829 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.829 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.829 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.829 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.829 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T22:18:07.829 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T22:18:07.829 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.830 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.830 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.830 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8", 2024-05-08T22:18:07.830 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.830 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.830 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.830 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.830 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.830 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.830 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.830 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.830 INFO:teuthology.orchestra.run.smithi131.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T22:18:07.830 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T22:18:07.830 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.831 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.831 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.831 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "45", 2024-05-08T22:18:07.831 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.831 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.831 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.831 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.831 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.831 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.831 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.831 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.831 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T22:18:07.831 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T22:18:07.831 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.831 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.832 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.832 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.832 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.832 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.832 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.832 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.832 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.832 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.832 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.832 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.832 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T22:18:07.832 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T22:18:07.832 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.832 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.832 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.833 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-05-08T22:18:07.833 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.833 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.833 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.833 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.833 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.833 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.833 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.833 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.833 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T22:18:07.833 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T22:18:07.833 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.833 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.833 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.833 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.834 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.834 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.834 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.834 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.834 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.834 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.834 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.834 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.834 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T22:18:07.834 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T22:18:07.834 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.834 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.834 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.835 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.835 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.835 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.835 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.835 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.835 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.835 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.835 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.835 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.835 INFO:teuthology.orchestra.run.smithi131.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T22:18:07.835 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T22:18:07.835 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.835 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.835 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.835 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.836 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.836 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.836 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.836 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.836 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.836 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.836 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.836 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.836 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T22:18:07.836 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T22:18:07.836 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.836 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.836 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.836 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-08T22:18:07.836 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.837 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.837 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.837 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.837 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.837 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.837 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.837 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.837 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T22:18:07.837 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T22:18:07.837 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.837 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.837 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.837 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-08T22:18:07.837 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.838 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.838 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.838 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.838 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.838 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.838 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.838 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.838 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T22:18:07.838 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T22:18:07.838 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.838 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.838 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.838 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-05-08T22:18:07.838 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.839 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.839 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.839 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.839 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.839 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.839 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.839 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.839 INFO:teuthology.orchestra.run.smithi131.stdout: "cross_origin_url": { 2024-05-08T22:18:07.839 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cross_origin_url", 2024-05-08T22:18:07.839 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.839 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.839 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.839 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.839 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.839 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.840 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.840 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.840 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.840 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.840 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.840 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.840 INFO:teuthology.orchestra.run.smithi131.stdout: "crt_file": { 2024-05-08T22:18:07.840 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crt_file", 2024-05-08T22:18:07.840 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.840 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.840 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.840 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.840 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.840 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.840 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.841 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.841 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.841 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.841 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.841 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.841 INFO:teuthology.orchestra.run.smithi131.stdout: "debug": { 2024-05-08T22:18:07.841 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "debug", 2024-05-08T22:18:07.841 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.841 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.841 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.841 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.841 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.841 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.841 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.841 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable/disable debug options", 2024-05-08T22:18:07.842 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.842 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.842 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.842 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.842 INFO:teuthology.orchestra.run.smithi131.stdout: "jwt_token_ttl": { 2024-05-08T22:18:07.842 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "jwt_token_ttl", 2024-05-08T22:18:07.842 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.842 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.842 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.842 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "28800", 2024-05-08T22:18:07.842 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.842 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.842 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.842 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.843 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.843 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.843 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.843 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.843 INFO:teuthology.orchestra.run.smithi131.stdout: "key_file": { 2024-05-08T22:18:07.843 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "key_file", 2024-05-08T22:18:07.843 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.843 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.843 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.843 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.843 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.843 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.843 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.843 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.843 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.844 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.844 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.844 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.844 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.844 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.844 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.844 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.844 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.844 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.844 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.844 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.844 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.844 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.844 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.845 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.845 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.845 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.845 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.845 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.845 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.845 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.845 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.845 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.845 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.845 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.845 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.845 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.845 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.845 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.845 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.846 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.846 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.846 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.846 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.846 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.846 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.846 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.846 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.846 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.846 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.846 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.846 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.846 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.846 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.847 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.847 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.847 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.847 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.847 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.847 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.847 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.847 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.847 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.847 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.847 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.847 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.847 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.847 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.847 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: "motd": { 2024-05-08T22:18:07.849 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "motd", 2024-05-08T22:18:07.849 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.849 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.849 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.849 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.849 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.849 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.849 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.849 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "The message of the day", 2024-05-08T22:18:07.849 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.849 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.849 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.849 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.849 INFO:teuthology.orchestra.run.smithi131.stdout: "redirect_resolve_ip_addr": { 2024-05-08T22:18:07.849 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T22:18:07.850 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.850 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.850 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.850 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.850 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.850 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.850 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.850 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.850 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.850 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.850 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.850 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.850 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-05-08T22:18:07.851 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-05-08T22:18:07.851 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.851 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.851 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.851 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "::", 2024-05-08T22:18:07.851 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.851 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.851 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.851 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.851 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.851 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.851 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.851 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.851 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-05-08T22:18:07.851 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-05-08T22:18:07.851 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.852 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.852 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.852 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8080", 2024-05-08T22:18:07.852 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.852 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.852 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.852 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.852 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.852 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.852 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.852 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.852 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl": { 2024-05-08T22:18:07.852 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl", 2024-05-08T22:18:07.852 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.852 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.853 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.853 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.853 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.853 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.853 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.853 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.853 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.853 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.853 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.853 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.853 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl_server_port": { 2024-05-08T22:18:07.853 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl_server_port", 2024-05-08T22:18:07.853 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.853 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.853 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.853 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8443", 2024-05-08T22:18:07.854 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.854 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.854 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.854 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.854 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.854 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.854 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.854 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.854 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_behaviour": { 2024-05-08T22:18:07.854 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_behaviour", 2024-05-08T22:18:07.854 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.854 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.854 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.854 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "redirect", 2024-05-08T22:18:07.854 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.855 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.855 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.855 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.855 INFO:teuthology.orchestra.run.smithi131.stdout: "redirect" 2024-05-08T22:18:07.855 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.855 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.855 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.855 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.855 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.855 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.855 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_error_status_code": { 2024-05-08T22:18:07.855 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_error_status_code", 2024-05-08T22:18:07.855 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.855 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.856 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.856 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "500", 2024-05-08T22:18:07.856 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "400", 2024-05-08T22:18:07.856 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "599", 2024-05-08T22:18:07.856 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.856 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.856 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.856 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.856 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.856 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.856 INFO:teuthology.orchestra.run.smithi131.stdout: "url_prefix": { 2024-05-08T22:18:07.856 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "url_prefix", 2024-05-08T22:18:07.856 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.856 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.856 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.857 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.857 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.857 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.857 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.857 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.857 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.857 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.857 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.857 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.857 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.857 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.857 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.857 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "devicehealth", 2024-05-08T22:18:07.857 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.857 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.857 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.858 INFO:teuthology.orchestra.run.smithi131.stdout: "enable_monitoring": { 2024-05-08T22:18:07.858 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enable_monitoring", 2024-05-08T22:18:07.858 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.858 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.858 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.858 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.858 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.858 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.858 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.858 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "monitor device health metrics", 2024-05-08T22:18:07.858 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.858 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.858 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.858 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.859 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.859 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.859 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.859 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.859 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.859 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.859 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.859 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.859 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.859 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.859 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.859 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.859 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.859 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.859 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.860 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.860 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.860 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.860 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.860 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.860 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.860 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.860 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.860 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.860 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.860 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.860 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.860 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.860 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.860 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.860 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.861 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.861 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.861 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.861 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.861 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.861 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.861 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.861 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.861 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.861 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.861 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.861 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.861 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.861 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.862 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.862 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.862 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.862 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.862 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.862 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.862 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.862 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.862 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.862 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.862 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.862 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.862 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.862 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.863 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.863 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.863 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.863 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.863 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.863 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.863 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.863 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.863 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.863 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.863 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.863 INFO:teuthology.orchestra.run.smithi131.stdout: "mark_out_threshold": { 2024-05-08T22:18:07.863 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mark_out_threshold", 2024-05-08T22:18:07.863 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.863 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.864 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.864 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "2419200", 2024-05-08T22:18:07.864 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.864 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.864 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.864 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T22:18:07.864 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.864 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.864 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.864 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.864 INFO:teuthology.orchestra.run.smithi131.stdout: "pool_name": { 2024-05-08T22:18:07.864 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pool_name", 2024-05-08T22:18:07.864 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.864 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.865 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.865 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "device_health_metrics", 2024-05-08T22:18:07.865 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.865 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.865 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.865 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T22:18:07.865 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.865 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.865 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.865 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.865 INFO:teuthology.orchestra.run.smithi131.stdout: "retention_period": { 2024-05-08T22:18:07.865 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "retention_period", 2024-05-08T22:18:07.865 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.866 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.866 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.866 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15552000", 2024-05-08T22:18:07.866 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.866 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.866 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.866 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long to retain device health metrics", 2024-05-08T22:18:07.866 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.866 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.866 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.866 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.866 INFO:teuthology.orchestra.run.smithi131.stdout: "scrape_frequency": { 2024-05-08T22:18:07.866 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "scrape_frequency", 2024-05-08T22:18:07.866 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.866 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.867 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.867 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "86400", 2024-05-08T22:18:07.867 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.867 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.867 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.867 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T22:18:07.867 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.867 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.867 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.867 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.867 INFO:teuthology.orchestra.run.smithi131.stdout: "self_heal": { 2024-05-08T22:18:07.867 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "self_heal", 2024-05-08T22:18:07.867 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.867 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.867 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.868 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.868 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.868 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.868 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.868 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T22:18:07.868 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.868 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.868 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.868 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.868 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-08T22:18:07.868 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-08T22:18:07.868 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.868 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.868 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.868 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-08T22:18:07.868 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.869 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.869 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.869 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T22:18:07.869 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.869 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.869 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.869 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.869 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_threshold": { 2024-05-08T22:18:07.869 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_threshold", 2024-05-08T22:18:07.869 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.869 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.869 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.869 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "7257600", 2024-05-08T22:18:07.869 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.869 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.869 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.870 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T22:18:07.870 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.870 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.870 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.870 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.870 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.870 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.870 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.870 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "diskprediction_local", 2024-05-08T22:18:07.870 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.870 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.870 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.870 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.870 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.870 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.871 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.871 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.871 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.871 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.871 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.871 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.871 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.871 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.871 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.871 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.871 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.871 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.871 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.871 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.871 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.871 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.871 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.872 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.872 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.872 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.872 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.872 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.872 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.872 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.872 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.872 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.872 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.872 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.872 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.872 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.872 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.872 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.872 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.873 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.873 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.873 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.873 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.873 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.873 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.873 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.873 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.873 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.873 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.873 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.873 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.873 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.873 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.873 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.873 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.874 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.874 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.874 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.874 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.874 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.874 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.874 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.874 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.874 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.874 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.874 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.874 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.874 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.874 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.875 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.875 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.875 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.875 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.875 INFO:teuthology.orchestra.run.smithi131.stdout: "predict_interval": { 2024-05-08T22:18:07.875 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "predict_interval", 2024-05-08T22:18:07.875 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.875 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.875 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.875 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "86400", 2024-05-08T22:18:07.875 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.875 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.875 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.875 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.875 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.876 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.876 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.876 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.876 INFO:teuthology.orchestra.run.smithi131.stdout: "predictor_model": { 2024-05-08T22:18:07.876 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "predictor_model", 2024-05-08T22:18:07.876 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.876 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.876 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.876 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "prophetstor", 2024-05-08T22:18:07.876 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.876 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.876 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.876 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.876 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.876 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.877 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.877 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.877 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-08T22:18:07.877 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-08T22:18:07.877 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.877 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.877 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.877 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-08T22:18:07.877 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.877 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.877 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.877 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.877 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.877 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.877 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.878 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.878 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.878 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.878 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.878 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "influx", 2024-05-08T22:18:07.878 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": false, 2024-05-08T22:18:07.878 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "influxdb python module not found", 2024-05-08T22:18:07.878 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.878 INFO:teuthology.orchestra.run.smithi131.stdout: "batch_size": { 2024-05-08T22:18:07.878 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "batch_size", 2024-05-08T22:18:07.878 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.878 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.878 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.878 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5000", 2024-05-08T22:18:07.878 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.879 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.879 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.879 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T22:18:07.879 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.879 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.879 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.879 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.879 INFO:teuthology.orchestra.run.smithi131.stdout: "database": { 2024-05-08T22:18:07.879 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "database", 2024-05-08T22:18:07.879 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.879 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.879 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.879 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "ceph", 2024-05-08T22:18:07.879 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.879 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.879 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.880 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T22:18:07.880 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.880 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.880 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.880 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.880 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": { 2024-05-08T22:18:07.880 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "hostname", 2024-05-08T22:18:07.880 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.880 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.880 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.880 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.880 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.880 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.880 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.880 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB server hostname", 2024-05-08T22:18:07.880 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.881 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.881 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.881 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.881 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-08T22:18:07.881 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-08T22:18:07.881 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.881 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.881 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.881 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "30", 2024-05-08T22:18:07.881 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "5", 2024-05-08T22:18:07.881 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.881 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.881 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T22:18:07.882 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.882 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.882 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.882 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.882 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.882 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.882 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.882 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.882 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.882 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.882 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.882 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.882 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.882 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.883 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.883 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.883 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.883 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.883 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.883 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.883 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.883 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.883 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.883 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.883 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.883 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.883 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.883 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.883 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.884 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.884 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.884 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.884 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.884 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.884 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.884 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.884 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.884 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.884 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.884 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.884 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.884 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.884 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.884 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.884 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.885 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.885 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.885 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.885 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.885 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.885 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.885 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.885 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.885 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.885 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.885 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.885 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.885 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.885 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.885 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.886 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.886 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.886 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.886 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.886 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.886 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.886 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.886 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.886 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.886 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.886 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.886 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.886 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.887 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.887 INFO:teuthology.orchestra.run.smithi131.stdout: "password": { 2024-05-08T22:18:07.887 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "password", 2024-05-08T22:18:07.887 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.887 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.887 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.887 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.887 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.887 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.887 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.887 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "password of InfluxDB server user", 2024-05-08T22:18:07.887 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.887 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.887 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.887 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.888 INFO:teuthology.orchestra.run.smithi131.stdout: "port": { 2024-05-08T22:18:07.888 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "port", 2024-05-08T22:18:07.888 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.888 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.888 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.888 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8086", 2024-05-08T22:18:07.888 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.888 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.888 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.888 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB server port", 2024-05-08T22:18:07.888 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.888 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.888 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.888 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.888 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl": { 2024-05-08T22:18:07.888 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl", 2024-05-08T22:18:07.888 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.889 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.889 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.889 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "false", 2024-05-08T22:18:07.889 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.889 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.889 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.889 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T22:18:07.889 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.889 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.889 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.889 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.889 INFO:teuthology.orchestra.run.smithi131.stdout: "threads": { 2024-05-08T22:18:07.889 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "threads", 2024-05-08T22:18:07.889 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.890 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.890 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.890 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-05-08T22:18:07.890 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-05-08T22:18:07.890 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "32", 2024-05-08T22:18:07.890 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.890 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T22:18:07.890 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.890 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.890 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.890 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.890 INFO:teuthology.orchestra.run.smithi131.stdout: "username": { 2024-05-08T22:18:07.890 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "username", 2024-05-08T22:18:07.890 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.890 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.890 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.891 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.891 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.891 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.891 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.891 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "username of InfluxDB server user", 2024-05-08T22:18:07.891 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.891 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.891 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.891 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.891 INFO:teuthology.orchestra.run.smithi131.stdout: "verify_ssl": { 2024-05-08T22:18:07.891 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "verify_ssl", 2024-05-08T22:18:07.891 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.891 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.891 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.891 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "true", 2024-05-08T22:18:07.891 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.892 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.892 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.892 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T22:18:07.892 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.892 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.892 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.892 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.892 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.892 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.892 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.892 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "insights", 2024-05-08T22:18:07.892 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.892 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.892 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.892 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.893 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.893 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.893 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.893 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.893 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.893 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.893 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.893 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.893 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.893 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.893 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.893 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.893 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.893 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.893 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.893 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.893 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.894 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.894 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.894 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.894 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.894 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.894 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.894 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.894 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.894 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.894 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.894 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.894 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.894 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.894 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.894 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.895 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.895 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.895 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.895 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.895 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.895 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.895 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.895 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.895 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.895 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.895 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.895 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.895 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.895 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.895 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.895 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.895 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.896 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.896 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.896 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.896 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.896 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.896 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.896 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.896 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.896 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.896 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.896 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.896 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.896 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.896 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.896 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.896 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.897 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.897 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.897 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.897 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.897 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.897 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.897 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.897 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "iostat", 2024-05-08T22:18:07.897 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.897 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.897 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.897 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.897 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.897 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.897 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.897 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.897 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.898 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.898 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.898 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.898 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.898 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.898 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.898 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.898 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.898 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.898 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.898 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.898 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.898 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.898 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.899 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.899 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.899 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.899 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.899 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.899 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.899 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.899 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.899 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.899 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.899 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.899 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.899 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.899 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.899 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.900 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.900 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.900 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.900 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.900 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.900 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.900 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.900 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.900 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.900 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.900 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.900 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.900 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.900 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.901 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.901 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.901 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.901 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.901 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.901 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.901 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.901 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.901 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.901 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.901 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.901 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.901 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.901 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.901 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.901 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.902 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.902 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.902 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.902 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.902 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.902 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.902 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.902 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.902 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "k8sevents", 2024-05-08T22:18:07.902 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.902 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.902 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.902 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_event_retention_days": { 2024-05-08T22:18:07.902 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ceph_event_retention_days", 2024-05-08T22:18:07.902 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.903 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.903 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.903 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "7", 2024-05-08T22:18:07.903 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.903 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.903 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.903 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T22:18:07.903 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.903 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.903 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.903 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.903 INFO:teuthology.orchestra.run.smithi131.stdout: "config_check_secs": { 2024-05-08T22:18:07.903 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_check_secs", 2024-05-08T22:18:07.903 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.903 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.903 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.904 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-08T22:18:07.904 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "10", 2024-05-08T22:18:07.904 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.904 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.904 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T22:18:07.904 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.904 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.904 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.904 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.904 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.904 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.904 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.904 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.904 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.904 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.904 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.905 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.905 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.905 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.905 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.905 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.905 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.905 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.905 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.905 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.905 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.905 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.905 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.905 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.905 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.905 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.905 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.905 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.906 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.906 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.906 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.906 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.906 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.906 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.906 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.906 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.906 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.906 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.906 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.906 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.906 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.906 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.907 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.907 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.907 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.907 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.907 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.907 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.907 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.907 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.907 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.907 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.907 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.907 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.907 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.907 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.907 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.908 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.908 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.908 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.908 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.908 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.908 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.908 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.908 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.908 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.908 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.908 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.908 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.908 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.908 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.908 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.909 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.909 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.909 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.909 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.909 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.909 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "localpool", 2024-05-08T22:18:07.909 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.909 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.909 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.909 INFO:teuthology.orchestra.run.smithi131.stdout: "failure_domain": { 2024-05-08T22:18:07.909 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "failure_domain", 2024-05-08T22:18:07.909 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.909 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.909 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.909 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "host", 2024-05-08T22:18:07.909 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.910 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.910 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.910 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "failure domain for any created local pool", 2024-05-08T22:18:07.910 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T22:18:07.910 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.910 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.910 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.910 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.910 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.910 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.910 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.910 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.910 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.910 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.910 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.910 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.911 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.911 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.911 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.911 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.911 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.911 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.911 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.911 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.911 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.911 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.911 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.911 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.911 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.911 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.911 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.911 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.912 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.912 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.912 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.912 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.912 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.912 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.912 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.912 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.912 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.912 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.912 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.912 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.912 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.912 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.912 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.912 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.913 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.913 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.913 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.913 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.913 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.913 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.913 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.913 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.913 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.913 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.913 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.913 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.913 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.913 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.913 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.913 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.914 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.914 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.914 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.914 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.914 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.914 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.914 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.914 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.914 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.914 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.914 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.914 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.914 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.914 INFO:teuthology.orchestra.run.smithi131.stdout: "min_size": { 2024-05-08T22:18:07.914 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "min_size", 2024-05-08T22:18:07.915 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.915 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.915 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.915 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.915 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.915 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.915 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.915 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default min_size for any created local pool", 2024-05-08T22:18:07.915 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T22:18:07.915 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.915 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.915 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.915 INFO:teuthology.orchestra.run.smithi131.stdout: "num_rep": { 2024-05-08T22:18:07.915 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "num_rep", 2024-05-08T22:18:07.915 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.915 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.915 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.916 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3", 2024-05-08T22:18:07.916 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.916 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.916 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.916 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default replica count for any created local pool", 2024-05-08T22:18:07.916 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.916 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.916 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.916 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.916 INFO:teuthology.orchestra.run.smithi131.stdout: "pg_num": { 2024-05-08T22:18:07.916 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pg_num", 2024-05-08T22:18:07.916 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.916 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.916 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.916 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "128", 2024-05-08T22:18:07.916 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.917 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.917 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.917 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T22:18:07.917 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.917 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.917 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.917 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.917 INFO:teuthology.orchestra.run.smithi131.stdout: "prefix": { 2024-05-08T22:18:07.917 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prefix", 2024-05-08T22:18:07.917 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.917 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.917 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.917 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.917 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.917 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.917 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.918 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "name prefix for any created local pool", 2024-05-08T22:18:07.918 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.918 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.918 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.918 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.918 INFO:teuthology.orchestra.run.smithi131.stdout: "subtree": { 2024-05-08T22:18:07.918 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "subtree", 2024-05-08T22:18:07.918 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.918 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.918 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.918 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "rack", 2024-05-08T22:18:07.918 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.918 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.918 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.918 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T22:18:07.918 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T22:18:07.919 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.919 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.919 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.919 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.919 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.919 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.919 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mds_autoscaler", 2024-05-08T22:18:07.919 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.919 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.919 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.919 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.919 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.919 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.919 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.919 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.919 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.920 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.920 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.920 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.920 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.920 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.920 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.920 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.920 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.920 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.920 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.920 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.920 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.920 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.920 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.920 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.920 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.920 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.921 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.921 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.921 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.921 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.921 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.921 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.921 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.921 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.921 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.921 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.921 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.921 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.921 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.921 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.921 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.921 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.921 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.922 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.922 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.922 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.922 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.922 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.922 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.922 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.922 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.922 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.922 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.922 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.922 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.923 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.923 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.923 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.923 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.923 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.923 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.923 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.923 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.923 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.923 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.923 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.923 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.923 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.923 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.923 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.924 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.924 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.924 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.924 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.924 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.924 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.924 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mirroring", 2024-05-08T22:18:07.924 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.924 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.924 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.924 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.924 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.924 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.924 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.924 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "nfs", 2024-05-08T22:18:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "orchestrator", 2024-05-08T22:18:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "fail_fs": { 2024-05-08T22:18:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "fail_fs", 2024-05-08T22:18:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T22:18:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.936 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.936 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.936 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.936 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.936 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.936 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.936 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.936 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.936 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.936 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.936 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.936 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.936 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.936 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.936 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.938 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.938 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.938 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.938 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.938 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.938 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.938 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.938 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.938 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.938 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.938 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.938 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.938 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.938 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "orchestrator": { 2024-05-08T22:18:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "orchestrator", 2024-05-08T22:18:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: "cephadm", 2024-05-08T22:18:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: "rook", 2024-05-08T22:18:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: "test_orchestrator" 2024-05-08T22:18:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Orchestrator backend", 2024-05-08T22:18:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "osd_perf_query", 2024-05-08T22:18:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.945 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.945 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.945 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.945 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.945 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.945 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.945 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.945 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.945 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.945 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.945 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.945 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.945 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.945 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.945 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.945 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.945 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.946 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.946 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.946 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.946 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.946 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.946 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.946 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.946 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.946 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.946 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "osd_support", 2024-05-08T22:18:07.946 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.946 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.946 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.946 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.946 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.946 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.949 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.949 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.949 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.949 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.949 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.949 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.949 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.949 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.949 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.949 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.949 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.949 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.949 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.949 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.949 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.949 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.950 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.950 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.950 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.950 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.950 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.950 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.950 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.950 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.950 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.950 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.950 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.950 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.950 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.950 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pg_autoscaler", 2024-05-08T22:18:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.952 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.952 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.952 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.952 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.952 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.952 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.952 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.952 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.952 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.952 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.952 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.952 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.952 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.952 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.952 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.953 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.953 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.953 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.953 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.953 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.953 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.953 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.953 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.953 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.953 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.953 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.953 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.953 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.953 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.956 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.956 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.957 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.957 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.957 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.957 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.957 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.957 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.957 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-08T22:18:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-08T22:18:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-08T22:18:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: "threshold": { 2024-05-08T22:18:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "threshold", 2024-05-08T22:18:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-08T22:18:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3.0", 2024-05-08T22:18:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1.0", 2024-05-08T22:18:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "scaling threshold", 2024-05-08T22:18:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T22:18:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "progress", 2024-05-08T22:18:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_pg_recovery_event": { 2024-05-08T22:18:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_pg_recovery_event", 2024-05-08T22:18:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T22:18:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "enabled": { 2024-05-08T22:18:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enabled", 2024-05-08T22:18:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.964 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.964 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.964 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.964 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.964 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.964 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.964 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.964 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.964 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.964 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.964 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.964 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.964 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "max_completed_events": { 2024-05-08T22:18:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_completed_events", 2024-05-08T22:18:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "50", 2024-05-08T22:18:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "number of past completed events to remember", 2024-05-08T22:18:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-08T22:18:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-08T22:18:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-05-08T22:18:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long the module is going to sleep", 2024-05-08T22:18:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prometheus", 2024-05-08T22:18:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "cache": { 2024-05-08T22:18:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cache", 2024-05-08T22:18:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "exclude_perf_counters": { 2024-05-08T22:18:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "exclude_perf_counters", 2024-05-08T22:18:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T22:18:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T22:18:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_stats_pools": { 2024-05-08T22:18:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_stats_pools", 2024-05-08T22:18:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T22:18:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T22:18:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "300", 2024-05-08T22:18:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.978 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.978 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.978 INFO:teuthology.orchestra.run.smithi131.stdout: "scrape_interval": { 2024-05-08T22:18:07.978 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "scrape_interval", 2024-05-08T22:18:07.978 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-08T22:18:07.978 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.978 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.978 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15.0", 2024-05-08T22:18:07.978 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.978 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.978 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.978 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.978 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.979 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.979 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.979 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.979 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-05-08T22:18:07.979 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-05-08T22:18:07.979 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.979 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.979 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.979 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "::", 2024-05-08T22:18:07.979 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.979 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.979 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.979 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T22:18:07.979 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.979 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.980 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.980 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.980 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-05-08T22:18:07.980 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-05-08T22:18:07.980 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.980 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.980 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.980 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "9283", 2024-05-08T22:18:07.980 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.980 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.980 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.980 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T22:18:07.980 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.980 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.981 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.981 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.981 INFO:teuthology.orchestra.run.smithi131.stdout: "stale_cache_strategy": { 2024-05-08T22:18:07.981 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "stale_cache_strategy", 2024-05-08T22:18:07.981 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.981 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.981 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.981 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "log", 2024-05-08T22:18:07.981 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.981 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.981 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.981 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.981 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.981 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.982 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.982 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.982 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_behaviour": { 2024-05-08T22:18:07.982 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_behaviour", 2024-05-08T22:18:07.982 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.982 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.982 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.982 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "default", 2024-05-08T22:18:07.982 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.982 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.982 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.982 INFO:teuthology.orchestra.run.smithi131.stdout: "default", 2024-05-08T22:18:07.982 INFO:teuthology.orchestra.run.smithi131.stdout: "error" 2024-05-08T22:18:07.982 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.983 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.983 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.983 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.983 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.983 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.983 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_error_status_code": { 2024-05-08T22:18:07.983 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_error_status_code", 2024-05-08T22:18:07.983 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.983 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.983 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.983 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "500", 2024-05-08T22:18:07.983 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "400", 2024-05-08T22:18:07.983 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "599", 2024-05-08T22:18:07.983 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.983 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.983 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.984 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.984 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.984 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.984 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.984 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.984 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.984 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_support", 2024-05-08T22:18:07.984 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.984 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.984 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.984 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.984 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.984 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.984 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.984 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.984 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.985 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.985 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.985 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.985 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.985 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.985 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.985 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.985 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.985 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.985 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.985 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.985 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.985 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.985 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.985 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.985 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.985 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.986 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.986 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.986 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.986 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.986 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.986 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.986 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.986 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.986 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.986 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.986 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.986 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.986 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.986 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.986 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.987 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.987 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.987 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.987 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.987 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.987 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.987 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.987 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.987 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.987 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.987 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.987 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.987 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.987 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.987 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.987 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.988 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.988 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.988 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.988 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.988 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.988 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.988 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.988 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.988 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.988 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.988 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.988 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.988 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.989 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.989 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.989 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.989 INFO:teuthology.orchestra.run.smithi131.stdout: "max_concurrent_snap_create": { 2024-05-08T22:18:07.989 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_concurrent_snap_create", 2024-05-08T22:18:07.989 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:07.989 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.989 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.989 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-08T22:18:07.989 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.989 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.989 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.989 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.989 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.989 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.989 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.990 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.990 INFO:teuthology.orchestra.run.smithi131.stdout: "mirror_snapshot_schedule": { 2024-05-08T22:18:07.990 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T22:18:07.990 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.990 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.990 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.990 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.990 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.990 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.990 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.990 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.990 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.990 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.990 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.991 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.991 INFO:teuthology.orchestra.run.smithi131.stdout: "trash_purge_schedule": { 2024-05-08T22:18:07.991 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "trash_purge_schedule", 2024-05-08T22:18:07.991 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.991 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.991 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.991 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.991 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.991 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.991 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.991 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.991 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.991 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.991 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.991 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.992 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:07.992 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.992 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:07.992 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "restful", 2024-05-08T22:18:07.992 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:07.992 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:07.992 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:07.992 INFO:teuthology.orchestra.run.smithi131.stdout: "enable_auth": { 2024-05-08T22:18:07.992 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enable_auth", 2024-05-08T22:18:07.992 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.992 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.992 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.992 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:07.992 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.992 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.993 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.993 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.993 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.993 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.993 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.993 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.993 INFO:teuthology.orchestra.run.smithi131.stdout: "key_file": { 2024-05-08T22:18:07.993 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "key_file", 2024-05-08T22:18:07.993 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.993 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.993 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.993 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.993 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.993 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.993 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.994 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.994 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.994 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.994 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.994 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.994 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:07.994 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:07.994 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.994 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.994 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.994 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.994 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.994 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.994 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.994 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.995 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.995 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.995 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.995 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.995 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.995 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.995 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.995 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.995 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.995 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.995 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.995 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:07.995 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:07.995 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.995 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.996 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.996 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.996 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.996 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.996 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.996 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.996 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.996 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.996 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.996 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.996 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:07.996 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:07.996 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.996 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.996 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.997 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:07.997 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.997 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.997 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:07.997 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:07.997 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:07.997 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:07.997 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:07.997 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:07.997 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:07.997 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:07.997 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.997 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.997 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.997 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.997 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.998 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:07.998 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:07.998 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:07.998 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.998 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:07.998 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:07.998 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.998 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.998 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.998 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.998 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.998 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.998 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.998 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.999 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-05-08T22:18:07.999 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-05-08T22:18:07.999 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:07.999 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:07.999 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:07.999 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:07.999 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:07.999 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:07.999 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:07.999 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:07.999 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:07.999 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:07.999 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:07.999 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:07.999 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-05-08T22:18:08.000 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-05-08T22:18:08.000 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.000 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.000 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.000 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.000 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.000 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.000 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.000 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.000 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.000 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.000 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.000 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.000 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.000 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.001 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.001 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rgw", 2024-05-08T22:18:08.001 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.001 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.001 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.001 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.001 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.001 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.001 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.001 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.001 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.001 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.001 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.001 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.002 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.002 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.002 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.002 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.002 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.002 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.002 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.002 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.002 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.002 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.002 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.002 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.002 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.003 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.003 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.003 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.003 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.003 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.003 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.003 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.003 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.003 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.003 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.003 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.003 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.003 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.004 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.004 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.004 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.004 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.004 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.004 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.004 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.004 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.004 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.004 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.004 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.004 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.004 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.004 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.004 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.005 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.005 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.005 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.005 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.005 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.005 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.005 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.005 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.005 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.005 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.005 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.005 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.005 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.005 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.006 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.006 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.006 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.006 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.006 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.006 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.006 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.006 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.006 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.006 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rook", 2024-05-08T22:18:08.006 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.006 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.006 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.006 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.007 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.007 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.007 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.007 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.007 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.007 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.007 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.007 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.007 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.007 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.007 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.007 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.007 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.007 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.008 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.008 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.008 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.008 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.008 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.008 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.008 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.008 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.008 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.008 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.008 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.008 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.008 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.008 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.009 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.009 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.009 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.009 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.009 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.009 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.009 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.009 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.009 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.009 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.009 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.009 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.009 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.009 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.010 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.010 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.010 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.010 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.010 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.010 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.010 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.010 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.010 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.010 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.010 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.010 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.010 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.011 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.011 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.011 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.011 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.011 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.011 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.011 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.011 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.011 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.011 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.011 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.011 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.011 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.011 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.011 INFO:teuthology.orchestra.run.smithi131.stdout: "storage_class": { 2024-05-08T22:18:08.012 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "storage_class", 2024-05-08T22:18:08.012 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.012 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.012 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.012 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "local", 2024-05-08T22:18:08.012 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.012 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.012 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.012 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T22:18:08.012 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.012 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.012 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.012 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.012 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.013 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.013 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.013 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "selftest", 2024-05-08T22:18:08.013 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.013 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.013 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.013 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.013 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.013 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.013 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.013 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.013 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.013 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.013 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.013 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.014 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.014 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.014 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.014 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.014 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.014 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.014 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.014 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.014 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.014 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.014 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.014 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.014 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.014 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.014 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.015 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.015 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.015 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.015 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.015 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.015 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.015 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.015 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.015 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.015 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.015 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.015 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.015 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.015 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.015 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.016 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.016 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.016 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.016 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.016 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.016 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.016 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.016 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.016 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.016 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.016 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.016 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.016 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.016 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.017 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.017 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.017 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.017 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.017 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.017 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.017 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.017 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.017 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.017 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.017 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.017 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.017 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.017 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.017 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.017 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.018 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.018 INFO:teuthology.orchestra.run.smithi131.stdout: "roption1": { 2024-05-08T22:18:08.018 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "roption1", 2024-05-08T22:18:08.018 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.018 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.018 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.018 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.018 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.018 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.018 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.018 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.018 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.018 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.018 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.019 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.019 INFO:teuthology.orchestra.run.smithi131.stdout: "roption2": { 2024-05-08T22:18:08.019 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "roption2", 2024-05-08T22:18:08.019 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.019 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.019 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.019 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "xyz", 2024-05-08T22:18:08.019 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.019 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.019 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.019 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.019 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.019 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.019 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.019 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.020 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption1": { 2024-05-08T22:18:08.020 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption1", 2024-05-08T22:18:08.020 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.020 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.020 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.020 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.020 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.020 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.020 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.020 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.020 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.020 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.020 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.020 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.020 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption2": { 2024-05-08T22:18:08.020 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption2", 2024-05-08T22:18:08.021 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.021 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.021 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.021 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.021 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.021 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.021 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.021 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.021 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.021 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.021 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.021 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.021 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption3": { 2024-05-08T22:18:08.021 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption3", 2024-05-08T22:18:08.021 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-08T22:18:08.021 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.021 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.022 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.022 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.022 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.022 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.022 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.022 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.022 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.022 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.022 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.022 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption4": { 2024-05-08T22:18:08.022 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption4", 2024-05-08T22:18:08.022 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.022 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.022 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.022 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.023 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.023 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.023 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.023 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.023 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.023 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.023 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.023 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.023 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption5": { 2024-05-08T22:18:08.023 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption5", 2024-05-08T22:18:08.023 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.023 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.023 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.023 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.023 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.023 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.024 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.024 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.024 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.024 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.024 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.024 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.024 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption6": { 2024-05-08T22:18:08.024 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption6", 2024-05-08T22:18:08.024 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.024 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.024 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.024 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.024 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.024 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.024 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.024 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.025 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.025 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.025 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.025 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.025 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption7": { 2024-05-08T22:18:08.025 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption7", 2024-05-08T22:18:08.025 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.025 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.025 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.025 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.025 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-05-08T22:18:08.025 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "42", 2024-05-08T22:18:08.025 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.025 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.025 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.025 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.026 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.026 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.026 INFO:teuthology.orchestra.run.smithi131.stdout: "testkey": { 2024-05-08T22:18:08.026 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testkey", 2024-05-08T22:18:08.026 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.026 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.026 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.026 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.026 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.026 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.026 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.026 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.026 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.026 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.026 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.027 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.027 INFO:teuthology.orchestra.run.smithi131.stdout: "testlkey": { 2024-05-08T22:18:08.027 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testlkey", 2024-05-08T22:18:08.027 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.027 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.027 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.027 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.027 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.027 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.027 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.027 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.027 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.027 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.027 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.027 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.028 INFO:teuthology.orchestra.run.smithi131.stdout: "testnewline": { 2024-05-08T22:18:08.028 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testnewline", 2024-05-08T22:18:08.028 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.028 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.028 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.028 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.028 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.028 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.028 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.028 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.028 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.028 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.028 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.028 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.029 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.029 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.029 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.029 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snap_schedule", 2024-05-08T22:18:08.029 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.029 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.029 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.029 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_m_granularity": { 2024-05-08T22:18:08.029 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_m_granularity", 2024-05-08T22:18:08.029 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.029 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.029 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.029 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.029 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.030 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.030 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.030 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T22:18:08.030 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.030 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.030 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.030 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.030 INFO:teuthology.orchestra.run.smithi131.stdout: "dump_on_update": { 2024-05-08T22:18:08.030 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "dump_on_update", 2024-05-08T22:18:08.030 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.030 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.030 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.030 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.030 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.031 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.031 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.031 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "dump database to debug log on update", 2024-05-08T22:18:08.031 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.031 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.031 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.031 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.031 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.031 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.031 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.031 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.031 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.031 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.031 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.031 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.032 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.032 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.032 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.032 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.032 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.032 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.032 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.032 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.032 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.032 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.032 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.032 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.032 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.032 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.032 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.033 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.033 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.033 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.033 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.033 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.033 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.033 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.033 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.033 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.033 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.033 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.033 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.033 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.033 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.033 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.033 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.034 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.034 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.034 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.034 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.034 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.034 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.034 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.034 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.034 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.034 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.034 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.034 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.034 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.034 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.035 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.035 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.035 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.035 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.035 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.035 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.035 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.035 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.035 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.035 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.035 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.035 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.035 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.035 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.035 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.035 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.036 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.036 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.036 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.036 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.036 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "stats", 2024-05-08T22:18:08.036 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.036 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.036 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.036 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.036 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.036 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.036 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.036 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.036 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.036 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.037 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.037 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.037 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.037 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.037 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.037 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.037 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.037 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.037 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.037 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.037 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.037 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.037 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.037 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.037 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.037 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.038 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.038 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.038 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.038 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.038 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.038 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.038 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.038 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.038 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.038 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.038 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.038 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.038 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.038 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.038 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.039 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.039 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.039 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.039 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.039 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.039 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.039 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.039 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.039 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.039 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.039 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.039 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.039 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.039 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.039 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.039 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.039 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.040 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "status", 2024-05-08T22:18:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.041 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.042 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.043 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.044 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.045 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "telegraf", 2024-05-08T22:18:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: "address": { 2024-05-08T22:18:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "address", 2024-05-08T22:18:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T22:18:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-08T22:18:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-08T22:18:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15", 2024-05-08T22:18:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.047 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.048 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.049 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.050 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.051 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.051 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.051 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.051 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.051 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.051 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.051 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.051 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.051 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.051 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.051 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.051 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.051 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.051 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.051 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.051 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.052 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.052 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.052 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.052 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.052 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.052 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.052 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.052 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.052 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.052 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.052 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.052 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.052 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.052 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.052 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.052 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.053 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.053 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.053 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "telemetry", 2024-05-08T22:18:08.053 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.053 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.053 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.053 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_basic": { 2024-05-08T22:18:08.053 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_basic", 2024-05-08T22:18:08.053 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.053 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.053 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.053 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.053 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.053 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.053 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.054 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T22:18:08.054 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.054 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.054 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.054 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.054 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_crash": { 2024-05-08T22:18:08.054 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_crash", 2024-05-08T22:18:08.054 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.054 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.054 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.054 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.054 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.054 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.054 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.055 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T22:18:08.055 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.055 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.055 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.055 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.055 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_device": { 2024-05-08T22:18:08.055 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_device", 2024-05-08T22:18:08.055 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.055 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.055 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.055 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.055 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.055 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.055 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.055 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T22:18:08.056 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.056 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.056 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.056 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.056 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_ident": { 2024-05-08T22:18:08.056 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_ident", 2024-05-08T22:18:08.056 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.056 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.056 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.056 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.056 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.056 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.056 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.056 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T22:18:08.056 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.057 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.057 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.057 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.057 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_perf": { 2024-05-08T22:18:08.057 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_perf", 2024-05-08T22:18:08.057 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.057 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.057 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.057 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.057 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.057 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.057 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.057 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T22:18:08.057 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.057 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.057 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.057 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.058 INFO:teuthology.orchestra.run.smithi131.stdout: "contact": { 2024-05-08T22:18:08.058 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "contact", 2024-05-08T22:18:08.058 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.058 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.058 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.058 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.058 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.058 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.058 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.058 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.058 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.058 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.058 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.058 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.058 INFO:teuthology.orchestra.run.smithi131.stdout: "description": { 2024-05-08T22:18:08.058 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "description", 2024-05-08T22:18:08.059 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.059 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.059 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.059 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.059 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.059 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.059 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.059 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.059 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.059 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.059 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.059 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.059 INFO:teuthology.orchestra.run.smithi131.stdout: "device_url": { 2024-05-08T22:18:08.059 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_url", 2024-05-08T22:18:08.059 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.059 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.060 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.060 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T22:18:08.060 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.060 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.060 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.060 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.060 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.060 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.060 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.060 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.060 INFO:teuthology.orchestra.run.smithi131.stdout: "enabled": { 2024-05-08T22:18:08.060 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enabled", 2024-05-08T22:18:08.060 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.060 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.060 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.060 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.061 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.061 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.061 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.061 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.061 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.061 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.061 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.061 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.061 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-08T22:18:08.061 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-08T22:18:08.061 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.061 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.061 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.061 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "24", 2024-05-08T22:18:08.061 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "8", 2024-05-08T22:18:08.062 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.062 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.062 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.062 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.062 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.062 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.062 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.062 INFO:teuthology.orchestra.run.smithi131.stdout: "last_opt_revision": { 2024-05-08T22:18:08.062 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "last_opt_revision", 2024-05-08T22:18:08.062 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.062 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.062 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.062 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1", 2024-05-08T22:18:08.062 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.062 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.062 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.063 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.063 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.063 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.063 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.063 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.063 INFO:teuthology.orchestra.run.smithi131.stdout: "leaderboard": { 2024-05-08T22:18:08.063 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "leaderboard", 2024-05-08T22:18:08.063 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.063 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.063 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.063 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.063 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.063 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.063 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.063 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.063 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.064 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.064 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.064 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.064 INFO:teuthology.orchestra.run.smithi131.stdout: "leaderboard_description": { 2024-05-08T22:18:08.064 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "leaderboard_description", 2024-05-08T22:18:08.064 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.064 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.064 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.064 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.064 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.064 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.064 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.064 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.064 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.064 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.064 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.064 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.065 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.065 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.065 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.065 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.065 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.065 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.065 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.065 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.065 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.065 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.065 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.065 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.065 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.065 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.065 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.065 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.066 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.066 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.066 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.066 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.066 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.066 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.066 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.066 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.066 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.066 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.066 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.066 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.066 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.066 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.066 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.067 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.067 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.067 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.067 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.067 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.067 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.067 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.067 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.067 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.067 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.067 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.067 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.067 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.068 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.068 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.068 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.068 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.068 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.068 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.068 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.068 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.068 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.068 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.068 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.068 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.068 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.068 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.068 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.068 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.069 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.069 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.069 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.069 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.069 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.069 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.069 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.069 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.069 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.069 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.069 INFO:teuthology.orchestra.run.smithi131.stdout: "organization": { 2024-05-08T22:18:08.069 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "organization", 2024-05-08T22:18:08.069 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.069 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.069 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.069 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.069 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.070 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.070 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.070 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.070 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.070 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.070 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.070 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.070 INFO:teuthology.orchestra.run.smithi131.stdout: "proxy": { 2024-05-08T22:18:08.070 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "proxy", 2024-05-08T22:18:08.070 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.070 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.070 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.070 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.070 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.070 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.071 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.071 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.071 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.071 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.071 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.071 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.071 INFO:teuthology.orchestra.run.smithi131.stdout: "url": { 2024-05-08T22:18:08.071 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "url", 2024-05-08T22:18:08.071 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.071 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.071 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.071 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T22:18:08.071 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.071 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.071 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.072 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.072 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.072 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.072 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.072 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.072 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.072 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.072 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.072 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "test_orchestrator", 2024-05-08T22:18:08.072 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.072 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.072 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.072 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.072 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.072 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.072 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.072 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.073 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.073 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.073 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.073 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.073 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.073 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.073 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.073 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.073 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.073 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.073 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.073 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.073 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.073 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.073 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.073 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.074 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.074 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.074 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.074 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.074 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.074 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.074 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.074 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.074 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.074 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.074 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.074 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.074 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.074 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.074 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.074 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.075 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.075 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.075 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.075 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.075 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.075 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.075 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.075 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.075 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.075 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.075 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.075 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.075 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.075 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.075 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.075 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.075 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.076 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.076 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.076 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.076 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.076 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.076 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.076 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.076 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.076 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.076 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.076 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.076 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.076 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.076 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.076 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.076 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.077 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.077 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.077 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.077 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "volumes", 2024-05-08T22:18:08.077 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.077 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.077 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.077 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.077 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.077 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.077 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.077 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.077 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.077 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.077 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.077 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.078 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.078 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.078 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.078 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.078 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.078 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.078 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.078 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.078 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.078 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.078 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.078 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.078 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.078 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.078 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.078 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.079 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.079 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.079 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.079 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.079 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.079 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.079 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.079 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.079 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.079 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.079 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.079 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.079 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.079 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.079 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.079 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.080 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.080 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.080 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.080 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.080 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.080 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.080 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.080 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.080 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.080 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.080 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.080 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.080 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.080 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.080 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.080 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.081 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.081 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.081 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.081 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.081 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.081 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.081 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.081 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.081 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.081 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.081 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.081 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.081 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.081 INFO:teuthology.orchestra.run.smithi131.stdout: "max_concurrent_clones": { 2024-05-08T22:18:08.082 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_concurrent_clones", 2024-05-08T22:18:08.082 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.082 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.082 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.082 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "4", 2024-05-08T22:18:08.082 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.082 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.082 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.082 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T22:18:08.082 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.082 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.082 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.082 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.082 INFO:teuthology.orchestra.run.smithi131.stdout: "snapshot_clone_delay": { 2024-05-08T22:18:08.082 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snapshot_clone_delay", 2024-05-08T22:18:08.082 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.083 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.083 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.083 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-08T22:18:08.083 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.083 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.083 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.083 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T22:18:08.083 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.083 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.083 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.083 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.083 INFO:teuthology.orchestra.run.smithi131.stdout: "snapshot_clone_no_wait": { 2024-05-08T22:18:08.083 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T22:18:08.083 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.083 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.084 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.084 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.084 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.084 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.084 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.084 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T22:18:08.084 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.084 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.084 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.084 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.084 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.084 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.084 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.084 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix", 2024-05-08T22:18:08.084 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.084 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.085 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.085 INFO:teuthology.orchestra.run.smithi131.stdout: "discovery_interval": { 2024-05-08T22:18:08.085 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "discovery_interval", 2024-05-08T22:18:08.085 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-08T22:18:08.085 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.085 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.085 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "100", 2024-05-08T22:18:08.085 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.085 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.085 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.085 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.085 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.085 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.085 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.085 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.085 INFO:teuthology.orchestra.run.smithi131.stdout: "identifier": { 2024-05-08T22:18:08.086 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "identifier", 2024-05-08T22:18:08.086 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.086 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.086 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.086 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.086 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.086 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.086 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.086 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.086 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.086 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.086 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.086 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.086 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-08T22:18:08.086 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-08T22:18:08.086 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:08.087 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.087 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.087 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-08T22:18:08.087 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.087 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.087 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.087 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.087 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.087 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.087 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.087 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.087 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.087 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.087 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.087 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.087 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.088 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.088 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.088 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.088 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.088 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.088 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.088 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.088 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.088 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.088 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.088 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.088 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.088 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.088 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.088 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.088 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.089 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.089 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.089 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.089 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.089 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.089 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.089 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.089 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.089 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.089 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.089 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.089 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.089 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.089 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.089 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.089 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.089 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.090 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.090 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.090 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.090 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.090 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.090 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.090 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.090 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.090 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.090 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.090 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.090 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.090 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.091 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.091 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.091 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.091 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.091 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.091 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.091 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.091 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.091 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.091 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.091 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.091 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.091 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.091 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.091 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.091 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.092 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.092 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.092 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.092 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_host": { 2024-05-08T22:18:08.092 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_host", 2024-05-08T22:18:08.092 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.092 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.092 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.092 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.092 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.092 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.092 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.092 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.092 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.092 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.092 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.093 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.093 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_port": { 2024-05-08T22:18:08.093 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_port", 2024-05-08T22:18:08.093 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.093 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.093 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.093 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10051", 2024-05-08T22:18:08.093 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.093 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.093 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.093 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.093 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.093 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.093 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.093 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.093 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_sender": { 2024-05-08T22:18:08.093 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_sender", 2024-05-08T22:18:08.094 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.094 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.094 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.094 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T22:18:08.094 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.094 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.094 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.094 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.094 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.094 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.094 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.094 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.094 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.094 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.094 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-08T22:18:08.095 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.095 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.095 INFO:teuthology.orchestra.run.smithi131.stdout: "modules": [ 2024-05-08T22:18:08.095 INFO:teuthology.orchestra.run.smithi131.stdout: "cephadm", 2024-05-08T22:18:08.095 INFO:teuthology.orchestra.run.smithi131.stdout: "dashboard", 2024-05-08T22:18:08.095 INFO:teuthology.orchestra.run.smithi131.stdout: "iostat", 2024-05-08T22:18:08.095 INFO:teuthology.orchestra.run.smithi131.stdout: "nfs", 2024-05-08T22:18:08.095 INFO:teuthology.orchestra.run.smithi131.stdout: "restful", 2024-05-08T22:18:08.095 INFO:teuthology.orchestra.run.smithi131.stdout: "snap_schedule" 2024-05-08T22:18:08.095 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.095 INFO:teuthology.orchestra.run.smithi131.stdout: "available_modules": [ 2024-05-08T22:18:08.095 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.095 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "alerts", 2024-05-08T22:18:08.095 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.095 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.095 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.096 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-08T22:18:08.096 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-08T22:18:08.096 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:08.096 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.096 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.096 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-08T22:18:08.096 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.096 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.096 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.096 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T22:18:08.096 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.096 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.096 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.096 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.097 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.097 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.097 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.097 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.097 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.097 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.097 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.097 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.097 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.097 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.097 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.097 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.097 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.097 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.098 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.098 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.098 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.098 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.098 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.098 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.098 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.098 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.098 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.098 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.098 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.098 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.098 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.098 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.099 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.099 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.099 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.099 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.099 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.099 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.099 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.099 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.099 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.099 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.099 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.099 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.099 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.099 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.099 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.100 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.100 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.100 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.100 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.100 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.100 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.100 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.100 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.100 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.100 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.100 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.100 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.100 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.100 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.100 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.100 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.101 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.101 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.101 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.101 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.101 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.101 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.101 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.101 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.101 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.101 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.101 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.101 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_destination": { 2024-05-08T22:18:08.101 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_destination", 2024-05-08T22:18:08.101 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.101 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.101 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.101 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.102 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.102 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.102 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.102 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Email address to send alerts to", 2024-05-08T22:18:08.102 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.102 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.102 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.102 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.102 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_from_name": { 2024-05-08T22:18:08.102 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_from_name", 2024-05-08T22:18:08.102 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.102 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.102 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.102 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "Ceph", 2024-05-08T22:18:08.102 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.103 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.103 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.103 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Email From: name", 2024-05-08T22:18:08.103 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.103 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.103 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.103 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.103 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_host": { 2024-05-08T22:18:08.103 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_host", 2024-05-08T22:18:08.103 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.103 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.103 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.103 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.103 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.103 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.103 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.104 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP server", 2024-05-08T22:18:08.104 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.104 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.104 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.104 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.104 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_password": { 2024-05-08T22:18:08.104 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_password", 2024-05-08T22:18:08.104 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.104 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.104 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.104 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.104 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.104 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.104 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.104 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Password to authenticate with", 2024-05-08T22:18:08.104 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.105 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.105 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.105 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.105 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_port": { 2024-05-08T22:18:08.105 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_port", 2024-05-08T22:18:08.105 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.105 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.105 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.105 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "465", 2024-05-08T22:18:08.105 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.105 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.105 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.105 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP port", 2024-05-08T22:18:08.105 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.105 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.105 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.105 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.106 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_sender": { 2024-05-08T22:18:08.106 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_sender", 2024-05-08T22:18:08.106 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.106 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.106 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.106 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.106 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.106 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.106 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.106 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP envelope sender", 2024-05-08T22:18:08.106 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.106 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.106 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.107 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.107 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_ssl": { 2024-05-08T22:18:08.107 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_ssl", 2024-05-08T22:18:08.107 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.107 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.107 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.107 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.107 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.107 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.107 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.107 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T22:18:08.107 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.107 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.107 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.107 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.107 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_user": { 2024-05-08T22:18:08.108 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_user", 2024-05-08T22:18:08.108 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.108 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.108 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.108 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.108 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.108 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.108 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.108 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "User to authenticate as", 2024-05-08T22:18:08.108 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.108 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.108 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.108 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.108 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.108 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.108 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.108 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "balancer", 2024-05-08T22:18:08.109 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.109 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.109 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.109 INFO:teuthology.orchestra.run.smithi131.stdout: "active": { 2024-05-08T22:18:08.109 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "active", 2024-05-08T22:18:08.109 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.109 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.109 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.109 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.109 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.109 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.109 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.109 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T22:18:08.109 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.109 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.109 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.110 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.110 INFO:teuthology.orchestra.run.smithi131.stdout: "begin_time": { 2024-05-08T22:18:08.110 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "begin_time", 2024-05-08T22:18:08.110 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.110 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.110 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.110 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0000", 2024-05-08T22:18:08.110 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.110 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.110 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.110 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T22:18:08.110 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T22:18:08.110 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.110 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.110 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.110 INFO:teuthology.orchestra.run.smithi131.stdout: "begin_weekday": { 2024-05-08T22:18:08.111 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "begin_weekday", 2024-05-08T22:18:08.111 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-08T22:18:08.111 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.111 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.111 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-08T22:18:08.111 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0", 2024-05-08T22:18:08.111 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "6", 2024-05-08T22:18:08.111 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.111 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T22:18:08.111 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T22:18:08.111 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.111 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.111 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.111 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_max_iterations": { 2024-05-08T22:18:08.111 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_max_iterations", 2024-05-08T22:18:08.111 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-08T22:18:08.111 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.112 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.112 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "25", 2024-05-08T22:18:08.112 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-05-08T22:18:08.112 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "250", 2024-05-08T22:18:08.112 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.112 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T22:18:08.112 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.112 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.112 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.112 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.112 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_metrics": { 2024-05-08T22:18:08.112 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_metrics", 2024-05-08T22:18:08.112 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.112 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.112 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.113 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T22:18:08.113 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.113 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.113 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.113 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T22:18:08.113 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T22:18:08.113 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.113 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.113 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.113 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_step": { 2024-05-08T22:18:08.113 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_step", 2024-05-08T22:18:08.113 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-08T22:18:08.113 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.113 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.113 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0.5", 2024-05-08T22:18:08.113 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0.001", 2024-05-08T22:18:08.114 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "0.999", 2024-05-08T22:18:08.114 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.114 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "aggressiveness of optimization", 2024-05-08T22:18:08.114 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T22:18:08.114 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.114 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.114 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.114 INFO:teuthology.orchestra.run.smithi131.stdout: "end_time": { 2024-05-08T22:18:08.114 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "end_time", 2024-05-08T22:18:08.114 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.114 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.114 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.114 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "2359", 2024-05-08T22:18:08.114 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.114 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.115 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.115 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T22:18:08.115 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T22:18:08.115 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.115 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.115 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.115 INFO:teuthology.orchestra.run.smithi131.stdout: "end_weekday": { 2024-05-08T22:18:08.115 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "end_weekday", 2024-05-08T22:18:08.115 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-08T22:18:08.115 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.115 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.115 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-08T22:18:08.115 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0", 2024-05-08T22:18:08.115 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "6", 2024-05-08T22:18:08.115 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.115 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T22:18:08.115 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T22:18:08.116 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.116 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.116 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.116 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.116 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.116 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.116 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.116 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.116 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.116 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.116 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.116 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.116 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.116 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.116 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.116 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.117 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.117 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.117 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.117 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.117 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.117 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.117 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.117 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.117 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.117 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.117 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.117 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.117 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.117 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.117 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.117 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.117 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.118 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.118 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.118 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.118 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.118 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.118 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.118 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.118 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.118 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.118 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.118 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.118 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.118 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.118 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.118 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.118 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.119 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.119 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.119 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.119 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.119 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.119 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.119 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.119 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.119 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.119 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.119 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.119 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.119 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.119 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.120 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.120 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.120 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.120 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.120 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.120 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.120 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.120 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.120 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.120 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.120 INFO:teuthology.orchestra.run.smithi131.stdout: "min_score": { 2024-05-08T22:18:08.120 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "min_score", 2024-05-08T22:18:08.120 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-08T22:18:08.120 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.120 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.120 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-08T22:18:08.120 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.121 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.121 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.121 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T22:18:08.121 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.121 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.121 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.121 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.121 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": { 2024-05-08T22:18:08.121 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mode", 2024-05-08T22:18:08.121 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.121 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.121 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.121 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "upmap", 2024-05-08T22:18:08.121 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.121 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.121 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.122 INFO:teuthology.orchestra.run.smithi131.stdout: "crush-compat", 2024-05-08T22:18:08.122 INFO:teuthology.orchestra.run.smithi131.stdout: "none", 2024-05-08T22:18:08.122 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap" 2024-05-08T22:18:08.122 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.122 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Balancer mode", 2024-05-08T22:18:08.122 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.122 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.122 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.122 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.122 INFO:teuthology.orchestra.run.smithi131.stdout: "pool_ids": { 2024-05-08T22:18:08.122 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pool_ids", 2024-05-08T22:18:08.122 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.122 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.122 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.122 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.122 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.122 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.123 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.123 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T22:18:08.123 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.123 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.123 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.123 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.123 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-08T22:18:08.123 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-08T22:18:08.123 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:08.123 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.123 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.123 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-08T22:18:08.123 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.123 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.123 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.124 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T22:18:08.124 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.124 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.124 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.124 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.124 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap_max_deviation": { 2024-05-08T22:18:08.124 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "upmap_max_deviation", 2024-05-08T22:18:08.124 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.124 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.124 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.124 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-05-08T22:18:08.124 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-05-08T22:18:08.124 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.124 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.124 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T22:18:08.125 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T22:18:08.125 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.125 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.125 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.125 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap_max_optimizations": { 2024-05-08T22:18:08.125 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "upmap_max_optimizations", 2024-05-08T22:18:08.125 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-08T22:18:08.125 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.125 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.125 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-08T22:18:08.125 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.125 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.125 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.125 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T22:18:08.125 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.125 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.125 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.126 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.126 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.126 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.126 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.126 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cephadm", 2024-05-08T22:18:08.126 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.126 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.126 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.126 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_down_multiplier": { 2024-05-08T22:18:08.126 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_down_multiplier", 2024-05-08T22:18:08.126 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-08T22:18:08.126 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.126 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.126 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3.0", 2024-05-08T22:18:08.126 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.127 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.127 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.127 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T22:18:08.127 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.127 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.127 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.127 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.127 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_refresh_rate": { 2024-05-08T22:18:08.127 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_refresh_rate", 2024-05-08T22:18:08.127 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:08.127 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.127 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.127 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "20", 2024-05-08T22:18:08.127 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.127 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.128 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.128 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T22:18:08.128 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.128 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.128 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.128 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.128 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_starting_port": { 2024-05-08T22:18:08.128 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_starting_port", 2024-05-08T22:18:08.128 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.128 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.128 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.128 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "4721", 2024-05-08T22:18:08.128 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.128 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.129 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.129 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T22:18:08.129 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.129 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.129 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.129 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.129 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_ptrace": { 2024-05-08T22:18:08.129 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_ptrace", 2024-05-08T22:18:08.129 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.129 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.129 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.129 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.129 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.129 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.129 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.130 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T22:18:08.130 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T22:18:08.130 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.130 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.130 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.130 INFO:teuthology.orchestra.run.smithi131.stdout: "autotune_interval": { 2024-05-08T22:18:08.130 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "autotune_interval", 2024-05-08T22:18:08.130 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:08.130 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.130 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.130 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-08T22:18:08.130 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.130 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.130 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.130 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T22:18:08.131 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.131 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.131 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.131 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.131 INFO:teuthology.orchestra.run.smithi131.stdout: "autotune_memory_target_ratio": { 2024-05-08T22:18:08.131 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T22:18:08.131 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-08T22:18:08.131 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.131 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.131 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0.7", 2024-05-08T22:18:08.131 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.131 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.131 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.131 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T22:18:08.131 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.131 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.132 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.132 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.132 INFO:teuthology.orchestra.run.smithi131.stdout: "cgroups_split": { 2024-05-08T22:18:08.132 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cgroups_split", 2024-05-08T22:18:08.132 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.132 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.132 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.132 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.132 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.132 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.132 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.132 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T22:18:08.132 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.132 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.133 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.133 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.133 INFO:teuthology.orchestra.run.smithi131.stdout: "config_checks_enabled": { 2024-05-08T22:18:08.133 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_checks_enabled", 2024-05-08T22:18:08.133 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.133 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.133 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.133 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.133 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.133 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.133 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.133 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T22:18:08.133 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.133 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.133 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.134 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.134 INFO:teuthology.orchestra.run.smithi131.stdout: "config_dashboard": { 2024-05-08T22:18:08.134 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_dashboard", 2024-05-08T22:18:08.134 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.134 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.134 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.134 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.134 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.134 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.134 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.134 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T22:18:08.134 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.134 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.134 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.134 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.134 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_alertmanager": { 2024-05-08T22:18:08.135 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_alertmanager", 2024-05-08T22:18:08.135 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.135 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.135 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.135 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T22:18:08.135 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.135 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.135 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.135 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-05-08T22:18:08.135 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.135 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.135 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.135 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.135 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_base": { 2024-05-08T22:18:08.135 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_base", 2024-05-08T22:18:08.135 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.136 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.136 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.136 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T22:18:08.136 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.136 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.136 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.136 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Container image name, without the tag", 2024-05-08T22:18:08.136 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.136 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.136 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.136 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.136 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_elasticsearch": { 2024-05-08T22:18:08.136 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_elasticsearch", 2024-05-08T22:18:08.136 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.136 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.137 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.137 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T22:18:08.137 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.137 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.137 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.137 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "elasticsearch container image", 2024-05-08T22:18:08.137 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.137 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.137 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.137 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.137 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_grafana": { 2024-05-08T22:18:08.137 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_grafana", 2024-05-08T22:18:08.137 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.137 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.137 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.138 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-08T22:18:08.138 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.138 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.138 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.138 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-05-08T22:18:08.138 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.138 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.138 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.138 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.138 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_haproxy": { 2024-05-08T22:18:08.138 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_haproxy", 2024-05-08T22:18:08.138 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.138 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.138 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.139 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T22:18:08.139 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.139 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.139 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.139 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "HAproxy container image", 2024-05-08T22:18:08.139 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.139 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.139 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.139 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.139 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_agent": { 2024-05-08T22:18:08.139 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_agent", 2024-05-08T22:18:08.139 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.139 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.139 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.139 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T22:18:08.139 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.140 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.140 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.140 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger agent container image", 2024-05-08T22:18:08.140 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.140 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.140 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.140 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.140 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_collector": { 2024-05-08T22:18:08.140 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_collector", 2024-05-08T22:18:08.140 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.140 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.140 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.140 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T22:18:08.140 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.140 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.140 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.141 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger collector container image", 2024-05-08T22:18:08.141 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.141 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.141 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.141 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.141 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_query": { 2024-05-08T22:18:08.141 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_query", 2024-05-08T22:18:08.141 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.141 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.141 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.141 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T22:18:08.141 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.141 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.141 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.141 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger query container image", 2024-05-08T22:18:08.141 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.142 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.142 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.142 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.142 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_keepalived": { 2024-05-08T22:18:08.142 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_keepalived", 2024-05-08T22:18:08.142 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.142 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.142 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.142 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T22:18:08.142 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.142 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.142 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.142 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Keepalived container image", 2024-05-08T22:18:08.142 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.142 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.142 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.143 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.143 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_loki": { 2024-05-08T22:18:08.143 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_loki", 2024-05-08T22:18:08.143 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.143 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.143 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.143 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T22:18:08.143 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.143 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.143 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.143 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Loki container image", 2024-05-08T22:18:08.143 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.143 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.143 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.143 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.143 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_node_exporter": { 2024-05-08T22:18:08.144 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_node_exporter", 2024-05-08T22:18:08.144 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.144 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.144 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.144 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T22:18:08.144 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.144 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.144 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.144 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-05-08T22:18:08.144 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.144 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.144 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.144 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.144 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_nvmeof": { 2024-05-08T22:18:08.145 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_nvmeof", 2024-05-08T22:18:08.145 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.145 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.145 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.145 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-08T22:18:08.145 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.145 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.145 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.145 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Nvme-of container image", 2024-05-08T22:18:08.145 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.145 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.145 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.145 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.146 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_prometheus": { 2024-05-08T22:18:08.146 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_prometheus", 2024-05-08T22:18:08.146 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.146 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.146 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.146 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T22:18:08.146 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.146 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.146 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.146 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-05-08T22:18:08.146 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.146 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.146 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.146 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.147 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_promtail": { 2024-05-08T22:18:08.147 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_promtail", 2024-05-08T22:18:08.147 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.147 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.147 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.147 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T22:18:08.147 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.147 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.147 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.147 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Promtail container image", 2024-05-08T22:18:08.147 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.147 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.147 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.147 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.147 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_snmp_gateway": { 2024-05-08T22:18:08.148 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_snmp_gateway", 2024-05-08T22:18:08.148 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.148 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.148 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.148 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T22:18:08.148 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.148 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.148 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.148 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SNMP Gateway container image", 2024-05-08T22:18:08.148 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.148 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.148 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.148 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.148 INFO:teuthology.orchestra.run.smithi131.stdout: "container_init": { 2024-05-08T22:18:08.148 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_init", 2024-05-08T22:18:08.149 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.149 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.149 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.149 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.149 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.149 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.149 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.149 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T22:18:08.149 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.149 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.149 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.149 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.149 INFO:teuthology.orchestra.run.smithi131.stdout: "daemon_cache_timeout": { 2024-05-08T22:18:08.149 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "daemon_cache_timeout", 2024-05-08T22:18:08.149 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:08.150 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.150 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.150 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-08T22:18:08.150 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.150 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.150 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.150 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T22:18:08.150 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.150 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.150 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.150 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.150 INFO:teuthology.orchestra.run.smithi131.stdout: "default_cephadm_command_timeout": { 2024-05-08T22:18:08.150 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T22:18:08.150 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.150 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.151 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.151 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "900", 2024-05-08T22:18:08.151 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.151 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.151 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.151 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T22:18:08.151 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.151 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.151 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.151 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.151 INFO:teuthology.orchestra.run.smithi131.stdout: "default_registry": { 2024-05-08T22:18:08.151 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "default_registry", 2024-05-08T22:18:08.151 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.151 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.151 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.151 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io", 2024-05-08T22:18:08.152 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.152 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.152 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.152 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T22:18:08.152 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.152 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.152 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.152 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.152 INFO:teuthology.orchestra.run.smithi131.stdout: "device_cache_timeout": { 2024-05-08T22:18:08.152 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_cache_timeout", 2024-05-08T22:18:08.152 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:08.152 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.152 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.152 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1800", 2024-05-08T22:18:08.153 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.153 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.153 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.153 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache device inventory", 2024-05-08T22:18:08.153 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.153 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.153 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.153 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.153 INFO:teuthology.orchestra.run.smithi131.stdout: "device_enhanced_scan": { 2024-05-08T22:18:08.153 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_enhanced_scan", 2024-05-08T22:18:08.153 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.153 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.153 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.153 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.153 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.154 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.154 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.154 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T22:18:08.154 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.154 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.154 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.154 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.154 INFO:teuthology.orchestra.run.smithi131.stdout: "facts_cache_timeout": { 2024-05-08T22:18:08.154 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "facts_cache_timeout", 2024-05-08T22:18:08.154 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:08.154 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.154 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.154 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-08T22:18:08.154 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.154 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.155 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.155 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache host facts data", 2024-05-08T22:18:08.155 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.155 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.155 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.155 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.155 INFO:teuthology.orchestra.run.smithi131.stdout: "host_check_interval": { 2024-05-08T22:18:08.155 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "host_check_interval", 2024-05-08T22:18:08.155 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:08.155 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.155 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.155 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-08T22:18:08.155 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.155 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.155 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.156 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to perform a host check", 2024-05-08T22:18:08.156 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.156 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.156 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.156 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.156 INFO:teuthology.orchestra.run.smithi131.stdout: "hw_monitoring": { 2024-05-08T22:18:08.156 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "hw_monitoring", 2024-05-08T22:18:08.156 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.156 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.156 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.156 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.156 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.156 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.156 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.156 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T22:18:08.157 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.157 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.157 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.157 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.157 INFO:teuthology.orchestra.run.smithi131.stdout: "inventory_list_all": { 2024-05-08T22:18:08.157 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "inventory_list_all", 2024-05-08T22:18:08.157 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.157 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.157 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.157 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.157 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.157 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.157 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.157 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T22:18:08.158 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.158 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.158 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.158 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.158 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.158 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.158 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.158 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.158 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.158 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.158 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.158 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.158 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.158 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.158 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.159 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.161 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.161 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.161 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.161 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.162 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.162 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.162 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.162 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.162 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.162 INFO:teuthology.orchestra.run.smithi131.stdout: "log_refresh_metadata": { 2024-05-08T22:18:08.162 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_refresh_metadata", 2024-05-08T22:18:08.162 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.162 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.162 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.162 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.162 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.162 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.162 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.163 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T22:18:08.163 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.163 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.163 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.163 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.163 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.163 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.163 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.163 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.163 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.163 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.163 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.163 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.163 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.164 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T22:18:08.164 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.164 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.164 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.164 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.164 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.164 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.164 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.164 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.164 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.164 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.164 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.164 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.164 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.165 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.165 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.165 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.165 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.165 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.165 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.165 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.165 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.165 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.165 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.165 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.165 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.165 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.165 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.165 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.166 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.166 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.166 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.166 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.166 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.166 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.166 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.166 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.166 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.166 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.166 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.166 INFO:teuthology.orchestra.run.smithi131.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T22:18:08.166 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T22:18:08.166 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.166 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.167 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.167 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.167 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.167 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.167 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.167 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T22:18:08.167 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.167 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.167 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.167 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.167 INFO:teuthology.orchestra.run.smithi131.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T22:18:08.167 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T22:18:08.167 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.168 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.168 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.168 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "*", 2024-05-08T22:18:08.168 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.168 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.168 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.168 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T22:18:08.168 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.168 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.168 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.168 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.168 INFO:teuthology.orchestra.run.smithi131.stdout: "max_count_per_host": { 2024-05-08T22:18:08.168 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_count_per_host", 2024-05-08T22:18:08.168 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.169 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.169 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.169 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-08T22:18:08.169 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.169 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.169 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.169 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "max number of daemons per service per host", 2024-05-08T22:18:08.169 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.169 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.169 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.169 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.169 INFO:teuthology.orchestra.run.smithi131.stdout: "max_osd_draining_count": { 2024-05-08T22:18:08.169 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_osd_draining_count", 2024-05-08T22:18:08.169 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.169 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.170 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.170 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-08T22:18:08.170 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.170 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.170 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.170 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T22:18:08.170 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.170 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.170 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.170 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.170 INFO:teuthology.orchestra.run.smithi131.stdout: "migration_current": { 2024-05-08T22:18:08.170 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "migration_current", 2024-05-08T22:18:08.170 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.170 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.171 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.171 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.171 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.171 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.171 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.171 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "internal - do not modify", 2024-05-08T22:18:08.171 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.171 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.171 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.171 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.171 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": { 2024-05-08T22:18:08.171 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mode", 2024-05-08T22:18:08.171 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.171 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.171 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.171 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "root", 2024-05-08T22:18:08.172 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.172 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.172 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.172 INFO:teuthology.orchestra.run.smithi131.stdout: "cephadm-package", 2024-05-08T22:18:08.172 INFO:teuthology.orchestra.run.smithi131.stdout: "root" 2024-05-08T22:18:08.172 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.172 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T22:18:08.172 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.172 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.172 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.172 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.172 INFO:teuthology.orchestra.run.smithi131.stdout: "oob_default_addr": { 2024-05-08T22:18:08.172 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "oob_default_addr", 2024-05-08T22:18:08.172 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.172 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.173 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.173 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "169.254.1.1", 2024-05-08T22:18:08.173 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.173 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.173 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.173 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T22:18:08.173 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.173 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.173 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.173 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.173 INFO:teuthology.orchestra.run.smithi131.stdout: "prometheus_alerts_path": { 2024-05-08T22:18:08.173 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prometheus_alerts_path", 2024-05-08T22:18:08.173 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.173 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.173 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.173 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T22:18:08.174 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.174 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.174 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.174 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T22:18:08.174 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.174 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.174 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.174 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.174 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_insecure": { 2024-05-08T22:18:08.174 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_insecure", 2024-05-08T22:18:08.174 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.174 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.174 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.175 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.175 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.175 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.175 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.175 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T22:18:08.175 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.175 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.175 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.175 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.175 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_password": { 2024-05-08T22:18:08.175 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_password", 2024-05-08T22:18:08.175 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.175 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.175 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.175 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.176 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.176 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.176 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.176 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T22:18:08.176 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.176 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.176 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.176 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.176 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_url": { 2024-05-08T22:18:08.176 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_url", 2024-05-08T22:18:08.176 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.176 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.176 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.176 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.177 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.177 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.177 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.177 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T22:18:08.177 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.177 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.177 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.177 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.177 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_username": { 2024-05-08T22:18:08.177 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_username", 2024-05-08T22:18:08.177 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.177 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.177 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.177 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.177 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.178 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.178 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.178 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T22:18:08.178 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.178 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.178 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.178 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.178 INFO:teuthology.orchestra.run.smithi131.stdout: "secure_monitoring_stack": { 2024-05-08T22:18:08.178 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "secure_monitoring_stack", 2024-05-08T22:18:08.178 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.178 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.178 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.178 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.178 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.179 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.179 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.179 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T22:18:08.179 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.179 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.179 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.179 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.179 INFO:teuthology.orchestra.run.smithi131.stdout: "service_discovery_port": { 2024-05-08T22:18:08.179 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "service_discovery_port", 2024-05-08T22:18:08.179 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.179 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.179 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.179 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8765", 2024-05-08T22:18:08.179 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.179 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.179 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.180 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "cephadm service discovery port", 2024-05-08T22:18:08.180 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.180 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.180 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.180 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.180 INFO:teuthology.orchestra.run.smithi131.stdout: "ssh_config_file": { 2024-05-08T22:18:08.180 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssh_config_file", 2024-05-08T22:18:08.180 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.180 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.180 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.180 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.180 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.180 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.180 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.180 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T22:18:08.181 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.181 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.181 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.181 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.181 INFO:teuthology.orchestra.run.smithi131.stdout: "use_agent": { 2024-05-08T22:18:08.181 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "use_agent", 2024-05-08T22:18:08.181 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.181 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.181 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.181 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.181 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.181 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.181 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.181 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T22:18:08.182 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.182 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.182 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.182 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.182 INFO:teuthology.orchestra.run.smithi131.stdout: "use_repo_digest": { 2024-05-08T22:18:08.182 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "use_repo_digest", 2024-05-08T22:18:08.182 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.182 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.182 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.182 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.182 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.182 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.182 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.182 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T22:18:08.183 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.183 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.183 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.183 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.183 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_failed_host_check": { 2024-05-08T22:18:08.183 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_failed_host_check", 2024-05-08T22:18:08.183 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.183 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.183 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.183 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.183 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.183 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.183 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.183 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T22:18:08.184 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.184 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.184 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.184 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.184 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_stray_daemons": { 2024-05-08T22:18:08.184 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_stray_daemons", 2024-05-08T22:18:08.184 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.184 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.184 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.184 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.184 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.184 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.184 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.184 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T22:18:08.184 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.185 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.185 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.185 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.185 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_stray_hosts": { 2024-05-08T22:18:08.185 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_stray_hosts", 2024-05-08T22:18:08.185 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.185 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.185 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.185 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.185 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.185 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.185 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.185 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T22:18:08.185 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.186 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.186 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.186 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.186 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.186 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.186 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.186 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crash", 2024-05-08T22:18:08.186 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.186 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.186 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.186 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.186 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.186 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.186 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.186 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.187 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.187 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.187 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.187 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.187 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.187 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.187 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.187 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.187 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.187 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.187 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.187 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.187 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.187 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.187 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.187 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.188 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.188 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.188 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.188 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.188 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.188 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.188 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.188 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.188 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.188 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.188 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.188 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.188 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.188 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.188 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.188 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.189 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.189 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.189 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.189 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.189 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.189 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.189 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.189 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.189 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.189 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.189 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.189 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.189 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.189 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.189 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.189 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.190 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.190 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.190 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.190 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.190 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.190 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.190 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.190 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.190 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.190 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.190 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.190 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.190 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.190 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.190 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.191 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.191 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.191 INFO:teuthology.orchestra.run.smithi131.stdout: "retain_interval": { 2024-05-08T22:18:08.191 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "retain_interval", 2024-05-08T22:18:08.191 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:08.191 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.191 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.191 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "31536000", 2024-05-08T22:18:08.191 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.191 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.191 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.191 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T22:18:08.191 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.191 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.191 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.191 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.192 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_recent_interval": { 2024-05-08T22:18:08.192 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_recent_interval", 2024-05-08T22:18:08.192 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:08.192 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.192 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.192 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1209600", 2024-05-08T22:18:08.192 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.192 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.192 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.192 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T22:18:08.192 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.192 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.192 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.192 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.192 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.192 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.192 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.193 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "dashboard", 2024-05-08T22:18:08.193 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.193 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.193 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.193 INFO:teuthology.orchestra.run.smithi131.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T22:18:08.193 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T22:18:08.193 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.193 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.193 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.193 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-08T22:18:08.193 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.193 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.193 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.193 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.193 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.194 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.194 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.194 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.194 INFO:teuthology.orchestra.run.smithi131.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T22:18:08.194 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T22:18:08.194 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.194 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.194 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.194 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.194 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.194 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.194 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.194 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.194 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.194 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.194 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.195 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.195 INFO:teuthology.orchestra.run.smithi131.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T22:18:08.195 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T22:18:08.195 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.195 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.195 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.195 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.195 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.195 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.195 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.195 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.195 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.195 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.195 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.195 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.196 INFO:teuthology.orchestra.run.smithi131.stdout: "AUDIT_API_ENABLED": { 2024-05-08T22:18:08.196 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T22:18:08.196 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.196 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.196 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.196 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.196 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.196 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.196 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.196 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.196 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.196 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.196 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.196 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.197 INFO:teuthology.orchestra.run.smithi131.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T22:18:08.197 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T22:18:08.197 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.197 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.197 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.197 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.197 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.197 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.197 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.197 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.197 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.197 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.197 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.197 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.198 INFO:teuthology.orchestra.run.smithi131.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T22:18:08.198 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T22:18:08.198 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.198 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.198 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.198 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.198 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.198 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.198 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.198 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.198 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.198 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.198 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.198 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.199 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T22:18:08.199 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T22:18:08.199 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.199 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.199 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.199 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.199 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.199 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.199 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.199 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.199 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.199 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.199 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.199 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.199 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T22:18:08.200 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T22:18:08.200 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.200 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.200 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.200 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.200 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.200 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.200 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.200 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.200 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.200 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.200 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.200 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.200 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T22:18:08.200 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T22:18:08.201 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.201 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.201 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.201 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.201 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.201 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.201 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.201 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.201 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.201 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.201 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.201 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.201 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T22:18:08.201 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T22:18:08.201 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.202 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.202 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.202 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.202 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.202 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.202 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.202 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.202 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.202 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.202 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.202 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.202 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T22:18:08.202 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T22:18:08.202 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.202 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.203 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.203 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.203 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.203 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.203 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.203 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.203 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.203 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.203 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.203 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.203 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T22:18:08.203 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T22:18:08.203 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.203 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.203 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.204 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.204 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.204 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.204 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.204 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.204 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.204 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.204 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.204 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.204 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T22:18:08.204 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T22:18:08.204 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.204 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.204 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.205 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.205 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.205 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.205 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.205 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.205 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.205 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.205 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.205 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.205 INFO:teuthology.orchestra.run.smithi131.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T22:18:08.205 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T22:18:08.205 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.205 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.205 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.206 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.206 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.206 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.206 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.206 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.206 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.206 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.206 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.206 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.206 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T22:18:08.206 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T22:18:08.206 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.206 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.207 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.207 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-05-08T22:18:08.207 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.207 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.207 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.207 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.207 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.207 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.207 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.207 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.207 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T22:18:08.207 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T22:18:08.207 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.207 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.207 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.208 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.208 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.208 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.208 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.208 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.208 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.208 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.208 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.208 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.208 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_URL": { 2024-05-08T22:18:08.208 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_URL", 2024-05-08T22:18:08.208 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.208 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.208 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.209 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.209 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.209 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.209 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.209 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.209 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.209 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.209 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.209 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.209 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T22:18:08.209 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T22:18:08.209 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.209 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.209 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.210 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-05-08T22:18:08.210 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.210 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.210 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.210 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.210 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.210 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.210 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.210 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.210 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T22:18:08.210 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T22:18:08.210 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.210 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.210 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.211 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.211 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.211 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.211 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.211 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.211 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.211 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.211 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.211 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.211 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T22:18:08.211 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T22:18:08.211 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.211 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.211 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.211 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.212 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.212 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.212 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.212 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.212 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.212 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.212 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.212 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.212 INFO:teuthology.orchestra.run.smithi131.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T22:18:08.212 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T22:18:08.212 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.212 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.212 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.212 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.213 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.213 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.213 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.213 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.213 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.213 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.213 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.213 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.213 INFO:teuthology.orchestra.run.smithi131.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T22:18:08.213 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T22:18:08.213 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.213 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.213 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.213 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.213 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.214 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.214 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.214 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.214 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.214 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.214 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.214 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.214 INFO:teuthology.orchestra.run.smithi131.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T22:18:08.214 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T22:18:08.214 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.214 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.214 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.214 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.214 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.215 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.215 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.215 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.215 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.215 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.215 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.215 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.215 INFO:teuthology.orchestra.run.smithi131.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T22:18:08.215 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T22:18:08.215 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.215 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.215 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.215 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.215 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.215 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.216 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.216 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.216 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.216 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.216 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.216 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.216 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T22:18:08.216 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T22:18:08.216 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.216 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.216 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.216 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.216 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.216 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.216 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.217 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.217 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.217 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.217 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.217 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.217 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T22:18:08.217 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T22:18:08.217 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.217 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.217 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.217 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.217 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.217 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.217 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.217 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.218 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.218 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.218 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.218 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.218 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T22:18:08.218 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T22:18:08.218 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.218 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.218 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.218 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.218 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.218 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.218 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.218 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.219 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.219 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.219 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.219 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.219 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T22:18:08.219 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T22:18:08.219 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.219 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.219 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.219 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.219 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.219 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.219 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.219 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.220 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.220 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.220 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.220 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.220 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T22:18:08.220 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T22:18:08.220 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.220 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.220 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.220 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.220 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.220 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.220 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.220 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.220 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.220 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.221 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.221 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.221 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T22:18:08.221 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T22:18:08.221 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.221 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.221 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.221 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.221 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.221 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.221 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.221 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.221 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.221 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.221 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.221 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.221 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T22:18:08.222 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T22:18:08.222 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.222 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.222 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.222 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.222 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.222 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.222 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.222 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.222 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.222 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.222 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.222 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.222 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T22:18:08.223 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T22:18:08.223 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.223 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.223 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.223 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.223 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.223 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.223 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.223 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.223 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.223 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.223 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.223 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.223 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T22:18:08.223 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T22:18:08.223 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.224 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.224 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.224 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T22:18:08.224 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.224 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.224 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.224 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.224 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.224 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.224 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.224 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.224 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T22:18:08.224 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T22:18:08.224 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.224 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.224 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.224 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-08T22:18:08.225 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.225 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.225 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.225 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.225 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.225 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.225 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.225 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.225 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T22:18:08.225 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T22:18:08.225 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.225 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.225 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.225 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8", 2024-05-08T22:18:08.225 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.225 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.225 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.226 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.226 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.226 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.226 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.226 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.226 INFO:teuthology.orchestra.run.smithi131.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T22:18:08.226 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T22:18:08.226 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.226 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.226 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.226 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "45", 2024-05-08T22:18:08.226 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.226 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.226 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.226 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.227 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.227 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.227 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.227 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.227 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T22:18:08.227 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T22:18:08.227 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.227 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.227 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.227 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.227 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.227 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.227 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.227 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.227 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.227 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.228 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.228 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.228 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T22:18:08.228 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T22:18:08.228 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.228 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.228 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.228 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-05-08T22:18:08.228 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.228 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.228 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.228 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.228 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.228 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.228 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.228 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.229 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T22:18:08.229 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T22:18:08.229 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.229 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.229 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.229 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.229 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.229 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.229 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.229 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.229 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.229 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.229 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.229 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.229 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T22:18:08.230 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T22:18:08.230 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.230 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.230 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.230 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.230 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.230 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.230 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.230 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.230 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.230 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.230 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.230 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.230 INFO:teuthology.orchestra.run.smithi131.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T22:18:08.230 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T22:18:08.231 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.231 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.231 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.231 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.231 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.231 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.231 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.231 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.231 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.231 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.231 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.231 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.231 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T22:18:08.231 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T22:18:08.231 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.232 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.232 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.232 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-08T22:18:08.232 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.232 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.232 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.232 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.232 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.232 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.232 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.232 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.232 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T22:18:08.232 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T22:18:08.232 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.232 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.233 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.233 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-08T22:18:08.233 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.233 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.233 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.233 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.233 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.233 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.233 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.233 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.233 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T22:18:08.233 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T22:18:08.233 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.233 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.233 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.234 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-05-08T22:18:08.234 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.234 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.234 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.234 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.234 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.234 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.234 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.234 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.234 INFO:teuthology.orchestra.run.smithi131.stdout: "cross_origin_url": { 2024-05-08T22:18:08.234 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cross_origin_url", 2024-05-08T22:18:08.234 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.234 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.234 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.235 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.235 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.235 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.235 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.235 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.235 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.235 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.235 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.235 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.235 INFO:teuthology.orchestra.run.smithi131.stdout: "crt_file": { 2024-05-08T22:18:08.235 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crt_file", 2024-05-08T22:18:08.235 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.235 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.235 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.235 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.235 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.236 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.236 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.236 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.236 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.236 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.236 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.236 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.236 INFO:teuthology.orchestra.run.smithi131.stdout: "debug": { 2024-05-08T22:18:08.236 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "debug", 2024-05-08T22:18:08.236 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.236 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.236 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.236 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.236 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.236 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.236 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.237 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable/disable debug options", 2024-05-08T22:18:08.237 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.237 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.237 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.237 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.237 INFO:teuthology.orchestra.run.smithi131.stdout: "jwt_token_ttl": { 2024-05-08T22:18:08.237 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "jwt_token_ttl", 2024-05-08T22:18:08.237 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.237 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.237 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.237 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "28800", 2024-05-08T22:18:08.237 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.237 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.237 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.237 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.237 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.238 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.238 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.238 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.238 INFO:teuthology.orchestra.run.smithi131.stdout: "key_file": { 2024-05-08T22:18:08.238 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "key_file", 2024-05-08T22:18:08.238 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.238 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.238 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.238 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.238 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.238 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.238 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.238 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.238 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.239 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.239 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.239 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.239 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.239 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.239 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.239 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.239 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.239 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.239 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.239 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.239 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.239 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.239 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.239 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.239 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.239 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.240 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.240 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.240 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.240 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.240 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.240 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.240 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.240 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.240 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.240 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.240 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.240 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.240 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.240 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.240 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.241 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.241 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.241 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.241 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.241 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.241 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.241 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.241 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.241 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.241 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.241 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.241 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.241 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.241 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.241 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.241 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.241 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.242 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.242 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.242 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.242 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.242 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.242 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.242 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.242 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.242 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.242 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.242 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.242 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.242 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.242 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.242 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.243 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.243 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.243 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.243 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.243 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.243 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.243 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.243 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.243 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.243 INFO:teuthology.orchestra.run.smithi131.stdout: "motd": { 2024-05-08T22:18:08.243 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "motd", 2024-05-08T22:18:08.243 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.243 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.243 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.243 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.243 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.244 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.244 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.244 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "The message of the day", 2024-05-08T22:18:08.244 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.244 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.244 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.244 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.244 INFO:teuthology.orchestra.run.smithi131.stdout: "redirect_resolve_ip_addr": { 2024-05-08T22:18:08.244 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T22:18:08.244 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.244 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.244 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.244 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.244 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.244 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.244 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.245 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.245 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.245 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.245 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.245 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.245 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-05-08T22:18:08.245 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-05-08T22:18:08.245 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.245 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.245 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.245 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "::", 2024-05-08T22:18:08.245 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.245 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.245 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.245 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.245 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.245 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.246 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.246 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.246 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-05-08T22:18:08.246 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-05-08T22:18:08.246 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.246 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.246 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.246 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8080", 2024-05-08T22:18:08.246 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.246 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.246 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.246 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.246 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.246 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.246 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.246 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.247 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl": { 2024-05-08T22:18:08.247 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl", 2024-05-08T22:18:08.247 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.247 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.247 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.247 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.247 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.247 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.247 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.247 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.247 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.247 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.247 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.247 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.248 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl_server_port": { 2024-05-08T22:18:08.248 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl_server_port", 2024-05-08T22:18:08.248 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.248 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.248 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.248 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8443", 2024-05-08T22:18:08.248 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.248 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.248 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.248 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.248 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.248 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.248 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.248 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.248 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_behaviour": { 2024-05-08T22:18:08.248 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_behaviour", 2024-05-08T22:18:08.249 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.249 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.249 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.249 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "redirect", 2024-05-08T22:18:08.249 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.249 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.249 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.249 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.249 INFO:teuthology.orchestra.run.smithi131.stdout: "redirect" 2024-05-08T22:18:08.249 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.249 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.249 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.249 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.249 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.249 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.250 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_error_status_code": { 2024-05-08T22:18:08.250 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_error_status_code", 2024-05-08T22:18:08.250 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.250 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.250 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.250 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "500", 2024-05-08T22:18:08.250 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "400", 2024-05-08T22:18:08.250 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "599", 2024-05-08T22:18:08.250 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.250 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.250 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.250 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.250 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.250 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.251 INFO:teuthology.orchestra.run.smithi131.stdout: "url_prefix": { 2024-05-08T22:18:08.251 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "url_prefix", 2024-05-08T22:18:08.251 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.251 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.251 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.251 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.251 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.251 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.251 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.251 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.251 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.251 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.251 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.251 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.251 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.252 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.252 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.252 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "devicehealth", 2024-05-08T22:18:08.252 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.252 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.252 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.252 INFO:teuthology.orchestra.run.smithi131.stdout: "enable_monitoring": { 2024-05-08T22:18:08.252 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enable_monitoring", 2024-05-08T22:18:08.252 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.252 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.252 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.252 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.252 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.252 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.252 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.252 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "monitor device health metrics", 2024-05-08T22:18:08.253 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.253 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.253 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.253 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.253 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.253 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.253 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.253 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.253 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.253 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.253 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.253 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.253 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.253 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.253 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.254 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.254 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.254 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.254 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.254 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.254 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.254 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.254 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.254 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.254 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.254 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.254 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.254 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.254 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.255 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.255 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.255 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.255 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.255 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.255 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.255 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.255 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.255 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.255 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.255 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.255 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.255 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.255 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.255 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.256 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.256 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.256 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.256 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.256 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.256 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.256 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.256 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.256 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.256 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.256 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.256 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.256 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.256 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.256 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.256 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.256 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.257 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.257 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.257 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.257 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.257 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.257 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.257 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.257 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.257 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.257 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.257 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.257 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.257 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.257 INFO:teuthology.orchestra.run.smithi131.stdout: "mark_out_threshold": { 2024-05-08T22:18:08.257 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mark_out_threshold", 2024-05-08T22:18:08.257 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:08.258 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.258 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.258 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "2419200", 2024-05-08T22:18:08.258 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.258 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.258 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.258 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T22:18:08.258 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.258 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.258 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.258 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.258 INFO:teuthology.orchestra.run.smithi131.stdout: "pool_name": { 2024-05-08T22:18:08.258 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pool_name", 2024-05-08T22:18:08.258 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.259 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.259 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.259 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "device_health_metrics", 2024-05-08T22:18:08.259 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.259 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.259 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.259 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T22:18:08.259 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.259 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.259 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.259 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.259 INFO:teuthology.orchestra.run.smithi131.stdout: "retention_period": { 2024-05-08T22:18:08.259 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "retention_period", 2024-05-08T22:18:08.259 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:08.260 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.260 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.260 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15552000", 2024-05-08T22:18:08.260 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.260 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.260 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.260 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long to retain device health metrics", 2024-05-08T22:18:08.260 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.260 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.260 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.260 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.260 INFO:teuthology.orchestra.run.smithi131.stdout: "scrape_frequency": { 2024-05-08T22:18:08.260 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "scrape_frequency", 2024-05-08T22:18:08.260 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:08.260 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.260 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.260 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "86400", 2024-05-08T22:18:08.261 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.261 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.261 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.261 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T22:18:08.261 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.261 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.261 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.261 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.261 INFO:teuthology.orchestra.run.smithi131.stdout: "self_heal": { 2024-05-08T22:18:08.261 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "self_heal", 2024-05-08T22:18:08.261 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.261 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.261 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.261 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.261 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.261 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.262 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.262 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T22:18:08.262 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.262 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.262 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.262 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.262 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-08T22:18:08.262 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-08T22:18:08.262 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:08.262 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.262 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.262 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-08T22:18:08.262 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.262 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.263 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.263 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T22:18:08.263 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.263 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.263 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.263 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.263 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_threshold": { 2024-05-08T22:18:08.263 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_threshold", 2024-05-08T22:18:08.263 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:08.263 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.263 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.263 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "7257600", 2024-05-08T22:18:08.263 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.263 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.263 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.263 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T22:18:08.264 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.264 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.264 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.264 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.264 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.264 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.264 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.264 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "diskprediction_local", 2024-05-08T22:18:08.264 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.264 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.264 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.264 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.264 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.264 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.264 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.265 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.265 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.265 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.265 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.265 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.265 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.265 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.265 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.265 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.265 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.265 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.265 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.265 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.265 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.266 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.266 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.266 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.266 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.266 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.266 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.266 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.266 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.266 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.266 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.266 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.266 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.266 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.266 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.266 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.266 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.267 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.267 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.267 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.267 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.267 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.267 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.267 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.267 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.267 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.267 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.267 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.267 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.267 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.267 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.267 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.268 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.268 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.268 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.268 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.268 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.268 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.268 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.268 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.268 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.268 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.268 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.268 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.268 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.268 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.268 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.269 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.269 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.269 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.269 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.269 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.269 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.269 INFO:teuthology.orchestra.run.smithi131.stdout: "predict_interval": { 2024-05-08T22:18:08.269 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "predict_interval", 2024-05-08T22:18:08.269 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.269 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.269 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.269 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "86400", 2024-05-08T22:18:08.269 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.269 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.269 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.270 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.270 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.270 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.270 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.270 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.270 INFO:teuthology.orchestra.run.smithi131.stdout: "predictor_model": { 2024-05-08T22:18:08.270 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "predictor_model", 2024-05-08T22:18:08.270 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.270 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.270 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.270 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "prophetstor", 2024-05-08T22:18:08.270 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.270 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.270 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.271 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.271 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.271 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.271 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.271 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.271 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-08T22:18:08.271 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-08T22:18:08.271 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.271 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.271 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.271 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-08T22:18:08.271 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.271 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.271 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.272 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.272 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.272 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.272 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.272 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.272 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.272 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.272 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.272 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "influx", 2024-05-08T22:18:08.272 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": false, 2024-05-08T22:18:08.272 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "influxdb python module not found", 2024-05-08T22:18:08.272 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.272 INFO:teuthology.orchestra.run.smithi131.stdout: "batch_size": { 2024-05-08T22:18:08.272 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "batch_size", 2024-05-08T22:18:08.272 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.272 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.273 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.273 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5000", 2024-05-08T22:18:08.273 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.273 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.273 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.273 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T22:18:08.273 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.273 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.273 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.273 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.273 INFO:teuthology.orchestra.run.smithi131.stdout: "database": { 2024-05-08T22:18:08.273 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "database", 2024-05-08T22:18:08.273 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.273 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.273 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.274 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "ceph", 2024-05-08T22:18:08.274 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.274 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.274 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.274 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T22:18:08.274 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.274 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.274 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.274 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.274 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": { 2024-05-08T22:18:08.274 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "hostname", 2024-05-08T22:18:08.274 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.274 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.274 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.275 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.275 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.275 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.275 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.275 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB server hostname", 2024-05-08T22:18:08.275 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.275 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.275 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.275 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.275 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-08T22:18:08.275 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-08T22:18:08.275 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:08.275 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.275 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.276 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "30", 2024-05-08T22:18:08.276 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "5", 2024-05-08T22:18:08.276 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.276 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.276 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T22:18:08.276 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.276 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.276 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.276 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.276 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.276 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.276 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.276 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.276 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.276 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.276 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.276 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.277 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.277 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.277 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.277 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.277 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.277 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.277 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.277 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.277 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.277 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.277 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.277 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.277 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.277 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.277 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.277 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.278 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.278 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.278 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.278 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.278 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.278 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.278 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.278 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.278 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.278 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.278 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.278 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.278 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.278 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.278 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.278 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.279 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.279 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.279 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.279 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.279 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.279 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.279 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.279 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.279 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.279 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.279 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.279 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.279 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.279 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.279 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.280 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.280 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.280 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.280 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.280 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.280 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.280 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.280 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.280 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.280 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.280 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.280 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.280 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.280 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.280 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.281 INFO:teuthology.orchestra.run.smithi131.stdout: "password": { 2024-05-08T22:18:08.281 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "password", 2024-05-08T22:18:08.281 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.281 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.281 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.281 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.281 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.281 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.281 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.281 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "password of InfluxDB server user", 2024-05-08T22:18:08.281 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.281 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.281 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.282 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.282 INFO:teuthology.orchestra.run.smithi131.stdout: "port": { 2024-05-08T22:18:08.282 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "port", 2024-05-08T22:18:08.282 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.282 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.282 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.282 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8086", 2024-05-08T22:18:08.282 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.282 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.282 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.282 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB server port", 2024-05-08T22:18:08.282 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.282 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.282 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.283 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.283 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl": { 2024-05-08T22:18:08.283 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl", 2024-05-08T22:18:08.283 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.283 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.283 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.283 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "false", 2024-05-08T22:18:08.283 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.283 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.283 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.283 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T22:18:08.283 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.283 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.283 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.283 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.283 INFO:teuthology.orchestra.run.smithi131.stdout: "threads": { 2024-05-08T22:18:08.284 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "threads", 2024-05-08T22:18:08.284 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.284 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.284 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.284 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-05-08T22:18:08.284 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-05-08T22:18:08.284 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "32", 2024-05-08T22:18:08.284 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.284 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T22:18:08.284 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.284 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.284 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.284 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.284 INFO:teuthology.orchestra.run.smithi131.stdout: "username": { 2024-05-08T22:18:08.285 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "username", 2024-05-08T22:18:08.285 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.285 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.285 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.285 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.285 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.285 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.285 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.285 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "username of InfluxDB server user", 2024-05-08T22:18:08.285 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.285 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.285 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.285 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.285 INFO:teuthology.orchestra.run.smithi131.stdout: "verify_ssl": { 2024-05-08T22:18:08.285 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "verify_ssl", 2024-05-08T22:18:08.285 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.286 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.286 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.286 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "true", 2024-05-08T22:18:08.286 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.286 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.286 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.286 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T22:18:08.286 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.286 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.286 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.286 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.286 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.286 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.286 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.286 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "insights", 2024-05-08T22:18:08.287 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.287 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.287 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.287 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.287 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.287 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.287 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.287 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.287 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.287 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.287 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.287 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.287 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.287 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.287 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.287 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.288 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.288 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.288 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.288 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.288 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.288 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.288 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.288 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.288 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.288 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.288 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.288 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.288 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.288 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.288 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.288 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.288 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.289 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.289 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.289 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.289 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.289 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.289 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.289 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.289 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.289 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.289 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.289 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.289 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.289 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.289 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.289 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.290 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.290 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.290 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.290 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.290 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.290 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.290 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.290 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.290 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.290 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.290 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.290 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.290 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.290 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.290 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.290 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.291 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.291 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.291 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.291 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.291 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.291 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.291 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.291 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.291 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.291 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.291 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.291 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.291 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "iostat", 2024-05-08T22:18:08.291 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.291 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.291 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.292 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.292 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.292 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.292 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.292 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.292 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.292 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.292 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.292 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.292 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.292 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.292 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.292 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.292 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.292 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.292 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.293 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.293 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.293 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.293 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.293 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.293 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.293 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.293 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.293 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.293 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.293 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.293 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.293 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.293 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.293 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.293 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.294 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.294 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.294 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.294 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.294 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.294 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.294 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.294 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.294 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.294 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.294 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.294 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.294 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.294 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.294 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.294 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.295 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.295 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.295 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.295 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.295 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.295 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.295 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.295 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.295 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.295 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.295 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.295 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.295 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.295 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.295 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.295 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.295 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.296 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.296 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.296 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.296 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.296 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.296 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.296 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.296 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.296 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "k8sevents", 2024-05-08T22:18:08.296 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.296 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.296 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.296 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_event_retention_days": { 2024-05-08T22:18:08.296 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ceph_event_retention_days", 2024-05-08T22:18:08.296 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.297 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.297 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.297 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "7", 2024-05-08T22:18:08.297 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.297 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.297 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.297 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T22:18:08.297 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.297 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.297 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.297 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.297 INFO:teuthology.orchestra.run.smithi131.stdout: "config_check_secs": { 2024-05-08T22:18:08.297 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_check_secs", 2024-05-08T22:18:08.297 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.297 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.297 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.298 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-08T22:18:08.298 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "10", 2024-05-08T22:18:08.298 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.298 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.298 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T22:18:08.298 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.298 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.298 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.298 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.298 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.298 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.298 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.298 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.298 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.299 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.299 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.299 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.299 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.299 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.299 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.299 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.299 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.299 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.299 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.299 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.299 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.299 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.300 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.300 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.300 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.300 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.300 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.300 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.300 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.300 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.300 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.300 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.300 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.300 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.300 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.300 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.300 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.301 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.301 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.301 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.301 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.301 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.301 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.301 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.301 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.301 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.301 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.301 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.301 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.301 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.301 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.301 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.302 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.302 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.302 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.302 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.302 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.302 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.302 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.302 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.302 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.302 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.302 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.302 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.302 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.302 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.302 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.303 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.303 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.303 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.303 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.303 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.303 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.303 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.303 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.303 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.303 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.303 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "localpool", 2024-05-08T22:18:08.303 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.303 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.303 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.303 INFO:teuthology.orchestra.run.smithi131.stdout: "failure_domain": { 2024-05-08T22:18:08.304 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "failure_domain", 2024-05-08T22:18:08.304 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.304 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.304 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.304 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "host", 2024-05-08T22:18:08.304 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.304 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.304 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.304 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "failure domain for any created local pool", 2024-05-08T22:18:08.304 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T22:18:08.304 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.304 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.304 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.304 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.304 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.304 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.305 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.305 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.305 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.305 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.305 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.305 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.305 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.305 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.305 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.305 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.305 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.305 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.305 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.305 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.305 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.305 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.306 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.306 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.306 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.306 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.306 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.306 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.306 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.306 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.306 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.306 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.306 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.306 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.306 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.306 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.307 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.307 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.307 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.307 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.307 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.307 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.307 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.307 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.307 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.307 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.307 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.307 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.307 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.307 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.307 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.307 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.307 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.308 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.308 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.308 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.308 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.308 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.308 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.308 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.308 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.308 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.308 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.308 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.308 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.308 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.308 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.308 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.309 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.309 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.309 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.309 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.309 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.309 INFO:teuthology.orchestra.run.smithi131.stdout: "min_size": { 2024-05-08T22:18:08.309 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "min_size", 2024-05-08T22:18:08.309 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.309 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.309 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.309 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.309 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.309 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.309 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.309 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default min_size for any created local pool", 2024-05-08T22:18:08.310 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T22:18:08.310 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.310 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.310 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.310 INFO:teuthology.orchestra.run.smithi131.stdout: "num_rep": { 2024-05-08T22:18:08.310 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "num_rep", 2024-05-08T22:18:08.310 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.310 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.310 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.310 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3", 2024-05-08T22:18:08.310 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.310 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.310 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.310 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default replica count for any created local pool", 2024-05-08T22:18:08.311 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.311 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.311 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.311 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.311 INFO:teuthology.orchestra.run.smithi131.stdout: "pg_num": { 2024-05-08T22:18:08.311 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pg_num", 2024-05-08T22:18:08.311 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.311 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.311 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.311 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "128", 2024-05-08T22:18:08.311 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.311 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.311 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.311 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T22:18:08.311 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.311 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.312 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.312 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.312 INFO:teuthology.orchestra.run.smithi131.stdout: "prefix": { 2024-05-08T22:18:08.312 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prefix", 2024-05-08T22:18:08.312 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.312 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.312 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.312 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.312 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.312 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.312 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.312 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "name prefix for any created local pool", 2024-05-08T22:18:08.312 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.312 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.312 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.313 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.313 INFO:teuthology.orchestra.run.smithi131.stdout: "subtree": { 2024-05-08T22:18:08.313 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "subtree", 2024-05-08T22:18:08.313 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.313 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.313 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.313 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "rack", 2024-05-08T22:18:08.313 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.313 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.313 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.313 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T22:18:08.313 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T22:18:08.313 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.313 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.313 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.314 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.314 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.314 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.314 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mds_autoscaler", 2024-05-08T22:18:08.314 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.314 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.314 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.314 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.314 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.314 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.314 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.314 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.314 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.314 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.315 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.315 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.315 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.315 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.315 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.315 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.315 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.315 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.315 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.315 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.315 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.315 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.315 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.315 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.315 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.315 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.316 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.316 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.316 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.316 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.316 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.316 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.316 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.316 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.316 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.316 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.316 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.316 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.316 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.316 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.316 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.316 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.317 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.317 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.317 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.317 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.317 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.317 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.317 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.317 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.317 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.317 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.317 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.317 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.317 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.317 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.317 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.317 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.318 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.318 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.318 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.318 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.318 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.318 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.318 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.318 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.318 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.318 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.318 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.318 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.318 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.318 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.318 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.319 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.319 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.319 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.319 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mirroring", 2024-05-08T22:18:08.319 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.319 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.319 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.319 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.319 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.319 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.319 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.319 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.319 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.319 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.319 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.319 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.319 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.320 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.320 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.320 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.320 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.320 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.320 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.320 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.320 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.320 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.320 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.320 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.320 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.320 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.320 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.321 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.321 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.321 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.321 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.321 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.321 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.321 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.321 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.321 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.321 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.321 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.321 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.321 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.321 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.321 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.321 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.322 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.322 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.322 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.322 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.322 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.322 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.322 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.322 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.322 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.322 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.322 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.322 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.322 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.322 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.322 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.322 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.322 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.323 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.323 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.323 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.323 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.323 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.323 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.323 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.323 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.323 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.323 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.323 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.323 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.323 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.323 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.323 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.323 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.324 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "nfs", 2024-05-08T22:18:08.324 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.324 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.324 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.324 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.324 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.324 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.324 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.324 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.324 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.324 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.324 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.324 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.324 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.324 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.324 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.324 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.325 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.325 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.325 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.325 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.325 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.325 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.325 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.325 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.325 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.325 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.325 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.325 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.325 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.325 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.325 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.325 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.325 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.326 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.326 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.326 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.326 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.326 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.326 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.326 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.326 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.326 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.326 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.326 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.326 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.326 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.326 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.327 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.327 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.327 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.327 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.327 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.327 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.327 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.327 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.327 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.327 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.327 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.327 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.327 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.327 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.327 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.327 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.328 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.328 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.328 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.328 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.328 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.328 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.328 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.328 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.328 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.328 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.328 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.328 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.328 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.328 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "orchestrator", 2024-05-08T22:18:08.328 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.329 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.329 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.329 INFO:teuthology.orchestra.run.smithi131.stdout: "fail_fs": { 2024-05-08T22:18:08.329 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "fail_fs", 2024-05-08T22:18:08.329 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.329 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.329 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.329 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.329 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.329 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.329 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.329 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T22:18:08.329 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.329 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.329 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.329 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.330 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.330 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.330 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.330 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.330 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.330 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.330 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.330 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.330 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.330 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.330 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.330 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.330 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.330 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.331 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.331 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.331 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.331 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.331 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.331 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.331 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.331 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.331 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.331 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.331 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.331 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.331 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.331 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.331 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.332 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.332 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.332 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.332 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.332 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.332 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.332 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.332 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.332 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.332 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.332 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.332 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.332 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.332 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.332 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.333 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.333 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.333 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.333 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.333 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.333 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.333 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.333 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.333 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.333 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.333 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.333 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.333 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.333 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.333 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.334 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.334 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.334 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.334 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.334 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.334 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.334 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.334 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.334 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.334 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.334 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.334 INFO:teuthology.orchestra.run.smithi131.stdout: "orchestrator": { 2024-05-08T22:18:08.334 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "orchestrator", 2024-05-08T22:18:08.334 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.334 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.334 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.335 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.335 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.335 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.335 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.335 INFO:teuthology.orchestra.run.smithi131.stdout: "cephadm", 2024-05-08T22:18:08.335 INFO:teuthology.orchestra.run.smithi131.stdout: "rook", 2024-05-08T22:18:08.335 INFO:teuthology.orchestra.run.smithi131.stdout: "test_orchestrator" 2024-05-08T22:18:08.335 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.335 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Orchestrator backend", 2024-05-08T22:18:08.335 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.335 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.335 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.335 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.335 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.335 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.335 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.336 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "osd_perf_query", 2024-05-08T22:18:08.336 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.336 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.336 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.336 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.336 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.336 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.336 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.336 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.336 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.336 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.336 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.336 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.336 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.336 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.336 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.336 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.337 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.337 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.337 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.337 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.337 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.337 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.337 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.337 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.337 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.337 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.337 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.337 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.337 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.337 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.337 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.337 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.338 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.338 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.338 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.338 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.338 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.338 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.338 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.338 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.338 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.338 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.338 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.338 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.338 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.338 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.338 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.338 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.339 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.339 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.339 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.339 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.339 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.339 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.339 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.339 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.339 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.339 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.339 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.339 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.339 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.339 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.339 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.339 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.339 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.340 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.340 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.340 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.340 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.340 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.340 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.340 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.340 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.340 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.340 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.340 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.340 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "osd_support", 2024-05-08T22:18:08.340 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.340 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.340 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.341 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.341 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.341 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.341 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.341 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.341 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.341 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.341 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.341 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.341 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.341 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.341 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.341 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.341 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.341 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.341 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.342 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.342 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.342 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.342 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.342 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.342 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.342 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.342 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.342 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.342 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.342 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.342 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.342 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.342 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.342 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.342 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.343 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.343 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.343 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.343 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.343 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.343 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.343 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.343 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.343 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.343 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.343 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.343 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.343 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.343 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.343 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.344 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.344 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.344 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.344 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.344 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.344 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.344 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.344 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.344 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.344 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.344 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.344 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.344 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.344 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.344 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.345 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.345 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.345 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.345 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.345 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.345 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.345 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.345 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.345 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.345 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.345 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.345 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pg_autoscaler", 2024-05-08T22:18:08.345 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.345 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.345 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.345 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.346 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.346 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.346 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.346 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.346 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.346 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.346 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.346 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.346 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.346 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.346 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.346 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.346 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.346 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.346 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.346 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.347 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.347 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.347 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.347 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.347 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.347 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.347 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.347 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.347 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.347 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.347 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.347 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.348 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.348 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.348 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.348 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.348 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.348 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.348 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.348 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.348 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.348 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.348 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.348 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.348 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.348 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.349 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.349 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.349 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.349 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.349 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.349 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.349 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.349 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.349 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.349 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.349 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.349 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.349 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.349 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.350 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.350 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.350 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.350 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.350 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.350 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.350 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.350 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.350 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.350 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.350 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.350 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.350 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.350 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-08T22:18:08.351 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-08T22:18:08.351 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:08.351 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.351 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.351 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-08T22:18:08.351 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.351 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.351 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.351 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.351 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.351 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.351 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.351 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.351 INFO:teuthology.orchestra.run.smithi131.stdout: "threshold": { 2024-05-08T22:18:08.352 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "threshold", 2024-05-08T22:18:08.352 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-08T22:18:08.352 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.352 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.352 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3.0", 2024-05-08T22:18:08.352 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1.0", 2024-05-08T22:18:08.352 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.352 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.352 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "scaling threshold", 2024-05-08T22:18:08.352 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T22:18:08.352 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.352 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.352 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.352 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.352 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.353 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.353 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "progress", 2024-05-08T22:18:08.353 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.353 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.353 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.353 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_pg_recovery_event": { 2024-05-08T22:18:08.353 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_pg_recovery_event", 2024-05-08T22:18:08.353 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.353 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.353 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.353 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.353 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.353 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.353 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.353 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T22:18:08.353 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.354 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.354 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.354 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.354 INFO:teuthology.orchestra.run.smithi131.stdout: "enabled": { 2024-05-08T22:18:08.354 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enabled", 2024-05-08T22:18:08.354 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.354 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.354 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.354 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.354 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.354 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.354 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.354 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.354 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.355 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.355 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.355 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.355 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.355 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.355 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.355 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.355 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.355 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.355 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.355 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.355 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.355 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.355 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.355 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.356 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.356 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.356 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.356 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.356 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.356 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.356 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.356 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.356 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.356 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.356 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.356 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.356 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.356 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.356 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.356 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.357 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.357 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.357 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.357 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.357 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.357 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.357 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.357 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.357 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.357 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.357 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.357 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.357 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.357 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.357 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.357 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.358 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.358 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.358 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.358 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.358 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.358 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.358 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.358 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.358 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.358 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.358 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.358 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.358 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.358 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.359 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.359 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.359 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.359 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.359 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.359 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.359 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.359 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.359 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.359 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.359 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.359 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.359 INFO:teuthology.orchestra.run.smithi131.stdout: "max_completed_events": { 2024-05-08T22:18:08.359 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_completed_events", 2024-05-08T22:18:08.360 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.360 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.360 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.360 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "50", 2024-05-08T22:18:08.360 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.360 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.360 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.360 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "number of past completed events to remember", 2024-05-08T22:18:08.360 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.360 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.360 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.360 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.360 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-08T22:18:08.360 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-08T22:18:08.360 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:08.361 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.361 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.361 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-05-08T22:18:08.361 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.361 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.361 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.361 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long the module is going to sleep", 2024-05-08T22:18:08.361 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.361 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.361 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.361 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.361 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.361 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.361 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.361 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prometheus", 2024-05-08T22:18:08.362 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.362 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.362 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.362 INFO:teuthology.orchestra.run.smithi131.stdout: "cache": { 2024-05-08T22:18:08.362 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cache", 2024-05-08T22:18:08.362 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.362 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.362 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.362 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.362 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.362 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.362 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.362 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.363 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.363 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.363 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.363 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.363 INFO:teuthology.orchestra.run.smithi131.stdout: "exclude_perf_counters": { 2024-05-08T22:18:08.363 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "exclude_perf_counters", 2024-05-08T22:18:08.363 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.363 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.363 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.363 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.363 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.363 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.363 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.363 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T22:18:08.363 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T22:18:08.364 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.364 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.364 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.366 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.366 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.366 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.366 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.366 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.366 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.366 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.366 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.367 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.367 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.367 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.367 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.367 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.367 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.367 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.367 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.367 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.367 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.367 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.367 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.367 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.367 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.367 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.368 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.368 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.368 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.368 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.368 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.368 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.368 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.368 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.368 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.368 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.368 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.368 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.368 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.368 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.369 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.369 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.369 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.369 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.369 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.369 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.369 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.369 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.369 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.369 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.369 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.369 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.369 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.369 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.369 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.370 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.370 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.370 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.370 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.370 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.370 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.370 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.370 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.370 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.370 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.370 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.370 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.370 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.370 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.371 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.371 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.371 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.371 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.371 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_stats_pools": { 2024-05-08T22:18:08.371 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_stats_pools", 2024-05-08T22:18:08.371 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.371 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.371 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.371 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.371 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.371 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.371 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.371 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.371 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.372 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.372 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.372 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.372 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T22:18:08.372 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T22:18:08.372 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.372 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.372 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.372 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "300", 2024-05-08T22:18:08.372 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.372 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.372 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.372 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.372 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.372 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.373 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.373 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.373 INFO:teuthology.orchestra.run.smithi131.stdout: "scrape_interval": { 2024-05-08T22:18:08.373 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "scrape_interval", 2024-05-08T22:18:08.373 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-08T22:18:08.373 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.373 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.373 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15.0", 2024-05-08T22:18:08.373 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.373 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.373 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.373 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.373 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.373 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.373 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.374 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.374 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-05-08T22:18:08.374 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-05-08T22:18:08.374 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.374 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.374 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.374 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "::", 2024-05-08T22:18:08.374 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.374 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.374 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.374 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T22:18:08.374 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.374 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.374 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.375 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.375 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-05-08T22:18:08.375 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-05-08T22:18:08.375 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.375 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.375 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.375 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "9283", 2024-05-08T22:18:08.375 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.375 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.375 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.375 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T22:18:08.375 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.375 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.375 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.375 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.376 INFO:teuthology.orchestra.run.smithi131.stdout: "stale_cache_strategy": { 2024-05-08T22:18:08.376 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "stale_cache_strategy", 2024-05-08T22:18:08.376 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.376 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.376 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.376 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "log", 2024-05-08T22:18:08.376 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.376 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.376 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.376 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.376 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.376 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.376 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.376 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.377 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_behaviour": { 2024-05-08T22:18:08.377 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_behaviour", 2024-05-08T22:18:08.377 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.377 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.377 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.377 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "default", 2024-05-08T22:18:08.377 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.377 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.377 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.377 INFO:teuthology.orchestra.run.smithi131.stdout: "default", 2024-05-08T22:18:08.377 INFO:teuthology.orchestra.run.smithi131.stdout: "error" 2024-05-08T22:18:08.377 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.377 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.377 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.378 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.378 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.378 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.378 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_error_status_code": { 2024-05-08T22:18:08.378 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_error_status_code", 2024-05-08T22:18:08.378 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.378 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.378 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.378 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "500", 2024-05-08T22:18:08.378 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "400", 2024-05-08T22:18:08.378 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "599", 2024-05-08T22:18:08.378 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.378 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.379 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.379 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.379 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.379 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.379 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.379 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.379 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.379 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_support", 2024-05-08T22:18:08.379 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.379 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.379 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.379 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.379 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.379 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.380 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.380 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.380 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.380 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.380 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.380 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.380 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.380 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.380 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.380 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.380 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.380 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.380 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.380 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.380 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.381 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.381 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.381 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.381 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.381 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.381 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.381 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.381 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.381 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.381 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.381 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.381 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.381 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.381 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.382 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.382 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.382 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.382 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.382 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.382 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.382 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.382 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.382 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.382 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.382 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.382 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.382 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.382 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.383 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.383 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.383 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.383 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.383 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.383 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.383 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.383 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.383 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.383 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.383 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.383 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.383 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.383 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.383 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.384 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.384 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.384 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.384 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.384 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.384 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.384 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.384 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.384 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.384 INFO:teuthology.orchestra.run.smithi131.stdout: "max_concurrent_snap_create": { 2024-05-08T22:18:08.384 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_concurrent_snap_create", 2024-05-08T22:18:08.384 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.384 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.384 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.384 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-08T22:18:08.384 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.385 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.385 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.385 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.385 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.385 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.385 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.385 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.385 INFO:teuthology.orchestra.run.smithi131.stdout: "mirror_snapshot_schedule": { 2024-05-08T22:18:08.385 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T22:18:08.385 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.385 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.385 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.385 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.385 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.385 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.386 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.386 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.386 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.386 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.386 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.386 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.386 INFO:teuthology.orchestra.run.smithi131.stdout: "trash_purge_schedule": { 2024-05-08T22:18:08.386 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "trash_purge_schedule", 2024-05-08T22:18:08.386 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.386 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.386 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.386 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.386 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.386 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.387 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.387 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.387 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.387 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.387 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.387 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.387 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.387 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.387 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.387 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "restful", 2024-05-08T22:18:08.387 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.387 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.387 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.387 INFO:teuthology.orchestra.run.smithi131.stdout: "enable_auth": { 2024-05-08T22:18:08.387 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enable_auth", 2024-05-08T22:18:08.388 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.388 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.388 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.388 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.388 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.388 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.388 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.388 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.388 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.388 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.388 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.388 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.388 INFO:teuthology.orchestra.run.smithi131.stdout: "key_file": { 2024-05-08T22:18:08.388 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "key_file", 2024-05-08T22:18:08.388 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.388 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.389 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.389 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.389 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.389 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.389 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.389 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.389 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.389 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.389 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.389 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.389 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.389 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.389 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.389 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.389 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.389 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.389 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.390 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.390 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.390 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.390 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.390 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.390 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.390 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.390 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.390 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.390 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.390 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.390 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.390 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.390 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.390 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.391 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.391 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.391 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.391 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.391 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.391 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.391 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.391 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.391 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.391 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.391 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.391 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.391 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.391 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.391 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.391 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.392 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.392 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.392 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.392 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.392 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.392 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.392 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.392 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.392 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.392 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.392 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.392 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.392 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.392 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.392 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.392 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.393 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.393 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.393 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.393 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.393 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.393 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.393 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.393 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.393 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.393 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.393 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.393 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.393 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.393 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.393 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.393 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.394 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-05-08T22:18:08.394 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-05-08T22:18:08.394 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.394 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.394 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.394 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.394 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.394 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.394 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.394 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.394 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.394 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.394 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.394 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.395 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-05-08T22:18:08.395 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-05-08T22:18:08.395 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.395 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.395 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.395 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.395 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.395 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.395 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.395 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.395 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.395 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.395 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.395 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.395 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.396 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.396 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.396 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rgw", 2024-05-08T22:18:08.396 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.396 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.396 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.396 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.396 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.396 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.396 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.396 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.396 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.396 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.396 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.397 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.397 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.397 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.397 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.397 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.397 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.397 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.397 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.397 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.397 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.397 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.397 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.397 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.397 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.397 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.398 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.398 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.398 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.398 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.398 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.398 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.398 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.398 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.398 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.398 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.398 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.398 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.398 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.398 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.399 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.399 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.399 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.399 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.399 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.399 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.399 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.399 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.399 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.399 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.399 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.399 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.399 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.399 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.399 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.400 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.400 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.400 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.400 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.400 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.400 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.400 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.400 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.400 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.400 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.400 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.400 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.400 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.400 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.400 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.400 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.400 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.401 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.401 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.401 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.401 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.401 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rook", 2024-05-08T22:18:08.401 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.401 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.401 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.401 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.401 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.401 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.401 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.401 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.401 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.401 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.401 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.402 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.402 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.402 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.402 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.402 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.402 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.402 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.402 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.402 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.402 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.402 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.402 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.402 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.402 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.402 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.403 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.403 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.403 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.403 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.403 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.403 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.403 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.403 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.403 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.403 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.403 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.403 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.403 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.403 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.403 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.403 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.404 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.404 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.404 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.404 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.404 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.404 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.404 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.404 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.404 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.404 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.404 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.404 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.404 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.404 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.404 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.404 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.405 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.405 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.405 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.405 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.405 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.405 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.405 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.405 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.405 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.405 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.405 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.405 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.405 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.405 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.405 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.405 INFO:teuthology.orchestra.run.smithi131.stdout: "storage_class": { 2024-05-08T22:18:08.406 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "storage_class", 2024-05-08T22:18:08.406 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.406 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.406 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.406 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "local", 2024-05-08T22:18:08.406 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.406 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.406 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.406 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T22:18:08.406 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.406 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.406 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.406 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.406 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.407 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.407 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.407 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "selftest", 2024-05-08T22:18:08.407 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.407 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.407 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.407 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.407 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.407 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.407 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.407 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.407 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.407 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.407 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.407 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.408 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.408 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.408 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.408 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.408 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.408 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.408 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.408 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.408 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.408 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.408 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.408 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.408 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.408 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.409 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.409 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.409 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.409 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.409 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.409 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.409 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.409 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.409 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.409 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.409 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.409 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.409 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.410 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.410 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.410 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.410 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.410 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.410 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.410 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.410 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.410 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.410 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.410 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.410 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.410 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.411 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.411 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.411 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.411 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.411 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.411 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.411 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.411 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.411 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.411 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.411 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.411 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.411 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.411 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.411 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.412 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.412 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.412 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.412 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.412 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.412 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.412 INFO:teuthology.orchestra.run.smithi131.stdout: "roption1": { 2024-05-08T22:18:08.412 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "roption1", 2024-05-08T22:18:08.412 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.412 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.412 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.412 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.412 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.412 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.413 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.413 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.413 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.413 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.413 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.413 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.413 INFO:teuthology.orchestra.run.smithi131.stdout: "roption2": { 2024-05-08T22:18:08.413 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "roption2", 2024-05-08T22:18:08.413 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.413 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.413 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.413 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "xyz", 2024-05-08T22:18:08.413 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.413 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.414 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.414 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.414 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.414 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.414 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.414 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.414 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption1": { 2024-05-08T22:18:08.414 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption1", 2024-05-08T22:18:08.414 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.414 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.414 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.414 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.414 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.414 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.415 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.415 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.415 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.415 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.415 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.415 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.415 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption2": { 2024-05-08T22:18:08.415 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption2", 2024-05-08T22:18:08.415 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.415 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.415 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.415 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.415 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.415 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.415 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.416 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.416 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.416 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.416 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.416 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.416 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption3": { 2024-05-08T22:18:08.416 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption3", 2024-05-08T22:18:08.416 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-08T22:18:08.416 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.416 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.416 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.416 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.416 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.416 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.416 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.417 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.417 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.417 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.417 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.417 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption4": { 2024-05-08T22:18:08.417 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption4", 2024-05-08T22:18:08.417 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.417 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.417 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.417 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.417 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.417 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.417 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.417 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.417 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.418 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.418 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.418 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.418 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption5": { 2024-05-08T22:18:08.418 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption5", 2024-05-08T22:18:08.418 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.418 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.418 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.418 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.418 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.418 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.418 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.418 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.418 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.419 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.419 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.419 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.419 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption6": { 2024-05-08T22:18:08.419 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption6", 2024-05-08T22:18:08.419 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.419 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.419 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.419 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.419 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.419 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.419 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.419 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.419 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.419 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.419 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.420 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.420 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption7": { 2024-05-08T22:18:08.420 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption7", 2024-05-08T22:18:08.420 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.420 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.420 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.420 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.420 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-05-08T22:18:08.420 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "42", 2024-05-08T22:18:08.420 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.420 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.420 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.420 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.420 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.420 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.421 INFO:teuthology.orchestra.run.smithi131.stdout: "testkey": { 2024-05-08T22:18:08.421 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testkey", 2024-05-08T22:18:08.421 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.421 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.421 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.421 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.421 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.421 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.421 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.421 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.421 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.421 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.421 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.421 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.422 INFO:teuthology.orchestra.run.smithi131.stdout: "testlkey": { 2024-05-08T22:18:08.422 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testlkey", 2024-05-08T22:18:08.422 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.422 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.422 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.422 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.422 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.422 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.422 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.422 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.422 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.422 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.422 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.422 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.422 INFO:teuthology.orchestra.run.smithi131.stdout: "testnewline": { 2024-05-08T22:18:08.422 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testnewline", 2024-05-08T22:18:08.423 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.423 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.423 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.423 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.423 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.423 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.423 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.423 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.423 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.423 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.423 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.423 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.423 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.423 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.423 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.423 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snap_schedule", 2024-05-08T22:18:08.424 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.424 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.424 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.424 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_m_granularity": { 2024-05-08T22:18:08.424 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_m_granularity", 2024-05-08T22:18:08.424 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.424 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.424 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.424 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.424 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.424 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.424 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.424 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T22:18:08.424 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.424 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.424 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.424 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.425 INFO:teuthology.orchestra.run.smithi131.stdout: "dump_on_update": { 2024-05-08T22:18:08.425 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "dump_on_update", 2024-05-08T22:18:08.425 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.425 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.425 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.425 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.425 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.425 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.425 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.425 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "dump database to debug log on update", 2024-05-08T22:18:08.425 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.425 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.425 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.425 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.425 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.425 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.426 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.426 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.426 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.426 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.426 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.426 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.426 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.426 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.426 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.426 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.426 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.426 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.426 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.426 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.427 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.427 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.427 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.427 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.427 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.427 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.427 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.427 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.427 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.427 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.427 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.427 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.427 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.427 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.427 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.427 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.427 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.428 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.428 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.428 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.428 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.428 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.428 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.428 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.428 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.428 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.428 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.428 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.428 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.428 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.428 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.428 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.429 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.429 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.429 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.429 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.429 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.429 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.429 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.429 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.429 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.429 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.429 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.429 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.429 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.429 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.429 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.429 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.430 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.430 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.430 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.430 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.430 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.430 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.430 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.430 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.430 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.430 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "stats", 2024-05-08T22:18:08.430 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.430 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.430 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.430 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.430 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.431 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.431 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.431 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.431 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.431 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.431 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.431 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.431 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.431 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.431 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.431 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.431 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.431 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.431 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.431 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.431 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.432 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.432 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.432 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.432 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.432 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.432 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.432 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.432 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.432 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.432 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.432 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.432 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.432 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.432 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.432 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.432 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.433 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.433 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.433 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.433 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.433 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.433 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.433 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.433 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.433 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.433 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.433 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.433 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.433 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.433 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.433 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.434 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.434 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.434 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.434 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.434 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.434 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.434 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.434 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.434 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.434 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.434 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.434 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.434 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.434 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.434 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.435 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.435 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.435 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.435 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.435 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.435 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.435 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.435 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.435 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.435 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "status", 2024-05-08T22:18:08.435 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.435 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.435 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.435 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.435 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.436 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.436 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.436 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.436 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.436 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.436 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.436 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.436 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.436 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.436 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.436 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.436 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.436 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.436 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.436 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.437 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.437 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.437 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.437 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.437 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.437 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.437 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.437 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.437 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.437 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.437 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.437 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.437 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.437 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.437 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.438 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.438 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.438 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.438 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.438 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.438 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.438 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.438 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.438 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.438 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.438 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.438 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.438 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.438 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.439 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.439 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.439 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.439 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.439 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.439 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.439 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.439 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.439 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.439 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.439 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.439 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.439 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.439 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.439 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.440 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.440 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.440 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.440 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.440 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.440 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.440 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.440 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.440 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.440 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.440 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.440 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.440 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "telegraf", 2024-05-08T22:18:08.440 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.441 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.441 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.441 INFO:teuthology.orchestra.run.smithi131.stdout: "address": { 2024-05-08T22:18:08.441 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "address", 2024-05-08T22:18:08.441 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.441 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.441 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.441 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T22:18:08.441 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.441 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.441 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.441 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.441 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.441 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.441 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.442 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.442 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-08T22:18:08.442 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-08T22:18:08.442 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:08.442 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.442 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.442 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15", 2024-05-08T22:18:08.442 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.442 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.442 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.442 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.442 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.442 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.442 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.443 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.443 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.443 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.443 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.443 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.443 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.443 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.443 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.443 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.443 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.443 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.443 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.443 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.443 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.443 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.444 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.444 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.444 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.444 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.444 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.444 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.444 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.444 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.444 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.444 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.444 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.444 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.444 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.444 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.445 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.445 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.445 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.445 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.445 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.445 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.445 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.445 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.445 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.445 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.445 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.445 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.445 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.445 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.445 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.446 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.446 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.446 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.446 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.446 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.446 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.446 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.446 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.446 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.446 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.446 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.446 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.446 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.446 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.447 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.447 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.447 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.447 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.447 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.447 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.447 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.447 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.447 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.447 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.447 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.447 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.447 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.447 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.448 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.448 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.448 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "telemetry", 2024-05-08T22:18:08.448 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.448 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.448 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.448 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_basic": { 2024-05-08T22:18:08.448 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_basic", 2024-05-08T22:18:08.448 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.448 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.448 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.448 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.448 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.448 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.449 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.449 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T22:18:08.449 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.449 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.449 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.449 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.449 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_crash": { 2024-05-08T22:18:08.449 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_crash", 2024-05-08T22:18:08.449 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.449 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.449 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.449 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.449 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.449 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.450 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.450 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T22:18:08.450 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.450 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.450 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.450 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.450 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_device": { 2024-05-08T22:18:08.450 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_device", 2024-05-08T22:18:08.450 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.450 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.450 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.450 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.450 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.450 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.450 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.451 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T22:18:08.451 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.451 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.451 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.451 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.451 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_ident": { 2024-05-08T22:18:08.451 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_ident", 2024-05-08T22:18:08.451 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.451 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.451 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.451 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.451 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.451 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.451 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.452 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T22:18:08.452 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.452 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.452 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.452 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.452 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_perf": { 2024-05-08T22:18:08.452 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_perf", 2024-05-08T22:18:08.452 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.452 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.452 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.452 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.452 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.452 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.452 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.452 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T22:18:08.453 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.453 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.453 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.453 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.453 INFO:teuthology.orchestra.run.smithi131.stdout: "contact": { 2024-05-08T22:18:08.453 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "contact", 2024-05-08T22:18:08.453 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.453 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.453 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.453 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.453 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.453 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.453 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.453 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.453 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.454 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.454 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.454 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.454 INFO:teuthology.orchestra.run.smithi131.stdout: "description": { 2024-05-08T22:18:08.454 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "description", 2024-05-08T22:18:08.454 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.454 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.454 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.454 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.454 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.454 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.454 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.454 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.454 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.455 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.455 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.455 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.455 INFO:teuthology.orchestra.run.smithi131.stdout: "device_url": { 2024-05-08T22:18:08.455 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_url", 2024-05-08T22:18:08.455 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.455 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.455 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.455 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T22:18:08.455 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.455 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.455 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.455 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.455 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.455 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.455 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.456 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.456 INFO:teuthology.orchestra.run.smithi131.stdout: "enabled": { 2024-05-08T22:18:08.456 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enabled", 2024-05-08T22:18:08.456 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.456 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.456 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.456 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.456 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.456 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.456 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.456 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.456 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.456 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.456 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.456 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.456 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-08T22:18:08.457 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-08T22:18:08.457 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.457 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.457 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.457 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "24", 2024-05-08T22:18:08.457 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "8", 2024-05-08T22:18:08.457 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.457 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.457 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.457 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.457 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.457 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.457 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.457 INFO:teuthology.orchestra.run.smithi131.stdout: "last_opt_revision": { 2024-05-08T22:18:08.457 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "last_opt_revision", 2024-05-08T22:18:08.457 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.458 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.458 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.458 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1", 2024-05-08T22:18:08.458 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.458 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.458 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.458 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.458 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.458 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.458 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.458 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.458 INFO:teuthology.orchestra.run.smithi131.stdout: "leaderboard": { 2024-05-08T22:18:08.458 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "leaderboard", 2024-05-08T22:18:08.458 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.458 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.459 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.459 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.459 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.459 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.459 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.459 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.459 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.459 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.459 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.459 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.459 INFO:teuthology.orchestra.run.smithi131.stdout: "leaderboard_description": { 2024-05-08T22:18:08.459 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "leaderboard_description", 2024-05-08T22:18:08.459 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.459 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.459 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.459 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.459 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.460 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.460 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.460 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.460 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.460 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.460 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.460 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.460 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.460 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.460 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.460 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.460 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.460 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.460 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.461 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.461 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.461 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.461 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.461 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.461 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.461 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.461 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.461 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.461 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.461 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.461 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.461 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.461 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.461 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.461 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.461 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.462 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.462 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.462 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.462 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.462 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.462 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.462 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.462 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.462 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.462 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.462 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.462 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.462 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.462 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.463 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.463 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.463 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.463 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.463 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.463 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.463 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.463 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.463 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.463 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.463 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.463 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.463 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.463 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.463 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.463 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.463 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.464 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.464 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.464 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.464 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.464 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.464 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.464 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.464 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.464 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.464 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.464 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.464 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.464 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.464 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.464 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.464 INFO:teuthology.orchestra.run.smithi131.stdout: "organization": { 2024-05-08T22:18:08.465 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "organization", 2024-05-08T22:18:08.465 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.465 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.465 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.465 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.465 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.465 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.465 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.465 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.465 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.465 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.465 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.465 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.465 INFO:teuthology.orchestra.run.smithi131.stdout: "proxy": { 2024-05-08T22:18:08.465 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "proxy", 2024-05-08T22:18:08.465 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.465 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.466 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.466 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.466 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.466 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.466 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.466 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.466 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.466 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.466 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.466 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.466 INFO:teuthology.orchestra.run.smithi131.stdout: "url": { 2024-05-08T22:18:08.466 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "url", 2024-05-08T22:18:08.466 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.466 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.466 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.467 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T22:18:08.467 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.467 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.467 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.467 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.467 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.467 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.467 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.467 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.467 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.467 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.467 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.467 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "test_orchestrator", 2024-05-08T22:18:08.467 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.467 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.467 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.468 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.468 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.468 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.468 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.468 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.468 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.468 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.468 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.468 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.468 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.468 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.468 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.468 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.468 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.468 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.469 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.469 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.469 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.469 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.469 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.469 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.469 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.469 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.469 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.469 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.469 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.469 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.469 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.469 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.469 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.469 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.469 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.470 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.470 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.470 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.470 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.470 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.470 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.470 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.470 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.470 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.470 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.470 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.470 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.470 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.470 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.470 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.471 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.471 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.471 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.471 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.471 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.471 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.471 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.471 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.471 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.471 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.471 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.471 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.471 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.471 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.471 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.472 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.472 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.472 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.472 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.472 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.472 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.472 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.472 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.472 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.472 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.472 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.472 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "volumes", 2024-05-08T22:18:08.472 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.472 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.472 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.472 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.473 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.473 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.473 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.473 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.473 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.473 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.473 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.473 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.473 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.473 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.473 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.473 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.473 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.473 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.473 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.474 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.474 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.474 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.474 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.474 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.474 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.474 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.474 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.474 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.474 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.474 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.474 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.474 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.474 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.474 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.475 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.475 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.475 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.475 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.475 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.475 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.475 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.475 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.475 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.475 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.475 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.475 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.475 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.475 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.475 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.476 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.476 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.476 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.476 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.476 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.476 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.476 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.476 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.476 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.476 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.476 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.476 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.476 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.476 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.476 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.477 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.477 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.477 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.477 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.477 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.477 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.477 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.477 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.477 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.477 INFO:teuthology.orchestra.run.smithi131.stdout: "max_concurrent_clones": { 2024-05-08T22:18:08.477 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_concurrent_clones", 2024-05-08T22:18:08.477 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.477 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.477 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.477 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "4", 2024-05-08T22:18:08.477 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.478 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.478 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.478 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T22:18:08.478 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.478 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.478 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.478 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.478 INFO:teuthology.orchestra.run.smithi131.stdout: "snapshot_clone_delay": { 2024-05-08T22:18:08.478 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snapshot_clone_delay", 2024-05-08T22:18:08.478 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.478 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.478 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.478 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-08T22:18:08.478 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.478 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.479 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.479 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T22:18:08.479 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.479 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.479 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.479 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.479 INFO:teuthology.orchestra.run.smithi131.stdout: "snapshot_clone_no_wait": { 2024-05-08T22:18:08.479 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T22:18:08.479 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.479 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.479 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.479 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-08T22:18:08.479 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.479 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.479 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.480 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T22:18:08.480 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.480 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.480 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.480 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.480 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.480 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.480 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.480 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix", 2024-05-08T22:18:08.480 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-08T22:18:08.480 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-08T22:18:08.480 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-08T22:18:08.480 INFO:teuthology.orchestra.run.smithi131.stdout: "discovery_interval": { 2024-05-08T22:18:08.480 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "discovery_interval", 2024-05-08T22:18:08.480 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-08T22:18:08.480 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.481 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.481 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "100", 2024-05-08T22:18:08.481 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.481 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.481 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.481 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.481 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.481 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.481 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.481 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.481 INFO:teuthology.orchestra.run.smithi131.stdout: "identifier": { 2024-05-08T22:18:08.481 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "identifier", 2024-05-08T22:18:08.481 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.481 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.481 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.481 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.482 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.482 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.482 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.482 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.482 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.482 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.482 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.482 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.482 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-08T22:18:08.482 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-08T22:18:08.482 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-08T22:18:08.482 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.482 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.482 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-08T22:18:08.483 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.483 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.483 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.483 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.483 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.483 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.483 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.483 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.483 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-08T22:18:08.483 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-08T22:18:08.483 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.483 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.483 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.483 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.483 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.484 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.484 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.484 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.484 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.484 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.484 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.484 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.484 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.484 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.484 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.484 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.484 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.484 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.484 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.485 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-08T22:18:08.485 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-08T22:18:08.485 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.485 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.485 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.485 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.485 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.485 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.485 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.485 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.485 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.485 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.485 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.485 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.486 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-08T22:18:08.486 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-08T22:18:08.486 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.486 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.486 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.486 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-08T22:18:08.486 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.486 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.486 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-08T22:18:08.486 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-08T22:18:08.486 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-08T22:18:08.486 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-08T22:18:08.486 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-08T22:18:08.486 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-08T22:18:08.486 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-08T22:18:08.486 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.487 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.487 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.487 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.487 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.487 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.487 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-08T22:18:08.487 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-08T22:18:08.487 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-08T22:18:08.487 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.487 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-08T22:18:08.487 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-08T22:18:08.487 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.487 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.487 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.487 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.488 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.488 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.488 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.488 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.488 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_host": { 2024-05-08T22:18:08.488 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_host", 2024-05-08T22:18:08.488 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.488 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.488 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.488 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-08T22:18:08.488 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.488 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.488 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.488 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.488 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.488 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.488 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.489 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.489 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_port": { 2024-05-08T22:18:08.489 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_port", 2024-05-08T22:18:08.489 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-08T22:18:08.489 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.489 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.489 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10051", 2024-05-08T22:18:08.489 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.489 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.489 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.489 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.489 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.489 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.489 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.489 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.489 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_sender": { 2024-05-08T22:18:08.490 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_sender", 2024-05-08T22:18:08.490 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-08T22:18:08.490 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-08T22:18:08.490 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-08T22:18:08.490 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T22:18:08.490 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-08T22:18:08.490 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-08T22:18:08.490 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-08T22:18:08.490 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-08T22:18:08.490 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-08T22:18:08.490 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-08T22:18:08.490 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-08T22:18:08.490 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.490 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.490 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.490 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.491 INFO:teuthology.orchestra.run.smithi131.stdout: "services": { 2024-05-08T22:18:08.491 INFO:teuthology.orchestra.run.smithi131.stdout: "dashboard": "https://172.21.15.131:8443/" 2024-05-08T22:18:08.491 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.491 INFO:teuthology.orchestra.run.smithi131.stdout: "always_on_modules": { 2024-05-08T22:18:08.491 INFO:teuthology.orchestra.run.smithi131.stdout: "octopus": [ 2024-05-08T22:18:08.491 INFO:teuthology.orchestra.run.smithi131.stdout: "balancer", 2024-05-08T22:18:08.491 INFO:teuthology.orchestra.run.smithi131.stdout: "crash", 2024-05-08T22:18:08.491 INFO:teuthology.orchestra.run.smithi131.stdout: "devicehealth", 2024-05-08T22:18:08.491 INFO:teuthology.orchestra.run.smithi131.stdout: "orchestrator", 2024-05-08T22:18:08.491 INFO:teuthology.orchestra.run.smithi131.stdout: "pg_autoscaler", 2024-05-08T22:18:08.491 INFO:teuthology.orchestra.run.smithi131.stdout: "progress", 2024-05-08T22:18:08.491 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_support", 2024-05-08T22:18:08.491 INFO:teuthology.orchestra.run.smithi131.stdout: "status", 2024-05-08T22:18:08.491 INFO:teuthology.orchestra.run.smithi131.stdout: "telemetry", 2024-05-08T22:18:08.491 INFO:teuthology.orchestra.run.smithi131.stdout: "volumes" 2024-05-08T22:18:08.491 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.492 INFO:teuthology.orchestra.run.smithi131.stdout: "pacific": [ 2024-05-08T22:18:08.492 INFO:teuthology.orchestra.run.smithi131.stdout: "balancer", 2024-05-08T22:18:08.492 INFO:teuthology.orchestra.run.smithi131.stdout: "crash", 2024-05-08T22:18:08.492 INFO:teuthology.orchestra.run.smithi131.stdout: "devicehealth", 2024-05-08T22:18:08.492 INFO:teuthology.orchestra.run.smithi131.stdout: "orchestrator", 2024-05-08T22:18:08.492 INFO:teuthology.orchestra.run.smithi131.stdout: "pg_autoscaler", 2024-05-08T22:18:08.492 INFO:teuthology.orchestra.run.smithi131.stdout: "progress", 2024-05-08T22:18:08.492 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_support", 2024-05-08T22:18:08.492 INFO:teuthology.orchestra.run.smithi131.stdout: "status", 2024-05-08T22:18:08.492 INFO:teuthology.orchestra.run.smithi131.stdout: "telemetry", 2024-05-08T22:18:08.492 INFO:teuthology.orchestra.run.smithi131.stdout: "volumes" 2024-05-08T22:18:08.492 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.492 INFO:teuthology.orchestra.run.smithi131.stdout: "quincy": [ 2024-05-08T22:18:08.492 INFO:teuthology.orchestra.run.smithi131.stdout: "balancer", 2024-05-08T22:18:08.492 INFO:teuthology.orchestra.run.smithi131.stdout: "crash", 2024-05-08T22:18:08.492 INFO:teuthology.orchestra.run.smithi131.stdout: "devicehealth", 2024-05-08T22:18:08.492 INFO:teuthology.orchestra.run.smithi131.stdout: "orchestrator", 2024-05-08T22:18:08.493 INFO:teuthology.orchestra.run.smithi131.stdout: "pg_autoscaler", 2024-05-08T22:18:08.493 INFO:teuthology.orchestra.run.smithi131.stdout: "progress", 2024-05-08T22:18:08.493 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_support", 2024-05-08T22:18:08.493 INFO:teuthology.orchestra.run.smithi131.stdout: "status", 2024-05-08T22:18:08.493 INFO:teuthology.orchestra.run.smithi131.stdout: "telemetry", 2024-05-08T22:18:08.493 INFO:teuthology.orchestra.run.smithi131.stdout: "volumes" 2024-05-08T22:18:08.493 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-08T22:18:08.493 INFO:teuthology.orchestra.run.smithi131.stdout: "reef": [ 2024-05-08T22:18:08.493 INFO:teuthology.orchestra.run.smithi131.stdout: "balancer", 2024-05-08T22:18:08.493 INFO:teuthology.orchestra.run.smithi131.stdout: "crash", 2024-05-08T22:18:08.493 INFO:teuthology.orchestra.run.smithi131.stdout: "devicehealth", 2024-05-08T22:18:08.493 INFO:teuthology.orchestra.run.smithi131.stdout: "orchestrator", 2024-05-08T22:18:08.493 INFO:teuthology.orchestra.run.smithi131.stdout: "pg_autoscaler", 2024-05-08T22:18:08.493 INFO:teuthology.orchestra.run.smithi131.stdout: "progress", 2024-05-08T22:18:08.493 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_support", 2024-05-08T22:18:08.494 INFO:teuthology.orchestra.run.smithi131.stdout: "status", 2024-05-08T22:18:08.494 INFO:teuthology.orchestra.run.smithi131.stdout: "telemetry", 2024-05-08T22:18:08.494 INFO:teuthology.orchestra.run.smithi131.stdout: "volumes" 2024-05-08T22:18:08.494 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-08T22:18:08.494 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.494 INFO:teuthology.orchestra.run.smithi131.stdout: "last_failure_osd_epoch": 88, 2024-05-08T22:18:08.494 INFO:teuthology.orchestra.run.smithi131.stdout: "active_clients": [ 2024-05-08T22:18:08.494 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.494 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "devicehealth", 2024-05-08T22:18:08.494 INFO:teuthology.orchestra.run.smithi131.stdout: "addrvec": [ 2024-05-08T22:18:08.494 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.494 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v2", 2024-05-08T22:18:08.494 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "172.21.15.131:0", 2024-05-08T22:18:08.494 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 4188763191 2024-05-08T22:18:08.495 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.495 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-08T22:18:08.495 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.495 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.495 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "libcephsqlite", 2024-05-08T22:18:08.495 INFO:teuthology.orchestra.run.smithi131.stdout: "addrvec": [ 2024-05-08T22:18:08.495 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.495 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v2", 2024-05-08T22:18:08.495 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "172.21.15.131:0", 2024-05-08T22:18:08.495 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 3082163846 2024-05-08T22:18:08.495 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.495 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-08T22:18:08.495 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.495 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.495 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_support", 2024-05-08T22:18:08.495 INFO:teuthology.orchestra.run.smithi131.stdout: "addrvec": [ 2024-05-08T22:18:08.496 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.496 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v2", 2024-05-08T22:18:08.496 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "172.21.15.131:0", 2024-05-08T22:18:08.496 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 3832375614 2024-05-08T22:18:08.496 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.496 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-08T22:18:08.496 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.496 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.496 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snap_schedule", 2024-05-08T22:18:08.496 INFO:teuthology.orchestra.run.smithi131.stdout: "addrvec": [ 2024-05-08T22:18:08.496 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.496 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v2", 2024-05-08T22:18:08.496 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "172.21.15.131:0", 2024-05-08T22:18:08.496 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 3572657118 2024-05-08T22:18:08.496 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.496 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-08T22:18:08.497 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.497 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.497 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snap_schedule", 2024-05-08T22:18:08.497 INFO:teuthology.orchestra.run.smithi131.stdout: "addrvec": [ 2024-05-08T22:18:08.497 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.497 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v2", 2024-05-08T22:18:08.497 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "172.21.15.131:0", 2024-05-08T22:18:08.497 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 1350374208 2024-05-08T22:18:08.497 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.497 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-08T22:18:08.497 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:08.497 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.497 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "volumes", 2024-05-08T22:18:08.497 INFO:teuthology.orchestra.run.smithi131.stdout: "addrvec": [ 2024-05-08T22:18:08.497 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-08T22:18:08.497 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v2", 2024-05-08T22:18:08.497 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "172.21.15.131:0", 2024-05-08T22:18:08.498 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 1996435468 2024-05-08T22:18:08.498 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.498 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-08T22:18:08.498 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:08.498 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-08T22:18:08.498 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:18:08.499 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:07 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3732064189' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:18:08.499 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:07 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2054894806' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T22:18:08.499 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:07 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3732064189' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:18:08.499 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:07 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2054894806' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T22:18:08.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.301+0000 7f40ee400700 1 -- 172.21.15.131:0/2054894806 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f40cc0a1290 msgr2=0x7f40cc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:08.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.301+0000 7f40ee400700 1 --2- 172.21.15.131:0/2054894806 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f40cc0a1290 0x7f40cc0a3750 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f40d8009fd0 tx=0x7f40d8019040 comp rx=0 tx=0).stop 2024-05-08T22:18:08.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.301+0000 7f40ee400700 1 -- 172.21.15.131:0/2054894806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40e8106170 msgr2=0x7f40e80737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:08.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.301+0000 7f40ee400700 1 --2- 172.21.15.131:0/2054894806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40e8106170 0x7f40e80737e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f40d400ec70 tx=0x7f40d400ef80 comp rx=0 tx=0).stop 2024-05-08T22:18:08.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.301+0000 7f40ee400700 1 -- 172.21.15.131:0/2054894806 shutdown_connections 2024-05-08T22:18:08.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.301+0000 7f40ee400700 1 --2- 172.21.15.131:0/2054894806 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f40cc0a1290 0x7f40cc0a3750 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:08.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.301+0000 7f40ee400700 1 --2- 172.21.15.131:0/2054894806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40e8073d20 0x7f40e81a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:08.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.301+0000 7f40ee400700 1 --2- 172.21.15.131:0/2054894806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40e8101050 0x7f40e80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:08.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.301+0000 7f40ee400700 1 --2- 172.21.15.131:0/2054894806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40e8106170 0x7f40e80737e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:08.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.301+0000 7f40ee400700 1 -- 172.21.15.131:0/2054894806 >> 172.21.15.131:0/2054894806 conn(0x7f40e80fc8f0 msgr2=0x7f40e80fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:08.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.301+0000 7f40ee400700 1 -- 172.21.15.131:0/2054894806 shutdown_connections 2024-05-08T22:18:08.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:07.301+0000 7f40ee400700 1 -- 172.21.15.131:0/2054894806 wait complete. 2024-05-08T22:18:08.506 DEBUG:tasks.check_counter:Getting stats from a 2024-05-08T22:18:08.506 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-08T22:18:08.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:07 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3732064189' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:18:08.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:07 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2054894806' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T22:18:08.781 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:08 smithi157 ceph-mon[77746]: pgmap v337: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-08T22:18:08.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:08 smithi132 ceph-mon[78180]: pgmap v337: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-08T22:18:08.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:08 smithi131 ceph-mon[37260]: pgmap v337: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-08T22:18:09.209 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:18:09.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:09 smithi131 ceph-mon[37260]: osdmap e98: 12 total, 12 up, 12 in 2024-05-08T22:18:09.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:09 smithi157 ceph-mon[77746]: osdmap e98: 12 total, 12 up, 12 in 2024-05-08T22:18:09.846 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:09 smithi132 ceph-mon[78180]: osdmap e98: 12 total, 12 up, 12 in 2024-05-08T22:18:10.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:10 smithi157 ceph-mon[77746]: pgmap v339: 129 pgs: 16 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 65 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-08T22:18:10.847 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:10 smithi132 ceph-mon[78180]: pgmap v339: 129 pgs: 16 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 65 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-08T22:18:10.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:10 smithi131 ceph-mon[37260]: pgmap v339: 129 pgs: 16 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 65 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-08T22:18:10.958 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:18:10.958 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:18:10.958 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 5338, 2024-05-08T22:18:10.958 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 5300, 2024-05-08T22:18:10.958 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 2529269, 2024-05-08T22:18:10.958 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 15382905, 2024-05-08T22:18:10.958 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 10, 2024-05-08T22:18:10.959 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 5, 2024-05-08T22:18:10.959 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.832341283, 2024-05-08T22:18:10.959 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.300732348, 2024-05-08T22:18:10.959 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.416820796, 2024-05-08T22:18:10.959 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.153153060, 2024-05-08T22:18:10.959 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:10.959 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 5298, 2024-05-08T22:18:10.959 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 1.743329051, 2024-05-08T22:18:10.959 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000329054 2024-05-08T22:18:10.959 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.959 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:10.959 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 6621, 2024-05-08T22:18:10.959 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.003966165, 2024-05-08T22:18:10.959 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000599 2024-05-08T22:18:10.959 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.960 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 3296, 2024-05-08T22:18:10.960 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 12192 2024-05-08T22:18:10.960 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.960 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:18:10.960 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 2468, 2024-05-08T22:18:10.960 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 2628, 2024-05-08T22:18:10.960 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 624751, 2024-05-08T22:18:10.960 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 1760178, 2024-05-08T22:18:10.960 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 39, 2024-05-08T22:18:10.960 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 6, 2024-05-08T22:18:10.960 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.393081186, 2024-05-08T22:18:10.960 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.135763145, 2024-05-08T22:18:10.960 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.098362119, 2024-05-08T22:18:10.960 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.116952657, 2024-05-08T22:18:10.961 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:10.961 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2625, 2024-05-08T22:18:10.961 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.715411084, 2024-05-08T22:18:10.961 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000272537 2024-05-08T22:18:10.961 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.961 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:10.961 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 40, 2024-05-08T22:18:10.961 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000007149, 2024-05-08T22:18:10.961 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000178 2024-05-08T22:18:10.961 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.961 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 2016, 2024-05-08T22:18:10.961 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 436432 2024-05-08T22:18:10.961 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.961 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:18:10.962 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 37638, 2024-05-08T22:18:10.962 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 43654, 2024-05-08T22:18:10.962 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 13590868, 2024-05-08T22:18:10.962 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 30314361, 2024-05-08T22:18:10.962 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 11, 2024-05-08T22:18:10.962 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 6, 2024-05-08T22:18:10.962 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 11.767521413, 2024-05-08T22:18:10.962 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 9.556337908, 2024-05-08T22:18:10.962 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 2.513951637, 2024-05-08T22:18:10.962 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.189889616, 2024-05-08T22:18:10.962 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:10.962 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 43654, 2024-05-08T22:18:10.962 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 19.918356880, 2024-05-08T22:18:10.962 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000456277 2024-05-08T22:18:10.963 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.963 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:10.963 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 30650, 2024-05-08T22:18:10.963 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.039075193, 2024-05-08T22:18:10.963 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000001274 2024-05-08T22:18:10.963 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.963 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 1493376, 2024-05-08T22:18:10.963 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 137072 2024-05-08T22:18:10.963 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.963 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-05-08T22:18:10.963 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-05-08T22:18:10.963 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-05-08T22:18:10.963 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.963 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-05-08T22:18:10.964 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-08T22:18:10.964 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-08T22:18:10.964 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3431, 2024-05-08T22:18:10.964 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 11.098225060, 2024-05-08T22:18:10.964 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.003234691 2024-05-08T22:18:10.964 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:10.964 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.964 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-05-08T22:18:10.964 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-08T22:18:10.964 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-08T22:18:10.964 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3137, 2024-05-08T22:18:10.964 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.693245713, 2024-05-08T22:18:10.964 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000220990 2024-05-08T22:18:10.964 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:10.964 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.965 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-05-08T22:18:10.965 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 21774, 2024-05-08T22:18:10.965 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 21771, 2024-05-08T22:18:10.965 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-05-08T22:18:10.965 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 21771, 2024-05-08T22:18:10.965 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 899.141231287, 2024-05-08T22:18:10.965 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.041299950 2024-05-08T22:18:10.965 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.965 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-05-08T22:18:10.965 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 3, 2024-05-08T22:18:10.965 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 4, 2024-05-08T22:18:10.965 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-05-08T22:18:10.965 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 65, 2024-05-08T22:18:10.965 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 6, 2024-05-08T22:18:10.966 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-05-08T22:18:10.966 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 12, 2024-05-08T22:18:10.966 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 1, 2024-05-08T22:18:10.966 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 1, 2024-05-08T22:18:10.966 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 10, 2024-05-08T22:18:10.966 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 10, 2024-05-08T22:18:10.966 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 4842, 2024-05-08T22:18:10.966 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 1, 2024-05-08T22:18:10.966 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 1, 2024-05-08T22:18:10.966 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 3, 2024-05-08T22:18:10.966 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 40701, 2024-05-08T22:18:10.966 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 32371, 2024-05-08T22:18:10.966 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 0, 2024-05-08T22:18:10.966 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 15, 2024-05-08T22:18:10.967 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-05-08T22:18:10.967 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-05-08T22:18:10.967 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 129, 2024-05-08T22:18:10.967 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 0, 2024-05-08T22:18:10.967 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-05-08T22:18:10.967 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 56, 2024-05-08T22:18:10.967 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 4044, 2024-05-08T22:18:10.967 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 14, 2024-05-08T22:18:10.967 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 1561, 2024-05-08T22:18:10.967 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-05-08T22:18:10.967 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 3939, 2024-05-08T22:18:10.967 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 1, 2024-05-08T22:18:10.967 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 4, 2024-05-08T22:18:10.967 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 234, 2024-05-08T22:18:10.968 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 6, 2024-05-08T22:18:10.968 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 3939, 2024-05-08T22:18:10.968 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-05-08T22:18:10.968 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:18:10.968 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:18:10.968 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-05-08T22:18:10.968 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T22:18:10.968 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T22:18:10.968 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 3939, 2024-05-08T22:18:10.968 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 94, 2024-05-08T22:18:10.968 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 1032, 2024-05-08T22:18:10.968 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 2768, 2024-05-08T22:18:10.968 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T22:18:10.968 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:18:10.968 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:18:10.969 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 2666, 2024-05-08T22:18:10.969 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 1143, 2024-05-08T22:18:10.969 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 106, 2024-05-08T22:18:10.969 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 5732 2024-05-08T22:18:10.969 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.969 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-05-08T22:18:10.969 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 2, 2024-05-08T22:18:10.969 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 3, 2024-05-08T22:18:10.969 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 2, 2024-05-08T22:18:10.969 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 11, 2024-05-08T22:18:10.969 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 9, 2024-05-08T22:18:10.969 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-05-08T22:18:10.969 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-05-08T22:18:10.969 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:18:10.969 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 1644, 2024-05-08T22:18:10.970 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 1644, 2024-05-08T22:18:10.970 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-05-08T22:18:10.970 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-05-08T22:18:10.970 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-05-08T22:18:10.970 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:18:10.970 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:18:10.970 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-05-08T22:18:10.970 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-05-08T22:18:10.970 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T22:18:10.970 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 56, 2024-05-08T22:18:10.970 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-05-08T22:18:10.970 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 6, 2024-05-08T22:18:10.970 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-05-08T22:18:10.970 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0 2024-05-08T22:18:10.971 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.971 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-05-08T22:18:10.971 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 8364, 2024-05-08T22:18:10.971 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-05-08T22:18:10.971 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-05-08T22:18:10.971 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 8364, 2024-05-08T22:18:10.971 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-05-08T22:18:10.971 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-05-08T22:18:10.971 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 10, 2024-05-08T22:18:10.971 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-05-08T22:18:10.971 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-05-08T22:18:10.971 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 10, 2024-05-08T22:18:10.971 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-05-08T22:18:10.971 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-05-08T22:18:10.971 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-05-08T22:18:10.972 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 20497971, 2024-05-08T22:18:10.972 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 0, 2024-05-08T22:18:10.972 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-05-08T22:18:10.972 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 315, 2024-05-08T22:18:10.972 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 5.774300114, 2024-05-08T22:18:10.972 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.018331111 2024-05-08T22:18:10.972 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.972 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 0 2024-05-08T22:18:10.972 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.972 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-05-08T22:18:10.972 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 16, 2024-05-08T22:18:10.972 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 3897, 2024-05-08T22:18:10.972 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 3881, 2024-05-08T22:18:10.972 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 15, 2024-05-08T22:18:10.972 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 102, 2024-05-08T22:18:10.973 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 87, 2024-05-08T22:18:10.973 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 12, 2024-05-08T22:18:10.973 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 6583, 2024-05-08T22:18:10.973 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 6571, 2024-05-08T22:18:10.973 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 1, 2024-05-08T22:18:10.973 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 2763, 2024-05-08T22:18:10.973 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 2762, 2024-05-08T22:18:10.973 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 70868, 2024-05-08T22:18:10.973 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 198964 2024-05-08T22:18:10.973 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.973 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-05-08T22:18:10.973 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 22322, 2024-05-08T22:18:10.973 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 137, 2024-05-08T22:18:10.973 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 21774, 2024-05-08T22:18:10.974 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 25, 2024-05-08T22:18:10.974 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 396, 2024-05-08T22:18:10.974 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-05-08T22:18:10.974 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 918, 2024-05-08T22:18:10.974 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 8.501122853, 2024-05-08T22:18:10.974 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.009260482 2024-05-08T22:18:10.974 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.974 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-05-08T22:18:10.974 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3, 2024-05-08T22:18:10.974 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000585762, 2024-05-08T22:18:10.974 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000195254 2024-05-08T22:18:10.974 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.974 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-05-08T22:18:10.974 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:10.974 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:10.975 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:10.975 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.975 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-05-08T22:18:10.975 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:10.975 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:10.975 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:10.975 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.975 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-05-08T22:18:10.975 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 17686, 2024-05-08T22:18:10.975 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 878.508686396, 2024-05-08T22:18:10.975 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.049672548 2024-05-08T22:18:10.975 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.975 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-05-08T22:18:10.975 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:10.976 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:10.976 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:10.976 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.976 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-05-08T22:18:10.976 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:10.976 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:10.976 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:10.976 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.976 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-05-08T22:18:10.976 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:10.976 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:10.976 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:10.976 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.976 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-05-08T22:18:10.977 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:10.977 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:10.977 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:10.977 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.977 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-05-08T22:18:10.977 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:10.977 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:10.977 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:10.977 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.977 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-05-08T22:18:10.977 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:10.977 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:10.977 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:10.977 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.978 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-05-08T22:18:10.978 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:10.978 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:10.978 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:10.978 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.978 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-05-08T22:18:10.978 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:10.978 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:10.978 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:10.978 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.978 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-05-08T22:18:10.978 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:10.978 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:10.978 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:10.979 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.979 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-05-08T22:18:10.979 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 911, 2024-05-08T22:18:10.979 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 3.020775283, 2024-05-08T22:18:10.979 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.003315889 2024-05-08T22:18:10.979 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.979 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-05-08T22:18:10.979 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 56, 2024-05-08T22:18:10.979 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.076806179, 2024-05-08T22:18:10.979 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001371538 2024-05-08T22:18:10.979 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.979 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-05-08T22:18:10.979 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 918, 2024-05-08T22:18:10.980 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 5.793093897, 2024-05-08T22:18:10.980 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.006310559 2024-05-08T22:18:10.980 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.980 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-05-08T22:18:10.980 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:10.980 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:10.980 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:10.980 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.980 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-05-08T22:18:10.980 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:10.980 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:10.981 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:10.981 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.981 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-05-08T22:18:10.981 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:10.981 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:10.981 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:10.981 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.981 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-05-08T22:18:10.981 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:10.981 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:10.981 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:10.981 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.982 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-05-08T22:18:10.982 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:10.982 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:10.982 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:10.982 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.982 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-05-08T22:18:10.982 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:10.982 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:10.982 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:10.982 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.982 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-05-08T22:18:10.983 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:10.983 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:10.983 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:10.983 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.983 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-05-08T22:18:10.983 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:10.983 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:10.983 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:10.983 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.983 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-05-08T22:18:10.983 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:10.983 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:10.983 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:10.983 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.984 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-05-08T22:18:10.984 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:10.984 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:10.984 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:10.984 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.984 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-05-08T22:18:10.984 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:10.984 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:10.984 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:10.984 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.984 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-05-08T22:18:10.985 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1279, 2024-05-08T22:18:10.985 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 3.240160917, 2024-05-08T22:18:10.985 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.002533354 2024-05-08T22:18:10.985 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.985 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:18:10.985 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:18:10.985 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-05-08T22:18:10.985 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:10.985 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:10.985 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:10.985 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:10.986 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.986 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-05-08T22:18:10.986 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 2, 2024-05-08T22:18:10.986 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 4, 2024-05-08T22:18:10.986 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 2, 2024-05-08T22:18:10.986 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 2, 2024-05-08T22:18:10.986 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-05-08T22:18:10.986 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 2058, 2024-05-08T22:18:10.986 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 1029, 2024-05-08T22:18:10.986 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 619, 2024-05-08T22:18:10.986 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-05-08T22:18:10.986 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.986 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-05-08T22:18:10.987 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:18:10.987 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-05-08T22:18:10.987 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:18:10.987 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:18:10.987 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:18:10.987 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:18:10.987 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:18:10.987 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:18:10.987 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:18:10.987 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:18:10.987 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:18:10.987 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:18:10.987 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:18:10.987 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:18:10.988 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:18:10.988 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-05-08T22:18:10.988 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:18:10.988 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-05-08T22:18:10.988 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:18:10.988 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:18:10.988 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:18:10.988 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:18:10.988 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:18:10.988 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:18:10.988 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:18:10.988 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-05-08T22:18:10.988 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:18:10.988 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:18:10.989 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:18:10.989 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-05-08T22:18:10.989 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-05-08T22:18:10.989 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-05-08T22:18:10.989 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:18:10.989 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:18:10.989 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:18:10.989 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:18:10.989 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 294653, 2024-05-08T22:18:10.989 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 129, 2024-05-08T22:18:10.989 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 704, 2024-05-08T22:18:10.989 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 8, 2024-05-08T22:18:10.989 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-05-08T22:18:10.989 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-05-08T22:18:10.989 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:18:10.990 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-05-08T22:18:10.990 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:18:10.990 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-05-08T22:18:10.990 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 9912, 2024-05-08T22:18:10.990 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 206, 2024-05-08T22:18:10.990 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:18:10.990 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:18:10.990 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-05-08T22:18:10.990 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-05-08T22:18:10.990 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 107112, 2024-05-08T22:18:10.990 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 429, 2024-05-08T22:18:10.990 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-05-08T22:18:10.990 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-05-08T22:18:10.990 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-05-08T22:18:10.991 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-05-08T22:18:10.991 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.991 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-05-08T22:18:10.991 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-05-08T22:18:10.991 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-05-08T22:18:10.991 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 7969, 2024-05-08T22:18:10.991 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 17679910, 2024-05-08T22:18:10.991 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 27, 2024-05-08T22:18:10.991 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 7969, 2024-05-08T22:18:10.991 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-05-08T22:18:10.991 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 7969, 2024-05-08T22:18:10.991 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 219.675875804, 2024-05-08T22:18:10.991 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.027566303 2024-05-08T22:18:10.991 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.991 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-05-08T22:18:10.992 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-05-08T22:18:10.992 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 7969, 2024-05-08T22:18:10.992 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 8144 2024-05-08T22:18:10.992 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.992 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 7969, 2024-05-08T22:18:10.992 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 4041, 2024-05-08T22:18:10.992 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 3928, 2024-05-08T22:18:10.992 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-05-08T22:18:10.992 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-05-08T22:18:10.992 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 0, 2024-05-08T22:18:10.992 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 10, 2024-05-08T22:18:10.992 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 94, 2024-05-08T22:18:10.992 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 409, 2024-05-08T22:18:10.992 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 77, 2024-05-08T22:18:10.992 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-05-08T22:18:10.993 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-05-08T22:18:10.993 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-05-08T22:18:10.993 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 6, 2024-05-08T22:18:10.993 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 3303, 2024-05-08T22:18:10.993 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-05-08T22:18:10.993 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-05-08T22:18:10.993 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-05-08T22:18:10.993 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 3939, 2024-05-08T22:18:10.993 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 20, 2024-05-08T22:18:10.993 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:18:10.993 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-05-08T22:18:10.993 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:18:10.993 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-05-08T22:18:10.993 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-05-08T22:18:10.994 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-05-08T22:18:10.994 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:18:10.994 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-05-08T22:18:10.994 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:18:10.994 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-05-08T22:18:10.994 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-05-08T22:18:10.994 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-05-08T22:18:10.994 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-05-08T22:18:10.994 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-05-08T22:18:10.994 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-05-08T22:18:10.994 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-05-08T22:18:10.994 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-05-08T22:18:10.994 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-05-08T22:18:10.994 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-05-08T22:18:10.995 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-05-08T22:18:10.995 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-05-08T22:18:10.995 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-05-08T22:18:10.995 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-05-08T22:18:10.995 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-05-08T22:18:10.995 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-05-08T22:18:10.995 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-05-08T22:18:10.995 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 98, 2024-05-08T22:18:10.995 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-05-08T22:18:10.995 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 15, 2024-05-08T22:18:10.995 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 12, 2024-05-08T22:18:10.995 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 12, 2024-05-08T22:18:10.995 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 0, 2024-05-08T22:18:10.995 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-05-08T22:18:10.995 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 227, 2024-05-08T22:18:10.996 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 12, 2024-05-08T22:18:10.996 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 47 2024-05-08T22:18:10.996 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.996 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-05-08T22:18:10.996 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 1, 2024-05-08T22:18:10.996 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:18:10.996 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-05-08T22:18:10.996 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-05-08T22:18:10.996 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.996 INFO:teuthology.orchestra.run.smithi131.stdout: "osd-slow-ops": { 2024-05-08T22:18:10.996 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 0 2024-05-08T22:18:10.996 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.996 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-05-08T22:18:10.996 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-05-08T22:18:10.997 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 16, 2024-05-08T22:18:10.997 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-05-08T22:18:10.997 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 8, 2024-05-08T22:18:10.997 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 1644, 2024-05-08T22:18:10.997 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-05-08T22:18:10.997 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.997 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:18:10.997 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-08T22:18:10.997 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-08T22:18:10.997 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-08T22:18:10.997 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 45440, 2024-05-08T22:18:10.997 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 13172808, 2024-05-08T22:18:10.997 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:10.997 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 45440, 2024-05-08T22:18:10.997 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-08T22:18:10.998 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-08T22:18:10.998 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 45440, 2024-05-08T22:18:10.998 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 13172808, 2024-05-08T22:18:10.998 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-08T22:18:10.998 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:10.998 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:10.998 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:10.998 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:10.998 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.998 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-05-08T22:18:10.998 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-08T22:18:10.998 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-08T22:18:10.998 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-08T22:18:10.998 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-08T22:18:10.999 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-08T22:18:10.999 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:10.999 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-08T22:18:10.999 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 7969, 2024-05-08T22:18:10.999 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 17895446, 2024-05-08T22:18:10.999 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 7969, 2024-05-08T22:18:10.999 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 17895446, 2024-05-08T22:18:10.999 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-08T22:18:10.999 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:10.999 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:10.999 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:10.999 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:10.999 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:10.999 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-05-08T22:18:10.999 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-08T22:18:11.000 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-05-08T22:18:11.000 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-08T22:18:11.000 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-08T22:18:11.000 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-08T22:18:11.000 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:11.000 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-08T22:18:11.000 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 7969, 2024-05-08T22:18:11.000 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 7969, 2024-05-08T22:18:11.000 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 7969, 2024-05-08T22:18:11.000 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 7969, 2024-05-08T22:18:11.000 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-08T22:18:11.000 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:11.000 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:11.000 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:11.001 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:11.001 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:11.001 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:18:11.001 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-08T22:18:11.001 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-08T22:18:11.001 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-08T22:18:11.001 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 1644, 2024-05-08T22:18:11.001 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 215740, 2024-05-08T22:18:11.001 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:11.001 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 1644, 2024-05-08T22:18:11.001 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-08T22:18:11.001 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-08T22:18:11.002 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 94, 2024-05-08T22:18:11.002 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 215740, 2024-05-08T22:18:11.002 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-08T22:18:11.002 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:11.002 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:11.002 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:11.002 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:11.002 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:11.002 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x562c520280a0": { 2024-05-08T22:18:11.002 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-08T22:18:11.002 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-08T22:18:11.002 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-08T22:18:11.002 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 8364, 2024-05-08T22:18:11.003 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 16303667, 2024-05-08T22:18:11.003 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:11.003 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 8364, 2024-05-08T22:18:11.003 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-08T22:18:11.003 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-08T22:18:11.003 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 315, 2024-05-08T22:18:11.003 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 16303667, 2024-05-08T22:18:11.003 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-08T22:18:11.003 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:11.003 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:11.003 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:11.003 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:11.003 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:11.003 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:18:11.391 INFO:tasks.check_counter:Daemon mds.a mds.exported=56 2024-05-08T22:18:11.391 INFO:tasks.check_counter:Daemon mds.a mds.imported=14 2024-05-08T22:18:11.391 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=6 2024-05-08T22:18:11.391 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-05-08T22:18:11.391 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T22:18:11.391 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=6 2024-05-08T22:18:11.392 DEBUG:tasks.check_counter:Getting stats from d 2024-05-08T22:18:11.392 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-08T22:18:12.086 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:18:12.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:12 smithi131 ceph-mon[37260]: pgmap v340: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-08T22:18:12.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:12 smithi157 ceph-mon[77746]: pgmap v340: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-08T22:18:12.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:12 smithi132 ceph-mon[78180]: pgmap v340: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-08T22:18:13.828 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:18:13.829 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:18:13.829 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 0, 2024-05-08T22:18:13.829 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 2, 2024-05-08T22:18:13.829 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 0, 2024-05-08T22:18:13.829 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 0, 2024-05-08T22:18:13.829 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 4, 2024-05-08T22:18:13.829 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 0, 2024-05-08T22:18:13.829 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.003632034, 2024-05-08T22:18:13.829 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T22:18:13.829 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.000704071, 2024-05-08T22:18:13.829 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T22:18:13.829 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:13.829 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:13.829 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:13.830 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:13.830 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:13.830 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:13.830 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:13.830 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:13.830 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:13.830 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:13.830 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T22:18:13.830 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T22:18:13.830 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:13.830 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:18:13.830 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 229, 2024-05-08T22:18:13.830 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 217, 2024-05-08T22:18:13.831 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 1427424, 2024-05-08T22:18:13.831 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 123168, 2024-05-08T22:18:13.831 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 2, 2024-05-08T22:18:13.831 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 1, 2024-05-08T22:18:13.831 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.067975319, 2024-05-08T22:18:13.831 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.029658608, 2024-05-08T22:18:13.831 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.039030078, 2024-05-08T22:18:13.831 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.006182454, 2024-05-08T22:18:13.831 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:13.831 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 217, 2024-05-08T22:18:13.831 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.034419077, 2024-05-08T22:18:13.831 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000158613 2024-05-08T22:18:13.831 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:13.831 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:13.831 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:13.831 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:13.832 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:13.832 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:13.832 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 1427424, 2024-05-08T22:18:13.832 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 123168 2024-05-08T22:18:13.832 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:13.832 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:18:13.832 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 2, 2024-05-08T22:18:13.832 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 165, 2024-05-08T22:18:13.832 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 320, 2024-05-08T22:18:13.832 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 181472, 2024-05-08T22:18:13.832 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 5, 2024-05-08T22:18:13.832 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 3, 2024-05-08T22:18:13.833 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.022118799, 2024-05-08T22:18:13.833 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.016297044, 2024-05-08T22:18:13.833 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.002930973, 2024-05-08T22:18:13.833 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T22:18:13.833 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:13.833 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 162, 2024-05-08T22:18:13.833 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.034816258, 2024-05-08T22:18:13.833 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000214915 2024-05-08T22:18:13.833 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:13.833 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:13.833 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:13.833 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:13.833 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:13.833 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:13.833 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T22:18:13.834 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 181472 2024-05-08T22:18:13.834 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:13.834 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-05-08T22:18:13.834 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 0, 2024-05-08T22:18:13.834 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-05-08T22:18:13.834 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:13.834 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-05-08T22:18:13.834 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:18:13.834 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-05-08T22:18:13.834 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:18:13.834 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:18:13.834 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:18:13.834 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:18:13.834 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:18:13.834 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:18:13.835 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:18:13.835 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:18:13.835 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:18:13.835 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:18:13.835 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:18:13.835 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:18:13.835 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:18:13.835 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-05-08T22:18:13.835 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:18:13.835 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-05-08T22:18:13.835 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:18:13.835 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:18:13.835 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:18:13.835 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:18:13.836 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:18:13.836 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:18:13.836 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:18:13.836 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-05-08T22:18:13.836 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:18:13.836 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:18:13.836 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:18:13.836 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-05-08T22:18:13.836 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-05-08T22:18:13.836 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-05-08T22:18:13.836 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:18:13.836 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:18:13.836 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:18:13.836 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:18:13.837 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 151039, 2024-05-08T22:18:13.837 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 32, 2024-05-08T22:18:13.837 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:18:13.837 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-05-08T22:18:13.837 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-05-08T22:18:13.837 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-05-08T22:18:13.837 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:18:13.837 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-05-08T22:18:13.837 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:18:13.837 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-05-08T22:18:13.837 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 0, 2024-05-08T22:18:13.837 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 0, 2024-05-08T22:18:13.837 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:18:13.837 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:18:13.837 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-05-08T22:18:13.838 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-05-08T22:18:13.838 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 928, 2024-05-08T22:18:13.838 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 2, 2024-05-08T22:18:13.838 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-05-08T22:18:13.838 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-05-08T22:18:13.838 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-05-08T22:18:13.838 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-05-08T22:18:13.838 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:13.838 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:18:13.838 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-08T22:18:13.838 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-08T22:18:13.838 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-08T22:18:13.838 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 227, 2024-05-08T22:18:13.838 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 1392683, 2024-05-08T22:18:13.838 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:13.838 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 227, 2024-05-08T22:18:13.839 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-08T22:18:13.839 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-08T22:18:13.839 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 227, 2024-05-08T22:18:13.839 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 1392683, 2024-05-08T22:18:13.839 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-08T22:18:13.839 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:13.839 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:13.839 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:13.839 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:13.839 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:13.839 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:18:14.253 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-08T22:18:14.253 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-08T22:18:14.253 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-05-08T22:18:14.253 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-05-08T22:18:14.254 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-05-08T22:18:14.254 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-05-08T22:18:14.254 DEBUG:tasks.check_counter:Getting stats from g 2024-05-08T22:18:14.254 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-08T22:18:14.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:14 smithi132 ceph-mon[78180]: pgmap v341: 129 pgs: 22 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 38 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-08T22:18:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:14 smithi131 ceph-mon[37260]: pgmap v341: 129 pgs: 22 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 38 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-08T22:18:14.933 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:18:15.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:14 smithi157 ceph-mon[77746]: pgmap v341: 129 pgs: 22 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 38 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-08T22:18:16.550 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:16 smithi131 ceph-mon[37260]: pgmap v342: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-08T22:18:16.685 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:18:16.686 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:18:16.686 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 5415, 2024-05-08T22:18:16.686 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 4950, 2024-05-08T22:18:16.686 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 3496541, 2024-05-08T22:18:16.686 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 7795729, 2024-05-08T22:18:16.686 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 28, 2024-05-08T22:18:16.686 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 6, 2024-05-08T22:18:16.686 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.775058471, 2024-05-08T22:18:16.686 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.255924155, 2024-05-08T22:18:16.686 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.488687068, 2024-05-08T22:18:16.686 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.111161944, 2024-05-08T22:18:16.687 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:16.687 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 4949, 2024-05-08T22:18:16.687 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 1.336225889, 2024-05-08T22:18:16.687 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000269999 2024-05-08T22:18:16.687 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.687 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:16.687 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 5650, 2024-05-08T22:18:16.687 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.003589755, 2024-05-08T22:18:16.687 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000635 2024-05-08T22:18:16.687 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.687 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 1487824, 2024-05-08T22:18:16.687 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 137504 2024-05-08T22:18:16.687 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.687 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:18:16.688 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 2176, 2024-05-08T22:18:16.688 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 2336, 2024-05-08T22:18:16.688 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 557944, 2024-05-08T22:18:16.688 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 5751530, 2024-05-08T22:18:16.688 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 11, 2024-05-08T22:18:16.688 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 6, 2024-05-08T22:18:16.688 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.335654092, 2024-05-08T22:18:16.688 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.122841826, 2024-05-08T22:18:16.688 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.061378962, 2024-05-08T22:18:16.688 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.117029780, 2024-05-08T22:18:16.688 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:16.688 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2334, 2024-05-08T22:18:16.688 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.607683039, 2024-05-08T22:18:16.688 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000260361 2024-05-08T22:18:16.688 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.689 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:16.689 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 15, 2024-05-08T22:18:16.689 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000002563, 2024-05-08T22:18:16.689 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000170 2024-05-08T22:18:16.689 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.689 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 5456, 2024-05-08T22:18:16.689 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 440432 2024-05-08T22:18:16.689 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.689 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:18:16.689 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 22652, 2024-05-08T22:18:16.689 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 27296, 2024-05-08T22:18:16.689 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 7639733, 2024-05-08T22:18:16.689 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 17004579, 2024-05-08T22:18:16.689 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 21, 2024-05-08T22:18:16.689 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 5, 2024-05-08T22:18:16.689 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 6.990938738, 2024-05-08T22:18:16.690 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 5.591913595, 2024-05-08T22:18:16.690 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 2.014756975, 2024-05-08T22:18:16.690 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.082053227, 2024-05-08T22:18:16.690 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:16.690 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 27294, 2024-05-08T22:18:16.690 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 12.940487695, 2024-05-08T22:18:16.690 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000474114 2024-05-08T22:18:16.690 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.690 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:16.690 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 19622, 2024-05-08T22:18:16.690 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.025287144, 2024-05-08T22:18:16.690 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000001288 2024-05-08T22:18:16.690 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.690 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 2784, 2024-05-08T22:18:16.691 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 3856 2024-05-08T22:18:16.691 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.691 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-05-08T22:18:16.691 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-05-08T22:18:16.691 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-05-08T22:18:16.691 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.691 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-05-08T22:18:16.691 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-08T22:18:16.691 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-08T22:18:16.691 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2253, 2024-05-08T22:18:16.691 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 7.654723537, 2024-05-08T22:18:16.691 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.003397569 2024-05-08T22:18:16.691 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:16.691 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.691 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-05-08T22:18:16.692 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-08T22:18:16.692 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-08T22:18:16.692 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2385, 2024-05-08T22:18:16.692 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.503778537, 2024-05-08T22:18:16.692 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000211227 2024-05-08T22:18:16.692 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:16.692 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.692 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-05-08T22:18:16.692 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 11233, 2024-05-08T22:18:16.692 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 11117, 2024-05-08T22:18:16.692 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-05-08T22:18:16.692 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 11117, 2024-05-08T22:18:16.692 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 174.453991543, 2024-05-08T22:18:16.692 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.015692542 2024-05-08T22:18:16.692 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.693 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-05-08T22:18:16.693 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 116, 2024-05-08T22:18:16.693 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 0, 2024-05-08T22:18:16.693 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-05-08T22:18:16.693 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 47, 2024-05-08T22:18:16.693 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 4, 2024-05-08T22:18:16.693 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-05-08T22:18:16.693 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 12, 2024-05-08T22:18:16.693 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 1, 2024-05-08T22:18:16.693 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 1, 2024-05-08T22:18:16.693 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 10, 2024-05-08T22:18:16.693 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 10, 2024-05-08T22:18:16.693 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 2914, 2024-05-08T22:18:16.693 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 0, 2024-05-08T22:18:16.693 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 0, 2024-05-08T22:18:16.694 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 2, 2024-05-08T22:18:16.694 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 20241, 2024-05-08T22:18:16.694 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 15961, 2024-05-08T22:18:16.694 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 116, 2024-05-08T22:18:16.694 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 10, 2024-05-08T22:18:16.694 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-05-08T22:18:16.694 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-05-08T22:18:16.694 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 175, 2024-05-08T22:18:16.694 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 0, 2024-05-08T22:18:16.694 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-05-08T22:18:16.694 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 32, 2024-05-08T22:18:16.694 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 2444, 2024-05-08T22:18:16.694 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 4, 2024-05-08T22:18:16.694 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 947, 2024-05-08T22:18:16.694 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-05-08T22:18:16.695 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 2359, 2024-05-08T22:18:16.695 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 0, 2024-05-08T22:18:16.695 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 4, 2024-05-08T22:18:16.695 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 234, 2024-05-08T22:18:16.695 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 6, 2024-05-08T22:18:16.695 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 2359, 2024-05-08T22:18:16.695 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-05-08T22:18:16.695 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:18:16.695 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:18:16.695 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-05-08T22:18:16.695 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T22:18:16.695 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T22:18:16.695 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 2359, 2024-05-08T22:18:16.695 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 50, 2024-05-08T22:18:16.696 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 985, 2024-05-08T22:18:16.696 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 2309, 2024-05-08T22:18:16.696 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T22:18:16.696 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:18:16.696 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:18:16.696 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 2384, 2024-05-08T22:18:16.696 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 1046, 2024-05-08T22:18:16.696 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 119, 2024-05-08T22:18:16.696 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 4490 2024-05-08T22:18:16.696 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.696 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-05-08T22:18:16.696 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 1, 2024-05-08T22:18:16.696 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 6, 2024-05-08T22:18:16.696 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 4, 2024-05-08T22:18:16.697 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 16, 2024-05-08T22:18:16.697 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 5, 2024-05-08T22:18:16.697 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-05-08T22:18:16.697 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-05-08T22:18:16.697 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:18:16.697 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 1241, 2024-05-08T22:18:16.697 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 1241, 2024-05-08T22:18:16.697 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-05-08T22:18:16.697 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-05-08T22:18:16.697 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-05-08T22:18:16.697 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:18:16.697 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:18:16.697 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-05-08T22:18:16.697 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-05-08T22:18:16.697 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T22:18:16.697 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 32, 2024-05-08T22:18:16.698 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-05-08T22:18:16.698 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 4, 2024-05-08T22:18:16.698 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-05-08T22:18:16.698 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0 2024-05-08T22:18:16.698 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.698 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-05-08T22:18:16.698 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 6023, 2024-05-08T22:18:16.698 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-05-08T22:18:16.698 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-05-08T22:18:16.698 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 6023, 2024-05-08T22:18:16.698 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-05-08T22:18:16.698 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-05-08T22:18:16.698 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 8, 2024-05-08T22:18:16.698 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-05-08T22:18:16.698 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-05-08T22:18:16.699 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 8, 2024-05-08T22:18:16.699 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-05-08T22:18:16.699 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-05-08T22:18:16.699 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-05-08T22:18:16.699 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 15973848, 2024-05-08T22:18:16.699 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 0, 2024-05-08T22:18:16.699 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-05-08T22:18:16.699 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 285, 2024-05-08T22:18:16.699 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 10.804288558, 2024-05-08T22:18:16.699 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.037909784 2024-05-08T22:18:16.699 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.699 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 0 2024-05-08T22:18:16.699 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.699 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-05-08T22:18:16.699 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 16, 2024-05-08T22:18:16.700 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 2751, 2024-05-08T22:18:16.700 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 2735, 2024-05-08T22:18:16.700 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 14, 2024-05-08T22:18:16.700 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 76, 2024-05-08T22:18:16.700 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 62, 2024-05-08T22:18:16.700 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 12, 2024-05-08T22:18:16.700 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 4286, 2024-05-08T22:18:16.700 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 4274, 2024-05-08T22:18:16.700 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 0, 2024-05-08T22:18:16.700 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 1776, 2024-05-08T22:18:16.700 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 1776, 2024-05-08T22:18:16.700 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 66160, 2024-05-08T22:18:16.700 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 198964 2024-05-08T22:18:16.700 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.700 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-05-08T22:18:16.700 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 11649, 2024-05-08T22:18:16.701 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 151, 2024-05-08T22:18:16.701 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 11233, 2024-05-08T22:18:16.701 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 20, 2024-05-08T22:18:16.701 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 378, 2024-05-08T22:18:16.701 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-05-08T22:18:16.701 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 843, 2024-05-08T22:18:16.701 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 7.775542136, 2024-05-08T22:18:16.701 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.009223656 2024-05-08T22:18:16.701 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.701 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-05-08T22:18:16.701 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2, 2024-05-08T22:18:16.701 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.002197564, 2024-05-08T22:18:16.701 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001098782 2024-05-08T22:18:16.701 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.701 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-05-08T22:18:16.702 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.702 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.702 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.702 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.702 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-05-08T22:18:16.702 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.702 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.702 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.702 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.702 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-05-08T22:18:16.702 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 7826, 2024-05-08T22:18:16.702 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 156.676336457, 2024-05-08T22:18:16.702 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.020019976 2024-05-08T22:18:16.702 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.702 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-05-08T22:18:16.703 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.703 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.703 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.703 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.703 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-05-08T22:18:16.703 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.703 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.703 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.703 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.703 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-05-08T22:18:16.703 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.703 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.703 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.703 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.703 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-05-08T22:18:16.703 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.704 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.704 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.704 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.704 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-05-08T22:18:16.704 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.704 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.704 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.704 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.704 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-05-08T22:18:16.704 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.704 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.704 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.704 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.704 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-05-08T22:18:16.704 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.704 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.705 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.705 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.705 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-05-08T22:18:16.705 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.705 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.705 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.705 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.705 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-05-08T22:18:16.705 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.705 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.705 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.705 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.705 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-05-08T22:18:16.705 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 879, 2024-05-08T22:18:16.706 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 2.812453056, 2024-05-08T22:18:16.706 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.003199605 2024-05-08T22:18:16.706 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.706 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-05-08T22:18:16.706 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 32, 2024-05-08T22:18:16.706 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.036653139, 2024-05-08T22:18:16.706 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001145410 2024-05-08T22:18:16.706 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.706 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-05-08T22:18:16.706 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 844, 2024-05-08T22:18:16.706 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 5.886511607, 2024-05-08T22:18:16.706 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.006974539 2024-05-08T22:18:16.706 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.706 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-05-08T22:18:16.707 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.707 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.707 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.707 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.707 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-05-08T22:18:16.707 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.707 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.707 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.707 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.707 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-05-08T22:18:16.707 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.707 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.707 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.707 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.708 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-05-08T22:18:16.708 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.708 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.708 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.708 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.708 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-05-08T22:18:16.708 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.708 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.708 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.708 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.708 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-05-08T22:18:16.708 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.708 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.708 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.708 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.709 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-05-08T22:18:16.709 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.709 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.709 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.709 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.709 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-05-08T22:18:16.709 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.709 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.709 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.709 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.709 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-05-08T22:18:16.709 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.709 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.709 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.709 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.709 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-05-08T22:18:16.710 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.710 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.710 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.710 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.710 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-05-08T22:18:16.710 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.710 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.710 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.710 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.710 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-05-08T22:18:16.710 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 691, 2024-05-08T22:18:16.710 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 1.264297584, 2024-05-08T22:18:16.710 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001829663 2024-05-08T22:18:16.710 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.710 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:18:16.711 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:18:16.711 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-05-08T22:18:16.711 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.711 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.711 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.711 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:16.711 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.711 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-05-08T22:18:16.711 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 1, 2024-05-08T22:18:16.711 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 4, 2024-05-08T22:18:16.711 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 3, 2024-05-08T22:18:16.711 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 1, 2024-05-08T22:18:16.711 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-05-08T22:18:16.711 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 1351, 2024-05-08T22:18:16.711 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 1351, 2024-05-08T22:18:16.712 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 2232, 2024-05-08T22:18:16.712 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-05-08T22:18:16.712 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.712 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-05-08T22:18:16.712 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:18:16.712 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-05-08T22:18:16.712 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:18:16.712 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:18:16.712 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:18:16.712 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:18:16.712 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:18:16.712 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:18:16.712 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:18:16.712 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:18:16.712 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:18:16.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:18:16.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:18:16.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:18:16.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:18:16.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-05-08T22:18:16.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:18:16.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-05-08T22:18:16.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:18:16.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:18:16.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:18:16.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:18:16.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:18:16.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:18:16.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:18:16.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-05-08T22:18:16.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:18:16.714 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:18:16.714 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:18:16.714 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-05-08T22:18:16.714 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-05-08T22:18:16.714 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-05-08T22:18:16.714 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:18:16.714 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:18:16.714 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:18:16.714 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:18:16.714 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 283101, 2024-05-08T22:18:16.714 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 117, 2024-05-08T22:18:16.714 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 616, 2024-05-08T22:18:16.714 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 7, 2024-05-08T22:18:16.714 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-05-08T22:18:16.715 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-05-08T22:18:16.715 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:18:16.715 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-05-08T22:18:16.715 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:18:16.715 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-05-08T22:18:16.715 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 10168, 2024-05-08T22:18:16.715 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 213, 2024-05-08T22:18:16.715 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:18:16.715 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:18:16.715 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-05-08T22:18:16.715 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-05-08T22:18:16.715 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 102952, 2024-05-08T22:18:16.715 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 382, 2024-05-08T22:18:16.715 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-05-08T22:18:16.715 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-05-08T22:18:16.716 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-05-08T22:18:16.716 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-05-08T22:18:16.716 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.716 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-05-08T22:18:16.716 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-05-08T22:18:16.716 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-05-08T22:18:16.716 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 5529, 2024-05-08T22:18:16.716 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 12814141, 2024-05-08T22:18:16.716 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 27, 2024-05-08T22:18:16.716 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 5529, 2024-05-08T22:18:16.716 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-05-08T22:18:16.716 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 5529, 2024-05-08T22:18:16.716 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 150.966225909, 2024-05-08T22:18:16.716 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.027304435 2024-05-08T22:18:16.716 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.716 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-05-08T22:18:16.717 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-05-08T22:18:16.717 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 5529, 2024-05-08T22:18:16.717 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 5687 2024-05-08T22:18:16.717 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.717 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 5529, 2024-05-08T22:18:16.717 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 2448, 2024-05-08T22:18:16.717 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 3081, 2024-05-08T22:18:16.717 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-05-08T22:18:16.717 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-05-08T22:18:16.717 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 0, 2024-05-08T22:18:16.717 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 10, 2024-05-08T22:18:16.717 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 89, 2024-05-08T22:18:16.717 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 374, 2024-05-08T22:18:16.717 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 83, 2024-05-08T22:18:16.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-05-08T22:18:16.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-05-08T22:18:16.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-05-08T22:18:16.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 4, 2024-05-08T22:18:16.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 2496, 2024-05-08T22:18:16.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-05-08T22:18:16.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-05-08T22:18:16.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-05-08T22:18:16.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 2359, 2024-05-08T22:18:16.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 20, 2024-05-08T22:18:16.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:18:16.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-05-08T22:18:16.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:18:16.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-05-08T22:18:16.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-05-08T22:18:16.719 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-05-08T22:18:16.719 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:18:16.719 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-05-08T22:18:16.719 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:18:16.719 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-05-08T22:18:16.719 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-05-08T22:18:16.719 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-05-08T22:18:16.719 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-05-08T22:18:16.719 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-05-08T22:18:16.719 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-05-08T22:18:16.719 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-05-08T22:18:16.719 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-05-08T22:18:16.719 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-05-08T22:18:16.719 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-05-08T22:18:16.719 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-05-08T22:18:16.720 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-05-08T22:18:16.720 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-05-08T22:18:16.720 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-05-08T22:18:16.720 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-05-08T22:18:16.720 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-05-08T22:18:16.720 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-05-08T22:18:16.720 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 98, 2024-05-08T22:18:16.720 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-05-08T22:18:16.720 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 15, 2024-05-08T22:18:16.720 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 12, 2024-05-08T22:18:16.720 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 12, 2024-05-08T22:18:16.720 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 0, 2024-05-08T22:18:16.720 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-05-08T22:18:16.720 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 202, 2024-05-08T22:18:16.720 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 0, 2024-05-08T22:18:16.721 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 50 2024-05-08T22:18:16.721 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.721 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-05-08T22:18:16.721 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 1, 2024-05-08T22:18:16.721 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:18:16.721 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-05-08T22:18:16.721 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-05-08T22:18:16.721 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.721 INFO:teuthology.orchestra.run.smithi131.stdout: "osd-slow-ops": { 2024-05-08T22:18:16.721 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 0 2024-05-08T22:18:16.721 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.721 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-05-08T22:18:16.721 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-05-08T22:18:16.721 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 16, 2024-05-08T22:18:16.721 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-05-08T22:18:16.722 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 8, 2024-05-08T22:18:16.722 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 1241, 2024-05-08T22:18:16.722 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-05-08T22:18:16.722 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.722 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:18:16.722 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-08T22:18:16.722 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-08T22:18:16.722 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-08T22:18:16.722 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 30239, 2024-05-08T22:18:16.722 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 9291939, 2024-05-08T22:18:16.722 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:16.722 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 30239, 2024-05-08T22:18:16.722 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-08T22:18:16.722 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-08T22:18:16.723 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 30239, 2024-05-08T22:18:16.723 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 9291939, 2024-05-08T22:18:16.723 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-08T22:18:16.723 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.723 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.723 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.723 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:16.723 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.723 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-05-08T22:18:16.723 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-08T22:18:16.723 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-08T22:18:16.723 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-08T22:18:16.723 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-08T22:18:16.723 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-08T22:18:16.723 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:16.724 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-08T22:18:16.724 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 5529, 2024-05-08T22:18:16.724 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 12970650, 2024-05-08T22:18:16.724 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 5529, 2024-05-08T22:18:16.724 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 12970650, 2024-05-08T22:18:16.724 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-08T22:18:16.724 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.724 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.724 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.725 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:16.725 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.725 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-05-08T22:18:16.725 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-08T22:18:16.725 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-05-08T22:18:16.725 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-08T22:18:16.725 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-08T22:18:16.725 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-08T22:18:16.725 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:16.725 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-08T22:18:16.725 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 5529, 2024-05-08T22:18:16.725 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 5529, 2024-05-08T22:18:16.725 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 5529, 2024-05-08T22:18:16.725 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 5529, 2024-05-08T22:18:16.725 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-08T22:18:16.725 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.726 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.726 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.726 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:16.726 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.726 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:18:16.726 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-08T22:18:16.726 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-08T22:18:16.726 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-08T22:18:16.726 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 1241, 2024-05-08T22:18:16.726 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 156509, 2024-05-08T22:18:16.726 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:16.726 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 1241, 2024-05-08T22:18:16.726 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-08T22:18:16.727 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-08T22:18:16.727 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 89, 2024-05-08T22:18:16.727 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 156509, 2024-05-08T22:18:16.727 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-08T22:18:16.727 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.727 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.727 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.727 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:16.727 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:16.727 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x5628d14080a0": { 2024-05-08T22:18:16.727 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-08T22:18:16.727 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-08T22:18:16.727 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-08T22:18:16.727 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 6023, 2024-05-08T22:18:16.727 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 11779544, 2024-05-08T22:18:16.728 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:16.728 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 6023, 2024-05-08T22:18:16.728 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-08T22:18:16.728 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-08T22:18:16.728 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 285, 2024-05-08T22:18:16.728 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 11779544, 2024-05-08T22:18:16.728 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-08T22:18:16.728 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:16.728 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:16.728 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:16.728 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:16.728 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:16.728 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:18:16.843 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:16 smithi132 ceph-mon[78180]: pgmap v342: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-08T22:18:17.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:16 smithi157 ceph-mon[77746]: pgmap v342: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-08T22:18:17.111 INFO:tasks.check_counter:Daemon mds.g mds.exported=32 2024-05-08T22:18:17.111 INFO:tasks.check_counter:Daemon mds.g mds.imported=4 2024-05-08T22:18:17.111 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=6 2024-05-08T22:18:17.111 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-05-08T22:18:17.111 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T22:18:17.111 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=4 2024-05-08T22:18:17.111 DEBUG:tasks.check_counter:Getting stats from j 2024-05-08T22:18:17.112 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-08T22:18:17.644 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:18:17 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:18:17.218+0000 7fecd6800700 -1 calc_pg_upmaps abort due to max <= 0 2024-05-08T22:18:17.644 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:18:17 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:18:17.218+0000 7fecd6800700 -1 calc_pg_upmaps abort due to max <= 0 2024-05-08T22:18:17.795 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:18:18.843 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:18 smithi132 ceph-mon[78180]: pgmap v343: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-08T22:18:18.894 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:18 smithi131 ceph-mon[37260]: pgmap v343: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-08T22:18:19.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:18 smithi157 ceph-mon[77746]: pgmap v343: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-08T22:18:19.522 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-08T22:18:19.522 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:18:19.522 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 0, 2024-05-08T22:18:19.522 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 2, 2024-05-08T22:18:19.522 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 0, 2024-05-08T22:18:19.522 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 0, 2024-05-08T22:18:19.522 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 4, 2024-05-08T22:18:19.522 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 0, 2024-05-08T22:18:19.522 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.003365095, 2024-05-08T22:18:19.522 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T22:18:19.523 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.000916784, 2024-05-08T22:18:19.523 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T22:18:19.523 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:19.523 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:19.523 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:19.523 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:19.523 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:19.523 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:19.523 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:19.523 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:19.523 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:19.523 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:19.523 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T22:18:19.523 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T22:18:19.523 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:19.524 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:18:19.524 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 227, 2024-05-08T22:18:19.524 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 217, 2024-05-08T22:18:19.524 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 1424976, 2024-05-08T22:18:19.524 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 123904, 2024-05-08T22:18:19.524 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 2, 2024-05-08T22:18:19.524 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 1, 2024-05-08T22:18:19.524 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.052248194, 2024-05-08T22:18:19.524 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.013586763, 2024-05-08T22:18:19.524 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.042649999, 2024-05-08T22:18:19.524 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.006571646, 2024-05-08T22:18:19.524 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:19.524 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 216, 2024-05-08T22:18:19.524 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.038218407, 2024-05-08T22:18:19.525 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000176937 2024-05-08T22:18:19.525 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:19.525 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:19.525 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:19.525 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:19.525 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:19.525 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:19.525 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 1424976, 2024-05-08T22:18:19.525 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 123904 2024-05-08T22:18:19.525 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:19.525 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:18:19.525 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 6, 2024-05-08T22:18:19.525 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 168, 2024-05-08T22:18:19.525 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 3312, 2024-05-08T22:18:19.525 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 182096, 2024-05-08T22:18:19.525 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 5, 2024-05-08T22:18:19.526 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 3, 2024-05-08T22:18:19.526 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.020517445, 2024-05-08T22:18:19.526 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.013973959, 2024-05-08T22:18:19.526 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.003503232, 2024-05-08T22:18:19.526 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T22:18:19.526 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:19.526 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 166, 2024-05-08T22:18:19.526 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.034665236, 2024-05-08T22:18:19.526 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000208826 2024-05-08T22:18:19.526 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:19.526 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:19.526 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:19.526 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:19.526 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:19.526 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:19.527 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-08T22:18:19.527 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 182096 2024-05-08T22:18:19.527 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:19.527 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-05-08T22:18:19.527 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 0, 2024-05-08T22:18:19.527 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-05-08T22:18:19.527 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:19.527 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-05-08T22:18:19.527 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:18:19.527 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-05-08T22:18:19.527 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:18:19.527 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:18:19.527 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:18:19.527 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:18:19.527 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:18:19.528 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:18:19.528 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:18:19.528 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:18:19.528 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:18:19.528 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:18:19.528 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:18:19.528 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:18:19.528 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:18:19.528 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-05-08T22:18:19.528 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:18:19.528 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-05-08T22:18:19.528 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:18:19.528 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:18:19.528 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:18:19.529 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:18:19.529 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:18:19.529 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:18:19.529 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:18:19.529 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-05-08T22:18:19.529 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:18:19.529 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:18:19.529 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:18:19.529 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-05-08T22:18:19.529 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-05-08T22:18:19.529 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-05-08T22:18:19.529 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:18:19.529 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:18:19.529 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:18:19.530 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:18:19.530 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 150048, 2024-05-08T22:18:19.530 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 32, 2024-05-08T22:18:19.530 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:18:19.530 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-05-08T22:18:19.530 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-05-08T22:18:19.530 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-05-08T22:18:19.530 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:18:19.530 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-05-08T22:18:19.530 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:18:19.530 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-05-08T22:18:19.530 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 0, 2024-05-08T22:18:19.530 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 0, 2024-05-08T22:18:19.530 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:18:19.531 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:18:19.531 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-05-08T22:18:19.531 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-05-08T22:18:19.531 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 928, 2024-05-08T22:18:19.531 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 2, 2024-05-08T22:18:19.531 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-05-08T22:18:19.531 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-05-08T22:18:19.531 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-05-08T22:18:19.531 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-05-08T22:18:19.531 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-08T22:18:19.531 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:18:19.531 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-08T22:18:19.531 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-08T22:18:19.531 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-08T22:18:19.531 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 229, 2024-05-08T22:18:19.531 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 1392943, 2024-05-08T22:18:19.532 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:19.532 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 229, 2024-05-08T22:18:19.532 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-08T22:18:19.532 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-08T22:18:19.532 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 229, 2024-05-08T22:18:19.532 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 1392943, 2024-05-08T22:18:19.532 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-08T22:18:19.532 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-08T22:18:19.532 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-08T22:18:19.532 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-08T22:18:19.532 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:19.532 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-08T22:18:19.532 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-08T22:18:19.949 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-08T22:18:19.949 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-08T22:18:19.949 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-05-08T22:18:19.950 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-05-08T22:18:19.950 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-05-08T22:18:19.950 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-05-08T22:18:19.950 DEBUG:tasks.check_counter:Getting stats from b 2024-05-08T22:18:19.950 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-08T22:18:20.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:20 smithi132 ceph-mon[78180]: pgmap v344: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 188 B/s wr, 7 op/s 2024-05-08T22:18:20.603 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:18:20.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:20 smithi131 ceph-mon[37260]: pgmap v344: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 188 B/s wr, 7 op/s 2024-05-08T22:18:21.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:20 smithi157 ceph-mon[77746]: pgmap v344: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 188 B/s wr, 7 op/s 2024-05-08T22:18:22.751 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-05-08T22:18:22.751 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:18:22.752 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 92528, 2024-05-08T22:18:22.752 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 91918, 2024-05-08T22:18:22.752 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 38291190, 2024-05-08T22:18:22.752 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 665694397, 2024-05-08T22:18:22.752 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 17, 2024-05-08T22:18:22.752 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 4, 2024-05-08T22:18:22.752 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 7.341806523, 2024-05-08T22:18:22.752 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 2.494084954, 2024-05-08T22:18:22.752 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 1.321186563, 2024-05-08T22:18:22.752 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 3.239758550, 2024-05-08T22:18:22.752 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:22.752 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 91918, 2024-05-08T22:18:22.752 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 96.023506782, 2024-05-08T22:18:22.753 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.001044664 2024-05-08T22:18:22.753 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.753 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:22.753 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 7470, 2024-05-08T22:18:22.753 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.001228479, 2024-05-08T22:18:22.753 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000164 2024-05-08T22:18:22.753 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.753 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 370400, 2024-05-08T22:18:22.753 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 98272 2024-05-08T22:18:22.753 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.753 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:18:22.753 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 115329, 2024-05-08T22:18:22.753 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 115930, 2024-05-08T22:18:22.753 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 45886357, 2024-05-08T22:18:22.753 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 456477522, 2024-05-08T22:18:22.754 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 25, 2024-05-08T22:18:22.754 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 6, 2024-05-08T22:18:22.754 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 7.840972609, 2024-05-08T22:18:22.754 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 2.325141260, 2024-05-08T22:18:22.754 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 1.511388840, 2024-05-08T22:18:22.754 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 3.639547046, 2024-05-08T22:18:22.754 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:22.754 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 115929, 2024-05-08T22:18:22.754 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 124.585184493, 2024-05-08T22:18:22.754 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.001074667 2024-05-08T22:18:22.754 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.754 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:22.755 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 5412, 2024-05-08T22:18:22.755 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.001001779, 2024-05-08T22:18:22.755 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000185 2024-05-08T22:18:22.755 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.755 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T22:18:22.755 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 448944 2024-05-08T22:18:22.755 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.755 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:18:22.755 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 454428, 2024-05-08T22:18:22.755 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 686877, 2024-05-08T22:18:22.755 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 159257629, 2024-05-08T22:18:22.755 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 561226616, 2024-05-08T22:18:22.755 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 110, 2024-05-08T22:18:22.755 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 10, 2024-05-08T22:18:22.755 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 22.019610391, 2024-05-08T22:18:22.755 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 10.096765709, 2024-05-08T22:18:22.756 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 11.784584839, 2024-05-08T22:18:22.756 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 3.051541054, 2024-05-08T22:18:22.756 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:22.756 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 686872, 2024-05-08T22:18:22.756 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 164.617211741, 2024-05-08T22:18:22.756 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000239662 2024-05-08T22:18:22.756 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.756 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:22.756 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 460371, 2024-05-08T22:18:22.756 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.112446813, 2024-05-08T22:18:22.756 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000244 2024-05-08T22:18:22.756 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.756 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 1651552, 2024-05-08T22:18:22.756 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 187376 2024-05-08T22:18:22.756 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.757 INFO:teuthology.orchestra.run.smithi132.stdout: "cct": { 2024-05-08T22:18:22.757 INFO:teuthology.orchestra.run.smithi132.stdout: "total_workers": 1, 2024-05-08T22:18:22.757 INFO:teuthology.orchestra.run.smithi132.stdout: "unhealthy_workers": 0 2024-05-08T22:18:22.757 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.757 INFO:teuthology.orchestra.run.smithi132.stdout: "finisher-MDSRank": { 2024-05-08T22:18:22.757 INFO:teuthology.orchestra.run.smithi132.stdout: "queue_len": 0, 2024-05-08T22:18:22.757 INFO:teuthology.orchestra.run.smithi132.stdout: "complete_latency": { 2024-05-08T22:18:22.757 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 59102, 2024-05-08T22:18:22.757 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 348.179437200, 2024-05-08T22:18:22.757 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.005891161 2024-05-08T22:18:22.757 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-05-08T22:18:22.757 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.757 INFO:teuthology.orchestra.run.smithi132.stdout: "finisher-PurgeQueue": { 2024-05-08T22:18:22.757 INFO:teuthology.orchestra.run.smithi132.stdout: "queue_len": 0, 2024-05-08T22:18:22.757 INFO:teuthology.orchestra.run.smithi132.stdout: "complete_latency": { 2024-05-08T22:18:22.757 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 100211, 2024-05-08T22:18:22.758 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 42.628009244, 2024-05-08T22:18:22.758 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000425382 2024-05-08T22:18:22.758 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-05-08T22:18:22.758 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.758 INFO:teuthology.orchestra.run.smithi132.stdout: "mds": { 2024-05-08T22:18:22.758 INFO:teuthology.orchestra.run.smithi132.stdout: "request": 194914, 2024-05-08T22:18:22.758 INFO:teuthology.orchestra.run.smithi132.stdout: "reply": 191044, 2024-05-08T22:18:22.758 INFO:teuthology.orchestra.run.smithi132.stdout: "reply_latency": { 2024-05-08T22:18:22.758 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 191044, 2024-05-08T22:18:22.758 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 6046.410344717, 2024-05-08T22:18:22.758 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.031649307 2024-05-08T22:18:22.758 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.758 INFO:teuthology.orchestra.run.smithi132.stdout: "slow_reply": 0, 2024-05-08T22:18:22.758 INFO:teuthology.orchestra.run.smithi132.stdout: "forward": 3870, 2024-05-08T22:18:22.758 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_fetch_complete": 5, 2024-05-08T22:18:22.758 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_fetch_keys": 2, 2024-05-08T22:18:22.759 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_commit": 3816, 2024-05-08T22:18:22.759 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_split": 192, 2024-05-08T22:18:22.759 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_merge": 27, 2024-05-08T22:18:22.759 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes": 89435, 2024-05-08T22:18:22.759 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_top": 29090, 2024-05-08T22:18:22.759 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_bottom": 60331, 2024-05-08T22:18:22.759 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_pin_tail": 14, 2024-05-08T22:18:22.759 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_pinned": 47877, 2024-05-08T22:18:22.759 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_expired": 3997, 2024-05-08T22:18:22.759 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_with_caps": 3, 2024-05-08T22:18:22.759 INFO:teuthology.orchestra.run.smithi132.stdout: "caps": 4, 2024-05-08T22:18:22.759 INFO:teuthology.orchestra.run.smithi132.stdout: "subtrees": 60, 2024-05-08T22:18:22.759 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse": 281311, 2024-05-08T22:18:22.759 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_hit": 260241, 2024-05-08T22:18:22.759 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_forward": 3851, 2024-05-08T22:18:22.760 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_discover": 10, 2024-05-08T22:18:22.760 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_dir_fetch": 0, 2024-05-08T22:18:22.760 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_remote_ino": 0, 2024-05-08T22:18:22.760 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_lock": 1341, 2024-05-08T22:18:22.760 INFO:teuthology.orchestra.run.smithi132.stdout: "load_cent": 0, 2024-05-08T22:18:22.760 INFO:teuthology.orchestra.run.smithi132.stdout: "q": 0, 2024-05-08T22:18:22.760 INFO:teuthology.orchestra.run.smithi132.stdout: "exported": 18, 2024-05-08T22:18:22.760 INFO:teuthology.orchestra.run.smithi132.stdout: "exported_inodes": 2508, 2024-05-08T22:18:22.760 INFO:teuthology.orchestra.run.smithi132.stdout: "imported": 88, 2024-05-08T22:18:22.760 INFO:teuthology.orchestra.run.smithi132.stdout: "imported_inodes": 6488, 2024-05-08T22:18:22.760 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_dir_fetch": 0, 2024-05-08T22:18:22.760 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_backtrace_fetch": 112763, 2024-05-08T22:18:22.760 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_peer_discover": 0, 2024-05-08T22:18:22.760 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rfiles": 4, 2024-05-08T22:18:22.760 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rbytes": 234, 2024-05-08T22:18:22.760 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rsnaps": 6, 2024-05-08T22:18:22.761 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_backtrace_fetch": 112763, 2024-05-08T22:18:22.761 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_set_tag": 0, 2024-05-08T22:18:22.761 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:18:22.761 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:18:22.761 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dir_inodes": 871, 2024-05-08T22:18:22.761 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dir_base_inodes": 13, 2024-05-08T22:18:22.761 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dirfrag_rstats": 858, 2024-05-08T22:18:22.761 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_file_inodes": 111892, 2024-05-08T22:18:22.761 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_inode_file_caps": 22, 2024-05-08T22:18:22.761 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_revoke": 57796, 2024-05-08T22:18:22.761 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_grant": 121646, 2024-05-08T22:18:22.761 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_trunc": 8, 2024-05-08T22:18:22.761 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:18:22.761 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:18:22.761 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_caps": 152803, 2024-05-08T22:18:22.761 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_caps_dirty": 59187, 2024-05-08T22:18:22.761 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_cap_release": 4563, 2024-05-08T22:18:22.762 INFO:teuthology.orchestra.run.smithi132.stdout: "process_request_cap_release": 122735 2024-05-08T22:18:22.762 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.762 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_cache": { 2024-05-08T22:18:22.762 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_update": 0, 2024-05-08T22:18:22.762 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_update_receipt": 0, 2024-05-08T22:18:22.762 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_try_discover": 0, 2024-05-08T22:18:22.762 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_send_discover": 10, 2024-05-08T22:18:22.762 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_handle_discover": 23, 2024-05-08T22:18:22.762 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays": 216, 2024-05-08T22:18:22.762 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays_delayed": 0, 2024-05-08T22:18:22.762 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:18:22.762 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_created": 57739, 2024-05-08T22:18:22.762 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_enqueued": 57523, 2024-05-08T22:18:22.762 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_reintegrated": 0, 2024-05-08T22:18:22.762 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_migrated": 0, 2024-05-08T22:18:22.762 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_processing": 0, 2024-05-08T22:18:22.763 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:18:22.763 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:18:22.763 INFO:teuthology.orchestra.run.smithi132.stdout: "recovery_started": 0, 2024-05-08T22:18:22.763 INFO:teuthology.orchestra.run.smithi132.stdout: "recovery_completed": 0, 2024-05-08T22:18:22.763 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_enqueue_scrub": 13, 2024-05-08T22:18:22.763 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_exportdir": 18, 2024-05-08T22:18:22.763 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_flush": 0, 2024-05-08T22:18:22.763 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_fragmentdir": 220, 2024-05-08T22:18:22.763 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_fragstats": 0, 2024-05-08T22:18:22.763 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_inodestats": 0 2024-05-08T22:18:22.763 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.763 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_log": { 2024-05-08T22:18:22.763 INFO:teuthology.orchestra.run.smithi132.stdout: "evadd": 326296, 2024-05-08T22:18:22.763 INFO:teuthology.orchestra.run.smithi132.stdout: "evex": 235136, 2024-05-08T22:18:22.764 INFO:teuthology.orchestra.run.smithi132.stdout: "evtrm": 235136, 2024-05-08T22:18:22.764 INFO:teuthology.orchestra.run.smithi132.stdout: "ev": 91160, 2024-05-08T22:18:22.764 INFO:teuthology.orchestra.run.smithi132.stdout: "evexg": 0, 2024-05-08T22:18:22.764 INFO:teuthology.orchestra.run.smithi132.stdout: "evexd": 2046, 2024-05-08T22:18:22.764 INFO:teuthology.orchestra.run.smithi132.stdout: "segadd": 464, 2024-05-08T22:18:22.764 INFO:teuthology.orchestra.run.smithi132.stdout: "segex": 336, 2024-05-08T22:18:22.764 INFO:teuthology.orchestra.run.smithi132.stdout: "segtrm": 336, 2024-05-08T22:18:22.764 INFO:teuthology.orchestra.run.smithi132.stdout: "seg": 131, 2024-05-08T22:18:22.764 INFO:teuthology.orchestra.run.smithi132.stdout: "segexg": 0, 2024-05-08T22:18:22.764 INFO:teuthology.orchestra.run.smithi132.stdout: "segexd": 3, 2024-05-08T22:18:22.764 INFO:teuthology.orchestra.run.smithi132.stdout: "expos": 847253757, 2024-05-08T22:18:22.764 INFO:teuthology.orchestra.run.smithi132.stdout: "wrpos": 1255524894, 2024-05-08T22:18:22.764 INFO:teuthology.orchestra.run.smithi132.stdout: "rdpos": 0, 2024-05-08T22:18:22.764 INFO:teuthology.orchestra.run.smithi132.stdout: "jlat": { 2024-05-08T22:18:22.764 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 3590, 2024-05-08T22:18:22.765 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 1071.624147906, 2024-05-08T22:18:22.765 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.298502548 2024-05-08T22:18:22.765 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.765 INFO:teuthology.orchestra.run.smithi132.stdout: "replayed": 0 2024-05-08T22:18:22.765 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.765 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_mem": { 2024-05-08T22:18:22.765 INFO:teuthology.orchestra.run.smithi132.stdout: "ino": 48908, 2024-05-08T22:18:22.765 INFO:teuthology.orchestra.run.smithi132.stdout: "ino+": 107584, 2024-05-08T22:18:22.765 INFO:teuthology.orchestra.run.smithi132.stdout: "ino-": 58676, 2024-05-08T22:18:22.765 INFO:teuthology.orchestra.run.smithi132.stdout: "dir": 1509, 2024-05-08T22:18:22.765 INFO:teuthology.orchestra.run.smithi132.stdout: "dir+": 1901, 2024-05-08T22:18:22.765 INFO:teuthology.orchestra.run.smithi132.stdout: "dir-": 392, 2024-05-08T22:18:22.765 INFO:teuthology.orchestra.run.smithi132.stdout: "dn": 89435, 2024-05-08T22:18:22.765 INFO:teuthology.orchestra.run.smithi132.stdout: "dn+": 181883, 2024-05-08T22:18:22.765 INFO:teuthology.orchestra.run.smithi132.stdout: "dn-": 92448, 2024-05-08T22:18:22.765 INFO:teuthology.orchestra.run.smithi132.stdout: "cap": 4, 2024-05-08T22:18:22.766 INFO:teuthology.orchestra.run.smithi132.stdout: "cap+": 59685, 2024-05-08T22:18:22.766 INFO:teuthology.orchestra.run.smithi132.stdout: "cap-": 59681, 2024-05-08T22:18:22.766 INFO:teuthology.orchestra.run.smithi132.stdout: "rss": 331412, 2024-05-08T22:18:22.766 INFO:teuthology.orchestra.run.smithi132.stdout: "heap": 207156 2024-05-08T22:18:22.766 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.766 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_server": { 2024-05-08T22:18:22.766 INFO:teuthology.orchestra.run.smithi132.stdout: "dispatch_client_request": 252645, 2024-05-08T22:18:22.766 INFO:teuthology.orchestra.run.smithi132.stdout: "dispatch_server_request": 202, 2024-05-08T22:18:22.766 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_request": 194914, 2024-05-08T22:18:22.766 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_session": 57, 2024-05-08T22:18:22.766 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_peer_request": 683, 2024-05-08T22:18:22.766 INFO:teuthology.orchestra.run.smithi132.stdout: "req_create_latency": { 2024-05-08T22:18:22.766 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 58641, 2024-05-08T22:18:22.766 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 819.913965595, 2024-05-08T22:18:22.766 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.013981923 2024-05-08T22:18:22.767 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.767 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getattr_latency": { 2024-05-08T22:18:22.767 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 4118, 2024-05-08T22:18:22.767 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 1526.347655642, 2024-05-08T22:18:22.767 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.370652660 2024-05-08T22:18:22.767 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.767 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getfilelock_latency": { 2024-05-08T22:18:22.767 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:22.767 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:22.767 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:22.767 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.767 INFO:teuthology.orchestra.run.smithi132.stdout: "req_link_latency": { 2024-05-08T22:18:22.767 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:22.767 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:22.768 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:22.768 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.768 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookup_latency": { 2024-05-08T22:18:22.768 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 20995, 2024-05-08T22:18:22.768 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 1525.072205528, 2024-05-08T22:18:22.768 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.072639781 2024-05-08T22:18:22.768 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.768 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookuphash_latency": { 2024-05-08T22:18:22.768 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:22.768 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:22.768 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:22.768 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.768 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupino_latency": { 2024-05-08T22:18:22.768 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 2, 2024-05-08T22:18:22.768 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000260637, 2024-05-08T22:18:22.768 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000130318 2024-05-08T22:18:22.769 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.769 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupname_latency": { 2024-05-08T22:18:22.769 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:22.769 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:22.769 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:22.769 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.769 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupparent_latency": { 2024-05-08T22:18:22.769 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:22.769 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:22.769 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:22.769 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.769 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupsnap_latency": { 2024-05-08T22:18:22.769 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:22.769 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:22.769 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:22.769 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.769 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lssnap_latency": { 2024-05-08T22:18:22.770 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 11, 2024-05-08T22:18:22.770 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.012080695, 2024-05-08T22:18:22.770 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.001098245 2024-05-08T22:18:22.770 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.770 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mkdir_latency": { 2024-05-08T22:18:22.770 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 229, 2024-05-08T22:18:22.770 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 10.022397037, 2024-05-08T22:18:22.770 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.043765925 2024-05-08T22:18:22.770 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.770 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mknod_latency": { 2024-05-08T22:18:22.770 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:22.770 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:22.770 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:22.771 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.771 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mksnap_latency": { 2024-05-08T22:18:22.771 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 11, 2024-05-08T22:18:22.771 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 7.430604169, 2024-05-08T22:18:22.771 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.675509469 2024-05-08T22:18:22.771 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.771 INFO:teuthology.orchestra.run.smithi132.stdout: "req_open_latency": { 2024-05-08T22:18:22.771 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 13958, 2024-05-08T22:18:22.771 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 230.328990919, 2024-05-08T22:18:22.771 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.016501575 2024-05-08T22:18:22.771 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.771 INFO:teuthology.orchestra.run.smithi132.stdout: "req_readdir_latency": { 2024-05-08T22:18:22.771 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 1465, 2024-05-08T22:18:22.771 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 7.839859717, 2024-05-08T22:18:22.772 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.005351440 2024-05-08T22:18:22.772 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.772 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rename_latency": { 2024-05-08T22:18:22.772 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 58248, 2024-05-08T22:18:22.772 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 1800.333786847, 2024-05-08T22:18:22.772 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.030908079 2024-05-08T22:18:22.772 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.772 INFO:teuthology.orchestra.run.smithi132.stdout: "req_renamesnap_latency": { 2024-05-08T22:18:22.772 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:22.772 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:22.772 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:22.772 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.772 INFO:teuthology.orchestra.run.smithi132.stdout: "req_snapdiff_latency": { 2024-05-08T22:18:22.772 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:22.772 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:22.772 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:22.773 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.773 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmdir_latency": { 2024-05-08T22:18:22.773 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 223, 2024-05-08T22:18:22.773 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 31.623760268, 2024-05-08T22:18:22.773 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.141810584 2024-05-08T22:18:22.773 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.773 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmsnap_latency": { 2024-05-08T22:18:22.773 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 5, 2024-05-08T22:18:22.773 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 10.256598563, 2024-05-08T22:18:22.773 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 2.051319712 2024-05-08T22:18:22.773 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.773 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmxattr_latency": { 2024-05-08T22:18:22.773 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:22.773 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:22.773 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:22.773 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.773 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setattr_latency": { 2024-05-08T22:18:22.774 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 10, 2024-05-08T22:18:22.774 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 1.777109912, 2024-05-08T22:18:22.774 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.177710991 2024-05-08T22:18:22.774 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.774 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setdirlayout_latency": { 2024-05-08T22:18:22.774 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:22.774 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:22.774 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:22.774 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.774 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setfilelock_latency": { 2024-05-08T22:18:22.774 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:22.774 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:22.774 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:22.774 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.774 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setlayout_latency": { 2024-05-08T22:18:22.774 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:22.775 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:22.775 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:22.775 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.775 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setxattr_latency": { 2024-05-08T22:18:22.775 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 9, 2024-05-08T22:18:22.775 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.023303963, 2024-05-08T22:18:22.775 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.002589329 2024-05-08T22:18:22.775 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.775 INFO:teuthology.orchestra.run.smithi132.stdout: "req_symlink_latency": { 2024-05-08T22:18:22.775 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 3, 2024-05-08T22:18:22.775 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.002478026, 2024-05-08T22:18:22.775 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000826008 2024-05-08T22:18:22.775 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.775 INFO:teuthology.orchestra.run.smithi132.stdout: "req_unlink_latency": { 2024-05-08T22:18:22.775 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 33116, 2024-05-08T22:18:22.776 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 75.425287199, 2024-05-08T22:18:22.776 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.002277608 2024-05-08T22:18:22.776 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.776 INFO:teuthology.orchestra.run.smithi132.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:18:22.776 INFO:teuthology.orchestra.run.smithi132.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:18:22.776 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getvxattr_latency": { 2024-05-08T22:18:22.776 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:22.776 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:22.776 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:22.776 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-05-08T22:18:22.776 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.776 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_sessions": { 2024-05-08T22:18:22.776 INFO:teuthology.orchestra.run.smithi132.stdout: "session_count": 2, 2024-05-08T22:18:22.777 INFO:teuthology.orchestra.run.smithi132.stdout: "session_add": 7, 2024-05-08T22:18:22.777 INFO:teuthology.orchestra.run.smithi132.stdout: "session_remove": 5, 2024-05-08T22:18:22.777 INFO:teuthology.orchestra.run.smithi132.stdout: "sessions_open": 2, 2024-05-08T22:18:22.777 INFO:teuthology.orchestra.run.smithi132.stdout: "sessions_stale": 0, 2024-05-08T22:18:22.777 INFO:teuthology.orchestra.run.smithi132.stdout: "total_load": 12198, 2024-05-08T22:18:22.777 INFO:teuthology.orchestra.run.smithi132.stdout: "average_load": 6099, 2024-05-08T22:18:22.777 INFO:teuthology.orchestra.run.smithi132.stdout: "avg_session_uptime": 640, 2024-05-08T22:18:22.777 INFO:teuthology.orchestra.run.smithi132.stdout: "mdthresh_evicted": 0 2024-05-08T22:18:22.777 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.777 INFO:teuthology.orchestra.run.smithi132.stdout: "mempool": { 2024-05-08T22:18:22.777 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:18:22.777 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_items": 0, 2024-05-08T22:18:22.777 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:18:22.777 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:18:22.777 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:18:22.778 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:18:22.778 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:18:22.778 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:18:22.778 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:18:22.778 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:18:22.778 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:18:22.778 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:18:22.778 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:18:22.778 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:18:22.778 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:18:22.778 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_items": 0, 2024-05-08T22:18:22.778 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:18:22.778 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_items": 0, 2024-05-08T22:18:22.778 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:18:22.778 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:18:22.779 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:18:22.779 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:18:22.779 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:18:22.779 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:18:22.779 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:18:22.779 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_items": 0, 2024-05-08T22:18:22.779 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:18:22.779 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:18:22.779 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:18:22.779 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_items": 0, 2024-05-08T22:18:22.779 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_bytes": 0, 2024-05-08T22:18:22.779 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_items": 0, 2024-05-08T22:18:22.779 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:18:22.779 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:18:22.779 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:18:22.780 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:18:22.780 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_bytes": 3693980, 2024-05-08T22:18:22.780 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_items": 48765, 2024-05-08T22:18:22.780 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:18:22.780 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_items": 0, 2024-05-08T22:18:22.780 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_bytes": 0, 2024-05-08T22:18:22.780 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_items": 0, 2024-05-08T22:18:22.780 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:18:22.780 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_items": 0, 2024-05-08T22:18:22.780 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:18:22.780 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_items": 0, 2024-05-08T22:18:22.780 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_bytes": 10168, 2024-05-08T22:18:22.780 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_items": 213, 2024-05-08T22:18:22.780 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:18:22.780 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:18:22.781 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_bytes": 0, 2024-05-08T22:18:22.781 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_items": 0, 2024-05-08T22:18:22.781 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_bytes": 179592869, 2024-05-08T22:18:22.781 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_items": 1885764, 2024-05-08T22:18:22.781 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_bytes": 0, 2024-05-08T22:18:22.781 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_items": 0, 2024-05-08T22:18:22.781 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_bytes": 0, 2024-05-08T22:18:22.781 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_items": 0 2024-05-08T22:18:22.781 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.781 INFO:teuthology.orchestra.run.smithi132.stdout: "objecter": { 2024-05-08T22:18:22.781 INFO:teuthology.orchestra.run.smithi132.stdout: "op_active": 0, 2024-05-08T22:18:22.781 INFO:teuthology.orchestra.run.smithi132.stdout: "op_laggy": 0, 2024-05-08T22:18:22.781 INFO:teuthology.orchestra.run.smithi132.stdout: "op_send": 291816, 2024-05-08T22:18:22.781 INFO:teuthology.orchestra.run.smithi132.stdout: "op_send_bytes": 1352235991, 2024-05-08T22:18:22.781 INFO:teuthology.orchestra.run.smithi132.stdout: "op_resend": 37, 2024-05-08T22:18:22.782 INFO:teuthology.orchestra.run.smithi132.stdout: "op_reply": 291816, 2024-05-08T22:18:22.782 INFO:teuthology.orchestra.run.smithi132.stdout: "op_latency": { 2024-05-08T22:18:22.782 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 291816, 2024-05-08T22:18:22.782 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 25555.840577613, 2024-05-08T22:18:22.782 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.087575186 2024-05-08T22:18:22.782 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.782 INFO:teuthology.orchestra.run.smithi132.stdout: "op_inflight": 0, 2024-05-08T22:18:22.782 INFO:teuthology.orchestra.run.smithi132.stdout: "oplen_avg": { 2024-05-08T22:18:22.782 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 291816, 2024-05-08T22:18:22.782 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 379469 2024-05-08T22:18:22.782 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.782 INFO:teuthology.orchestra.run.smithi132.stdout: "op": 291816, 2024-05-08T22:18:22.782 INFO:teuthology.orchestra.run.smithi132.stdout: "op_r": 113685, 2024-05-08T22:18:22.782 INFO:teuthology.orchestra.run.smithi132.stdout: "op_w": 178131, 2024-05-08T22:18:22.782 INFO:teuthology.orchestra.run.smithi132.stdout: "op_rmw": 0, 2024-05-08T22:18:22.782 INFO:teuthology.orchestra.run.smithi132.stdout: "op_pg": 0, 2024-05-08T22:18:22.783 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_stat": 2026, 2024-05-08T22:18:22.783 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_create": 52874, 2024-05-08T22:18:22.783 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_read": 902, 2024-05-08T22:18:22.783 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_write": 4483, 2024-05-08T22:18:22.783 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_writefull": 397, 2024-05-08T22:18:22.783 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_writesame": 0, 2024-05-08T22:18:22.783 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_append": 0, 2024-05-08T22:18:22.783 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_zero": 0, 2024-05-08T22:18:22.783 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_truncate": 192, 2024-05-08T22:18:22.783 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_delete": 116232, 2024-05-08T22:18:22.783 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_mapext": 0, 2024-05-08T22:18:22.783 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_sparse_read": 0, 2024-05-08T22:18:22.783 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_clonerange": 0, 2024-05-08T22:18:22.783 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_getxattr": 112783, 2024-05-08T22:18:22.783 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_setxattr": 79436, 2024-05-08T22:18:22.784 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:18:22.784 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_rmxattr": 0, 2024-05-08T22:18:22.784 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:18:22.784 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_call": 0, 2024-05-08T22:18:22.784 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_watch": 0, 2024-05-08T22:18:22.784 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_notify": 0, 2024-05-08T22:18:22.784 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:18:22.784 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_pgls": 0, 2024-05-08T22:18:22.784 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:18:22.784 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_other": 8, 2024-05-08T22:18:22.784 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_active": 0, 2024-05-08T22:18:22.784 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_send": 0, 2024-05-08T22:18:22.784 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_resend": 0, 2024-05-08T22:18:22.784 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_ping": 0, 2024-05-08T22:18:22.784 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_active": 0, 2024-05-08T22:18:22.785 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_send": 0, 2024-05-08T22:18:22.785 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_resend": 0, 2024-05-08T22:18:22.785 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_active": 0, 2024-05-08T22:18:22.785 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_send": 0, 2024-05-08T22:18:22.785 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_resend": 0, 2024-05-08T22:18:22.785 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_active": 0, 2024-05-08T22:18:22.785 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_send": 0, 2024-05-08T22:18:22.785 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_resend": 0, 2024-05-08T22:18:22.785 INFO:teuthology.orchestra.run.smithi132.stdout: "command_active": 0, 2024-05-08T22:18:22.785 INFO:teuthology.orchestra.run.smithi132.stdout: "command_send": 0, 2024-05-08T22:18:22.785 INFO:teuthology.orchestra.run.smithi132.stdout: "command_resend": 0, 2024-05-08T22:18:22.785 INFO:teuthology.orchestra.run.smithi132.stdout: "map_epoch": 98, 2024-05-08T22:18:22.785 INFO:teuthology.orchestra.run.smithi132.stdout: "map_full": 0, 2024-05-08T22:18:22.785 INFO:teuthology.orchestra.run.smithi132.stdout: "map_inc": 20, 2024-05-08T22:18:22.786 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_sessions": 12, 2024-05-08T22:18:22.786 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_session_open": 12, 2024-05-08T22:18:22.786 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_session_close": 0, 2024-05-08T22:18:22.786 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_laggy": 0, 2024-05-08T22:18:22.786 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_wr": 7831, 2024-05-08T22:18:22.786 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_rd": 14, 2024-05-08T22:18:22.786 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_del": 2291 2024-05-08T22:18:22.786 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.786 INFO:teuthology.orchestra.run.smithi132.stdout: "oft": { 2024-05-08T22:18:22.786 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_objs": 1, 2024-05-08T22:18:22.786 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:18:22.786 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_updates": 0, 2024-05-08T22:18:22.786 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_removes": 0 2024-05-08T22:18:22.786 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.787 INFO:teuthology.orchestra.run.smithi132.stdout: "osd-slow-ops": { 2024-05-08T22:18:22.787 INFO:teuthology.orchestra.run.smithi132.stdout: "slow_ops_count": 32 2024-05-08T22:18:22.787 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.787 INFO:teuthology.orchestra.run.smithi132.stdout: "purge_queue": { 2024-05-08T22:18:22.787 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_ops": 0, 2024-05-08T22:18:22.787 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_ops_high_water": 23, 2024-05-08T22:18:22.787 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing": 0, 2024-05-08T22:18:22.787 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_high_water": 9, 2024-05-08T22:18:22.787 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executed": 57523, 2024-05-08T22:18:22.787 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_item_in_journal": 0 2024-05-08T22:18:22.787 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.787 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:18:22.787 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-05-08T22:18:22.787 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2024-05-08T22:18:22.788 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-05-08T22:18:22.788 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 662281, 2024-05-08T22:18:22.788 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 189555773, 2024-05-08T22:18:22.788 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:22.788 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 662281, 2024-05-08T22:18:22.788 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2024-05-08T22:18:22.788 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2024-05-08T22:18:22.788 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 662281, 2024-05-08T22:18:22.788 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 189555773, 2024-05-08T22:18:22.788 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-05-08T22:18:22.788 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:22.788 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:22.788 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:22.788 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-05-08T22:18:22.788 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.789 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-objecter_bytes": { 2024-05-08T22:18:22.789 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-05-08T22:18:22.789 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2024-05-08T22:18:22.789 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-05-08T22:18:22.789 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2024-05-08T22:18:22.789 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2024-05-08T22:18:22.789 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:22.789 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2024-05-08T22:18:22.789 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 291816, 2024-05-08T22:18:22.789 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 1359540334, 2024-05-08T22:18:22.789 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 291816, 2024-05-08T22:18:22.789 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 1359540334, 2024-05-08T22:18:22.789 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-05-08T22:18:22.789 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:22.790 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:22.790 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:22.790 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-05-08T22:18:22.790 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.790 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-objecter_ops": { 2024-05-08T22:18:22.790 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-05-08T22:18:22.790 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 1024, 2024-05-08T22:18:22.790 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-05-08T22:18:22.790 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2024-05-08T22:18:22.790 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2024-05-08T22:18:22.790 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:22.790 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2024-05-08T22:18:22.790 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 291816, 2024-05-08T22:18:22.790 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 291816, 2024-05-08T22:18:22.791 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 291816, 2024-05-08T22:18:22.791 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 291816, 2024-05-08T22:18:22.791 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-05-08T22:18:22.791 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:22.791 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:22.791 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:22.791 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-05-08T22:18:22.791 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.791 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:18:22.791 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-05-08T22:18:22.791 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 3758096384, 2024-05-08T22:18:22.791 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-05-08T22:18:22.791 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 57523, 2024-05-08T22:18:22.791 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 7304431, 2024-05-08T22:18:22.791 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:22.791 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 57523, 2024-05-08T22:18:22.792 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2024-05-08T22:18:22.792 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2024-05-08T22:18:22.792 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 893, 2024-05-08T22:18:22.792 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 7304431, 2024-05-08T22:18:22.792 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-05-08T22:18:22.792 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:22.792 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:22.792 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:22.792 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-05-08T22:18:22.792 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:22.792 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-write_buf_throttle-0x55972a8e20a0": { 2024-05-08T22:18:22.792 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-05-08T22:18:22.792 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 3758096384, 2024-05-08T22:18:22.792 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-05-08T22:18:22.792 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 326296, 2024-05-08T22:18:22.793 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 1251330590, 2024-05-08T22:18:22.793 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:22.793 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 326296, 2024-05-08T22:18:22.793 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2024-05-08T22:18:22.793 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2024-05-08T22:18:22.793 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 3590, 2024-05-08T22:18:22.793 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 1251330590, 2024-05-08T22:18:22.793 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-05-08T22:18:22.793 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:22.793 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:22.793 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:22.793 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-05-08T22:18:22.793 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-05-08T22:18:22.793 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-05-08T22:18:22.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:22 smithi131 ceph-mon[37260]: pgmap v345: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T22:18:22.965 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:22 smithi132 ceph-mon[78180]: pgmap v345: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T22:18:23.027 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:22 smithi157 ceph-mon[77746]: pgmap v345: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T22:18:23.152 INFO:tasks.check_counter:Daemon mds.b mds.exported=18 2024-05-08T22:18:23.152 INFO:tasks.check_counter:Daemon mds.b mds.imported=88 2024-05-08T22:18:23.152 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-05-08T22:18:23.152 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-05-08T22:18:23.152 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-05-08T22:18:23.152 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=192 2024-05-08T22:18:23.152 DEBUG:tasks.check_counter:Getting stats from e 2024-05-08T22:18:23.153 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-08T22:18:23.607 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:18:24.685 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:24 smithi132 ceph-mon[78180]: pgmap v346: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-08T22:18:24.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:24 smithi131 ceph-mon[37260]: pgmap v346: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-08T22:18:24.973 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-05-08T22:18:24.973 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:18:24.973 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 227, 2024-05-08T22:18:24.973 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 217, 2024-05-08T22:18:24.974 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 1424976, 2024-05-08T22:18:24.974 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 123904, 2024-05-08T22:18:24.974 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 4, 2024-05-08T22:18:24.974 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 0, 2024-05-08T22:18:24.974 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.038464779, 2024-05-08T22:18:24.974 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.010800243, 2024-05-08T22:18:24.974 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.029370016, 2024-05-08T22:18:24.974 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.005075148, 2024-05-08T22:18:24.974 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:24.974 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 216, 2024-05-08T22:18:24.974 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.023152285, 2024-05-08T22:18:24.974 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000107186 2024-05-08T22:18:24.974 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:24.974 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:24.975 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:24.975 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:24.975 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:24.975 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:24.975 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 1424976, 2024-05-08T22:18:24.975 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 123904 2024-05-08T22:18:24.975 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:24.975 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:18:24.975 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 2, 2024-05-08T22:18:24.975 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 166, 2024-05-08T22:18:24.975 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 320, 2024-05-08T22:18:24.975 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 181648, 2024-05-08T22:18:24.975 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 5, 2024-05-08T22:18:24.975 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 3, 2024-05-08T22:18:24.976 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.008928068, 2024-05-08T22:18:24.976 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.005839705, 2024-05-08T22:18:24.976 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.001286413, 2024-05-08T22:18:24.976 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T22:18:24.976 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:24.976 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 163, 2024-05-08T22:18:24.976 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.039125090, 2024-05-08T22:18:24.976 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000240031 2024-05-08T22:18:24.976 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:24.976 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:24.976 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:24.976 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:24.976 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:24.977 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:24.977 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T22:18:24.977 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 181648 2024-05-08T22:18:24.977 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:24.977 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:18:24.977 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 4, 2024-05-08T22:18:24.977 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 4, 2024-05-08T22:18:24.977 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 2992, 2024-05-08T22:18:24.977 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 448, 2024-05-08T22:18:24.977 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 2, 2024-05-08T22:18:24.977 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 1, 2024-05-08T22:18:24.977 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.000748086, 2024-05-08T22:18:24.977 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.000043548, 2024-05-08T22:18:24.977 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.000687775, 2024-05-08T22:18:24.977 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T22:18:24.978 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:24.978 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 3, 2024-05-08T22:18:24.978 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000860041, 2024-05-08T22:18:24.978 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000286680 2024-05-08T22:18:24.978 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:24.978 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:24.978 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:24.978 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:24.978 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:24.978 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:24.978 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-08T22:18:24.978 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T22:18:24.978 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:24.978 INFO:teuthology.orchestra.run.smithi132.stdout: "cct": { 2024-05-08T22:18:24.979 INFO:teuthology.orchestra.run.smithi132.stdout: "total_workers": 0, 2024-05-08T22:18:24.979 INFO:teuthology.orchestra.run.smithi132.stdout: "unhealthy_workers": 0 2024-05-08T22:18:24.979 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:24.979 INFO:teuthology.orchestra.run.smithi132.stdout: "mempool": { 2024-05-08T22:18:24.979 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:18:24.979 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_items": 0, 2024-05-08T22:18:24.979 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:18:24.979 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:18:24.979 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:18:24.979 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:18:24.979 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:18:24.979 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:18:24.979 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:18:24.979 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:18:24.979 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:18:24.980 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:18:24.980 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:18:24.980 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:18:24.980 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:18:24.980 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_items": 0, 2024-05-08T22:18:24.980 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:18:24.980 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_items": 0, 2024-05-08T22:18:24.980 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:18:24.980 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:18:24.980 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:18:24.980 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:18:24.980 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:18:24.980 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:18:24.980 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:18:24.981 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_items": 0, 2024-05-08T22:18:24.981 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:18:24.981 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:18:24.981 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:18:24.981 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_items": 0, 2024-05-08T22:18:24.981 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_bytes": 0, 2024-05-08T22:18:24.981 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_items": 0, 2024-05-08T22:18:24.981 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:18:24.981 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:18:24.981 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:18:24.981 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:18:24.981 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_bytes": 149017, 2024-05-08T22:18:24.981 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_items": 31, 2024-05-08T22:18:24.981 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:18:24.981 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_items": 0, 2024-05-08T22:18:24.982 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_bytes": 0, 2024-05-08T22:18:24.982 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_items": 0, 2024-05-08T22:18:24.982 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:18:24.982 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_items": 0, 2024-05-08T22:18:24.982 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:18:24.982 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_items": 0, 2024-05-08T22:18:24.982 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_bytes": 0, 2024-05-08T22:18:24.982 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_items": 0, 2024-05-08T22:18:24.982 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:18:24.982 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:18:24.982 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_bytes": 0, 2024-05-08T22:18:24.982 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_items": 0, 2024-05-08T22:18:24.982 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_bytes": 928, 2024-05-08T22:18:24.982 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_items": 2, 2024-05-08T22:18:24.983 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_bytes": 0, 2024-05-08T22:18:24.983 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_items": 0, 2024-05-08T22:18:24.983 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_bytes": 0, 2024-05-08T22:18:24.983 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_items": 0 2024-05-08T22:18:24.983 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:24.983 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:18:24.983 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-05-08T22:18:24.983 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2024-05-08T22:18:24.983 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-05-08T22:18:24.983 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 229, 2024-05-08T22:18:24.983 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 1392943, 2024-05-08T22:18:24.983 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:24.983 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 229, 2024-05-08T22:18:24.983 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2024-05-08T22:18:24.983 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2024-05-08T22:18:24.984 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 229, 2024-05-08T22:18:24.984 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 1392943, 2024-05-08T22:18:24.984 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-05-08T22:18:24.984 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:24.984 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:24.984 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:24.984 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-05-08T22:18:24.984 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-05-08T22:18:24.984 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-05-08T22:18:25.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:24 smithi157 ceph-mon[77746]: pgmap v346: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-08T22:18:26.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:26 smithi131 ceph-mon[37260]: pgmap v347: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-08T22:18:27.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:26 smithi157 ceph-mon[77746]: pgmap v347: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-08T22:18:27.092 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:26 smithi132 ceph-mon[78180]: pgmap v347: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-08T22:18:27.783 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-05-08T22:18:27.784 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-05-08T22:18:27.784 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-05-08T22:18:27.784 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-05-08T22:18:27.784 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-05-08T22:18:27.784 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.e 2024-05-08T22:18:27.784 DEBUG:tasks.check_counter:Getting stats from h 2024-05-08T22:18:27.784 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-08T22:18:28.271 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:18:29.028 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:28 smithi157 ceph-mon[77746]: pgmap v348: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-08T22:18:29.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:28 smithi132 ceph-mon[78180]: pgmap v348: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-08T22:18:29.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:28 smithi131 ceph-mon[37260]: pgmap v348: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-08T22:18:31.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:30 smithi157 ceph-mon[77746]: pgmap v349: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-08T22:18:31.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:30 smithi132 ceph-mon[78180]: pgmap v349: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-08T22:18:31.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:30 smithi131 ceph-mon[37260]: pgmap v349: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-08T22:18:31.943 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-05-08T22:18:31.943 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:18:31.943 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 2102, 2024-05-08T22:18:31.944 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 2104, 2024-05-08T22:18:31.944 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 13185801, 2024-05-08T22:18:31.944 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 653729, 2024-05-08T22:18:31.944 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 6, 2024-05-08T22:18:31.944 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 2, 2024-05-08T22:18:31.944 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.157627786, 2024-05-08T22:18:31.944 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.044066283, 2024-05-08T22:18:31.944 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.047469424, 2024-05-08T22:18:31.944 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.053424147, 2024-05-08T22:18:31.944 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:31.944 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 2102, 2024-05-08T22:18:31.944 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.151417371, 2024-05-08T22:18:31.944 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000072034 2024-05-08T22:18:31.945 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.945 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:31.945 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.945 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.945 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.945 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.945 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T22:18:31.945 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T22:18:31.945 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.945 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:18:31.946 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 510, 2024-05-08T22:18:31.946 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 493, 2024-05-08T22:18:31.946 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 5264093, 2024-05-08T22:18:31.946 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 207335, 2024-05-08T22:18:31.946 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 4, 2024-05-08T22:18:31.946 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 3, 2024-05-08T22:18:31.946 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.063684472, 2024-05-08T22:18:31.946 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.016920064, 2024-05-08T22:18:31.946 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.040906751, 2024-05-08T22:18:31.946 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.012588366, 2024-05-08T22:18:31.947 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:31.947 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 492, 2024-05-08T22:18:31.947 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.057440308, 2024-05-08T22:18:31.947 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000116748 2024-05-08T22:18:31.947 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.947 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:31.947 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.947 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.947 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.947 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.947 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 1477792, 2024-05-08T22:18:31.947 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 127968 2024-05-08T22:18:31.947 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.948 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:18:31.948 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 7, 2024-05-08T22:18:31.948 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 169, 2024-05-08T22:18:31.948 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 3580, 2024-05-08T22:18:31.948 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 440199, 2024-05-08T22:18:31.948 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 6, 2024-05-08T22:18:31.948 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 4, 2024-05-08T22:18:31.948 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.009468397, 2024-05-08T22:18:31.948 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.005687073, 2024-05-08T22:18:31.948 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.002073723, 2024-05-08T22:18:31.948 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.000023973, 2024-05-08T22:18:31.948 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:31.949 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 167, 2024-05-08T22:18:31.949 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.047007425, 2024-05-08T22:18:31.949 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000281481 2024-05-08T22:18:31.949 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.949 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:31.949 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.949 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.949 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.949 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.949 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-08T22:18:31.949 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 439888 2024-05-08T22:18:31.949 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.949 INFO:teuthology.orchestra.run.smithi132.stdout: "cct": { 2024-05-08T22:18:31.950 INFO:teuthology.orchestra.run.smithi132.stdout: "total_workers": 1, 2024-05-08T22:18:31.950 INFO:teuthology.orchestra.run.smithi132.stdout: "unhealthy_workers": 0 2024-05-08T22:18:31.950 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.950 INFO:teuthology.orchestra.run.smithi132.stdout: "finisher-MDSRank": { 2024-05-08T22:18:31.950 INFO:teuthology.orchestra.run.smithi132.stdout: "queue_len": 0, 2024-05-08T22:18:31.950 INFO:teuthology.orchestra.run.smithi132.stdout: "complete_latency": { 2024-05-08T22:18:31.950 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 3996, 2024-05-08T22:18:31.950 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.091060161, 2024-05-08T22:18:31.950 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000022787 2024-05-08T22:18:31.950 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-05-08T22:18:31.950 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.950 INFO:teuthology.orchestra.run.smithi132.stdout: "finisher-PurgeQueue": { 2024-05-08T22:18:31.951 INFO:teuthology.orchestra.run.smithi132.stdout: "queue_len": 0, 2024-05-08T22:18:31.951 INFO:teuthology.orchestra.run.smithi132.stdout: "complete_latency": { 2024-05-08T22:18:31.951 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.951 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.951 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.951 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-05-08T22:18:31.951 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.951 INFO:teuthology.orchestra.run.smithi132.stdout: "mds": { 2024-05-08T22:18:31.951 INFO:teuthology.orchestra.run.smithi132.stdout: "request": 0, 2024-05-08T22:18:31.951 INFO:teuthology.orchestra.run.smithi132.stdout: "reply": 0, 2024-05-08T22:18:31.951 INFO:teuthology.orchestra.run.smithi132.stdout: "reply_latency": { 2024-05-08T22:18:31.951 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.952 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.952 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.952 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.952 INFO:teuthology.orchestra.run.smithi132.stdout: "slow_reply": 0, 2024-05-08T22:18:31.952 INFO:teuthology.orchestra.run.smithi132.stdout: "forward": 0, 2024-05-08T22:18:31.952 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_fetch_complete": 0, 2024-05-08T22:18:31.952 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_fetch_keys": 0, 2024-05-08T22:18:31.952 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_commit": 0, 2024-05-08T22:18:31.952 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_split": 0, 2024-05-08T22:18:31.952 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_merge": 0, 2024-05-08T22:18:31.952 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes": 1654, 2024-05-08T22:18:31.952 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_top": 0, 2024-05-08T22:18:31.953 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_bottom": 1654, 2024-05-08T22:18:31.953 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_pin_tail": 0, 2024-05-08T22:18:31.953 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_pinned": 1654, 2024-05-08T22:18:31.953 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_expired": 0, 2024-05-08T22:18:31.953 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_with_caps": 0, 2024-05-08T22:18:31.953 INFO:teuthology.orchestra.run.smithi132.stdout: "caps": 0, 2024-05-08T22:18:31.953 INFO:teuthology.orchestra.run.smithi132.stdout: "subtrees": 1, 2024-05-08T22:18:31.953 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse": 0, 2024-05-08T22:18:31.953 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_hit": 0, 2024-05-08T22:18:31.953 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_forward": 0, 2024-05-08T22:18:31.953 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_discover": 0, 2024-05-08T22:18:31.953 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_dir_fetch": 0, 2024-05-08T22:18:31.954 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_remote_ino": 0, 2024-05-08T22:18:31.954 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_lock": 0, 2024-05-08T22:18:31.954 INFO:teuthology.orchestra.run.smithi132.stdout: "load_cent": 0, 2024-05-08T22:18:31.954 INFO:teuthology.orchestra.run.smithi132.stdout: "q": 0, 2024-05-08T22:18:31.954 INFO:teuthology.orchestra.run.smithi132.stdout: "exported": 0, 2024-05-08T22:18:31.954 INFO:teuthology.orchestra.run.smithi132.stdout: "exported_inodes": 0, 2024-05-08T22:18:31.954 INFO:teuthology.orchestra.run.smithi132.stdout: "imported": 0, 2024-05-08T22:18:31.954 INFO:teuthology.orchestra.run.smithi132.stdout: "imported_inodes": 0, 2024-05-08T22:18:31.954 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_dir_fetch": 0, 2024-05-08T22:18:31.954 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_backtrace_fetch": 0, 2024-05-08T22:18:31.954 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_peer_discover": 0, 2024-05-08T22:18:31.954 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rfiles": 0, 2024-05-08T22:18:31.954 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rbytes": 0, 2024-05-08T22:18:31.955 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rsnaps": 0, 2024-05-08T22:18:31.955 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T22:18:31.955 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_set_tag": 0, 2024-05-08T22:18:31.955 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:18:31.955 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:18:31.955 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dir_inodes": 0, 2024-05-08T22:18:31.955 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T22:18:31.955 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T22:18:31.955 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_file_inodes": 0, 2024-05-08T22:18:31.955 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_inode_file_caps": 0, 2024-05-08T22:18:31.955 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T22:18:31.955 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_grant": 0, 2024-05-08T22:18:31.956 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T22:18:31.956 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:18:31.956 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:18:31.956 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_caps": 0, 2024-05-08T22:18:31.956 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_caps_dirty": 0, 2024-05-08T22:18:31.956 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_cap_release": 0, 2024-05-08T22:18:31.956 INFO:teuthology.orchestra.run.smithi132.stdout: "process_request_cap_release": 0 2024-05-08T22:18:31.956 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.956 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_cache": { 2024-05-08T22:18:31.956 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_update": 0, 2024-05-08T22:18:31.956 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_update_receipt": 0, 2024-05-08T22:18:31.956 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_try_discover": 0, 2024-05-08T22:18:31.957 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_send_discover": 0, 2024-05-08T22:18:31.957 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_handle_discover": 0, 2024-05-08T22:18:31.957 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays": 0, 2024-05-08T22:18:31.957 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays_delayed": 0, 2024-05-08T22:18:31.957 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:18:31.957 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_created": 0, 2024-05-08T22:18:31.957 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_enqueued": 0, 2024-05-08T22:18:31.957 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_reintegrated": 0, 2024-05-08T22:18:31.957 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_migrated": 0, 2024-05-08T22:18:31.957 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_processing": 0, 2024-05-08T22:18:31.957 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:18:31.957 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:18:31.958 INFO:teuthology.orchestra.run.smithi132.stdout: "recovery_started": 0, 2024-05-08T22:18:31.958 INFO:teuthology.orchestra.run.smithi132.stdout: "recovery_completed": 0, 2024-05-08T22:18:31.958 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T22:18:31.958 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_exportdir": 0, 2024-05-08T22:18:31.958 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_flush": 0, 2024-05-08T22:18:31.958 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_fragmentdir": 0, 2024-05-08T22:18:31.958 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_fragstats": 0, 2024-05-08T22:18:31.958 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_inodestats": 0 2024-05-08T22:18:31.958 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.958 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_log": { 2024-05-08T22:18:31.958 INFO:teuthology.orchestra.run.smithi132.stdout: "evadd": 0, 2024-05-08T22:18:31.958 INFO:teuthology.orchestra.run.smithi132.stdout: "evex": 0, 2024-05-08T22:18:31.959 INFO:teuthology.orchestra.run.smithi132.stdout: "evtrm": 0, 2024-05-08T22:18:31.959 INFO:teuthology.orchestra.run.smithi132.stdout: "ev": 8364, 2024-05-08T22:18:31.959 INFO:teuthology.orchestra.run.smithi132.stdout: "evexg": 0, 2024-05-08T22:18:31.959 INFO:teuthology.orchestra.run.smithi132.stdout: "evexd": 0, 2024-05-08T22:18:31.959 INFO:teuthology.orchestra.run.smithi132.stdout: "segadd": 0, 2024-05-08T22:18:31.959 INFO:teuthology.orchestra.run.smithi132.stdout: "segex": 0, 2024-05-08T22:18:31.959 INFO:teuthology.orchestra.run.smithi132.stdout: "segtrm": 0, 2024-05-08T22:18:31.959 INFO:teuthology.orchestra.run.smithi132.stdout: "seg": 10, 2024-05-08T22:18:31.959 INFO:teuthology.orchestra.run.smithi132.stdout: "segexg": 0, 2024-05-08T22:18:31.959 INFO:teuthology.orchestra.run.smithi132.stdout: "segexd": 0, 2024-05-08T22:18:31.959 INFO:teuthology.orchestra.run.smithi132.stdout: "expos": 4194304, 2024-05-08T22:18:31.959 INFO:teuthology.orchestra.run.smithi132.stdout: "wrpos": 20497971, 2024-05-08T22:18:31.959 INFO:teuthology.orchestra.run.smithi132.stdout: "rdpos": 20492572, 2024-05-08T22:18:31.960 INFO:teuthology.orchestra.run.smithi132.stdout: "jlat": { 2024-05-08T22:18:31.960 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.960 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.960 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.960 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.960 INFO:teuthology.orchestra.run.smithi132.stdout: "replayed": 8364 2024-05-08T22:18:31.960 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.960 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_mem": { 2024-05-08T22:18:31.960 INFO:teuthology.orchestra.run.smithi132.stdout: "ino": 13, 2024-05-08T22:18:31.960 INFO:teuthology.orchestra.run.smithi132.stdout: "ino+": 3900, 2024-05-08T22:18:31.960 INFO:teuthology.orchestra.run.smithi132.stdout: "ino-": 3887, 2024-05-08T22:18:31.960 INFO:teuthology.orchestra.run.smithi132.stdout: "dir": 11, 2024-05-08T22:18:31.961 INFO:teuthology.orchestra.run.smithi132.stdout: "dir+": 110, 2024-05-08T22:18:31.961 INFO:teuthology.orchestra.run.smithi132.stdout: "dir-": 99, 2024-05-08T22:18:31.961 INFO:teuthology.orchestra.run.smithi132.stdout: "dn": 1654, 2024-05-08T22:18:31.961 INFO:teuthology.orchestra.run.smithi132.stdout: "dn+": 6550, 2024-05-08T22:18:31.961 INFO:teuthology.orchestra.run.smithi132.stdout: "dn-": 4896, 2024-05-08T22:18:31.961 INFO:teuthology.orchestra.run.smithi132.stdout: "cap": 0, 2024-05-08T22:18:31.961 INFO:teuthology.orchestra.run.smithi132.stdout: "cap+": 0, 2024-05-08T22:18:31.961 INFO:teuthology.orchestra.run.smithi132.stdout: "cap-": 0, 2024-05-08T22:18:31.961 INFO:teuthology.orchestra.run.smithi132.stdout: "rss": 58740, 2024-05-08T22:18:31.961 INFO:teuthology.orchestra.run.smithi132.stdout: "heap": 207156 2024-05-08T22:18:31.961 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.961 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_server": { 2024-05-08T22:18:31.961 INFO:teuthology.orchestra.run.smithi132.stdout: "dispatch_client_request": 0, 2024-05-08T22:18:31.962 INFO:teuthology.orchestra.run.smithi132.stdout: "dispatch_server_request": 0, 2024-05-08T22:18:31.962 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_request": 0, 2024-05-08T22:18:31.962 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_session": 0, 2024-05-08T22:18:31.962 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_peer_request": 0, 2024-05-08T22:18:31.962 INFO:teuthology.orchestra.run.smithi132.stdout: "req_create_latency": { 2024-05-08T22:18:31.962 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.962 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.962 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.962 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.962 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getattr_latency": { 2024-05-08T22:18:31.962 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.963 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.963 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.963 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.963 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getfilelock_latency": { 2024-05-08T22:18:31.963 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.963 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.963 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.963 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.963 INFO:teuthology.orchestra.run.smithi132.stdout: "req_link_latency": { 2024-05-08T22:18:31.963 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.963 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.963 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.964 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.964 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookup_latency": { 2024-05-08T22:18:31.964 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.964 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.964 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.964 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.964 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookuphash_latency": { 2024-05-08T22:18:31.964 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.964 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.964 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.964 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.964 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupino_latency": { 2024-05-08T22:18:31.965 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.965 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.965 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.965 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.965 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupname_latency": { 2024-05-08T22:18:31.965 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.965 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.965 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.965 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.965 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupparent_latency": { 2024-05-08T22:18:31.965 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.965 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.966 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.966 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.966 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupsnap_latency": { 2024-05-08T22:18:31.966 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.966 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.966 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.966 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.966 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lssnap_latency": { 2024-05-08T22:18:31.966 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.966 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.966 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.967 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.967 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mkdir_latency": { 2024-05-08T22:18:31.967 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.967 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.967 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.967 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.967 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mknod_latency": { 2024-05-08T22:18:31.967 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.967 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.967 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.967 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.968 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mksnap_latency": { 2024-05-08T22:18:31.968 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.968 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.968 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.968 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.968 INFO:teuthology.orchestra.run.smithi132.stdout: "req_open_latency": { 2024-05-08T22:18:31.968 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.968 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.968 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.968 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.968 INFO:teuthology.orchestra.run.smithi132.stdout: "req_readdir_latency": { 2024-05-08T22:18:31.968 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.969 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.969 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.969 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.969 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rename_latency": { 2024-05-08T22:18:31.969 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.969 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.969 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.969 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.969 INFO:teuthology.orchestra.run.smithi132.stdout: "req_renamesnap_latency": { 2024-05-08T22:18:31.969 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.969 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.969 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.970 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.970 INFO:teuthology.orchestra.run.smithi132.stdout: "req_snapdiff_latency": { 2024-05-08T22:18:31.970 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.970 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.970 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.970 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.970 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmdir_latency": { 2024-05-08T22:18:31.970 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.970 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.970 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.970 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.970 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmsnap_latency": { 2024-05-08T22:18:31.971 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.971 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.971 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.971 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.971 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmxattr_latency": { 2024-05-08T22:18:31.971 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.971 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.971 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.971 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.971 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setattr_latency": { 2024-05-08T22:18:31.972 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.972 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.972 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.972 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.972 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setdirlayout_latency": { 2024-05-08T22:18:31.972 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.972 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.972 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.972 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.972 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setfilelock_latency": { 2024-05-08T22:18:31.972 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.972 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.973 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.973 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.973 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setlayout_latency": { 2024-05-08T22:18:31.973 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.973 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.973 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.973 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.973 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setxattr_latency": { 2024-05-08T22:18:31.973 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.973 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.973 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.974 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.974 INFO:teuthology.orchestra.run.smithi132.stdout: "req_symlink_latency": { 2024-05-08T22:18:31.974 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.974 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.974 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.974 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.974 INFO:teuthology.orchestra.run.smithi132.stdout: "req_unlink_latency": { 2024-05-08T22:18:31.974 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.974 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.974 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.974 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.974 INFO:teuthology.orchestra.run.smithi132.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:18:31.975 INFO:teuthology.orchestra.run.smithi132.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:18:31.975 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getvxattr_latency": { 2024-05-08T22:18:31.975 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.975 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.975 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.975 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-05-08T22:18:31.975 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.975 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_sessions": { 2024-05-08T22:18:31.975 INFO:teuthology.orchestra.run.smithi132.stdout: "session_count": 2, 2024-05-08T22:18:31.975 INFO:teuthology.orchestra.run.smithi132.stdout: "session_add": 4, 2024-05-08T22:18:31.975 INFO:teuthology.orchestra.run.smithi132.stdout: "session_remove": 2, 2024-05-08T22:18:31.975 INFO:teuthology.orchestra.run.smithi132.stdout: "sessions_open": 3, 2024-05-08T22:18:31.976 INFO:teuthology.orchestra.run.smithi132.stdout: "sessions_stale": 0, 2024-05-08T22:18:31.976 INFO:teuthology.orchestra.run.smithi132.stdout: "total_load": 0, 2024-05-08T22:18:31.976 INFO:teuthology.orchestra.run.smithi132.stdout: "average_load": 0, 2024-05-08T22:18:31.976 INFO:teuthology.orchestra.run.smithi132.stdout: "avg_session_uptime": 3644, 2024-05-08T22:18:31.976 INFO:teuthology.orchestra.run.smithi132.stdout: "mdthresh_evicted": 0 2024-05-08T22:18:31.976 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.976 INFO:teuthology.orchestra.run.smithi132.stdout: "mempool": { 2024-05-08T22:18:31.976 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:18:31.976 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_items": 0, 2024-05-08T22:18:31.976 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:18:31.976 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:18:31.976 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:18:31.977 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:18:31.977 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:18:31.977 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:18:31.977 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:18:31.977 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:18:31.977 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:18:31.977 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:18:31.977 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:18:31.977 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:18:31.977 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:18:31.977 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_items": 0, 2024-05-08T22:18:31.977 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:18:31.977 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_items": 0, 2024-05-08T22:18:31.978 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:18:31.978 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:18:31.978 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:18:31.978 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:18:31.978 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:18:31.978 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:18:31.978 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:18:31.978 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_items": 0, 2024-05-08T22:18:31.978 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:18:31.978 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:18:31.978 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:18:31.978 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_items": 0, 2024-05-08T22:18:31.979 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_bytes": 0, 2024-05-08T22:18:31.979 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_items": 0, 2024-05-08T22:18:31.979 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:18:31.979 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:18:31.979 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:18:31.979 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:18:31.979 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_bytes": 190177, 2024-05-08T22:18:31.979 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_items": 53, 2024-05-08T22:18:31.979 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:18:31.979 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_items": 0, 2024-05-08T22:18:31.979 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_bytes": 0, 2024-05-08T22:18:31.979 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_items": 0, 2024-05-08T22:18:31.980 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:18:31.980 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_items": 0, 2024-05-08T22:18:31.980 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:18:31.980 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_items": 0, 2024-05-08T22:18:31.980 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_bytes": 10168, 2024-05-08T22:18:31.980 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_items": 213, 2024-05-08T22:18:31.980 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:18:31.980 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:18:31.980 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_bytes": 0, 2024-05-08T22:18:31.980 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_items": 0, 2024-05-08T22:18:31.980 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_bytes": 1226832, 2024-05-08T22:18:31.980 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_items": 6780, 2024-05-08T22:18:31.981 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_bytes": 0, 2024-05-08T22:18:31.981 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_items": 0, 2024-05-08T22:18:31.981 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_bytes": 0, 2024-05-08T22:18:31.981 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_items": 0 2024-05-08T22:18:31.981 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.981 INFO:teuthology.orchestra.run.smithi132.stdout: "objecter": { 2024-05-08T22:18:31.981 INFO:teuthology.orchestra.run.smithi132.stdout: "op_active": 0, 2024-05-08T22:18:31.981 INFO:teuthology.orchestra.run.smithi132.stdout: "op_laggy": 0, 2024-05-08T22:18:31.981 INFO:teuthology.orchestra.run.smithi132.stdout: "op_send": 2358, 2024-05-08T22:18:31.981 INFO:teuthology.orchestra.run.smithi132.stdout: "op_send_bytes": 21, 2024-05-08T22:18:31.981 INFO:teuthology.orchestra.run.smithi132.stdout: "op_resend": 3, 2024-05-08T22:18:31.981 INFO:teuthology.orchestra.run.smithi132.stdout: "op_reply": 2358, 2024-05-08T22:18:31.982 INFO:teuthology.orchestra.run.smithi132.stdout: "op_latency": { 2024-05-08T22:18:31.982 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 2358, 2024-05-08T22:18:31.982 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 5.245858782, 2024-05-08T22:18:31.982 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.002224706 2024-05-08T22:18:31.982 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.982 INFO:teuthology.orchestra.run.smithi132.stdout: "op_inflight": 0, 2024-05-08T22:18:31.982 INFO:teuthology.orchestra.run.smithi132.stdout: "oplen_avg": { 2024-05-08T22:18:31.982 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 2358, 2024-05-08T22:18:31.982 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 2359 2024-05-08T22:18:31.982 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.982 INFO:teuthology.orchestra.run.smithi132.stdout: "op": 2358, 2024-05-08T22:18:31.983 INFO:teuthology.orchestra.run.smithi132.stdout: "op_r": 2358, 2024-05-08T22:18:31.983 INFO:teuthology.orchestra.run.smithi132.stdout: "op_w": 0, 2024-05-08T22:18:31.983 INFO:teuthology.orchestra.run.smithi132.stdout: "op_rmw": 0, 2024-05-08T22:18:31.983 INFO:teuthology.orchestra.run.smithi132.stdout: "op_pg": 0, 2024-05-08T22:18:31.983 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_stat": 1436, 2024-05-08T22:18:31.983 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_create": 0, 2024-05-08T22:18:31.983 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_read": 920, 2024-05-08T22:18:31.983 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_write": 0, 2024-05-08T22:18:31.983 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_writefull": 0, 2024-05-08T22:18:31.983 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_writesame": 0, 2024-05-08T22:18:31.983 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_append": 0, 2024-05-08T22:18:31.983 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_zero": 0, 2024-05-08T22:18:31.983 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_truncate": 0, 2024-05-08T22:18:31.983 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_delete": 0, 2024-05-08T22:18:31.984 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_mapext": 0, 2024-05-08T22:18:31.984 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_sparse_read": 0, 2024-05-08T22:18:31.984 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_clonerange": 0, 2024-05-08T22:18:31.984 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_getxattr": 1, 2024-05-08T22:18:31.984 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_setxattr": 0, 2024-05-08T22:18:31.984 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:18:31.984 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_rmxattr": 0, 2024-05-08T22:18:31.984 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:18:31.984 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_call": 0, 2024-05-08T22:18:31.984 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_watch": 0, 2024-05-08T22:18:31.984 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_notify": 0, 2024-05-08T22:18:31.984 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:18:31.984 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_pgls": 0, 2024-05-08T22:18:31.985 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:18:31.985 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_other": 0, 2024-05-08T22:18:31.985 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_active": 0, 2024-05-08T22:18:31.985 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_send": 0, 2024-05-08T22:18:31.985 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_resend": 0, 2024-05-08T22:18:31.985 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_ping": 0, 2024-05-08T22:18:31.985 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_active": 0, 2024-05-08T22:18:31.985 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_send": 0, 2024-05-08T22:18:31.985 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_resend": 0, 2024-05-08T22:18:31.985 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_active": 0, 2024-05-08T22:18:31.985 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_send": 0, 2024-05-08T22:18:31.985 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_resend": 0, 2024-05-08T22:18:31.985 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_active": 0, 2024-05-08T22:18:31.986 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_send": 0, 2024-05-08T22:18:31.986 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_resend": 0, 2024-05-08T22:18:31.986 INFO:teuthology.orchestra.run.smithi132.stdout: "command_active": 0, 2024-05-08T22:18:31.986 INFO:teuthology.orchestra.run.smithi132.stdout: "command_send": 0, 2024-05-08T22:18:31.986 INFO:teuthology.orchestra.run.smithi132.stdout: "command_resend": 0, 2024-05-08T22:18:31.986 INFO:teuthology.orchestra.run.smithi132.stdout: "map_epoch": 98, 2024-05-08T22:18:31.986 INFO:teuthology.orchestra.run.smithi132.stdout: "map_full": 0, 2024-05-08T22:18:31.986 INFO:teuthology.orchestra.run.smithi132.stdout: "map_inc": 15, 2024-05-08T22:18:31.986 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_sessions": 5, 2024-05-08T22:18:31.986 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_session_open": 5, 2024-05-08T22:18:31.986 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_session_close": 0, 2024-05-08T22:18:31.987 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_laggy": 0, 2024-05-08T22:18:31.987 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_wr": 0, 2024-05-08T22:18:31.987 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_rd": 2, 2024-05-08T22:18:31.987 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_del": 0 2024-05-08T22:18:31.987 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.987 INFO:teuthology.orchestra.run.smithi132.stdout: "oft": { 2024-05-08T22:18:31.987 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_objs": 0, 2024-05-08T22:18:31.987 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:18:31.987 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_updates": 0, 2024-05-08T22:18:31.987 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_removes": 0 2024-05-08T22:18:31.987 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.987 INFO:teuthology.orchestra.run.smithi132.stdout: "osd-slow-ops": { 2024-05-08T22:18:31.987 INFO:teuthology.orchestra.run.smithi132.stdout: "slow_ops_count": 0 2024-05-08T22:18:31.987 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.988 INFO:teuthology.orchestra.run.smithi132.stdout: "purge_queue": { 2024-05-08T22:18:31.988 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_ops": 0, 2024-05-08T22:18:31.988 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T22:18:31.988 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing": 0, 2024-05-08T22:18:31.988 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_high_water": 0, 2024-05-08T22:18:31.988 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executed": 0, 2024-05-08T22:18:31.988 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_item_in_journal": 0 2024-05-08T22:18:31.988 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.988 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:18:31.988 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-05-08T22:18:31.988 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2024-05-08T22:18:31.988 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-05-08T22:18:31.989 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 2615, 2024-05-08T22:18:31.989 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 18202072, 2024-05-08T22:18:31.989 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:31.989 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 2615, 2024-05-08T22:18:31.989 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2024-05-08T22:18:31.989 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2024-05-08T22:18:31.989 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 2615, 2024-05-08T22:18:31.989 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 18202072, 2024-05-08T22:18:31.989 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-05-08T22:18:31.989 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.989 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.989 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.989 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-05-08T22:18:31.990 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.990 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-objecter_bytes": { 2024-05-08T22:18:31.990 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-05-08T22:18:31.990 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2024-05-08T22:18:31.990 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-05-08T22:18:31.990 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2024-05-08T22:18:31.990 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2024-05-08T22:18:31.990 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:31.990 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2024-05-08T22:18:31.990 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 2358, 2024-05-08T22:18:31.990 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 16309256, 2024-05-08T22:18:31.990 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 2358, 2024-05-08T22:18:31.990 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 16309256, 2024-05-08T22:18:31.991 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-05-08T22:18:31.991 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.991 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.991 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.991 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-05-08T22:18:31.991 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.991 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-objecter_ops": { 2024-05-08T22:18:31.991 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-05-08T22:18:31.991 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 1024, 2024-05-08T22:18:31.991 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-05-08T22:18:31.991 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2024-05-08T22:18:31.991 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2024-05-08T22:18:31.991 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:31.991 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2024-05-08T22:18:31.992 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 2358, 2024-05-08T22:18:31.992 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 2358, 2024-05-08T22:18:31.992 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 2358, 2024-05-08T22:18:31.992 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 2358, 2024-05-08T22:18:31.992 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-05-08T22:18:31.992 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.992 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.992 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.992 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-05-08T22:18:31.992 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.992 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:18:31.992 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-05-08T22:18:31.992 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 3758096384, 2024-05-08T22:18:31.992 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-05-08T22:18:31.993 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2024-05-08T22:18:31.993 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2024-05-08T22:18:31.993 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:31.993 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2024-05-08T22:18:31.993 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2024-05-08T22:18:31.993 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2024-05-08T22:18:31.993 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 0, 2024-05-08T22:18:31.993 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 0, 2024-05-08T22:18:31.993 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-05-08T22:18:31.993 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.993 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.993 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.993 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-05-08T22:18:31.993 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:31.994 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-write_buf_throttle-0x56125fb20500": { 2024-05-08T22:18:31.994 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-05-08T22:18:31.994 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 3758096384, 2024-05-08T22:18:31.994 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-05-08T22:18:31.994 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2024-05-08T22:18:31.994 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2024-05-08T22:18:31.994 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:31.994 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2024-05-08T22:18:31.994 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2024-05-08T22:18:31.994 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2024-05-08T22:18:31.994 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 0, 2024-05-08T22:18:31.994 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 0, 2024-05-08T22:18:31.994 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-05-08T22:18:31.995 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:31.995 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:31.995 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:31.995 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-05-08T22:18:31.995 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-05-08T22:18:31.995 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-05-08T22:18:32.335 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-08T22:18:32.335 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-08T22:18:32.335 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-05-08T22:18:32.335 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-05-08T22:18:32.335 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T22:18:32.335 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-05-08T22:18:32.335 DEBUG:tasks.check_counter:Getting stats from k 2024-05-08T22:18:32.336 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-08T22:18:32.688 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:18:33.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:32 smithi157 ceph-mon[77746]: pgmap v350: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:18:33.034 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:32 smithi132 ceph-mon[78180]: pgmap v350: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:18:33.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:32 smithi131 ceph-mon[37260]: pgmap v350: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:18:33.886 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-05-08T22:18:33.887 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:18:33.887 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 4, 2024-05-08T22:18:33.887 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 4, 2024-05-08T22:18:33.887 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 2992, 2024-05-08T22:18:33.887 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 448, 2024-05-08T22:18:33.887 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 4, 2024-05-08T22:18:33.887 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 0, 2024-05-08T22:18:33.887 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.001439304, 2024-05-08T22:18:33.887 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.000035888, 2024-05-08T22:18:33.887 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.000601281, 2024-05-08T22:18:33.887 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T22:18:33.887 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:33.887 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 3, 2024-05-08T22:18:33.887 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000703682, 2024-05-08T22:18:33.888 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000234560 2024-05-08T22:18:33.888 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:33.888 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:33.888 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:33.888 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:33.888 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:33.888 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:33.888 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-08T22:18:33.888 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T22:18:33.888 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:33.888 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:18:33.888 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 2, 2024-05-08T22:18:33.888 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 170, 2024-05-08T22:18:33.888 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 320, 2024-05-08T22:18:33.888 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 182352, 2024-05-08T22:18:33.889 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 5, 2024-05-08T22:18:33.889 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 3, 2024-05-08T22:18:33.889 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.009350897, 2024-05-08T22:18:33.889 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.005898906, 2024-05-08T22:18:33.889 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.001561537, 2024-05-08T22:18:33.889 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T22:18:33.889 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:33.889 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 167, 2024-05-08T22:18:33.889 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.030395772, 2024-05-08T22:18:33.889 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000182010 2024-05-08T22:18:33.889 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:33.889 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:33.889 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:33.889 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:33.889 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:33.889 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:33.890 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T22:18:33.890 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 182352 2024-05-08T22:18:33.890 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:33.890 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:18:33.890 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 231, 2024-05-08T22:18:33.890 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 221, 2024-05-08T22:18:33.890 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 1426064, 2024-05-08T22:18:33.890 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 126256, 2024-05-08T22:18:33.890 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 2, 2024-05-08T22:18:33.890 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 1, 2024-05-08T22:18:33.890 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.038455505, 2024-05-08T22:18:33.890 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.011067818, 2024-05-08T22:18:33.890 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.029852408, 2024-05-08T22:18:33.890 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.005239170, 2024-05-08T22:18:33.890 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:33.890 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 220, 2024-05-08T22:18:33.891 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.019186172, 2024-05-08T22:18:33.891 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000087209 2024-05-08T22:18:33.891 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:33.891 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:33.891 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:33.891 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:33.891 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:33.891 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:33.891 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 1426064, 2024-05-08T22:18:33.891 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 126256 2024-05-08T22:18:33.891 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:33.891 INFO:teuthology.orchestra.run.smithi132.stdout: "cct": { 2024-05-08T22:18:33.891 INFO:teuthology.orchestra.run.smithi132.stdout: "total_workers": 0, 2024-05-08T22:18:33.891 INFO:teuthology.orchestra.run.smithi132.stdout: "unhealthy_workers": 0 2024-05-08T22:18:33.891 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:33.892 INFO:teuthology.orchestra.run.smithi132.stdout: "mempool": { 2024-05-08T22:18:33.892 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:18:33.892 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_items": 0, 2024-05-08T22:18:33.892 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:18:33.892 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:18:33.892 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:18:33.892 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:18:33.892 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:18:33.892 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:18:33.892 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:18:33.892 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:18:33.892 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:18:33.892 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:18:33.892 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:18:33.892 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:18:33.892 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:18:33.893 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_items": 0, 2024-05-08T22:18:33.893 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:18:33.893 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_items": 0, 2024-05-08T22:18:33.893 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:18:33.893 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:18:33.893 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:18:33.893 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:18:33.893 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:18:33.893 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:18:33.893 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:18:33.893 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_items": 0, 2024-05-08T22:18:33.893 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:18:33.893 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:18:33.893 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:18:33.893 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_items": 0, 2024-05-08T22:18:33.893 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_bytes": 0, 2024-05-08T22:18:33.894 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_items": 0, 2024-05-08T22:18:33.894 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:18:33.894 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:18:33.894 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:18:33.894 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:18:33.894 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_bytes": 149017, 2024-05-08T22:18:33.894 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_items": 31, 2024-05-08T22:18:33.894 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:18:33.894 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_items": 0, 2024-05-08T22:18:33.894 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_bytes": 0, 2024-05-08T22:18:33.894 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_items": 0, 2024-05-08T22:18:33.894 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:18:33.894 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_items": 0, 2024-05-08T22:18:33.895 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:18:33.895 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_items": 0, 2024-05-08T22:18:33.895 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_bytes": 0, 2024-05-08T22:18:33.895 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_items": 0, 2024-05-08T22:18:33.895 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:18:33.895 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:18:33.895 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_bytes": 0, 2024-05-08T22:18:33.895 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_items": 0, 2024-05-08T22:18:33.895 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_bytes": 928, 2024-05-08T22:18:33.895 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_items": 2, 2024-05-08T22:18:33.895 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_bytes": 0, 2024-05-08T22:18:33.895 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_items": 0, 2024-05-08T22:18:33.895 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_bytes": 0, 2024-05-08T22:18:33.895 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_items": 0 2024-05-08T22:18:33.896 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-05-08T22:18:33.896 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:18:33.896 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-05-08T22:18:33.896 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2024-05-08T22:18:33.896 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-05-08T22:18:33.896 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 233, 2024-05-08T22:18:33.896 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 1393463, 2024-05-08T22:18:33.896 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:33.896 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 233, 2024-05-08T22:18:33.896 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2024-05-08T22:18:33.896 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2024-05-08T22:18:33.896 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 233, 2024-05-08T22:18:33.896 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 1393463, 2024-05-08T22:18:33.896 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-05-08T22:18:33.896 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-05-08T22:18:33.897 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-05-08T22:18:33.897 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-05-08T22:18:33.897 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-05-08T22:18:33.897 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-05-08T22:18:33.897 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-05-08T22:18:33.938 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:33 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:18:34.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:33 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:18:34.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:33 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:18:34.279 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-08T22:18:34.279 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-08T22:18:34.279 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-05-08T22:18:34.279 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-05-08T22:18:34.279 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-05-08T22:18:34.279 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-05-08T22:18:34.280 DEBUG:tasks.check_counter:Getting stats from c 2024-05-08T22:18:34.280 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-08T22:18:34.650 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:18:34.702 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:34 smithi157 ceph-mon[77746]: pgmap v351: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T22:18:35.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:34 smithi132 ceph-mon[78180]: pgmap v351: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T22:18:35.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:34 smithi131 ceph-mon[37260]: pgmap v351: 129 pgs: 21 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 41 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T22:18:35.849 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-05-08T22:18:35.850 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:18:35.850 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 261, 2024-05-08T22:18:35.850 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 242, 2024-05-08T22:18:35.850 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 1481890, 2024-05-08T22:18:35.850 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 129325, 2024-05-08T22:18:35.850 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 5, 2024-05-08T22:18:35.850 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 1, 2024-05-08T22:18:35.850 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.043197966, 2024-05-08T22:18:35.850 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.012207975, 2024-05-08T22:18:35.850 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.033269501, 2024-05-08T22:18:35.850 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.005293685, 2024-05-08T22:18:35.850 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:35.850 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 242, 2024-05-08T22:18:35.851 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.021110008, 2024-05-08T22:18:35.851 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000087231 2024-05-08T22:18:35.851 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.851 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:35.851 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.851 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.851 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.851 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.851 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1481056, 2024-05-08T22:18:35.851 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 129008 2024-05-08T22:18:35.851 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.851 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:18:35.851 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 728, 2024-05-08T22:18:35.851 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 893, 2024-05-08T22:18:35.852 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 4379992, 2024-05-08T22:18:35.852 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 665407, 2024-05-08T22:18:35.852 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 7, 2024-05-08T22:18:35.852 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 5, 2024-05-08T22:18:35.852 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.073798473, 2024-05-08T22:18:35.852 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.022651686, 2024-05-08T22:18:35.852 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.020056167, 2024-05-08T22:18:35.852 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.023694457, 2024-05-08T22:18:35.852 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:35.852 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 890, 2024-05-08T22:18:35.852 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.073566446, 2024-05-08T22:18:35.852 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000082658 2024-05-08T22:18:35.852 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.852 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:35.853 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.853 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.853 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.853 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.853 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T22:18:35.853 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 439616 2024-05-08T22:18:35.853 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.853 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:18:35.853 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 1623, 2024-05-08T22:18:35.853 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 1625, 2024-05-08T22:18:35.853 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 8064567, 2024-05-08T22:18:35.853 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 504811, 2024-05-08T22:18:35.853 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 5, 2024-05-08T22:18:35.854 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 4, 2024-05-08T22:18:35.854 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.146970636, 2024-05-08T22:18:35.854 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.040046739, 2024-05-08T22:18:35.854 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.044595777, 2024-05-08T22:18:35.854 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.049183591, 2024-05-08T22:18:35.854 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:35.854 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1623, 2024-05-08T22:18:35.854 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.127322718, 2024-05-08T22:18:35.854 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000078448 2024-05-08T22:18:35.854 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.854 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:35.854 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.854 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.854 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.855 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.855 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T22:18:35.855 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T22:18:35.855 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.855 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-05-08T22:18:35.855 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-05-08T22:18:35.855 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-05-08T22:18:35.855 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.855 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-05-08T22:18:35.855 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-05-08T22:18:35.855 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-05-08T22:18:35.855 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 3966, 2024-05-08T22:18:35.855 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.103648522, 2024-05-08T22:18:35.855 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000026134 2024-05-08T22:18:35.855 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-08T22:18:35.855 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.856 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-05-08T22:18:35.856 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-05-08T22:18:35.856 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-05-08T22:18:35.856 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.856 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.856 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.856 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-08T22:18:35.856 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.856 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-05-08T22:18:35.856 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 0, 2024-05-08T22:18:35.856 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 0, 2024-05-08T22:18:35.856 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-05-08T22:18:35.856 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.856 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.856 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.857 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.857 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-05-08T22:18:35.857 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 0, 2024-05-08T22:18:35.857 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2024-05-08T22:18:35.857 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-05-08T22:18:35.857 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 0, 2024-05-08T22:18:35.857 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2024-05-08T22:18:35.857 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2024-05-08T22:18:35.857 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 1248, 2024-05-08T22:18:35.857 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 0, 2024-05-08T22:18:35.857 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 1248, 2024-05-08T22:18:35.857 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 0, 2024-05-08T22:18:35.857 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 1248, 2024-05-08T22:18:35.857 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 0, 2024-05-08T22:18:35.857 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 0, 2024-05-08T22:18:35.858 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 0, 2024-05-08T22:18:35.858 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 1, 2024-05-08T22:18:35.858 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 0, 2024-05-08T22:18:35.858 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 0, 2024-05-08T22:18:35.858 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2024-05-08T22:18:35.858 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 0, 2024-05-08T22:18:35.858 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2024-05-08T22:18:35.858 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-05-08T22:18:35.858 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 0, 2024-05-08T22:18:35.858 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-05-08T22:18:35.858 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-05-08T22:18:35.858 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 0, 2024-05-08T22:18:35.858 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2024-05-08T22:18:35.859 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 0, 2024-05-08T22:18:35.859 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 0, 2024-05-08T22:18:35.859 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-05-08T22:18:35.859 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 0, 2024-05-08T22:18:35.859 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-05-08T22:18:35.859 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 0, 2024-05-08T22:18:35.859 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 0, 2024-05-08T22:18:35.859 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 0, 2024-05-08T22:18:35.859 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T22:18:35.859 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-05-08T22:18:35.859 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:18:35.859 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:18:35.859 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 0, 2024-05-08T22:18:35.859 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T22:18:35.859 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T22:18:35.859 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 0, 2024-05-08T22:18:35.860 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2024-05-08T22:18:35.860 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T22:18:35.860 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 0, 2024-05-08T22:18:35.860 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T22:18:35.860 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:18:35.860 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:18:35.860 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 0, 2024-05-08T22:18:35.860 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 0, 2024-05-08T22:18:35.860 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 0, 2024-05-08T22:18:35.860 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 0 2024-05-08T22:18:35.860 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.860 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-05-08T22:18:35.860 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2024-05-08T22:18:35.860 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2024-05-08T22:18:35.860 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2024-05-08T22:18:35.861 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 0, 2024-05-08T22:18:35.861 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2024-05-08T22:18:35.861 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 0, 2024-05-08T22:18:35.861 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-05-08T22:18:35.861 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:18:35.861 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 0, 2024-05-08T22:18:35.861 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 0, 2024-05-08T22:18:35.861 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-05-08T22:18:35.861 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-05-08T22:18:35.861 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-05-08T22:18:35.861 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:18:35.861 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:18:35.861 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-05-08T22:18:35.861 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-05-08T22:18:35.862 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T22:18:35.862 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 0, 2024-05-08T22:18:35.862 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-05-08T22:18:35.862 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2024-05-08T22:18:35.862 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-05-08T22:18:35.862 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-05-08T22:18:35.862 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.862 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-05-08T22:18:35.862 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 0, 2024-05-08T22:18:35.862 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 0, 2024-05-08T22:18:35.862 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 0, 2024-05-08T22:18:35.862 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 6023, 2024-05-08T22:18:35.862 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-05-08T22:18:35.862 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 0, 2024-05-08T22:18:35.862 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 0, 2024-05-08T22:18:35.863 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 0, 2024-05-08T22:18:35.863 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 0, 2024-05-08T22:18:35.863 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 8, 2024-05-08T22:18:35.863 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-05-08T22:18:35.863 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 0, 2024-05-08T22:18:35.863 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 4194304, 2024-05-08T22:18:35.863 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 15973848, 2024-05-08T22:18:35.863 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 15968922, 2024-05-08T22:18:35.863 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-05-08T22:18:35.863 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.863 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.863 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.863 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.863 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 6023 2024-05-08T22:18:35.863 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.863 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-05-08T22:18:35.864 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 10, 2024-05-08T22:18:35.864 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 2748, 2024-05-08T22:18:35.864 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 2738, 2024-05-08T22:18:35.864 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 8, 2024-05-08T22:18:35.864 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 72, 2024-05-08T22:18:35.864 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 64, 2024-05-08T22:18:35.864 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 1248, 2024-05-08T22:18:35.864 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 4205, 2024-05-08T22:18:35.864 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 2957, 2024-05-08T22:18:35.864 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 0, 2024-05-08T22:18:35.864 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 0, 2024-05-08T22:18:35.864 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 0, 2024-05-08T22:18:35.864 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 56108, 2024-05-08T22:18:35.864 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 207156 2024-05-08T22:18:35.864 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.864 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-05-08T22:18:35.865 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 0, 2024-05-08T22:18:35.865 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 0, 2024-05-08T22:18:35.865 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 0, 2024-05-08T22:18:35.865 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 0, 2024-05-08T22:18:35.865 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 0, 2024-05-08T22:18:35.865 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-05-08T22:18:35.865 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.865 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.865 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.865 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.865 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-05-08T22:18:35.865 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.865 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.865 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.865 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.865 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-05-08T22:18:35.865 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.866 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.866 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.866 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.866 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-05-08T22:18:35.866 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.866 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.866 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.866 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.866 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-05-08T22:18:35.866 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.866 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.866 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.866 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.867 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-05-08T22:18:35.867 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.867 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.867 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.867 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.867 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-05-08T22:18:35.867 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.867 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.867 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.867 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.867 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-05-08T22:18:35.867 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.867 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.867 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.867 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.867 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-05-08T22:18:35.868 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.868 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.868 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.868 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.868 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-05-08T22:18:35.868 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.868 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.868 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.868 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.868 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-05-08T22:18:35.868 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.868 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.868 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.868 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.868 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-05-08T22:18:35.868 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.869 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.869 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.869 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.869 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-05-08T22:18:35.869 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.869 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.869 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.869 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.869 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-05-08T22:18:35.869 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.869 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.869 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.869 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.869 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-05-08T22:18:35.869 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.869 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.870 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.870 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.870 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-05-08T22:18:35.870 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.870 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.870 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.870 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.870 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-05-08T22:18:35.870 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.870 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.870 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.870 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.870 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-05-08T22:18:35.870 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.871 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.871 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.871 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.871 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-05-08T22:18:35.871 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.871 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.871 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.871 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.871 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-05-08T22:18:35.871 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.871 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.871 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.871 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.871 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-05-08T22:18:35.871 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.871 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.872 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.872 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.872 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-05-08T22:18:35.872 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.872 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.872 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.872 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.872 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-05-08T22:18:35.872 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.872 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.872 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.872 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.872 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-05-08T22:18:35.872 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.872 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.872 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.873 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.873 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-05-08T22:18:35.873 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.873 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.873 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.873 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.873 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-05-08T22:18:35.873 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.873 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.873 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.873 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.873 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-05-08T22:18:35.873 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.873 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.873 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.873 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.874 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-05-08T22:18:35.874 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.874 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.874 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.874 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.874 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-05-08T22:18:35.874 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.874 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.874 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.874 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.874 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:18:35.874 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:18:35.874 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-05-08T22:18:35.874 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.874 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.875 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.875 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-08T22:18:35.875 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.875 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-05-08T22:18:35.875 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 1, 2024-05-08T22:18:35.875 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 4, 2024-05-08T22:18:35.875 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 3, 2024-05-08T22:18:35.875 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 1, 2024-05-08T22:18:35.875 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-05-08T22:18:35.875 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 0, 2024-05-08T22:18:35.875 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 0, 2024-05-08T22:18:35.875 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 2615, 2024-05-08T22:18:35.875 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-05-08T22:18:35.875 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.875 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-05-08T22:18:35.875 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:18:35.876 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-05-08T22:18:35.876 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:18:35.876 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:18:35.876 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:18:35.876 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:18:35.876 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:18:35.876 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:18:35.876 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:18:35.876 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:18:35.876 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:18:35.876 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:18:35.876 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:18:35.876 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:18:35.876 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:18:35.876 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-05-08T22:18:35.876 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:18:35.877 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-05-08T22:18:35.877 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:18:35.877 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:18:35.877 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:18:35.877 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:18:35.877 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:18:35.877 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:18:35.877 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:18:35.877 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-05-08T22:18:35.877 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:18:35.877 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:18:35.877 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:18:35.877 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-05-08T22:18:35.877 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-05-08T22:18:35.877 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-05-08T22:18:35.878 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:18:35.878 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:18:35.878 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:18:35.878 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:18:35.878 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 198257, 2024-05-08T22:18:35.878 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 57, 2024-05-08T22:18:35.878 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:18:35.878 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-05-08T22:18:35.878 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-05-08T22:18:35.878 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-05-08T22:18:35.878 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:18:35.878 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-05-08T22:18:35.878 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:18:35.878 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-05-08T22:18:35.878 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 10168, 2024-05-08T22:18:35.878 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 213, 2024-05-08T22:18:35.879 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:18:35.879 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:18:35.879 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-05-08T22:18:35.879 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-05-08T22:18:35.879 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 927424, 2024-05-08T22:18:35.879 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 5116, 2024-05-08T22:18:35.879 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-05-08T22:18:35.879 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-05-08T22:18:35.879 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-05-08T22:18:35.879 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-05-08T22:18:35.879 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.879 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-05-08T22:18:35.879 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-05-08T22:18:35.879 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-05-08T22:18:35.879 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 2350, 2024-05-08T22:18:35.879 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 21, 2024-05-08T22:18:35.880 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 3, 2024-05-08T22:18:35.880 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 2350, 2024-05-08T22:18:35.880 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-05-08T22:18:35.880 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2350, 2024-05-08T22:18:35.880 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 9.708827889, 2024-05-08T22:18:35.880 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.004131416 2024-05-08T22:18:35.880 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.880 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-05-08T22:18:35.880 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-05-08T22:18:35.880 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2350, 2024-05-08T22:18:35.880 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 2351 2024-05-08T22:18:35.880 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.880 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 2350, 2024-05-08T22:18:35.880 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 2350, 2024-05-08T22:18:35.880 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 0, 2024-05-08T22:18:35.880 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-05-08T22:18:35.880 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-05-08T22:18:35.881 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 1436, 2024-05-08T22:18:35.881 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 0, 2024-05-08T22:18:35.881 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 912, 2024-05-08T22:18:35.881 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 0, 2024-05-08T22:18:35.881 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 0, 2024-05-08T22:18:35.881 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-05-08T22:18:35.881 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-05-08T22:18:35.881 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-05-08T22:18:35.881 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2024-05-08T22:18:35.881 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 0, 2024-05-08T22:18:35.881 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-05-08T22:18:35.881 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-05-08T22:18:35.881 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-05-08T22:18:35.881 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 1, 2024-05-08T22:18:35.882 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 0, 2024-05-08T22:18:35.882 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:18:35.882 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-05-08T22:18:35.882 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:18:35.882 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-05-08T22:18:35.882 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-05-08T22:18:35.882 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-05-08T22:18:35.882 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:18:35.882 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-05-08T22:18:35.882 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:18:35.882 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2024-05-08T22:18:35.882 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-05-08T22:18:35.882 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-05-08T22:18:35.882 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-05-08T22:18:35.882 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-05-08T22:18:35.882 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-05-08T22:18:35.883 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-05-08T22:18:35.883 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-05-08T22:18:35.883 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-05-08T22:18:35.883 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-05-08T22:18:35.883 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-05-08T22:18:35.883 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-05-08T22:18:35.883 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-05-08T22:18:35.883 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-05-08T22:18:35.883 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-05-08T22:18:35.883 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-05-08T22:18:35.883 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-05-08T22:18:35.883 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 98, 2024-05-08T22:18:35.883 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-05-08T22:18:35.883 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 15, 2024-05-08T22:18:35.883 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 6, 2024-05-08T22:18:35.883 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 6, 2024-05-08T22:18:35.883 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2024-05-08T22:18:35.884 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-05-08T22:18:35.884 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 0, 2024-05-08T22:18:35.884 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 2, 2024-05-08T22:18:35.884 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 0 2024-05-08T22:18:35.884 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.884 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-05-08T22:18:35.884 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 0, 2024-05-08T22:18:35.884 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:18:35.884 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-05-08T22:18:35.884 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-05-08T22:18:35.884 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.884 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-05-08T22:18:35.884 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 0 2024-05-08T22:18:35.884 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.884 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-05-08T22:18:35.884 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-05-08T22:18:35.885 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T22:18:35.885 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-05-08T22:18:35.885 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 0, 2024-05-08T22:18:35.885 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 0, 2024-05-08T22:18:35.885 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-05-08T22:18:35.885 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.885 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:18:35.885 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-08T22:18:35.885 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-05-08T22:18:35.885 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-08T22:18:35.885 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 2608, 2024-05-08T22:18:35.885 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 13675625, 2024-05-08T22:18:35.885 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:35.885 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 2608, 2024-05-08T22:18:35.885 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-08T22:18:35.885 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-08T22:18:35.886 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2608, 2024-05-08T22:18:35.886 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 13675625, 2024-05-08T22:18:35.886 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-08T22:18:35.886 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.886 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.886 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.886 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-08T22:18:35.886 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.886 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-05-08T22:18:35.886 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-08T22:18:35.886 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-05-08T22:18:35.886 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-08T22:18:35.886 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-08T22:18:35.886 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-08T22:18:35.886 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:35.886 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-08T22:18:35.887 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 2350, 2024-05-08T22:18:35.887 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 11784231, 2024-05-08T22:18:35.887 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2350, 2024-05-08T22:18:35.887 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 11784231, 2024-05-08T22:18:35.887 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-08T22:18:35.887 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.887 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.887 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.887 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-08T22:18:35.887 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.887 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-05-08T22:18:35.887 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-08T22:18:35.887 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-05-08T22:18:35.887 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-08T22:18:35.888 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-08T22:18:35.888 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-08T22:18:35.888 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:35.888 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-08T22:18:35.888 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 2350, 2024-05-08T22:18:35.888 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 2350, 2024-05-08T22:18:35.888 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2350, 2024-05-08T22:18:35.888 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 2350, 2024-05-08T22:18:35.888 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-08T22:18:35.888 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.888 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.888 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.888 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-08T22:18:35.888 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.888 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:18:35.888 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-08T22:18:35.889 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-05-08T22:18:35.889 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-08T22:18:35.889 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-08T22:18:35.889 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-08T22:18:35.889 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:35.889 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-08T22:18:35.889 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-08T22:18:35.889 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-08T22:18:35.889 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-05-08T22:18:35.889 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-05-08T22:18:35.889 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-08T22:18:35.889 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.889 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.889 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.889 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-08T22:18:35.889 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:35.890 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x55621b8fe500": { 2024-05-08T22:18:35.890 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-08T22:18:35.890 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-05-08T22:18:35.890 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-08T22:18:35.890 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-08T22:18:35.890 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-08T22:18:35.890 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:35.890 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-08T22:18:35.890 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-08T22:18:35.890 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-08T22:18:35.890 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-05-08T22:18:35.890 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-05-08T22:18:35.890 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-08T22:18:35.890 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:35.890 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:35.890 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:35.891 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-08T22:18:35.891 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-08T22:18:35.891 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-05-08T22:18:35.891 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:35 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:18:35.891 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:35 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:18:35.891 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:35 smithi157 ceph-mon[77746]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:18:35.891 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:35 smithi157 ceph-mon[77746]: pgmap v352: 129 pgs: 16 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 58 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:18:36.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:35 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:18:36.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:35 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:18:36.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:35 smithi132 ceph-mon[78180]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:18:36.090 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:35 smithi132 ceph-mon[78180]: pgmap v352: 129 pgs: 16 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 58 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:18:36.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:35 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:18:36.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:35 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:18:36.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:35 smithi131 ceph-mon[37260]: from='mgr.15366 ' entity='mgr.x' 2024-05-08T22:18:36.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:35 smithi131 ceph-mon[37260]: pgmap v352: 129 pgs: 16 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 58 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:18:36.255 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-08T22:18:36.255 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-08T22:18:36.255 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-05-08T22:18:36.255 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-05-08T22:18:36.255 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T22:18:36.255 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-05-08T22:18:36.255 DEBUG:tasks.check_counter:Getting stats from f 2024-05-08T22:18:36.255 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-08T22:18:36.667 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:18:37.927 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-05-08T22:18:37.927 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:18:37.927 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 0, 2024-05-08T22:18:37.927 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 2, 2024-05-08T22:18:37.927 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 0, 2024-05-08T22:18:37.927 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 0, 2024-05-08T22:18:37.927 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 4, 2024-05-08T22:18:37.927 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 0, 2024-05-08T22:18:37.928 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.001214997, 2024-05-08T22:18:37.928 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T22:18:37.928 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.000122330, 2024-05-08T22:18:37.928 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T22:18:37.928 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:37.928 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:37.928 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:37.928 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:37.928 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:37.928 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:37.928 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:37.928 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:37.928 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:37.928 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:37.928 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T22:18:37.929 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T22:18:37.929 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:37.929 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:18:37.929 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 234, 2024-05-08T22:18:37.929 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 222, 2024-05-08T22:18:37.929 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 1428784, 2024-05-08T22:18:37.929 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 126128, 2024-05-08T22:18:37.929 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 2, 2024-05-08T22:18:37.929 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 1, 2024-05-08T22:18:37.929 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.041059323, 2024-05-08T22:18:37.929 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.011772535, 2024-05-08T22:18:37.929 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.031949492, 2024-05-08T22:18:37.929 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.005375090, 2024-05-08T22:18:37.929 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:37.929 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 222, 2024-05-08T22:18:37.929 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.024759804, 2024-05-08T22:18:37.930 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000111530 2024-05-08T22:18:37.930 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:37.930 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:37.930 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:37.930 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:37.930 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:37.930 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:37.930 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1428784, 2024-05-08T22:18:37.930 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 126128 2024-05-08T22:18:37.930 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:37.930 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:18:37.930 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 2, 2024-05-08T22:18:37.930 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 168, 2024-05-08T22:18:37.930 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 320, 2024-05-08T22:18:37.930 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 182000, 2024-05-08T22:18:37.931 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 5, 2024-05-08T22:18:37.931 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 3, 2024-05-08T22:18:37.931 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.009985640, 2024-05-08T22:18:37.931 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.006283431, 2024-05-08T22:18:37.931 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.001641277, 2024-05-08T22:18:37.931 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T22:18:37.931 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:37.931 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 165, 2024-05-08T22:18:37.931 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.045737793, 2024-05-08T22:18:37.931 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000277198 2024-05-08T22:18:37.931 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:37.931 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:37.931 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:37.931 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:37.931 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:37.931 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:37.932 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T22:18:37.932 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 182000 2024-05-08T22:18:37.932 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:37.932 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-05-08T22:18:37.932 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 0, 2024-05-08T22:18:37.932 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-05-08T22:18:37.932 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:37.932 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-05-08T22:18:37.932 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:18:37.932 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-05-08T22:18:37.932 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:18:37.932 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:18:37.932 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:18:37.932 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:18:37.932 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:18:37.932 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:18:37.933 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:18:37.933 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:18:37.933 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:18:37.933 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:18:37.933 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:18:37.933 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:18:37.933 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:18:37.933 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-05-08T22:18:37.933 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:18:37.933 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-05-08T22:18:37.933 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:18:37.933 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:18:37.933 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:18:37.933 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:18:37.934 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:18:37.934 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:18:37.934 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:18:37.934 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-05-08T22:18:37.934 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:18:37.934 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:18:37.934 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:18:37.934 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-05-08T22:18:37.934 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-05-08T22:18:37.934 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-05-08T22:18:37.934 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:18:37.934 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:18:37.934 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:18:37.935 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:18:37.935 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 148995, 2024-05-08T22:18:37.935 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 31, 2024-05-08T22:18:37.935 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:18:37.935 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-05-08T22:18:37.935 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-05-08T22:18:37.935 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-05-08T22:18:37.935 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:18:37.935 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-05-08T22:18:37.935 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:18:37.935 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-05-08T22:18:37.935 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 0, 2024-05-08T22:18:37.935 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 0, 2024-05-08T22:18:37.935 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:18:37.935 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:18:37.936 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-05-08T22:18:37.936 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-05-08T22:18:37.936 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 928, 2024-05-08T22:18:37.936 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 2, 2024-05-08T22:18:37.936 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-05-08T22:18:37.936 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-05-08T22:18:37.936 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-05-08T22:18:37.936 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-05-08T22:18:37.936 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:37.936 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:18:37.936 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-08T22:18:37.936 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-05-08T22:18:37.936 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-08T22:18:37.937 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 232, 2024-05-08T22:18:37.937 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 1393333, 2024-05-08T22:18:37.937 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:37.937 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 232, 2024-05-08T22:18:37.937 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-08T22:18:37.937 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-08T22:18:37.937 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 232, 2024-05-08T22:18:37.937 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 1393333, 2024-05-08T22:18:37.937 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-08T22:18:37.937 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:37.937 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:37.937 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:37.937 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-08T22:18:37.937 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-08T22:18:37.938 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-05-08T22:18:38.308 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2024-05-08T22:18:38.308 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2024-05-08T22:18:38.308 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-05-08T22:18:38.308 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-05-08T22:18:38.308 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-05-08T22:18:38.308 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.f 2024-05-08T22:18:38.308 DEBUG:tasks.check_counter:Getting stats from i 2024-05-08T22:18:38.308 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-08T22:18:38.527 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:38 smithi157 ceph-mon[77746]: pgmap v353: 129 pgs: 16 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 58 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:18:38.708 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:18:38.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:38 smithi132 ceph-mon[78180]: pgmap v353: 129 pgs: 16 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 58 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:18:38.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:38 smithi131 ceph-mon[37260]: pgmap v353: 129 pgs: 16 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 58 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:18:39.948 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-05-08T22:18:39.948 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:18:39.948 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 1909, 2024-05-08T22:18:39.948 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 1911, 2024-05-08T22:18:39.948 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 471811006, 2024-05-08T22:18:39.948 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 593709, 2024-05-08T22:18:39.948 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 8, 2024-05-08T22:18:39.948 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 4, 2024-05-08T22:18:39.948 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.385174098, 2024-05-08T22:18:39.949 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.045331299, 2024-05-08T22:18:39.949 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.267207901, 2024-05-08T22:18:39.949 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.055993075, 2024-05-08T22:18:39.949 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:39.949 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1909, 2024-05-08T22:18:39.949 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.182783700, 2024-05-08T22:18:39.949 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000095748 2024-05-08T22:18:39.949 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.949 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:39.949 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.949 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.949 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.949 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.949 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T22:18:39.950 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T22:18:39.950 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.950 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:18:39.950 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 1155, 2024-05-08T22:18:39.950 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 1136, 2024-05-08T22:18:39.950 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 446425014, 2024-05-08T22:18:39.950 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 407268, 2024-05-08T22:18:39.950 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 6, 2024-05-08T22:18:39.950 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 5, 2024-05-08T22:18:39.950 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.326432779, 2024-05-08T22:18:39.950 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.030489999, 2024-05-08T22:18:39.950 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.263088810, 2024-05-08T22:18:39.950 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.034053010, 2024-05-08T22:18:39.950 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:39.951 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1136, 2024-05-08T22:18:39.951 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.099632394, 2024-05-08T22:18:39.951 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000087704 2024-05-08T22:18:39.951 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.951 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:39.951 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.951 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.951 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.951 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.951 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1481216, 2024-05-08T22:18:39.951 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 129168 2024-05-08T22:18:39.951 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.951 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:18:39.951 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 876, 2024-05-08T22:18:39.951 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 1042, 2024-05-08T22:18:39.952 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 336013685, 2024-05-08T22:18:39.952 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 714743, 2024-05-08T22:18:39.952 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 8, 2024-05-08T22:18:39.952 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 6, 2024-05-08T22:18:39.952 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.216067546, 2024-05-08T22:18:39.952 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.027518091, 2024-05-08T22:18:39.952 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.156551567, 2024-05-08T22:18:39.952 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.023456325, 2024-05-08T22:18:39.952 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:39.952 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1039, 2024-05-08T22:18:39.952 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.088843529, 2024-05-08T22:18:39.952 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000085508 2024-05-08T22:18:39.952 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.952 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:39.952 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.952 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.953 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.953 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.953 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T22:18:39.953 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 442928 2024-05-08T22:18:39.953 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.953 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-05-08T22:18:39.953 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-05-08T22:18:39.953 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-05-08T22:18:39.953 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.953 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-05-08T22:18:39.953 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-05-08T22:18:39.953 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-05-08T22:18:39.953 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 5454, 2024-05-08T22:18:39.953 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.645331968, 2024-05-08T22:18:39.953 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000118322 2024-05-08T22:18:39.954 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-08T22:18:39.954 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.954 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-05-08T22:18:39.954 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-05-08T22:18:39.954 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-05-08T22:18:39.954 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.954 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.954 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.954 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-08T22:18:39.954 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.954 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-05-08T22:18:39.954 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 0, 2024-05-08T22:18:39.954 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 0, 2024-05-08T22:18:39.955 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-05-08T22:18:39.955 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.955 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.955 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.955 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.955 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-05-08T22:18:39.955 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 0, 2024-05-08T22:18:39.955 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2024-05-08T22:18:39.955 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-05-08T22:18:39.955 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 0, 2024-05-08T22:18:39.955 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2024-05-08T22:18:39.956 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2024-05-08T22:18:39.956 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 174525, 2024-05-08T22:18:39.956 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 58511, 2024-05-08T22:18:39.956 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 116014, 2024-05-08T22:18:39.956 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 0, 2024-05-08T22:18:39.956 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 88159, 2024-05-08T22:18:39.956 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 0, 2024-05-08T22:18:39.956 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 0, 2024-05-08T22:18:39.956 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 0, 2024-05-08T22:18:39.956 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 2, 2024-05-08T22:18:39.957 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 0, 2024-05-08T22:18:39.957 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 0, 2024-05-08T22:18:39.957 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2024-05-08T22:18:39.957 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 0, 2024-05-08T22:18:39.957 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2024-05-08T22:18:39.957 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-05-08T22:18:39.957 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 0, 2024-05-08T22:18:39.957 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-05-08T22:18:39.957 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-05-08T22:18:39.957 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 0, 2024-05-08T22:18:39.957 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2024-05-08T22:18:39.957 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 0, 2024-05-08T22:18:39.957 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 0, 2024-05-08T22:18:39.957 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-05-08T22:18:39.957 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 0, 2024-05-08T22:18:39.957 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-05-08T22:18:39.957 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 4, 2024-05-08T22:18:39.958 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 234, 2024-05-08T22:18:39.958 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 6, 2024-05-08T22:18:39.958 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T22:18:39.958 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-05-08T22:18:39.958 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:18:39.958 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:18:39.958 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 0, 2024-05-08T22:18:39.958 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T22:18:39.958 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T22:18:39.958 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 0, 2024-05-08T22:18:39.958 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2024-05-08T22:18:39.958 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T22:18:39.958 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 0, 2024-05-08T22:18:39.958 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T22:18:39.959 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:18:39.959 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:18:39.959 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 0, 2024-05-08T22:18:39.959 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 0, 2024-05-08T22:18:39.959 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 0, 2024-05-08T22:18:39.959 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 0 2024-05-08T22:18:39.959 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.959 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-05-08T22:18:39.959 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2024-05-08T22:18:39.959 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2024-05-08T22:18:39.959 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2024-05-08T22:18:39.959 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 0, 2024-05-08T22:18:39.959 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2024-05-08T22:18:39.959 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 0, 2024-05-08T22:18:39.959 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-05-08T22:18:39.959 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:18:39.960 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 0, 2024-05-08T22:18:39.960 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 0, 2024-05-08T22:18:39.960 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-05-08T22:18:39.960 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-05-08T22:18:39.960 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-05-08T22:18:39.960 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:18:39.960 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:18:39.960 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-05-08T22:18:39.960 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-05-08T22:18:39.960 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T22:18:39.960 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 0, 2024-05-08T22:18:39.960 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-05-08T22:18:39.960 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2024-05-08T22:18:39.960 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-05-08T22:18:39.960 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-05-08T22:18:39.961 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.961 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-05-08T22:18:39.961 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 0, 2024-05-08T22:18:39.961 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 0, 2024-05-08T22:18:39.961 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 0, 2024-05-08T22:18:39.961 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 326296, 2024-05-08T22:18:39.961 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-05-08T22:18:39.961 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 0, 2024-05-08T22:18:39.961 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 0, 2024-05-08T22:18:39.961 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 0, 2024-05-08T22:18:39.961 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 0, 2024-05-08T22:18:39.961 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 136, 2024-05-08T22:18:39.961 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-05-08T22:18:39.961 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 0, 2024-05-08T22:18:39.961 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 847253757, 2024-05-08T22:18:39.962 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 1255524894, 2024-05-08T22:18:39.962 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 1255524758, 2024-05-08T22:18:39.962 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-05-08T22:18:39.962 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.962 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.962 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.962 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.962 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 326296 2024-05-08T22:18:39.962 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.962 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-05-08T22:18:39.962 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 48893, 2024-05-08T22:18:39.963 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 107540, 2024-05-08T22:18:39.963 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 58647, 2024-05-08T22:18:39.963 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 1494, 2024-05-08T22:18:39.963 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 1884, 2024-05-08T22:18:39.963 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 390, 2024-05-08T22:18:39.963 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 174525, 2024-05-08T22:18:39.963 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 177450, 2024-05-08T22:18:39.963 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 2925, 2024-05-08T22:18:39.963 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 0, 2024-05-08T22:18:39.963 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 0, 2024-05-08T22:18:39.963 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 0, 2024-05-08T22:18:39.963 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 331404, 2024-05-08T22:18:39.963 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 207156 2024-05-08T22:18:39.963 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.964 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-05-08T22:18:39.964 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 0, 2024-05-08T22:18:39.964 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 0, 2024-05-08T22:18:39.964 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 0, 2024-05-08T22:18:39.964 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 0, 2024-05-08T22:18:39.964 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 0, 2024-05-08T22:18:39.964 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-05-08T22:18:39.964 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.964 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.964 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.964 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.964 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-05-08T22:18:39.964 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.964 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.965 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.965 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.965 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-05-08T22:18:39.965 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.965 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.965 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.965 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.965 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-05-08T22:18:39.965 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.965 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.965 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.965 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.965 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-05-08T22:18:39.965 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.966 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.966 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.966 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.966 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-05-08T22:18:39.966 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.966 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.966 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.966 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.966 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-05-08T22:18:39.966 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.966 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.966 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.967 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.967 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-05-08T22:18:39.967 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.967 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.967 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.967 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.967 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-05-08T22:18:39.967 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.967 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.967 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.967 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.968 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-05-08T22:18:39.968 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.968 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.968 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.968 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.968 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-05-08T22:18:39.968 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.968 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.968 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.968 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.968 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-05-08T22:18:39.968 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.968 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.968 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.968 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.968 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-05-08T22:18:39.968 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.969 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.969 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.969 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.969 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-05-08T22:18:39.969 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.969 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.969 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.969 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.969 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-05-08T22:18:39.969 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.969 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.969 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.969 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.969 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-05-08T22:18:39.969 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.969 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.970 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.970 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.970 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-05-08T22:18:39.970 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.970 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.970 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.970 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.970 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-05-08T22:18:39.970 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.970 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.970 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.970 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.970 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-05-08T22:18:39.970 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.971 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.971 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.971 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.971 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-05-08T22:18:39.971 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.971 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.971 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.971 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.971 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-05-08T22:18:39.971 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.971 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.971 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.971 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.971 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-05-08T22:18:39.971 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.971 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.972 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.972 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.972 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-05-08T22:18:39.972 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.972 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.972 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.972 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.972 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-05-08T22:18:39.972 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.972 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.972 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.972 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.972 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-05-08T22:18:39.972 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.973 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.973 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.973 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.973 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-05-08T22:18:39.973 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.973 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.973 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.973 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.973 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-05-08T22:18:39.973 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.973 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.973 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.973 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.973 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-05-08T22:18:39.973 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.974 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.974 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.974 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.974 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-05-08T22:18:39.974 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.974 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.974 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.974 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.974 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:18:39.974 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:18:39.974 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-05-08T22:18:39.975 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.975 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.975 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.975 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-08T22:18:39.975 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.975 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-05-08T22:18:39.975 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 2, 2024-05-08T22:18:39.975 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 7, 2024-05-08T22:18:39.975 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 5, 2024-05-08T22:18:39.975 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 5, 2024-05-08T22:18:39.975 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-05-08T22:18:39.975 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 0, 2024-05-08T22:18:39.976 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 0, 2024-05-08T22:18:39.976 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 10142, 2024-05-08T22:18:39.976 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-05-08T22:18:39.976 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.976 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-05-08T22:18:39.976 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:18:39.976 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-05-08T22:18:39.976 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:18:39.976 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:18:39.976 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:18:39.976 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:18:39.976 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:18:39.976 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:18:39.976 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:18:39.976 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:18:39.977 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:18:39.977 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:18:39.977 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:18:39.977 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:18:39.977 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:18:39.977 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-05-08T22:18:39.977 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:18:39.977 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-05-08T22:18:39.977 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:18:39.977 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:18:39.978 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:18:39.978 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:18:39.978 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:18:39.978 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:18:39.978 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:18:39.978 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-05-08T22:18:39.978 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:18:39.978 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:18:39.978 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:18:39.978 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-05-08T22:18:39.979 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-05-08T22:18:39.979 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-05-08T22:18:39.979 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:18:39.979 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:18:39.979 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:18:39.979 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:18:39.979 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 1929801, 2024-05-08T22:18:39.979 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 49817, 2024-05-08T22:18:39.979 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:18:39.979 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-05-08T22:18:39.979 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-05-08T22:18:39.979 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-05-08T22:18:39.979 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:18:39.979 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-05-08T22:18:39.980 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:18:39.980 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-05-08T22:18:39.980 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 10168, 2024-05-08T22:18:39.980 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 213, 2024-05-08T22:18:39.980 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:18:39.980 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:18:39.980 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-05-08T22:18:39.980 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-05-08T22:18:39.980 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 238115700, 2024-05-08T22:18:39.980 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 2537648, 2024-05-08T22:18:39.980 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-05-08T22:18:39.980 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-05-08T22:18:39.980 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-05-08T22:18:39.980 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-05-08T22:18:39.980 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.981 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-05-08T22:18:39.981 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-05-08T22:18:39.981 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-05-08T22:18:39.981 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 3677, 2024-05-08T22:18:39.981 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 26, 2024-05-08T22:18:39.981 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 4, 2024-05-08T22:18:39.981 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 3677, 2024-05-08T22:18:39.981 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-05-08T22:18:39.981 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 3677, 2024-05-08T22:18:39.981 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 29.370479359, 2024-05-08T22:18:39.981 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.007987620 2024-05-08T22:18:39.981 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.981 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-05-08T22:18:39.981 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-05-08T22:18:39.981 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 3677, 2024-05-08T22:18:39.981 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 3678 2024-05-08T22:18:39.982 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.982 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 3677, 2024-05-08T22:18:39.982 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 3677, 2024-05-08T22:18:39.982 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 0, 2024-05-08T22:18:39.982 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-05-08T22:18:39.982 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-05-08T22:18:39.982 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 2238, 2024-05-08T22:18:39.982 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 0, 2024-05-08T22:18:39.982 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 1436, 2024-05-08T22:18:39.982 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 0, 2024-05-08T22:18:39.982 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 0, 2024-05-08T22:18:39.982 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-05-08T22:18:39.982 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-05-08T22:18:39.982 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-05-08T22:18:39.982 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2024-05-08T22:18:39.983 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 0, 2024-05-08T22:18:39.983 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-05-08T22:18:39.983 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-05-08T22:18:39.983 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-05-08T22:18:39.983 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 2, 2024-05-08T22:18:39.983 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 0, 2024-05-08T22:18:39.983 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:18:39.983 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-05-08T22:18:39.983 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:18:39.983 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-05-08T22:18:39.983 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-05-08T22:18:39.983 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-05-08T22:18:39.983 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:18:39.983 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-05-08T22:18:39.983 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:18:39.984 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2024-05-08T22:18:39.984 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-05-08T22:18:39.984 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-05-08T22:18:39.984 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-05-08T22:18:39.984 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-05-08T22:18:39.984 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-05-08T22:18:39.984 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-05-08T22:18:39.984 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-05-08T22:18:39.984 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-05-08T22:18:39.984 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-05-08T22:18:39.984 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-05-08T22:18:39.984 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-05-08T22:18:39.984 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-05-08T22:18:39.984 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-05-08T22:18:39.984 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-05-08T22:18:39.985 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-05-08T22:18:39.985 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-05-08T22:18:39.985 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 98, 2024-05-08T22:18:39.985 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-05-08T22:18:39.985 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 15, 2024-05-08T22:18:39.985 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 11, 2024-05-08T22:18:39.985 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 11, 2024-05-08T22:18:39.985 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2024-05-08T22:18:39.985 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-05-08T22:18:39.985 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 0, 2024-05-08T22:18:39.985 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 2, 2024-05-08T22:18:39.985 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 0 2024-05-08T22:18:39.985 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.985 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-05-08T22:18:39.985 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 0, 2024-05-08T22:18:39.985 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:18:39.986 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-05-08T22:18:39.986 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-05-08T22:18:39.986 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.986 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-05-08T22:18:39.986 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 0 2024-05-08T22:18:39.986 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.986 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-05-08T22:18:39.986 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-05-08T22:18:39.986 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T22:18:39.986 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-05-08T22:18:39.986 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 0, 2024-05-08T22:18:39.986 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 0, 2024-05-08T22:18:39.986 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-05-08T22:18:39.986 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.987 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:18:39.987 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-08T22:18:39.987 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-05-08T22:18:39.987 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-08T22:18:39.987 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 3936, 2024-05-08T22:18:39.987 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 1253879315, 2024-05-08T22:18:39.987 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:39.987 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 3936, 2024-05-08T22:18:39.987 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-08T22:18:39.987 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-08T22:18:39.987 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 3936, 2024-05-08T22:18:39.987 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 1253879315, 2024-05-08T22:18:39.987 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-08T22:18:39.988 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.988 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.988 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.988 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-08T22:18:39.988 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.988 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-05-08T22:18:39.988 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-08T22:18:39.988 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-05-08T22:18:39.988 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-08T22:18:39.989 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-08T22:18:39.989 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-08T22:18:39.989 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:39.989 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-08T22:18:39.989 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 3677, 2024-05-08T22:18:39.989 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 1251769191, 2024-05-08T22:18:39.989 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 3677, 2024-05-08T22:18:39.989 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 1251769191, 2024-05-08T22:18:39.989 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-08T22:18:39.989 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.989 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.989 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.989 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-08T22:18:39.989 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.990 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-05-08T22:18:39.990 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-08T22:18:39.990 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-05-08T22:18:39.990 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-08T22:18:39.990 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-08T22:18:39.990 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-08T22:18:39.990 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:39.990 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-08T22:18:39.990 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 3677, 2024-05-08T22:18:39.990 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 3677, 2024-05-08T22:18:39.990 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 3677, 2024-05-08T22:18:39.990 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 3677, 2024-05-08T22:18:39.990 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-08T22:18:39.990 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.991 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.991 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.991 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-08T22:18:39.991 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.991 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:18:39.991 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-08T22:18:39.991 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-05-08T22:18:39.991 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-08T22:18:39.991 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-08T22:18:39.991 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-08T22:18:39.991 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:39.991 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-08T22:18:39.991 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-08T22:18:39.991 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-08T22:18:39.991 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-05-08T22:18:39.991 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-05-08T22:18:39.991 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-08T22:18:39.992 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.992 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.992 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.992 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-08T22:18:39.992 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:39.992 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x55f647f54640": { 2024-05-08T22:18:39.992 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-08T22:18:39.992 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-05-08T22:18:39.992 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-08T22:18:39.992 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-08T22:18:39.992 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-08T22:18:39.992 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:39.992 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-08T22:18:39.992 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-08T22:18:39.992 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-08T22:18:39.992 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-05-08T22:18:39.992 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-05-08T22:18:39.993 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-08T22:18:39.993 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:39.993 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:39.993 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:39.993 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-08T22:18:39.993 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-08T22:18:39.993 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-05-08T22:18:40.336 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-05-08T22:18:40.337 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-05-08T22:18:40.337 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-05-08T22:18:40.337 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-05-08T22:18:40.337 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T22:18:40.337 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-05-08T22:18:40.337 DEBUG:tasks.check_counter:Getting stats from l 2024-05-08T22:18:40.338 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-08T22:18:40.529 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:40 smithi157 ceph-mon[77746]: pgmap v354: 129 pgs: 16 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 58 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:18:40.794 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:18:40.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:40 smithi132 ceph-mon[78180]: pgmap v354: 129 pgs: 16 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 58 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:18:40.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:40 smithi131 ceph-mon[37260]: pgmap v354: 129 pgs: 16 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 58 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T22:18:42.019 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-05-08T22:18:42.019 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:18:42.019 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 0, 2024-05-08T22:18:42.019 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 2, 2024-05-08T22:18:42.019 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 0, 2024-05-08T22:18:42.020 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 0, 2024-05-08T22:18:42.020 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 4, 2024-05-08T22:18:42.020 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 0, 2024-05-08T22:18:42.020 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.001336790, 2024-05-08T22:18:42.020 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T22:18:42.020 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.000341255, 2024-05-08T22:18:42.020 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T22:18:42.020 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:42.020 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:42.020 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:42.020 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:42.020 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:42.020 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:42.020 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:42.020 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:42.021 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:42.021 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:42.021 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T22:18:42.021 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T22:18:42.021 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:42.021 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:18:42.021 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 238, 2024-05-08T22:18:42.021 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 226, 2024-05-08T22:18:42.021 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 1429872, 2024-05-08T22:18:42.021 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 128496, 2024-05-08T22:18:42.021 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 2, 2024-05-08T22:18:42.021 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 1, 2024-05-08T22:18:42.021 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.043378034, 2024-05-08T22:18:42.021 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.009586088, 2024-05-08T22:18:42.022 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.034189628, 2024-05-08T22:18:42.022 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.006946819, 2024-05-08T22:18:42.022 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:42.022 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 226, 2024-05-08T22:18:42.022 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.030168488, 2024-05-08T22:18:42.022 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000133488 2024-05-08T22:18:42.022 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:42.022 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:42.022 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:42.022 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:42.022 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:42.022 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:42.022 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1429872, 2024-05-08T22:18:42.023 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 128496 2024-05-08T22:18:42.023 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:42.023 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:18:42.023 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 2, 2024-05-08T22:18:42.023 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 171, 2024-05-08T22:18:42.023 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 320, 2024-05-08T22:18:42.023 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 182528, 2024-05-08T22:18:42.023 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 5, 2024-05-08T22:18:42.023 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 3, 2024-05-08T22:18:42.023 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.010462503, 2024-05-08T22:18:42.023 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.006616845, 2024-05-08T22:18:42.023 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.001700606, 2024-05-08T22:18:42.023 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T22:18:42.024 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:18:42.024 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 168, 2024-05-08T22:18:42.024 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.047513091, 2024-05-08T22:18:42.024 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000282816 2024-05-08T22:18:42.024 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:42.024 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:18:42.024 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:42.024 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:42.024 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:42.024 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:42.024 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T22:18:42.024 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 182528 2024-05-08T22:18:42.024 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:42.025 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-05-08T22:18:42.025 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 0, 2024-05-08T22:18:42.025 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-05-08T22:18:42.025 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:42.025 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-05-08T22:18:42.025 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:18:42.025 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-05-08T22:18:42.025 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:18:42.025 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:18:42.025 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:18:42.025 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:18:42.025 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:18:42.025 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:18:42.025 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:18:42.025 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:18:42.026 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:18:42.026 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:18:42.026 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:18:42.026 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:18:42.026 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:18:42.026 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-05-08T22:18:42.026 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:18:42.026 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-05-08T22:18:42.026 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:18:42.026 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:18:42.026 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:18:42.026 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:18:42.026 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:18:42.026 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:18:42.027 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:18:42.027 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-05-08T22:18:42.027 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:18:42.027 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:18:42.027 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:18:42.027 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-05-08T22:18:42.027 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-05-08T22:18:42.027 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-05-08T22:18:42.027 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:18:42.027 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:18:42.027 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:18:42.028 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:18:42.028 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 148977, 2024-05-08T22:18:42.028 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 30, 2024-05-08T22:18:42.028 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:18:42.028 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-05-08T22:18:42.028 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-05-08T22:18:42.028 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-05-08T22:18:42.028 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:18:42.028 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-05-08T22:18:42.028 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:18:42.028 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-05-08T22:18:42.028 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 0, 2024-05-08T22:18:42.028 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 0, 2024-05-08T22:18:42.028 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:18:42.029 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:18:42.029 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-05-08T22:18:42.029 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-05-08T22:18:42.029 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 928, 2024-05-08T22:18:42.029 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 2, 2024-05-08T22:18:42.029 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-05-08T22:18:42.029 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-05-08T22:18:42.029 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-05-08T22:18:42.029 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-05-08T22:18:42.029 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-08T22:18:42.029 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:18:42.029 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-08T22:18:42.029 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-05-08T22:18:42.029 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-08T22:18:42.030 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 236, 2024-05-08T22:18:42.030 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 1393853, 2024-05-08T22:18:42.030 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-08T22:18:42.030 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 236, 2024-05-08T22:18:42.030 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-08T22:18:42.030 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-08T22:18:42.030 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 236, 2024-05-08T22:18:42.030 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 1393853, 2024-05-08T22:18:42.030 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-08T22:18:42.030 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-08T22:18:42.030 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-08T22:18:42.030 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-08T22:18:42.030 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-08T22:18:42.030 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-08T22:18:42.031 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-05-08T22:18:42.409 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-05-08T22:18:42.409 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-05-08T22:18:42.409 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-05-08T22:18:42.410 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-05-08T22:18:42.410 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-05-08T22:18:42.410 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-05-08T22:18:42.410 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-08T22:18:42.420 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-08T22:18:42.420 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-08T22:18:42.421 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:18:42.506 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-08T22:18:42.506 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:18:42.591 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:18:42.737 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:42 smithi131 ceph-mon[37260]: pgmap v355: 129 pgs: 15 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 65 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:18:42.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:42 smithi157 ceph-mon[77746]: pgmap v355: 129 pgs: 15 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 65 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:18:42.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:42 smithi132 ceph-mon[78180]: pgmap v355: 129 pgs: 15 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 65 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:18:42.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.980+0000 7f7f89600700 1 -- 172.21.15.131:0/236256708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f84115fa0 msgr2=0x7f7f84118390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:42.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.980+0000 7f7f89600700 1 --2- 172.21.15.131:0/236256708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f84115fa0 0x7f7f84118390 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c00b0a0 tx=0x7f7f7c00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:18:42.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.980+0000 7f7f89600700 1 -- 172.21.15.131:0/236256708 shutdown_connections 2024-05-08T22:18:42.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.980+0000 7f7f89600700 1 --2- 172.21.15.131:0/236256708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f84115fa0 0x7f7f84118390 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:42.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.980+0000 7f7f89600700 1 --2- 172.21.15.131:0/236256708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f8410b5d0 0x7f7f84107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:42.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.980+0000 7f7f89600700 1 --2- 172.21.15.131:0/236256708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f841084e0 0x7f7f84108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:42.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.980+0000 7f7f89600700 1 -- 172.21.15.131:0/236256708 >> 172.21.15.131:0/236256708 conn(0x7f7f8406bf20 msgr2=0x7f7f8406c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:42.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.982+0000 7f7f89600700 1 -- 172.21.15.131:0/236256708 shutdown_connections 2024-05-08T22:18:42.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.982+0000 7f7f89600700 1 -- 172.21.15.131:0/236256708 wait complete. 2024-05-08T22:18:42.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.984+0000 7f7f89600700 1 Processor -- start 2024-05-08T22:18:42.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.984+0000 7f7f89600700 1 -- start start 2024-05-08T22:18:42.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.985+0000 7f7f89600700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f841084e0 0x7f7f841a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:42.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.985+0000 7f7f89600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f8410b5d0 0x7f7f841a2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:42.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.985+0000 7f7f89600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f84115fa0 0x7f7f841a6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:42.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.985+0000 7f7f89600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f841a6cd0 con 0x7f7f8410b5d0 2024-05-08T22:18:42.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.985+0000 7f7f89600700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f841a6e40 con 0x7f7f841084e0 2024-05-08T22:18:42.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.985+0000 7f7f89600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f841a7000 con 0x7f7f84115fa0 2024-05-08T22:18:42.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.985+0000 7f7f82000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f8410b5d0 0x7f7f841a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:42.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.985+0000 7f7f83400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f84115fa0 0x7f7f841a6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:42.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.985+0000 7f7f83400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f84115fa0 0x7f7f841a6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34602/0 (socket says 172.21.15.131:34602) 2024-05-08T22:18:42.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.985+0000 7f7f83400700 1 -- 172.21.15.131:0/458428849 learned_addr learned my addr 172.21.15.131:0/458428849 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:42.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.985+0000 7f7f82a00700 1 --2- 172.21.15.131:0/458428849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f841084e0 0x7f7f841a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:42.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.986+0000 7f7f83400700 1 -- 172.21.15.131:0/458428849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f841084e0 msgr2=0x7f7f841a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:42.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.986+0000 7f7f83400700 1 --2- 172.21.15.131:0/458428849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f841084e0 0x7f7f841a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:42.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.986+0000 7f7f83400700 1 -- 172.21.15.131:0/458428849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f8410b5d0 msgr2=0x7f7f841a2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:42.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.986+0000 7f7f83400700 1 --2- 172.21.15.131:0/458428849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f8410b5d0 0x7f7f841a2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:42.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.986+0000 7f7f83400700 1 -- 172.21.15.131:0/458428849 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7c009d00 con 0x7f7f84115fa0 2024-05-08T22:18:42.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.986+0000 7f7f82a00700 1 --2- 172.21.15.131:0/458428849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f841084e0 0x7f7f841a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:18:42.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.986+0000 7f7f83400700 1 --2- 172.21.15.131:0/458428849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f84115fa0 0x7f7f841a6690 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c015000 tx=0x7f7f7c0078a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:42.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.986+0000 7f7f7b400700 1 -- 172.21.15.131:0/458428849 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c00e040 con 0x7f7f84115fa0 2024-05-08T22:18:42.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.987+0000 7f7f89600700 1 -- 172.21.15.131:0/458428849 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f841a7280 con 0x7f7f84115fa0 2024-05-08T22:18:42.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.987+0000 7f7f89600700 1 -- 172.21.15.131:0/458428849 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f841aeb20 con 0x7f7f84115fa0 2024-05-08T22:18:42.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.987+0000 7f7f7b400700 1 -- 172.21.15.131:0/458428849 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f7c004e60 con 0x7f7f84115fa0 2024-05-08T22:18:42.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.987+0000 7f7f7b400700 1 -- 172.21.15.131:0/458428849 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c00e040 con 0x7f7f84115fa0 2024-05-08T22:18:42.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.988+0000 7f7f89600700 1 -- 172.21.15.131:0/458428849 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f640052f0 con 0x7f7f84115fa0 2024-05-08T22:18:42.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.992+0000 7f7f7b400700 1 -- 172.21.15.131:0/458428849 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f7f7c005030 con 0x7f7f84115fa0 2024-05-08T22:18:42.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.995+0000 7f7f7b400700 1 --2- 172.21.15.131:0/458428849 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7f680a13d0 0x7f7f680a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:42.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.995+0000 7f7f7b400700 1 -- 172.21.15.131:0/458428849 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f7f7c0cddf0 con 0x7f7f84115fa0 2024-05-08T22:18:42.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.997+0000 7f7f82a00700 1 --2- 172.21.15.131:0/458428849 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7f680a13d0 0x7f7f680a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:42.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:42.998+0000 7f7f82a00700 1 --2- 172.21.15.131:0/458428849 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7f680a13d0 0x7f7f680a3890 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7f74005e10 tx=0x7f7f74005d80 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:43.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.003+0000 7f7f7b400700 1 -- 172.21.15.131:0/458428849 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7f7c096e30 con 0x7f7f84115fa0 2024-05-08T22:18:43.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.414+0000 7f7f89600700 1 -- 172.21.15.131:0/458428849 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f64005160 con 0x7f7f84115fa0 2024-05-08T22:18:43.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.416+0000 7f7f7b400700 1 -- 172.21.15.131:0/458428849 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+25481 (secure 0 0 0) 0x7f7f7c09a450 con 0x7f7f84115fa0 2024-05-08T22:18:43.418 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:18:43.418 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":98,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:18:08.471284+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:05:12.469858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:05:13.484446+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:05:17.016186+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/2704707055":"2024-05-09T22:07:17.096274+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6801/2485665042":"2024-05-09T22:07:17.096274+0000","172.21.15.131:0/126236497":"2024-05-09T22:07:17.096274+0000","172.21.15.131:0/25489954":"2024-05-09T22:07:17.096274+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/1773942606":"2024-05-09T22:07:17.096274+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/2485665042":"2024-05-09T22:07:17.096274+0000","172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3120268590":"2024-05-09T22:07:17.096274+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:18:43.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.420+0000 7f7f89600700 1 -- 172.21.15.131:0/458428849 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7f680a13d0 msgr2=0x7f7f680a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:43.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.421+0000 7f7f89600700 1 --2- 172.21.15.131:0/458428849 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7f680a13d0 0x7f7f680a3890 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7f74005e10 tx=0x7f7f74005d80 comp rx=0 tx=0).stop 2024-05-08T22:18:43.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.421+0000 7f7f89600700 1 -- 172.21.15.131:0/458428849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f84115fa0 msgr2=0x7f7f841a6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:43.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.421+0000 7f7f89600700 1 --2- 172.21.15.131:0/458428849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f84115fa0 0x7f7f841a6690 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c015000 tx=0x7f7f7c0078a0 comp rx=0 tx=0).stop 2024-05-08T22:18:43.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.421+0000 7f7f89600700 1 -- 172.21.15.131:0/458428849 shutdown_connections 2024-05-08T22:18:43.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.422+0000 7f7f89600700 1 --2- 172.21.15.131:0/458428849 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f7f680a13d0 0x7f7f680a3890 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:43.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.422+0000 7f7f89600700 1 --2- 172.21.15.131:0/458428849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f84115fa0 0x7f7f841a6690 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:43.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.422+0000 7f7f89600700 1 --2- 172.21.15.131:0/458428849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f841084e0 0x7f7f841a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:43.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.422+0000 7f7f89600700 1 --2- 172.21.15.131:0/458428849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f8410b5d0 0x7f7f841a2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:43.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.422+0000 7f7f89600700 1 -- 172.21.15.131:0/458428849 >> 172.21.15.131:0/458428849 conn(0x7f7f8406bf20 msgr2=0x7f7f84116380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:43.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.423+0000 7f7f89600700 1 -- 172.21.15.131:0/458428849 shutdown_connections 2024-05-08T22:18:43.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.423+0000 7f7f89600700 1 -- 172.21.15.131:0/458428849 wait complete. 2024-05-08T22:18:43.458 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:01:41.271078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:05:12.469858+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:05:13.484446+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 98, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:05:17.016186+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 98, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:18:43.458 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:18:43.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:43 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/458428849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:18:43.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.811+0000 7fb3cfc00700 1 -- 172.21.15.131:0/3513400671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3c81140f0 msgr2=0x7fb3c81164e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:43.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.811+0000 7fb3cfc00700 1 --2- 172.21.15.131:0/3513400671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3c81140f0 0x7fb3c81164e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009f90 tx=0x7fb3c000b330 comp rx=0 tx=0).stop 2024-05-08T22:18:43.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.811+0000 7fb3cfc00700 1 -- 172.21.15.131:0/3513400671 shutdown_connections 2024-05-08T22:18:43.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.812+0000 7fb3cfc00700 1 --2- 172.21.15.131:0/3513400671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3c81140f0 0x7fb3c81164e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:43.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.812+0000 7fb3cfc00700 1 --2- 172.21.15.131:0/3513400671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3c8107fa0 0x7fb3c8108380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:43.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.812+0000 7fb3cfc00700 1 --2- 172.21.15.131:0/3513400671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3c81088c0 0x7fb3c8113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:43.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.812+0000 7fb3cfc00700 1 -- 172.21.15.131:0/3513400671 >> 172.21.15.131:0/3513400671 conn(0x7fb3c806bff0 msgr2=0x7fb3c806c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:43.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.812+0000 7fb3cfc00700 1 -- 172.21.15.131:0/3513400671 shutdown_connections 2024-05-08T22:18:43.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.812+0000 7fb3cfc00700 1 -- 172.21.15.131:0/3513400671 wait complete. 2024-05-08T22:18:43.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.813+0000 7fb3cfc00700 1 Processor -- start 2024-05-08T22:18:43.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.814+0000 7fb3cfc00700 1 -- start start 2024-05-08T22:18:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.814+0000 7fb3cfc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3c8107fa0 0x7fb3c81a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.815+0000 7fb3cfc00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3c81088c0 0x7fb3c81a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.815+0000 7fb3cfc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3c81140f0 0x7fb3c81a6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.815+0000 7fb3cfc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c81a6da0 con 0x7fb3c81140f0 2024-05-08T22:18:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.815+0000 7fb3cfc00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c81a6f10 con 0x7fb3c81088c0 2024-05-08T22:18:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.815+0000 7fb3cfc00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c81a70d0 con 0x7fb3c8107fa0 2024-05-08T22:18:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.815+0000 7fb3cd200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3c8107fa0 0x7fb3c81a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.815+0000 7fb3cd200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3c8107fa0 0x7fb3c81a1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37198/0 (socket says 172.21.15.131:37198) 2024-05-08T22:18:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.815+0000 7fb3cd200700 1 -- 172.21.15.131:0/4051133085 learned_addr learned my addr 172.21.15.131:0/4051133085 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:43.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.815+0000 7fb3c7e00700 1 --2- 172.21.15.131:0/4051133085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3c81088c0 0x7fb3c81a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:43.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.815+0000 7fb3cd200700 1 -- 172.21.15.131:0/4051133085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3c81088c0 msgr2=0x7fb3c81a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:43.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.815+0000 7fb3cd200700 1 --2- 172.21.15.131:0/4051133085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3c81088c0 0x7fb3c81a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:43.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.815+0000 7fb3cd200700 1 -- 172.21.15.131:0/4051133085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3c81140f0 msgr2=0x7fb3c81a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:43.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.815+0000 7fb3cd200700 1 --2- 172.21.15.131:0/4051133085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3c81140f0 0x7fb3c81a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:43.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.816+0000 7fb3cd200700 1 -- 172.21.15.131:0/4051133085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c0009c40 con 0x7fb3c8107fa0 2024-05-08T22:18:43.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.816+0000 7fb3c7e00700 1 --2- 172.21.15.131:0/4051133085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3c81088c0 0x7fb3c81a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:18:43.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.819+0000 7fb3cd200700 1 --2- 172.21.15.131:0/4051133085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3c8107fa0 0x7fb3c81a1e80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc00d980 tx=0x7fb3bc00dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:43.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.820+0000 7fb3c5600700 1 -- 172.21.15.131:0/4051133085 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc009b40 con 0x7fb3c8107fa0 2024-05-08T22:18:43.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.820+0000 7fb3c5600700 1 -- 172.21.15.131:0/4051133085 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3bc00de60 con 0x7fb3c8107fa0 2024-05-08T22:18:43.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.820+0000 7fb3c5600700 1 -- 172.21.15.131:0/4051133085 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc00f620 con 0x7fb3c8107fa0 2024-05-08T22:18:43.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.820+0000 7fb3cfc00700 1 -- 172.21.15.131:0/4051133085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c81a73b0 con 0x7fb3c8107fa0 2024-05-08T22:18:43.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.820+0000 7fb3cfc00700 1 -- 172.21.15.131:0/4051133085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c81aebf0 con 0x7fb3c8107fa0 2024-05-08T22:18:43.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.824+0000 7fb3cfc00700 1 -- 172.21.15.131:0/4051133085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c810f300 con 0x7fb3c8107fa0 2024-05-08T22:18:43.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.830+0000 7fb3c5600700 1 -- 172.21.15.131:0/4051133085 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fb3bc010460 con 0x7fb3c8107fa0 2024-05-08T22:18:43.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.833+0000 7fb3c5600700 1 --2- 172.21.15.131:0/4051133085 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb3b00a13e0 0x7fb3b00a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:43.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.833+0000 7fb3c5600700 1 -- 172.21.15.131:0/4051133085 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fb3bc0ccf00 con 0x7fb3c8107fa0 2024-05-08T22:18:43.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.834+0000 7fb3c7e00700 1 --2- 172.21.15.131:0/4051133085 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb3b00a13e0 0x7fb3b00a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:43.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.835+0000 7fb3c7e00700 1 --2- 172.21.15.131:0/4051133085 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb3b00a13e0 0x7fb3b00a38a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb3c81a33d0 tx=0x7fb3b8005f50 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:43.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:43.836+0000 7fb3c5600700 1 -- 172.21.15.131:0/4051133085 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb3bc095f10 con 0x7fb3c8107fa0 2024-05-08T22:18:43.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:43 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/458428849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:18:43.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:43 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/458428849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:18:44.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.234+0000 7fb3cfc00700 1 -- 172.21.15.131:0/4051133085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb3c8109690 con 0x7fb3c8107fa0 2024-05-08T22:18:44.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.234+0000 7fb3c5600700 1 -- 172.21.15.131:0/4051133085 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb3bc0200a0 con 0x7fb3c8107fa0 2024-05-08T22:18:44.236 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-08T22:18:44.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.238+0000 7fb3cfc00700 1 -- 172.21.15.131:0/4051133085 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb3b00a13e0 msgr2=0x7fb3b00a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:44.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.238+0000 7fb3cfc00700 1 --2- 172.21.15.131:0/4051133085 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb3b00a13e0 0x7fb3b00a38a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb3c81a33d0 tx=0x7fb3b8005f50 comp rx=0 tx=0).stop 2024-05-08T22:18:44.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.238+0000 7fb3cfc00700 1 -- 172.21.15.131:0/4051133085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3c8107fa0 msgr2=0x7fb3c81a1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:44.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.238+0000 7fb3cfc00700 1 --2- 172.21.15.131:0/4051133085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3c8107fa0 0x7fb3c81a1e80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc00d980 tx=0x7fb3bc00dd40 comp rx=0 tx=0).stop 2024-05-08T22:18:44.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.240+0000 7fb3cfc00700 1 -- 172.21.15.131:0/4051133085 shutdown_connections 2024-05-08T22:18:44.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.240+0000 7fb3cfc00700 1 --2- 172.21.15.131:0/4051133085 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fb3b00a13e0 0x7fb3b00a38a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:44.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.240+0000 7fb3cfc00700 1 --2- 172.21.15.131:0/4051133085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3c81140f0 0x7fb3c81a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:44.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.240+0000 7fb3cfc00700 1 --2- 172.21.15.131:0/4051133085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3c8107fa0 0x7fb3c81a1e80 secure :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc00d980 tx=0x7fb3bc00dd40 comp rx=0 tx=0).stop 2024-05-08T22:18:44.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.240+0000 7fb3cfc00700 1 --2- 172.21.15.131:0/4051133085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3c81088c0 0x7fb3c81a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:44.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.240+0000 7fb3cfc00700 1 -- 172.21.15.131:0/4051133085 >> 172.21.15.131:0/4051133085 conn(0x7fb3c806bff0 msgr2=0x7fb3c806fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:44.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.241+0000 7fb3cfc00700 1 -- 172.21.15.131:0/4051133085 shutdown_connections 2024-05-08T22:18:44.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.241+0000 7fb3cfc00700 1 -- 172.21.15.131:0/4051133085 wait complete. 2024-05-08T22:18:44.275 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:18:44.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.619+0000 7f5df5c00700 1 -- 172.21.15.131:0/729729139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5df0107de0 msgr2=0x7f5df0108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:44.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.619+0000 7f5df5c00700 1 --2- 172.21.15.131:0/729729139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5df0107de0 0x7f5df0108260 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0099c0 tx=0x7f5ddc009cd0 comp rx=0 tx=0).stop 2024-05-08T22:18:44.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.619+0000 7f5df5c00700 1 -- 172.21.15.131:0/729729139 shutdown_connections 2024-05-08T22:18:44.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.619+0000 7f5df5c00700 1 --2- 172.21.15.131:0/729729139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df01087a0 0x7f5df01143e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:44.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.619+0000 7f5df5c00700 1 --2- 172.21.15.131:0/729729139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df010d520 0x7f5df010d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:44.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.619+0000 7f5df5c00700 1 --2- 172.21.15.131:0/729729139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5df0107de0 0x7f5df0108260 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:44.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.619+0000 7f5df5c00700 1 -- 172.21.15.131:0/729729139 >> 172.21.15.131:0/729729139 conn(0x7f5df0076af0 msgr2=0x7f5df0078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:44.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.619+0000 7f5df5c00700 1 -- 172.21.15.131:0/729729139 shutdown_connections 2024-05-08T22:18:44.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.620+0000 7f5df5c00700 1 -- 172.21.15.131:0/729729139 wait complete. 2024-05-08T22:18:44.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.621+0000 7f5df5c00700 1 Processor -- start 2024-05-08T22:18:44.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.621+0000 7f5df5c00700 1 -- start start 2024-05-08T22:18:44.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.622+0000 7f5df5c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df0107de0 0x7f5df00838f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:44.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.622+0000 7f5df5c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df01087a0 0x7f5df007d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:44.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.622+0000 7f5df5c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5df010d520 0x7f5df007df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:44.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.623+0000 7f5dee000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df01087a0 0x7f5df007d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:44.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.623+0000 7f5dee000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df01087a0 0x7f5df007d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37204/0 (socket says 172.21.15.131:37204) 2024-05-08T22:18:44.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.623+0000 7f5dee000700 1 -- 172.21.15.131:0/931135736 learned_addr learned my addr 172.21.15.131:0/931135736 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:44.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.623+0000 7f5df5c00700 1 -- 172.21.15.131:0/931135736 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df007e730 con 0x7f5df0107de0 2024-05-08T22:18:44.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.623+0000 7f5df5c00700 1 -- 172.21.15.131:0/931135736 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df007e8a0 con 0x7f5df010d520 2024-05-08T22:18:44.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.623+0000 7f5df5c00700 1 -- 172.21.15.131:0/931135736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df01b2700 con 0x7f5df01087a0 2024-05-08T22:18:44.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.623+0000 7f5deea00700 1 --2- 172.21.15.131:0/931135736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df0107de0 0x7f5df00838f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:44.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.623+0000 7f5def400700 1 --2- 172.21.15.131:0/931135736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5df010d520 0x7f5df007df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:44.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.624+0000 7f5dee000700 1 -- 172.21.15.131:0/931135736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5df010d520 msgr2=0x7f5df007df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:44.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.624+0000 7f5dee000700 1 --2- 172.21.15.131:0/931135736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5df010d520 0x7f5df007df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:44.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.624+0000 7f5dee000700 1 -- 172.21.15.131:0/931135736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df0107de0 msgr2=0x7f5df00838f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:44.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.624+0000 7f5dee000700 1 --2- 172.21.15.131:0/931135736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df0107de0 0x7f5df00838f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:44.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.624+0000 7f5dee000700 1 -- 172.21.15.131:0/931135736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ddc0095f0 con 0x7f5df01087a0 2024-05-08T22:18:44.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.624+0000 7f5def400700 1 --2- 172.21.15.131:0/931135736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5df010d520 0x7f5df007df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:18:44.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.624+0000 7f5deea00700 1 --2- 172.21.15.131:0/931135736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df0107de0 0x7f5df00838f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:18:44.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.625+0000 7f5dee000700 1 --2- 172.21.15.131:0/931135736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df01087a0 0x7f5df007d9e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc000c00 tx=0x7f5ddc00bbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:44.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.625+0000 7f5de7400700 1 -- 172.21.15.131:0/931135736 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc01d070 con 0x7f5df01087a0 2024-05-08T22:18:44.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.625+0000 7f5df5c00700 1 -- 172.21.15.131:0/931135736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df01b2980 con 0x7f5df01087a0 2024-05-08T22:18:44.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.625+0000 7f5de7400700 1 -- 172.21.15.131:0/931135736 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ddc003be0 con 0x7f5df01087a0 2024-05-08T22:18:44.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.625+0000 7f5de7400700 1 -- 172.21.15.131:0/931135736 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc00fb50 con 0x7f5df01087a0 2024-05-08T22:18:44.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.625+0000 7f5df5c00700 1 -- 172.21.15.131:0/931135736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df01b2d50 con 0x7f5df01087a0 2024-05-08T22:18:44.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.627+0000 7f5df5c00700 1 -- 172.21.15.131:0/931135736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5df010fb10 con 0x7f5df01087a0 2024-05-08T22:18:44.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.630+0000 7f5de7400700 1 -- 172.21.15.131:0/931135736 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f5ddc003720 con 0x7f5df01087a0 2024-05-08T22:18:44.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.632+0000 7f5de7400700 1 --2- 172.21.15.131:0/931135736 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f5dd40a1350 0x7f5dd40a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:44.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.632+0000 7f5de7400700 1 -- 172.21.15.131:0/931135736 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f5ddc0ce120 con 0x7f5df01087a0 2024-05-08T22:18:44.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.633+0000 7f5deea00700 1 --2- 172.21.15.131:0/931135736 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f5dd40a1350 0x7f5dd40a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:44.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.634+0000 7f5deea00700 1 --2- 172.21.15.131:0/931135736 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f5dd40a1350 0x7f5dd40a3810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5de0009c90 tx=0x7f5de0009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:44.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:44.639+0000 7f5de7400700 1 -- 172.21.15.131:0/931135736 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5ddc097160 con 0x7f5df01087a0 2024-05-08T22:18:44.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:44 smithi131 ceph-mon[37260]: pgmap v356: 129 pgs: 14 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 74 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T22:18:44.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:44 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/4051133085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:18:44.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:44 smithi157 ceph-mon[77746]: pgmap v356: 129 pgs: 14 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 74 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T22:18:44.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:44 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/4051133085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:18:44.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:44 smithi132 ceph-mon[78180]: pgmap v356: 129 pgs: 14 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 74 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T22:18:44.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:44 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/4051133085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:18:45.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.031+0000 7f5df5c00700 1 -- 172.21.15.131:0/931135736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5df007ee70 con 0x7f5df01087a0 2024-05-08T22:18:45.033 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-08T22:18:45.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.032+0000 7f5de7400700 1 -- 172.21.15.131:0/931135736 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5ddc016e80 con 0x7f5df01087a0 2024-05-08T22:18:45.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.036+0000 7f5df5c00700 1 -- 172.21.15.131:0/931135736 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f5dd40a1350 msgr2=0x7f5dd40a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:45.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.036+0000 7f5df5c00700 1 --2- 172.21.15.131:0/931135736 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f5dd40a1350 0x7f5dd40a3810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5de0009c90 tx=0x7f5de0009260 comp rx=0 tx=0).stop 2024-05-08T22:18:45.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.036+0000 7f5df5c00700 1 -- 172.21.15.131:0/931135736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df01087a0 msgr2=0x7f5df007d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:45.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.036+0000 7f5df5c00700 1 --2- 172.21.15.131:0/931135736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df01087a0 0x7f5df007d9e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc000c00 tx=0x7f5ddc00bbb0 comp rx=0 tx=0).stop 2024-05-08T22:18:45.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.037+0000 7f5df5c00700 1 -- 172.21.15.131:0/931135736 shutdown_connections 2024-05-08T22:18:45.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.037+0000 7f5df5c00700 1 --2- 172.21.15.131:0/931135736 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f5dd40a1350 0x7f5dd40a3810 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:45.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.037+0000 7f5df5c00700 1 --2- 172.21.15.131:0/931135736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5df010d520 0x7f5df007df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:45.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.037+0000 7f5df5c00700 1 --2- 172.21.15.131:0/931135736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df0107de0 0x7f5df00838f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:45.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.037+0000 7f5df5c00700 1 --2- 172.21.15.131:0/931135736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5df01087a0 0x7f5df007d9e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:45.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.037+0000 7f5df5c00700 1 -- 172.21.15.131:0/931135736 >> 172.21.15.131:0/931135736 conn(0x7f5df0076af0 msgr2=0x7f5df0079230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:45.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.037+0000 7f5df5c00700 1 -- 172.21.15.131:0/931135736 shutdown_connections 2024-05-08T22:18:45.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.037+0000 7f5df5c00700 1 -- 172.21.15.131:0/931135736 wait complete. 2024-05-08T22:18:45.074 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:18:45.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.412+0000 7f7161c00700 1 -- 172.21.15.131:0/1286778602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f715c105160 msgr2=0x7f715c105610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:45.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.412+0000 7f7161c00700 1 --2- 172.21.15.131:0/1286778602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f715c105160 0x7f715c105610 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f715400b0a0 tx=0x7f715400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:18:45.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.412+0000 7f7161c00700 1 -- 172.21.15.131:0/1286778602 shutdown_connections 2024-05-08T22:18:45.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.412+0000 7f7161c00700 1 --2- 172.21.15.131:0/1286778602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f715c105160 0x7f715c105610 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:45.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.412+0000 7f7161c00700 1 --2- 172.21.15.131:0/1286778602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f715c0ffd60 0x7f715c100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:45.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.412+0000 7f7161c00700 1 --2- 172.21.15.131:0/1286778602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f715c100680 0x7f715c104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:45.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.412+0000 7f7161c00700 1 -- 172.21.15.131:0/1286778602 >> 172.21.15.131:0/1286778602 conn(0x7f715c0fb770 msgr2=0x7f715c0fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:45.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.414+0000 7f7161c00700 1 -- 172.21.15.131:0/1286778602 shutdown_connections 2024-05-08T22:18:45.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.414+0000 7f7161c00700 1 -- 172.21.15.131:0/1286778602 wait complete. 2024-05-08T22:18:45.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.416+0000 7f7161c00700 1 Processor -- start 2024-05-08T22:18:45.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.416+0000 7f7161c00700 1 -- start start 2024-05-08T22:18:45.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.417+0000 7f7161c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f715c0ffd60 0x7f715c1a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:45.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.417+0000 7f7161c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f715c100680 0x7f715c1a3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:45.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.417+0000 7f7161c00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f715c105160 0x7f715c19d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:45.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.417+0000 7f7161c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c19de20 con 0x7f715c100680 2024-05-08T22:18:45.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.417+0000 7f7161c00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c19df90 con 0x7f715c105160 2024-05-08T22:18:45.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.417+0000 7f7161c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c19e150 con 0x7f715c0ffd60 2024-05-08T22:18:45.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.418+0000 7f715b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f715c100680 0x7f715c1a3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:45.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.418+0000 7f715b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f715c100680 0x7f715c1a3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45518/0 (socket says 172.21.15.131:45518) 2024-05-08T22:18:45.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.418+0000 7f715b400700 1 -- 172.21.15.131:0/767589109 learned_addr learned my addr 172.21.15.131:0/767589109 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:45.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.418+0000 7f7161200700 1 --2- 172.21.15.131:0/767589109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f715c105160 0x7f715c19d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:45.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.418+0000 7f715b400700 1 -- 172.21.15.131:0/767589109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f715c0ffd60 msgr2=0x7f715c1a3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:45.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.418+0000 7f715b400700 1 --2- 172.21.15.131:0/767589109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f715c0ffd60 0x7f715c1a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:45.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.418+0000 7f715b400700 1 -- 172.21.15.131:0/767589109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f715c105160 msgr2=0x7f715c19d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:45.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.418+0000 7f715b400700 1 --2- 172.21.15.131:0/767589109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f715c105160 0x7f715c19d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:45.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.418+0000 7f715b400700 1 -- 172.21.15.131:0/767589109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7154009d00 con 0x7f715c100680 2024-05-08T22:18:45.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.419+0000 7f7161200700 1 --2- 172.21.15.131:0/767589109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f715c105160 0x7f715c19d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:18:45.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.419+0000 7f715b400700 1 --2- 172.21.15.131:0/767589109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f715c100680 0x7f715c1a3bc0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f714c00e9f0 tx=0x7f714c00edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:45.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.422+0000 7f7158c00700 1 -- 172.21.15.131:0/767589109 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c0096e0 con 0x7f715c100680 2024-05-08T22:18:45.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.422+0000 7f7158c00700 1 -- 172.21.15.131:0/767589109 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f714c00cbd0 con 0x7f715c100680 2024-05-08T22:18:45.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.422+0000 7f7158c00700 1 -- 172.21.15.131:0/767589109 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c018680 con 0x7f715c100680 2024-05-08T22:18:45.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.422+0000 7f7161c00700 1 -- 172.21.15.131:0/767589109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715c19e430 con 0x7f715c100680 2024-05-08T22:18:45.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.422+0000 7f7161c00700 1 -- 172.21.15.131:0/767589109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715c1aa670 con 0x7f715c100680 2024-05-08T22:18:45.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.425+0000 7f7161c00700 1 -- 172.21.15.131:0/767589109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f715c0688d0 con 0x7f715c100680 2024-05-08T22:18:45.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.427+0000 7f7158c00700 1 -- 172.21.15.131:0/767589109 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f714c00cd40 con 0x7f715c100680 2024-05-08T22:18:45.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.430+0000 7f7158c00700 1 --2- 172.21.15.131:0/767589109 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f71400a1460 0x7f71400a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:45.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.430+0000 7f7158c00700 1 -- 172.21.15.131:0/767589109 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f714c014070 con 0x7f715c100680 2024-05-08T22:18:45.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.430+0000 7f715be00700 1 --2- 172.21.15.131:0/767589109 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f71400a1460 0x7f71400a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:45.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.433+0000 7f715be00700 1 --2- 172.21.15.131:0/767589109 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f71400a1460 0x7f71400a3920 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7148005950 tx=0x7f71480058e0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:45.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.438+0000 7f7158c00700 1 -- 172.21.15.131:0/767589109 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f714c052d60 con 0x7f715c100680 2024-05-08T22:18:45.781 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:45 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/931135736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:18:45.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.829+0000 7f7161c00700 1 -- 172.21.15.131:0/767589109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f715c19ec70 con 0x7f715c100680 2024-05-08T22:18:45.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.830+0000 7f7158c00700 1 -- 172.21.15.131:0/767589109 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) v1 ==== 100+0+11 (secure 0 0 0) 0x7f714c096210 con 0x7f715c100680 2024-05-08T22:18:45.831 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-08T22:18:45.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.835+0000 7f7161c00700 1 -- 172.21.15.131:0/767589109 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f71400a1460 msgr2=0x7f71400a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:45.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.835+0000 7f7161c00700 1 --2- 172.21.15.131:0/767589109 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f71400a1460 0x7f71400a3920 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7148005950 tx=0x7f71480058e0 comp rx=0 tx=0).stop 2024-05-08T22:18:45.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.835+0000 7f7161c00700 1 -- 172.21.15.131:0/767589109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f715c100680 msgr2=0x7f715c1a3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:45.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.835+0000 7f7161c00700 1 --2- 172.21.15.131:0/767589109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f715c100680 0x7f715c1a3bc0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f714c00e9f0 tx=0x7f714c00edb0 comp rx=0 tx=0).stop 2024-05-08T22:18:45.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.835+0000 7f7161c00700 1 -- 172.21.15.131:0/767589109 shutdown_connections 2024-05-08T22:18:45.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.835+0000 7f7161c00700 1 --2- 172.21.15.131:0/767589109 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f71400a1460 0x7f71400a3920 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:45.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.835+0000 7f7161c00700 1 --2- 172.21.15.131:0/767589109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f715c105160 0x7f715c19d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:45.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.835+0000 7f7161c00700 1 --2- 172.21.15.131:0/767589109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f715c0ffd60 0x7f715c1a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:45.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.835+0000 7f7161c00700 1 --2- 172.21.15.131:0/767589109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f715c100680 0x7f715c1a3bc0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:45.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.835+0000 7f7161c00700 1 -- 172.21.15.131:0/767589109 >> 172.21.15.131:0/767589109 conn(0x7f715c0fb770 msgr2=0x7f715c0fc760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:45.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.836+0000 7f7161c00700 1 -- 172.21.15.131:0/767589109 shutdown_connections 2024-05-08T22:18:45.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:45.836+0000 7f7161c00700 1 -- 172.21.15.131:0/767589109 wait complete. 2024-05-08T22:18:45.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:45 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/931135736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:18:45.873 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:45 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/931135736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:18:45.877 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:18:46.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.215+0000 7fcda3a00700 1 -- 172.21.15.131:0/1680721692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd9c107de0 msgr2=0x7fcd9c108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:46.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.215+0000 7fcda3a00700 1 --2- 172.21.15.131:0/1680721692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd9c107de0 0x7fcd9c108260 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcd9000b0a0 tx=0x7fcd9000b3b0 comp rx=0 tx=0).stop 2024-05-08T22:18:46.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.216+0000 7fcda3a00700 1 -- 172.21.15.131:0/1680721692 shutdown_connections 2024-05-08T22:18:46.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.216+0000 7fcda3a00700 1 --2- 172.21.15.131:0/1680721692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd9c1087a0 0x7fcd9c1143e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:46.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.216+0000 7fcda3a00700 1 --2- 172.21.15.131:0/1680721692 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd9c10d520 0x7fcd9c10d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:46.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.216+0000 7fcda3a00700 1 --2- 172.21.15.131:0/1680721692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd9c107de0 0x7fcd9c108260 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:46.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.216+0000 7fcda3a00700 1 -- 172.21.15.131:0/1680721692 >> 172.21.15.131:0/1680721692 conn(0x7fcd9c076af0 msgr2=0x7fcd9c078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:46.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.217+0000 7fcda3a00700 1 -- 172.21.15.131:0/1680721692 shutdown_connections 2024-05-08T22:18:46.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.217+0000 7fcda3a00700 1 -- 172.21.15.131:0/1680721692 wait complete. 2024-05-08T22:18:46.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.222+0000 7fcda3a00700 1 Processor -- start 2024-05-08T22:18:46.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.222+0000 7fcda3a00700 1 -- start start 2024-05-08T22:18:46.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.223+0000 7fcda3a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd9c107de0 0x7fcd9c072570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:46.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.223+0000 7fcda3a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd9c1087a0 0x7fcd9c072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:46.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.223+0000 7fcda3a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd9c10d520 0x7fcd9c06b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:46.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.223+0000 7fcda3a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c06bf00 con 0x7fcd9c10d520 2024-05-08T22:18:46.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.223+0000 7fcda3a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c06c070 con 0x7fcd9c107de0 2024-05-08T22:18:46.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.223+0000 7fcda3a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c06c230 con 0x7fcd9c1087a0 2024-05-08T22:18:46.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.224+0000 7fcd9be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd9c1087a0 0x7fcd9c072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:46.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.224+0000 7fcd9be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd9c1087a0 0x7fcd9c072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37240/0 (socket says 172.21.15.131:37240) 2024-05-08T22:18:46.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.224+0000 7fcd9be00700 1 -- 172.21.15.131:0/3521899554 learned_addr learned my addr 172.21.15.131:0/3521899554 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:46.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.224+0000 7fcda1000700 1 --2- 172.21.15.131:0/3521899554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd9c107de0 0x7fcd9c072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:46.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.224+0000 7fcda1000700 1 -- 172.21.15.131:0/3521899554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd9c1087a0 msgr2=0x7fcd9c072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:46.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.224+0000 7fcda1000700 1 --2- 172.21.15.131:0/3521899554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd9c1087a0 0x7fcd9c072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:46.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.224+0000 7fcda1000700 1 -- 172.21.15.131:0/3521899554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd9c10d520 msgr2=0x7fcd9c06b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:46.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.224+0000 7fcda1a00700 1 --2- 172.21.15.131:0/3521899554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd9c10d520 0x7fcd9c06b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:46.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.225+0000 7fcda1000700 1 --2- 172.21.15.131:0/3521899554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd9c10d520 0x7fcd9c06b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:46.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.225+0000 7fcda1000700 1 -- 172.21.15.131:0/3521899554 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd90009d00 con 0x7fcd9c107de0 2024-05-08T22:18:46.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.225+0000 7fcda1a00700 1 --2- 172.21.15.131:0/3521899554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd9c10d520 0x7fcd9c06b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:18:46.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.225+0000 7fcda1000700 1 --2- 172.21.15.131:0/3521899554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd9c107de0 0x7fcd9c072570 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00c970 tx=0x7fcd8c00cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:46.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.226+0000 7fcd99600700 1 -- 172.21.15.131:0/3521899554 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c007b00 con 0x7fcd9c107de0 2024-05-08T22:18:46.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.226+0000 7fcd99600700 1 -- 172.21.15.131:0/3521899554 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd8c004cf0 con 0x7fcd9c107de0 2024-05-08T22:18:46.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.226+0000 7fcd99600700 1 -- 172.21.15.131:0/3521899554 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c0054b0 con 0x7fcd9c107de0 2024-05-08T22:18:46.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.226+0000 7fcda3a00700 1 -- 172.21.15.131:0/3521899554 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd9c06c510 con 0x7fcd9c107de0 2024-05-08T22:18:46.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.226+0000 7fcda3a00700 1 -- 172.21.15.131:0/3521899554 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd9c070480 con 0x7fcd9c107de0 2024-05-08T22:18:46.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.228+0000 7fcda3a00700 1 -- 172.21.15.131:0/3521899554 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd9c108df0 con 0x7fcd9c107de0 2024-05-08T22:18:46.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.231+0000 7fcd99600700 1 -- 172.21.15.131:0/3521899554 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fcd8c004750 con 0x7fcd9c107de0 2024-05-08T22:18:46.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.234+0000 7fcd99600700 1 --2- 172.21.15.131:0/3521899554 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fcd840a13d0 0x7fcd840a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:46.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.234+0000 7fcd99600700 1 -- 172.21.15.131:0/3521899554 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fcd8c0ccc80 con 0x7fcd9c107de0 2024-05-08T22:18:46.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.234+0000 7fcd9be00700 1 --2- 172.21.15.131:0/3521899554 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fcd840a13d0 0x7fcd840a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:46.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.238+0000 7fcd99600700 1 -- 172.21.15.131:0/3521899554 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcd8c0d3080 con 0x7fcd9c107de0 2024-05-08T22:18:46.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.239+0000 7fcd9be00700 1 --2- 172.21.15.131:0/3521899554 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fcd840a13d0 0x7fcd840a3890 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcd900060b0 tx=0x7fcd90006000 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:46.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.625+0000 7fcda3a00700 1 -- 172.21.15.131:0/3521899554 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcd9c070ed0 con 0x7fcd9c107de0 2024-05-08T22:18:46.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.626+0000 7fcd99600700 1 -- 172.21.15.131:0/3521899554 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v98) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcd8c095c30 con 0x7fcd9c107de0 2024-05-08T22:18:46.628 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-08T22:18:46.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.630+0000 7fcda3a00700 1 -- 172.21.15.131:0/3521899554 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fcd840a13d0 msgr2=0x7fcd840a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:46.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.630+0000 7fcda3a00700 1 --2- 172.21.15.131:0/3521899554 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fcd840a13d0 0x7fcd840a3890 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcd900060b0 tx=0x7fcd90006000 comp rx=0 tx=0).stop 2024-05-08T22:18:46.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.630+0000 7fcda3a00700 1 -- 172.21.15.131:0/3521899554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd9c107de0 msgr2=0x7fcd9c072570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:46.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.630+0000 7fcda3a00700 1 --2- 172.21.15.131:0/3521899554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd9c107de0 0x7fcd9c072570 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00c970 tx=0x7fcd8c00cd30 comp rx=0 tx=0).stop 2024-05-08T22:18:46.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.630+0000 7fcda3a00700 1 -- 172.21.15.131:0/3521899554 shutdown_connections 2024-05-08T22:18:46.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.630+0000 7fcda3a00700 1 --2- 172.21.15.131:0/3521899554 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fcd840a13d0 0x7fcd840a3890 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:46.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.630+0000 7fcda3a00700 1 --2- 172.21.15.131:0/3521899554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd9c10d520 0x7fcd9c06b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:46.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.630+0000 7fcda3a00700 1 --2- 172.21.15.131:0/3521899554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd9c107de0 0x7fcd9c072570 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:46.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.630+0000 7fcda3a00700 1 --2- 172.21.15.131:0/3521899554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd9c1087a0 0x7fcd9c072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:46.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.630+0000 7fcda3a00700 1 -- 172.21.15.131:0/3521899554 >> 172.21.15.131:0/3521899554 conn(0x7fcd9c076af0 msgr2=0x7fcd9c0792d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:46.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.631+0000 7fcda3a00700 1 -- 172.21.15.131:0/3521899554 shutdown_connections 2024-05-08T22:18:46.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.631+0000 7fcda3a00700 1 -- 172.21.15.131:0/3521899554 wait complete. 2024-05-08T22:18:46.663 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:18:46.758 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:46 smithi131 ceph-mon[37260]: pgmap v357: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:18:46.758 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:46 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/767589109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:18:46.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:46 smithi157 ceph-mon[77746]: pgmap v357: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:18:46.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:46 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/767589109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:18:46.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:46 smithi132 ceph-mon[78180]: pgmap v357: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T22:18:46.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:46 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/767589109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:18:46.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.983+0000 7f3db3200700 1 -- 172.21.15.131:0/1885161309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dac106280 msgr2=0x7f3dac10de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:46.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.983+0000 7f3db3200700 1 --2- 172.21.15.131:0/1885161309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dac106280 0x7f3dac10de00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3da400b0a0 tx=0x7f3da400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:18:46.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.983+0000 7f3db3200700 1 -- 172.21.15.131:0/1885161309 shutdown_connections 2024-05-08T22:18:46.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.983+0000 7f3db3200700 1 --2- 172.21.15.131:0/1885161309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dac106280 0x7f3dac10de00 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:46.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.983+0000 7f3db3200700 1 --2- 172.21.15.131:0/1885161309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dac101160 0x7f3dac101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:46.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.983+0000 7f3db3200700 1 --2- 172.21.15.131:0/1885161309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dac101b10 0x7f3dac105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:46.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.983+0000 7f3db3200700 1 -- 172.21.15.131:0/1885161309 >> 172.21.15.131:0/1885161309 conn(0x7f3dac0fc9c0 msgr2=0x7f3dac0fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:46.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.983+0000 7f3db3200700 1 -- 172.21.15.131:0/1885161309 shutdown_connections 2024-05-08T22:18:46.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.983+0000 7f3db3200700 1 -- 172.21.15.131:0/1885161309 wait complete. 2024-05-08T22:18:46.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.985+0000 7f3db3200700 1 Processor -- start 2024-05-08T22:18:46.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.985+0000 7f3db3200700 1 -- start start 2024-05-08T22:18:46.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.985+0000 7f3db3200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dac101160 0x7f3dac1151f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:46.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.985+0000 7f3db3200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dac106280 0x7f3dac11af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:46.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.986+0000 7f3db3200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dac1158b0 0x7f3dac1103a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:46.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.986+0000 7f3db3200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac1109f0 con 0x7f3dac101160 2024-05-08T22:18:46.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.986+0000 7f3db3200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac110b60 con 0x7f3dac1158b0 2024-05-08T22:18:46.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.986+0000 7f3db3200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac110d20 con 0x7f3dac106280 2024-05-08T22:18:46.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.986+0000 7f3dab400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dac106280 0x7f3dac11af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:46.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.986+0000 7f3dab400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dac106280 0x7f3dac11af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37258/0 (socket says 172.21.15.131:37258) 2024-05-08T22:18:46.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.986+0000 7f3dab400700 1 -- 172.21.15.131:0/974284146 learned_addr learned my addr 172.21.15.131:0/974284146 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:46.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.986+0000 7f3db1200700 1 --2- 172.21.15.131:0/974284146 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dac1158b0 0x7f3dac1103a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:46.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.986+0000 7f3dabe00700 1 --2- 172.21.15.131:0/974284146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dac101160 0x7f3dac1151f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:46.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.987+0000 7f3dab400700 1 -- 172.21.15.131:0/974284146 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dac1158b0 msgr2=0x7f3dac1103a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:46.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.987+0000 7f3dab400700 1 --2- 172.21.15.131:0/974284146 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dac1158b0 0x7f3dac1103a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:46.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.987+0000 7f3dab400700 1 -- 172.21.15.131:0/974284146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dac101160 msgr2=0x7f3dac1151f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:46.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.987+0000 7f3dab400700 1 --2- 172.21.15.131:0/974284146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dac101160 0x7f3dac1151f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:46.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.987+0000 7f3dab400700 1 -- 172.21.15.131:0/974284146 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da4009d00 con 0x7f3dac106280 2024-05-08T22:18:46.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.987+0000 7f3dab400700 1 --2- 172.21.15.131:0/974284146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dac106280 0x7f3dac11af70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800d980 tx=0x7f3d9800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:46.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.987+0000 7f3da8c00700 1 -- 172.21.15.131:0/974284146 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d98009930 con 0x7f3dac106280 2024-05-08T22:18:46.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.987+0000 7f3db3200700 1 -- 172.21.15.131:0/974284146 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dac111000 con 0x7f3dac106280 2024-05-08T22:18:46.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.988+0000 7f3db3200700 1 -- 172.21.15.131:0/974284146 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dac1b33f0 con 0x7f3dac106280 2024-05-08T22:18:46.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.988+0000 7f3da8c00700 1 -- 172.21.15.131:0/974284146 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d98010460 con 0x7f3dac106280 2024-05-08T22:18:46.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.988+0000 7f3da8c00700 1 -- 172.21.15.131:0/974284146 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9800f620 con 0x7f3dac106280 2024-05-08T22:18:46.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.989+0000 7f3db3200700 1 -- 172.21.15.131:0/974284146 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d8c0052f0 con 0x7f3dac106280 2024-05-08T22:18:46.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.992+0000 7f3da8c00700 1 -- 172.21.15.131:0/974284146 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f3d98009a90 con 0x7f3dac106280 2024-05-08T22:18:46.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.994+0000 7f3da8c00700 1 --2- 172.21.15.131:0/974284146 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3d900a13d0 0x7f3d900a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:46.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.995+0000 7f3da8c00700 1 -- 172.21.15.131:0/974284146 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f3d980cce50 con 0x7f3dac106280 2024-05-08T22:18:46.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.995+0000 7f3dabe00700 1 --2- 172.21.15.131:0/974284146 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3d900a13d0 0x7f3d900a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:46.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.996+0000 7f3dabe00700 1 --2- 172.21.15.131:0/974284146 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3d900a13d0 0x7f3d900a3890 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c009910 tx=0x7f3d9c008040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:46.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:46.997+0000 7f3da8c00700 1 -- 172.21.15.131:0/974284146 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3d98099910 con 0x7f3dac106280 2024-05-08T22:18:47.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:47.412+0000 7f3db3200700 1 -- 172.21.15.131:0/974284146 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3d8c005160 con 0x7f3dac106280 2024-05-08T22:18:47.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:47.415+0000 7f3da8c00700 1 -- 172.21.15.131:0/974284146 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v98) v1 ==== 81+0+1017 (secure 0 0 0) 0x7f3d98017020 con 0x7f3dac106280 2024-05-08T22:18:47.417 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/2704707055 2024-05-09T22:07:17.096274+0000 2024-05-08T22:18:47.418 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6800/1564023164 2024-05-09T21:59:15.548383+0000 2024-05-08T22:18:47.418 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/3082367829 2024-05-09T21:59:40.641373+0000 2024-05-08T22:18:47.418 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6801/2485665042 2024-05-09T22:07:17.096274+0000 2024-05-08T22:18:47.418 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/126236497 2024-05-09T22:07:17.096274+0000 2024-05-08T22:18:47.418 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/25489954 2024-05-09T22:07:17.096274+0000 2024-05-08T22:18:47.418 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6801/1564023164 2024-05-09T21:59:15.548383+0000 2024-05-08T22:18:47.418 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/1773942606 2024-05-09T22:07:17.096274+0000 2024-05-08T22:18:47.418 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/1981506111 2024-05-09T21:59:15.548383+0000 2024-05-08T22:18:47.418 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/1716895483 2024-05-09T21:59:40.641373+0000 2024-05-08T22:18:47.419 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/1241655400 2024-05-09T21:59:40.641373+0000 2024-05-08T22:18:47.419 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6800/1418340798 2024-05-09T21:59:40.641373+0000 2024-05-08T22:18:47.419 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6800/2485665042 2024-05-09T22:07:17.096274+0000 2024-05-08T22:18:47.419 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6801/1418340798 2024-05-09T21:59:40.641373+0000 2024-05-08T22:18:47.419 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/3309822030 2024-05-09T21:59:15.548383+0000 2024-05-08T22:18:47.419 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/976983862 2024-05-09T21:59:15.548383+0000 2024-05-08T22:18:47.419 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/3120268590 2024-05-09T22:07:17.096274+0000 2024-05-08T22:18:47.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:47.419+0000 7f3db3200700 1 -- 172.21.15.131:0/974284146 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3d900a13d0 msgr2=0x7f3d900a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:47.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:47.419+0000 7f3db3200700 1 --2- 172.21.15.131:0/974284146 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3d900a13d0 0x7f3d900a3890 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c009910 tx=0x7f3d9c008040 comp rx=0 tx=0).stop 2024-05-08T22:18:47.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:47.419+0000 7f3db3200700 1 -- 172.21.15.131:0/974284146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dac106280 msgr2=0x7f3dac11af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:47.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:47.419+0000 7f3db3200700 1 --2- 172.21.15.131:0/974284146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dac106280 0x7f3dac11af70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800d980 tx=0x7f3d9800dc90 comp rx=0 tx=0).stop 2024-05-08T22:18:47.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:47.420+0000 7f3db3200700 1 -- 172.21.15.131:0/974284146 shutdown_connections 2024-05-08T22:18:47.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:47.420+0000 7f3db3200700 1 --2- 172.21.15.131:0/974284146 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f3d900a13d0 0x7f3d900a3890 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:47.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:47.420+0000 7f3db3200700 1 --2- 172.21.15.131:0/974284146 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dac1158b0 0x7f3dac1103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:47.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:47.420+0000 7f3db3200700 1 --2- 172.21.15.131:0/974284146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dac101160 0x7f3dac1151f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:47.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:47.420+0000 7f3db3200700 1 --2- 172.21.15.131:0/974284146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dac106280 0x7f3dac11af70 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:47.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:47.421+0000 7f3db3200700 1 -- 172.21.15.131:0/974284146 >> 172.21.15.131:0/974284146 conn(0x7f3dac0fc9c0 msgr2=0x7f3dac0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:47.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:47.421+0000 7f3db3200700 1 -- 172.21.15.131:0/974284146 shutdown_connections 2024-05-08T22:18:47.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:47.421+0000 7f3db3200700 1 -- 172.21.15.131:0/974284146 wait complete. 2024-05-08T22:18:47.424 INFO:teuthology.orchestra.run.smithi131.stderr:listed 17 entries 2024-05-08T22:18:47.453 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T22:18:47.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:18:47.454 DEBUG:teuthology.orchestra.run.smithi131:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T22:18:47.716 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:47 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3521899554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:18:47.717 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:47 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/974284146' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T22:18:47.722 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-05-08T22:18:47.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:18:47.722 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T22:18:47.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:47 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3521899554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:18:47.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:47 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/974284146' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T22:18:47.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:18:47.817 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-05-08T22:18:47.843 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:47 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3521899554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T22:18:47.843 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:47 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/974284146' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T22:18:47.915 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T22:18:47.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:18:47.915 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T22:18:48.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:18:48.051 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link delete ceph-brx 2024-05-08T22:18:48.206 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-08T22:18:48.217 INFO:tasks.cephadm:Teardown begin 2024-05-08T22:18:48.217 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T22:18:48.217 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:18:48.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.532+0000 7f56aee00700 1 -- 172.21.15.131:0/677458912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56a8107fa0 msgr2=0x7f56a8108420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:48.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.532+0000 7f56aee00700 1 --2- 172.21.15.131:0/677458912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56a8107fa0 0x7f56a8108420 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f56a000b0a0 tx=0x7f56a000b3b0 comp rx=0 tx=0).stop 2024-05-08T22:18:48.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.532+0000 7f56aee00700 1 -- 172.21.15.131:0/677458912 shutdown_connections 2024-05-08T22:18:48.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.532+0000 7f56aee00700 1 --2- 172.21.15.131:0/677458912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56a8107fa0 0x7f56a8108420 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:48.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.532+0000 7f56aee00700 1 --2- 172.21.15.131:0/677458912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56a810f5c0 0x7f56a810f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:48.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.532+0000 7f56aee00700 1 --2- 172.21.15.131:0/677458912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56a810d2b0 0x7f56a810d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:48.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.532+0000 7f56aee00700 1 -- 172.21.15.131:0/677458912 >> 172.21.15.131:0/677458912 conn(0x7f56a806bf20 msgr2=0x7f56a806c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:48.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.532+0000 7f56aee00700 1 -- 172.21.15.131:0/677458912 shutdown_connections 2024-05-08T22:18:48.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.533+0000 7f56aee00700 1 -- 172.21.15.131:0/677458912 wait complete. 2024-05-08T22:18:48.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.534+0000 7f56aee00700 1 Processor -- start 2024-05-08T22:18:48.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.534+0000 7f56aee00700 1 -- start start 2024-05-08T22:18:48.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.535+0000 7f56aee00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56a8107fa0 0x7f56a81a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:48.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.535+0000 7f56aee00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56a810d2b0 0x7f56a81a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:48.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.535+0000 7f56aee00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56a810f5c0 0x7f56a81aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:48.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.535+0000 7f56aee00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a81ab2e0 con 0x7f56a810d2b0 2024-05-08T22:18:48.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.535+0000 7f56aee00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a81ab450 con 0x7f56a810f5c0 2024-05-08T22:18:48.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.535+0000 7f56aee00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a81ab610 con 0x7f56a8107fa0 2024-05-08T22:18:48.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.535+0000 7f56a7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56a8107fa0 0x7f56a81a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:48.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.535+0000 7f56a7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56a8107fa0 0x7f56a81a63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37274/0 (socket says 172.21.15.131:37274) 2024-05-08T22:18:48.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.535+0000 7f56a7400700 1 --2- 172.21.15.131:0/2451517578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56a810d2b0 0x7f56a81a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:48.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.535+0000 7f56a7e00700 1 -- 172.21.15.131:0/2451517578 learned_addr learned my addr 172.21.15.131:0/2451517578 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:48.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.535+0000 7f56ace00700 1 --2- 172.21.15.131:0/2451517578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56a810f5c0 0x7f56a81aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:48.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.536+0000 7f56a7e00700 1 -- 172.21.15.131:0/2451517578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56a810f5c0 msgr2=0x7f56a81aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:48.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.536+0000 7f56a7e00700 1 --2- 172.21.15.131:0/2451517578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56a810f5c0 0x7f56a81aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:48.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.536+0000 7f56a7e00700 1 -- 172.21.15.131:0/2451517578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56a810d2b0 msgr2=0x7f56a81a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:48.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.537+0000 7f56a7e00700 1 --2- 172.21.15.131:0/2451517578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56a810d2b0 0x7f56a81a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:48.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.537+0000 7f56a7e00700 1 -- 172.21.15.131:0/2451517578 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a0009d00 con 0x7f56a8107fa0 2024-05-08T22:18:48.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.537+0000 7f56a7e00700 1 --2- 172.21.15.131:0/2451517578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56a8107fa0 0x7f56a81a63c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f569800d950 tx=0x7f569800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:48.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.538+0000 7f56a4c00700 1 -- 172.21.15.131:0/2451517578 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569800f890 con 0x7f56a8107fa0 2024-05-08T22:18:48.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.538+0000 7f56a4c00700 1 -- 172.21.15.131:0/2451517578 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5698010460 con 0x7f56a8107fa0 2024-05-08T22:18:48.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.538+0000 7f56a4c00700 1 -- 172.21.15.131:0/2451517578 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569800e610 con 0x7f56a8107fa0 2024-05-08T22:18:48.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.538+0000 7f56aee00700 1 -- 172.21.15.131:0/2451517578 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a81ab8f0 con 0x7f56a8107fa0 2024-05-08T22:18:48.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.538+0000 7f56aee00700 1 -- 172.21.15.131:0/2451517578 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a81b3130 con 0x7f56a8107fa0 2024-05-08T22:18:48.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.541+0000 7f56aee00700 1 -- 172.21.15.131:0/2451517578 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56a806d280 con 0x7f56a8107fa0 2024-05-08T22:18:48.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.543+0000 7f56a4c00700 1 -- 172.21.15.131:0/2451517578 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f56980108d0 con 0x7f56a8107fa0 2024-05-08T22:18:48.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.545+0000 7f56a4c00700 1 --2- 172.21.15.131:0/2451517578 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f568c0a1460 0x7f568c0a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:48.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.546+0000 7f56a4c00700 1 -- 172.21.15.131:0/2451517578 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f56980cce40 con 0x7f56a8107fa0 2024-05-08T22:18:48.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.546+0000 7f56a7400700 1 --2- 172.21.15.131:0/2451517578 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f568c0a1460 0x7f568c0a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:48.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.547+0000 7f56a7400700 1 --2- 172.21.15.131:0/2451517578 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f568c0a1460 0x7f568c0a3920 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f56a81a7910 tx=0x7f5694009190 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:48.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:48.552+0000 7f56a4c00700 1 -- 172.21.15.131:0/2451517578 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5698095df0 con 0x7f56a8107fa0 2024-05-08T22:18:48.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:48 smithi131 ceph-mon[37260]: pgmap v358: 129 pgs: 11 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 91 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:18:48.779 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:48 smithi157 ceph-mon[77746]: pgmap v358: 129 pgs: 11 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 91 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:18:48.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:48 smithi132 ceph-mon[78180]: pgmap v358: 129 pgs: 11 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 91 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T22:18:49.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.006+0000 7f56aee00700 1 -- 172.21.15.131:0/2451517578 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56a8067c10 con 0x7f56a8107fa0 2024-05-08T22:18:49.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.007+0000 7f56a4c00700 1 -- 172.21.15.131:0/2451517578 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10891 (secure 0 0 0) 0x7f5698020090 con 0x7f56a8107fa0 2024-05-08T22:18:49.009 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:18:49.009 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:05:14.034908+0000","modified":"2024-05-08T22:16:38.400949+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:18:49.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.011+0000 7f56aee00700 1 -- 172.21.15.131:0/2451517578 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f568c0a1460 msgr2=0x7f568c0a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:49.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.011+0000 7f56aee00700 1 --2- 172.21.15.131:0/2451517578 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f568c0a1460 0x7f568c0a3920 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f56a81a7910 tx=0x7f5694009190 comp rx=0 tx=0).stop 2024-05-08T22:18:49.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.011+0000 7f56aee00700 1 -- 172.21.15.131:0/2451517578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56a8107fa0 msgr2=0x7f56a81a63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:49.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.011+0000 7f56aee00700 1 --2- 172.21.15.131:0/2451517578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56a8107fa0 0x7f56a81a63c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f569800d950 tx=0x7f569800dc60 comp rx=0 tx=0).stop 2024-05-08T22:18:49.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.011+0000 7f56aee00700 1 -- 172.21.15.131:0/2451517578 shutdown_connections 2024-05-08T22:18:49.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.011+0000 7f56aee00700 1 --2- 172.21.15.131:0/2451517578 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f568c0a1460 0x7f568c0a3920 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:49.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.011+0000 7f56aee00700 1 --2- 172.21.15.131:0/2451517578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56a810f5c0 0x7f56a81aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:49.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.011+0000 7f56aee00700 1 --2- 172.21.15.131:0/2451517578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56a8107fa0 0x7f56a81a63c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:49.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.011+0000 7f56aee00700 1 --2- 172.21.15.131:0/2451517578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56a810d2b0 0x7f56a81a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:49.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.011+0000 7f56aee00700 1 -- 172.21.15.131:0/2451517578 >> 172.21.15.131:0/2451517578 conn(0x7f56a806bf20 msgr2=0x7f56a810aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:49.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.011+0000 7f56aee00700 1 -- 172.21.15.131:0/2451517578 shutdown_connections 2024-05-08T22:18:49.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.011+0000 7f56aee00700 1 -- 172.21.15.131:0/2451517578 wait complete. 2024-05-08T22:18:49.016 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 30 2024-05-08T22:18:49.043 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:18:49.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.349+0000 7f087dc00700 1 -- 172.21.15.131:0/3358730578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0878107fa0 msgr2=0x7f0878108420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:49.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.349+0000 7f087dc00700 1 --2- 172.21.15.131:0/3358730578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0878107fa0 0x7f0878108420 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0868009f90 tx=0x7f086800b330 comp rx=0 tx=0).stop 2024-05-08T22:18:49.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.349+0000 7f087dc00700 1 -- 172.21.15.131:0/3358730578 shutdown_connections 2024-05-08T22:18:49.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.349+0000 7f087dc00700 1 --2- 172.21.15.131:0/3358730578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0878108960 0x7f0878114700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:49.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.349+0000 7f087dc00700 1 --2- 172.21.15.131:0/3358730578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f087810d8f0 0x7f087810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:49.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.349+0000 7f087dc00700 1 --2- 172.21.15.131:0/3358730578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0878107fa0 0x7f0878108420 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:49.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.349+0000 7f087dc00700 1 -- 172.21.15.131:0/3358730578 >> 172.21.15.131:0/3358730578 conn(0x7f087806bf20 msgr2=0x7f087806c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:49.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.350+0000 7f087dc00700 1 -- 172.21.15.131:0/3358730578 shutdown_connections 2024-05-08T22:18:49.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.350+0000 7f087dc00700 1 -- 172.21.15.131:0/3358730578 wait complete. 2024-05-08T22:18:49.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.351+0000 7f087dc00700 1 Processor -- start 2024-05-08T22:18:49.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.352+0000 7f087dc00700 1 -- start start 2024-05-08T22:18:49.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.352+0000 7f087dc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0878107fa0 0x7f08781a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:49.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.352+0000 7f087dc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0878108960 0x7f087819d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:49.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.352+0000 7f087dc00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f087810d8f0 0x7f087819deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:49.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.352+0000 7f087dc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087819e420 con 0x7f0878108960 2024-05-08T22:18:49.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.352+0000 7f087dc00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087819e590 con 0x7f087810d8f0 2024-05-08T22:18:49.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.352+0000 7f087dc00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087819e750 con 0x7f0878107fa0 2024-05-08T22:18:49.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.353+0000 7f0877400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f087810d8f0 0x7f087819deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:49.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.353+0000 7f0877400700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f087810d8f0 0x7f087819deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:45788/0 (socket says 172.21.15.131:45788) 2024-05-08T22:18:49.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.353+0000 7f0877400700 1 -- 172.21.15.131:0/2708685108 learned_addr learned my addr 172.21.15.131:0/2708685108 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:49.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.353+0000 7f0876a00700 1 --2- 172.21.15.131:0/2708685108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0878107fa0 0x7f08781a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:49.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.353+0000 7f0876000700 1 --2- 172.21.15.131:0/2708685108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0878108960 0x7f087819d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:49.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.353+0000 7f0877400700 1 -- 172.21.15.131:0/2708685108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0878107fa0 msgr2=0x7f08781a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:49.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.353+0000 7f0877400700 1 --2- 172.21.15.131:0/2708685108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0878107fa0 0x7f08781a4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:49.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.353+0000 7f0877400700 1 -- 172.21.15.131:0/2708685108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0878108960 msgr2=0x7f087819d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:49.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.353+0000 7f0877400700 1 --2- 172.21.15.131:0/2708685108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0878108960 0x7f087819d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:49.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.353+0000 7f0877400700 1 -- 172.21.15.131:0/2708685108 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08700071c0 con 0x7f087810d8f0 2024-05-08T22:18:49.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.354+0000 7f0877400700 1 --2- 172.21.15.131:0/2708685108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f087810d8f0 0x7f087819deb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f086000d980 tx=0x7f086000dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:49.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.354+0000 7f086f400700 1 -- 172.21.15.131:0/2708685108 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0860009b40 con 0x7f087810d8f0 2024-05-08T22:18:49.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.354+0000 7f087dc00700 1 -- 172.21.15.131:0/2708685108 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0868009c40 con 0x7f087810d8f0 2024-05-08T22:18:49.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.354+0000 7f087dc00700 1 -- 172.21.15.131:0/2708685108 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08781a27b0 con 0x7f087810d8f0 2024-05-08T22:18:49.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.354+0000 7f086f400700 1 -- 172.21.15.131:0/2708685108 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f086000be40 con 0x7f087810d8f0 2024-05-08T22:18:49.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.355+0000 7f086f400700 1 -- 172.21.15.131:0/2708685108 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08600054c0 con 0x7f087810d8f0 2024-05-08T22:18:49.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.357+0000 7f087dc00700 1 -- 172.21.15.131:0/2708685108 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08540052f0 con 0x7f087810d8f0 2024-05-08T22:18:49.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.360+0000 7f086f400700 1 -- 172.21.15.131:0/2708685108 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f0860009ca0 con 0x7f087810d8f0 2024-05-08T22:18:49.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.362+0000 7f086f400700 1 --2- 172.21.15.131:0/2708685108 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f085c0a1290 0x7f085c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:49.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.362+0000 7f086f400700 1 -- 172.21.15.131:0/2708685108 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f08600ccb90 con 0x7f087810d8f0 2024-05-08T22:18:49.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.363+0000 7f0876a00700 1 --2- 172.21.15.131:0/2708685108 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f085c0a1290 0x7f085c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:49.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.364+0000 7f0876a00700 1 --2- 172.21.15.131:0/2708685108 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f085c0a1290 0x7f085c0a3750 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f087819f120 tx=0x7f087000c040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:49.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.368+0000 7f086f400700 1 -- 172.21.15.131:0/2708685108 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0860095ba0 con 0x7f087810d8f0 2024-05-08T22:18:49.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:49 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2451517578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:18:49.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.793+0000 7f087dc00700 1 -- 172.21.15.131:0/2708685108 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0854005160 con 0x7f087810d8f0 2024-05-08T22:18:49.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.794+0000 7f086f400700 1 -- 172.21.15.131:0/2708685108 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10891 (secure 0 0 0) 0x7f086001f090 con 0x7f087810d8f0 2024-05-08T22:18:49.796 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:18:49.796 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1062828950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1062828950},{"type":"v1","addr":"172.21.15.131:6837","nonce":1062828950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24461,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034338270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034338270},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034338270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/505027564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":505027564},{"type":"v1","addr":"172.21.15.132:6833","nonce":505027564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2147999088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2147999088},{"type":"v1","addr":"172.21.15.131:6835","nonce":2147999088}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2610471069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2610471069},{"type":"v1","addr":"172.21.15.132:6835","nonce":2610471069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1436814390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1436814390},{"type":"v1","addr":"172.21.15.157:6835","nonce":1436814390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:05:14.034908+0000","modified":"2024-05-08T22:16:38.400949+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2693198274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2693198274},{"type":"v1","addr":"172.21.15.157:6837","nonce":2693198274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2685833178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2685833178},{"type":"v1","addr":"172.21.15.132:6837","nonce":2685833178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/3040193644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3040193644},{"type":"v1","addr":"172.21.15.157:6839","nonce":3040193644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.131:6839/1622880274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":1622880274},{"type":"v1","addr":"172.21.15.131:6839","nonce":1622880274}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6841/1686013987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":1686013987},{"type":"v1","addr":"172.21.15.131:6841","nonce":1686013987}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/4205858754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4205858754},{"type":"v1","addr":"172.21.15.132:6839","nonce":4205858754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:18:49.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.797+0000 7f087dc00700 1 -- 172.21.15.131:0/2708685108 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f085c0a1290 msgr2=0x7f085c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:49.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.797+0000 7f087dc00700 1 --2- 172.21.15.131:0/2708685108 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f085c0a1290 0x7f085c0a3750 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f087819f120 tx=0x7f087000c040 comp rx=0 tx=0).stop 2024-05-08T22:18:49.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.797+0000 7f087dc00700 1 -- 172.21.15.131:0/2708685108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f087810d8f0 msgr2=0x7f087819deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:49.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.797+0000 7f087dc00700 1 --2- 172.21.15.131:0/2708685108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f087810d8f0 0x7f087819deb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f086000d980 tx=0x7f086000dd40 comp rx=0 tx=0).stop 2024-05-08T22:18:49.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.798+0000 7f087dc00700 1 -- 172.21.15.131:0/2708685108 shutdown_connections 2024-05-08T22:18:49.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.798+0000 7f087dc00700 1 --2- 172.21.15.131:0/2708685108 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f085c0a1290 0x7f085c0a3750 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:49.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.798+0000 7f087dc00700 1 --2- 172.21.15.131:0/2708685108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f087810d8f0 0x7f087819deb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:49.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.798+0000 7f087dc00700 1 --2- 172.21.15.131:0/2708685108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0878107fa0 0x7f08781a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:49.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.798+0000 7f087dc00700 1 --2- 172.21.15.131:0/2708685108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0878108960 0x7f087819d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:49.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.798+0000 7f087dc00700 1 -- 172.21.15.131:0/2708685108 >> 172.21.15.131:0/2708685108 conn(0x7f087806bf20 msgr2=0x7f0878112eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:49.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.798+0000 7f087dc00700 1 -- 172.21.15.131:0/2708685108 shutdown_connections 2024-05-08T22:18:49.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:49.798+0000 7f087dc00700 1 -- 172.21.15.131:0/2708685108 wait complete. 2024-05-08T22:18:49.801 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 30 2024-05-08T22:18:49.824 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:49 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2451517578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:18:49.826 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:18:49.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:49 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2451517578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:18:50.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.100+0000 7f6a1d200700 1 -- 172.21.15.131:0/1374316198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a18106170 msgr2=0x7f6a1810dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:50.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.100+0000 7f6a1d200700 1 --2- 172.21.15.131:0/1374316198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a18106170 0x7f6a1810dcf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6a10009f90 tx=0x7f6a1000b330 comp rx=0 tx=0).stop 2024-05-08T22:18:50.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.100+0000 7f6a1d200700 1 -- 172.21.15.131:0/1374316198 shutdown_connections 2024-05-08T22:18:50.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.100+0000 7f6a1d200700 1 --2- 172.21.15.131:0/1374316198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a18106170 0x7f6a1810dcf0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:50.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.100+0000 7f6a1d200700 1 --2- 172.21.15.131:0/1374316198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a18101050 0x7f6a18101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:50.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.100+0000 7f6a1d200700 1 --2- 172.21.15.131:0/1374316198 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a18101a00 0x7f6a18105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:50.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.100+0000 7f6a1d200700 1 -- 172.21.15.131:0/1374316198 >> 172.21.15.131:0/1374316198 conn(0x7f6a180fc8f0 msgr2=0x7f6a180fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:50.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.100+0000 7f6a1d200700 1 -- 172.21.15.131:0/1374316198 shutdown_connections 2024-05-08T22:18:50.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.100+0000 7f6a1d200700 1 -- 172.21.15.131:0/1374316198 wait complete. 2024-05-08T22:18:50.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.102+0000 7f6a1d200700 1 Processor -- start 2024-05-08T22:18:50.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.103+0000 7f6a1d200700 1 -- start start 2024-05-08T22:18:50.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.103+0000 7f6a1d200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a18101050 0x7f6a181a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:50.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.103+0000 7f6a1d200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a18106170 0x7f6a181a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:50.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.103+0000 7f6a1d200700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a1810ceb0 0x7f6a1810d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:50.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.104+0000 7f6a1d200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1810d940 con 0x7f6a18106170 2024-05-08T22:18:50.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.104+0000 7f6a1d200700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1810dab0 con 0x7f6a1810ceb0 2024-05-08T22:18:50.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.104+0000 7f6a1d200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1810dc70 con 0x7f6a18101050 2024-05-08T22:18:50.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.104+0000 7f6a15400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a18106170 0x7f6a181a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:50.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.104+0000 7f6a15e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a18101050 0x7f6a181a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:50.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.104+0000 7f6a16800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a1810ceb0 0x7f6a1810d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:50.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.104+0000 7f6a15e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a18101050 0x7f6a181a3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37310/0 (socket says 172.21.15.131:37310) 2024-05-08T22:18:50.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.104+0000 7f6a15e00700 1 -- 172.21.15.131:0/19689352 learned_addr learned my addr 172.21.15.131:0/19689352 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:50.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.105+0000 7f6a15e00700 1 -- 172.21.15.131:0/19689352 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a1810ceb0 msgr2=0x7f6a1810d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:50.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.105+0000 7f6a15e00700 1 --2- 172.21.15.131:0/19689352 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a1810ceb0 0x7f6a1810d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:50.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.105+0000 7f6a15e00700 1 -- 172.21.15.131:0/19689352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a18106170 msgr2=0x7f6a181a3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:50.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.105+0000 7f6a15e00700 1 --2- 172.21.15.131:0/19689352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a18106170 0x7f6a181a3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:50.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.105+0000 7f6a15e00700 1 -- 172.21.15.131:0/19689352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a10009c40 con 0x7f6a18101050 2024-05-08T22:18:50.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.105+0000 7f6a15e00700 1 --2- 172.21.15.131:0/19689352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a18101050 0x7f6a181a3690 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6a0800e9c0 tx=0x7f6a0800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:50.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.106+0000 7f6a0ea00700 1 -- 172.21.15.131:0/19689352 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0800cbd0 con 0x7f6a18101050 2024-05-08T22:18:50.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.106+0000 7f6a0ea00700 1 -- 172.21.15.131:0/19689352 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a0800eeb0 con 0x7f6a18101050 2024-05-08T22:18:50.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.106+0000 7f6a0ea00700 1 -- 172.21.15.131:0/19689352 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a08005580 con 0x7f6a18101050 2024-05-08T22:18:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.106+0000 7f6a1d200700 1 -- 172.21.15.131:0/19689352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a181aa540 con 0x7f6a18101050 2024-05-08T22:18:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.107+0000 7f6a1d200700 1 -- 172.21.15.131:0/19689352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a181aaa00 con 0x7f6a18101050 2024-05-08T22:18:50.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.108+0000 7f6a1d200700 1 -- 172.21.15.131:0/19689352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a181051c0 con 0x7f6a18101050 2024-05-08T22:18:50.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.110+0000 7f6a0ea00700 1 -- 172.21.15.131:0/19689352 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f6a08004750 con 0x7f6a18101050 2024-05-08T22:18:50.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.113+0000 7f6a0ea00700 1 --2- 172.21.15.131:0/19689352 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f69fc0a1310 0x7f69fc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:50.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.113+0000 7f6a0ea00700 1 -- 172.21.15.131:0/19689352 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f6a08014070 con 0x7f6a18101050 2024-05-08T22:18:50.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.114+0000 7f6a15400700 1 --2- 172.21.15.131:0/19689352 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f69fc0a1310 0x7f69fc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:50.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.115+0000 7f6a15400700 1 --2- 172.21.15.131:0/19689352 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f69fc0a1310 0x7f69fc0a37d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6a180752c0 tx=0x7f6a04009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:50.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.116+0000 7f6a0ea00700 1 -- 172.21.15.131:0/19689352 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6a08096300 con 0x7f6a18101050 2024-05-08T22:18:50.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.441+0000 7f6a1d200700 1 -- 172.21.15.131:0/19689352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a1819ed60 con 0x7f6a18101050 2024-05-08T22:18:50.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.443+0000 7f6a0ea00700 1 -- 172.21.15.131:0/19689352 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+25481 (secure 0 0 0) 0x7f6a08099920 con 0x7f6a18101050 2024-05-08T22:18:50.445 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-08T22:18:50.445 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":98,"fsid":"06f1a696-0d86-11ef-bc97-c7b262605968","created":"2024-05-08T21:58:43.079038+0000","modified":"2024-05-08T22:18:08.471284+0000","last_up_change":"2024-05-08T22:04:35.106613+0000","last_in_change":"2024-05-08T22:04:18.930191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:01:41.271078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:05:12.469858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:05:13.484446+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:05:17.016186+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1fba15ea-c8d0-49f9-a7cc-80efdeff2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6803","nonce":206657594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6805","nonce":206657594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6809","nonce":206657594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":206657594},{"type":"v1","addr":"172.21.15.131:6807","nonce":206657594}]},"public_addr":"172.21.15.131:6803/206657594","cluster_addr":"172.21.15.131:6805/206657594","heartbeat_back_addr":"172.21.15.131:6809/206657594","heartbeat_front_addr":"172.21.15.131:6807/206657594","state":["exists","up"]},{"osd":1,"uuid":"f6d2536e-d9a6-4073-8297-987d9e885657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6801","nonce":2358880332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6803","nonce":2358880332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6807","nonce":2358880332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2358880332},{"type":"v1","addr":"172.21.15.132:6805","nonce":2358880332}]},"public_addr":"172.21.15.132:6801/2358880332","cluster_addr":"172.21.15.132:6803/2358880332","heartbeat_back_addr":"172.21.15.132:6807/2358880332","heartbeat_front_addr":"172.21.15.132:6805/2358880332","state":["exists","up"]},{"osd":2,"uuid":"992291bf-2d07-42f7-8722-efdbe3dafb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6801","nonce":3561264016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6803","nonce":3561264016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6807","nonce":3561264016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3561264016},{"type":"v1","addr":"172.21.15.157:6805","nonce":3561264016}]},"public_addr":"172.21.15.157:6801/3561264016","cluster_addr":"172.21.15.157:6803/3561264016","heartbeat_back_addr":"172.21.15.157:6807/3561264016","heartbeat_front_addr":"172.21.15.157:6805/3561264016","state":["exists","up"]},{"osd":3,"uuid":"b8281ef9-2390-4b3e-9ba8-0f7e71ca3616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6811","nonce":1075134307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6813","nonce":1075134307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6817","nonce":1075134307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1075134307},{"type":"v1","addr":"172.21.15.131:6815","nonce":1075134307}]},"public_addr":"172.21.15.131:6811/1075134307","cluster_addr":"172.21.15.131:6813/1075134307","heartbeat_back_addr":"172.21.15.131:6817/1075134307","heartbeat_front_addr":"172.21.15.131:6815/1075134307","state":["exists","up"]},{"osd":4,"uuid":"f1592d54-01e0-4201-905f-89664e2da391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6809","nonce":1317083238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6811","nonce":1317083238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6815","nonce":1317083238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1317083238},{"type":"v1","addr":"172.21.15.132:6813","nonce":1317083238}]},"public_addr":"172.21.15.132:6809/1317083238","cluster_addr":"172.21.15.132:6811/1317083238","heartbeat_back_addr":"172.21.15.132:6815/1317083238","heartbeat_front_addr":"172.21.15.132:6813/1317083238","state":["exists","up"]},{"osd":5,"uuid":"2d54acce-6950-42eb-b65b-97931044a3f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6809","nonce":1488005925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6811","nonce":1488005925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6815","nonce":1488005925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1488005925},{"type":"v1","addr":"172.21.15.157:6813","nonce":1488005925}]},"public_addr":"172.21.15.157:6809/1488005925","cluster_addr":"172.21.15.157:6811/1488005925","heartbeat_back_addr":"172.21.15.157:6815/1488005925","heartbeat_front_addr":"172.21.15.157:6813/1488005925","state":["exists","up"]},{"osd":6,"uuid":"fecf0f79-034b-4ad3-a194-eb7c3ed23776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6819","nonce":260285513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6821","nonce":260285513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6825","nonce":260285513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":260285513},{"type":"v1","addr":"172.21.15.131:6823","nonce":260285513}]},"public_addr":"172.21.15.131:6819/260285513","cluster_addr":"172.21.15.131:6821/260285513","heartbeat_back_addr":"172.21.15.131:6825/260285513","heartbeat_front_addr":"172.21.15.131:6823/260285513","state":["exists","up"]},{"osd":7,"uuid":"0fbdd862-cf84-4929-a74c-5c4be1ce7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6817","nonce":464003238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6819","nonce":464003238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6823","nonce":464003238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":464003238},{"type":"v1","addr":"172.21.15.132:6821","nonce":464003238}]},"public_addr":"172.21.15.132:6817/464003238","cluster_addr":"172.21.15.132:6819/464003238","heartbeat_back_addr":"172.21.15.132:6823/464003238","heartbeat_front_addr":"172.21.15.132:6821/464003238","state":["exists","up"]},{"osd":8,"uuid":"47077c11-3248-442f-b15c-692931137f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6817","nonce":3594097089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6819","nonce":3594097089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6823","nonce":3594097089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3594097089},{"type":"v1","addr":"172.21.15.157:6821","nonce":3594097089}]},"public_addr":"172.21.15.157:6817/3594097089","cluster_addr":"172.21.15.157:6819/3594097089","heartbeat_back_addr":"172.21.15.157:6823/3594097089","heartbeat_front_addr":"172.21.15.157:6821/3594097089","state":["exists","up"]},{"osd":9,"uuid":"b027ba0a-8aec-45af-8137-b261c1588499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6827","nonce":4217805693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6829","nonce":4217805693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6833","nonce":4217805693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4217805693},{"type":"v1","addr":"172.21.15.131:6831","nonce":4217805693}]},"public_addr":"172.21.15.131:6827/4217805693","cluster_addr":"172.21.15.131:6829/4217805693","heartbeat_back_addr":"172.21.15.131:6833/4217805693","heartbeat_front_addr":"172.21.15.131:6831/4217805693","state":["exists","up"]},{"osd":10,"uuid":"430013a4-d468-49b2-9162-165ac5a4a7f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6825","nonce":306651644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6827","nonce":306651644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6831","nonce":306651644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":306651644},{"type":"v1","addr":"172.21.15.132:6829","nonce":306651644}]},"public_addr":"172.21.15.132:6825/306651644","cluster_addr":"172.21.15.132:6827/306651644","heartbeat_back_addr":"172.21.15.132:6831/306651644","heartbeat_front_addr":"172.21.15.132:6829/306651644","state":["exists","up"]},{"osd":11,"uuid":"76622ed0-d3b9-4f55-9206-fc96c56e18ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6825","nonce":2884744696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6827","nonce":2884744696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6831","nonce":2884744696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2884744696},{"type":"v1","addr":"172.21.15.157:6829","nonce":2884744696}]},"public_addr":"172.21.15.157:6825/2884744696","cluster_addr":"172.21.15.157:6827/2884744696","heartbeat_back_addr":"172.21.15.157:6831/2884744696","heartbeat_front_addr":"172.21.15.157:6829/2884744696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:02.028669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:20.800828+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:01:38.538511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:00.408791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:19.242933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:36.981300+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:02:58.711369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:17.524065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:35.197589+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:03:57.576712+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:15.887503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:04:33.500770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/2704707055":"2024-05-09T22:07:17.096274+0000","172.21.15.131:6800/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3082367829":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6801/2485665042":"2024-05-09T22:07:17.096274+0000","172.21.15.131:0/126236497":"2024-05-09T22:07:17.096274+0000","172.21.15.131:0/25489954":"2024-05-09T22:07:17.096274+0000","172.21.15.131:6801/1564023164":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/1773942606":"2024-05-09T22:07:17.096274+0000","172.21.15.131:0/1981506111":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/1716895483":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/1241655400":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:6800/2485665042":"2024-05-09T22:07:17.096274+0000","172.21.15.131:6801/1418340798":"2024-05-09T21:59:40.641373+0000","172.21.15.131:0/3309822030":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/976983862":"2024-05-09T21:59:15.548383+0000","172.21.15.131:0/3120268590":"2024-05-09T22:07:17.096274+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:18:50.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.446+0000 7f6a1d200700 1 -- 172.21.15.131:0/19689352 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f69fc0a1310 msgr2=0x7f69fc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:50.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.446+0000 7f6a1d200700 1 --2- 172.21.15.131:0/19689352 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f69fc0a1310 0x7f69fc0a37d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6a180752c0 tx=0x7f6a04009260 comp rx=0 tx=0).stop 2024-05-08T22:18:50.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.446+0000 7f6a1d200700 1 -- 172.21.15.131:0/19689352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a18101050 msgr2=0x7f6a181a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:50.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.446+0000 7f6a1d200700 1 --2- 172.21.15.131:0/19689352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a18101050 0x7f6a181a3690 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6a0800e9c0 tx=0x7f6a0800ecd0 comp rx=0 tx=0).stop 2024-05-08T22:18:50.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.446+0000 7f6a1d200700 1 -- 172.21.15.131:0/19689352 shutdown_connections 2024-05-08T22:18:50.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.446+0000 7f6a1d200700 1 --2- 172.21.15.131:0/19689352 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f69fc0a1310 0x7f69fc0a37d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:50.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.446+0000 7f6a1d200700 1 --2- 172.21.15.131:0/19689352 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a1810ceb0 0x7f6a1810d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:50.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.446+0000 7f6a1d200700 1 --2- 172.21.15.131:0/19689352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a18101050 0x7f6a181a3690 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:50.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.446+0000 7f6a1d200700 1 --2- 172.21.15.131:0/19689352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a18106170 0x7f6a181a3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:50.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.446+0000 7f6a1d200700 1 -- 172.21.15.131:0/19689352 >> 172.21.15.131:0/19689352 conn(0x7f6a180fc8f0 msgr2=0x7f6a180fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:50.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.447+0000 7f6a1d200700 1 -- 172.21.15.131:0/19689352 shutdown_connections 2024-05-08T22:18:50.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.447+0000 7f6a1d200700 1 -- 172.21.15.131:0/19689352 wait complete. 2024-05-08T22:18:50.469 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T22:18:50.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.703+0000 7feef1800700 1 -- 172.21.15.131:0/1529159725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feeec101b10 msgr2=0x7feeec105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:50.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.703+0000 7feef1800700 1 --2- 172.21.15.131:0/1529159725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feeec101b10 0x7feeec105b50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7feed4009990 tx=0x7feed4009ca0 comp rx=0 tx=0).stop 2024-05-08T22:18:50.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.703+0000 7feef1800700 1 -- 172.21.15.131:0/1529159725 shutdown_connections 2024-05-08T22:18:50.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.703+0000 7feef1800700 1 --2- 172.21.15.131:0/1529159725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feeec106280 0x7feeec10de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:50.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.703+0000 7feef1800700 1 --2- 172.21.15.131:0/1529159725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeec101160 0x7feeec101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:50.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.703+0000 7feef1800700 1 --2- 172.21.15.131:0/1529159725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feeec101b10 0x7feeec105b50 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:50.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.703+0000 7feef1800700 1 -- 172.21.15.131:0/1529159725 >> 172.21.15.131:0/1529159725 conn(0x7feeec0fc9c0 msgr2=0x7feeec0fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:50.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.703+0000 7feef1800700 1 -- 172.21.15.131:0/1529159725 shutdown_connections 2024-05-08T22:18:50.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.703+0000 7feef1800700 1 -- 172.21.15.131:0/1529159725 wait complete. 2024-05-08T22:18:50.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.705+0000 7feef1800700 1 Processor -- start 2024-05-08T22:18:50.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.705+0000 7feef1800700 1 -- start start 2024-05-08T22:18:50.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.705+0000 7feef1800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feeec101160 0x7feeec1a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:50.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.705+0000 7feef1800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeec106280 0x7feeec1a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:50.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.705+0000 7feef1800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feeec10ce90 0x7feeec10d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:50.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.705+0000 7feef1800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec10d910 con 0x7feeec10ce90 2024-05-08T22:18:50.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.705+0000 7feef1800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec10da50 con 0x7feeec101160 2024-05-08T22:18:50.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.705+0000 7feef1800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec10dbe0 con 0x7feeec106280 2024-05-08T22:18:50.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.706+0000 7feeeaa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feeec101160 0x7feeec1a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:50.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.706+0000 7feee3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeec106280 0x7feeec1a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:50.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.706+0000 7feeeaa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feeec101160 0x7feeec1a36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:45818/0 (socket says 172.21.15.131:45818) 2024-05-08T22:18:50.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.706+0000 7feeeaa00700 1 -- 172.21.15.131:0/2925331374 learned_addr learned my addr 172.21.15.131:0/2925331374 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:50.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.706+0000 7feeeb400700 1 --2- 172.21.15.131:0/2925331374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feeec10ce90 0x7feeec10d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:50.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.706+0000 7feee3e00700 1 -- 172.21.15.131:0/2925331374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feeec101160 msgr2=0x7feeec1a36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:50.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.706+0000 7feee3e00700 1 --2- 172.21.15.131:0/2925331374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feeec101160 0x7feeec1a36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:50.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.706+0000 7feee3e00700 1 -- 172.21.15.131:0/2925331374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feeec10ce90 msgr2=0x7feeec10d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:50.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.706+0000 7feee3e00700 1 --2- 172.21.15.131:0/2925331374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feeec10ce90 0x7feeec10d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:50.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.706+0000 7feee3e00700 1 -- 172.21.15.131:0/2925331374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feedc009c40 con 0x7feeec106280 2024-05-08T22:18:50.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.706+0000 7feeeaa00700 1 --2- 172.21.15.131:0/2925331374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feeec101160 0x7feeec1a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:18:50.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.706+0000 7feeeb400700 1 --2- 172.21.15.131:0/2925331374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feeec10ce90 0x7feeec10d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:18:50.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.706+0000 7feee3e00700 1 --2- 172.21.15.131:0/2925331374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeec106280 0x7feeec1a3c00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7feed4000c00 tx=0x7feed40057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:50.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.707+0000 7feee3400700 1 -- 172.21.15.131:0/2925331374 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed401d070 con 0x7feeec106280 2024-05-08T22:18:50.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.707+0000 7feee3400700 1 -- 172.21.15.131:0/2925331374 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feed4021410 con 0x7feeec106280 2024-05-08T22:18:50.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.707+0000 7feef1800700 1 -- 172.21.15.131:0/2925331374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed40095f0 con 0x7feeec106280 2024-05-08T22:18:50.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.707+0000 7feee3400700 1 -- 172.21.15.131:0/2925331374 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed4017650 con 0x7feeec106280 2024-05-08T22:18:50.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.707+0000 7feef1800700 1 -- 172.21.15.131:0/2925331374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeec1aa820 con 0x7feeec106280 2024-05-08T22:18:50.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.708+0000 7feef1800700 1 -- 172.21.15.131:0/2925331374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeec102630 con 0x7feeec106280 2024-05-08T22:18:50.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.710+0000 7feee3400700 1 -- 172.21.15.131:0/2925331374 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7feed4021810 con 0x7feeec106280 2024-05-08T22:18:50.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.711+0000 7feee3400700 1 --2- 172.21.15.131:0/2925331374 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7feed00a1290 0x7feed00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:50.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.711+0000 7feee3400700 1 -- 172.21.15.131:0/2925331374 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9613+0+0 (secure 0 0 0) 0x7feed40ce360 con 0x7feeec106280 2024-05-08T22:18:50.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.712+0000 7feeeaa00700 1 --2- 172.21.15.131:0/2925331374 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7feed00a1290 0x7feed00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:50.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.712+0000 7feeeaa00700 1 --2- 172.21.15.131:0/2925331374 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7feed00a1290 0x7feed00a3750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7feeec0fd270 tx=0x7feedc00b040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:50.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:50.715+0000 7feee3400700 1 -- 172.21.15.131:0/2925331374 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7feed40973a0 con 0x7feeec106280 2024-05-08T22:18:50.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:50 smithi157 ceph-mon[77746]: pgmap v359: 129 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 111 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-08T22:18:50.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:50 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2708685108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:18:50.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:50 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/19689352' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:18:50.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:50 smithi132 ceph-mon[78180]: pgmap v359: 129 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 111 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-08T22:18:50.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:50 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2708685108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:18:50.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:50 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/19689352' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:18:50.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:50 smithi131 ceph-mon[37260]: pgmap v359: 129 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 111 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-08T22:18:50.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:50 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2708685108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:18:50.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:50 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/19689352' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:18:51.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.027+0000 7feef1800700 1 -- 172.21.15.131:0/2925331374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7feeec19ec90 con 0x7feeec106280 2024-05-08T22:18:51.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.042+0000 7feee3400700 1 -- 172.21.15.131:0/2925331374 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) v1 ==== 161+0+0 (secure 0 0 0) 0x7feed409a9c0 con 0x7feeec106280 2024-05-08T22:18:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.046+0000 7feef1800700 1 -- 172.21.15.131:0/2925331374 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7feed00a1290 msgr2=0x7feed00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.046+0000 7feef1800700 1 --2- 172.21.15.131:0/2925331374 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7feed00a1290 0x7feed00a3750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7feeec0fd270 tx=0x7feedc00b040 comp rx=0 tx=0).stop 2024-05-08T22:18:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.046+0000 7feef1800700 1 -- 172.21.15.131:0/2925331374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeec106280 msgr2=0x7feeec1a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.046+0000 7feef1800700 1 --2- 172.21.15.131:0/2925331374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeec106280 0x7feeec1a3c00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7feed4000c00 tx=0x7feed40057c0 comp rx=0 tx=0).stop 2024-05-08T22:18:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.046+0000 7feef1800700 1 -- 172.21.15.131:0/2925331374 shutdown_connections 2024-05-08T22:18:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.046+0000 7feef1800700 1 --2- 172.21.15.131:0/2925331374 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7feed00a1290 0x7feed00a3750 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.046+0000 7feef1800700 1 --2- 172.21.15.131:0/2925331374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feeec10ce90 0x7feeec10d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:51.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.046+0000 7feef1800700 1 --2- 172.21.15.131:0/2925331374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feeec101160 0x7feeec1a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:51.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.046+0000 7feef1800700 1 --2- 172.21.15.131:0/2925331374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeec106280 0x7feeec1a3c00 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:51.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.046+0000 7feef1800700 1 -- 172.21.15.131:0/2925331374 >> 172.21.15.131:0/2925331374 conn(0x7feeec0fc9c0 msgr2=0x7feeec0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:51.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.046+0000 7feef1800700 1 -- 172.21.15.131:0/2925331374 shutdown_connections 2024-05-08T22:18:51.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.046+0000 7feef1800700 1 -- 172.21.15.131:0/2925331374 wait complete. 2024-05-08T22:18:51.050 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T22:18:51.076 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T22:18:51.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.320+0000 7f46dba00700 1 -- 172.21.15.131:0/145985931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46d4101b10 msgr2=0x7f46d4105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:51.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.320+0000 7f46dba00700 1 --2- 172.21.15.131:0/145985931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46d4101b10 0x7f46d4105b50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009990 tx=0x7f46c0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:18:51.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.320+0000 7f46dba00700 1 -- 172.21.15.131:0/145985931 shutdown_connections 2024-05-08T22:18:51.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.320+0000 7f46dba00700 1 --2- 172.21.15.131:0/145985931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46d4106280 0x7f46d410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:51.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.320+0000 7f46dba00700 1 --2- 172.21.15.131:0/145985931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46d4101160 0x7f46d4101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:51.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.320+0000 7f46dba00700 1 --2- 172.21.15.131:0/145985931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46d4101b10 0x7f46d4105b50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:51.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.320+0000 7f46dba00700 1 -- 172.21.15.131:0/145985931 >> 172.21.15.131:0/145985931 conn(0x7f46d40fc9c0 msgr2=0x7f46d40fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:51.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.320+0000 7f46dba00700 1 -- 172.21.15.131:0/145985931 shutdown_connections 2024-05-08T22:18:51.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.320+0000 7f46dba00700 1 -- 172.21.15.131:0/145985931 wait complete. 2024-05-08T22:18:51.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.322+0000 7f46dba00700 1 Processor -- start 2024-05-08T22:18:51.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.322+0000 7f46dba00700 1 -- start start 2024-05-08T22:18:51.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.322+0000 7f46dba00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46d4101160 0x7f46d4199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:51.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.322+0000 7f46dba00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46d4106280 0x7f46d4199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:51.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.322+0000 7f46dba00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46d419dee0 0x7f46d419e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:51.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.322+0000 7f46dba00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d419e960 con 0x7f46d4106280 2024-05-08T22:18:51.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.322+0000 7f46dba00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d419ead0 con 0x7f46d4101160 2024-05-08T22:18:51.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.322+0000 7f46dba00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d419ec90 con 0x7f46d419dee0 2024-05-08T22:18:51.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.322+0000 7f46d9000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46d4101160 0x7f46d4199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:51.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.323+0000 7f46d9000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46d4101160 0x7f46d4199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:45846/0 (socket says 172.21.15.131:45846) 2024-05-08T22:18:51.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.323+0000 7f46d9000700 1 -- 172.21.15.131:0/3494270109 learned_addr learned my addr 172.21.15.131:0/3494270109 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:51.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.323+0000 7f46d3e00700 1 --2- 172.21.15.131:0/3494270109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46d4106280 0x7f46d4199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:51.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.323+0000 7f46d9a00700 1 --2- 172.21.15.131:0/3494270109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46d419dee0 0x7f46d419e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:51.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.323+0000 7f46d9000700 1 -- 172.21.15.131:0/3494270109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46d419dee0 msgr2=0x7f46d419e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:51.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.323+0000 7f46d9000700 1 --2- 172.21.15.131:0/3494270109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46d419dee0 0x7f46d419e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:51.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.323+0000 7f46d9000700 1 -- 172.21.15.131:0/3494270109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46d4106280 msgr2=0x7f46d4199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:51.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.323+0000 7f46d9000700 1 --2- 172.21.15.131:0/3494270109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46d4106280 0x7f46d4199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:51.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.323+0000 7f46d9000700 1 -- 172.21.15.131:0/3494270109 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c00095f0 con 0x7f46d4101160 2024-05-08T22:18:51.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.323+0000 7f46d3e00700 1 --2- 172.21.15.131:0/3494270109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46d4106280 0x7f46d4199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:18:51.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.323+0000 7f46d9000700 1 --2- 172.21.15.131:0/3494270109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46d4101160 0x7f46d4199600 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f46c800e9a0 tx=0x7f46c800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:51.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.323+0000 7f46d1600700 1 -- 172.21.15.131:0/3494270109 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c8009790 con 0x7f46d4101160 2024-05-08T22:18:51.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.323+0000 7f46d1600700 1 -- 172.21.15.131:0/3494270109 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46c800cbd0 con 0x7f46d4101160 2024-05-08T22:18:51.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.323+0000 7f46dba00700 1 -- 172.21.15.131:0/3494270109 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d41a61f0 con 0x7f46d4101160 2024-05-08T22:18:51.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.324+0000 7f46d1600700 1 -- 172.21.15.131:0/3494270109 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c8018680 con 0x7f46d4101160 2024-05-08T22:18:51.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.324+0000 7f46dba00700 1 -- 172.21.15.131:0/3494270109 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d41a6720 con 0x7f46d4101160 2024-05-08T22:18:51.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.325+0000 7f46dba00700 1 -- 172.21.15.131:0/3494270109 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d4102630 con 0x7f46d4101160 2024-05-08T22:18:51.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.327+0000 7f46d1600700 1 -- 172.21.15.131:0/3494270109 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f46c800cd40 con 0x7f46d4101160 2024-05-08T22:18:51.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.328+0000 7f46d1600700 1 --2- 172.21.15.131:0/3494270109 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f46bc0a1310 0x7f46bc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:51.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.329+0000 7f46d1600700 1 -- 172.21.15.131:0/3494270109 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9783+0+0 (secure 0 0 0) 0x7f46c8014070 con 0x7f46d4101160 2024-05-08T22:18:51.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.329+0000 7f46d3e00700 1 --2- 172.21.15.131:0/3494270109 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f46bc0a1310 0x7f46bc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:51.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.330+0000 7f46d3e00700 1 --2- 172.21.15.131:0/3494270109 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f46bc0a1310 0x7f46bc0a37d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f46c0006010 tx=0x7f46c000b260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:51.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.332+0000 7f46d1600700 1 -- 172.21.15.131:0/3494270109 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f46c8052d60 con 0x7f46d4101160 2024-05-08T22:18:51.340 INFO:journalctl@ceph.mds.b.smithi132.stdout:May 08 22:18:51 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-b[114823]: ignoring --setuser ceph since I am not root 2024-05-08T22:18:51.340 INFO:journalctl@ceph.mds.b.smithi132.stdout:May 08 22:18:51 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-b[114823]: ignoring --setgroup ceph since I am not root 2024-05-08T22:18:51.340 INFO:journalctl@ceph.mds.b.smithi132.stdout:May 08 22:18:51 smithi132 ceph-mds[114846]: -- 172.21.15.132:0/22653615 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56028350a1a0 con 0x5602834e0800 2024-05-08T22:18:51.340 INFO:journalctl@ceph.mds.b.smithi132.stdout:May 08 22:18:51 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-b[114823]: starting mds.b at 2024-05-08T22:18:51.340 INFO:journalctl@ceph.mds.h.smithi132.stdout:May 08 22:18:51 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-h[114482]: ignoring --setuser ceph since I am not root 2024-05-08T22:18:51.340 INFO:journalctl@ceph.mds.h.smithi132.stdout:May 08 22:18:51 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-h[114482]: ignoring --setgroup ceph since I am not root 2024-05-08T22:18:51.340 INFO:journalctl@ceph.mds.h.smithi132.stdout:May 08 22:18:51 smithi132 ceph-mds[114505]: -- 172.21.15.132:0/3351516956 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d15c6961a0 con 0x55d15c66d000 2024-05-08T22:18:51.340 INFO:journalctl@ceph.mds.h.smithi132.stdout:May 08 22:18:51 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-h[114482]: starting mds.h at 2024-05-08T22:18:51.391 INFO:journalctl@ceph.mds.g.smithi131.stdout:May 08 22:18:51 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-g[87250]: ignoring --setuser ceph since I am not root 2024-05-08T22:18:51.391 INFO:journalctl@ceph.mds.g.smithi131.stdout:May 08 22:18:51 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-g[87250]: ignoring --setgroup ceph since I am not root 2024-05-08T22:18:51.391 INFO:journalctl@ceph.mds.g.smithi131.stdout:May 08 22:18:51 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-g[87250]: starting mds.g at 2024-05-08T22:18:51.391 INFO:journalctl@ceph.mds.a.smithi131.stdout:May 08 22:18:51 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-a[87641]: ignoring --setuser ceph since I am not root 2024-05-08T22:18:51.391 INFO:journalctl@ceph.mds.a.smithi131.stdout:May 08 22:18:51 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-a[87641]: ignoring --setgroup ceph since I am not root 2024-05-08T22:18:51.391 INFO:journalctl@ceph.mds.a.smithi131.stdout:May 08 22:18:51 smithi131 ceph-mds[87665]: -- 172.21.15.131:0/2843554190 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cd590721a0 con 0x55cd5904a800 2024-05-08T22:18:51.391 INFO:journalctl@ceph.mds.a.smithi131.stdout:May 08 22:18:51 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-a[87641]: starting mds.a at 2024-05-08T22:18:51.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:51 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mon-a[37237]: 2024-05-08T22:18:51.029+0000 7fbf43200700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:18:51.520 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 08 22:18:51 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-c[115254]: ignoring --setuser ceph since I am not root 2024-05-08T22:18:51.520 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 08 22:18:51 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-c[115254]: ignoring --setgroup ceph since I am not root 2024-05-08T22:18:51.520 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 08 22:18:51 smithi157 ceph-mds[115277]: -- 172.21.15.157:0/4016995901 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562097db61a0 con 0x562097d8c800 2024-05-08T22:18:51.520 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 08 22:18:51 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-c[115254]: starting mds.c at 2024-05-08T22:18:51.520 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 08 22:18:51 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-i[115591]: ignoring --setuser ceph since I am not root 2024-05-08T22:18:51.521 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 08 22:18:51 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-i[115591]: ignoring --setgroup ceph since I am not root 2024-05-08T22:18:51.521 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 08 22:18:51 smithi157 ceph-mds[115614]: -- 172.21.15.157:0/593049859 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b48669c1a0 con 0x55b486672c00 2024-05-08T22:18:51.521 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 08 22:18:51 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-i[115591]: starting mds.i at 2024-05-08T22:18:51.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:51.651+0000 7f46dba00700 1 -- 172.21.15.131:0/3494270109 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f46d4067cf0 con 0x7f46d4101160 2024-05-08T22:18:51.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:51 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2925331374' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T22:18:51.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:51 smithi157 ceph-mon[77746]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T22:18:51.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:51 smithi157 ceph-mon[77746]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T22:18:51.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:51 smithi157 ceph-mon[77746]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:18:51.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:51 smithi157 ceph-mon[77746]: osdmap e99: 12 total, 12 up, 12 in 2024-05-08T22:18:51.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:51 smithi157 ceph-mon[77746]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T22:18:51.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:51 smithi157 ceph-mon[77746]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-08T22:18:51.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:51 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2925331374' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T22:18:51.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:51 smithi132 ceph-mon[78180]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T22:18:51.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:51 smithi132 ceph-mon[78180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T22:18:51.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:51 smithi132 ceph-mon[78180]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:18:51.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:51 smithi132 ceph-mon[78180]: osdmap e99: 12 total, 12 up, 12 in 2024-05-08T22:18:51.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:51 smithi132 ceph-mon[78180]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T22:18:51.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:51 smithi132 ceph-mon[78180]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-08T22:18:51.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:51 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2925331374' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T22:18:51.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:51 smithi131 ceph-mon[37260]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T22:18:51.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:51 smithi131 ceph-mon[37260]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T22:18:51.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:51 smithi131 ceph-mon[37260]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:18:51.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:51 smithi131 ceph-mon[37260]: osdmap e99: 12 total, 12 up, 12 in 2024-05-08T22:18:51.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:51 smithi131 ceph-mon[37260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T22:18:51.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:51 smithi131 ceph-mon[37260]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-08T22:18:52.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.048+0000 7f46d1600700 1 -- 172.21.15.131:0/3494270109 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) v1 ==== 104+0+0 (secure 0 0 0) 0x7f46c8096340 con 0x7f46d4101160 2024-05-08T22:18:52.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.051+0000 7f46dba00700 1 -- 172.21.15.131:0/3494270109 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f46bc0a1310 msgr2=0x7f46bc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:52.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.051+0000 7f46dba00700 1 --2- 172.21.15.131:0/3494270109 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f46bc0a1310 0x7f46bc0a37d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f46c0006010 tx=0x7f46c000b260 comp rx=0 tx=0).stop 2024-05-08T22:18:52.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.051+0000 7f46dba00700 1 -- 172.21.15.131:0/3494270109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46d4101160 msgr2=0x7f46d4199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:52.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.051+0000 7f46dba00700 1 --2- 172.21.15.131:0/3494270109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46d4101160 0x7f46d4199600 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f46c800e9a0 tx=0x7f46c800ecb0 comp rx=0 tx=0).stop 2024-05-08T22:18:52.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.051+0000 7f46dba00700 1 -- 172.21.15.131:0/3494270109 shutdown_connections 2024-05-08T22:18:52.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.051+0000 7f46dba00700 1 --2- 172.21.15.131:0/3494270109 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f46bc0a1310 0x7f46bc0a37d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:52.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.051+0000 7f46dba00700 1 --2- 172.21.15.131:0/3494270109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46d419dee0 0x7f46d419e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:52.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.051+0000 7f46dba00700 1 --2- 172.21.15.131:0/3494270109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46d4101160 0x7f46d4199600 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:52.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.051+0000 7f46dba00700 1 --2- 172.21.15.131:0/3494270109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46d4106280 0x7f46d4199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:52.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.051+0000 7f46dba00700 1 -- 172.21.15.131:0/3494270109 >> 172.21.15.131:0/3494270109 conn(0x7f46d40fc9c0 msgr2=0x7f46d40feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:52.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.052+0000 7f46dba00700 1 -- 172.21.15.131:0/3494270109 shutdown_connections 2024-05-08T22:18:52.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.052+0000 7f46dba00700 1 -- 172.21.15.131:0/3494270109 wait complete. 2024-05-08T22:18:52.073 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T22:18:52.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.271+0000 7f25c3000700 1 -- 172.21.15.131:0/1495656648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25bc101b10 msgr2=0x7f25bc105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:52.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.271+0000 7f25c3000700 1 --2- 172.21.15.131:0/1495656648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25bc101b10 0x7f25bc105b50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f25a4009990 tx=0x7f25a4009ca0 comp rx=0 tx=0).stop 2024-05-08T22:18:52.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.271+0000 7f25c3000700 1 -- 172.21.15.131:0/1495656648 shutdown_connections 2024-05-08T22:18:52.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.271+0000 7f25c3000700 1 --2- 172.21.15.131:0/1495656648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25bc106280 0x7f25bc10de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:52.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.271+0000 7f25c3000700 1 --2- 172.21.15.131:0/1495656648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25bc101160 0x7f25bc101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:52.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.271+0000 7f25c3000700 1 --2- 172.21.15.131:0/1495656648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25bc101b10 0x7f25bc105b50 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:52.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.271+0000 7f25c3000700 1 -- 172.21.15.131:0/1495656648 >> 172.21.15.131:0/1495656648 conn(0x7f25bc0fc9c0 msgr2=0x7f25bc0fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:52.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.271+0000 7f25c3000700 1 -- 172.21.15.131:0/1495656648 shutdown_connections 2024-05-08T22:18:52.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.271+0000 7f25c3000700 1 -- 172.21.15.131:0/1495656648 wait complete. 2024-05-08T22:18:52.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.272+0000 7f25c3000700 1 Processor -- start 2024-05-08T22:18:52.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.272+0000 7f25c3000700 1 -- start start 2024-05-08T22:18:52.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.272+0000 7f25c3000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25bc101160 0x7f25bc0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:52.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.272+0000 7f25c3000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25bc106280 0x7f25bc0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:52.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.272+0000 7f25c3000700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25bc073d20 0x7f25bc1a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:52.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.272+0000 7f25c3000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc1a5be0 con 0x7f25bc106280 2024-05-08T22:18:52.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.272+0000 7f25c3000700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc1a5d20 con 0x7f25bc073d20 2024-05-08T22:18:52.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.272+0000 7f25c3000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc1a5ee0 con 0x7f25bc101160 2024-05-08T22:18:52.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.273+0000 7f25bbe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25bc101160 0x7f25bc0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:52.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.273+0000 7f25b3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25bc106280 0x7f25bc0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:52.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.273+0000 7f25bbe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25bc101160 0x7f25bc0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37398/0 (socket says 172.21.15.131:37398) 2024-05-08T22:18:52.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.273+0000 7f25bbe00700 1 -- 172.21.15.131:0/2176182391 learned_addr learned my addr 172.21.15.131:0/2176182391 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:52.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.273+0000 7f25c1000700 1 --2- 172.21.15.131:0/2176182391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25bc073d20 0x7f25bc1a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:52.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.273+0000 7f25bbe00700 1 -- 172.21.15.131:0/2176182391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25bc073d20 msgr2=0x7f25bc1a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:52.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.273+0000 7f25bbe00700 1 --2- 172.21.15.131:0/2176182391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25bc073d20 0x7f25bc1a5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:52.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.273+0000 7f25bbe00700 1 -- 172.21.15.131:0/2176182391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25bc106280 msgr2=0x7f25bc0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:52.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.273+0000 7f25bbe00700 1 --2- 172.21.15.131:0/2176182391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25bc106280 0x7f25bc0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:52.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.273+0000 7f25bbe00700 1 -- 172.21.15.131:0/2176182391 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a40095f0 con 0x7f25bc101160 2024-05-08T22:18:52.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.273+0000 7f25b3e00700 1 --2- 172.21.15.131:0/2176182391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25bc106280 0x7f25bc0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:18:52.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.273+0000 7f25bbe00700 1 --2- 172.21.15.131:0/2176182391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25bc101160 0x7f25bc0732a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00e9a0 tx=0x7f25ac00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:52.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.273+0000 7f25b9600700 1 -- 172.21.15.131:0/2176182391 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac009790 con 0x7f25bc101160 2024-05-08T22:18:52.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.273+0000 7f25c3000700 1 -- 172.21.15.131:0/2176182391 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25bc1a61c0 con 0x7f25bc101160 2024-05-08T22:18:52.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.273+0000 7f25b9600700 1 -- 172.21.15.131:0/2176182391 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25ac010430 con 0x7f25bc101160 2024-05-08T22:18:52.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.274+0000 7f25b9600700 1 -- 172.21.15.131:0/2176182391 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac018680 con 0x7f25bc101160 2024-05-08T22:18:52.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.274+0000 7f25c3000700 1 -- 172.21.15.131:0/2176182391 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25bc1a6720 con 0x7f25bc101160 2024-05-08T22:18:52.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.275+0000 7f25c3000700 1 -- 172.21.15.131:0/2176182391 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25bc109550 con 0x7f25bc101160 2024-05-08T22:18:52.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.276+0000 7f25b9600700 1 -- 172.21.15.131:0/2176182391 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f25ac0105a0 con 0x7f25bc101160 2024-05-08T22:18:52.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.278+0000 7f25b9600700 1 --2- 172.21.15.131:0/2176182391 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f25a00a1310 0x7f25a00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:52.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.278+0000 7f25b9600700 1 -- 172.21.15.131:0/2176182391 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9783+0+0 (secure 0 0 0) 0x7f25ac014070 con 0x7f25bc101160 2024-05-08T22:18:52.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.278+0000 7f25b3e00700 1 --2- 172.21.15.131:0/2176182391 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f25a00a1310 0x7f25a00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:52.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.278+0000 7f25b3e00700 1 --2- 172.21.15.131:0/2176182391 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f25a00a1310 0x7f25a00a37d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f25bc074980 tx=0x7f25a400b260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:52.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.282+0000 7f25b9600700 1 -- 172.21.15.131:0/2176182391 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f25ac052d60 con 0x7f25bc101160 2024-05-08T22:18:52.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:52.493+0000 7f25c3000700 1 -- 172.21.15.131:0/2176182391 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f25bc074730 con 0x7f25bc101160 2024-05-08T22:18:52.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:52 smithi157 ceph-mon[77746]: pgmap v361: 129 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 111 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 716 B/s wr, 7 op/s 2024-05-08T22:18:52.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:52 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/3494270109' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:18:52.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:52 smithi157 ceph-mon[77746]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:18:52.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:52 smithi157 ceph-mon[77746]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T22:18:52.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:52 smithi157 ceph-mon[77746]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T22:18:52.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:52 smithi157 ceph-mon[77746]: Cluster is now healthy 2024-05-08T22:18:52.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:52 smithi157 ceph-mon[77746]: mds.? [v2:172.21.15.132:6836/99680116,v1:172.21.15.132:6837/99680116] up:boot 2024-05-08T22:18:52.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:52 smithi157 ceph-mon[77746]: mds.? [v2:172.21.15.157:6836/1700892232,v1:172.21.15.157:6837/1700892232] up:boot 2024-05-08T22:18:52.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:52 smithi157 ceph-mon[77746]: mds.? [v2:172.21.15.132:6838/4198276407,v1:172.21.15.132:6839/4198276407] up:boot 2024-05-08T22:18:52.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:52 smithi157 ceph-mon[77746]: mds.? [v2:172.21.15.157:6838/2596177841,v1:172.21.15.157:6839/2596177841] up:boot 2024-05-08T22:18:52.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:52 smithi157 ceph-mon[77746]: mds.? [v2:172.21.15.131:6838/630271346,v1:172.21.15.131:6839/630271346] up:boot 2024-05-08T22:18:52.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:52 smithi157 ceph-mon[77746]: mds.? [v2:172.21.15.131:6840/1389788646,v1:172.21.15.131:6841/1389788646] up:boot 2024-05-08T22:18:52.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:52 smithi157 ceph-mon[77746]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T22:18:52.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:52 smithi157 ceph-mon[77746]: fsmap 12 up:standby 2024-05-08T22:18:52.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:52 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T22:18:52.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:52 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T22:18:52.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:52 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T22:18:52.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:52 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T22:18:52.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:52 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T22:18:52.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:52 smithi157 ceph-mon[77746]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T22:18:52.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:52 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2176182391' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:18:52.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:52 smithi157 ceph-mon[77746]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:18:52.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:52 smithi132 ceph-mon[78180]: pgmap v361: 129 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 111 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 716 B/s wr, 7 op/s 2024-05-08T22:18:52.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:52 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/3494270109' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:18:52.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:52 smithi132 ceph-mon[78180]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:18:52.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:52 smithi132 ceph-mon[78180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T22:18:52.840 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:52 smithi132 ceph-mon[78180]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T22:18:52.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:52 smithi132 ceph-mon[78180]: Cluster is now healthy 2024-05-08T22:18:52.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:52 smithi132 ceph-mon[78180]: mds.? [v2:172.21.15.132:6836/99680116,v1:172.21.15.132:6837/99680116] up:boot 2024-05-08T22:18:52.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:52 smithi132 ceph-mon[78180]: mds.? [v2:172.21.15.157:6836/1700892232,v1:172.21.15.157:6837/1700892232] up:boot 2024-05-08T22:18:52.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:52 smithi132 ceph-mon[78180]: mds.? [v2:172.21.15.132:6838/4198276407,v1:172.21.15.132:6839/4198276407] up:boot 2024-05-08T22:18:52.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:52 smithi132 ceph-mon[78180]: mds.? [v2:172.21.15.157:6838/2596177841,v1:172.21.15.157:6839/2596177841] up:boot 2024-05-08T22:18:52.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:52 smithi132 ceph-mon[78180]: mds.? [v2:172.21.15.131:6838/630271346,v1:172.21.15.131:6839/630271346] up:boot 2024-05-08T22:18:52.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:52 smithi132 ceph-mon[78180]: mds.? [v2:172.21.15.131:6840/1389788646,v1:172.21.15.131:6841/1389788646] up:boot 2024-05-08T22:18:52.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:52 smithi132 ceph-mon[78180]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T22:18:52.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:52 smithi132 ceph-mon[78180]: fsmap 12 up:standby 2024-05-08T22:18:52.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:52 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T22:18:52.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:52 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T22:18:52.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:52 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T22:18:52.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:52 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T22:18:52.841 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:52 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T22:18:52.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:52 smithi132 ceph-mon[78180]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T22:18:52.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:52 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2176182391' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:18:52.842 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:52 smithi132 ceph-mon[78180]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:18:52.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:52 smithi131 ceph-mon[37260]: pgmap v361: 129 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 111 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 716 B/s wr, 7 op/s 2024-05-08T22:18:52.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:52 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/3494270109' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:18:52.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:52 smithi131 ceph-mon[37260]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:18:52.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:52 smithi131 ceph-mon[37260]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T22:18:52.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:52 smithi131 ceph-mon[37260]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T22:18:52.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:52 smithi131 ceph-mon[37260]: Cluster is now healthy 2024-05-08T22:18:52.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:52 smithi131 ceph-mon[37260]: mds.? [v2:172.21.15.132:6836/99680116,v1:172.21.15.132:6837/99680116] up:boot 2024-05-08T22:18:52.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:52 smithi131 ceph-mon[37260]: mds.? [v2:172.21.15.157:6836/1700892232,v1:172.21.15.157:6837/1700892232] up:boot 2024-05-08T22:18:52.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:52 smithi131 ceph-mon[37260]: mds.? [v2:172.21.15.132:6838/4198276407,v1:172.21.15.132:6839/4198276407] up:boot 2024-05-08T22:18:52.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:52 smithi131 ceph-mon[37260]: mds.? [v2:172.21.15.157:6838/2596177841,v1:172.21.15.157:6839/2596177841] up:boot 2024-05-08T22:18:52.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:52 smithi131 ceph-mon[37260]: mds.? [v2:172.21.15.131:6838/630271346,v1:172.21.15.131:6839/630271346] up:boot 2024-05-08T22:18:52.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:52 smithi131 ceph-mon[37260]: mds.? [v2:172.21.15.131:6840/1389788646,v1:172.21.15.131:6841/1389788646] up:boot 2024-05-08T22:18:52.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:52 smithi131 ceph-mon[37260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T22:18:52.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:52 smithi131 ceph-mon[37260]: fsmap 12 up:standby 2024-05-08T22:18:52.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:52 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T22:18:52.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:52 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T22:18:52.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:52 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T22:18:52.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:52 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T22:18:52.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:52 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T22:18:52.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:52 smithi131 ceph-mon[37260]: from='mgr.15366 172.21.15.131:0/2318137946' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T22:18:52.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:52 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2176182391' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:18:52.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:52 smithi131 ceph-mon[37260]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:18:53.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.054+0000 7f25b9600700 1 -- 172.21.15.131:0/2176182391 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v100) v1 ==== 181+0+0 (secure 0 0 0) 0x7f25ac099d10 con 0x7f25bc101160 2024-05-08T22:18:53.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.059+0000 7f25c3000700 1 -- 172.21.15.131:0/2176182391 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f25a00a1310 msgr2=0x7f25a00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:53.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.059+0000 7f25c3000700 1 --2- 172.21.15.131:0/2176182391 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f25a00a1310 0x7f25a00a37d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f25bc074980 tx=0x7f25a400b260 comp rx=0 tx=0).stop 2024-05-08T22:18:53.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.060+0000 7f25c3000700 1 -- 172.21.15.131:0/2176182391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25bc101160 msgr2=0x7f25bc0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:53.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.060+0000 7f25c3000700 1 --2- 172.21.15.131:0/2176182391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25bc101160 0x7f25bc0732a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00e9a0 tx=0x7f25ac00ecb0 comp rx=0 tx=0).stop 2024-05-08T22:18:53.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.061+0000 7f25c3000700 1 -- 172.21.15.131:0/2176182391 shutdown_connections 2024-05-08T22:18:53.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.061+0000 7f25c3000700 1 --2- 172.21.15.131:0/2176182391 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f25a00a1310 0x7f25a00a37d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:53.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.061+0000 7f25c3000700 1 --2- 172.21.15.131:0/2176182391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25bc073d20 0x7f25bc1a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:53.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.061+0000 7f25c3000700 1 --2- 172.21.15.131:0/2176182391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25bc101160 0x7f25bc0732a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:53.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.061+0000 7f25c3000700 1 --2- 172.21.15.131:0/2176182391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25bc106280 0x7f25bc0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:53.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.061+0000 7f25c3000700 1 -- 172.21.15.131:0/2176182391 >> 172.21.15.131:0/2176182391 conn(0x7f25bc0fc9c0 msgr2=0x7f25bc0fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:53.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.061+0000 7f25c3000700 1 -- 172.21.15.131:0/2176182391 shutdown_connections 2024-05-08T22:18:53.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.061+0000 7f25c3000700 1 -- 172.21.15.131:0/2176182391 wait complete. 2024-05-08T22:18:53.065 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2024-05-08T22:18:53.092 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T22:18:53.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.306+0000 7f1db2800700 1 -- 172.21.15.131:0/3788447190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dac073c90 msgr2=0x7f1dac10d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:53.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.306+0000 7f1db2800700 1 --2- 172.21.15.131:0/3788447190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dac073c90 0x7f1dac10d270 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009990 tx=0x7f1d98009ca0 comp rx=0 tx=0).stop 2024-05-08T22:18:53.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.306+0000 7f1db2800700 1 -- 172.21.15.131:0/3788447190 shutdown_connections 2024-05-08T22:18:53.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.306+0000 7f1db2800700 1 --2- 172.21.15.131:0/3788447190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dac10d7b0 0x7f1dac10fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:53.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.306+0000 7f1db2800700 1 --2- 172.21.15.131:0/3788447190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dac073370 0x7f1dac073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:53.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.306+0000 7f1db2800700 1 --2- 172.21.15.131:0/3788447190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dac073c90 0x7f1dac10d270 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:53.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.306+0000 7f1db2800700 1 -- 172.21.15.131:0/3788447190 >> 172.21.15.131:0/3788447190 conn(0x7f1dac0fc9e0 msgr2=0x7f1dac0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:53.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.307+0000 7f1db2800700 1 -- 172.21.15.131:0/3788447190 shutdown_connections 2024-05-08T22:18:53.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.307+0000 7f1db2800700 1 -- 172.21.15.131:0/3788447190 wait complete. 2024-05-08T22:18:53.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.307+0000 7f1db2800700 1 Processor -- start 2024-05-08T22:18:53.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.308+0000 7f1db2800700 1 -- start start 2024-05-08T22:18:53.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.308+0000 7f1db2800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dac073370 0x7f1dac199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:53.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.308+0000 7f1db2800700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dac073c90 0x7f1dac199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:53.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.308+0000 7f1db2800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dac10d7b0 0x7f1dac19de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:53.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.308+0000 7f1db2800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac19e4c0 con 0x7f1dac073370 2024-05-08T22:18:53.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.308+0000 7f1db2800700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac19e630 con 0x7f1dac073c90 2024-05-08T22:18:53.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.308+0000 7f1db2800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac19e7f0 con 0x7f1dac10d7b0 2024-05-08T22:18:53.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.308+0000 7f1daaa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dac073c90 0x7f1dac199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:53.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.308+0000 7f1daaa00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dac073c90 0x7f1dac199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.131:45888/0 (socket says 172.21.15.131:45888) 2024-05-08T22:18:53.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.308+0000 7f1daaa00700 1 -- 172.21.15.131:0/2903402830 learned_addr learned my addr 172.21.15.131:0/2903402830 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:53.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.308+0000 7f1dabe00700 1 --2- 172.21.15.131:0/2903402830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dac10d7b0 0x7f1dac19de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:53.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.308+0000 7f1dab400700 1 --2- 172.21.15.131:0/2903402830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dac073370 0x7f1dac199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:53.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.309+0000 7f1daaa00700 1 -- 172.21.15.131:0/2903402830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dac10d7b0 msgr2=0x7f1dac19de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:53.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.309+0000 7f1daaa00700 1 --2- 172.21.15.131:0/2903402830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dac10d7b0 0x7f1dac19de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:53.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.309+0000 7f1daaa00700 1 -- 172.21.15.131:0/2903402830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dac073370 msgr2=0x7f1dac199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:53.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.309+0000 7f1daaa00700 1 --2- 172.21.15.131:0/2903402830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dac073370 0x7f1dac199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:53.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.309+0000 7f1daaa00700 1 -- 172.21.15.131:0/2903402830 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d980095f0 con 0x7f1dac073c90 2024-05-08T22:18:53.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.309+0000 7f1dab400700 1 --2- 172.21.15.131:0/2903402830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dac073370 0x7f1dac199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:18:53.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.309+0000 7f1dabe00700 1 --2- 172.21.15.131:0/2903402830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dac10d7b0 0x7f1dac19de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:18:53.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.309+0000 7f1daaa00700 1 --2- 172.21.15.131:0/2903402830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dac073c90 0x7f1dac199b50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1d98005770 tx=0x7f1d980057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:53.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.309+0000 7f1da3e00700 1 -- 172.21.15.131:0/2903402830 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9801d070 con 0x7f1dac073c90 2024-05-08T22:18:53.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.309+0000 7f1da3e00700 1 -- 172.21.15.131:0/2903402830 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d98021410 con 0x7f1dac073c90 2024-05-08T22:18:53.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.309+0000 7f1db2800700 1 -- 172.21.15.131:0/2903402830 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dac19ea70 con 0x7f1dac073c90 2024-05-08T22:18:53.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.309+0000 7f1da3e00700 1 -- 172.21.15.131:0/2903402830 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d98017650 con 0x7f1dac073c90 2024-05-08T22:18:53.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.309+0000 7f1db2800700 1 -- 172.21.15.131:0/2903402830 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dac1a6310 con 0x7f1dac073c90 2024-05-08T22:18:53.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.311+0000 7f1db2800700 1 -- 172.21.15.131:0/2903402830 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dac1089c0 con 0x7f1dac073c90 2024-05-08T22:18:53.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.312+0000 7f1da3e00700 1 -- 172.21.15.131:0/2903402830 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f1d9800fb00 con 0x7f1dac073c90 2024-05-08T22:18:53.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.313+0000 7f1da3e00700 1 --2- 172.21.15.131:0/2903402830 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1d900a1290 0x7f1d900a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:53.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.314+0000 7f1da3e00700 1 -- 172.21.15.131:0/2903402830 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9229+0+0 (secure 0 0 0) 0x7f1d980ce390 con 0x7f1dac073c90 2024-05-08T22:18:53.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.314+0000 7f1dab400700 1 --2- 172.21.15.131:0/2903402830 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1d900a1290 0x7f1d900a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:53.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.314+0000 7f1dab400700 1 --2- 172.21.15.131:0/2903402830 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1d900a1290 0x7f1d900a3750 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009cb0 tx=0x7f1d9c009260 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:53.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.317+0000 7f1da3e00700 1 -- 172.21.15.131:0/2903402830 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1d98097550 con 0x7f1dac073c90 2024-05-08T22:18:53.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:53.546+0000 7f1db2800700 1 -- 172.21.15.131:0/2903402830 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1dac19a910 con 0x7f1dac073c90 2024-05-08T22:18:54.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.062+0000 7f1da3e00700 1 -- 172.21.15.131:0/2903402830 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v101) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1d9809ab70 con 0x7f1dac073c90 2024-05-08T22:18:54.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.066+0000 7f1db2800700 1 -- 172.21.15.131:0/2903402830 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1d900a1290 msgr2=0x7f1d900a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:54.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.066+0000 7f1db2800700 1 --2- 172.21.15.131:0/2903402830 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1d900a1290 0x7f1d900a3750 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009cb0 tx=0x7f1d9c009260 comp rx=0 tx=0).stop 2024-05-08T22:18:54.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.066+0000 7f1db2800700 1 -- 172.21.15.131:0/2903402830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dac073c90 msgr2=0x7f1dac199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:54.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.066+0000 7f1db2800700 1 --2- 172.21.15.131:0/2903402830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dac073c90 0x7f1dac199b50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1d98005770 tx=0x7f1d980057a0 comp rx=0 tx=0).stop 2024-05-08T22:18:54.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.067+0000 7f1db2800700 1 -- 172.21.15.131:0/2903402830 shutdown_connections 2024-05-08T22:18:54.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.067+0000 7f1db2800700 1 --2- 172.21.15.131:0/2903402830 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7f1d900a1290 0x7f1d900a3750 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:54.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.067+0000 7f1db2800700 1 --2- 172.21.15.131:0/2903402830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1dac10d7b0 0x7f1dac19de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:54.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.067+0000 7f1db2800700 1 --2- 172.21.15.131:0/2903402830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dac073370 0x7f1dac199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:54.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.067+0000 7f1db2800700 1 --2- 172.21.15.131:0/2903402830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dac073c90 0x7f1dac199b50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:54.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.067+0000 7f1db2800700 1 -- 172.21.15.131:0/2903402830 >> 172.21.15.131:0/2903402830 conn(0x7f1dac0fc9e0 msgr2=0x7f1dac0fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:54.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.067+0000 7f1db2800700 1 -- 172.21.15.131:0/2903402830 shutdown_connections 2024-05-08T22:18:54.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.067+0000 7f1db2800700 1 -- 172.21.15.131:0/2903402830 wait complete. 2024-05-08T22:18:54.070 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2024-05-08T22:18:54.101 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-08T22:18:54.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.305+0000 7fa362e00700 1 -- 172.21.15.131:0/2796040721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa35c1062a0 msgr2=0x7fa35c10de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:54.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.305+0000 7fa362e00700 1 --2- 172.21.15.131:0/2796040721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa35c1062a0 0x7fa35c10de20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa35400b0a0 tx=0x7fa35400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:18:54.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.305+0000 7fa362e00700 1 -- 172.21.15.131:0/2796040721 shutdown_connections 2024-05-08T22:18:54.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.305+0000 7fa362e00700 1 --2- 172.21.15.131:0/2796040721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa35c1062a0 0x7fa35c10de20 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:54.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.305+0000 7fa362e00700 1 --2- 172.21.15.131:0/2796040721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35c101180 0x7fa35c101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:54.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.305+0000 7fa362e00700 1 --2- 172.21.15.131:0/2796040721 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa35c101b30 0x7fa35c105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:54.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.305+0000 7fa362e00700 1 -- 172.21.15.131:0/2796040721 >> 172.21.15.131:0/2796040721 conn(0x7fa35c0fc9e0 msgr2=0x7fa35c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:54.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.305+0000 7fa362e00700 1 -- 172.21.15.131:0/2796040721 shutdown_connections 2024-05-08T22:18:54.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.305+0000 7fa362e00700 1 -- 172.21.15.131:0/2796040721 wait complete. 2024-05-08T22:18:54.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.306+0000 7fa362e00700 1 Processor -- start 2024-05-08T22:18:54.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.306+0000 7fa362e00700 1 -- start start 2024-05-08T22:18:54.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.307+0000 7fa362e00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa35c101180 0x7fa35c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:54.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.307+0000 7fa362e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35c1062a0 0x7fa35c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:54.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.307+0000 7fa362e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa35c074f10 0x7fa35c073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:54.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.307+0000 7fa362e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c1aa200 con 0x7fa35c074f10 2024-05-08T22:18:54.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.307+0000 7fa362e00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c1aa370 con 0x7fa35c101180 2024-05-08T22:18:54.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.307+0000 7fa362e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c1aa530 con 0x7fa35c1062a0 2024-05-08T22:18:54.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.307+0000 7fa360e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa35c074f10 0x7fa35c073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:54.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.307+0000 7fa360e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa35c074f10 0x7fa35c073d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45174/0 (socket says 172.21.15.131:45174) 2024-05-08T22:18:54.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.307+0000 7fa360e00700 1 -- 172.21.15.131:0/2851871342 learned_addr learned my addr 172.21.15.131:0/2851871342 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-08T22:18:54.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.307+0000 7fa353e00700 1 --2- 172.21.15.131:0/2851871342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35c1062a0 0x7fa35c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:54.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.307+0000 7fa360e00700 1 -- 172.21.15.131:0/2851871342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35c1062a0 msgr2=0x7fa35c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:54.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.307+0000 7fa360e00700 1 --2- 172.21.15.131:0/2851871342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35c1062a0 0x7fa35c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:54.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.307+0000 7fa360e00700 1 -- 172.21.15.131:0/2851871342 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa35c101180 msgr2=0x7fa35c0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:54.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.307+0000 7fa360e00700 1 --2- 172.21.15.131:0/2851871342 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa35c101180 0x7fa35c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:54.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.307+0000 7fa360e00700 1 -- 172.21.15.131:0/2851871342 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa354009d00 con 0x7fa35c074f10 2024-05-08T22:18:54.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.308+0000 7fa360e00700 1 --2- 172.21.15.131:0/2851871342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa35c074f10 0x7fa35c073d20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa35400b070 tx=0x7fa354007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:54.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.308+0000 7fa359600700 1 -- 172.21.15.131:0/2851871342 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35400e040 con 0x7fa35c074f10 2024-05-08T22:18:54.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.308+0000 7fa359600700 1 -- 172.21.15.131:0/2851871342 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa35400bdc0 con 0x7fa35c074f10 2024-05-08T22:18:54.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.308+0000 7fa362e00700 1 -- 172.21.15.131:0/2851871342 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35c1aa7b0 con 0x7fa35c074f10 2024-05-08T22:18:54.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.308+0000 7fa359600700 1 -- 172.21.15.131:0/2851871342 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3540052a0 con 0x7fa35c074f10 2024-05-08T22:18:54.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.308+0000 7fa362e00700 1 -- 172.21.15.131:0/2851871342 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35c1aabd0 con 0x7fa35c074f10 2024-05-08T22:18:54.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.309+0000 7fa362e00700 1 -- 172.21.15.131:0/2851871342 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa35c109570 con 0x7fa35c074f10 2024-05-08T22:18:54.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.311+0000 7fa359600700 1 -- 172.21.15.131:0/2851871342 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fa35401d480 con 0x7fa35c074f10 2024-05-08T22:18:54.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.312+0000 7fa359600700 1 --2- 172.21.15.131:0/2851871342 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa3400a1310 0x7fa3400a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:18:54.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.312+0000 7fa359600700 1 -- 172.21.15.131:0/2851871342 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8641+0+0 (secure 0 0 0) 0x7fa3540cdfa0 con 0x7fa35c074f10 2024-05-08T22:18:54.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.312+0000 7fa35be00700 1 --2- 172.21.15.131:0/2851871342 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa3400a1310 0x7fa3400a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:18:54.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.313+0000 7fa35be00700 1 --2- 172.21.15.131:0/2851871342 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa3400a1310 0x7fa3400a37d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa34c005950 tx=0x7fa34c0058e0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:18:54.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.316+0000 7fa359600700 1 -- 172.21.15.131:0/2851871342 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa3540973a0 con 0x7fa35c074f10 2024-05-08T22:18:54.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:54 smithi132 ceph-mon[78180]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T22:18:54.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:54 smithi132 ceph-mon[78180]: osdmap e100: 12 total, 12 up, 12 in 2024-05-08T22:18:54.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:54 smithi132 ceph-mon[78180]: pgmap v363: 97 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 79 active+clean; 658 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:18:54.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:54 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2903402830' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:18:54.340 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:54 smithi132 ceph-mon[78180]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:18:54.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:54 smithi131 ceph-mon[37260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T22:18:54.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:54 smithi131 ceph-mon[37260]: osdmap e100: 12 total, 12 up, 12 in 2024-05-08T22:18:54.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:54 smithi131 ceph-mon[37260]: pgmap v363: 97 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 79 active+clean; 658 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:18:54.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:54 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2903402830' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:18:54.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:54 smithi131 ceph-mon[37260]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:18:54.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:54 smithi157 ceph-mon[77746]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T22:18:54.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:54 smithi157 ceph-mon[77746]: osdmap e100: 12 total, 12 up, 12 in 2024-05-08T22:18:54.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:54 smithi157 ceph-mon[77746]: pgmap v363: 97 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 79 active+clean; 658 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:18:54.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:54 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2903402830' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:18:54.526 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:54 smithi157 ceph-mon[77746]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:18:54.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:54.530+0000 7fa362e00700 1 -- 172.21.15.131:0/2851871342 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa35c0747f0 con 0x7fa35c074f10 2024-05-08T22:18:55.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:55.064+0000 7fa359600700 1 -- 172.21.15.131:0/2851871342 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v102) v1 ==== 178+0+0 (secure 0 0 0) 0x7fa35409a9c0 con 0x7fa35c074f10 2024-05-08T22:18:55.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:55.069+0000 7fa362e00700 1 -- 172.21.15.131:0/2851871342 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa3400a1310 msgr2=0x7fa3400a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:55.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:55.069+0000 7fa362e00700 1 --2- 172.21.15.131:0/2851871342 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa3400a1310 0x7fa3400a37d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa34c005950 tx=0x7fa34c0058e0 comp rx=0 tx=0).stop 2024-05-08T22:18:55.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:55.069+0000 7fa362e00700 1 -- 172.21.15.131:0/2851871342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa35c074f10 msgr2=0x7fa35c073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:18:55.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:55.069+0000 7fa362e00700 1 --2- 172.21.15.131:0/2851871342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa35c074f10 0x7fa35c073d20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa35400b070 tx=0x7fa354007cf0 comp rx=0 tx=0).stop 2024-05-08T22:18:55.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:55.069+0000 7fa362e00700 1 -- 172.21.15.131:0/2851871342 shutdown_connections 2024-05-08T22:18:55.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:55.069+0000 7fa362e00700 1 --2- 172.21.15.131:0/2851871342 >> [v2:172.21.15.131:6800/1157588501,v1:172.21.15.131:6801/1157588501] conn(0x7fa3400a1310 0x7fa3400a37d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:55.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:55.069+0000 7fa362e00700 1 --2- 172.21.15.131:0/2851871342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa35c074f10 0x7fa35c073d20 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:55.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:55.069+0000 7fa362e00700 1 --2- 172.21.15.131:0/2851871342 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa35c101180 0x7fa35c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:55.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:55.069+0000 7fa362e00700 1 --2- 172.21.15.131:0/2851871342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35c1062a0 0x7fa35c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:18:55.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:55.069+0000 7fa362e00700 1 -- 172.21.15.131:0/2851871342 >> 172.21.15.131:0/2851871342 conn(0x7fa35c0fc9e0 msgr2=0x7fa35c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:18:55.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:55.069+0000 7fa362e00700 1 -- 172.21.15.131:0/2851871342 shutdown_connections 2024-05-08T22:18:55.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-08T22:18:55.070+0000 7fa362e00700 1 -- 172.21.15.131:0/2851871342 wait complete. 2024-05-08T22:18:55.072 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data_ec' removed 2024-05-08T22:18:55.108 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T22:18:55.204 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T22:18:55.274 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T22:18:55.327 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:55 smithi157 ceph-mon[77746]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T22:18:55.327 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:55 smithi157 ceph-mon[77746]: osdmap e101: 12 total, 12 up, 12 in 2024-05-08T22:18:55.327 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:55 smithi157 ceph-mon[77746]: from='client.? 172.21.15.131:0/2851871342' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:18:55.335 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-08T22:18:55.335 DEBUG:teuthology.orchestra.run.smithi131:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T22:18:55.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:55 smithi132 ceph-mon[78180]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T22:18:55.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:55 smithi132 ceph-mon[78180]: osdmap e101: 12 total, 12 up, 12 in 2024-05-08T22:18:55.341 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:55 smithi132 ceph-mon[78180]: from='client.? 172.21.15.131:0/2851871342' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:18:55.390 DEBUG:teuthology.orchestra.run.smithi132:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T22:18:55.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:55 smithi131 ceph-mon[37260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T22:18:55.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:55 smithi131 ceph-mon[37260]: osdmap e101: 12 total, 12 up, 12 in 2024-05-08T22:18:55.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:55 smithi131 ceph-mon[37260]: from='client.? 172.21.15.131:0/2851871342' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:18:55.418 DEBUG:teuthology.orchestra.run.smithi157:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T22:18:55.449 INFO:tasks.cephadm:Stopping all daemons... 2024-05-08T22:18:55.449 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-08T22:18:55.450 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mon.a 2024-05-08T22:18:55.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:55 smithi131 systemd[1]: Stopping Ceph mon.a for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:18:56.302 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:55 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mon-a[37237]: 2024-05-08T22:18:55.959+0000 7fbf4a000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:18:56.302 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:55 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mon-a[37237]: 2024-05-08T22:18:55.959+0000 7fbf4a000700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-08T22:18:56.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 08 22:18:56 smithi131 bash[119246]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mon-a 2024-05-08T22:18:56.766 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mon.a.service' 2024-05-08T22:18:56.886 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:18:56.887 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-08T22:18:56.887 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-08T22:18:56.887 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mon.b 2024-05-08T22:18:57.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:56 smithi132 systemd[1]: Stopping Ceph mon.b for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:18:57.590 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:57 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mon-b[78157]: 2024-05-08T22:18:57.198+0000 7fccb1ea6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:18:57.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:57 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mon-b[78157]: 2024-05-08T22:18:57.198+0000 7fccb1ea6700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-08T22:18:57.591 INFO:journalctl@ceph.mon.b.smithi132.stdout:May 08 22:18:57 smithi132 bash[120380]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mon-b 2024-05-08T22:18:57.742 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mon.b.service' 2024-05-08T22:18:57.786 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:18:57.786 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-08T22:18:57.786 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-08T22:18:57.786 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mon.c 2024-05-08T22:18:58.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:57 smithi157 systemd[1]: Stopping Ceph mon.c for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:18:58.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:58 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mon-c[77723]: 2024-05-08T22:18:58.297+0000 7f139c350700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:18:58.776 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:58 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mon-c[77723]: 2024-05-08T22:18:58.297+0000 7f139c350700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-08T22:19:00.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 08 22:18:59 smithi157 bash[121557]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mon-c 2024-05-08T22:19:00.641 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:19:00 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:19:00.313+0000 7fec63e00700 -1 client.15387: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-05-08T22:19:00.724 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mon.c.service' 2024-05-08T22:19:00.914 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:19:00.915 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-08T22:19:00.915 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-08T22:19:00.915 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mgr.x 2024-05-08T22:19:01.391 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:19:01 smithi131 systemd[1]: Stopping Ceph mgr.x for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:19:02.891 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:19:02 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:19:02.540+0000 7fed3bc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:19:02.891 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:19:02 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x[37512]: 2024-05-08T22:19:02.540+0000 7fed3bc00700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-08T22:19:04.641 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 08 22:19:04 smithi131 bash[119383]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-x 2024-05-08T22:19:04.804 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-08T22:19:04.888 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:19:04.888 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-08T22:19:04.888 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-08T22:19:04.889 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mgr.y 2024-05-08T22:19:05.266 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:19:04 smithi132 systemd[1]: Stopping Ceph mgr.y for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:19:05.580 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:19:05 smithi132 bash[120520]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-y 2024-05-08T22:19:05.580 INFO:journalctl@ceph.mgr.y.smithi132.stdout:May 08 22:19:05 smithi132 systemd[1]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-05-08T22:19:05.587 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-08T22:19:05.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:19:05.623 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-08T22:19:05.623 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-08T22:19:05.623 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mgr.z 2024-05-08T22:19:05.976 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:19:05 smithi157 systemd[1]: Stopping Ceph mgr.z for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:19:06.276 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:19:06 smithi157 bash[121698]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mgr-z 2024-05-08T22:19:06.276 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 08 22:19:06 smithi157 systemd[1]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-05-08T22:19:06.292 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-08T22:19:06.330 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:19:06.330 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-08T22:19:06.330 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-08T22:19:06.330 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.0 2024-05-08T22:19:06.641 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 08 22:19:06 smithi131 systemd[1]: Stopping Ceph osd.0 for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:19:07.141 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 08 22:19:06 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-0[55470]: 2024-05-08T22:19:06.696+0000 7f0eaec00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:19:07.141 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 08 22:19:06 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-0[55470]: 2024-05-08T22:19:06.696+0000 7f0eaec00700 -1 osd.0 102 *** Got signal Terminated *** 2024-05-08T22:19:07.141 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 08 22:19:06 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-0[55470]: 2024-05-08T22:19:06.696+0000 7f0eaec00700 -1 osd.0 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:19:12.891 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 08 22:19:12 smithi131 bash[119521]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-0 2024-05-08T22:19:14.240 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.0.service' 2024-05-08T22:19:14.332 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:19:14.332 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-08T22:19:14.332 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-08T22:19:14.332 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.1 2024-05-08T22:19:14.655 INFO:journalctl@ceph.osd.1.smithi132.stdout:May 08 22:19:14 smithi132 systemd[1]: Stopping Ceph osd.1 for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:19:15.090 INFO:journalctl@ceph.osd.1.smithi132.stdout:May 08 22:19:14 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-1[83287]: 2024-05-08T22:19:14.653+0000 7f2992b85700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:19:15.090 INFO:journalctl@ceph.osd.1.smithi132.stdout:May 08 22:19:14 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-1[83287]: 2024-05-08T22:19:14.653+0000 7f2992b85700 -1 osd.1 102 *** Got signal Terminated *** 2024-05-08T22:19:15.090 INFO:journalctl@ceph.osd.1.smithi132.stdout:May 08 22:19:14 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-1[83287]: 2024-05-08T22:19:14.653+0000 7f2992b85700 -1 osd.1 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:19:20.591 INFO:journalctl@ceph.osd.1.smithi132.stdout:May 08 22:19:20 smithi132 bash[120663]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-1 2024-05-08T22:19:21.641 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.1.service' 2024-05-08T22:19:21.682 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:19:21.682 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-08T22:19:21.683 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-08T22:19:21.683 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.2 2024-05-08T22:19:22.026 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 08 22:19:21 smithi157 systemd[1]: Stopping Ceph osd.2 for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:19:22.026 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 08 22:19:21 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-2[83563]: 2024-05-08T22:19:21.908+0000 7fd7059ea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:19:22.026 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 08 22:19:21 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-2[83563]: 2024-05-08T22:19:21.908+0000 7fd7059ea700 -1 osd.2 102 *** Got signal Terminated *** 2024-05-08T22:19:22.026 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 08 22:19:21 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-2[83563]: 2024-05-08T22:19:21.908+0000 7fd7059ea700 -1 osd.2 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:19:27.776 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 08 22:19:27 smithi157 bash[121843]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-2 2024-05-08T22:19:28.929 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.2.service' 2024-05-08T22:19:28.971 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:19:28.972 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-08T22:19:28.972 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-08T22:19:28.972 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.3 2024-05-08T22:19:29.384 INFO:journalctl@ceph.osd.3.smithi131.stdout:May 08 22:19:29 smithi131 systemd[1]: Stopping Ceph osd.3 for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:19:29.641 INFO:journalctl@ceph.osd.3.smithi131.stdout:May 08 22:19:29 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-3[63378]: 2024-05-08T22:19:29.382+0000 7f46a9400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:19:29.641 INFO:journalctl@ceph.osd.3.smithi131.stdout:May 08 22:19:29 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-3[63378]: 2024-05-08T22:19:29.382+0000 7f46a9400700 -1 osd.3 102 *** Got signal Terminated *** 2024-05-08T22:19:29.641 INFO:journalctl@ceph.osd.3.smithi131.stdout:May 08 22:19:29 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-3[63378]: 2024-05-08T22:19:29.382+0000 7f46a9400700 -1 osd.3 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:19:33.641 INFO:journalctl@ceph.osd.3.smithi131.stdout:May 08 22:19:33 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-3[63378]: 2024-05-08T22:19:33.356+0000 7f46a4c00700 -1 osd.3 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:06.957494+0000 front 2024-05-08T22:19:06.957243+0000 (oldest deadline 2024-05-08T22:19:32.857328+0000) 2024-05-08T22:19:34.641 INFO:journalctl@ceph.osd.3.smithi131.stdout:May 08 22:19:34 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-3[63378]: 2024-05-08T22:19:34.332+0000 7f46a4c00700 -1 osd.3 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:06.957494+0000 front 2024-05-08T22:19:06.957243+0000 (oldest deadline 2024-05-08T22:19:32.857328+0000) 2024-05-08T22:19:34.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:34 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:34.492+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:19:34.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:34 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:34.347+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:19:35.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:34 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:34.876+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.182697+0000 front 2024-05-08T22:19:10.182627+0000 (oldest deadline 2024-05-08T22:19:34.282795+0000) 2024-05-08T22:19:35.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:35 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:35.468+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:19:35.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:35 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:35.353+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:19:35.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:35 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:35.456+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:19:36.391 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:35 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:35.914+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.182697+0000 front 2024-05-08T22:19:10.182627+0000 (oldest deadline 2024-05-08T22:19:34.282795+0000) 2024-05-08T22:19:36.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:36 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:36.433+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:19:36.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:36 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:36.357+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:19:36.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:36 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:36.415+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:19:36.891 INFO:journalctl@ceph.osd.3.smithi131.stdout:May 08 22:19:36 smithi131 bash[119818]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-3 2024-05-08T22:19:37.217 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:36 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:36.941+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.182697+0000 front 2024-05-08T22:19:10.182627+0000 (oldest deadline 2024-05-08T22:19:34.282795+0000) 2024-05-08T22:19:37.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:37 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:37.415+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:19:37.835 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:37 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:37.461+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:19:37.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:37 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:37.395+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:19:38.038 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.3.service' 2024-05-08T22:19:38.098 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:37 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:37.943+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.182697+0000 front 2024-05-08T22:19:10.182627+0000 (oldest deadline 2024-05-08T22:19:34.282795+0000) 2024-05-08T22:19:38.099 INFO:journalctl@ceph.osd.3.smithi131.stdout:May 08 22:19:38 smithi131 systemd[1]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.3.service: Succeeded. 2024-05-08T22:19:38.099 INFO:journalctl@ceph.osd.3.smithi131.stdout:May 08 22:19:38 smithi131 systemd[1]: Stopped Ceph osd.3 for 06f1a696-0d86-11ef-bc97-c7b262605968. 2024-05-08T22:19:38.125 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:19:38.125 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-08T22:19:38.125 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-08T22:19:38.125 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.4 2024-05-08T22:19:38.439 INFO:journalctl@ceph.osd.4.smithi132.stdout:May 08 22:19:38 smithi132 systemd[1]: Stopping Ceph osd.4 for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:19:38.457 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:38 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:38.122+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:19:38.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:38 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:38.456+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:19:38.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:38 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:38.438+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:19:38.840 INFO:journalctl@ceph.osd.4.smithi132.stdout:May 08 22:19:38 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-4[91853]: 2024-05-08T22:19:38.479+0000 7f1fa3077700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:19:38.840 INFO:journalctl@ceph.osd.4.smithi132.stdout:May 08 22:19:38 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-4[91853]: 2024-05-08T22:19:38.479+0000 7f1fa3077700 -1 osd.4 102 *** Got signal Terminated *** 2024-05-08T22:19:38.840 INFO:journalctl@ceph.osd.4.smithi132.stdout:May 08 22:19:38 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-4[91853]: 2024-05-08T22:19:38.479+0000 7f1fa3077700 -1 osd.4 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:19:38.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:38 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:38.413+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:19:39.367 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:38 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:38.903+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.182697+0000 front 2024-05-08T22:19:10.182627+0000 (oldest deadline 2024-05-08T22:19:34.282795+0000) 2024-05-08T22:19:39.435 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:39 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:39.084+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:19:39.641 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:39 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:39.367+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:19:39.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:39 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:39.435+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:19:39.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:39 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:39.400+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:19:40.391 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:39 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:39.895+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.182697+0000 front 2024-05-08T22:19:10.182627+0000 (oldest deadline 2024-05-08T22:19:34.282795+0000) 2024-05-08T22:19:40.416 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:40 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:40.088+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:19:40.776 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:40 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:40.430+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:19:40.776 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:40 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:40.430+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:19:40.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:40 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:40.416+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:19:40.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:40 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:40.393+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:19:41.090 INFO:journalctl@ceph.osd.4.smithi132.stdout:May 08 22:19:40 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-4[91853]: 2024-05-08T22:19:40.775+0000 7f1f9f66d700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:16.296398+0000 front 2024-05-08T22:19:16.296485+0000 (oldest deadline 2024-05-08T22:19:40.396769+0000) 2024-05-08T22:19:41.090 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:40 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:40.966+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:19:41.391 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:40 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:40.916+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.182697+0000 front 2024-05-08T22:19:10.182627+0000 (oldest deadline 2024-05-08T22:19:34.282795+0000) 2024-05-08T22:19:41.441 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:41 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:41.133+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:19:41.764 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:41 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:41.447+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:19:41.764 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:41 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:41.447+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:19:41.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:41 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:41.441+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:19:41.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:41 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:41.419+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:19:42.016 INFO:journalctl@ceph.osd.4.smithi132.stdout:May 08 22:19:41 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-4[91853]: 2024-05-08T22:19:41.764+0000 7f1f9f66d700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:16.296398+0000 front 2024-05-08T22:19:16.296485+0000 (oldest deadline 2024-05-08T22:19:40.396769+0000) 2024-05-08T22:19:42.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:42 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:42.016+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:19:42.391 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:41 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:41.935+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.182697+0000 front 2024-05-08T22:19:10.182627+0000 (oldest deadline 2024-05-08T22:19:34.282795+0000) 2024-05-08T22:19:42.466 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:42 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:42.106+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:19:42.717 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:42 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:42.439+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:19:42.717 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:42 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:42.439+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:19:42.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:42 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:42.466+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:19:42.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:42 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:42.439+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:19:42.973 INFO:journalctl@ceph.osd.4.smithi132.stdout:May 08 22:19:42 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-4[91853]: 2024-05-08T22:19:42.715+0000 7f1f9f66d700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:16.296398+0000 front 2024-05-08T22:19:16.296485+0000 (oldest deadline 2024-05-08T22:19:40.396769+0000) 2024-05-08T22:19:43.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:42 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:42.971+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:19:43.391 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:42 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:42.906+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.182697+0000 front 2024-05-08T22:19:10.182627+0000 (oldest deadline 2024-05-08T22:19:34.282795+0000) 2024-05-08T22:19:43.461 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:43 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:43.153+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:19:43.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:43 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:43.460+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:19:43.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:43 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:43.454+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:19:43.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:43 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:43.454+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:19:43.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:43 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:43.421+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:19:44.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:43 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:43.976+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:19:44.391 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:43 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:43.938+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.182697+0000 front 2024-05-08T22:19:10.182627+0000 (oldest deadline 2024-05-08T22:19:34.282795+0000) 2024-05-08T22:19:44.391 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:43 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:43.938+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.884603+0000 front 2024-05-08T22:19:18.884922+0000 (oldest deadline 2024-05-08T22:19:43.584710+0000) 2024-05-08T22:19:44.469 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:44 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:44.152+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:19:44.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:44 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:44.468+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:19:44.840 INFO:journalctl@ceph.osd.4.smithi132.stdout:May 08 22:19:44 smithi132 bash[120917]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-4 2024-05-08T22:19:44.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:44 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:44.485+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:19:44.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:44 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:44.485+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:19:44.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:44 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:44.457+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:19:44.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:44 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:44.457+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:19:45.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:45 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:45.000+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:19:45.391 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:44 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:44.978+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.182697+0000 front 2024-05-08T22:19:10.182627+0000 (oldest deadline 2024-05-08T22:19:34.282795+0000) 2024-05-08T22:19:45.391 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:44 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:44.978+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.884603+0000 front 2024-05-08T22:19:18.884922+0000 (oldest deadline 2024-05-08T22:19:43.584710+0000) 2024-05-08T22:19:45.449 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 08 22:19:45 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-5[91506]: 2024-05-08T22:19:45.262+0000 7f173ce26700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.294482+0000 front 2024-05-08T22:19:19.294598+0000 (oldest deadline 2024-05-08T22:19:45.194552+0000) 2024-05-08T22:19:45.449 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:45 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:45.128+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:19:45.449 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:45 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:45.128+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:19:45.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:45 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:45.448+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:19:45.815 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:45 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:45.535+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:19:45.815 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:45 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:45.535+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:19:45.822 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.4.service' 2024-05-08T22:19:45.863 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:19:45.863 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-08T22:19:45.863 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-08T22:19:45.864 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.5 2024-05-08T22:19:45.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:45 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:45.421+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:19:45.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:45 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:45.421+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:19:46.090 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:46 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:46.032+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:19:46.276 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 08 22:19:45 smithi157 systemd[1]: Stopping Ceph osd.5 for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:19:46.276 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 08 22:19:46 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-5[91506]: 2024-05-08T22:19:46.105+0000 7f1744838700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:19:46.276 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 08 22:19:46 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-5[91506]: 2024-05-08T22:19:46.105+0000 7f1744838700 -1 osd.5 102 *** Got signal Terminated *** 2024-05-08T22:19:46.277 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 08 22:19:46 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-5[91506]: 2024-05-08T22:19:46.105+0000 7f1744838700 -1 osd.5 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:19:46.277 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:46 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:46.113+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:19:46.277 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:46 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:46.113+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:19:46.391 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:45 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:45.936+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.182697+0000 front 2024-05-08T22:19:10.182627+0000 (oldest deadline 2024-05-08T22:19:34.282795+0000) 2024-05-08T22:19:46.391 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:45 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:45.936+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.884603+0000 front 2024-05-08T22:19:18.884922+0000 (oldest deadline 2024-05-08T22:19:43.584710+0000) 2024-05-08T22:19:46.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:46 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:46.411+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:19:46.776 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 08 22:19:46 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-5[91506]: 2024-05-08T22:19:46.300+0000 7f173ce26700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.294482+0000 front 2024-05-08T22:19:19.294598+0000 (oldest deadline 2024-05-08T22:19:45.194552+0000) 2024-05-08T22:19:46.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:46 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:46.510+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:19:46.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:46 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:46.510+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:19:46.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:46 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:46.449+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:19:46.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:46 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:46.449+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:19:47.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:47 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:47.057+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:19:47.391 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:46 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:46.914+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.182697+0000 front 2024-05-08T22:19:10.182627+0000 (oldest deadline 2024-05-08T22:19:34.282795+0000) 2024-05-08T22:19:47.391 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:46 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:46.914+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.884603+0000 front 2024-05-08T22:19:18.884922+0000 (oldest deadline 2024-05-08T22:19:43.584710+0000) 2024-05-08T22:19:47.436 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 08 22:19:47 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-5[91506]: 2024-05-08T22:19:47.250+0000 7f173ce26700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.294482+0000 front 2024-05-08T22:19:19.294598+0000 (oldest deadline 2024-05-08T22:19:45.194552+0000) 2024-05-08T22:19:47.436 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:47 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:47.131+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:19:47.436 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:47 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:47.131+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:19:47.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:47 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:47.435+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:19:47.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:47 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:47.527+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:19:47.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:47 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:47.527+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:19:47.883 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:47 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:47.465+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:19:47.883 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:47 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:47.465+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:19:48.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:47 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:47.882+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.182697+0000 front 2024-05-08T22:19:10.182627+0000 (oldest deadline 2024-05-08T22:19:34.282795+0000) 2024-05-08T22:19:48.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:47 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:47.882+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.884603+0000 front 2024-05-08T22:19:18.884922+0000 (oldest deadline 2024-05-08T22:19:43.584710+0000) 2024-05-08T22:19:48.471 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 08 22:19:48 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-5[91506]: 2024-05-08T22:19:48.299+0000 7f173ce26700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.294482+0000 front 2024-05-08T22:19:19.294598+0000 (oldest deadline 2024-05-08T22:19:45.194552+0000) 2024-05-08T22:19:48.472 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:48 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:48.163+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:19:48.472 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:48 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:48.163+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:19:48.545 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:48 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:48.097+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:19:48.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:48 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:48.470+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:19:48.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:48 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:48.470+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.361459+0000 front 2024-05-08T22:19:23.361491+0000 (oldest deadline 2024-05-08T22:19:47.461727+0000) 2024-05-08T22:19:48.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:48 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:48.544+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:19:48.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:48 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:48.544+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:19:48.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:48 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:48.544+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:19:48.849 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:48 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:48.481+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:19:48.849 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:48 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:48.481+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:19:49.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:48 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:48.847+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.182697+0000 front 2024-05-08T22:19:10.182627+0000 (oldest deadline 2024-05-08T22:19:34.282795+0000) 2024-05-08T22:19:49.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:48 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:48.847+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.884603+0000 front 2024-05-08T22:19:18.884922+0000 (oldest deadline 2024-05-08T22:19:43.584710+0000) 2024-05-08T22:19:49.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:48 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:48.847+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.585131+0000 front 2024-05-08T22:19:23.585250+0000 (oldest deadline 2024-05-08T22:19:48.285366+0000) 2024-05-08T22:19:49.514 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 08 22:19:49 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-5[91506]: 2024-05-08T22:19:49.297+0000 7f173ce26700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.294482+0000 front 2024-05-08T22:19:19.294598+0000 (oldest deadline 2024-05-08T22:19:45.194552+0000) 2024-05-08T22:19:49.514 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 08 22:19:49 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-5[91506]: 2024-05-08T22:19:49.297+0000 7f173ce26700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:25.695500+0000 front 2024-05-08T22:19:25.695445+0000 (oldest deadline 2024-05-08T22:19:48.595615+0000) 2024-05-08T22:19:49.514 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:49 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:49.203+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:19:49.515 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:49 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:49.203+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:19:49.515 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:49 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:49.203+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:19:49.555 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:49 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:49.089+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:19:49.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:49 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:49.513+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:19:49.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:49 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:49.513+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.361459+0000 front 2024-05-08T22:19:23.361491+0000 (oldest deadline 2024-05-08T22:19:47.461727+0000) 2024-05-08T22:19:49.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:49 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:49.553+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:19:49.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:49 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:49.553+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:19:49.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:49 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:49.553+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:19:49.841 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:49 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:49.492+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:19:49.841 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:49 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:49.492+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:19:49.841 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:49 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:49.492+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:19:50.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:49 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:49.840+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.182697+0000 front 2024-05-08T22:19:10.182627+0000 (oldest deadline 2024-05-08T22:19:34.282795+0000) 2024-05-08T22:19:50.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:49 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:49.840+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.884603+0000 front 2024-05-08T22:19:18.884922+0000 (oldest deadline 2024-05-08T22:19:43.584710+0000) 2024-05-08T22:19:50.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:49 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:49.840+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.585131+0000 front 2024-05-08T22:19:23.585250+0000 (oldest deadline 2024-05-08T22:19:48.285366+0000) 2024-05-08T22:19:50.497 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 08 22:19:50 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-5[91506]: 2024-05-08T22:19:50.334+0000 7f173ce26700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.294482+0000 front 2024-05-08T22:19:19.294598+0000 (oldest deadline 2024-05-08T22:19:45.194552+0000) 2024-05-08T22:19:50.498 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 08 22:19:50 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-5[91506]: 2024-05-08T22:19:50.334+0000 7f173ce26700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:25.695500+0000 front 2024-05-08T22:19:25.695445+0000 (oldest deadline 2024-05-08T22:19:48.595615+0000) 2024-05-08T22:19:50.498 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:50 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:50.211+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:19:50.498 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:50 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:50.211+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:19:50.498 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:50 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:50.211+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:19:50.529 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:50 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:50.126+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:19:50.529 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:50 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:50.126+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:26.119819+0000 front 2024-05-08T22:19:26.119890+0000 (oldest deadline 2024-05-08T22:19:49.619911+0000) 2024-05-08T22:19:50.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:50 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:50.496+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:19:50.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:50 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:50.496+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.361459+0000 front 2024-05-08T22:19:23.361491+0000 (oldest deadline 2024-05-08T22:19:47.461727+0000) 2024-05-08T22:19:50.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:50 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:50.527+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:19:50.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:50 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:50.527+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:19:50.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:50 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:50.527+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:19:50.874 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:50 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:50.488+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:19:50.874 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:50 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:50.488+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:19:50.874 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:50 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:50.488+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:19:51.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:50 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:50.873+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.182697+0000 front 2024-05-08T22:19:10.182627+0000 (oldest deadline 2024-05-08T22:19:34.282795+0000) 2024-05-08T22:19:51.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:50 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:50.873+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.884603+0000 front 2024-05-08T22:19:18.884922+0000 (oldest deadline 2024-05-08T22:19:43.584710+0000) 2024-05-08T22:19:51.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:50 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:50.873+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.585131+0000 front 2024-05-08T22:19:23.585250+0000 (oldest deadline 2024-05-08T22:19:48.285366+0000) 2024-05-08T22:19:51.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:51 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:51.083+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:19:51.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:51 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:51.083+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:26.119819+0000 front 2024-05-08T22:19:26.119890+0000 (oldest deadline 2024-05-08T22:19:49.619911+0000) 2024-05-08T22:19:51.526 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:51 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:51.478+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:19:51.526 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:51 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:51.478+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.361459+0000 front 2024-05-08T22:19:23.361491+0000 (oldest deadline 2024-05-08T22:19:47.461727+0000) 2024-05-08T22:19:51.527 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:51 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:51.231+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:19:51.527 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:51 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:51.231+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:19:51.527 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:51 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:51.231+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:19:51.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:51 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:51.529+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:19:51.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:51 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:51.529+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:19:51.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:51 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:51.529+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:19:51.857 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:51 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:51.457+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:19:51.857 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:51 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:51.457+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:19:51.858 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:51 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:51.457+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:19:52.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:51 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:51.856+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.182697+0000 front 2024-05-08T22:19:10.182627+0000 (oldest deadline 2024-05-08T22:19:34.282795+0000) 2024-05-08T22:19:52.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:51 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:51.856+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.884603+0000 front 2024-05-08T22:19:18.884922+0000 (oldest deadline 2024-05-08T22:19:43.584710+0000) 2024-05-08T22:19:52.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:51 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:51.856+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.585131+0000 front 2024-05-08T22:19:23.585250+0000 (oldest deadline 2024-05-08T22:19:48.285366+0000) 2024-05-08T22:19:52.222 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 08 22:19:51 smithi157 bash[122096]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-5 2024-05-08T22:19:52.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:52 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:52.074+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:19:52.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:52 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:52.074+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:26.119819+0000 front 2024-05-08T22:19:26.119890+0000 (oldest deadline 2024-05-08T22:19:49.619911+0000) 2024-05-08T22:19:52.522 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:52 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:52.220+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:19:52.522 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:52 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:52.220+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:19:52.523 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:52 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:52.220+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:19:52.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:52 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:52.521+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:19:52.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:52 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:52.521+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.361459+0000 front 2024-05-08T22:19:23.361491+0000 (oldest deadline 2024-05-08T22:19:47.461727+0000) 2024-05-08T22:19:52.835 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:52 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:52.435+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:19:52.835 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:52 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:52.435+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:19:52.835 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:52 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:52.435+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:19:52.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:52 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:52.576+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:19:52.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:52 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:52.576+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:19:52.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:52 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:52.576+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:19:52.994 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.5.service' 2024-05-08T22:19:53.035 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:19:53.035 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-08T22:19:53.035 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-08T22:19:53.035 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.6 2024-05-08T22:19:53.100 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:52 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:52.833+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.182697+0000 front 2024-05-08T22:19:10.182627+0000 (oldest deadline 2024-05-08T22:19:34.282795+0000) 2024-05-08T22:19:53.100 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:52 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:52.833+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.884603+0000 front 2024-05-08T22:19:18.884922+0000 (oldest deadline 2024-05-08T22:19:43.584710+0000) 2024-05-08T22:19:53.101 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:52 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:52.833+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.585131+0000 front 2024-05-08T22:19:23.585250+0000 (oldest deadline 2024-05-08T22:19:48.285366+0000) 2024-05-08T22:19:53.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:53 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:53.044+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:19:53.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:53 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:53.044+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:26.119819+0000 front 2024-05-08T22:19:26.119890+0000 (oldest deadline 2024-05-08T22:19:49.619911+0000) 2024-05-08T22:19:53.391 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:53 smithi131 systemd[1]: Stopping Ceph osd.6 for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:19:53.526 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:53 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:53.480+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:19:53.526 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:53 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:53.480+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.361459+0000 front 2024-05-08T22:19:23.361491+0000 (oldest deadline 2024-05-08T22:19:47.461727+0000) 2024-05-08T22:19:53.527 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:53 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:53.232+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:19:53.527 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:53 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:53.232+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:19:53.527 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:53 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:53.232+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:19:53.866 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:53 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:53.483+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:19:53.866 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:53 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:53.483+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:19:53.867 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:53 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:53.483+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:19:53.867 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:53 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:53.441+0000 7ffa0be00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:19:53.867 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:53 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:53.441+0000 7ffa0be00700 -1 osd.6 102 *** Got signal Terminated *** 2024-05-08T22:19:53.867 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:53 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:53.441+0000 7ffa0be00700 -1 osd.6 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:19:54.069 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:53 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:53.591+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:19:54.069 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:53 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:53.591+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:19:54.069 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:53 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:53.591+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:19:54.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:53 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:53.865+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.182697+0000 front 2024-05-08T22:19:10.182627+0000 (oldest deadline 2024-05-08T22:19:34.282795+0000) 2024-05-08T22:19:54.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:53 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:53.865+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.884603+0000 front 2024-05-08T22:19:18.884922+0000 (oldest deadline 2024-05-08T22:19:43.584710+0000) 2024-05-08T22:19:54.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:53 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:53.865+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.585131+0000 front 2024-05-08T22:19:23.585250+0000 (oldest deadline 2024-05-08T22:19:48.285366+0000) 2024-05-08T22:19:54.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:54 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:54.068+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:19:54.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:54 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:54.068+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:26.119819+0000 front 2024-05-08T22:19:26.119890+0000 (oldest deadline 2024-05-08T22:19:49.619911+0000) 2024-05-08T22:19:54.519 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:54 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:54.201+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:19:54.519 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:54 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:54.201+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:19:54.519 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:54 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:54.201+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:19:54.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:54 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:54.518+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:19:54.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:54 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:54.518+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.361459+0000 front 2024-05-08T22:19:23.361491+0000 (oldest deadline 2024-05-08T22:19:47.461727+0000) 2024-05-08T22:19:54.838 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:54 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:54.459+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:19:54.838 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:54 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:54.459+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:19:54.839 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:54 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:54.459+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:19:55.024 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:54 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:54.638+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:19:55.024 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:54 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:54.638+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:19:55.024 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:54 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:54.638+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:19:55.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:54 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:54.838+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.182697+0000 front 2024-05-08T22:19:10.182627+0000 (oldest deadline 2024-05-08T22:19:34.282795+0000) 2024-05-08T22:19:55.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:54 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:54.838+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.884603+0000 front 2024-05-08T22:19:18.884922+0000 (oldest deadline 2024-05-08T22:19:43.584710+0000) 2024-05-08T22:19:55.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:54 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:54.838+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.585131+0000 front 2024-05-08T22:19:23.585250+0000 (oldest deadline 2024-05-08T22:19:48.285366+0000) 2024-05-08T22:19:55.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:55 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:55.022+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:19:55.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:55 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:55.022+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:26.119819+0000 front 2024-05-08T22:19:26.119890+0000 (oldest deadline 2024-05-08T22:19:49.619911+0000) 2024-05-08T22:19:55.502 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:55 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:55.243+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:19:55.502 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:55 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:55.243+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:19:55.502 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:55 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:55.243+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:19:55.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:55 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:55.501+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:19:55.777 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:55 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:55.501+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.361459+0000 front 2024-05-08T22:19:23.361491+0000 (oldest deadline 2024-05-08T22:19:47.461727+0000) 2024-05-08T22:19:55.809 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:55 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:55.420+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:19:55.809 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:55 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:55.420+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:19:55.809 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:55 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:55.420+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:19:55.985 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:55 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:55.597+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:19:55.985 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:55 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:55.597+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:19:55.986 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:55 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:55.597+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:19:56.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:55 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:55.808+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.182697+0000 front 2024-05-08T22:19:10.182627+0000 (oldest deadline 2024-05-08T22:19:34.282795+0000) 2024-05-08T22:19:56.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:55 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:55.808+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.884603+0000 front 2024-05-08T22:19:18.884922+0000 (oldest deadline 2024-05-08T22:19:43.584710+0000) 2024-05-08T22:19:56.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:55 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:55.808+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.585131+0000 front 2024-05-08T22:19:23.585250+0000 (oldest deadline 2024-05-08T22:19:48.285366+0000) 2024-05-08T22:19:56.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:55 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:55.808+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:32.386776+0000 front 2024-05-08T22:19:32.386832+0000 (oldest deadline 2024-05-08T22:19:55.287028+0000) 2024-05-08T22:19:56.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:55 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:55.984+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:19:56.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:55 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:55.984+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:26.119819+0000 front 2024-05-08T22:19:26.119890+0000 (oldest deadline 2024-05-08T22:19:49.619911+0000) 2024-05-08T22:19:56.526 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:56 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:56.455+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:19:56.526 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:56 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:56.455+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.361459+0000 front 2024-05-08T22:19:23.361491+0000 (oldest deadline 2024-05-08T22:19:47.461727+0000) 2024-05-08T22:19:56.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:56 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:56.221+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:19:56.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:56 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:56.222+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:19:56.527 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:56 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:56.222+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:19:56.828 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:56 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:56.434+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:19:56.829 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:56 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:56.434+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:19:56.829 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:56 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:56.434+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:19:56.829 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:56 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:56.434+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.371986+0000 front 2024-05-08T22:19:30.372008+0000 (oldest deadline 2024-05-08T22:19:56.272139+0000) 2024-05-08T22:19:56.959 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:56 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:56.634+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:19:56.959 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:56 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:56.634+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:19:56.959 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:56 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:56.634+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:19:56.959 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:56 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:56.634+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:19:57.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:56 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:56.827+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.182697+0000 front 2024-05-08T22:19:10.182627+0000 (oldest deadline 2024-05-08T22:19:34.282795+0000) 2024-05-08T22:19:57.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:56 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:56.827+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.884603+0000 front 2024-05-08T22:19:18.884922+0000 (oldest deadline 2024-05-08T22:19:43.584710+0000) 2024-05-08T22:19:57.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:56 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:56.827+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.585131+0000 front 2024-05-08T22:19:23.585250+0000 (oldest deadline 2024-05-08T22:19:48.285366+0000) 2024-05-08T22:19:57.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:56 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:56.827+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:32.386776+0000 front 2024-05-08T22:19:32.386832+0000 (oldest deadline 2024-05-08T22:19:55.287028+0000) 2024-05-08T22:19:57.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:56 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:56.957+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:19:57.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:56 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:56.957+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:26.119819+0000 front 2024-05-08T22:19:26.119890+0000 (oldest deadline 2024-05-08T22:19:49.619911+0000) 2024-05-08T22:19:57.341 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:56 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:56.957+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.721100+0000 front 2024-05-08T22:19:30.721129+0000 (oldest deadline 2024-05-08T22:19:56.021232+0000) 2024-05-08T22:19:57.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:57 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:57.235+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:19:57.504 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:57 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:57.235+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:19:57.504 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:57 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:57.235+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:19:57.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:57 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:57.502+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:19:57.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:57 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:57.502+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.361459+0000 front 2024-05-08T22:19:23.361491+0000 (oldest deadline 2024-05-08T22:19:47.461727+0000) 2024-05-08T22:19:57.829 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:57 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:57.431+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:19:57.829 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:57 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:57.431+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:19:57.829 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:57 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:57.431+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:19:57.829 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:57 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:57.431+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.371986+0000 front 2024-05-08T22:19:30.372008+0000 (oldest deadline 2024-05-08T22:19:56.272139+0000) 2024-05-08T22:19:57.962 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:57 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:57.619+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:19:57.962 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:57 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:57.619+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:19:57.962 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:57 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:57.619+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:19:57.963 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:57 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:57.619+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:19:58.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:57 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:57.828+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.182697+0000 front 2024-05-08T22:19:10.182627+0000 (oldest deadline 2024-05-08T22:19:34.282795+0000) 2024-05-08T22:19:58.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:57 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:57.828+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.884603+0000 front 2024-05-08T22:19:18.884922+0000 (oldest deadline 2024-05-08T22:19:43.584710+0000) 2024-05-08T22:19:58.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:57 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:57.828+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.585131+0000 front 2024-05-08T22:19:23.585250+0000 (oldest deadline 2024-05-08T22:19:48.285366+0000) 2024-05-08T22:19:58.141 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:57 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6[72008]: 2024-05-08T22:19:57.828+0000 7ffa07600700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:32.386776+0000 front 2024-05-08T22:19:32.386832+0000 (oldest deadline 2024-05-08T22:19:55.287028+0000) 2024-05-08T22:19:58.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:57 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:57.961+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:19:58.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:57 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:57.961+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:26.119819+0000 front 2024-05-08T22:19:26.119890+0000 (oldest deadline 2024-05-08T22:19:49.619911+0000) 2024-05-08T22:19:58.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:57 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:57.961+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.721100+0000 front 2024-05-08T22:19:30.721129+0000 (oldest deadline 2024-05-08T22:19:56.021232+0000) 2024-05-08T22:19:58.519 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:58 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:58.232+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:19:58.519 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:58 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:58.232+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:19:58.519 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:58 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:58.232+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:19:58.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:58 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:58.518+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:19:58.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:58 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:58.518+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.361459+0000 front 2024-05-08T22:19:23.361491+0000 (oldest deadline 2024-05-08T22:19:47.461727+0000) 2024-05-08T22:19:58.776 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:58 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:58.518+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:33.263559+0000 front 2024-05-08T22:19:33.263549+0000 (oldest deadline 2024-05-08T22:19:57.963489+0000) 2024-05-08T22:19:58.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:58 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:58.448+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:19:58.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:58 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:58.448+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:19:58.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:58 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:58.448+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:19:58.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:58 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:58.449+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.371986+0000 front 2024-05-08T22:19:30.372008+0000 (oldest deadline 2024-05-08T22:19:56.272139+0000) 2024-05-08T22:19:58.989 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:58 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:58.648+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:19:58.989 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:58 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:58.648+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:19:58.989 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:58 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:58.648+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:19:58.989 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:58 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:58.648+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:19:59.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:58 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:58.988+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:19:59.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:58 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:58.988+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:26.119819+0000 front 2024-05-08T22:19:26.119890+0000 (oldest deadline 2024-05-08T22:19:49.619911+0000) 2024-05-08T22:19:59.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:19:58 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:19:58.988+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.721100+0000 front 2024-05-08T22:19:30.721129+0000 (oldest deadline 2024-05-08T22:19:56.021232+0000) 2024-05-08T22:19:59.391 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 08 22:19:59 smithi131 bash[120073]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-6 2024-05-08T22:19:59.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:59 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:59.271+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:19:59.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:59 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:59.271+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:19:59.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:19:59 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:19:59.271+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:19:59.886 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:59 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:59.453+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:19:59.886 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:59 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:59.453+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:19:59.886 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:59 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:59.453+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:19:59.886 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:19:59 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:19:59.453+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.371986+0000 front 2024-05-08T22:19:30.372008+0000 (oldest deadline 2024-05-08T22:19:56.272139+0000) 2024-05-08T22:20:00.016 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:59 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:59.601+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:00.016 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:59 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:59.601+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:00.016 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:59 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:59.601+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:00.016 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:19:59 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:19:59.601+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:00.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:59 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:59.560+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:20:00.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:59 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:59.560+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.361459+0000 front 2024-05-08T22:19:23.361491+0000 (oldest deadline 2024-05-08T22:19:47.461727+0000) 2024-05-08T22:20:00.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:19:59 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:19:59.560+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:33.263559+0000 front 2024-05-08T22:19:33.263549+0000 (oldest deadline 2024-05-08T22:19:57.963489+0000) 2024-05-08T22:20:00.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:00 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:00.015+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:20:00.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:00 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:00.015+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:26.119819+0000 front 2024-05-08T22:19:26.119890+0000 (oldest deadline 2024-05-08T22:19:49.619911+0000) 2024-05-08T22:20:00.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:00 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:00.015+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.721100+0000 front 2024-05-08T22:19:30.721129+0000 (oldest deadline 2024-05-08T22:19:56.021232+0000) 2024-05-08T22:20:00.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:00 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:00.270+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:00.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:00 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:00.270+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:00.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:00 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:00.270+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:00.641 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:00 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:00.501+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:20:00.641 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:00 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:00.501+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:20:00.641 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:00 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:00.501+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:20:00.641 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:00 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:00.501+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.371986+0000 front 2024-05-08T22:19:30.372008+0000 (oldest deadline 2024-05-08T22:19:56.272139+0000) 2024-05-08T22:20:00.718 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.6.service' 2024-05-08T22:20:00.807 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:20:00.807 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-08T22:20:00.807 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-08T22:20:00.808 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.7 2024-05-08T22:20:00.922 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:00 smithi132 systemd[1]: Stopping Ceph osd.7 for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:20:00.923 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:00 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:00.630+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:00.923 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:00 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:00.630+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:00.923 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:00 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:00.630+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:00.923 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:00 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:00.630+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:01.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:00 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:00.576+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:20:01.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:00 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:00.576+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.361459+0000 front 2024-05-08T22:19:23.361491+0000 (oldest deadline 2024-05-08T22:19:47.461727+0000) 2024-05-08T22:20:01.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:00 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:00.576+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:33.263559+0000 front 2024-05-08T22:19:33.263549+0000 (oldest deadline 2024-05-08T22:19:57.963489+0000) 2024-05-08T22:20:01.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:01 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:01.055+0000 7f5affbaa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:20:01.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:01 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:01.055+0000 7f5affbaa700 -1 osd.7 102 *** Got signal Terminated *** 2024-05-08T22:20:01.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:01 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:01.055+0000 7f5affbaa700 -1 osd.7 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:20:01.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:01 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:01.057+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:20:01.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:01 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:01.057+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:26.119819+0000 front 2024-05-08T22:19:26.119890+0000 (oldest deadline 2024-05-08T22:19:49.619911+0000) 2024-05-08T22:20:01.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:01 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:01.057+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.721100+0000 front 2024-05-08T22:19:30.721129+0000 (oldest deadline 2024-05-08T22:19:56.021232+0000) 2024-05-08T22:20:01.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:01 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:01.263+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:01.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:01 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:01.263+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:01.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:01 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:01.263+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:01.893 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:01 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:01.520+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:20:01.894 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:01 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:01.520+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:20:01.894 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:01 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:01.520+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:20:01.894 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:01 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:01.520+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.371986+0000 front 2024-05-08T22:19:30.372008+0000 (oldest deadline 2024-05-08T22:19:56.272139+0000) 2024-05-08T22:20:02.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:01 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:01.542+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:20:02.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:01 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:01.542+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.361459+0000 front 2024-05-08T22:19:23.361491+0000 (oldest deadline 2024-05-08T22:19:47.461727+0000) 2024-05-08T22:20:02.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:01 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:01.542+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:33.263559+0000 front 2024-05-08T22:19:33.263549+0000 (oldest deadline 2024-05-08T22:19:57.963489+0000) 2024-05-08T22:20:02.069 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:01 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:01.615+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:02.069 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:01 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:01.615+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:02.069 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:01 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:01.615+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:02.069 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:01 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:01.615+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:02.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:02 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:02.068+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:20:02.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:02 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:02.068+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:26.119819+0000 front 2024-05-08T22:19:26.119890+0000 (oldest deadline 2024-05-08T22:19:49.619911+0000) 2024-05-08T22:20:02.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:02 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:02.068+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.721100+0000 front 2024-05-08T22:19:30.721129+0000 (oldest deadline 2024-05-08T22:19:56.021232+0000) 2024-05-08T22:20:02.567 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:02 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:02.284+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:02.567 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:02 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:02.284+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:02.567 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:02 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:02.284+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:02.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:02 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:02.509+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:20:02.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:02 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:02.509+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:20:02.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:02 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:02.509+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:20:02.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:02 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:02.509+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.371986+0000 front 2024-05-08T22:19:30.372008+0000 (oldest deadline 2024-05-08T22:19:56.272139+0000) 2024-05-08T22:20:03.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:02 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:02.566+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:20:03.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:02 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:02.566+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.361459+0000 front 2024-05-08T22:19:23.361491+0000 (oldest deadline 2024-05-08T22:19:47.461727+0000) 2024-05-08T22:20:03.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:02 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:02.566+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:33.263559+0000 front 2024-05-08T22:19:33.263549+0000 (oldest deadline 2024-05-08T22:19:57.963489+0000) 2024-05-08T22:20:03.043 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:02 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:02.604+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:03.044 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:02 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:02.604+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:03.044 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:02 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:02.604+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:03.044 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:02 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:02.604+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:03.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:03 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:03.043+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:20:03.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:03 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:03.043+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:26.119819+0000 front 2024-05-08T22:19:26.119890+0000 (oldest deadline 2024-05-08T22:19:49.619911+0000) 2024-05-08T22:20:03.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:03 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:03.043+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.721100+0000 front 2024-05-08T22:19:30.721129+0000 (oldest deadline 2024-05-08T22:19:56.021232+0000) 2024-05-08T22:20:03.615 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:03 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:03.304+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:03.615 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:03 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:03.304+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:03.615 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:03 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:03.304+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:03.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:03 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:03.500+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:20:03.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:03 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:03.500+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:20:03.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:03 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:03.500+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:20:03.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:03 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:03.500+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.371986+0000 front 2024-05-08T22:19:30.372008+0000 (oldest deadline 2024-05-08T22:19:56.272139+0000) 2024-05-08T22:20:04.003 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:03 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:03.596+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:04.004 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:03 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:03.596+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:04.004 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:03 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:03.596+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:04.004 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:03 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:03.596+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:04.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:03 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:03.614+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:20:04.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:03 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:03.614+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.361459+0000 front 2024-05-08T22:19:23.361491+0000 (oldest deadline 2024-05-08T22:19:47.461727+0000) 2024-05-08T22:20:04.027 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:03 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:03.614+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:33.263559+0000 front 2024-05-08T22:19:33.263549+0000 (oldest deadline 2024-05-08T22:19:57.963489+0000) 2024-05-08T22:20:04.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:04 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:04.003+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:20:04.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:04 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:04.003+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:26.119819+0000 front 2024-05-08T22:19:26.119890+0000 (oldest deadline 2024-05-08T22:19:49.619911+0000) 2024-05-08T22:20:04.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:04 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:04.003+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.721100+0000 front 2024-05-08T22:19:30.721129+0000 (oldest deadline 2024-05-08T22:19:56.021232+0000) 2024-05-08T22:20:04.623 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:04 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:04.279+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:04.623 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:04 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:04.279+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:04.623 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:04 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:04.279+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:04.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:04 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:04.586+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:04.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:04 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:04.586+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:04.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:04 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:04.586+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:04.840 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:04 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:04.586+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:04.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:04 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:04.481+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:20:04.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:04 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:04.481+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:20:04.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:04 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:04.481+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:20:04.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:04 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:04.481+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.371986+0000 front 2024-05-08T22:19:30.372008+0000 (oldest deadline 2024-05-08T22:19:56.272139+0000) 2024-05-08T22:20:05.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:04 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:04.622+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:20:05.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:04 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:04.622+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.361459+0000 front 2024-05-08T22:19:23.361491+0000 (oldest deadline 2024-05-08T22:19:47.461727+0000) 2024-05-08T22:20:05.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:04 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:04.622+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:33.263559+0000 front 2024-05-08T22:19:33.263549+0000 (oldest deadline 2024-05-08T22:19:57.963489+0000) 2024-05-08T22:20:05.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:04 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:04.993+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:20:05.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:04 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:04.993+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:26.119819+0000 front 2024-05-08T22:19:26.119890+0000 (oldest deadline 2024-05-08T22:19:49.619911+0000) 2024-05-08T22:20:05.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:04 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:04.993+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.721100+0000 front 2024-05-08T22:19:30.721129+0000 (oldest deadline 2024-05-08T22:19:56.021232+0000) 2024-05-08T22:20:05.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:04 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:04.993+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:41.922101+0000 front 2024-05-08T22:19:41.922124+0000 (oldest deadline 2024-05-08T22:20:04.222265+0000) 2024-05-08T22:20:05.649 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:05 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:05.324+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:05.649 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:05 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:05.324+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:05.649 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:05 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:05.324+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:05.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:05 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:05.521+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:20:05.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:05 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:05.521+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:20:05.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:05 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:05.521+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:20:05.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:05 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:05.521+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.371986+0000 front 2024-05-08T22:19:30.372008+0000 (oldest deadline 2024-05-08T22:19:56.272139+0000) 2024-05-08T22:20:06.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:05 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:05.648+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:20:06.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:05 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:05.648+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.361459+0000 front 2024-05-08T22:19:23.361491+0000 (oldest deadline 2024-05-08T22:19:47.461727+0000) 2024-05-08T22:20:06.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:05 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:05.648+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:33.263559+0000 front 2024-05-08T22:19:33.263549+0000 (oldest deadline 2024-05-08T22:19:57.963489+0000) 2024-05-08T22:20:06.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:05 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:05.648+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:41.464828+0000 front 2024-05-08T22:19:41.464948+0000 (oldest deadline 2024-05-08T22:20:04.964705+0000) 2024-05-08T22:20:06.028 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:05 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:05.598+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:06.028 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:05 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:05.598+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:06.028 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:05 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:05.598+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:06.028 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:05 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:05.598+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:06.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:06 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:06.027+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:15.518745+0000 front 2024-05-08T22:19:15.518729+0000 (oldest deadline 2024-05-08T22:19:40.218864+0000) 2024-05-08T22:20:06.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:06 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:06.027+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:26.119819+0000 front 2024-05-08T22:19:26.119890+0000 (oldest deadline 2024-05-08T22:19:49.619911+0000) 2024-05-08T22:20:06.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:06 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:06.027+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.721100+0000 front 2024-05-08T22:19:30.721129+0000 (oldest deadline 2024-05-08T22:19:56.021232+0000) 2024-05-08T22:20:06.340 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:06 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7[99818]: 2024-05-08T22:20:06.027+0000 7f5afc1a0700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:41.922101+0000 front 2024-05-08T22:19:41.922124+0000 (oldest deadline 2024-05-08T22:20:04.222265+0000) 2024-05-08T22:20:06.659 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:06 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:06.303+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:06.660 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:06 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:06.303+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:06.660 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:06 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:06.303+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:06.660 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:06 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:06.303+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:06.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:06 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:06.530+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:20:06.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:06 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:06.530+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:20:06.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:06 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:06.530+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:20:06.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:06 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:06.530+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.371986+0000 front 2024-05-08T22:19:30.372008+0000 (oldest deadline 2024-05-08T22:19:56.272139+0000) 2024-05-08T22:20:07.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:06 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:06.658+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:20:07.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:06 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:06.658+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.361459+0000 front 2024-05-08T22:19:23.361491+0000 (oldest deadline 2024-05-08T22:19:47.461727+0000) 2024-05-08T22:20:07.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:06 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:06.658+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:33.263559+0000 front 2024-05-08T22:19:33.263549+0000 (oldest deadline 2024-05-08T22:19:57.963489+0000) 2024-05-08T22:20:07.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:06 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:06.658+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:41.464828+0000 front 2024-05-08T22:19:41.464948+0000 (oldest deadline 2024-05-08T22:20:04.964705+0000) 2024-05-08T22:20:07.090 INFO:journalctl@ceph.osd.7.smithi132.stdout:May 08 22:20:06 smithi132 bash[121181]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-7 2024-05-08T22:20:07.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:06 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:06.626+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:07.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:06 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:06.626+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:07.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:06 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:06.626+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:07.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:06 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:06.626+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:07.692 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:07 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:07.310+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:07.692 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:07 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:07.310+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:07.692 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:07 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:07.310+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:07.692 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:07 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:07.310+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:07.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:07 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:07.574+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:20:07.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:07 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:07.574+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:20:07.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:07 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:07.574+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:20:07.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:07 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:07.574+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.371986+0000 front 2024-05-08T22:19:30.372008+0000 (oldest deadline 2024-05-08T22:19:56.272139+0000) 2024-05-08T22:20:07.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:07 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:07.574+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:41.473774+0000 front 2024-05-08T22:19:41.473785+0000 (oldest deadline 2024-05-08T22:20:06.773867+0000) 2024-05-08T22:20:07.978 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:07 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:07.673+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:07.978 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:07 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:07.673+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:07.978 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:07 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:07.673+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:07.978 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:07 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:07.673+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:07.978 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:07 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:07.673+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:43.000435+0000 front 2024-05-08T22:19:43.000460+0000 (oldest deadline 2024-05-08T22:20:07.100597+0000) 2024-05-08T22:20:07.985 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.7.service' 2024-05-08T22:20:08.024 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:20:08.024 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-08T22:20:08.025 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-08T22:20:08.025 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.8 2024-05-08T22:20:08.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:07 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:07.690+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:20:08.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:07 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:07.690+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.361459+0000 front 2024-05-08T22:19:23.361491+0000 (oldest deadline 2024-05-08T22:19:47.461727+0000) 2024-05-08T22:20:08.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:07 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:07.690+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:33.263559+0000 front 2024-05-08T22:19:33.263549+0000 (oldest deadline 2024-05-08T22:19:57.963489+0000) 2024-05-08T22:20:08.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:07 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:07.690+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:41.464828+0000 front 2024-05-08T22:19:41.464948+0000 (oldest deadline 2024-05-08T22:20:04.964705+0000) 2024-05-08T22:20:08.311 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:08 smithi157 systemd[1]: Stopping Ceph osd.8 for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:20:08.311 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:08 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:08.259+0000 7f0cd1504700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:20:08.311 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:08 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:08.259+0000 7f0cd1504700 -1 osd.8 102 *** Got signal Terminated *** 2024-05-08T22:20:08.311 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:08 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:08.259+0000 7f0cd1504700 -1 osd.8 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:20:08.738 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:08 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:08.310+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:08.739 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:08 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:08.310+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:08.739 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:08 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:08.310+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:08.739 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:08 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:08.310+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:08.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:08 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:08.596+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:20:08.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:08 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:08.596+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:20:08.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:08 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:08.596+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:20:08.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:08 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:08.596+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.371986+0000 front 2024-05-08T22:19:30.372008+0000 (oldest deadline 2024-05-08T22:19:56.272139+0000) 2024-05-08T22:20:08.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:08 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:08.596+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:41.473774+0000 front 2024-05-08T22:19:41.473785+0000 (oldest deadline 2024-05-08T22:20:06.773867+0000) 2024-05-08T22:20:09.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:08 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:08.737+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:20:09.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:08 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:08.737+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.361459+0000 front 2024-05-08T22:19:23.361491+0000 (oldest deadline 2024-05-08T22:19:47.461727+0000) 2024-05-08T22:20:09.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:08 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:08.737+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:33.263559+0000 front 2024-05-08T22:19:33.263549+0000 (oldest deadline 2024-05-08T22:19:57.963489+0000) 2024-05-08T22:20:09.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:08 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:08.737+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:41.464828+0000 front 2024-05-08T22:19:41.464948+0000 (oldest deadline 2024-05-08T22:20:04.964705+0000) 2024-05-08T22:20:09.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:08 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:08.704+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:09.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:08 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:08.704+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:09.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:08 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:08.704+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:09.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:08 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:08.704+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:09.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:08 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:08.704+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:43.000435+0000 front 2024-05-08T22:19:43.000460+0000 (oldest deadline 2024-05-08T22:20:07.100597+0000) 2024-05-08T22:20:09.762 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:09 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:09.280+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:09.762 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:09 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:09.280+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:09.762 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:09 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:09.280+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:09.762 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:09 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:09.280+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:09.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:09 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:09.634+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:20:09.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:09 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:09.634+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:20:09.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:09 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:09.634+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:20:09.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:09 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:09.634+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.371986+0000 front 2024-05-08T22:19:30.372008+0000 (oldest deadline 2024-05-08T22:19:56.272139+0000) 2024-05-08T22:20:09.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:09 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:09.634+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:41.473774+0000 front 2024-05-08T22:19:41.473785+0000 (oldest deadline 2024-05-08T22:20:06.773867+0000) 2024-05-08T22:20:10.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:09 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:09.761+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:20:10.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:09 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:09.761+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.361459+0000 front 2024-05-08T22:19:23.361491+0000 (oldest deadline 2024-05-08T22:19:47.461727+0000) 2024-05-08T22:20:10.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:09 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:09.761+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:33.263559+0000 front 2024-05-08T22:19:33.263549+0000 (oldest deadline 2024-05-08T22:19:57.963489+0000) 2024-05-08T22:20:10.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:09 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:09.761+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:41.464828+0000 front 2024-05-08T22:19:41.464948+0000 (oldest deadline 2024-05-08T22:20:04.964705+0000) 2024-05-08T22:20:10.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:09 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:09.738+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:10.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:09 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:09.738+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:10.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:09 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:09.738+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:10.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:09 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:09.738+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:10.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:09 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:09.738+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:43.000435+0000 front 2024-05-08T22:19:43.000460+0000 (oldest deadline 2024-05-08T22:20:07.100597+0000) 2024-05-08T22:20:10.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:10 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:10.261+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:10.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:10 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:10.261+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:10.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:10 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:10.261+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:10.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:10 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:10.261+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:11.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:10 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:10.735+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:20:11.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:10 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:10.735+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.361459+0000 front 2024-05-08T22:19:23.361491+0000 (oldest deadline 2024-05-08T22:19:47.461727+0000) 2024-05-08T22:20:11.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:10 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:10.735+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:33.263559+0000 front 2024-05-08T22:19:33.263549+0000 (oldest deadline 2024-05-08T22:19:57.963489+0000) 2024-05-08T22:20:11.026 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:10 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:10.735+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:41.464828+0000 front 2024-05-08T22:19:41.464948+0000 (oldest deadline 2024-05-08T22:20:04.964705+0000) 2024-05-08T22:20:11.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:10 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:10.768+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:11.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:10 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:10.768+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:11.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:10 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:10.768+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:11.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:10 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:10.768+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:11.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:10 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:10.768+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:43.000435+0000 front 2024-05-08T22:19:43.000460+0000 (oldest deadline 2024-05-08T22:20:07.100597+0000) 2024-05-08T22:20:11.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:10 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:10.667+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:20:11.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:10 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:10.667+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:20:11.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:10 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:10.667+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:20:11.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:10 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:10.667+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.371986+0000 front 2024-05-08T22:19:30.372008+0000 (oldest deadline 2024-05-08T22:19:56.272139+0000) 2024-05-08T22:20:11.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:10 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:10.667+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:41.473774+0000 front 2024-05-08T22:19:41.473785+0000 (oldest deadline 2024-05-08T22:20:06.773867+0000) 2024-05-08T22:20:11.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:11 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:11.247+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:11.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:11 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:11.247+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:11.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:11 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:11.247+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:11.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:11 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:11.247+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:12.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:11 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:11.758+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:12.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:11 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:11.758+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:12.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:11 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:11.758+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:12.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:11 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:11.758+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:12.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:11 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:11.758+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:43.000435+0000 front 2024-05-08T22:19:43.000460+0000 (oldest deadline 2024-05-08T22:20:07.100597+0000) 2024-05-08T22:20:12.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:11 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:11.695+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:20:12.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:11 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:11.695+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:20:12.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:11 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:11.695+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:20:12.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:11 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:11.695+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.371986+0000 front 2024-05-08T22:19:30.372008+0000 (oldest deadline 2024-05-08T22:19:56.272139+0000) 2024-05-08T22:20:12.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:11 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:11.695+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:41.473774+0000 front 2024-05-08T22:19:41.473785+0000 (oldest deadline 2024-05-08T22:20:06.773867+0000) 2024-05-08T22:20:12.200 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:11 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:11.783+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:20:12.200 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:11 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:11.783+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.361459+0000 front 2024-05-08T22:19:23.361491+0000 (oldest deadline 2024-05-08T22:19:47.461727+0000) 2024-05-08T22:20:12.201 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:11 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:11.783+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:33.263559+0000 front 2024-05-08T22:19:33.263549+0000 (oldest deadline 2024-05-08T22:19:57.963489+0000) 2024-05-08T22:20:12.201 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:11 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:11.783+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:41.464828+0000 front 2024-05-08T22:19:41.464948+0000 (oldest deadline 2024-05-08T22:20:04.964705+0000) 2024-05-08T22:20:12.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:12 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:12.200+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:12.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:12 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:12.200+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:12.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:12 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:12.200+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:12.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:12 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:12.200+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:13.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:12 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:12.773+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:13.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:12 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:12.773+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:13.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:12 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:12.773+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:13.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:12 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:12.773+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:13.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:12 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:12.773+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:43.000435+0000 front 2024-05-08T22:19:43.000460+0000 (oldest deadline 2024-05-08T22:20:07.100597+0000) 2024-05-08T22:20:13.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:12 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:12.675+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:20:13.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:12 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:12.675+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:20:13.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:12 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:12.675+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:20:13.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:12 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:12.675+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.371986+0000 front 2024-05-08T22:19:30.372008+0000 (oldest deadline 2024-05-08T22:19:56.272139+0000) 2024-05-08T22:20:13.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:12 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:12.675+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:41.473774+0000 front 2024-05-08T22:19:41.473785+0000 (oldest deadline 2024-05-08T22:20:06.773867+0000) 2024-05-08T22:20:13.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:12 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:12.675+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.075206+0000 front 2024-05-08T22:19:49.075344+0000 (oldest deadline 2024-05-08T22:20:11.875662+0000) 2024-05-08T22:20:13.175 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:12 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:12.833+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.058559+0000 front 2024-05-08T22:19:10.058511+0000 (oldest deadline 2024-05-08T22:19:33.558403+0000) 2024-05-08T22:20:13.176 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:12 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:12.833+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.361459+0000 front 2024-05-08T22:19:23.361491+0000 (oldest deadline 2024-05-08T22:19:47.461727+0000) 2024-05-08T22:20:13.176 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:12 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:12.833+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:33.263559+0000 front 2024-05-08T22:19:33.263549+0000 (oldest deadline 2024-05-08T22:19:57.963489+0000) 2024-05-08T22:20:13.176 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:12 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:12.833+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:41.464828+0000 front 2024-05-08T22:19:41.464948+0000 (oldest deadline 2024-05-08T22:20:04.964705+0000) 2024-05-08T22:20:13.176 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:12 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8[100160]: 2024-05-08T22:20:12.833+0000 7f0ccdafa700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:48.465762+0000 front 2024-05-08T22:19:48.465741+0000 (oldest deadline 2024-05-08T22:20:12.565781+0000) 2024-05-08T22:20:13.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:13 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:13.174+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:13.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:13 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:13.174+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:13.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:13 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:13.174+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:13.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:13 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:13.174+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:13.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:13 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:13.174+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.591380+0000 front 2024-05-08T22:19:49.591408+0000 (oldest deadline 2024-05-08T22:20:13.091544+0000) 2024-05-08T22:20:13.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:13 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:13.626+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:20:13.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:13 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:13.626+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:20:13.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:13 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:13.626+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:20:13.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:13 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:13.626+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.371986+0000 front 2024-05-08T22:19:30.372008+0000 (oldest deadline 2024-05-08T22:19:56.272139+0000) 2024-05-08T22:20:13.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:13 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:13.626+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:41.473774+0000 front 2024-05-08T22:19:41.473785+0000 (oldest deadline 2024-05-08T22:20:06.773867+0000) 2024-05-08T22:20:13.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:13 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:13.626+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.075206+0000 front 2024-05-08T22:19:49.075344+0000 (oldest deadline 2024-05-08T22:20:11.875662+0000) 2024-05-08T22:20:14.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:13 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:13.767+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:14.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:13 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:13.767+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:14.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:13 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:13.767+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:14.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:13 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:13.767+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:14.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:13 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:13.767+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:43.000435+0000 front 2024-05-08T22:19:43.000460+0000 (oldest deadline 2024-05-08T22:20:07.100597+0000) 2024-05-08T22:20:14.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:13 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:13.767+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:47.101441+0000 front 2024-05-08T22:19:47.101419+0000 (oldest deadline 2024-05-08T22:20:12.901707+0000) 2024-05-08T22:20:14.276 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 08 22:20:14 smithi157 bash[122362]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-8 2024-05-08T22:20:14.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:14 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:14.163+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:14.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:14 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:14.163+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:14.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:14 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:14.163+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:14.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:14 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:14.163+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:14.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:14 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:14.163+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.591380+0000 front 2024-05-08T22:19:49.591408+0000 (oldest deadline 2024-05-08T22:20:13.091544+0000) 2024-05-08T22:20:15.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:14 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:14.790+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:15.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:14 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:14.790+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:15.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:14 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:14.790+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:15.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:14 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:14.790+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:15.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:14 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:14.790+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:43.000435+0000 front 2024-05-08T22:19:43.000460+0000 (oldest deadline 2024-05-08T22:20:07.100597+0000) 2024-05-08T22:20:15.091 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:14 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:14.790+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:47.101441+0000 front 2024-05-08T22:19:47.101419+0000 (oldest deadline 2024-05-08T22:20:12.901707+0000) 2024-05-08T22:20:15.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:14 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:14.657+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:20:15.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:14 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:14.657+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:20:15.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:14 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:14.657+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:20:15.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:14 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:14.657+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.371986+0000 front 2024-05-08T22:19:30.372008+0000 (oldest deadline 2024-05-08T22:19:56.272139+0000) 2024-05-08T22:20:15.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:14 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:14.657+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:41.473774+0000 front 2024-05-08T22:19:41.473785+0000 (oldest deadline 2024-05-08T22:20:06.773867+0000) 2024-05-08T22:20:15.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:14 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:14.657+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.075206+0000 front 2024-05-08T22:19:49.075344+0000 (oldest deadline 2024-05-08T22:20:11.875662+0000) 2024-05-08T22:20:15.149 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:15 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:15.147+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:15.149 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:15 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:15.147+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:15.230 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.8.service' 2024-05-08T22:20:15.270 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:20:15.270 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-08T22:20:15.271 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-08T22:20:15.271 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.9 2024-05-08T22:20:15.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:15 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:15.147+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:15.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:15 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:15.147+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:15.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:15 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:15.147+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.591380+0000 front 2024-05-08T22:19:49.591408+0000 (oldest deadline 2024-05-08T22:20:13.091544+0000) 2024-05-08T22:20:15.641 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:15 smithi131 systemd[1]: Stopping Ceph osd.9 for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:20:16.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:15 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:15.754+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:16.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:15 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:15.754+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:16.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:15 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:15.754+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:16.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:15 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:15.754+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:16.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:15 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:15.754+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:43.000435+0000 front 2024-05-08T22:19:43.000460+0000 (oldest deadline 2024-05-08T22:20:07.100597+0000) 2024-05-08T22:20:16.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:15 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:15.754+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:47.101441+0000 front 2024-05-08T22:19:47.101419+0000 (oldest deadline 2024-05-08T22:20:12.901707+0000) 2024-05-08T22:20:16.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:15 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:15.656+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:20:16.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:15 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:15.656+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:20:16.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:15 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:15.656+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:20:16.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:15 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:15.656+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.371986+0000 front 2024-05-08T22:19:30.372008+0000 (oldest deadline 2024-05-08T22:19:56.272139+0000) 2024-05-08T22:20:16.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:15 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:15.656+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:41.473774+0000 front 2024-05-08T22:19:41.473785+0000 (oldest deadline 2024-05-08T22:20:06.773867+0000) 2024-05-08T22:20:16.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:15 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:15.656+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.075206+0000 front 2024-05-08T22:19:49.075344+0000 (oldest deadline 2024-05-08T22:20:11.875662+0000) 2024-05-08T22:20:16.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:15 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:15.694+0000 7f4000e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:20:16.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:15 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:15.694+0000 7f4000e00700 -1 osd.9 102 *** Got signal Terminated *** 2024-05-08T22:20:16.142 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:15 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:15.694+0000 7f4000e00700 -1 osd.9 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:20:16.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:16 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:16.196+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:16.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:16 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:16.196+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:16.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:16 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:16.196+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:16.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:16 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:16.196+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:16.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:16 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:16.196+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.591380+0000 front 2024-05-08T22:19:49.591408+0000 (oldest deadline 2024-05-08T22:20:13.091544+0000) 2024-05-08T22:20:17.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:16 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:16.722+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:17.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:16 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:16.722+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:17.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:16 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:16.722+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:17.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:16 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:16.722+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:17.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:16 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:16.722+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:43.000435+0000 front 2024-05-08T22:19:43.000460+0000 (oldest deadline 2024-05-08T22:20:07.100597+0000) 2024-05-08T22:20:17.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:16 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:16.722+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:47.101441+0000 front 2024-05-08T22:19:47.101419+0000 (oldest deadline 2024-05-08T22:20:12.901707+0000) 2024-05-08T22:20:17.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:16 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:16.700+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:20:17.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:16 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:16.700+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:20:17.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:16 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:16.700+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:20:17.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:16 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:16.700+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.371986+0000 front 2024-05-08T22:19:30.372008+0000 (oldest deadline 2024-05-08T22:19:56.272139+0000) 2024-05-08T22:20:17.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:16 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:16.700+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:41.473774+0000 front 2024-05-08T22:19:41.473785+0000 (oldest deadline 2024-05-08T22:20:06.773867+0000) 2024-05-08T22:20:17.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:16 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:16.700+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.075206+0000 front 2024-05-08T22:19:49.075344+0000 (oldest deadline 2024-05-08T22:20:11.875662+0000) 2024-05-08T22:20:17.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:17 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:17.195+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:17.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:17 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:17.195+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:17.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:17 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:17.195+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:17.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:17 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:17.195+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:17.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:17 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:17.195+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.591380+0000 front 2024-05-08T22:19:49.591408+0000 (oldest deadline 2024-05-08T22:20:13.091544+0000) 2024-05-08T22:20:18.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:17 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:17.700+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:18.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:17 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:17.700+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:18.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:17 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:17.700+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:18.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:17 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:17.700+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:18.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:17 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:17.700+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:43.000435+0000 front 2024-05-08T22:19:43.000460+0000 (oldest deadline 2024-05-08T22:20:07.100597+0000) 2024-05-08T22:20:18.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:17 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:17.700+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:47.101441+0000 front 2024-05-08T22:19:47.101419+0000 (oldest deadline 2024-05-08T22:20:12.901707+0000) 2024-05-08T22:20:18.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:17 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:17.716+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:20:18.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:17 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:17.716+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:20:18.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:17 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:17.716+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:20:18.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:17 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:17.716+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.371986+0000 front 2024-05-08T22:19:30.372008+0000 (oldest deadline 2024-05-08T22:19:56.272139+0000) 2024-05-08T22:20:18.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:17 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:17.716+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:41.473774+0000 front 2024-05-08T22:19:41.473785+0000 (oldest deadline 2024-05-08T22:20:06.773867+0000) 2024-05-08T22:20:18.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:17 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:17.716+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.075206+0000 front 2024-05-08T22:19:49.075344+0000 (oldest deadline 2024-05-08T22:20:11.875662+0000) 2024-05-08T22:20:18.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:18 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:18.219+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:18.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:18 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:18.219+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:18.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:18 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:18.219+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:18.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:18 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:18.219+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:18.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:18 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:18.219+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.591380+0000 front 2024-05-08T22:19:49.591408+0000 (oldest deadline 2024-05-08T22:20:13.091544+0000) 2024-05-08T22:20:19.091 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:18 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:18.697+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:19.091 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:18 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:18.697+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:19.091 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:18 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:18.697+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:19.091 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:18 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:18.697+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:19.091 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:18 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:18.697+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:43.000435+0000 front 2024-05-08T22:19:43.000460+0000 (oldest deadline 2024-05-08T22:20:07.100597+0000) 2024-05-08T22:20:19.092 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:18 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:18.697+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:47.101441+0000 front 2024-05-08T22:19:47.101419+0000 (oldest deadline 2024-05-08T22:20:12.901707+0000) 2024-05-08T22:20:19.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:18 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:18.704+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:20:19.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:18 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:18.704+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:20:19.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:18 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:18.704+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:20:19.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:18 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:18.704+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.371986+0000 front 2024-05-08T22:19:30.372008+0000 (oldest deadline 2024-05-08T22:19:56.272139+0000) 2024-05-08T22:20:19.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:18 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:18.704+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:41.473774+0000 front 2024-05-08T22:19:41.473785+0000 (oldest deadline 2024-05-08T22:20:06.773867+0000) 2024-05-08T22:20:19.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:18 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:18.704+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.075206+0000 front 2024-05-08T22:19:49.075344+0000 (oldest deadline 2024-05-08T22:20:11.875662+0000) 2024-05-08T22:20:19.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:19 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:19.213+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:19.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:19 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:19.213+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:19.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:19 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:19.213+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:19.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:19 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:19.213+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:19.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:19 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:19.213+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.591380+0000 front 2024-05-08T22:19:49.591408+0000 (oldest deadline 2024-05-08T22:20:13.091544+0000) 2024-05-08T22:20:20.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:19 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:19.729+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:20.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:19 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:19.729+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:20.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:19 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:19.729+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:20.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:19 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:19.729+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:20.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:19 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:19.729+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:43.000435+0000 front 2024-05-08T22:19:43.000460+0000 (oldest deadline 2024-05-08T22:20:07.100597+0000) 2024-05-08T22:20:20.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:19 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:19.729+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:47.101441+0000 front 2024-05-08T22:19:47.101419+0000 (oldest deadline 2024-05-08T22:20:12.901707+0000) 2024-05-08T22:20:20.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:19.680+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:20:20.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:19.680+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:20:20.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:19.680+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:20:20.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:19.680+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.371986+0000 front 2024-05-08T22:19:30.372008+0000 (oldest deadline 2024-05-08T22:19:56.272139+0000) 2024-05-08T22:20:20.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:19.680+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:41.473774+0000 front 2024-05-08T22:19:41.473785+0000 (oldest deadline 2024-05-08T22:20:06.773867+0000) 2024-05-08T22:20:20.141 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:19 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:19.680+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.075206+0000 front 2024-05-08T22:19:49.075344+0000 (oldest deadline 2024-05-08T22:20:11.875662+0000) 2024-05-08T22:20:20.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:20 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:20.180+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:20.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:20 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:20.180+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:20.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:20 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:20.180+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:20.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:20 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:20.180+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:20.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:20 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:20.180+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.591380+0000 front 2024-05-08T22:19:49.591408+0000 (oldest deadline 2024-05-08T22:20:13.091544+0000) 2024-05-08T22:20:20.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:20 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:20.180+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-08T22:19:55.392640+0000 front 2024-05-08T22:19:55.392688+0000 (oldest deadline 2024-05-08T22:20:19.492575+0000) 2024-05-08T22:20:20.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:20 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:20.640+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:08.267444+0000 front 2024-05-08T22:19:08.267458+0000 (oldest deadline 2024-05-08T22:19:35.268562+0000) 2024-05-08T22:20:20.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:20 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:20.640+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.670875+0000 front 2024-05-08T22:19:18.670855+0000 (oldest deadline 2024-05-08T22:19:43.970620+0000) 2024-05-08T22:20:20.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:20 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:20.640+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:23.971405+0000 front 2024-05-08T22:19:23.971458+0000 (oldest deadline 2024-05-08T22:19:49.271116+0000) 2024-05-08T22:20:20.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:20 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:20.640+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:30.371986+0000 front 2024-05-08T22:19:30.372008+0000 (oldest deadline 2024-05-08T22:19:56.272139+0000) 2024-05-08T22:20:20.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:20 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:20.640+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:41.473774+0000 front 2024-05-08T22:19:41.473785+0000 (oldest deadline 2024-05-08T22:20:06.773867+0000) 2024-05-08T22:20:20.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:20 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9[79929]: 2024-05-08T22:20:20.640+0000 7f3ffc600700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.075206+0000 front 2024-05-08T22:19:49.075344+0000 (oldest deadline 2024-05-08T22:20:11.875662+0000) 2024-05-08T22:20:21.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:20 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:20.695+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:21.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:20 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:20.695+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:21.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:20 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:20.695+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:21.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:20 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:20.695+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:21.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:20 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:20.695+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:43.000435+0000 front 2024-05-08T22:19:43.000460+0000 (oldest deadline 2024-05-08T22:20:07.100597+0000) 2024-05-08T22:20:21.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:20 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:20.695+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:47.101441+0000 front 2024-05-08T22:19:47.101419+0000 (oldest deadline 2024-05-08T22:20:12.901707+0000) 2024-05-08T22:20:21.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:20 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:20.695+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-08T22:19:58.202947+0000 front 2024-05-08T22:19:58.202994+0000 (oldest deadline 2024-05-08T22:20:19.902689+0000) 2024-05-08T22:20:21.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:21 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:21.137+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:21.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:21 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:21.137+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:21.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:21 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:21.137+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:21.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:21 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:21.137+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:21.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:21 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:21.137+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.591380+0000 front 2024-05-08T22:19:49.591408+0000 (oldest deadline 2024-05-08T22:20:13.091544+0000) 2024-05-08T22:20:21.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:21 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:21.137+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-08T22:19:55.392640+0000 front 2024-05-08T22:19:55.392688+0000 (oldest deadline 2024-05-08T22:20:19.492575+0000) 2024-05-08T22:20:21.891 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 08 22:20:21 smithi131 bash[120346]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-9 2024-05-08T22:20:22.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:21 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:21.703+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:22.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:21 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:21.703+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:22.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:21 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:21.703+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:22.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:21 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:21.704+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:22.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:21 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:21.704+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:43.000435+0000 front 2024-05-08T22:19:43.000460+0000 (oldest deadline 2024-05-08T22:20:07.100597+0000) 2024-05-08T22:20:22.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:21 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:21.704+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:47.101441+0000 front 2024-05-08T22:19:47.101419+0000 (oldest deadline 2024-05-08T22:20:12.901707+0000) 2024-05-08T22:20:22.091 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:21 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:21.704+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-08T22:19:58.202947+0000 front 2024-05-08T22:19:58.202994+0000 (oldest deadline 2024-05-08T22:20:19.902689+0000) 2024-05-08T22:20:22.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:22 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:22.134+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:22.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:22 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:22.134+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:22.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:22 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:22.134+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:22.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:22 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:22.134+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:22.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:22 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:22.134+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.591380+0000 front 2024-05-08T22:19:49.591408+0000 (oldest deadline 2024-05-08T22:20:13.091544+0000) 2024-05-08T22:20:22.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:22 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:22.134+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-08T22:19:55.392640+0000 front 2024-05-08T22:19:55.392688+0000 (oldest deadline 2024-05-08T22:20:19.492575+0000) 2024-05-08T22:20:23.043 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.9.service' 2024-05-08T22:20:23.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:22 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:22.656+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:23.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:22 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:22.656+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:23.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:22 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:22.656+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:23.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:22 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:22.656+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:23.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:22 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:22.656+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:43.000435+0000 front 2024-05-08T22:19:43.000460+0000 (oldest deadline 2024-05-08T22:20:07.100597+0000) 2024-05-08T22:20:23.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:22 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:22.656+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:47.101441+0000 front 2024-05-08T22:19:47.101419+0000 (oldest deadline 2024-05-08T22:20:12.901707+0000) 2024-05-08T22:20:23.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:22 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:22.656+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-08T22:19:58.202947+0000 front 2024-05-08T22:19:58.202994+0000 (oldest deadline 2024-05-08T22:20:19.902689+0000) 2024-05-08T22:20:23.131 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:20:23.131 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-08T22:20:23.131 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-08T22:20:23.131 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.10 2024-05-08T22:20:23.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:23 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:23.098+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:23.527 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:23 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:23.098+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:23.527 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:23 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:23.098+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:23.527 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:23 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:23.098+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:23.527 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:23 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:23.098+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.591380+0000 front 2024-05-08T22:19:49.591408+0000 (oldest deadline 2024-05-08T22:20:13.091544+0000) 2024-05-08T22:20:23.527 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:23 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:23.098+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-08T22:19:55.392640+0000 front 2024-05-08T22:19:55.392688+0000 (oldest deadline 2024-05-08T22:20:19.492575+0000) 2024-05-08T22:20:23.590 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:23 smithi132 systemd[1]: Stopping Ceph osd.10 for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:20:23.590 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:23 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:23.384+0000 7f02edfa0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:20:23.590 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:23 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:23.384+0000 7f02edfa0700 -1 osd.10 102 *** Got signal Terminated *** 2024-05-08T22:20:23.590 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:23 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:23.384+0000 7f02edfa0700 -1 osd.10 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:20:24.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:23 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:23.616+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:24.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:23 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:23.616+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:24.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:23 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:23.616+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:24.091 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:23 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:23.616+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:24.091 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:23 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:23.616+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:43.000435+0000 front 2024-05-08T22:19:43.000460+0000 (oldest deadline 2024-05-08T22:20:07.100597+0000) 2024-05-08T22:20:24.091 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:23 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:23.616+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:47.101441+0000 front 2024-05-08T22:19:47.101419+0000 (oldest deadline 2024-05-08T22:20:12.901707+0000) 2024-05-08T22:20:24.091 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:23 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:23.616+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-08T22:19:58.202947+0000 front 2024-05-08T22:19:58.202994+0000 (oldest deadline 2024-05-08T22:20:19.902689+0000) 2024-05-08T22:20:24.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:24 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:24.053+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:24.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:24 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:24.053+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:24.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:24 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:24.053+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:24.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:24 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:24.053+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:24.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:24 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:24.053+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.591380+0000 front 2024-05-08T22:19:49.591408+0000 (oldest deadline 2024-05-08T22:20:13.091544+0000) 2024-05-08T22:20:24.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:24 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:24.053+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-08T22:19:55.392640+0000 front 2024-05-08T22:19:55.392688+0000 (oldest deadline 2024-05-08T22:20:19.492575+0000) 2024-05-08T22:20:25.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:24 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:24.661+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:25.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:24 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:24.661+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:25.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:24 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:24.661+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:25.091 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:24 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:24.661+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:25.091 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:24 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:24.661+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:43.000435+0000 front 2024-05-08T22:19:43.000460+0000 (oldest deadline 2024-05-08T22:20:07.100597+0000) 2024-05-08T22:20:25.091 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:24 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:24.661+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:47.101441+0000 front 2024-05-08T22:19:47.101419+0000 (oldest deadline 2024-05-08T22:20:12.901707+0000) 2024-05-08T22:20:25.091 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:24 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:24.661+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-08T22:19:58.202947+0000 front 2024-05-08T22:19:58.202994+0000 (oldest deadline 2024-05-08T22:20:19.902689+0000) 2024-05-08T22:20:25.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:25 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:25.013+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:25.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:25 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:25.013+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:25.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:25 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:25.013+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:25.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:25 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:25.013+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:25.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:25 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:25.013+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.591380+0000 front 2024-05-08T22:19:49.591408+0000 (oldest deadline 2024-05-08T22:20:13.091544+0000) 2024-05-08T22:20:25.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:25 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:25.013+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-08T22:19:55.392640+0000 front 2024-05-08T22:19:55.392688+0000 (oldest deadline 2024-05-08T22:20:19.492575+0000) 2024-05-08T22:20:26.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:25 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:25.614+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:26.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:25 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:25.614+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:26.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:25 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:25.614+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:26.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:25 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:25.614+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:26.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:25 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:25.614+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:43.000435+0000 front 2024-05-08T22:19:43.000460+0000 (oldest deadline 2024-05-08T22:20:07.100597+0000) 2024-05-08T22:20:26.091 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:25 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:25.614+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:47.101441+0000 front 2024-05-08T22:19:47.101419+0000 (oldest deadline 2024-05-08T22:20:12.901707+0000) 2024-05-08T22:20:26.091 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:25 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:25.614+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-08T22:19:58.202947+0000 front 2024-05-08T22:19:58.202994+0000 (oldest deadline 2024-05-08T22:20:19.902689+0000) 2024-05-08T22:20:26.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:26 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:26.007+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:26.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:26 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:26.007+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:26.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:26 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:26.007+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:26.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:26 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:26.007+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:26.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:26 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:26.007+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.591380+0000 front 2024-05-08T22:19:49.591408+0000 (oldest deadline 2024-05-08T22:20:13.091544+0000) 2024-05-08T22:20:26.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:26 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:26.007+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-08T22:19:55.392640+0000 front 2024-05-08T22:19:55.392688+0000 (oldest deadline 2024-05-08T22:20:19.492575+0000) 2024-05-08T22:20:27.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:26 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:26.631+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:27.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:26 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:26.631+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:27.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:26 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:26.631+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:27.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:26 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:26.631+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:27.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:26 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:26.631+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:43.000435+0000 front 2024-05-08T22:19:43.000460+0000 (oldest deadline 2024-05-08T22:20:07.100597+0000) 2024-05-08T22:20:27.091 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:26 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:26.631+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:47.101441+0000 front 2024-05-08T22:19:47.101419+0000 (oldest deadline 2024-05-08T22:20:12.901707+0000) 2024-05-08T22:20:27.091 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:26 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:26.631+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-08T22:19:58.202947+0000 front 2024-05-08T22:19:58.202994+0000 (oldest deadline 2024-05-08T22:20:19.902689+0000) 2024-05-08T22:20:27.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:26 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:26.965+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:27.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:26 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:26.965+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:27.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:26 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:26.965+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:27.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:26 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:26.965+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:27.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:26 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:26.965+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.591380+0000 front 2024-05-08T22:19:49.591408+0000 (oldest deadline 2024-05-08T22:20:13.091544+0000) 2024-05-08T22:20:27.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:26 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:26.965+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-08T22:19:55.392640+0000 front 2024-05-08T22:19:55.392688+0000 (oldest deadline 2024-05-08T22:20:19.492575+0000) 2024-05-08T22:20:28.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:27 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:27.625+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:10.494058+0000 front 2024-05-08T22:19:10.494142+0000 (oldest deadline 2024-05-08T22:19:33.994290+0000) 2024-05-08T22:20:28.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:27 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:27.625+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:19.096520+0000 front 2024-05-08T22:19:19.096543+0000 (oldest deadline 2024-05-08T22:19:40.196631+0000) 2024-05-08T22:20:28.090 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:27 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:27.625+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.897799+0000 front 2024-05-08T22:19:24.897768+0000 (oldest deadline 2024-05-08T22:19:48.397796+0000) 2024-05-08T22:20:28.091 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:27 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:27.625+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-08T22:19:31.299008+0000 front 2024-05-08T22:19:31.298972+0000 (oldest deadline 2024-05-08T22:19:56.598913+0000) 2024-05-08T22:20:28.091 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:27 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:27.625+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:43.000435+0000 front 2024-05-08T22:19:43.000460+0000 (oldest deadline 2024-05-08T22:20:07.100597+0000) 2024-05-08T22:20:28.091 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:27 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:27.625+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:47.101441+0000 front 2024-05-08T22:19:47.101419+0000 (oldest deadline 2024-05-08T22:20:12.901707+0000) 2024-05-08T22:20:28.091 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:27 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10[108465]: 2024-05-08T22:20:27.625+0000 7f02e658e700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-08T22:19:58.202947+0000 front 2024-05-08T22:19:58.202994+0000 (oldest deadline 2024-05-08T22:20:19.902689+0000) 2024-05-08T22:20:28.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:27 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:27.939+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:28.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:27 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:27.939+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:28.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:27 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:27.939+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:28.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:27 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:27.939+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:28.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:27 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:27.939+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.591380+0000 front 2024-05-08T22:19:49.591408+0000 (oldest deadline 2024-05-08T22:20:13.091544+0000) 2024-05-08T22:20:28.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:27 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:27.939+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-08T22:19:55.392640+0000 front 2024-05-08T22:19:55.392688+0000 (oldest deadline 2024-05-08T22:20:19.492575+0000) 2024-05-08T22:20:29.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:28 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:28.916+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:29.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:28 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:28.916+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:29.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:28 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:28.916+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:29.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:28 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:28.916+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:29.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:28 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:28.916+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.591380+0000 front 2024-05-08T22:19:49.591408+0000 (oldest deadline 2024-05-08T22:20:13.091544+0000) 2024-05-08T22:20:29.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:28 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:28.916+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-08T22:19:55.392640+0000 front 2024-05-08T22:19:55.392688+0000 (oldest deadline 2024-05-08T22:20:19.492575+0000) 2024-05-08T22:20:29.340 INFO:journalctl@ceph.osd.10.smithi132.stdout:May 08 22:20:29 smithi132 bash[121437]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-10 2024-05-08T22:20:30.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:29 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:29.877+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:30.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:29 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:29.877+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:30.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:29 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:29.877+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:30.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:29 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:29.877+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:30.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:29 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:29.877+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.591380+0000 front 2024-05-08T22:19:49.591408+0000 (oldest deadline 2024-05-08T22:20:13.091544+0000) 2024-05-08T22:20:30.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:29 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:29.877+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-08T22:19:55.392640+0000 front 2024-05-08T22:19:55.392688+0000 (oldest deadline 2024-05-08T22:20:19.492575+0000) 2024-05-08T22:20:30.347 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.10.service' 2024-05-08T22:20:30.388 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:20:30.388 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-08T22:20:30.388 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-08T22:20:30.388 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.11 2024-05-08T22:20:30.776 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:30 smithi157 systemd[1]: Stopping Ceph osd.11 for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:20:30.776 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:30 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:30.613+0000 7f77b0b4e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:20:30.776 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:30 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:30.613+0000 7f77b0b4e700 -1 osd.11 102 *** Got signal Terminated *** 2024-05-08T22:20:30.776 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:30 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:30.613+0000 7f77b0b4e700 -1 osd.11 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:20:31.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:30 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:30.892+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:31.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:30 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:30.892+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:31.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:30 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:30.892+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:31.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:30 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:30.892+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:31.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:30 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:30.892+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.591380+0000 front 2024-05-08T22:19:49.591408+0000 (oldest deadline 2024-05-08T22:20:13.091544+0000) 2024-05-08T22:20:31.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:30 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:30.892+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-08T22:19:55.392640+0000 front 2024-05-08T22:19:55.392688+0000 (oldest deadline 2024-05-08T22:20:19.492575+0000) 2024-05-08T22:20:31.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:30 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:30.892+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2024-05-08T22:20:02.393490+0000 front 2024-05-08T22:20:02.393463+0000 (oldest deadline 2024-05-08T22:20:30.593897+0000) 2024-05-08T22:20:32.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:31 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:31.933+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:32.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:31 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:31.933+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:32.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:31 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:31.933+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:32.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:31 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:31.933+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:32.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:31 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:31.933+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.591380+0000 front 2024-05-08T22:19:49.591408+0000 (oldest deadline 2024-05-08T22:20:13.091544+0000) 2024-05-08T22:20:32.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:31 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:31.933+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-08T22:19:55.392640+0000 front 2024-05-08T22:19:55.392688+0000 (oldest deadline 2024-05-08T22:20:19.492575+0000) 2024-05-08T22:20:32.277 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:31 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:31.933+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2024-05-08T22:20:02.393490+0000 front 2024-05-08T22:20:02.393463+0000 (oldest deadline 2024-05-08T22:20:30.593897+0000) 2024-05-08T22:20:33.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:32 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:32.919+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:33.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:32 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:32.919+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:33.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:32 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:32.919+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:33.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:32 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:32.919+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:33.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:32 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:32.919+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.591380+0000 front 2024-05-08T22:19:49.591408+0000 (oldest deadline 2024-05-08T22:20:13.091544+0000) 2024-05-08T22:20:33.277 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:32 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:32.919+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-08T22:19:55.392640+0000 front 2024-05-08T22:19:55.392688+0000 (oldest deadline 2024-05-08T22:20:19.492575+0000) 2024-05-08T22:20:33.277 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:32 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:32.919+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2024-05-08T22:20:02.393490+0000 front 2024-05-08T22:20:02.393463+0000 (oldest deadline 2024-05-08T22:20:30.593897+0000) 2024-05-08T22:20:34.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:33 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:33.958+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:34.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:33 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:33.958+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:34.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:33 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:33.958+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:34.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:33 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:33.958+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:34.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:33 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:33.958+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.591380+0000 front 2024-05-08T22:19:49.591408+0000 (oldest deadline 2024-05-08T22:20:13.091544+0000) 2024-05-08T22:20:34.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:33 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:33.958+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-08T22:19:55.392640+0000 front 2024-05-08T22:19:55.392688+0000 (oldest deadline 2024-05-08T22:20:19.492575+0000) 2024-05-08T22:20:34.276 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:33 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:33.958+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2024-05-08T22:20:02.393490+0000 front 2024-05-08T22:20:02.393463+0000 (oldest deadline 2024-05-08T22:20:30.593897+0000) 2024-05-08T22:20:35.278 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:34 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:34.935+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-08T22:19:11.486276+0000 front 2024-05-08T22:19:11.486178+0000 (oldest deadline 2024-05-08T22:19:37.386240+0000) 2024-05-08T22:20:35.278 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:34 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:34.935+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2024-05-08T22:19:18.487332+0000 front 2024-05-08T22:19:18.487310+0000 (oldest deadline 2024-05-08T22:19:44.387412+0000) 2024-05-08T22:20:35.278 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:34 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:34.935+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-08T22:19:24.387909+0000 front 2024-05-08T22:19:24.387929+0000 (oldest deadline 2024-05-08T22:19:49.088019+0000) 2024-05-08T22:20:35.278 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:34 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:34.935+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2024-05-08T22:19:39.590314+0000 front 2024-05-08T22:19:39.590207+0000 (oldest deadline 2024-05-08T22:20:05.490316+0000) 2024-05-08T22:20:35.278 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:34 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:34.935+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-08T22:19:49.591380+0000 front 2024-05-08T22:19:49.591408+0000 (oldest deadline 2024-05-08T22:20:13.091544+0000) 2024-05-08T22:20:35.278 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:34 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:34.935+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-08T22:19:55.392640+0000 front 2024-05-08T22:19:55.392688+0000 (oldest deadline 2024-05-08T22:20:19.492575+0000) 2024-05-08T22:20:35.278 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:34 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11[108915]: 2024-05-08T22:20:34.935+0000 7f77a913c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2024-05-08T22:20:02.393490+0000 front 2024-05-08T22:20:02.393463+0000 (oldest deadline 2024-05-08T22:20:30.593897+0000) 2024-05-08T22:20:36.526 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 08 22:20:36 smithi157 bash[122618]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-osd-11 2024-05-08T22:20:37.343 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@osd.11.service' 2024-05-08T22:20:37.383 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:20:37.383 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-08T22:20:37.383 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-08T22:20:37.383 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.a 2024-05-08T22:20:37.776 INFO:journalctl@ceph.mds.a.smithi131.stdout:May 08 22:20:37 smithi131 systemd[1]: Stopping Ceph mds.a for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:20:38.142 INFO:journalctl@ceph.mds.a.smithi131.stdout:May 08 22:20:37 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-a[87641]: 2024-05-08T22:20:37.773+0000 7fbe8a800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:20:38.143 INFO:journalctl@ceph.mds.a.smithi131.stdout:May 08 22:20:37 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-a[87641]: 2024-05-08T22:20:37.773+0000 7fbe8a800700 -1 mds.a *** got signal Terminated *** 2024-05-08T22:20:39.641 INFO:journalctl@ceph.mds.a.smithi131.stdout:May 08 22:20:39 smithi131 bash[120606]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-a 2024-05-08T22:20:39.722 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.a.service' 2024-05-08T22:20:39.804 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:20:39.804 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-08T22:20:39.804 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-08T22:20:39.804 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.d 2024-05-08T22:20:40.141 INFO:journalctl@ceph.mds.d.smithi131.stdout:May 08 22:20:39 smithi131 systemd[1]: Stopping Ceph mds.d for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:20:40.641 INFO:journalctl@ceph.mds.d.smithi131.stdout:May 08 22:20:40 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-d[86681]: 2024-05-08T22:20:40.156+0000 7f1ace200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:20:40.641 INFO:journalctl@ceph.mds.d.smithi131.stdout:May 08 22:20:40 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-d[86681]: 2024-05-08T22:20:40.156+0000 7f1ace200700 -1 mds.d *** got signal Terminated *** 2024-05-08T22:20:44.641 INFO:journalctl@ceph.mds.d.smithi131.stdout:May 08 22:20:44 smithi131 bash[120746]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-d 2024-05-08T22:20:44.716 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.d.service' 2024-05-08T22:20:44.796 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:20:44.796 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-08T22:20:44.797 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-08T22:20:44.797 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.g 2024-05-08T22:20:45.141 INFO:journalctl@ceph.mds.g.smithi131.stdout:May 08 22:20:44 smithi131 systemd[1]: Stopping Ceph mds.g for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:20:45.641 INFO:journalctl@ceph.mds.g.smithi131.stdout:May 08 22:20:45 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-g[87250]: 2024-05-08T22:20:45.146+0000 7f52a0800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:20:45.641 INFO:journalctl@ceph.mds.g.smithi131.stdout:May 08 22:20:45 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-g[87250]: 2024-05-08T22:20:45.147+0000 7f52a0800700 -1 mds.g *** got signal Terminated *** 2024-05-08T22:20:47.640 INFO:journalctl@ceph.mds.g.smithi131.stdout:May 08 22:20:47 smithi131 bash[120884]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-g 2024-05-08T22:20:47.794 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.g.service' 2024-05-08T22:20:47.874 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:20:47.875 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-08T22:20:47.875 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-08T22:20:47.875 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.j 2024-05-08T22:20:48.141 INFO:journalctl@ceph.mds.j.smithi131.stdout:May 08 22:20:47 smithi131 systemd[1]: Stopping Ceph mds.j for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:20:48.640 INFO:journalctl@ceph.mds.j.smithi131.stdout:May 08 22:20:48 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-j[86291]: 2024-05-08T22:20:48.254+0000 7f6b1fa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:20:48.641 INFO:journalctl@ceph.mds.j.smithi131.stdout:May 08 22:20:48 smithi131 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-j[86291]: 2024-05-08T22:20:48.254+0000 7f6b1fa00700 -1 mds.j *** got signal Terminated *** 2024-05-08T22:20:50.140 INFO:journalctl@ceph.mds.j.smithi131.stdout:May 08 22:20:49 smithi131 bash[121022]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-j 2024-05-08T22:20:50.195 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.j.service' 2024-05-08T22:20:50.277 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:20:50.277 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-08T22:20:50.277 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-08T22:20:50.278 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.b 2024-05-08T22:20:50.590 INFO:journalctl@ceph.mds.b.smithi132.stdout:May 08 22:20:50 smithi132 systemd[1]: Stopping Ceph mds.b for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:20:51.090 INFO:journalctl@ceph.mds.b.smithi132.stdout:May 08 22:20:50 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-b[114823]: 2024-05-08T22:20:50.598+0000 7f9d989fe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:20:51.090 INFO:journalctl@ceph.mds.b.smithi132.stdout:May 08 22:20:50 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-b[114823]: 2024-05-08T22:20:50.598+0000 7f9d989fe700 -1 mds.b *** got signal Terminated *** 2024-05-08T22:20:55.082 INFO:journalctl@ceph.mds.b.smithi132.stdout:May 08 22:20:54 smithi132 bash[121701]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-b 2024-05-08T22:20:55.088 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.b.service' 2024-05-08T22:20:55.126 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:20:55.126 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-08T22:20:55.126 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-08T22:20:55.126 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.e 2024-05-08T22:20:55.340 INFO:journalctl@ceph.mds.e.smithi132.stdout:May 08 22:20:55 smithi132 systemd[1]: Stopping Ceph mds.e for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:20:55.840 INFO:journalctl@ceph.mds.e.smithi132.stdout:May 08 22:20:55 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-e[114138]: 2024-05-08T22:20:55.380+0000 7f6c28697700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:20:55.840 INFO:journalctl@ceph.mds.e.smithi132.stdout:May 08 22:20:55 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-e[114138]: 2024-05-08T22:20:55.380+0000 7f6c28697700 -1 mds.e *** got signal Terminated *** 2024-05-08T22:20:58.340 INFO:journalctl@ceph.mds.e.smithi132.stdout:May 08 22:20:58 smithi132 bash[121839]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-e 2024-05-08T22:20:58.417 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.e.service' 2024-05-08T22:20:58.451 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:20:58.451 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-08T22:20:58.451 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-08T22:20:58.452 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.h 2024-05-08T22:20:58.705 INFO:journalctl@ceph.mds.h.smithi132.stdout:May 08 22:20:58 smithi132 systemd[1]: Stopping Ceph mds.h for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:20:59.090 INFO:journalctl@ceph.mds.h.smithi132.stdout:May 08 22:20:58 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-h[114482]: 2024-05-08T22:20:58.704+0000 7fbbafd59700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:20:59.090 INFO:journalctl@ceph.mds.h.smithi132.stdout:May 08 22:20:58 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-h[114482]: 2024-05-08T22:20:58.704+0000 7fbbafd59700 -1 mds.h *** got signal Terminated *** 2024-05-08T22:21:03.223 INFO:journalctl@ceph.mds.h.smithi132.stdout:May 08 22:21:02 smithi132 bash[121978]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-h 2024-05-08T22:21:03.228 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.h.service' 2024-05-08T22:21:03.263 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:21:03.263 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-08T22:21:03.263 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-08T22:21:03.263 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.k 2024-05-08T22:21:03.521 INFO:journalctl@ceph.mds.k.smithi132.stdout:May 08 22:21:03 smithi132 systemd[1]: Stopping Ceph mds.k for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:21:03.840 INFO:journalctl@ceph.mds.k.smithi132.stdout:May 08 22:21:03 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-k[113794]: 2024-05-08T22:21:03.519+0000 7f2f96876700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:21:03.840 INFO:journalctl@ceph.mds.k.smithi132.stdout:May 08 22:21:03 smithi132 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-k[113794]: 2024-05-08T22:21:03.519+0000 7f2f96876700 -1 mds.k *** got signal Terminated *** 2024-05-08T22:21:07.590 INFO:journalctl@ceph.mds.k.smithi132.stdout:May 08 22:21:07 smithi132 bash[122117]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-k 2024-05-08T22:21:07.661 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.k.service' 2024-05-08T22:21:07.694 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:21:07.694 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-08T22:21:07.694 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-08T22:21:07.694 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.c 2024-05-08T22:21:08.017 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 08 22:21:07 smithi157 systemd[1]: Stopping Ceph mds.c for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:21:08.276 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 08 22:21:08 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-c[115254]: 2024-05-08T22:21:08.015+0000 7f7f7f730700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:21:08.276 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 08 22:21:08 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-c[115254]: 2024-05-08T22:21:08.015+0000 7f7f7f730700 -1 mds.c *** got signal Terminated *** 2024-05-08T22:21:11.642 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 08 22:21:11 smithi157 bash[122884]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-c 2024-05-08T22:21:11.648 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.c.service' 2024-05-08T22:21:11.685 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:21:11.685 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-08T22:21:11.686 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-08T22:21:11.686 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.f 2024-05-08T22:21:11.950 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 08 22:21:11 smithi157 systemd[1]: Stopping Ceph mds.f for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:21:12.276 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 08 22:21:11 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-f[114914]: 2024-05-08T22:21:11.948+0000 7fda504d3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:21:12.276 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 08 22:21:11 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-f[114914]: 2024-05-08T22:21:11.948+0000 7fda504d3700 -1 mds.f *** got signal Terminated *** 2024-05-08T22:21:16.220 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 08 22:21:15 smithi157 bash[123024]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-f 2024-05-08T22:21:16.225 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.f.service' 2024-05-08T22:21:16.260 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:21:16.260 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-08T22:21:16.260 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-08T22:21:16.261 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.i 2024-05-08T22:21:16.506 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 08 22:21:16 smithi157 systemd[1]: Stopping Ceph mds.i for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:21:16.776 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 08 22:21:16 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-i[115591]: 2024-05-08T22:21:16.505+0000 7f6331448700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:21:16.776 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 08 22:21:16 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-i[115591]: 2024-05-08T22:21:16.505+0000 7f6331448700 -1 mds.i *** got signal Terminated *** 2024-05-08T22:21:19.615 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 08 22:21:19 smithi157 bash[123162]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-i 2024-05-08T22:21:19.621 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.i.service' 2024-05-08T22:21:19.655 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:21:19.656 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-08T22:21:19.656 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-08T22:21:19.656 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.l 2024-05-08T22:21:19.922 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 08 22:21:19 smithi157 systemd[1]: Stopping Ceph mds.l for 06f1a696-0d86-11ef-bc97-c7b262605968... 2024-05-08T22:21:20.276 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 08 22:21:19 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-l[114564]: 2024-05-08T22:21:19.921+0000 7f3968cfd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:21:20.276 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 08 22:21:19 smithi157 ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-l[114564]: 2024-05-08T22:21:19.921+0000 7f3968cfd700 -1 mds.l *** got signal Terminated *** 2024-05-08T22:21:21.777 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 08 22:21:21 smithi157 bash[123302]: ceph-06f1a696-0d86-11ef-bc97-c7b262605968-mds-l 2024-05-08T22:21:21.877 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06f1a696-0d86-11ef-bc97-c7b262605968@mds.l.service' 2024-05-08T22:21:21.911 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:21:21.911 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-08T22:21:21.912 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T22:21:22.171 INFO:teuthology.orchestra.run.smithi131.stdout:Deleting cluster with fsid: 06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T22:21:25.125 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T22:21:25.421 INFO:teuthology.orchestra.run.smithi132.stdout:Deleting cluster with fsid: 06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T22:21:26.884 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T22:21:27.072 INFO:teuthology.orchestra.run.smithi157.stdout:Deleting cluster with fsid: 06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T22:21:28.448 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T22:21:28.524 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T22:21:28.560 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T22:21:28.590 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-08T22:21:28.593 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698072/remote/smithi131/crash 2024-05-08T22:21:28.594 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/crash -- . 2024-05-08T22:21:28.660 INFO:teuthology.orchestra.run.smithi131.stderr:tar: /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T22:21:28.660 INFO:teuthology.orchestra.run.smithi131.stderr:tar: Error is not recoverable: exiting now 2024-05-08T22:21:28.664 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698072/remote/smithi132/crash 2024-05-08T22:21:28.665 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/crash -- . 2024-05-08T22:21:28.774 INFO:teuthology.orchestra.run.smithi132.stderr:tar: /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T22:21:28.774 INFO:teuthology.orchestra.run.smithi132.stderr:tar: Error is not recoverable: exiting now 2024-05-08T22:21:28.776 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698072/remote/smithi157/crash 2024-05-08T22:21:28.777 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/crash -- . 2024-05-08T22:21:28.810 INFO:teuthology.orchestra.run.smithi157.stderr:tar: /var/lib/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T22:21:28.810 INFO:teuthology.orchestra.run.smithi157.stderr:tar: Error is not recoverable: exiting now 2024-05-08T22:21:28.811 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-08T22:21:28.811 DEBUG:teuthology.orchestra.run.smithi131:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-08T22:21:28.891 INFO:teuthology.orchestra.run.smithi131.stderr:grep: /var/log/ceph/06f1a696-0d86-11ef-bc97-c7b262605968/ceph.log: No such file or directory 2024-05-08T22:21:28.896 INFO:tasks.cephadm:Compressing logs... 2024-05-08T22:21:28.896 DEBUG:teuthology.orchestra.run.smithi131:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T22:21:28.900 DEBUG:teuthology.orchestra.run.smithi132:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T22:21:28.903 DEBUG:teuthology.orchestra.run.smithi157:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T22:21:28.931 INFO:teuthology.orchestra.run.smithi157.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-08T22:21:28.945 INFO:teuthology.orchestra.run.smithi132.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-08T22:21:28.960 INFO:teuthology.orchestra.run.smithi131.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-08T22:29:01.246 INFO:tasks.cephadm:Archiving logs... 2024-05-08T22:29:01.248 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698072/remote/smithi131/log 2024-05-08T22:29:01.249 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T22:29:28.195 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698072/remote/smithi132/log 2024-05-08T22:29:28.196 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T22:30:29.219 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698072/remote/smithi157/log 2024-05-08T22:30:29.220 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T22:30:55.701 INFO:tasks.cephadm:Removing cluster... 2024-05-08T22:30:55.702 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 --force 2024-05-08T22:30:55.963 INFO:teuthology.orchestra.run.smithi131.stdout:Deleting cluster with fsid: 06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T22:30:56.985 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 --force 2024-05-08T22:30:57.537 INFO:teuthology.orchestra.run.smithi132.stdout:Deleting cluster with fsid: 06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T22:30:58.144 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 06f1a696-0d86-11ef-bc97-c7b262605968 --force 2024-05-08T22:30:58.335 INFO:teuthology.orchestra.run.smithi157.stdout:Deleting cluster with fsid: 06f1a696-0d86-11ef-bc97-c7b262605968 2024-05-08T22:30:58.779 INFO:tasks.cephadm:Removing cephadm ... 2024-05-08T22:30:58.779 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T22:30:58.827 DEBUG:teuthology.orchestra.run.smithi132:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T22:30:58.857 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T22:30:58.875 INFO:tasks.cephadm:Teardown complete 2024-05-08T22:30:58.875 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-08T22:30:58.887 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-08T22:30:58.888 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T22:30:58.892 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T22:30:58.900 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T22:30:59.026 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-08T22:30:59.037 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-08T22:30:59.038 DEBUG:teuthology.orchestra.run.smithi131:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T22:30:59.042 DEBUG:teuthology.orchestra.run.smithi132:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T22:30:59.044 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T22:30:59.059 INFO:teuthology.orchestra.run.smithi132.stderr:bash: ntpq: command not found 2024-05-08T22:30:59.060 INFO:teuthology.orchestra.run.smithi157.stderr:bash: ntpq: command not found 2024-05-08T22:30:59.064 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T22:30:59.064 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-05-08T22:30:59.064 INFO:teuthology.orchestra.run.smithi157.stdout:^* hv01.front.sepia.ceph.com 3 6 377 24 +1813us[+1759us] +/- 72ms 2024-05-08T22:30:59.065 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 24 -1141us[-1141us] +/- 66ms 2024-05-08T22:30:59.065 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 -395us[ -395us] +/- 61ms 2024-05-08T22:30:59.065 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:30:59.078 INFO:teuthology.orchestra.run.smithi131.stderr:bash: ntpq: command not found 2024-05-08T22:30:59.085 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T22:30:59.085 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2024-05-08T22:30:59.086 INFO:teuthology.orchestra.run.smithi131.stdout:^* hv01.front.sepia.ceph.com 3 6 377 25 +1789us[+1781us] +/- 72ms 2024-05-08T22:30:59.086 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 26 -1164us[-1172us] +/- 66ms 2024-05-08T22:30:59.086 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 31 -441us[ -451us] +/- 61ms 2024-05-08T22:30:59.086 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:30:59.097 INFO:teuthology.orchestra.run.smithi132.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T22:30:59.097 INFO:teuthology.orchestra.run.smithi132.stdout:=============================================================================== 2024-05-08T22:30:59.097 INFO:teuthology.orchestra.run.smithi132.stdout:^* hv01.front.sepia.ceph.com 3 6 377 28 +1815us[+1752us] +/- 72ms 2024-05-08T22:30:59.097 INFO:teuthology.orchestra.run.smithi132.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 29 -1162us[-1225us] +/- 66ms 2024-05-08T22:30:59.097 INFO:teuthology.orchestra.run.smithi132.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 -401us[ -401us] +/- 61ms 2024-05-08T22:30:59.097 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:30:59.098 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-08T22:30:59.109 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-08T22:30:59.110 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-08T22:30:59.133 DEBUG:teuthology.orchestra.run.smithi131:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T22:30:59.271 DEBUG:teuthology.orchestra.run.smithi132:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T22:30:59.374 DEBUG:teuthology.orchestra.run.smithi157:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T22:30:59.441 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T22:30:59.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:30:59.512 DEBUG:teuthology.orchestra.run.smithi132:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T22:30:59.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:30:59.545 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T22:30:59.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:30:59.576 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-08T22:30:59.588 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-08T22:30:59.599 INFO:teuthology.task.internal:Duration was 2941.304011 seconds 2024-05-08T22:30:59.599 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-08T22:30:59.612 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-08T22:30:59.612 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T22:30:59.616 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T22:30:59.619 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T22:30:59.666 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T22:30:59.693 INFO:teuthology.orchestra.run.smithi132.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T22:30:59.724 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T22:31:00.125 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-08T22:31:00.126 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi131.front.sepia.ceph.com 2024-05-08T22:31:00.126 DEBUG:teuthology.orchestra.run.smithi131:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T22:31:00.182 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi132.front.sepia.ceph.com 2024-05-08T22:31:00.182 DEBUG:teuthology.orchestra.run.smithi132:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T22:31:00.211 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2024-05-08T22:31:00.212 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T22:31:00.235 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-08T22:31:00.236 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T22:31:00.240 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T22:31:00.254 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T22:31:00.337 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-08T22:31:00.337 DEBUG:teuthology.orchestra.run.smithi131:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T22:31:00.341 DEBUG:teuthology.orchestra.run.smithi132:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T22:31:00.344 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T22:31:00.502 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-08T22:31:00.514 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-08T22:31:00.514 DEBUG:teuthology.orchestra.run.smithi131:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T22:31:00.581 DEBUG:teuthology.orchestra.run.smithi132:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T22:31:00.646 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T22:31:00.674 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-08T22:31:00.687 DEBUG:teuthology.orchestra.run.smithi131:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T22:31:00.691 DEBUG:teuthology.orchestra.run.smithi132:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T22:31:00.693 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T22:31:00.731 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = core 2024-05-08T22:31:00.741 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2024-05-08T22:31:00.751 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = core 2024-05-08T22:31:00.786 DEBUG:teuthology.orchestra.run.smithi131:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T22:31:00.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:31:00.826 DEBUG:teuthology.orchestra.run.smithi132:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T22:31:00.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:31:00.843 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T22:31:00.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:31:00.860 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-08T22:31:00.874 INFO:teuthology.task.internal:Transferring archived files... 2024-05-08T22:31:00.876 DEBUG:teuthology.misc:Transferring archived files from smithi131:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698072/remote/smithi131 2024-05-08T22:31:00.876 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T22:31:00.960 DEBUG:teuthology.misc:Transferring archived files from smithi132:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698072/remote/smithi132 2024-05-08T22:31:00.961 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T22:31:01.002 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698072/remote/smithi157 2024-05-08T22:31:01.002 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T22:31:01.043 INFO:teuthology.task.internal:Removing archive directory... 2024-05-08T22:31:01.043 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T22:31:01.048 DEBUG:teuthology.orchestra.run.smithi132:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T22:31:01.050 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T22:31:01.093 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-08T22:31:01.106 INFO:teuthology.task.internal:Not uploading archives. 2024-05-08T22:31:01.106 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-08T22:31:01.118 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-08T22:31:01.118 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T22:31:01.131 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T22:31:01.133 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T22:31:01.147 INFO:teuthology.orchestra.run.smithi132.stdout: 262157 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 22:31 /home/ubuntu/cephtest 2024-05-08T22:31:01.149 INFO:teuthology.orchestra.run.smithi157.stdout: 262157 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 22:31 /home/ubuntu/cephtest 2024-05-08T22:31:01.168 INFO:teuthology.orchestra.run.smithi131.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 22:31 /home/ubuntu/cephtest 2024-05-08T22:31:01.171 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-08T22:31:01.184 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-08T22:31:01.253 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/blogbench}} duration: 2941.304010629654 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-08T22:31:01.253 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T22:31:01.339 INFO:teuthology.run:pass